Page 1 of 1
Looking for feedback
Posted: Tue Apr 29, 2025 3:44 pm America/New_York
by andrewsurf
Hello everyone,
I’m currently working on a new library focused on significantly improving the performance of large satellite image processing workflows.
I would like to ask the community:
• Which operations when using GDAL or rasterio (etc) feel the slowest or most frustrating for you?
• Are there particular bottlenecks you encounter when working with large GeoTIFFs or cloud-optimized datasets?
Any insights — even small examples — would be incredibly helpful to better understand real-world pain points.
Thank you very much for your help!
Re: Looking for feedback
Posted: Thu May 15, 2025 3:55 pm America/New_York
by GES DISC - jimacker
To andrewsurf:
We apologize for the delay in our reply to your post. We do suggest adding other DAAC filters to this post (if you would like me to do that, please reply, or you can post your questions again), as the filter for "GES DISC" may have limited the visibility or responses to your questions in the community which may have experience and/or opinion.
I have two responses from my GES DISC colleagues, which have been edited for clarity:
"I have minimal experience in working with GDAL and GeoTIFFS, most likely due to the fact that we at GES DISC don’t host any STAC collections, or COGs. This is also why we don’t have any tutorials with rasterio accessing cloud-optimized formats. I can’t really give an opinion on rasterio bottlenecks either. In the limited times I’ve used it it’s been pretty efficient, but I’ve never performed a huge analysis with this data format to be able to tell a difference.
Other DAACs like LP DAAC host CMR-STAC COGs for HLS-Landsat."
Second response:
"We haven’t experienced any real bottlenecks or pain points when using GeoTIFFs or GDAL. Most of my (and our) experiences with GDAL with our own GES DISC data has been pretty successful."
"We use GDAL MRF drivers for the IMERG service with no issues."
"The only issue I have come across is that GDAL translate doesn’t always seem to like NetCDF 4 for-mats (*.nc4), but it likes NetCDF formats (*.nc)."
"In my graduate work, 0.5-meter spatial resolution GeoTIFFs operate slowly when using QGIS (or GDAL) processing tools, but that was due to the sheer size of the GeoTIFFs, and I was using my personal laptop."
"I think GeoTIFFs are among the easiest raster data formats to work with in GIS, especially if they are georeferenced correctly. GDAL is a bit of a learning curve, but works great once you understand the commands."