TEMPO Proxy NO2 in QGIS

[lschwize]

Software: QGIS 3.20.3
Data Products: TEMPO_NO2-PROXY_L3_PRO_20130810T160000Z_S008

Issue: CRS Undefined/Unknown and overall Projection errors (Note: file comes in without issue via Add Multidimensional Data -> Multiband Raster in ArcGIS Pro)

Workflow:
Add Raster Layer - successful
Assign Raster - fails
Warp (Reproject) - error, output flipped
and
Add Mesh Layer - program repeatedly crashes prior to display

Is there a preferred workaround to use this data in QGIS?

[joseph.f.koch]

Hi Leah,

Thanks for notifying us of the issue you are encountering when displaying the TEMPO proxy data (NO2) in QGIS.

We'll take a closer look at this and get back to you as soon as possible with a workaround or solution to the problems you are experiencing.

Thanks,
Joe

[joseph.f.koch]

Hi Leah,

Here is an overview of our findings after taking a closer look.

It appears the version of QGIS that you are running (3.20.3) was released in September of this year, so we were able to quickly rule out that this is not an issue due to the software being outdated (or software in general). We encountered the same issues even with the latest version of QGIS (3.22.0).

We then took a look at the preliminary TEMPO L3 proxy data in Panoply and were able to see that there were some discrepancies that we'll need to coordinate with the TEMPO science team (data producers) to address, which we are planning to do to get ahead of things. When compared to the TEMPO L2 proxy data, the L3 data lacks a significant amount of metadata and is quite sparse. We also noticed some different fields/values were used to populate the L3 data which is likely what is causing the interoperability issues you are seeing - this was also the case when working with the data using ArcGIS Pro.

In any case, the ASDC will be working internally and with the TEMPO data producers to correct these metadata issues to make the data more interoperable with tools like QGIS/ArcGIS, and will keep you updated as we learn more and things progress!

Thanks,
Joe Koch
ASDC User Services

[davidglenn]

I'm using QGIS 3.34. I've downloaded a subset file created by HARMONY and checked it in Panoply. In Panoply, the image is where it should be. In QGIS it has no CRS when opened. I've changed the CRS to EPSG:3857, the same CRS as OSM. The layer places the image off the African coast. I've saved the raster file as a Geotiff and assigned the 3857 CRS and the image still goes to the African Coast. What is the solution to this problem? Many thanks. I attended the webinar yesterday and it was an excellent presentation on TEMPO that I was able to quickly implement.

[ASDC - ghayescrepps]

Thank you for your post and we are glad to hear you enjoyed the TEMPO Earthdata webinar! Can you please confirm that you are using TEMPO V03 beta data? Is there a specific collection (e.g., NO2 L3, formaldehyde (HCHO) L2) you are working with? The TEMPO V03 data is in WGS 1984 (CRS: 4326). Please apply this CRS and see if this addresses this issue.

If you would like to reference the CRS in the data, this can be seen in the global attributes as: geospatial_bounds_crs = "EPSG:4326." You can see the global attributes in the viewing pane in Panoply (as well as other data software).

[davidglenn]

I hate to sound incompetent but I've used QGIS for years and never had this problem. I am downloading a level 3, subsetted TEMPO file generated by HARMONY. I've set the CRS for it and OSM at 4326. Examining the information for each layer confirms that the CRS is 4326. The TEMPO layer should be over the US but is in the Atlantic ocean off the coast of Africa. I've used other background maps with the same results. I've set OSM and the TEMPO file to CRS 3857 with the same results. Panoply does present the layer correctly over the US and has the CRS set at 4329. I've gone back to an older version of QGIS (3.16) with the same results. I'm at a loss for other approaches. Any suggestions?

[ASDC - ghayescrepps]

Hello,
We have been looking into this issue. I am able to replicate your same error using the TEMPO V03 L3 NO2 product. The TEMPO data is a NetCDF4 with grouped/hierarchical variables (e.g., most of the variables are not at the root/home level of the file, but are instead in sub-groups). In the L3 NO2, there is one variable "weight" that is at the root level of the file; for this variable, once I import the file into QGIS (Layer > Data Source Manager > Raster), and set the CRS to 4326, this one variable appears in the correct ___location over North America. However, if I import any of the other variables (e.g., product/vertical_column_troposphere), setting the CRS is not sufficient to correctly display the data. In addition, when I look at the properties>information for these two variables, weight has a origin and pixel size listed, while the product/vertical_column_troposphere does not. This leads me to believe that QGIS is not reading the grouped variables in the NetCDF correctly. Using the warp tool, I can set the CRS for the product/vertical_column_troposphere to 4326 and the layer does appear correctly in the map viewer; however, in my initial tests, it looks like there may be some data loss due to the interpolation occurring (e.g., the min/max of the reprojected data are not the same as in the original). While I think I may have found the cause of this issue, I have yet to find a great solution; however, we are still looking into this.

Can you provide any more details about your use case--which data products you are using and how you would like to process/apply them? We might be able to assist in finding an alternate solution as well.

Thanks,
Georgina

[davidglenn]

I appreciate you taking the time to re-create the problem. I've tried to warp reproject a subsetted file to 4326 and it still does not display correctly. I've even exported the warped file to a the same geotiff at 4326 and it still does not display correctly. How do I plan to use TEMPO? I am a private citizen (tho a retired USDA-ARS research scientist) examining the pollution from a manufacturing plant recently established in the panhandle of WV. I would like to use the hourly images for this AOI (subsetted by HARMONY) to detect plumes of O3 NO2 and/or HCOH. It is a shot in the dark but other satellite imagery I've used has lacked the temporal resolution of TEMPO (although the spatial resolution of TEMPO may be too limiting for my purposes). This area is not particularly windy so I may get lucky and have day(s) with little atmospheric turbulance. Any suggestions will be greatly appreciated.

[ASDC - ghayescrepps]

Thank you for providing additional information. We have been investigating the QGIS documentation as well as the TEMPO data. At this point in time, my assumption is that QGIS is not reading the data from the grouped variables in the TEMPO netcdf 4 files correctly. We were able to use python to flatten the groups of a sample L3 TEMPO file before importing the .nc into QGIS; this appears to have resolved the issue as the data are located properly on the map. While this may not be ideal to have to add in an additional step to your workflow, is it possible for you to test flattening the data to see if that resolves your issue?

In addition, we do currently have one ArcGIS image service for Level 3 NO2 troposphere (https://www.arcgis.com/home/item.html?id=6a1bdd0c076d499da69e867732ed2ab7). Utilizing these pre-filtered data may be of use to your interest case in calculating aggregates, as you can do analyses against the service rest URL (e.g., Get Samples) without an ArcGIS license. ArcGIS image services in general are supported in QGIS, although this TEMPO image service is time-enabled and we have been having some difficulty getting it to work properly in QGIS so using this image service may still require a workflow outside of QGIS for the time being.

We are continuing to investigate best practices for utilizing TEMPO data in QGIS, but this may take some time to make this assessment.

[davidglenn]

Thank you for continuing to look into this. I'm sure I'm not the only QGIS user. Unfortunately, my python programming skills are very limited and 'flattening' those files is beyond my skill level. I'll be patient and thanks for all you are doing.