Earthdata Forum

Hi!

I'm trying to subset data from the MUR-JPL-L4-GLOB-v4.1 set (https://podaac.jpl.nasa.gov/dataset/MUR-JPL-L4-GLOB-v4.1?ids=&values=#). However, since the migration to cloud-based storage, I've found it increasingly difficult to do so. Are there any tips or tutorials on how to actually do this? I tried the podaac_data_downloader just to find out it doesn't subset the data at all.

Thanks!

Hi,

PO.DAAC is currently working on adding the subsetting function to the podaac_data_downloader script. It should be available in a few weeks. Meanwhile, you could try to use the cloud OPeNDAP service, which enables the subsetting on a single granule. Following the https://cmr.earthdata.nasa.gov/virtual-directory/collections/C1996881146-POCLOUD URL and drill down to the granule (file), you will see the OPeNDAP Link and click it. On the OPeNDAP page, you can select the variables and define the spacial range of interest (bounding box). You can either download the subsetted file here or use the wget command. Here is an example for wget command.

wget wget --http-user=username --ask-password --keep-session-cookies -O mur-sebset.nc4 'https://opendap.earthdata.nasa.gov/collections/C1996881146-POCLOUD/granules/20230711090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.dap.nc4?dap4.ce=/analysed_sst%5B0:1:0%5D%5B6500:1:8000%5D%5B15000:1:20000%5D;/lon%5B15000:1:20000%5D;/lat%5B6500:1:8000%5D'

To download multiple granules, you can write a script to execute multiple time of the wget command by simply changing the date string (20230711090000) of the filename.

Please let us know if you have any questions.

Best Regards,

Wenhao Li

Hi!

Thanks for the response! I too managed to reach the same conclusion as you did after some testing, it works alright and is relatively fast. I just thought there was a more elegant solution.

Thanks again! Cheers!

Hi,
I have tried to download the data using podaac-data-downloader for the Northern Indian Ocean, but when I download, I get whole global data. Is the subsetting issue resolved yet? If not, how can I download specific area data over 10 years?

Thank you.

Hi,

The PO.DAAC team is working on adding the subsetting function right now. It should be available in next released version. But, you could try to use the alpha version, which can be installed by following command
pip install podaac-data-subscriber==1.15.0a3

Following is an example of subsetting the MUR-JPL-L4-GLOB-v4.1 dataset.
podaac-data-downloader -c MUR-JPL-L4-GLOB-v4.1 -d ./data --start-date 2022-01-01T09:00:00Z --end-date 2022-01-10T09:00:00Z -b="-75.61, 38.4,84.1,50" --subset

Please let us know if you have any questions.

Thanks,

Please, can you help me about this error ?

I followed the instructions provided by this link:
https://search.earthdata.nasa.gov/search?p=!C1996881146-POCLOUD&pg[1][a]=2579461934!POCLOUD&pg[1][v]=t&pg[1][gsk]=-start_date&pg[1][m]=opendap&pg[1][of]=nc4&q=MUR%20SST&sb[0]=-44.54297%2C-24.18514%2C-41.2207%2C-22.21682&qt=2023-01-01T00%3A00%3A00.000Z%2C2023-01-05T23%3A59%3A59.999Z&lat=-22.92260767577794&long=-49.4296875&zoom=5

Select a data access method:
Customize OPeNDAP

Output Format:
NetCDF4

the following link was provided:
https://opendap.earthdata.nasa.gov/collections/C1996881146-POCLOUD/granules/20230105090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc4

This error showed up:
It appears that you have submitted a Bad Request.
The specific error message associated with your request was:
ERROR: Unable to convert a DAP4 DMR for this dataset to a DAP2 DDS object. This dataset contains variables and/or attributes whose data types are not compatible with the DAP2 data model. There are 13 incompatible variables and/or attributes referenced in your request. Incompatible variables: Int8 mask[time=1][lat=17999][lon=36000] Int8 mask@_FillValue Int8 mask@valid_min Int8 mask@valid_max Int8 mask@flag_masks Int8 sea_ice_fraction[time=1][lat=17999][lon=36000] Int8 sea_ice_fraction@_FillValue Int8 sea_ice_fraction@valid_min Int8 sea_ice_fraction@valid_max Int8 dt_1km_data[time=1][lat=17999][lon=36000] Int8 dt_1km_data@_FillValue Int8 dt_1km_data@valid_min Int8 dt_1km_data@valid_max

There may simply be problem with the syntax of your OPeNDAP URL. If you are using server side functions in your constraint expression you should double check the syntax of the functions that you are attempting to use.

It may also be that the URL extension did not match any that are known by this server.
Here is a list of the six URL extensions that are be recognized by all DAP servers:

dds - The DAP2 syntactic (structural) metadata response.
dds - The DAP2 semantic metadata response.
dods - The DAP2 data response.
info - The DAP2 HTML dataset information page.
html - The DAP2 HTML data request form.
ascii - The DAP2 ASCII data response.

In addition Hyrax and other new servers support the following DAP4 URL extensions:
dmr - The DAP4 dataset metadata response.
dap - The DAP4 data response.
dsr - The DAP4 dataset services response.
ddx - The DAP3.2 dataset metadata response.

Hi,

The Earthdata OpeNDAP uses the DAP2 as default, which is not working for MUR SST dataset. You have to use the DAP4 version. To enable the DAP4, you need to modify the HTTP link,

https://opendap.earthdata.nasa.gov/collections/C1996881146-POCLOUD/granules/20230105090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc4
To
https://opendap.earthdata.nasa.gov/collections/C1996881146-POCLOUD/granules/20230105090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.dap.nc4?dap4

Thanks,

PODAAC - wenhaoli wrote:
> Hi,
>
> The PO.DAAC team is working on adding the subsetting function right now. It
> should be available in next released version. But, you could try to use the
> alpha version, which can be installed by following command
> pip install podaac-data-subscriber==1.15.0a3
>
> Following is an example of subsetting the MUR-JPL-L4-GLOB-v4.1 dataset.
> podaac-data-downloader -c MUR-JPL-L4-GLOB-v4.1 -d ./data --start-date
> 2022-01-01T09:00:00Z --end-date 2022-01-10T09:00:00Z -b="-75.61,
> 38.4,84.1,50" --subset
>
> Please let us know if you have any questions.
>
> Thanks,

Hi,
This option is not subsetting/cropping the area, rather than downloading the global dataset. Is there any Python script for downloading bulk data for specific regions for a specific period? Thanks.

Hi,

Did you installed the alpha version? "pip install podaac-data-subscriber==1.15.0a3"

Yes, First stage I used the following command
pip install podaac-data-subscriber. But it doest work to crop a ___domain.

Then I used
pip install podaac-data-subscriber==1.15.0a3

and trying to crop data as follows:
podaac-data-downloader -c MUR-JPL-L4-GLOB-v4.1 -d ./data --start-date 2003-01-01T09:00:00Z --end-date 2022-12-31T09:00:00Z
b="135,-50,175,-10" --subset

Still, the global data is downloading but not getting the cropping ___domain as specified.

Thanks for your further instructions.