Earthdata Forum

Hello,

I am trying to use the API to automate download of DORIS ephemeris data.

I figured out how to make a query to list the files:
https://cmr.earthdata.nasa.gov/search/granules.json?collection_concept_id=C1602851234-CDDIS&page_size=2000&pretty=true&created_at=2022-01-01T00:00:00Z,2022-06-01T00:00:00Z
(Return 170 hits)

However some time periods fail, and i am not sure why:
https://cmr.earthdata.nasa.gov/search/granules.json?collection_concept_id=C1602851234-CDDIS&page_size=2000&pretty=true&created_at=2022-01-01T00:00:00Z,2022-09-01T00:00:00Z
(Maxes out at 2000 hits with most of them pre 2022)

Is this a bug?

Regards

Hello scmember,

To better assist you, we need to ask for a point of clarification. You stated that some time periods fail, but you also state that it maxes out at 2,000 hits, with most of them pre-2022.

So your search appears to have been successful, but that you ran into a max? Or were you specifically looking for the pre-2022 time periods for results?

Ross Bagwell
CDDIS

(Deleted)

Upon closer inspection it looks like the query is working but it's filtering on the "updated" field.

I am looking through the API but struggling to figure out how to filter on "start_time".
Ideally, I would also like to be able to filter on object name and analysis center name.

API doc: https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html

Any suggestions?

I have completely given up on using the API...

In theory it is convenient but in practice I have encountered too many issues, including:
1) It only returns a small subset of files listed on the web.
2) Many of the files it lists do not exist.
3) The file sizes it return are often incorrect.
4) There is no obvious way to filter on fields, so user has to download large listings and filter manually.

I have reverted to scraping 100s of html pages to generate file lists.
I have resolved my issue with not being able to load some html pages due to this error:
"curl: (56) Recv failure: Connection was reset"

Solution was to update the default windows 10 curl to the latest version from:
8.7.1 (Windows) libcurl/8.7.1 Schannel zlib/1.3 WinIDN, Release-Date: 2024-03-27
to:
curl 8.10.1 (x86_64-w64-mingw32) libcurl/8.10.1 LibreSSL/3.9.2 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 WinIDN libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.63.0 ngtcp2/1.7.0 nghttp3/1.5.0 Release-Date: 2024-09-18