MYD03.061 CMR entry changed, but the file didn't

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
earthengine_urs
Posts: 87
Joined: Mon Jan 27, 2020 10:36 am America/New_York
Answers: 0
Has thanked: 3 times
Been thanked: 2 times

MYD03.061 CMR entry changed, but the file didn't

by earthengine_urs » Tue Aug 27, 2024 7:44 pm America/New_York

Hi,

We just noticed that some MYD03.061 entries like MYD03.A2010123.2320.061.2018059034522.hdf have new values of the "updated" field in CMR.

Eg, https://cmr.earthdata.nasa.gov/search/granules.json?collection_concept_id=C1379841358-LAADS&temporal=2010-05-03T23:20:00 shows the "updated" field with the value of 2024-08-22, but in an earlier scan we saw the same filename with the "updated" field of 2022-12-10.

The actual file's checksum didn't change compared to the version we downloaded earlier, so the change in the "updated" field feels unwarranted. But if this is an expected situation, we can adjust our download procedure.

Advice appreciated!

Thanks,
Simon

Filters:

LAADS-EDL_UserServices_M
Posts: 385
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 2
Has thanked: 3 times

Re: MYD03.061 CMR entry changed, but the file didn't

by LAADS-EDL_UserServices_M » Wed Aug 28, 2024 6:55 am America/New_York

Yes, CMR records for all LAADS data products have been updated starting August 1, 2024. The update is due to changing data access URL from on-premise to Cloud, see announcement at:
https://ladsweb.modaps.eosdis.nasa.gov/cloud/
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

earthengine_urs
Posts: 87
Joined: Mon Jan 27, 2020 10:36 am America/New_York
Answers: 0
Has thanked: 3 times
Been thanked: 2 times

Re: MYD03.061 CMR entry changed, but the file didn't

by earthengine_urs » Wed Aug 28, 2024 6:33 pm America/New_York

Thank you, noted. I would argue that because the actual data haven't changed, the "updated" field shouldn't change - the switch from one storage system to another is an implementation detail. Instead, the "revisionDate" field indicating the CMR metadata update time should change. But I realize this is too late to give feedback on the process.

LAADS-EDL_UserServices_M
Posts: 385
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 2
Has thanked: 3 times

Re: MYD03.061 CMR entry changed, but the file didn't

by LAADS-EDL_UserServices_M » Thu Aug 29, 2024 8:40 am America/New_York

Good point! However, a change in data is reflected by the change in last part of the granule id (the production date, e.g., "2018059034713" in MYD03.A2010123.2315.061.2018059034713.hdf) and the change in the "updated" field shows updates in metadata.
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

earthengine_urs
Posts: 87
Joined: Mon Jan 27, 2020 10:36 am America/New_York
Answers: 0
Has thanked: 3 times
Been thanked: 2 times

Re: MYD03.061 CMR entry changed, but the file didn't

by earthengine_urs » Thu Aug 29, 2024 3:42 pm America/New_York

The problem is that not every dataset uses the filename to indicate the data version (and if they do, there's no consistenvcy - eg, SMAP uses _01, _02, _03 etc instead of the timestamp). So it would have been better to rely on more structured entries in CMR, but I realize that achieving consistency across thousands of datasets is hard.

LAADS-EDL_UserServices_M
Posts: 385
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 2
Has thanked: 3 times

Re: MYD03.061 CMR entry changed, but the file didn't

by LAADS-EDL_UserServices_M » Fri Aug 30, 2024 9:40 am America/New_York

"I realize that achieving consistency across thousands of datasets is hard." --True!
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

Post Reply