MYD03.061 CMR entry changed, but the file didn't
-
- Posts: 87
- Joined: Mon Jan 27, 2020 10:36 am America/New_York
- Has thanked: 3 times
- Been thanked: 2 times
MYD03.061 CMR entry changed, but the file didn't
Hi,
We just noticed that some MYD03.061 entries like MYD03.A2010123.2320.061.2018059034522.hdf have new values of the "updated" field in CMR.
Eg, https://cmr.earthdata.nasa.gov/search/granules.json?collection_concept_id=C1379841358-LAADS&temporal=2010-05-03T23:20:00 shows the "updated" field with the value of 2024-08-22, but in an earlier scan we saw the same filename with the "updated" field of 2022-12-10.
The actual file's checksum didn't change compared to the version we downloaded earlier, so the change in the "updated" field feels unwarranted. But if this is an expected situation, we can adjust our download procedure.
Advice appreciated!
Thanks,
Simon
We just noticed that some MYD03.061 entries like MYD03.A2010123.2320.061.2018059034522.hdf have new values of the "updated" field in CMR.
Eg, https://cmr.earthdata.nasa.gov/search/granules.json?collection_concept_id=C1379841358-LAADS&temporal=2010-05-03T23:20:00 shows the "updated" field with the value of 2024-08-22, but in an earlier scan we saw the same filename with the "updated" field of 2022-12-10.
The actual file's checksum didn't change compared to the version we downloaded earlier, so the change in the "updated" field feels unwarranted. But if this is an expected situation, we can adjust our download procedure.
Advice appreciated!
Thanks,
Simon
Filters:
-
- Posts: 385
- Joined: Mon Sep 30, 2019 8:33 am America/New_York
- Has thanked: 3 times
Re: MYD03.061 CMR entry changed, but the file didn't
Yes, CMR records for all LAADS data products have been updated starting August 1, 2024. The update is due to changing data access URL from on-premise to Cloud, see announcement at:
https://ladsweb.modaps.eosdis.nasa.gov/cloud/
https://ladsweb.modaps.eosdis.nasa.gov/cloud/
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
-
- Posts: 87
- Joined: Mon Jan 27, 2020 10:36 am America/New_York
- Has thanked: 3 times
- Been thanked: 2 times
Re: MYD03.061 CMR entry changed, but the file didn't
Thank you, noted. I would argue that because the actual data haven't changed, the "updated" field shouldn't change - the switch from one storage system to another is an implementation detail. Instead, the "revisionDate" field indicating the CMR metadata update time should change. But I realize this is too late to give feedback on the process.
-
- Posts: 385
- Joined: Mon Sep 30, 2019 8:33 am America/New_York
- Has thanked: 3 times
Re: MYD03.061 CMR entry changed, but the file didn't
Good point! However, a change in data is reflected by the change in last part of the granule id (the production date, e.g., "2018059034713" in MYD03.A2010123.2315.061.2018059034713.hdf) and the change in the "updated" field shows updates in metadata.
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
-
- Posts: 87
- Joined: Mon Jan 27, 2020 10:36 am America/New_York
- Has thanked: 3 times
- Been thanked: 2 times
Re: MYD03.061 CMR entry changed, but the file didn't
The problem is that not every dataset uses the filename to indicate the data version (and if they do, there's no consistenvcy - eg, SMAP uses _01, _02, _03 etc instead of the timestamp). So it would have been better to rely on more structured entries in CMR, but I realize that achieving consistency across thousands of datasets is hard.
-
- Posts: 385
- Joined: Mon Sep 30, 2019 8:33 am America/New_York
- Has thanked: 3 times
Re: MYD03.061 CMR entry changed, but the file didn't
"I realize that achieving consistency across thousands of datasets is hard." --True!
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.