All realtime processing failing

oo_processing · by **oo_processing** » Wed Dec 02, 2020 10:13 am America/New_York

I have 286 passes that have MODIS failures since doy 336 in all my subscription area. VIIRS is unaffected using the same software.
It seems that the L1B files produced have errors?
This is what I am seeing:

$ ll -ltrh
total 379M
-rw-rw-r-- 1 bmurch usfuser 379M Dec 2 09:45 MOD00.P2020336.1755_1.PDS
-rw-r--r-- 1 bmurch usfuser 4.0K Dec 2 09:53 ancillary_data.db
$ modis_L1A.py ./MOD00.P2020336.1755_1.PDS -o ./MOD00.P2020336.1755.BIGBEND_FULL.L1A_LAC --startnudge=0 --stopnudge=10
MODIS L1A version 6.0.6, built Aug 16 2019 12:46:26
scan: 0 out of 201 Wed Dec 2 10:03:07 2020
...
scan: 190 out of 201 Wed Dec 2 10:03:15 2020
[bmurch@optics0 P2020336.1755]$ ll -ltrh
total 908M
-rw-rw-r-- 1 bmurch usfuser 379M Dec 2 09:45 MOD00.P2020336.1755_1.PDS
-rw-r--r-- 1 bmurch usfuser 4.0K Dec 2 09:53 ancillary_data.db
-rw-rw-r-- 1 bmurch usfuser 530M Dec 2 10:03 MOD00.P2020336.1755.BIGBEND_FULL.L1A_LAC
[bmurch@optics0 P2020336.1755]$ modis_GEO.py MOD00.P2020336.1755.BIGBEND_FULL.L1A_LAC -o MOD00.P2020337.1755.BIGBEND_FULL.GEO --threshold=95 --ancdb=./ancillary_data.db
MODIS GEO version 6.1.0, built Aug 16 2019 12:46:24
scan: 0 out of 196 Wed Dec 2 10:03:41 2020
...
scan: 190 out of 196 Wed Dec 2 10:03:42 2020
Percent valid data (0.00) is less than threshold (95.00)
ERROR: MODIS geolocation processing failed.

There are error files created:
$ ll -ltrh
total 911M
-rw-rw-r-- 1 bmurch usfuser 379M Dec 2 09:45 MOD00.P2020336.1755_1.PDS
-rw-r--r-- 1 bmurch usfuser 4.0K Dec 2 10:03 ancillary_data.db
-rw-rw-r-- 1 bmurch usfuser 23K Dec 2 10:03 MOD00.P2020337.1755.BIGBEND_FULL.GEO.pcf
-rw-rw-r-- 1 bmurch usfuser 1.2K Dec 2 10:03 LogReport.MOD00.P2020337.1755.BIGBEND_FULL.GEO
-rw-rw-r-- 1 bmurch usfuser 2.2M Dec 2 10:03 MOD00.P2020337.1755.BIGBEND_FULL.GEO
-rw-rw-r-- 1 bmurch usfuser 530M Dec 2 10:03 MOD00.P2020336.1755.BIGBEND_FULL.L1A_LAC
-rw-rw-r-- 1 bmurch usfuser 266K Dec 2 10:03 LogStatus.MOD00.P2020337.1755.BIGBEND_FULL.GEO
-rw-rw-r-- 1 bmurch usfuser 1.2K Dec 2 10:03 LogUser.MOD00.P2020337.1755.BIGBEND_FULL.GEO

See attached files but ... the all seem to have this

$ cat LogReport.MOD00.P2020337.1755.BIGBEND_FULL.GEO

FILE: /shares/cms_optics/virtual_ant/S4P/S4P_MODIS_H5/seadas_l1a_geo_extract_h5/failed/P2020336.1755/LogReport.MOD00.P2020337.1755.BIGBEND_FULL.GEO
LOGGING: status message logging enabled
TRACE_LEVEL: tracing disabled
PID_LOGGING: disabled
DISABLED_LEVELS: none
DISABLED_SEEDS: none
DISABLED_CODES: PGSCSC_W_PREDICTED_UT1 PGSDEM_M_FILLVALUE_INCLUDED
THREAD-SAFE MODE: disabled
TOOLKIT_VERSION: SCF TK5.2.19
****************************************

Wed Dec 2 10:03:42 2020

Input file name: /shares/cms_optics/virtual_ant/S4P/S4P_MODIS_H5/seadas_l1a_geo_extract_h5/failed/P2020336.1755/MOD00.P2020336.1755.BIGBEND_FULL.L1A_LAC
Output file name: /shares/cms_optics/virtual_ant/S4P/S4P_MODIS_H5/seadas_l1a_geo_extract_h5/failed/P2020336.1755/MOD00.P2020337.1755.BIGBEND_FULL.GEO
Local Granule ID: MYD01.A2020336.1755.061.2020337150315.hdf
scans: 196
pixels: 0
missing: 0
out of bounds: 0
Cumulated gflags: 0, 0, 0, 0, 0, 0, 0, 0
Bounding coordinates
east: 3.141593
west: -3.141593
north: -1.570796
south: 1.570796
==========================================================
AND:

$ cat LogReport.MOD00.P2020337.1155.ANGOLA_FULL.GEO

****************************************
BEGIN_PGE: Wed Dec 2 10:09:16 2020
MSG_TAG: 11
FILE: /shares/cms_optics/virtual_ant/S4P/S4P_MODIS_H5/seadas_l1a_geo_extract_h5/failed/P2020337.1155/LogReport.MOD00.P2020337.1155.ANGOLA_FULL.GEO
LOGGING: status message logging enabled
TRACE_LEVEL: tracing disabled
PID_LOGGING: disabled
DISABLED_LEVELS: none
DISABLED_SEEDS: none
DISABLED_CODES: PGSCSC_W_PREDICTED_UT1 PGSDEM_M_FILLVALUE_INCLUDED
THREAD-SAFE MODE: disabled
TOOLKIT_VERSION: SCF TK5.2.19
****************************************

Wed Dec 2 10:09:18 2020

Input file name: /shares/cms_optics/virtual_ant/S4P/S4P_MODIS_H5/seadas_l1a_geo_extract_h5/failed/P2020337.1155/MOD00.P2020337.1155.ANGOLA_FULL.L1A_LAC
Output file name: /shares/cms_optics/virtual_ant/S4P/S4P_MODIS_H5/seadas_l1a_geo_extract_h5/failed/P2020337.1155/MOD00.P2020337.1155.ANGOLA_FULL.GEO
Local Granule ID: MYD01.A2020337.1155.061.2020337150908.hdf
scans: 196
pixels: 0
missing: 0
out of bounds: 0
Cumulated gflags: 0, 0, 0, 0, 0, 0, 0, 0
Bounding coordinates
east: 3.141593
west: -3.141593
north: -1.570796
south: 1.570796
==========================================================
attachment 1

attachment 2

attachment 3

by **OB SeaDAS - dshea** » Wed Dec 02, 2020 11:08 am America/New_York

Please see:

https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=12848

don

oo_processing · by **oo_processing** » Thu Dec 03, 2020 12:45 pm America/New_York

Don,

Still seeing a couple like this:

[bmurch@optics0 P2020338.0955]$ modis_L1A.py MOD00.P2020338.0955_1.PDS -o MOD00.P2020338.0955.GBAY_FULL.L1A_LAC --startnudge=0 --stopnudge=10
MODIS L1A version 6.0.6, built Aug 16 2019 12:46:26
scan: 0 out of 201 Thu Dec 3 12:42:20 2020
scan: 10 out of 201 Thu Dec 3 12:42:21 2020
scan: 20 out of 201 Thu Dec 3 12:42:21 2020
scan: 30 out of 201 Thu Dec 3 12:42:22 2020
scan: 40 out of 201 Thu Dec 3 12:42:22 2020
scan: 50 out of 201 Thu Dec 3 12:42:23 2020
scan: 60 out of 201 Thu Dec 3 12:42:23 2020
scan: 70 out of 201 Thu Dec 3 12:42:24 2020
scan: 80 out of 201 Thu Dec 3 12:42:24 2020
scan: 90 out of 201 Thu Dec 3 12:42:25 2020
scan: 100 out of 201 Thu Dec 3 12:42:25 2020
scan: 110 out of 201 Thu Dec 3 12:42:26 2020
scan: 120 out of 201 Thu Dec 3 12:42:26 2020
scan: 130 out of 201 Thu Dec 3 12:42:27 2020
scan: 140 out of 201 Thu Dec 3 12:42:27 2020
scan: 150 out of 201 Thu Dec 3 12:42:28 2020
scan: 160 out of 201 Thu Dec 3 12:42:28 2020
scan: 170 out of 201 Thu Dec 3 12:42:29 2020
scan: 180 out of 201 Thu Dec 3 12:42:29 2020
scan: 190 out of 201 Thu Dec 3 12:42:30 2020
[bmurch@optics0 P2020338.0955]$ modis_GEO.py MOD00.P2020338.0955.GBAY_FULL.L1A_LAC -o MOD00.P2020338.0955.GBAY_FULL.GEO --threshold=95 --ancdb=./ancillary_data.db
MODIS GEO version 6.1.0, built Aug 16 2019 12:46:24
scan: 0 out of 197 Thu Dec 3 12:43:04 2020
scan: 10 out of 197 Thu Dec 3 12:43:05 2020
scan: 20 out of 197 Thu Dec 3 12:43:05 2020
scan: 30 out of 197 Thu Dec 3 12:43:05 2020
scan: 40 out of 197 Thu Dec 3 12:43:06 2020
scan: 50 out of 197 Thu Dec 3 12:43:06 2020
scan: 60 out of 197 Thu Dec 3 12:43:06 2020
scan: 70 out of 197 Thu Dec 3 12:43:07 2020
scan: 80 out of 197 Thu Dec 3 12:43:07 2020
scan: 90 out of 197 Thu Dec 3 12:43:07 2020
scan: 100 out of 197 Thu Dec 3 12:43:08 2020
scan: 110 out of 197 Thu Dec 3 12:43:08 2020
scan: 120 out of 197 Thu Dec 3 12:43:08 2020
scan: 130 out of 197 Thu Dec 3 12:43:08 2020
scan: 140 out of 197 Thu Dec 3 12:43:09 2020
scan: 150 out of 197 Thu Dec 3 12:43:09 2020
scan: 160 out of 197 Thu Dec 3 12:43:09 2020
scan: 170 out of 197 Thu Dec 3 12:43:10 2020
scan: 180 out of 197 Thu Dec 3 12:43:10 2020
scan: 190 out of 197 Thu Dec 3 12:43:10 2020
Percent valid data (0.00) is less than threshold (95.00)
ERROR: MODIS geolocation processing failed.
[bmurch@optics0 P2020338.0955]$ ll -ltrh
total 371M
-rw-rw-r-- 1 bmurch cms_optics 113M Dec 3 09:46 MOD00.P2020338.0955_1.PDS
-rw-r--r-- 1 bmurch usfuser    63K Dec 3 12:41 SEADAS_L1A_GEO_EXTRACT_H5.GBAY.MODISA.MOD00.P2020338.0955_1.log
-rw-r--r-- 1 bmurch usfuser 4.0K Dec 3 12:43 ancillary_data.db
-rw-rw-r-- 1 bmurch usfuser    23K Dec 3 12:43 MOD00.P2020338.0955.GBAY_FULL.GEO.pcf
-rw-rw-r-- 1 bmurch usfuser 1008 Dec 3 12:43 LogReport.MOD00.P2020338.0955.GBAY_FULL.GEO
-rw-rw-r-- 1 bmurch usfuser    87M Dec 3 12:43 MOD00.P2020338.0955.GBAY_FULL.GEO
-rw-rw-r-- 1 bmurch usfuser 175M Dec 3 12:43 MOD00.P2020338.0955.GBAY_FULL.L1A_LAC
-rw-rw-r-- 1 bmurch usfuser 304K Dec 3 12:43 LogStatus.MOD00.P2020338.0955.GBAY_FULL.GEO
-rw-rw-r-- 1 bmurch usfuser 1.1K Dec 3 12:43 LogUser.MOD00.P2020338.0955.GBAY_FULL.GEO

****************************************
BEGIN_PGE: Thu Dec 3 12:43:03 2020
MSG_TAG: 11
FILE: /work/b/bmurch/test/P2020338.0955/LogReport.MOD00.P2020338.0955.GBAY_FULL.GEO
LOGGING: status message logging enabled
TRACE_LEVEL: tracing disabled
PID_LOGGING: disabled
DISABLED_LEVELS: none
DISABLED_SEEDS: none
DISABLED_CODES: PGSCSC_W_PREDICTED_UT1 PGSDEM_M_FILLVALUE_INCLUDED
THREAD-SAFE MODE: disabled
TOOLKIT_VERSION: SCF TK5.2.19
****************************************

Thu Dec 3 12:43:10 2020

Input file name: /work/b/bmurch/test/P2020338.0955/MOD00.P2020338.0955.GBAY_FULL.L1A_LAC
Output file name: /work/b/bmurch/test/P2020338.0955/MOD00.P2020338.0955.GBAY_FULL.GEO
Local Granule ID: MYD01.A2020338.0955.061.2020338174230.hdf
scans: 197
pixels: 2667380
missing: 2667380
out of bounds: 0
Cumulated gflags: 0, 0, 0, 2667380, 0, 2667380, 0, 2667380
Bounding coordinates
east: 0.000000
west: 3.141593
north: -1.570796
south: 1.570796
==========================================================