MODIS Data processing with OCSSW

[woodbri]

I am upgrading my processing server to Ubuntu 22.04 and I'm having issues processing data using the OCSSW software.

Code: Select all

Linux 389F910 5.15.0-112-generic #122-Ubuntu SMP Thu May 23 07:48:21 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

# I installed like this

mkdir -p /u/ocssw
cd /u/ocssw

wget -N https://oceandata.sci.gsfc.nasa.gov/manifest/install_ocssw
wget -N https://oceandata.sci.gsfc.nasa.gov/manifest/manifest.py

chmod +x install_ocssw

./install_ocssw --install_dir /u/ocssw --tag T2024.20 --bin --opt --luts --modisa --viirsj1 --viirsj2 --viirsn --common --root --viirsdem

# and later

./install_ocssw --install_dir /u/ocssw --tag V2024.0 --update

A few problems with install_ocssw:

Code: Select all

woodbri@389F910:/u/ocssw$ ./install_ocssw --installed_tag

WARNING: --tag is required to get the proper bundle list.

installedTag = Unknown
woodbri@389F910:/u/ocssw$ ./install_ocssw --installed_tag --tag V2024.0
installedTag = Unknown

woodbri@389F910:/u/ocssw$ ./install_ocssw --status --tag V2024.0
Traceback (most recent call last):
  File "/data/u/ocssw/./install_ocssw", line 1007, in <module>
    sys.exit(run())
  File "/data/u/ocssw/./install_ocssw", line 776, in run
    if status(options):
  File "/data/u/ocssw/./install_ocssw", line 542, in status
    bundleDir = os.path.abspath(os.path.join(options.install_dir, bundleInfo["dir"]))
  File "/usr/lib/python3.10/posixpath.py", line 76, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

The problem with MODIS processing is:

Code: Select all

 /u/ocssw/ocssw_runner --ocsswroot /u/ocssw/ modis_L1A MOD00.P2024177.1600_1.PDS -o MOD00.P2024177.1600_1.PDS.L1A_LAC
Traceback (most recent call last):
  File "/u/ocssw//bin/modis_L1A", line 75, in <module>
    m.l0()
  File "/data/u/ocssw/bin/modis/modis_L1A_utils.py", line 159, in l0
    status = self.get_constructor()
  File "/data/u/ocssw/bin/modis/modis_L1A_utils.py", line 107, in get_constructor
    line = line.replace('LOGDIR', self.dirs['run'].resolve().as_posix())
AttributeError: 'str' object has no attribute 'resolve'

My typical processing recipe for MODIS is:

Code: Select all

wget -q 'https://oceandata.sci.gsfc.nasa.gov/getfile/MOD00.P2024177.1600_1.PDS.bz2'

bunzip2 MOD00.P2024177.1600_1.PDS.bz2

modis_L1A MOD00.P2024177.1600_1.PDS -o MOD00.P2024177.1600_1.PDS.L1A_LAC

getanc --timeout=60 --refreshDB MOD00.P2024177.1600_1.PDS.L1A_LAC

modis_atteph --timeout=60 --refreshDB -v MOD00.P2024177.1600_1.PDS.L1A_LAC

modis_GEO --time=60 MOD00.P2024177.1600_1.PDS.L1A_LAC -o MOD00.P2024177.1600_1.PDS.GEO

modis_L1B MOD00.P2024177.1600_1.PDS.L1A_LAC MOD00.P2024177.1600_1.PDS.GEO -o MOD00.P2024177.1600_1.PDS.L1B_LAC -q MOD00.P2024177.1600_1.PDS.L1B_QKM -k MOD00.P2024177.1600_1.PDS.L1B_HKM

l2gen ifile=MOD00.P2024177.1600_1.PDS.L1B_LAC geofile=MOD00.P2024177.1600_1.PDS.GEO ofile=MOD00.P2024177.1600_1.PDS.chl.L2.nc suite=OC l2prod=chlor_a,Kd_490 parfile=MOD00.P2024177.1600_1.PDS.L1A_LAC.anc resolution=250

l2gen ifile=MOD00.P2024177.1600_1.PDS.L1B_LAC geofile=MOD00.P2024177.1600_1.PDS.GEO ofile=MOD00.P2024177.1600_1.PDS.sst.L2.nc suite=SST l2prod=sst,qual_sst,rhos_vvv parfile=MOD00.P2024177.1600_1.PDS.L1A_LAC.anc resolution=250

followed by l3mapgen commands, like:

Code: Select all

    "l3mapgen interp=area 'projection=$proj' threshold=0.5 ofile=$name.L2_K490.tif north=$north south=$south west=$west east=$east ifile=$name.chl.L3b.nc oformat=tiff fudge=3.0 product=Kd_490 apply_pal=1 quiet=0 resolution=qkm",
    # --------- process chlor_a image
    "l3mapgen interp=area 'projection=$proj' threshold=0.5 ofile=$name.L2_OC.tif north=$north south=$south west=$west east=$east ifile=$name.chl.L3b.nc oformat=tiff fudge=3.0 product=chlor_a apply_pal=1 quiet=0 resolution=qkm ",
    # --------- process sst image
    "l3mapgen interp=area 'projection=$proj' threshold=0.5 ofile=$name.L2_SST.tif north=$north south=$south west=$west east=$east ifile=$name.sst.L3b.nc oformat=tiff fudge=3.0 suite=SST product=sst apply_pal=0 quiet=0 resolution=qkm ",
    # --------- process true color image
    "l3mapgen suite=SFREFL 'projection=$proj' threshold=0.5 ofile=$name.L2_TC.tif north=$north south=$south west=$west east=$east ifile=$name.tc.L3b.nc oformat=tiff fudge=3.0 apply_pal=1 quiet=0 resolution=qkm",

The *.sst.L3b.nc get deleted because:

Code: Select all

total_filled_bins: 0
rm -f MOD00.P2024177.1600_1.PDS.sst.L3b.nc*

Failed: 'l2bin prodtype=regional ofile=MOD00.P2024177.1600_1.PDS.sst.L3b.nc infile=MOD00.P2024177.1600_1.PDS.sst.L2.nc resolution=Q average=area suite=SST l3bprod=sst', error 28160

This is working on Ubuntu 18.04 using OCSSW git branch v7.5
On Ubuntu 22.04, I installed v7.5 installed in a separate directories and it works for MODIS EXCEPT the generation of SST files fails. I installed V2024.0 to work around VIIRS processing issues, but that's a different problem. I tried to use V2024.0 for MODIS hoping it might solve the total_filled_bins: 0 problem.

[OB ODPS - towens]

One thing I noticed:

Code: Select all

Failed: 'l2bin prodtype=regional ofile=MOD00.P2024177.1600_1.PDS.sst.L3b.nc infile=MOD00.P2024177.1600_1.PDS.sst.L2.nc resolution=Q average=area suite=SST l3bprod=sst', error 28160

average=area is not an l2bin option

You can apply interp=area to map the L3b to L3m

Tommy

[woodbri]

Actually, average=area is a valid option on git branch v7.5 for l2bin 5.0.0
but not in tag V2024.0 l2bin 7.0.7

I had been running modis files using v7.5 but something broke after doing LUT updates. I load V2024.0 to deal is VIIRS isses and thought I would try to use it for MODIS but it dies as stated above.