How to optimize processing many L1 files into GeoTIFFS (l3mapgen)?

[ebradbury]

Hello,

I'm preparing to process several years of MODIS Aqua L1 data into high resolution (250m) true color GeoTIFF images for a visualization/mapping project. Before I process the whole dataset I want to make sure that I've optimized each step.

Here's how I'm currently generating an image:

modis_GEO A2021137180500.L1A_LAC.x.hdf
modis_L1B A2021137180500.L1A_LAC.x.hdf A2021137180500.GEO.x.hdf
l2gen ifile=A2021137180500.L1B_LAC.x.hdf geofile=A2021137180500.GEO.x.hdf ofile=l2gen_out.nc l2prod=rhos_nnn north=45.9 south=40.5 east=-63.0 west=-71.2 resolution=250
l2bin ifile=l2gen_out.nc ofile=l2bin_out.nc prodtype=regional latnorth=46 latsouth=40 lonwest=-72 loneast=-63 resolution=Q
l3mapgen ifile=l2bin_out.nc ofile=truecolor.tiff oformat=tiff product=rgb use_transparency=yes interp=bin resolution=250.0 projection=mercator trimNSEW=1 north=45.9 south=40.5 east=-63.0 west=-71.2

I've already taken a couple steps to optimize:
- Requested "extracted" data when ordering the data from the portal
- Using crop/trim options where available (l2gen, l2bin, l3mapgen)

Knowing that the end goal is to generate high resolution, cloud-free images, are there any parameters to the commands that I'm using that will speed things up?

Is there a tool or API that can estimate cloud cover from L1 (or pre-generated L3 products) such that I could skip processing days that are too cloudy? It seems like cloud cover data isn't available until after the l2gen step (time consuming).

Once I get my dataset and parameters figured out I'll wrap this into a multiprocessing python script.

Thanks for any help.

[OB SeaDAS - xuanyang02]

At this moment, l2gen cannot take subset options (north= south= east= west=) with resolution=250 or 500. If your ordered data is already extracted, you can just omit the subset options for l2gen. Otherwise, you need to do l1aextract_modis on the L1A files and create corresponding geo and L1B files as the input for l2gen. See this post -- viewtopic.php?t=3077&sid=ecf7eab77de183379b2a653b4489f90f#p11063

To save time, you can turn off atmosphere correction by setting atmocor=0 for l2gen.

To screen out cloudy days, you can try our L1/2 browser https://oceancolor.gsfc.nasa.gov/cgi/browse.pl?sen=amod
You can see the quasi true color image of the scene you are interested in by choosing MODIS Aqua 5/17/2021 and ChesapeakeBay, and then decide if it's too cloudy for your case.

SeaDAS-OCSSW also provides multilevel_processor on command line and in GUI -- https://seadas.gsfc.nasa.gov/help-8.2.0/processors/ProcessMultilevel_processor.html

[ebradbury]

Good to know re: l2gen bounds. I'll just omit it.

Is atmospheric correction relevant for SST or can it also be omitted in that case?

I figured that I could use the L1/L2 browser for manual filtering for clouds but was looking for a programmatic option. That will work though.

Thank you.

[OB SeaDAS - xuanyang02]

Atmospheric correction is relevant for SST.