l3bin fails with error code -11 when there are 475 input files

[efujioka]

I'm trying to generate climatology layers (e.e. January mean / median of chl-a over 2020-2024) using MODIS Aqua, Sentinel-3 and VIIRS NetCDFs downloaded using Python earthaccess library. My workflow is l2bin -> l3in -> l3mapgen.
l3bin works fine if there are around 200 input files generated by l2bin. But if there are 475 files, it fails with error code -11. l3bin's output looks like:

L3BIN 5.14 (May 4 2025 16:57:23)
475 input files
0 .../AQUA_MODIS.20200101T170000.L2.CHLA.L3b
1 .../AQUA_MODIS.20200101T170500.L2.CHLA.L3b
...
474 .../AQUA_MODIS.20240131T192000.L2.CHLA.L3b
irow: 0 of 4320 Fri Aug 22 09:24:58 2025

I could split the input files into chunks and merge them at the final step, which was successful. But the reason I need to process such a large number of files at once is to calculate median not mean (l3bin median=1).

Here is an excerpt of the Python code:

l3bin_bin = "l3bin"
final_list_file = "...."
l3b_output = "...L3b"
product = "chlor_a"
resolution = "4"
cmd = [
l3bin_bin,
f"ifile={final_list_file}",
f"ofile={l3b_output}",
f"prod={product}",
f"resolve={resolution}",
#"median=1", # Use median binning
"verbose=1"
]
result = subprocess.run(cmd, capture_output=True, text=True)


OSCCW tools are installed on a Ubuntu server with 40GB RAM. So, physical RAM should be large enough.
I wonder if l3bin itself has a limit regarding the number of files it can process at once?
Any workaround or alternative approach?
SeaDAS GUI Raster -> Geometric -> L3-Level Binning can process 475 files. So, I also wonder what the differences are between SeaDAS and l3bin.

SeaDAS 9.2.0
l2bin 8.0.2
l3bin 5.14
l3mapgen 2.4.0-T2025.8

Thanks for your help!

Ei

[OB.DAAC - SeanBailey]

The l3bin code is some old code and sets up a number of static arrays with what is normally enough space to do its work...and as such it has a hard-coded file limit of 256.
Line 26 of l3bin.cpp :

Code: Select all

#define MAXNFILES 256

I suppose it could be recompiled setting that #define to 512...it should work...

(BTW, it's not really C++...the core code is really just old C that has had a minor cosmetic makeover as C++)

Sean

[OB SeaDAS - xuanyang02]

SeaDAS is a comprehensive software package for the processing, display, analysis, and quality control of remote-sensing Earth data - https://seadas.gsfc.nasa.gov/help-9.2.0/general/overview/SeadasOverview.html#GeneralOverview

You can find tutorial materials here - https://seadas.gsfc.nasa.gov/tutorials/ and an introduction video to SeaDAS - https://www.youtube.com/watchv=GZG2saE9ecc&list=PLf60TttfDm30SjrRVBSXptm0tGRax5fKq

SeaDAS includes general tools to visualize and analyze remote sensing data and science processors to process data. l3bin is one of the science processors. You can find help here - https://seadas.gsfc.nasa.gov/help/