Processing using l2gen: several questions

[madjidh]

Hello,

I am using l2gen to generate L2 files for some specific variables. An example of my linux command line code:

l2gen ifile=$L1BFILE geofile=$GEOFILE ofile=$L2FILE \ l2prod1='senz,l2_flags,chl_gsm,chlor_a' \

1) When I use ncdump / ncdisp on my freshly created file, I can see the line:

"mask_names = 'ATMFAIL,LAND,CLDICE,HILT'.

However, when I process files at L3b, I use:

echo "l2bin : Processing $L3BINFILE to Level 3 binned.."
l2bin ifile=$L2FILE ofile=$L3BINFILE \
resolve=1 \
flaguse=ATMFAIL \

using only the ATMFAIL flag.
Does it mean that the l2 files used to generate the L3bfile applied HILT/CLDICE prior to merging, or will the L3b file only apply the ATM FAIL flag?

2) Second question, there is a threshold on the solar / sensor zenith angle when applying standard atmospheric correction, set to

sunzen = '70.000'
satzen = '60.000'

But I still have access to the information above 70° for the solar zenith angle for example. Is it really applying it, or do I have to activate it somewhere, like using the HISOLZEN?

The code I run, on my Linux:

##########################################################################
################ START THE PROCESSING FROM L1A to L3m ####################
##########################################################################
echo "Start processing from L1A to L3m.."
cd $fullpath # Go to the folder where you need to process

################ START THE PROCESSING FROM L1A to GEO #####################
# Process the L1A subscene files to GEO
echo "modis_GEO : Processing L1A to GEO.."
modis_GEO.py $FILE -o $GEOFILE
echo "modis_GEO Done!"

################ START THE PROCESSING FROM L1A to L1B #####################
echo "modis_L1B : Processing Level 1A to Level 1B.."
# Process the L1A/GEO subscene files to L1B
modis_L1B.py $FILE $GEOFILE -o $L1BFILE --del-hkm --del-qkm
# Determine ancillary data
# Create a file in l2gen's par file format called L1BFILE.anc
getanc.py $L1BFILE
echo "modis_L1B Done!"

################ START PROCESSING FROM L1B to L2A #####################
echo "l2gen : Processing $L1BFILE to Level 2.."
l2gen ifile=$L1BFILE geofile=$GEOFILE ofile=$L2FILE \ l2prod1='senz,l2_flags,chl_gsm,chlor_a' \
echo "l2gen Done!"

################ START PROCESSING FROM L2A to L3BIN #####################
echo "l2bin : Processing $L3BINFILE to Level 3 binned.."
l2bin ifile=$L2FILE ofile=$L3BINFILE \
resolve=1 \
flaguse=ATMFAIL \
echo "l2bin Done!"

################ START PROCESSING FROM L3BIN to L3MAP #####################
echo "l3mapgen : Processing $L3BINFILE to Level 3 mapped.."
l3mapgen ifile=$L3BINFILE ofile=$L3MAPFILE \
resolution=1km \
interp=area \
north=65 \
south=48 \
east=13 \
west=-25 \
fudge=3 \

Edit: updated, found my previous issues. Only 2 questions remaining

[OB.DAACx - SeanBailey]

The mask_names attribute in the L2 metadata indicate which flags were masked during L2 processing.
The flaguse parameter in l2bin indicates which flags to use when excluding data from binning. If a flag was set as a mask at L2, pixels with that flag set will be fill data, and if binned, will be excluded (we don't bin fill data) even if you don't set those flags in the flaguse parameter.

The satzen and solzen parameters in l2gen define the threshold for setting the HISATZEN and HISOLZEN flags respectively. If you don't pass those flags to l2bin, the data flagged as HISATZEN and HISOLZEN will be binned (unless flagged for another reason that *is* excluded).

Regards,
Sean

[madjidh]

Thanks for the previous reply Sean, but I am sorry to say I am still unsure what masks have been applied to my Level 3 output. My processing steps are: After using L2GEN my L2 product shows "CLDICE" and "HILT" . I then use L2BIN and I apply "ATM FAIL" to mask data where AC fails. Does my final product have only had the ATM FAIL mask applied, or have the CLDICE and HILT masks also been applied by L2GEN by default?

If the answer is that CLDICE and HILT have been applied to the L3 product, is there an option to give to L2GEN not to apply it ?

Best

[OB.DAACx - SeanBailey]

By default, l2gen masks pixels flagged as CLDICE or HILT (well, depending on the sensor for HILT). This is controlled by the parameters:
maskland (boolean) (default=on) = land mask option
maskbath (boolean) (default=off) = shallow water mask option
maskcloud (boolean) (default=on) = cloud mask option
maskglint (boolean) (default=off) = glint mask option
masksunzen (boolean) (default=off) = large sun zenith angle mask option
masksatzen (boolean) (default=off) = large satellite zenith angle mask option
maskhilt (boolean) (default=on) = high Lt masking
maskstlight (boolean) (default=on) = stray light masking

There are sensor specific default files that change the above (and in the case of maskstlight, the l2gen common default sets it off, even though the code by default enables it...)

These control whether or not l2gen will process atmospheric correction (and any impacted derived products). So, by default, the code will not run the atmospheric correction on pixels flagged as CLDICE or HILT. Some products will be available without atmospheric correction (e.g. Lt, rhot, rhos, ...). These will still have valid values. Any product requiring atmospheric correction will have fill values. The binner will exclude fill value data. SO, you don't need to set the flaguse to exclude CLDICE, if you're binning those products (for pixels masked at L2 there is no data to bin). If you want derived products under clouds or high Lt conditions, unset the maskcloud and maskhilt options in l2gen. The atmospheric correction is likely to fail anyway, so the data will be invalid and thus not binned, but you can try...
If you're wanting to bin data that do not require atmospheric correction, then you'll want to NOT set those flags in the l2bin flaguse parameter.

Keep in mind, l2bin is an all or nothing process. Every product selected to bin together must have valid data for a bin to be filled. For example, if you are binning the PAR product (par) with the chlorophyll product (chlor_a), you'll only get PAR where there was a successful chlororophyll retrieval. Even though PAR doesn't need atmospheric correction, if you combine it in the same bin process you will not get par where the atmospheric correction failed (or wasn't attempted - i.e. maskcloud/maskhilt were set in l2gen)

Sean

[madjidh]

Thank you very much Sean for the explanations !