l2gen: valid l2prod values

[loicdtx]

Hi,

Where can I find a list of valid products that can be passed to l2gen (l2prod argument)? I'm particularly interested in FAI/AFAI ((Alternative) Floating Algae index ), hu, 2009; Wang and Hu, 2016. In case a product does not exist, is there a way to pass a formula to l2gen?

Thanks,
Loïc

[OB.DAACx - SeanBailey]

There is a program distributed with SeaDAS that does the trick:

get_product_info 2.0 (Jul 14 2017 15:57:00)
Program to list products or detailed information about a single product
Usage: get_product_info [option=val] <productName>
Options:
   help (boolean) (alias=h) (default=false) = print usage information
   version (boolean) (default=false) = print the version
        information
   dump_options (boolean) (default=false) = print
        information about each option
   dump_options_paramfile (ofile) = print
        information about each option to paramfile
   dump_options_xmlfile (ofile) = print
        information about each option to XML file
   par (ifile) = input parameter file
   l (boolean) (default=false) = list all of the products
   r (boolean) (default=false) = list all of the products recursing through
        the wavelengths of the sensor specified
   sensor (string) (default=MODISA) = sensor name (or ID) to use for wavelength
        expansion or product lookup
   <productName> product to print detailed information about

But, no, Chuanmin's FAI is not a product l2gen produces.  It *could*, but no one has yet coded it up.
That said, it's a pretty simple algorithm and could be done using SeaDAS's "Band Math" capability....you;d just have to generate an L2 file with the appropriate inputs (e.g. rhos_[645,859,1240]

Sean

[loicdtx]

Thanks a lot for the pointers Sean! I ended up doing as below (random viirs file for reproducible example; not yet validated). Suggestions are very welcome.


wget https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/V2017241160000.L1A_SNPP.nc
wget https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/V2017241160000.GEO-M_SNPP.nc

l2gen ifile=V2017241160000.L1A_SNPP.nc geofile=V2017241160000.GEO-M_SNPP.nc ofile=V2017241160000.L2_SNPP_FAI.nc l2prod=rhos_nnn



# In python
import netCDF4 as nc

filename = '/path/to/file/V2017241160000.L2_SNPP_FAI.nc'

def l2_append(x, input_bands, formula, short_name, long_name, standard_name):
    """Compute a new array and append it to an existing OBPG L2 file

    Args:
        x (str): Input L2 file in netCDF format
        input_bands (list): List of strings corresponding to the names of the
            netCDF dataset variables used to compute the new array. They must
            be provided in the same order than used in the formula argument
        formula (func): The function used to compute the new variable. Each argument
            must be an array, and len(args) must equal len(input_bands)
        short_name (str): Short name for the newly create dataset (used as netCDF
            variable name)
        long_name (str): Name of the newly create dataset
        standard_name (str): Name of the newly create dataset

    Return:
        None: The function is used for its side effect of appending a new variable
            to an existing netCDF file.
    """
    with nc.Dataset(x, 'a') as src:
        geo = src['geophysical_data']
        newVar = geo.createVariable(short_name, 'f4', ('number_of_lines', 'pixels_per_line'))
        newVar.long_name = long_name
        newVar.standard_name = standard_name
        newVar[:] = formula(*[geo[x][:] for x in input_bands])

def afai_viirs(red, nir, swir):
    nir_line = red + (swir - red) * (745 - 671) / (862 - 671)
    return nir - nir_line

l2_append(x=filename, input_bands=['rhos_671', 'rhos_745', 'rhos_862'],
          short_name='afai', long_name='Alternative floating algae index',
          standard_name='afai', formula=afai_viirs)


Somehow if I try to continue via the l2bin + l3mapgen route, I get a l3mapgen error: "-E- product afai not found in XML product table". Any idea what's happening there?


l2bin infile=V2017241160000.L2_SNPP_FAI.nc ofile=V2017241.L3b_SNPP_FAI.nc resolve=1 flaguse=CLDICE,LAND,HIGLINT l3bprod=afai
l3mapgen ifile=V2017241.L3b_SNPP_FAI.nc ofile=V2017241.L3m_SNPP_FAI.nc resolution=1km south=3 north=33 west=-122 east=-72 projection="+proj=laea +lat_0=18 +lon_0=-100" product=afai


Cheers,
Loïc

[gnwiii]

l3mapgen uses the XML product table ($OCSSWROOT/run/data/common/product.xml) so when you create a new product you need to add an entry to the product table.  See $OCSSWROOT/build/src/l2gen/HOWTO_Add_a_product.txt, which refers to "product.xsd".  The latter file is actually $OCSSWROOT/run/data/common/product-1.0.xsd.  You can choose a similar existing product entry to use as a template for the new product.

[OB.DAACx - SeanBailey]

Mostly correct.  The xsd file is the XML schema.  There is a corresponding product.xml file that is the one to edit to include information on the new product.

Here's an example of what you could add for this product:

    <product name="afai">
        <units>unitless</units>
        <category>Derived</category>
        <range>
            <validMin>-0.5</validMin> <!-- Change these as appropriate -->
            <validMax>5.0</validMax>
            <displayMin>0</displayMin>
            <displayMax>0.5</displayMax>
        </range>
        <algorithm>
            <cat_ix>-1</cat_ix><!-- The cat_ix is required by l2gen - Since you're not creating the product within l2gen, leave this alone -->
            <description>Floating Algal Index</description>
            <reference>A novel ocean color index to detect floating algae in the global oceans. C Hu - Remote Sensing of Environment, doi:10.1016/j.rse.2009.05.012, 2009</reference>
            </algorithm>
    </product>

Once you add the product.xml entry, the get_product_info program will know about it:
$ get_product_info afai
sensorName=HMODISA
fullProductName=afai
description=Floating Algal Index
units=unitless
palette=default
paramDesignator=none
paramWaveMin=-1
paramWaveMax=-1
standardName=(null)
category=Derived
dataType=float
prefix=afai
suffix=
algorithmName=
productName=afai
cat_ix=-1
prod_ix=-1
rank=2
fillValue=-32767
validMin=-0.5
validMax=5
displayScale=linear
displayMin=0
displayMax=0.5
scaleFactor=1
reference=A novel ocean color index to detect floating algae in the global oceans. C Hu - Remote Sensing of Environment, doi:10.1016/j.rse.2009.05.012, 2009
keywords=(null)
comment=(null)
titleFormat=Floating Algal Index

Regards,
Sean

[loicdtx]

Thanks for the help about l3mapgen. Does l2mapgen similarly refer to an external parameter file? I've been trying to reproject my user generated afai, with interpolation, to produce a "pretty image" but the values in the output geotiff are nothing like what is in the input L2 file.

Is there any other route than l2mapgen to produce a pretty georeferenced image (free of binning moire patterns).

Thanks,
Loïc

[gnwiii]

If you run "l2mapgen" without any arguments you get a usage summary.  You can supply your own product table using the argument list keyword "product_table". The default ($OCDATAROOT/common/smigen_product_table.dat) has comments that explain the fields.

[OB.DAACx - SeanBailey]

Loïc,

If you use the apply_pal option in l3mapgen, the output TIFF image will be scaled to match the indexed color palette used in making the pretty picture.
If you want to use the TIFF to represent geophysical values, set apply_pal=0 (the output will be grayscale, but the values will be floating point and match
the input) 

Sean