Bowtie question

[obdaac_forum_user]

Greetings all
I feel a bit silly asking this yet will do so anyway to try to ave myself a lot of work. 

I am interested in using MODIS-Aqua data to estimate and map chlorophyll-a from different bands as well as the chlor_a values already present.   The goal is to use these data to do things ranging from looking for spatial patterns to calculating mean values for entire Great Lakes.  I know about potential limitations (or not) of MODIS data in such waters and am not worried about those.  I feel I have different problems/worries.

My initial approaches were 1) NOT in SeaDAS but rather R.  My approach was limiting in that it started with L2 data and did not remedy for the MODIS bowtie in any way. The first image shows the L2 chlor_a displayed in SeaDAS.

From what I've read on the forum, I ought to be able to remedy the bowtie effect by 1) projecting the data, which I could do with SeaDAS, one file at a time, 2) reprojecting with GPT (if I could get it to work for multiple files), 3) reprojecting with SNAP (which lets one batch-reproject from the GUI), or 4) use l2bin followed by l3mapgen within shell scripts to batch process.  I've spent way too much time on this already and in some ways feel I know less now than I did a year ago.  But I have the 3rd and 4th approaches working so they win out.

So my questions are
1) Should I get the same Rrs and chlor_a values for a given pixel in a given scene for L2 data that I have reprojected and L3 data resulting from l2bin and l3mapgen?
2) Is that even the pertinent question?
3) If it is "best" to use the approach of l2bin then l3mapgen, is there some way to make l3mapgen process all bands of the l2bin result via shell script?  I seem to only be able to make it work on one band at a time on my Mac.  I can get l2bin to do all bands though. 
4) How ought I choose from "Preserve resolution" or not when using Reproject.  Is it better to resample all scenes in a time series from one place to the same origin/extent/resolution after reprjection or should I at least set a common resolution during reprojection?
5) If I am interested in taking the OCSSW approach and obtaining resolution of perhaps <2km, what should I choose for resolutions in l2bin and l3mapgen, respectively?

I know that was a lot.  Sorry.  Thanks to anyone who reads this and for your patience.  I was not sure this was the right board.
Dave Warner

[OB SeaDAS - knowles]

It would take a lot to answer all these permutations  to your questions, but here's a few of the questions:

1) Should I get the same Rrs and chlor_a values for a given pixel in a given scene for L2 data that I have reprojected and L3 data resulting from l2bin and l3mapgen?
No, for the first you are mapping directly to SMI (or whatever projection you choose), the second you are doing an integerized sinusoidal bin followed by a mapping to SMI (or whatever projection you choose).

3) If it is "best" to use the approach of l2bin then l3mapgen, is there some way to make l3mapgen process all bands of the l2bin result via shell script?  I seem to only be able to make it work on one band at a time on my Mac.  I can get l2bin to do all bands though.
Here's an example of how to do more than one band:
l2bin ifile=A2010283180500.L2_LAC_OC.nc ofile=A2010283180500.L3b_chlor_a_Rrs_443.nc l3bprod="chlor_a,Rrs_443"
l3mapgen ifile=A2010283180500.L3b_chlor_a_Rrs_443.nc ofile=A2010283180500.L3m_chlor_a_Rrs_443.nc product="chlor_a,Rrs_443"

5) If I am interested in taking the OCSSW approach and obtaining resolution of perhaps <2km, what should I choose for resolutions in l2bin and l3mapgen, respectively?
There's no single right answer.  The following is just a general suggestion:

l2bin (resolution=2) and l3mapgen (resolution=2 and interp=area)
Or the following variant will likely be similar but on a higher resolution raster (useful when visually overlaying higher resolution land masks)
l2bin (resolution=2) and l3mapgen (resolution=1 with fudge=2 and interp=area)

Here's an example:
l2bin ifile=A2010283180500.L2_LAC_OC.nc ofile=A2010283180500.L3b_chlor_a.nc l3bprod=chlor_a resolution=2 interp=area

Then either:
l3mapgen ifile=A2010283180500.L3b_chlor_a.nc ofile=A2010283180500.L3m_chlor_a__2km.nc product=chlor_a resolution=2km interp=area
or the variant:
l3mapgen ifile=A2010283180500.L3b_chlor_a.nc ofile=A2010283180500.L3m_chlor_a__1km_fudge2.nc product=chlor_a resolution=1km fudge=2 interp=area

Note fudge could be 1.5 or some other value.  This is all subjective depending on how many data holes you want to fill.

Danny

[obdaac_forum_user]

Thanks much Danny.  I know that for much of this, "how best to do this" depends on the question one is trying to ask with the data.

Perhaps I can ask here then what the scientific advantage is to L3 data relative to L2 (or vice versa).  I know at least one person whose expertise I trust uses only L2 data for matchups with in situ data. 

Is there a disadvantage to using mapped L2 as long as reasonable flag use is involved?  I've tried to figure this out from papers and NASA sources and I have nothing definitive yet. 

Again, thank you for helping my thinking on this set of topics.

Dave

[OB SeaDAS - knowles]

A single point in situ matchup (meaning where in situ measurement surface area is less than the satellite pixel) is better done directly on l2files and not a mapped l2file.  The mapped pixels are not unique and each mapped pixel derives from any number of level2 source pixels thus altering your statistics.  Depending on the circumstances, you could easily have 2 adjacent mapped pixels, each derived exclusively from a single source pixel.

Take a look at this paper:
Bailey, S.W., and Werdell, P.J. (2006). A multi-sensor approach for the on-orbit validation of ocean color satellite data products. Rem. Sens. Environ. 102, 12-23.

Danny

[obdaac_forum_user]

Thanks again Danny.  I started working on trying to amass a MODISA data set for my AOI several years ago.  I knew virtually nothing about programming.  I've really learned a lot and am getting better.

In R and managed to create code that does exactly what you suggested (matchups with unmapped L2 data) byreading in the L2 nc files and creating dataframes of the different bands and lat/long assuming each lat/long pair corresponded to a pixel so these could be chosen by spatial adjacency (within a buffer distance) to in situ data (in our case a LRAUV track of 1,000 km).  I did not worry about the bowtie issue as I was ignorant of the need. 

I am apparently still well down on the learning curve as I thought that I needed to remove bowtie artifact (scan overlap) prior to using the data in this way.  I don't see that done in the paper you mentioned, but that paper used SeaWiFS data, which don't have this issue (I don't think) https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=1320. 
Dave

[OB SeaDAS - knowles]

Yes, MODIS is a problem.  For MODIS with its 10 detector scans the bow tie effect is complicated such that adjacent satellite pixels are not necessarily next to each regarding their ground projections.  For MODIS, you would be best to bin the data (at 1km) and then map it if needed (at 1km).  You may get holes of missing data, but all the satellite data near your in situ point will be accounted for and included in your statistics. 

Danny

[obdaac_forum_user]

Thanks again Danny! 

Is the difference between binning and mapping just that in mapping, resampling (nearest, bilinear interpolation, etc.) takes place so ALL output pixels have values?

I prefer to think of MODIS as a challenge!  Heck, I have a challenge just posting and reading this forum!  I work for the US government and in my office I can't connect to oceancolor.gsfc.nasa.gov!  For some reason the site is blocked.  IT staff solution is use wifi or do it at home...:confused:

Dave

[OB SeaDAS - knowles]

Neither mapping nor binning will guarantee ALL output pixels have values.

Binning is essentially a specific and OBPG official form of mapping with very specific rules and file format.  For this binning methodology, at least currently, each source pixel gets put into precisely one and only one bin (provided it passes validation criteria such as all other products being binned also must have valid data for that source pixel).  Mapping can involve the source pixel getting spread out into multiple mapped pixels.

Also the bin file format is optimized and stores the data by bin number instead of storing a full raster of data.

For more on the specifics of our binning methodology see:
https://oceancolor.gsfc.nasa.gov/docs/format/l3bins/
and see:
"Level 3 SeaWiFS Data Products: Spatial and Temporal Binning Algorithms" J.W.Campbell, J.M. Blaisdell, and M.Darzi , NASA Technical Memorandum 104566, Vol.32.

Danny