Earthdata Forum

Hello,

I am trying to generate my own climatology file for the years 2002-2010.
I downloaded the summer months for the years in question. I organized the data as follows:
2002 - summer months data
2003 - summer months data
2004 - summer months data
etc etc
I have already processed the data through l3bin and l3mapgen and have collocated my files into one big file.
My plan was to use the math expression function to add 2002 + 2003 ..... + 2010 and then divide by 9. To get the mean. Right?

I tried it, and it appears to work...but I'm not sure sure about the climatology that is generated from that. I have my doubts about its accuracy. Could I be doing something wrong?

When I use my generated climatology to calculate anomaly, I do the following:

observed - climatology = anomaly for that time frame

And the resulting image simply cannot be right.
Any help is much appreciated. I'm loving the program so far, have generated some really cool images for 2011, 2015, and 2018 but that was using the climatology on the ocean color website. I am trying to show deviation from before 2011.

Thank you!
B

Your use of the math expression is incorrect.  You cannot just sum the values for a group of bands for a given pixel and then divide by the number of bands because some bands may likely have NaN values for that pixel, hence the number you divide by would be pixel dependent.  The correct math expression would be huge and ugly.  The Math expression tool has it's place and use but it is not a good tool to use for aggregating files.  The primary tools for aggregating a lot of files are l3bin and the Mosaic Tool.

Since you are starting with level3 bin files the best technique to achieve what you are trying to accomplish is:
1. Create a level3 bin file for each summer by running l3bin on the set of bin files for each year's summer months
2. Create your "climatology" bin file by then running l3bin on the complete set of your summer bin files (the output files of step 1)
3. Run l3mapgen 

Danny

Alright, I have created the level3 bin file for each summer by running l3bin on the set of files for each year's summer months. Now, I'm trying to complete step 2, but I'm a little confused. Do I use another text file to unite the years I'm interested in using for my climatology? How do I run l3bin on a complete set of output files?
Also, I just wanted to say, this software is amazing. I'm still figuring it all out but it runs extremely smoothly on my Mac.

Bianca

Many people find these workflows confusing so a bit of explanation may help you and hopefully others who encounter this thread.     Computers are very fussy about little details.   A general overview of the processing workflow may help distinguish between fussy details (formatting input text files on MacOS) as opposed to those caused by trying to do something the software wasn't designed to handle (e.g., hoping to view binned files in the SeaDAS GUI or using band maths to compute climatologies).

Level-2 files are difficult to use.  Although the data are stored in an array, the locations of the individual pixels are obtained from relatively complicated calculations and vary with each level-2 file.   Binning combines data that originated from multiple level-2 files into a common spatial structure (but not an array).  Each "bin" represents a region with that contains statistics for data in a certain range of longitudes and latitudes.   The bins are defined by dividing the earth's sea-level surface into a fixed number of bands, and then dividing up each latitude band into bins.  The "height" (latitude difference) of each bin is the same, while the width varies slightly in order to have an integer number of bins within each latitude band.  L2bin reads level-2 data from multiple files and saves  statistics (counts, sum, and sum-of-squares) for all the level-2 pixels that fall in each bin.   L3bin combines binned files by combining the statistics from all the input files for each bin.  It is simpler and faster than l2bin since the input files all have a common structure.    L3mapgen uses the statistics stored in a binned file to compute means/averages, variances, etc. as well as mapping the binned data into a 2-D array suitable for viewing.

Hopefully this is enough to allow you to answer your own question.

Here's a specific example which creates an aggregated "climatology" file for 2003-2005 using the June, July and August level3 monthly bin files as input.

Step 1a:
l3bin ifile=2003_monthlies.txt ofile=A20031522003243.L3b_SNSU_CHL.nc
where 2003_monthlies.txt:
A20031522003181.L3b_MO_CHL.nc
A20031822003212.L3b_MO_CHL.nc
A20032132003243.L3b_MO_CHL.nc

Step 1b:
l3bin ifile=2004_monthlies.txt ofile=A20041532004244.L3b_SNSU_CHL.nc
where 2004_monthlies.txt:
A20041532004182.L3b_MO_CHL.nc
A20041832004213.L3b_MO_CHL.nc
A20042142004244.L3b_MO_CHL.nc

Step 1c:
l3bin ifile=2005_monthlies.txt ofile=A20051522005243.L3b_SNSU_CHL.nc
where 2005_monthlies.txt:
A20051522005181.L3b_MO_CHL.nc
A20051822005212.L3b_MO_CHL.nc
A20052132005243.L3b_MO_CHL.nc

Step 2:
l3bin ifile=2003_2005_summers.txt ofile=A20031522005243.L3b_SCSU_CHL.nc
where 2003_2005_summers.txt:
A20031522003243.L3b_SNSU_CHL.nc
A20041532004244.L3b_SNSU_CHL.nc
A20051522005243.L3b_SNSU_CHL.nc

Note that you cannot actually make a "true" summer aggregated level3 file if your input is limited to monthlies.  For instance, the span of days when using June, July and August Monthly files for 2003 is from day 152 (June 1) to day 243 (August 31).  If you were to download the OBPG summer seasonal level3 file (A20031722003263.L3b_SNSU_CHL.nc) this actually spans from day 172 to day 263 (which is from June 21 summer solstice to September 20 autumn equinox).  In order to make a "true" summer file you could use the daily level3 files as your input to step 1.

Danny

Okay that's sort of what I was envisioning. I am trying to make a "climatology" for 2002-2010 so that I can then look for anomalies from 2011-2018. So I l3binned the summer months as you show, and that worked. So I tried to make a text document that looks as follows:

A20021852002244.L3b_CU_OC
A20031522003244.L3b_CU_OC
A20041532004245.L3b_CU_OC
A20051522005244.L3b_CU_OC
A20061522006244.L3b_CU_OC
A20071522007244.L3b_CU_OC
A20081532008245.L3b_CU_OC
A20091522009244.L3b_CU_OC
A20101522010244.L3b_CU_OC

(because those are the output results of the l3bin from step 1).

You're suggesting that I run that in l3bin, then l3mapgen, and then THAT output will represent a rough climatology for that time frame?

I am trying to run the files in l3bin, and I keep getting an error. I have attached an image of my files and the respective text file I am trying to use. Thank you!
B

Are you able to see my screenshots? I cannot seem to see them but I have tried to post them.
Bianca

/Users/biancabroman/Desktop/Screen Shot 2019-03-04 at 4.11.29 PM.png
/Users/biancabroman/Desktop/Screen Shot 2019-03-04 at 4.11.33 PM.png

The images are not visible.   If you run the command in a terminal then you can paste the output as text directly into a post.   It may be useful to add verbose=on to the l3bin command line.

What's the error?   To rule out a problem with your text file (which is a list of files) being incorrectly formatted you can instead try a single file to see if it works. 

Also the file name /Users/biancabroman/Desktop/Screen Shot 2019-03-04 at 4.11.29 PM.png contains spaces, perhaps this is causing your file upload problem.

Danny