Earthdata Forum

Hello,
I want to create a composite image from 17 years chl-a monthly binned (L3) data. For example, I want one image of June, which is composite of 2002-2018 June months. For this I have just install ocssw in my WSL (windows 10 and ubuntu 18..). For test, I tried to make a composite image of June from 2012-2016 as shown in video tutorial and as written in "l3bin" under help page. But it showed such error:
execution exception: java.io.IOException: l3bin failed with exit code 139.
Check log for more details.
HDF5-DIAG: Error detected in HDF5 (1.8.18) thread 0:
L3BIN 5.12 (Nov 30 2018 09:53:07)
  #000: /home/seadas/ocssw/opt/src/hdf5/hdf5-1.8.18/src/H5F.c line 439 in H5Fis_hdf5(): unable open file
5 input files
    major: File accessibilty
0 C:\Users\antu\seadasClientServerShared\A20121532012182.L3b_MO_CHL.nc
    minor: Not an HDF5 file
  #001: /home/seadas/ocssw/opt/src/hdf5/hdf5-1.8.18/src/H5Fint.c line 554 in H5F_is_hdf5(): unable to open file
    major: Low-level I/O
    minor: Unable to initialize object
  #002: /home/seadas/ocssw/opt/src/hdf5/hdf5-1.8.18/src/H5FD.c line 993 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
  #003: /home/seadas/ocssw/opt/src/hdf5/hdf5-1.8.18/src/H5FDsec2.c line 339 in H5FD_sec2_open(): unable to open file: name = 'C:\Users\antu\seadasClientServerShared\A20121532012182.L3b_MO_CHL.nc', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
    major: File accessibilty
    minor: Unable to open file
/mnt/c/Users/antu/seadasClientServerShared/ocssw/scripts/ocssw_runner: line 23:  1259 Segmentation fault      (core dumped) $*

In this case, for selecting "input files", I just marked and selected them all (don't create txt file before), "prob=chlor_a" , " loneast, lonwest, latnorth, latsouth= Default Values" "pversion= keep blank" "oformat= netcdf4".
After running, there created a "_inputFiles.lst" file and showed the above error.
In "_inputFiles.lst"
E:\Research\3. Onshore and offshore\Working\Composite CC\5 years\June\A20121532012182.L3b_MO_CHL.nc
E:\Research\3. Onshore and offshore\Working\Composite CC\5 years\June\A20131522013181.L3b_MO_CHL.nc
E:\Research\3. Onshore and offshore\Working\Composite CC\5 years\June\A20141522014181.L3b_MO_CHL.nc
E:\Research\3. Onshore and offshore\Working\Composite CC\5 years\June\A20151522015181.L3b_MO_CHL.nc
E:\Research\3. Onshore and offshore\Working\Composite CC\5 years\June\A20161532016182.L3b_MO_CHL.nc

Then, I tried in another way. I created txt file as showed in that video and as in the post suggested (https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=9904)
My "Text.txt" .... ( as in video)
A20121532012182.L3b_MO_CHL.nc
A20131522013181.L3b_MO_CHL.nc
A20141522014181.L3b_MO_CHL.nc
A20151522015181.L3b_MO_CHL.nc
A20161532016182.L3b_MO_CHL.nc

"Text.txt" .... ( as in post)
$ echo X | cat Text.txt -
A20121532012182.L3b_MO_CHL.nc
A20131522013181.L3b_MO_CHL.nc
A20141522014181.L3b_MO_CHL.nc
A20151522015181.L3b_MO_CHL.nc
A20161532016182.L3b_MO_CHL.ncX

In this case, for selecting "input files", I just selected the "Text.txt" file, "prob=chlor_a" , " loneast, lonwest, latnorth, latsouth= Default Values" "pversion= keep blank" "oformat= netcdf4".
after running it showed the error as below....
execution exception: java.io.IOException: l3bin failed with exit code 255.
Check log for more details.
L3BIN 5.12 (Nov 30 2018 09:53:07)
Input listing file: "" not found.
HDF5-DIAG: Error detected in HDF5 (1.8.18) thread 0:
  #000: /home/seadas/ocssw/opt/src/hdf5/hdf5-1.8.18/src/H5F.c line 435 in H5Fis_hdf5(): no file name specified
    major: Invalid arguments to routine
    minor: Out of range

How I can solve this problem?

Regards,
Antu

You have:
#003: /home/seadas/ocssw/opt/src/hdf5/hdf5-1.8.18/src/H5FDsec2.c line 339 in H5FD_sec2_open(): unable to open file: name = 'C:\Users\antu\seadasClientServerShared\A20121532012182.L3b_MO_CHL.nc', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
    major: File accessibilty
    minor: Unable to open file
/mnt/c/Users/antu/seadasClientServerShared/ocssw/scripts/ocssw_runner: line 23:  1259 Segmentation fault      (core dumped) $*

Linux does not use the Microsoft drive letter notation, so instead of "C:\Users\antu\seadasClientServerShared\A20121532012182.L3b_MO_CHL.nc" you need "/mnt/c/Users/antu/seadasClientServerShared/A20121532012182.L3b_MO_CHL.nc".  In the WSL terminal, try "ls /mnt/c/Users/antu/seadasClientServerShared" to check that the files can be found by the WSL linux system. 

You appear to have installed the OCSSW software outside WSL.   This may not work as expected, as portion of the filesystem used by WSL has some differences from the usual Windows filesystem.  I have only used installations to my WSL "home" directory.

You also have "E:\Research\3. Onshore and offshore\Working\Composite CC\5 years\June\A20121532012182.L3b_MO_CHL.nc".  Depending on the
WSL version and type of drive (external USB, network share, etc) you may need extra steps to make the files accessible to WSL.  You can use the "Search" tool at Microsoft Command Line to find postings relevant to the type of drive you are using for "E:".  The OCSSW software sometimes has problems when there are spaces in the file path so if you continue have problems you may want to replace  the spaces with underscores.

thanks gnwiii ,
You are correct. Linux has problem with "space issue", when I remove the space, it works.
And
after "ls" command in " /mnt/c/Users/antu/seadasClientServerShared" directory, the files showed up (as below)

antu@DESKTOP-7K5SEI6:/mnt/c/Users/Antu/seadasClientServerShared$ ls
A20121532012182.L3b_MO_CHL.nc           antu
A20131522013181.L3b_MO_CHL.nc           install_ocssw.py
A20141522014181.L3b_MO_CHL.nc           ocssw
A20151522015181.L3b_MO_CHL.nc           ocssw.db
A20161532016182.L3b_MO_CHL.nc           ocsswserver.config
A20161532016183.L3m_CU_chlor_a_9km.nc   productInfo.xml
'New Text Document.txt'                  seadas-ocsswserver.jar
_inputFiles.lst

But the problem is, when I run the "processing" 5 bin file into 1 file with filling up fields like this, "input files= _inputFiles.lst file", "prob=chlor_a" , " loneast, lonwest, latnorth, latsouth= Default Values" "pversion= keep blank" "oformat= netcdf4", it was taking so long time. I mean, after click "run" , I waited 9 hours for processing this 5 files. But it didnt show any output within 9 hours.

Then, I tried with only 2 bin files and focus on my study area i.e., "input files= _inputFiles.lst file", "prob=chlor_a" , " loneast= +93", "lonwest=+89", "latnorth=+23", "latsouth=+20"  "pversion= keep blank" "oformat= netcdf4". But this is also taking so long. I waited more than 1 hr, after click "run" and I dont find any output even I waited more longer.
After click "run", some codes are working and moving continuously in WSL bash, no result showed up. So, I stopped the server.. Such as...
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100

Why it is taking so long time?
If considering such two bin files and my small study area takes so much time, How much time will need to combine 17 bin files with little bigger study area. I am really getting confused. I need some help.

My laptop configuration:
HP core i3 , 2nd gen, 4 gb ram, windows 10 with WSL, 500gb,
Regards,
Antu

This type of code (below) is running repeatedly for 4 hours just for combining 2 bin files for my small study area ( 88-93 E, 20-23 N).
.
.
.
.
process status: -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Operation done successfully
process status: -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Operation done successfully
process status: -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Operation done successfully
process status: -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Operation done successfully
process status: -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Retrieved item name : -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS
Operation done successfully
process status: -100
Retrieved item name : -100
Operation done successfully
process status: -100
Operating on table PROCESS_TABLE  jobID = 1987149b25cf19447260ccfe82a4a220 searching for STATUS

After this, I stop the server and a massage popped up in seadas.
execution exception: java.io.IOException: l3bin failed with exit code -1.
Check log for more details.

Urgently need give me some solution.

Your machine has only 4G RAM, but the server configuration has "java -Xmx4G" which tells java it can use all of your 4G RAM.  On a system with 8G RAM, the "performance" screen in the task manager shows 7.5G used with the SeaDAS GUI, firefox (to post this response), and the ocsswserver running, so I'd guess that 8G is the minimum practical requirement to use the ocsswserver with WSL.

There is also the possibility that the ___location of your ocssw installation outside WSL is causing a problem.   If the ocssw software is working,
you may be able to do your task using the WSL linux command line directly.   If you can install native linux (e.g., in a dual boot configuration),  simple OCSSW processing should run nicely on your hardware. 

With your current configuration, the output from the server will include the command-lines.   You can start the server using:

java -Xmx4G -jar seadas-ocsswserver.jar ocsswserver.config 2>&1 | tee ossswserver.log


This will save a copy of the output to ossswserver.log.  In the log, look for lines that start with <ocssw_install_dir>/ocssw/scripts/ocssw_runner.   You can paste one of these lines into the WSL bash terminal to run the job without the memory overhead of the ocsswserver process.

Thank you very much Mr. gnwiii for letting me know about the ossswserver.log. I am trying it. Hope, it will work.
I think, I should reinstall ocssw in my home directory as I cant do anything about RAM.
Thanks again for altruistic response.