Page 1 of 2
SeaDAS new l2gen very slow
Posted: Wed May 07, 2025 8:33 am America/New_York
by melinfr
Dear support,
I am experiencing something that's never happened before, i.e., l2gen is now excruciatingly slow (with very little CPU capacity used, ~1%). This has happened since I installed V2025.1, while the previously installed l2gen, associated with V2023.3, is running fine (with close to 100% CPU). I noticed that the new l2gen versions are 30% bigger but this is not an explanation. Would you see a reason for this? This is deeply worrying as my current installation is useless. Thanks a lot in advance for any help,
Best, F.
Re: SeaDAS new l2gen very slow
Posted: Wed May 07, 2025 9:48 am America/New_York
by OB SeaDAS - knowles
Can you let's us know what mission you are processing?
Also if you run l2gen from the command line, is it also at the command line?
Re: SeaDAS new l2gen very slow
Posted: Thu May 08, 2025 5:08 am America/New_York
by melinfr
Dear support,
thanks a lot for the reply! I am using l2gen exclusively from the command line (with default options), and so far I've noticed problems with Sentinel-3A and MODIS-A. I have seen that l2gen has taken some weight between the 2 versions. However, the screen outputs appear to be the same (so it does not seem that the new version does something weird). Does the new version access resources on the internet (like your web site) while processing?
Thanks for any suggestion,
Best, F.
Re: SeaDAS new l2gen very slow
Posted: Fri May 16, 2025 12:18 pm America/New_York
by OB SeaDAS - dshea
What system are you running on?
l2gen does not get anything from the network.
don
Re: SeaDAS new l2gen very slow
Posted: Fri Jun 20, 2025 9:00 am America/New_York
by melinfr
Dear Support,
sorry for the slow reply. Our machines have Xeon CPUs running RHEL 8.10 as OS. When I source SeaDAS to V2023.3, l2gen goes fast, while if it is V2025.1, it is very slow. Do you have an idea of why this could happen?
Thnaks,
Best, F.
Re: SeaDAS new l2gen very slow
Posted: Tue Jul 22, 2025 11:37 am America/New_York
by melinfr
Dear Support,
I am still looking desperately for a reason for the decrease in speed for l2gen between V2023 and 2024-2025. For instance the processing of a S3A large image takes 8 times longer. I am using only default options, and the only difference I've seen is the addition of absaer_opt=0 is the more recent defaults, but I don't see how this could affect speed. Do you see any reason why l2gen could have become much slower after 2023 ? Thanks a lot, Best, F.
Re: SeaDAS new l2gen very slow
Posted: Wed Jul 23, 2025 10:05 am America/New_York
by melinfr
Dear Support,
to continue testing, could you please update me on which OS versions are recommended to run V2025 ? I see references in some help pages but they seem to point to rather old releases of Ubuntu or CentOS .
Thanks a lot!
Best, f.
Re: SeaDAS new l2gen very slow
Posted: Wed Jul 23, 2025 11:45 am America/New_York
by OB ScienceSW - alex
Could you please provide the file name and explain exactly how you run the code?
Re: SeaDAS new l2gen very slow
Posted: Thu Jul 24, 2025 5:10 am America/New_York
by melinfr
Dear Support,
the example I'm using (processing a S3A image) is
l2gen par=./l2gen.par
(l2gen Version 9.11.0-V2025.1 )
with l2gen.par containing:
ifile=S3A_OL_1_ERR____20170516T112404_20170516T120828_20180420T162232_2664_017_351______LR2_R_NT_002_x/xfdumanifest.xml
ofile1=./O2017136112404.L2_S3A.nc
l2prod1=aot_443 aot_865 angstrom_510 Rrs_400 Rrs_412 Rrs_443 Rrs_490 Rrs_510 Rrs_560 Rrs_620 Rrs_665 Rrs_674 Rrs_681 Rrs_709 chlor_a Kd_490 Kd_412_lee Kd_490_lee Zeu_lee Zsd_lee bbp_443_qaa bbp_560_qaa a_412_qaa a_490_qaa a_560_qaa aph_443_qaa aph_490_qaa adg_412_qaa adg_443_qaa bbp_443_giop bbp_560_giop a_412_giop a_490_giop
a_560_giop chl_giop aph_443_giop aph_490_giop adg_412_giop adg_443_giop l2_flags
met1=/oc10/data/Meteorological/2017/136/GMAO_MERRA2.20170516T110000.MET.nc
met2=/oc10/data/Meteorological/2017/136/GMAO_MERRA2.20170516T120000.MET.nc
met3=/oc10/data/Meteorological/2017/136/GMAO_MERRA2.20170516T130000.MET.nc
ozone1=/oc10/data/Meteorological/2017/136/GMAO_MERRA2.20170516T110000.MET.nc
ozone2=/oc10/data/Meteorological/2017/136/GMAO_MERRA2.20170516T120000.MET.nc
ozone3=/oc10/data/Meteorological/2017/136/GMAO_MERRA2.20170516T130000.MET.nc
sstfile=/oc10/data/Meteorological/2017/136/20170516120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
With V2025, it takes 2 hours, versus less than 20' with V2023. Please notice that this is not only true for this file or S3A; this change of speed also affects S3B or MODIS-A. The output on the command line looks the same and the resulting L2 file is similar to the file I'm getting with V2023. Just the speed is different.... I am afraid that it be associated with the Operating System.
Thanks a lot for your help,
Best,
F.
Re: SeaDAS new l2gen very slow
Posted: Thu Jul 24, 2025 4:00 pm America/New_York
by OB ScienceSW - alex
I ran l2gen 9.11.0-V2025.1 (Jan 30 2025 23:47:26) on a Red Hat Enterprise Linux release 8.5 (Ootpa) with AMD EPYC 7352 24-Core Processor. CPU usage was close to 100% all the time and the RSS was around 4.3 Gb
I used the same input file and the same input parameter as you did. I ran the code directly from a shell
Code: Select all
[avsemeno@poseidon S3A_OL_1_ERR____20170516T112404_20170516T120828_20180420T162232_2664_017_351______LR2_R_NT_002.SEN3]$ l2gen par=par.par
Below is a part of the code output:
Loading default parameters from /home/avsemeno/ocssw/share/common/msl12_defaults.par
Found OLCI Sentinel-3A product: SENTINEL-3 OLCI Level 1 Earth Observation Reduced Resolution Product
Loading characteristics for OLCIS3A
Opening sensor information file /home/avsemeno/ocssw/share/olci/s3a/msl12_sensor_info.dat
......
Loading Raman coefficients from: /home/avsemeno/ocssw/share/olci/s3a/raman.nc.
Reading aph* from /home/avsemeno/ocssw/share/common/aph_bricaud_1998.txt.
Processing scan # 100 (101 of 15139) after 8 seconds
Processing scan # 150 (151 of 15139) after 11 seconds
Processing scan # 200 (201 of 15139) after 13 seconds
....
Processing scan # 14900 (14901 of 15139) after 2180 seconds
Processing scan # 14950 (14951 of 15139) after 2186 seconds
Processing scan # 15000 (15001 of 15139) after 2192 seconds
Processing scan # 15050 (15051 of 15139) after 2198 seconds
Processing scan # 15100 (15101 of 15139) after 2204 seconds
Percentage of pixels flagged:
Flag # 1: ATMFAIL 2735 0.0148
Flag # 2: LAND 827577 4.4918
Flag # 3: PRODWARN 2082 0.0113
Flag # 4: HIGLINT 1754838 9.5247
Flag # 5: HILT 557016 3.0233
Flag # 6: HISATZEN 0 0.0000
Flag # 7: COASTZ 249059 1.3518
Flag # 8: SPARE 0 0.0000
Flag # 9: STRAYLIGHT 2219583 12.0471
Flag #10: CLDICE 13012473 70.6272
Flag #11: COCCOLITH 75641 0.4106
Flag #12: TURBIDW 152959 0.8302
Flag #13: HISOLZEN 2942187 15.9692
Flag #14: SPARE 0 0.0000
Flag #15: LOWLW 83150 0.4513
Flag #16: CHLFAIL 152767 0.8292
Flag #17: NAVWARN 0 0.0000
Flag #18: ABSAER 0 0.0000
Flag #19: SPARE 0 0.0000
Flag #20: MAXAERITER 172137 0.9343
Flag #21: MODGLINT 4338970 23.5504
Flag #22: CHLWARN 55 0.0003
Flag #23: ATMWARN 245552 1.3328
Flag #24: SPARE 18637 0.1012
Flag #25: SEAICE 3386083 18.3785
Flag #26: NAVFAIL 657693 3.5697
Flag #27: FILTER 0 0.0000
Flag #28: SPARE 0 0.0000
Flag #29: BOWTIEDEL 0 0.0000
Flag #30: HIPOL 0 0.0000
Flag #31: PRODFAIL 14547791 78.9604
Flag #32: SPARE 0 0.0000
End MSl12 processing at 2025205195346000
Processing Rate = 6.859538 scans/sec
Processing Completed
It took 2204s or 37 minutes. It's not 20 minutes but not 2 hours either. Could you run the code from a shell?
Thanks