SeaDAS 7.5 update_luts.py is broken

[liamgumley]

I am attempting a new SeaDAS 7.5 install on 64-bit CentOS 7.7 today. This is the install command:

curl -O https://oceandata.sci.gsfc.nasa.gov/ocssw/install_ocssw.py
chmod ugo+x install_ocssw.py
./install_ocssw.py --verbose --install-dir=$PWD/ocssw --arch=linux_64 --terra --aqua --viirsn --viirsj1 --direct-broadcast

The install is failing at the point where it tries to download Aqua LUTS as shown below:

Installing aqua-luts (14 of 17)
Traceback (most recent call last):
  File "/home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/scripts/update_luts.py", line 64, in <module>
    luts.get_luts()
  File "/home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/scripts/modules/LutUtils.py", line 134, in get_luts
    check_times=True)
  File "/home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/scripts/modules/JsonUtils.py", line 203, in download_allfiles
    all_links = self.get_links(url, regex=regex)
  File "/home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/scripts/modules/JsonUtils.py", line 170, in get_links
    with session.get(url, stream=True, timeout=self.timeout) as response:
AttributeError: __exit__
Error - Could not install luts for aqua

Note the same error is seen if I try to download the Aqua LUTS using update_luts.py as shown below:

[gumley@bolt test]$ export OCSSWROOT=$PWD/ocssw
[gumley@bolt test]$ source $OCSSWROOT/OCSSW_bash.env
[gumley@bolt test]$ update_luts.py -v aqua

Downloading files into /home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/var/common
OBPG session started
Traceback (most recent call last):
  File "/home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/scripts/update_luts.py", line 64, in <module>
    luts.get_luts()
  File "/home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/scripts/modules/LutUtils.py", line 134, in get_luts
    check_times=True)
  File "/home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/scripts/modules/JsonUtils.py", line 203, in download_allfiles
    all_links = self.get_links(url, regex=regex)
  File "/home/gumley/dbvm/ocssw_build/tarfiles/test/ocssw/scripts/modules/JsonUtils.py", line 170, in get_links
    with session.get(url, stream=True, timeout=self.timeout) as response:
AttributeError: __exit__

[OB.DAACx - SeanBailey]

Have you installed the python requests library?

[liamgumley]

Yes.

[OB.DAACx - SeanBailey]

Liam,
Before I released the code, I made sure it worked on our CentOS 7.7 machine.  ...used the system installed python and yum installed requests library.
What version of python are you running?

Sean

[gnwiii]

I have been using the "-b v7.5" option with install_ocssw.py.  To check your installed  version:
$ cd $OCSSWROOT/scripts
$ git status
On branch v7.5
Your branch is up to date with 'origin/v7.5'.

[liamgumley]

python --version
Python 2.7.5

sudo yum install python-requests | grep python-requests
Package python-requests-2.6.0-8.el7_7.noarch already installed and latest version

[OB.DAACx - SeanBailey]

Liam,

I have no clue.  Here's what I get:


$ cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)

$ python --version
Python 2.7.5

$ yum list python-requests
...
Installed Packages
python-requests.noarch                                  2.6.0-8.el7_7                                   @updates

$ install_ocssw.py --verbose --install-dir=/mnt/beegfs/poseidon/sbailey/ocssw-test --arch=linux_64 --terra --aqua --viirsn --viirsj1
....
Checking Python version                                                                               
Python version acceptable                                                           
Installing aqua-luts (14 of 17)                   
Installing terra-luts (15 of 17)                                                  
Installing viirsnpp-luts (16 of 17)                                                               
Installing viirsj1-luts (17 of 17)


I hate saying "works for me", but...it works for me...

Sean

EDIT:  BTW, it's just the update_luts.py that failed for you, so the rest of ocssw should be fine.  Maybe there is a network issue...
Oh, and --direct-broadcast just forces --aqua --terra and --arch is only necessary if for some reason the script can't determine your architecture...less of a problem these days since we're down to macOS and linux64...

[liamgumley]

Do I need to have something in place for NASA Earthdata authentication in order to download the LUTS?

[OB.DAACx - SeanBailey]

No, you shouldn't need that as the update_luts.py does not access any data that require authentication.
You'll need it for the ancillary files (including MODIS att/eph files) retrieved via getanc.py or modis_atteph.py, but not for the LUTs.

Sean

[OB.DAACx - SeanBailey]

Liam,

Since I was baffled by the discrepancy between my run and yours, I followed the definition of insanity approach and repeated my steps to see if I got a different result.  Nope.  The install_ocssw.py run completed successfully.  But, then I did a manual update_luts.py.  Guess what?  It failed exactly as you report.  So, I can say both "works for me" and "issue confirmed, investigating now" :eek:

I believe I figured out why it "worked for me".  Despite the fact that I made sure I ran from a clean shell with system level installed python,  the installer uses the  ocssw_runner script to call update_luts.py...that was picking up my user configuration - which has a miniconda install of python3 (3.7.5).  So update_luts.py works with python 3, but isn't happy with 2.7.5

We'll put this back in the pot and stew  it a little longer in the hopes that we can get it working with python2.7.  If we can't, you'll need to install python3.

Sean