Errors accessing HLS data

[venusinkant]

I have a same error. I am trying to access HLS data according to the https://github.com/nasa/HLS-Data-Resources/blob/main/python/tutorials/HLS_Tutorial.ipynb. But I get the error
vsi curl error: '/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T22KDB.2018308T133219.v2.0/HLS.S30.T22KDB.2018308T133219.v2.0.B04.tif' not recognized as a supported file format.. Retrying...

Could you tell me how do you deal with this problem？

Thank you

[LP DAACx - dgolon]

Hi @venusinkant Our science team is looking into this. We'll report here when we have more info. Thanks --Danielle

[LP DAACx - dgolon]

Our science team continues to look into the error you are seeing but are having trouble replicating it. In the meantime, they recommend using AppEEARS or the AppEEARS API to access the HLS data: https://appeears.earthdatacloud.nasa.gov/

[venusinkant]

Thank you for the reply!

I have found the solutuion to my problem. The python version in my cmd is 3.6, but the environment where I run the code is python 3.11. When I run the code to logging into my earthdata account, the program create the netrc file in my computer by the python3.6, but when I run the following code, the environment is python 3.11. So comes the error.

Thank you again for your help！

[kvenkataramani]

My '.netrc' file was missing a newline character at the end of the line containing my earthdata credentials. Adding this to my file on my Windows machine fixed the issue.

The gdalinfo command did note that it was unable to open the remote URL, pinpointing the error a credentials issue. Then I tested out changing the read/write permissions for the file as well as adding the newline character at the end of the line. The latter fixed the issue.

[rbrassfield]

I'm running into the same issues. I'm working within fully updated packages/environments following this GDAL config:
gdal.SetConfigOption('GDAL_HTTP_COOKIEFILE','~/cookies.txt')
gdal.SetConfigOption('GDAL_HTTP_COOKIEJAR', '~/cookies.txt')
gdal.SetConfigOption('GDAL_DISABLE_READDIR_ON_OPEN','EMPTY_DIR')
gdal.SetConfigOption('CPL_VSIL_CURL_ALLOWED_EXTENSIONS','tif')
gdal.SetConfigOption('GDAL_HTTP_UNSAFESSL', 'YES')

with the following lines for building/double checking the netrc file:
urs = 'urs.earthdata.nasa.gov' # Earthdata URL endpoint for authentication
prompts = ['Enter NASA Earthdata Login Username: ',
'Enter NASA Earthdata Login Password: ']

netrc_name = ".netrc"

# Determine if netrc file exists, and if so, if it includes NASA Earthdata Login Credentials
try:
netrcDir = os.path.expanduser(f"~/{netrc_name}")
netrc(netrcDir).authenticators(urs)[0]

# Below, create a netrc file and prompt user for NASA Earthdata Login Username and Password
except FileNotFoundError:
homeDir = os.path.expanduser("~")
Popen('touch {0}{2} | echo machine {1} >> {0}{2}'.format(homeDir + os.sep, urs, netrc_name), shell=True)
Popen('echo login {} >> {}{}'.format(getpass(prompt=prompts[0]), homeDir + os.sep, netrc_name), shell=True)
Popen('echo \'password {} \'>> {}{}'.format(getpass(prompt=prompts[1]), homeDir + os.sep, netrc_name), shell=True)
# Set restrictive permissions
Popen('chmod 0600 {0}{1}'.format(homeDir + os.sep, netrc_name), shell=True)

# Determine OS and edit netrc file if it exists but is not set up for NASA Earthdata Login
except TypeError:
homeDir = os.path.expanduser("~")
Popen('echo machine {1} >> {0}{2}'.format(homeDir + os.sep, urs, netrc_name), shell=True)
Popen('echo login {} >> {}{}'.format(getpass(prompt=prompts[0]), homeDir + os.sep, netrc_name), shell=True)
Popen('echo \'password {} \'>> {}{}'.format(getpass(prompt=prompts[1]), homeDir + os.sep, netrc_name), shell=True)

And I am still getting the following error:
---------------------------------------------------------------------------
CPLE_OpenFailedError Traceback (most recent call last)
rasterio/_base.pyx in rasterio._base.DatasetBase.__init__()

rasterio/_base.pyx in rasterio._base.open_dataset()

rasterio/_err.pyx in rasterio._err.exc_wrap_pointer()

CPLE_OpenFailedError: '/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif' not recognized as a supported file format.

During handling of the above exception, another exception occurred:

RasterioIOError Traceback (most recent call last)
<ipython-input-29-7e3d3a89d91f> in <cell line: 1>()
----> 1 rio.open("https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif")

1 frames
/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py in open(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, **kwargs)
302
303 if mode == "r":
--> 304 dataset = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
305 elif mode == "r+":
306 dataset = get_writer_for_path(path, driver=driver)(

rasterio/_base.pyx in rasterio._base.DatasetBase.__init__()

RasterioIOError: '/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif' not recognized as a supported file format.

I have too much data for the AppEEARS download methods.

[LP DAACx - dgolon]

Hi @rbrassfield We have worked with a few users now that previously found this error and it seems resetting their Earthdata Login password (https://urs.earthdata.nasa.gov/reset_passwords/new) has resolved the issue. When creating a password please avoid using the "#", "&" or "%" symbols or other similar characters in it, as in some cases that also seems to have caused an issue. If resetting your password and then trying again does not work, please email us a copy of the exact script you are using to lpdaac@usgs.gov and reference this post. Thanks --Danielle

[jamesbrown941]

It seems like the error might be due to compatibility issues between the file format and the versions of the libraries you're using. The .tif files being accessed via vsicurl may require specific versions of rasterio or rioxarray that support this type of remote file handling.

Here are a few suggestions to troubleshoot:

Library Versions: Check the versions of rasterio, rioxarray, and other dependencies in your environment. Try matching them with the versions in your professor's environment.
GDAL Compatibility: Ensure that your GDAL version is compatible with the .tif files. Sometimes, older or newer versions cause issues.
Environment Setup: If possible, clone your professor’s environment or recreate it using tools like conda or pip.
Direct File Access: As a workaround, try downloading the .tif file locally and then load it to check if the issue is related to the remote access via vsicurl.