Earthdata Forum

I am trying to access some EarthDATA hosted on AWS S3. (https://search.earthdata.nasa.gov/downloads/6685456403?tl=1684009604!3!!). I am using this code:

library(aws.s3)

Sys.setenv("AWS_ACCESS_KEY_ID" = "xxx")
Sys.setenv("AWS_SECRET_ACCESS_KEY" = "yyy")
Sys.setenv("AWS_DEFAULT_REGION" = "us-west-2")
Sys.setenv("AWS_SESSION_TOKEN" = "zzz")

bucketlist()

List of 4
$ Code : chr "AccessDenied"
$ Message : chr "Access Denied"
$ RequestId: chr "XVYCE9WF0ZFHYD8X"
$ HostId : chr "+UVjWq+bum9j1f13Q2/SAe5mqDD+Ts7ToBQ2KIA4Lyev0QWHhe9CZ319D9NmaTczvZV96NakxQM="
- attr(*, "headers")=List of 6
..$ x-amz-request-id : chr "XVYCE9WF0ZFHYD8X"
..$ x-amz-id-2 : chr "+UVjWq+bum9j1f13Q2/SAe5mqDD+Ts7ToBQ2KIA4Lyev0QWHhe9CZ319D9NmaTczvZV96NakxQM="
..$ content-type : chr "application/xml"
..$ transfer-encoding: chr "chunked"
..$ date : chr "Sun, 14 May 2023 11:13:18 GMT"
..$ server : chr "AmazonS3"
..- attr(*, "class")= chr [1:2] "insensitive" "list"
- attr(*, "class")= chr "aws_error"
- attr(*, "request_canonical")= chr "GET\n/\n\nhost:s3-us-west-2.amazonaws.com\nx-amz-date:20230514T111319Z\nx-amz-security-token:FwoGZXIvYXdzEOX///"| __truncated__
- attr(*, "request_string_to_sign")= chr "AWS4-HMAC-SHA256\n20230514T111319Z\n20230514/us-west-2/s3/aws4_request\ndd966ae1f22cc7f8a1bc589f5f718b1ea90187d"| __truncated__
- attr(*, "request_signature")= chr "AWS4-HMAC-SHA256 Credential=ASIAZLX6ZES4XW7JRPFE/20230514/us-west-2/s3/aws4_request,SignedHeaders=host;x-amz-da"| __truncated__
NULL
Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
Forbidden (HTTP 403).
I got credential info here: https://data.lpdaac.earthdatacloud.nasa.gov/s3credentials (which are valid for 1 hour). Ultimately, I would like to read that geotif: https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T19UEP.2023126T152446.v2.0/HLS.L30.T19UEP.2023126T152446.v2.0.B05.tif. I have also tried
r$> res <- aws.s3::get_object(
object = "https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T19UEP.2023126T152446.v2.0/HLS.L30.T19UEP.2023
126T152446.v2.0.B05.tif",
bucket = "s3://lp-prod-public/",
)

rawToChar(res)
[1] "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>JYYRT8KTB62G5JW8</RequestId><HostId>mYqOP0hkmNZAeJZO39LHn/jn1Y34wB+pJebA0EMXKO0XoM5Gm2cl8Rs7EKVtmCfY60SexBO2wbM=</HostId></Error>"

Any ideas why I am getting access denied?

Cheers,
Phil

Hi @filoche We have one of our science staff looking into this based on a few other users writing in on the forum: viewtopic.php?p=14261# I've passed along your additional information to him. We'll let you know once we have an answer. Thanks.

Thank you very much, it is appreciated!

Hi @filoche Just wanted to make sure you saw Aaron's response in the other thread:

Hey All,

For those having issues with boto3 in Python when trying to download from and list objects in S3, please make sure you check the following:
1. Make sure you are working from an cloud compute instance running in AWS us-west-2
2. Make sure your credentials are not expired. Temporary S3 credentials are only valid for 1 hour.
3. Make sure to include the "Prefix" option in the download and list objects methods from boto3. Please refer to the following resources: https://github.com/nasa/LPDAAC-Data-Resources/blob/main/python/how-tos/Earthdata_Cloud__Download_file_from_S3.ipynb & https://github.com/nasa/LPDAAC-Data-Resources/blob/main/python/how-tos/Earthdata_Cloud__List_bucket_objects.ipynb

Thanks!