Earthdata Forum

Hi,
I am using wget to download datasets using api key from AWS instance, we notice that we are not able to resolve the ___domain name, whilst this works well from my work machine.

Any leads on this will be appreciated.

Thanks,

Hi @zbegg To help us look into the issue more, could you please tell us which datasets are you trying to download? Thanks -- Danielle

Hi, I am using this command to download the data,

wget -nc https://oceandata.sci.gsfc.nasa.gov/ob/getfile/AQUA_MODIS.20230821.L3m.DAY.CHL.chlor_a.4km.nc?appkey={my_api_key} --no-check-certificate -O AQUA_MODIS.20230821.L3m.DAY.CHL.chlor_a.4km.nc

Error:
Resolving oceandata.sci.gsfc.nasa.gov...
169.154.128.84, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov|169.154.128.84|:443... failed: Connection timed out.
Connecting to oceandata.sci.gsfc.nasa.gov|2001:4d0:2418:128::84|:443... failed: Network is unreachable.

This server is sitting in amazon AWS cloud.

Thanking you in advance.

Please email "connection_problems@oceancolor.gsfc.nasa.gov" with the public ip address you are connection from, the time you tried to connect and a short description of the network issue you are experiencing. The NASA OBPG network team will assist you.

Although I see the file via file search, it does appear that the file is missing, resulting in an error page. I've alerted the data production team of this issue.

If you are experiencing issues resolving a ___domain using wget, there are a few things you can check and troubleshoot. Here are some steps you can take:

Check Internet Connection:
Ensure that your internet connection is stable and working. You can try opening a website in a web browser to verify connectivity.

DNS Resolution:
Confirm that the ___domain name can be resolved to an IP address. You can use the nslookup or dig command to check DNS resolution. For example:


nslookup website ___domain name
DNS Server Configuration:
Verify that your DNS server settings are correct. You can check the DNS configuration in your system's network settings or in the configuration files, depending on your operating system.

Firewall or Proxy Settings:
Make sure that there are no firewall or proxy settings blocking the wget command. If you are behind a proxy, you may need to configure wget to use the proxy.

Check wget Command:
Ensure that the wget command is correctly formatted. Here is a basic example:


wget website ___domain name
HTTP vs HTTPS:
Verify if the website uses HTTPS. If it does, make sure to use https:// in the URL when using wget. Some websites may automatically redirect HTTP to HTTPS.


wget website ___domain name
User-Agent:
Some websites may block requests without a proper User-Agent header. You can try setting a User-Agent using the -U or --user-agent option:


wget -U "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3" http://example.com
SSL/TLS Issues:
If the website uses HTTPS, there might be SSL/TLS-related issues. Make sure your system has the necessary certificates installed.

Network Configuration:
Check if there are any network configuration issues on your system, such as incorrect DNS settings or routing problems.

Debugging Output:
Use the --debug option with wget to get more detailed information about what might be causing the issue:


wget --debug website ___domain name
By going through these steps, you should be able to identify and resolve the issue preventing wget from resolving the ___domain. If the problem persists, there might be specific details about the error that can help in further troubleshooting.

Hello,

After re-reading this, I think the issue may be your appkey. Please try your wget command with an appkey created here: https://oceandata.sci.gsfc.nasa.gov/appkey/

Our team has tested and at least 3 of us have been able to pull the desired file down by repeating your wget command from our machines.