Bulk downloading encounters connection errors
-
- Posts: 4
- Joined: Thu Jul 14, 2022 5:58 am America/New_York
Bulk downloading encounters connection errors
I am bulk downloading some GPM SSMI data from GES DISC with wget and am experiencing connection timeouts:
Connecting to gpm1.gesdisc.eosdis.nasa.gov (gpm1.gesdisc.eosdis.nasa.gov)|198.118.197.50|:443... failed: Connection timed out.
Connecting to gpm1.gesdisc.eosdis.nasa.gov (gpm1.gesdisc.eosdis.nasa.gov)|2001:4d0:241a:4041::50|:443... failed: Network is unreachable.
Is this because of a limit on the number of sequential (or concurrent) requests in a certain period of time? And if so, can this limit be increased?
Connecting to gpm1.gesdisc.eosdis.nasa.gov (gpm1.gesdisc.eosdis.nasa.gov)|198.118.197.50|:443... failed: Connection timed out.
Connecting to gpm1.gesdisc.eosdis.nasa.gov (gpm1.gesdisc.eosdis.nasa.gov)|2001:4d0:241a:4041::50|:443... failed: Network is unreachable.
Is this because of a limit on the number of sequential (or concurrent) requests in a certain period of time? And if so, can this limit be increased?
Filters:
Re: Bulk downloading encounters connection errors
Thank you for your interest in using GPM data.
In regards to the error you received, the system may issue service temporarily
unavailable for more than one concurrent connection in certain circumstances.
Please review/reduce the number of concurrent connections you are submitting,
and try again.
In regards to the error you received, the system may issue service temporarily
unavailable for more than one concurrent connection in certain circumstances.
Please review/reduce the number of concurrent connections you are submitting,
and try again.
-
- Posts: 4
- Joined: Thu Jul 14, 2022 5:58 am America/New_York
Re: Bulk downloading encounters connection errors
Thank you for you reaction. Indeed when I decrease the number of wget processes running in parallel I get much less Connection timeouts. But when I have only one wget running I still get the Connection timeouts. It runs for 30 seconds or so and downloads a couple of files but then stalls more than a minute.
Re: Bulk downloading encounters connection errors
Our staff are working on this issue. Once I hear back from them I will be in contact with you.
Thanks
Thanks