multi-folder download using wget download command
multi-folder download using wget download command
Hi all, I'm trying to realize multi-file (multi-folder) download using wget download command for specific days from the folder https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD03/2020/.
The command works properly when downloading multi-file for only one day.
On the other hand, I am not able to download data for determined multiple days (e.g., using list option for wget).
In particular, after completing the multi-file download of a specific day, lets' say day 091, the system starts automatically the downloads from day 001, after creating all the 365 daily folders.
Any suggestion to avoid this?
I would like wget to enter a list of URL (e.g., using wget -i url.txt in the command line) and let's say download files into folder 092 and 093. But wget always starts to download day 001 files after completing the day 091.
Many thanks for your kind help,
Peppe
The command works properly when downloading multi-file for only one day.
On the other hand, I am not able to download data for determined multiple days (e.g., using list option for wget).
In particular, after completing the multi-file download of a specific day, lets' say day 091, the system starts automatically the downloads from day 001, after creating all the 365 daily folders.
Any suggestion to avoid this?
I would like wget to enter a list of URL (e.g., using wget -i url.txt in the command line) and let's say download files into folder 092 and 093. But wget always starts to download day 001 files after completing the day 091.
Many thanks for your kind help,
Peppe
Filters:
-
- Posts: 379
- Joined: Mon Sep 30, 2019 8:33 am America/New_York
- Has thanked: 3 times
Re: multi-folder download using wget download command
Dear Peppe:
If you are using the URL in your wget command the way you sent (https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD03/2020/) it is going to download all days of the year starting from day 001. However, if you are using a different URL, send your complete wget command (without your token) for us to investigate.
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
If you are using the URL in your wget command the way you sent (https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD03/2020/) it is going to download all days of the year starting from day 001. However, if you are using a different URL, send your complete wget command (without your token) for us to investigate.
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
Re: multi-folder download using wget download command
Many thanks for your support.
The wget command I'm using is:
wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD21/2012/061" --header "Authorization: OMISSIS" -P F:\
Alternatively, I tried using the command wget -i list.txt with the same results (i.e., downloading day 061 and after that all days since 001.
In the list.txt I placed all the URL of my interest, such as:
https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD21/2012/061
https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD21/2012/091
https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD21/2012/121
... etc...
Hope you can help me.
Best,
Peppe
The wget command I'm using is:
wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD21/2012/061" --header "Authorization: OMISSIS" -P F:\
Alternatively, I tried using the command wget -i list.txt with the same results (i.e., downloading day 061 and after that all days since 001.
In the list.txt I placed all the URL of my interest, such as:
https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD21/2012/061
https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD21/2012/091
https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD21/2012/121
... etc...
Hope you can help me.
Best,
Peppe
-
- Posts: 379
- Joined: Mon Sep 30, 2019 8:33 am America/New_York
- Has thanked: 3 times
Re: multi-folder download using wget download command
Dear Peppe:
Thanks very much for sending the details. We investigate and will get back to you ASAP.
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
Thanks very much for sending the details. We investigate and will get back to you ASAP.
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
Regards,
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.