Page 1 of 1
Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Wed Dec 18, 2024 4:58 am America/New_York
by shimonhypervine
When i try to download sentinel-1 data using python script for bulk download it shows:
Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
And previously if we restart the downloads it skips the files that have been already downloaded and restarts wherever the files are partially downloaded but now if we rerun the script it just starts from initial asset and move towards the next after completing the first. It then takes a lot of time. Can someone point a solution for this issue.
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Thu Dec 19, 2024 3:59 pm America/New_York
by ASF - bhauer
This is a bug in the bulk download script that we recently became aware of and hope to have fixed soon. I'll let you know when this has happened.
Thank you for your patience.
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Mon Dec 23, 2024 9:14 am America/New_York
by mike_chan
Hey there
I have been bothered by the same issue lately, so I rewrite the script to solve it, you can find it at:
https://github.com/chenyw-insar/bulk_downloader_asf
Remember to paste the urls inside your original script into the "urls = []" position before execute it.
Hope it helps :)
YW Chen
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Fri Jan 10, 2025 12:24 pm America/New_York
by shimonhypervine
Hi @mike_chan, Thank you for the script it works fine. Is there any update from the team regarding the issue?
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Fri Jan 10, 2025 3:06 pm America/New_York
by ASF - bhauer
The bug is still being worked on, and I'll let you know just as soon as we've got a fix. Thanks for your patience!
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Thu Mar 20, 2025 7:45 am America/New_York
by radar_rob
This issue persists to this day. I was using python files downloaded from the ASF (search.asf.alaska.edu) website.
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Wed Jul 02, 2025 12:56 pm America/New_York
by nadyanadya1111
The same error. The script from Chen is not working either.
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Wed Jul 02, 2025 4:31 pm America/New_York
by ASF - bhauer
Attached is a document that outlines the steps for running the alternate bulk download script in a Windows command line environment. I hope this helps.
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Thu Jul 03, 2025 5:24 am America/New_York
by nadyanadya1111
No, my problem is not about the authentication cookie, but about the file head check. My problem is exactly what the first post described. The script from ASF finds my login cookie sucessfuly (if not many days have passed), runs, but re-downloads everything again from the beginning, not checking the failed files like a year ago and downloading only them. Hope this bug will be fixed soon because I am downloading long timeseries.
I am on Linux, the script of Chen runs, logs me in, identifies failed files, but fails to actually download them.
Thank you.
Re: Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden
Posted: Thu Jul 03, 2025 8:00 pm America/New_York
by ASF - bhauer
You're correct, the error is a result of the bug that is causing previously downloaded files to be overwritten. Thanks for identifying that.