Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden

[shimonhypervine]

When i try to download sentinel-1 data using python script for bulk download it shows:
Unknown Error, Could not get file HEAD: HTTP Error 403: Forbidden

And previously if we restart the downloads it skips the files that have been already downloaded and restarts wherever the files are partially downloaded but now if we rerun the script it just starts from initial asset and move towards the next after completing the first. It then takes a lot of time. Can someone point a solution for this issue.

[ASF - bhauer]

This is a bug in the bulk download script that we recently became aware of and hope to have fixed soon. I'll let you know when this has happened.

Thank you for your patience.

[mike_chan]

Hey there

I have been bothered by the same issue lately, so I rewrite the script to solve it, you can find it at:

https://github.com/chenyw-insar/bulk_downloader_asf

Remember to paste the urls inside your original script into the "urls = []" position before execute it.

Hope it helps :)
YW Chen

[shimonhypervine]

Hi @mike_chan, Thank you for the script it works fine. Is there any update from the team regarding the issue?

[ASF - bhauer]

The bug is still being worked on, and I'll let you know just as soon as we've got a fix. Thanks for your patience!

[radar_rob]

This issue persists to this day. I was using python files downloaded from the ASF (search.asf.alaska.edu) website.

[nadyanadya1111]

The same error. The script from Chen is not working either.

[ASF - bhauer]

Attached is a document that outlines the steps for running the alternate bulk download script in a Windows command line environment. I hope this helps.

Using the alternate download script in Windows.pdf

(103.19 KiB) Downloaded 20 times

[nadyanadya1111]

No, my problem is not about the authentication cookie, but about the file head check. My problem is exactly what the first post described. The script from ASF finds my login cookie sucessfuly (if not many days have passed), runs, but re-downloads everything again from the beginning, not checking the failed files like a year ago and downloading only them. Hope this bug will be fixed soon because I am downloading long timeseries.
I am on Linux, the script of Chen runs, logs me in, identifies failed files, but fails to actually download them.
Thank you.

[ASF - bhauer]

You're correct, the error is a result of the bug that is causing previously downloaded files to be overwritten. Thanks for identifying that.