Wget timestamp option no longer works since authentication upgrade

[b.detracey]

Sean,Using the --timestamp option with wget now causes downloads to fail.$ wget -v --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge=on --content-disposition https://oceandata.sci.gsfc.nasa.gov (/cgi/getfile/V2019335133000.L2_SNPP_OC.nc--2020-01-27 10:26:23-- ) https://oceandata.sci.gsfc.nasa.gov (/cgi/getfile/V2019335133000.L2_SNPP_OC.nc)Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... xx.xxx.xx.xx, 2001:4d0128::84Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|xx.xxx.xx.xx|:443... connected.HTTP request sent, awaiting response... 302 FoundLocation: /ob/getfile/V2019335133000.L2_SNPP_OC.nc [following]--2020-01-27 10:26:23--  https://oceandata.sci.gsfc.nasa.gov(/ob/getfile/V2019335133000.L2_SNPP_OC.nc)Reusing existing connection to oceandata.sci.gsfc.nasa.gov:443.HTTP request sent, awaiting response... 302 FoundLocation: https://urs.earthdata.nasa.gov/oauth/authorize?client_id=Z0u-MdLNypXBjiDREZ3roA&redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov(%2Fob%2Fgetfile%2Frestrict&response_type=code&required_scope=study_area,country [following]--2020-01-27 10:26:24--)  https://urs.earthdata.nasa.gov/oauth/authorize?client_id=Z0u-MdLNypXBjiDREZ3roA&redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov (%2Fob%2Fgetfile%2Frestrict&response_type=code&required_scope=study_area,country) Resolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... xx.xxx.xx.xx, 2001:4d04081::89Connecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|xx.xxx.xx.xx|:443... connected.HTTP request sent, awaiting response... 302 FoundLocation: https://oceandata.sci.gsfc.nasa.gov (/ob/getfile/restrict?code=cc128f531cf035e33370a4fa6e247e35ee83c24d02ebd91fb44c8e9c9d62368e )[following]--2020-01-27 10:26:24--  https://oceandata.sci.gsfc.nasa.gov (/ob/getfile/restrict?)code=cc128f531cf035e33370a4fa6e247e35ee83c24d02ebd91fb44c8e9c9d62368eConnecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|xx.xxx.xx.xx|:443... connected.HTTP request sent, awaiting response... 302 FoundLocation: /ob/getfile/V2019335133000.L2_SNPP_OC.nc [following]--2020-01-27 10:26:24--  https://oceandata.sci.gsfc.nasa.gov (/ob/getfile/V2019335133000.L2_SNPP_OC.nc) Reusing existing connection to oceandata.sci.gsfc.nasa.gov:443.HTTP request sent, awaiting response... 200 OKLength: 77120538 (74M) [application/octet-stream]Saving to: ‘V2019335133000.L2_SNPP_OC.nc’100%[==========================================================================================>] 77,120,538  22.1MB/s   in 3.4s2020-01-27 10:26:34 (21.4 MB/s) - ‘V2019335133000.L2_SNPP_OC.nc’ saved [77120538/77120538]$ wget -v --timestamp --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge=on --content-disposition https://oceandata.sci.gsfc.nasa.gov(/cgi/getfile/V2019335133000.L2_SNPP_OC.nc--2020-01-27 10:26:40-- ) https://oceandata.sci.gsfc.nasa.gov (/cgi/getfile/V2019335133000.L2_SNPP_OC.nc) Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... xx.xxx.xx.xx, 2001:4d0128::84Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|xx.xxx.xx.xx|:443... connected.HTTP request sent, awaiting response... 302 FoundLocation: /ob/getfile/V2019335133000.L2_SNPP_OC.nc [following]--2020-01-27 10:26:41--  https://oceandata.sci.gsfc.nasa.gov (/ob/getfile/V2019335133000.L2_SNPP_OC.nc) Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|xx.xxx.xx.xx|:443... connected.HTTP request sent, awaiting response... 302 FoundLocation: https://urs.earthdata.nasa.gov/oauth/authorize?redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov (%2Fob%2Fgetfile%2Frestrict&response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA&required_scope=study_area,country) [following]--2020-01-27 10:26:44--  https://urs.earthdata.nasa.gov/oauth/authorize?redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict&response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA&required_scope=study_area,countryResolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... xx.xxx.xx.xx, 2001:4d04081::89Connecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|xx.xxx.xx.xx|:443... connected.HTTP request sent, awaiting response... 400 Bad Request2020-01-27 10:26:45 ERROR 400: Bad Request.

[OB.DAACx - SeanBailey]

Ben,

I cannot replicate the problem...works for me:

$  wget -v --timestamp --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge=on --content-disposition https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/V2019335133000.L2_SNPP_OC.nc
--2020-01-27 15:06:18--  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/V2019335133000.L2_SNPP_OC.nc
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... fddd:0:0:2::33, xx.xxx.xx.xx
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|fddd:0:0:2::33|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: /ob/getfile/V2019335133000.L2_SNPP_OC.nc [following]
--2020-01-27 15:06:18--  https://oceandata.sci.gsfc.nasa.gov/ob/getfile/V2019335133000.L2_SNPP_OC.nc
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|fddd:0:0:2::33|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 77120538 (74M) [application/octet-stream]
--2020-01-27 15:06:19--  https://oceandata.sci.gsfc.nasa.gov/ob/getfile/V2019335133000.L2_SNPP_OC.nc
Reusing existing connection to [oceandata.sci.gsfc.nasa.gov]:443.
HTTP request sent, awaiting response... 200 OK
Length: 77120538 (74M) [application/octet-stream]
Saving to: ‘V2019335133000.L2_SNPP_OC.nc’

V2019335133000.L2_SNPP_OC.n 100%[===========================================>]  73.55M  44.4MB/s    in 1.7s

$  wget -v --timestamp --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge=on --content-disposition https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/V2019335133000.L2_SNPP_OC.nc
--2020-01-27 15:06:49--  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/V2019335133000.L2_SNPP_OC.nc
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... fddd:0:0:2::33, xx.xxx.xx.xx
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|fddd:0:0:2::33|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: /ob/getfile/V2019335133000.L2_SNPP_OC.nc [following]
--2020-01-27 15:06:49--  https://oceandata.sci.gsfc.nasa.gov/ob/getfile/V2019335133000.L2_SNPP_OC.nc
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|fddd:0:0:2::33|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 77120538 (74M) [application/octet-stream]
Server file no newer than local file ‘V2019335133000.L2_SNPP_OC.nc’ -- not retrieving.


[b.detracey]

Looks like my version of wget(1.15) was too old. Have had success with a more recent version (1.19.4).
Thanks Sean

[OB.DAACx - SeanBailey]

I'm glad you figured it out!  Seems lots of things working differently with various versions of wget...I should have asked the version question!

Sean