Elimination of FTP data access

[OB.DAACx - SeanBailey]

Our publicly-accessible ftp server (samoa.gsfc.nasa.gov) will be going out of
service permanently in mid to late March. The ftp protocol has certain
uncorrectable security issues which make it a high risk.  This will finalize
the transition from FTP that we began in March of 2010 and continued in June of
2013.   

MOST of our data distribution services will be unaffected by this change, as
the primary mechanism is via HTTPS.  However, users with extracted data
subscriptions currently served via ftp://samoa.gsfc.nasa.gov/pub/subscriptions
will need to change their data retrieval method.  Non-extracted subscriptions
are not impacted, as those were already using HTTPS.

We are currently working on modifications that will present the extract
subscriptions in the same manner as non-extracted subscriptions and hope to
have this in place concurrent with the existing FTP distribution within a week.
Once that is in place, another notification will be sent and we encourage you
to begin your transition to the new method to avoid any data access delays.

We apologize for any inconvenience this change may cause, but look forward to
being able to present a consistent distribution mechanism for all
subscriptions.

Regards,
Sean

[OB.DAACx - SeanBailey]

OBPG Data Users,

The updated method for retrieving extracted subscription data previously staged on our FTP server is now in place.
You may begin using this method immediately.  The existing FTP server will remain in operations until 19 March 2018.
After then, this new method will be the sole means for accessing the data for these subscriptions.

The method is now the same as used for non-extracted subscriptions and is summarized below.

NOTE: For requests that result in multiple output files per source file, the files will be staged in a compressed tar file to
reduce the number of files required to download.

Regards,
Sean

The new extracted subscription access method uses the same HTTP service used by extracted subscriptions. 
This service returns a list of URLs to the files in your subscription, and its output can be piped directly into
another wget command to download the files.  If you are a curl user, it is a bit more complicated but still reasonable.

The HTTP service is called with wget like this, where NNNN is replaced by your subscription ID.

wget --post-data="subID=NNNN&subType=2&addurl=1&results_as_file=1" -O -  https://oceandata.sci.gsfc.nasa.gov/api/file_search

Extracted subscriptions are indicated by the inclusion of subType=2
Non-extracted can either set subType=1 or not include the subType element, as it will default to 1.

To actually download the files for the above, pipe that command to another wget command:

wget --post-data="subID=NNNN&subType&addurl=1&results_as_file=1" -O - https://oceandata.sci.gsfc.nasa.gov/api/file_search | wget -i -

Curl users can get the list of file URLs with this command:

curl -d "subID=NNNN&subType=2&addurl=1&results_as_file=1"  https://oceandata.sci.gsfc.nasa.gov/api/file_search

Each file in the list can be downloaded with a small script like this:

for file in $(<curl cmd from above> | grep getfile)
do
    curl -L -O $file
done