How to download release data


#1

This was asked in a different thread. I think it worth its own topic.

Currently I use the following methds to find out the URL to the latest data:

wget https://download.icgc.org/exports
bzuo@lxv-icgc-download01:/data/brady$ cat exports | jq '.'
[
  {
    "date": 1473912995726,
    "release_number": 22,
    "access": "open",
    "type": "repository",
    "id": "repository.tar.gz",
    "url": "https://download.icgc.org/exports/repository.tar.gz"
  },
  {
    "date": 1471287424702,
    "release_number": 22,
    "access": "open",
    "type": "data",
    "id": "data.open.tar",
    "url": "https://download.icgc.org/exports/data.open.tar"
  },
  {
    "date": 1471382920174,
    "release_number": 22,
    "access": "open",
    "type": "release",
    "id": "release.tar",
    "url": "https://download.icgc.org/exports/release.tar"
  }
]

Then I download all of them. The release.tar and repository.tar.gz seem to be the latest release 22. However, the data.open.tar is not:

wget --continue --no-check-certificate https://download.icgc.org/exports/data.open.tar
 tar tvf data.open.tar
-rw-r--r-- 0/0             351 2016-09-15 23:02 release_21/README.txt
-rw-r--r-- 0/0             126 2016-09-15 23:02 release_21/data/ALL-US/DO1/donor/part-00001.gz
-rw-r--r-- 0/0          214267 2016-09-15 23:02 release_21/data/ALL-US/DO1/exp_array/part-00001.gz

Something is wrong on download server. It looks its information is inconsistent.

The links only allows download latest release. If I want to download past release, how do I download them?


#2

@brady Yes, looks like the download server contains the old data. I’m working on this now.


#3

@brady, this issue is fixed now. The latest data can be downloaded from the server.


#4

@vitalii, thanks for the quick fix!

Yes, it is now release_22:

tar tvf data.open.tar 

-rw-r--r-- 0/0            1646 2016-09-16 00:30 release_22/README.txt
-rw-r--r-- 0/0             126 2016-09-16 00:30 release_22/data/ALL-US/DO1/donor/part-00001.gz
-rw-r--r-- 0/0          214267 2016-09-16 00:30 release_22/data/ALL-US/DO1/exp_array/part-00001.gz

Hopefully I will be able to download to China in 24 hours.