Stale data release information on portal

Portal
#1

After installation, we have a dummpy release_22 in hdfs, and full populated release_21. Elasticsearch has release_22 data. Release_21 was imported into hdfs due to wrong data downloaded from ICGC download server. On portal data release page, it shows pretty much empty release_22 project directory, and fully populated release_21 project directory. This is expected.

Recently we deleted the dummy release_22 in hdfs, and imported fully populated release_22 data. In HDFS, it shows a lot of projects. However, in portal release page, it shows the same release_22 data. release_22 project directory is the same as before. It seems to me some sort index was generated for release_22 when it had dummy data. The index hasn’t been regenerated after we reimported the release_22 data.

I have a few questions:

  1. How do we update the data release information on portal page after reimporting a release?

  2. How do we change the current release on data release page? Is there a config for this? I couldn’t find in portal server or download server config files. For example, if I import a new release into HDFS, I want to change portal to use the data from the release, we want the “current” on data release page points to the new release instead of last old release.

Brady

#2

@brady After a new release data is added to the download server there are 2 ways to refresh it and start serving the new data:

  • Restart the download server. The server will scan its input directory and load the required files in memory.
  • Run the following command on the download server to load the added release into the server memory.
curl -XPUT <hostname>:<port>/srv-info/release/release_22

You will a message in the log file that the new release has been added.

#3

Thanks, @vitalii. I restarted the download server, it now sees the new release 22 data.

Do you know how the “current” link is generated? Currently it points to release 22 which works for us for now. If we add release 23 data, will it points to release 23 automatically or we will have to change some config?

#4

@brady, after a new release is added to the download service(like you did with the release_22). The current link will still point to the old release(release_21 in your case). It will point to the old one until you do either a restart of the server or execute the command which I mentioned in the previous post.

P.S. I advise you to upgrade the download server to the latest version. It contains a bugfix related to importing data with the curl command I mentioned earlier.

#5

@vitalii, if I understand correctly, on restart download server will scan its input directory and point current release to the latest release from all the releases it finds during scanning. Is this correct?