Dcc-portal-server in docker container

1

Hey all,

I’m working on initializing the dcc-portal-server in it’s own Docker container, which communicates to the portal-ui and elasticsearch outside of the container. The Dockerfile is as follows:

FROM java:8-jre

ENV DCC_HOME /opt/dcc
RUN useradd -m dcc &&  mkdir -p $DCC_HOME

ENV RELEASE=4.3.27

WORKDIR $DCC_HOME

ADD https://artifacts.oicr.on.ca/artifactory/dcc-release/org/icgc/dcc/dcc-portal-server/${RELEASE}/dcc-portal-server-${RELEASE}-dist.tar.gz $DCC_HOME

RUN tar xf dcc-portal-server-${RELEASE}-dist.tar.gz && \
    rm dcc-portal-server-${RELEASE}-dist.tar.gz

RUN sed -i.bak s/spring.profiles.active=.*$/spring.profiles.active=ohsu-dev/ ${DCC_HOME}/dcc-portal-server-${RELEASE}/conf/wrapper.conf
RUN cat dcc-portal-server-${RELEASE}/conf/wrapper.conf
COPY conf/portal.yml ${DCC_HOME}/dcc-portal-server-${RELEASE}/conf/application.yml

WORKDIR $DCC_HOME/dcc-portal-server-${RELEASE}

EXPOSE 9000 9200 9300
CMD bin/dcc-portal-server start && tail -F logs/dcc-portal-server.log

And I am starting it in compose like this:

# run DCC-portal-server
  server:
    container_name: server
    build:
      context: services/server
    ports:
      - "${SERVER_PORT}:${SERVER_PORT}"
      - 9200:9200
      - 9300:9300
    volumes:
      - "./.resource_data:/resource"

But I am continually getting an error that looks like this:

Unsatisfied dependency expressed through field 'hazelcast':
  Error creating bean with name 'hazelcastProperties' defined in class path resource [org/icgc/dcc/portal/server/config/ServerConfig.class]:
   Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException:
    Failed to instantiate [org.icgc.dcc.portal.server.config.ServerProperties$HazelcastProperties]:
     Factory method 'hazelcastProperties' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException:
      Error creating bean with name 'properties':
       Could not bind properties to ServerProperties (prefix=, ignoreInvalidFields=false, ignoreUnknownFields=true, ignoreNestedProperties=false); nested exception is org.springframework.validation.BindException:
        org.springframework.boot.bind.RelaxedDataBinder$RelaxedBeanPropertyBindingResult:
         1 errors
Field error in object 'target' on field 'release':
          rejected value [4.3.27]; codes [typeMismatch.target.release,typeMismatch.release,typeMismatch.org.icgc.dcc.portal.server.config.ServerProperties$ReleaseProperties,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable:
           codes [target.release,release]; arguments []; default message [release]]; default message [Failed to convert property value of type [java.lang.String] to required type [org.icgc.dcc.portal.server.config.ServerProperties$ReleaseProperties] for property 'release'; nested exception is java.lang.IllegalStateException:
            Cannot convert value of type [java.lang.String] to required type [org.icgc.dcc.portal.server.config.ServerProperties$ReleaseProperties] for property 'release':
             no matching editors or conversion strategy found]; nested exception is org.springframework.beans.factory.BeanCreationException:
              Error creating bean with name 'hazelcastProperties' defined in class path resource [org/icgc/dcc/portal/server/config/ServerConfig.class]:
               Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException:
                Failed to instantiate [org.icgc.dcc.portal.server.config.ServerProperties$HazelcastProperties]:
                 Factory method 'hazelcastProperties' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException:
                  Error creating bean with name 'properties':
                   Could not bind properties to ServerProperties (prefix=, ignoreInvalidFields=false, ignoreUnknownFields=true, ignoreNestedProperties=false); nested exception is org.springframework.validation.BindException:
                    org.springframework.boot.bind.RelaxedDataBinder$RelaxedBeanPropertyBindingResult:
                     1 errors
Field error in object 'target' on field 'release':
                      rejected value [4.3.27]; codes [typeMismatch.target.release,typeMismatch.release,typeMismatch.org.icgc.dcc.portal.server.config.ServerProperties$ReleaseProperties,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable:
                       codes [target.release,release]; arguments []; default message [release]]; default message [Failed to convert property value of type [java.lang.String] to required type [org.icgc.dcc.portal.server.config.ServerProperties$ReleaseProperties] for property 'release'; nested exception is java.lang.IllegalStateException:
                        Cannot convert value of type [java.lang.String] to required type [org.icgc.dcc.portal.server.config.ServerProperties$ReleaseProperties] for property 'release':
                         no matching editors or conversion strategy found]
"


Action:

Review the stack trace and configuration

What’s confusing is the same set-up and configuration, I don’t have this problem on bare metal. Is it possible I have some networking error in connecting to elastic or something? Maybe there’s a dependency missing from my Dockerfile? I’m using the test-application.yml conf file.

Thanks,
Georgia

2

Any chance you can copy the portal.yml file that you are copying over, with any sensitive bits excluded.

Nothing jumps out to me as wrong with the dockerfile.

3

Yes, thanks. Here it is:

#
#
# DCC Portal Server - OHSU Development Configuration
#

###############################################################################
# Profile - "production"
###############################################################################

spring.profiles: ohsu-dev

# API
server:
  port: 9000
#  ssl:
#    enabled: true
#    keyStore: "<location>"
#    keyStorePassword: "<pass>"

# Datasource
spring.datasource:
  driver-class-name: org.h2.Driver
  url: jdbc:h2:mem:portal;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;INIT=runscript from 'classpath:sql/schema.sql'
  username:
  password:
  max-active: 10
  max-idle: 1
  min-idle: 1

# Authorization
crowd:
  ssoUrl: "https://ssoUrl?continue="
  ssoUrlGoogle: "https://ssoUrlGoogle?continue="

# ICGC auth server conig
oauth:
  serviceUrl: "http://localhost:8443"
  clientId: "mgmt"
  clientSecret: "pass"
  enableStrictSSL: false
  enableHttpLogging: true

# Authorization
auth:
  enabled: false

hazelcast:
  # Enable session replication?
  enabled: false

  # NB: Below hazelcast settings do not have default values. The portal won't start if they're unset

  # Credentials to enter a group. E.g. dev, uat, prog groups
  groupName: ""
  groupPassword: ""

  # Time-to-live settings
  # Maximum number of seconds for each entry to stay in the cache. Entries that are
  # older than the set value and not updated for that time
  # will get automatically evicted from the cache.

  # Expire inactive users in 15 minutes
  usersCacheTTL: 900

  # Expire openTd authentication attempt in 2 minutes
  openidAuthTTL: 120

  # Enable multicast? If false then hosts must be provided. hosts not used otherwise
  # Multicast must be disabled in an OpenStack /  AWS environment
  multicast: false
  hosts:
    - 127.0.0.1

# ElasticSearch index and hosts
elastic:
  indexName: icgc-release
  repoIndexName: icgc-repository
  nodeAddresses:
    - host: "<host ip>"
      port: 9300

  # List of TransportClient settings.
  # E.g.
  # "client.transport.sniff": true
  client:
    "client.transport.sniff": true

# Mail
mail:
  enabled: false
  recipientEmail:

web:
  # Defines an external URL when the portal is behind a reverse proxy / load balancer. E.g. shortUrl resource uses it for generation of valid URLs
  baseUrl: "http://localhost:8080"


# ICGC Client configuration
icgc:

  # CGP/DACO API endpoint
  cgpUrl: "http://localhost"

  # ShortUrl API endpoint
  shortUrl: "http://localhost"

  # CUD API endpoint
  cudUrl: "http://localhost"

  # CMS API endpoint
  cmsUrl: "http://localhost"

  # CUD credentials. Obtained from webdev@oicr.on.ca
  cudAppId: "<fill in>"
  cudUser: "<fill in>"
  cudPassword: "<fill in>"

  #OAuth credentials
  # To generate the credentials go to icgc.org, log inm go to the "OAuth Settings" tab and request "Regenerate all tokens"
  consumerKey: "<fill in>"
  consumerSecret: "<fill in>"
  accessToken: "<fill in>"
  accessSecret: "<fill in>"

  # Log HTTP requests to the API
  enableHttpLogging: false

  # Disallow self-signed SSL certificates
  enableStrictSSL: false

# Portal release configuration
release:
  releaseDate: "Month Day, Year"
  dataVersion: 2
4

It’s good practice that before any profile declaration you use --- which denotes the start of a new config document for spring.

So

---

spring.profiles: ohsu-dev

However I’d be doubtful if that is the cause of the issue.

5

Actually, that might be it exactly, give that a try.

6

I added that in into the conf file and I still get the same error. Any other thoughts?

7

Docker container originally used LinuX Containers (LXC), but later switched to runC (formerly known as libcontainer), which runs in the same operating system as its host. This allows it to share a lot of the host operating system resources. Also, it uses a layered filesystem (AuFS) and manages networking.

AuFS is a layered file system, so you can have a read only part and a write part which are merged together. One could have the common parts of the operating system as read only (and shared amongst all of your containers) and then give each container its own mount for writing.

So, let’s say you have a 1 GB container image; if you wanted to use a full VM, you would need to have 1 GB times x number of VMs you want. With Docker and AuFS you can share the bulk of the 1 GB between all the containers and if you have 1000 containers you still might only have a little over 1 GB of space for the containers OS (assuming they are all running the same OS image).

A full virtualized system gets its own set of resources allocated to it, and does minimal sharing. You get more isolation, but it is much heavier (requires more resources). With Docker you get less isolation, but the containers are lightweight (require fewer resources). So you could easily run thousands of containers on a host, and it won’t even blink. Try doing that with Xen, and unless you have a really big host, I don’t think it is possible.

A full virtualized system usually takes minutes to start, whereas Docker/LXC/runC containers take seconds, and often even less than a second.

There are pros and cons for each type of virtualized system. If you want full isolation with guaranteed resources, a full VM is the way to go. If you just want to isolate processes from each other and want to run a ton of them on a reasonably sized host, then Docker/LXC/runC seems to be the way to go.