Skip to main content

Manual Installation using Docker

info

This install process involves direct interaction with Docker, and is targeted at advanced users who have some experience with Docker. For an easier experience, consider using ChemCLI.

If you face problems, please let us know; we are there to support you. You may contact us directly if you plan to install Chemotion or have problems installing/using it.

Latest Version​

We distribute the ELN as docker images for ease of distribution and use. The latest images can be found on DockerHub and installed using a docker-compose.yml file. A version of docker-compose.yml file that uses these latest images can be found here.

Fresh Installations​

Important

Certain installations require may require additional/different steps. Please check this section to see if your desired version needs such treatment.

To install the latest version of Chemotion on a blank system, follow these steps:

  • download the docker-compose.yml to a directory of your choice:

    wget https://raw.githubusercontent.com/Chemotion/ChemCLI/main/payload/docker-compose.yml

  • edit the file to change these keys if they exist in the downloaded file:

    • services.worker.environment.SECRET_KEY_BASE and services.eln.environment.SECRET_KEY_BASE (both should be same)
    • services.converter.environment.SECRET_KEY

  • download all images and create the containers, data volumes and networks by running this command in the same folder you downloaded the compose file to:

    docker compose up --no-start

  • Start the ELN containers with

    docker compose up # outputs to stdout

    or

    docker compose up -d #starts the ELN as a background service logging to the docker's log daemon
    • after a short startup period, the ELN will available on port 4000 and can be accessed on <your host IP>:4000
    • The application is running on http://localhost:4000, the seeded administration account is ADM (all caps!) with password PleaseChangeYourPassword.
    • Proceed with Configuration
info

We rely on docker volumes to preserve data. Your user data will be stored in the volumes chemotion_data and chemotion_db. DO NOT delete them: Having backups is essential to making your data FAIR.

Controlling Services​

  • Services automatically restart when the docker daemon restarts. This can be configured by removing the lines containing restart: unless-stopped in the docker-compose.yml file or disabling autostart of your docker daemon on your system. Please refer to the docker's documentation on how this property works.

    services:
      db:
        ...
      worker:
        restart: unless-stopped
        ...
      eln:
        restart: unless-stopped
        ...

  • Following additional services are included in the docker-compose.yml file.

    ServiceIntroduced in VersionOptional
    Chemotion file format converter1.4.1Yes
    Ketcher render backend (ketchersvc)1.4.1Yes
    Spectra (spectra)1.1.2p220401Yes
    NMRium1.5.0Yes

    Optional services can be disabled by removing/commenting-out their entry in the docker-compose.yml file.

Network Configuration​

We strongly recommend getting in touch with the IT services for this part of the installation. Getting it wrong can have security implications for your data/network.

This documentation assumes that the exposed port is port 4000. If you change this, for example, by modifying the docker-compose.yml file, then please keep that in consideration.

The ELN runs inside a container and is therefore not available on your computer's network; other than on one exposed port, generally the port number 4000. To access the ELN locally, you need to go to http://localhost:4000. To make the installation available to the public, the container's port should be then linked to the external network. This can be achieved in two ways.

  • In case you have a fixed IP, other computers might be able to reach the ELN on ###.#.#.#:4000 where the hashes represent your IP. This generally only works within your local network or while you are using VPN.
  • Recommended: We suggest that you setup a reverse-proxy service. Such a service forwards a domain name (e.g. https://chemotion.dept.uni.de) to your ELN so that your users can visit the domain name to use the ELN. A rough guideline on how to do so is provided here.

Setting up a Reverse-Proxy​

We suggest to run an nginx reverse-proxy server if you don’t already have one. In case you want to use Shibboleth, please use Apache. You can also choose to run something similar like Traefik if you are familiar with the process.

The simplest way to setup nginx is as follows:

  1. Include nginx as a service (in form of a container) in the docker-compose.yml file that you use to run Chemotion ELN. In case you are using ChemCLI, the please include the service in docker-compose.cli.yml file instead. The service's description should look like as follows:
# This block is to be included in the `services` block.
nginx:
  image: nginx
  restart: always
  volumes:
    - ./nginx.conf:/etc/nginx/conf.d/nginx.conf:ro
    - ./ssl/:/ssl/:ro
  ports:
    - "443:443"
  networks:
    - proxy

Here we name a new network called proxy which is used to bridge the ELN's network and the host's network. To use it successfully, please also include the following at the end of your file:

networks:
  proxy:
    external: true

This declares the proxy network as external i.e. it is allowed to talk to the host's network. To link this proxy network with the ELN's network, include the it as a key-value pair for the eln service i.e. create a new YAML key called services.eln.networks: [proxy].

Additionally, you also need to create such a network explicitly by running the following command in your shell:

docker network create proxy
  1. Then, you need to create the nginx.conf file in the same folder as where you have the docker-compose file. The contents of the file should look as follows:
server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name chemotion.dept.uni.de;

    ssl_certificate /ssl/chemotion.dept.uni.de.crt;
    ssl_certificate_key /ssl/chemotion.dept.uni.de.key;

    resolver 127.0.0.11; # this bit is very important as it allows nginx and docker to cooperate

    location / {
        proxy_pass http://eln:4000;
    }
}

Here, you are accessing the files ....crt and ....key. These files are certificates that allow your connection to use encryption (i.e. use https). These certificate files should be provided to you by your IT department, or you can use a service like Let's Encrypt to generate them as long as your IT and firewall policies allow that. Please put these files in a folder called ssh and put this folder in the folder where you have your docker-compose.yml (and `docker-compose.cli.yml) file.

Further information on how to configure nginx, please refer to tutorials online (e.g. here and here).

NB: When setting the CSP headers, be sure to allow commonchemistry.cas.org (for creating sample by cas-rn) and dx.doi.org doi.org and api.crossref.org (to fetch literature references by doi). For example the directive could look like:

add_header Content-Security-Policy "default-src 'self'  'unsafe-inline' 'unsafe-eval';connect-src 'self' ws: blob:  commonchemistry.cas.org dx.doi.org doi.org api.crossref.org; font-src 'self' data: ; frame-src 'self' nmrium.chemotion.scc.kit.edu ; img-src 'self' data:" ;

To summarize, you have to make sure that:​

  • Reverse proxy is setup properly (i.e. DNS resolves to the URL to your server's IP address).
  • Any potential firewall/networking rules do not get in the way.
  • Docker exposes and listens on the correct ports.
  • The ELN is running.

Upgrades​

Important

Certain upgrades require may require additional/different steps. Please check this section to see if your current version and/or desired version needs such treatment.

Before Upgrades
  • ALWAYS perform a backup before doing upgrades. We cannot help you restore data if there is no backup and things go wrong during upgrade processes.
  • Inform your users of downtime that may accompany the upgrade.

  • Stop all running containers:

    docker compose down --remove-orphans

  • Delete the APP volume:

    docker volume rm chemotion_app

  • Follow instructions for a fresh installation in the same directory. However, your admin/user credentials and user data should remain same as they were before the upgrade.

Managing your instance​

To get access to the inside of the container, i.e to perform tasks based on the Rails console etc., one can use the following commands:

  • Getting a shell with loaded Chemotion environment variables

    This will drop you to a root shell inside the container. You are now free to perform any administrative tasks in the container context, but be aware that all changes are ephemeral and lost when the container is stopped. To access the host file system, a mount point has to be used, i.e. such as the already configured /shared (which - by default - maps to ./shared on the host).

    docker compose exec eln chemotion shell

  • Getting a Ruby on Rails console:

    docker compose exec eln chemotion railsc

  • Drop to postgreSQL console in the Chemotion database

    docker compose exec eln chemotion psql

  • Display basic version information

    docker compose exec eln chemotion info

  • Resetting the Administrator account's password

    docker compose exec eln chemotion resetAdminPW

Furthermore, you can use docker compose logs or check the ./shared/logs folder to access docker logs of your system.

Backing up and Restoring your data​

info

It is advised to setup an automatic system task (i.e. cron job) to backup regularly. This is highly system specific and thus out of the scope of this documentation. Please refer to your OS's manual to find out how to setup scheduled tasks.

The backup process, if using ChemCLI is described on its page.

In addition, you should always backup before upgrading your installation.

Warning

Backup scripts do not backup data that reside outside docker volumes e.g. services and shared folders. These folders need to be backed up separately by you.

To backup that resides inside docker volumes, it is sufficient to run the following command on your host machine in the folder where your Compose file resides:

docker compose run eln chemotion backup

This will place two files: backup.sql.gz and backup.data.tar.gz in the ./shared/backup folder.

To restore a backup, clean out the the folder where your Compose file resides, keeping only the backup folder and its files in place. Then run the following:

docker compose down --remove-orphans                           # stop all services
docker compose run -e FORCE_DB_RESET=1 eln chemotion restore   # restore the latest backup while resetting any database corresponding to this compose file, if already created
Details

How backup and restore works?This backup script is meant to simplify your life a bit when it comes to backups. However it doesn't do anything magical and if it suits your needs better, you can also do manual backups. To create one, you need to dump your database, i.e. with pg_dump or by copying the chemotion_db volume when the database is not running.



In addition, you need to save all your data, in the folders: /uploads and /public/images (the latter being more a convenience thing preventing you from having to recreate a lots of thumbnails after restoration). These folders are stored in chemotion_data volume, so you can mount it somewhere and use rsync/tar/cp to copy the data. If you already have a container mounting this volume, such as (such as the eln or worker services), you can also simply use docker compose cp or docker cp to copy the data to the host machine.



To restore a backup, invert the process: For restoration, you need to stop your services, then remount the backed-up volumes. If you have backup created using pg_dump then use pg_restore to restore it. Similarly, restore the data files into their destination using using cp/tar/rsync etc.

Special Cases​

Only when installing or upgrading to version 1.4.1​

This is not applicable in any other case, e.g. when moving from version v1.3.1 to v1.5.1.

Finish your installation or upgrade by following the usual steps. Then configure the converter service by following the steps described here:

In short, the following steps have to be taken:

  • create a directory to hold all its configuration
  • create a user for the ELN to authenticate at the service
  • configure the ELN to use the service and the created user

These steps would boil down to this:

mkdir -p ./shared/pullin/config/
# this makes sure a configuration directory for the ELN exists

mkdir -p ./services/converter/
# this creates a directory holding service configurations

docker run --entrypoint htpasswd httpd:2 -sbn mysuser mypassword | grep ':' > ./services/converter/htpasswd
# this creates a user in the service's authentication database

cat >> ./shared/pullin/config/converter.yml <<EOF
production:
  :url: 'http://converter/'
  :profile: MYUSER
  :secret_key: MYPASS
  :timeout: 300
  :ext:
    - '.xy'
    - '.xls'
    - '.xlsx'
    - '.txt'
    - '.brml'
    - '.dta'
    - '.pssession'
EOF

Upgrading from 1.0.3D0.1 to higher versions​

To upgrade from a previous version to this release, a few manual steps have to be done. In the new release, we changed to make use of docker volumes instead of bind mounts for all user data and shared storage. It is up to the user to transfer previous user data to these volumes.

  • You need to do is to copy/merge your old folders to/with the data volume (mounted at /chemotion/data in the ELN container):
    • ./shared/eln/uploads → /chemotion/data/uploads
    • ./shared/eln/public/images → /chemotion/data/public/images

We recommend the following steps:

  • Bring down your current instance, backup everything. In case anything goes wrong, you can always fall back to this backup by simply extracting the archive.

      docker compose down --remove-orphans
      sudo tar cvzf /tmp/pre-upgrade.tar.gz ./db-data ./shared ./docker-compose.yml

  • clean your directory, since the new containers will also write to ./shared.

    rm docker-compose.yml
    sudo mv ./shared ./old

  • download the docker-compose.yml

    wget https://raw.githubusercontent.com/Chemotion/ChemCLI/main/payload/docker-compose.yml

  • download all images and create the containers, data volumes and networks by running this command in the same folder you downloaded the compose file to

    docker compose create

  • start a disposable sidecar container with an interactive shell attaching your old data and old db stores in addition to the storages defined in docker-compose.yml:

    docker run -v $(pwd)/old:/old \
              -v chemotion_data:/new \
              -v $(pwd)/db-data:/old/db \
              -v chemotion_db:/new/db \
              -it --rm ubuntu:latest bash

  • You will find yourself in the root of a fresh container. The point here is that you copy everything from your old uploads and public/images folder to the new location. By default those are stored in ./eln/, if you configured things differently, adjust accordingly.

    cp -Rf /old/eln/uploads/.        /new/uploads/
    cp -Rf /old/eln/public/images/.  /new/public/images/
    cp -Rf /old/db/.                 /new/db/

  • Your data is now stored on the data volumes. Type exit to quit the interactive shell.

  • start your new instances:

    docker compose up -d

  • after a short startup/migration period, the ELN will available on port <your host IP>:4000

  • Proceed with Configuration.