Skip to content

DEPLOY.md

Latest commit

 

History

History
185 lines (140 loc) · 6.49 KB

DEPLOY.md

File metadata and controls

185 lines (140 loc) · 6.49 KB
Raw

Deployment

Deployment uses dokku and requires the environment variables defined in dotenv-sample. It is deployed to our dokku3 instance.

Deployment instructions

Create app

On dokku3, as the dokku user:

dokku$ dokku apps:create opencodelists
dokku$ dokku domains:add opencodelists www.opencodelists.org

Create storage for sqlite db and backups and load db into it

dokku$ mkdir /var/lib/dokku/data/storage/opencodelists
dokku$ chown dokku:dokku /var/lib/dokku/data/storage/opencodelists
# If we have an existing db to load in
dokku$ cp ./opencodelists-db.sqlite3 /var/lib/dokku/data/storage/opencodelists/db.sqlite3
dokku$ chown dokku:dokku /var/lib/dokku/data/storage/opencodelists/*
dokku$ dokku storage:mount opencodelists /var/lib/dokku/data/storage/opencodelists/:/storage

Add block storage to the droplet with the following steps:

  • Log in to the DigitalOcean Dashboard
  • Go to Droplets
  • Select dokku3
  • Select Volumes tab
  • Press the Add Volume button
  • Use the modal to configure, create, and mount the volume, selecting the droplet name, desired volume size, "Automatically format and mount", and "EXT4".

Run the following commands to set up the mount manually:

mkdir -p /mnt/volume_opencodelists_backups/opencodelists
sudo chown -R 10003:10003 /mnt/volume_opencodelists_backups/opencodelists
sudo chmod 755 /mnt/volume_opencodelists_backups/opencodelists
dokku storage:mount opencodelists /mnt/volume_opencodelists_backups/opencodelists:/block_storage

You can resize block storage to increase its size but never decrease. You must ensure that nothing is writing to the volume while you do this to avoid data corruption. Steps:

  • Log in to the DigitalOcean Dashboard
  • Go to Droplets
  • Select dokku3
  • Select Volumes tab
  • Select "Increase storage size" from the "..." menu against the volume.
  • Increase by the required amount in the modal and click the "Increase storage size" button.

Then SSH to the droplet to resize the file system to use all available blocks:

# Check the free disk on the mounted device before your change.
df /mnt/volume_opencodelists_backups/ -h
# Check the size of the block device.
lsblk
# Resize the device in the file system to use all available blocks.
sudo resize2fs /dev/sda
# Check the free disk on the mounted device after your change.
df /mnt/volume_opencodelists_backups/ -h

Configure app

# set environment variables
dokku$ dokku config:set opencodelists IN_PRODUCTION=True
dokku$ dokku config:set opencodelists BASE_URLS='https://www.opencodelists.org'
dokku$ dokku config:set opencodelists DATABASE_URL='sqlite:////storage/db.sqlite3'
dokku$ dokku config:set opencodelists DATABASE_DIR='/storage'
dokku$ dokku config:set opencodelists SECRET_KEY='xxx'
dokku$ dokku config:set opencodelists SENTRY_DSN='https://xxx@xxx.ingest.sentry.io/xxx'
dokku$ dokku config:set opencodelists OTEL_EXPORTER_OLTP_ENDPOINT='https://api.honeycomb.io'
dokku$ dokku config:set opencodelists OTEL_EXPORTER_OTLP_HEADERS='x-honeycomb-team=VALIDKEY1234,x-honeycomb-dataset=opencodelists'
dokku$ dokku config:set opencodelists OTEL_SERVICE_NAME='opencodelists'
dokku$ dokku config:set opencodelists DJANGO_SETTINGS_MODULE="opencodelists.settings"
dokku$ dokku config:set opencodelists EMAIL_BACKEND='anymail.backends.mailgun.EmailBackend'
dokku$ dokku config:set opencodelists MAILGUN_API_KEY='xxx'
dokku$ dokku config:set opencodelists BLOCK_STORAGE_DIR='/block_storage'

Configure nginx

Dokku does most of the nginx configuration for us, however specific config can be updated with the nginx:set command.

To allow uploads of CSV files that are larger than the default allowed by nginx, run:

dokku$ dokku nginx:set opencodelists client-max-body-size 20m

Restart the app to regenerate the nginx config:

dokku$ dokku ps:restart opencodelists

View nginx config:

dokku$ dokku nginx:show-config opencodelists

Backups

Backups are defined as cron jobs in app.json, and managed by dokku

Check cron tasks:

dokku$ dokku cron:list opencodelists

Backups are saved to /var/lib/dokku/data/storage/opencodelists/backup on dokku3.

Manually deploying

Merges to the main branch will trigger an auto-deploy via GitHub actions.

Note this deploys by building the prod docker image (see docker/docker-compose.yaml) and using the dokku git:from-image command.

To deploy manually:

# build prod image locally
just docker-build prod

# tag image and push
export PUBLIC_IMAGE_NAME=ghcr.io/opensafely-core/opencodelists
docker tag opencodelists "$PUBLIC_IMAGE_NAME:latest"
docker push "$PUBLIC_IMAGE_NAME:latest"

# get the SHA for the latest image
SHA=$(docker inspect --format='{{join .RepoDigests "\n"}}' "$PUBLIC_IMAGE_NAME:latest" \
| grep --fixed-strings "$PUBLIC_IMAGE_NAME")

On dokku3, as the dokku user:

dokku$ dokku git:from-image opencodelists <SHA>

extras

Requires the sentry-webhook and letsencrypt plugins.

# Check plugins installed:
dokku$ dokku plugin:list

# enable letsencrypt (must be run as root)
root$ dokku config:set --no-restart opencodelists DOKKU_LETSENCRYPT_EMAIL=<e-mail>
root$ dokku letsencrypt:enable opencodelists

# turn on/off HTTP auth (also requires restarting the app)
dokku$ dokku http-auth:on opencodelists <user> <password>
dokku$ dokku http-auth:off opencodelists

Mapping source files

When updating a mapping you can copy the mapping file(s) to dokku via scp. You should probably test that you can import it in a local development environment first to ensure that there are no issues with it. By convention, we place the mapping files to be imported in /var/lib/dokku/data/storage/mappings/bnfdmd which is mapped within the container to /storage/mappings/bnfdmd.

cron jobs

We use a mixture of dokku-managed (see Backups ) and self-managedcron jobs.

For the latter, these are configured via a cronfile in deploy/bin/ which should be copied to /etc/cron.d/ using copy_cronfile.sh as sudo.

All cron jobs (both dokku- and self-managed) call functions from sentry_cron_functions.sh to enable Sentry cron job monitoring. The contents of deploy/bin should be copied to /var/lib/dokku/data/storage/opencodelists/deploy/bin (outside container) or /storage/deploy/bin (within container) to support this.

This sentry monitoring requires that the SENTRY_DSN environment variable is set (see above)