hi, i'm tmac

Docker with Nginx and Let's Encrypt

Posted Wednesday, February 10, 2021.

Serving web content over https is a must these days. If your webserver is a Docker container, it might be tempting to install nginx and Let's Encrypt in that image. However your image would no longer be encapsulating a single function. Instead, we can use multiple containers and Docker Compose to achieve our goal.

Docker Compose

First, let's take a look at the docker-compose.yml setup and then discuss each part. In this example we use 4 containers:

  • certbot/certbot:latest - Let's Encrypt container for certificates
  • nginx:alpine - nginx webserver using the slim alpine variant
  • hiimtmac/api:latest (fake api container)
  • hiimtmac/web:latest (fake web container)

Note:

  • Both api and web containers are not necessary, you could have one or the other.
  • In this example, the web container talks to the api container, but the api container is also available for external services.
  • I chose to have both web and api use the same domain/subdomain. This is also not required, but you would need separate server blocks if the domains/subdomains were to be handled differently.
  • Volumes are mounted in ro (read only) mode wherever possible
version: "3.7"

services:
  certbot:
    image: certbot/certbot:latest
    volumes:
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
  nginx:
    image: nginx:alpine
    volumes:
      - ./data/nginx:/etc/nginx/templates:ro
      - ./data/certbot/conf:/etc/letsencrypt:ro
      - ./data/certbot/www:/var/www/certbot:ro
      - shared-data:/var/www/app:ro
    ports:
      - "80:80"
      - "443:443"
    environment:
      - SERVER_NAME=hiimtmac.com
    depends_on:
      - certbot
      - api
      - web
  api:
    image: hiimtmac/api:latest
    env_file:
      - .env.api
  web:
    image: hiimtmac/web:latest
    env_file:
      - .env.web
    depends_on:
      - api
    volumes:
      - shared-data:/app

volumes:
  shared-data:

File Setup:

.
├── data
│   ├── certbot
│   |   ├── conf/
│   │   └── www/
│   └── nginx
│       ├── includes
│       │   ├── ssl.conf.template
│       │   ├── proxy.conf.template
│       │   └── ...
│       └── nginx.conf.template
├── .env.web
├── .env.api
└── docker-compose.yml

Normally, you would not have runtime configuration for a React application as it builds in all environment variables when compiling. However, in a previous post, we discussed how this could be achieved, hence the .env.web environment file.

Nginx Setup

We will be using the template feature of the nginx docker image, which will allow us to have some runtime conditions and variables (this is not necessary, if you do not want to use templates you can map your nginx configs right to /etc/nginx/conf.d/).

nginx.conf.template

...

# default block
server {
    listen 80 default_server;
    listen [::]:80 default_server;
    listen 443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;
    server_name _;

    include conf.d/includes/ssl.conf;

    return 404;
}

# 80 block
server {
    listen 80;
    listen [::]:80;
    server_name ${SERVER_NAME};

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

# 443 block
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name ${SERVER_NAME};

    include conf.d/includes/ssl.conf;
    include conf.d/includes/proxy.conf;
    ...

    resolver 127.0.0.11;

    # back end
    location /api/ {
        ...
        set $upstream_api api;
        proxy_pass http://$upstream_api:8080;
        ...
    }

    root /var/www/app;

    # front end
    location / {
        ...
        try_files $uri $uri/ @proxy;
        ...
    }

    location @proxy {
        set $upstream_web web;
        proxy_pass http://$upstream_web:8080;
    }
}

Let's break it down into smaller pieces.

# default block
server {
    listen 80 default_server;
    listen [::]:80 default_server;
    listen 443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;
    server_name _;

    include conf.d/includes/ssl.conf;

    return 404;
}

The default server is setup to return 404 on any server name by default. We setup listeners on IPV4 and IPV6 on both port 80 and port 443 with ssl and http2. We include the ssl.conf file as it is needed for https traffic.

...
# 80 block
server {
    listen 80;
    listen [::]:80;
    server_name ${SERVER_NAME};

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

The 80 server block accepts IPV4 and IPV6 traffic on port 80 (regular http) for server name we specified in the docker-compose.yml file. In this example it would resolve to server_name hiimtmac.com;. The first location block is used to handle certificates from Let's Encrypt. The second location block is used to redirect and upgrade any http traffic to https - ie sending it to the 443 server block.

# 443 block
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name ${SERVER_NAME};

    include conf.d/includes/ssl.conf;
    include conf.d/includes/proxy.conf;

    resolver 127.0.0.11;
    ...
}

Similar to the 80 server block, the 443 server block block accepts IPV4 and IPV6 traffic on the server name we specified in the docker-compose.yml file (hiimtmac.com;). Instead of port 80, it uses port 443 and specifies ssl and http2. We include the ssl.conf file which is needed for https traffic. We also include the proxy.conf file. The contents of each of these files could be inlined, but it groups functionality together and makes the main conf file slightly shorter. The resolver line is necessary as we need Docker's internal DNS for routing to our containers.

# 443 block
server {
    ...
    # back end
    location /api/ {
        set $upstream_api api;
        proxy_pass http://$upstream_api:8080;
    }
    ...
}

The location /api/ block will send any matching url to our backend api container. For example hiimtmac.com/api/test and hiimtmac.com/api/cool would match, however hiimtmac.com/test/api would not. This way, the web container could make a call to hiimtmac.com/api/users and it would be routed to the api container, and not call itself. When we are bringing up our containers to get certificates (in the next section), we want the nginx configuration to be valid even when the web and api containers are not live. In order to satisfy this need, we use set $upstream_api api; and then proxy_pass http://$upstream_api:8080 as opposed to just proxy_pass http://api:8080. This way we don't have to start all the containers in order for nginx to be happy.

# 443 block
server {
    ...
    root /var/www/app;

    # front end
    location / {
        ...
        try_files $uri $uri/ @proxy;
        ...
    }

    location @proxy {
        set $upstream_web web;
        proxy_pass http://$upstream_web:8080;
    }
}

The location / block will catch anything that our location /api/ block did not. We have set the root to what we mapped in from the web container, which we can then use for try_files. This will attempt to serve static files from the specified directory before falling back to the web container. We use the @proxy block to achieve the same goal as the api block for the proxy_pass.

includes/ssl.conf.template

ssl_certificate /etc/letsencrypt/live/${SERVER_NAME}/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/${SERVER_NAME}/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

This configuration file will substitute the server name from your compose file (in this case hiitmac.com), as well as provide the configurations that nginx needs (we will download in next section) to work with Let's Encrypt to serve https traffic.

includes/proxy.conf.template

# Basic Proxy Config
proxy_set_header    Host                $http_host;
proxy_set_header    X-Real-IP           $remote_addr;
proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;

This is more basic information. You can add advanced settings as well.

General

Additionally, you will want to include other settings, for example the options below:

charset              utf-8;
server_tokens        off;
log_not_found        off;

You can either inline them into your nginx.conf.template or you could make them as a separate file in the includes directory, and then include them in the 443 block along where you link the ssl.conf and proxy.conf. I am by no means an expert at nginx configuration, but this has worked for me so far.

Certbot Setup

I got a great deal of information from GitHub user wmnnd's repo on setting this up, and have since made some tweaks myself. I've separated it into 3 steps:

  • Setup - download configurations, setup folders, create keys
  • Staging - try to get a ssl certificate from Let's Encrypt, but in staging mode (production only allows a certain amount of attempts per day so tinkering with settings will block you after a while)
  • Production - do it for real and get the production certificate

I would recommend creating a scripts directory alongside your data directory with the following 3 files (if you don't house them there, be sure to adjust the cd ../ command in the following scripts).

Setup

The setup.sh script looks like this:

#!/bin/bash

if ! [ -x "$(command -v docker-compose)" ]; then
  echo 'Error: docker-compose is not installed.' >&2
  exit 1
fi

CERT_NAME=$1
cd ../

echo "Download TLS parameters"
mkdir -p "./data/certbot/conf"
curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf > "./data/certbot/conf/options-ssl-nginx.conf"
curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot/certbot/ssl-dhparams.pem > "./data/certbot/conf/ssl-dhparams.pem"
echo

echo "make keys"
mkdir -p "./data/certbot/conf/live/$CERT_NAME"
docker-compose run --rm --no-deps --entrypoint "\
  openssl req -x509 -nodes -newkey rsa:1024 -days 1\
    -keyout '/etc/letsencrypt/live/$CERT_NAME/privkey.pem' \
    -out '/etc/letsencrypt/live/$CERT_NAME/fullchain.pem' \
    -subj '/CN=localhost'" certbot

When you run it, you will include the root url for the certificate name. In our example, I would run this script with ./setup.sh hiimtmac.com.

Staging

The staging.sh script looks like this:

#!/bin/bash

if ! [ -x "$(command -v docker-compose)" ]; then
  echo 'Error: docker-compose is not installed.' >&2
  exit 1
fi

MAIN_DOMAIN=$1
ADDL_DOMAINS=$2

if [ ! -z "$ADDL_DOMAINS" ]
then
DOMAINS="$MAIN_DOMAIN,$ADDL_DOMAINS"
else
DOMAINS=$MAIN_DOMAIN
fi

cd ../

echo "run nginx"
docker-compose up --force-recreate --no-deps -d nginx
echo

echo "delete old certiticates"
docker-compose run --rm --no-deps --entrypoint "\
  rm -Rf /etc/letsencrypt/live/$MAIN_DOMAIN && \
  rm -Rf /etc/letsencrypt/archive/$MAIN_DOMAIN && \
  rm -Rf /etc/letsencrypt/renewal/$MAIN_DOMAIN.conf" certbot
echo

echo "get staging certificate to test config"
docker-compose run --rm --no-deps --entrypoint "\
  certbot certonly \
    --webroot \
    --register-unsafely-without-email \
    --agree-tos \
    --staging \
    --webroot-path /var/www/certbot \
    --rsa-key-size 4096 \
    --force-renewal \
    --domains $DOMAINS" certbot
echo

echo "stop nginx"
docker-compose down --remove-orphans

When you run it, you will include the root url for the certificate name, but also you have the option to include other domains in the certificate. In our example, I would run this script with ./staging.sh hiimtmac.com, but I could also do ./staging.sh hiimtmac.com blog.hiimtmac.com (and then I would need to include the blog subdomain somewhere in the nginx config).

Production

The production.sh script looks like this:

#!/bin/bash

if ! [ -x "$(command -v docker-compose)" ]; then
  echo 'Error: docker-compose is not installed.' >&2
  exit 1
fi

EMAIL=$1
MAIN_DOMAIN=$2
ADDL_DOMAINS=$3

if [ ! -z "$ADDL_DOMAINS" ]
then
DOMAINS="$MAIN_DOMAIN,$ADDL_DOMAINS"
else
DOMAINS=$MAIN_DOMAIN
fi

cd ../

echo "run nginx"
docker-compose up --force-recreate --no-deps -d nginx
echo

echo "delete old certiticates"
docker-compose run --rm --no-deps --entrypoint "\
  rm -Rf /etc/letsencrypt/live/$MAIN_DOMAIN && \
  rm -Rf /etc/letsencrypt/archive/$MAIN_DOMAIN && \
  rm -Rf /etc/letsencrypt/renewal/$MAIN_DOMAIN.conf" certbot
echo

echo "get production certificate"
docker-compose run --rm --no-deps --entrypoint "\
  certbot certonly \
    --webroot \
    --email $EMAIL \
    --no-eff-email \
    --agree-tos \
    --webroot-path /var/www/certbot \
    --rsa-key-size 4096 \
    --force-renewal \
    --domains $DOMAINS" certbot
echo

echo "stop nginx"
docker-compose down --remove-orphans

The production script is almost the same as the staging one, except there are a couple parameters changed when grabbing the certificate, such as providing an email. This script would be used as such: ./production.sh taylor@hiimtmac.com hiimtmac.com.

When you run this, it will generate a production SSL certificate for you. Now you can run docker-compose up -d and serve your content with https! The custom entrypoint for the certbot image will handle renewals for you so you do not have to worry about that.

Volumes

In order to serve static content from the web container through nginx, we created a volume and then mounted it to the web and nginx containers. This creates a volume in which the content from the web's public folder can then be accessed by the nginx container. I'm not sure this is the best way to achieve this goal as it comes with a few caviats:

  • You can't mount this directory to the host. If you do, the empty files of the host will overwrite the contents of the web container, and then the nginx container will just see an empty directory.
  • When re-deploying changes, you need to docker-compose down -v (remove volumes) before docker-compose up -d. If you don't, then the nginx container still has a copy of those files in it's container, and overwrites the newer files of the web container when you bring it up.
  • If your api container included a public directory and you were not using a separate front-end container, you would map its public folder instead of how we did with the web container.

If you don't need this functionality, you can delete the shared-data volume from the compose file, the volume section from both web and nginx containers, and the root/try_files parts of the nginx.conf.template file.

In Closing

This deployment has been resiliant and easily replicatable for me in multiple production servers. I've configured it with the following setups without much change between configs:

  • Nginx, Certbot, Laravel*
  • Nginx, Certbot, Rails (puma)
  • Nginx, Certbot, Vapor
  • Nginx, Certbot, Vapor, React

*The Laravel config for nginx was different because of php-fpm.

I had to figure this stuff out from a lot of trial and error, and research. If there is a better way to go about this, please let me know!

Tagged With: