hi, i'm tmac

GitHub Actions with Private SPM Dependencies

With the projects that I work on, I found myself repeating the same steps over and over for deployment:

• Git tag release X.X.X
• Build Docker tagged release organization/repo:X.X.X
• Build (or just retag previous) Docker image as organization/repo:develop for staging servers

Given that the git tags always correspond to a Docker image tag, I wondered if it would be possible to have GitHub actions build and tag the images for me when a new tag was pushed to the repo. Turns out this wasn't too difficult (with a couple gotcha's).

From searching other repo examples I was able to get 95% of the way there just copying existing actions. The main problems came from getting the git tag in the action as a variable, and setting up permissions on both the DockerHub and GitHub sides of things.

SSH setup

In a previous post Docker with private SPM dependencies I discussed the extra user I setup for my GitHub organization:

For my organization, I have setup a dummy build user that only has read access to all our private repos. I generated a new ssh key on my computer (storing it in ~/.ssh/github_rsa). Then I uploaded the public key to github under that build user's account. I didn't want my private key to be in the dockerfile at any point so this is why I chose that route.

For this setup I generated another SSH key for that account (where you store it does not matter as you will not use it directly, but this one was ~/.ssh/github_ci). The reason I did not reuse the existing one is I wanted to separate who uses each key so that they could be revoked if needed and it wouldn't ruin everything. For example github_rsa.pub is named hiimtmac builder on the dummy user's account, with the idea being that others could generate keys for that user as well. This new key github_ci.pub is named cicd builder on the dummy user's account. If you don't have access to a dummy build user, you can just create another key and add it to your personal account.

You will need to get the value of the private key so it can be added to GitHub as a secret. The gotcha here is that all the line breaks in the file need to be replaced with \n so that its a single string with no whitespace.

For example the below key:

-----BEGIN OPENSSH PRIVATE KEY-----
aaabbbcccdddeeefffggghhhiiijjjkkklllmmm
nnnooopppqqqrrrssstttuuuvvvwwwxxxyyyzzz
abcdefghijklmnopqrstuvwxyz==
-----END OPENSSH PRIVATE KEY-----

Needs to be represented like this:

-----BEGIN OPENSSH PRIVATE KEY-----\naaabbbcccdddeeefffggghhhiiijjjkkklllmmm\nnnnooopppqqqrrrssstttuuuvvvwwwxxxyyyzzz\nabcdefghijklmnopqrstuvwxyz==\n-----END OPENSSH PRIVATE KEY-----

I will refer to this key as SSH_KEY

Docker Setup

Following along with the GitHub setup, I did the same for DockerHub. I created a new dummy DockerHub account (named DOCKER_USER for the purposes of this tutorial). Then I added it to our organization, placing it in a team that I called CICD which has read + write access to the repos I want to build with GitHub actions. I also generated an access token for the user (called DOCKER_TOKEN for the purposes of this tutorial).

Github Secrets

Now you can create GitHub secrets to use in your action files. You can create these secrets at the organization level, or at the repo level. If you want to use them for many repos, making them on the organization level and then granting repo access to said secrets seems like the way to go.

I created 3 secrets and then delegated access to these secrets to the repos that needed them:

• DOCKERHUB_USERNAME with the value of DOCKER_USER
• DOCKERHUB_TOKEN with the value of DOCKER_TOKEN
• CICD_SSH with the value of SSH_KEY

Actions File

Now you can create your actions file. This is pretty stock but the first step is what took a while to figure out through searching various examples. The gotcha is the ${GITHUB_REF} variable by default looks something like refs/tags/X.X.X but I needed the tag only (ie X.X.X). This step strips the tag only and applies it to the environment where it can be used later and applied with ${{ env.TAG }}. Below is the file I settled on. It should be stored as .github/workflows/docker.yml in your project's root directory (feel free to change the file name or action name).

name: docker-deploy

on:
  push:
    tags:
      - "*"

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - name: Set Tag
        run: echo "TAG=${GITHUB_REF##*/}" >> $GITHUB_ENV
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v1
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1
      - name: Login to DockerHub
        uses: docker/login-action@v1
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - name: Build and push
        id: docker_build
        uses: docker/build-push-action@v2
        with:
          push: true
          tags: |
            hiimtmac/fake-repo:${{ env.TAG }}
            hiimtmac/fake-repo:develop
          build-args: |
            GITHUB_SSH=${{ secrets.CICD_SSH }}
      - name: Image digest
        run: echo ${{ steps.docker_build.outputs.digest }}

This action is set to only run when a tag is applied. I know that's not really true CI/CD but it accomplishes what I needed so that I only have to do 1 of the steps mentioned at the top and theres no way to screw up tagging between git and docker. Note the SSH key being grabbed from the secrets instead of docker build --build-arg GITHUB_SSH=... like we did locally.

Closing

Make sure the GitHub dummy user has at least read access to the repositories you want to build (and their dependencies if they are private). Also make sure the DockerHub dummy user has write privileges to the repositories you want to push the built images to.

With a little tweaking, I have implemented this strategy to build Vapor, NPM, Laravel, and Rails projects. If you have private NPM dependencies, you can use a similar strategy to what we discussed in docker with private NPM dependencies. Instead of an SSH key, you can create another Personal access token, add it to secrets, and use that as your build argument (assuming this private NPM dependency is hosted on GitHub as it was in that example).

As always, if you know of a better way to do something, please let me know!