April 3, 2020

Deploying a Gitlab Runner on AWS with Packer

Introduction

This post was originally published on the 56k.cloud blog.

Gitlab.com allows its users to run their own, self-hosted, Gitlab Runners as part of their Gitlab CI feature. This comes in handy when the CI jobs have to run in a user-owned environment or more powerful runners are needed. If that environment is AWS and you are looking for an automated way of setting up the Runner(s) on EC2, then stay tuned for a Hostatic recipe!

Requirements

  • a gitlab.com account
  • a repository that needs a CI / CD set up
  • the Gitlab runner registration token. This can be found in the above mentioned repository settings -> CI / CD -> Runners -> Specific Runners -> Set up a specific runner manually
  • install Hashicorp’s Packer on your build machine

Steps

git clone https://github.com/danakim/packer.git

  • build the Gitlab runner image using Packer:

cd packer/gitlab-runner packer build -var 'aws_access_key=<your_access_key_id>' -var 'aws_secret_key=<your_access_key>' gitlab-runner.json

  • login the AWS Console and check the EC2 -> AMIs section to confirm that the new Gitlab runner AMI is there. Make a note of the AMI ID.
  • launch a new EC2 instance using the AMI ID found in the previous step. Call the following script from the User data section of EC2’s instance launch options to register the runner with gitlab.com:

/home/ubuntu/gitlab-runner-register.sh <environment (e.g.: dev)> <gitlab_cicd_token>

  • going back to the repository’s settings -> CI / CD -> Runners -> Specific Runners, this new runner should be available in the list. Click on Enable for this project to make the runner usable by the repository.

Next steps

At this point, the Gitlab runner is attached to the project. When defining the Gitlab CI / CD pipeline in .gitlab-ci.yml each job will have to use a tag to run that job on the new Gitlab runner. This tag matches the environment passed to the gitlab-runner-register.sh script above. Example:

job: script: - echo "Hello world!" tags: - <environment (e.g. dev)>

Summary

Now that we have a reusable image (AMI) for a self-hosted Gitlab runner, it can be used as a building block for further automation.

One possible next improvement is to use this AMI and the User Data script in an infrastructure-as-code DSL like Terraform or Cloudformation. But that’s a subject for a future blog post!