Category Archives: GIT

GitLab CI: Deployment & Environments

This post is a success story of one imaginary news portal, and you’re the happy owner, the editor, and the only developer. Luckily, you already host your project code on GitLab.com and know that you can run tests with GitLab CI. Now you’re curious if it can be used for deployment, and how far can you go with it.

To keep our story technology stack-agnostic, let’s assume that the app is just a set of HTML files. No server-side code, no fancy JS assets compilation.

Destination platform is also simplistic – we will use Amazon S3.

The goal of the article is not to give you a bunch of copypasteable snippets. The goal is to show principles and features of GitLab CI so that you could easily apply them to your technology stack.

Let’s start from the beginning: no CI in our story yet.

A Starting Point

Deployment: in your case, it means that a bunch of HTML files should appear on your S3 bucket (which is already configured for static website hosting).

There’re a million ways to do it. We’ll use the awscli library, provided by Amazon.

The full command looks like this:

aws s3 cp ./ s3://yourbucket/ --recursive --exclude "*" --include "*.html"

Manual deploymentPushing code to repository & deploing are separate processes

Important detail: the command expects you to provide AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. Also you might need to specify AWS_DEFAULT_REGION.

Let’s try to automate it using GitLab CI.

First Automated Deployment

With GitLab, there’s no difference on what commands to run. You can setup GitLab CI according to your needs as if it was your local terminal on your computer. As long as you execute commands there, you can tell CI to do the same for you in GitLab. Put your script to.gitlab-ci.yml and push your code – that’s it: CI triggers a job and your commands are executed.

Let’s add some context to our story: our website is small, there is 20-30 daily visitors and the code repository has only one branch:master.

Let’s start by specifying a job with the command from above in .gitlab-ci.yml:

deploy:
  script: aws s3 cp ./ s3://yourbucket/ --recursive --exclude "*" --include "*.html"

No luck: Failed command

It is our job to ensure that there is an aws executable. To install awscli we need pip, which is a tool for Python packages installation. Let’s specify Docker image with preinstalled Python, which should contain pip as well:

deploy:
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://yourbucket/ --recursive --exclude "*" --include "*.html"

Automated deploymentYou push your code to GitLab, and it is automatically deployed by CI

The installation of awscli extends the job execution time, but it is not a big deal for now. If you need to speed up the process, you can always look for a Docker image with preinstalled awscli, or create an image by yourself.

Also, let’s not forget about these environment variables, which you’ve just grabbed from AWS Console:

variables:
  AWS_ACCESS_KEY_ID: "AKIAIOSFODNN7EXAMPLE"
  AWS_SECRET_ACCESS_KEY: “wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY”
deploy:
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://yourbucket/ --recursive --exclude "*" --include "*.html"

It should work, however keeping secret keys open, even in a private repository, is not a good idea. Let’s see how to deal with it.

Keeping Secret Things Secret

GitLab has a special place for secret variables: Settings > Variables

Picture of Variables page

Whatever you put there will be turned into environment variables. Only an administrator of a project has access to this section.

We could remove variables section from our CI configuration. However, let’s use it for another purpose.

Specifying and Using Non-secret Variables

When your configuration gets bigger, it is convenient to keep some of the parameters as variables at the beginning of your configuration. Especially if you use them in multiple places. Although it is not the case in our situation yet, let’s set the S3 bucket name as a variable, for demonstration purposes:

variables:
  S3_BUCKET_NAME: "yourbucket"
deploy:
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://$S3_BUCKET_NAME/ --recursive --exclude "*" --include "*.html"

So far so good:

Successful build

Because the audience of your website grew, you’ve hired a developer to help you. Now you have a team. Let’s see how teamwork changes the workflow.

Dealing with Teamwork

Now there’s two of you working in the same repository. It is no longer convenient to use the master branch for development. You decide to use separate branches for both new features and new articles and merge them into master when they are ready.

The problem is that your current CI config doesn’t care about branches at all. Whenever you push anything to GitLab, it will be deployed to S3.

Preventing it is straightforward. Just add only: master to your deploy job.

Automated deployment of master branchYou don’t want to deploy every branch to the production website

But it would also be nice to preview your changes from feature-branches somehow.

Setting Up a Separate Place for Testing

Patrick (the guy you recently hired) reminds you that there is such a thing called GitLab Pages. It looks like a perfect candidate for a place to preview your work in progress.

To host websites on GitLab Pages your CI configuration should satisfy these simple rules:

  • The job should be named pages
  • There should be an artifacts section with folder public in it
  • Everything you want to host should be in this public folder

The contents of the public folder will be hosted at http://<username>.gitlab.io/<projectname>/

After applying the example config for plain-html websites, the full CI configuration looks like this:

variables:
  S3_BUCKET_NAME: "yourbucket"

deploy:
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://$S3_BUCKET_NAME/ --recursive --exclude "*" --include "*.html"
  only:
  - master

pages:
  image: alpine:latest
  script:
  - mkdir -p ./public
  - cp ./*.html ./public/
  artifacts:
    paths:
    - public
  except:
  - master

We specified two jobs. One job deploys the website for your customers to S3 (deploy). The other one (pages) deploys the website to GitLab Pages. We can name them “Production environment” and “Staging environment”, respectively.

Deployment to two placesAll branches, except master, will be deployed to GitLab Pages

Introducing Environments

GitLab offers support for environments, and all you need to do it to specify the corresponding environment for each deployment job:

variables:
  S3_BUCKET_NAME: "yourbucket"

deploy to production:
  environment: production
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://$S3_BUCKET_NAME/ --recursive --exclude "*" --include "*.html"
  only:
  - master

pages:
  image: alpine:latest
  environment: staging
  script:
  - mkdir -p ./public
  - cp ./*.html ./public/
  artifacts:
    paths:
    - public
  except:
  - master

GitLab keeps track of your deployments, so you always know what is currently being deployed on your servers:

List of environments

GitLab provides full history of your deployments per every environment:

List of deployments to staging environment

Environments

Now, with everything automated and set up, we’re ready for the new challenges that are just around the corner.

Deal with Teamwork Part 2

It has just happened again. You’ve pushed your feature-branch to preview it on staging; a minute later Patrick pushed his branch, so the Staging was re-written with his work. Aargh!! It was the third time today!

Idea! Let’s use Slack to notify us of deployments, so that people will not push their stuff if another one has been just deployed!

Slack Notifications

Setting up Slack notifications is a straightforward process.

The whole idea is to take the Incoming WebHook URL from Slack… Grabbing Incoming WebHook URL in Slack

…and put it into Settings > Services > Slack together with your Slack username: Configuring Slack Service in GitLab

Since the only thing you want to be notified of is deployments, you can uncheck all the checkboxes except the “Build” in the settings above. That’s it. Now you’re notified for every deployment:

Deployment notifications in Slack

Teamwork at Scale

As the time passed, your website became really popular, and your team has grown from 2 to 8 people. People develop in parallel, so the situation when people wait for each other to preview something on Staging has become pretty common. “Deploy every branch to staging” stopped working.

Queue of branches for review on Staging

It’s time to modify the process one more time. You and your team agreed that if someone wants to see his/her changes on the staging server, he/she should first merge the changes to the “staging” branch.

The change of .gitlab-ci.yml is minimal:

except:
- master

is now changed to

only:
- staging

Staging branchPeople have to merge their feature branches before preview on Staging

Of course, it requires additional time and effort for merging, but everybody agreed that it is better than waiting.

Handling Emergencies

You can’t control everything, so sometimes things go wrong. Someone merged branches incorrectly and pushed the result straight to production exactly when your site was on top of HackerNews. Thousands of people saw your completely broken layout instead of your shiny main page.

Luckily, someone found the Rollback button, so the website was fixed a minute after the problem was discovered.

List of environmentsRollback relaunches the previous job with the previous commit

Anyway, you felt that you needed to react to the problem and decided to turn off auto deployment to production and switch to manual deployment. To do that, you needed to add when: manual to your job.

As you expected, there will be no automatic deployment to Production after that. To deploy manually go to Pipelines > Builds, and click the button:

Skipped job is available for manual launch

Finally, your company has turned into a corporation. You have hundreds of people working on the website, so all the previous compromises are not working anymore.

Review Apps

The next logical step is to boot up a temporary instance of the application per feature-branch for review.

In our case, we set up another bucket on S3 for that. The only difference that we copy the contents of our website to a “folder” named by a name of the development branch, so that the URL looks like this:

http://<REVIEW_S3_BUCKET_NAME>.s3-website-us-east-1.amazonaws.com/<branchname>/

Here’s the replacement for the pages job we used before:

review apps:
  variables:
    S3_BUCKET_NAME: "reviewbucket"
  image: python:latest
  environment: review
  script:
  - pip install awscli
  - mkdir -p ./$CI_BUILD_REF_NAME
  - cp ./*.html ./$CI_BUILD_REF_NAME/
  - aws s3 cp ./ s3://$S3_BUCKET_NAME/ --recursive --exclude "*" --include "*.html"

The interesting thing is where we got this $CI_BUILD_REF_NAME variable from. GitLab predefines many environment variables so that you can use them in your jobs.

Note that we defined the S3_BUCKET_NAME variable inside the job. You can do this to rewrite top-level definitions.

Visual representation of this configuration: Review apps

The details of Review Apps implementation depend widely on your real technology stack and on your deployment process, which is out of the scope of this blog post.

It will not be that straightforward, as it is with our static HTML website. For example, you had to make these instances temporary, and booting up these instances with all required software and services automatically on the fly is not a trivial task. However, it is doable, especially if you use Docker, or at least Chef or Ansible.

We’ll cover deployment with Docker in another blog post. To be fair, I feel a bit guilty for simplifying the deployment process to a simple HTML files copying, and not adding some hardcore scenarios. If you need some right now, I recommend you to read article “Building an Elixir Release into a Docker image using GitLab CI”

For now, let’s talk about one final thing.

Deploying to Different Platforms

In real life, we are not limited to S3 and GitLab Pages. We host, and therefore, deploy our apps and packages to various services.

Moreover, at some point, you could decide to move to a new platform and thus need to rewrite all your deployment scripts. You can use a gem called dpl to minimize the damage.

In the examples above we used awscli as a tool to deliver code to an example service (Amazon S3). However, no matter what tool and what destination system you use, the principle is the same: you run a command with some parameters and somehow pass a secret key for authentication purposes.

The dpl deployment tool utilizes this principle and provides a unified interface for this list of providers.

Here’s how a production deployment job would look if we use dpl:

variables:
  S3_BUCKET_NAME: "yourbucket"

deploy to production:
  environment: production
  image: ruby:latest
  script:
  - gem install dpl
  - dpl --provider=s3 --bucket=$S3_BUCKET_NAME
  only:
  - master

If you deploy to different systems or change destination platform frequently, consider using dpl to make your deployment scripts look uniform.

Summary

  1. Deployment is just a command (or a set of commands) that is regularly executed. Therefore it can run inside GitLab CI
  2. Most times you’ll need to provide some secret key(s) to the command you execute. Store these secret keys in Settings > Variables
  3. With GitLab CI, you can flexibly specify which branches to deploy to
  4. If you deploy to multiple environments, GitLab will conserve the history of deployments, which gives you the ability to rollback to any previous version
  5. For critical parts of your infrastructure, you can enable manual deployment from GitLab interface, instead of automated deployment

Testing a PHP application in GitLab CI

This guide covers basic building instructions for PHP projects.

There are covered two cases: testing using the Docker executor and testing using the Shell executor.

Test PHP projects using the Docker executor

While it is possible to test PHP apps on any system, this would require manual configuration from the developer. To overcome this we will be using the official PHP docker image that can be found in Docker Hub.

This will allow us to test PHP projects against different versions of PHP. However, not everything is plug ‘n’ play, you still need to configure some things manually.

As with every build, you need to create a valid .gitlab-ci.yml describing the build environment.

Let’s first specify the PHP image that will be used for the build process (you can read more about what an image means in the Runner’s lingo reading about Using Docker images).

Start by adding the image to your .gitlab-ci.yml:

image: php:5.6

The official images are great, but they lack a few useful tools for testing. We need to first prepare the build environment. A way to overcome this is to create a script which installs all prerequisites prior the actual testing is done.

Let’s create a ci/docker_install.sh file in the root directory of our repository with the following content:

#!/bin/bash

# We need to install dependencies only for Docker
[[ ! -e /.dockerenv ]] && [[ ! -e /.dockerinit ]] && exit 0

set -xe

# Install git (the php image doesn't have it) which is required by composer
apt-get update -yqq
apt-get install git -yqq

# Install phpunit, the tool that we will use for testing
curl --location --output /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar
chmod +x /usr/local/bin/phpunit

# Install mysql driver
# Here you can install any other extension that you need
docker-php-ext-install pdo_mysql

You might wonder what docker-php-ext-install is. In short, it is a script provided by the official php docker image that you can use to easilly install extensions. For more information read the the documentation at https://hub.docker.com/r/_/php/.

Now that we created the script that contains all prerequisites for our build environment, let’s add it in .gitlab-ci.yml:

...

before_script:
- bash ci/docker_install.sh > /dev/null

...

Last step, run the actual tests using phpunit:

...

test:app:
  script:
  - phpunit --configuration phpunit_myapp.xml

...

Finally, commit your files and push them to GitLab to see your build succeeding (or failing).

The final .gitlab-ci.yml should look similar to this:

# Select image from https://hub.docker.com/r/_/php/
image: php:5.6

before_script:
# Install dependencies
- bash ci/docker_install.sh > /dev/null

test:app:
  script:
  - phpunit --configuration phpunit_myapp.xml

Test against different PHP versions in Docker builds

Testing against multiple versions of PHP is super easy. Just add another job with a different docker image version and the runner will do the rest:

before_script:
# Install dependencies
- bash ci/docker_install.sh > /dev/null

# We test PHP5.6
test:5.6:
  image: php:5.6
  script:
  - phpunit --configuration phpunit_myapp.xml

# We test PHP7.0 (good luck with that)
test:7.0:
  image: php:7.0
  script:
  - phpunit --configuration phpunit_myapp.xml

Custom PHP configuration in Docker builds

There are times where you will need to customise your PHP environment by putting your .ini file into/usr/local/etc/php/conf.d/. For that purpose add a before_script action:

before_script:
- cp my_php.ini /usr/local/etc/php/conf.d/test.ini

Of course, my_php.ini must be present in the root directory of your repository.

Test PHP projects using the Shell executor

The shell executor runs your builds in a terminal session on your server. Thus, in order to test your projects you first need to make sure that all dependencies are installed.

For example, in a VM running Debian 8 we first update the cache, then we install phpunit and php5-mysql:

sudo apt-get update -y
sudo apt-get install -y phpunit php5-mysql

Next, add the following snippet to your .gitlab-ci.yml:

test:app:
  script:
  - phpunit --configuration phpunit_myapp.xml

Finally, push to GitLab and let the tests begin!

Test against different PHP versions in Shell builds

The phpenv project allows you to easily manage different versions of PHP each with its own config. This is specially usefull when testing PHP projects with the Shell executor.

You will have to install it on your build machine under the gitlab-runner user following the upstream installation guide.

Using phpenv also allows to easily configure the PHP environment with:

phpenv config-add my_config.ini

Important note: It seems phpenv/phpenv is abandoned. There is a fork at madumlao/phpenv that tries to bring the project back to life. CHH/phpenv also seems like a good alternative. Picking any of the mentioned tools will work with the basic phpenv commands. Guiding you to choose the right phpenv is out of the scope of this tutorial.

Install custom extensions

Since this is a pretty bare installation of the PHP environment, you may need some extensions that are not currently present on the build machine.

To install additional extensions simply execute:

pecl install <extension>

It’s not advised to add this to .gitlab-ci.yml. You should execute this command once, only to setup the build environment.

Extend your tests

Using atoum

Instead of PHPUnit, you can use any other tool to run unit tests. For example you can use atoum:

before_script:
- wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar

test:atoum:
  script:
  - php mageekguy.atoum.phar

Using Composer

The majority of the PHP projects use Composer for managing their PHP packages. In order to execute Composer before running your tests, simply add the following in your .gitlab-ci.yml:

...

# Composer stores all downloaded packages in the vendor/ directory.
# Do not use the following if the vendor/ directory is commited to
# your git repository.
cache:
  paths:
  - vendor/

before_script:
# Install composer dependencies
- curl --silent --show-error https://getcomposer.org/installer | php
- php composer.phar install

...

Access private packages / dependencies

If your test suite needs to access a private repository, you need to configure the SSH keys in order to be able to clone it.

Use databases or other services

Most of the time you will need a running database in order for your tests to run. If you are using the Docker executor you can leverage Docker’s ability to link to other containers. In GitLab Runner lingo, this can be achieved by defining a service.

This functionality is covered in the CI services documentation.

Testing things locally

With GitLab Runner 1.0 you can also test any changes locally. From your terminal execute:

# Check using docker executor
gitlab-ci-multi-runner exec docker test:app

# Check using shell executor
gitlab-ci-multi-runner exec shell test:app

Example project

We have set up an Example PHP Project for your convenience that runs on GitLab.com using our publicly available shared runners.

Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin.

Decreasing build time from 8 minutes 33 seconds to just 10 seconds

I setup GitLab to host several projects at work and I have been quite pleased with it. I read that setting GitLab CI for test and deployment was easy so I decided to try it to automatically run the test suite and the sphinx documentation.

I found the official documentation to be quite good to setup a runner so I won’t go into details here. I chose the Docker executor.

Here is my first .gitlab-ci.yml test:

image: python:3.4

before_script:
  - pip install -r requirements.txt

tests:
  stage: test
  script:
    - python -m unittest discover -v

Success, it works! Nice. But… 8 minutes 33 seconds build time for a test suite that runs in less than 1 second… that’s a bit long.

Let’s try using some caching to avoid having to download all the pip requirements every time. After googling, I found this post explaining that the cache path must be inside the build directory:

image: python:3.4

before_script:
  - export PIP_CACHE_DIR="pip-cache"
  - pip install -r requirements.txt

cache:
  paths:
    - pip-cache

tests:
  stage: test
  script:
    - python -m unittest discover -v

With the pip cache, the build time went down to about 6 minutes. A bit better, but far from acceptable.

Of course I knew the problem was not the download, but the installation of the pip requirements. I use pandas which explains why it takes a while to compile.

So how do you install pandas easily? With conda of course! There are even some nice docker images created by Continuum Analytics ready to be used.

So let’s try again:

image: continuumio/miniconda3:latest

before_script:
  - conda env create -f environment.yml
  - source activate koopa

tests:
  stage: test
  script:
    - python -m unittest discover -v

Build time: 2 minutes 55 seconds. Nice but we need some cache to avoid downloading all the packages everytime. The first problem is that the cache path has to be in the build directory. Conda packages are saved in /opt/conda/pkgs by default. A solution is to replace that directory with a link to a local directory. It works but the problem is that Gitlab makes a compressed archive to save and restore the cache which takes quite some time in this case…

How to get a fast cache? Let’s use a docker volume! I modified my /etc/gitlab-runner/config.toml to add two volumes:

[runners.docker]
  tls_verify = false
  image = "continuumio/miniconda3:latest"
  privileged = false
  disable_cache = false
  volumes = ["/cache", "/opt/cache/conda/pkgs:/opt/conda/pkgs:rw", "/opt/cache/pip:/opt/cache/pip:rw"]

One volume for conda packages and one for pip. My new .gitlab-ci.yml:

image: continuumio/miniconda3:latest

before_script:
  - export PIP_CACHE_DIR="/opt/cache/pip"
  - conda env create -f environment.yml
  - source activate koopa

tests:
  stage: test
  script:
    - python -m unittest discover -v

The build time is about 10 seconds!

Just a few days after my tests, GitLab announced GitLab Container Registry. I already thought about building my own docker image and this new feature would make it even easier than before. But I would have to remember to update my image if I change my requirements. Which I don’t have to think about with the current solution.

A Quick Start On GitLab Continuous Integration

GitLab offers a continuous integration service. If you add a .gitlab-ci.yml file to the root directory of your repository, and configure your GitLab project to use a Runner, then each merge request or push triggers your CI pipeline.

The .gitlab-ci.yml file tells the GitLab runner what to do. By default it runs a pipeline with three stages: build, test, and deploy. You don’t need to use all three stages; stages with no jobs are simply ignored.

If everything runs OK (no non-zero return values), you’ll get a nice green checkmark associated with the pushed commit or merge request. This makes it easy to see whether a merge request caused any of the tests to fail before you even look at the code.

Most projects use GitLab’s CI service to run the test suite so that developers get immediate feedback if they broke something.

There’s a growing trend to use continuous delivery and continuous deployment to automatically deploy tested code to staging and production environments.

So in brief, the steps needed to have a working CI can be summed up to:

  1. Add .gitlab-ci.yml to the root directory of your repository
  2. Configure a Runner

From there on, on every push to your Git repository, the Runner will automagically start the pipeline and the pipeline will appear under the project’s /pipelines page.


This guide assumes that you:

  • have a working GitLab instance of version 8.0 or higher or are using GitLab.com
  • have a project in GitLab that you would like to use CI for

Let’s break it down to pieces and work on solving the GitLab CI puzzle.

Creating a .gitlab-ci.yml file

Before you create .gitlab-ci.yml let’s first explain in brief what this is all about.

What is .gitlab-ci.yml

The .gitlab-ci.yml file is where you configure what CI does with your project. It lives in the root of your repository.

On any push to your repository, GitLab will look for the .gitlab-ci.yml file and start builds on Runners according to the contents of the file, for that commit.

Because .gitlab-ci.yml is in the repository and is version controlled, old versions still build successfully, forks can easily make use of CI, branches can have different pipelines and jobs, and you have a single source of truth for CI. You can read more about the reasons why we are using .gitlab-ci.yml in our blog about it.

Note: .gitlab-ci.yml is a YAML file so you have to pay extra attention to indentation. Always use spaces, not tabs.

Creating a simple .gitlab-ci.yml file

You need to create a file named .gitlab-ci.yml in the root directory of your repository. Below is an example for a Ruby on Rails project.

before_script:
  - apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs
  - ruby -v
  - which ruby
  - gem install bundler --no-ri --no-rdoc
  - bundle install --jobs $(nproc)  "${FLAGS[@]}"

rspec:
  script:
    - bundle exec rspec

rubocop:
  script:
    - bundle exec rubocop

This is the simplest possible build configuration that will work for most Ruby applications:

  1. Define two jobs rspec and rubocop (the names are arbitrary) with different commands to be executed.
  2. Before every job, the commands defined by before_script are executed.

The .gitlab-ci.yml file defines sets of jobs with constraints of how and when they should be run. The jobs are defined as top-level elements with a name (in our case rspec and rubocop) and always have to contain the script keyword. Jobs are used to create builds, which are then picked by Runners and executed within the environment of the Runner.

What is important is that each job is run independently from each other.

If you want to check whether your .gitlab-ci.yml file is valid, there is a Lint tool under the page /ci/lint of your GitLab instance. You can also find a “CI Lint” button to go to this page under Pipelines > Pipelines and Pipelines > Builds in your project.

For more information and a complete .gitlab-ci.yml syntax, please read the documentation on .gitlab-ci.yml.

Push .gitlab-ci.yml to GitLab

Once you’ve created .gitlab-ci.yml, you should add it to your git repository and push it to GitLab.

git add .gitlab-ci.yml
git commit -m "Add .gitlab-ci.yml"
git push origin master

Now if you go to the Pipelines page you will see that the pipeline is pending.

You can also go to the Commits page and notice the little clock icon next to the commit SHA.

New commit pending

Clicking on the clock icon you will be directed to the builds page for that specific commit.

Single commit builds page

Notice that there are two jobs pending which are named after what we wrote in .gitlab-ci.yml. The red triangle indicates that there is no Runner configured yet for these builds.

The next step is to configure a Runner so that it picks the pending builds.

Configuring a Runner

In GitLab, Runners run the builds that you define in .gitlab-ci.yml. A Runner can be a virtual machine, a VPS, a bare-metal machine, a docker container or even a cluster of containers. GitLab and the Runners communicate through an API, so the only requirement is that the Runner’s machine has Internet access.

A Runner can be specific to a certain project or serve multiple projects in GitLab. If it serves all projects it’s called a Shared Runner.

Find more information about different Runners in the Runners documentation.

You can find whether any Runners are assigned to your project by going to Settings > Runners. Setting up a Runner is easy and straightforward. The official Runner supported by GitLab is written in Go and can be found at https://gitlab.com/gitlab-org/gitlab-ci-multi-runner.

In order to have a functional Runner you need to follow two steps:

  1. Install it
  2. Configure it

Follow the links above to set up your own Runner or use a Shared Runner as described in the next section.

For other types of unofficial Runners written in other languages, see the instructions for the various GitLab Runners.

Once the Runner has been set up, you should see it on the Runners page of your project, following Settings > Runners.

Activated runners

Shared Runners

If you use GitLab.com you can use Shared Runners provided by GitLab Inc.

These are special virtual machines that run on GitLab’s infrastructure and can build any project.

To enable Shared Runners you have to go to your project’s Settings > Runners and click Enable shared runners.

Read more on Shared Runners.

Seeing the status of your pipeline and builds

After configuring the Runner successfully, you should see the status of your last commit change from pending to either running,success or failed.

You can view all pipelines by going to the Pipelines page in your project.

Commit status

Or you can view all builds, by going to the Pipelines > Builds page.

Commit status

By clicking on a Build ID, you will be able to see the log of that build. This is important to diagnose why a build failed or acted differently than you expected.

Build log

You are also able to view the status of any commit in the various pages in GitLab, such as Commits and Merge Requests.

Enabling build emails

If you want to receive e-mail notifications about the result status of the builds, you should explicitly enable the Builds Emails service under your project’s settings.

For more information read the Builds emails service documentation.

Examples

Visit the examples README to see a list of examples using GitLab CI with various languages.

Awesome! You started using CI in GitLab!

GitLab Continuous Integration

What Are The Advantages?

  • Integrated: GitLab CI is part of GitLab. You can use it for free on GitLab.com
  • Easy to learn: See our Quick Start guide
  • Beautiful: GitLab CI offers the same great experience as GitLab. Familiar, easy to use, and beautiful.
  • Scalable: Tests run distributed on separate machines of which you can add as many as you want
  • Faster results: Each build can be split in multiple jobs that run in parallel on multiple machines
  • Continuous Delivery (CD): multiple stages, manual deploys, environments, and variables
  • Open source: CI is included with both the open source GitLab Community Edition and the proprietary GitLab Enterprise Edition

Features

  • Multi-platform: you can execute builds on Unix, Windows, OSX, and any other platform that supports Go.
  • Multi-language: build scripts are command line driven and work with Java, PHP, Ruby, C, and any other language.
  • Stable: your builds run on a different machine than GitLab.
  • Parallel builds: GitLab CI splits builds over multiple machines, for fast execution.
  • Realtime logging: a link in the merge request takes you to the current build log that updates dynamically.
  • Versioned tests: a .gitlab-ci.yml file that contains your tests, allowing everyone to contribute changes and ensuring every branch gets the tests it needs.
  • Pipeline: you can define multiple jobs per stage and you cantrigger other builds.
  • Autoscaling: you can automatically spin up and down VM’sto make sure your builds get processed immediately and minimize costs.
  • Build artifacts: you can upload binaries and other build artifacts to GitLab and browse and download them.
  • Test locally there are multiple executors and you canreproduce tests locally.
  • Docker support: you can easily spin up other Docker containers as a service as part of the test and build docker images.

Fully integrated with GitLab

  • Fully integrated with GitLab.
  • Quick project setup: Add projects with a single click, all hooks are setup automatically via the GitLab API.
  • Merge request integration: See the status of each build within the Merge Request in GitLab.

Architecture

GitLab CI is a part of GitLab, a web application with an API that stores its state in a database. It manages projects/builds and provides a nice user interface, besides all the features of GitLab.

GitLab Runner is an application which processes builds. It can be deployed separately and works with GitLab CI through an API.

In order to run tests, you need at least one GitLab instance and one GitLab Runner.


GitLab Runner

To perform the actual build, you need to install GitLab Runner which is written in Go.

It can run on any platform for which you can build Go binaries, including Linux, OSX, Windows, FreeBSD and Docker.

It can test any programming language including .Net, Java, Python, C, PHP and others.

GitLab Runner has many features including autoscaling,

Install GitLab Runner

Jenkins to GitLab CI

The railway world is a fast-moving environment. To bring you the latest improvements and fixes as quick as possible, Captain Train’s web-app is often updated, sometimes several times per day.

Did you always wonder how we manage building and deploying all of this without a jolt? Then read-on: here is a technical peek into our engineering process.

Note: this post tells the customer story of Captain Train.

From Jenkins to GitLab CI

We used to build our web-app using Jenkins. A robust and proven solution—which was polling our repositories every minute, and built the appropriate integration and production branches.

However we recently switched to a new system for building our web-app. To host our source-code and perform merge-requests, we’re using a self-hosted instance of GitLab. It’s nice, open-source—and features an integrated build system: GitLab CI.

See it like Travis, but integrated: just add a custom .gitlab-ci.yml file at the root of your repository, and GitLab will automatically start building your app in the way you specified.

Now what’s cool about this?

Reliable dockerized builds

Jenkins builds were all executed on a resource-constrained server—and this made builds slow and unreliable. For instance, we observed several times PhantomJS crashing randomly during tests: apparently it didn’t like several builds running on the same machine at the same time—and a single PhantomJS process crashing would bring all of the others down.

So the first step of our migration was to insulate builds into Docker containers. In this way:

  • Every build is isolated from the others, and processes don’t crash each other randomly.
  • Building the same project on different architectures is easy, and that’s good news, because we need this to support multiple Debian versions.
  • Project maintainers have greater control on the setup of their build environment: no need to bother an admin when upgrading an SDK on the shared build machine.

It scales

GitLab CI allows us to add more runners very easily. And now that builds are performed in Docker containers, we don’t have to configure the runners specifically with our build tools: any out-of-the-box server will do.

Once a new runner is declared, scaling is automatic: the most available runner will be picked to start every new build. It’s so simple that you can even add your own machine to build locally.

We’ve already reduced our build time by switching to a more powerful runner—a migration that would have been more difficult to do using Jenkins. Although we regularly optimize the run time of our test suite, sometimes you also need to just throw more CPU at it.

Easier to control

With Jenkins, the configuration of the build job is stored in an external admin-restricted tool. You need the right credentials to edit the build configuration, and it’s not obvious how to do it.

Using GitLab CI, the build jobs are determined solely from the .gitlab-ci.yml file in the repository. This makes it really simple to edit, and you get all the niceties of your usual git work-flow: versioning, merge requests, and so on. You don’t need to ask permission to add CI to your project. Lowering the barrier to entry for CI is definitely a good thing for engineering quality and developer happiness.

Tests on merge requests

GitLab CI makes it really easy to build and test the branch of a merge request (or a “Pull request” in GitHub slang). Just a few lines added to our .gitlab-ci.yml file, and we were running tests for every push to a merge request.

Merge automatically when the build succeeds

We get nice red-or-green-status, the quite useful “Merge automatically when the build succeeds” button — and, as branches are now tested before being merged, much less build breakage.

Build Passed

A slick UI

GitLab CI provides “Pipelines”, an overview of all your build jobs. This points you quickly to a failing build, and the stage where the problem occurs. Plus it gets you this warm and fuzzy feeling of safeness when everything is green.

Pipelines

In a nutshell

We found the overall experience quite positive. Once the initial hurdle of making the build pass in a Docker container, integrating it into GitLab CI was really easy. And it gave us tons of positive signals, new features and neat integrations. 10/10, would build again.👍

Our Android team also migrated their pipeline, and are now building the integration and production Android APK with GitLab CI.

For further reading, you can find on the official website a nice overview of GitLab CI features, and some examples of .gitlab-ci.ymlfiles.

Continuous Integration Newbie

Let’s assume that you don’t know anything about what Continuous Integration is and why it’s needed. Or, you just forgot. Anyway, we’re starting from scratch here.

Imagine that you work on a project, where all the code consists of two text files. Moreover, it is super-critical that the concatenation of these two files contains the phrase “Hello world.”

If there’s no such phrase, the whole development team stays without a salary for a month. Yeah, it is that serious!

The most responsible developer wrote a small script to run every time we are about to send our code to customers. The code is pretty sophisticated:

cat file1.txt file2.txt | grep -q "Hello world"

The problem is that there are ten developers in the team, and, you know, human factors can hit hard.

A week ago, a new guy forgot to run the script and three clients got broken builds. So you decided to solve the problem once and for all. Luckily, your code is already on GitLab, and you remember that there is a built-in CI system. Moreover, you heard at a conference that people use CI to run tests…

Run our first test inside CI

After a couple minutes to find and read the docs, it seems like all we need is these two lines of code in a file called .gitlab-ci.yml:

test:
  script: cat file1.txt file2.txt | grep -q 'Hello world'

Committing it, and hooray! Our build is successful: Build succeeded

Let’s change “world” to “Africa” in the second file and check what happens: Build failed

The build fails as expected!

Okay, we now have automated tests here! GitLab CI will run our test script every time we push new code to the repository.

Make results of builds downloadable

The next business requirement is to package the code before sending it to our customers. Let’s automate that as well!

All we need to do is define another job for CI. Let’s name the job “package”:

test:
  script: cat file1.txt file2.txt | grep -q 'Hello world'

package:
  script: cat file1.txt file2.txt | gzip > package.gz

We have two tabs now: Two tabs - generated from two jobs

However, we forgot to specify that the new file is a build artifact, so that it could be downloaded. We can fix it by adding an artifactssection:

test:
  script: cat file1.txt file2.txt | grep -q 'Hello world'

package:
  script: cat file1.txt file2.txt | gzip > packaged.gz
  artifacts:
    paths:
    - packaged.gz

Checking… It is there: Checking the download buttons

Perfect! However, we have a problem to fix: the jobs are running in parallel, but we do not want to package our application if our tests fail.

Run jobs sequentially

We only want to run the ‘package’ job if the tests are successful. Let’s define the order by specifying stages:

stages:
  - test
  - package

test:
  stage: test
  script: cat file1.txt file2.txt | grep -q 'Hello world'

package:
  stage: package
  script: cat file1.txt file2.txt | gzip > packaged.gz
  artifacts:
    paths:
    - packaged.gz

That should be good!

Also, we forgot to mention, that compilation (which is represented by concatenation in our case) takes a while, so we don’t want to run it twice. Let’s define a separate step for it:

stages:
  - compile
  - test
  - package

compile:
  stage: compile
  script: cat file1.txt file2.txt > compiled.txt
  artifacts:
    paths:
    - compiled.txt

test:
  stage: test
  script: cat compiled.txt | grep -q 'Hello world'

package:
  stage: package
  script: cat compiled.txt | gzip > packaged.gz
  artifacts:
    paths:
    - packaged.gz

Let’s take a look at our artifacts:

Unnecessary artifact

Hmm, we do not need that “compile” file to be downloadable. Let’s make our temporary artifacts expire by setting expire_in to ’20 minutes’:

compile:
  stage: compile
  script: cat file1.txt file2.txt > compiled.txt
  artifacts:
    paths:
    - compiled.txt
    expire_in: 20 minutes

Now our config looks pretty impressive:

  • We have three sequential stages to compile, test, and package our application.
  • We are passing the compiled app to the next stages so that there’s no need to run compilation twice (so it will run faster).
  • We are storing a packaged version of our app in build artifacts for further usage.

Learning which Docker image to use

So far so good. However, it appears our builds are still slow. Let’s take a look at the logs.

Ruby 2.1 is the logs

Wait, what is this? Ruby 2.1?

Why do we need Ruby at all? Oh, GitLab.com uses Docker images to run our builds, and by default it uses the ruby:2.1 image. For sure, this image contains many packages we don’t need. After a minute of googling, we figure out that there’s an image called alpinewhich is an almost blank Linux image.

OK, let’s explicitly specify that we want to use this image by adding image: alpine to .gitlab-ci.yml. Now we’re talking! We shaved almost 3 minutes off:

Build speed improved

It looks like there’s a lot of public images around. So we can just grab one for our technology stack. It makes sense to specify an image which contains no extra software because it minimizes download time.

Dealing with complex scenarios

So far so good. However, let’s suppose we have a new client who wants us to package our app into .iso image instead of .gz Since CI does the whole work, we can just add one more job to it. ISO images can be created using the mkisofs command. Here’s how our config should look:

image: alpine

stages:
  - compile
  - test
  - package

# ... "compile" and "test" jobs are skipped here for the sake of compactness

pack-gz:
  stage: package
  script: cat compiled.txt | gzip > packaged.gz
  artifacts:
    paths:
    - packaged.gz

pack-iso:
  stage: package
  script:
  - mkisofs -o ./packaged.iso ./compiled.txt
  artifacts:
    paths:
    - packaged.iso

Note that job names shouldn’t necessarily be the same. In fact if they were the same, it wouldn’t be possible to make the jobs run in parallel inside the same stage. Hence, think of same names of jobs & stages as coincidence.

Anyhow, the build is failing: Failed build because of missing mkisofs

The problem is that mkisofs is not included in the alpine image, so we need to install it first.

Dealing with missing software/packages

According to the Alpine Linux website mkisofs is a part of the xorriso and cdrkit packages. These are the magic commands that we need to run to install a package:

echo "ipv6" >> /etc/modules  # enable networking
apk update                   # update packages list
apk add xorriso              # install package

For CI, these are just like any other commands. The full list of commands we need to pass to script section should look like this:

script:
- echo "ipv6" >> /etc/modules
- apk update
- apk add xorriso
- mkisofs -o ./packaged.iso ./compiled.txt

However, to make it semantically correct, let’s put commands related to package installation in before_script. Note that if you usebefore_script at the top level of a configuration, then the commands will run before all jobs. In our case, we just want it to run before one specific job.

Our final version of .gitlab-ci.yml:

image: alpine

stages:
  - compile
  - test
  - package

compile:
  stage: compile
  script: cat file1.txt file2.txt > compiled.txt
  artifacts:
    paths:
    - compiled.txt
    expire_in: 20 minutes

test:
  stage: test
  script: cat compiled.txt | grep -q 'Hello world'

pack-gz:
  stage: package
  script: cat compiled.txt | gzip > packaged.gz
  artifacts:
    paths:
    - packaged.gz

pack-iso:
  stage: package
  before_script:
  - echo "ipv6" >> /etc/modules
  - apk update
  - apk add xorriso
  script:
  - mkisofs -o ./packaged.iso ./compiled.txt
  artifacts:
    paths:
    - packaged.iso

Wow, it looks like we have just created a pipeline! We have three sequential stages, but jobs pack-gz and pack-iso, inside thepackage stage, are running in parallel:

Pipelines illustration

Summary

There’s much more to cover but let’s stop here for now. I hope you liked this short story. All examples were made intentionally trivial so that you could learn the concepts of GitLab CI without being distracted by an unfamiliar technology stack. Let’s wrap up what we have learned:

  1. To delegate some work to GitLab CI you should define one or more jobs in .gitlab-ci.yml.
  2. Jobs should have names and it’s your responsibility to come up with good ones.
  3. Every job contains a set of rules & instructions for GitLab CI, defined by special keywords.
  4. Jobs can run sequentially, in parallel, or you can define a custom pipeline.
  5. You can pass files between jobs and store them in build artifacts so that they can be downloaded from the interface.

Below is the last section containing a more formal description of terms and keywords we used, as well as links to the detailed description of GitLab CI functionality.

Keywords description & links to the documentation

Keyword/term Description
.gitlab-ci.yml File containing all definitions of how your project should be built
script Defines a shell script to be executed
before_script Used to define the command that should be run before (all) jobs
image Defines what docker image to use
stage Defines a pipeline stage (default: test)
artifacts Defines a list of build artifacts
artifacts:expire_in Used to delete uploaded artifacts after the specified time
pipelines A pipeline is a group of builds that get executed in stages (batches)