UI Testing of a Docker Compose application

In a previous article we saw an example of a simple CI/CD stack that we are using at TRAXxs.

ci-cd-2

Each of our internal services follows the above workflow:

  • code pushed to GitHub
  • CircleCI tests triggered
  • Docker image creation
  • image deployment on our cloud provider

With this simple setup in place (knowing that it is not optimal) it’s a good base on which we could iterate and leverage.

Overview of our services

Basically, all our services are grouped together within a Docker Compose file. To keep it simple, we will only consider a subset of the services we are using and see how to run tests against the front-end.

Let’s then consider the following docker-compose.yml file.


version: '2'
services:

  # Data store
  db:
    image: mongo:3.2
    volumes:
      - mongo-data:/data/db
    expose:
      - "27017"

  # Session store
  kv:
    image: redis:3.0.7-alpine
    volumes:
      - redis-data:/data
    exposes:
      - "6379"

  # web front-end
  web:
    build: web
    expose:
      - "80"
    links:
      - kv
    environment:
      - KV_STORE=kv

  # api back-end
  api:
    build: api
    expose:
      - "80"
    links:
      - db
      - kv
    environment:
      - MONGODB_URL=mongodb://db/testdb
      - KV_STORE=kv

  # Proxy
  proxy:
    build: proxy
    links:
      - api
      - web
    ports:
      - "80:80"
      - "8000:8000"

volumes:
  mongo-data:
  redis-data:

Basically, a web interface can access data through an API backed by a mongo data store. API and web services also uses Redis for session storage (I left that here for the example but we’ll not consider it in the following).
On top of the stack, a reverse proxy that redirects traffic towards the web or the api service. Nothing really fancy here, but enough to illustrate some interesting stuff.

Our testing stack: chimp / cucumber / selenium

For our automated tests, we are using chimp a great javascript library that integrates several tools. Among them, cucumber.js and Selenium, the one that we will use.

One of the great thing about Chimp is that it embeds a Selenium standalone server so we do not have to care about setting this up.

Anatomy of a test

By default, all our cucumber tests need to be defined in a features folder. At the very simple, each test is made up of one feature file (~ description) and one step definition file (~ implementation).

Let’s have a closer look to this and consider our first 2 tests of the web interface:

  • test if the homepage page can be displayed
  • test if the login action can be done

Note: the second test requires an interaction with the back-end api as the signin will need to be verified against the user’s data.

— Homepage —

Let’s see the 2 files defining the test for the homepage


$ cat features/1-homepage.feature
Feature: Display the homepage

  As a user
  I want to display the home page
  So I can see the site is available

  Scenario: Display the homepage
    Given I have visited the homepage of my application
    When I look at the interface
    Then I should see the signin form

A test description with Cucumber is a feature, it is defined at a very high level, nothing technical here and that’s really good so non technical guys can easily create them. Reminds me of the definition of a User Story in Scrum methodology (but that’s another story…).

Now, comes the part when the description of the test needs to be implemented in javascript.


$ cat features/step_definitions/homepage.js
module.exports = function() {

  // Display the home page
  this.Given(/^I have visited the homepage of my application$/, function () {
    browser.url('http://' + process.env.HOST);
  });

  this.When(/^I look at the interface$/, function () {
  });

  this.Then(/^I should see the signin form$/, function() {
    this.client.waitForExist('.login-form');
  });
}

We can see in this file that each step of the scenario defined in the description are used sequentially. The first thing done is pointing the browser to the host the tests needs to be ran against and then verify that a class named login-form is present on the page.

Note: if the login-form class is not found, the test timeout and fails.

Once the tests is setup, it can be ran with the following command


./node_modules/.bin/chimp --browser=firefox

The browser parameter indicates the browser’s driver that should be used to perform the tests. Chrome, Firefox or phantomjs (headless) are among the possible choices.

When the test succeed, the following output should be displayed.

homepage-test

This test is very simple and only relies on the availability of the web server serving the content of the web service. If the service is not available, the test will output

homepage-test-fails

— Login —

The login test is a little bit more complicated in the sense that it relies on the api to know wether or not a user can log into the application. Let’s first see the file used to defined the login service.


Feature: Signin to the application

  As a human
  I want to use the signin form
  So I can signin into the application

  Scenario: Enter wrong credentials
    Given I enter wrong credentials in the signin form
    When I validate the form with wrong creds
    Then I should get an error message

  Scenario: Enter correct credentials
    Given I enter correct credentials in the signin form
    When I validate the form with correct creds
    Then I should see the map page

In this feature, 2 scenarios are defined: the first one provides wrong credentials when the second provides correct ones. Let’s see how the implementation looks like.


$ cat features/step_definitions/signin.js
module.exports = function() {

  // Enter wrong credentials

  this.Given(/^I enter wrong credentials in the signin form$/, function () {
    this.client.setValue('input[id="form-username"]', "wrong_me@gmail.com");
    this.client.setValue('input[id="form-password"]', "password");
  });

  this.When(/^I validate the form with wrong creds$/, function () {
    this.client.waitForVisible('#button_signin');
    this.client.click('#button_signin');
  });

  this.Then(/^I should get an error message$/, function () {
    this.client.waitForExist('.input-error', 10000);
    this.client.waitForVisible('.input-error', 10000);
  });


  // Enter correct credentials

  this.Given(/^I enter correct credentials in the signin form$/, function () {
    this.client.setValue('input[id="form-username"]', "luc@me.com");
    this.client.setValue('input[id="form-password"]', process.env.USER_PASSWD);
  });

  this.When(/^I validate the form with correct creds$/, function () {
    this.client.waitForVisible('#button_signin');
    this.client.click('#button_signin');
  });

  this.Then(/^I should see the map page$/, function () {
    this.client.waitForExist('#map', 15000);
  });
};

Basically, the credentials are set in the login form and the signin button is clicked (a check is done to make sure the signin button is there).

Before running this second tests, we need to consider several options:

  • run the test against a mock api
  • run the test on the whole application

Testing the web interface with a mock API

The first approach I setup was to use a mock API when testing the web front-end. At first it was ok, but it quickly became tedious to maintain the mock and make it evolve with the real api and web services.

The think that bothered me was that testing against the mock api does not reflect the real context of a running application (obviously) and it cannot be considered as integration tests as well.

As our application is defined as a Docker Compose application, and thus can be ran « as is » on every environment, let’s just use Compose api to run the test against the whole application.

Testing the web interface in the whole application

Viktor Farcic, in his great book The DevOps 2.0 Toolkit, describes in great details the setup of a CICD pipeline using Docker. He shows how to run tests using Docker Compose and we will do just that now.

Remember, at the beginning of this article, I dumped a simplified version of the Docker Compose file describing our application.

We will then add a new service, web-test, that will ran the test of our web front-end using all the other services of the stack.

But before adding this service, we need to create an image of our web-test service. We’ll see that there are several « subtleties » that we need to be careful of.

— Code of the test application —

The service that runs the test is developed in Node.js.

The test application is only composed of

  • a very simple package.json file
  • a folder containing the tests

The package.json is the following one. Please note that we use $BROWSER environment variable to be able to run the tests against firefox or chrome.


{
  "name": "web-test",
  "private": true,
  "version": "0.0.1",
  "description": "Test for web ui",
  "keywords": [],
  "dependencies": {
    "chimp": "0.37.1"
  },
  "scripts": {
    "test": "./node_modules/.bin/chimp --browser=$BROWSER"
  },
  "main": "test.js",
  "author": "luc",
  "license": "",
  "engines": {
    "node": "4.4.7"
  }
}

Our feature folder contains the 2 tests described above.


$ find features/
features/
features/1-homepage.feature
features/2-signin.feature
features//step_definitions
features//step_definitions/homepage.js
features//step_definitions/signin.js

Note: the number in the filename ensure the tests are performed in a given order.

— Init Dockerfile for the test image —

As we are using Node.js, let’s initiate a Dockerfile with the following content.


FROM node:4.4.7

# Copy list of server side dependencies
COPY package.json /tmp/package.json

# Install dependencies
RUN cd /tmp && npm install

# Copy dependencies libraries
RUN mkdir /app && cp -a /tmp/node_modules /app/

# Copy src files
COPY . /app/

# Use /app working directory
WORKDIR /app

# Run application
CMD ["npm", "test"]

Basically, the node dependencies (only chimp in this case) are installed and the code (the features folder containing the descriptions and the implementations of the tests) is copied in the container’s app folder.

In the section below, we’ll add all the elements that are needed.

— Install Java JDK 1.8 —

Chimp needs JDK 1.8 to run. At the top of the Dockerfile (below the instruction FROM node:4.4.7), we add the command to install the JDK.


# Install jdk1.8
RUN echo "===> add webupd8 repository..."  && \
    echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu trusty main" | tee /etc/apt/sources.list.d/webupd8team-java.list  && \
    echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu trusty main" | tee -a /etc/apt/sources.list.d/webupd8team-java.list  && \
    apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886  && \
    apt-get update && \
    echo "===> install Java"  && \
    echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections  && \
    echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections  && \
    DEBIAN_FRONTEND=noninteractive  apt-get install -y --force-yes oracle-java8-installer oracle-java8-set-default  && \
    echo "===> clean up..."  && rm -rf /var/cache/oracle-jdk8-installer && apt-get clean && rm -rf /var/lib/apt/lists/*

— Install Xvfb: X Virtual Framebuffer —

Our new container will need to open a browser to run the tests. As there is no X server, we’ll install Xvfb (kinda X emulator) that will be used for this purpose. We’ll add the following instruction in our Dockerfile.


# Install xvfb - X server emulator
RUN apt-get update
RUN apt-get install -y xvfb

Prior to run the tests, Xvfb needs to be started first. We’ll add this instruction in the « test » section of our package.json file, prior to run the tests.


"scripts": {
    "test": "/sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_99.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -ac -screen 0 1027x768x16 && sleep 3 && export DISPLAY=:99.0 && ./node_modules/.bin/chimp --browser=$BROWSER"
  },

Note: thanks to Sam Hatoum and to Łukasz Gandecki (Xolv.io Slack Community) to pointing me towards Xvfb and for the command.

To keep it simple, a X server emulator will be started on display :99.0 and the tests will run on this same display.

— Install Chrome and Firefox —

The last thing to install in our container are the browsers that will be used to test the application. Adding those new instructions, our Dockerfile will look like the following.


FROM node:4.4.7

# Install Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
RUN apt-get update
RUN apt-get install -y --fix-missing google-chrome-stable

# Install Firefox
RUN echo "deb http://packages.linuxmint.com debian import" >> /etc/apt/sources.list
RUN apt-get update
RUN apt-get install -y --force-yes firefox

# Install xvfb - X server emulator
RUN apt-get update
RUN apt-get install -y xvfb

# Install jdk1.8
RUN echo "===> add webupd8 repository..."  && \
    echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu trusty main" | tee /etc/apt/sources.list.d/webupd8team-java.list  && \
    echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu trusty main" | tee -a /etc/apt/sources.list.d/webupd8team-java.list  && \
    apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886  && \
    apt-get update && \
    echo "===> install Java"  && \
    echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections  && \
    echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections  && \
    DEBIAN_FRONTEND=noninteractive  apt-get install -y --force-yes oracle-java8-installer oracle-java8-set-default  && \
    echo "===> clean up..."  && rm -rf /var/cache/oracle-jdk8-installer && apt-get clean && rm -rf /var/lib/apt/lists/*

# Copy list of server side dependencies
COPY package.json /tmp/package.json

# Install dependencies
RUN cd /tmp && npm install

# Copy dependencies libraries
RUN mkdir /app && cp -a /tmp/node_modules /app/

# Copy src files
COPY . /app/

# Use /app working directory
WORKDIR /app

# Run application
CMD ["npm", "test"]

— Add the test service into our Docker Compose file —

We can now add, in our application Docker Compose file, our newly created service whose only purpose is to test the web service.


  web-test:
    build: test
    links:
      - proxy
    depends_on:
      - web
      - proxy
    environment:
      - HOST=proxy
      - BROWSER=firefox
      - USER_PASSWD=test
    privileged: true
    command: npm test

This service depends on the web service, as this is the one it will test, and also on the proxy service as this is the entry point of the application.

Note: a very important thing to notice is the usage of the privileged flag set to true. If this flag is not present, Chrome will not be able to run within the container.

— Running the test using Docker Compose —

Now that everything is setup, the tests can be ran from a single Docker Compose command.


docker-compose run web-test

Compose will make sure all the services are started prior the web-test service.

Note: when the api is ran, a bunch of dummy data are created inside the underlying database. The test are tested against this data set.

This will produce the following output.

test-application-compose

Summary

We have seen in this article how to use Docker Compose to run a test suite (very small one but still 🙂 ) with Chimp on a whole application.
It’s then very easy to create a wrapper around the call to docker-compose run to check the result of the tests (using $?). Other actions can then be triggered to keep on going in the CICD pipeline.

4 réflexions au sujet de « UI Testing of a Docker Compose application »

Laisser un commentaire

Votre adresse de messagerie ne sera pas publiée. Les champs obligatoires sont indiqués avec *