1. Cider-CI
2. Articles
3. Testing Services
4. Atom feed

• Introduction
• Backgrounding - A Poor Solution
• Scripts, Processes, Events and Control in Cider-CI
• Dependency Declaration by Example
• Cleaning Up and Managing Background Services
• Conclusion and Wrap-up

Introduction

Integration tests often require that multiple processes are being run in parallel. The conventional approach is to send services to the background. This makes it hard to control and shutdown the service¹. In Cider-CI several scrips can be run in the same context while each script runs in its own process. Cider-CI automatically manges the scripts according to the dependencies declared in the project configuration.

Backgrounding - A Poor Solution

If we send a process into background we will loose direct control over it. Inspecting the state and terminating the process becomes brittle. Capturing the output and making it available to the user becomes unnecessary complicated. Debugging is at least very time consuming and likely just impractical with reasonable effort.

Shutting down a background process is difficult. If the parent of a background process gets terminated the background process will keep on running. Even worse than that: it will become a child of the main system process on Unix like operating systems. It is now next to impossible for a CI system to differentiate processes which should be killed from those which are still used by ongoing tests.

Scripts, Processes, Events and Control in Cider-CI

In Cider-CI a task can have several scripts. When it comes to control on the executor every script runs in its own process in the foreground.

The output for every script its captured and directly visible in the user interface of Cider-CI. At the end of a trial² every script belonging to it is guaranteed to have been terminated. The scripts belonging to a trial are always executed on the same executor in the same context. They share the working directory for example.

By default a script is immediately started after a trial has been dispatched to an executor. The start of a script can be deferred. It can depend on a subset of states of any subset of other scripts within the same trial. The resulting dependency graph is automatically managed by Cider-CI.

Dependency Declaration by Example

We will use the integration tests for the Madek project to illustrate how dependencies are formulated in the Cider-CI project configuration.

scripts:
 precomile-api: {⋯}
 run-api:
    start_when:
      the api has been precompiled:
        script_key: precompile-api
    ⋯

The keys used to reference scripts, (here precompile-api, and run-api) are of free choice. The key for the dependency the api has been precompiled can be chosen to be mnemonic but is otherwise irrelevant.

Multiple dependencies are combined with logical conjunction. A dependency always includes one or many states combined with logical disjunction. Possible script states are pending, executing, witing, aborted, defective, skipped, failed, and passed. The five latter ones are the terminal states. The declaration states: [passed] is very common. It is the implicit default.

The following example lets the script api-is-running started once run-api is executing. The loop inside the body of the scripts is exited once it is confirmed that the API is is running. The interesting part is that run-api will keep on executing in the foreground.

scripts
  api-is-running:
    body: |
      set -eux
      until curl --silent --fail --user x:secret -I  \
        http://localhost:${API_HTTP_PORT}/api; do sleep 1; done
    start_when:
      run-api is executing:
        script_key: run-api
        states: [executing]

A script will never keep executing forever, e.g. in the case above when curl will keep on failing. There is always timeout directive in place. The implicit timeout value is 3 Minutes. After the timeout has passed the script will be forcefully terminated and assume the state defective.

Cleaning Up and Managing Background Services

To clean up a database for example after the test has finished we would give all terminal states as a dependency as in the following example:

scripts:
  delete-database:
    body: |
      #!/usr/bin/env bash
      set -eux
      dropdb "$DATABASE"
    start_when:
      test is in termial state:
        script_key: test
        states: [aborted, defective,failed, passed, skipped]
        ignore_abort: true

The parameter ignore_abort: true is essential. It ensures that scripts are invoked even if the job or trial is aborted. Jobs are automatically aborted when all related commits become orphans. This can happen frequently if git reflow or a similar workflow is applied.

Programs which only work in background mode would be stopped by adding a stop script with very similar dependency settings as in the delete-database example.

Conclusion and Wrap-up

We can run run multiple scripts for one test execution with Cider-CI. The scripts are controlled by declaring dependencies. Multiple scripts can be used to speed up tests by executing things in parallel. They also provide means to keep services running in the foreground. This keeps complex integration testing transparent and simplifies it as much as possible.

Cider-CI provides features to ensure that background process are terminated when not needed anymore and other clean-up duties are executed reliably.

It is still possible to run everything in one single large script. This makes migrating to Cider-CI a quick an painless process while keeping the possibility to take the testing to a higher level later on.