Cider-CI encourages reproducibility in many ways. But sometimes a job can depend on external resources which may change over time in unpredictable ways. This article discusses how to configure a project for Cider-CI to work with a potentially volatile environment.
A job in Cider-CI is bound to the tree_id of the repository. The tree_id
is fingerprint of the contents of the repository for a given commit. The key
of the job and the tree_id are combined to a unique identity in Cider-CI.
This is an important feature of Cider-CI as it provides transparency,
reproducibility and avoids overhead.
It can also imposes some inconvenience when working with changing external resources as it requires to change and submit the repository contents to rerun a job and in consequence forces us to rerun all related jobs.
Consider the usage of an external API for example. In an ideal world the
external API will use dedicated versions which in turn would be specified in
our project. This causes the creation of a new tree_id which links our
project to the external API in a unique and traceable way. In reality the
external API might not proceed in such stringent ways but change ad hoc without
publishing changes. In such a case we probably might not want to create a new
commit and rerun all associated jobs but just rerun a particular job that
covers the interaction with the external service.
We will illustrate the project configuration by the example of a deploy job. There are many volatile properties involved. For example the state of the server, or the state of the database which undergoing migrations during the deploy.
We will focus on the two directives exclusive_global_resources
and aggregate_state in the following sections.
The shown deploy configuration is a slightly simplified variant of the original which can be found in the Madek Deploy Project.
jobs:
deploy-to-test:
name: Deploy to "test.madek"
context:
tasks:
deploy:
exclusive_global_resources:
"test.madek.zhdk.ch": true
aggregate_state: satisfy-last
traits:
Ansible 2: yes
scripts:
deploy:
body: >
ansible-playbook
play_setup-and-deploy.yml
exclusive_global_resources PropertyThe exclusive_global_resources prevents concurrent access to the same
resource. The syntax based on maps is used to provide composability and is
discussed in the Composing Data page of the documentation.
exclusive_global_resources:
"test.madek.zhdk.ch": true
Running concurrent deploys to the same server would almost certainly cause
problems and can even leave the server in a non recoverable state. Cider-CI
will ensure that at most one trial is executing at any time with respect to the
string test.madek.zhdk.ch. This spans over jobs and even projects. It is
therefore important to specify the resource precisely. Using simple strings
like test can easily lead to unintended locking and delays. However,
deadlocks are impossible because tasks never depend on each other.
aggregate-state PropertyA task features the aggregate_state property. The value controls how
the state of a task is aggregated.
aggregate-state: satisfy-last
The default value satisfy-any will
let the corresponding task to assume the passed state if any of its trial
has passed. This is the core functionality on which the resilience
against false test negatives is build.
For working with changing external environments the value can be set to
satisfy-last. Then the state of the task corresponds to the state of the
trial created most recently.
A job where all tasks are set to satisfy-any will always stay passed if it
has reached this state once. The state of a job where the aggregate_state
property of some task is set to satisfy-last is volatile. It can possibly
switch to any state at any time later. It can switch from passed to
failed, and temporarily from a terminal state to a non terminal state.
We likely need to act on external events to make use of of the statisfy-last
setting.
Cider-CI provides internal mechanisms to trigger the execution of a job via branch updates or changes of the state where the target job depends on. Jobs and retries can also be triggered via user interaction or posting actions via the API. The latter is apparently suitable for interaction from external events. We will show a typical scenario in the following. We will use the Ruby JSON-ROA Client which in turn uses the JSON-ROA extension of the Cider-CI API.
We initialize the client in the first step.
@client = JSON_ROA::Client.connect API_BASE_URL do |conn|
conn.basic_auth(API_LOGIN, API_PASSWORD)
end
Next, we retrieve the current tree_id of the targeted branch
and repository.
def get_tree_id(repository_url, name_branch_head)
@client.get.relation(:commits) \
.get(repository_url: repository_url,
branch_head: name_branch_head) \
.collection.first.get.data[:tree_id]
end
We either want execute a job if it does not exist yet, or create a new trial
otherwise. This code will return an id if the job exists or nil otherwise.
def get_job_id(tree_id, key)
job_rel = @client.get.relation(:jobs) \
.get(tree_id: tree_id, key: key) \
.collection.first
job_rel && job_rel.get.data[:id]
end
Creating a new job can be performed by sending the tree_id and the key as
defined in the project configuration of the job via a post request.
def create_job(tree_id, key)
@client.get.relation(:create_job) \
.post({}, { tree_id: tree_id, key: key }.to_json,
content_type: 'application/json')
end
Jobs like a deploy will often have only one task. This code creates a retry for the first (and only) task.
def retry(job_id)
@client.get.relation(:job).get(id: job_id) \
.relation(:tasks).get.collection.first.get \
.relation(:trials).get.relation(:retry).post
end
The examples above have been extracted from the Madek Nightly-Deploy-Script. The code given in the project provides a complete example.
The default project configuration in Cider-CI encourages to take the source code as fundamental truth of a project which solely determines the outcome of a job. Cider-CI provides configuration properties to additionally take volatile environments into account.