Quilk CI

Quilk CI is a super lightweight automated run tool, typically used in conjunction with git webhooks.

Index

• For who
• What is Quilk CI
• Installing Quilk CI and setting up and instance
• Updating the Quilk CI core after installation
• How to configure Quilk CI
• Quilk CI security
• Adding new sites

Who

Quilk CI is a lightweight continuous integration system build in node for linux servers. It might work on a windows box and there is no reason it should not.

What

Quilk CI responds to http requests eg from a webhook and then runs a series of configurable tasks. A typical use case is to add a webhook to say bitbucket or github which is called on a git push to the main branch, this url maps to quilk ci which in turn automatically git pulls a specific branch, pulls any new dependencies and runs a https://www.npmjs.com/package/quilk file to build client side js or css files.

Quilk CI runs as a HTTP(S) service via the popular package express. The rest API maps to a json config file name (minus the file extension): <your domain>/ci/<file name>. The config files are site specific, in other words one config file per site and the config files for each site are stored in a directory named sites.

There are currently 3 modules available for any site json:

1. git_pull. This resets and pulls a specific branch.
2. quilk_build. This will run quilk in the given project and also accepts arguments to build for a specific release eg: release=dev.
3. command_run. This will basically run any program with arguments on commandline.

(when you add a site via the command line helper you will have an example .json file with examples for all 3 modules)

Installing

First up

Install quilk_ci globally to your system:

npm install -g quilk_ci

Second

With quilk_ci installed globally you must now create a quilk_ci instance somewhere on you system.

Navigate to an empty directory you wish to use for your quilk_ci instance and then run:

quilk_ci install

You will be asked if you wish to continue, select Y for both questions. This will install and setup you quilk_ci.

Third

You are now ready to start quilk_ci. In the directory you installed quilk_ci to you should see a file labeled server.js, this is the main server instance you need to run to start the server. It is recommended that you use a node instance manager such as pm2 (npm install -g pm2). Starting quilk_ci with pm2:

pm2 start server.js --watch

Then check out the logs to see it in action, pm2 logs.

See https://www.npmjs.com/package/pm2 for all the pretty awesome tools pm2 offers for node application management.

Updating Quilk CI core

The Quilk CI app is continually growing and adding more and more functionality, however there is one thing that will not change and this the JSON based configuration. So to keep up to date with all the new goodies in Quilk CI but retain your own configurations just update the globally installed quilk_ci then update your local running instance.

1. Update the globally installed quilk_ci tool npm install -g quilk_ci
2. IMPORTANT: Now as the linux user your quilk_ci instance is running as eg, su www-data. Else you will have permission issues.
3. The update a running instance from the directory: quilk_ci update

If you are running pm2 with the watch flag you should now be updated and done. Else you may have to restart your node app with pm2 restart

Configuration

Quilk CI is configured via std JSON files. For example, your quilk instance once started will be listening on the port that is listed in the config/default.json.

After the installation of quilk_ci you will be left with a directory named sites with a sample config file within. The sample config is configured to use two inbuilt modules, git_pull and quilk_build, and you will also note two different blocks, deploy and build.

NB: The git_pull feature only knows how to not pull when the webhook payload is from bitbucket. When you add hosting_provider to the mix in the git pull, the module will not continue if the branch in the webhook post data is not from dev. In other words, a commit to branch feature/jira_id_123 will not match so will not build. eg:

{
  "deploy": [{
    "module": "git_pull",
    "branch": "dev",
    "hosting_provider" : "bitbucket",
    "dir":"/some/path/to/git/pull"
  }],
  "build": [{
    "module": "quilk_build",
    "arguments": ["release=dev"],
    "dir": "/some/path/to/quilk/build"
  }]
}

When the url to your instance is called, the module runner will first run the array of modules in the deploy block, once completed it will run the array of modules in the build block. The reason behind this is to allow for a repository to first update its code before rebuilding.

As a test and with the quilk_ci instance running on your machine point a browser at <your domain>/ci/sample-site.

Security

To view the root of the quilk_ci in a browser you will need to add a user: quilk_ci auth add-username

There are also the following commands: quilk_ci auth change-username quilk_ci auth change-password

To whitelist or blacklist an IP or IP range your conf might look like:

{
 "ip_whitelist":{
  "12.71.23.52" : "bitbucket"
 },
 "ip_blacklist": {
  "::ffff:127.0.0.0/6": "local"
 },
 "users": {
   "john": "$2a$10$YPXLZGXDIQN3G64Na51oI.w4vNaM56RUHbnXHs2q99pi4iCDBrgFa"
 }
}

NB: For now you must have at least and empty object for each top level key.

I will add some commandline tools to manage whitelisting of IPs later.

Adding new sites

Adding a new site to the instance is as easy as adding a new json file to the sites folder. This can be done manually or via the command line helper: quilk_ci sites add. Just follow the prompts.