@idearium/cli
The Idearium cli, which makes working with our projects, super easy :)
Documentation
The cli is self documenting and you'll find limited information about the commands within the cli here.
Install
Via NPM:
$ npm install -DE [-g] @idearium/cli
We actually recommend installing @idearium/cli
locally in your project. This allows each project to use a different version of the cli. If you do however, we also recommend adding ./node_modules/.bin
to your path, so that you can run the cli via c
without a global install.
Usage
Once installed, simply run c
, and you'll get the help output for all commands. Running a sub-command and then --help
will give you the help output for that command (i.e. c dc --help
).
Requirements
The cli has been designed to work closely with infrastructure-common. Infrastructure provides MongoDB, Redis, RabbitMQ and importantly for the cli, Kubernetes ingress.
You can read more about how this all hangs together at https://github.com/idearium/developers/tree/master/stack
Commands
The following is a summary of the top level commands.
-
c d
is for everything Docker. -
c dc
is for everything Docker Compose. -
c hosts
helps with hosts management. -
c kc
is for everything kubectrl. -
c mk
is for everything Minikube. -
c mongo
is for MongoDB connections. -
c npm
is for everything NPM. -
c project
is for project management. -
c yarn
is for everything is for everything Yarn.
Be aware that kubectl
uses a global configuration, but the c kc
command supercedes those where possible and ensures kubectl runs in the context of the project you're within.
Concepts
The cli has two concepts to be aware of:
- Configuration
- State
The cli can be highly configured within information about your project, and work with Docker, NPM and Kubernetes with the information provided. All configuration is stored within a c.js
file that lives in the root of your project. This file should be stored in Git.
The cli is designed to work with multiple environments. You put your project into a particular environment, and the commands run within the context of that environment. All state is stored in devops/state.json
(this file should be added to .gitignore
).
Configuration
The Idearium cli can be customised through configurations. Configurations are provided through a c.js
file located in projects root folder.
The c.js
should be a standard Node.js file. It will be loaded through require
. The c.js
should export an object, with the following keys representing configuration:
module.exports = {
docker: {
// ...
},
environments: {
// ...
},
kubernetes: {
// ...
},
mongo: {
// ...
},
npm: {
// ...
},
project: {
// ...
},
};
The command c project init
can be used to initialise a project. It will generate a c.js
file as a starting point, although, it will need to be completed.
Docker configuration
The Idearium cli supports a Docker configuration. The Docker configuration can be used to define locations of multiple Dockerfiles within your project.
An example Docker configuration:
'use strict';
const { exec } = require('shelljs');
module.exports = {
docker: {
locations: {
'app': {
buildArgs: {
NPM_AUTH_TOKEN: () => exec('c npm auth -n', { silent: true }).stdout,
},
path: './app',
useTar: true,
},
'static': {
path: './static',
},
},
},
};
The Docker configuration supports the following keys.
locations
The locations key should be an object, describing all Dockerfile locations in your project. Each key should be the name of a Dockerfile location. You can use the name to reference the location when using the cli. Each location supports the following structure.
path
A path to the Dockerfile location.
buildArgs
An object represent key=value --build-arg
flags to pass to docker build
. Each property should present a build argument name, and the value for the build argument. The value can be either a static value (i.e. string, number) or a function.
useTar
If useTar
is set to true, tar
will be used to prepare the Docker context and package and send files to the Docker daemon. This allows for some additional flexibility such as using symlinks which point outside the Dockerfile context.
test
You can supply a test
object to enable the c kc test <location>
command. test
has the following properties:
-
cmd
containing the command to run, inside the Docker container. It defaults tonpm test
. -
params
containing an object keyed bydocker run
parameter name. For example:{ '-e': ['foo="bar"', 'bar="foo"'], '-w': '/app' }
. This allows you to customise thedocker run
command used when testing the image.
Environments configuration
The Idearium cli supports multiple environments. The environments can be whatever you need them to be as long as you define them in c.js
. Here is an example environments configuration:
'use strict';
module.exports = {
environments: {
local: { url: 'http://domain.name.local' },
production: { url: 'https://domain.name.com' },
},
};
For each environment, you should provide a URL for the environment.
Kubernetes configuration
You can supply a Kubernetes configuration. The configuration allows you to define contexts and namespaces for each environment your project supports, along with Kubernetes locations which describe the Kubernetes objects to deploy. Here is an example kubernetes configuration:
'use strict';
module.exports = {
kubernetes: {
environments: {
local: {
context: 'minikube',
locations: {
/* eslint-disable sort-keys */
'namespace': [
{
path: 'namespace',
templateLocals: ['environment', 'namespace'],
type: 'namespace',
},
],
'app': [
{
path: 'app.deployment',
templateLocals: ['namespace', 'prefix', 'tag'],
type: 'deployment',
},
{
path: 'app.service',
templateLocals: ['namespace'],
type: 'service',
},
],
'static': [
{
path: 'static.deployment',
templateLocals: ['namespace', 'prefix', 'tag'],
type: 'deployment',
},
{
path: 'static.service',
templateLocals: ['environment', 'namespace'],
type: 'service',
},
],
/* eslint-enable sort-keys */
},
path: './manifests/local'
},
production: {
context: 'gke_focus-booster_us-east1-b_focus-booster',
namespace: 'override',
region: 'us-east1',
},
}
},
};
The kubernetes.environments
path holds a descrete environment. Each Kubernetes environment supports the following keys:
- context, the kubectl context that should be used.
- locations, the Kubernetes objects to work with.
- namespace, if provied, will override the generated namespace.
- path, the path to the folder containing Kuberentes manifest YAML files.
- region, an optional region, mostly required for production.
Kubernetes locations
The Kuberentes locations object describes all Kubernetes objects for that Kuberentes environment. Each location should contain the following ({}
used to represent meaning):
locations: {
{location-name}: [
{
dockerLocation: '',
path: '',
templateLocals: [],
type: '',
}
]
}
The location-name
should be unique, and can be used within the cli commands to target a specific Kubernetes location.
Each location should be an array of services. One Kubernetes location can contain multiple services.
Each service object should provide:
-
dockerLocation
(optional): The Docker location that a particular Kubernetes service object is associated with. This is usually provided with adeployment
orpod
object. -
path
: The filename within the Kubernetes environment path. The filename should not include the extension; but onlyyaml
andyaml.tmpl
are supported. -
templateLocals
: An array to provide a list of locals that should be passed to atmpl
file to create ayaml
file. -
type
: The type of Kubernetes object this service describes (i.e.pod
,deployment
,namespace
).
YAML templates
The c kc apply
command supports templates, which will generate yaml
files which will be used to deploy Kuberntes objects.
It supports templates, because you'll often want to substitute specific information for Kubernetes during development. A good example is the tag
that should be used with a particular image.
If you'd like to use a template create Kubernetes manifest file ending in .yaml.tmpl
rather than tmpl
. The path
in the Kubernetes location service object, should not contain the extension. In the template file use {{tag}}
to substitute with an actual value before being applied to Kubernetes.
You'll then need to supply all of the values that the template file requires. You provide this via the templateLocals
array in the Kubernetes location service object. Here is an example:
{
path: 'app.deployment',
templateLocals: ['namespace', 'prefix', 'tag', () => { label: 'a', value: 'b' }],
type: 'deployment'
}
The c kc apply
will automatically provide the values for namespace
, prefix
and tag
. If you'd like to provide something else, simply write a function that returns an object with label
and value
. Then use the value of label
within a template placeholder (i.e. {{a}}
) and it will be updated with the value
(i.e. {{b}}
).
MongoDB configuration
The Idearium cli supports a MongoDB configuration. The MongoDB configuration can be used to access local and remote databases.
An example MongoDB configuration:
'use strict';
module.exports = {
mongo: {
beta: {
host: 'dbhost.com',
name: 'db-beta',
password: 'password',
port: '12345',
ssl: true,
user: 'user',
},
local: {
host: 'db.common.idearium.local',
name: 'db',
port: '27017',
ssl: false,
},
beta: {
host: 'dbhost.com',
name: 'db',
password: 'password',
port: '12345',
ssl: true,
user: 'user',
},
},
};
The MongoDB configuration supports the following keys.
Environment
Each top level key will be used as an env
variable, setting the context for the commands. e.g. c mongo connect local
.
Each environment can contain the following keys host
, name
, password
, port
, ssl
, and user
.
password
and user
are currently optional for local connections.
NPM configuration
The Idearium cli supports an NPM configuration. The configuration can be used to provide the cli without information about where NPM commands can be run. To provide this information, add an npm
property in c.js
like so:
'use strict';
module.exports = {
npm: {
locations: {
name: './app/root/app/',
project: './',
},
},
};
You should customise it, but you need to provide a locations
key, containing an object with a name and folder for each NPM location in your project. You'll use the name to reference the location.
For example, the c npm proxy
command can be used to run a NPM command, at a specific location or all locations:
- Execute
c npm proxy all install -SE logentries
to install thelogentries
module at all NPM locations in your project. - Execute
c npm proxy project install -DE jest
to install thejest
module at theproject
location in your project.
Project configuration
The Idearium cli supports a project configuration. This configuration is used to provide some general information about the project. This information is also used to automatically generate some strings such as the project prefix, and Kubernetes namespaces. Here is an example project configuration:
'use strict';
module.exports = {
project: {
name: 'www',
organisation: 'fb'
},
};
State
The Idearium cli, understands the concept of state. The only state it manages at present is the environment of your project. The current environment of your project will be used in certain commands, but not all commands.
State is stored in a JSON file at ./devops/state.json
, and is created and written to by the c project env set
and c project setup
commands.
Ngrok
The Idearium cli can be used to easily expose a Kubernetes service to the world via Ngrok, and the c kc ngrok
command.
To use this command, however, you need to provide a configuration file for Ngrok. Follow these steps.
- Create a file at
./manifests/local/ngrok.configmap.yaml.tmpl
. - Add the code from the Manifest template (below) to the file and update
<authtoken>
with your Ngrok authtoken (or remove that line) and update<hostname>
with the hostname you'd like the tunnel to be exposed at. - Added the code from the Kubernetes location (below) to
c.js
in thekubernetes.environments.local.locations
array. - Then run
c kc ngrok start
in a new terminal (after runningc mk docker-env
). - To stop, issue
CTRL + C
, then runc kc ngrok stop
to stop ngrok.
Manifest template:
apiVersion: v1
kind: ConfigMap
metadata:
name: ngrok
namespace: {{namespace}}
data:
ngrok.yaml: |
authtoken: <authtoken>
console_ui: false
log: stdout
region: au
tunnels:
http:
proto: http
bind_tls: false
addr: static:80
hostname: <hostname>
update: false
Kubernetes location:
'ngrok': [
{
path: 'ngrok.configmap',
templateLocals: ['namespace'],
type: 'configmap',
},
],