This package has been deprecated

Author message:

deprecated

@mortonprod/consul-communication

Parameter from node consul

We use the consul function below to produce the network information further down.

consul.catalog.service.list(function(err, result) {
  if (err) throw err;
  console.log(result)
});

{ consul: [],
  fabio: [],
  gab: [ 'urlprefix-/gab strip=/gab' ],
  gba: [ 'urlprefix-/gba strip=/gba' ],
  gcb: [ 'urlprefix-/gcb strip=/gcb' ],
  oauth: 
   [ 'urlprefix-/oauth/token',
     'urlprefix-/oauth/authorize',
     'urlprefix-/oauth/authenticate',
     'urlprefix-/oauth/static' ],
  oh: [ 'urlprefix-/oh strip=/oh' ],
  rule: 
   [ 'urlprefix-/rules/fact strip=/rules',
     'urlprefix-/rules/logs/ strip=/rules',
     'urlprefix-/rules/stop strip=/rules',
     'urlprefix-/rules/configs strip=/rules',
     'urlprefix-/rules strip=/rules',
     'urlprefix-/rules/run strip=/rules',
     'urlprefix-/rules/networks strip=/rules',
     'urlprefix-/rules/network strip=/rules',
     'urlprefix-/rules/config strip=/rules' ] }

We can then use the information to get the ip and ports of your service.

consul.catalog.service.nodes('rule', function(err, result) {
  if (err) throw err;
  console.log(result)
});

[ { ID: 'b56b4882-390e-24c1-1c7e-3dbadf054539',
    Node: 'i-083a0561357891ba1',
    Address: '10.0.0.140',
    Datacenter: 'eu-west-2',
    TaggedAddresses: { lan: '10.0.0.140', wan: '10.0.0.140' },
    NodeMeta: { 'consul-network-segment': '' },
    ServiceID: 'rule-d3c1d2a0',
    ServiceName: 'rule',
    ServiceTags: 
     [ 'urlprefix-/rules/fact strip=/rules',
       'urlprefix-/rules/logs/ strip=/rules',
       'urlprefix-/rules/logs/ strip=/rules',
       'urlprefix-/rules/run strip=/rules',
       'urlprefix-/rules/stop strip=/rules',
       'urlprefix-/rules/networks strip=/rules',
       'urlprefix-/rules/network strip=/rules',
       'urlprefix-/rules/config strip=/rules',
       'urlprefix-/rules/configs strip=/rules',
       'urlprefix-/rules strip=/rules' ],
    ServiceAddress: '',
    ServicePort: 3001,
    ServiceEnableTagOverride: false,
    CreateIndex: 97,
    ModifyIndex: 97 },
  { ID: 'cc2a215d-2340-2e64-6f04-5a7700cf89d9',
    Node: 'i-0e037e4022f53f7f8',
    Address: '10.0.2.38',
    Datacenter: 'eu-west-2',
    TaggedAddresses: { lan: '10.0.2.38', wan: '10.0.2.38' },
    NodeMeta: { 'consul-network-segment': '' },
    ServiceID: 'rule-968d31d2',
    ServiceName: 'rule',
    ServiceTags: 
     [ 'urlprefix-/rules/fact strip=/rules',
       'urlprefix-/rules/logs/ strip=/rules',
       'urlprefix-/rules/logs/ strip=/rules',
       'urlprefix-/rules/run strip=/rules',
       'urlprefix-/rules/stop strip=/rules',
       'urlprefix-/rules/networks strip=/rules',
       'urlprefix-/rules/network strip=/rules',
       'urlprefix-/rules/config strip=/rules',
       'urlprefix-/rules/configs strip=/rules',
       'urlprefix-/rules strip=/rules' ],
    ServiceAddress: '',
    ServicePort: 3001,
    ServiceEnableTagOverride: false,
    CreateIndex: 94,
    ModifyIndex: 94 } ]

Installing with docker

So you don't have to worry about installing all consul dependencies install the docker image:

Docker image of consul

Using docker and consul 😉

Raft and server nodes 😂

In consul, client nodes(agents) will forward requests to servers. This is to reduce the number of peer nodes which is related to the the quorum as:

quorum = (n/2) + 1

The quorum is the number of nodes which need to agree on reproducing ordered log files deterministically

Of course you will need to install docker first!

Building

We want to setup the container using some configuration. This will also setup the container with some docker volume which we can attach to the host system. See below...

docker run -d --name=dev-consul -e CONSUL_BIND_INTERFACE=eth0 consul

• -d: This just tells docker to run the container in detached mode like you could with any process.
• --name=dev-consul : This will specify the container name.
• -e CONSUL_BIND_INTERFACE=eth0 consul: This will look up client or bind address by interface name.
  • Similar to DNS resolution...

Learn about the containers known ip addresses

If you want to learn about the other agents the current agent knows about.

docker exec -t dev-consul consul members

This is just a docker command to run the "consul members" in the dev-consul using bash command line.

This is useful for two reasons:

• You learn about the current known members of the cluster tha agent is apart of.
• You can use the IP address to allow other agents to join that agent.

Joining agents

Assuming the IP address of the member you searched above was 172.17.0.2. Not a large stretch of the imagination since it was on my computer 😃 .

Then, run the command below to attach a new agent to the agent at that IP address.

docker run -d -e CONSUL_BIND_INTERFACE=eth0 consul agent -dev -join=172.17.0.2

Web UI.

Note if you have the UI running then this might stop the clients from being created. Need to investigate. Something to do with port conflicts.

Produce the UI with:

MAC/UNIX

docker run -d -p 8500:8500  -e CONSUL_BIND_INTERFACE=eth0 consul agent -ui  -dev -join=172.17.0.2 -client=0.0.0.0 -bind='{{ GetPrivateIP }}'

Windows

 docker run -d -p 8500:8500  -e CONSUL_BIND_INTERFACE=eth0 consul agent -ui  -dev -join=172.17.0.2 -client=0.0.0.0

Creating Client agents

Up until know we have been creating server agents since this is the default behaviour of consul. To create client agents we would run:

docker run -d -e CONSUL_BIND_INTERFACE=eth0 consul agent -retry-join=172.17.0.2

In the docker consul readme specifies to use host network but then proceeds to use the default bridge for the first server!!!???!!!

Storage

If you run in production then the container will use /consul/data as persistent storage. What is saved will depend on if the container was initialised as a server or client agent. For client agents, this stores some information about the cluster and the client's health checks in case the container is restarted. For server agents, this stores the client information plus snapshots and data related to the consensus algorithm and other state like Consul's key/value store and catalog. For servers it is highly desirable to keep this volume's data around when restarting containers to recover from outage scenarios. If this is bind mounted then ownership will be changed to the consul user when the container starts.

The persistant storage can be of three forms

• Volumes: This is storing within the container.
  • This ties the data to the host system.
  • It means the container must be running to access the data.
  • Require an inefficient storage driver.
• Bind mounts: Mount a directory from your computer to the container. *
• tmpfs: Store to non-persistant memory for the lifetime of the container.

Configuration

The configuration of the container is found at /consul/config. The simplest way of updating this is by binding a host directory to that mount point.

Agents

Every service will have a consul client agent. Each agent can communicate with one or more consul servers. Consul servers will elect a leader which coordinates between the servers. A cluster of servers can communicate with a datacentre.

An agent or server can be queried about services. An agent will forward a request to a server by default. Communication is done using the consensus and gossip protocols. These have their own concept of cluster and client IP addresses.

Cluster

A cluster address is the basically has the same functionality as a gateway address. However there is additional functionality provided by the consul protocols.

Some of that functionality is that a agent can join any other agent and learn about the cluster as a whole my gossiping

Finally...

Consul should always be run with --net=host in Docker because Consul's consensus and gossip protocols are sensitive to delays and packet loss, so the extra layers involved with other networking types are usually undesirable and unnecessary

Including Terraform

A repository which links terraform, AWS and docker with consul.