|
|
@@ -21,9 +21,9 @@ Consul is a distributed, highly-available, and multi-datacenter aware tool for s
|
|
|
|
|
|
Consul has several moving parts so we'll start with a brief introduction to Consul's architecture and then detail how Consul interacts with Docker. Please see the [Consul Architecture](https://www.consul.io/docs/internals/architecture.html) guide for more detail on all these concepts.
|
|
|
|
|
|
-Each host in a Consul cluster runs the Consul agent, a long running daemon that can be started in client or server mode. Each cluster has at least 1 agent in server mode, and usually 3 or 5 for high availability. The server agents participate in a [consensus protocol](https://www.consul.io/docs/internals/consensus.html), maintain a centralized view of the the cluster's state, and respond to queries about the cluster from other agents in the cluster. The rest of the agents in client mode participate in a [gossip protocol](https://www.consul.io/docs/internals/gossip.html) to discover other agents and check them for failures, and they forward queries about the cluster to the server agents.
|
|
|
+Each host in a Consul cluster runs the Consul agent, a long running daemon that can be started in client or server mode. Each cluster has at least 1 agent in server mode, and usually 3 or 5 for high availability. The server agents participate in a [consensus protocol](https://www.consul.io/docs/internals/consensus.html), maintain a centralized view of the cluster's state, and respond to queries from other agents in the cluster. The rest of the agents in client mode participate in a [gossip protocol](https://www.consul.io/docs/internals/gossip.html) to discover other agents and check them for failures, and they forward queries about the cluster to the server agents.
|
|
|
|
|
|
-Applications running on a given host communicate only with their local Consul agent, using its HTTP APIs or DNS interface. Services on the host are also registered with the local Consul agent, which shares the information with the Consul servers. Doing the most basic DNS-based service discovery using Consul, an application queries for `foo.service.consul` and gets a randomly shuffled subset of all the hosts providing service "foo". This allows services to locate each other and balance the load without any intermediate proxies. Several HTTP APIs are also available for applications doing a deeper integration with Consul's service discovery capabilities, as well as its other features such as the key/value store.
|
|
|
+Applications running on a given host communicate only with their local Consul agent, using its HTTP APIs or DNS interface. Services on the host are also registered with the local Consul agent, which syncs the information with the Consul servers. Doing the most basic DNS-based service discovery using Consul, an application queries for `foo.service.consul` and gets a randomly shuffled subset of all the hosts providing service "foo". This allows applications to locate services and balance the load without any intermediate proxies. Several HTTP APIs are also available for applications doing a deeper integration with Consul's service discovery capabilities, as well as its other features such as the key/value store.
|
|
|
|
|
|
These concepts also apply when running Consul in Docker. Typically, you'll run a single Consul agent container on each host, running alongside the Docker daemon. You'll also need to configure some of the agents as servers (at least 3 for a basic HA setup). Consul should always be run with `--net=host` in Docker because Consul's consensus and gossip protocols are sensitive to delays and packet loss, so the extra layers involved with other networking types are usually undesirable and unnecessary. We will talk more about this below.
|
|
|
|
|
|
@@ -35,9 +35,9 @@ We chose Alpine as a lightweight base with a reasonably small surface area for s
|
|
|
|
|
|
Consul always runs under [dumb-init](https://github.com/Yelp/dumb-init), which handles reaping zombie processes and forwards signals on to all processes running in the container. We also use [gosu](https://github.com/tianon/gosu) to run Consul as a non-root "consul" user for better security. These binaries are all built by HashiCorp and signed with our [GPG key](https://www.hashicorp.com/security.html), so you can verify the signed package used to build a given base image.
|
|
|
|
|
|
-Running the Consul container with no arguments will give you a Consul server in [development mode](https://www.consul.io/docs/agent/options.html#_dev). The provided entry point script will also look for Consul subcommands and run `consul` as the correct user and that subcommand. For example, you can excecute `docker run consul members` and it will run the `consul members` using the container. The adds some special configration options as detailed in the sections below when running the `agent` subcommand. Any other command gets `exec`-ed inside the container under `dumb-init`.
|
|
|
+Running the Consul container with no arguments will give you a Consul server in [development mode](https://www.consul.io/docs/agent/options.html#_dev). The provided entry point script will also look for Consul subcommands and run `consul` as the correct user and with that subcommand. For example, you can execute `docker run consul members` and it will run the `consul members` command inside the container. The entry point also adds some special configuration options as detailed in the sections below when running the `agent` subcommand. Any other command gets `exec`-ed inside the container under `dumb-init`.
|
|
|
|
|
|
-The container exposes `VOLUME /consul/data`, which is a path were Consul will place its persisted state. This isn't used in any way when running in development mode. For client agents, this stores some information about the cluster and the client's health checks, in case the container is restarted. For server agents, this stores the client information plus snapshots and data related to the consensus algorithm and Consul's catalog of services. For servers it is highly desirable to keep this volume's data around when restarting the containers to recover from outage scenarios.
|
|
|
+The container exposes `VOLUME /consul/data`, which is a path were Consul will place its persisted state. This isn't used in any way when running in development mode. For client agents, this stores some information about the cluster and the client's health checks in case the container is restarted. For server agents, this stores the client information plus snapshots and data related to the consensus algorithm and other state like Consul's key/value store and catalog. For servers it is highly desirable to keep this volume's data around when restarting containers to recover from outage scenarios.
|
|
|
|
|
|
The container has a Consul configuration directory set up at `/consul/config` and the agent will load any configuration files placed here by binding a volume or by composing a new image and adding files. Alternatively, configuration can be added by passing the configuration JSON via environment variable `CONSUL_LOCAL_CONFIG`.
|
|
|
|
|
|
@@ -66,7 +66,6 @@ Then we can query for all the members in the cluster by running a Consul CLI com
|
|
|
$ docker exec -t dev-consul consul members
|
|
|
Node Address Status Type Build Protocol DC
|
|
|
579db72c1ae1 172.17.0.3:8301 alive server 0.6.3 2 dc1
|
|
|
-7e185aebe4e6 172.17.0.3:8301 left server 0.6.3 2 dc1
|
|
|
93fe2309ef19 172.17.0.4:8301 alive server 0.6.3 2 dc1
|
|
|
c9caabfd4c2a 172.17.0.2:8301 alive server 0.6.3 2 dc1
|
|
|
```
|
|
|
@@ -159,15 +158,15 @@ This runs a Consul server agent sharing the host's network. All of the network c
|
|
|
|
|
|
Just like the client agent, the `-retry-join` parameter specifies the external IP of one other agent in the cluster to use to join at startup. There are several ways to control how an agent joins the cluster, see the [agent configuration](https://www.consul.io/docs/agent/options.html) guide for more details on the `-join`, `-retry-join`, and `-atlas-join` options. The server agent also consumes a `-bootstrap-expect` option that specifies how many server agents to watch for before bootstrapping the cluster for the first time. This provides an easy way to get an orderly startup with a new cluster. See the [agent configuration](https://www.consul.io/docs/agent/options.html) guide for more details on the `-bootstrap` and `-bootstrap-expect` options.
|
|
|
|
|
|
-Note also we've set [`skip_leave_on_interrupt`](https://www.consul.io/docs/agent/options.html#skip_leave_on_interrupt) using the `CONSUL_LOCAL_CONFIG` environment variable. This is recommended for servers to and will be defaulted to `true` in Consul 0.7 and later, so this will no longer be necessary.
|
|
|
+Note also we've set [`skip_leave_on_interrupt`](https://www.consul.io/docs/agent/options.html#skip_leave_on_interrupt) using the `CONSUL_LOCAL_CONFIG` environment variable. This is recommended for servers and will be defaulted to `true` in Consul 0.7 and later, so this will no longer be necessary.
|
|
|
|
|
|
At startup, the agent will read config JSON files from `/consul/config`. Data will be persisted in the `/consul/data` volume.
|
|
|
|
|
|
-Once the cluster is bootstrapped and quorum is achieved, you must use care to try to keep the minimum number of servers operating in order to avoid an outage state for the cluster. The deployment table in the [consensus](https://www.consul.io/docs/internals/consensus.html) guide outlines the number of servers required for different configurations. There's also an [adding/removing servers](https://www.consul.io/docs/guides/servers.html) guide that describes that process, which is relevant to Docker configurations as well. The [outage recovery](https://www.consul.io/docs/guides/outage.html) guide has steps to perform if servers are permanently lost. In general it's best to restart or replace servers one at a time, making sure servers are healthy before proceeding to the next server.
|
|
|
+Once the cluster is bootstrapped and quorum is achieved, you must use care to keep the minimum number of servers operating in order to avoid an outage state for the cluster. The deployment table in the [consensus](https://www.consul.io/docs/internals/consensus.html) guide outlines the number of servers required for different configurations. There's also an [adding/removing servers](https://www.consul.io/docs/guides/servers.html) guide that describes that process, which is relevant to Docker configurations as well. The [outage recovery](https://www.consul.io/docs/guides/outage.html) guide has steps to perform if servers are permanently lost. In general it's best to restart or replace servers one at a time, making sure servers are healthy before proceeding to the next server.
|
|
|
|
|
|
## Exposing Consul's DNS Server on Port 53
|
|
|
|
|
|
-By default the dev, client, and server modes started by the endpoint will expose Consul's DNS server on port 8600. Because this is cumbersome to configure with facilities like `resolv.conf`, you may want to expose DNS on port 53 using port arguments on your run command:
|
|
|
+By default, Consul's DNS server is exposed on port 8600. Because this is cumbersome to configure with facilities like `resolv.conf`, you may want to expose DNS on port 53 using port arguments on your run command:
|
|
|
|
|
|
```console
|
|
|
$ docker run -d --net=host -p 53:8600/tcp -p 53:8600/udp consul
|
|
|
@@ -195,7 +194,7 @@ In the example above, adding the bridge address to the host's `/etc/resolv.conf`
|
|
|
|
|
|
## Service Discovery with Containers
|
|
|
|
|
|
-There are several approaches you can use to register services running in containers with Consul. For manual configuration, your containers can use the local agent's APIs to register and deregister themselves, see the [Agent API](https://www.consul.io/docs/agent/http/agent.html) for more details. Another strategy could be to create a derived Consul container for each host type which includes JSON config files for Consul to parse at startup, see [Services](https://www.consul.io/docs/agent/services.html) for more information. Both of these approaches are fairly cumbersome, and the configured services may fall out of sync if containers die or additional containers are started.
|
|
|
+There are several approaches you can use to register services running in containers with Consul. For manual configuration, your containers can use the local agent's APIs to register and deregister themselves, see the [Agent API](https://www.consul.io/docs/agent/http/agent.html) for more details. Another strategy is to create a derived Consul container for each host type which includes JSON config files for Consul to parse at startup, see [Services](https://www.consul.io/docs/agent/services.html) for more information. Both of these approaches are fairly cumbersome, and the configured services may fall out of sync if containers die or additional containers are started.
|
|
|
|
|
|
If you run your containers under [HashiCorp's Nomad](https://www.nomadproject.io/) scheduler, it has [first class support for Consul](https://www.nomadproject.io/docs/jobspec/servicediscovery.html). The Nomad agent runs on each host alongside the Consul agent. When jobs are scheduled on a given host, the Nomad agent automatically takes care of syncing the Consul agent with the service information. This is very easy to manage, and even services on hosts running outside of Docker containers can be managed by Nomad and registered with Consul. You can find out more about running Docker under Nomad in the [Docker Driver](https://www.nomadproject.io/docs/drivers/docker.html) guide.
|
|
|
|