Load-balancing Docker containers with Nginx and Consul-Template

This article has been cross-posted here from my employer’s technical blog.

We are investing a lot of research and development time into leveraging Docker in the next generation of our internal infrastructure. One of the next components we need to build out to full maturity is being able to dynamically route web traffic from our [Nginx][nginx] load balancers to internal Docker containers in a performant way.

We are very passionate fans of the work of HashiCorp at Belly, and they recently published a new project named Consul-Template. We were using an earlier HashiCorp tool named consul-haproxy to reconfigure our Nginx load-balancers based on Consul data. Consul-Template is a slightly more generalized tool that was fairly smooth to adopt.

Let me walk you through a proof of concept I whipped up last week. Starting from an OSX computer with Homebrew and VirtualBox installed, we will be able to spin up a Docker-based environment that will load-balance HTTP traffic via Nginx to an arbitrary number of backend processes, all running in separate Docker containers.

Homelab: SmartOS Lenovo TS140 build

Today I received the final component to complete my first homelab-specific PC build.

Docker-Machine Lazy Loading

Like many, I use docker-machine (installed via Docker Toolbox) to develop Docker images on OSX. I am terrible about remembering to launch the docker-machine VM ahead of time, and I don’t like having STDERR output or delays from launching new shell tabs, so I wrote a quick lazy-load function inspired by Spacemacs.

Running a Phoenix app via Docker-Compose