Like most industries computer programming has cycles. One of these cycles is how we write or compile programs. We fundamentally have two methods: static vs. dynamic. Neither is perfect. Because each has benefits and drawbacks the wheel is slowly turning between which is perceived as “better” or is more common. There are changes afoot right now which to me signal the wheel is turning away from the current preference and moving toward static.

Connected Wheels

Before going into the static vs. dynamic wheel, there is a related wheel I want to mention. Big framework or OS install vs trim and minimal. In a sense these two wheels are closely related. A static binary is easy to deploy as it has minimal (if any) dependencies. A simple Python script using only the included batteries, for example, is also easier to deploy and has minimal dependencies. Compare this to something like Django, Rails, etc.. There is no question more is required operationally for the larger setups. Whether we get enough out of the added complexity is the question we have to ask for our specific circumstance. Right now the wheel has the more complex and larger setups as the incumbent. However, like dynamic compiling I think the wheel is turning - and for the very same reasons.

First, the resurgence of containerization. Docker is unquestionably the main driver in this resurgence. While the concept, and indeed much of the tech, isn’t brand new the (re)discovery of it’s benefits and the applicability of those benefits is driving the industry toward containerization.

Initially this containerization will keep the complexity of operational demands such as “you will install this to a large Ubuntu installation with SSH, Syslog, your services, etc.”. This happens because it is one small step, and it fits with our current dominant notion of the computer as the unit. The next steps however, diverge slightly but point in the same direction.

The next logical step for many is to split their application into services - thus the “Micro-Services” push. But these services are still being deployed in heavyweight images. For example consider Redis. Redis is a simple service which requires little more than the OS. For running in a Docker image all you need is something as small as Busybox. Yet the image most people base from, going by the Docker Hub Dockerfiles, is Ubuntu.

Nearly all of Redis can be managed by talking to it over it’s native port and protocol. Nearly all of the information you need from it can be obtained the same way. No need for SSH, no need for LDAP, no need for syslog even. So why do we build it on Ubuntu? The simple answer is our comfort zone is being able to SSH in and “do stuff”. This combined with the mentality of the machine as the fundamental operations unit keeps us in this zone. This, however, is about to change.

As more of us understand the underlying thought process powering Docker we realize we want trim images with only that which is actually required. As we break up complex monolithic services into independent but interrelated services we see the advantage of not deploying a 1-3 GB OS image to run a service which itself comes in at a few MB.

Yet there is another development afoot which will push the wheels to turn faster, bringing the cycle back around to small systems and even static compiling.

Static Vs. Dynamic - the song that never ends.

This development is called “Go” - yes the Go programming language. I make no attempt to hide the fact that Go is the first programming language I’ve truly enjoyed and been inspired by since Python a couple decades ago. But the reasons I like it include the compiling factors. It builds fast, and builds a “mostly static” binary which can be “fully static” with a few options and judicious choices of library usage.

Left to it’s own devices, as it were, Go’s relative simplicity in deployments is a much needed and welcome improvement in the world of operations. It is a great thing, especially in large environments, for development to hand over a simple binary or collection of binaries to Operations which doesn’t come with “and install these 22 libraries and 7 system services, then make them all running correctly and working together on the same machine”. But I think most of us have not yet seen the combination of Go and Docker and the change it can bring to the industry - and the implements in system robustness, flexibility, and the ability of Operations to sleep through the night.

With Go you can build a Docker image which requires no in-image OS. In Dockerfile-speak you base from “scratch” - an empty image. Your image is essentially a packaging of the binary your Go code produced. It is, perhaps, as close to the ideal containerization of an application as we’ve yet achieved.

Take, for example, Commissar’s Latency tool - a Redis latency checker. It is built in Go and the resulting scratch-based Docker image is around 3MB. Place it in your repo, or use the Hub, and operationally deployment is a docker pull followed by a docker run. And you can do this remotely. That, from an operations perspective, is brilliant. Further, the simplicity can pervade further into the operations stack. How much do you really need something like Chef or Puppet if your deployments are Docker pull + Docker run? If you don’t have a complex chain of dependencies, do you need a complex OS manager?

At the host level we have tools such as run levels, systemd, upstart, init scripts, etc. and these are all trying to do one thing at the host level: orchestration. Ultimately, this is what tools such as Puppet do - they set up service orchestration at the host level. Once you remove the host level you’re orchestrating your services directly. Things get much simpler at that point.

Now, we don’t have to have Go for this. You can certainly build static binaries in C and C++ for example. But you need a good reason to go these routes. I maintain the benefits of thinking in the single-service model and even the no-OS-Image model provide these benefits. More time on code combined with less complexity for ops is a recipe for better service and better code. With static compilation and minimal-to-zero OS images showing such tremendous benefits and advantages I suspect we’re looking at the balance shifting in that direction - the wheel is turning and picking up speed.

But there is yet one more development I want to highlight. Self-contained images.

Docker w/o Docker

A self-contained image is a binary you execute on the host system which has the image and needed containerization bits built into it. It’s Docker w/o Docker. You achieve this through use of libcontainer - the default container engine for Docker. You write, in Go, an app which embeds the image and contains all which is necessary to launching a container. And it builds down into a single binary. With this you don’t even need Docker installed on the host. For an example of this you can check out “sc-redis” on Github.

This repo builds Redis in a tiny image, though Redis not statically compiled, and results in a binary that runs in a container an ephemeral Redis - one that goes completely away when you close out the command. This binary requires no dependencies beyond the ability to execute it on the host. It’s dependencies are kernel and iptables level items. If you can deploy your services like this, it means operations spends it’s brainpower on making the interconnectivity and monitoring/recovery routines better rather than building systems and chasing dependencies.

For those who pick up on pairing and synergy of these developments the wheel will turn faster and their wheels will turn faster in turn. Does it mean you need to drop your complex web of Ruby, Python, or Java recipes and switch? Probably not, or at least not right away. But it does mean that if you can migrate components to this type of setup you’ll reap some excellent rewards and make the Ops half of “DevOps” much simpler and happier. It does mean the automatic dismissal of static compilation and default of “Giant Docker Images” (GDI) should be shifting.

For decades the power of Unix has been in it’s core principles such as “do one thing and do it well” and connecting tools - such as pipelines. Bringing this up a level in the stack to applications and applying it to services and applications will let us take the lessons learned and the power Unix has enjoyed deeper into our “stack”.

And somewhere, those who’ve been silently using small images and never switched fully to big installs and complex dependencies will smile, nod, and possibly refer to this “new hotness” as perhaps a bit hipster-ish.

And the wheel will continue to turn.