X
Popular Searches

How to Simplify Docker Compose Files With YAML Anchors and Extensions

Docker Compose lets you manage multiple Docker containers and their associated resources such as volumes and networks. You write declarative YAML files which Compose uses to create your container stack.

Your docker-compose.yml files can become repetitive when you’re working with a complex stack. Services might share configuration options, causing you to duplicate sections of your file. Later updates can lead to mistakes if you forget to update every instance of a section.

Because Compose files are plain YAML files, you can take advantage of built-in YAML features to modularise your stack definitions. Anchors, aliases and extensions let you abstract YAML sections into reusable blocks. You can add a reference to the section in each place it’s needed.

What Is An Anchor?

YAML anchors are a feature which let you identify an item and then reference it elsewhere in your file. Anchors are created using the & sign. The sign is followed by an alias name. You can use this alias later to reference the value following the anchor.

Here’s how you could use an anchor to avoid repeating container restart policies:

services:
  httpd:
    image: httpd:latest
    restart: &restartpolicy unless-stopped
  mysql:
    image: mysql:latest
    restart: *restartpolicy

The anchor is referenced using the * character and its alias. You must ensure there’s no space between the &/* characters and the following alias name.

This example shows how a single-line value can be reused with anchors. Changing the stack restart policy can now be done in one place, without editing the services individually.

Multi-Line Anchors

Anchors can have multi-line values. You create them using the same syntax as a single-line anchor. This is useful when you need to provide a set of configuration details to multiple services.

services:
  first:
    image: my-image:latest
    environment: &env
      - CONFIG_KEY
      - EXAMPLE_KEY
      - DEMO_VAR
  second:
    image: another-image:latest
    environment: *env

The second service will now pull in the same environment variables as first. We haven’t had to repeat the list of environment variables, making it much more maintainable in the future.

Extending Anchor Values

The environment example above takes the anchor’s value and uses it as-is. You’ll often want to extend the anchor to add additional values. You can do this with an alternative syntax.

Modify the second service as follows:

services:
  second:
    image: another-image:latest
    environment:
      <<: *env
      - AN_EXTRA_KEY
      - SECOND_SPECIFIC_KEY

The service now pulls in the base environment configuration from the env anchor. Additional keys are then added to the environment list. You may also override existing keys defined by the anchor.

Using Extension Fields

Another approach to modularisation is extension fields. These are special top-level YAML fragments which will be ignored by Docker.

Docker usually tries to interpret any node at the root of a Compose file. The parser will ignore extension fields prefixed with x-. You can use these fields to encapsulate shared configuration for later reference. Combine extension fields with anchors to abstract sections out of your service definitions.

x-env: &env
  environment:
    - CONFIG_KEY
    - EXAMPLE_KEY
 
services:
  first:
    <<: *env
    image: my-image:latest
  second:
    <<: *env
    image: another-image:latest

This Compose file is a further refinement over the example shown above. The environment variables no longer belong to either of the services. They’ve been lifted out completely, into the x-env extension field.

This defines a new node which contains the environment field. A YAML anchor is used (&env) so both services can reference the extension field’s value.

Composability

Making use of these features lets you break your Compose files into self-contained chunks. This helps you avoid overly repetitive service definitions. Anything common to more than one service should be lifted into an extension field.

Besides aiding maintainability, this practice communicates your intentions to other collaborators. It’s clear that any top-level extension fields contain generic fields. They’re not tied to any particular service and can be freely reused.

Anchors and extension fields let you compose your service definitions out of reusable blocks of YAML. By keeping each field small, your services can mix and match configuration sections from the available anchors. Maintaining your Compose files should become less of a chore.

Other Approaches to Modularity

Besides anchors and extensions, don’t forget you can always split your Compose definitions into multiple Compose files. This may become necessary when you have more than a handful of individual services.

Using multiple Compose files lets you allocate each service its own file. You can also create override files, where a node’s values get replaced or extended. Compose will merge all the files together to create the final runtime configuration.

service.yml

services:
  service:
    image: my-image:latest

service-dev.yml

services:
  service:
    environment:
      - DEV_MODE=true

In this example, applying both Compose files would result in one service, my-image:latest, with the DEV_MODE environment variable set. To use multiple files with the Compose CLI, pass the -f flag:

docker-compose -f service.yml -f service-dev.yml up -d

Files are merged in the order specified.

Summary

Docker Compose files can become unwieldy and repetitive. If you’re spending time copying values about, consider abstracting sections of your services into dedicated YAML blocks.

Features such as anchors and extensions aid maintainability and make for an easier authoring experience. Not every Compose file will benefit – some services may have little in common with each other – so assess your specific stack before you start.

James Walker James Walker
James Walker is a contributor to CloudSavvy IT. He is the founder of Heron Web, a UK-based digital agency providing bespoke software development services to SMEs. He has experience of managing complete end-to-end web development workflows, using technologies including Linux, GitLab, Docker and Kubernetes. Read Full Bio »

The above article may contain affiliate links, which help support CloudSavvy IT.