Split docker compose files

2023-09-29 · 515 words · 3 minute read

All my services that I selfhost are docker containers which I manage using docker compose . Until recently I had all of them in one big docker-compose.yaml file which started to be a hassle to manage.

For a while I looked for ways to split the file into multiple files but nothing really statisfied me.

I even mad an attempt to to have a bash script that makes use of the -f parameter to merge multiple files which kind of worked but had some strange side effects such as containers were not added to networks every now an then and depends_on didn’t work.

As of docker compose version 2.20 it supports the include function which is exactly what I was looking for 🤩

All my container config lies in /opt/docker/, this is the structure I use now:

 1.
 2├── docker-compose.yaml
 3├── filesharing
 4│   ├── filesharing.yaml
 5│   ├── syncthing
 6│   └── webdav
 7├── internal
 8│   ├── homer
 9│   └── internal.yaml
10├── network
11│   ├── blocky
12│   ├── diun
13│   ├── network.yaml
14│   ├── omada
15│   └── wg-easy
16├── public
17│   ├── photoview
18│   ├── public.yaml
19│   ├── remark42
20│   ├── shiori
21│   ├── vaultwarden
22│   └── vikunja
23├── smarthome
24│   ├── homeassistant
25│   ├── influxdb
26│   ├── mosquitto
27│   ├── postgresql
28│   └── smarthome.yaml
29└── webserver
30    ├── caddy
31    └── www

The main docker-compose.yaml is placed directly in /opt/docker. Caddy is my favourit webserver and the corner stone of my setup. It acts as my reverse proxy so I decided to place it as the “main” container in that file:

 1version: "3"
 2
 3include:
 4  - /opt/docker/filesharing/filesharing.yaml
 5  - /opt/docker/internal/internal.yaml
 6  - /opt/docker/network/network.yaml
 7  - /opt/docker/public/public.yaml
 8  - /opt/docker/smarthome/smarthome.yaml
 9
10services:
11  caddy:
12    container_name: caddy
13    image: custom-caddy
14    build: /opt/docker/webserver/caddy
15    env_file:
16      - /opt/docker/webserver/caddy/caddy.env
17    volumes:
18      - /opt/docker/webserver/caddy/Caddyfile:/etc/caddy/Caddyfile
19      - /opt/docker/webserver/caddy/data:/data
20      - /opt/docker/webserver/caddy/config:/config
21      - /opt/docker/webserver/www:/www
22    restart: unless-stopped
23    ports:
24      - 80:80
25      - 443:443
26    networks:
27      - webserver
28
29networks:
30  webserver:
31    name: webserver
32    driver: bridge

You can see how I included the other yaml files, filesharing.yaml for example looks like this:

 1services:
 2  syncthing:
 3    image: linuxserver/syncthing
 4    container_name: syncthing
 5    env_file:
 6      - /opt/docker/filesharing/syncthing/syncthing.env
 7    volumes:
 8      - /opt/docker/filesharing/syncthing:/config
 9      - /storage/syncthing:/data1
10    restart: unless-stopped
11    ports:
12      - 8384:8384
13      - 22000:22000/tcp
14      - 22000:22000/udp
15      - 21027:21027/udp
16    networks:
17      - webserver
18    depends_on:
19      - caddy
20
21  webdav:
22    container_name: webdav
23    image: hacdias/webdav:latest
24    user: 1000:1000
25    restart: unless-stopped
26    volumes:
27      - /storage/webdav:/data
28      - /opt/docker/filesharing/webdav/config.yaml:/config.yaml
29    command: --config /config.yaml
30    networks:
31      - webserver
32    depends_on:
33      - caddy
34
35networks:
36  filesharing:
37    name: filesharing

The main benefit is that depends_on: caddy works as expected as well as networks: webserver.

In order to manage my containers I can now cd into /opt/docker and use the normal docker compose commands, in my case I use the oh-my-zsh shortcuts like dcup -d and so on ( complete list ).

I’m really happy with how this works and will stick with it for the forseable future 😎