Traefik is an open-source router and load-balancer that sits in front of your web services. You can set it up to automatically encrypt your websites with SSL certificates. It’s also easy to add new web services to an existing Traefik cluster.

Traefik image image from the official Traefik website

I discovered Traefik via Jakub Svehla’s post Building a Heroku-like infrastructure for $5 a month. He shows you how to use Docker to install a Traefik infrastructure on a cheap VPS like DigitalOcean. With his setup, you can easily deploy new projects to your VPS on your domain.

Jakub uses plain Docker with docker-compose files. While that’s a feasible option, I dislike using the docker-compose command for production, as it’s a tool for local development.

You can use docker-compose files in production, but only with using docker stack and Docker swarm clusters. Further reading: The Difference Between Docker Compose And Docker Stack.

Using Docker in swarm mode adds a new layer of complexity.

In this post, I will show you a working example of Traefik 2 in Docker swarm mode with a docker socket proxy, running the Traefik dashboard on a different port, basic auth and IP whitelisting.

This post is not for beginners, but for those who got a basic setup working, but can’t figure out how to tie all pieces together.

Getting Started

Here are the articles that you will need for basic setup and understanding of Traefik 2 and Docker Swarm:

Why Docker Socket?

Security concerns, see: Protecting Your Docker Socket With Traefik 2.

Why Is The Dashboard on Port 8000?

Security concerns. My setup uses port 8000 for the dashboard and IP whitelisting. Only my public IP can reach the dashboard, and only on port 8000. That already minimizes the risk of exposure.

If you try to reach the dashboard from a different IP, you can’t even reach the login middleware(403: Forbidden).

The dashboard also has basic auth.

Additionally, there is a rate limit in place.

Working Example

Docker Swarm Init

Follow the setup guide on dockerswarm.rocks to create a Docker swarm cluster.

Create Networks

I use 3 networks:

  1. cloud-edge (for Traefik 2)
  2. cloud-public (for all web services that I want to expose to the internet)
  3. cloud-socket-proxy (for the docker socket proxy)

Remove default ingress network and re-create it with encryption:

docker network create --ingress --driver overlay \
   --opt encrypted --subnet 10.10.0.0./16 ingress

Add the two other networks as overlay networks:

# host network for outside of docker
docker network create --subnet 10.11.0.0/16 --driver overlay \
  --scope swarm --opt encrypted --attachable cloud-edge
# network hosting the socket proxy
docker network create --subnet 10.12.0.0/16 --driver overlay \
  --scope swarm --opt encrypted --attachable cloud-socket-proxy
# network hosting the services that are routed by traefik
docker network create --subnet 10.13.0.0/16 --driver overlay \
  --scope swarm --opt encrypted --attachable cloud-public

Add Node Labels And Environment Variables

The following steps are from the superb guide on dockerswarm.rocks. We create a node label to make sure that Traefik will deploy on the same node that has the volume for the SSL certificates.

export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add cloud-public.traefik-certificates=true $NODE_ID

You also have to create environment variables for DOMAIN, USERNAME, EMAIL, PASSWORD and HASHED_PASSWORD.

TLS

Create a folder traefik_conf with a file called dynamic_conf.toml. This is needed to specify the minimal TLS version for your setup. You need to bind mount the file.

More info here.

The TLS setup is part of Traefik’s dynamic configuration.

[tls.options]
  [tls.options.default]
    minVersion = "VersionTLS13"
    sniStrict = true

  [tls.options.tls12]
    minVersion = "VersionTLS12"
    sinStrict = true
    cipherSuites = [
      "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
      "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
      "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
      "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
      "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305",
      "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305"
    ]

Docker Compose

Here is the final docker-compose.yml:

version: '3.8'

# volume for the SSL certificates from Let's Encrypt
volumes:
  traefik-certificates:

networks:
  cloud-edge:
    external: true
  cloud-public:
    external: true
  cloud-socket-proxy:
    external: true

services:
  reverse-proxy:
    image: traefik:v2.2
    command:
      - --providers.docker
      # Use the secure docker socket proxy
      - --providers.docker.endpoint=tcp://socket-proxy:2375
      # Add a constraint to only use services with the label "traefik.constraint-label=cloud-public"
      - --providers.docker.constraints=Label(`traefik.constraint-label`, `cloud-public`)
      # Don't expose containers per default
      - --providers.docker.exposedByDefault=false
      - --providers.docker.swarmMode=true
      # fileprovider needed for TLS configuration
      # see https://github.com/containous/traefik/issues/5507
      - --providers.file.filename=traefik_conf/dynamic_conf.toml
      # Entrypoints (ports) for the routers
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      # Entrypoint for the dashboard on port 8000
      - --entrypoints.api.address=:8000
      # Create the certificate resolver "letsencrypt" for Let's Encrypt, uses the environment variable EMAIL
      - --certificatesresolvers.letsencrypt.acme.email=${EMAIL?Variable not set}
      - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
      - --certificatesresolvers.letsencrypt.acme.tlschallenge=true
      # Only for development to avoid hitting the rate limit on certificates
      - --certificatesresolvers.letsencrypt.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory
      # Logging
      - --accesslog
      - --log.level=debug
      # Enable the dashboard
      - --api
    deploy:
      restart_policy:
        condition: on-failure
      placement:
        constraints:
          # Use the custom label "traefik.constraint-label=traefik-public"
          # This public Traefik will only use services with this label
          # That way you can add other internal Traefik instances per stack if needed
          - node.labels.cloud-public.traefik-certificates == true
          - node.role == manager
      labels:
        # traefik.enable is required because we don't expose all containers automatically
        - traefik.enable=true
        - traefik.docker.network=cloud-public
        - traefik.constraint-label=cloud-public

        # Global redirection: HTTP to HTTPS
        - traefik.http.routers.http-redirects.entrypoints=web
        - traefik.http.routers.http-redirects.rule=hostregexp(`{host:(www\.)?.+}`)
        - traefik.http.routers.http-redirects.middlewares=traefik-ratelimit,redirect-to-non-www-https

        # Global redirection: HTTPS www to HTTPS non-www
        - traefik.http.routers.www-redirects.entrypoints=websecure
        - traefik.http.routers.www-redirects.rule=hostregexp(`{host:(www\.).+}`)
        - traefik.http.routers.www-redirects.tls=true
        - traefik.http.routers.www-redirects.tls.options=default
        - traefik.http.routers.www-redirects.middlewares=traefik-ratelimit,redirect-to-non-www-https

        # Middleware to redirect to bare https
        - traefik.http.middlewares.redirect-to-non-www-https.redirectregex.regex=^https?://(?:www\.)?(.+)
        - traefik.http.middlewares.redirect-to-non-www-https.redirectregex.replacement=https://$${1}
        - traefik.http.middlewares.redirect-to-non-www-https.redirectregex.permanent=true

        # Dashboard on port 8000
        - traefik.http.routers.api.entrypoints=api
        - traefik.http.routers.api.rule=Host(`${DOMAIN?Variable not set}`)
        - traefik.http.routers.api.service=api@internal
        - traefik.http.routers.api.tls=true
        - traefik.http.routers.api.tls.options=default
        - traefik.http.routers.api.tls.certresolver=letsencrypt
        # middlewares: use IP whitelisting, ratelimit and basic authentication
        - traefik.http.routers.api.middlewares=api-ipwhitelist,traefik-ratelimit,api-auth
        - traefik.http.middlewares.api-auth.basicauth.users=${USERNAME?Variable not set}:${HASHED_PASSWORD?Variable not set}
        # whitelist your public ip, see https://icanhazip.com
        # replace with _your IP_
        - traefik.http.middlewares.api-ipwhitelist.ipwhitelist.sourcerange=192.168.178.2/32
        - traefik.http.services.api.loadbalancer.server.port=8000

        # Extra middleware (ratelimit, ip whitelisting)
        - traefik.http.middlewares.traefik-ratelimit.ratelimit.average=100
        - traefik.http.middlewares.traefik-ratelimit.ratelimit.burst=50
    # use host mode for network ports for ip whitelisting
    # see https://community.containo.us/t/whitelist-swarm-cant-get-real-source-ip/3897
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
      - target: 8000
        published: 8000
        protocol: tcp
        mode: host
    volumes:
      - traefik-certificates:/letsencrypt
      # bind mount the directory for your traefik configuration
      - /home/$USER/traefik_conf:/traefik_conf
    networks:
      - cloud-edge
      - cloud-public
      - cloud-socket-proxy

  socket-proxy:
    image: tecnativa/docker-socket-proxy:latest
    deploy:
      restart_policy:
        condition: on-failure
      placement:
        constraints: [node.role == manager]
    environment:
      # permssions needed
      NETWORKS: 1
      SERVICES: 1
      TASKS: 1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      cloud-socket-proxy:
        aliases:
          - socket-proxy

Please note that the labels entries are under the entry deployment. This structure is needed for docker-compose.

Deploy on your VPS with docker stack deploy -c <name-of-your-swarm> (for example: traefik).

Let’s say your domain is my.traefik.com. You can reach your dashboard at https://my.traefik.com:8000/dashboard/ (the trailing slash is important).

For deployment, remove the following line:

- -- certificatesresolvers.letsencrypt.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory`

It is only needed for development to avoid hitting the limit on Let’s Encrypt.
You’ll probably have to remove the bound volume as well. Docker will “cache” the development certificates.

I suggest accessing your server (via SSH) and run:

docker volume prune

Deploy New Services

Let’s say you want to deploy a new container on the url whoami.traefik.com.

Use your web provider to add a new A record. Here’s a guide for DigitalOcean.

Add a new docker compose file.

Example whoami.yml:

version: '3.8'

services:
  whoami:
    image: containous/whoami:latest
    command:
      - --port=8082
    deploy:
      labels:
        # traefik.enable=true and constraint label are needed because
        # of the restrictions we enforced in our traefik configuration
        - traefik.enable=true
        - traefik.constraint-label=cloud-public
        - traefik.http.routers.whoami.entrypoints=websecure
        - traefik.http.routers.whoami.rule=Host(`whoami.traefik.com`)
        - traefik.http.routers.whoami.tls=true
        # min TLS version
        - traefik.http.routers.whoami.tls.options=tls12@file
        - traefik.http.routers.whoami.tls.certresolver=letsencrypt
        - traefik.http.routers.whoami.middlewares=traefik-ratelimit
        - traefik.http.services.whoami.loadbalancer.server.port=8082
    networks:
      - cloud-public

networks:
  cloud-public:
    external: true

Deploy:

docker stack deploy -c whoami.yaml <name-of-your-swarm>

Docker stack will add the new service to the existing stack and will re-use the configuration from your main traefik installation.

Final Thoughts

Traefik 2 Setup is very complicated. The web is full of examples where people can’t figure out how to configure it.

It took me too many hours to count to create a working setup that goes beyond the bare minimum.

There are a lot of moving parts: Docker, Docker swarm mode, networking, volume binding, static and dynamic configuration of Traefik 2, middlewares, routers, etc.

At first, I was excited by Traefik, but the complexity is mind-boggling. Be prepared for a lot of frustration.

It’s a cool piece of tech, but you will need a lot of time to understand how to configure it.

Beware.