Docker Compose, Postgres and Travis CI

07/28/20191 Min Read — In DevOps, Docker, Python

Today I tried to set up a dockerized Flask app with Postgres database with Travis CI.

It shouldn't have been difficult because I followed a tutorial.

Setup

Here is the docker-compose.yml:

version: '3.7'
services:
users:
build:
context: ./services/users
dockerfile: Dockerfile
volumes:
- './services/users:/usr/src/app'
ports:
- 5001:5000
environment:
- FLASK_ENV=development
- APP_SETTINGS=project.config.DevelopmentConfig
- DATABASE_URL=postgres://postgres:postgres@users-db:5432/users_dev
- DATABASE_TEST_URL=postgres://postgres:postgres@users-db:5432/users_test
depends_on:
- users-db
users-db:
build:
context: ./services/users/project/db
dockerfile: Dockerfile
ports:
- 5435:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
nginx:
build:
context: ./services/nginx
dockerfile: Dockerfile
restart: always
ports:
- 80:80
depends_on:
- users

There is a Python Flask app that runs as "users". It depends on a PostgresQL database called "users-db". The db maps the port 5435 to the standard postgres port 5432.

Here is the Dockerfile (/services/users/Dockerfile):

# base image
FROM python:3.7.2-alpine
# install dependencies
RUN apk update && \
apk add --virtual build-deps gcc python-dev musl-dev && \
apk add postgresql-dev && \
apk add netcat-openbsd
# set working directory
WORKDIR /usr/src/app
# add and install requirements
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
# add entrypoint.sh
COPY ./entrypoint.sh /usr/src/app/entrypoint.sh
RUN chmod +x /usr/src/app/entrypoint.sh
# add app
COPY . /usr/src/app
# run server
CMD ["/usr/src/app/entrypoint.sh"]

The script for the entrypoint makes sure that the database is ready:

#!/bin/sh
echo "Waiting for postgres..."
while ! nc -z users-db 5432; do
sleep 0.1
done
echo "PostgreSQL started"

For docker-compose 2 you could add a "healthcheck" condition to your configuration. This condition ensured that the container waited until Postgres was ready.
Read more about it at docker-compose-healthcheck.

The condition flag doesn't work for docker-compose 3.

Now, you have to create shell scripts to wait for the database connection. See "Can this work with v3 of docker compose?" for details.

Travis-CI

Running this build on Travis gave the following error:

could not connect to server: Connection refused
Is the server running on host "users-db" (172.18.0.2) and accepting
TCP/IP connections on port 5432?

This is the configuration (.travis.yml):

sudo: required
services:
- docker
env:
DOCKER_COMPOSE_VERSION: 1.23.2
before_install:
- sudo rm /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
before_script:
- docker-compose up -d --build
script:
- docker-compose exec users python manage.py test
- docker-compose exec users black project
- docker-compose exec users flake8 project
after_script:
- docker-compose down

Add Postgres Image to docker-compose

I spent some time searching for a solution.

The only thing I came up with was to specify the image for the Postgres database:

.docker-compose.yml:

users-db:
+ image: postgres:10-alpine
build:
context: ./services/users/project/db
dockerfile: Dockerfile
ports:
- 5435:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres

Update: Looks like this doesn't solve the problem. Sometimes the builds of Travis-CI fail because the database isn't responsive.
If I trigger a new build manually inside the Travis-CI-web-console, the test often work.