Docker Development

Installation

Follow instructions in https://docs.docker.com/get-docker/

Network Troubleshooting

using https://github.com/nicolaka/netshoot

docker run -it --network [network-name] nicolaka/netshoot

MySQL

I was seeing some password issues in production, so I’m saving a few links

CI/CD Workflow

also see

Compose repository

strconv.Atoi: parsing “”: invalid syntax

secrets management

docker recommends swarm mode

security

deploy compose application

From https://docs.docker.com/compose/production/, “You can use Compose to deploy an app to a remote Docker host by setting the DOCKER_HOST, DOCKER_TLS_VERIFY, and DOCKER_CERT_PATH environment variables appropriately. See also Compose CLI environment variables.”

Ref: https://www.docker.com/blog/how-to-deploy-on-remote-docker-hosts-with-docker-compose/

Reference code at https://github.com/louking/webmodules/tree/docker-skeleton-v1.1

docker context create <contextname> --docker "host=ssh://<user>@<remotemachine>"

Make sure identify file is in ssh_config (C:\Users\<username>\\.ssh\config) (https://forums.docker.com/t/docker-context-problem/95105/2)

Host <remotemachine>
    User <user>
    HostName <remotemachine>
    IdentityFile <sshkey file>

On target machine give user permissions to access docker without using sudo (https://phoenixnap.com/kb/cannot-connect-to-the-docker-daemon-error method 4)

sudo usermod -aG docker <user>

Using context on the local machine, build and bring up the compose app on the remote machine

docker --context <contextname> compose -f docker-compose.yml -f docker-compose.prod.yml up --build -d

When there is a new release, the application must be brought down before building and bringing it up

docker --context <contextname> compose -f docker-compose.yml -f docker-compose.prod.yml down

or

docker --context <contextname> compose down

Debugging a Docker Service Locally

Note

This has been made to work in https://github.com/louking/tm-csv-connector. If there are any problems below, please use that repo as an example.

Some Required Files

These allow the database to be upgraded before running the app.

app/src/app-initdb.d/create-database.sh (see create-database.sh)

app/dbupgrade_and_run.sh

#!/bin/sh

# NOTE: file end of line characters must be LF, not CRLF (see https://stackoverflow.com/a/58220487/799921)

# create database if necessary
while ! ./app-initdb.d/create-database.sh
do
    sleep 5
done

# initial volume create may cause flask db upgrade to fail
while ! flask db upgrade
do
    sleep 5
done
exec "$@"

web/nginx-longtimeout.conf

# see https://serverfault.com/a/777753

fastcgi_read_timeout 3600;
proxy_read_timeout 3600;

Launch Debugger from vscode

In order for vscode to access the service, a Docker compose file similar to the following should be used. This configures nginx to have a long timeout to allow time spent with the service halted, starts the service in the debugger, and exposes the port 5678 which matches the debugger –listen port. This allows the service to be initialized by docker compose without being started.

docker-compose.debug.yml

services:
  web:
    volumes:
        - ./web/nginx-longtimeout.conf:/etc/nginx/conf.d/nginx-longtimeout.conf
  app:
    ports:
    - 5000:5000
    - 5678:5678
    environment:
    - FLASK_APP=/app/app.py
    volumes:
    - ./app/src:/app
    command: ["./dbupgrade_and_run.sh", "sh", "-c", "pip install debugpy -t /tmp && python /tmp/debugpy --wait-for-client --listen 0.0.0.0:5678 -m flask run --no-debugger --no-reload --host 0.0.0.0 --port 5000"]

The application is started with docker compose, including these files, as appropriate. E.g.,

docker compose -f docker-compose.yml -f docker-compose.debug.yml up --build -d

or alternately in tasks.json run task docker-compose: debug

"tasks": [
    {
        "label": "build app",
        "type": "shell",
        "dependsOn": [
            "build docs"
        ],
        "command": "docker compose -f docker-compose.yml build",
        "problemMatcher": []
    },
    {
        "type": "docker-compose",
        "label": "docker-compose: debug",
        "dependsOn": [
            "build app"
        ],
        "dockerCompose": {
            "up": {
              "detached": true,
              "build": false,
            },
            "files": [
              "${workspaceFolder}/docker-compose.yml",
              "${workspaceFolder}/docker-compose.debug.yml"
            ]
      },
    },
]

(in tasks.json, for completeness)

"tasks": [
    {
        "type": "docker-compose",
        "label": "docker-compose: up",
        "dependsOn": [
            "build app"
        ],
        "dockerCompose": {
            "up": {
              "detached": true,
              "build": false,
            },
            "files": [
              "${workspaceFolder}/docker-compose.yml",
            ]
      },
    },
    {
        "type": "docker-compose",
        "label": "docker-compose: down",
        // "dependsOn": [
        //     "build app"
        // ],
        "dockerCompose": {
            "down": {
            //   "services": ["app"]
            },
            "files": [
              "${workspaceFolder}/docker-compose.yml",
              "${workspaceFolder}/docker-compose.debug.yml"
            ]
      },
    },
]

Assuming breakpoints are desired and docker-compose.debug.yml is used, vscode needs to launch accordingly. The following must be added to vscode’s launch.json. Note the port number 5678 matches that which was used within docker-compose.debug.yml. Also note that the remoteRoot value must match the python version which was used within the service.

launch.json

{
    "configurations": [
        // see https://code.visualstudio.com/docs/containers/docker-compose#_python
        {
            "name": "Python: Remote Attach",
            "type": "python",
            "request": "attach",
            "host": "localhost",
            "port": 5678,
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}/app/src",
                    "remoteRoot": "/app"
                },
                // allow debugging of pip installed packages
                {
                    "localRoot": "${workspaceFolder}/.venv/Lib/site-packages",
                    "remoteRoot": "/usr/local/lib/python3.10/site-packages"
                }
            ],
            "justMyCode": false
        },
    ]
}

Debug pypi Package Stored Locally

If a separate package that is normally loaded via pypi is in the workspace, but is being developed along with the main package (e.g., loutilities), an additional docker compose file and launch.json configuration is required.

docker-compose.loutilities.yml

# use editable loutilities
services:
  app:
    build:
      args:
        - PYTHON_LIB_VER=${PYTHON_LIB_VER}
    volumes:
      - ..\..\loutilities\loutilities\loutilities:/usr/local/lib/python${PYTHON_LIB_VER}/site-packages/loutilities

launch.json

{
    "configurations":  [
        // see https://code.visualstudio.com/docs/containers/docker-compose#_python
        {
            "name": "Python: Remote Attach (loutilities)",
            "type": "python",
            "request": "attach",
            "port": 5678,
            "host": "localhost",
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}/app/src",
                    "remoteRoot": "/app"
                },
                // allow debugging of pip installed packages
                {
                    "localRoot": "${workspaceFolder}/.venv/Lib/site-packages",
                    "remoteRoot": "/usr/local/lib/python3.10/site-packages"
                },
                // see https://code.visualstudio.com/docs/editor/variables-reference#_variables-scoped-per-workspace-folder
                {
                    "localRoot": "${workspaceFolder:loutilities}/loutilities/",
                    "remoteRoot": "/usr/local/lib/python3.10/site-packages/loutilities/"
                },

            ],
            "justMyCode": false
        },
    ]
}

and the application is started with docker compose including these files, as appropriate. E.g.,

docker compose -f docker-compose.yml -f docker-compose.loutilities.yml -f docker-compose.debug.yml up --build -d

Restart Service

To restart the service, e.g., after changes have been made in initialization code, use docker compose <files> restart or docker compose <files> stop, docker compose <files> start. Also see https://www.shellhacks.com/docker-compose-start-stop-restart-build-single-service/

Apache Configuration

The server has apache running natively. We’ll proxy via apache, then take care of routing within the docker compose application with an nginx container.

Apache setup example for production host:

# www.<vhost>

<VirtualHost *:80>
    ServerName www.<vhost>
    ServerAlias <vhost>
    # comment out when creating certificate
    Redirect permanent / https://www.<vhost>/
</VirtualHost>


<VirtualHost *:443>
    ServerAdmin lking@pobox.com
    ServerName www.<vhost>
    ServerAlias <vhost>

    # comment out when creating certificate
    SSLProxyEngine on
    SSLCertificateFile /etc/letsencrypt/live/www.<vhost>/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/www.<vhost>/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateChainFile /etc/letsencrypt/live/www.<vhost>/chain.pem

    ProxyPreserveHost on
    ProxyPass / http://localhost:<port>/
    ProxyPassReverse / http://localhost:<port>/
    RequestHeader set X-Forwarded-Port 443
    RequestHeader set X-Forwarded-Scheme https

</VirtualHost>

flask db migrations

Execute flask db migrations in the development shell container

# initialize migrations environment
docker exec webmodules-shell-1 flask db init --multidb

# create migration
docker exec webmodules-shell-1 flask db migrate -m "migration comment"

To add additional database binds to single database, follow https://github.com/miguelgrinberg/Flask-Migrate/issues/179#issuecomment-355344826

Initial Deploy of Docker Web App to Server

Create apache configuration (e.g., /etc/httpd/sites-available/www.<vhost>.conf), via Apache Configuration

Set up user account (once per server)

(appuser) vim .bashrc
    export HISTTIMEFORMAT="%Y-%m-%d %H:%M "

(appuser) mkdir .ssh
(appuser) chmod 700 .ssh
(appuser) touch .ssh/authorized_keys
(appuser) chmod 600 .ssh/authorized_keys

sudo usermod -aG docker appuser

Follow instructions to update DNS (Update DNS (optional)), and in Create VHOST enable VHOST, set up VHOST SSL

Create server base directory

sudo mkdir /var/www/www.<vhost>
sudo mkdir /var/www/www.<vhost>/logs
sudo a2ensite www.<vhost>
sudo apachectl configtest # test configuration created above
sudo apachectl restart

Create git environment

sudo mkdir -p /var/www/www.<vhost>/<appname>
cd /var/www/www.<vhost>/<appname>
sudo git clone https://github.com/louking/<appname>
cd /var/www/www.<vhost>
sudo chown -R appuser:appuser <appname>
sudo mkdir /var/www/www.<vhost>/applogs
sudo chown -R appuser:appuser applogs
sudo mkdir /var/www/www.<vhost>/<appname>/<appname>/config
sudo mkdir /var/www/www.<vhost>.loutilities.us/<appname>/<appname>/config/db
sudo chmod 700 /var/www/www.<vhost>.loutilities.us/<appname>/<appname>/config/db
# create config file(s)
sudo chown -R appuser:appuser /var/www/www.<vhost>/<appname>/<appname>/config
# create <appname>.cfg
# if needed, create users.cfg
# if needed, create /var/www/www.<vhost>/<appname>/<appname>/.env
# if needed, create password.txt file(s)

Build application

(appuser) docker compose -f docker-compose.yml -f docker-compose.<qualifier>.yml build

where:
    <qualifier> is one of prod, sandbox, dev

Start application

(appuser) docker compose -f docker-compose.yml -f docker-compose.<qualifier>.yml up -d

where:
    <qualifier> is one of prod, sandbox, dev

Push to docker hub

docker login
docker compose push