digiserver-v2/docs/07-deployment.md

# 07 · Deployment

DigiServer v2 runs as a **Docker Compose** stack (app + Caddy) with optional remote SSH provisioning of players.

---

## 1. Architecture

```mermaid
flowchart LR
    subgraph Host["Docker Host"]
        subgraph Net["digiserver-network"]
            APP["digiserver-app\nFlask + Gunicorn :5000"]
            CAD["caddy:2-alpine\n:80 / :443"]
        end
        VOL1["./data/instance → /app/instance\n(SQLite DBs)"]
        VOL2["./data/uploads → /app/app/static/uploads"]
        VOL3["./data/Caddyfile → /etc/caddy/Caddyfile"]
        VOL4["./data/caddy-data → /data\n./data/caddy-config → /config"]
    end
    Browser["Browser"] --> CAD
    CAD --> APP
    APP --> VOL1
    APP --> VOL2
    CAD --> VOL3
    CAD --> VOL4
    APP -.SSH/rsync.-> Players["Remote player hosts"]
```

---

## 2. `docker-compose.yml`

### Service `digiserver-app`
- **Build:** `.` (Dockerfile, `python:3.13-slim`)
- **Ports:** `5000:5000` (direct dev access; also `expose: 5000`)
- **Volumes:** `./data/instance:/app/instance`, `./data/uploads:/app/app/static/uploads`
- **Env:** `FLASK_ENV=production`, `SECRET_KEY`, `ADMIN_USERNAME`, `ADMIN_PASSWORD`
- **Healthcheck:** HTTP GET `http://localhost:5000/` (30s interval, 40s start)
- **Restart:** `unless-stopped`

### Service `caddy`
- **Image:** `caddy:2-alpine`
- **Ports:** `8080:80`, `8443:443`
- **Volumes:** `./data/Caddyfile:/etc/caddy/Caddyfile:rw`, `./data/caddy-data:/data`, `./data/caddy-config:/config`, `./data/caddy-logs:/var/log/caddy`
- **Depends on:** app (service started)
- **Healthcheck:** `wget` on port 80

---

## 3. Dockerfile

| Stage | Content |
|---|---|
| Base | `python:3.13-slim` |
| System deps | `poppler-utils`, `ffmpeg`, `libmagic1`, `sudo`, `fonts-noto-color-emoji`, `LibreOffice` (core/impress/writer), `sshpass`, `openssh-client`, `rsync`, `git` |
| Python | `COPY requirements.txt` → `pip install` (cached layer), then `COPY . .` |
| App config | `FLASK_APP=app.app:create_app`, `FLASK_ENV=production`, `EXPOSE 5000` |
| User | Non-root `appuser` (UID 1000) with **passwordless sudo** limited to `apt-get`, `install_libreoffice.sh`, `install_emoji_fonts.sh` |
| Runtime | `HEALTHCHECK` (HTTP 5000), `ENTRYPOINT /docker-entrypoint.sh` |

---

## 4. `docker-entrypoint.sh`

1. **Pin the database**: export `DATABASE_URL` (default `sqlite:////app/instance/dashboard.db`) so migrations and the app resolve to the *same* file. Required because the config classes default to different files (`dev.db` vs `dashboard.db`) and most migration scripts call `create_app()` without an argument (→ development config), while the app runs `create_app('production')`.
2. Create `/app/instance` and `/app/app/static/uploads`.
3. Ensure schema + admin user — runs on **every** start (idempotent): `db.create_all()`, then create the admin from `ADMIN_USERNAME` / `ADMIN_PASSWORD` or refresh its password.
4. Run the **migration chain** (idempotent, ordered — table-creating migrations run before those that alter them):

```
add_https_config_table.py
add_player_user_table.py
add_email_to_https_config.py
migrate_player_user_global.py
add_url_to_content.py
add_original_filename_to_content.py
add_deployment_fields_to_player.py
```

   Migrations are **non-fatal**: failures are logged as `⚠️ WARNING` and startup continues, so one bad migration can't strand the container in a restart loop.

5. Start **Gunicorn**: `--bind 0.0.0.0:5000 --workers 4 --timeout 120 app.app:create_app('production')`.

> Because migrations now run automatically on startup, `deploy.sh` step 4 is redundant (harmless — the scripts are idempotent).

### Data layout (bind mounts)

| Host path | Container path | Contents |
|---|---|---|
| `data/instance` | `/app/instance` | SQLite DB (`dashboard.db`), `player_build.json` |
| `data/uploads` | `/app/app/static/uploads` | Media files + `edited_media/` |
| `data/Caddyfile` | `/etc/caddy/Caddyfile` | Reverse-proxy config (**a file, not a directory**) |
| `data/caddy-data` | `/data` | Caddy state (instance UUID, certificates) |
| `data/caddy-config` | `/config` | Caddy autosave |
| `data/caddy-logs` | `/var/log/caddy` | Access logs |

> ⚠️ **`data/Caddyfile` must exist before `docker compose up`.** It is bind-mounted as a *file*;
> if it is missing, Docker creates a **directory** in its place and Caddy fails to start.
> `deploy.sh` seeds it from the version-controlled `Caddyfile.example`. For a manual start:
> ```
> mkdir -p data/instance data/uploads data/caddy-data data/caddy-config data/caddy-logs
> cp Caddyfile.example data/Caddyfile
> docker compose up -d --build
> ```

> ℹ️ The legacy `data/nginx-*` and `data/certbot` folders are **obsolete** — the reverse proxy is
> Caddy. They are no longer created by `deploy.sh`.

### Clean start (wipe all runtime data)

`data/` is **gitignored**, so wiping it is irreversible. To reset to a pristine deployment:

```
docker compose down
docker rmi digiserver-v2-digiserver-app:latest        # drop stale image
docker image prune -f && docker builder prune -a -f   # reclaim build cache
rm -rf data                                            # WIPES db, uploads, certs
mkdir -p data/instance data/uploads data/caddy-data data/caddy-config data/caddy-logs
cp Caddyfile.example data/Caddyfile
./deploy.sh
```

> ⚠️ Files under `data/caddy-*` are created **root-owned** by the Caddy container, so a plain
> `rm -rf data` may fail with *Permission denied*. Remove them via a helper container:
> ```
> docker run --rm -v "$PWD/data:/data" caddy:2-alpine sh -c 'rm -rf /data/caddy-config /data/caddy-data'
> ```
> Avoid `docker system prune --volumes` — this host also holds volumes for **other** projects.

---

## 5. `deploy.sh` (One-Shot Deployment)

```
1. Detect compose: `docker compose` (plugin) or `docker-compose` (v1 fallback)
   — stored in $COMPOSE and used for every subsequent call
2. Create data/ subdirs (instance, uploads, caddy-data, caddy-config, caddy-logs)
   and seed data/Caddyfile from Caddyfile.example
3. $COMPOSE up -d + verify containers "Up"
4. Run migration scripts (add_https_config_table, add_player_user_table,
   add_email_to_https_config, migrate_player_user_global,
   add_original_filename_to_content)
   ↳ NOTE: the container entrypoint already applies all seven on startup.
     This step is idempotent and therefore redundant.
5. /app/https_manager.py enable <hostname> <domain> <email> <ip> <port>
   ↳ Exit code 2 ("config applied, Caddy not reloaded") is non-fatal.
6. Verify DB tables via SQLAlchemy inspector; caddy validate;
   https_manager.py status; print access URLs + default creds
```

### Configuration variables

| Variable | Default | Meaning |
|---|---|---|
| `HOSTNAME` | `digiserver` | Display hostname |
| `HTTPS_MODE` | `internal` | `internal` \| `acme` \| `off` |
| `DOMAIN` | *(empty)* | Required only when `HTTPS_MODE=acme` |
| `IP_ADDRESS` | **auto-detected** | Primary LAN IP (override if needed) |
| `EMAIL` | `admin@example.com` | ACME account email (unused by internal CA) |
| `PORT` | `8443` | Externally published HTTPS port |

> If `IP_ADDRESS` is unset, `deploy.sh` auto-detects it
> (`ip -4 route get 1.1.1.1` → `src` address, falling back to `hostname -I`).
> The old hard-coded defaults (`10.76.152.164`, a `.intra` domain) were wrong for
> most hosts and have been removed.

---

## 6. HTTPS Setup (Caddy)

Three equivalent entry points drive the **same** code path
(`HTTPSConfig` + `CaddyConfigGenerator`) so CLI, env bootstrap and UI cannot diverge:

1. **Env bootstrap (deploy time)** — the container entrypoint runs
   `python /app/https_manager.py bootstrap`, which reads `HOSTNAME_INTERNAL` and
   `HOST_IP` from the environment and configures Caddy for HTTPS automatically.
2. **CLI** — `python /app/https_manager.py enable … | verify | status | disable`
3. **UI** — *Admin → HTTPS Configuration* (reloads Caddy live; the ongoing source of truth)

### Addressing model — one HTTP endpoint, one HTTPS endpoint

| Port | What it does |
|---|---|
| **80** | Always answers. A catch-all `:80` block serves **any** Host header, plus explicit blocks for the IP and hostname so both work. |
| **443** | HTTPS for the same names, using the internal CA (or ACME for a public domain). |

If HTTPS is disabled or never configured, port 80 simply serves the app — there is
no separate "HTTP mode" to set.

> ⚠️ **`default_sni` is required for IP access.** Browsers send **no SNI** when the
> URL is an IP address (an IP is not a valid SNI hostname). Caddy then identifies the
> connection by the container's own internal IP and aborts the handshake with
> `no certificate available for '<container-ip>'`. To prevent this, the generator emits
> `default_sni <ip>` whenever internal-CA mode is used, so `https://<ip>` works in a
> plain browser. This was found by end-to-end testing — see
> `docs/tools/test_http_https_runtime.sh`.

### Mode selection

| Condition | Result |
|---|---|
| HTTPS off, or no IP/domain | Plain HTTP on port 80 |
| HTTPS on + `ip_address` / `hostname` | `tls internal` per name (no DNS, no ACME) |
| HTTPS on + `domain` set | Let's Encrypt for that name |

### Automatic fallback if HTTPS does not work

After applying a config, `https_manager.py` probes the HTTPS endpoint
(`/api/health`, certificate validation deliberately disabled). If the TLS listener
does not come up, the configuration is **automatically reverted to plain HTTP** so a
failed certificate can never make the site unreachable:

```
enable HTTPS → reload Caddy → probe https://<host>:<port>/api/health
                               ├─ OK   → keep HTTPS
                               └─ FAIL → revert to HTTP-only, log a warning
```

Disable the probe with `HTTPS_VERIFY=false` (or `enable --no-verify`).
Re-check at any time with `python /app/https_manager.py verify`.

### Deploy-time bootstrap via `.env`

Copy `.env.example` → `.env` and set the host address. `docker-compose.yml`
forwards these to the app container:

| Variable | Effect |
|---|---|
| `HOSTNAME_INTERNAL` | Hostname served (both HTTP and HTTPS) |
| `HOST_IP` | IP served and certified |
| `DOMAIN` | **Leave empty for an intranet name** → internal CA. Set only for Let's Encrypt |
| `SSL_EMAIL` | ACME contact (ignored by internal CA) |
| `HTTP_PORT` / `HTTPS_PORT` | Host ports mapped to Caddy's 80/443 (default `80`/`443`) |
| `HTTPS_HTTP_FALLBACK` | `true` (default) also serves plain HTTP; `false` redirects instead |
| `HTTPS_VERIFY` | `true` (default) probe HTTPS and auto-fall back on failure |

```bash
cp .env.example .env
# set HOSTNAME_INTERNAL and HOST_IP
docker compose up -d --build
```

> **If either `HOSTNAME_INTERNAL` or `HOST_IP` is missing the bootstrap is a no-op** —
> the app starts on plain HTTP and stays reachable. HTTPS can then be enabled from
> *Admin → HTTPS Configuration*, which regenerates the Caddyfile and reloads Caddy
> **without a restart**.

### Who owns the config (env vs admin UI)

The admin UI is the **ongoing source of truth**. The bootstrap runs on every container
start, so it guards against silently overwriting an admin's change by tracking
provenance in `HTTPSConfig.updated_by`:

| Current config | Bootstrap behaviour |
|---|---|
| *(none — first deploy)* | Apply from env ✅ |
| Written by env/CLI (`updated_by='deploy.sh'`) | Apply from env ✅ — so editing `HOST_IP` and redeploying works |
| Written by a user (`updated_by='<username>'`) | **Skip** — the admin's setting is preserved |

So an admin change made in the UI survives restarts even while the env vars remain set.

### Why internal CA (and not Let's Encrypt) for `.intra`

An intranet name such as `digiserver.sibiusb.harting.intra` is **not resolvable from the
public internet**, so Let's Encrypt's HTTP-01/TLS-ALPN challenge cannot succeed. Setting
`DOMAIN=` empty makes Caddy sign the certificate itself with its local CA — no external
dependency at all.

**Trust caveat:** the internal CA is not in any client trust store, so browsers show a
warning and a Kivy player with `verify_ssl: true` will **reject the connection**. Options:

1. **Use HTTP** — port 80 always serves the app, so players need no trust configuration.
2. **Install the root CA** on each device:
   ```
   docker compose cp caddy:/data/caddy/pki/authorities/local/root.crt ./caddy-root.crt
   ```

### Ports

| Host → Container | Purpose |
|---|---|
| `80 → 80` | HTTP (always available) |
| `443 → 443` | HTTPS |
| `5000 → 5000` | Direct app access (bypasses Caddy; dev/testing) |

> Ports are configurable via `HTTP_PORT`/`HTTPS_PORT` so the stack also works where
> 80/443 are already taken (e.g. `HTTP_PORT=8080 HTTPS_PORT=8443`).




---

## 7. `verify-deployment.sh` — Pre/Post-Deployment Checks

Sections checked (pass/fail/warn counters):
- git status
- `.env` / `.env.example`
- Docker + Compose versions (plugin **or** v1) + `compose config` syntax
- Dockerfile best practices (HEALTHCHECK, non-root, slim base)
- `requirements.txt` critical packages + versions
- migrations directory
- **TLS certificate** — Caddy internal CA expiry (`data/caddy-data/caddy/pki/authorities/local/root.crt`)
- Flask config (`ProductionConfig`, `SESSION_COOKIE_SECURE`)
- `data/Caddyfile` checks (reverse_proxy, admin API, TLS mode)
- runtime container health + live HTTP/HTTPS endpoint probes
- security best practices

> The script now detects `docker compose` (plugin) or `docker-compose` (v1) and warns when
> buildx is too old for compose v1 builds — matching `deploy.sh`, which then falls back to
> `docker build`.

---

## 8. Player Deployment Pipeline

```mermaid
sequenceDiagram
    participant U as Admin
    participant A as App
    participant B as Background
    participant H as Player host
    U->>A: Add player (name, hostname, password/quickconnect)
    A->>B: background_player_deployment()
    B->>B: build/refresh staged code (PLAYER_CODE_DIR)
    B->>H: sshpass + ssh test
    B->>H: rsync code (or git clone/pull fallback)
    B->>H: write config/app_config.json (server_ip, quickconnect, https...)
    B->>H: temporary passwordless sudo
    B->>H: ./install.sh && ./start.sh
    B->>H: remove temp sudoers
    B-->>A: player.deployment_status = 'deployed' | 'failed'
    U->>A: poll /players/deployment-status
```

Statuses: `pending → deploying → deployed | failed`, with `last_deployment_at/status/message` persisted on the player row. Receiving player feedback also auto-marks deployment `deployed`.

---

## 9. Ports & Volumes Summary

| Item | Value |
|---|---|
| App port (container) | 5000 |
| App port (host, dev) | 5000 |
| Caddy HTTP | 8080 → 80 |
| Caddy HTTPS | 8443 → 443 |
| Caddy admin API | 2019 (container) |
| DB volume | `./data/instance` |
| Uploads volume | `./data/uploads` |
| Caddy data/config/logs | `./data/caddy-*` |

---

> Next: [08 · Workflows](08-workflows.md)