digiserver-v2/docs/06-utils-services.md

# 06 · Utilities & Services

All shared services live in `app/utils/`. This document details each module, its main functions, and how they fit together.

---

## Overview Table

| Module | Community | Responsibility | Key symbols |
|---|---|---|---|
| `logger.py` | C0 | DB-backed audit logging | `log_action()`, `get_recent_logs()`, `clear_old_logs()` |
| `group_player_management.py` | C0 | Player status reporting | `get_player_status_info()` |
| `caddy_manager.py` | C1 | HTTPS Caddyfile generation | `CaddyConfigGenerator`, `write_caddyfile()`, `reload_caddy()` |
| `background_tasks.py` | C10 | Async task execution | `run_background_task()`, `background_player_deployment()` |
| `ssh_deploy.py` | C10 | Remote player provisioning | `deploy_player_to_host()`, `test_ssh_connection()`, `generate_player_config()` |
| `player_build.py` | C11 | Stage player source code | `build_player_files()`, `write_base_config()`, `load/save_build_settings()` |
| `pptx_converter.py` | C12 | PPTX → PDF → PNG | `pptx_to_pdf_libreoffice()`, `validate_pptx_file()`, `cleanup_libreoffice_processes()` |
| `uploads.py` | C9 | Upload progress + file ops | `get/set/clear_upload_progress()`, `save_uploaded_file()`, `process_video_file()` |
| `portal_sso.py` | C4 | SSO auto-login | `init_portal_sso()`, `_get_or_create_user()` |
| `script_name_fix.py` | C4 | WSGI sub-path middleware | `ScriptNameFix` |

> `nginx_config_reader.py` (C7) was **removed** during the sanitization pass — the
> reverse proxy is Caddy. See [SANITIZATION-REVIEW.md](SANITIZATION-REVIEW.md).

---

## 1. `logger.py` — Audit Logging ⭐ God Node (116 edges)

The most-connected module in the system. Every meaningful action across all blueprints is recorded here.

| Function | Purpose |
|---|---|
| `log_action(level, message)` | Create + commit a `ServerLog` row |
| `log_info(message)` / `log_warning(message)` / `log_error(message)` | Convenience wrappers |
| `get_recent_logs(limit, level)` | Query recent logs (used by dashboard/admin) |
| `clear_old_logs(days)` | Housekeeping |

**Usage pattern:** admin actions (`reset_user_password`, `upload_header_logo`, `delete_editing_user`, `delete_playlist`, login failures, etc.) all funnel through `log_action()`.

---

## 2. `background_tasks.py` — Async Execution

| Function | Purpose |
|---|---|
| `run_background_task(func, *args)` | Spawns a **daemon thread** with a Flask app context pushed, then runs `func` |
| `background_player_deployment(...)` | Runs SSH deployment in background; updates `player.deployment_status` (`deploying → deployed|failed`) |

Used by `players.add_player` to avoid blocking the HTTP request during long remote installs.

---

## 3. `ssh_deploy.py` — Remote Player Provisioning ⭐ (heat 0.614)

The deployment engine. Full pipeline implemented by `deploy_player_to_host(...)`:

```mermaid
flowchart TD
    A["deploy_player_to_host()"] --> B["test_ssh_connection() (sshpass)"]
    B --> C["mkdir remote dir"]
    C --> D{"staged code present?"}
    D -- yes --> E["rsync pre-staged code"]
    D -- no --> F["git clone / pull"]
    E --> G["write config/app_config.json"]
    F --> G
    G --> H["temp passwordless sudo"]
    H --> I["run install.sh"]
    I --> J["run start.sh"]
    J --> K["cleanup sudoers"]
    K --> L["return steps[]"]
```

Other helpers:
- `get_local_player_code_status()` — inspect staged code state.
- `detect_server_ip()` / `parse_server_address()` — resolve the server address players should reach.
- `generate_player_config()` / `generate_app_config()` — write player's `config/app_config.json` with `server_ip`, `port`, `screen_name`, `quickconnect_key`, `orientation`, `use_https`, `verify_ssl`.

---

## 4. `caddy_manager.py` — HTTPS Automation ⭐ God Node

`CaddyConfigGenerator` produces the Caddyfile for the reverse proxy and reloads Caddy without restart.

| Method | Purpose |
|---|---|
| `generate_caddyfile(config, http_fallback=True)` | Pick template by mode: **HTTP-only** (`:80`), **domain** (Let's Encrypt), or **IP-only** (internal CA `tls internal`). Includes `reverse_proxy digiserver-app:5000`, 2 GB body limit, gzip, security headers. `http_fallback` also serves plain HTTP alongside internal-CA TLS so clients that cannot trust the local CA still work |
| `write_caddyfile(content, path=/etc/caddy/Caddyfile)` | Write to disk |
| `reload_caddy()` | POST to Caddy admin API `http://caddy:2019/load` |

Triggered from `admin.update_https_config` (Admin UI) **or** `https_manager.py` (CLI),
both of which save `HTTPSConfig` first so the two paths stay in sync.

> **Internal CA vs Let's Encrypt:** an intranet name (e.g. `*.harting.intra`) is not
> resolvable publicly, so ACME challenges cannot succeed. Leaving `domain` empty selects
> `tls internal`, which needs no DNS and no external service. See
> [07 · Deployment §6](07-deployment.md#6-https-setup-caddy).

---

## 4b. `https_manager.py` — HTTPS CLI (repo root)

Command-line equivalent of the Admin HTTPS page, used by `deploy.sh`.

| Command | Purpose |
|---|---|
| `enable <hostname> <domain> <email> <ip> [port]` | Persist `HTTPSConfig`, regenerate + write the Caddyfile, hot-reload Caddy. Empty `<domain>` → internal CA. `--redirect-only` to disable the HTTP fallback; `--no-https` for HTTP only |
| `disable` | Turn HTTPS off (HTTP only) |
| `status` | Print the stored configuration and resolved mode |

Exit codes: `0` success · `1` bad args/config · `2` config applied but Caddy did not reload.

---

## 5. `player_build.py` — Staging Player Code

| Function | Purpose |
|---|---|
| `build_player_files(dir, repo_url, branch)` | Clone or `fetch + reset` the player repository into `PLAYER_CODE_DIR` |
| `write_base_config(...)` | Write `config/app_config.json` (blank screen / quickconnect) |
| `load_build_settings()` / `save_build_settings()` | JSON at `instance/player_build.json` |
| `get_player_server_settings` / `make_build_record` / `get_short_head` | Build metadata helpers |

---

## 6. `uploads.py` — Upload & Media Processing

| Function | Purpose |
|---|---|
| `get_upload_progress` / `set_upload_progress` / `clear_upload_progress` | In-memory per-file progress (for the upload page) |
| `save_uploaded_file` | Save a multipart upload |
| `process_video_file` | FFmpeg → H.264 main, 30 fps, ≤1080p, faststart |
| `process_pdf_file` | PDF processing (stub — real logic in content.py) |
| `get_file_size` / `delete_file` | FS helpers |

---

## 7. `pptx_converter.py` — LibreOffice Integration

| Function | Purpose |
|---|---|
| `pptx_to_pdf_libreoffice(pptx_path, output_dir)` | Headless LibreOffice → PDF (300 s timeout) |
| `validate_pptx_file()` | Validate a file is a real PPTX |
| `cleanup_libreoffice_processes()` | `pkill soffice` — clean hanging processes |

Used by the upload pipeline: **PPTX → PDF → PNG slides (Full HD)**.

---

## 8. `portal_sso.py` & `script_name_fix.py` — Gateway Integration

- **`portal_sso.py`** — `before_request` reads `X-Auth-Username`/`X-Auth-Role` from the umbrella nginx and auto-logs-in the local user (creating it on first arrival). See [04 · Application Core §11](04-application-core.md#11-portal-sso--apputilsportal_sso_py).
- **`script_name_fix.py`** — WSGI middleware mapping `X-Script-Name` → `SCRIPT_NAME` so `url_for()` is correct behind a path-prefixed gateway.

---

## 9. `group_player_management.py` — Player Status

Only `get_player_status_info(player_id)` remains: it returns the online flag
(5-minute window), status, last-seen plus a humanised "time ago", and the latest
`PlayerFeedback`. Used by `players.list` and `players.manage_player`.

The group helpers (`get_group_statistics`, `assign_player_to_group`,
`bulk_assign_players_to_group`) and the status-list helpers
(`get_online_players_count`, `get_players_by_status`) were **removed** with the
archived Group subsystem.

---

> Next: [07 · Deployment](07-deployment.md)