updates to digiserver app server
This commit is contained in:
@@ -0,0 +1,198 @@
|
||||
# 01 · System Architecture
|
||||
|
||||
> Generated from the Graphify knowledge graph (41 communities) and source analysis of `digiserver-v2`.
|
||||
|
||||
---
|
||||
|
||||
## 1. System Overview
|
||||
|
||||
DigiServer v2 is a Flask application serving **three audiences**:
|
||||
|
||||
1. **Humans (admins)** — manage content, playlists, players, users, HTTPS via the web UI.
|
||||
2. **Players (physical signage)** — poll the REST API for playlists, send status feedback, upload edited media.
|
||||
3. **Operators** — deploy/update player software remotely over SSH.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Browser["Web UI (Admins)"]
|
||||
A[Login]
|
||||
B[Dashboard / Admin / Content / Players]
|
||||
end
|
||||
|
||||
subgraph App["digiserver-app (Flask :5000)"]
|
||||
C[Gunicorn x4]
|
||||
D[7 Blueprints]
|
||||
E[SQLAlchemy + SQLite]
|
||||
F[Cache / Background tasks]
|
||||
end
|
||||
|
||||
subgraph Players["Signage Players"]
|
||||
G[Player 1]
|
||||
H[Player N]
|
||||
end
|
||||
|
||||
subgraph Proxy["Reverse Proxy"]
|
||||
I[Caddy :80/:443]
|
||||
J[Umbrella nginx gateway - optional SSO]
|
||||
end
|
||||
|
||||
I -->|reverse_proxy digiserver-app:5000| C
|
||||
J -.X-Auth-Username .-> C
|
||||
D --> E
|
||||
G <-->|"/api/*"| D
|
||||
H <-->|"/api/*"| D
|
||||
Browser --> I
|
||||
C -.SSH deploy/rsync.-> G
|
||||
C -.SSH deploy/rsync.-> H
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Layered Architecture
|
||||
|
||||
Graphify assigns each node a **level** (0 = entry/global → 3 = utility):
|
||||
|
||||
| Layer | Level | Components |
|
||||
|---|---|---|
|
||||
| **L0 — Entry / Global** | 0 | `create_app()`, `app.py`, config classes, error handlers, CLI commands |
|
||||
| **L1 — Strategic / Core** | 1 | Blueprint route handlers (players, content, api, admin), model classes |
|
||||
| **L2 — Implementation** | 2 | Playlist/group management helpers, processing helpers, model methods |
|
||||
| **L3 — Utility** | 3 | `logger.py`, `ssh_deploy.py`, `caddy_manager.py`, `uploads.py`, `pptx_converter.py`, migrations |
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph L0["L0 · Entry"]
|
||||
create_app["create_app()"]
|
||||
ProxyFix
|
||||
ScriptNameFix
|
||||
portal_sso["init_portal_sso()"]
|
||||
end
|
||||
subgraph L1["L1 · Blueprints + Models"]
|
||||
bp["main · auth · admin · players · content · playlist · api"]
|
||||
models["User Player Content Playlist PlayerEdit PlayerFeedback PlayerUser HTTPSConfig ServerLog"]
|
||||
end
|
||||
subgraph L2["L2 · Implementation"]
|
||||
bg["run_background_task()"]
|
||||
dep["deploy_player_to_host()"]
|
||||
caddy["CaddyConfigGenerator"]
|
||||
pb["build_player_files() / write_base_config()"]
|
||||
end
|
||||
subgraph L3["L3 · Utility"]
|
||||
log["log_action()"]
|
||||
up["uploads.py"]
|
||||
pptx["pptx_converter.py"]
|
||||
nginx["NginxConfigReader"]
|
||||
end
|
||||
create_app --> bp
|
||||
create_app --> models
|
||||
bp --> log
|
||||
bp --> bg
|
||||
bp --> dep
|
||||
bp --> caddy
|
||||
bp --> pb
|
||||
bp --> up
|
||||
bp --> pptx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Component Map (files → communities)
|
||||
|
||||
Graphify clustered the code into **41 communities**. The 12 meaningful ones are shown below (communities 13–40 are mostly migrations, isolated helper functions, and archived code).
|
||||
|
||||
| Community | Domain (derived) | Files | Role |
|
||||
|---|---|---|---|
|
||||
| **C0** (90) | **Authentication + legacy groups** | `blueprints/auth.py`, `blueprints/content_old.py`, `models/server_log.py`, `utils/logger.py`, `utils/group_player_management.py`, `old_code_documentation/blueprint_groups.py` | Auth flows + audit logging + (legacy) group features |
|
||||
| **C1** (80) | **Admin + HTTPS + player users** | `blueprints/admin.py`, `models/https_config.py`, `models/player_user.py`, `utils/caddy_manager.py`, `migrations/add_player_user_table.py` | Admin panel, Caddy HTTPS generation, editing-user registry |
|
||||
| **C2** (68) | **Playlist & content workflows** | `blueprints/content.py`, `blueprints/playlist.py`, `models/playlist.py` | Modern playlist-centric content management + legacy redirects |
|
||||
| **C3** (62) | **Player API + edits** | `blueprints/api.py`, `models/player_edit.py` | Player-facing REST, edited-media pipeline |
|
||||
| **C4** (55) | **Application core** | `app.py`, `config.py`, `models/user.py`, `utils/portal_sso.py`, `utils/script_name_fix.py` | App factory, config, auth identity, middleware |
|
||||
| **C5** (54) | **Content & player models** | `models/content.py`, `models/player.py`, `models/player_feedback.py`, `blueprints/main.py` | Media model, player model, feedback, dashboard |
|
||||
| **C6** (51) | **Player management UI** | `blueprints/players.py`, `migrations/add_https_config_table.py` | Player CRUD, manage page, deployment polling |
|
||||
| **C7** (42) | **Groups (legacy)** | `models/group.py`, `utils/nginx_config_reader.py` | Archived groups feature + legacy nginx reader |
|
||||
| **C8** (24) | **Dev tooling** | `old_code_documentation/test_edit_media_api.py`, `Colors`, integrity checker | Test/analysis utilities |
|
||||
| **C9** (21) | **Upload processing** | `utils/uploads.py` | Upload progress, video/image processing |
|
||||
| **C10** (20) | **Deployment** | `utils/background_tasks.py`, `utils/ssh_deploy.py` | Background SSH deployment engine |
|
||||
| **C11** (19) | **Player build/staging** | `utils/player_build.py` | Stage player source code on server |
|
||||
| **C12** (8) | **PPTX conversion** | `utils/pptx_converter.py` | LibreOffice PPTX → PDF → PNG |
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
C0["C0 · Auth & logging"]
|
||||
C1["C1 · Admin & HTTPS"]
|
||||
C2["C2 · Playlists & content"]
|
||||
C3["C3 · Player API & edits"]
|
||||
C4["C4 · App core"]
|
||||
C5["C5 · Content/player models"]
|
||||
C6["C6 · Player management UI"]
|
||||
C9["C9 · Upload processing"]
|
||||
C10["C10 · Deployment"]
|
||||
C11["C11 · Player build"]
|
||||
C12["C12 · PPTX conversion"]
|
||||
|
||||
C1 -->|27 edges| C3
|
||||
C2 -->|25 edges| C3
|
||||
C2 -->|25 edges| C0
|
||||
C3 -->|25 edges| C1
|
||||
C1 -->|21 edges| C0
|
||||
C6 -->|16 edges| C1
|
||||
C6 -->|16 edges| C3
|
||||
C3 -->|14 edges| C0
|
||||
C6 -->|14 edges| C0
|
||||
C5 -->|9 edges| C3
|
||||
C9 -->|5 edges| C0
|
||||
C1 -->|3 edges| C11
|
||||
C10 --> C6
|
||||
C11 --> C10
|
||||
C12 --> C9
|
||||
```
|
||||
|
||||
*(edge weights from `graphify-out/metadata.json` communityLinks)*
|
||||
|
||||
---
|
||||
|
||||
## 4. Middleware & Request Pipeline
|
||||
|
||||
Every request flows through three layers before reaching a blueprint:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
R["Raw request"] --> PF["werkzeug ProxyFix"]
|
||||
PF --> SF["ScriptNameFix"]
|
||||
SF --> SSO["portal_sso (before_request)"]
|
||||
SSO --> BP["Blueprint route"]
|
||||
BP --> DB[("SQLAlchemy / SQLite")]
|
||||
```
|
||||
|
||||
1. **ProxyFix** — trusts `X-Forwarded-For/Proto/Host/Port` from Caddy/nginx (1 hop).
|
||||
2. **ScriptNameFix** — reads `X-Script-Name` (e.g. `/digiserver`) so `url_for()` produces correct prefixed URLs behind an umbrella nginx gateway.
|
||||
3. **Portal SSO** (`portal_sso.py`) — if the umbrella nginx sends `X-Auth-Username` / `X-Auth-Role`, automatically create + log in the local user.
|
||||
|
||||
---
|
||||
|
||||
## 5. Key Architectural Decisions
|
||||
|
||||
| Decision | Implementation |
|
||||
|---|---|
|
||||
| **Application factory** | `create_app(config_name)` — clean testability, per-env config |
|
||||
| **Blueprint isolation** | 7 blueprints with clear URL prefixes; API separate from UI |
|
||||
| **Playlist versioning** | `Playlist.version` incremented on every mutation → players poll `/api/playlist-version/<id>` for refresh |
|
||||
| **In-memory caching** | Flask-Caching `@cache.memoize(300)` on playlist builders; `cache.clear()` on mutations |
|
||||
| **DB-backed audit log** | `log_action()` writes `ServerLog` rows; surfaced on dashboard & admin |
|
||||
| **Two auth systems** | Humans: Flask-Login (`User`). Players: `auth_code` Bearer + `quickconnect_code` (bcrypt) |
|
||||
| **Remote provisioning** | SSH + rsync/git staged code → write `app_config.json` → run install/start scripts |
|
||||
| **HTTPS automation** | `CaddyConfigGenerator` writes `Caddyfile`; reloads Caddy via admin API on `:2019` |
|
||||
|
||||
---
|
||||
|
||||
## 6. Runtime Dependencies (requirements.txt)
|
||||
|
||||
**Runtime:** Flask 3.1, Werkzeug 3.1, SQLAlchemy 2.0.37, Flask-SQLAlchemy 3.1.1, Flask-Migrate, Flask-Bcrypt, Flask-Login, Flask-Caching, Flask-Cors, Flask-Talisman, pdf2image, Pillow, ffmpeg-python, python-magic, bcrypt, cryptography, gunicorn 23, psutil, python-dotenv.
|
||||
|
||||
**System binaries (Docker):** `poppler-utils`, `ffmpeg`, `libmagic1`, `LibreOffice` (core/impress/writer), `fonts-noto-color-emoji`, `sshpass`, `openssh-client`, `rsync`, `git`.
|
||||
|
||||
**Dev:** black, flake8, pytest, pytest-cov. *(`gevent` is commented out — incompatible with Python 3.13.)*
|
||||
|
||||
---
|
||||
|
||||
> Next: [02 · Knowledge Graph](02-knowledge-graph.md)
|
||||
@@ -0,0 +1,148 @@
|
||||
# 02 · The Graphify Knowledge Graph
|
||||
|
||||
This document explains the **interactive knowledge graph** generated by the [Graphify](https://marketplace.visualstudio.com/items?itemName=anytechiestudio.graphify-vscode) extension, and how to use it to explore and maintain DigiServer v2.
|
||||
|
||||
---
|
||||
|
||||
## 1. What Was Generated
|
||||
|
||||
Running `Graphify: Build Knowledge Graph` on the project root produced `graphify-out/`:
|
||||
|
||||
| Artifact | Description |
|
||||
|---|---|
|
||||
| `graph.html` | **Interactive visualizer** — zoom/pan/filter nodes, click to jump to source |
|
||||
| `graph.json` | Raw graph data (nodes, links, communities, heat) |
|
||||
| `GRAPH_REPORT.md` | God nodes, surprising connections, community list, knowledge gaps |
|
||||
| `COMPASS.md` | Token-optimized architecture summary (god nodes, layers) |
|
||||
| `DOMAINS.md` | Community ID → domain table |
|
||||
| `intelligence.json` | AI-detected intelligence (god nodes, surprising links) |
|
||||
| `graph.compact.txt` | Compact graph dump, ideal for LLM context |
|
||||
| `wiki/` | Markdown articles per community + per god-node, with Mermaid diagrams |
|
||||
|
||||
**Graph statistics:**
|
||||
|
||||
```
|
||||
631 nodes · 1162 edges · 41 communities
|
||||
Extraction: 56% EXTRACTED · 44% INFERRED (avg confidence 0.62)
|
||||
Node types: 313 FUNCTION · 300 FILE · 18 CLASS
|
||||
Link types: 311 uses · 290 calls · 275 rationale_for · 243 contains · 39 method · 4 inherits
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Opening the Visualizer
|
||||
|
||||
1. In VS Code, click the **Graphify** activity-bar icon (sidebar).
|
||||
2. Click **Build Knowledge Graph** (or right-click any folder → *Graphify: Build Knowledge Graph*).
|
||||
3. Once built, click **Open Interactive Visualizer** — or open `graphify-out/graph.html` directly in a browser.
|
||||
|
||||
> ⚠️ The visualizer opens best inside the VS Code webview (activity bar). Opening `graph.html` via `file://` may show a minor console error but still renders the graph data (nodes/edges/communities listed in the sidebar).
|
||||
|
||||
**View modes:** Deep Dive · Full Detail · Heatmap · Simulation (component-removal impact).
|
||||
|
||||
---
|
||||
|
||||
## 3. God Nodes — The Core Abstractions
|
||||
|
||||
These are the most-connected nodes (graph centrality). They are the architectural "load-bearing" abstractions:
|
||||
|
||||
| Rank | Node | Degree | File | Why it matters |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `log_action()` | 116 | `app/utils/logger.py` | Every important action across all blueprints logs here |
|
||||
| 2 | `PlayerEdit` | 99 | `app/models/player_edit.py` | Central record of on-player media edits |
|
||||
| 3 | `PlayerUser` | 74 | `app/models/player_user.py` | Maps player edit user codes to names |
|
||||
| 4 | `Playlist` | 33 | `app/models/playlist.py` | Core content organization + version sync |
|
||||
| 5 | `CaddyConfigGenerator` | 31 | `app/utils/caddy_manager.py` | HTTPS Caddyfile generation/reload |
|
||||
| 6 | `HTTPSConfig` | 30 | `app/models/https_config.py` | Persisted HTTPS settings |
|
||||
| 7 | `Content` | 24 | `app/models/content.py` | Media model used by everything |
|
||||
| 8 | `User` | 19 | `app/models/user.py` | Human accounts (admin/user/viewer) |
|
||||
| 9 | `create_app()` | 13 | `app/app.py` | Application factory |
|
||||
| 10 | `models/__init__.py` | 13 | `app/models/__init__.py` | Package export hub |
|
||||
|
||||
---
|
||||
|
||||
## 4. Heatmap — Hottest Code
|
||||
|
||||
`heat` (0–1) reflects how central/complex a node is in the graph. The hottest functions drive most of the system's behaviour:
|
||||
|
||||
| Heat | Function | File |
|
||||
|---|---|---|
|
||||
| 0.620 | `receive_edited_media()` | `app/blueprints/api.py` |
|
||||
| 0.614 | `deploy_player_to_host()` | `app/utils/ssh_deploy.py` |
|
||||
| 0.592 | `add_player()` | `app/blueprints/players.py` |
|
||||
| 0.537 | `upload_media()` | `app/blueprints/content.py` |
|
||||
| 0.429 | `manage_player()` | `app/blueprints/players.py` |
|
||||
| 0.425 | `receive_player_feedback()` | `app/blueprints/api.py` |
|
||||
| 0.407 | `update_https_config()` | `app/blueprints/admin.py` |
|
||||
| 0.376 | `process_file_in_background()` | `app/blueprints/content.py` |
|
||||
| 0.365 | `CaddyConfigGenerator` | `app/utils/caddy_manager.py` |
|
||||
| 0.354 | `get_playlist_by_quickconnect()` | `app/blueprints/api.py` |
|
||||
|
||||
---
|
||||
|
||||
## 5. Communities (Clusters)
|
||||
|
||||
Graphify groups related code into communities. See [01-architecture.md §3](01-architecture.md#3-component-map-files--communities) for the full mapping. The largest:
|
||||
|
||||
- **C0 · Auth & logging** (90 nodes) — login/logout/register + audit logging + legacy groups
|
||||
- **C1 · Admin & HTTPS** (80 nodes) — admin panel, Caddy generation, editing users
|
||||
- **C2 · Playlists & content** (68 nodes) — the modern content/playlist workflow
|
||||
- **C3 · Player API & edits** (62 nodes) — player-facing REST + edited-media pipeline
|
||||
- **C4 · Application core** (55 nodes) — `create_app`, config, user, middleware
|
||||
- **C5 · Content/player models** (54 nodes) — `Content`, `Player`, feedback, dashboard
|
||||
|
||||
Each community has a wiki article with a **Mermaid class diagram**, key concepts, source files, and an audit trail (extracted vs inferred edges). Example: `graphify-out/wiki/Community_4.md`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Surprising Connections (AI-detected)
|
||||
|
||||
Graphify flags links that are non-obvious. Examples from `GRAPH_REPORT.md`:
|
||||
|
||||
- `reset_user_password()` → `log_action()` — admin password resets are audited
|
||||
- `upload_header_logo()` → `log_action()` — logo changes are audited
|
||||
- `delete_editing_user()` → `log_action()` — editing-user deletion is audited
|
||||
- `delete_playlist()` → `log_action()` — playlist deletion is audited
|
||||
- `Main playlist management page` uses `PlayerEdit` — the content UI surfaces edit counts
|
||||
|
||||
**Pattern:** virtually every destructive/admin action flows through `log_action()`, which is why it is the #1 god node.
|
||||
|
||||
---
|
||||
|
||||
## 7. Known Gaps in the Graph
|
||||
|
||||
- **178 isolated nodes** (≤1 connection) — mostly standalone migration scripts and model helper methods.
|
||||
- **Thin communities 13–40** — single-file migrations, isolated utility functions, and archived `old_code_documentation/` scripts. These are not "broken" — they are intentionally decoupled.
|
||||
- The graph does **not** index templates or static assets (Python AST extraction only).
|
||||
|
||||
---
|
||||
|
||||
## 8. Querying the Graph (CLI)
|
||||
|
||||
The `anytechie-graphify` engine (installed in `.venv`) ships a CLI for graph-aware questions:
|
||||
|
||||
```bash
|
||||
# Shortest path between two nodes
|
||||
python -m graphify path "Content" "Playlist"
|
||||
|
||||
# Explain a node and its neighbours
|
||||
python -m graphify explain "deploy_player_to_host()"
|
||||
|
||||
# BFS traversal to answer a question (no LLM)
|
||||
python -m graphify query "how is a playlist synced to a player?"
|
||||
|
||||
# Grounded chat over the graph (requires an LLM API key)
|
||||
python -m graphify ask "what happens when a player uploads edited media?"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Keeping the Graph Fresh
|
||||
|
||||
- **After code changes:** `Graphify: Build Knowledge Graph` again (or run `python -m graphify update <path>`).
|
||||
- **Git hooks:** `python -m graphify hook install` auto-rebuilds on `post-commit`/`post-checkout`.
|
||||
- The graph is stored in `graphify-out/` (not committed to git by default).
|
||||
|
||||
---
|
||||
|
||||
> Next: [03 · Data Model](03-data-model.md)
|
||||
@@ -0,0 +1,198 @@
|
||||
# 03 · Data Model
|
||||
|
||||
DigiServer v2 uses **SQLAlchemy 2.0** with **SQLite** (production: `instance/dashboard.db`, dev: `instance/dev.db`, tests: in-memory). All models live in `app/models/` and are re-exported from `app/models/__init__.py`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Entity-Relationship Overview
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
user ||--o{ server_log : "writes"
|
||||
player ||--o{ player_feedback : "sends (cascade)"
|
||||
player ||--o{ player_edit : "edits (cascade)"
|
||||
player }o--o| playlist : "assigned_to"
|
||||
content ||--o{ player_edit : "edited (cascade)"
|
||||
content ||--o{ player_feedback : "playing"
|
||||
playlist ||--o{ content : "playlist_content M2M (position,duration,muted,edit_on_player_enabled)"
|
||||
content }o--o{ group : "group_content M2M (legacy)"
|
||||
player_user ||--o{ player_edit : "user_code"
|
||||
https_config ||--o| https_config : "singleton row"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Tables & Columns
|
||||
|
||||
### `user` — human accounts
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `username` | String(80) | unique, NOT NULL, indexed |
|
||||
| `password` | String(120) | bcrypt hash, NOT NULL |
|
||||
| `role` | String(20) | default `'user'`; also `'admin'`, `'viewer'`; indexed |
|
||||
| `theme` | String(20) | default `'light'` |
|
||||
| `created_at` | DateTime | NOT NULL |
|
||||
| `last_login` | DateTime | nullable |
|
||||
|
||||
Key methods: `is_admin` (property), `update_last_login()`. Mixin: `UserMixin` (Flask-Login).
|
||||
|
||||
### `player` — signage devices
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `name` | String(255) | NOT NULL |
|
||||
| `hostname` | String(255) | unique, NOT NULL, indexed |
|
||||
| `location` | String(255) | nullable |
|
||||
| `auth_code` | String(255) | unique, NOT NULL, indexed (legacy Bearer auth) |
|
||||
| `password_hash` | String(255) | NOT NULL (bcrypt) |
|
||||
| `quickconnect_code` | String(255) | nullable, bcrypt-hashed |
|
||||
| `orientation` | String(16) | default `'Landscape'` |
|
||||
| `status` | String(50) | default `'offline'`, indexed |
|
||||
| `last_seen` | DateTime | indexed |
|
||||
| `last_heartbeat` | DateTime | indexed |
|
||||
| `created_at` | DateTime | NOT NULL |
|
||||
| `playlist_id` | Integer | FK → `playlist.id` ON DELETE SET NULL, indexed |
|
||||
| `deployment_status` | String(50) | default `'pending'`; pending/deploying/deployed/failed |
|
||||
| `last_deployment_at` | DateTime | nullable |
|
||||
| `last_deployment_status` | String(50) | nullable |
|
||||
| `last_deployment_message` | Text | nullable |
|
||||
|
||||
Relationships: `playlist`, `feedback` (→ PlayerFeedback, cascade delete-orphan).
|
||||
Methods: `is_online` (5-min window), `update_status()`, `set_password()/check_password()`, `set_quickconnect_code()/check_quickconnect_code()`, static `authenticate(hostname, password, quickconnect_code)`.
|
||||
|
||||
### `content` — media items (also weblinks)
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `filename` | String(255) | unique, NOT NULL, indexed; may point to `edited_media/<id>/...` after a player edit |
|
||||
| `original_filename` | String(255) | nullable, indexed — pristine upload name |
|
||||
| `content_type` | String(50) | NOT NULL, indexed; image/video/pdf/pptx/weblink/other |
|
||||
| `duration` | Integer | default 10, nullable |
|
||||
| `file_size` | BigInteger | nullable |
|
||||
| `url` | String(2048) | nullable — target URL for `weblink` content |
|
||||
| `description` | Text | nullable |
|
||||
| `uploaded_at` | DateTime | NOT NULL, indexed |
|
||||
|
||||
Relationships: `playlists` (M2M via `playlist_content`), `groups` (M2M via `group_content`).
|
||||
Properties/methods: `file_size_mb`, `group_count`, `original_display_name`, `original_media_path`, `current_media_path`, `is_image()/is_video()/is_pdf()/is_weblink()`, `has_player_edits`.
|
||||
|
||||
### `playlist` — ordered collections of content
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `name` | String(100) | unique, NOT NULL, indexed |
|
||||
| `description` | Text | nullable |
|
||||
| `orientation` | String(20) | default `'Landscape'` |
|
||||
| `version` | Integer | default 1, NOT NULL — **sync detection** |
|
||||
| `is_active` | Boolean | default True |
|
||||
| `created_at` | DateTime | NOT NULL |
|
||||
| `updated_at` | DateTime | NOT NULL, onupdate |
|
||||
|
||||
Methods: `player_count`, `content_count`, `total_duration` (properties), `increment_version()`, `get_content_ordered()`.
|
||||
|
||||
### `playlist_content` — association table (M2M playlist ↔ content)
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `playlist_id` | Integer | FK → `playlist.id` CASCADE, composite PK |
|
||||
| `content_id` | Integer | FK → `content.id` CASCADE, composite PK |
|
||||
| `position` | Integer | default 0 — ordering |
|
||||
| `duration` | Integer | default 10 — per-playlist override |
|
||||
| `muted` | Boolean | default True |
|
||||
| `edit_on_player_enabled` | Boolean | default False — whether player-side editing allowed |
|
||||
|
||||
### `player_edit` — on-player media edits
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `player_id` | Integer | FK → `player.id` CASCADE, indexed |
|
||||
| `content_id` | Integer | FK → `content.id` CASCADE, indexed |
|
||||
| `original_name` | String(255) | NOT NULL |
|
||||
| `new_name` | String(255) | NOT NULL |
|
||||
| `version` | Integer | default 1 |
|
||||
| `user` | String(255) | nullable (user code) |
|
||||
| `time_of_modification` | DateTime | nullable |
|
||||
| `metadata_path` | String(512) | nullable |
|
||||
| `edited_file_path` | String(512) | NOT NULL |
|
||||
| `created_at` | DateTime | NOT NULL, indexed |
|
||||
|
||||
Relationships: `player`, `content`. Method: `to_dict()`.
|
||||
|
||||
### `player_feedback` — status messages from players
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `player_id` | Integer | FK → `player.id`, NOT NULL, indexed |
|
||||
| `status` | String(50) | default `'unknown'` |
|
||||
| `current_content_id` | Integer | FK → `content.id`, nullable |
|
||||
| `message` | Text | nullable |
|
||||
| `error` | Text | nullable |
|
||||
| `timestamp` | DateTime | NOT NULL, indexed |
|
||||
|
||||
Properties/classmethod: `is_error`, `age_seconds`, `get_latest_for_player()`.
|
||||
|
||||
### `player_user` — mapping of player edit user codes
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `user_code` | String(255) | **globally unique**, NOT NULL, indexed |
|
||||
| `user_name` | String(255) | nullable |
|
||||
| `created_at` | DateTime | NOT NULL |
|
||||
| `updated_at` | DateTime | NOT NULL, onupdate |
|
||||
|
||||
Method: `to_dict()`. *(Migrated from a per-player table — see `migrate_player_user_global.py`.)*
|
||||
|
||||
### `server_log` — DB-backed audit log
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `level` | String(20) | NOT NULL, indexed, default `'info'` |
|
||||
| `message` | Text | NOT NULL |
|
||||
| `timestamp` | DateTime | NOT NULL, indexed |
|
||||
|
||||
Classmethods: `log_info`, `log_warning`, `log_error`.
|
||||
|
||||
### `https_config` — HTTPS settings (singleton row)
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `https_enabled` | Boolean | default False |
|
||||
| `hostname` | String(255) | nullable |
|
||||
| `domain` | String(255) | nullable |
|
||||
| `ip_address` | String(45) | nullable (IPv6-capable) |
|
||||
| `email` | String(255) | nullable (Let's Encrypt contact) |
|
||||
| `port` | Integer | default 443 |
|
||||
| `created_at` | DateTime | NOT NULL |
|
||||
| `updated_at` | DateTime | NOT NULL, onupdate |
|
||||
| `updated_by` | String(255) | nullable |
|
||||
|
||||
Classmethods: `get_config()` (first row), `create_or_update(...)`. Method: `to_dict()`.
|
||||
|
||||
### `group` + `group_content` — **ARCHIVED / LEGACY**
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | Integer | PK |
|
||||
| `name` | String(100) | unique, NOT NULL, indexed |
|
||||
| `description` | Text | nullable |
|
||||
| `created_at` / `updated_at` | DateTime | NOT NULL |
|
||||
|
||||
`group_content(group_id, content_id)` composite-PK M2M. The current `Player` model has **no** `group_id` column — group features are archived (see [09 · Legacy](09-legacy-and-migrations.md)).
|
||||
|
||||
---
|
||||
|
||||
## 3. Relationship Summary
|
||||
|
||||
| Relationship | Cardinality | FK / Mechanism |
|
||||
|---|---|---|
|
||||
| `playlist` → `content` | M:N | `playlist_content` (positioned, with extras) |
|
||||
| `group` → `content` | M:N | `group_content` (legacy) |
|
||||
| `player` → `playlist` | N:1 | `player.playlist_id` (ON DELETE SET NULL) |
|
||||
| `player` → `player_feedback` | 1:N | `player_feedback.player_id` (cascade) |
|
||||
| `player` → `player_edit` | 1:N | `player_edit.player_id` (cascade) |
|
||||
| `content` → `player_edit` | 1:N | `player_edit.content_id` (cascade) |
|
||||
| `player_user` → `player_edit` | 1:N | `player_edit.user` = `player_user.user_code` (logical) |
|
||||
| `https_config` | 1 row | singleton via `get_config()` |
|
||||
|
||||
---
|
||||
|
||||
> Next: [04 · Application Core](04-application-core.md)
|
||||
@@ -0,0 +1,178 @@
|
||||
# 04 · Application Core
|
||||
|
||||
Covers the application factory, configuration, extensions, middleware, CLI commands, context processors, and template layer. This is **Community 4** in the Graphify knowledge graph.
|
||||
|
||||
---
|
||||
|
||||
## 1. Application Factory — `app/app.py`
|
||||
|
||||
The entire app is constructed by `create_app(config_name=None)`:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["create_app()"] --> B["Set instance_path"]
|
||||
B --> C["Load config: Dev / Prod / Test"]
|
||||
C --> D["ProxyFix middleware"]
|
||||
D --> E["ScriptNameFix middleware"]
|
||||
E --> F["init extensions: db bcrypt login migrate cache cors"]
|
||||
F --> G["configure_login_manager()"]
|
||||
G --> H["register_blueprints()"]
|
||||
H --> I["register_error_handlers()"]
|
||||
I --> J["register_commands()"]
|
||||
J --> K["register_context_processors()"]
|
||||
K --> L["register_template_filters()"]
|
||||
L --> M["init_portal_sso(app)"]
|
||||
M --> N["db.create_all() (idempotent)"]
|
||||
N --> O["return app"]
|
||||
```
|
||||
|
||||
### Blueprint registration
|
||||
|
||||
```python
|
||||
def register_blueprints(app):
|
||||
from app.blueprints.main import main_bp
|
||||
from app.blueprints.auth import auth_bp
|
||||
from app.blueprints.admin import admin_bp
|
||||
from app.blueprints.players import players_bp
|
||||
from app.blueprints.content import content_bp
|
||||
from app.blueprints.playlist import playlist_bp
|
||||
from app.blueprints.api import api_bp
|
||||
...
|
||||
```
|
||||
|
||||
> Note: `app.blueprints.content_old` is **not** imported — it is dead/legacy code.
|
||||
|
||||
---
|
||||
|
||||
## 2. Configuration — `app/config.py`
|
||||
|
||||
Four classes: `Config` (base) → `DevelopmentConfig`, `ProductionConfig`, `TestingConfig`.
|
||||
|
||||
| Setting | Value | Notes |
|
||||
|---|---|---|
|
||||
| `MAX_CONTENT_LENGTH` | 2 GB | upload cap |
|
||||
| `UPLOAD_FOLDER` | `app/static/uploads` | media |
|
||||
| `UPLOAD_FOLDERLOGO` | `app/static/resurse` | logos |
|
||||
| `ALLOWED_EXTENSIONS` | png jpg jpeg gif bmp mp4 avi mkv mov webm pdf ppt pptx | |
|
||||
| `PERMANENT_SESSION_LIFETIME` | 30 min | |
|
||||
| `SESSION_COOKIE_SECURE` | False (True in prod) | |
|
||||
| `ITEMS_PER_PAGE` | 20 | |
|
||||
| `SERVER_VERSION` | 2.0.0 | shown in UI footer |
|
||||
| `PLAYER_CODE_DIR` | `/app/data/player` | staged player source |
|
||||
| `PLAYER_REPO_URL` | env `PLAYER_REPO_URL` | Kiwy-Signage repo |
|
||||
| DB (dev) | `instance/dev.db` | |
|
||||
| DB (prod) | `instance/dashboard.db` | |
|
||||
| DB (test) | in-memory | |
|
||||
|
||||
---
|
||||
|
||||
## 3. Extensions — `app/extensions.py`
|
||||
|
||||
Centralizes shared singletons:
|
||||
|
||||
```python
|
||||
db = SQLAlchemy()
|
||||
bcrypt = Bcrypt()
|
||||
login_manager = LoginManager() # login_view='auth.login'
|
||||
migrate = Migrate()
|
||||
cache = Cache()
|
||||
cors = CORS()
|
||||
```
|
||||
|
||||
CORS is configured in `create_app` for `/api/*`: all origins, GET/POST/OPTIONS/PUT/DELETE, supports credentials.
|
||||
|
||||
---
|
||||
|
||||
## 4. Middleware
|
||||
|
||||
### ProxyFix (Werkzeug)
|
||||
`app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_port=1)` — trusts one reverse-proxy hop (Caddy / umbrella nginx).
|
||||
|
||||
### ScriptNameFix — `app/utils/script_name_fix.py`
|
||||
`ScriptNameFix` WSGI middleware sets `SCRIPT_NAME` from `HTTP_X_SCRIPT_NAME` so `url_for()` generates correct paths when the app is mounted under a sub-path (e.g. `/digiserver`).
|
||||
|
||||
---
|
||||
|
||||
## 5. Login Manager
|
||||
|
||||
```python
|
||||
login_manager.login_view = 'auth.login'
|
||||
login_manager.login_message = 'Please log in to access this page.'
|
||||
|
||||
@login_manager.user_loader
|
||||
def load_user(user_id):
|
||||
return User.query.get(int(user_id))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Error Handlers
|
||||
|
||||
| Code | Template | Behaviour |
|
||||
|---|---|---|
|
||||
| 404 | `errors/404.html` | Not found |
|
||||
| 403 | `errors/403.html` | Forbidden |
|
||||
| 500 | `errors/500.html` | Rolls back DB session |
|
||||
| 413 | `errors/413.html` | Payload too large |
|
||||
| 408 | `errors/408.html` | Request timeout |
|
||||
|
||||
The `/api` blueprint registers its own **JSON** 404/405/500 handlers.
|
||||
|
||||
---
|
||||
|
||||
## 7. CLI Commands
|
||||
|
||||
| Command | Purpose |
|
||||
|---|---|
|
||||
| `flask init-db` | `db.create_all()` |
|
||||
| `flask create-admin --username X` | Create admin user (prompts for password) |
|
||||
| `flask seed-db` | Seed sample data (blocked in production) |
|
||||
|
||||
---
|
||||
|
||||
## 8. Context Processors
|
||||
|
||||
- **`inject_config`** — exposes `server_version`, `build_date`, `logo_exists` to all templates.
|
||||
- **`inject_user_theme`** — exposes the authenticated user's `theme` (`light`/`dark`) for the UI toggle.
|
||||
|
||||
---
|
||||
|
||||
## 9. Template Filters
|
||||
|
||||
- **`localtime`** — converts naive/UTC datetimes to local time with a configurable `strftime` format (default `%Y-%m-%d %H:%M`).
|
||||
|
||||
---
|
||||
|
||||
## 10. Template & Static Layout
|
||||
|
||||
```
|
||||
app/templates/
|
||||
├── base.html ← main layout (theme, logos, nav)
|
||||
├── dashboard.html
|
||||
├── auth/ login.html, register.html, change_password.html
|
||||
├── admin/ admin.html, user_management.html, leftover_media.html,
|
||||
│ dependencies.html, customize_logos.html, editing_users.html,
|
||||
│ https_config.html, build_player.html
|
||||
├── content/ content_list.html (legacy), content_list_new.html (modern),
|
||||
│ media_library.html, upload_content.html (legacy),
|
||||
│ upload_media.html, manage_playlist_content.html, edit_content.html
|
||||
├── players/ players_list.html, add_player.html, edit_player.html,
|
||||
│ manage_player.html, player_page.html, player_fullscreen.html,
|
||||
│ edited_media.html, edited_media_report.html, _deploy_badge.html
|
||||
└── errors/ 403.html, 404.html, 500.html (+413/408)
|
||||
|
||||
app/static/
|
||||
├── icons/ edit, home, info, monitor, moon, playlist, sun, trash, upload, warning (SVG)
|
||||
├── uploads/ uploaded media + edited_media/<content_id>/ (versioned edits)
|
||||
└── (resurse/ logo storage — referenced by config)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Portal SSO — `app/utils/portal_sso.py`
|
||||
|
||||
`init_portal_sso(app)` registers a `before_request` hook that reads `X-Auth-Username` / `X-Auth-Role` headers (set by the umbrella nginx gateway). If present, it auto-creates and logs in the local `User` (`_get_or_create_user`). This lets the app sit behind an existing corporate SSO portal.
|
||||
|
||||
---
|
||||
|
||||
> Next: [05 · Blueprints & API](05-blueprints-api.md)
|
||||
@@ -0,0 +1,280 @@
|
||||
# 05 · Blueprints & REST API
|
||||
|
||||
DigiServer v2 registers **7 active blueprints** (`content_old.py` is legacy dead code and is not registered).
|
||||
|
||||
---
|
||||
|
||||
## 1. Blueprint Overview
|
||||
|
||||
| Blueprint | URL prefix | Module | Focus |
|
||||
|---|---|---|---|
|
||||
| `main_bp` | `/` | `blueprints/main.py` | Dashboard, health |
|
||||
| `auth_bp` | `/` | `blueprints/auth.py` | Login/logout/register/password |
|
||||
| `admin_bp` | `/admin` | `blueprints/admin.py` | Admin panel, HTTPS, build, users, logos |
|
||||
| `players_bp` | `/players` | `blueprints/players.py` | Player management |
|
||||
| `content_bp` | `/content` | `blueprints/content.py` | Media library + modern playlist management |
|
||||
| `playlist_bp` | `/playlist` | `blueprints/playlist.py` | Legacy per-player playlist (redirects) |
|
||||
| `api_bp` | `/api` | `blueprints/api.py` | Player-facing REST + deployment API |
|
||||
|
||||
---
|
||||
|
||||
## 2. `main` — Dashboard & Health (`/`)
|
||||
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/` | GET | `dashboard` | Counts (players/playlists/content), storage MB, 20 recent logs (cached 60s except `viewer` role) |
|
||||
| `/health` | GET | `health` | Health JSON: `SELECT 1` DB ping + free disk space |
|
||||
|
||||
---
|
||||
|
||||
## 3. `auth` — Authentication (`/`)
|
||||
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/login` | GET/POST | `login` | bcrypt login, `remember`, `next` redirect, failed-attempt logging |
|
||||
| `/logout` | GET | `logout` | Logs out + logs action |
|
||||
| `/register` | GET/POST | `register` | Self-registration → default `viewer` role |
|
||||
| `/change-password` | GET/POST | `change_password` | Change own password (verifies current) |
|
||||
|
||||
---
|
||||
|
||||
## 4. `admin` — Admin Panel (`/admin`)
|
||||
|
||||
**Security:** `@login_required` on all routes; `@admin_required` (custom decorator, `role == 'admin'`) on user-management.
|
||||
|
||||
### Users
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/` | GET | `admin_panel` | Stats, recent logs, users, storage |
|
||||
| `/user/create` | POST | `create_user` | Create user (username ≥3, password ≥6) |
|
||||
| `/user/<id>/role` | POST | `change_user_role` | Change role (not own) |
|
||||
| `/user/<id>/delete` | POST | `delete_user` | Delete user (not own) |
|
||||
| `/user/<id>/password` | POST | `reset_user_password` | Admin reset password |
|
||||
| `/users` | GET | `user_management` | User list page |
|
||||
| `/theme` | POST | `change_theme` | light/dark theme |
|
||||
|
||||
### Media housekeeping
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/leftover-media` | GET | `leftover_media` | Content not assigned to any playlist, by type + size |
|
||||
| `/delete-leftover-images` | POST | — | Bulk-delete leftover images (+ archive + PlayerEdit) |
|
||||
| `/delete-leftover-videos` | POST | — | Bulk-delete leftover videos |
|
||||
| `/delete-single-leftover/<id>` | POST | — | Delete one leftover item |
|
||||
|
||||
### System / dependencies
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/system/info` | GET | `system_info` | JSON: platform + psutil (CPU/mem/disk) |
|
||||
| `/dependencies` | GET | `dependencies` | Checks LibreOffice, Poppler, FFmpeg, emoji fonts |
|
||||
| `/install-libreoffice` | POST | — | `sudo -n install_libreoffice.sh` |
|
||||
| `/install-emoji-fonts` | POST | — | `sudo -n install_emoji_fonts.sh` |
|
||||
| `/logo/upload` | POST | `upload_logo` | Upload logo.png |
|
||||
| `/logs/clear` | POST | `clear_logs` | Delete all `ServerLog` rows |
|
||||
| `/customize-logos` | GET | `customize_logos` | Logo customization page |
|
||||
| `/upload-header-logo` | POST | — | Save header_logo.png |
|
||||
| `/upload-login-logo` | POST | — | Save login_logo.png |
|
||||
|
||||
### Editing users
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/editing-users` | GET | `manage_editing_users` | List `PlayerUser` + per-user edit counts |
|
||||
| `/editing-users/<id>/update` | POST | — | Rename editing user |
|
||||
| `/editing-users/<id>/delete` | POST | — | Delete editing user |
|
||||
|
||||
### HTTPS (Caddy)
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/https-config` | GET | `https_config` | HTTPS config page (auto-corrects DB flag if request is HTTPS) |
|
||||
| `/https-config/update` | POST | `update_https_config` | Validate + save config → regenerate Caddyfile → reload Caddy |
|
||||
| `/https-config/status` | GET | `https_config_status` | Config as JSON |
|
||||
|
||||
### Player build & deployment
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/build-player` | GET | `build_player` | Build page + staged-code status |
|
||||
| `/build-player` | POST | `build_player_action` | `build_files` / `save_config` / `build_and_config` → clone/refresh code, write `app_config.json`, persist `instance/player_build.json` |
|
||||
|
||||
---
|
||||
|
||||
## 5. `players` — Player Management (`/players`)
|
||||
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/` and `/list` | GET | `list` | Player list + live deployment-status polling data |
|
||||
| `/add` | GET/POST | `add_player` | Create player; optional **background SSH deployment** |
|
||||
| `/bulk/delete` | POST | — | JSON bulk delete |
|
||||
| `/bulk/assign-playlist` | POST | — | JSON bulk assign to playlist |
|
||||
| `/deployment-status` | GET | — | JSON deployment status for all players (polling) |
|
||||
| `/<int:player_id>` | GET | — | Redirect to `manage_player` |
|
||||
| `/<id>/edit` | GET/POST | — | Edit name/location (legacy) |
|
||||
| `/<id>/delete` | POST | — | Delete player + feedback |
|
||||
| `/<id>/regenerate-auth` | POST | — | New auth code |
|
||||
| `/<id>/manage` | GET/POST | `manage_player` | Manage page: `update_credentials` / `assign_playlist`; feedback, edits, status |
|
||||
| `/<id>/edited-media` | GET | `edited_media` | All edited media + user mapping |
|
||||
| `/<id>/edited-media-report` | GET | `edited_media_report` | Tabular report |
|
||||
| `/<id>/fullscreen` | GET | — | Player fullscreen (NO auth; optional `?auth=` check), cached playlist |
|
||||
| `/<id>/reorder` | POST | — | Legacy stub → 400 "use Playlists page" |
|
||||
| `/<id>/playlist/reorder` | POST | — | Legacy per-player reorder (JSON up/down) |
|
||||
| `/<id>/playlist/remove` | POST | — | Legacy removal (bumps `playlist_version`) |
|
||||
|
||||
Helper: `get_player_playlist(player_id)` — `@cache.memoize(300)` builds playlist dicts (`url`, `type`, `duration`, `position`, `muted`, `audio`).
|
||||
|
||||
---
|
||||
|
||||
## 6. `content` — Media Library & Modern Playlist Management (`/content`)
|
||||
|
||||
> This is the **primary workflow** in v2 (1,519 lines).
|
||||
|
||||
### Library & uploads
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/` | GET | `content_list` | Main page `content_list_new.html`: playlists, last-3 media, counts, players |
|
||||
| `/media-library` | GET | `media_library` | All media by type + edit counts |
|
||||
| `/media/<id>/delete` | POST | `delete_media` | Delete file + archive + edits; removes from playlists + bumps versions |
|
||||
| `/upload-media-page` | GET | `upload_media_page` | Upload page (playlist selector) |
|
||||
| `/upload-media` | POST | `upload_media` | Core upload: images optimized, videos validated, **PDF/PPTX → Full-HD PNGs**, large files → background threads |
|
||||
|
||||
### Web links
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/add-weblink` | POST | `add_weblink` | Create weblink Content (http/https validated), optional playlist add |
|
||||
| `/playlist/<id>/add-weblink` | POST | — | Create weblink + append to playlist |
|
||||
|
||||
### Playlist management
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/playlist/create` | POST | — | Create playlist (name, description, orientation) |
|
||||
| `/playlist/<id>/delete` | POST | — | Delete + unassign players |
|
||||
| `/playlist/<id>/manage` | GET | `manage_playlist_content` | Ordered content + available library (no weblinks) |
|
||||
| `/playlist/<id>/add-content` | POST | — | Add library content at next position |
|
||||
| `/playlist/<id>/remove-content/<content_id>` | POST | — | Remove content (deletes orphan weblinks) |
|
||||
| `/playlist/<id>/bulk-remove` | POST | — | JSON bulk remove |
|
||||
| `/playlist/<id>/reorder` | POST | — | JSON reorder (position = index) |
|
||||
| `/playlist/<id>/update-muted/<content_id>` | POST | — | Toggle muted |
|
||||
| `/playlist/<id>/update-edit-enabled/<content_id>` | POST | — | Toggle `edit_on_player_enabled` |
|
||||
| `/playlist/<id>/update-duration/<content_id>` | POST | — | Per-playlist duration (≥1s) |
|
||||
| `/player/<id>/assign-playlist` | POST | — | Assign/unassign player to playlist |
|
||||
|
||||
**All playlist mutations** call `Playlist.increment_version()` and `cache.clear()`.
|
||||
|
||||
### Processing helpers (module-level)
|
||||
`process_image_file`, `process_video_file_extended` (ffprobe), `process_pdf_file` (pdf2image @300 DPI → `_pageNNN.png`), `process_presentation_file` (LibreOffice → PDF → pdftoppm → Full-HD PNG), `create_fullhd_image`, `resize_image_to_fullhd`, `optimize_image_to_fullhd`, `process_file_in_background` (thread wrapper).
|
||||
|
||||
---
|
||||
|
||||
## 7. `playlist` — Legacy Per-Player Routes (`/playlist`)
|
||||
|
||||
Redirects/legacy — kept for compatibility:
|
||||
| Route | Methods | Purpose |
|
||||
|---|---|---|
|
||||
| `/<int:player_id>` | GET | Redirect to modern manage page |
|
||||
| `/<id>/add` | POST | Add content to player's playlist |
|
||||
| `/<id>/remove/<content_id>` | POST | Remove + renumber |
|
||||
| `/<id>/reorder` | POST | JSON reorder from `content_ids` |
|
||||
| `/<id>/update-duration/<content_id>` | POST | Update duration |
|
||||
| `/<id>/update-muted/<content_id>` | POST | Update muted |
|
||||
| `/<id>/clear` | POST | Clear all items |
|
||||
|
||||
---
|
||||
|
||||
## 8. `api` — Player-Facing REST + Deployment API (`/api`)
|
||||
|
||||
**Custom decorators:** `rate_limit(max_requests, window)` (in-memory, IP/Bearer keyed) and `verify_player_auth` (validates Bearer `auth_code` → sets `request.player`).
|
||||
|
||||
### Health & auth
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/health` | GET | `health_check` | API health (`version: 2.0.0`) |
|
||||
| `/certificate` | GET | — | TLS certificate test |
|
||||
| `/auth/player` | POST | `authenticate_player` | hostname + password/quickconnect → auth_code, playlist_id, orientation; sets online |
|
||||
| `/auth/verify` | POST | `verify_auth_code` | Verify auth code → player info |
|
||||
|
||||
### Playlist delivery
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/playlists` | GET | `get_playlist_by_quickconnect` | Kivy-compatible: `?hostname=&quickconnect_code=` → playlist + version + hashed quickconnect |
|
||||
| `/playlists/<player_id>` | GET | `get_player_playlist` | Bearer-authed fetch (cached) |
|
||||
| `/playlist-version/<player_id>` | GET | `get_playlist_version` | Lightweight refresh-poll version check |
|
||||
|
||||
**Payload keys the players expect** (from `get_cached_playlist`): `file_name`, `type`, `duration`, `position`, `url`, `description`, `edit_on_player_enabled`, `muted`, `audio` (`"on"`/`"off"`).
|
||||
|
||||
### Feedback & monitoring
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/player-feedback` | POST | `receive_player_feedback` | Status (playing/paused/error/restarting); infers player; **auto-marks deployment `deployed`** |
|
||||
| `/player-status/<player_id>` | GET | `get_player_status` | Online (5-min), latest feedback |
|
||||
| `/system-info` | GET | — | Counts: players online/total, groups, content, 24h logs |
|
||||
| `/content` | GET | — | List content with counts |
|
||||
| `/logs` | GET | `get_logs` | Query logs (`limit`/`level`/`since`) |
|
||||
|
||||
### Edited media
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/player-edit-media` | POST | `receive_edited_media` | Bearer-authed multipart upload of edited image + metadata; versionizes to `edited_media/<content_id>/`; records `PlayerEdit`; auto-creates `PlayerUser`; repoints `Content.filename`; bumps playlist version; clears cache |
|
||||
|
||||
### SSH deployment
|
||||
| Route | Methods | Endpoint | Purpose |
|
||||
|---|---|---|---|
|
||||
| `/deploy/test-ssh` | POST | `test_ssh_connection` | Test SSH (sshpass/ssh) |
|
||||
| `/deploy/player` | POST | `deploy_player` | Full remote deploy (`deploy_player_to_host`); API key = `sha256(name:hostname)[:32]` |
|
||||
|
||||
**Error handlers:** JSON 404 / 405 / 500.
|
||||
|
||||
---
|
||||
|
||||
## 9. API Call Flows (Mermaid)
|
||||
|
||||
### Player bootstrap (auth → playlist)
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant P as Player
|
||||
participant A as /api
|
||||
P->>A: POST /api/auth/player {hostname, password|quickconnect}
|
||||
A->>A: Player.authenticate()
|
||||
A-->>P: {auth_code, playlist_id, orientation}
|
||||
P->>A: GET /api/playlists/<id> (Bearer auth_code)
|
||||
A-->>P: playlist items + version
|
||||
loop refresh poll
|
||||
P->>A: GET /api/playlist-version/<id>
|
||||
A-->>P: {version}
|
||||
end
|
||||
```
|
||||
|
||||
### Edited media upload
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant P as Player
|
||||
participant A as /api
|
||||
participant DB as SQLite
|
||||
P->>A: POST /api/player-edit-media (Bearer, multipart image + metadata)
|
||||
A->>DB: locate Content (filename / edited_media/<id>/ regex / last PlayerEdit)
|
||||
A->>DB: move original → edited_media/<id>/original_*
|
||||
A->>DB: save new version + side-car JSON
|
||||
A->>DB: get_or_create PlayerUser (user_code)
|
||||
A->>DB: create PlayerEdit
|
||||
A->>DB: repoint Content.filename → latest edit (preserve original_filename)
|
||||
A->>DB: Playlist.increment_version()
|
||||
A-->>P: 200 OK
|
||||
```
|
||||
|
||||
### Deployment
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Admin as Admin UI
|
||||
participant S as Server (app)
|
||||
participant T as Background task
|
||||
participant H as Player host
|
||||
Admin->>S: add player / deploy
|
||||
S->>T: background_player_deployment()
|
||||
T->>H: sshpass test_ssh_connection()
|
||||
T->>H: rsync staged code (or git clone/pull)
|
||||
T->>H: write config/app_config.json
|
||||
T->>H: temp passwordless sudo → install.sh → start.sh
|
||||
T->>H: cleanup sudoers
|
||||
T-->>S: update player.deployment_status = deployed|failed
|
||||
Admin->>S: GET /players/deployment-status (poll)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
> Next: [06 · Utils & Services](06-utils-services.md)
|
||||
@@ -0,0 +1,140 @@
|
||||
# 06 · Utilities & Services
|
||||
|
||||
All shared services live in `app/utils/`. This document details each module, its main functions, and how they fit together.
|
||||
|
||||
---
|
||||
|
||||
## Overview Table
|
||||
|
||||
| Module | Community | Responsibility | Key symbols |
|
||||
|---|---|---|---|
|
||||
| `logger.py` | C0 | DB-backed audit logging | `log_action()`, `get_recent_logs()`, `clear_old_logs()` |
|
||||
| `group_player_management.py` | C0 | Group/player stats (legacy) | `get_player_status_info()`, `assign_player_to_group()`, `get_online_players_count()` |
|
||||
| `caddy_manager.py` | C1 | HTTPS Caddyfile generation | `CaddyConfigGenerator`, `write_caddyfile()`, `reload_caddy()` |
|
||||
| `background_tasks.py` | C10 | Async task execution | `run_background_task()`, `background_player_deployment()` |
|
||||
| `ssh_deploy.py` | C10 | Remote player provisioning | `deploy_player_to_host()`, `test_ssh_connection()`, `generate_player_config()` |
|
||||
| `player_build.py` | C11 | Stage player source code | `build_player_files()`, `write_base_config()`, `load/save_build_settings()` |
|
||||
| `pptx_converter.py` | C12 | PPTX → PDF → PNG | `pptx_to_pdf_libreoffice()`, `validate_pptx_file()`, `cleanup_libreoffice_processes()` |
|
||||
| `uploads.py` | C9 | Upload progress + file ops | `get/set/clear_upload_progress()`, `save_uploaded_file()`, `process_video_file()` |
|
||||
| `portal_sso.py` | C4 | SSO auto-login | `init_portal_sso()`, `_get_or_create_user()` |
|
||||
| `script_name_fix.py` | C4 | WSGI sub-path middleware | `ScriptNameFix` |
|
||||
| `nginx_config_reader.py` | C7 | Legacy nginx status parsing | `NginxConfigReader`, `get_nginx_status()` |
|
||||
|
||||
---
|
||||
|
||||
## 1. `logger.py` — Audit Logging ⭐ God Node (116 edges)
|
||||
|
||||
The most-connected module in the system. Every meaningful action across all blueprints is recorded here.
|
||||
|
||||
| Function | Purpose |
|
||||
|---|---|
|
||||
| `log_action(level, message)` | Create + commit a `ServerLog` row |
|
||||
| `log_info(message)` / `log_warning(message)` / `log_error(message)` | Convenience wrappers |
|
||||
| `get_recent_logs(limit, level)` | Query recent logs (used by dashboard/admin) |
|
||||
| `clear_old_logs(days)` | Housekeeping |
|
||||
|
||||
**Usage pattern:** admin actions (`reset_user_password`, `upload_header_logo`, `delete_editing_user`, `delete_playlist`, login failures, etc.) all funnel through `log_action()`.
|
||||
|
||||
---
|
||||
|
||||
## 2. `background_tasks.py` — Async Execution
|
||||
|
||||
| Function | Purpose |
|
||||
|---|---|
|
||||
| `run_background_task(func, *args)` | Spawns a **daemon thread** with a Flask app context pushed, then runs `func` |
|
||||
| `background_player_deployment(...)` | Runs SSH deployment in background; updates `player.deployment_status` (`deploying → deployed|failed`) |
|
||||
|
||||
Used by `players.add_player` to avoid blocking the HTTP request during long remote installs.
|
||||
|
||||
---
|
||||
|
||||
## 3. `ssh_deploy.py` — Remote Player Provisioning ⭐ (heat 0.614)
|
||||
|
||||
The deployment engine. Full pipeline implemented by `deploy_player_to_host(...)`:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["deploy_player_to_host()"] --> B["test_ssh_connection() (sshpass)"]
|
||||
B --> C["mkdir remote dir"]
|
||||
C --> D{"staged code present?"}
|
||||
D -- yes --> E["rsync pre-staged code"]
|
||||
D -- no --> F["git clone / pull"]
|
||||
E --> G["write config/app_config.json"]
|
||||
F --> G
|
||||
G --> H["temp passwordless sudo"]
|
||||
H --> I["run install.sh"]
|
||||
I --> J["run start.sh"]
|
||||
J --> K["cleanup sudoers"]
|
||||
K --> L["return steps[]"]
|
||||
```
|
||||
|
||||
Other helpers:
|
||||
- `get_local_player_code_status()` — inspect staged code state.
|
||||
- `detect_server_ip()` / `parse_server_address()` — resolve the server address players should reach.
|
||||
- `generate_player_config()` / `generate_app_config()` — write player's `config/app_config.json` with `server_ip`, `port`, `screen_name`, `quickconnect_key`, `orientation`, `use_https`, `verify_ssl`.
|
||||
|
||||
---
|
||||
|
||||
## 4. `caddy_manager.py` — HTTPS Automation ⭐ God Node
|
||||
|
||||
`CaddyConfigGenerator` produces the Caddyfile for the reverse proxy and reloads Caddy without restart.
|
||||
|
||||
| Method | Purpose |
|
||||
|---|---|
|
||||
| `generate_caddyfile(config)` | Pick template by mode: **HTTP-only** (`:80`), **domain** (Let's Encrypt), or **IP** (internal CA self-signed). Includes `reverse_proxy digiserver-app:5000`, 2 GB body limit, gzip, security headers |
|
||||
| `write_caddyfile(content, path=/etc/caddy/Caddyfile)` | Write to disk |
|
||||
| `reload_caddy()` | POST to Caddy admin API `http://caddy:2019/load` |
|
||||
|
||||
Triggered from `admin.update_https_config` after saving `HTTPSConfig`.
|
||||
|
||||
---
|
||||
|
||||
## 5. `player_build.py` — Staging Player Code
|
||||
|
||||
| Function | Purpose |
|
||||
|---|---|
|
||||
| `build_player_files(dir, repo_url, branch)` | Clone or `fetch + reset` the player repository into `PLAYER_CODE_DIR` |
|
||||
| `write_base_config(...)` | Write `config/app_config.json` (blank screen / quickconnect) |
|
||||
| `load_build_settings()` / `save_build_settings()` | JSON at `instance/player_build.json` |
|
||||
| `get_player_server_settings` / `make_build_record` / `get_short_head` | Build metadata helpers |
|
||||
|
||||
---
|
||||
|
||||
## 6. `uploads.py` — Upload & Media Processing
|
||||
|
||||
| Function | Purpose |
|
||||
|---|---|
|
||||
| `get_upload_progress` / `set_upload_progress` / `clear_upload_progress` | In-memory per-file progress (for the upload page) |
|
||||
| `save_uploaded_file` | Save a multipart upload |
|
||||
| `process_video_file` | FFmpeg → H.264 main, 30 fps, ≤1080p, faststart |
|
||||
| `process_pdf_file` | PDF processing (stub — real logic in content.py) |
|
||||
| `get_file_size` / `delete_file` | FS helpers |
|
||||
|
||||
---
|
||||
|
||||
## 7. `pptx_converter.py` — LibreOffice Integration
|
||||
|
||||
| Function | Purpose |
|
||||
|---|---|
|
||||
| `pptx_to_pdf_libreoffice(pptx_path, output_dir)` | Headless LibreOffice → PDF (300 s timeout) |
|
||||
| `validate_pptx_file()` | Validate a file is a real PPTX |
|
||||
| `cleanup_libreoffice_processes()` | `pkill soffice` — clean hanging processes |
|
||||
|
||||
Used by the upload pipeline: **PPTX → PDF → PNG slides (Full HD)**.
|
||||
|
||||
---
|
||||
|
||||
## 8. `portal_sso.py` & `script_name_fix.py` — Gateway Integration
|
||||
|
||||
- **`portal_sso.py`** — `before_request` reads `X-Auth-Username`/`X-Auth-Role` from the umbrella nginx and auto-logs-in the local user (creating it on first arrival). See [04 · Application Core §11](04-application-core.md#11-portal-sso--apputilsportal_sso_py).
|
||||
- **`script_name_fix.py`** — WSGI middleware mapping `X-Script-Name` → `SCRIPT_NAME` so `url_for()` is correct behind a path-prefixed gateway.
|
||||
|
||||
---
|
||||
|
||||
## 9. `nginx_config_reader.py` — Legacy (informational)
|
||||
|
||||
`NginxConfigReader` parses an `nginx.conf` and reports `ssl_enabled`, ports, upstreams, `server_names`, `ssl_protocols`, `client_max_body_size`, `gzip`. **Legacy** — the current reverse proxy is Caddy; retained for reference and the old deployment stack.
|
||||
|
||||
---
|
||||
|
||||
> Next: [07 · Deployment](07-deployment.md)
|
||||
@@ -0,0 +1,162 @@
|
||||
# 07 · Deployment
|
||||
|
||||
DigiServer v2 runs as a **Docker Compose** stack (app + Caddy) with optional remote SSH provisioning of players.
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Host["Docker Host"]
|
||||
subgraph Net["digiserver-network"]
|
||||
APP["digiserver-app\nFlask + Gunicorn :5000"]
|
||||
CAD["caddy:2-alpine\n:80 / :443"]
|
||||
end
|
||||
VOL1["./data/instance → /app/instance\n(SQLite DBs)"]
|
||||
VOL2["./data/uploads → /app/app/static/uploads"]
|
||||
VOL3["./data/Caddyfile → /etc/caddy/Caddyfile"]
|
||||
VOL4["./data/caddy-data → /data\n./data/caddy-config → /config"]
|
||||
end
|
||||
Browser["Browser"] --> CAD
|
||||
CAD --> APP
|
||||
APP --> VOL1
|
||||
APP --> VOL2
|
||||
CAD --> VOL3
|
||||
CAD --> VOL4
|
||||
APP -.SSH/rsync.-> Players["Remote player hosts"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. `docker-compose.yml`
|
||||
|
||||
### Service `digiserver-app`
|
||||
- **Build:** `.` (Dockerfile, `python:3.13-slim`)
|
||||
- **Ports:** `5000:5000` (direct dev access; also `expose: 5000`)
|
||||
- **Volumes:** `./data/instance:/app/instance`, `./data/uploads:/app/app/static/uploads`
|
||||
- **Env:** `FLASK_ENV=production`, `SECRET_KEY`, `ADMIN_USERNAME`, `ADMIN_PASSWORD`
|
||||
- **Healthcheck:** HTTP GET `http://localhost:5000/` (30s interval, 40s start)
|
||||
- **Restart:** `unless-stopped`
|
||||
|
||||
### Service `caddy`
|
||||
- **Image:** `caddy:2-alpine`
|
||||
- **Ports:** `8080:80`, `8443:443`
|
||||
- **Volumes:** `./data/Caddyfile:/etc/caddy/Caddyfile:rw`, `./data/caddy-data:/data`, `./data/caddy-config:/config`, `./data/caddy-logs:/var/log/caddy`
|
||||
- **Depends on:** app (service started)
|
||||
- **Healthcheck:** `wget` on port 80
|
||||
|
||||
---
|
||||
|
||||
## 3. Dockerfile
|
||||
|
||||
| Stage | Content |
|
||||
|---|---|
|
||||
| Base | `python:3.13-slim` |
|
||||
| System deps | `poppler-utils`, `ffmpeg`, `libmagic1`, `sudo`, `fonts-noto-color-emoji`, `LibreOffice` (core/impress/writer), `sshpass`, `openssh-client`, `rsync`, `git` |
|
||||
| Python | `COPY requirements.txt` → `pip install` (cached layer), then `COPY . .` |
|
||||
| App config | `FLASK_APP=app.app:create_app`, `FLASK_ENV=production`, `EXPOSE 5000` |
|
||||
| User | Non-root `appuser` (UID 1000) with **passwordless sudo** limited to `apt-get`, `install_libreoffice.sh`, `install_emoji_fonts.sh` |
|
||||
| Runtime | `HEALTHCHECK` (HTTP 5000), `ENTRYPOINT /docker-entrypoint.sh` |
|
||||
|
||||
---
|
||||
|
||||
## 4. `docker-entrypoint.sh`
|
||||
|
||||
1. Create `/app/instance` and `/app/app/static/uploads`.
|
||||
2. If `dashboard.db` is missing: create app + `db.create_all()`, then create/update the admin user from `ADMIN_USERNAME` / `ADMIN_PASSWORD`.
|
||||
3. Start **Gunicorn**: `--bind 0.0.0.0:5000 --workers 4 --timeout 120 app.app:create_app('production')`.
|
||||
|
||||
---
|
||||
|
||||
## 5. `deploy.sh` (One-Shot Deployment)
|
||||
|
||||
```
|
||||
1. Validate compose + project; create data/ subdirs; copy nginx configs
|
||||
2. docker compose up -d + verify containers "Up"
|
||||
3. Run migration scripts (add_https_config_table, add_player_user_table,
|
||||
add_email_to_https_config, migrate_player_user_global,
|
||||
add_original_filename_to_content)
|
||||
4. Run /app/https_manager.py enable <hostname> <domain> <email> <ip> <port>
|
||||
⚠ https_manager.py is NOT in the current repo — this step needs attention
|
||||
5. Verify DB tables via SQLAlchemy inspector
|
||||
6. caddy validate + https_manager.py status; print access URLs + default creds
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. HTTPS Setup (Caddy)
|
||||
|
||||
HTTPS is configured through the **Admin → HTTPS Configuration** page, which:
|
||||
|
||||
1. Saves `HTTPSConfig` (hostname, domain, IP, email, port, enabled).
|
||||
2. `CaddyConfigGenerator.generate_caddyfile(config)` picks a template:
|
||||
- **HTTP-only** → `:80` reverse proxy
|
||||
- **Domain** → automatic Let's Encrypt
|
||||
- **IP** → internal CA self-signed
|
||||
3. Writes `/etc/caddy/Caddyfile` and reloads Caddy via `POST http://caddy:2019/load`.
|
||||
|
||||
The current `data/Caddyfile` (HTTP mode) includes: admin API on `0.0.0.0:2019`, `:80` block → `digiserver-app:5000`, 2 GB body limit, gzip, security headers, access log.
|
||||
|
||||
---
|
||||
|
||||
## 7. `verify-deployment.sh` — Pre/Post-Deployment Checks
|
||||
|
||||
Sections checked (pass/fail/warn counters):
|
||||
- git status
|
||||
- `.env` / `.env.example`
|
||||
- Docker + Compose versions + `compose config` syntax
|
||||
- Dockerfile best practices (HEALTHCHECK, non-root, slim base)
|
||||
- `requirements.txt` critical packages + versions
|
||||
- migrations directory
|
||||
- **SSL cert expiry** (openssl)
|
||||
- Flask config (`ProductionConfig`, `SESSION_COOKIE_SECURE`)
|
||||
- nginx.conf checks
|
||||
- runtime container health
|
||||
- security best practices
|
||||
|
||||
> ⚠ Note: the script still references `docker-compose` (v1) and `digiserver-nginx` — the current stack uses Compose v2 + Caddy.
|
||||
|
||||
---
|
||||
|
||||
## 8. Player Deployment Pipeline
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant U as Admin
|
||||
participant A as App
|
||||
participant B as Background
|
||||
participant H as Player host
|
||||
U->>A: Add player (name, hostname, password/quickconnect)
|
||||
A->>B: background_player_deployment()
|
||||
B->>B: build/refresh staged code (PLAYER_CODE_DIR)
|
||||
B->>H: sshpass + ssh test
|
||||
B->>H: rsync code (or git clone/pull fallback)
|
||||
B->>H: write config/app_config.json (server_ip, quickconnect, https...)
|
||||
B->>H: temporary passwordless sudo
|
||||
B->>H: ./install.sh && ./start.sh
|
||||
B->>H: remove temp sudoers
|
||||
B-->>A: player.deployment_status = 'deployed' | 'failed'
|
||||
U->>A: poll /players/deployment-status
|
||||
```
|
||||
|
||||
Statuses: `pending → deploying → deployed | failed`, with `last_deployment_at/status/message` persisted on the player row. Receiving player feedback also auto-marks deployment `deployed`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Ports & Volumes Summary
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| App port (container) | 5000 |
|
||||
| App port (host, dev) | 5000 |
|
||||
| Caddy HTTP | 8080 → 80 |
|
||||
| Caddy HTTPS | 8443 → 443 |
|
||||
| Caddy admin API | 2019 (container) |
|
||||
| DB volume | `./data/instance` |
|
||||
| Uploads volume | `./data/uploads` |
|
||||
| Caddy data/config/logs | `./data/caddy-*` |
|
||||
|
||||
---
|
||||
|
||||
> Next: [08 · Workflows](08-workflows.md)
|
||||
@@ -0,0 +1,150 @@
|
||||
# 08 · End-to-End Workflows
|
||||
|
||||
This document traces the main user journeys through the system, tying together blueprints, models, utils, and the Graphify graph.
|
||||
|
||||
---
|
||||
|
||||
## 1. Content Upload → Playlist → Player
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
U["Admin uploads media"] --> UF["/content/upload-media"]
|
||||
UF -->|image| IMG["optimize_image_to_fullhd"]
|
||||
UF -->|video| VID["validate via ffprobe"]
|
||||
UF -->|pdf| PDF["pdf2image @300dpi → page PNGs"]
|
||||
UF -->|pptx| PPT["LibreOffice → PDF → PNG slides"]
|
||||
UF -->|large file| BG["process_file_in_background (thread)"]
|
||||
UF --> DB[("Content row")]
|
||||
DB --> PLA["/content/playlist/<id>/add-content"]
|
||||
PLA --> PLC["playlist_content (position, duration, muted, edit_on_player_enabled)"]
|
||||
PLC --> VER["Playlist.increment_version() + cache.clear()"]
|
||||
VER --> API["/api/playlists/<id>"]
|
||||
API --> P["Player fetches new playlist"]
|
||||
```
|
||||
|
||||
**Key files:** `app/blueprints/content.py`, `app/utils/uploads.py`, `app/utils/pptx_converter.py`, `app/models/content.py`, `app/models/playlist.py`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Playlist Synchronization (the "version" mechanism)
|
||||
|
||||
Every playlist mutation bumps `Playlist.version`. Players detect changes by polling:
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant U as Admin UI
|
||||
participant App
|
||||
participant DB as SQLite
|
||||
participant P as Player
|
||||
|
||||
U->>App: reorder / mute / duration / add / remove
|
||||
App->>DB: mutate playlist_content
|
||||
App->>DB: playlist.increment_version()
|
||||
App->>App: cache.clear()
|
||||
loop while player runs
|
||||
P->>App: GET /api/playlist-version/<player_id>
|
||||
App-->>P: {version: N}
|
||||
alt version changed
|
||||
P->>App: GET /api/playlists/<player_id>
|
||||
App-->>P: new playlist payload
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Playlist payload items** include: `file_name`, `type`, `duration`, `position`, `url` (weblink or static file), `description`, `edit_on_player_enabled`, `muted`, `audio`.
|
||||
|
||||
---
|
||||
|
||||
## 3. On-Player Media Editing Pipeline ⭐ (heat 0.620)
|
||||
|
||||
When a signage operator edits an image directly on the player device:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Player posts edited image"] --> B["POST /api/player-edit-media (Bearer + multipart)"]
|
||||
B --> C{"Locate Content"}
|
||||
C -->|by filename| D["match filename"]
|
||||
C -->|edited_media regex| E["match edited_media/<id>/"]
|
||||
C -->|fallback| F["last PlayerEdit"]
|
||||
D/E/F --> G["Move original → edited_media/<content_id>/original_*"]
|
||||
G --> H["Save new version + side-car metadata JSON"]
|
||||
H --> I{"PlayerUser exists for user_code?"}
|
||||
I -- no --> J["auto-create PlayerUser"]
|
||||
I -- yes --> K["reuse"]
|
||||
J/K --> L["Create PlayerEdit (version, original_name, new_name...)"]
|
||||
L --> M["Repoint Content.filename → latest edit"]
|
||||
M --> N["Preserve original_filename"]
|
||||
N --> O["Playlist.increment_version() + cache.clear()"]
|
||||
O --> P["Player sees updated media on refresh"]
|
||||
```
|
||||
|
||||
**Why this is a god node:** `PlayerEdit` (99 edges) and `PlayerUser` (74 edges) make this the most-connected data flow in the system. See `app/models/player_edit.py`, `app/models/player_user.py`, `app/blueprints/api.py`.
|
||||
|
||||
---
|
||||
|
||||
## 4. HTTPS Configuration (Caddy)
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Admin → /admin/https-config"] --> B["POST /admin/https-config/update"]
|
||||
B --> C["Validate + save HTTPSConfig (email, domain, ip, port)"]
|
||||
C --> D["CaddyConfigGenerator.generate_caddyfile()"]
|
||||
D --> E{"Mode"}
|
||||
E -->|HTTP| F[":80 reverse proxy"]
|
||||
E -->|domain| G["Let's Encrypt automatic HTTPS"]
|
||||
E -->|IP| H["Internal CA self-signed"]
|
||||
F/G/H --> I["write_caddyfile(/etc/caddy/Caddyfile)"]
|
||||
I --> J["POST http://caddy:2019/load (hot reload)"]
|
||||
J --> K["HTTPS live without restart"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. User Management & Auditing
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Admin panel"] --> B["/admin/user/create"]
|
||||
A --> C["/admin/user/<id>/role"]
|
||||
A --> D["/admin/user/<id>/password"]
|
||||
A --> E["/admin/user/<id>/delete"]
|
||||
B/C/D/E --> L["log_action(level, message)"]
|
||||
L --> S[("server_log")]
|
||||
S --> F["dashboard / admin recent logs"]
|
||||
S --> G["/api/logs (limit, level, since)"]
|
||||
```
|
||||
|
||||
**Roles:** `admin` (full), `user` (manage content/players), `viewer` (read-only dashboard).
|
||||
|
||||
---
|
||||
|
||||
## 6. Portal SSO (optional corporate gateway)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant B as Browser
|
||||
participant G as Umbrella nginx gateway
|
||||
participant A as DigiServer
|
||||
B->>G: request (authenticated by portal)
|
||||
G->>A: proxy + headers X-Auth-Username, X-Auth-Role
|
||||
A->>A: init_portal_sso before_request
|
||||
A->>A: _get_or_create_user(username, role)
|
||||
A-->>B: dashboard (auto logged-in)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Media Housekeeping (Leftovers)
|
||||
|
||||
Content not assigned to any playlist is flagged as "leftover":
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
L["/admin/leftover-media"] --> T["grouped by type: image/video/pdf/pptx + sizes"]
|
||||
T --> D["/admin/delete-leftover-images|videos"]
|
||||
D --> X["delete file + edited_media archive + PlayerEdit rows"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
> Next: [09 · Legacy & Migrations](09-legacy-and-migrations.md)
|
||||
@@ -0,0 +1,64 @@
|
||||
# 09 · Migrations & Legacy Components
|
||||
|
||||
---
|
||||
|
||||
## 1. Migrations
|
||||
|
||||
Migrations are **standalone Python scripts** (run via `docker compose exec ... python`, **not** Alembic revisions — there is no `migrations/versions/`).
|
||||
|
||||
| Script | Table(s) | What it does |
|
||||
|---|---|---|
|
||||
| `add_https_config_table.py` | `https_config` | Creates table via `db.create_all()` |
|
||||
| `add_player_user_table.py` | `player_user` | Creates table via `db.create_all()` |
|
||||
| `add_email_to_https_config.py` | `https_config` | `ALTER TABLE ... ADD COLUMN email VARCHAR(255)` (idempotent) |
|
||||
| `add_deployment_fields_to_player.py` | `player` | Adds `deployment_status`, `last_deployment_at`, `last_deployment_status`, `last_deployment_message` (idempotent) |
|
||||
| `add_url_to_content.py` | `content` | `ALTER TABLE ... ADD COLUMN url VARCHAR(2048)` (weblink support) |
|
||||
| `add_original_filename_to_content.py` | `content` | Adds `original_filename` then **backfills**: from first (v1) `PlayerEdit.original_name` for edited content, else `filename` |
|
||||
| `migrate_player_user_global.py` | `player_user` | **Drop & recreate**: removes old `player_id` FK, makes `user_code` globally unique, `user_name` nullable |
|
||||
|
||||
**Order (from `deploy.sh`):**
|
||||
```
|
||||
add_https_config_table.py
|
||||
add_player_user_table.py
|
||||
add_email_to_https_config.py
|
||||
migrate_player_user_global.py
|
||||
add_original_filename_to_content.py
|
||||
```
|
||||
|
||||
> ⚠ `deploy.sh` also calls `/app/https_manager.py` — **that file is not present in this repo**, so that deployment step needs attention.
|
||||
|
||||
---
|
||||
|
||||
## 2. Legacy / Archived Components
|
||||
|
||||
| Component | Status | Notes |
|
||||
|---|---|---|
|
||||
| `app/blueprints/content_old.py` | **Dead code** | Legacy per-player content routes (`/`, `/upload`, `/<id>/edit`, `/bulk/delete`, `/upload-progress`, `/preview`, `/statistics`, `/check-duplicates`, `/<id>/groups`). Not imported by `create_app`. |
|
||||
| `app/blueprints/playlist.py` | **Active but legacy** | Per-player playlist routes kept as redirects to the modern content workflow. |
|
||||
| `Group` model + group routes | **Archived** | `Player` no longer has `group_id`; group routes commented out; `group_player_management.py` and `group_content` association remain for reference. |
|
||||
| `nginx` stack | **Replaced by Caddy** | `data/nginx.conf`, `data/nginx-custom-domains.conf`, `data/nginx-logs/`, `data/nginx-ssl/` retained. `utils/nginx_config_reader.py` still parses it. |
|
||||
| `https_manager.py` | **Missing** | Referenced by `deploy.sh` but not in repo — likely merged into `CaddyConfigGenerator`. |
|
||||
| `old_code_documentation/` | **Archive** | Full legacy docs, old scripts (`blueprint_groups.py`, `add_muted_column.py`, `fix_player_user_schema.py`, `test_edit_media_*.py`, `check_fix_player.py`, `migrate_add_edit_enabled.py`), deployment guides, HTTPS analysis, player analysis. |
|
||||
| `QUICK_DEPLOYMENT.md`, `deployment-commands-reference.sh` | **Active reference** | Manual deployment notes. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommended Cleanup (optional)
|
||||
|
||||
- Remove `content_old.py` and `old_code_documentation/*.py` scripts that are no longer needed (keep the `.md` docs).
|
||||
- Resolve the missing `https_manager.py` in `deploy.sh` (use `CaddyConfigGenerator` equivalents).
|
||||
- Update `verify-deployment.sh` to reference Compose v2 and Caddy instead of `docker-compose`/nginx.
|
||||
|
||||
---
|
||||
|
||||
## 📚 Back to Index
|
||||
|
||||
- [README · Documentation Home](README.md)
|
||||
- [01 · Architecture](01-architecture.md)
|
||||
- [02 · Knowledge Graph](02-knowledge-graph.md)
|
||||
- [03 · Data Model](03-data-model.md)
|
||||
- [04 · Application Core](04-application-core.md)
|
||||
- [05 · Blueprints & API](05-blueprints-api.md)
|
||||
- [06 · Utils & Services](06-utils-services.md)
|
||||
- [07 · Deployment](07-deployment.md)
|
||||
- [08 · Workflows](08-workflows.md)
|
||||
+108
@@ -0,0 +1,108 @@
|
||||
# DigiServer v2 — Project Documentation
|
||||
|
||||
> **Comprehensive technical documentation generated with [Graphify](https://marketplace.visualstudio.com/items?itemName=anytechiestudio.graphify-vscode)** — an interactive knowledge-graph engine for AI-assisted coding.
|
||||
|
||||
**Version:** 2.0.0 · **Build date:** 2025-11-12 · **Generated:** 2026-08-16
|
||||
|
||||
DigiServer v2 is a **digital signage management server** built with **Flask**. It manages media content (images, videos, PDFs, presentations), organizes it into **playlists**, assigns **players** (physical signage displays), and remotely **deploys and monitors** those players over SSH — including HTTPS/SSL provisioning via Caddy and an on-player photo-editing pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 🗂 Documentation Index
|
||||
|
||||
| Document | Purpose |
|
||||
|---|---|
|
||||
| [**01-architecture.md**](01-architecture.md) | System overview, layers, component map, architecture diagrams |
|
||||
| [**02-knowledge-graph.md**](02-knowledge-graph.md) | Graphify graph: 41 communities, god nodes, and how to use the visualizer |
|
||||
| [**03-data-model.md**](03-data-model.md) | Database schema: all 10 tables, relationships, ER diagrams |
|
||||
| [**04-application-core.md**](04-application-core.md) | App factory, config, extensions, middleware, CLI, templates |
|
||||
| [**05-blueprints-api.md**](05-blueprints-api.md) | All 7 blueprints + full REST API reference |
|
||||
| [**06-utils-services.md**](06-utils-services.md) | Background tasks, SSH deploy, Caddy manager, player build, SSO, uploads |
|
||||
| [**07-deployment.md**](07-deployment.md) | Docker, Caddy, deployment pipeline, HTTPS setup, verify script |
|
||||
| [**08-workflows.md**](08-workflows.md) | End-to-end flows: upload, playlists, player edits, deployment, sync |
|
||||
| [**09-legacy-and-migrations.md**](09-legacy-and-migrations.md) | Migration scripts, deprecated/archived components |
|
||||
|
||||
**Auto-generated Graphify artifacts** (regenerate anytime — see [02-knowledge-graph.md](02-knowledge-graph.md)):
|
||||
|
||||
```
|
||||
graphify-out/
|
||||
├── graph.html ← Interactive visualizer (open in browser)
|
||||
├── graph.json ← Raw graph data (631 nodes / 1162 edges)
|
||||
├── GRAPH_REPORT.md ← God nodes, communities, knowledge gaps
|
||||
├── COMPASS.md ← Token-optimized architecture summary
|
||||
├── DOMAINS.md ← Community → domain mapping
|
||||
├── intelligence.json ← AI-detected insights (god nodes, surprises)
|
||||
├── graph.compact.txt ← Compact graph dump for LLM context
|
||||
└── wiki/ ← Per-community + per-god-node Markdown articles
|
||||
├── index.md
|
||||
├── Community_0.md … Community_40.md
|
||||
├── CaddyConfigGenerator.md, Content.md, HTTPSConfig.md,
|
||||
├── PlayerEdit.md, PlayerUser.md, Playlist.md, User.md,
|
||||
├── log_action().md, create_app().md …
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Quick Facts
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| Framework | Flask 3.1 + SQLAlchemy 2.0 (SQLite) + Gunicorn |
|
||||
| Application type | Digital signage content/playlist/player management |
|
||||
| Graph size | **631 nodes · 1162 edges · 41 communities** |
|
||||
| Blueprints | 7 active (`main`, `auth`, `admin`, `players`, `content`, `playlist`, `api`) |
|
||||
| Database tables | 10 (`user`, `player`, `player_edit`, `player_feedback`, `player_user`, `content`, `group`, `playlist`, `server_log`, `https_config`) |
|
||||
| Reverse proxy | Caddy 2 (automatic HTTPS / Let's Encrypt) |
|
||||
| Deployment | Docker Compose (app + Caddy) + remote SSH player provisioning |
|
||||
| Key externals | LibreOffice, Poppler (pdf2image), FFmpeg, sshpass/rsync |
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Core Abstractions (Graphify "God Nodes")
|
||||
|
||||
Detected automatically from graph centrality:
|
||||
|
||||
1. `log_action()` — 116 edges · DB-backed audit logging (`app/utils/logger.py`)
|
||||
2. `PlayerEdit` — 99 edges · on-player edit records (`app/models/player_edit.py`)
|
||||
3. `PlayerUser` — 74 edges · player edit user mapping (`app/models/player_user.py`)
|
||||
4. `Playlist` — 33 edges · playlist + versioning (`app/models/playlist.py`)
|
||||
5. `CaddyConfigGenerator` — 31 edges · HTTPS Caddyfile generation (`app/utils/caddy_manager.py`)
|
||||
6. `HTTPSConfig` — 30 edges · HTTPS settings model (`app/models/https_config.py`)
|
||||
7. `Content` — 24 edges · media content model (`app/models/content.py`)
|
||||
8. `User` — 19 edges · admin/user/viewer accounts (`app/models/user.py`)
|
||||
9. `create_app()` — 13 edges · application factory (`app/app.py`)
|
||||
10. `app/models/__init__.py` — 13 edges · models package export
|
||||
|
||||
---
|
||||
|
||||
## 🧭 Quick Navigation
|
||||
|
||||
### The 7 Active Blueprints
|
||||
|
||||
| Blueprint | Prefix | Responsibility |
|
||||
|---|---|---|
|
||||
| `main` | `/` | Dashboard + health check |
|
||||
| `auth` | `/` | Login / logout / register / change password |
|
||||
| `admin` | `/admin` | Users, HTTPS config, player build, logos, logs, leftovers |
|
||||
| `players` | `/players` | Player CRUD, manage, edited media, deployment status |
|
||||
| `content` | `/content` | Media library, uploads, playlist management (modern) |
|
||||
| `playlist` | `/playlist` | Legacy per-player playlist routes (redirects) |
|
||||
| `api` | `/api` | Player-facing REST API (auth, playlists, feedback, edits, deploy) |
|
||||
|
||||
### Data Model at a Glance
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
user ||--o{ server_log : ""
|
||||
player ||--o{ player_feedback : "cascade"
|
||||
player ||--o{ player_edit : "cascade"
|
||||
player }o--o| playlist : "assigned"
|
||||
content ||--o{ player_edit : "cascade"
|
||||
content ||--o{ player_feedback : ""
|
||||
playlist ||--o{ content : "playlist_content (M2M)"
|
||||
content }o--o{ group : "group_content (M2M)"
|
||||
player_user ||--o{ player_edit : "user_code"
|
||||
https_config ||--|| https_config : "single row config"
|
||||
```
|
||||
|
||||
> **Proceed to [01-architecture.md](01-architecture.md)** for the full system overview and diagrams.
|
||||
Reference in New Issue
Block a user