ske087/digiserver-v2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Sanitize codebase, reorganize docs, and add missing deploy files

Browse Source 
Remove dead code identified in docs/SANITIZATION-REVIEW.md:
- app/blueprints/content_old.py and app/blueprints/playlist.py
- app/models/group.py, app/utils/nginx_config_reader.py
- orphaned templates (content_list, edit_content, upload_content,
  player_page) and the related group/Template references

Result: 6 blueprints, 82 routes, no dead modules or orphan templates.

Add files that deploy.sh and docker-entrypoint.sh already require but
which were never tracked:
- https_manager.py       (referenced by deploy.sh, migrate_network.sh,
                          docker-entrypoint.sh)
- Caddyfile.example      (seeded by deploy.sh; its absence aborts deploy)

Relocate generated Graphify artifacts from graphify-out/ to
docs/graphify-out/ (110 files, no content change) and archive the
superseded docs under docs/.

Ignore hygiene:
- ignore ad-hoc .env backups (.env.bak*) — they contain live secrets
- keep the pre-sanitization snapshots (docs/legacy code/,
  docs/old_code_documentation/) on disk but out of the repo

Fix .env.example: drop a duplicated config block, genericize the
hardcoded host IP, and document HOSTNAME_INTERNAL.
This commit is contained in:
scheianu
2026-09-11 12:18:34 +03:00
parent 1c5186463a
commit 46602f1933
226 changed files with 3999 additions and 15737 deletions
Download Patch File Download Diff File Expand all files Collapse all files
https_manager.py
+390
View File
@@ -0,0 +1,390 @@
#!/usr/bin/env python
"""HTTPS management CLI.
Invoked by ``deploy.sh`` after the containers are healthy:
python /app/https_manager.py enable <hostname> <domain> <email> <ip> [port]
python /app/https_manager.py disable
python /app/https_manager.py status
python /app/https_manager.py bootstrap # from environment variables
This is a thin wrapper around the very same code path the Admin UI uses
(``HTTPSConfig`` + ``CaddyConfigGenerator``), so a command-line deployment and a
UI-driven change always produce an identical Caddyfile.
Mode selection (delegated to ``CaddyConfigGenerator``):
* ``--domain`` given → Let's Encrypt (needs a publicly resolvable name)
* IP only, ``--domain`` empty → Caddy **internal CA** (no DNS, no ACME).
Correct for intranet servers.
* ``--no-https`` → HTTP only.
Exit codes:
0 success
1 invalid arguments / configuration error
2 configuration applied, but Caddy could not be reloaded
"""
from __future__ import annotations
import argparse
import os
import ssl
import sys
import time
import urllib.error
import urllib.request
sys.path.insert(0, '/app')
from app.app import create_app # noqa: E402
from app.models.https_config import HTTPSConfig # noqa: E402
from app.utils.caddy_manager import CaddyConfigGenerator # noqa: E402
from app.utils.logger import log_action # noqa: E402
CADDYFILE_PATH = '/etc/caddy/Caddyfile'
def _truthy(value: str | None, default: bool = True) -> bool:
"""Interpret a string env var as a boolean."""
if value is None or value == '':
return default
return value.strip().lower() in ('1', 'true', 'yes', 'on')
def verify_https(hostname: str, port: int = 443, timeout: float = 6.0,
attempts: int = 3) -> tuple[bool, str]:
"""Check that HTTPS actually answers on *hostname*:*port*.
The certificate is deliberately **not** validated: for an intranet name we
expect Caddy's internal CA, which is not in this container's trust store.
What matters is that the TLS listener is up and serving.
Retries a few times because Caddy may still be obtaining a certificate.
Args:
hostname: Name or IP to connect to (e.g. the host IP or domain).
port: TLS port.
timeout: Per-attempt timeout in seconds.
attempts: Number of attempts before giving up.
Returns:
``(ok, detail)`` — ``detail`` is a human-readable reason.
"""
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
last = 'no attempt made'
for attempt in range(1, attempts + 1):
url = f"https://{hostname}:{port}/api/health"
try:
req = urllib.request.Request(url, method='GET')
with urllib.request.urlopen(req, timeout=timeout, context=ctx) as r:
if r.status < 500:
return True, f'HTTP {r.status} from {url}'
last = f'HTTP {r.status} from {url}'
except urllib.error.HTTPError as e:
# An HTTP error still means TLS terminated successfully.
if e.code < 500:
return True, f'HTTP {e.code} from {url}'
last = f'HTTP {e.code} from {url}'
except Exception as e: # noqa: BLE001
last = f'{type(e).__name__}: {e}'
if attempt < attempts:
time.sleep(2)
return False, last
def bootstrap_from_env(app) -> int:
"""Configure HTTPS from environment variables at container startup.
Reads ``HOSTNAME_INTERNAL`` and ``HOST_IP``. Both must be set for HTTPS to
be provisioned; when either is missing the app deliberately stays on the
plain-HTTP fallback so it is always reachable, and HTTPS can be enabled
later from Admin → HTTPS Configuration (which reloads Caddy live).
Ownership / precedence
----------------------
The admin UI is the ongoing source of truth. To avoid the environment
silently overwriting an admin's change on every restart, the bootstrap only
applies while the environment still *owns* the configuration:
* No config yet → apply from env (first deploy).
* Last written by the env bootstrap → apply from env (env still owns it,
so changing ``HOST_IP`` and redeploying works).
* Last written by a real user → SKIP; the admin's settings are kept.
Provenance is taken from ``HTTPSConfig.updated_by`` (``'deploy.sh'`` marks an
env/CLI write), so no schema change is needed.
Idempotent: re-running with an unchanged environment rewrites the same
Caddyfile and reloads Caddy, which is harmless.
Returns one of the standard exit codes (0/1/2).
"""
hostname = (os.getenv('HOSTNAME_INTERNAL') or '').strip()
ip_address = (os.getenv('HOST_IP') or '').strip()
if not hostname or not ip_address:
missing = [n for n, v in (('HOSTNAME_INTERNAL', hostname),
('HOST_IP', ip_address)) if not v]
print(f'[https] {" and ".join(missing)} not set — staying on the '
f'plain-HTTP fallback.')
print('[https] Enable HTTPS later from Admin → HTTPS Configuration.')
return 0
# Respect an explicit admin change: only seed while the env owns the config.
with app.app_context():
existing = HTTPSConfig.get_config()
if existing is not None and existing.updated_by not in (None, '', 'deploy.sh'):
print(f'[https] HTTPS already configured by "{existing.updated_by}" — '
f'environment bootstrap skipped so the admin setting is kept.')
return 0
domain = (os.getenv('DOMAIN') or '').strip()
email = (os.getenv('SSL_EMAIL') or '').strip() or None
port = int(os.getenv('HTTPS_PORT') or 443)
http_fallback = _truthy(os.getenv('HTTPS_HTTP_FALLBACK'), default=True)
# HTTS_VERIFY lets an operator skip the post-reload probe (e.g. when the
# server is not reachable from inside the container, or in CI).
do_verify = _truthy(os.getenv('HTTPS_VERIFY'), default=True)
print(f'[https] Bootstrapping from environment: host={hostname!r} ip={ip_address} '
f'domain={domain!r} port={port} http_fallback={http_fallback}')
return _apply(app, https_enabled=True, hostname=hostname, domain=domain,
email=email, ip_address=ip_address, port=port,
http_fallback=http_fallback, verify=do_verify)
def _apply(app, https_enabled: bool, hostname: str | None, domain: str | None,
email: str | None, ip_address: str | None, port: int,
http_fallback: bool, verify: bool = True) -> int:
"""Persist the config, regenerate the Caddyfile and hot-reload Caddy.
When *verify* is set and HTTPS is being enabled, the TLS listener is probed
after the reload. If it does not come up, the configuration is automatically
reverted to plain HTTP so the server is never left unreachable — HTTPS then
falls back to HTTP exactly as intended, and can be retried from the admin UI.
"""
with app.app_context():
# An empty domain is meaningful: it selects the internal-CA path.
config = HTTPSConfig.create_or_update(
https_enabled=https_enabled,
hostname=hostname,
domain=domain or None,
ip_address=ip_address,
email=email,
port=port,
updated_by='deploy.sh',
)
mode = ('HTTP only' if not https_enabled
else 'Let\'s Encrypt' if config.domain
else 'internal CA')
print(f' Mode: {mode}')
print(f' Hostname: {config.hostname or "-"}')
print(f' Domain: {config.domain or "(none)"}')
print(f' IP address: {config.ip_address or "-"}')
print(f' Email: {config.email or "-"}')
print(f' HTTPS port: {config.port}')
caddyfile = CaddyConfigGenerator.generate_caddyfile(
config, http_fallback=http_fallback,
http_port=int(os.getenv('HTTP_PORT') or 80),
https_port=int(os.getenv('HTTPS_PORT') or port or 443))
if not CaddyConfigGenerator.write_caddyfile(caddyfile):
print(' ✗ Failed to write the Caddyfile', file=sys.stderr)
return 1
print(' ✓ Caddyfile written')
if not CaddyConfigGenerator.reload_caddy():
print(' ⚠ Caddyfile written but Caddy reload failed — restart the '
'caddy container to apply.', file=sys.stderr)
log_action('warning', 'Caddy reload failed during CLI HTTPS setup')
if not verify:
return 2
reload_ok = False
else:
print(' ✓ Caddy reloaded')
reload_ok = True
# ── Verify the TLS listener, then fall back to HTTP if it failed ────
if verify and https_enabled and reload_ok:
probe_host = config.ip_address or config.hostname or domain
# Probe whichever port the host actually publishes.
probe_port = int(os.getenv('HTTPS_PORT') or config.port or 443)
print(f' … Verifying HTTPS on {probe_host}:{probe_port}')
ok, detail = verify_https(probe_host, probe_port)
if ok:
print(f' ✓ HTTPS verified ({detail})')
log_action('info', f'HTTPS configured and verified (mode={mode})')
return 0
print(f' ✗ HTTPS verification failed: {detail}', file=sys.stderr)
print(' ↩ Falling back to plain HTTP so the server stays reachable.',
file=sys.stderr)
log_action('warning', f'HTTPS verification failed ({detail}); '
f'fell back to HTTP')
# Revert to HTTP-only and re-apply so port 80 keeps serving.
HTTPSConfig.create_or_update(
https_enabled=False,
hostname=config.hostname,
domain=None,
ip_address=config.ip_address,
email=config.email,
port=config.port,
updated_by='deploy.sh',
)
http_only = CaddyConfigGenerator.generate_caddyfile(
HTTPSConfig.get_config())
CaddyConfigGenerator.write_caddyfile(http_only)
CaddyConfigGenerator.reload_caddy()
return 1
if verify and https_enabled and not reload_ok:
log_action('warning', 'HTTPS configured but Caddy reload failed')
return 2
log_action('info', f'HTTPS configured via CLI (mode={mode})')
return 0
def _status(app) -> int:
with app.app_context():
config = HTTPSConfig.get_config()
if not config:
print(' No HTTPS configuration found (HTTP-only defaults).')
return 0
if not config.https_enabled:
mode = 'HTTP only'
elif config.domain:
mode = "Let's Encrypt"
else:
mode = 'internal CA'
print(f' Enabled: {config.https_enabled}')
print(f' Mode: {mode}')
print(f' Hostname: {config.hostname or "-"}')
print(f' Domain: {config.domain or "(none)"}')
print(f' IP address: {config.ip_address or "-"}')
print(f' Email: {config.email or "-"}')
print(f' HTTPS port: {config.port}')
print(f' Updated by: {config.updated_by or "-"}')
if config.updated_at:
print(f' Updated at: {config.updated_at.isoformat()}')
return 0
def _verify_current(app) -> int:
"""Probe the currently configured HTTPS endpoint; fall back to HTTP if down.
Useful as a post-deploy check and as a self-healing step after a restart
(e.g. certificate issuance failed, or the request path changed).
"""
with app.app_context():
config = HTTPSConfig.get_config()
if not config or not config.https_enabled:
print(' HTTPS is not enabled — nothing to verify.')
return 0
probe_host = config.ip_address or config.hostname or config.domain
probe_port = int(os.getenv('HTTPS_PORT') or config.port or 443)
print(f' Probing https://{probe_host}:{probe_port} …')
ok, detail = verify_https(probe_host, probe_port)
if ok:
print(f' ✓ HTTPS is working ({detail})')
return 0
print(f' ✗ HTTPS is NOT working: {detail}', file=sys.stderr)
print(' ↩ Falling back to plain HTTP so the server stays reachable.',
file=sys.stderr)
log_action('warning', f'HTTPS verify failed ({detail}); fell back to HTTP')
HTTPSConfig.create_or_update(
https_enabled=False,
hostname=config.hostname,
domain=None,
ip_address=config.ip_address,
email=config.email,
port=config.port,
updated_by='deploy.sh',
)
http_only = CaddyConfigGenerator.generate_caddyfile(HTTPSConfig.get_config())
CaddyConfigGenerator.write_caddyfile(http_only)
CaddyConfigGenerator.reload_caddy()
return 1
def main() -> int:
parser = argparse.ArgumentParser(description='DigiServer HTTPS manager')
sub = parser.add_subparsers(dest='command', required=True)
p_enable = sub.add_parser('enable', help='enable HTTPS and reload Caddy')
p_enable.add_argument('hostname', nargs='?', default=None)
p_enable.add_argument('domain', nargs='?', default=None,
help='Public domain; leave empty for internal CA')
p_enable.add_argument('email', nargs='?', default=None)
p_enable.add_argument('ip_address', nargs='?', default=None)
p_enable.add_argument('port', nargs='?', type=int, default=443)
p_enable.add_argument('--redirect-only', action='store_true',
help='Do not serve plain HTTP alongside HTTPS; '
'redirect to HTTPS instead')
p_enable.add_argument('--no-https', action='store_true',
help='Configure HTTP only (no TLS)')
p_enable.add_argument('--no-verify', action='store_true',
help='Skip the post-reload HTTPS probe (no auto-fallback)')
sub.add_parser('disable', help='disable HTTPS (HTTP only)')
sub.add_parser('status', help='print the current configuration')
sub.add_parser(
'verify',
help='probe the HTTPS endpoint and fall back to HTTP if it is broken')
sub.add_parser(
'bootstrap',
help='configure HTTPS from HOSTNAME_INTERNAL/HOST_IP env vars '
'(no-op when unset)')
args = parser.parse_args()
app = create_app()
if args.command == 'status':
return _status(app)
if args.command == 'bootstrap':
return bootstrap_from_env(app)
if args.command == 'verify':
return _verify_current(app)
if args.command == 'disable':
return _apply(app, https_enabled=False, hostname=None, domain=None,
email=None, ip_address=None, port=443,
http_fallback=True)
# enable
if args.no_https:
return _apply(app, https_enabled=False, hostname=args.hostname,
domain=None, email=args.email,
ip_address=args.ip_address, port=args.port,
http_fallback=True)
if not args.ip_address:
print('error: ip_address is required to enable HTTPS', file=sys.stderr)
return 1
return _apply(app, https_enabled=True, hostname=args.hostname,
domain=args.domain, email=args.email,
ip_address=args.ip_address, port=args.port,
http_fallback=not args.redirect_only,
verify=not args.no_verify)
if __name__ == '__main__':
sys.exit(main())