ske087
9f5409685d
Web links previously launched a separate Chrome/Edge kiosk process, which
caused the whole class of bugs in the tracker: the browser opening behind the
player, being handed off to an already-running instance and exiting instantly,
fighting for foreground/z-order, and leaking msedge.exe/chrome.exe processes
that were never closed.
WebView2 renders as a CHILD HWND of Kivy's own SDL window instead, so there is
no separate top-level browser to open behind the player, nothing to hand the
URL off to, no z-order contest, and no leaked browser process.
Windows/webview2_browser.py
- Environment -> controller -> navigate, driven through pythonnet.
- Async .NET Tasks are polled from Kivy's Clock. Calling GetAwaiter()
.GetResult() would deadlock: the continuation needs the same thread's
message pump.
- The controller is a .NET IntPtr, not a Python int (CreateAsync overloads
do not match otherwise).
- NavigationCompleted is tracked so a page that never loads can be told
apart from one that did. This matters on a closed network: an unreachable
host paints a Chromium error page, and without this the player would show
a blank/error screen for the item's whole slot instead of skipping it.
- is_alive() reports True while starting up. Start-up is async, so a
controller that does not exist yet is not a dead browser; treating it as
one made the first web link after a cold start be skipped instantly.
Windows/webview2_runtime.py
- Detects the Runtime (registry pv value, SDK probe as fallback) and
installs it silently when missing, unelevated, which produces a per-user
install and therefore never raises a UAC prompt on the signage display.
- Success is decided by RE-READING the installed version, not by the
installer exit code: Edge Update returns a non-zero HRESULT
(-2147219416) when the Runtime is already current, which is not a failure.
- On a closed network the online bootstrapper can never succeed, so it fails
fast with an actionable message instead of hanging for the full timeout.
- Failed attempts are cooldown-gated so a broken machine does not re-run an
installer on every start.
Offline hardening
- Browser arguments disable component updates, field trials, safe-browsing
list fetches, translate and other internet chatter. On an isolated LAN
each of those would otherwise have to time out, costing start-up latency.
Pages on the local server are unaffected.
Engine order (best first): WebView2 -> CEF -> Chrome/Edge subprocess. CEF has
no wheels past Python 3.9 so it is dormant on this build; the subprocess engine
remains only as a last resort.
Also fixes the reason the Windows adapters were never used at all:
SignagePlayer.__init__ assigned self.weblink_adapter_factory = None, which
shadowed the CLASS attribute that run_win.py injects. play_weblink() therefore
fell back to the generic adapter, whose find_browser() uses shutil.which() and
finds nothing on Windows because Chrome/Edge are not on PATH. The instance
attribute is now only set when the class attribute is absent.
Verified: windows/test_webview2_embed.py, test_webview2_navigation.py and
test_webview2_offline.py all pass (a locally served page renders with all
internet traffic disabled), and the packaged exe reports
"weblink_launch engine=webview2-embedded" -> "weblink_launched" on every cycle
with no leaked browser processes.
381 lines
16 KiB
Markdown
381 lines
16 KiB
Markdown
# Web Link Playlist Items — Player Integration Guide
|
||
|
||
This document describes how the **Kiwy-Signage player**
|
||
(<https://gitea.moto-adv.com/ske087/Kiwy-Signage.git>) supports the **`weblink`**
|
||
playlist item type (display a live web page / URL instead of an uploaded media
|
||
file).
|
||
|
||
> **Status: implemented.** The player supports `weblink` items on both
|
||
> Raspberry Pi (`chromium` subprocess) and Windows (embedded **WebView2**, with
|
||
> the CEF and Chrome/Edge subprocess engines as fallbacks). Sections 1–4
|
||
> describe the original design plan; section 6 documents the shipped
|
||
> architecture and the interaction model.
|
||
|
||
---
|
||
|
||
## 1. Background — how items flow
|
||
|
||
```
|
||
DigiServer API ──JSON──▶ player sync (get_playlists_v2.py) ──▶ playlist.json ──▶ main.py renders
|
||
/api/playlists downloads files to media/ by item type
|
||
```
|
||
|
||
Each playlist item the server returns currently looks like:
|
||
|
||
```json
|
||
{
|
||
"id": 42,
|
||
"file_name": "promo.jpg",
|
||
"type": "image",
|
||
"duration": 10,
|
||
"position": 1,
|
||
"url": "https://server/digiserver/static/uploads/promo.jpg",
|
||
"edit_on_player": false
|
||
}
|
||
```
|
||
|
||
The player:
|
||
1. **Syncs** (`src/get_playlists_v2.py` → `download_media_files()`): downloads
|
||
`url` into the local `media/` directory, then rewrites each item keeping only
|
||
`file_name`, `url` (now a **local relative path**), `duration`,
|
||
`edit_on_player`. **Note: the `type` field is currently discarded here.**
|
||
2. **Renders** (`src/main.py` → `play_current_media()`): opens the local file
|
||
and chooses a Kivy widget purely by **file extension**
|
||
(`.mp4/.avi/...` → `Video`, `.jpg/.png/...` → `AsyncImage`). Unknown
|
||
extensions are skipped as "unsupported".
|
||
|
||
A web link breaks all three assumptions: there is no file to download, no
|
||
extension to switch on, and no widget that renders a web page.
|
||
|
||
---
|
||
|
||
## 2. New server contract (what DigiServer will send)
|
||
|
||
A web-link playlist item will look like this:
|
||
|
||
```json
|
||
{
|
||
"id": 91,
|
||
"file_name": "weblink-3f9c1a2b",
|
||
"type": "weblink",
|
||
"duration": 30,
|
||
"position": 4,
|
||
"url": "https://example.com/dashboard",
|
||
"edit_on_player": false
|
||
}
|
||
```
|
||
|
||
Key differences vs. a file item:
|
||
|
||
| Field | File item | Web-link item |
|
||
|--------------|-----------------------------------|----------------------------------------|
|
||
| `type` | `image` / `video` | **`weblink`** |
|
||
| `url` | Path to a file on the server | **The web page to display (the link)** |
|
||
| `file_name` | Real filename on disk | Synthetic id (`weblink-<uuid>`), **no file exists** |
|
||
|
||
The player must branch on `type == "weblink"` and treat `url` as the page to
|
||
open — **never** try to download it as a file.
|
||
|
||
---
|
||
|
||
## 3. Required player changes
|
||
|
||
### 3.1 Sync step — `src/get_playlists_v2.py`
|
||
|
||
**Function:** `download_media_files(playlist, media_dir, ...)`
|
||
|
||
1. **Skip download for web links.** At the top of the per-item loop, detect
|
||
`media.get('type') == 'weblink'` and do **not** call `session.get()` / write
|
||
any file for it.
|
||
2. **Preserve `type` and the original `url`.** The `updated_media` dict that is
|
||
appended to `updated_playlist` currently drops `type` and rewrites `url` to a
|
||
local path. It must now carry `type` through, and for web links keep `url`
|
||
as the original web address (do not convert to a local relative path).
|
||
|
||
Suggested shape of the per-item logic:
|
||
|
||
```python
|
||
item_type = media.get('type', '')
|
||
|
||
if item_type == 'weblink':
|
||
# No file to download — pass the web link through unchanged.
|
||
updated_playlist.append({
|
||
'file_name': media.get('file_name', ''),
|
||
'type': 'weblink',
|
||
'url': media.get('url', ''), # the actual web page
|
||
'duration': media.get('duration', 10),
|
||
'edit_on_player': False,
|
||
})
|
||
continue
|
||
|
||
# ... existing download logic for file items ...
|
||
updated_playlist.append({
|
||
'file_name': file_name,
|
||
'type': item_type, # <-- now preserved
|
||
'url': os.path.relpath(local_path, os.path.dirname(media_dir)),
|
||
'duration': duration,
|
||
'edit_on_player': media.get('edit_on_player', False),
|
||
})
|
||
```
|
||
|
||
3. **`delete_unused_media()`** walks `media/` using `file_name`. Web links have
|
||
no file, so they simply won't match anything on disk — no change strictly
|
||
required, but make sure a missing local file for a `weblink` item does not
|
||
trigger a re-download or an error elsewhere.
|
||
|
||
### 3.2 Render step — `src/main.py`
|
||
|
||
**Function:** `play_current_media(self, force_reload=False)`
|
||
|
||
The current logic builds `media_path = os.path.join(self.media_dir, file_name)`
|
||
and then does `os.stat(media_path)` — which will fail for a web link (no file).
|
||
Add a **web-link branch before the file-existence check**:
|
||
|
||
```python
|
||
media_item = self.playlist[self.current_index]
|
||
file_name = media_item.get('file_name', '')
|
||
duration = media_item.get('duration', 10)
|
||
|
||
# NEW: handle web links before any file/path handling
|
||
if media_item.get('type') == 'weblink':
|
||
self.play_weblink(media_item.get('url', ''), duration)
|
||
return
|
||
|
||
# ... existing file existence check + extension branching ...
|
||
```
|
||
|
||
Then add a new method `play_weblink(self, url, duration)`:
|
||
|
||
- Validate the scheme is `http`/`https` (reject anything else, e.g. `file://`).
|
||
- Open the page for `duration` seconds, then advance with `self.next_media()`.
|
||
- Wrap in `try/except`; on failure increment `self.consecutive_errors` and call
|
||
`self.next_media()`, matching the existing error-handling pattern.
|
||
- Make sure the previous widget (`self.current_widget`) is removed/stopped just
|
||
like the image/video paths do.
|
||
|
||
#### Rendering approach (pick one)
|
||
|
||
Kivy has **no production-grade embedded web view**, especially on Raspberry Pi.
|
||
Recommended options, in order of robustness:
|
||
|
||
1. **Chromium kiosk overlay (recommended).** Launch Chromium over the Kivy
|
||
window for the item's duration, then close it and return to Kivy:
|
||
|
||
```python
|
||
import subprocess, shutil
|
||
from urllib.parse import urlparse
|
||
from kivy.clock import Clock
|
||
|
||
def play_weblink(self, url, duration):
|
||
scheme = urlparse(url).scheme.lower()
|
||
if scheme not in ('http', 'https'):
|
||
Logger.warning(f"SignagePlayer: Refusing non-http(s) weblink: {url}")
|
||
self.next_media()
|
||
return
|
||
try:
|
||
browser = shutil.which('chromium-browser') or shutil.which('chromium')
|
||
self._weblink_proc = subprocess.Popen([
|
||
browser,
|
||
'--kiosk', '--app=' + url,
|
||
'--noerrdialogs', '--disable-infobars',
|
||
'--incognito', '--no-first-run',
|
||
'--check-for-update-interval=31536000',
|
||
])
|
||
Clock.schedule_once(lambda dt: self._close_weblink_and_next(), duration)
|
||
except Exception as e:
|
||
Logger.error(f"SignagePlayer: Error opening weblink: {e}")
|
||
self.consecutive_errors += 1
|
||
self.next_media()
|
||
|
||
def _close_weblink_and_next(self):
|
||
proc = getattr(self, '_weblink_proc', None)
|
||
if proc and proc.poll() is None:
|
||
proc.terminate()
|
||
try:
|
||
proc.wait(timeout=5)
|
||
except Exception:
|
||
proc.kill()
|
||
self._weblink_proc = None
|
||
self.next_media()
|
||
```
|
||
|
||
Requirements / notes:
|
||
- Install Chromium on the player image (`chromium-browser` on Raspberry Pi OS).
|
||
- Ensure Chromium gets window focus over Kivy and is fully killed before the
|
||
next item, including on pause/stop/restart paths and on app shutdown
|
||
(`on_stop`) so no stray browser window is left behind.
|
||
- On Wayland/X11 the player already sets `SDL_VIDEODRIVER`; verify Chromium
|
||
launches on the same display/session.
|
||
|
||
2. **Embedded web view widget** (`kivy_garden.webview`, WebKit/GTK, or WebView2).
|
||
Cleaner UX (stays inside the Kivy widget tree) but fragile and poorly
|
||
supported on Pi/Wayland — only pursue if option 1 is unacceptable.
|
||
|
||
3. **Server-side screenshot fallback (no player change).** If embedding a live
|
||
browser is not desirable, DigiServer can periodically screenshot the URL and
|
||
store it as a normal `image` item; the player then needs no changes. This
|
||
loses live/animated content. Documented here for completeness only.
|
||
|
||
---
|
||
|
||
## 4. Checklist for the player update
|
||
|
||
- [ ] `get_playlists_v2.py`: skip download when `type == 'weblink'`.
|
||
- [ ] `get_playlists_v2.py`: preserve `type` in the rewritten playlist items
|
||
(fixes the current loss of `type`).
|
||
- [ ] `get_playlists_v2.py`: keep the original web `url` for weblink items.
|
||
- [ ] `main.py` `play_current_media()`: branch to `play_weblink()` **before** the
|
||
`os.stat()` file check.
|
||
- [ ] `main.py`: implement `play_weblink(url, duration)` (Chromium kiosk).
|
||
- [ ] `main.py`: validate scheme is `http`/`https`; reject others.
|
||
- [ ] Kill/cleanup the browser process on next item, pause, stop, restart, and
|
||
`on_stop`.
|
||
- [ ] Install Chromium on the player image / document it in the player README.
|
||
- [ ] Test: mixed playlist (image → video → weblink → image) cycles correctly
|
||
and respects per-item `duration`.
|
||
|
||
---
|
||
|
||
## 5. Security notes
|
||
|
||
- Only allow `http`/`https` schemes on both server and player; never open
|
||
`file://`, `chrome://`, etc.
|
||
- The server validates and stores the URL when the operator adds it; the player
|
||
should still re-validate the scheme before launching the browser (defence in
|
||
depth).
|
||
- Consider running Chromium with `--incognito` (no persistent cookies/cache) as
|
||
shown above.
|
||
|
||
---
|
||
|
||
## 6. Shipped architecture (`src/weblink_session.py`)
|
||
|
||
The player-side implementation lives in **one** module, so launch, verification,
|
||
timing and teardown have a single owner instead of being duplicated per
|
||
platform:
|
||
|
||
| Piece | Responsibility |
|
||
|--------------------------|----------------|
|
||
| `WeblinkSession` | Owns one weblink item: validate → launch → verify → watch → teardown. Generation-tokened so stale callbacks are ignored, and `atexit`-safe. |
|
||
| `WeblinkAdapter` | The only platform-specific part: launch / wait for the window / is it alive / tear it down / pre-warm. `extra_launch_args()` lets a subclass add browser flags without copying `launch()`. |
|
||
| `ChromiumSubprocessAdapter` | Default engine (Raspberry Pi `chromium`; on Windows the Chrome/Edge fallback). |
|
||
| `InteractionWatcher` | Decides when the item is finished (see the interaction model below). |
|
||
| `WebInputSources` | Reads `/dev/input/event*` (Linux) and does a pointer-position tap (Windows, needed for embedded engines). |
|
||
| `webview2_browser.py` | **Windows, preferred**: embeds WebView2 as a child HWND of the Kivy window. |
|
||
| `webview2_runtime.py` | **Windows**: detects the WebView2 Runtime and installs it silently when missing. |
|
||
|
||
Platform wrappers inject their engines through
|
||
`SignagePlayer.weblink_adapter_factory`:
|
||
|
||
* **Raspberry Pi / Linux** — built-in Chromium subprocess adapter.
|
||
* **Windows** (`windows/run_win.py`) — engines are tried in this order:
|
||
|
||
| Order | Engine | Renders | Notes |
|
||
|-------|--------|---------|-------|
|
||
| 1 | **WebView2** (`webview2_browser.py`) | child window **inside** Kivy | Preferred. No subprocess, so no background/z-order/hand-off/leak problems. |
|
||
| 2 | CEF (`cef_browser.py`) | child window inside Kivy | Dormant: `cefpython3` has no wheels past Python 3.9. |
|
||
| 3 | Chrome/Edge subprocess | separate window | Last resort only; retains the old drawbacks. |
|
||
|
||
`WeblinkAdapter.extra_launch_args()` is the hook subclasses use to add flags
|
||
without duplicating `launch()` — the Chrome adapter uses it for
|
||
`--user-data-dir` + `--kiosk`.
|
||
|
||
> **Do not give the factory a class-level `None` default combined with an
|
||
> unconditional instance assignment.** `SignagePlayer.__init__` originally set
|
||
> `self.weblink_adapter_factory = None`, which shadowed the class attribute the
|
||
> Windows wrapper installs — so the platform adapters were silently ignored and
|
||
> every weblink fell back to the generic adapter and failed. It now only sets
|
||
> the instance attribute when the class attribute is absent.
|
||
|
||
### 6.2 Windows: WebView2 Runtime
|
||
|
||
WebView2 is two separate things, and they ship differently:
|
||
|
||
* the **SDK** (`Microsoft.Web.WebView2.Core.dll`, `WebView2Loader.dll`) — the
|
||
API surface, bundled in the exe from `windows\webview2_sdk\` (~860 KB);
|
||
* the **Runtime** (`msedgewebview2.exe`) — the actual Chromium engine, shipped
|
||
by Microsoft and **verified/installed at start-up** by
|
||
`windows\webview2_runtime.py`.
|
||
|
||
If the Runtime is absent the player runs an installer silently
|
||
(`/silent /install`) and **unelevated**, which produces a *per-user* install and
|
||
therefore never raises a UAC prompt on the signage display. A ~1.7 MB online
|
||
bootstrapper is bundled by default; a ~203 MB offline standalone installer can
|
||
be bundled instead (see `windows\webview2_runtime\download_runtime_installers.ps1`)
|
||
for machines with no internet.
|
||
|
||
Install success is decided by **re-reading the installed version**, not by the
|
||
installer exit code — Edge Update returns a non-zero HRESULT (e.g.
|
||
`-2147219416`) when the Runtime is already current, which is not a failure.
|
||
|
||
### 6.1 Interaction model — web links are not passive media
|
||
|
||
`duration` on a weblink is **not** a hard cut-off. The player advances only when
|
||
**both** conditions are true:
|
||
|
||
1. the configured `duration` has elapsed; **and**
|
||
2. the viewer has not interacted with the page for `interaction_postpone`
|
||
seconds (default **10 s**), measured from the **most recent** interaction.
|
||
|
||
Consequences:
|
||
|
||
- A viewer who taps, scrolls or navigates the page during the final seconds of
|
||
the slot **keeps the page on screen** — the advance is pushed 10 s past that
|
||
touch, and every further touch pushes it again. The link is never pulled out
|
||
from under someone who is using it.
|
||
- An untouched page still advances on schedule, exactly like a media item.
|
||
- A multi-event burst (a drag, a page transition) counts as **one** interaction
|
||
but the countdown is measured from the **last** event of that burst, so an
|
||
item can never be cut off mid-gesture.
|
||
- `max_dwell` (duration × `max_dwell_factor`, at least `min_max_dwell`) is an
|
||
absolute backstop so a wedged browser or a jammed touchscreen cannot park the
|
||
playlist forever.
|
||
|
||
**Pause/play does not apply to web links.** A web link is an interactive
|
||
surface, so `toggle_pause()` is a no-op while one is on screen — the interaction
|
||
watcher owns its lifecycle. The pause button continues to work normally for
|
||
images and videos.
|
||
|
||
### 6.2 Verified start-up
|
||
|
||
Launching a browser is not the same as displaying a page. The session therefore
|
||
does **not** report success immediately after spawning the process (that used to
|
||
reset the error counter and leave a black screen for the whole duration). The
|
||
watcher thread — never the Kivy main thread — waits for the browser window to
|
||
appear, and if it never does the item is reported as failed and skipped.
|
||
|
||
### 6.3 Configuration
|
||
|
||
All timings are tunable in `config/app_config.json` under `weblink`:
|
||
|
||
```json
|
||
"weblink": {
|
||
"engine": "auto",
|
||
"interaction_postpone": 10,
|
||
"interaction_debounce": 0.5,
|
||
"interaction_grace": 5.0,
|
||
"max_dwell_factor": 6.0,
|
||
"min_max_dwell": 300,
|
||
"launch_timeout": 15,
|
||
"prewarm": true
|
||
}
|
||
```
|
||
|
||
| Key | Meaning |
|
||
|-----|---------|
|
||
| `engine` | Preferred engine (`auto`, `cef`, `subprocess`). |
|
||
| `interaction_postpone` | Seconds the advance is postponed, measured from each interaction (default 10). |
|
||
| `interaction_debounce` | Logging/trace throttle for continuous drags (default 0.5). |
|
||
| `interaction_grace` | Settle window after the last raw event still counted as interacting (default 5). |
|
||
| `max_dwell_factor` | Hard ceiling = `duration × factor`. |
|
||
| `min_max_dwell` | Floor for that hard ceiling, in seconds. |
|
||
| `launch_timeout` | How long to wait for the browser window to appear. |
|
||
| `prewarm` | Pre-warm the next weblink (disabled on Windows). |
|
||
|
||
### 6.4 Diagnostics
|
||
|
||
The watcher traces structured events through `playback_trace.py`:
|
||
`weblink_launch`, `weblink_visible`, `weblink_interaction`, `weblink_end`
|
||
(with reason `viewer_idle`, `browser_exited` or `max_dwell`),
|
||
`weblink_not_visible` and `weblink_failed`.
|