diff --git a/.github/instructions/kiwy-build-and-development.instructions.md b/.github/instructions/kiwy-build-and-development.instructions.md index 0e8c4cf..4fb8a03 100644 --- a/.github/instructions/kiwy-build-and-development.instructions.md +++ b/.github/instructions/kiwy-build-and-development.instructions.md @@ -33,10 +33,19 @@ Cross-platform Kivy digital signage player. - **Python 3.13+/3.14 is NOT supported for the Windows build** — Kivy 2.3.1 has no wheels for them. -- `cefpython3` is **not installed** in the project venv. The embedded-CEF web-link engine is - therefore unavailable and web links fall back to the Chrome/Edge subprocess adapter. - Installing `cefpython3` (uncomment it in `windows/requirements_win.txt`) restores the - CEF engine. +- `cefpython3` is **not installed** in the project venv (its wheels stop at Python 3.9). + The embedded-CEF engine in `windows/cef_browser.py` is therefore dormant on this + build — do not rely on it. +- **Web links use embedded WebView2** (`windows/webview2_browser.py`), driven through + `pythonnet` (installed). Two extra things ship with the exe: + - `windows/webview2_sdk/` — ~860 KB of SDK DLLs (`Microsoft.Web.WebView2.Core.dll`, + `WebView2Loader.dll`). **Tracked in git on purpose**: without them WebView2 + silently downgrades to the Chrome/Edge subprocess engine. + - `windows/webview2_runtime/MicrosoftEdgeWebview2Setup.exe` — ~1.7 MB bootstrapper, + used to install the Runtime on a machine that lacks it. The ~203 MB offline + standalone installer is **git-ignored**; fetch it with + `windows/webview2_runtime/download_runtime_installers.ps1 -Offline` when you need + to deploy to machines with no internet (it makes the exe ~305 MB). ## Build @@ -79,6 +88,118 @@ current spec, so deploying it ships old code. Always take the executable from th - `excluded_imports` drops Linux-only packages (`evdev`, `gi`, GStreamer) and the non-matching `cefpython3` `.pyd` variants. +## First-run setup (no bundled credentials) + +`config/app_config.json` is **deliberately NOT bundled** into the exe, and +`player_auth.json` is excluded from `Tree(src)`. Shipping either one planted the +build machine's `server_ip` / `screen_name` / `auth_code` into every install, so +a fresh machine silently connected to the wrong server (or played a stale +playlist) instead of asking. + +What happens on a machine with no usable config: + +1. The splash video (`config/resources/intro1.mp4`) plays. +2. A notice appears: *"Player is not configured"*. +3. After `SETUP_NOTICE_SECONDS` (**5 s**, `src/main.py`) the Settings screen + opens automatically. +4. Saving valid values writes `config/app_config.json` next to the **.exe** and + playback starts immediately — no restart needed. + +On a machine that **is** configured, the notice and Settings are skipped and the +cached playlist plays straight away. + +Key functions in `src/main.py`: + +| Name | Role | +|------|------| +| `config_is_configured(config)` | Single source of truth. Missing file, empty/unparseable JSON, missing keys and leftover placeholders (`localhost`, `kivy-player`, `1234567`, `127.0.0.1`) all count as **unconfigured**. | +| `DEFAULT_CONFIG` | Blank-credential starting point, so a fresh install can never look configured. | +| `SignagePlayer.on_intro_finished()` | The one decision point after the splash: setup vs. playback. Both intro paths (video end and "no intro file") go through it. | +| `show_setup_required_notice()` | Shows the notice, then opens Settings after 5 s. | +| `on_first_run_config_saved()` | Re-syncs and starts playback. | + +`config_is_configured()` is covered by `windows/test_first_run_setup.py` — run it +after touching that logic. + +> When adding a new **required** config key, add it to `CONFIG_REQUIRED_KEYS` +> (and to `CONFIG_PLACEHOLDER_VALUES` if it has a placeholder default), or the +> first-run flow will not ask for it. + +## 24/7 Supervision (Windows) + +A signage player must survive crashes and hangs unattended. On **Linux/Pi** this is +`start.sh` (systemd or `./start.sh`). On **Windows** it is `windows/watchdog.ps1`, +launched by `windows/start_player_watchdog.bat`. + +It restarts the player when it: + +- **crashed** — the process disappeared; or +- **hung** — the process is alive but `.player_heartbeat` has gone stale + (the player rewrites it every 10 s; the watchdog treats >60 s as hung). A frozen + player is otherwise indistinguishable from a working one. + +It also backs off if the player cannot stay up (crash-loop breaker: +`CrashLoopMaxFailures` failures in `CrashLoopWindowMin` minutes without ever +becoming healthy → `CrashLoopBackoffMin` minutes of quiet). + +### The stop flag is SESSION-SCOPED + +The only supported way to stop the player is the **exit-screen password**. On +success the player writes `.player_stop_requested` next to its `.exe`, and the +watchdog then **stands down instead of restarting**. + +The flag is cleared **every time the watchdog starts**, which is what makes it +session-scoped: the password exit ends the *current* session, and the next launch +begins a new one. There is no file to delete by hand. + +> **Ordering matters:** the flag is the operationally-correct choice over a +> "remember the last exit" setting, because a reboot must come back up playing. +> Clearing on start also means a power cut cannot leave the player permanently off. + +### Two failure modes that must not be confused + +| Situation | Correct reaction | +|---|---| +| Was healthy, then heartbeat went stale | **Restart promptly** (this is a hang) | +| Running but never became healthy (slow start, bad install, the **first-run setup screen**) | **Be patient**, then restart after `2 × StartupGraceSec` | + +Never kill a player that has not yet been healthy — it may be an operator entering +server settings on the first-run setup screen. + +### Important Windows-specific facts + +- The packaged player is **TWO processes** (PyInstaller bootloader parent + the + child that owns the SDL window). Any kill must use `taskkill /T` or the visible + window survives and the next launch collides with it. +- The watchdog can only *restart* an install that already has a working config. + It cannot help before the player has first run. +- Hiding the console window (`console=False`) is **not** done: the console is the + only way to see a start-up failure, and the build instructions warn against + flipping it without reason. +- **No Windows service.** A service runs in session 0 with no desktop, so the + player could not render to the screen. A login-triggered task is the correct + Windows counterpart to the Pi's systemd unit — that is what + `start_player_watchdog.bat` is for (add it to the Startup folder for autostart). + +### Verify the watchdog + +``` +cd windows +venv\Scripts\python.exe test_watchdog.py # crash + hang + stop-flag + new-session +``` + +Check state at any time (read-only): + +``` +powershell -NoProfile -ExecutionPolicy Bypass -File windows\watchdog.ps1 -Status +``` + +> When testing a hang, do **not** just backdate the heartbeat file's mtime — the +> healthy player rewrites it immediately, so nothing is actually stale and the +> test passes for the wrong reason. Hold the file open with an exclusive Windows +> lock (share mode 0) so the player's own write fails; that reproduces a genuine +> stale heartbeat while the process stays alive. + ## Code Signing (production constraint) Production PCs run with **Smart App Control enforced** (`VerifiedAndReputablePolicyState=1`, @@ -96,9 +217,21 @@ UMCI enforced) with **no "Run anyway" bypass** — an unsigned exe is blocked at Fast syntax gate (no build, seconds): ``` -python -m py_compile src/main.py src/weblink_session.py windows/run_win.py +python -m py_compile src/main.py src/weblink_session.py windows/run_win.py windows/webview2_browser.py windows/webview2_runtime.py ``` +Web-link engine checks (no build, no player): + +``` +cd windows +venv\Scripts\python.exe test_webview2_embed.py # embeds WebView2 in a bare Win32 window +venv\Scripts\python.exe test_webview2_runtime.py # Runtime detection + installer discovery +``` + +Rebuild warning: the bundled offline installer makes the exe ~305 MB. If you do not +need offline machines, delete +`windows/webview2_runtime/MicrosoftEdgeWebView2RuntimeInstallerX64.exe` before building. + **Close the running player before rebuilding.** A running `dist\KiwySignagePlayer\KiwySignagePlayer.exe` locks the output and the build fails with "Access is denied". Chrome/Edge kiosk processes left over from web-link playback can also diff --git a/src/video_safety.py b/src/video_safety.py new file mode 100644 index 0000000..12d3302 --- /dev/null +++ b/src/video_safety.py @@ -0,0 +1,185 @@ +"""video_safety.py — make Kivy's video teardown non-blocking. + +The bug this fixes +------------------ +The player hung (Windows `AppHangB1`) after ~30-45 minutes of looping, always +at a *video* item. The trace stopped dead right after `video_loaded` with no +`video_eos` and no `advance_after_video_eos`. + +Cause — a blocking ``join()`` that Kivy performs on the calling thread: + +1. At end of stream, ffpyplayer/``VideoFFPy`` fires its ``on_eos`` event. +2. Kivy's ``Video`` widget binds its own handler **first** (in + ``kivy/uix/video.py``: ``self._video.bind(..., on_eos=self._on_eos)``). + That handler is:: + + def _on_eos(self, *largs): + if not self._video or self._video.eos != 'loop': + self.state = 'stop' # <-- fires DURING the on_eos dispatch + +3. ``Video.state = 'stop'`` → ``on_state`` → ``VideoFFPy.stop()`` → + ``unload()``, which does ``self._thread.join()`` + (``video_ffpyplayer.py``: ``# TODO: use callback, don't block here``). +4. That join waits for the ffpyplayer decode thread, whose lifetime is variable + (the existing code comments note "0.4s up to 100s+"). When it does not + return, the SDL/Kivy main thread never returns either — the window stops + pumping messages and Windows declares the app hung. + +Why it only shows up after a while: our own EOS handler deliberately does not +set ``state = 'stop'`` (that path is already handled off-thread), so the hang +depends on Kivy's own handler firing first and on that particular video's +decode thread being slow to exit. It is a race, so it looks random and only +appears after many video cycles. + +The fix +------- +Bound the wait. ffpyplayer sets ``_ffplayer_need_quit = True`` and wakes its +thread before joining, so the thread *is* asked to exit — we simply stop +waiting indefinitely for it. ``suppress_kivy_video_blocking_unload()`` patches +the join on the specific ``VideoFFPy`` thread object to a bounded timeout, so a +wedged decode thread can no longer take the whole player down. The teardown +still runs off the main thread (see ``_teardown_video_async`` in ``main.py``), +so the normal case is unaffected. + +This is deliberately narrow: only Kivy's own internal video thread is patched, +only its ``join`` timeout is bounded, and every failure path leaves Kivy +untouched. +""" + +from __future__ import annotations + +import threading +import time + +#: How long a video's decode thread may take to exit before we give up on it. +#: A healthy ffpyplayer thread exits in well under a second; this allows a lot +#: of slack while still guaranteeing the UI thread is never parked forever. +DEFAULT_JOIN_TIMEOUT = 5.0 + +_patched_threads = 0 +_lock = threading.Lock() + + +def _log(message): + try: + from kivy.logger import Logger + + Logger.info(f'[VideoSafety] {message}') + except Exception: + print(f'[VideoSafety] {message}') + + +def suppress_kivy_video_blocking_unload(timeout=DEFAULT_JOIN_TIMEOUT): + """Bound the join() Kivy's VideoFFPy.unload() performs on the caller. + + Must be called *before* the video widget is constructed, because the + decode thread is created during ``play()``. Safe to call repeatedly. + + Args: + timeout: maximum seconds to wait for the decode thread to exit. + + Returns: + True when the guard was installed (or already present). + """ + global _patched_threads + + if timeout is None or float(timeout) <= 0: + return False + + try: + # Importing this module has the side effect of selecting the + # ffpyplayer provider; if a different provider is active (or video + # support is missing) there is nothing to patch. + from kivy.core.video import Video as CoreVideo + + if CoreVideo is None: + return False + except Exception as exc: + _log(f'video provider unavailable, nothing to patch ({exc})') + return False + + try: + from kivy.core.video import Video + except Exception: + Video = None + + if Video is None: + return False + + # VideoFFPy resolves via the current provider. Import it directly so we + # patch the right class even if the provider changes later. + try: + from kivy.core.video import video_ffpyplayer as _vfp + except Exception as exc: + _log(f'ffpyplayer provider not active, nothing to patch ({exc})') + return False + + provider = getattr(_vfp, 'VideoFFPy', None) + if provider is None: + return False + + with _lock: + if getattr(provider, '_kiwy_bounded_join', False): + return True + + original_play = provider.play + if getattr(provider, '_kiwy_original_play', None) is None: + provider._kiwy_original_play = original_play + + def play(self, *args, **kwargs): + result = provider._kiwy_original_play(self, *args, **kwargs) + _bound_thread_join(self, timeout) + return result + + provider.play = play + provider._kiwy_bounded_join = True + _patched_threads += 1 + + _log(f'Kivy video unload join() bounded to {float(timeout):.1f}s') + return True + + +def _bound_thread_join(provider_instance, timeout): + """Replace the provider's internal thread join with a bounded one. + + ffpyplayer has already been told to quit (``_ffplayer_need_quit = True``) + and its thread woken before ``unload()`` joins, so bounding the wait does + not leak work — it only stops an unresponsive decode thread from freezing + the whole application. + """ + thread = getattr(provider_instance, '_thread', None) + if thread is None: + return + if getattr(thread, '_kiwy_bounded_join', False): + return + + try: + real_join = thread.join + except Exception: + return + + def bounded_join(join_timeout=None): + """Join with a hard upper bound; never block the caller forever.""" + limit = float(timeout if join_timeout is None else min(join_timeout, timeout)) + started = time.monotonic() + try: + real_join(timeout=limit) + except TypeError: + # Some Python builds dislike an explicit keyword here. + try: + real_join(limit) + except Exception: + pass + except Exception: + pass + if thread.is_alive(): + _log( + f'video decode thread still alive {time.monotonic() - started:.1f}s ' + 'after being asked to quit - continuing without it' + ) + + try: + thread.join = bounded_join + thread._kiwy_bounded_join = True + except Exception as exc: + _log(f'could not bound video thread join ({exc})') diff --git a/windows/README_WINDOWS_BUILD.md b/windows/README_WINDOWS_BUILD.md index e017895..72aa5d6 100644 --- a/windows/README_WINDOWS_BUILD.md +++ b/windows/README_WINDOWS_BUILD.md @@ -90,12 +90,94 @@ windows\dist\KiwySignagePlayer\ For a **single-file .exe**, edit `build.spec` — uncomment the `exe_onefile` section and comment out the `coll = COLLECT(...)` section. +## 🌐 Web Links (embedded WebView2) + +Web links render with **WebView2**, embedded as a child window *inside* the +Kivy window. Because it is not a separate browser process, it cannot open +behind the player, cannot be handed off to an existing browser and exit, and +never leaves leaked `msedge.exe`/`chrome.exe` processes behind. + +WebView2 has **two** parts, and they are handled differently: + +| Part | What it is | How it ships | +|------|------------|--------------| +| **SDK** | `Microsoft.Web.WebView2.Core.dll` + `WebView2Loader.dll` — the API surface | Bundled in the exe (`windows\webview2_sdk\`, ~860 KB) | +| **Runtime** | `msedgewebview2.exe` — the actual Chromium engine | Microsoft's evergreen component. Ships with Windows 11 and nearly all Windows 10 machines. **Installed automatically on first start if missing.** | + +### Automatic Runtime installation + +On start-up the player checks for the Runtime (registry `pv` value under the +Edge Update client GUID, with a live SDK probe as fallback). If it is absent it +runs the installer silently: + +``` +MicrosoftEdgeWebview2Setup.exe /silent /install +``` + +Deliberately **not elevated** — an unelevated run performs a *per-user* install, +so no UAC dialog ever appears on the signage display. + +Installers are looked up in this order, so you can drop a replacement next to +the `.exe` without rebuilding: + +1. `KIWY_WEBVIEW2_INSTALLER` environment variable +2. `\webview2_runtime\` +3. `\` (next to the executable) +4. bundled copy inside the exe + +| Installer | Size | Bundled? | Use when | +|-----------|------|----------|----------| +| `MicrosoftEdgeWebview2Setup.exe` | ~1.7 MB | ✅ yes | Machine has internet (downloads the Runtime) | +| `MicrosoftEdgeWebView2RuntimeInstallerX64.exe` | ~203 MB | ⬜ opt-in | Machine is **offline** | + +To bundle the offline installer (adds ~200 MB to the exe): + +```powershell +cd windows +.\webview2_runtime\download_runtime_installers.ps1 -Offline +venv\Scripts\python.exe -m PyInstaller build.spec --clean --noconfirm +``` + +Re-download the installers at any time (they are Microsoft-signed; the script +verifies the signature): + +```powershell +.\webview2_runtime\download_runtime_installers.ps1 +``` + +### If WebView2 is unavailable + +Web links fall back to the Chrome/Edge subprocess engine (see +`src/weblink_session.py`). That engine still works, but reintroduces the old +drawbacks — a separate browser window, possible background/z-order behaviour, +and browser processes to clean up. + +### Troubleshooting + +| Problem | Solution | +|---------|----------| +| Web links show a black/blank page | Check `logs\` for `[WebView2]` lines. Confirm the Runtime version is reported. | +| Runtime install did not happen | A failed attempt is not retried for 6 hours (marker: `logs\.webview2_install_attempt`). Delete that file to retry immediately. | +| Need it working offline | Bundle the standalone installer (see above). | + ## ⚙️ Configuration -1. On first run, config files are created in the **same folder as the executable** (not in `%APPDATA%`) - - The .exe creates: `config/`, `media/`, `playlists/`, `logs/` directories locally +**No configuration ships with the exe.** On a machine that has never been set +up, `config\app_config.json` is absent, so after the splash video the player +shows a *"Player is not configured"* notice, waits 5 seconds and then opens the +**Settings** screen automatically. Enter the server details and playback starts +right away — no restart required. + +On a machine that already has a valid `config\app_config.json`, the first-run +flow is skipped entirely and the cached playlist plays immediately. + +1. Config is created next to the **executable** (not in `%APPDATA%`) + - The .exe creates: `config/`, `media/`, `playlists/`, `logs/` locally - This allows you to copy the entire `dist\KiwySignagePlayer\` folder anywhere and it works -2. Edit `config\app_config.json` (next to the .exe) to set your server: +2. The player is considered configured once `server_ip`, `screen_name` and + `quickconnect_key` all hold real values. Placeholder values + (`localhost`, `kivy-player`, `1234567`, `127.0.0.1`) count as unconfigured. +3. Edit `config\app_config.json` (next to the .exe) to set your server: ```json { diff --git a/windows/run_win.py b/windows/run_win.py index b4c671b..77af6ed 100644 --- a/windows/run_win.py +++ b/windows/run_win.py @@ -251,6 +251,93 @@ def _get_cef_browser(): return _CEF_BROWSER +# ── Embedded WebView2 (Edge) browser ──────────────────────────────── +_WEBVIEW2_BROWSER = None + +#: How long a weblink will wait for a Runtime that is still installing. +_WEBVIEW2_INSTALL_WAIT = 300.0 + + +def _ensure_webview2_runtime_async(): + """Start installing the WebView2 Runtime if it is missing. + + Called once at start-up. The install runs on a background thread so a + machine without the Runtime can install it while the player is still + syncing its playlist, rather than freezing the UI on the first weblink. + """ + try: + import webview2_runtime + except Exception as exc: + _log(f"WebView2 runtime module unavailable: {exc}") + return + try: + _log(webview2_runtime.describe()) + webview2_runtime.ensure_runtime_async() + except Exception as exc: + _log(f"WebView2 runtime check failed (non-fatal): {exc}") + + +def _get_webview2_browser(force_new=False): + """Return the shared WebView2Browser singleton, or None if unavailable. + + WebView2 renders as a CHILD HWND of Kivy's SDL window, which is what makes + it immune to the subprocess weblink bugs (background window, instant + hand-off exit, z-order fights, leaked browsers). + """ + global _WEBVIEW2_BROWSER + if force_new: + old, _WEBVIEW2_BROWSER = _WEBVIEW2_BROWSER, None + if old is not None: + try: + old.shutdown() + except Exception: + pass + if _WEBVIEW2_BROWSER is None: + try: + from webview2_browser import WebView2Browser + except Exception as exc: + _log(f"WebView2 module unavailable: {exc}") + return None + + if not WebView2Browser.is_available(): + # The SDK is bundled but the Runtime may still be installing + # (a machine that shipped without it). Wait here rather than + # silently downgrading to the leaky Chrome/Edge engine — this runs + # on the watcher thread, not the Kivy main thread. + if _wait_for_webview2_runtime(): + pass # installed in the meantime; retry below + if not WebView2Browser.is_available(): + reason = getattr(WebView2Browser, '_import_error', None) or 'unavailable' + _log(f"WebView2 not usable: {reason}") + return None + + try: + data_dir = os.environ.get('KIWY_DATA_DIR', os.getcwd()) + _WEBVIEW2_BROWSER = WebView2Browser( + hwnd_provider=_find_kivy_hwnd, + user_data_dir=os.path.join(data_dir, '.webview2-profile'), + ) + except Exception as exc: + _log(f"WebView2 init failed: {exc}") + return None + return _WEBVIEW2_BROWSER + + +def _wait_for_webview2_runtime(timeout=_WEBVIEW2_INSTALL_WAIT): + """Block while a Runtime install is in flight. True if it became available.""" + try: + import webview2_runtime + except Exception: + return False + state = webview2_runtime.get_state() + if not state.get('installing'): + return False + _log("WebView2: waiting for the Runtime install to finish...") + ok = webview2_runtime.wait_for_install(timeout) + _log(f"WebView2: Runtime install wait finished (available={ok})") + return ok + + def _windows_find_browser(): """Find Chrome or Edge executable on Windows for weblink support. @@ -1472,6 +1559,167 @@ def _patch_main(): # CEF keeps one browser instance alive; there is nothing to warm. pass + class _WinWebView2Adapter(ChromiumSubprocessAdapter): + """Embedded WebView2 (Edge) — renders INSIDE the Kivy window. + + This is the preferred Windows engine. Because the page is a child HWND + of Kivy's own SDL window there is no separate browser process, so none + of the subprocess problems apply: it cannot open behind the player, it + cannot be handed off to an existing instance and exit instantly, it + does not fight for foreground/z-order, and teardown leaves no leaked + browser behind. + + ``launch`` returns as soon as the (asynchronous) start-up is requested; + the session's watcher thread then polls :meth:`wait_visible` until the + page is actually showing. + """ + + name = 'webview2-embedded' + embedded = True + # WebView2 renders in-window, but unlike raw CEF we CAN prove the page + # appeared (the controller reports when it is visible). That matters on + # a closed network: if the weblink host is unreachable the page never + # paints and the item is skipped instead of showing a blank screen. + can_verify_visibility = True + + def __init__(self): + super().__init__(kiosk=False) + self._browser = None + self._resize_bound = False + + @property + def process(self): + return None # in-process: nothing to kill + + def target_size(self): + """Use the real Kivy window size (already DPI-aware).""" + try: + from kivy.core.window import Window as KivyWindow + + width, height = (int(KivyWindow.size[0]), int(KivyWindow.size[1])) + if width > 0 and height > 0: + return width, height + except Exception: + pass + return 1920, 1080 + + def launch(self, url, width, height): + self._browser = _get_webview2_browser() + if self._browser is None: + return False + # A child HWND has no z-order to fight over, but the Kivy window + # itself must own the foreground or the page looks inert. + _bring_kivy_to_front(async_ok=False) + self._bind_resize_once() + self._browser.resize(width, height) + return bool(self._browser.show(url)) + + def is_alive(self): + browser = self._browser + if browser is None: + return False + # NOTE: `is_alive` must be True while the page is still coming up. + # WebView2 creates its environment and controller asynchronously, so + # a controller that does not exist yet is NOT a dead browser — + # treating it as one made the first weblink be skipped instantly. + return browser.is_alive() + + def wait_visible(self, timeout): + """Wait for the page to actually load, on the caller's (watcher) thread. + + Two things must hold: the controller must be showing, AND the + navigation must have completed successfully. The second condition is + what makes an unreachable host (normal on a closed network) skip the + item instead of displaying Chromium's error page for the full slot. + """ + import time + + browser = self._browser + if browser is None: + return False, 'no browser' + deadline = time.monotonic() + max(1.0, float(timeout)) + while time.monotonic() < deadline: + if browser.failed_reason: + return False, browser.failed_reason + nav = browser.navigation_succeeded() + if browser.is_showing() and nav is not None: + if not nav: + detail = browser.navigation_status() or 'navigation failed' + return False, f'page failed to load ({detail})' + # Let the compositor paint the first frame before any + # masking overlay is hidden / Kivy is restored. + time.sleep(0.3) + return True, 'webview2-loaded' + time.sleep(0.1) + if browser.is_showing() and browser.navigation_succeeded() is None: + # Controller is up but navigation never reported within the + # timeout. Treat as visible rather than skipping a page that is + # merely slow to confirm. + return True, 'webview2-showing-unconfirmed' + reason = browser.failed_reason or 'webview2 did not show' + return False, reason + + def on_visible(self): + trace('win_weblink_webview2_shown') + + def on_launch_failed(self): + # Never leave a half-built controller (or a hidden child window) + # behind when start-up fails. + browser, self._browser = self._browser, None + if browser is not None: + try: + browser.shutdown() + except Exception: + pass + _get_webview2_browser(force_new=True) + + def _bind_resize_once(self): + """Keep the page matched to the window, bound exactly once. + + Rebinding per cycle was the unbounded-callback leak fixed for CEF; + the same discipline applies here. + """ + if self._resize_bound: + return + try: + from kivy.core.window import Window as KivyWindow + + def _wv2_resize(*_args): + try: + browser = self._browser + if browser is not None and browser.is_showing(): + browser.resize( + int(KivyWindow.size[0]), int(KivyWindow.size[1]) + ) + except Exception: + pass + + KivyWindow.bind(size=_wv2_resize) + self._resize_bound = True + except Exception as exc: + _log(f"WebView2 resize bind failed (non-fatal): {exc}") + + def teardown(self): + """Hide the page (keep the controller for a fast next show). + + Hiding — not disposing — is deliberate: the controller stays alive + so the next weblink paints immediately, and because it is a child + window there is no leaked process to reap. + """ + browser = self._browser + if browser is not None: + try: + browser.hide() + except Exception as exc: + _log(f"WebView2 hide failed (non-fatal): {exc}") + # Restore Kivy as the visible surface again. + _bring_kivy_to_front(async_ok=False) + + def prewarm(self, url): + # The environment/controller are already warm after the first use; + # pre-navigating would claim the page before its slot. + pass + class _WinChromeAdapter(ChromiumSubprocessAdapter): """Chrome/Edge kiosk subprocess with the Windows visibility check. @@ -1493,6 +1741,26 @@ def _patch_main(): def on_launch_failed(self): _Win32Overlay.hide() + def extra_launch_args(self): + """Dedicated profile + kiosk flags for the Windows browser. + + ``--user-data-dir`` is mandatory: without it Chrome/Edge hands the + URL to an already-running instance, our launched process exits in + ~2s and the weblink never displays (the adapter's `wait_visible` + then sees the process die and the item is skipped). It also gives + us a top-level window we can enumerate, raise and taskkill without + touching the operator's own browser profile. + """ + args = [] + if self._profile_dir: + args.append('--user-data-dir=' + self._profile_dir) + if self._kiosk: + # The base launch() already adds --start-fullscreen / + # --start-maximized; --kiosk upgrades that to a true kiosk + # window (no browser UI, locks to the screen). + args.append('--kiosk') + return args + def launch(self, url, width, height): # A dedicated --user-data-dir is mandatory: without it Chrome hands # the URL to an existing process, the launched process exits @@ -1556,15 +1824,28 @@ def _patch_main(): def _windows_weblink_adapter_factory(player): """Choose the Windows web-link engines, best first. - CEF (embedded) is preferred when available because it cannot fight for - z-order or foreground; the Chrome/Edge subprocess is the fallback. + Order matters: + + 1. **WebView2 (embedded)** — a child HWND of the Kivy window. It cannot + open behind the player, cannot be handed off to an existing browser + and exit instantly, does not fight for z-order/foreground, and + leaves no leaked process behind. This is the preferred engine. + 2. **CEF (embedded)** — same in-window model, but cefpython3 has no + wheels past Python 3.9 so it is dormant on this build. + 3. **Chrome/Edge subprocess** — last resort only. Kept so a machine + without the WebView2 runtime still shows weblinks. """ adapters = [] + if _get_webview2_browser() is not None: + adapters.append(_WinWebView2Adapter()) if _get_cef_browser() is not None: adapters.append(_WinCefAdapter()) browser = _windows_find_browser() if browser: adapters.append(_WinChromeAdapter(browser)) + if not adapters: + _log("WebView2/CEF unavailable and no Chrome/Edge found — " + "web links will be skipped") return adapters signage_main.SignagePlayer.weblink_adapter_factory = staticmethod( @@ -1657,6 +1938,85 @@ def _patch_main(): signage_main.SettingsPopup.test_connection = _windows_test_connection + # ── Keep player auth OUT of the bundle ─────────────────────────── + # `player_auth.json` used to be a tracked file inside src/, so PyInstaller + # bundled it and, because the frozen app runs with cwd = _internal/, the + # player loaded it as its live auth state. A stale snapshot therefore made + # a fresh build boot "already authenticated" against an old server and play + # an outdated playlist. + # + # Fix: never read a relative auth path from the bundle. Any relative + # 'player_auth.json' is redirected to the data dir next to the .exe, which + # is the single source of truth for this install. + import player_auth as _player_auth_module + + _bundled_auth = os.path.join( + getattr(sys, '_MEIPASS', os.path.dirname(os.path.abspath(__file__))), + 'player_auth.json', + ) + _local_auth = os.path.join(DATA_DIR, 'player_auth.json') + + # If an older build left the bundled snapshot next to the exe, drop it: + # it names another server and would be trusted on the next start-up. + try: + if os.path.isfile(_local_auth): + import json as _json + + with open(_local_auth, 'r') as _f: + _existing = _json.load(_f) + _existing_url = str(_existing.get('server_url') or '') + _wanted_ip = str(os.environ.get('KIWY_SERVER_IP') or '') + if _wanted_ip and _wanted_ip not in _existing_url: + Logger.warning( + "SignagePlayer: stored auth points at %s but the configured " + "server is %s - clearing it so the player re-authenticates", + _existing_url or '(none)', _wanted_ip, + ) + os.remove(_local_auth) + except Exception as _exc: + Logger.debug(f"SignagePlayer: auth pre-check skipped: {_exc}") + + _original_auth_init = _player_auth_module.PlayerAuth.__init__ + + def _windows_auth_init(self, config_file='player_auth.json', + use_https=True, verify_ssl=True): + """Force the auth file to live in the data dir next to the .exe.""" + try: + if not os.path.isabs(config_file): + config_file = os.path.join(DATA_DIR, os.path.basename(config_file)) + except Exception: + config_file = _local_auth + _original_auth_init(self, config_file, use_https=use_https, verify_ssl=verify_ssl) + + _player_auth_module.PlayerAuth.__init__ = _windows_auth_init + Logger.info(f"SignagePlayer: player auth file -> {_local_auth}") + + # `get_playlists_v2` caches a global auth instance created with the default + # relative path; make sure its cache is empty so the redirect above applies. + try: + import get_playlists_v2 as _gp + + if _gp._auth_instance is not None: + _gp._auth_instance = None + except Exception as _exc: + Logger.debug(f"SignagePlayer: could not reset auth cache: {_exc}") + + # `reset_player_auth` deleted the file next to main.py (i.e. inside the + # bundle, which is read-only and not what we load). Point it at the real + # auth file so the "Reset auth" button actually works. + def _windows_reset_player_auth(self): + try: + if os.path.exists(_local_auth): + os.remove(_local_auth) + Logger.info(f"SettingsPopup: Deleted authentication file: {_local_auth}") + self._show_temp_message( + '✓ Authentication reset - will reauthenticate on restart', (0, 1, 0, 1) + ) + except Exception as exc: + Logger.error(f"SettingsPopup: Failed to reset auth: {exc}") + + signage_main.SettingsPopup.reset_player_auth = _windows_reset_player_auth + # ── Patch apply_kiosk_mode for Windows ────────────────────────── # Wrap the base implementation (exit_on_escape + close guard + Ctrl+C) # and add the low-level keyboard hook that swallows Alt+F4 / Alt+Tab / @@ -1702,6 +2062,13 @@ def _patch_main(): Logger.info("SignagePlayer: Windows card reader shut down") except Exception as e: Logger.debug(f"SignagePlayer: card reader shutdown error: {e}") + # Tear down the embedded web engine. Because it is a child HWND (not a + # subprocess) this is what stops it surviving the player on exit. + try: + _get_webview2_browser(force_new=True) + Logger.info("SignagePlayer: WebView2 browser shut down") + except Exception as e: + Logger.debug(f"SignagePlayer: WebView2 shutdown error: {e}") _original_on_stop(self) signage_main.SignagePlayerApp.on_stop = _windows_on_stop @@ -1829,6 +2196,14 @@ if __name__ == '__main__': Logger.info(f"Data directory: {DATA_DIR}") Logger.info("=" * 80) + # ── Ensure the WebView2 Runtime is present ─────────────────── + # The bundled SDK is only an API surface; the Runtime holds the actual + # engine. Windows 11 and most Windows 10 machines already have it, but + # when it is missing we install it silently (per-user, so no UAC prompt + # appears on the signage display). Runs in the background so start-up + # is not blocked. + _ensure_webview2_runtime_async() + # Patch base_dir in SignagePlayer instances to point to local data folder _original_init = patched_main.SignagePlayer.__init__ diff --git a/windows/start_player_watchdog.bat b/windows/start_player_watchdog.bat new file mode 100644 index 0000000..63b7f64 --- /dev/null +++ b/windows/start_player_watchdog.bat @@ -0,0 +1,48 @@ +@echo off +REM ============================================================ +REM Kiwy Signage Player - Watchdog Launcher (Windows) +REM ============================================================ +REM Starts watchdog.ps1, which keeps the player running 24/7: +REM * restarts it if it crashes +REM * restarts it if it hangs (stale heartbeat) +REM * backs off if it cannot stay up (crash-loop breaker) +REM +REM The WATCHDOG runs in this window. Closing it stops supervision +REM (it does NOT stop the player). +REM +REM The ONLY supported way to stop the player is the exit screen +REM password. That writes .player_stop_requested next to the .exe, +REM which tells the watchdog not to restart it. Launching again +REM starts a fresh session and clears that flag. +REM +REM Usage: +REM start_player_watchdog.bat start supervising +REM start_player_watchdog.bat -Status report state, then exit +REM ============================================================ + +setlocal +cd /d "%~dp0" + +echo ============================================ +echo Kiwy Signage Player - Watchdog +echo ============================================ +echo. +echo Stop the WATCHDOG with Ctrl+C. +echo Stop the PLAYER via the exit password. +echo. + +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0watchdog.ps1" %* + +set "RC=%ERRORLEVEL%" + +if not "%RC%"=="0" ( + echo. + echo [ERROR] Watchdog exited with code %RC%. + echo Check logs\watchdog.log next to the player executable. + echo. +) + +REM Keep the window open so a crash is readable when double-clicked. +if "%~1"=="" pause +endlocal +exit /b %RC% diff --git a/windows/test_video_hang.py b/windows/test_video_hang.py new file mode 100644 index 0000000..b580cf3 --- /dev/null +++ b/windows/test_video_hang.py @@ -0,0 +1,139 @@ +"""Proves src/video_safety.py stops the hang at the end of a video. + +Reproduces the exact failure: + Kivy's VideoFFPy.unload() does `self._thread.join()` with NO timeout. If the + ffpyplayer decode thread does not exit, the calling thread (the Kivy main + thread, via Kivy's own on_eos handler setting state='stop') blocks forever and + Windows reports the app as hung (AppHangB1). + +Two checks: + 1. The guard installs on the REAL Kivy provider (VideoFFPy.play wrapped). + 2. A deliberately wedged decode thread makes unload() return promptly + instead of blocking forever. + +Run: windows\\venv\\Scripts\\python.exe windows\\test_video_hang.py +Exit code 0 = PASS (the hang is prevented). +""" + +import sys +import threading +import time +from pathlib import Path + +SRC = Path(__file__).resolve().parent.parent / 'src' +sys.path.insert(0, str(SRC)) + +import video_safety # noqa: E402 + +# A short-lived "decode thread" that ignores the quit request, standing in for +# an ffpyplayer thread stuck inside a codec/close call. +WEDGE_SECONDS = 30 + + +class _WedgedProvider: + """Minimal stand-in for VideoFFPy, with the same blocking unload().""" + + def __init__(self): + self._thread = threading.Thread(target=self._decode_loop, daemon=True) + self._thread.start() + + def _decode_loop(self): + # Ignores any "please quit" flag and holds the thread — this is what a + # slow ffpyplayer teardown looks like to unload(). + time.sleep(WEDGE_SECONDS) + + def play(self, *args, **kwargs): + return True + + def unload(self): + # Verbatim shape of kivy/core/video/video_ffpyplayer.py unload(): + # if self._thread: + # self._thread.join() # <-- no timeout: hangs forever + if self._thread: + self._thread.join() + self._thread = None + + +def main(): + print('=' * 68) + print(' Kivy video-teardown hang test') + print('=' * 68) + + ok = True + + # ── 1. Does the guard install on the real provider? ────────────── + print('\n[1] guard installation') + try: + from kivy.core.video import video_ffpyplayer as vfp + + provider = vfp.VideoFFPy + print(f' provider: {provider.__module__}.{provider.__name__}') + except Exception as exc: + print(f' SKIP: ffpyplayer provider not available ({exc})') + print(' (the packaged app uses it, so this must pass there)') + return 0 + + installed = video_safety.suppress_kivy_video_blocking_unload(timeout=2.0) + print(f' suppress_kivy_video_blocking_unload() -> {installed}') + wrapped = getattr(provider, '_kiwy_bounded_join', False) + print(f' provider.play wrapped -> {wrapped}') + if not (installed and wrapped): + print(' FAIL: guard not installed') + ok = False + + # The existing Video._do_video_load path must be untouched (no source change). + try: + from kivy.uix.video import Video + + print(f' kivy.uix.video.Video unload: ' + f'{"unload" in dir(Video)}') + except Exception as exc: + print(f' note: could not import Video ({exc})') + + # ── 2. Does a wedged thread still block? ───────────────────────── + print(f'\n[2] wedged decode thread (holds {WEDGE_SECONDS}s)') + prov = _WedgedProvider() + # Install the same bound join the guard installs on a real provider. + video_safety._bound_thread_join(prov, 2.0) + patched = getattr(prov._thread, '_kiwy_bounded_join', False) + print(f' thread join bounded -> {patched}') + if not patched: + print(' FAIL: thread join was not bounded') + ok = False + + started = time.monotonic() + prov.unload() # would hang forever without the fix + elapsed = time.monotonic() - started + print(f' unload() returned after {elapsed:.2f}s') + if elapsed > 5.0: + print(f' FAIL: unload() blocked for {elapsed:.1f}s (expected < 5s)') + ok = False + else: + print(' OK: unload() no longer blocks the caller indefinitely') + + # ── 3. Control: show the unpatched case really would hang ─────── + print('\n[3] control (unpatched join, 2s probe to prove it blocks)') + prov2 = _WedgedProvider() + blocked = True + t = threading.Thread(target=prov2.unload, daemon=True) + t.start() + t.join(timeout=2.0) + blocked = t.is_alive() + print(f' unpatched unload() still blocked after 2s -> {blocked}') + if not blocked: + print(' note: control did not block (timing); fix still valid') + else: + print(' OK: confirms the original join() is the hang, and the fix ' + 'is what prevents it') + + print('=' * 68) + print(' RESULT:', 'PASS' if ok else 'FAIL') + if ok: + print(' A slow/wedged ffpyplayer teardown can no longer freeze the') + print(' player at the end of a video item.') + print('=' * 68) + return 0 if ok else 1 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/windows/test_watchdog.py b/windows/test_watchdog.py new file mode 100644 index 0000000..2534e01 --- /dev/null +++ b/windows/test_watchdog.py @@ -0,0 +1,438 @@ +"""Verifies windows/watchdog.ps1 actually recovers the player. + +Three scenarios, each with a real player process: + + 1. CRASH - kill the player (simulating a crash) and confirm the watchdog + brings it back and it becomes healthy again. + 2. HANG - make the heartbeat go stale while the process stays alive, and + confirm the watchdog kills it and restarts it. + 3. STOP FLAG - write the stop-flag file (what the password exit does) and + confirm the watchdog stands down instead of restarting. + Then confirm a fresh watchdog run CLEARS the flag (new session). + +The watchdog is started as a child process with a short check interval so the +test is quick. It is stopped at the end; the stop flag it may have created is +removed so the machine is left as it was found. + +Run: windows\\venv\\Scripts\\python.exe windows\\test_watchdog.py +Exit code 0 = PASS. +""" + +import subprocess +import sys +import time +from pathlib import Path + +WIN = Path(__file__).resolve().parent +PLAYER_DIR = WIN / 'dist' / 'KiwySignagePlayer' +EXE = PLAYER_DIR / 'KiwySignagePlayer.exe' +HEARTBEAT = PLAYER_DIR / '.player_heartbeat' +STOP_FLAG = PLAYER_DIR / '.player_stop_requested' +WATCHDOG = WIN / 'watchdog.ps1' +LOG = PLAYER_DIR / 'logs' / 'watchdog.log' + +PS = ['powershell', '-NoProfile', '-ExecutionPolicy', 'Bypass', '-File'] + +# Fast timings so the test finishes in a reasonable time. +FAST = ['-HealthCheckIntervalSec', '3', '-HeartbeatStaleSec', '10', + '-StartupGraceSec', '45', '-RestartDelaySec', '2', + '-CrashLoopMaxFailures', '50', '-CrashLoopBackoffMin', '1'] + + +def kill_players(): + subprocess.run(['taskkill', '/F', '/T', '/IM', 'KiwySignagePlayer.exe'], + capture_output=True, text=True) + + +def player_count(): + out = subprocess.run( + ['powershell', '-NoProfile', '-Command', + "(Get-Process -Name KiwySignagePlayer -ErrorAction SilentlyContinue " + "| Measure-Object).Count"], + capture_output=True, text=True) + try: + return int((out.stdout or '0').strip() or 0) + except ValueError: + return 0 + + +def hb_age(): + if not HEARTBEAT.is_file(): + return -1 + return time.time() - HEARTBEAT.stat().st_mtime + + +def get_root_pid(): + """Return the pid of the player process that owns the window. + + The packaged player is TWO processes (PyInstaller bootloader + child). + """ + out = subprocess.run( + ['powershell', '-NoProfile', '-Command', + "Get-Process -Name KiwySignagePlayer -ErrorAction SilentlyContinue | " + "Where-Object { $_.MainWindowHandle -ne 0 } | " + "Select-Object -First 1 -ExpandProperty Id"], + capture_output=True, text=True) + try: + return int((out.stdout or '').strip()) + except ValueError: + # Fall back to any player process. + out = subprocess.run( + ['powershell', '-NoProfile', '-Command', + "Get-Process -Name KiwySignagePlayer -ErrorAction SilentlyContinue | " + "Select-Object -First 1 -ExpandProperty Id"], + capture_output=True, text=True) + try: + return int((out.stdout or '').strip()) + except ValueError: + return None + + +# ── Exclusive file lock (real Windows share-mode lock) ─────────────── +# msvcrt.locking only locks a byte RANGE within the file, and Python's +# open() already shares the file for writing, so the player can still replace +# its contents. Opening with share mode 0 denies ALL other access instead, +# which is what actually stops the heartbeat being rewritten. +import ctypes # noqa: E402 +import ctypes.wintypes as wintypes # noqa: E402 + +GENERIC_READ = 0x80000000 +GENERIC_WRITE = 0x40000000 +OPEN_EXISTING = 3 +FILE_ATTRIBUTE_NORMAL = 0x80 +INVALID_HANDLE_VALUE = ctypes.c_void_p(-1).value + +_kernel32 = ctypes.WinDLL('kernel32', use_last_error=True) +_kernel32.CreateFileW.restype = ctypes.c_void_p +_kernel32.CreateFileW.argtypes = [ + ctypes.c_wchar_p, wintypes.DWORD, wintypes.DWORD, ctypes.c_void_p, + wintypes.DWORD, wintypes.DWORD, ctypes.c_void_p, +] +_kernel32.CloseHandle.argtypes = [ctypes.c_void_p] + + +def lock_file_exclusive(path): + """Open ``path`` denying all sharing. Returns the handle, or None.""" + handle = _kernel32.CreateFileW( + str(path), + GENERIC_READ | GENERIC_WRITE, + 0, # share mode 0: nobody else may touch it + None, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL, + None, + ) + if handle in (None, INVALID_HANDLE_VALUE): + err = ctypes.get_last_error() + print(f' note: CreateFileW failed (winerror={err})') + return None + return handle + + +def unlock_file(handle): + if handle: + try: + _kernel32.CloseHandle(handle) + except Exception: + pass + + +def tail_log(n=14): + if not LOG.is_file(): + return [] + lines = LOG.read_text(encoding='utf-8', errors='replace').splitlines() + return lines[-n:] + + +def main(): + # Tee stdout to a UTF-8 report file: the console in this environment + # mangles encoding between processes, so the file is the reliable record. + report_path = WIN / 'watchdog_test_report.txt' + + class _Tee: + def __init__(self, stream, path): + self._stream = stream + self._path = path + + def write(self, text): + self._stream.write(text) + try: + with open(self._path, 'a', encoding='utf-8') as fh: + fh.write(text) + except Exception: + pass + + def flush(self): + try: + self._stream.flush() + except Exception: + pass + + try: + report_path.unlink() + except Exception: + pass + sys.stdout = _Tee(sys.__stdout__, report_path) + + print('=' * 70) + print(' Watchdog recovery test') + print('=' * 70) + + if not EXE.is_file(): + print(f'FAIL: player exe not found at {EXE}') + return 1 + if not WATCHDOG.is_file(): + print(f'FAIL: watchdog not found at {WATCHDOG}') + return 1 + + ok = True + watchdog = None + + # Clean slate: no player, no leftover stop flag. + kill_players() + if STOP_FLAG.exists(): + STOP_FLAG.unlink() + time.sleep(2) + + try: + # --------------------------------------------------------------- + # Start the watchdog; it should launch the player itself. + # --------------------------------------------------------------- + print('\n[setup] starting watchdog (it should launch the player)') + watchdog = subprocess.Popen( + PS + [str(WATCHDOG)] + FAST, + cwd=str(WIN), + creationflags=getattr(subprocess, 'CREATE_NEW_CONSOLE', 0), + ) + + # Wait for the player to come up and report a heartbeat. + deadline = time.time() + 90 + healthy = False + while time.time() < deadline: + time.sleep(3) + if player_count() > 0 and 0 <= hb_age() < 10: + healthy = True + break + print(f' player processes={player_count()} hb_age={hb_age():.0f}s') + if not healthy: + print(' FAIL: player never became healthy under the watchdog') + ok = False + else: + print(' OK: watchdog launched the player and it is healthy') + + # --------------------------------------------------------------- + # 1. CRASH recovery + # --------------------------------------------------------------- + if healthy: + print('\n[1] CRASH: killing the player (simulating a crash)') + kill_players() + time.sleep(2) + print(f' after kill: processes={player_count()}') + + recovered = False + deadline = time.time() + 150 + while time.time() < deadline: + time.sleep(3) + if player_count() > 0 and 0 <= hb_age() < 10: + recovered = True + break + print(f' after recovery: processes={player_count()} hb_age={hb_age():.0f}s') + if recovered: + print(' OK: watchdog restarted the player after the crash') + else: + print(' FAIL: watchdog did not restore a healthy player') + ok = False + + # ----------------------------------------------------------- + # 2. HANG recovery (stale heartbeat, process still alive) + # ----------------------------------------------------------- + # IMPORTANT: backdating the heartbeat's mtime does NOT simulate a + # hang - the healthy player rewrites it on its next tick, so the + # watchdog never sees it stale (that was a false pass in an earlier + # version of this test). + # + # A genuine hang means "process alive, but it stopped updating the + # heartbeat". We reproduce exactly that: hold the heartbeat open + # with an exclusive Windows lock (share mode 0), so the player's + # own write fails. The player catches that write error, logs a + # warning and KEEPS RUNNING - the mtime simply stops advancing. + # That is precisely the condition the watchdog tests for. + print('\n[2] HANG: freezing the heartbeat (player stays alive)') + pid_before = get_root_pid() + if pid_before is None: + print(' FAIL: could not find the player process') + ok = False + else: + handle = lock_file_exclusive(HEARTBEAT) + if handle is None: + print(' FAIL: could not lock the heartbeat file') + ok = False + else: + print(f' holding an exclusive lock on the heartbeat; ' + f'player pid={pid_before} should stay alive') + + # Wait for the heartbeat to go stale while the process lives. + stale_seen = False + deadline = time.time() + 90 + while time.time() < deadline: + time.sleep(2) + if hb_age() > 15: + stale_seen = True + break + still_alive = player_count() > 0 + print(f' heartbeat age={hb_age():.0f}s stale={stale_seen}; ' + f'player still running={still_alive}') + if not (stale_seen and still_alive): + print(' FAIL: could not create a genuine stale-heartbeat hang') + ok = False + + # The watchdog should kill the frozen player... + old_killed = False + deadline = time.time() + 90 + while time.time() < deadline: + time.sleep(2) + if get_root_pid() != pid_before: + old_killed = True + break + print(f' watchdog killed the stale player: {old_killed}') + + # ...release the lock so the replacement can write... + unlock_file(handle) + + # ...and confirm a fresh, healthy player is now running. + restarted = False + deadline = time.time() + 150 + while time.time() < deadline: + time.sleep(3) + if player_count() > 0 and 0 <= hb_age() < 10: + restarted = True + break + print(f' after hang recovery: processes={player_count()} ' + f'hb_age={hb_age():.0f}s') + if old_killed and restarted: + print(' OK: watchdog detected the hang and restarted the player') + else: + print(' FAIL: watchdog did not recover from the hang') + ok = False + + # --------------------------------------------------------------- + # 3. STOP FLAG: the password exit must not be undone + # --------------------------------------------------------------- + print('\n[3] STOP FLAG: simulating the password exit') + STOP_FLAG.write_text('User requested exit via password', encoding='utf-8') + print(' wrote the stop flag') + + # The player is running; give the watchdog time to notice and stand down. + stood_down = False + deadline = time.time() + 90 + while time.time() < deadline: + time.sleep(3) + if watchdog.poll() is not None: + stood_down = True + break + print(f' watchdog exited={stood_down} (rc={watchdog.poll()})') + if stood_down: + print(' OK: watchdog stood down instead of restarting') + else: + print(' FAIL: watchdog did not stand down on the stop flag') + ok = False + + # Confirm it really stops restarting: kill the player and verify it + # stays dead now that the watchdog is gone. + kill_players() + time.sleep(8) + if player_count() == 0: + print(' OK: player stayed stopped (no supervisor resurrecting it)') + else: + print(' FAIL: player was restarted despite the stop flag') + ok = False + + # --------------------------------------------------------------- + # 4. NEW SESSION: a fresh watchdog run clears the flag + # --------------------------------------------------------------- + print('\n[4] NEW SESSION: starting the watchdog again') + if not STOP_FLAG.exists(): + print(' FAIL: stop flag vanished unexpectedly') + ok = False + session = subprocess.Popen( + PS + [str(WATCHDOG), '-Status'], + cwd=str(WIN), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, + text=True) + # -Status must NOT clear the flag (it is read-only). + try: + session.wait(timeout=60) + except subprocess.TimeoutExpired: + session.kill() + if STOP_FLAG.exists(): + print(' OK: -Status is read-only (flag untouched)') + else: + print(' FAIL: -Status cleared the flag (should be read-only)') + ok = False + + # A real run clears it, then launches the player. + fresh = subprocess.Popen( + PS + [str(WATCHDOG)] + FAST, + cwd=str(WIN), + creationflags=getattr(subprocess, 'CREATE_NEW_CONSOLE', 0)) + cleared = False + deadline = time.time() + 60 + while time.time() < deadline: + time.sleep(2) + if not STOP_FLAG.exists(): + cleared = True + break + print(f' stop flag cleared by the new session: {cleared}') + if cleared: + print(' OK: a fresh launch starts a new session (flag cleared)') + else: + print(' FAIL: the new session did not clear the stop flag') + ok = False + + # And the player comes back up. + back_up = False + deadline = time.time() + 120 + while time.time() < deadline: + time.sleep(3) + if player_count() > 0 and 0 <= hb_age() < 10: + back_up = True + break + print(f' player back up: {back_up} (processes={player_count()})') + if back_up: + print(' OK: player resumed supervision after the new session') + else: + print(' FAIL: player did not come back in the new session') + ok = False + + fresh.terminate() + + finally: + # --------------------------------------------------------------- + # Leave the machine as we found it. + # --------------------------------------------------------------- + for p in (watchdog,): + if p is not None and p.poll() is None: + p.terminate() + subprocess.run( + ['powershell', '-NoProfile', '-Command', + "Get-CimInstance Win32_Process -Filter \"Name='powershell.exe'\" | " + "Where-Object { $_.CommandLine -like '*watchdog.ps1*' } | " + "ForEach-Object { Stop-Process -Id $_.ProcessId -Force }"], + capture_output=True, text=True) + kill_players() + if STOP_FLAG.exists(): + STOP_FLAG.unlink() + print('\n[cleanup] player stopped, watchdog stopped, stop flag removed') + + print('\n=== last watchdog log lines ===') + for line in tail_log(12): + print(' ' + line) + + print('=' * 70) + print(' RESULT:', 'PASS' if ok else 'FAIL') + print('=' * 70) + return 0 if ok else 1 + + +if __name__ == '__main__': + sys.exit(main()) + diff --git a/windows/watchdog.ps1 b/windows/watchdog.ps1 new file mode 100644 index 0000000..7a76ff7 --- /dev/null +++ b/windows/watchdog.ps1 @@ -0,0 +1,441 @@ +<# +===================================================================== + watchdog.ps1 - keep the Kiwy Signage Player running 24/7 on Windows +===================================================================== + +WHAT THIS DOES + Supervises KiwySignagePlayer.exe and restarts it when it: + + * CRASHES - the process disappeared without the user asking it to. + * HANGS - the process is alive but its heartbeat file has gone stale, + which means the UI thread is wedged (a frozen player looks + exactly like a working one from the outside). + + This is the Windows counterpart of the Linux `start.sh` watchdog, which has + been running the player on the Raspberry Pi deployments. + +HOW THE "PASSWORD ONLY" EXIT IS PRESERVED + The only supported way to stop the player is the exit screen's password. + On success the player writes a stop-flag file next to its .exe: + + .player_stop_requested + + The flag is SESSION SCOPED: + + * While the flag exists, this watchdog will NOT restart the player; it + stops supervising and exits (the machine is left with the player off). + * Every time the watchdog starts, it CLEARS the flag first, which is what + makes a fresh launch a fresh session. So starting the player again is + all it takes to resume - there is no file to delete by hand. + +CRASH-LOOP BREAKER + A player that cannot stay up (bad config, missing media, GPU problem) would + otherwise be restarted forever. If the player dies `CrashLoopMaxFailures` + times inside `CrashLoopWindowMin` minutes WITHOUT ever becoming healthy, + the watchdog backs off for `CrashLoopBackoffMin` minutes before retrying, + and keeps the reason in the log. + +WHAT IT DOES NOT DO + It does not install itself to run at boot/login - start it with + `start_player_watchdog.bat`. It also cannot watch the player before the + player has ever run, so a totally broken install simply logs and backs off. + +USAGE + powershell -NoProfile -ExecutionPolicy Bypass -File watchdog.ps1 + powershell -NoProfile -ExecutionPolicy Bypass -File watchdog.ps1 -Status + + Stop it with Ctrl+C in its window (this does NOT stop the player; it only + stops supervising). +#> + +[CmdletBinding()] +param( + # Path to the player executable. Defaults to the standard build output. + [string]$ExePath, + + # How often the supervisor checks on the player. + [int]$HealthCheckIntervalSec = 15, + + # A heartbeat older than this means the player is wedged, since the player + # rewrites it every 10 seconds. Matches the proven Linux value of 60s. + [int]$HeartbeatStaleSec = 60, + + # After launching, give the player this long to write its first heartbeat + # (startup shows a splash video, so it is not instant) before health + # checks count against it. + [int]$StartupGraceSec = 120, + + # Pause before restarting a player that died. + [int]$RestartDelaySec = 5, + + # Crash-loop breaker (see header). + [int]$CrashLoopWindowMin = 10, + [int]$CrashLoopMaxFailures = 5, + [int]$CrashLoopBackoffMin = 10, + + # Report current state and exit - never launches or kills anything. + [switch]$Status +) + +$ErrorActionPreference = 'Stop' + +# UTF-8 so accented paths in logs are readable. +$OutputEncoding = [System.Text.Encoding]::UTF8 + +# --------------------------------------------------------------------- +# Paths +# --------------------------------------------------------------------- +function Resolve-ExePath { + param([string]$Override) + + if ($Override) { + if (-not (Test-Path -LiteralPath $Override)) { + throw "Player executable not found: $Override" + } + return (Resolve-Path -LiteralPath $Override).Path + } + + $candidates = @( + (Join-Path $PSScriptRoot 'dist\KiwySignagePlayer\KiwySignagePlayer.exe'), + (Join-Path $PSScriptRoot 'KiwySignagePlayer.exe') + ) + foreach ($c in $candidates) { + if (Test-Path -LiteralPath $c) { + return (Resolve-Path -LiteralPath $c).Path + } + } + throw ("Could not find KiwySignagePlayer.exe. Looked in:`n " + + ($candidates -join "`n ")) +} + +$ExeFull = Resolve-ExePath -Override $ExePath +$PlayerDir = Split-Path -Parent $ExeFull +$HeartbeatFile = Join-Path $PlayerDir '.player_heartbeat' +$StopFlagFile = Join-Path $PlayerDir '.player_stop_requested' +$LogDir = Join-Path $PlayerDir 'logs' +$LogFile = Join-Path $LogDir 'watchdog.log' +$ProcessName = [System.IO.Path]::GetFileNameWithoutExtension($ExeFull) + +New-Item -ItemType Directory -Force -Path $LogDir | Out-Null + +# --------------------------------------------------------------------- +# Logging (kept tiny and never throws) +# --------------------------------------------------------------------- +function Write-Log { + param( + [Parameter(Mandatory = $true)][string]$Message, + [ValidateSet('INFO', 'WARN', 'ERROR')][string]$Level = 'INFO' + ) + $line = "[{0}] [{1}] {2}" -f (Get-Date -Format 'yyyy-MM-dd HH:mm:ss'), $Level, $Message + Write-Host $line + try { + Add-Content -LiteralPath $LogFile -Value $line -Encoding UTF8 + } catch { + # Logging must never take the supervisor down. + } +} + +# --------------------------------------------------------------------- +# Player process helpers +# --------------------------------------------------------------------- +function Get-PlayerProcesses { + # NOTE: the packaged player is TWO processes on Windows - the PyInstaller + # bootloader parent and the child that owns the SDL window. Both share the + # image name, so selecting by name covers the whole tree. + @(Get-Process -Name $ProcessName -ErrorAction SilentlyContinue) +} + +function Get-PlayerRootPid { + $procs = Get-PlayerProcesses + if ($procs.Count -eq 0) { return $null } + # Prefer the child (it has the main window); fall back to any. + $withWindow = $procs | Where-Object { $_.MainWindowHandle -ne 0 } | Select-Object -First 1 + if ($withWindow) { return $withWindow.Id } + return ($procs | Select-Object -First 1).Id +} + +function Test-HeartbeatFresh { + param([int]$MaxAgeSec) + + if (-not (Test-Path -LiteralPath $HeartbeatFile)) { + return $false + } + try { + $age = (Get-Date).ToUniversalTime() - ` + (Get-Item -LiteralPath $HeartbeatFile).LastWriteTimeUtc + return ($age.TotalSeconds -lt $MaxAgeSec) + } catch { + return $false + } +} + +function Get-HeartbeatAgeSec { + if (-not (Test-Path -LiteralPath $HeartbeatFile)) { return -1 } + try { + $age = (Get-Date).ToUniversalTime() - ` + (Get-Item -LiteralPath $HeartbeatFile).LastWriteTimeUtc + return [int]$age.TotalSeconds + } catch { + return -1 + } +} + +function Stop-PlayerTree { + param([int]$RootPid) + + # /T kills the PyInstaller child too; without it the visible player window + # would survive and the next launch would collide with it. + try { + & taskkill.exe /F /T /PID $RootPid 2>&1 | Out-Null + } catch { + Write-Log "taskkill failed for pid $RootPid ($($_.Exception.Message)); forcing by name" 'WARN' + } + + # Belt and braces: make sure no stragglers remain. + $deadline = (Get-Date).AddSeconds(15) + while ((Get-Date) -lt $deadline) { + if ((Get-PlayerProcesses).Count -eq 0) { return $true } + Start-Sleep -Milliseconds 500 + } + foreach ($p in Get-PlayerProcesses) { + try { Stop-Process -Id $p.Id -Force -ErrorAction SilentlyContinue } catch { } + } + Start-Sleep -Milliseconds 500 + return ((Get-PlayerProcesses).Count -eq 0) +} + +function Start-Player { + Write-Log "Launching player: $ExeFull" + # Start in the player's own folder: the app resolves config/media/playlists + # relative to its working directory. + Start-Process -FilePath $ExeFull -WorkingDirectory $PlayerDir | Out-Null +} + +# --------------------------------------------------------------------- +# -Status : report and exit (never mutates anything) +# --------------------------------------------------------------------- +if ($Status) { + $procs = Get-PlayerProcesses + $pid0 = Get-PlayerRootPid + $age = Get-HeartbeatAgeSec + Write-Host '==========================================' + Write-Host ' Kiwy Signage Player - Watchdog Status' + Write-Host '==========================================' + Write-Host ("Executable : {0}" -f $ExeFull) + Write-Host ("Processes : {0}" -f $procs.Count) + if ($pid0) { Write-Host ("Root PID : {0}" -f $pid0) } + if ($age -ge 0) { + Write-Host ("Heartbeat : {0}s old" -f $age) + if ($age -lt $HeartbeatStaleSec) { + Write-Host 'Health : HEALTHY' -ForegroundColor Green + } else { + Write-Host 'Health : STALE (player may be hung)' -ForegroundColor Yellow + } + } else { + Write-Host 'Heartbeat : (not present - player has not started)' + } + if (Test-Path -LiteralPath $StopFlagFile) { + Write-Host 'Stop flag : PRESENT (password exit was used)' + Write-Host ' a new launch clears it' -ForegroundColor Yellow + } else { + Write-Host 'Stop flag : absent (watchdog would restart on failure)' + } + Write-Host ("Log : {0}" -f $LogFile) + exit 0 +} + +# --------------------------------------------------------------------- +# Session start: clear the stop flag +# --------------------------------------------------------------------- +# This is what makes the flag session-scoped: the user's password exit ends +# the CURRENT session, and the next launch begins a new one. +if (Test-Path -LiteralPath $StopFlagFile) { + try { + Remove-Item -LiteralPath $StopFlagFile -Force + Write-Log 'Cleared the stop flag from the previous session - new session started' + } catch { + Write-Log "Could not clear the stop flag ($($_.Exception.Message))" 'WARN' + } +} + +Write-Log '==================================================' +Write-Log "Watchdog starting (exe=$ProcessName, check=${HealthCheckIntervalSec}s, stale=${HeartbeatStaleSec}s)" +Write-Log "Crash-loop breaker: ${CrashLoopMaxFailures} failures / ${CrashLoopWindowMin} min -> ${CrashLoopBackoffMin} min backoff" +Write-Log 'Stop the WATCHDOG with Ctrl+C. Stop the PLAYER via the exit password.' +Write-Log '==================================================' + +# --------------------------------------------------------------------- +# Crash-loop breaker +# --------------------------------------------------------------------- +# A player that cannot stay up (broken config, missing media, GPU fault) would +# otherwise be restarted forever, filling the log and thrashing the machine. +# If it fails CrashLoopMaxFailures times inside CrashLoopWindowMin minutes +# WITHOUT ever becoming healthy, wait CrashLoopBackoffMin minutes before the +# next attempt. Returns $true when a backoff was performed. +function Invoke-CrashLoopBackoff { + param([System.Collections.ArrayList]$Failures) + + if ($Failures.Count -lt $CrashLoopMaxFailures) { return $false } + + $windowStart = (Get-Date).AddMinutes(-$CrashLoopWindowMin) + $recent = @($Failures | Where-Object { $_ -gt $windowStart }) + if ($recent.Count -lt $CrashLoopMaxFailures) { + # Only old failures remain; forget them so they cannot accumulate. + $Failures.Clear() + return $false + } + + Write-Log ("Player failed $($recent.Count) times in the last " + + "${CrashLoopWindowMin} minutes without ever becoming healthy.") 'ERROR' + Write-Log "Backing off for ${CrashLoopBackoffMin} minutes before the next attempt." 'ERROR' + Write-Log "Investigate $LogDir (and the player's own logs next to it)." 'ERROR' + + $backoffEnd = (Get-Date).AddMinutes($CrashLoopBackoffMin) + while ((Get-Date) -lt $backoffEnd) { + Start-Sleep -Seconds 5 + if (Test-Path -LiteralPath $StopFlagFile) { + Write-Log 'Stop flag appeared during backoff - standing down.' + $Failures.Clear() + return $true + } + } + $Failures.Clear() + Write-Log 'Backoff finished - resuming supervision.' + return $true +} + +# --------------------------------------------------------------------- +# Main supervision loop +# --------------------------------------------------------------------- +# Two DIFFERENT failures need two different reactions, and confusing them +# would be harmful: +# +# * "never became healthy" - a slow start-up, a bad install, or the +# first-run setup screen. Killing on this would restart forever and could +# interrupt an operator entering settings. Handled by the crash-loop +# breaker, with a long hard cap before we intervene. +# * "was healthy, then went silent" - the app is wedged. This is the case +# that must be restarted promptly. +# +# So a hang is only declared once we have actually SEEN a fresh heartbeat. +$failureTimestamps = New-Object System.Collections.ArrayList +$hasBeenHealthy = $false + +while ($true) { + + # ---- The operator asked to exit: stand down ---------------------- + if (Test-Path -LiteralPath $StopFlagFile) { + Write-Log 'Stop flag present - the operator exited with the password.' + Write-Log 'Watchdog will NOT restart the player. Start it again to resume.' + break + } + + $procs = Get-PlayerProcesses + + # ================================================================ + # Case 1: the player is not running + # ================================================================ + if ($procs.Count -eq 0) { + Write-Log 'Player is not running - starting it.' + + try { + Start-Player + } catch { + Write-Log "Failed to launch the player: $($_.Exception.Message)" 'ERROR' + [void]$failureTimestamps.Add((Get-Date)) + [void](Invoke-CrashLoopBackoff -Failures $failureTimestamps) + Start-Sleep -Seconds $RestartDelaySec + continue + } + + $launchTime = Get-Date + $hasBeenHealthy = $false + + # Watch the start-up window: break early the moment it is healthy, + # and notice immediately if it dies. + $diedEarly = $false + $graceEnd = $launchTime.AddSeconds($StartupGraceSec) + while ((Get-Date) -lt $graceEnd) { + Start-Sleep -Seconds 2 + if (Test-Path -LiteralPath $StopFlagFile) { break } + if ((Get-PlayerProcesses).Count -eq 0) { $diedEarly = $true; break } + if (Test-HeartbeatFresh -MaxAgeSec $HeartbeatStaleSec) { + $hasBeenHealthy = $true + break + } + } + + if ($diedEarly -and -not (Test-Path -LiteralPath $StopFlagFile)) { + [void]$failureTimestamps.Add((Get-Date)) + Write-Log ("Player exited during start-up (within the ${StartupGraceSec}s grace window).") 'WARN' + [void](Invoke-CrashLoopBackoff -Failures $failureTimestamps) + Write-Log ("Waiting ${RestartDelaySec}s before retrying...") + Start-Sleep -Seconds $RestartDelaySec + } elseif ($hasBeenHealthy) { + Write-Log 'Player started and is reporting a fresh heartbeat.' + $failureTimestamps.Clear() + } else { + Write-Log 'Player is running but has not reported a heartbeat yet - continuing to watch.' 'WARN' + } + continue + } + + # ================================================================ + # Case 2: the player is running - is it actually working? + # ================================================================ + if (Test-HeartbeatFresh -MaxAgeSec $HeartbeatStaleSec) { + # Healthy. Forget past failures - only CONSECUTIVE ones matter. + if (-not $hasBeenHealthy -or $failureTimestamps.Count -gt 0) { + Write-Log 'Player is healthy.' + } + $hasBeenHealthy = $true + $failureTimestamps.Clear() + Start-Sleep -Seconds $HealthCheckIntervalSec + continue + } + + $age = Get-HeartbeatAgeSec + $ageText = if ($age -lt 0) { 'no heartbeat file' } else { "${age}s old" } + + if ($hasBeenHealthy) { + # ---- The real hang: it WAS working and has gone silent ---------- + Write-Log ("Player was healthy but its heartbeat is now stale ({0}, limit ${HeartbeatStaleSec}s) - it is hung." -f $ageText) 'ERROR' + + $rootPid = Get-PlayerRootPid + if ($rootPid) { + Write-Log "Killing the hung player (pid $rootPid) and its children..." + [void](Stop-PlayerTree -RootPid $rootPid) + } + $hasBeenHealthy = $false + [void]$failureTimestamps.Add((Get-Date)) + + if (Test-Path -LiteralPath $StopFlagFile) { continue } + [void](Invoke-CrashLoopBackoff -Failures $failureTimestamps) + Write-Log ("Waiting ${RestartDelaySec}s before restarting...") + Start-Sleep -Seconds $RestartDelaySec + continue + } + + # ---- Running but never healthy: be patient, then give up ----------- + # Deliberately generous: a slow machine still inside its start-up window + # must not be killed, and neither must the first-run setup screen. + $stuckLimitSec = $StartupGraceSec * 2 + if ($null -eq $launchTime) { $launchTime = Get-Date } + $upSec = ((Get-Date) - $launchTime).TotalSeconds + + if ($upSec -lt $stuckLimitSec) { + Write-Log ("Player is up but not yet healthy ({0}); still inside the ${stuckLimitSec}s start-up allowance." -f $ageText) 'WARN' + Start-Sleep -Seconds $HealthCheckIntervalSec + continue + } + + Write-Log ("Player has not become healthy within ${stuckLimitSec}s ({0}) - restarting it." -f $ageText) 'ERROR' + $rootPid = Get-PlayerRootPid + if ($rootPid) { [void](Stop-PlayerTree -RootPid $rootPid) } + [void]$failureTimestamps.Add((Get-Date)) + [void](Invoke-CrashLoopBackoff -Failures $failureTimestamps) + Write-Log ("Waiting ${RestartDelaySec}s before retrying...") + Start-Sleep -Seconds $RestartDelaySec +} + +Write-Log 'Watchdog exited.'