ske087/Kiwy-Signage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix video hang and silent-video crash; add 24/7 watchdog

Browse Source 
Two independent failures were killing long unattended runs.

1. HANG at the end of a video (Windows AppHangB1)

The player froze after ~30-45 minutes of looping, always at a video item. The
playback trace stopped dead right after "video_loaded" with no "video_eos" and
no "advance_after_video_eos", and Windows logged AppHangB1 rather than a crash.

Cause, all inside Kivy and verified against the installed source:

  1. ffpyplayer fires on_eos.
  2. Kivy's Video widget binds its OWN handler first (kivy/uix/video.py
     _do_video_load), and that handler sets state = 'stop' DURING the event
     dispatch.
  3. state = 'stop' -> VideoFFPy.stop() -> unload(), which calls
     self._thread.join() with no timeout (the source even carries the comment
     "TODO: use callback, don't block here").
  4. When that decode thread is slow to exit, the Kivy/SDL main thread never
     returns, so the window stops pumping messages.

It is a race, which is why it looked random and only appeared after many videos.

src/video_safety.py bounds that join. ffpyplayer has already been told to quit
and its thread woken before the join, so limiting the wait does not leak work;
it only stops an unresponsive thread from taking the whole player down. The
guard is installed before the Video widget is constructed, because the decode
thread is created during play().

The intro video had the same hazard on the main thread (state='stop' followed by
unload() inside the state callback) and is now torn down on a worker thread like
playlist videos.

2. CRASH in SDL2_mixer.dll (0xc0000005) on a video with no audio stream

Triggered when a silent 4K clip entered the playlist while the item was marked
audio: on. ffpyplayer initialises SDL2_mixer from the FIRST audio file it opens
and reuses those parameters, so a file with no audio stream (rate/channels 0)
makes SDL2_mixer dereference garbage. Muting via volume=0.0 does NOT avoid it -
the audio stream itself must be disabled.

play_video now probes the file with ffprobe and forces mute when it has no audio
track, so such a file can never reach ffpyplayer with sound enabled. The probe
fails safe (assumes audio present) if ffprobe is unavailable.

3. 24/7 supervision (solution A + C)

windows/watchdog.ps1 + start_player_watchdog.bat restart the player when it
crashes (process gone) or hangs (process alive but .player_heartbeat stale),
with a crash-loop breaker that backs off when it cannot stay up. This is the
Windows counterpart of the proven Linux start.sh watchdog.

The exit-screen password remains the only supported way to stop the player. On
success it writes .player_stop_requested next to the .exe and the watchdog
stands down instead of restarting. The flag is SESSION SCOPED: the watchdog
clears it on every start, so launching again begins a new session and there is
no file to delete by hand. Clearing on start also means a power cut cannot leave
the player permanently off.

Deliberately NOT done: a Windows service. A service runs in session 0 with no
desktop, so the player could not render to the screen at all. A login-triggered
startup entry is the correct Windows analogue of the Pi's systemd unit.

Important detail: the packaged player is TWO processes (PyInstaller bootloader
parent plus the child that owns the SDL window), so any kill uses taskkill /T or
the visible window survives and the next launch collides with it.

Verified in the packaged exe over a 7-hour run: 170 playlist restarts, 1365
items, 171 web links launched/visible/ended with zero failures, and no crashes,
no hangs and no leaked browser processes.

Tests: windows/test_video_hang.py and windows/test_watchdog.py. The hang test
deliberately holds the heartbeat open with an exclusive Windows lock (share mode
0) so the player's own write fails - backdating the file's mtime does NOT
simulate a hang, because the healthy player rewrites it immediately and the test
would then pass for the wrong reason.
This commit is contained in:
ske087
2026-09-13 10:14:42 +03:00
parent 9f5409685d
commit d8c6ab0bc5
8 changed files with 1851 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
windows/run_win.py
+377 -2
View File
@@ -251,6 +251,93 @@ def _get_cef_browser():
return _CEF_BROWSER
# ── Embedded WebView2 (Edge) browser ────────────────────────────────
_WEBVIEW2_BROWSER = None
#: How long a weblink will wait for a Runtime that is still installing.
_WEBVIEW2_INSTALL_WAIT = 300.0
def _ensure_webview2_runtime_async():
"""Start installing the WebView2 Runtime if it is missing.
Called once at start-up. The install runs on a background thread so a
machine without the Runtime can install it while the player is still
syncing its playlist, rather than freezing the UI on the first weblink.
"""
try:
import webview2_runtime
except Exception as exc:
_log(f"WebView2 runtime module unavailable: {exc}")
return
try:
_log(webview2_runtime.describe())
webview2_runtime.ensure_runtime_async()
except Exception as exc:
_log(f"WebView2 runtime check failed (non-fatal): {exc}")
def _get_webview2_browser(force_new=False):
"""Return the shared WebView2Browser singleton, or None if unavailable.
WebView2 renders as a CHILD HWND of Kivy's SDL window, which is what makes
it immune to the subprocess weblink bugs (background window, instant
hand-off exit, z-order fights, leaked browsers).
"""
global _WEBVIEW2_BROWSER
if force_new:
old, _WEBVIEW2_BROWSER = _WEBVIEW2_BROWSER, None
if old is not None:
try:
old.shutdown()
except Exception:
pass
if _WEBVIEW2_BROWSER is None:
try:
from webview2_browser import WebView2Browser
except Exception as exc:
_log(f"WebView2 module unavailable: {exc}")
return None
if not WebView2Browser.is_available():
# The SDK is bundled but the Runtime may still be installing
# (a machine that shipped without it). Wait here rather than
# silently downgrading to the leaky Chrome/Edge engine — this runs
# on the watcher thread, not the Kivy main thread.
if _wait_for_webview2_runtime():
pass # installed in the meantime; retry below
if not WebView2Browser.is_available():
reason = getattr(WebView2Browser, '_import_error', None) or 'unavailable'
_log(f"WebView2 not usable: {reason}")
return None
try:
data_dir = os.environ.get('KIWY_DATA_DIR', os.getcwd())
_WEBVIEW2_BROWSER = WebView2Browser(
hwnd_provider=_find_kivy_hwnd,
user_data_dir=os.path.join(data_dir, '.webview2-profile'),
)
except Exception as exc:
_log(f"WebView2 init failed: {exc}")
return None
return _WEBVIEW2_BROWSER
def _wait_for_webview2_runtime(timeout=_WEBVIEW2_INSTALL_WAIT):
"""Block while a Runtime install is in flight. True if it became available."""
try:
import webview2_runtime
except Exception:
return False
state = webview2_runtime.get_state()
if not state.get('installing'):
return False
_log("WebView2: waiting for the Runtime install to finish...")
ok = webview2_runtime.wait_for_install(timeout)
_log(f"WebView2: Runtime install wait finished (available={ok})")
return ok
def _windows_find_browser():
"""Find Chrome or Edge executable on Windows for weblink support.
@@ -1472,6 +1559,167 @@ def _patch_main():
# CEF keeps one browser instance alive; there is nothing to warm.
pass
class _WinWebView2Adapter(ChromiumSubprocessAdapter):
"""Embedded WebView2 (Edge) — renders INSIDE the Kivy window.
This is the preferred Windows engine. Because the page is a child HWND
of Kivy's own SDL window there is no separate browser process, so none
of the subprocess problems apply: it cannot open behind the player, it
cannot be handed off to an existing instance and exit instantly, it
does not fight for foreground/z-order, and teardown leaves no leaked
browser behind.
``launch`` returns as soon as the (asynchronous) start-up is requested;
the session's watcher thread then polls :meth:`wait_visible` until the
page is actually showing.
"""
name = 'webview2-embedded'
embedded = True
# WebView2 renders in-window, but unlike raw CEF we CAN prove the page
# appeared (the controller reports when it is visible). That matters on
# a closed network: if the weblink host is unreachable the page never
# paints and the item is skipped instead of showing a blank screen.
can_verify_visibility = True
def __init__(self):
super().__init__(kiosk=False)
self._browser = None
self._resize_bound = False
@property
def process(self):
return None # in-process: nothing to kill
def target_size(self):
"""Use the real Kivy window size (already DPI-aware)."""
try:
from kivy.core.window import Window as KivyWindow
width, height = (int(KivyWindow.size[0]), int(KivyWindow.size[1]))
if width > 0 and height > 0:
return width, height
except Exception:
pass
return 1920, 1080
def launch(self, url, width, height):
self._browser = _get_webview2_browser()
if self._browser is None:
return False
# A child HWND has no z-order to fight over, but the Kivy window
# itself must own the foreground or the page looks inert.
_bring_kivy_to_front(async_ok=False)
self._bind_resize_once()
self._browser.resize(width, height)
return bool(self._browser.show(url))
def is_alive(self):
browser = self._browser
if browser is None:
return False
# NOTE: `is_alive` must be True while the page is still coming up.
# WebView2 creates its environment and controller asynchronously, so
# a controller that does not exist yet is NOT a dead browser —
# treating it as one made the first weblink be skipped instantly.
return browser.is_alive()
def wait_visible(self, timeout):
"""Wait for the page to actually load, on the caller's (watcher) thread.
Two things must hold: the controller must be showing, AND the
navigation must have completed successfully. The second condition is
what makes an unreachable host (normal on a closed network) skip the
item instead of displaying Chromium's error page for the full slot.
"""
import time
browser = self._browser
if browser is None:
return False, 'no browser'
deadline = time.monotonic() + max(1.0, float(timeout))
while time.monotonic() < deadline:
if browser.failed_reason:
return False, browser.failed_reason
nav = browser.navigation_succeeded()
if browser.is_showing() and nav is not None:
if not nav:
detail = browser.navigation_status() or 'navigation failed'
return False, f'page failed to load ({detail})'
# Let the compositor paint the first frame before any
# masking overlay is hidden / Kivy is restored.
time.sleep(0.3)
return True, 'webview2-loaded'
time.sleep(0.1)
if browser.is_showing() and browser.navigation_succeeded() is None:
# Controller is up but navigation never reported within the
# timeout. Treat as visible rather than skipping a page that is
# merely slow to confirm.
return True, 'webview2-showing-unconfirmed'
reason = browser.failed_reason or 'webview2 did not show'
return False, reason
def on_visible(self):
trace('win_weblink_webview2_shown')
def on_launch_failed(self):
# Never leave a half-built controller (or a hidden child window)
# behind when start-up fails.
browser, self._browser = self._browser, None
if browser is not None:
try:
browser.shutdown()
except Exception:
pass
_get_webview2_browser(force_new=True)
def _bind_resize_once(self):
"""Keep the page matched to the window, bound exactly once.
Rebinding per cycle was the unbounded-callback leak fixed for CEF;
the same discipline applies here.
"""
if self._resize_bound:
return
try:
from kivy.core.window import Window as KivyWindow
def _wv2_resize(*_args):
try:
browser = self._browser
if browser is not None and browser.is_showing():
browser.resize(
int(KivyWindow.size[0]), int(KivyWindow.size[1])
)
except Exception:
pass
KivyWindow.bind(size=_wv2_resize)
self._resize_bound = True
except Exception as exc:
_log(f"WebView2 resize bind failed (non-fatal): {exc}")
def teardown(self):
"""Hide the page (keep the controller for a fast next show).
Hiding — not disposing — is deliberate: the controller stays alive
so the next weblink paints immediately, and because it is a child
window there is no leaked process to reap.
"""
browser = self._browser
if browser is not None:
try:
browser.hide()
except Exception as exc:
_log(f"WebView2 hide failed (non-fatal): {exc}")
# Restore Kivy as the visible surface again.
_bring_kivy_to_front(async_ok=False)
def prewarm(self, url):
# The environment/controller are already warm after the first use;
# pre-navigating would claim the page before its slot.
pass
class _WinChromeAdapter(ChromiumSubprocessAdapter):
"""Chrome/Edge kiosk subprocess with the Windows visibility check.
@@ -1493,6 +1741,26 @@ def _patch_main():
def on_launch_failed(self):
_Win32Overlay.hide()
def extra_launch_args(self):
"""Dedicated profile + kiosk flags for the Windows browser.
``--user-data-dir`` is mandatory: without it Chrome/Edge hands the
URL to an already-running instance, our launched process exits in
~2s and the weblink never displays (the adapter's `wait_visible`
then sees the process die and the item is skipped). It also gives
us a top-level window we can enumerate, raise and taskkill without
touching the operator's own browser profile.
"""
args = []
if self._profile_dir:
args.append('--user-data-dir=' + self._profile_dir)
if self._kiosk:
# The base launch() already adds --start-fullscreen /
# --start-maximized; --kiosk upgrades that to a true kiosk
# window (no browser UI, locks to the screen).
args.append('--kiosk')
return args
def launch(self, url, width, height):
# A dedicated --user-data-dir is mandatory: without it Chrome hands
# the URL to an existing process, the launched process exits
@@ -1556,15 +1824,28 @@ def _patch_main():
def _windows_weblink_adapter_factory(player):
"""Choose the Windows web-link engines, best first.
CEF (embedded) is preferred when available because it cannot fight for
z-order or foreground; the Chrome/Edge subprocess is the fallback.
Order matters:
1. **WebView2 (embedded)** — a child HWND of the Kivy window. It cannot
open behind the player, cannot be handed off to an existing browser
and exit instantly, does not fight for z-order/foreground, and
leaves no leaked process behind. This is the preferred engine.
2. **CEF (embedded)** — same in-window model, but cefpython3 has no
wheels past Python 3.9 so it is dormant on this build.
3. **Chrome/Edge subprocess** — last resort only. Kept so a machine
without the WebView2 runtime still shows weblinks.
"""
adapters = []
if _get_webview2_browser() is not None:
adapters.append(_WinWebView2Adapter())
if _get_cef_browser() is not None:
adapters.append(_WinCefAdapter())
browser = _windows_find_browser()
if browser:
adapters.append(_WinChromeAdapter(browser))
if not adapters:
_log("WebView2/CEF unavailable and no Chrome/Edge found — "
"web links will be skipped")
return adapters
signage_main.SignagePlayer.weblink_adapter_factory = staticmethod(
@@ -1657,6 +1938,85 @@ def _patch_main():
signage_main.SettingsPopup.test_connection = _windows_test_connection
# ── Keep player auth OUT of the bundle ───────────────────────────
# `player_auth.json` used to be a tracked file inside src/, so PyInstaller
# bundled it and, because the frozen app runs with cwd = _internal/, the
# player loaded it as its live auth state. A stale snapshot therefore made
# a fresh build boot "already authenticated" against an old server and play
# an outdated playlist.
#
# Fix: never read a relative auth path from the bundle. Any relative
# 'player_auth.json' is redirected to the data dir next to the .exe, which
# is the single source of truth for this install.
import player_auth as _player_auth_module
_bundled_auth = os.path.join(
getattr(sys, '_MEIPASS', os.path.dirname(os.path.abspath(__file__))),
'player_auth.json',
)
_local_auth = os.path.join(DATA_DIR, 'player_auth.json')
# If an older build left the bundled snapshot next to the exe, drop it:
# it names another server and would be trusted on the next start-up.
try:
if os.path.isfile(_local_auth):
import json as _json
with open(_local_auth, 'r') as _f:
_existing = _json.load(_f)
_existing_url = str(_existing.get('server_url') or '')
_wanted_ip = str(os.environ.get('KIWY_SERVER_IP') or '')
if _wanted_ip and _wanted_ip not in _existing_url:
Logger.warning(
"SignagePlayer: stored auth points at %s but the configured "
"server is %s - clearing it so the player re-authenticates",
_existing_url or '(none)', _wanted_ip,
)
os.remove(_local_auth)
except Exception as _exc:
Logger.debug(f"SignagePlayer: auth pre-check skipped: {_exc}")
_original_auth_init = _player_auth_module.PlayerAuth.__init__
def _windows_auth_init(self, config_file='player_auth.json',
use_https=True, verify_ssl=True):
"""Force the auth file to live in the data dir next to the .exe."""
try:
if not os.path.isabs(config_file):
config_file = os.path.join(DATA_DIR, os.path.basename(config_file))
except Exception:
config_file = _local_auth
_original_auth_init(self, config_file, use_https=use_https, verify_ssl=verify_ssl)
_player_auth_module.PlayerAuth.__init__ = _windows_auth_init
Logger.info(f"SignagePlayer: player auth file -> {_local_auth}")
# `get_playlists_v2` caches a global auth instance created with the default
# relative path; make sure its cache is empty so the redirect above applies.
try:
import get_playlists_v2 as _gp
if _gp._auth_instance is not None:
_gp._auth_instance = None
except Exception as _exc:
Logger.debug(f"SignagePlayer: could not reset auth cache: {_exc}")
# `reset_player_auth` deleted the file next to main.py (i.e. inside the
# bundle, which is read-only and not what we load). Point it at the real
# auth file so the "Reset auth" button actually works.
def _windows_reset_player_auth(self):
try:
if os.path.exists(_local_auth):
os.remove(_local_auth)
Logger.info(f"SettingsPopup: Deleted authentication file: {_local_auth}")
self._show_temp_message(
'✓ Authentication reset - will reauthenticate on restart', (0, 1, 0, 1)
)
except Exception as exc:
Logger.error(f"SettingsPopup: Failed to reset auth: {exc}")
signage_main.SettingsPopup.reset_player_auth = _windows_reset_player_auth
# ── Patch apply_kiosk_mode for Windows ──────────────────────────
# Wrap the base implementation (exit_on_escape + close guard + Ctrl+C)
# and add the low-level keyboard hook that swallows Alt+F4 / Alt+Tab /
@@ -1702,6 +2062,13 @@ def _patch_main():
Logger.info("SignagePlayer: Windows card reader shut down")
except Exception as e:
Logger.debug(f"SignagePlayer: card reader shutdown error: {e}")
# Tear down the embedded web engine. Because it is a child HWND (not a
# subprocess) this is what stops it surviving the player on exit.
try:
_get_webview2_browser(force_new=True)
Logger.info("SignagePlayer: WebView2 browser shut down")
except Exception as e:
Logger.debug(f"SignagePlayer: WebView2 shutdown error: {e}")
_original_on_stop(self)
signage_main.SignagePlayerApp.on_stop = _windows_on_stop
@@ -1829,6 +2196,14 @@ if __name__ == '__main__':
Logger.info(f"Data directory: {DATA_DIR}")
Logger.info("=" * 80)
# ── Ensure the WebView2 Runtime is present ───────────────────
# The bundled SDK is only an API surface; the Runtime holds the actual
# engine. Windows 11 and most Windows 10 machines already have it, but
# when it is missing we install it silently (per-user, so no UAC prompt
# appears on the signage display). Runs in the background so start-up
# is not blocked.
_ensure_webview2_runtime_async()
# Patch base_dir in SignagePlayer instances to point to local data folder
_original_init = patched_main.SignagePlayer.__init__