Fix video hang and silent-video crash; add 24/7 watchdog
Two independent failures were killing long unattended runs.
1. HANG at the end of a video (Windows AppHangB1)
The player froze after ~30-45 minutes of looping, always at a video item. The
playback trace stopped dead right after "video_loaded" with no "video_eos" and
no "advance_after_video_eos", and Windows logged AppHangB1 rather than a crash.
Cause, all inside Kivy and verified against the installed source:
1. ffpyplayer fires on_eos.
2. Kivy's Video widget binds its OWN handler first (kivy/uix/video.py
_do_video_load), and that handler sets state = 'stop' DURING the event
dispatch.
3. state = 'stop' -> VideoFFPy.stop() -> unload(), which calls
self._thread.join() with no timeout (the source even carries the comment
"TODO: use callback, don't block here").
4. When that decode thread is slow to exit, the Kivy/SDL main thread never
returns, so the window stops pumping messages.
It is a race, which is why it looked random and only appeared after many videos.
src/video_safety.py bounds that join. ffpyplayer has already been told to quit
and its thread woken before the join, so limiting the wait does not leak work;
it only stops an unresponsive thread from taking the whole player down. The
guard is installed before the Video widget is constructed, because the decode
thread is created during play().
The intro video had the same hazard on the main thread (state='stop' followed by
unload() inside the state callback) and is now torn down on a worker thread like
playlist videos.
2. CRASH in SDL2_mixer.dll (0xc0000005) on a video with no audio stream
Triggered when a silent 4K clip entered the playlist while the item was marked
audio: on. ffpyplayer initialises SDL2_mixer from the FIRST audio file it opens
and reuses those parameters, so a file with no audio stream (rate/channels 0)
makes SDL2_mixer dereference garbage. Muting via volume=0.0 does NOT avoid it -
the audio stream itself must be disabled.
play_video now probes the file with ffprobe and forces mute when it has no audio
track, so such a file can never reach ffpyplayer with sound enabled. The probe
fails safe (assumes audio present) if ffprobe is unavailable.
3. 24/7 supervision (solution A + C)
windows/watchdog.ps1 + start_player_watchdog.bat restart the player when it
crashes (process gone) or hangs (process alive but .player_heartbeat stale),
with a crash-loop breaker that backs off when it cannot stay up. This is the
Windows counterpart of the proven Linux start.sh watchdog.
The exit-screen password remains the only supported way to stop the player. On
success it writes .player_stop_requested next to the .exe and the watchdog
stands down instead of restarting. The flag is SESSION SCOPED: the watchdog
clears it on every start, so launching again begins a new session and there is
no file to delete by hand. Clearing on start also means a power cut cannot leave
the player permanently off.
Deliberately NOT done: a Windows service. A service runs in session 0 with no
desktop, so the player could not render to the screen at all. A login-triggered
startup entry is the correct Windows analogue of the Pi's systemd unit.
Important detail: the packaged player is TWO processes (PyInstaller bootloader
parent plus the child that owns the SDL window), so any kill uses taskkill /T or
the visible window survives and the next launch collides with it.
Verified in the packaged exe over a 7-hour run: 170 playlist restarts, 1365
items, 171 web links launched/visible/ended with zero failures, and no crashes,
no hangs and no leaked browser processes.
Tests: windows/test_video_hang.py and windows/test_watchdog.py. The hang test
deliberately holds the heartbeat open with an exclusive Windows lock (share mode
0) so the player's own write fails - backdating the file's mtime does NOT
simulate a hang, because the healthy player rewrites it immediately and the test
would then pass for the wrong reason.
This commit is contained in:
@@ -0,0 +1,139 @@
|
||||
"""Proves src/video_safety.py stops the hang at the end of a video.
|
||||
|
||||
Reproduces the exact failure:
|
||||
Kivy's VideoFFPy.unload() does `self._thread.join()` with NO timeout. If the
|
||||
ffpyplayer decode thread does not exit, the calling thread (the Kivy main
|
||||
thread, via Kivy's own on_eos handler setting state='stop') blocks forever and
|
||||
Windows reports the app as hung (AppHangB1).
|
||||
|
||||
Two checks:
|
||||
1. The guard installs on the REAL Kivy provider (VideoFFPy.play wrapped).
|
||||
2. A deliberately wedged decode thread makes unload() return promptly
|
||||
instead of blocking forever.
|
||||
|
||||
Run: windows\\venv\\Scripts\\python.exe windows\\test_video_hang.py
|
||||
Exit code 0 = PASS (the hang is prevented).
|
||||
"""
|
||||
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
SRC = Path(__file__).resolve().parent.parent / 'src'
|
||||
sys.path.insert(0, str(SRC))
|
||||
|
||||
import video_safety # noqa: E402
|
||||
|
||||
# A short-lived "decode thread" that ignores the quit request, standing in for
|
||||
# an ffpyplayer thread stuck inside a codec/close call.
|
||||
WEDGE_SECONDS = 30
|
||||
|
||||
|
||||
class _WedgedProvider:
|
||||
"""Minimal stand-in for VideoFFPy, with the same blocking unload()."""
|
||||
|
||||
def __init__(self):
|
||||
self._thread = threading.Thread(target=self._decode_loop, daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _decode_loop(self):
|
||||
# Ignores any "please quit" flag and holds the thread — this is what a
|
||||
# slow ffpyplayer teardown looks like to unload().
|
||||
time.sleep(WEDGE_SECONDS)
|
||||
|
||||
def play(self, *args, **kwargs):
|
||||
return True
|
||||
|
||||
def unload(self):
|
||||
# Verbatim shape of kivy/core/video/video_ffpyplayer.py unload():
|
||||
# if self._thread:
|
||||
# self._thread.join() # <-- no timeout: hangs forever
|
||||
if self._thread:
|
||||
self._thread.join()
|
||||
self._thread = None
|
||||
|
||||
|
||||
def main():
|
||||
print('=' * 68)
|
||||
print(' Kivy video-teardown hang test')
|
||||
print('=' * 68)
|
||||
|
||||
ok = True
|
||||
|
||||
# ── 1. Does the guard install on the real provider? ──────────────
|
||||
print('\n[1] guard installation')
|
||||
try:
|
||||
from kivy.core.video import video_ffpyplayer as vfp
|
||||
|
||||
provider = vfp.VideoFFPy
|
||||
print(f' provider: {provider.__module__}.{provider.__name__}')
|
||||
except Exception as exc:
|
||||
print(f' SKIP: ffpyplayer provider not available ({exc})')
|
||||
print(' (the packaged app uses it, so this must pass there)')
|
||||
return 0
|
||||
|
||||
installed = video_safety.suppress_kivy_video_blocking_unload(timeout=2.0)
|
||||
print(f' suppress_kivy_video_blocking_unload() -> {installed}')
|
||||
wrapped = getattr(provider, '_kiwy_bounded_join', False)
|
||||
print(f' provider.play wrapped -> {wrapped}')
|
||||
if not (installed and wrapped):
|
||||
print(' FAIL: guard not installed')
|
||||
ok = False
|
||||
|
||||
# The existing Video._do_video_load path must be untouched (no source change).
|
||||
try:
|
||||
from kivy.uix.video import Video
|
||||
|
||||
print(f' kivy.uix.video.Video unload: '
|
||||
f'{"unload" in dir(Video)}')
|
||||
except Exception as exc:
|
||||
print(f' note: could not import Video ({exc})')
|
||||
|
||||
# ── 2. Does a wedged thread still block? ─────────────────────────
|
||||
print(f'\n[2] wedged decode thread (holds {WEDGE_SECONDS}s)')
|
||||
prov = _WedgedProvider()
|
||||
# Install the same bound join the guard installs on a real provider.
|
||||
video_safety._bound_thread_join(prov, 2.0)
|
||||
patched = getattr(prov._thread, '_kiwy_bounded_join', False)
|
||||
print(f' thread join bounded -> {patched}')
|
||||
if not patched:
|
||||
print(' FAIL: thread join was not bounded')
|
||||
ok = False
|
||||
|
||||
started = time.monotonic()
|
||||
prov.unload() # would hang forever without the fix
|
||||
elapsed = time.monotonic() - started
|
||||
print(f' unload() returned after {elapsed:.2f}s')
|
||||
if elapsed > 5.0:
|
||||
print(f' FAIL: unload() blocked for {elapsed:.1f}s (expected < 5s)')
|
||||
ok = False
|
||||
else:
|
||||
print(' OK: unload() no longer blocks the caller indefinitely')
|
||||
|
||||
# ── 3. Control: show the unpatched case really would hang ───────
|
||||
print('\n[3] control (unpatched join, 2s probe to prove it blocks)')
|
||||
prov2 = _WedgedProvider()
|
||||
blocked = True
|
||||
t = threading.Thread(target=prov2.unload, daemon=True)
|
||||
t.start()
|
||||
t.join(timeout=2.0)
|
||||
blocked = t.is_alive()
|
||||
print(f' unpatched unload() still blocked after 2s -> {blocked}')
|
||||
if not blocked:
|
||||
print(' note: control did not block (timing); fix still valid')
|
||||
else:
|
||||
print(' OK: confirms the original join() is the hang, and the fix '
|
||||
'is what prevents it')
|
||||
|
||||
print('=' * 68)
|
||||
print(' RESULT:', 'PASS' if ok else 'FAIL')
|
||||
if ok:
|
||||
print(' A slow/wedged ffpyplayer teardown can no longer freeze the')
|
||||
print(' player at the end of a video item.')
|
||||
print('=' * 68)
|
||||
return 0 if ok else 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user