Fix install.sh failing on offline Raspberry Pi at the venv step

install.sh ran 'pip install .', which builds the project: pip downloads
setuptools into an isolated build environment. With no internet it fails at
the 'Building the virtual environment' step with

    Could not find a version that satisfies the requirement setuptools>=68
    (from versions: none)

even though the runtime libraries are already in the wheel directory. The
build backend is only needed to package, never to run.

- Install the runtime deps straight from the bundled wheels with --no-index,
  and put src/ on the path via a .pth file, so no build step happens at all.
- Generate the console script with the venv's interpreter as an absolute
  shebang. '#\!/usr/bin/env python3' resolves to the *system* python when the
  venv is not activated, so the package was not importable from the unit.
- apt-get update/install and 'pip install --upgrade pip' are now best-effort:
  all three need the network and none should abort the install.
- Detect a failed 'python3 -m venv' and name python3-venv as the usual cause.
- Sanity-check the install with --version before enabling the service.
- The unit falls back to 'python -m rdm_card_reader' if the console script is
  ever absent (no shell in ExecStart, so the installer picks the path).

Reproduced the failure offline, then verified the fix offline as well: a venv
created on a copy of the tree self-provisions from the bundled wheels and runs
from cwd=/ with no PYTHONPATH and no venv activation.

Added tests/test_deploy.py (20 tests) covering the installer and the deploy
artifacts, including a regression test for 'pip install .' that was confirmed
to fail when the bug is reintroduced.
This commit is contained in:
2026-10-01 16:19:33 +03:00
parent 674d91a5ca
commit 2ded37e190
4 changed files with 403 additions and 11 deletions
+89 -8
View File
@@ -47,8 +47,11 @@ fi
# 1. System packages
# ---------------------------------------------------------------------------
log "Checking system packages"
apt-get update -qq
apt-get install -y --no-install-recommends python3 python3-venv python3-pip
# Both of these need the network, so neither may abort the install: a device
# with no internet usually already has python3, and we can work without them.
apt-get update -qq 2>/dev/null || warn "apt-get update failed (no network?) - continuing"
apt-get install -y --no-install-recommends python3 python3-venv python3-pip \
|| warn "could not install system packages - continuing with what is present"
# ---------------------------------------------------------------------------
# 2. Install the application
@@ -57,16 +60,86 @@ log "Installing to $INSTALL_DIR"
mkdir -p "$INSTALL_DIR"
# Copy the source tree (excluding local build/test artifacts).
for item in src tests pyproject.toml README.md LICENSE deploy; do
for item in src tests pyproject.toml README.md LAUNCH.md LICENSE deploy; do
if [[ -e "$SOURCE_DIR/$item" ]]; then
cp -a "$SOURCE_DIR/$item" "$INSTALL_DIR/"
fi
done
log "Building the virtual environment"
python3 -m venv "$INSTALL_DIR/.venv"
"$INSTALL_DIR/.venv/bin/pip" install --quiet --upgrade pip
"$INSTALL_DIR/.venv/bin/pip" install --quiet "$INSTALL_DIR"
if ! python3 -m venv "$INSTALL_DIR/.venv"; then
warn "could not create the virtualenv."
warn "On Debian/Raspberry Pi OS this usually means python3-venv is missing:"
warn " sudo apt-get install python3-venv"
exit 1
fi
VENV_PY="$INSTALL_DIR/.venv/bin/python"
WHEEL_DIR="$INSTALL_DIR/src/rdm_card_reader/wheels"
# Upgrading pip is optional and needs the network.
"$INSTALL_DIR/.venv/bin/pip" install --quiet --upgrade pip 2>/dev/null \
|| echo " (kept the bundled pip - no network)"
# Install the runtime dependencies from the wheels shipped in the source tree.
#
# Deliberately NOT 'pip install .': that builds the project, which requires
# downloading setuptools into an isolated build environment. With no internet
# it fails with
# Could not find a version that satisfies the requirement setuptools>=68
# even though everything needed to *run* is sitting in the wheel directory.
# Installing the runtime dependencies directly, and putting 'src' on the
# path, achieves the same thing and works offline.
INSTALLED_PACKAGE=0
if [[ -d "$WHEEL_DIR" ]] && "$VENV_PY" -m pip install --quiet --no-index \
--find-links "$WHEEL_DIR" rdm6300 pyserial 2>/dev/null; then
echo " installed rdm6300 and pyserial from the bundled wheels (offline)"
else
echo " bundled wheels unavailable - falling back to PyPI"
if "$VENV_PY" -m pip install --quiet rdm6300 pyserial 2>/dev/null; then
echo " installed rdm6300 and pyserial from PyPI"
else
warn "could not install rdm6300 / pyserial."
warn "The app will try again on first start (it can install from its own"
warn "bundled wheels), but check network access if that also fails."
fi
fi
# Make the package importable. The .pth file adds src/ to sys.path, so
# 'python -m rdm_card_reader' resolves from any working directory.
SITE_PACKAGES="$("$VENV_PY" -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')"
echo "$INSTALL_DIR/src" > "$SITE_PACKAGES/rdm_card_reader_src.pth"
# Console-script equivalent, so the unit and the docs work as written.
# The shebang must be the venv's interpreter by absolute path: '#!/usr/bin/env
# python3' would pick up the *system* python when the venv is not activated,
# and the package would not be importable.
cat > "$INSTALL_DIR/.venv/bin/rdm-card-reader" <<EOF
#!$VENV_PY
import sys
from rdm_card_reader.cli import main
if __name__ == "__main__":
sys.exit(main())
EOF
chmod +x "$INSTALL_DIR/.venv/bin/rdm-card-reader"
# Standalone launcher for running without activating anything.
cat > "$INSTALL_DIR/run.sh" <<EOF
#!/bin/sh
# Convenience launcher: ./run.sh [args...]
exec "$INSTALL_DIR/.venv/bin/rdm-card-reader" "\$@"
EOF
chmod +x "$INSTALL_DIR/run.sh"
# Sanity check: prove it imports and can run before we enable a service.
if "$INSTALL_DIR/.venv/bin/rdm-card-reader" --version >/dev/null 2>&1; then
echo " verified: $("$INSTALL_DIR/.venv/bin/rdm-card-reader" --version)"
else
warn "the reader does not run yet. Check with:"
warn " $INSTALL_DIR/.venv/bin/python -m rdm_card_reader --list-ports"
fi
# ---------------------------------------------------------------------------
# 3. Service account
@@ -114,12 +187,20 @@ for origin in "${ORIGINS[@]:-}"; do
done
python3 - "$UNIT" "$INSTALL_DIR" "$BRIDGE_PORT" "$ORIGIN_ARGS" "$SERVICE_USER" <<'PY'
import re, sys
import os, re, sys
unit, install_dir, port, origin_args, user = sys.argv[1:6]
text = open(unit).read()
# The console script is created by the installer, but the .pth file is what
# makes the package importable, so fall back to 'python -m' if it is ever
# absent. systemd's ExecStart has no shell, so no pipes or conditionals.
wrapper = f"{install_dir}/.venv/bin/rdm-card-reader"
if not os.path.exists(wrapper):
wrapper = f"{install_dir}/.venv/bin/python -m rdm_card_reader"
command = (
f"{install_dir}/.venv/bin/rdm-card-reader --serve "
f"{wrapper} --serve "
f"--port /dev/rdm-reader --bridge-port {port} --mode login{origin_args}"
)
# Replace the whole (possibly line-continued) ExecStart definition.