diff --git a/LAUNCH.md b/LAUNCH.md index 1dd5dcd..bc74a34 100644 --- a/LAUNCH.md +++ b/LAUNCH.md @@ -30,6 +30,70 @@ All commands run from the project root. The virtualenv is already built. ``` After `pip install .` the same thing is available as `rdm-card-reader …`. +`pip install .` needs the network — see **Troubleshooting** below for the +offline route. + +--- + +## Troubleshooting install + +### "Could not find a version that satisfies the requirement setuptools>=68" + +Seen when running `deploy/install.sh` on a Pi with no internet, at the +"Building the virtual environment" step: + +``` +ERROR: Could not find a version that satisfies the requirement setuptools>=68 + (from versions: none) +ERROR: No matching distribution found for setuptools>=68 +``` + +**Cause:** `pip install .` builds the project. pip creates an isolated build +environment and downloads the *build* backend (`setuptools`) into it — even +though the *runtime* libraries are already sitting in the wheel directory. With +no network that download cannot happen. + +**It is not** a problem with the bundled wheels, and not a Python version +mismatch. Two separate confusions produce a similar-looking error: + +| Situation | Error | +| --- | --- | +| Build isolation, no network | `Could not find a version ... setuptools>=68` | +| `--no-build-isolation`, and setuptools absent | `Cannot import 'setuptools.build_meta'` | + +Note that Python 3.12+ venvs no longer include `setuptools`, so the second case +is increasingly common. + +**Fix:** the current `install.sh` avoids the build entirely — it installs the +bundled runtime wheels and puts `src/` on the path with a `.pth` file. If you +are on an older copy, update it (`git pull`) or run these by hand on the Pi: + +```bash +python3 -m venv .venv +.venv/bin/pip install --no-index \ + --find-links src/rdm_card_reader/wheels rdm6300 pyserial +SITE=$(.venv/bin/python -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])') +echo "$PWD/src" > "$SITE/rdm_card_reader_src.pth" +.venv/bin/python -m rdm_card_reader --demo # verify, no hardware needed +``` + +No build step, no network, and the same result. + +### "Could not create the virtualenv" + +On Debian and Raspberry Pi OS, `python3 -m venv` needs the `python3-venv` +package: + +```bash +sudo apt-get install python3-venv +``` + +### apt-get failing on the first step + +`apt-get update` and `apt-get install` need the network. The installer now +treats both as best-effort and continues, because a device typically already +has `python3`. If they fail and something really is missing, install it while +online and re-run. --- @@ -240,6 +304,27 @@ journalctl -u rdm-card-reader -f curl http://127.0.0.1:8765/api/card # tap a card first ``` +The installer needs no internet: the runtime libraries ship inside the +source tree (`src/rdm_card_reader/wheels/`). + +### Copying the project to the Pi + +`install.sh` resolves its source directory from its own location, so copy the +whole tree (including `src/` — that is where the wheels live) and run it from +there: + +```bash +# from this machine +scp -r rdm-card-reader pi@192.168.0.123:~/Desktop/ + +# on the Pi +cd ~/Desktop/rdm-card-reader +sudo ./deploy/install.sh --origin https://your.web.app +``` + +If you transferred only a subset of the tree, `--dir` can point the install +somewhere else, but `src/` must be present or the wheels will be missing. + --- ## Testing the reader path without a reader diff --git a/README.md b/README.md index bed777b..0b95d65 100644 --- a/README.md +++ b/README.md @@ -53,18 +53,45 @@ without its library is worse than one that refuses to start. ## Install +On a machine **with internet**: + ```bash pip install . ``` -On a Raspberry Pi OS install with an externally-managed Python, add -`--break-system-packages` or use a virtual environment: +`rdm6300` and `pyserial` are pulled in automatically. On Raspberry Pi OS with +an externally-managed Python, add `--break-system-packages` or use a venv: ```bash python3 -m venv .venv && . .venv/bin/activate && pip install . ``` -`rdm6300` and `pyserial` are pulled in automatically. +### On a device with no internet + +`pip install .` will **not** work offline. It builds the project, which makes +pip download `setuptools` into an isolated build environment, and that fails +with: + +``` +ERROR: Could not find a version that satisfies the requirement setuptools>=68 + (from versions: none) +``` + +The runtime dependencies are bundled in `src/rdm_card_reader/wheels/`, so +install those directly and put `src` on the path — no build step, no network: + +```bash +python3 -m venv .venv +.venv/bin/pip install --no-index \ + --find-links src/rdm_card_reader/wheels rdm6300 pyserial +echo "$PWD/src" > "$( .venv/bin/python -c \ + 'import sysconfig; print(sysconfig.get_paths()["purelib"])' )/rdm_card_reader_src.pth" +.venv/bin/python -m rdm_card_reader --demo # verify, no hardware needed +``` + +`deploy/install.sh` does exactly this, plus the service setup. Running the +app directly also works: it installs anything missing from the same bundled +wheels on first start. --- diff --git a/deploy/install.sh b/deploy/install.sh index 6c6134c..37d49b3 100755 --- a/deploy/install.sh +++ b/deploy/install.sh @@ -47,8 +47,11 @@ fi # 1. System packages # --------------------------------------------------------------------------- log "Checking system packages" -apt-get update -qq -apt-get install -y --no-install-recommends python3 python3-venv python3-pip +# Both of these need the network, so neither may abort the install: a device +# with no internet usually already has python3, and we can work without them. +apt-get update -qq 2>/dev/null || warn "apt-get update failed (no network?) - continuing" +apt-get install -y --no-install-recommends python3 python3-venv python3-pip \ + || warn "could not install system packages - continuing with what is present" # --------------------------------------------------------------------------- # 2. Install the application @@ -57,16 +60,86 @@ log "Installing to $INSTALL_DIR" mkdir -p "$INSTALL_DIR" # Copy the source tree (excluding local build/test artifacts). -for item in src tests pyproject.toml README.md LICENSE deploy; do +for item in src tests pyproject.toml README.md LAUNCH.md LICENSE deploy; do if [[ -e "$SOURCE_DIR/$item" ]]; then cp -a "$SOURCE_DIR/$item" "$INSTALL_DIR/" fi done log "Building the virtual environment" -python3 -m venv "$INSTALL_DIR/.venv" -"$INSTALL_DIR/.venv/bin/pip" install --quiet --upgrade pip -"$INSTALL_DIR/.venv/bin/pip" install --quiet "$INSTALL_DIR" +if ! python3 -m venv "$INSTALL_DIR/.venv"; then + warn "could not create the virtualenv." + warn "On Debian/Raspberry Pi OS this usually means python3-venv is missing:" + warn " sudo apt-get install python3-venv" + exit 1 +fi + +VENV_PY="$INSTALL_DIR/.venv/bin/python" +WHEEL_DIR="$INSTALL_DIR/src/rdm_card_reader/wheels" + +# Upgrading pip is optional and needs the network. +"$INSTALL_DIR/.venv/bin/pip" install --quiet --upgrade pip 2>/dev/null \ + || echo " (kept the bundled pip - no network)" + +# Install the runtime dependencies from the wheels shipped in the source tree. +# +# Deliberately NOT 'pip install .': that builds the project, which requires +# downloading setuptools into an isolated build environment. With no internet +# it fails with +# Could not find a version that satisfies the requirement setuptools>=68 +# even though everything needed to *run* is sitting in the wheel directory. +# Installing the runtime dependencies directly, and putting 'src' on the +# path, achieves the same thing and works offline. +INSTALLED_PACKAGE=0 +if [[ -d "$WHEEL_DIR" ]] && "$VENV_PY" -m pip install --quiet --no-index \ + --find-links "$WHEEL_DIR" rdm6300 pyserial 2>/dev/null; then + echo " installed rdm6300 and pyserial from the bundled wheels (offline)" +else + echo " bundled wheels unavailable - falling back to PyPI" + if "$VENV_PY" -m pip install --quiet rdm6300 pyserial 2>/dev/null; then + echo " installed rdm6300 and pyserial from PyPI" + else + warn "could not install rdm6300 / pyserial." + warn "The app will try again on first start (it can install from its own" + warn "bundled wheels), but check network access if that also fails." + fi +fi + +# Make the package importable. The .pth file adds src/ to sys.path, so +# 'python -m rdm_card_reader' resolves from any working directory. +SITE_PACKAGES="$("$VENV_PY" -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')" +echo "$INSTALL_DIR/src" > "$SITE_PACKAGES/rdm_card_reader_src.pth" + +# Console-script equivalent, so the unit and the docs work as written. +# The shebang must be the venv's interpreter by absolute path: '#!/usr/bin/env +# python3' would pick up the *system* python when the venv is not activated, +# and the package would not be importable. +cat > "$INSTALL_DIR/.venv/bin/rdm-card-reader" < "$INSTALL_DIR/run.sh" </dev/null 2>&1; then + echo " verified: $("$INSTALL_DIR/.venv/bin/rdm-card-reader" --version)" +else + warn "the reader does not run yet. Check with:" + warn " $INSTALL_DIR/.venv/bin/python -m rdm_card_reader --list-ports" +fi # --------------------------------------------------------------------------- # 3. Service account @@ -114,12 +187,20 @@ for origin in "${ORIGINS[@]:-}"; do done python3 - "$UNIT" "$INSTALL_DIR" "$BRIDGE_PORT" "$ORIGIN_ARGS" "$SERVICE_USER" <<'PY' -import re, sys +import os, re, sys + unit, install_dir, port, origin_args, user = sys.argv[1:6] text = open(unit).read() +# The console script is created by the installer, but the .pth file is what +# makes the package importable, so fall back to 'python -m' if it is ever +# absent. systemd's ExecStart has no shell, so no pipes or conditionals. +wrapper = f"{install_dir}/.venv/bin/rdm-card-reader" +if not os.path.exists(wrapper): + wrapper = f"{install_dir}/.venv/bin/python -m rdm_card_reader" + command = ( - f"{install_dir}/.venv/bin/rdm-card-reader --serve " + f"{wrapper} --serve " f"--port /dev/rdm-reader --bridge-port {port} --mode login{origin_args}" ) # Replace the whole (possibly line-continued) ExecStart definition. diff --git a/tests/test_deploy.py b/tests/test_deploy.py new file mode 100644 index 0000000..f537463 --- /dev/null +++ b/tests/test_deploy.py @@ -0,0 +1,199 @@ +"""Tests for the deploy artifacts. + +These are the files that only ever run on the target device, so a mistake in +them is found late and painfully. The checks here are the ones that would have +caught real bugs: the install must not depend on the network, and the service +must not reference something the installer does not create. +""" + +import os +import re +import stat + +HERE = os.path.dirname(os.path.abspath(__file__)) +PROJECT = os.path.dirname(HERE) +DEPLOY = os.path.join(PROJECT, "deploy") + +INSTALL_SH = os.path.join(DEPLOY, "install.sh") +UNIT = os.path.join(DEPLOY, "rdm-card-reader.service") +RULES = os.path.join(DEPLOY, "99-rdm-reader.rules") + + +def read(path): + with open(path, "r") as handle: + return handle.read() + + +# -- install.sh ------------------------------------------------------------- + +def test_installer_exists_and_is_executable(): + assert os.path.exists(INSTALL_SH) + mode = os.stat(INSTALL_SH).st_mode + assert mode & stat.S_IXUSR, "install.sh should be executable" + + +def test_installer_does_not_pip_install_the_project(): + """`pip install .` needs setuptools from the network, so it breaks offline. + + This is the bug that made installation fail on an air-gapped Pi with + "Could not find a version that satisfies the requirement setuptools>=68". + """ + text = read(INSTALL_SH) + code = "\n".join( + line for line in text.splitlines() if not line.strip().startswith("#") + ) + assert not re.search(r'pip"?\s+install[^\n]*"\$INSTALL_DIR"', code), ( + "install.sh must not 'pip install' the project directory: it requires " + "downloading setuptools and fails with no network" + ) + assert not re.search(r"pip\s+install\s+\.\s*$", code, re.M) + + +def test_installer_installs_runtime_deps_from_bundled_wheels(): + text = read(INSTALL_SH) + assert "--no-index" in text + assert "--find-links" in text + assert "wheels" in text + + +def test_installer_adds_src_to_path(): + """Without this the package is not importable at all.""" + text = read(INSTALL_SH) + assert ".pth" in text + + +def test_installer_network_steps_are_not_fatal(): + """A device with no internet must still get a working install.""" + text = read(INSTALL_SH) + for line in text.splitlines(): + stripped = line.strip() + if stripped.startswith("apt-get"): + assert "|| " in line or line.rstrip().endswith("\\"), ( + "network-dependent step should not abort the install: " + stripped + ) + # The pip self-upgrade is best-effort too. The fallback may sit on the + # following line (line continuation), so join continuations first. + joined = text.replace("\\\n", " ") + assert re.search(r"upgrade pip[^\n]*\|\|", joined), ( + "'pip install --upgrade pip' needs the network, so it must not abort " + "the install; add '|| ...' to it" + ) + + +def test_installer_copies_the_wheels_with_the_source(): + """The bundled wheels live under src/, so src/ must be copied.""" + text = read(INSTALL_SH) + match = re.search(r"for item in ([^\n;]+);", text) + assert match, "could not find the copy list in install.sh" + assert "src" in match.group(1) + + +def test_installer_checks_venv_creation_can_fail_clearly(): + text = read(INSTALL_SH) + assert "python3-venv" in text, "should mention the usual cause of venv failure" + + +def test_installer_verifies_the_install_before_enabling_service(): + text = read(INSTALL_SH) + assert "--version" in text, "installer should sanity-check the install" + + +def test_installer_copies_launch_doc(): + text = read(INSTALL_SH) + assert "LAUNCH.md" in text + + +# -- service unit ----------------------------------------------------------- + +def test_unit_exists(): + assert os.path.exists(UNIT) + + +def test_unit_restarts_on_failure(): + text = read(UNIT) + assert "Restart=always" in text + assert "RestartSec=" in text + + +def test_unit_runs_as_a_non_root_user_in_dialout(): + text = read(UNIT) + assert re.search(r"^User=", text, re.M) + assert re.search(r"^Group=dialout", text, re.M) + + +def _sections(text): + """Split an INI-style unit into {section: body}. + + Splitting on the literal string "[Service]" is wrong: the file mentions it + in a comment above the real section header. + """ + sections = {} + current = None + for line in text.splitlines(): + header = re.match(r"^\[([^\]]+)\]\s*$", line) + if header: + current = header.group(1) + sections[current] = [] + elif current is not None: + sections[current].append(line) + return {name: "\n".join(body) for name, body in sections.items()} + + +def test_unit_start_limits_are_in_the_unit_section(): + """systemd silently ignores these in [Service].""" + sections = _sections(read(UNIT)) + assert "Unit" in sections and "Service" in sections + for key in ("StartLimitIntervalSec", "StartLimitBurst"): + assert key not in sections["Service"], ( + "{} belongs in [Unit]; systemd ignores it in [Service]".format(key) + ) + assert "StartLimitBurst" in sections["Unit"] + + +def test_unit_has_the_expected_sections(): + sections = _sections(read(UNIT)) + assert set(sections) >= {"Unit", "Service", "Install"} + + +def test_unit_uses_loopback_by_default(): + text = read(UNIT) + # No --host flag means the CLI default (127.0.0.1) applies. + assert "--host" not in text + + +def test_unit_serves_in_login_mode(): + text = read(UNIT) + assert "--serve" in text + assert "--mode login" in text + + +def test_unit_points_at_the_stable_device_alias(): + text = read(UNIT) + assert "/dev/rdm-reader" in text + + +# -- udev rule -------------------------------------------------------------- + +def test_udev_rule_creates_the_alias_the_unit_expects(): + text = read(RULES) + assert 'SYMLINK+="rdm-reader"' in text + unit = read(UNIT) + assert "/dev/rdm-reader" in unit, "unit and rule must agree on the alias name" + + +def test_udev_rule_sets_group_and_mode(): + text = read(RULES) + assert 'GROUP="dialout"' in text + assert 'MODE="0660"' in text + + +def test_udev_rule_only_has_one_active_match(): + """Multiple active rules would fight over the same alias.""" + active = [ + line + for line in read(RULES).splitlines() + if line.strip().startswith('SUBSYSTEM=="tty"') + ] + assert len(active) == 1, "exactly one rule should be active, found {}".format( + len(active) + )