ske087/rdm-card-reader
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix install.sh failing on offline Raspberry Pi at the venv step

Browse Source 
install.sh ran 'pip install .', which builds the project: pip downloads
setuptools into an isolated build environment. With no internet it fails at
the 'Building the virtual environment' step with

    Could not find a version that satisfies the requirement setuptools>=68
    (from versions: none)

even though the runtime libraries are already in the wheel directory. The
build backend is only needed to package, never to run.

- Install the runtime deps straight from the bundled wheels with --no-index,
  and put src/ on the path via a .pth file, so no build step happens at all.
- Generate the console script with the venv's interpreter as an absolute
  shebang. '#\!/usr/bin/env python3' resolves to the *system* python when the
  venv is not activated, so the package was not importable from the unit.
- apt-get update/install and 'pip install --upgrade pip' are now best-effort:
  all three need the network and none should abort the install.
- Detect a failed 'python3 -m venv' and name python3-venv as the usual cause.
- Sanity-check the install with --version before enabling the service.
- The unit falls back to 'python -m rdm_card_reader' if the console script is
  ever absent (no shell in ExecStart, so the installer picks the path).

Reproduced the failure offline, then verified the fix offline as well: a venv
created on a copy of the tree self-provisions from the bundled wheels and runs
from cwd=/ with no PYTHONPATH and no venv activation.

Added tests/test_deploy.py (20 tests) covering the installer and the deploy
artifacts, including a regression test for 'pip install .' that was confirmed
to fail when the bug is reintroduced.
This commit is contained in:
scheianu
2026-10-01 16:19:33 +03:00
parent 674d91a5ca
commit 2ded37e190
4 changed files with 403 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LAUNCH.md
+85
View File
@@ -30,6 +30,70 @@ All commands run from the project root. The virtualenv is already built.
```
After `pip install .` the same thing is available as `rdm-card-reader …`.
`pip install .` needs the network — see **Troubleshooting** below for the
offline route.
---
## Troubleshooting install
### "Could not find a version that satisfies the requirement setuptools>=68"
Seen when running `deploy/install.sh` on a Pi with no internet, at the
"Building the virtual environment" step:
```
ERROR: Could not find a version that satisfies the requirement setuptools>=68
(from versions: none)
ERROR: No matching distribution found for setuptools>=68
```
**Cause:** `pip install .` builds the project. pip creates an isolated build
environment and downloads the *build* backend (`setuptools`) into it — even
though the *runtime* libraries are already sitting in the wheel directory. With
no network that download cannot happen.
**It is not** a problem with the bundled wheels, and not a Python version
mismatch. Two separate confusions produce a similar-looking error:
| Situation | Error |
| --- | --- |
| Build isolation, no network | `Could not find a version ... setuptools>=68` |
| `--no-build-isolation`, and setuptools absent | `Cannot import 'setuptools.build_meta'` |
Note that Python 3.12+ venvs no longer include `setuptools`, so the second case
is increasingly common.
**Fix:** the current `install.sh` avoids the build entirely — it installs the
bundled runtime wheels and puts `src/` on the path with a `.pth` file. If you
are on an older copy, update it (`git pull`) or run these by hand on the Pi:
```bash
python3 -m venv .venv
.venv/bin/pip install --no-index \
--find-links src/rdm_card_reader/wheels rdm6300 pyserial
SITE=$(.venv/bin/python -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
echo "$PWD/src" > "$SITE/rdm_card_reader_src.pth"
.venv/bin/python -m rdm_card_reader --demo # verify, no hardware needed
```
No build step, no network, and the same result.
### "Could not create the virtualenv"
On Debian and Raspberry Pi OS, `python3 -m venv` needs the `python3-venv`
package:
```bash
sudo apt-get install python3-venv
```
### apt-get failing on the first step
`apt-get update` and `apt-get install` need the network. The installer now
treats both as best-effort and continues, because a device typically already
has `python3`. If they fail and something really is missing, install it while
online and re-run.
---
@@ -240,6 +304,27 @@ journalctl -u rdm-card-reader -f
curl http://127.0.0.1:8765/api/card # tap a card first
```
The installer needs no internet: the runtime libraries ship inside the
source tree (`src/rdm_card_reader/wheels/`).
### Copying the project to the Pi
`install.sh` resolves its source directory from its own location, so copy the
whole tree (including `src/` — that is where the wheels live) and run it from
there:
```bash
# from this machine
scp -r rdm-card-reader pi@192.168.0.123:~/Desktop/
# on the Pi
cd ~/Desktop/rdm-card-reader
sudo ./deploy/install.sh --origin https://your.web.app
```
If you transferred only a subset of the tree, `--dir` can point the install
somewhere else, but `src/` must be present or the wheels will be missing.
---
## Testing the reader path without a reader
README.md
+30 -3
View File
@@ -53,18 +53,45 @@ without its library is worse than one that refuses to start.
## Install
On a machine **with internet**:
```bash
pip install .
```
On a Raspberry Pi OS install with an externally-managed Python, add
`--break-system-packages` or use a virtual environment:
`rdm6300` and `pyserial` are pulled in automatically. On Raspberry Pi OS with
an externally-managed Python, add `--break-system-packages` or use a venv:
```bash
python3 -m venv .venv && . .venv/bin/activate && pip install .
```
`rdm6300` and `pyserial` are pulled in automatically.
### On a device with no internet
`pip install .` will **not** work offline. It builds the project, which makes
pip download `setuptools` into an isolated build environment, and that fails
with:
```
ERROR: Could not find a version that satisfies the requirement setuptools>=68
(from versions: none)
```
The runtime dependencies are bundled in `src/rdm_card_reader/wheels/`, so
install those directly and put `src` on the path — no build step, no network:
```bash
python3 -m venv .venv
.venv/bin/pip install --no-index \
--find-links src/rdm_card_reader/wheels rdm6300 pyserial
echo "$PWD/src" > "$( .venv/bin/python -c \
'import sysconfig; print(sysconfig.get_paths()["purelib"])' )/rdm_card_reader_src.pth"
.venv/bin/python -m rdm_card_reader --demo # verify, no hardware needed
```
`deploy/install.sh` does exactly this, plus the service setup. Running the
app directly also works: it installs anything missing from the same bundled
wheels on first start.
---
deploy/install.sh
+89 -8
View File
@@ -47,8 +47,11 @@ fi
# 1. System packages
# ---------------------------------------------------------------------------
log "Checking system packages"
apt-get update -qq
apt-get install -y --no-install-recommends python3 python3-venv python3-pip
# Both of these need the network, so neither may abort the install: a device
# with no internet usually already has python3, and we can work without them.
apt-get update -qq 2>/dev/null || warn "apt-get update failed (no network?) - continuing"
apt-get install -y --no-install-recommends python3 python3-venv python3-pip \
|| warn "could not install system packages - continuing with what is present"
# ---------------------------------------------------------------------------
# 2. Install the application
@@ -57,16 +60,86 @@ log "Installing to $INSTALL_DIR"
mkdir -p "$INSTALL_DIR"
# Copy the source tree (excluding local build/test artifacts).
for item in src tests pyproject.toml README.md LICENSE deploy; do
for item in src tests pyproject.toml README.md LAUNCH.md LICENSE deploy; do
if [[ -e "$SOURCE_DIR/$item" ]]; then
cp -a "$SOURCE_DIR/$item" "$INSTALL_DIR/"
fi
done
log "Building the virtual environment"
python3 -m venv "$INSTALL_DIR/.venv"
"$INSTALL_DIR/.venv/bin/pip" install --quiet --upgrade pip
"$INSTALL_DIR/.venv/bin/pip" install --quiet "$INSTALL_DIR"
if ! python3 -m venv "$INSTALL_DIR/.venv"; then
warn "could not create the virtualenv."
warn "On Debian/Raspberry Pi OS this usually means python3-venv is missing:"
warn " sudo apt-get install python3-venv"
exit 1
fi
VENV_PY="$INSTALL_DIR/.venv/bin/python"
WHEEL_DIR="$INSTALL_DIR/src/rdm_card_reader/wheels"
# Upgrading pip is optional and needs the network.
"$INSTALL_DIR/.venv/bin/pip" install --quiet --upgrade pip 2>/dev/null \
|| echo " (kept the bundled pip - no network)"
# Install the runtime dependencies from the wheels shipped in the source tree.
#
# Deliberately NOT 'pip install .': that builds the project, which requires
# downloading setuptools into an isolated build environment. With no internet
# it fails with
# Could not find a version that satisfies the requirement setuptools>=68
# even though everything needed to *run* is sitting in the wheel directory.
# Installing the runtime dependencies directly, and putting 'src' on the
# path, achieves the same thing and works offline.
INSTALLED_PACKAGE=0
if [[ -d "$WHEEL_DIR" ]] && "$VENV_PY" -m pip install --quiet --no-index \
--find-links "$WHEEL_DIR" rdm6300 pyserial 2>/dev/null; then
echo " installed rdm6300 and pyserial from the bundled wheels (offline)"
else
echo " bundled wheels unavailable - falling back to PyPI"
if "$VENV_PY" -m pip install --quiet rdm6300 pyserial 2>/dev/null; then
echo " installed rdm6300 and pyserial from PyPI"
else
warn "could not install rdm6300 / pyserial."
warn "The app will try again on first start (it can install from its own"
warn "bundled wheels), but check network access if that also fails."
fi
fi
# Make the package importable. The .pth file adds src/ to sys.path, so
# 'python -m rdm_card_reader' resolves from any working directory.
SITE_PACKAGES="$("$VENV_PY" -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')"
echo "$INSTALL_DIR/src" > "$SITE_PACKAGES/rdm_card_reader_src.pth"
# Console-script equivalent, so the unit and the docs work as written.
# The shebang must be the venv's interpreter by absolute path: '#!/usr/bin/env
# python3' would pick up the *system* python when the venv is not activated,
# and the package would not be importable.
cat > "$INSTALL_DIR/.venv/bin/rdm-card-reader" <<EOF
#!$VENV_PY
import sys
from rdm_card_reader.cli import main
if __name__ == "__main__":
sys.exit(main())
EOF
chmod +x "$INSTALL_DIR/.venv/bin/rdm-card-reader"
# Standalone launcher for running without activating anything.
cat > "$INSTALL_DIR/run.sh" <<EOF
#!/bin/sh
# Convenience launcher: ./run.sh [args...]
exec "$INSTALL_DIR/.venv/bin/rdm-card-reader" "\$@"
EOF
chmod +x "$INSTALL_DIR/run.sh"
# Sanity check: prove it imports and can run before we enable a service.
if "$INSTALL_DIR/.venv/bin/rdm-card-reader" --version >/dev/null 2>&1; then
echo " verified: $("$INSTALL_DIR/.venv/bin/rdm-card-reader" --version)"
else
warn "the reader does not run yet. Check with:"
warn " $INSTALL_DIR/.venv/bin/python -m rdm_card_reader --list-ports"
fi
# ---------------------------------------------------------------------------
# 3. Service account
@@ -114,12 +187,20 @@ for origin in "${ORIGINS[@]:-}"; do
done
python3 - "$UNIT" "$INSTALL_DIR" "$BRIDGE_PORT" "$ORIGIN_ARGS" "$SERVICE_USER" <<'PY'
import re, sys
import os, re, sys
unit, install_dir, port, origin_args, user = sys.argv[1:6]
text = open(unit).read()
# The console script is created by the installer, but the .pth file is what
# makes the package importable, so fall back to 'python -m' if it is ever
# absent. systemd's ExecStart has no shell, so no pipes or conditionals.
wrapper = f"{install_dir}/.venv/bin/rdm-card-reader"
if not os.path.exists(wrapper):
wrapper = f"{install_dir}/.venv/bin/python -m rdm_card_reader"
command = (
f"{install_dir}/.venv/bin/rdm-card-reader --serve "
f"{wrapper} --serve "
f"--port /dev/rdm-reader --bridge-port {port} --mode login{origin_args}"
)
# Replace the whole (possibly line-continued) ExecStart definition.
tests/test_deploy.py
+199
View File
@@ -0,0 +1,199 @@
"""Tests for the deploy artifacts.
These are the files that only ever run on the target device, so a mistake in
them is found late and painfully. The checks here are the ones that would have
caught real bugs: the install must not depend on the network, and the service
must not reference something the installer does not create.
"""
import os
import re
import stat
HERE = os.path.dirname(os.path.abspath(__file__))
PROJECT = os.path.dirname(HERE)
DEPLOY = os.path.join(PROJECT, "deploy")
INSTALL_SH = os.path.join(DEPLOY, "install.sh")
UNIT = os.path.join(DEPLOY, "rdm-card-reader.service")
RULES = os.path.join(DEPLOY, "99-rdm-reader.rules")
def read(path):
with open(path, "r") as handle:
return handle.read()
# -- install.sh -------------------------------------------------------------
def test_installer_exists_and_is_executable():
assert os.path.exists(INSTALL_SH)
mode = os.stat(INSTALL_SH).st_mode
assert mode & stat.S_IXUSR, "install.sh should be executable"
def test_installer_does_not_pip_install_the_project():
"""`pip install .` needs setuptools from the network, so it breaks offline.
This is the bug that made installation fail on an air-gapped Pi with
"Could not find a version that satisfies the requirement setuptools>=68".
"""
text = read(INSTALL_SH)
code = "\n".join(
line for line in text.splitlines() if not line.strip().startswith("#")
)
assert not re.search(r'pip"?\s+install[^\n]*"\$INSTALL_DIR"', code), (
"install.sh must not 'pip install' the project directory: it requires "
"downloading setuptools and fails with no network"
)
assert not re.search(r"pip\s+install\s+\.\s*$", code, re.M)
def test_installer_installs_runtime_deps_from_bundled_wheels():
text = read(INSTALL_SH)
assert "--no-index" in text
assert "--find-links" in text
assert "wheels" in text
def test_installer_adds_src_to_path():
"""Without this the package is not importable at all."""
text = read(INSTALL_SH)
assert ".pth" in text
def test_installer_network_steps_are_not_fatal():
"""A device with no internet must still get a working install."""
text = read(INSTALL_SH)
for line in text.splitlines():
stripped = line.strip()
if stripped.startswith("apt-get"):
assert "|| " in line or line.rstrip().endswith("\\"), (
"network-dependent step should not abort the install: " + stripped
)
# The pip self-upgrade is best-effort too. The fallback may sit on the
# following line (line continuation), so join continuations first.
joined = text.replace("\\\n", " ")
assert re.search(r"upgrade pip[^\n]*\|\|", joined), (
"'pip install --upgrade pip' needs the network, so it must not abort "
"the install; add '|| ...' to it"
)
def test_installer_copies_the_wheels_with_the_source():
"""The bundled wheels live under src/, so src/ must be copied."""
text = read(INSTALL_SH)
match = re.search(r"for item in ([^\n;]+);", text)
assert match, "could not find the copy list in install.sh"
assert "src" in match.group(1)
def test_installer_checks_venv_creation_can_fail_clearly():
text = read(INSTALL_SH)
assert "python3-venv" in text, "should mention the usual cause of venv failure"
def test_installer_verifies_the_install_before_enabling_service():
text = read(INSTALL_SH)
assert "--version" in text, "installer should sanity-check the install"
def test_installer_copies_launch_doc():
text = read(INSTALL_SH)
assert "LAUNCH.md" in text
# -- service unit -----------------------------------------------------------
def test_unit_exists():
assert os.path.exists(UNIT)
def test_unit_restarts_on_failure():
text = read(UNIT)
assert "Restart=always" in text
assert "RestartSec=" in text
def test_unit_runs_as_a_non_root_user_in_dialout():
text = read(UNIT)
assert re.search(r"^User=", text, re.M)
assert re.search(r"^Group=dialout", text, re.M)
def _sections(text):
"""Split an INI-style unit into {section: body}.
Splitting on the literal string "[Service]" is wrong: the file mentions it
in a comment above the real section header.
"""
sections = {}
current = None
for line in text.splitlines():
header = re.match(r"^\[([^\]]+)\]\s*$", line)
if header:
current = header.group(1)
sections[current] = []
elif current is not None:
sections[current].append(line)
return {name: "\n".join(body) for name, body in sections.items()}
def test_unit_start_limits_are_in_the_unit_section():
"""systemd silently ignores these in [Service]."""
sections = _sections(read(UNIT))
assert "Unit" in sections and "Service" in sections
for key in ("StartLimitIntervalSec", "StartLimitBurst"):
assert key not in sections["Service"], (
"{} belongs in [Unit]; systemd ignores it in [Service]".format(key)
)
assert "StartLimitBurst" in sections["Unit"]
def test_unit_has_the_expected_sections():
sections = _sections(read(UNIT))
assert set(sections) >= {"Unit", "Service", "Install"}
def test_unit_uses_loopback_by_default():
text = read(UNIT)
# No --host flag means the CLI default (127.0.0.1) applies.
assert "--host" not in text
def test_unit_serves_in_login_mode():
text = read(UNIT)
assert "--serve" in text
assert "--mode login" in text
def test_unit_points_at_the_stable_device_alias():
text = read(UNIT)
assert "/dev/rdm-reader" in text
# -- udev rule --------------------------------------------------------------
def test_udev_rule_creates_the_alias_the_unit_expects():
text = read(RULES)
assert 'SYMLINK+="rdm-reader"' in text
unit = read(UNIT)
assert "/dev/rdm-reader" in unit, "unit and rule must agree on the alias name"
def test_udev_rule_sets_group_and_mode():
text = read(RULES)
assert 'GROUP="dialout"' in text
assert 'MODE="0660"' in text
def test_udev_rule_only_has_one_active_match():
"""Multiple active rules would fight over the same alias."""
active = [
line
for line in read(RULES).splitlines()
if line.strip().startswith('SUBSYSTEM=="tty"')
]
assert len(active) == 1, "exactly one rule should be active, found {}".format(
len(active)
)