diff --git a/gitea-python-ci-cd.md b/gitea-python-ci-cd.md index 1933825..604c09c 100644 --- a/gitea-python-ci-cd.md +++ b/gitea-python-ci-cd.md @@ -84,6 +84,41 @@ asyncio_mode = "auto" addopts = "-v --cov=your_package --cov-report=html --cov-report=term" ``` +### 1.1. Backend Pytest in Isolated venv (for monorepos and persistent runners) + +The pattern in section 1 above works for single-package projects on ephemeral runners. For monorepo backends (multiple `packages/*` with shared deps) or any setup on a **persistent** Gitea Act Runner (TrueNAS, self-hosted), install into a per-run venv at `/tmp/venv` instead of the toolcache Python: + +```yaml +- name: Build wheels in isolated venv + run: | + python -m venv /tmp/venv + /tmp/venv/bin/pip install --upgrade pip build + mkdir -p /tmp/wheels + /tmp/venv/bin/pip wheel --no-deps -w /tmp/wheels packages/core packages/server + +- name: Install backend + test deps + run: | + /tmp/venv/bin/pip install /tmp/wheels/*.whl pytest pytest-asyncio httpx aioresponses + +- name: Run pytest + env: + YOUR_APP_DATA_DIR: /tmp/app-test-data + YOUR_APP_SECRET_KEY: ci-secret-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx + run: | + cd packages/server + /tmp/venv/bin/pytest tests --tb=short +``` + +**Why a venv (and not just the toolcache Python):** + +- `actions/setup-python@v5` reuses the toolcache between runs on persistent runners. A previous broken-wheel install (e.g. a hatchling wheel with missing METADATA Version — see section 10.1) leaves a `dist-info/` dir with no RECORD file. The next run's `pip install` tries to upgrade, fails with `error: uninstall-no-record-file`, and the test gate is permanently red until someone SSHes in to clean site-packages by hand. +- Each CI run gets a fresh `/tmp/venv` with empty site-packages → no leftover state, no broken installs to uninstall. +- `pip install --force-reinstall` / `--ignore-installed` are partial workarounds but don't help when the RECORD file is missing — pip still refuses to overwrite. The venv is the only reliable fix. + +**Why wheels and not editable for monorepos:** measured 4-6x slower with `pip install -e packages/core packages/server` because hatchling's editable build hook re-resolves on every pytest collection. Wheels build once, install once, then pytest runs at native speed. + +**Wheel cache trade-off:** the wheels themselves are NOT worth caching — their hashes depend on every file under `packages/`, so the cache invalidates on basically every PR. Pip's HTTP cache for the test deps is enough. + ## 2. Release Workflow **Trigger:** Push a tag matching `v*` (e.g., `v0.2.0`, `v0.2.0-alpha.1`). @@ -884,6 +919,90 @@ except PackageNotFoundError: This reads whatever version `pip install` baked in. No manual syncing needed. +### 10.1.1. Defense-in-depth: expose `__version__` as a fallback + +Section 10.1 above is the right primary pattern, but `pip wheel --no-deps` on certain hatchling configurations has been observed to produce wheels whose installed METADATA file is **missing the `Version` field entirely**. `importlib.metadata.version()` then returns `None` (not `PackageNotFoundError`), and any resolver that doesn't handle that fall-through advertises `0.0.0+unknown` (or whatever its fallback is) in the UI — even though the package itself is correctly versioned, only the metadata is broken. + +Observed symptoms: + +- `importlib.metadata.version("your-package")` returns `None` with `DeprecationWarning: Implicit None on return values is deprecated and will raise KeyErrors. return self.metadata['Version']`. +- `pip` reports the install as `Found existing installation: your-package None`. +- Reproduced on Python 3.12.12 with hatchling-built wheels in a monorepo. Same monorepo, same `pip wheel --no-deps` invocation: the `core` package wheel had proper Version metadata; the `server` package wheel did not. Root cause not fully pinned (possibly a `[project.scripts]` interaction). + +**Bulletproof workaround:** expose `__version__` from `__init__.py` as well, and pick the highest of (importlib metadata, `__version__`, source pyproject) in the resolver: + +```python +# src/your_package/__init__.py +__version__ = "0.8.1" # synced with pyproject.toml on every release +``` + +```python +# src/your_package/version.py +from importlib.metadata import version as _pkg_version, PackageNotFoundError +from pathlib import Path + +_UNKNOWN = "0.0.0+unknown" + + +def _read_package_version() -> str | None: + """Read ``__version__`` from the package's own ``__init__.py``.""" + try: + from . import __version__ as pkg_version + except ImportError: + return None + return str(pkg_version) if pkg_version else None + + +def _read_source_version() -> str | None: + """Best-effort read of source pyproject.toml — only finds it in editable + installs, not in production wheels.""" + pyproject = Path(__file__).resolve().parents[2] / "pyproject.toml" + if not pyproject.is_file(): + return None + import tomllib + return tomllib.loads(pyproject.read_text(encoding="utf-8")).get("project", {}).get("version") + + +def _segments(version: str) -> tuple[int, ...]: + head = version.split("+", 1)[0].split("-", 1)[0] + out = [] + for piece in head.split("."): + digits = "".join(c for c in piece if c.isdigit()) + if digits: + out.append(int(digits)) + return tuple(out) + + +def resolve_version() -> str: + try: + installed = _pkg_version("your-package-name") + except PackageNotFoundError: + installed = None + package = _read_package_version() + source = _read_source_version() + + candidates = [v for v in (installed, package, source) if v] + if not candidates: + return _UNKNOWN + best = candidates[0] + for cand in candidates[1:]: + if _segments(cand) >= _segments(best): + best = cand + return best +``` + +**Cost:** `__version__` becomes a third file to sync on every release (along with `pyproject.toml` and any frontend `package.json`). Add it to the release-publisher's version-bump scan list. + +**When you DON'T need this:** + +- Single-package project where wheels are produced via `python -m build` (not `pip wheel --no-deps`) and the test gate has actually verified the install works on a fresh runner. +- Projects using a different build backend (setuptools, poetry-core, pdm) that hasn't been observed to drop the Version field. + +**When you DO need this:** + +- Monorepo with multiple hatchling-built packages installed via the `pip wheel + pip install /tmp/wheels/*.whl` pattern (section 1.1). +- Any project where the runtime version string is user-visible (Settings page, `/health` endpoint, log line) AND missing it would be a bug. + ### 10.2. CI Stamps the Tag into `pyproject.toml` Build scripts resolve the version (see section 3), then stamp it before building: @@ -1173,6 +1292,31 @@ the `dist/` directory is a snapshot and won't pick up source changes automatical See section 13.3. The `.onInit` function in section 6 shows how to detect a locked `python.exe` and prompt the user before proceeding. +### `error: uninstall-no-record-file` in backend pytest + +**Symptom:** Second or later run of the backend test job fails at the install step: + +``` +notify-bridge-core is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel. +Installing collected packages: notify-bridge-server + Attempting uninstall: notify-bridge-server + Found existing installation: notify-bridge-server None +error: uninstall-no-record-file +``` + +The version is reported as `None` because the wheel's METADATA file is missing the Version field (hatchling + `pip wheel --no-deps` quirk — see section 10.1.1). The dist-info dir was written without a RECORD file, so pip can't uninstall and refuses to upgrade. + +**Fix:** Install into `/tmp/venv` instead of the toolcache Python — see section 1.1. Each CI run gets a fresh site-packages so there's nothing broken to uninstall. + +**One-shot recovery on a stuck runner:** SSH into the runner and delete the stale dist-info: + +```bash +PYTHON_SITE=$(python3 -c "import site; print(site.getsitepackages()[0])") +rm -rf "$PYTHON_SITE"/your_package*-*.dist-info +``` + +Then the next CI run can install cleanly. But adopt the venv pattern in 1.1 to prevent the problem from recurring. + ### Pillow / package version mismatch on upgrade **Symptom (at runtime, after upgrading the installed app):**