docs(gitea-python-ci-cd): venv install for monorepos + hatchling METADATA workaround
Two real failures hit during notify-bridge v0.8.1 release that section 1 and section 10.1 didn't cover: 1. New section 1.1 — backend pytest in a per-run /tmp/venv. Single-package ephemeral runners get away with pip install -e into the toolcache; monorepos on persistent Gitea Act Runners (TrueNAS) hit "error: uninstall-no-record-file" the second time a broken wheel install leaks across runs. Venv per run, fresh site-packages, no leak. 2. New section 10.1.1 — pip wheel --no-deps on hatchling has been observed to produce wheels with METADATA missing the Version field. importlib.metadata.version() returns None and the UI advertises 0.0.0+unknown. Defense-in-depth: also expose __version__ from __init__.py and pick the max of (metadata, __version__, source pyproject) in the resolver. Documented when it's needed and when section 10.1 alone is fine. 3. New section 14 troubleshooting entry — quotes the exact pip error message and points at section 1.1 (prevention) and a one-shot site-packages cleanup recipe (recovery on a stuck runner).
This commit is contained in:
@@ -84,6 +84,41 @@ asyncio_mode = "auto"
|
||||
addopts = "-v --cov=your_package --cov-report=html --cov-report=term"
|
||||
```
|
||||
|
||||
### 1.1. Backend Pytest in Isolated venv (for monorepos and persistent runners)
|
||||
|
||||
The pattern in section 1 above works for single-package projects on ephemeral runners. For monorepo backends (multiple `packages/*` with shared deps) or any setup on a **persistent** Gitea Act Runner (TrueNAS, self-hosted), install into a per-run venv at `/tmp/venv` instead of the toolcache Python:
|
||||
|
||||
```yaml
|
||||
- name: Build wheels in isolated venv
|
||||
run: |
|
||||
python -m venv /tmp/venv
|
||||
/tmp/venv/bin/pip install --upgrade pip build
|
||||
mkdir -p /tmp/wheels
|
||||
/tmp/venv/bin/pip wheel --no-deps -w /tmp/wheels packages/core packages/server
|
||||
|
||||
- name: Install backend + test deps
|
||||
run: |
|
||||
/tmp/venv/bin/pip install /tmp/wheels/*.whl pytest pytest-asyncio httpx aioresponses
|
||||
|
||||
- name: Run pytest
|
||||
env:
|
||||
YOUR_APP_DATA_DIR: /tmp/app-test-data
|
||||
YOUR_APP_SECRET_KEY: ci-secret-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||
run: |
|
||||
cd packages/server
|
||||
/tmp/venv/bin/pytest tests --tb=short
|
||||
```
|
||||
|
||||
**Why a venv (and not just the toolcache Python):**
|
||||
|
||||
- `actions/setup-python@v5` reuses the toolcache between runs on persistent runners. A previous broken-wheel install (e.g. a hatchling wheel with missing METADATA Version — see section 10.1) leaves a `dist-info/` dir with no RECORD file. The next run's `pip install` tries to upgrade, fails with `error: uninstall-no-record-file`, and the test gate is permanently red until someone SSHes in to clean site-packages by hand.
|
||||
- Each CI run gets a fresh `/tmp/venv` with empty site-packages → no leftover state, no broken installs to uninstall.
|
||||
- `pip install --force-reinstall` / `--ignore-installed` are partial workarounds but don't help when the RECORD file is missing — pip still refuses to overwrite. The venv is the only reliable fix.
|
||||
|
||||
**Why wheels and not editable for monorepos:** measured 4-6x slower with `pip install -e packages/core packages/server` because hatchling's editable build hook re-resolves on every pytest collection. Wheels build once, install once, then pytest runs at native speed.
|
||||
|
||||
**Wheel cache trade-off:** the wheels themselves are NOT worth caching — their hashes depend on every file under `packages/`, so the cache invalidates on basically every PR. Pip's HTTP cache for the test deps is enough.
|
||||
|
||||
## 2. Release Workflow
|
||||
|
||||
**Trigger:** Push a tag matching `v*` (e.g., `v0.2.0`, `v0.2.0-alpha.1`).
|
||||
@@ -884,6 +919,90 @@ except PackageNotFoundError:
|
||||
|
||||
This reads whatever version `pip install` baked in. No manual syncing needed.
|
||||
|
||||
### 10.1.1. Defense-in-depth: expose `__version__` as a fallback
|
||||
|
||||
Section 10.1 above is the right primary pattern, but `pip wheel --no-deps` on certain hatchling configurations has been observed to produce wheels whose installed METADATA file is **missing the `Version` field entirely**. `importlib.metadata.version()` then returns `None` (not `PackageNotFoundError`), and any resolver that doesn't handle that fall-through advertises `0.0.0+unknown` (or whatever its fallback is) in the UI — even though the package itself is correctly versioned, only the metadata is broken.
|
||||
|
||||
Observed symptoms:
|
||||
|
||||
- `importlib.metadata.version("your-package")` returns `None` with `DeprecationWarning: Implicit None on return values is deprecated and will raise KeyErrors. return self.metadata['Version']`.
|
||||
- `pip` reports the install as `Found existing installation: your-package None`.
|
||||
- Reproduced on Python 3.12.12 with hatchling-built wheels in a monorepo. Same monorepo, same `pip wheel --no-deps` invocation: the `core` package wheel had proper Version metadata; the `server` package wheel did not. Root cause not fully pinned (possibly a `[project.scripts]` interaction).
|
||||
|
||||
**Bulletproof workaround:** expose `__version__` from `__init__.py` as well, and pick the highest of (importlib metadata, `__version__`, source pyproject) in the resolver:
|
||||
|
||||
```python
|
||||
# src/your_package/__init__.py
|
||||
__version__ = "0.8.1" # synced with pyproject.toml on every release
|
||||
```
|
||||
|
||||
```python
|
||||
# src/your_package/version.py
|
||||
from importlib.metadata import version as _pkg_version, PackageNotFoundError
|
||||
from pathlib import Path
|
||||
|
||||
_UNKNOWN = "0.0.0+unknown"
|
||||
|
||||
|
||||
def _read_package_version() -> str | None:
|
||||
"""Read ``__version__`` from the package's own ``__init__.py``."""
|
||||
try:
|
||||
from . import __version__ as pkg_version
|
||||
except ImportError:
|
||||
return None
|
||||
return str(pkg_version) if pkg_version else None
|
||||
|
||||
|
||||
def _read_source_version() -> str | None:
|
||||
"""Best-effort read of source pyproject.toml — only finds it in editable
|
||||
installs, not in production wheels."""
|
||||
pyproject = Path(__file__).resolve().parents[2] / "pyproject.toml"
|
||||
if not pyproject.is_file():
|
||||
return None
|
||||
import tomllib
|
||||
return tomllib.loads(pyproject.read_text(encoding="utf-8")).get("project", {}).get("version")
|
||||
|
||||
|
||||
def _segments(version: str) -> tuple[int, ...]:
|
||||
head = version.split("+", 1)[0].split("-", 1)[0]
|
||||
out = []
|
||||
for piece in head.split("."):
|
||||
digits = "".join(c for c in piece if c.isdigit())
|
||||
if digits:
|
||||
out.append(int(digits))
|
||||
return tuple(out)
|
||||
|
||||
|
||||
def resolve_version() -> str:
|
||||
try:
|
||||
installed = _pkg_version("your-package-name")
|
||||
except PackageNotFoundError:
|
||||
installed = None
|
||||
package = _read_package_version()
|
||||
source = _read_source_version()
|
||||
|
||||
candidates = [v for v in (installed, package, source) if v]
|
||||
if not candidates:
|
||||
return _UNKNOWN
|
||||
best = candidates[0]
|
||||
for cand in candidates[1:]:
|
||||
if _segments(cand) >= _segments(best):
|
||||
best = cand
|
||||
return best
|
||||
```
|
||||
|
||||
**Cost:** `__version__` becomes a third file to sync on every release (along with `pyproject.toml` and any frontend `package.json`). Add it to the release-publisher's version-bump scan list.
|
||||
|
||||
**When you DON'T need this:**
|
||||
|
||||
- Single-package project where wheels are produced via `python -m build` (not `pip wheel --no-deps`) and the test gate has actually verified the install works on a fresh runner.
|
||||
- Projects using a different build backend (setuptools, poetry-core, pdm) that hasn't been observed to drop the Version field.
|
||||
|
||||
**When you DO need this:**
|
||||
|
||||
- Monorepo with multiple hatchling-built packages installed via the `pip wheel + pip install /tmp/wheels/*.whl` pattern (section 1.1).
|
||||
- Any project where the runtime version string is user-visible (Settings page, `/health` endpoint, log line) AND missing it would be a bug.
|
||||
|
||||
### 10.2. CI Stamps the Tag into `pyproject.toml`
|
||||
|
||||
Build scripts resolve the version (see section 3), then stamp it before building:
|
||||
@@ -1173,6 +1292,31 @@ the `dist/` directory is a snapshot and won't pick up source changes automatical
|
||||
See section 13.3. The `.onInit` function in section 6 shows how to detect a locked
|
||||
`python.exe` and prompt the user before proceeding.
|
||||
|
||||
### `error: uninstall-no-record-file` in backend pytest
|
||||
|
||||
**Symptom:** Second or later run of the backend test job fails at the install step:
|
||||
|
||||
```
|
||||
notify-bridge-core is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
|
||||
Installing collected packages: notify-bridge-server
|
||||
Attempting uninstall: notify-bridge-server
|
||||
Found existing installation: notify-bridge-server None
|
||||
error: uninstall-no-record-file
|
||||
```
|
||||
|
||||
The version is reported as `None` because the wheel's METADATA file is missing the Version field (hatchling + `pip wheel --no-deps` quirk — see section 10.1.1). The dist-info dir was written without a RECORD file, so pip can't uninstall and refuses to upgrade.
|
||||
|
||||
**Fix:** Install into `/tmp/venv` instead of the toolcache Python — see section 1.1. Each CI run gets a fresh site-packages so there's nothing broken to uninstall.
|
||||
|
||||
**One-shot recovery on a stuck runner:** SSH into the runner and delete the stale dist-info:
|
||||
|
||||
```bash
|
||||
PYTHON_SITE=$(python3 -c "import site; print(site.getsitepackages()[0])")
|
||||
rm -rf "$PYTHON_SITE"/your_package*-*.dist-info
|
||||
```
|
||||
|
||||
Then the next CI run can install cleanly. But adopt the venv pattern in 1.1 to prevent the problem from recurring.
|
||||
|
||||
### Pillow / package version mismatch on upgrade
|
||||
|
||||
**Symptom (at runtime, after upgrading the installed app):**
|
||||
|
||||
Reference in New Issue
Block a user