From 0a9c270c6b26136230913fb3bd991fb28174e4a9 Mon Sep 17 00:00:00 2001 From: "dolgolyov.alexei" Date: Wed, 25 Mar 2026 12:51:07 +0300 Subject: [PATCH] docs: add version management and in-app auto-update sections - Section 10: Single source of truth via pyproject.toml + importlib.metadata, CI version stamping, Docker build args, updated fallback chain - Section 11: Release provider abstraction, PEP 440 version normalization, install type detection, update service pattern, NSIS silent install, portable ZIP/tarball swap scripts, API endpoints, frontend integration --- gitea-python-ci-cd.md | 253 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 245 insertions(+), 8 deletions(-) diff --git a/gitea-python-ci-cd.md b/gitea-python-ci-cd.md index 737e74d..fcc93bb 100644 --- a/gitea-python-ci-cd.md +++ b/gitea-python-ci-cd.md @@ -507,24 +507,259 @@ git push origin v0.2.0-alpha.1 | Secrets | `${{ secrets.NAME }}` | `${{ secrets.NAME }}` (same) | | Docker Buildx | Available | May not work (runner networking) | -## 10. Checklist for New Projects +## 10. Version Management — Single Source of Truth + +The version must be consistent across the UI, package metadata, installer filename, and Docker label. The simplest approach: **`pyproject.toml` is the single source of truth**, and everything else derives from it. + +### 10.1. Runtime Version via `importlib.metadata` + +Instead of hardcoding `__version__` in `__init__.py`, read it from installed package metadata: + +```python +# src/your_package/__init__.py +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package-name") +except PackageNotFoundError: + __version__ = "0.0.0-dev" # Running from source without pip install +``` + +This reads whatever version `pip install` baked in. No manual syncing needed. + +### 10.2. CI Stamps the Tag into `pyproject.toml` + +Build scripts resolve the version (see section 3), then stamp it before building: + +```bash +VERSION_CLEAN="${VERSION#v}" # "0.3.0" from "v0.3.0" + +# Stamp into pyproject.toml so pip install bakes the correct version +sed -i "s/^version = .*/version = \"${VERSION_CLEAN}\"/" server/pyproject.toml +``` + +This happens in both `build-dist.sh` and `build-dist-windows.sh`, before `pip install` or wheel extraction. + +### 10.3. Docker Version Stamping + +Use a build arg so the version is stamped during the Docker build: + +```dockerfile +ARG APP_VERSION=0.0.0 + +# Stamp before pip install (which bakes version into metadata) +RUN sed -i "s/^version = .*/version = \"${APP_VERSION}\"/" pyproject.toml \ + && pip install --no-cache-dir ".[notifications]" +``` + +In CI: + +```yaml +docker build --build-arg APP_VERSION="${VERSION}" ./server +``` + +The `--label org.opencontainers.image.version` flag can also be set dynamically: + +```yaml +docker build \ + --build-arg APP_VERSION="${VERSION}" \ + --label "org.opencontainers.image.version=${VERSION}" \ + ./server +``` + +### 10.4. Version Fallback Chain (Updated) + +The build script fallback (section 3) should read from `pyproject.toml` instead of `__init__.py`: + +```bash +# Fallback 4: Read from pyproject.toml (single source of truth) +if [ -z "$VERSION" ]; then + VERSION=$(grep -oP '^version\s*=\s*"\K[^"]+' server/pyproject.toml 2>/dev/null || echo "0.0.0") +fi +``` + +### 10.5. Version Flow Summary + +```text +git tag v0.3.0 + → CI reads tag + → sed stamps pyproject.toml with "0.3.0" + → pip install bakes "0.3.0" into package metadata + → importlib.metadata.version() reads "0.3.0" at runtime + → UI shows "v0.3.0", update checker compares against it +``` + +## 11. In-App Auto-Update + +For self-hosted apps distributed as installers or portable ZIPs, you can implement in-app update checking and one-click updates. The design separates the platform-specific hosting API (Gitea, GitHub) from the update logic via an abstraction layer. + +### 11.1. Architecture + +```text +UpdateService (background task) + ├── ReleaseProvider (abstract) + │ ├── GiteaReleaseProvider — GET /api/v1/repos/{repo}/releases + │ └── GitHubReleaseProvider — GET api.github.com/repos/{owner}/{repo}/releases + ├── VersionCheck — normalize semver tags, compare via packaging.version + └── InstallType — detect installer vs portable vs Docker vs dev +``` + +### 11.2. Release Provider Abstraction + +The abstract interface has two methods: + +```python +class ReleaseProvider(ABC): + @abstractmethod + async def get_releases(self, limit: int = 10) -> list[ReleaseInfo]: ... + + @abstractmethod + def get_releases_page_url(self) -> str: ... +``` + +`ReleaseInfo` is a frozen dataclass with `tag`, `version`, `name`, `body`, `prerelease`, `published_at`, and `assets` (tuple of `AssetInfo` with `name`, `size`, `download_url`). + +**Gitea implementation** hits `GET {base_url}/api/v1/repos/{repo}/releases?limit=N`. **GitHub** would hit `api.github.com/repos/{owner}/{repo}/releases`. The rest of the system only depends on the abstract interface — swapping providers is a config change. + +### 11.3. Version Comparison + +Gitea/GitHub tags like `v0.3.0-alpha.1` must be normalized to PEP 440 for comparison: + +```python +import re +from packaging.version import Version + +_PRE_PATTERN = re.compile(r"^(\d+\.\d+\.\d+)[-.]?(alpha|beta|rc)[.-]?(\d+)$", re.IGNORECASE) +_PRE_MAP = {"alpha": "a", "beta": "b", "rc": "rc"} + +def normalize_version(raw: str) -> Version: + cleaned = raw.lstrip("v").strip() + m = _PRE_PATTERN.match(cleaned) + if m: + base, pre_label, pre_num = m.group(1), m.group(2).lower(), m.group(3) + cleaned = f"{base}{_PRE_MAP[pre_label]}{pre_num}" + return Version(cleaned) +``` + +Examples: `v0.3.0-alpha.1` → `0.3.0a1`, `v0.3.0-rc.3` → `0.3.0rc3`, `v1.0.0` → `1.0.0`. + +### 11.4. Install Type Detection + +Detect at startup by checking filesystem markers: + +| Marker | Install type | Auto-update strategy | +|--------|-------------|---------------------| +| `uninstall.exe` in CWD | `installer` | Download `-setup.exe`, run `/S /D=` (silent NSIS reinstall) | +| `python/python.exe` in CWD (no uninstaller) | `portable` (Windows) | Download ZIP, extract, swap `app/` + `python/` via detached bat script | +| `venv/` + `run.sh` in CWD | `portable` (Linux) | Download tarball, extract, swap `app/` + `venv/` via detached shell script | +| `/.dockerenv` exists | `docker` | No auto-update — show "pull new image" instructions | +| None of above | `dev` | No auto-update — link to releases page | + +### 11.5. Update Service (Background Task) + +Follows the same pattern as other background services (e.g., auto-backup): + +- **Settings** persisted in DB: `enabled`, `check_interval_hours`, `include_prerelease` +- **30s startup delay** to avoid slowing boot +- **60s debounce** on manual checks to prevent API spam +- **State machine**: `IDLE → CHECKING → UPDATE_AVAILABLE → DOWNLOADING → APPLYING` +- **WebSocket events**: `update_available` (triggers banner), `update_download_progress` (progress bar) +- **Dismissed version** persisted in DB (survives restarts) + +### 11.6. Update Application Strategies + +**NSIS installer (Windows):** + +```python +# Launch installer silently, detached from server process +subprocess.Popen( + [str(exe_path), "/S", f"/D={install_dir}"], + creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, +) +# Shut down server — installer waits for python.exe to unlock, then installs +request_shutdown() +``` + +The NSIS installer's `.onInit` detects the locked `python.exe` and handles it. After install, the VBS launcher or startup shortcut restarts the app. + +**Portable ZIP (Windows):** + +Can't replace files while the server is running (DLL locks). Solution: write a `.bat` script that runs after shutdown: + +```batch +@echo off +timeout /t 5 /nobreak >nul +rmdir /s /q "C:\path\app" 2>nul +move /y "C:\path\staging\LedGrab\app" "C:\path\app" +rmdir /s /q "C:\path\python" 2>nul +move /y "C:\path\staging\LedGrab\python" "C:\path\python" +rmdir /s /q "C:\path\staging" 2>nul +start "" wscript.exe "C:\path\scripts\start-hidden.vbs" +del /f /q "%~f0" +``` + +Launch it detached via `subprocess.Popen`, then shut down the server. + +**Portable tarball (Linux):** + +Same pattern but with a shell script: + +```bash +#!/bin/bash +sleep 3 +rm -rf "$APP_ROOT/app" && mv "$STAGING/LedGrab/app" "$APP_ROOT/app" +rm -rf "$APP_ROOT/venv" && mv "$STAGING/LedGrab/venv" "$APP_ROOT/venv" +rm -rf "$STAGING" +cd "$APP_ROOT" && exec ./run.sh +``` + +### 11.7. API Endpoints + +| Method | Path | Purpose | +|--------|------|---------| +| `GET` | `/system/update/status` | Current state, available version, install type, progress | +| `POST` | `/system/update/check` | Trigger immediate check | +| `POST` | `/system/update/dismiss` | Dismiss notification for a version | +| `POST` | `/system/update/apply` | Download + install + restart | +| `GET/PUT` | `/system/update/settings` | Auto-check settings | + +### 11.8. Frontend Integration + +- **Version badge**: Add CSS class `has-update` to the version pill in the header — changes color + pulse animation, clickable to open settings +- **Banner**: Persistent dismissible bar below header with icon buttons (external link to releases, download to apply, X to dismiss). Stored in `localStorage` so dismissed state survives page refresh +- **Settings tab**: "Updates" tab showing current version, install type, status, check button, "Update Now" button (visible only when `can_auto_update` is true), download progress bar, release notes preview, auto-check toggle + interval + channel (stable/pre-release) +- **WebSocket events**: `update_available` triggers banner + badge, `update_download_progress` updates progress bar, `server_restarting` shows reconnect overlay + +### 11.9. Key Design Decisions + +1. **Provider abstraction** — Gitea today, GitHub/GitLab tomorrow. One interface, swap implementations via config. +2. **No data directory changes** — Updates replace `app/` and `python/` (or `venv/`) but **never touch `data/`**. User configuration survives every update. +3. **Detached post-update scripts** — The server can't replace its own files while running. A detached script waits for shutdown, does the swap, then restarts. +4. **Silent NSIS reinstall** — NSIS `/S` flag runs the full installer flow without UI, preserving the `data/` directory because the uninstaller section is never triggered during an upgrade. +5. **Asset matching by filename pattern** — e.g., `*-setup.exe` for installer, `*-win-x64.zip` for portable. Platform + install type determines the pattern. +6. **Pre-release channel** — Users opt in to alpha/beta/RC builds via settings. Default is stable only. + +## 12. Checklist for New Projects - [ ] Create `.gitea/workflows/test.yml` — lint + test on push/PR - [ ] Create `.gitea/workflows/release.yml` — build + release on `v*` tag - [ ] Add `GITEA_TOKEN` secret to repository - [ ] Set up version detection in build scripts (tag → env → source) +- [ ] Set up `importlib.metadata` version in `__init__.py` (section 10.1) +- [ ] Add `sed` version stamp step in build scripts (section 10.2) - [ ] Create `build-dist.sh` for Linux (venv + tarball) - [ ] Create `build-dist-windows.sh` for Windows (embedded Python + ZIP) - [ ] Create `installer.nsi` for Windows installer (optional) - [ ] Create `Dockerfile` for Docker builds (optional) - [ ] Configure `pyproject.toml` with `[tool.ruff]` and `[tool.pytest]` - [ ] Set up `.pre-commit-config.yaml` with ruff + black +- [ ] Add in-app update checker if self-hosted (section 11) -## 11. Local Build Testing (Windows) +## 13. Local Build Testing (Windows) The CI builds on Linux, but you can build locally on Windows for faster iteration. -### 11.1. Prerequisites +### 13.1. Prerequisites Install NSIS (for installer builds): @@ -534,7 +769,7 @@ Install NSIS (for installer builds): # Installs to: C:\Program Files (x86)\NSIS\makensis.exe ``` -### 11.2. Build Steps +### 13.2. Build Steps ```bash # 1. Build frontend @@ -548,7 +783,7 @@ bash build-dist-windows.sh v1.0.0 # Output: build/MediaServer-v1.0.0-setup.exe ``` -### 11.3. Common Issues +### 13.3. Common Issues | Issue | Cause | Fix | |-------|-------|-----| @@ -559,8 +794,10 @@ bash build-dist-windows.sh v1.0.0 | `winget` not recognized | `winget.exe` exists but isn't on shell PATH | Use full path: `$env:LOCALAPPDATA\Microsoft\WindowsApps\winget.exe` | | NSIS warning: `install function not referenced` | `MUI_FINISHPAGE_RUN` define is missing | `MUI_FINISHPAGE_RUN_FUNCTION` still requires `MUI_FINISHPAGE_RUN` to be defined (controls checkbox visibility). Set it to `""` | | `dist/` has stale files after code changes | `build-dist-windows.sh` copies from source at build time | Re-run the full build script, or manually copy changed files into `dist/` | +| Audio visualizer returns all zeros / `numpy.fromstring` removed | `pip download` pulls numpy 2.x as a transitive dependency; `soundcard` uses `numpy.fromstring` which was removed in numpy 2.0 | Pin `numpy>=1.24.0,<2.0` in both `pyproject.toml` and `build-dist-windows.sh`. Also add a post-download cleanup step to remove numpy 2.x wheels pulled transitively: `for f in "$WHEEL_DIR"/numpy-2*; do rm "$f"; done` | +| Wrong numpy version despite pinning `<2.0` | The `pip download` fallback (no `--only-binary`) ignores platform constraints and pulls the host Python's numpy. Also, transitive deps from other packages (e.g. `soundcard`, `uvicorn`) can pull unconstrained numpy | Add `--python-version` to fallback command, use `--no-cache-dir`, and strip numpy 2.x wheels after the download loop | -### 11.4. Iterating on Installer Only +### 13.4. Iterating on Installer Only If you only changed `installer.nsi` (not app code), skip the full rebuild: @@ -572,11 +809,11 @@ If you only changed `installer.nsi` (not app code), skip the full rebuild: If you changed app code or dependencies, you must re-run `build-dist-windows.sh` first — the `dist/` directory is a snapshot and won't pick up source changes automatically. -## 12. Troubleshooting +## 14. Troubleshooting ### Running server blocks installation -See section 11.3. The `.onInit` function in section 6 shows how to detect a locked +See section 13.3. The `.onInit` function in section 6 shows how to detect a locked `python.exe` and prompt the user before proceeding. ### Release already exists for tag