# CI/CD for Python Apps on Gitea A reusable reference for building CI pipelines, release automation, and installer packaging for Python-based applications hosted on Gitea. Extracted from a production project using Gitea Actions (GitHub Actions-compatible). **This is a modular reference — pick only the sections you need.** Not every project requires all build targets. Common combinations: | Project type | Sections to use | |---|---| | Docker-only service | 1, 2 (docker job only), 7, 8 | | Desktop app (Windows + Linux) | 1, 2, 3, 4, 5, 6, 8 | | Desktop + Docker | All sections | | Python library/CLI | 1, 2 (release job only), 8 | ## Prerequisites - Gitea instance with Actions enabled - Runner(s) tagged `ubuntu-latest` (e.g., TrueNAS-hosted Gitea runners) - `DEPLOY_TOKEN` secret configured in the repository (Settings > Secrets). **Do NOT use `GITEA_TOKEN`** — it is a reserved name in Gitea and will be rejected by the UI and API. ## Pipeline Architecture Two workflows, triggered by different events: ```text push/PR to master ──► test.yml (lint + test) push tag v* ──► release.yml (build + release + Docker) ``` ## 1. Lint & Test Workflow **Trigger:** Push to `master` or pull request. ```yaml # .gitea/workflows/test.yml name: Lint & Test on: push: branches: [master] pull_request: branches: [master] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: '3.11' - name: Install system dependencies run: | sudo apt-get update sudo apt-get install -y --no-install-recommends - name: Install dependencies working-directory: server run: | pip install --upgrade pip pip install -e ".[dev]" - name: Lint working-directory: server run: ruff check src/ tests/ - name: Test working-directory: server run: pytest --tb=short -q ``` **Key points:** - Keep the test job fast — lint first, then test - Use `ruff` for linting (fast, replaces flake8 + isort + pyupgrade) - Configure pytest in `pyproject.toml` with coverage: ```toml [tool.pytest.ini_options] testpaths = ["tests"] asyncio_mode = "auto" addopts = "-v --cov=your_package --cov-report=html --cov-report=term" ``` ## 2. Release Workflow **Trigger:** Push a tag matching `v*` (e.g., `v0.2.0`, `v0.2.0-alpha.1`). ### 2.1. Create Release Job Creates a Gitea release via REST API with a formatted description. ```yaml create-release: runs-on: ubuntu-latest outputs: release_id: ${{ steps.create.outputs.release_id }} steps: - name: Create Gitea release id: create env: DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }} run: | TAG="${{ gitea.ref_name }}" BASE_URL="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}" # Detect pre-release IS_PRE="false" if echo "$TAG" | grep -qE '(alpha|beta|rc)'; then IS_PRE="true" fi # Build release body (use Python to avoid YAML escaping issues) BODY_JSON=$(python3 -c " import json, textwrap tag = '$TAG' body = f'''## Downloads | Platform | File | |----------|------| | Windows (installer) | \`App-{tag}-setup.exe\` | | Windows (portable) | \`App-{tag}-win-x64.zip\` | | Linux | \`App-{tag}-linux-x64.tar.gz\` | ''' print(json.dumps(textwrap.dedent(body).strip())) ") RELEASE=$(curl -s -X POST "\$BASE_URL/releases" \ -H "Authorization: token \$DEPLOY_TOKEN" \ -H "Content-Type: application/json" \ -d "{ \"tag_name\": \"\$TAG\", \"name\": \"App \$TAG\", \"body\": \$BODY_JSON, \"draft\": false, \"prerelease\": \$IS_PRE }") # Fallback: if release already exists for this tag, fetch it instead RELEASE_ID=$(echo "\$RELEASE" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])" 2>/dev/null) if [ -z "\$RELEASE_ID" ]; then echo "::warning::Release already exists for tag \$TAG — reusing existing release" RELEASE=$(curl -s "\$BASE_URL/releases/tags/\$TAG" \ -H "Authorization: token \$DEPLOY_TOKEN") RELEASE_ID=$(echo "\$RELEASE" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])") fi echo "release_id=\$RELEASE_ID" >> "\$GITHUB_OUTPUT" ``` **Key points:** - Gitea uses `${{ gitea.ref_name }}` instead of GitHub's `${{ github.ref_name }}` - Use `${{ gitea.server_url }}` and `${{ gitea.repository }}` for API URLs - Output vars use `$GITHUB_OUTPUT` (Gitea Actions is compatible with GitHub Actions syntax) - Use Python for body templating to avoid shell/YAML escaping hell ### 2.2. Build Jobs (Parallel) All build jobs run in parallel, gated on `create-release`: ```yaml build-windows: needs: create-release runs-on: ubuntu-latest # ... build-linux: needs: create-release runs-on: ubuntu-latest # ... build-docker: needs: create-release runs-on: ubuntu-latest # ... ``` ### 2.3. Upload Assets to Gitea Release **IMPORTANT:** Gitea does not reject duplicate asset names — it silently appends duplicates. When re-triggering a failed release workflow (where `create-release` reuses an existing release), build jobs will upload a second copy of every artifact. Always **delete existing assets with the same name before uploading**. ```yaml - name: Attach assets to release env: DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }} run: | RELEASE_ID="${{ needs.create-release.outputs.release_id }}" BASE_URL="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}" upload_asset() { local FILE="$1" local NAME=$(basename "$FILE") # Delete existing asset with the same name (prevents duplicates on re-run) EXISTING_ID=$(curl -s "$BASE_URL/releases/$RELEASE_ID/assets" \ -H "Authorization: token $DEPLOY_TOKEN" \ | python3 -c "import sys,json; assets=json.load(sys.stdin); print(next((str(a['id']) for a in assets if a['name']=='$NAME'),''))" 2>/dev/null) if [ -n "$EXISTING_ID" ]; then curl -s -X DELETE "$BASE_URL/releases/$RELEASE_ID/assets/$EXISTING_ID" \ -H "Authorization: token $DEPLOY_TOKEN" echo "Replaced existing asset: $NAME" fi curl -s -X POST \ "$BASE_URL/releases/$RELEASE_ID/assets?name=$NAME" \ -H "Authorization: token $DEPLOY_TOKEN" \ -H "Content-Type: application/octet-stream" \ --data-binary "@$FILE" echo "Uploaded: $NAME" } FILE=$(ls build/App-*.zip | head -1) [ -f "$FILE" ] && upload_asset "$FILE" ``` ## 3. Version Detection Pattern A robust fallback chain for detecting the version in build scripts: ```bash # 1. CLI argument VERSION="${1:-}" # 2. Exact git tag if [ -z "$VERSION" ]; then VERSION=$(git describe --tags --exact-match 2>/dev/null || true) fi # 3. CI environment variable (Gitea or GitHub) if [ -z "$VERSION" ]; then VERSION="${GITEA_REF_NAME:-${GITHUB_REF_NAME:-}}" fi # 4. Hardcoded in source if [ -z "$VERSION" ]; then VERSION=$(grep -oP '__version__\s*=\s*"\K[^"]+' \ src/your_package/__init__.py 2>/dev/null || echo "0.0.0") fi VERSION_CLEAN="${VERSION#v}" # Strip leading 'v' ``` ## 4. Cross-Building Windows from Linux No Windows runner needed. Download Windows embedded Python and win_amd64 wheels on Linux. ### 4.1. Embedded Python Setup ```bash # Download Windows embedded Python PYTHON_VERSION="3.11.9" curl -sL "https://www.python.org/ftp/python/${PYTHON_VERSION}/python-${PYTHON_VERSION}-embed-amd64.zip" \ -o python-embed.zip unzip -qo python-embed.zip -d dist/python # Patch ._pth to enable site-packages PTH_FILE=$(ls dist/python/python*._pth | head -1) sed -i 's/^#\s*import site/import site/' "$PTH_FILE" echo 'Lib\site-packages' >> "$PTH_FILE" echo '../app/src' >> "$PTH_FILE" # App source on PYTHONPATH ``` **Why embedded Python?** It's a minimal, standalone Python distribution (~15 MB) without an installer. Perfect for bundling into portable apps. ### 4.2. Cross-Platform Wheel Download ```bash WHEEL_DIR="build/win-wheels" SITE_PACKAGES="dist/python/Lib/site-packages" mkdir -p "$WHEEL_DIR" "$SITE_PACKAGES" # Download win_amd64 wheels (fallback to source for pure Python) for dep in "fastapi>=0.115.0" "uvicorn>=0.32.0" "numpy>=2.1.3"; do pip download --quiet --dest "$WHEEL_DIR" \ --platform win_amd64 --python-version "3.11" \ --implementation cp --only-binary :all: \ "$dep" 2>/dev/null || \ pip download --quiet --dest "$WHEEL_DIR" "$dep" done # Install wheels into site-packages (just unzip — .whl is a zip) for whl in "$WHEEL_DIR"/*.whl; do unzip -qo "$whl" -d "$SITE_PACKAGES" done ``` ### 4.3. Size Optimization Strip unnecessary files from site-packages to reduce archive size: ```bash # Generic cleanup find "$SITE_PACKAGES" -type d -name __pycache__ -exec rm -rf {} + find "$SITE_PACKAGES" -type d -name tests -exec rm -rf {} + find "$SITE_PACKAGES" -type d -name "*.dist-info" -exec rm -rf {} + find "$SITE_PACKAGES" -name "*.pyi" -delete # Remove build-time-only packages rm -rf "$SITE_PACKAGES"/{pip,setuptools,pkg_resources,_distutils_hack}* # OpenCV: remove ffmpeg DLL (~28 MB), Haar cascades, dev files rm -f "$SITE_PACKAGES"/cv2/opencv_videoio_ffmpeg*.dll rm -rf "$SITE_PACKAGES"/cv2/{data,gapi,misc,utils,typing_stubs,typing} # NumPy: remove unused submodules (only keep core, fft, random) for mod in polynomial linalg ma lib distutils f2py typing _pyinstaller; do rm -rf "$SITE_PACKAGES/numpy/$mod" done # zeroconf: remove service type database (~1-2 MB) rm -rf "$SITE_PACKAGES/zeroconf/_services" # Strip debug symbols from native extensions find "$SITE_PACKAGES" -name "*.pyd" -exec strip --strip-debug {} \; # Remove .py source files (keep compiled .pyc only) find "$SITE_PACKAGES" -name "*.py" ! -name "__init__.py" -delete ``` **Tip:** If a library is only needed for one feature (e.g., Pillow for system tray icons), move it to an optional dependency group and strip unused plugins. Replace its core usage with a library already in the dependency tree (e.g., use cv2 for JPEG encoding instead of Pillow). ### 4.4. Bundling tkinter (Optional) Embedded Python doesn't include tkinter. Extract it from the official MSI packages: ```bash # Requires: apt install msitools MSI_BASE="https://www.python.org/ftp/python/${PYTHON_VERSION}/amd64" curl -sL "$MSI_BASE/tcltk.msi" -o tcltk.msi curl -sL "$MSI_BASE/lib.msi" -o lib.msi msiextract tcltk.msi # → _tkinter.pyd, tcl86t.dll, tk86t.dll, tcl8.6/, tk8.6/ msiextract lib.msi # → tkinter/ Python package # Copy to embedded Python directory cp _tkinter.pyd tcl86t.dll tk86t.dll dist/python/ cp -r tkinter dist/python/Lib/ cp -r tcl8.6 tk8.6 dist/python/ ``` ## 5. Linux Build (venv-based) ```bash # Create self-contained virtualenv python3 -m venv dist/venv source dist/venv/bin/activate pip install ".[camera,notifications]" # Remove the installed package (PYTHONPATH handles app code at runtime) rm -rf dist/venv/lib/python*/site-packages/your_package* # Launcher script cat > dist/run.sh << 'EOF' #!/usr/bin/env bash set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" export PYTHONPATH="$SCRIPT_DIR/app/src" source "$SCRIPT_DIR/venv/bin/activate" exec python -m your_package.main EOF chmod +x dist/run.sh # Package tar -czf "App-v${VERSION}-linux-x64.tar.gz" dist/ ``` ### 5.1. Systemd Service Script Include install/uninstall scripts for running as a service: ```bash # install-service.sh creates: # /etc/systemd/system/your-app.service # Type=simple, Restart=on-failure, RestartSec=5 # ExecStart=/path/to/run.sh # Then: systemctl daemon-reload && systemctl enable && systemctl start ``` ### 5.2. Shared Build Logic When building for both Windows and Linux, extract shared logic into a `build-common.sh` sourced by both platform scripts. This avoids duplicating version detection, site-packages cleanup, frontend builds, and app file copying. ```bash # build-common.sh — shared functions detect_version() { ... } # git tag → env var → pyproject.toml fallback clean_dist() { ... } # rm -rf + mkdir build_frontend() { ... } # npm ci && npm run build copy_app_files() { ... } # cp src/, config/, clean .map/__pycache__ # Parameterized by platform — extension suffix differs: # cleanup_site_packages # Windows: cleanup_site_packages "$SP" "pyd" "dll" # Linux: cleanup_site_packages "$SP" "so" "so" cleanup_site_packages() { local sp_dir="$1" ext_suffix="${2:-so}" lib_suffix="${3:-so}" # Generic: __pycache__, tests/, *.dist-info, *.pyi # NumPy: remove unused submodules (polynomial, linalg, ma, etc.) # OpenCV: remove ffmpeg, Haar cascades, dev files # Pillow: remove unused image format plugins # zeroconf: remove _services/ database # strip --strip-debug on *.$ext_suffix # Remove .py source (keep .pyc only) } ``` ```bash # build-dist-windows.sh / build-dist.sh source "$(dirname "$0")/build-common.sh" detect_version "${1:-}" clean_dist # ... platform-specific Python setup + dependency install ... cleanup_site_packages "$SITE_PACKAGES" "pyd" "dll" # or "so" "so" build_frontend copy_app_files # ... platform-specific launcher + packaging ... ``` **Key principle:** Both scripts run on Linux (Windows build is cross-compiled). Keep platform-specific code (embedded Python, ._pth patching, .bat launchers, systemd services) in the platform scripts. Keep size optimization, cleanup, and build steps that are identical in the common file. ## 6. NSIS Installer (Windows) Cross-compilable on Linux: `apt install nsis && makensis installer.nsi` Key design decisions: ```nsi ; User-scoped install (no admin required) InstallDir "$LOCALAPPDATA\${APPNAME}" RequestExecutionLevel user ; Optional: launch app after install (checkbox on finish page) ; IMPORTANT: Do NOT use MUI_FINISHPAGE_RUN with MUI_FINISHPAGE_RUN_PARAMETERS — ; NSIS Exec command chokes on the quoting. Use MUI_FINISHPAGE_RUN_FUNCTION instead: !define MUI_FINISHPAGE_RUN "" !define MUI_FINISHPAGE_RUN_TEXT "Launch ${APPNAME}" !define MUI_FINISHPAGE_RUN_FUNCTION LaunchApp Function LaunchApp ExecShell "open" "wscript.exe" '"$INSTDIR\scripts\start-hidden.vbs"' Sleep 2000 ExecShell "open" "http://localhost:8765/" ; Open Web UI after server starts FunctionEnd ; Detect running instance before install (file lock check) Function .onInit IfFileExists "$INSTDIR\python\python.exe" 0 done ClearErrors FileOpen $0 "$INSTDIR\python\python.exe" a IfErrors locked FileClose $0 Goto done locked: MessageBox MB_YESNOCANCEL|MB_ICONEXCLAMATION \ "${APPNAME} is currently running.$\n$\nYes = Stop and continue$\nNo = Continue anyway (may cause errors)$\nCancel = Abort" \ IDYES kill IDNO done Abort kill: nsExec::ExecToLog 'wmic process where "ExecutablePath like $\'%AppName%python%$\'" call terminate' Sleep 2000 done: FunctionEnd ; Sections Section "!Core (required)" SecCore ; App files + uninstaller Section "Desktop shortcut" SecDesktop ; Optional Section "Start with Windows" SecAutostart ; Optional — Startup folder shortcut ; Uninstall preserves user data ; RMDir /r $INSTDIR\python, $INSTDIR\app, $INSTDIR\logs ; But NOT $INSTDIR\data (user config) ``` ### Hidden Launcher (VBS) Bat files briefly flash a console window even with `@echo off`. To avoid this, use a VBS wrapper that all shortcuts and the finish page point to: ```vbs Set fso = CreateObject("Scripting.FileSystemObject") Set WshShell = CreateObject("WScript.Shell") scriptDir = fso.GetParentFolderName(WScript.ScriptFullName) appRoot = fso.GetParentFolderName(scriptDir) WshShell.CurrentDirectory = appRoot ' Use embedded Python if present (installed dist), otherwise system Python embeddedPython = appRoot & "\python\python.exe" If fso.FileExists(embeddedPython) Then WshShell.Run """" & embeddedPython & """ -m your_package.main", 0, False Else WshShell.Run "python -m your_package.main", 0, False End If ``` Place in `scripts/start-hidden.vbs` and bundle it in the build script. All NSIS shortcuts use: `"wscript.exe" '"$INSTDIR\scripts\start-hidden.vbs"'` **Important:** The embedded Python fallback is critical — the installed distribution doesn't have Python on PATH, so `python` alone won't work. The dev environment uses system Python, so the fallback handles both cases. **CI dependencies:** `sudo apt-get install -y nsis msitools zip` Build: `makensis -DVERSION="${VERSION}" installer.nsi` ## 7. Docker Build ### Multi-stage Dockerfile ```dockerfile # Stage 1: Build frontend FROM node:20-slim AS frontend WORKDIR /app COPY package.json package-lock.json ./ RUN npm ci COPY . . RUN npm run build # Stage 2: Runtime FROM python:3.11-slim RUN apt-get update && apt-get install -y --no-install-recommends \ && rm -rf /var/lib/apt/lists/* WORKDIR /app COPY pyproject.toml . RUN pip install --no-cache-dir ".[notifications]" COPY src/ src/ COPY --from=frontend /app/src/your_package/static/ src/your_package/static/ RUN useradd -r -u 1000 appuser USER appuser HEALTHCHECK --interval=30s --timeout=5s CMD curl -f http://localhost:8080/health || exit 1 EXPOSE 8080 CMD ["uvicorn", "your_package.main:app", "--host", "0.0.0.0", "--port", "8080"] ``` ### Docker in CI (Gitea Registry) ```yaml - name: Login to Gitea Container Registry id: docker-login continue-on-error: true # Graceful degradation if registry unavailable run: | echo "${{ secrets.DEPLOY_TOKEN }}" | docker login \ "$SERVER_HOST" -u "${{ gitea.actor }}" --password-stdin - name: Build and tag if: steps.docker-login.outcome == 'success' run: | docker build -t "$REGISTRY:$TAG" -t "$REGISTRY:$VERSION" ./server # Tag as 'latest' only for stable releases if ! echo "$TAG" | grep -qE '(alpha|beta|rc)'; then docker tag "$REGISTRY:$TAG" "$REGISTRY:latest" fi - name: Push if: steps.docker-login.outcome == 'success' run: docker push "$REGISTRY" --all-tags - name: Trigger Portainer redeploy if: steps.docker-login.outcome == 'success' continue-on-error: true # Optional — webhook may not be configured run: | if [ -n "${{ secrets.DOCKER_REDEPLOY_WEBHOOK_URL }}" ]; then echo "Triggering Portainer redeploy..." curl -sf -X POST "${{ secrets.DOCKER_REDEPLOY_WEBHOOK_URL }}" \ --max-time 30 || echo "::warning::Portainer webhook failed" else echo "DOCKER_REDEPLOY_WEBHOOK_URL not set — skipping auto-deploy" fi ``` **Registry URL pattern:** `{gitea-host}/{owner}/{repo}:{tag}` ### Auto-Deploy via Portainer Webhook (Optional) Portainer can automatically redeploy a stack/service when a new image is pushed. Each stack has its own unique webhook URL generated in the Portainer UI. **Setup:** 1. In Portainer, open your stack → **Webhooks** → enable and copy the URL 2. In Gitea, go to repo **Settings → Secrets** → add `DOCKER_REDEPLOY_WEBHOOK_URL` 3. The CI step above calls the webhook after `docker push` — Portainer pulls the new image and recreates the container **Notes:** - The webhook URL itself acts as authentication — no extra token needed - The step uses `continue-on-error: true` so missing webhooks don't fail the build - Each service/stack gets its own webhook — one secret per repo - If `DOCKER_REDEPLOY_WEBHOOK_URL` is not set, the step is silently skipped ### Docker Network on TrueNAS If Docker builds fail with `route for the gateway 0.0.0.1 could not be found: network is unreachable`, the Docker address pool is misconfigured. In TrueNAS 25.10+: 1. Go to **Apps → Configuration** (gear icon) 2. Change Address Pool base from `0.0.0.0` to `172.17.0.0` (keep `/12`, size `24`) 3. Save — Docker restarts automatically Verify: `sudo docker run --rm alpine ping -c 1 google.com` ### Package-Repository Linking Gitea does not auto-link container packages to repositories on first push, even with the `org.opencontainers.image.source` label in the Dockerfile. You must manually link the package once: 1. Go to your profile → **Packages** → click the package 2. Go to **Settings** → link it to the repository Subsequent pushes to the same package name stay linked automatically. ## 8. Release Versioning | Tag format | Example | Pre-release? | Docker `:latest`? | | --- | --- | --- | --- | | `v{major}.{minor}.{patch}` | `v0.2.0` | No | Yes | | `v{major}.{minor}.{patch}-alpha.{n}` | `v0.2.0-alpha.1` | Yes | No | | `v{major}.{minor}.{patch}-beta.{n}` | `v0.2.0-beta.2` | Yes | No | | `v{major}.{minor}.{patch}-rc.{n}` | `v0.2.0-rc.1` | Yes | No | **Release process:** ```bash # Stable release git tag v0.2.0 git push origin v0.2.0 # Pre-release git tag v0.2.0-alpha.1 git push origin v0.2.0-alpha.1 ``` ### 8.1. Release Notes from File Instead of hardcoding release notes in the workflow, keep a `RELEASE_NOTES.md` in the repo root. The CI fetches only that file (via sparse-checkout for speed) and prepends its content to the auto-generated Downloads section. **Workflow:** 1. Before tagging, write `RELEASE_NOTES.md` with changes for this release 2. Commit, tag, push — CI picks up the file automatically 3. Release body = your notes + auto-generated download/Docker instructions **CI implementation:** ```yaml - name: Fetch RELEASE_NOTES.md only uses: actions/checkout@v4 with: sparse-checkout: RELEASE_NOTES.md sparse-checkout-cone-mode: false - name: Create release run: | if [ -f RELEASE_NOTES.md ]; then export RELEASE_NOTES=$(cat RELEASE_NOTES.md) echo "Found RELEASE_NOTES.md" else export RELEASE_NOTES="" echo "No RELEASE_NOTES.md found" fi # Python reads RELEASE_NOTES from env, prepends to Downloads section BODY_JSON=$(python3 -c " import json, os, textwrap notes = os.environ.get('RELEASE_NOTES', '') sections = [] if notes.strip(): sections.append(notes.strip()) sections.append('## Downloads\n...') # auto-generated part print(json.dumps('\n\n'.join(sections))) ") # Create release with combined body curl -s -X POST ... -d "{\"body\": $BODY_JSON, ...}" ``` > **Why sparse-checkout?** The `create-release` job only needs one file. A sparse-checkout skips downloading the full repo history and working tree, making the step significantly faster — especially in repos with large static assets or many files. If `RELEASE_NOTES.md` is absent, the release uses only the auto-generated Downloads section — no manual notes needed for quick pre-releases. ## 9. Gitea vs GitHub Actions Differences | Feature | GitHub Actions | Gitea Actions | | --- | --- | --- | | Context prefix | `github.*` | `gitea.*` | | Ref name | `${{ github.ref_name }}` | `${{ gitea.ref_name }}` | | Server URL | `${{ github.server_url }}` | `${{ gitea.server_url }}` | | Repository | `${{ github.repository }}` | `${{ gitea.repository }}` | | Actor | `${{ github.actor }}` | `${{ gitea.actor }}` | | SHA | `${{ github.sha }}` | `${{ gitea.sha }}` | | Output vars | `$GITHUB_OUTPUT` | `$GITHUB_OUTPUT` (same) | | Secrets | `${{ secrets.NAME }}` | `${{ secrets.NAME }}` (same) | | Docker Buildx | Available | May not work (runner networking) | ## 10. Version Management — Single Source of Truth The version must be consistent across the UI, package metadata, installer filename, and Docker label. The simplest approach: **`pyproject.toml` is the single source of truth**, and everything else derives from it. ### 10.1. Runtime Version via `importlib.metadata` Instead of hardcoding `__version__` in `__init__.py`, read it from installed package metadata: ```python # src/your_package/__init__.py from importlib.metadata import version, PackageNotFoundError try: __version__ = version("your-package-name") except PackageNotFoundError: __version__ = "0.0.0-dev" # Running from source without pip install ``` This reads whatever version `pip install` baked in. No manual syncing needed. ### 10.2. CI Stamps the Tag into `pyproject.toml` Build scripts resolve the version (see section 3), then stamp it before building: ```bash VERSION_CLEAN="${VERSION#v}" # "0.3.0" from "v0.3.0" # Stamp into pyproject.toml so pip install bakes the correct version sed -i "s/^version = .*/version = \"${VERSION_CLEAN}\"/" server/pyproject.toml ``` This happens in both `build-dist.sh` and `build-dist-windows.sh`, before `pip install` or wheel extraction. ### 10.3. Docker Version Stamping Use a build arg so the version is stamped during the Docker build: ```dockerfile ARG APP_VERSION=0.0.0 # Stamp before pip install (which bakes version into metadata) RUN sed -i "s/^version = .*/version = \"${APP_VERSION}\"/" pyproject.toml \ && pip install --no-cache-dir ".[notifications]" ``` In CI: ```yaml docker build --build-arg APP_VERSION="${VERSION}" ./server ``` The `--label org.opencontainers.image.version` flag can also be set dynamically: ```yaml docker build \ --build-arg APP_VERSION="${VERSION}" \ --label "org.opencontainers.image.version=${VERSION}" \ ./server ``` ### 10.4. Version Fallback Chain (Updated) The build script fallback (section 3) should read from `pyproject.toml` instead of `__init__.py`: ```bash # Fallback 4: Read from pyproject.toml (single source of truth) if [ -z "$VERSION" ]; then VERSION=$(grep -oP '^version\s*=\s*"\K[^"]+' server/pyproject.toml 2>/dev/null || echo "0.0.0") fi ``` ### 10.5. Version Flow Summary ```text git tag v0.3.0 → CI reads tag → sed stamps pyproject.toml with "0.3.0" → pip install bakes "0.3.0" into package metadata → importlib.metadata.version() reads "0.3.0" at runtime → UI shows "v0.3.0", update checker compares against it ``` ## 11. In-App Auto-Update For self-hosted apps distributed as installers or portable ZIPs, you can implement in-app update checking and one-click updates. The design separates the platform-specific hosting API (Gitea, GitHub) from the update logic via an abstraction layer. ### 11.1. Architecture ```text UpdateService (background task) ├── ReleaseProvider (abstract) │ ├── GiteaReleaseProvider — GET /api/v1/repos/{repo}/releases │ └── GitHubReleaseProvider — GET api.github.com/repos/{owner}/{repo}/releases ├── VersionCheck — normalize semver tags, compare via packaging.version └── InstallType — detect installer vs portable vs Docker vs dev ``` ### 11.2. Release Provider Abstraction The abstract interface has two methods: ```python class ReleaseProvider(ABC): @abstractmethod async def get_releases(self, limit: int = 10) -> list[ReleaseInfo]: ... @abstractmethod def get_releases_page_url(self) -> str: ... ``` `ReleaseInfo` is a frozen dataclass with `tag`, `version`, `name`, `body`, `prerelease`, `published_at`, and `assets` (tuple of `AssetInfo` with `name`, `size`, `download_url`). **Gitea implementation** hits `GET {base_url}/api/v1/repos/{repo}/releases?limit=N`. **GitHub** would hit `api.github.com/repos/{owner}/{repo}/releases`. The rest of the system only depends on the abstract interface — swapping providers is a config change. ### 11.3. Version Comparison Gitea/GitHub tags like `v0.3.0-alpha.1` must be normalized to PEP 440 for comparison: ```python import re from packaging.version import Version _PRE_PATTERN = re.compile(r"^(\d+\.\d+\.\d+)[-.]?(alpha|beta|rc)[.-]?(\d+)$", re.IGNORECASE) _PRE_MAP = {"alpha": "a", "beta": "b", "rc": "rc"} def normalize_version(raw: str) -> Version: cleaned = raw.lstrip("v").strip() m = _PRE_PATTERN.match(cleaned) if m: base, pre_label, pre_num = m.group(1), m.group(2).lower(), m.group(3) cleaned = f"{base}{_PRE_MAP[pre_label]}{pre_num}" return Version(cleaned) ``` Examples: `v0.3.0-alpha.1` → `0.3.0a1`, `v0.3.0-rc.3` → `0.3.0rc3`, `v1.0.0` → `1.0.0`. ### 11.4. Install Type Detection Detect at startup by checking filesystem markers: | Marker | Install type | Auto-update strategy | |--------|-------------|---------------------| | `uninstall.exe` in CWD | `installer` | Download `-setup.exe`, run `/S /D=` (silent NSIS reinstall) | | `python/python.exe` in CWD (no uninstaller) | `portable` (Windows) | Download ZIP, extract, swap `app/` + `python/` via detached bat script | | `venv/` + `run.sh` in CWD | `portable` (Linux) | Download tarball, extract, swap `app/` + `venv/` via detached shell script | | `/.dockerenv` exists | `docker` | No auto-update — show "pull new image" instructions | | None of above | `dev` | No auto-update — link to releases page | ### 11.5. Update Service (Background Task) Follows the same pattern as other background services (e.g., auto-backup): - **Settings** persisted in DB: `enabled`, `check_interval_hours`, `include_prerelease` - **30s startup delay** to avoid slowing boot - **60s debounce** on manual checks to prevent API spam - **State machine**: `IDLE → CHECKING → UPDATE_AVAILABLE → DOWNLOADING → APPLYING` - **WebSocket events**: `update_available` (triggers banner), `update_download_progress` (progress bar) - **Dismissed version** persisted in DB (survives restarts) ### 11.6. Update Application Strategies **NSIS installer (Windows):** ```python # Launch installer silently, detached from server process subprocess.Popen( [str(exe_path), "/S", f"/D={install_dir}"], creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, ) # Shut down server — installer waits for python.exe to unlock, then installs request_shutdown() ``` The NSIS installer's `.onInit` detects the locked `python.exe` and handles it. After install, the VBS launcher or startup shortcut restarts the app. **Portable ZIP (Windows):** Can't replace files while the server is running (DLL locks). Solution: write a `.bat` script that runs after shutdown: ```batch @echo off timeout /t 5 /nobreak >nul rmdir /s /q "C:\path\app" 2>nul move /y "C:\path\staging\LedGrab\app" "C:\path\app" rmdir /s /q "C:\path\python" 2>nul move /y "C:\path\staging\LedGrab\python" "C:\path\python" rmdir /s /q "C:\path\staging" 2>nul start "" wscript.exe "C:\path\scripts\start-hidden.vbs" del /f /q "%~f0" ``` Launch it detached via `subprocess.Popen`, then shut down the server. **Portable tarball (Linux):** Same pattern but with a shell script: ```bash #!/bin/bash sleep 3 rm -rf "$APP_ROOT/app" && mv "$STAGING/LedGrab/app" "$APP_ROOT/app" rm -rf "$APP_ROOT/venv" && mv "$STAGING/LedGrab/venv" "$APP_ROOT/venv" rm -rf "$STAGING" cd "$APP_ROOT" && exec ./run.sh ``` ### 11.7. API Endpoints | Method | Path | Purpose | |--------|------|---------| | `GET` | `/system/update/status` | Current state, available version, install type, progress | | `POST` | `/system/update/check` | Trigger immediate check | | `POST` | `/system/update/dismiss` | Dismiss notification for a version | | `POST` | `/system/update/apply` | Download + install + restart | | `GET/PUT` | `/system/update/settings` | Auto-check settings | ### 11.8. Frontend Integration - **Version badge**: Add CSS class `has-update` to the version pill in the header — changes color + pulse animation, clickable to open settings - **Banner**: Persistent dismissible bar below header with icon buttons (external link to releases, download to apply, X to dismiss). Stored in `localStorage` so dismissed state survives page refresh - **Settings tab**: "Updates" tab showing current version, install type, status, check button, "Update Now" button (visible only when `can_auto_update` is true), download progress bar, release notes preview, auto-check toggle + interval + channel (stable/pre-release) - **WebSocket events**: `update_available` triggers banner + badge, `update_download_progress` updates progress bar, `server_restarting` shows reconnect overlay ### 11.9. Key Design Decisions 1. **Provider abstraction** — Gitea today, GitHub/GitLab tomorrow. One interface, swap implementations via config. 2. **No data directory changes** — Updates replace `app/` and `python/` (or `venv/`) but **never touch `data/`**. User configuration survives every update. 3. **Detached post-update scripts** — The server can't replace its own files while running. A detached script waits for shutdown, does the swap, then restarts. 4. **Silent NSIS reinstall** — NSIS `/S` flag runs the full installer flow without UI, preserving the `data/` directory because the uninstaller section is never triggered during an upgrade. 5. **Asset matching by filename pattern** — e.g., `*-setup.exe` for installer, `*-win-x64.zip` for portable. Platform + install type determines the pattern. 6. **Pre-release channel** — Users opt in to alpha/beta/RC builds via settings. Default is stable only. ## 12. Checklist for New Projects - [ ] Create `.gitea/workflows/test.yml` — lint + test on push/PR - [ ] Create `.gitea/workflows/release.yml` — build + release on `v*` tag - [ ] Add `DEPLOY_TOKEN` secret to repository - [ ] Set up version detection in build scripts (tag → env → source) - [ ] Set up `importlib.metadata` version in `__init__.py` (section 10.1) - [ ] Add `sed` version stamp step in build scripts (section 10.2) - [ ] Create `build-dist.sh` for Linux (venv + tarball) - [ ] Create `build-dist-windows.sh` for Windows (embedded Python + ZIP) - [ ] Create `installer.nsi` for Windows installer (optional) - [ ] Create `Dockerfile` for Docker builds (optional) - [ ] Configure `pyproject.toml` with `[tool.ruff]` and `[tool.pytest]` - [ ] Set up `.pre-commit-config.yaml` with ruff + black - [ ] Add in-app update checker if self-hosted (section 11) ## 13. Local Build Testing (Windows) The CI builds on Linux, but you can build locally on Windows for faster iteration. ### 13.1. Prerequisites Install NSIS (for installer builds): ```powershell # If winget is not on PATH, use the full path: & "$env:LOCALAPPDATA\Microsoft\WindowsApps\winget.exe" install NSIS.NSIS # Installs to: C:\Program Files (x86)\NSIS\makensis.exe ``` ### 13.2. Build Steps ```bash # 1. Build frontend npm ci && npm run build # 2. Build Windows distribution (from Git Bash) bash build-dist-windows.sh v1.0.0 # 3. Build NSIS installer "/c/Program Files (x86)/NSIS/makensis.exe" -DVERSION="1.0.0" installer.nsi # Output: build/MediaServer-v1.0.0-setup.exe ``` ### 13.3. Common Issues | Issue | Cause | Fix | |-------|-------|-----| | `zip: command not found` | Git Bash doesn't include `zip` | Harmless — only affects the portable ZIP, not the installer. Install `zip` via MSYS2 if needed | | `Exec expects 1 parameters, got 2` | `MUI_FINISHPAGE_RUN_PARAMETERS` quoting breaks NSIS `Exec` | Use `MUI_FINISHPAGE_RUN_FUNCTION` instead (see section 6) | | `Error opening file for writing: ...python\\_asyncio.pyd` | Server is running and has DLLs locked | Stop the server before installing. Add `.onInit` file-lock check (see section 6) | | App doesn't start after install (Launch checkbox) | VBS uses `python` but embedded Python isn't on PATH | Use embedded Python fallback in VBS (see Hidden Launcher section) | | `winget` not recognized | `winget.exe` exists but isn't on shell PATH | Use full path: `$env:LOCALAPPDATA\Microsoft\WindowsApps\winget.exe` | | NSIS warning: `install function not referenced` | `MUI_FINISHPAGE_RUN` define is missing | `MUI_FINISHPAGE_RUN_FUNCTION` still requires `MUI_FINISHPAGE_RUN` to be defined (controls checkbox visibility). Set it to `""` | | `dist/` has stale files after code changes | `build-dist-windows.sh` copies from source at build time | Re-run the full build script, or manually copy changed files into `dist/` | | Audio visualizer returns all zeros / `numpy.fromstring` removed | `pip download` pulls numpy 2.x as a transitive dependency; `soundcard` uses `numpy.fromstring` which was removed in numpy 2.0 | Pin `numpy>=1.24.0,<2.0` in both `pyproject.toml` and `build-dist-windows.sh`. Also add a post-download cleanup step to remove numpy 2.x wheels pulled transitively: `for f in "$WHEEL_DIR"/numpy-2*; do rm "$f"; done` | | Wrong numpy version despite pinning `<2.0` | The `pip download` fallback (no `--only-binary`) ignores platform constraints and pulls the host Python's numpy. Also, transitive deps from other packages (e.g. `soundcard`, `uvicorn`) can pull unconstrained numpy | Add `--python-version` to fallback command, use `--no-cache-dir`, and strip numpy 2.x wheels after the download loop | ### 13.4. Iterating on Installer Only If you only changed `installer.nsi` (not app code), skip the full rebuild: ```bash # Just rebuild the installer using existing dist/ "/c/Program Files (x86)/NSIS/makensis.exe" -DVERSION="1.0.0" installer.nsi ``` If you changed app code or dependencies, you must re-run `build-dist-windows.sh` first — the `dist/` directory is a snapshot and won't pick up source changes automatically. ## 14. Troubleshooting ### Running server blocks installation See section 13.3. The `.onInit` function in section 6 shows how to detect a locked `python.exe` and prompt the user before proceeding. ### Release already exists for tag If the `create-release` job fails with `KeyError: 'id'`, the Gitea API returned an error because a release already exists for that tag (e.g., from a previous failed run). **Prevention:** The fallback logic in section 2.1 handles this automatically — if creation fails, it fetches the existing release by tag and reuses its ID. A `::warning::` annotation is emitted in the workflow log. **Re-triggering a failed release workflow:** ```bash # Option A: Delete and re-push the same tag git push origin :refs/tags/v0.1.0-alpha.2 # delete remote tag # Delete the release in Gitea UI or via API git tag -f v0.1.0-alpha.2 # recreate local tag git push origin v0.1.0-alpha.2 # push again # Option B: Just bump the version (simpler) git tag v0.1.0-alpha.3 git push origin v0.1.0-alpha.3 ```