claude-code-facts/gitea-python-ci-cd.md

# CI/CD for Python Apps on Gitea

A reusable reference for building CI pipelines, release automation, and installer packaging for Python-based applications hosted on Gitea. Extracted from a production project using Gitea Actions (GitHub Actions-compatible).

**This is a modular reference — pick only the sections you need.** Not every project requires all build targets. Common combinations:

| Project type | Sections to use |
| --- | --- |
| Docker-only service | 1, 2 (docker job only), 7, 8 |
| Desktop app (Windows + Linux) | 1, 2, 3, 4, 5, 6, 8 |
| Desktop + Docker | All sections |
| Python library/CLI | 1, 2 (release job only), 8 |

## Prerequisites

- Gitea instance with Actions enabled
- Runner(s) tagged `ubuntu-latest` (e.g., TrueNAS-hosted Gitea runners)
- `DEPLOY_TOKEN` secret configured in the repository (Settings > Secrets). **Do NOT use `GITEA_TOKEN`** — it is a reserved name in Gitea and will be rejected by the UI and API.

## Pipeline Architecture

Two workflows, triggered by different events:

```text
push/PR to master  ──►  test.yml    (lint + test)
push tag v*        ──►  release.yml (build + release + Docker)
manual trigger     ──►  build.yml   (build artifacts only, no release)
```

## 1. Lint & Test Workflow

**Trigger:** Push to `master` or pull request.

```yaml
# .gitea/workflows/test.yml
name: Lint & Test

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install system dependencies
        run: |
          sudo apt-get update
          sudo apt-get install -y --no-install-recommends <your-system-deps>

      - name: Install dependencies
        working-directory: server
        run: |
          pip install --upgrade pip
          pip install -e ".[dev]"

      - name: Lint
        working-directory: server
        run: ruff check src/ tests/

      - name: Test
        working-directory: server
        run: pytest --tb=short -q
```

**Key points:**

- Keep the test job fast — lint first, then test
- Use `ruff` for linting (fast, replaces flake8 + isort + pyupgrade)
- Configure pytest in `pyproject.toml` with coverage:

```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"
addopts = "-v --cov=your_package --cov-report=html --cov-report=term"
```

## 2. Release Workflow

**Trigger:** Push a tag matching `v*` (e.g., `v0.2.0`, `v0.2.0-alpha.1`).

### 2.1. Create Release Job

Creates a Gitea release via REST API with a formatted description.

```yaml
create-release:
  runs-on: ubuntu-latest
  outputs:
    release_id: ${{ steps.create.outputs.release_id }}
  steps:
    - name: Create Gitea release
      id: create
      env:
        DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
      run: |
        TAG="${{ gitea.ref_name }}"
        BASE_URL="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"

        # Detect pre-release
        IS_PRE="false"
        if echo "$TAG" | grep -qE '(alpha|beta|rc)'; then
          IS_PRE="true"
        fi

        # Build release body (use Python to avoid YAML escaping issues)
        BODY_JSON=$(python3 -c "
        import json, textwrap
        tag = '$TAG'
        body = f'''## Downloads
        | Platform | File |
        |----------|------|
        | Windows (installer) | \`App-{tag}-setup.exe\` |
        | Windows (portable) | \`App-{tag}-win-x64.zip\` |
        | Linux | \`App-{tag}-linux-x64.tar.gz\` |
        '''
        print(json.dumps(textwrap.dedent(body).strip()))
        ")

        RELEASE=$(curl -s -X POST "\$BASE_URL/releases" \
          -H "Authorization: token \$DEPLOY_TOKEN" \
          -H "Content-Type: application/json" \
          -d "{
            \"tag_name\": \"\$TAG\",
            \"name\": \"App \$TAG\",
            \"body\": \$BODY_JSON,
            \"draft\": false,
            \"prerelease\": \$IS_PRE
          }")

        # Fallback: if release already exists for this tag, fetch it instead
        RELEASE_ID=$(echo "\$RELEASE" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])" 2>/dev/null)
        if [ -z "\$RELEASE_ID" ]; then
          echo "::warning::Release already exists for tag \$TAG — reusing existing release"
          RELEASE=$(curl -s "\$BASE_URL/releases/tags/\$TAG" \
            -H "Authorization: token \$DEPLOY_TOKEN")
          RELEASE_ID=$(echo "\$RELEASE" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])")
        fi
        echo "release_id=\$RELEASE_ID" >> "\$GITHUB_OUTPUT"
```

**Key points:**

- Gitea uses `${{ gitea.ref_name }}` instead of GitHub's `${{ github.ref_name }}`
- Use `${{ gitea.server_url }}` and `${{ gitea.repository }}` for API URLs
- Output vars use `$GITHUB_OUTPUT` (Gitea Actions is compatible with GitHub Actions syntax)
- Use Python for body templating to avoid shell/YAML escaping hell

### 2.2. Build Jobs (Parallel)

All build jobs run in parallel, gated on `create-release`:

```yaml
build-windows:
  needs: create-release
  runs-on: ubuntu-latest
  # ...

build-linux:
  needs: create-release
  runs-on: ubuntu-latest
  # ...

build-docker:
  needs: create-release
  runs-on: ubuntu-latest
  # ...
```

### 2.3. Upload Assets to Gitea Release

**IMPORTANT:** Gitea does not reject duplicate asset names — it silently appends duplicates. When re-triggering a failed release workflow (where `create-release` reuses an existing release), build jobs will upload a second copy of every artifact. Always **delete existing assets with the same name before uploading**.

```yaml
- name: Attach assets to release
  env:
    DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
  run: |
    RELEASE_ID="${{ needs.create-release.outputs.release_id }}"
    BASE_URL="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"

    upload_asset() {
      local FILE="$1"
      local NAME=$(basename "$FILE")

      # Delete existing asset with the same name (prevents duplicates on re-run)
      EXISTING_ID=$(curl -s "$BASE_URL/releases/$RELEASE_ID/assets" \
        -H "Authorization: token $DEPLOY_TOKEN" \
        | python3 -c "import sys,json; assets=json.load(sys.stdin); print(next((str(a['id']) for a in assets if a['name']=='$NAME'),''))" 2>/dev/null)

      if [ -n "$EXISTING_ID" ]; then
        curl -s -X DELETE "$BASE_URL/releases/$RELEASE_ID/assets/$EXISTING_ID" \
          -H "Authorization: token $DEPLOY_TOKEN"
        echo "Replaced existing asset: $NAME"
      fi

      curl -s -X POST \
        "$BASE_URL/releases/$RELEASE_ID/assets?name=$NAME" \
        -H "Authorization: token $DEPLOY_TOKEN" \
        -H "Content-Type: application/octet-stream" \
        --data-binary "@$FILE"
      echo "Uploaded: $NAME"
    }

    FILE=$(ls build/App-*.zip | head -1)
    [ -f "$FILE" ] && upload_asset "$FILE"
```

## 3. Version Detection Pattern

A robust fallback chain for detecting the version in build scripts:

```bash
# 1. CLI argument
VERSION="${1:-}"

# 2. Exact git tag
if [ -z "$VERSION" ]; then
    VERSION=$(git describe --tags --exact-match 2>/dev/null || true)
fi

# 3. CI environment variable (Gitea or GitHub)
if [ -z "$VERSION" ]; then
    VERSION="${GITEA_REF_NAME:-${GITHUB_REF_NAME:-}}"
fi

# 4. Hardcoded in source
if [ -z "$VERSION" ]; then
    VERSION=$(grep -oP '__version__\s*=\s*"\K[^"]+' \
      src/your_package/__init__.py 2>/dev/null || echo "0.0.0")
fi

VERSION_CLEAN="${VERSION#v}"  # Strip leading 'v'

# Normalize non-PEP440 labels (e.g. "dev", "nightly", "snapshot") to a
# valid PEP440 dev release. Without this, pip/setuptools rejects
# pyproject.toml with: `project.version` must be pep440.
if ! [[ "$VERSION_CLEAN" =~ ^[0-9]+(\.[0-9]+)*((a|b|rc|\.dev|\.post)[0-9]+)*(\+[a-zA-Z0-9.]+)?$ ]]; then
    echo "  Warning: '$VERSION_CLEAN' is not PEP440-compliant, using 0.0.0.dev0"
    VERSION_CLEAN="0.0.0.dev0"
fi
```

> **PEP 440 gotcha:** if a build script stamps a bare label like `dev` or `nightly` into `pyproject.toml`, the next `pip install` fails with `configuration error: project.version must be pep440`. Section 10.2 stamps the resolved version into `pyproject.toml`, so the version *must* be PEP 440-compliant before stamping. Valid forms: `1.2.3`, `1.2.3a1`, `1.2.3rc2`, `1.2.3.dev0`, `1.2.3.post1`, optionally with `+local` suffix. Invalid: `dev`, `vdev`, `nightly`, `snapshot-2024`. Always normalize before stamping.

## 4. Cross-Building Windows from Linux

No Windows runner needed. Download Windows embedded Python and win_amd64 wheels on Linux.

### 4.1. Embedded Python Setup

```bash
# Download Windows embedded Python
PYTHON_VERSION="3.11.9"
curl -sL "https://www.python.org/ftp/python/${PYTHON_VERSION}/python-${PYTHON_VERSION}-embed-amd64.zip" \
  -o python-embed.zip
unzip -qo python-embed.zip -d dist/python

# Patch ._pth to enable site-packages
PTH_FILE=$(ls dist/python/python*._pth | head -1)
sed -i 's/^#\s*import site/import site/' "$PTH_FILE"
echo 'Lib\site-packages' >> "$PTH_FILE"
echo '../app/src' >> "$PTH_FILE"       # App source on PYTHONPATH
```

**Why embedded Python?** It's a minimal, standalone Python distribution (~15 MB) without an installer. Perfect for bundling into portable apps.

### 4.2. Cross-Platform Wheel Download

```bash
WHEEL_DIR="build/win-wheels"
SITE_PACKAGES="dist/python/Lib/site-packages"
mkdir -p "$WHEEL_DIR" "$SITE_PACKAGES"

# Download win_amd64 wheels (fallback to source for pure Python)
for dep in "fastapi>=0.115.0" "uvicorn>=0.32.0" "numpy>=2.1.3"; do
    pip download --quiet --dest "$WHEEL_DIR" \
        --platform win_amd64 --python-version "3.11" \
        --implementation cp --only-binary :all: \
        "$dep" 2>/dev/null || \
    pip download --quiet --dest "$WHEEL_DIR" "$dep"
done

# Install wheels into site-packages (just unzip — .whl is a zip)
for whl in "$WHEEL_DIR"/*.whl; do
    unzip -qo "$whl" -d "$SITE_PACKAGES"
done
```

### 4.3. Size Optimization

Strip unnecessary files from site-packages to reduce archive size:

```bash
# Generic cleanup
find "$SITE_PACKAGES" -type d -name __pycache__ -exec rm -rf {} +
find "$SITE_PACKAGES" -type d -name tests -exec rm -rf {} +
find "$SITE_PACKAGES" -type d -name "*.dist-info" -exec rm -rf {} +
find "$SITE_PACKAGES" -name "*.pyi" -delete

# Remove build-time-only packages
rm -rf "$SITE_PACKAGES"/{pip,setuptools,pkg_resources,_distutils_hack}*

# OpenCV: remove ffmpeg DLL (~28 MB), Haar cascades, dev files
rm -f "$SITE_PACKAGES"/cv2/opencv_videoio_ffmpeg*.dll
rm -rf "$SITE_PACKAGES"/cv2/{data,gapi,misc,utils,typing_stubs,typing}

# NumPy: remove unused submodules (only keep core, fft, random)
for mod in polynomial linalg ma lib distutils f2py typing _pyinstaller; do
    rm -rf "$SITE_PACKAGES/numpy/$mod"
done

# zeroconf: remove service type database (~1-2 MB)
rm -rf "$SITE_PACKAGES/zeroconf/_services"

# Strip debug symbols from native extensions
find "$SITE_PACKAGES" -name "*.pyd" -exec strip --strip-debug {} \;
```

> ⚠️ **Do NOT delete `.py` source files** without first running `python -m compileall`.
> A previous version of this guide recommended `find ... -name "*.py" ! -name "__init__.py" -delete`
> as a "30-40% size win". Without a prior `compileall` step, the dist ships with no
> `.py` and no `.pyc`, so every package's submodules become unimportable. The bug is
> latent on fresh installs (clean ImportError) but manifests as a confusing **version
> mismatch** on in-place upgrades — e.g. PIL `_imaging` extension vs `_version.py`
> mismatch — because the new install's missing files leave stale files from the old
> install in place. See Troubleshooting → "Pillow / package version mismatch on upgrade".
>
> If you really want the size win, do it correctly:
> ```bash
> python -m compileall -b -q "$SITE_PACKAGES"   # compile in-place (-b strips __pycache__/)
> find "$SITE_PACKAGES" -name "*.py" -delete    # then drop sources
> find "$SITE_PACKAGES" -type d -name __pycache__ -exec rm -rf {} +
> ```
> Note `-b` writes `foo.pyc` next to `foo.py` instead of `__pycache__/foo.cpython-311.pyc`,
> so the .pyc files aren't orphaned when you delete `__pycache__`.

**Tip:** If a library is only needed for one feature (e.g., Pillow for system tray icons), move it to an optional dependency group and strip unused plugins. Replace its core usage with a library already in the dependency tree (e.g., use cv2 for JPEG encoding instead of Pillow).

### 4.4. Bundling tkinter (Optional)

Embedded Python doesn't include tkinter. Extract it from the official MSI packages:

```bash
# Requires: apt install msitools
MSI_BASE="https://www.python.org/ftp/python/${PYTHON_VERSION}/amd64"
curl -sL "$MSI_BASE/tcltk.msi" -o tcltk.msi
curl -sL "$MSI_BASE/lib.msi" -o lib.msi

msiextract tcltk.msi   # → _tkinter.pyd, tcl86t.dll, tk86t.dll, tcl8.6/, tk8.6/
msiextract lib.msi      # → tkinter/ Python package

# Copy to embedded Python directory
cp _tkinter.pyd tcl86t.dll tk86t.dll dist/python/
cp -r tkinter dist/python/Lib/
cp -r tcl8.6 tk8.6 dist/python/
```

## 5. Linux Build (venv-based)

```bash
# Create self-contained virtualenv
python3 -m venv dist/venv
source dist/venv/bin/activate
pip install ".[camera,notifications]"

# Remove the installed package (PYTHONPATH handles app code at runtime)
rm -rf dist/venv/lib/python*/site-packages/your_package*

# Launcher script
cat > dist/run.sh << 'EOF'
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
export PYTHONPATH="$SCRIPT_DIR/app/src"
source "$SCRIPT_DIR/venv/bin/activate"
exec python -m your_package.main
EOF
chmod +x dist/run.sh

# Package
tar -czf "App-v${VERSION}-linux-x64.tar.gz" dist/
```

### 5.1. Systemd Service Script

Include install/uninstall scripts for running as a service:

```bash
# install-service.sh creates:
# /etc/systemd/system/your-app.service
#   Type=simple, Restart=on-failure, RestartSec=5
#   ExecStart=/path/to/run.sh
# Then: systemctl daemon-reload && systemctl enable && systemctl start
```

### 5.2. Shared Build Logic

When building for both Windows and Linux, extract shared logic into a `build-common.sh` sourced by both platform scripts. This avoids duplicating version detection, site-packages cleanup, frontend builds, and app file copying.

```bash
# build-common.sh — shared functions
detect_version() { ... }      # git tag → env var → pyproject.toml fallback
clean_dist() { ... }          # rm -rf + mkdir
build_frontend() { ... }     # npm ci && npm run build
copy_app_files() { ... }     # cp src/, config/, clean .map/__pycache__

# Parameterized by platform — extension suffix differs:
#   cleanup_site_packages <path> <ext_suffix> <lib_suffix>
#   Windows: cleanup_site_packages "$SP" "pyd" "dll"
#   Linux:   cleanup_site_packages "$SP" "so"  "so"
cleanup_site_packages() {
    local sp_dir="$1" ext_suffix="${2:-so}" lib_suffix="${3:-so}"
    # Generic: __pycache__, tests/, *.dist-info, *.pyi
    # NumPy: remove unused submodules (polynomial, linalg, ma, etc.)
    # OpenCV: remove ffmpeg, Haar cascades, dev files
    # Pillow: remove unused image format plugins
    # zeroconf: remove _services/ database
    # strip --strip-debug on *.$ext_suffix
    # Remove .py source (keep .pyc only)
}
```

```bash
# build-dist-windows.sh / build-dist.sh
source "$(dirname "$0")/build-common.sh"
detect_version "${1:-}"
clean_dist
# ... platform-specific Python setup + dependency install ...
cleanup_site_packages "$SITE_PACKAGES" "pyd" "dll"  # or "so" "so"
build_frontend
copy_app_files
# ... platform-specific launcher + packaging ...
```

**Key principle:** Both scripts run on Linux (Windows build is cross-compiled). Keep platform-specific code (embedded Python, ._pth patching, .bat launchers, systemd services) in the platform scripts. Keep size optimization, cleanup, and build steps that are identical in the common file.

## 6. NSIS Installer (Windows)

Cross-compilable on Linux: `apt install nsis && makensis installer.nsi`

Key design decisions:

```nsi
; User-scoped install (no admin required)
InstallDir "$LOCALAPPDATA\${APPNAME}"
RequestExecutionLevel user

; Optional: launch app after install (checkbox on finish page)
; IMPORTANT: Do NOT use MUI_FINISHPAGE_RUN with MUI_FINISHPAGE_RUN_PARAMETERS —
; NSIS Exec command chokes on the quoting. Use MUI_FINISHPAGE_RUN_FUNCTION instead:
!define MUI_FINISHPAGE_RUN ""
!define MUI_FINISHPAGE_RUN_TEXT "Launch ${APPNAME}"
!define MUI_FINISHPAGE_RUN_FUNCTION LaunchApp

Function LaunchApp
    ExecShell "open" "wscript.exe" '"$INSTDIR\scripts\start-hidden.vbs"'
    Sleep 2000
    ExecShell "open" "http://localhost:8765/"  ; Open Web UI after server starts
FunctionEnd

; Detect running instance before install (file lock check)
Function .onInit
    IfFileExists "$INSTDIR\python\python.exe" 0 done
    ClearErrors
    FileOpen $0 "$INSTDIR\python\python.exe" a
    IfErrors locked
    FileClose $0
    Goto done
    locked:
        MessageBox MB_YESNOCANCEL|MB_ICONEXCLAMATION \
            "${APPNAME} is currently running.$\n$\nYes = Stop and continue$\nNo = Continue anyway (may cause errors)$\nCancel = Abort" \
            IDYES kill IDNO done
        Abort
        kill:
            nsExec::ExecToLog 'wmic process where "ExecutablePath like $\'%AppName%python%$\'" call terminate'
            Sleep 2000
    done:
FunctionEnd

; Sections
Section "!Core (required)" SecCore        ; App files + uninstaller
    SetOutPath "$INSTDIR"

    ; CRITICAL: wipe previous payload BEFORE extracting. NSIS `File /r` is a
    ; merge, not a replace — it overwrites files that exist in the new payload
    ; but leaves behind any file that was in the old install but not the new
    ; one. On Python apps this produces a half-old/half-new site-packages
    ; (e.g. new PIL/__init__.py + old PIL/_version.py) which crashes at import.
    ; Preserve $INSTDIR\config.yaml (user data) by deleting only known payload dirs.
    RMDir /r "$INSTDIR\python"
    RMDir /r "$INSTDIR\app"
    RMDir /r "$INSTDIR\scripts"
    Delete "$INSTDIR\${EXENAME}"
    Delete "$INSTDIR\VERSION"
    Delete "$INSTDIR\config.example.yaml"

    File /r "dist\app\*.*"
SectionEnd

Section "Desktop shortcut" SecDesktop     ; Optional
Section "Start with Windows" SecAutostart ; Optional — Startup folder shortcut

; Uninstall preserves user data
; RMDir /r $INSTDIR\python, $INSTDIR\app, $INSTDIR\logs
; But NOT $INSTDIR\data (user config)
```

### Shortcut Icons

By default, shortcuts that launch via `wscript.exe` or `python.exe` inherit the Python/Windows Script Host icon in the taskbar. To use a custom icon, set the 4th parameter of `CreateShortcut` to your `.ico` file:

```nsi
; Set installer/uninstaller icons (shown during install wizard)
!define MUI_ICON "src\static\icons\icon.ico"
!define MUI_UNICON "src\static\icons\icon.ico"

; Shortcut with custom icon (4th param = icon path, 5th = icon index)
CreateShortcut "$DESKTOP\${APPNAME}.lnk" \
    "wscript.exe" '"$INSTDIR\scripts\start-hidden.vbs"' \
    "$INSTDIR\app\static\icons\icon.ico" 0
```

The `.ico` file must be bundled in the distribution (copied by the build script). The same icon path applies to Start Menu, Desktop, and Startup shortcuts.

### Hidden Launcher (VBS)

Bat files briefly flash a console window even with `@echo off`. To avoid this,
use a VBS wrapper that all shortcuts and the finish page point to:

```vbs
Set fso = CreateObject("Scripting.FileSystemObject")
Set WshShell = CreateObject("WScript.Shell")
scriptDir = fso.GetParentFolderName(WScript.ScriptFullName)
appRoot = fso.GetParentFolderName(scriptDir)
WshShell.CurrentDirectory = appRoot
' Use embedded Python if present (installed dist), otherwise system Python
embeddedPython = appRoot & "\python\python.exe"
If fso.FileExists(embeddedPython) Then
    WshShell.Run """" & embeddedPython & """ -m your_package.main", 0, False
Else
    WshShell.Run "python -m your_package.main", 0, False
End If
```

Place in `scripts/start-hidden.vbs` and bundle it in the build script.
All NSIS shortcuts use: `"wscript.exe" '"$INSTDIR\scripts\start-hidden.vbs"'`

**Important:** The embedded Python fallback is critical — the installed distribution
doesn't have Python on PATH, so `python` alone won't work. The dev environment uses
system Python, so the fallback handles both cases.

#### VBS launcher gotchas

Three separate things have to be right or the launcher fails silently (often with a misleading error):

1. **Pure ASCII only.** No em-dashes, smart quotes, or any non-ASCII character — not even in comments. `wscript.exe` parses the file as ANSI. If it hits UTF-8 bytes it aborts with:

    > Execution of the Windows Script Host failed. (Not enough memory resources are available to complete this operation.)

    That dialog has nothing to do with memory; it's an encoding parse failure. If you write the VBS with a tool that defaults to UTF-8, strip any non-ASCII before shipping.

2. **CRLF line endings.** LF-only files parse inconsistently across WSH versions. Run `unix2dos` (or `sed -i 's/$/\r/'`) on the VBS after writing it. Your build script should do this unconditionally — don't rely on whatever produced the file.

3. **Use `python.exe` with `WindowStyle=0`, not `pythonw.exe`.** `pythonw.exe` has null stdout/stderr handles. Any library that touches those at startup (uvicorn, structlog, warnings) can silently crash the process before it writes a single log line. `python.exe` with `WshShell.Run cmd, 0, False` gives you a real console attached to a hidden window — same visual result, real stream handles, reliable startup.

**Debugging tip:** if the shortcut "does nothing", run `file path\to\start-hidden.vbs` from Git Bash. It must report `ASCII text, with CRLF line terminators`. Any other encoding is a red flag. Also run the VBS through `cscript //nologo` from a terminal to see actual error output — wscript swallows errors by design.

**Build-script enforcement:**

```bash
# In build-dist-windows.sh, right after copying scripts/ into the dist:
cp server/scripts/start-hidden.vbs "$DIST_DIR/scripts/"
# Paranoid: strip any accidental non-ASCII and normalize line endings
LC_ALL=C sed -i 's/[^\x00-\x7F]//g' "$DIST_DIR/scripts/start-hidden.vbs"
unix2dos "$DIST_DIR/scripts/start-hidden.vbs" 2>/dev/null
```

**CI dependencies:** `sudo apt-get install -y nsis msitools zip`

Build: `makensis -DVERSION="${VERSION}" installer.nsi`

### 6.1. Code Signing the Installer

Sign the `.exe` after `makensis` but before uploading assets to avoid SmartScreen and browser download warnings. See [windows-code-signing.md](windows-code-signing.md) for signing options and CI integration examples.

### 6.2. Build Without Release (Manual Trigger)

A separate workflow that builds installers and artifacts on demand, without creating a Gitea release. Useful for testing builds before tagging, verifying installer changes, or grabbing a dev build.

```yaml
# .gitea/workflows/build.yml
name: Build Artifacts

on:
  workflow_dispatch:
    inputs:
      version:
        description: 'Version label (e.g. dev, 0.3.0-test)'
        required: false
        default: 'dev'

jobs:
  build-windows:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build Windows distribution
        run: bash build-dist-windows.sh "${{ inputs.version }}"

      - name: Build NSIS installer
        run: makensis -DVERSION="${{ inputs.version }}" installer.nsi

      - uses: actions/upload-artifact@v4
        with:
          name: windows-installer
          path: build/*.exe

      - uses: actions/upload-artifact@v4
        with:
          name: windows-portable
          path: build/*.zip

  build-linux:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build Linux distribution
        run: bash build-dist.sh "${{ inputs.version }}"

      - uses: actions/upload-artifact@v4
        with:
          name: linux-tarball
          path: build/*.tar.gz
```

**Key differences from `release.yml`:**

- **No `create-release` job** — build jobs run independently
- **`actions/upload-artifact`** instead of Gitea release API — artifacts are downloadable from the workflow run page
- **`workflow_dispatch`** — triggered manually from Gitea UI (Actions → Build Artifacts → Run)
- **Same build scripts** — `build-dist-windows.sh` and `build-dist.sh` accept the version as an argument

> **Note:** `actions/upload-artifact` stores artifacts temporarily (default retention: 90 days). For permanent distribution, use the release workflow with a `v*` tag.

**Docker-only projects** don't need artifact uploads — just verify the image builds:

```yaml
# .gitea/workflows/build.yml
name: Build Docker Image

on:
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build Docker image
        run: docker build -t your-app:dev .
```

## 7. Docker Build

### Multi-stage Dockerfile

```dockerfile
# Stage 1: Build frontend
FROM node:20-slim AS frontend
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: Runtime
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends <system-deps> \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY pyproject.toml .
RUN pip install --no-cache-dir ".[notifications]"
COPY src/ src/
COPY --from=frontend /app/src/your_package/static/ src/your_package/static/

RUN useradd -r -u 1000 appuser
USER appuser

HEALTHCHECK --interval=30s --timeout=5s CMD curl -f http://localhost:8080/health || exit 1
EXPOSE 8080
CMD ["uvicorn", "your_package.main:app", "--host", "0.0.0.0", "--port", "8080"]
```

### Docker in CI (Gitea Registry)

```yaml
- name: Login to Gitea Container Registry
  id: docker-login
  continue-on-error: true  # Graceful degradation if registry unavailable
  run: |
    echo "${{ secrets.DEPLOY_TOKEN }}" | docker login \
      "$SERVER_HOST" -u "${{ gitea.actor }}" --password-stdin

- name: Build and tag
  if: steps.docker-login.outcome == 'success'
  run: |
    docker build -t "$REGISTRY:$TAG" -t "$REGISTRY:$VERSION" ./server
    # Tag as 'latest' only for stable releases
    if ! echo "$TAG" | grep -qE '(alpha|beta|rc)'; then
      docker tag "$REGISTRY:$TAG" "$REGISTRY:latest"
    fi

- name: Push
  if: steps.docker-login.outcome == 'success'
  run: docker push "$REGISTRY" --all-tags

- name: Trigger redeploy webhook
  if: steps.docker-login.outcome == 'success'
  continue-on-error: true  # Optional — webhook may not be configured
  run: |
    if [ -n "${{ secrets.DOCKER_REDEPLOY_WEBHOOK_URL }}" ]; then
      echo "Triggering redeploy webhook..."
      curl -sf -X POST "${{ secrets.DOCKER_REDEPLOY_WEBHOOK_URL }}" \
        --max-time 30 || echo "::warning::Redeploy webhook failed"
    else
      echo "DOCKER_REDEPLOY_WEBHOOK_URL not set — skipping auto-deploy"
    fi
```

**Registry URL pattern:** `{gitea-host}/{owner}/{repo}:{tag}`

### Auto-Deploy via Portainer Webhook (Optional)

Portainer can automatically redeploy a stack/service when a new image is pushed.
Each stack has its own unique webhook URL generated in the Portainer UI.

**Setup:**

1. In Portainer, open your stack → **Webhooks** → enable and copy the URL
2. In Gitea, go to repo **Settings → Secrets** → add `DOCKER_REDEPLOY_WEBHOOK_URL`
3. The CI step above calls the webhook after `docker push` — Portainer pulls
   the new image and recreates the container

**Notes:**

- The webhook URL itself acts as authentication — no extra token needed
- The step uses `continue-on-error: true` so missing webhooks don't fail the build
- Each service/stack gets its own webhook — one secret per repo
- If `DOCKER_REDEPLOY_WEBHOOK_URL` is not set, the step is silently skipped

### Docker Network on TrueNAS

If Docker builds fail with `route for the gateway 0.0.0.1 could not be found: network is unreachable`, the Docker address pool is misconfigured. In TrueNAS 25.10+:

1. Go to **Apps → Configuration** (gear icon)
2. Change Address Pool base from `0.0.0.0` to `172.17.0.0` (keep `/12`, size `24`)
3. Save — Docker restarts automatically

Verify: `sudo docker run --rm alpine ping -c 1 google.com`

### Package-Repository Linking

Gitea does not auto-link container packages to repositories on first push, even with the `org.opencontainers.image.source` label in the Dockerfile. You must manually link the package once:

1. Go to your profile → **Packages** → click the package
2. Go to **Settings** → link it to the repository

Subsequent pushes to the same package name stay linked automatically.

## 8. Release Versioning

| Tag format | Example | Pre-release? | Docker `:latest`? |
| --- | --- | --- | --- |
| `v{major}.{minor}.{patch}` | `v0.2.0` | No | Yes |
| `v{major}.{minor}.{patch}-alpha.{n}` | `v0.2.0-alpha.1` | Yes | No |
| `v{major}.{minor}.{patch}-beta.{n}` | `v0.2.0-beta.2` | Yes | No |
| `v{major}.{minor}.{patch}-rc.{n}` | `v0.2.0-rc.1` | Yes | No |

**Release process:**

```bash
# Stable release
git tag v0.2.0
git push origin v0.2.0

# Pre-release
git tag v0.2.0-alpha.1
git push origin v0.2.0-alpha.1
```

### 8.1. Release Notes from File

Instead of hardcoding release notes in the workflow, keep a `RELEASE_NOTES.md` in the repo root. The CI fetches only that file (via sparse-checkout for speed) and prepends its content to the auto-generated Downloads section.

**Workflow:**

1. Before tagging, write `RELEASE_NOTES.md` with changes for this release
2. Commit, tag, push — CI picks up the file automatically
3. Release body = your notes + auto-generated download/Docker instructions

**CI implementation:**

```yaml
- name: Fetch RELEASE_NOTES.md only
  uses: actions/checkout@v4
  with:
    sparse-checkout: RELEASE_NOTES.md
    sparse-checkout-cone-mode: false

- name: Create release
  run: |
    if [ -f RELEASE_NOTES.md ]; then
      export RELEASE_NOTES=$(cat RELEASE_NOTES.md)
      echo "Found RELEASE_NOTES.md"
    else
      export RELEASE_NOTES=""
      echo "No RELEASE_NOTES.md found"
    fi

    # Python reads RELEASE_NOTES from env, prepends to Downloads section
    BODY_JSON=$(python3 -c "
    import json, os, textwrap
    notes = os.environ.get('RELEASE_NOTES', '')
    sections = []
    if notes.strip():
        sections.append(notes.strip())
    sections.append('## Downloads\n...')  # auto-generated part
    print(json.dumps('\n\n'.join(sections)))
    ")

    # Create release with combined body
    curl -s -X POST ... -d "{\"body\": $BODY_JSON, ...}"
```

> **Why sparse-checkout?** The `create-release` job only needs one file. A sparse-checkout skips downloading the full repo history and working tree, making the step significantly faster — especially in repos with large static assets or many files.

If `RELEASE_NOTES.md` is absent, the release uses only the auto-generated Downloads section — no manual notes needed for quick pre-releases.

## 9. Gitea vs GitHub Actions Differences

| Feature | GitHub Actions | Gitea Actions |
| --- | --- | --- |
| Context prefix | `github.*` | `gitea.*` |
| Ref name | `${{ github.ref_name }}` | `${{ gitea.ref_name }}` |
| Server URL | `${{ github.server_url }}` | `${{ gitea.server_url }}` |
| Repository | `${{ github.repository }}` | `${{ gitea.repository }}` |
| Actor | `${{ github.actor }}` | `${{ gitea.actor }}` |
| SHA | `${{ github.sha }}` | `${{ gitea.sha }}` |
| Output vars | `$GITHUB_OUTPUT` | `$GITHUB_OUTPUT` (same) |
| Secrets | `${{ secrets.NAME }}` | `${{ secrets.NAME }}` (same) |
| Docker Buildx | Available | May not work (runner networking) |

## 10. Version Management — Single Source of Truth

The version must be consistent across the UI, package metadata, installer filename, and Docker label. The simplest approach: **`pyproject.toml` is the single source of truth**, and everything else derives from it.

### 10.1. Runtime Version via `importlib.metadata`

Instead of hardcoding `__version__` in `__init__.py`, read it from installed package metadata:

```python
# src/your_package/__init__.py
from importlib.metadata import version, PackageNotFoundError

try:
    __version__ = version("your-package-name")
except PackageNotFoundError:
    __version__ = "0.0.0-dev"  # Running from source without pip install
```

This reads whatever version `pip install` baked in. No manual syncing needed.

### 10.2. CI Stamps the Tag into `pyproject.toml`

Build scripts resolve the version (see section 3), then stamp it before building:

```bash
VERSION_CLEAN="${VERSION#v}"  # "0.3.0" from "v0.3.0"

# Stamp into pyproject.toml so pip install bakes the correct version
sed -i "s/^version = .*/version = \"${VERSION_CLEAN}\"/" server/pyproject.toml
```

This happens in both `build-dist.sh` and `build-dist-windows.sh`, before `pip install` or wheel extraction.

### 10.3. Docker Version Stamping

Use a build arg so the version is stamped during the Docker build:

```dockerfile
ARG APP_VERSION=0.0.0

# Stamp before pip install (which bakes version into metadata)
RUN sed -i "s/^version = .*/version = \"${APP_VERSION}\"/" pyproject.toml \
    && pip install --no-cache-dir ".[notifications]"
```

In CI:

```yaml
docker build --build-arg APP_VERSION="${VERSION}" ./server
```

The `--label org.opencontainers.image.version` flag can also be set dynamically:

```yaml
docker build \
  --build-arg APP_VERSION="${VERSION}" \
  --label "org.opencontainers.image.version=${VERSION}" \
  ./server
```

### 10.4. Version Fallback Chain (Updated)

The build script fallback (section 3) should read from `pyproject.toml` instead of `__init__.py`:

```bash
# Fallback 4: Read from pyproject.toml (single source of truth)
if [ -z "$VERSION" ]; then
    VERSION=$(grep -oP '^version\s*=\s*"\K[^"]+' server/pyproject.toml 2>/dev/null || echo "0.0.0")
fi
```

### 10.5. Version Flow Summary

```text
git tag v0.3.0
  → CI reads tag
  → sed stamps pyproject.toml with "0.3.0"
  → pip install bakes "0.3.0" into package metadata
  → importlib.metadata.version() reads "0.3.0" at runtime
  → UI shows "v0.3.0", update checker compares against it
```

## 11. In-App Auto-Update

For self-hosted apps distributed as installers or portable ZIPs, you can implement in-app update checking and one-click updates. The design separates the platform-specific hosting API (Gitea, GitHub) from the update logic via an abstraction layer.

### 11.1. Architecture

```text
UpdateService (background task)
  ├── ReleaseProvider (abstract)
  │     ├── GiteaReleaseProvider  — GET /api/v1/repos/{repo}/releases
  │     └── GitHubReleaseProvider — GET api.github.com/repos/{owner}/{repo}/releases
  ├── VersionCheck — normalize semver tags, compare via packaging.version
  └── InstallType  — detect installer vs portable vs Docker vs dev
```

### 11.2. Release Provider Abstraction

The abstract interface has two methods:

```python
class ReleaseProvider(ABC):
    @abstractmethod
    async def get_releases(self, limit: int = 10) -> list[ReleaseInfo]: ...

    @abstractmethod
    def get_releases_page_url(self) -> str: ...
```

`ReleaseInfo` is a frozen dataclass with `tag`, `version`, `name`, `body`, `prerelease`, `published_at`, and `assets` (tuple of `AssetInfo` with `name`, `size`, `download_url`).

**Gitea implementation** hits `GET {base_url}/api/v1/repos/{repo}/releases?limit=N`. **GitHub** would hit `api.github.com/repos/{owner}/{repo}/releases`. The rest of the system only depends on the abstract interface — swapping providers is a config change.

### 11.3. Version Comparison

Gitea/GitHub tags like `v0.3.0-alpha.1` must be normalized to PEP 440 for comparison:

```python
import re
from packaging.version import Version

_PRE_PATTERN = re.compile(r"^(\d+\.\d+\.\d+)[-.]?(alpha|beta|rc)[.-]?(\d+)$", re.IGNORECASE)
_PRE_MAP = {"alpha": "a", "beta": "b", "rc": "rc"}

def normalize_version(raw: str) -> Version:
    cleaned = raw.lstrip("v").strip()
    m = _PRE_PATTERN.match(cleaned)
    if m:
        base, pre_label, pre_num = m.group(1), m.group(2).lower(), m.group(3)
        cleaned = f"{base}{_PRE_MAP[pre_label]}{pre_num}"
    return Version(cleaned)
```

Examples: `v0.3.0-alpha.1` → `0.3.0a1`, `v0.3.0-rc.3` → `0.3.0rc3`, `v1.0.0` → `1.0.0`.

### 11.4. Install Type Detection

Detect at startup by checking filesystem markers:

| Marker | Install type | Auto-update strategy |
| -------- | ------------- | --------------------- |
| `uninstall.exe` in CWD | `installer` | Download `-setup.exe`, run `/S /D=<dir>` (silent NSIS reinstall) |
| `python/python.exe` in CWD (no uninstaller) | `portable` (Windows) | Download ZIP, extract, swap `app/` + `python/` via detached bat script |
| `venv/` + `run.sh` in CWD | `portable` (Linux) | Download tarball, extract, swap `app/` + `venv/` via detached shell script |
| `/.dockerenv` exists | `docker` | No auto-update — show "pull new image" instructions |
| None of above | `dev` | No auto-update — link to releases page |

### 11.5. Update Service (Background Task)

Follows the same pattern as other background services (e.g., auto-backup):

- **Settings** persisted in DB: `enabled`, `check_interval_hours`, `include_prerelease`
- **30s startup delay** to avoid slowing boot
- **60s debounce** on manual checks to prevent API spam
- **State machine**: `IDLE → CHECKING → UPDATE_AVAILABLE → DOWNLOADING → APPLYING`
- **WebSocket events**: `update_available` (triggers banner), `update_download_progress` (progress bar)
- **Dismissed version** persisted in DB (survives restarts)

### 11.6. Update Application Strategies

**NSIS installer (Windows):**

```python
# Launch installer silently, detached from server process
subprocess.Popen(
    [str(exe_path), "/S", f"/D={install_dir}"],
    creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP,
)
# Shut down server — installer waits for python.exe to unlock, then installs
request_shutdown()
```

The NSIS installer's `.onInit` detects the locked `python.exe` and handles it. After install, the VBS launcher or startup shortcut restarts the app.

**Portable ZIP (Windows):**

Can't replace files while the server is running (DLL locks). Solution: write a `.bat` script that runs after shutdown:

```batch
@echo off
timeout /t 5 /nobreak >nul
rmdir /s /q "C:\path\app" 2>nul
move /y "C:\path\staging\YourApp\app" "C:\path\app"
rmdir /s /q "C:\path\python" 2>nul
move /y "C:\path\staging\YourApp\python" "C:\path\python"
rmdir /s /q "C:\path\staging" 2>nul
start "" wscript.exe "C:\path\scripts\start-hidden.vbs"
del /f /q "%~f0"
```

Launch it detached via `subprocess.Popen`, then shut down the server.

**Portable tarball (Linux):**

Same pattern but with a shell script:

```bash
#!/bin/bash
sleep 3
rm -rf "$APP_ROOT/app" && mv "$STAGING/YourApp/app" "$APP_ROOT/app"
rm -rf "$APP_ROOT/venv" && mv "$STAGING/YourApp/venv" "$APP_ROOT/venv"
rm -rf "$STAGING"
cd "$APP_ROOT" && exec ./run.sh
```

### 11.7. API Endpoints

| Method | Path | Purpose |
| -------- | ------ | --------- |
| `GET` | `/system/update/status` | Current state, available version, install type, progress |
| `POST` | `/system/update/check` | Trigger immediate check |
| `POST` | `/system/update/dismiss` | Dismiss notification for a version |
| `POST` | `/system/update/apply` | Download + install + restart |
| `GET/PUT` | `/system/update/settings` | Auto-check settings |

### 11.8. Frontend Integration

- **Version badge**: Add CSS class `has-update` to the version pill in the header — changes color + pulse animation, clickable to open settings
- **Banner**: Persistent dismissible bar below header with icon buttons (external link to releases, download to apply, X to dismiss). Stored in `localStorage` so dismissed state survives page refresh
- **Settings tab**: "Updates" tab showing current version, install type, status, check button, "Update Now" button (visible only when `can_auto_update` is true), download progress bar, release notes preview, auto-check toggle + interval + channel (stable/pre-release)
- **WebSocket events**: `update_available` triggers banner + badge, `update_download_progress` updates progress bar, `server_restarting` shows reconnect overlay

### 11.9. Key Design Decisions

1. **Provider abstraction** — Gitea today, GitHub/GitLab tomorrow. One interface, swap implementations via config.
2. **No data directory changes** — Updates replace `app/` and `python/` (or `venv/`) but **never touch `data/`**. User configuration survives every update.
3. **Detached post-update scripts** — The server can't replace its own files while running. A detached script waits for shutdown, does the swap, then restarts.
4. **Silent NSIS reinstall** — NSIS `/S` flag runs the full installer flow without UI, preserving the `data/` directory because the uninstaller section is never triggered during an upgrade.
5. **Asset matching by filename pattern** — e.g., `*-setup.exe` for installer, `*-win-x64.zip` for portable. Platform + install type determines the pattern.
6. **Pre-release channel** — Users opt in to alpha/beta/RC builds via settings. Default is stable only.

## 12. Checklist for New Projects

- [ ] Create `.gitea/workflows/test.yml` — lint + test on push/PR
- [ ] Create `.gitea/workflows/release.yml` — build + release on `v*` tag
- [ ] Add `DEPLOY_TOKEN` secret to repository
- [ ] Set up version detection in build scripts (tag → env → source)
- [ ] Set up `importlib.metadata` version in `__init__.py` (section 10.1)
- [ ] Add `sed` version stamp step in build scripts (section 10.2)
- [ ] Create `build-dist.sh` for Linux (venv + tarball)
- [ ] Create `build-dist-windows.sh` for Windows (embedded Python + ZIP)
- [ ] Create `installer.nsi` for Windows installer (optional)
- [ ] Create `Dockerfile` for Docker builds (optional)
- [ ] Configure `pyproject.toml` with `[tool.ruff]` and `[tool.pytest]`
- [ ] Set up `.pre-commit-config.yaml` with ruff + black
- [ ] Add in-app update checker if self-hosted (section 11)

## 13. Local Build Testing (Windows)

The CI builds on Linux, but you can build locally on Windows for faster iteration.

### 13.1. Prerequisites

Install NSIS (for installer builds):

```powershell
# If winget is not on PATH, use the full path:
& "$env:LOCALAPPDATA\Microsoft\WindowsApps\winget.exe" install NSIS.NSIS
# Installs to: C:\Program Files (x86)\NSIS\makensis.exe
```

### 13.2. Build Steps

```bash
# 1. Build frontend
npm ci && npm run build

# 2. Build Windows distribution (from Git Bash)
bash build-dist-windows.sh v1.0.0

# 3. Build NSIS installer
"/c/Program Files (x86)/NSIS/makensis.exe" -DVERSION="1.0.0" installer.nsi
# Output: build/YourApp-v1.0.0-setup.exe
```

### 13.3. Common Issues

| Issue | Cause | Fix |
| ------- | ------- | ----- |
| `zip: command not found` | Git Bash doesn't include `zip` | Harmless — only affects the portable ZIP, not the installer. Install `zip` via MSYS2 if needed |
| `Exec expects 1 parameters, got 2` | `MUI_FINISHPAGE_RUN_PARAMETERS` quoting breaks NSIS `Exec` | Use `MUI_FINISHPAGE_RUN_FUNCTION` instead (see section 6) |
| `Error opening file for writing: ...python\\_asyncio.pyd` | Server is running and has DLLs locked | Stop the server before installing. Add `.onInit` file-lock check (see section 6) |
| App doesn't start after install (Launch checkbox) | VBS uses `python` but embedded Python isn't on PATH | Use embedded Python fallback in VBS (see Hidden Launcher section) |
| `winget` not recognized | `winget.exe` exists but isn't on shell PATH | Use full path: `$env:LOCALAPPDATA\Microsoft\WindowsApps\winget.exe` |
| NSIS warning: `install function not referenced` | `MUI_FINISHPAGE_RUN` define is missing | `MUI_FINISHPAGE_RUN_FUNCTION` still requires `MUI_FINISHPAGE_RUN` to be defined (controls checkbox visibility). Set it to `""` |
| `dist/` has stale files after code changes | `build-dist-windows.sh` copies from source at build time | Re-run the full build script, or manually copy changed files into `dist/` |
| Audio visualizer returns all zeros / `numpy.fromstring` removed | `pip download` pulls numpy 2.x as a transitive dependency; `soundcard` uses `numpy.fromstring` which was removed in numpy 2.0 | Pin `numpy>=1.24.0,<2.0` in both `pyproject.toml` and `build-dist-windows.sh`. Also add a post-download cleanup step to remove numpy 2.x wheels pulled transitively: `for f in "$WHEEL_DIR"/numpy-2*; do rm "$f"; done` |
| Wrong numpy version despite pinning `<2.0` | The `pip download` fallback (no `--only-binary`) ignores platform constraints and pulls the host Python's numpy. Also, transitive deps from other packages (e.g. `soundcard`, `uvicorn`) can pull unconstrained numpy | Add `--python-version` to fallback command, use `--no-cache-dir`, and strip numpy 2.x wheels after the download loop |

### 13.4. Iterating on Installer Only

If you only changed `installer.nsi` (not app code), skip the full rebuild:

```bash
# Just rebuild the installer using existing dist/
"/c/Program Files (x86)/NSIS/makensis.exe" -DVERSION="1.0.0" installer.nsi
```

If you changed app code or dependencies, you must re-run `build-dist-windows.sh` first —
the `dist/` directory is a snapshot and won't pick up source changes automatically.

## 14. Troubleshooting

### Running server blocks installation

See section 13.3. The `.onInit` function in section 6 shows how to detect a locked
`python.exe` and prompt the user before proceeding.

### Pillow / package version mismatch on upgrade

**Symptom (at runtime, after upgrading the installed app):**

```
RuntimeWarning: The _imaging extension was built for another version of Pillow or PIL:
Core version: 12.2.0
Pillow version: 12.1.1
ImportError: The _imaging extension was built for another version of Pillow or PIL
```

A fresh install works (or fails with a different error like `ModuleNotFoundError: PIL._version`),
but **upgrading from a previous version** produces the mismatch above. The two reported
versions are unrelated to anything pinned in your build script.

**Two cooperating bugs cause this:**

1. **`cleanup_site_packages` deletes `.py` files without `compileall`-ing them first.**
   The dist ships with `PIL/__init__.py` + `PIL/_imaging.pyd` only — no `_version.py`,
   no `Image.py`, no `ImageDraw.py`. See section 4.3.

2. **The NSIS installer's `File /r` is a merge, not a replace.** It overwrites files
   that exist in the new payload but leaves any file that was in the old install but
   missing from the new one. So upgrading produces a half-old/half-new site-packages:

   | File                              | After upgrade  |
   |-----------------------------------|----------------|
   | `PIL/__init__.py` (in new dist)   | NEW (12.2.0)   |
   | `PIL/_imaging.cp311.pyd` (in new) | NEW (12.2.0)   |
   | `PIL/_version.py` (NOT in new)    | OLD (12.1.1)   |
   | `PIL/Image.py` (NOT in new)       | OLD (12.1.1)   |

   The new `__init__.py` runs `from . import _version`, reads the **old** `_version.py`,
   reports `__version__ == "12.1.1"`, and then PIL/Image.py compares it against the
   `.pyd` (built for 12.2.0). Mismatch.

**Why it's confusing:** the symptom points at "duplicate Pillow wheels in the build" or
"pip resolver instability" — neither is the actual cause. The fix is in two places that
look unrelated to Pillow.

**Fix:**

- **Stop deleting `.py` files in `cleanup_site_packages`** (or do it correctly with
  `python -m compileall -b` first — see section 4.3).
- **Add explicit `RMDir /r` of payload dirs** at the top of the NSIS Core section,
  before `File /r` — see section 6.

**Why v0.1.x worked before this surfaced:** if the app's only consumer of the broken
package is something like a tray icon import, the bug is invisible until a transitive
dep version bumps (Pillow 12.1.1 → 12.2.0) AND a user upgrades in place. Greenfield
installs may also coincidentally work if the user once ran a dev install that left
`.py` files behind.

### Release already exists for tag

If the `create-release` job fails with `KeyError: 'id'`, the Gitea API returned an error
because a release already exists for that tag (e.g., from a previous failed run).

**Prevention:** The fallback logic in section 2.1 handles this automatically — if creation
fails, it fetches the existing release by tag and reuses its ID. A `::warning::` annotation
is emitted in the workflow log.

**Re-triggering a failed release workflow:**

```bash
# Option A: Delete and re-push the same tag
git push origin :refs/tags/v0.1.0-alpha.2   # delete remote tag
# Delete the release in Gitea UI or via API
git tag -f v0.1.0-alpha.2                    # recreate local tag
git push origin v0.1.0-alpha.2               # push again

# Option B: Just bump the version (simpler)
git tag v0.1.0-alpha.3
git push origin v0.1.0-alpha.3
```