From f4d7f7b6bb1e479ca0fcb70ce7d8096b3dc1427d Mon Sep 17 00:00:00 2001 From: "dolgolyov.alexei" Date: Tue, 7 Apr 2026 23:03:29 +0300 Subject: [PATCH] docs: warn against deleting .py without compileall + NSIS upgrade safety MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two cooperating bugs we hit in production caused a Pillow version mismatch error on in-place upgrades. Documenting both so future projects do not reproduce them. - Section 4.3: removed the "delete .py, keep .pyc" snippet (no compileall step ever ran, so the dist shipped with no .py and no .pyc — every package's submodules were unimportable). Added a warning box and the correct compileall -b pattern. - Section 6: NSIS Core section now shows explicit RMDir /r of payload dirs before File /r. NSIS File /r is a merge, not a replace, so upgrades produce half-old/half-new site-packages. - Section 14: new troubleshooting entry "Pillow / package version mismatch on upgrade" with the symptom, both root causes, and a file-state table showing how the mismatch arises. --- gitea-python-ci-cd.md | 92 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 89 insertions(+), 3 deletions(-) diff --git a/gitea-python-ci-cd.md b/gitea-python-ci-cd.md index 54c34d5..5b92077 100644 --- a/gitea-python-ci-cd.md +++ b/gitea-python-ci-cd.md @@ -324,11 +324,26 @@ rm -rf "$SITE_PACKAGES/zeroconf/_services" # Strip debug symbols from native extensions find "$SITE_PACKAGES" -name "*.pyd" -exec strip --strip-debug {} \; - -# Remove .py source files (keep compiled .pyc only) -find "$SITE_PACKAGES" -name "*.py" ! -name "__init__.py" -delete ``` +> ⚠️ **Do NOT delete `.py` source files** without first running `python -m compileall`. +> A previous version of this guide recommended `find ... -name "*.py" ! -name "__init__.py" -delete` +> as a "30-40% size win". Without a prior `compileall` step, the dist ships with no +> `.py` and no `.pyc`, so every package's submodules become unimportable. The bug is +> latent on fresh installs (clean ImportError) but manifests as a confusing **version +> mismatch** on in-place upgrades — e.g. PIL `_imaging` extension vs `_version.py` +> mismatch — because the new install's missing files leave stale files from the old +> install in place. See Troubleshooting → "Pillow / package version mismatch on upgrade". +> +> If you really want the size win, do it correctly: +> ```bash +> python -m compileall -b -q "$SITE_PACKAGES" # compile in-place (-b strips __pycache__/) +> find "$SITE_PACKAGES" -name "*.py" -delete # then drop sources +> find "$SITE_PACKAGES" -type d -name __pycache__ -exec rm -rf {} + +> ``` +> Note `-b` writes `foo.pyc` next to `foo.py` instead of `__pycache__/foo.cpython-311.pyc`, +> so the .pyc files aren't orphaned when you delete `__pycache__`. + **Tip:** If a library is only needed for one feature (e.g., Pillow for system tray icons), move it to an optional dependency group and strip unused plugins. Replace its core usage with a library already in the dependency tree (e.g., use cv2 for JPEG encoding instead of Pillow). ### 4.4. Bundling tkinter (Optional) @@ -474,6 +489,24 @@ FunctionEnd ; Sections Section "!Core (required)" SecCore ; App files + uninstaller + SetOutPath "$INSTDIR" + + ; CRITICAL: wipe previous payload BEFORE extracting. NSIS `File /r` is a + ; merge, not a replace — it overwrites files that exist in the new payload + ; but leaves behind any file that was in the old install but not the new + ; one. On Python apps this produces a half-old/half-new site-packages + ; (e.g. new PIL/__init__.py + old PIL/_version.py) which crashes at import. + ; Preserve $INSTDIR\config.yaml (user data) by deleting only known payload dirs. + RMDir /r "$INSTDIR\python" + RMDir /r "$INSTDIR\app" + RMDir /r "$INSTDIR\scripts" + Delete "$INSTDIR\${EXENAME}" + Delete "$INSTDIR\VERSION" + Delete "$INSTDIR\config.example.yaml" + + File /r "dist\app\*.*" +SectionEnd + Section "Desktop shortcut" SecDesktop ; Optional Section "Start with Windows" SecAutostart ; Optional — Startup folder shortcut @@ -1114,6 +1147,59 @@ the `dist/` directory is a snapshot and won't pick up source changes automatical See section 13.3. The `.onInit` function in section 6 shows how to detect a locked `python.exe` and prompt the user before proceeding. +### Pillow / package version mismatch on upgrade + +**Symptom (at runtime, after upgrading the installed app):** + +``` +RuntimeWarning: The _imaging extension was built for another version of Pillow or PIL: +Core version: 12.2.0 +Pillow version: 12.1.1 +ImportError: The _imaging extension was built for another version of Pillow or PIL +``` + +A fresh install works (or fails with a different error like `ModuleNotFoundError: PIL._version`), +but **upgrading from a previous version** produces the mismatch above. The two reported +versions are unrelated to anything pinned in your build script. + +**Two cooperating bugs cause this:** + +1. **`cleanup_site_packages` deletes `.py` files without `compileall`-ing them first.** + The dist ships with `PIL/__init__.py` + `PIL/_imaging.pyd` only — no `_version.py`, + no `Image.py`, no `ImageDraw.py`. See section 4.3. + +2. **The NSIS installer's `File /r` is a merge, not a replace.** It overwrites files + that exist in the new payload but leaves any file that was in the old install but + missing from the new one. So upgrading produces a half-old/half-new site-packages: + + | File | After upgrade | + |-----------------------------------|----------------| + | `PIL/__init__.py` (in new dist) | NEW (12.2.0) | + | `PIL/_imaging.cp311.pyd` (in new) | NEW (12.2.0) | + | `PIL/_version.py` (NOT in new) | OLD (12.1.1) | + | `PIL/Image.py` (NOT in new) | OLD (12.1.1) | + + The new `__init__.py` runs `from . import _version`, reads the **old** `_version.py`, + reports `__version__ == "12.1.1"`, and then PIL/Image.py compares it against the + `.pyd` (built for 12.2.0). Mismatch. + +**Why it's confusing:** the symptom points at "duplicate Pillow wheels in the build" or +"pip resolver instability" — neither is the actual cause. The fix is in two places that +look unrelated to Pillow. + +**Fix:** + +- **Stop deleting `.py` files in `cleanup_site_packages`** (or do it correctly with + `python -m compileall -b` first — see section 4.3). +- **Add explicit `RMDir /r` of payload dirs** at the top of the NSIS Core section, + before `File /r` — see section 6. + +**Why v0.1.x worked before this surfaced:** if the app's only consumer of the broken +package is something like a tray icon import, the bug is invisible until a transitive +dep version bumps (Pillow 12.1.1 → 12.2.0) AND a user upgrades in place. Greenfield +installs may also coincidentally work if the user once ran a dev install that left +`.py` files behind. + ### Release already exists for tag If the `create-release` job fails with `KeyError: 'id'`, the Gitea API returned an error