Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 10eb24b2ce | |||
| 66b85b0175 | |||
| bc42604045 | |||
| 3645216669 | |||
| 85da2e538d | |||
| e4d24a02da |
@@ -98,6 +98,9 @@ jobs:
|
||||
print(json.dumps('\n\n'.join(sections)))
|
||||
")
|
||||
|
||||
# Created as draft so the release isn't user-visible until every
|
||||
# build job has attached its assets. The publish-release job at
|
||||
# the end of the workflow flips draft=false once all builds pass.
|
||||
RELEASE=$(curl -s -X POST "$BASE_URL/releases" \
|
||||
-H "Authorization: token $GITEA_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
@@ -105,7 +108,7 @@ jobs:
|
||||
\"tag_name\": \"$TAG\",
|
||||
\"name\": \"LedGrab $TAG\",
|
||||
\"body\": $BODY_JSON,
|
||||
\"draft\": false,
|
||||
\"draft\": true,
|
||||
\"prerelease\": $IS_PRE
|
||||
}")
|
||||
|
||||
@@ -350,3 +353,25 @@ jobs:
|
||||
if ! echo "$TAG" | grep -qE '(alpha|beta|rc)'; then
|
||||
docker push "$REGISTRY:latest"
|
||||
fi
|
||||
|
||||
# ── Publish the release (flip draft=false) ─────────────────
|
||||
# Runs only after every build job succeeded so users never see a
|
||||
# release that's missing artifacts or sha256 sidecars (the in-app
|
||||
# updater refuses to install without them).
|
||||
publish-release:
|
||||
needs: [create-release, build-windows, build-linux, build-docker]
|
||||
if: github.event_name == 'push' && success()
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Promote draft release to published
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
|
||||
run: |
|
||||
RELEASE_ID="${{ needs.create-release.outputs.release_id }}"
|
||||
BASE_URL="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
curl -s -X PATCH "$BASE_URL/releases/$RELEASE_ID" \
|
||||
-H "Authorization: token $GITEA_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"draft": false}'
|
||||
echo "Published release $RELEASE_ID"
|
||||
|
||||
@@ -0,0 +1,249 @@
|
||||
# Dashboard Reconciliation — Review Notes
|
||||
|
||||
*Captured 2026-05-26. Session focused on dashboard + perf-card flicker and per-poll re-rendering.*
|
||||
|
||||
*Updated 2026-05-27 — widened the audit beyond the two poll timers and found a **second driver** (server push) plus the **highest-blast-radius site** (`entity-events.ts`). Added §3.5, corrected the "out of scope" reasoning in §5, and confirmed the decision: **commit to the Lit migration**. Implementation deferred — this is still a planning doc, not a spec.*
|
||||
|
||||
This is a thinking-aloud document for whoever picks up reconciliation work next (likely me). It captures the bug class, what's already shipped, what's still latent, the decision ladder we walked through, and the recommendation we landed on. It is **not** a spec — treat any code shown as illustrative.
|
||||
|
||||
---
|
||||
|
||||
## 1. The bug class in one sentence
|
||||
|
||||
> Every place a data-driven render — a poll timer **or** a server-pushed `server:*` event — writes `el.innerHTML = ...`, the existing DOM is torn down — even when the new HTML equals the old — which restarts CSS animations, drops focus, skips transitions, and burns wasted DOM mutation cycles.
|
||||
|
||||
The symptom only becomes visually loud when the destroyed subtree contains a CSS keyframe animation (e.g. the pulsing `.perf-patches-empty-dot`). Everywhere else the cost is silent: lost transitions, broken focus, wasted layout work. The bug is **load-bearing in the architecture**, not in any single call site — that's why we keep coming back to it.
|
||||
|
||||
---
|
||||
|
||||
## 2. What landed in commit `f6486f9` (this session)
|
||||
|
||||
Tactical work — solves the worst cases, does not change the architecture.
|
||||
|
||||
### `server/src/ledgrab/static/js/features/dashboard.ts`
|
||||
- Collapsed the two fast-path branches into one. Fast path runs when `structureUnchanged && !forceFullRender` regardless of `running.length`. Previously, **zero running targets meant every poll rebuilt the entire dashboard** even when nothing changed.
|
||||
- `_lastSyncClockIds` no longer fingerprints `is_running` — pausing/resuming a clock no longer tears down every card. `_updateSyncClocksInPlace` already handles the toggle.
|
||||
- `_updateAutomationsInPlace` now called from the unified fast path. Automation badges were silently going stale on the fast path.
|
||||
- `_initFpsCharts` rewritten diff-based: only destroy charts for ids that left or whose canvas was detached by a DOM swap; only create for new ids; only fetch `/api/metrics/history` when there are genuinely new ids needing seed data.
|
||||
- Sync-clock pause/resume/reset callers + `server:automation_state_changed` SSE handler now use `loadDashboard()` (no force) — `forceFullRender` is now actually load-bearing, meaning "settings changed, full rebuild required."
|
||||
|
||||
### `server/src/ledgrab/static/js/features/perf-charts.ts`
|
||||
- `_renderChartSvg` no longer rewrites `innerHTML` per poll. The SVG skeleton (ref line + sys area/line + app line) is built once via `_ensureSparkNodes` and mutated thereafter. WeakMap cache (`_sparkNodeCache`) keyed by host element avoids the per-tick `querySelector` cost.
|
||||
- Hidden cards (env-disabled GPU/Temp) skip render entirely.
|
||||
- `_fetchPerformance` switched to `fetchWithAuth`.
|
||||
- Hardcoded English strings replaced with `t()` calls. New keys: `perf.no_captures`, `perf.captures_count.{one,few,many,other}`, `perf.ratio_of_requested`, `perf.total_count`, `perf.skipped_per_sec`, `perf.tip.now`, `perf.tip.ago` (en/ru/zh).
|
||||
- Tooltip reads `dashboardPollInterval` per mousemove tick (was captured at bind time).
|
||||
- Dead `<defs><linearGradient>` block removed.
|
||||
- `updateTotalCaptureFpsActual` now delegates to `_paintCaptureFpsActualValue` — single code path.
|
||||
- `updateActivePatches` / `updateDevices` skip the `innerHTML` write when content signature hasn't changed. This is the direct fix for the "READY TO LAUNCH flickers every update" report — the empty-state dot's CSS pulse no longer resets.
|
||||
- Two missing semicolons in `_seedAggregateHistories` (ASI was saving us).
|
||||
|
||||
### Reviewer findings addressed (typescript-reviewer pass)
|
||||
- **HIGH:** `_metricLabel` was looking up `dashboard.perf.${key}` but the FPS family uses `dashboard.perf.total_fps`, `total_capture_fps`, `total_capture_fps_actual`. Tooltip would have shouted `FPS` / `CAPTURE_FPS` / `CAPTURE_FPS_ACTUAL`. Fixed via explicit `METRIC_LABEL_KEYS` map.
|
||||
- **HIGH:** `_ensureSparkNodes` silently coerced `null` children to non-null when the SVG existed but a child was missing. Hardened to validate all four children and rebuild if any are missing.
|
||||
|
||||
---
|
||||
|
||||
## 3. Hot spots still latent
|
||||
|
||||
These are the call sites where `innerHTML` is still written every poll. None are flickering today (no CSS animations on their inner elements), but every one is the same bug shape and will bite the next time someone adds a keyframe / transition / focus target inside.
|
||||
|
||||
### `perf-charts.ts`
|
||||
|
||||
| Line | Site | Fires per poll? | Notes |
|
||||
|------|------|-----------------|-------|
|
||||
| 462 | `updateActivePatches` → `listEl.innerHTML` | yes | guarded by signature compare (✓) |
|
||||
| 493 | `updateTotalFps` → `valEl.innerHTML` | yes | FPS value, no inner animation |
|
||||
| 526 | `updateTotalCaptureFps` → `valEl.innerHTML` | yes | same |
|
||||
| 638 | `_paintNetworkValue` → `valEl.innerHTML` | yes | bytes/s value |
|
||||
| 655 | `_paintDeviceLatencyValue` → `valEl.innerHTML` (no-devices hint) | yes | hint span |
|
||||
| 657 | `_paintDeviceLatencyValue` → `valEl.innerHTML` (offline hint) | yes | hint span |
|
||||
| 660 | `_paintDeviceLatencyValue` → `valEl.innerHTML` (ms value) | yes | value |
|
||||
| 676 | `_paintSendTimingValue` → `valEl.innerHTML` (idle hint) | yes | hint span |
|
||||
| 679 | `_paintSendTimingValue` → `valEl.innerHTML` (ms value) | yes | value |
|
||||
| 738 | `_paintErrorsValue` → `valEl.innerHTML` | yes | rate value |
|
||||
| 806 | `updateDevices` → `dotsEl.innerHTML` | yes | guarded by signature compare (✓) |
|
||||
| 1086 | `_renderValuePair` → `mainEl.innerHTML = appVal` | yes | dual sys/app value |
|
||||
| 1088 | `_renderValuePair` → `mainEl.innerHTML = sysVal` | yes | dual sys/app value |
|
||||
| 1094 | `_renderValuePair` → `tagEl.innerHTML` (App tag) | mode='both' only | App tag in `both` mode |
|
||||
| 1181 | `_applyPerfDataToDom` temp hint | only when cpu_temp_hint_key changes | rare |
|
||||
| 1449 | `_paintFpsValue` | seed only | once per init |
|
||||
| 1456 | `_paintCaptureFpsValue` | seed only | once per init |
|
||||
| 1463 | `_paintCaptureFpsActualValue` (no-captures hint) | yes via live updater | now goes through painter |
|
||||
| 1469 | `_paintCaptureFpsActualValue` (value) | yes via live updater | same |
|
||||
| 1499 | `_paintErrorsValue` (duplicate of 738) | seed only | once per init |
|
||||
| 1823 | tooltip `tip.innerHTML` | per mousemove | rate-limited by hover only |
|
||||
|
||||
### `dashboard.ts`
|
||||
|
||||
| Line | Site | Fires per poll? | Notes |
|
||||
|------|------|-----------------|-------|
|
||||
| 275 | `_updateRunningMetrics` → `fpsEl.innerHTML` | per running target | live FPS pill — visible churn |
|
||||
| 293 | `_updateRunningMetrics` → `labelEl.innerHTML` (errors label) | per running target | rebuilt each poll |
|
||||
| 340 | `_updateAutomationsInPlace` → `btn.innerHTML` | only on enable/disable change | low frequency |
|
||||
| 366 | `_updateSyncClocksInPlace` → `btn.innerHTML` | per poll for every clock | wasteful |
|
||||
| 975 | `loadDashboard` first-load → `container.innerHTML` | once per init | fine |
|
||||
| 989 | `loadDashboard` slow path → `dynamic.innerHTML = dynamicHtml` | only when slow path fires | the **big** swap, scoped already |
|
||||
| 1010 | `loadDashboard` error path | rare | fine |
|
||||
| 1416 | `subscribeDashboardLayout` clear | rare | fine |
|
||||
|
||||
### What this list tells us
|
||||
|
||||
- The remaining innerHTML writes are **per-cell value updates** that paint formatted spans (`{value}<span class="perf-fps-unit">fps</span>`). Each rewrite destroys two text nodes + a span every poll across ~10 cells. Not flickering today; will flicker the moment anyone adds an animation to `.perf-fps-unit` or `.perf-fps-ceiling`.
|
||||
- The pattern can be killed without architectural change by splitting these into a stable structure (number text node + static unit span) and only updating `textContent` of the number. That's what L3 / Lit would force naturally.
|
||||
|
||||
---
|
||||
|
||||
## 3.5 Beyond dashboard/perf — push-driven reconciliation
|
||||
|
||||
*Added 2026-05-27. The §3 audit was scoped to the two poll timers we were debugging. Widening the `\.innerHTML\s*=` search showed the bug class has a **second driver** and lives outside dashboard/perf too.*
|
||||
|
||||
### Two drivers, not one
|
||||
|
||||
The teardown is triggered by anything that re-renders **without user intent**:
|
||||
|
||||
- **Poll timers** (`setInterval`) — what §2/§3 covered (`dashboard.ts` `_uptimeTimer` + main refresh, `perf-charts.ts` `_pollTimer`).
|
||||
- **Server-pushed `server:*` events** — `core/events-ws.ts` turns each WS message into a `server:*` CustomEvent; feature modules listen and re-render through the *same* `innerHTML` paths.
|
||||
|
||||
So the one-line bug class in §1 reads "poll- **or** push-driven," not just poll.
|
||||
|
||||
### Genuinely-affected sites outside dashboard/perf
|
||||
|
||||
| Site | Driver | Shape | Notes |
|
||||
| ---- | ------ | ----- | ----- |
|
||||
| `core/entity-events.ts` `_invalidateAndReload` | push (`server:entity_changed`, `server:device_health_changed`) | full-**tab** rebuild via `loadTargetsTab` / `loadPictureSources` / `loadAutomations` / `loadIntegrations` | **highest blast radius.** A single pushed entity change tears down and rebuilds an entire tab — losing scroll, focus, open inline editors, restarting card-enter animations. |
|
||||
| `features/game-integration.ts` event feed (`_eventMonitorTimer`) | poll (2 s) | `feed.innerHTML = events.slice(0,20).map(...)` | full 20-item list rebuild every 2 s while the panel is open. |
|
||||
| `features/game-integration.ts` connection test (`_connectionTestTimer`) | poll | `panel.innerHTML = …` per tick | transient, low frequency. |
|
||||
|
||||
`entity-events.ts` already has the **L1 floor applied by hand**: a 600 ms debounce plus a diff check (`oldData === newData`, then length + `id` + `updated_at` compare) that skips the reload when nothing changed. That kills the *no-op* case — but a **real** change still does the full-tab teardown. This is exactly the §4-L1 limitation ("still tears down when content *does* differ"), live across the whole app.
|
||||
|
||||
### Counter-examples that already do it right
|
||||
|
||||
Two poll loops never flicker because they mutate `textContent` on a **stable structure** instead of rewriting `innerHTML`:
|
||||
|
||||
- `core/api.ts` `loadServerInfo` (connection-check poll) — `versionEl.textContent` / `statusEl.textContent`.
|
||||
- `features/color-strips/test.ts` FPS sampler (1 s) — `valueEl.textContent` / `avgEl.textContent`.
|
||||
|
||||
These are live proof that "stable structure + mutate text node" is the fix — i.e. what L3 / Lit force by construction.
|
||||
|
||||
### What this changes about the plan
|
||||
|
||||
The §4 ladder was reasoned entirely around **per-cell** rendering, because that was the visible flicker. The push-driven finding surfaces a second, qualitatively different problem:
|
||||
|
||||
- **Problem A — cell value churn:** every poll, one value span. Loud only with animations. *Mostly fixed in `f6486f9`.* → wants `setText` / skip-if-unchanged.
|
||||
- **Problem B — list/tab teardown:** on change/push, an entire list or tab. Loses scroll/focus/open editors. *Unaddressed.* `entity-events.ts` and the game feed are Problem B. → wants **keyed list reconciliation**.
|
||||
|
||||
Problem B is a **list-level** concern, not a cell-level one. In Lit terms it maps to a keyed `repeat()` directive over the tab/list body — the dashboard-card work in Phase 2 already needs this, but `entity-events.ts` needs it for tabs that §5 used to list as "out of scope." This does **not** change the chosen direction (Lit); it adds `entity-events.ts` as a first-class, high-priority target.
|
||||
|
||||
---
|
||||
|
||||
## 4. Decision ladder
|
||||
|
||||
Walked through with the user 2026-05-26. Captured here so we don't re-litigate.
|
||||
|
||||
### L1 — drop-in `setInnerHtmlIfChanged` helper
|
||||
- **Shape:** `WeakMap<Element, string>` cache; replace every `el.innerHTML = x` with `setInnerHtmlIfChanged(el, x)`.
|
||||
- **Wins:** stops the no-change rewrites globally; zero behavior risk; ~30 call-site changes.
|
||||
- **Misses:** still tears down DOM when content *does* differ (e.g. FPS row values change every tick); doesn't preserve focus/transition state inside a list.
|
||||
- **Verdict:** floor, not ceiling. Worth doing for cells that don't get migrated to L3/Lit.
|
||||
|
||||
### L2 — lint guard
|
||||
- **Shape:** pre-commit script greps `\.innerHTML\s*=` in `static/js/` outside an allowlist, fails the commit.
|
||||
- **Wins:** keeps the discipline; cheap.
|
||||
- **Misses:** only useful as a pair with L1+; bare guard with no helper makes contributors angry.
|
||||
- **Verdict:** pair with whatever helper we land on.
|
||||
|
||||
### L3 — hand-rolled cell-component pattern
|
||||
- **Shape:** `defineCell({ html, refs, mount, update, unmount })` + `reconcileList(host, items, binding)` + `setText/setClass/setAttr` mutators. ~150–300 lines of runtime.
|
||||
- **Wins:** correct by construction; no dependencies; explicit about what mutates; composes with existing customize panel / color picker.
|
||||
- **Misses:** we own the abstraction — it grows over time as we need transitions, async data, focus, devtools, error boundaries. Death by a thousand features.
|
||||
- **Verdict:** second-best. Strong contender if zero-deps is a hard constraint.
|
||||
|
||||
### Lit migration of polling modules — **recommended**
|
||||
- **Shape:** convert each perf cell + each dashboard card cell to a Lit web component. Use `html\`<span>${value}</span>\`` tagged-template + targeted diff. ~5KB gzip added to bundle, no new build step (esbuild handles it).
|
||||
- **Wins:** solves the bug class by design; maintained by Google + community; web-components-based so no framework lock-in; composes with vanilla DOM trivially; mental model is close to current template-string idiom; non-polling code can stay vanilla forever.
|
||||
- **Misses:** introduces a dependency; contributors learn one more thing; rare edge cases (`@html`-equivalent exists and reintroduces the bug if misused).
|
||||
- **Verdict:** best ceiling-to-cost ratio for a small team. Recommended.
|
||||
|
||||
### Full framework rewrite (React / Vue / Solid)
|
||||
- **Verdict:** overkill. The bug class lives in polling paths; the rest of the app is fine. Spending the migration budget on rebuilding IconSelect / EntitySelect / modals / customize panel / graph editor — none of which are broken — is a bad trade.
|
||||
|
||||
---
|
||||
|
||||
## 5. Recommendation
|
||||
|
||||
**Lit for the polling-heavy modules.**
|
||||
|
||||
Migration plan:
|
||||
|
||||
### Phase 0 — spike (2-hour time-box)
|
||||
- Convert `patches` cell to a Lit component, end to end.
|
||||
- Verify it plays nicely with: color picker integration, customize panel layout reorder, `rerenderPerfGrid` reconciliation, `setPerfMode` toggle, hidden-by-env state, the spark tooltip handler.
|
||||
- If any of those break in an unfixable way → pivot to L3.
|
||||
- If they work → commit to the migration.
|
||||
|
||||
### Phase 1 — perf-charts cells
|
||||
1. `patches` (already spiked)
|
||||
2. `devices`
|
||||
3. `fps` / `capture_fps` / `capture_fps_actual` (share a sparkline base class)
|
||||
4. `cpu` / `ram` / `gpu` / `temp` (share `_sparkCardHtml` template family)
|
||||
5. `network` / `device_latency` / `send_timing` / `errors`
|
||||
|
||||
Each is its own PR, dashboard stays working at every step. `renderPerfSection` becomes a registry of Lit components; `rerenderPerfGrid` becomes "reorder existing elements in the grid" (which it mostly already does).
|
||||
|
||||
### Phase 2 — dashboard card cells
|
||||
6. Output target cards (running variant — biggest payoff, has live FPS + uptime + errors)
|
||||
7. Output target cards (stopped variant)
|
||||
8. Sync clock cards
|
||||
9. Automation cards
|
||||
10. Integration (HA / MQTT) cards
|
||||
|
||||
These get bigger wins from the migration because they have nested mutable state (FPS pill, errors cell, health dot, action button) that's currently rebuilt per poll via the `_updateRunningMetrics` path.
|
||||
|
||||
### Highest-impact: `entity-events.ts` tab reconciliation (sequence early)
|
||||
|
||||
`entity-events.ts` (§3.5) is the single highest-blast-radius site and is **not** on the dashboard — it re-renders the Targets / Integrations / Automations tabs on server push. Whether or not those tabs' cells become Lit components, the loader path (`loadTargetsTab` / `loadIntegrations` / `loadAutomations`) should switch from a full `innerHTML` rebuild to a **keyed list reconcile** (a Lit `repeat()` over the tab body). This preserves scroll / focus / open inline editors across pushes. If the goal is "biggest UX win first" rather than "lowest-risk first," sequence this ahead of Phase 2.
|
||||
|
||||
### Phase 3 — stopgap helper for the rest
|
||||
Add `setInnerHtmlIfChanged` and apply to any remaining vanilla polling sites we don't plan to migrate. Add the L2 lint guard at this point — by now everything that polls is either Lit-managed or uses the helper.
|
||||
|
||||
### Out of scope (deliberately) — with one correction (2026-05-27)
|
||||
|
||||
- Targets tab, automations editor, integrations, scene presets — these render on-demand, **but they are ALSO re-rendered on server push** via `entity-events.ts` (see §3.5). The original claim that "the bug class doesn't bite them" was **wrong**: a pushed `server:entity_changed` does a full-tab `innerHTML` teardown. The *editor / on-demand views* can stay vanilla, but the **list/tab render that entity-events triggers needs reconciliation** (a keyed list diff) regardless of whether those cells become Lit components. Treat the entity-events reload path as **in-scope** — it is the highest-blast-radius Problem B site.
|
||||
- Color strips editor, graph editor, settings — genuinely on-demand, no push re-render path, stay vanilla.
|
||||
- Transport bar cells (CPU/Mem chip in the top bar) — read from the same perf payload, can be migrated opportunistically but not urgent.
|
||||
|
||||
---
|
||||
|
||||
## 6. Open questions to settle before committing
|
||||
|
||||
These came up during the discussion and weren't resolved:
|
||||
|
||||
1. **Bundle-size budget.** Is +5KB acceptable? Current bundle is 2.7MB so this is noise — but worth confirming there isn't a strict cap (e.g. for slow networks / Android Chaquopy embed).
|
||||
2. **Contributor model.** If the project will grow to multiple contributors, Lit's smaller community vs React's is a recruiting tradeoff. Currently solo-ish, so probably moot.
|
||||
3. **Android TV target.** Chaquopy embed serves the same bundle. Lit works fine in any modern browser — Android TV WebView is Chromium-based. Should be a no-op but verify in Phase 0 spike.
|
||||
4. **Long-term framework intent.** If there's a chance we ever migrate to React/Vue/Solid for the rest of the app, doing Lit now is *not* lock-in (web components are standard), but it does add a second mental model. Probably fine; just naming the tradeoff.
|
||||
5. **Customize panel.** The drag-reorder code in `dashboard-customize.ts` mutates `.dashboard-section` DOM directly. Lit components reorder cleanly via `moveBefore` / `insertBefore` since they're just elements, but the dnd library needs to treat them as opaque drag handles. Phase 0 spike should confirm.
|
||||
|
||||
---
|
||||
|
||||
## 7. Pointers
|
||||
|
||||
- Source files most relevant:
|
||||
- `server/src/ledgrab/static/js/features/dashboard.ts`
|
||||
- `server/src/ledgrab/static/js/features/perf-charts.ts`
|
||||
- `server/src/ledgrab/static/js/features/dashboard-layout.ts` (cell ordering + visibility)
|
||||
- `server/src/ledgrab/static/js/features/dashboard-customize.ts` (drag-reorder UI)
|
||||
- `server/src/ledgrab/static/js/core/card-modes.ts` (mode toggle that hangs off section headers)
|
||||
- `server/src/ledgrab/static/js/core/entity-events.ts` (push-driven tab reloads — §3.5, highest blast radius)
|
||||
- `server/src/ledgrab/static/js/core/events-ws.ts` (WS → `server:*` CustomEvent dispatch)
|
||||
- `server/src/ledgrab/static/js/features/game-integration.ts` (2 s event-feed list rebuild — §3.5)
|
||||
- Most recent reconciliation commit: `f6486f9`.
|
||||
- Related skill files in `~/.claude/skills/`: `frontend-patterns`, `documentation-lookup` (for Lit docs via Context7).
|
||||
- Locale convention: `perf.*` for cross-card primitives, `dashboard.perf.*` for cell titles.
|
||||
|
||||
---
|
||||
|
||||
## 8. If this doc gets stale
|
||||
|
||||
If you read this and the perf cells are already Lit components — delete this file. If you read this and there's a new flicker / focus / transition bug nobody can explain — search for `\.innerHTML\s*=` in `static/js/features/` **and `static/js/core/`** (`entity-events` lives in core) and you've probably found it. For *state loss on a server event* (scroll jump, focus drop, an inline editor closing itself), look at the `server:*` listeners in `core/entity-events.ts` first.
|
||||
@@ -0,0 +1,327 @@
|
||||
"""Generate LedGrab app icon assets.
|
||||
|
||||
Concept: "Spectrum Aperture" — a rounded-square frame (the screen/display)
|
||||
traced by a continuous RGB color-wheel stroke (the bias-light LED strip),
|
||||
on a near-black canvas with a soft chromatic bloom behind it.
|
||||
|
||||
Outputs:
|
||||
server/src/ledgrab/static/icons/icon-512.png (standard, opaque vignette bg)
|
||||
server/src/ledgrab/static/icons/icon-192.png (downscale of 512)
|
||||
server/src/ledgrab/static/icons/icon-512-maskable.png (safe-area padded, opaque)
|
||||
server/src/ledgrab/static/icons/icon-tray.png (256, transparent bg, frame + glow)
|
||||
server/src/ledgrab/static/icons/icon.ico (16/24/32/48/64/128/256)
|
||||
|
||||
Run from repo root:
|
||||
py -3.13 build/generate_icon.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import colorsys
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image, ImageDraw, ImageFilter
|
||||
|
||||
# ── Tunables ────────────────────────────────────────────────────────────
|
||||
SUPERSAMPLE = 4 # render at 4x and downsample for crispness
|
||||
BASE = 1024 # logical canvas size
|
||||
HQ = BASE * SUPERSAMPLE # render canvas
|
||||
|
||||
BG_TOP = (12, 14, 22) # near-black, faint cool tint
|
||||
BG_BOTTOM = (6, 7, 12) # darker at edges (vignette feel)
|
||||
|
||||
FRAME_INSET = 0.18 # margin from canvas edge to frame (fraction)
|
||||
FRAME_RADIUS = 0.22 # corner radius (fraction of frame side)
|
||||
FRAME_STROKE = 0.085 # stroke width (fraction of canvas)
|
||||
BLOOM_OPACITY = 0.62 # outer bloom strength (0–1)
|
||||
INNER_GLOW_OPACITY = 0.38 # inner chromatic reflection strength
|
||||
|
||||
# Hue rotation offset so red sits at the top
|
||||
HUE_OFFSET = -90.0 # degrees (negative = counter-clockwise shift)
|
||||
|
||||
|
||||
def lerp(a: float, b: float, t: float) -> float:
|
||||
return a + (b - a) * t
|
||||
|
||||
|
||||
def hue_to_rgb(hue_deg: float) -> tuple[int, int, int]:
|
||||
"""Bright, slightly desaturated spectral color (LED-like)."""
|
||||
h = (hue_deg % 360) / 360.0
|
||||
r, g, b = colorsys.hls_to_rgb(h, 0.58, 0.92)
|
||||
return int(r * 255), int(g * 255), int(b * 255)
|
||||
|
||||
|
||||
def vignette_background(size: int) -> Image.Image:
|
||||
"""Dark canvas with a soft radial vignette + faint scanline noise."""
|
||||
img = Image.new("RGB", (size, size), BG_TOP)
|
||||
px = img.load()
|
||||
cx, cy = size / 2, size / 2
|
||||
max_r = math.hypot(cx, cy)
|
||||
for y in range(size):
|
||||
for x in range(size):
|
||||
d = math.hypot(x - cx, y - cy) / max_r
|
||||
t = min(1.0, d**1.6)
|
||||
px[x, y] = (
|
||||
int(lerp(BG_TOP[0], BG_BOTTOM[0], t)),
|
||||
int(lerp(BG_TOP[1], BG_BOTTOM[1], t)),
|
||||
int(lerp(BG_TOP[2], BG_BOTTOM[2], t)),
|
||||
)
|
||||
return img
|
||||
|
||||
|
||||
def draw_chromatic_bloom(size: int) -> Image.Image:
|
||||
"""Soft, large chromatic glow behind the frame — the bias-light effect."""
|
||||
layer = Image.new("RGBA", (size, size), (0, 0, 0, 0))
|
||||
draw = ImageDraw.Draw(layer)
|
||||
|
||||
cx, cy = size / 2, size / 2
|
||||
radius = size * 0.36
|
||||
blob_r = int(size * 0.30)
|
||||
n_blobs = 24
|
||||
|
||||
for i in range(n_blobs):
|
||||
a = i / n_blobs * 360.0
|
||||
bx = cx + math.cos(math.radians(a - 90)) * radius
|
||||
by = cy + math.sin(math.radians(a - 90)) * radius
|
||||
r, g, b = hue_to_rgb(a + HUE_OFFSET)
|
||||
alpha = int(255 * BLOOM_OPACITY * 0.55)
|
||||
draw.ellipse(
|
||||
(bx - blob_r, by - blob_r, bx + blob_r, by + blob_r),
|
||||
fill=(r, g, b, alpha),
|
||||
)
|
||||
|
||||
# Heavy blur → continuous, dreamy halo
|
||||
layer = layer.filter(ImageFilter.GaussianBlur(radius=size * 0.10))
|
||||
return layer
|
||||
|
||||
|
||||
def rounded_rect_mask(size: int, inset: int, radius: int, stroke: int) -> Image.Image:
|
||||
"""L-mode mask of a rounded-rect ring (the frame stroke region)."""
|
||||
mask = Image.new("L", (size, size), 0)
|
||||
draw = ImageDraw.Draw(mask)
|
||||
box_outer = (inset, inset, size - inset, size - inset)
|
||||
box_inner = (
|
||||
inset + stroke,
|
||||
inset + stroke,
|
||||
size - inset - stroke,
|
||||
size - inset - stroke,
|
||||
)
|
||||
r_outer = radius
|
||||
r_inner = max(0, radius - stroke)
|
||||
draw.rounded_rectangle(box_outer, radius=r_outer, fill=255)
|
||||
draw.rounded_rectangle(box_inner, radius=r_inner, fill=0)
|
||||
return mask
|
||||
|
||||
|
||||
def draw_spectrum_frame(size: int) -> Image.Image:
|
||||
"""Draw the rounded-square frame stroke filled with a hue-rotation gradient.
|
||||
|
||||
Strategy: paint a full-canvas angular hue gradient (centered), then
|
||||
clip it with the rounded-ring mask. This guarantees a continuous,
|
||||
seam-free color flow around the entire frame.
|
||||
"""
|
||||
cx, cy = size / 2, size / 2
|
||||
|
||||
gradient = Image.new("RGB", (size, size), (0, 0, 0))
|
||||
gpx = gradient.load()
|
||||
for y in range(size):
|
||||
dy = y - cy
|
||||
for x in range(size):
|
||||
dx = x - cx
|
||||
ang = math.degrees(math.atan2(dy, dx)) + 90.0 # 0° = top
|
||||
r, g, b = hue_to_rgb(ang + HUE_OFFSET)
|
||||
gpx[x, y] = (r, g, b)
|
||||
|
||||
inset = int(size * FRAME_INSET)
|
||||
frame_side = size - 2 * inset
|
||||
stroke = int(size * FRAME_STROKE)
|
||||
radius = int(frame_side * FRAME_RADIUS)
|
||||
|
||||
mask = rounded_rect_mask(size, inset, radius, stroke)
|
||||
|
||||
out = Image.new("RGBA", (size, size), (0, 0, 0, 0))
|
||||
out.paste(gradient, (0, 0), mask)
|
||||
return out
|
||||
|
||||
|
||||
def draw_inner_screen(size: int) -> Image.Image:
|
||||
"""Subtle dark rounded square inside the frame, with faint chromatic
|
||||
inner reflection along the edges — like a screen catching ambient light."""
|
||||
inset = int(size * FRAME_INSET)
|
||||
stroke = int(size * FRAME_STROKE)
|
||||
frame_side = size - 2 * inset
|
||||
radius = int(frame_side * FRAME_RADIUS)
|
||||
|
||||
pad = int(stroke * 0.35)
|
||||
box = (
|
||||
inset + stroke + pad,
|
||||
inset + stroke + pad,
|
||||
size - inset - stroke - pad,
|
||||
size - inset - stroke - pad,
|
||||
)
|
||||
r_inner = max(0, radius - stroke - pad)
|
||||
|
||||
layer = Image.new("RGBA", (size, size), (0, 0, 0, 0))
|
||||
draw = ImageDraw.Draw(layer)
|
||||
# Dark fill, very slight cool tint
|
||||
draw.rounded_rectangle(box, radius=r_inner, fill=(10, 12, 18, 255))
|
||||
|
||||
# Inner chromatic glow: same spectrum, very soft, clipped to the screen
|
||||
bloom = draw_chromatic_bloom(size)
|
||||
screen_mask = Image.new("L", (size, size), 0)
|
||||
ImageDraw.Draw(screen_mask).rounded_rectangle(box, radius=r_inner, fill=255)
|
||||
|
||||
bloom_alpha = bloom.split()[-1].point(lambda v: int(v * INNER_GLOW_OPACITY))
|
||||
bloom.putalpha(bloom_alpha)
|
||||
|
||||
masked_bloom = Image.new("RGBA", (size, size), (0, 0, 0, 0))
|
||||
masked_bloom.paste(bloom, (0, 0), screen_mask)
|
||||
layer.alpha_composite(masked_bloom)
|
||||
|
||||
# Faint highlight glint top-left
|
||||
glint = Image.new("RGBA", (size, size), (0, 0, 0, 0))
|
||||
gdraw = ImageDraw.Draw(glint)
|
||||
glint_box = (
|
||||
box[0] + int(frame_side * 0.04),
|
||||
box[1] + int(frame_side * 0.04),
|
||||
box[0] + int(frame_side * 0.42),
|
||||
box[1] + int(frame_side * 0.18),
|
||||
)
|
||||
gdraw.rounded_rectangle(glint_box, radius=int(frame_side * 0.05), fill=(255, 255, 255, 22))
|
||||
glint = glint.filter(ImageFilter.GaussianBlur(radius=size * 0.012))
|
||||
masked_glint = Image.new("RGBA", (size, size), (0, 0, 0, 0))
|
||||
masked_glint.paste(glint, (0, 0), screen_mask)
|
||||
layer.alpha_composite(masked_glint)
|
||||
|
||||
return layer
|
||||
|
||||
|
||||
def add_outer_frame_glow(frame_rgba: Image.Image) -> Image.Image:
|
||||
"""Take the spectrum frame and produce a blurred, brightened copy for glow."""
|
||||
glow = frame_rgba.copy()
|
||||
# Slightly inflate brightness for glow
|
||||
r, g, b, a = glow.split()
|
||||
glow = Image.merge("RGBA", (r, g, b, a.point(lambda v: min(255, int(v * 0.85)))))
|
||||
glow = glow.filter(ImageFilter.GaussianBlur(radius=glow.width * 0.025))
|
||||
return glow
|
||||
|
||||
|
||||
def render_tray(size: int) -> Image.Image:
|
||||
"""Render a tray-optimised icon: transparent background, bolder frame,
|
||||
tight outer glow. Designed to read clearly at 16–32 px on top of any
|
||||
taskbar color."""
|
||||
hq = size * SUPERSAMPLE
|
||||
|
||||
# Pull the frame inward a touch and beef up the stroke so it reads at 16 px.
|
||||
global FRAME_INSET, FRAME_STROKE
|
||||
saved_inset, saved_stroke = FRAME_INSET, FRAME_STROKE
|
||||
FRAME_INSET = 0.13
|
||||
FRAME_STROKE = 0.115
|
||||
try:
|
||||
frame = draw_spectrum_frame(hq)
|
||||
finally:
|
||||
FRAME_INSET, FRAME_STROKE = saved_inset, saved_stroke
|
||||
|
||||
# Tight, bright glow that doesn't bleed past the tray cell.
|
||||
glow = frame.copy()
|
||||
r, g, b, a = glow.split()
|
||||
glow = Image.merge("RGBA", (r, g, b, a.point(lambda v: min(255, int(v * 0.95)))))
|
||||
glow = glow.filter(ImageFilter.GaussianBlur(radius=hq * 0.012))
|
||||
|
||||
canvas = Image.new("RGBA", (hq, hq), (0, 0, 0, 0))
|
||||
canvas.alpha_composite(glow)
|
||||
canvas.alpha_composite(frame)
|
||||
|
||||
return canvas.resize((size, size), Image.LANCZOS)
|
||||
|
||||
|
||||
def render(size: int, *, maskable: bool = False) -> Image.Image:
|
||||
"""Render the full icon at the given size."""
|
||||
hq = size * SUPERSAMPLE
|
||||
|
||||
if maskable:
|
||||
# Maskable: pad inward so the entire icon survives a circular crop.
|
||||
# We render the standard composition at 80% of canvas size, centered.
|
||||
bg = Image.new("RGB", (hq, hq), BG_BOTTOM).convert("RGBA")
|
||||
bg.paste(vignette_background(hq), (0, 0))
|
||||
|
||||
inner = render(size, maskable=False).resize((int(hq * 0.78), int(hq * 0.78)), Image.LANCZOS)
|
||||
# Strip the bg from the inner render: composite the spectrum
|
||||
# parts on top of our maskable background.
|
||||
ox = (hq - inner.width) // 2
|
||||
oy = (hq - inner.height) // 2
|
||||
bg.alpha_composite(inner, (ox, oy))
|
||||
return bg.resize((size, size), Image.LANCZOS)
|
||||
|
||||
bg = vignette_background(hq).convert("RGBA")
|
||||
bloom = draw_chromatic_bloom(hq)
|
||||
frame = draw_spectrum_frame(hq)
|
||||
frame_glow = add_outer_frame_glow(frame)
|
||||
inner_screen = draw_inner_screen(hq)
|
||||
|
||||
# Composite order: bg → bloom → frame_glow → inner_screen → frame
|
||||
canvas = Image.new("RGBA", (hq, hq), (0, 0, 0, 0))
|
||||
canvas.alpha_composite(bg)
|
||||
canvas.alpha_composite(bloom)
|
||||
canvas.alpha_composite(frame_glow)
|
||||
canvas.alpha_composite(inner_screen)
|
||||
canvas.alpha_composite(frame)
|
||||
|
||||
return canvas.resize((size, size), Image.LANCZOS)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
repo_root = Path(__file__).resolve().parent.parent
|
||||
targets = [
|
||||
repo_root / "server" / "src" / "ledgrab" / "static" / "icons",
|
||||
repo_root
|
||||
/ "android"
|
||||
/ "app"
|
||||
/ "build"
|
||||
/ "python"
|
||||
/ "sources"
|
||||
/ "debug"
|
||||
/ "ledgrab"
|
||||
/ "static"
|
||||
/ "icons",
|
||||
]
|
||||
|
||||
print("Rendering 1024 master...")
|
||||
master = render(1024, maskable=False)
|
||||
|
||||
print("Rendering maskable 1024 master...")
|
||||
maskable_master = render(1024, maskable=True)
|
||||
|
||||
print("Rendering tray 512 master (transparent bg)...")
|
||||
tray_master = render_tray(512)
|
||||
|
||||
for icons_dir in targets:
|
||||
if not icons_dir.exists():
|
||||
print(f" skip (missing): {icons_dir}")
|
||||
continue
|
||||
|
||||
out_512 = icons_dir / "icon-512.png"
|
||||
out_192 = icons_dir / "icon-192.png"
|
||||
out_mask = icons_dir / "icon-512-maskable.png"
|
||||
out_tray = icons_dir / "icon-tray.png"
|
||||
out_ico = icons_dir / "icon.ico"
|
||||
|
||||
master.resize((512, 512), Image.LANCZOS).save(out_512, "PNG", optimize=True)
|
||||
master.resize((192, 192), Image.LANCZOS).save(out_192, "PNG", optimize=True)
|
||||
maskable_master.resize((512, 512), Image.LANCZOS).save(out_mask, "PNG", optimize=True)
|
||||
tray_master.save(out_tray, "PNG", optimize=True)
|
||||
|
||||
# Pre-resize each frame from the 1024 master for maximum crispness.
|
||||
# Pass them via the `sizes` arg so Pillow embeds every variant.
|
||||
ico_sizes = [(16, 16), (24, 24), (32, 32), (48, 48), (64, 64), (128, 128), (256, 256)]
|
||||
# Use the tray (transparent-bg) variant for ICO frames so the file/
|
||||
# taskbar icon doesn't show a dark tile against light backgrounds.
|
||||
ico_source = tray_master.resize((256, 256), Image.LANCZOS)
|
||||
ico_source.save(out_ico, format="ICO", sizes=ico_sizes)
|
||||
|
||||
print(f" wrote: {icons_dir}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -49,7 +49,8 @@ from ledgrab.utils.win_shutdown import WindowsShutdownGuard # noqa: E402
|
||||
setup_logging()
|
||||
logger = get_logger(__name__)
|
||||
|
||||
_ICON_PATH = Path(__file__).parent / "static" / "icons" / "icon-192.png"
|
||||
_ICON_PATH = Path(__file__).parent / "static" / "icons" / "icon-tray.png"
|
||||
_ICON_FALLBACK_PATH = Path(__file__).parent / "static" / "icons" / "icon-192.png"
|
||||
|
||||
|
||||
def _run_server(server: uvicorn.Server) -> None:
|
||||
@@ -154,8 +155,9 @@ def main() -> None:
|
||||
).start()
|
||||
|
||||
# Tray on main thread (blocking)
|
||||
tray_icon = _ICON_PATH if _ICON_PATH.exists() else _ICON_FALLBACK_PATH
|
||||
tray = TrayManager(
|
||||
icon_path=_ICON_PATH,
|
||||
icon_path=tray_icon,
|
||||
port=config.server.port,
|
||||
on_exit=lambda: _request_shutdown(server),
|
||||
)
|
||||
|
||||
@@ -11,6 +11,7 @@ import sys
|
||||
import threading
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
|
||||
from fastapi.responses import StreamingResponse
|
||||
@@ -38,28 +39,59 @@ _SERVER_DIR = Path(__file__).resolve().parents[4]
|
||||
|
||||
|
||||
def _schedule_restart() -> None:
|
||||
"""Spawn a restart script after a short delay so the HTTP response completes."""
|
||||
"""Spawn a restart script after a short delay so the HTTP response completes.
|
||||
|
||||
def _restart():
|
||||
stdout/stderr of the spawned script are redirected to ``<server>/restart.log``
|
||||
so a silent failure (PowerShell not on PATH, restart.ps1 erroring, etc.)
|
||||
leaves evidence on disk instead of vanishing into a detached child.
|
||||
"""
|
||||
|
||||
def _restart() -> None:
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
|
||||
# Annotated as ``dict[str, Any]`` because the value union spans
|
||||
# int flags (Windows ``creationflags``) and bool (POSIX
|
||||
# ``start_new_session``); a narrower union confuses ``**`` unpacking.
|
||||
popen_kwargs: dict[str, Any]
|
||||
if sys.platform == "win32":
|
||||
subprocess.Popen(
|
||||
[
|
||||
"powershell",
|
||||
"-ExecutionPolicy",
|
||||
"Bypass",
|
||||
"-File",
|
||||
str(_SERVER_DIR / "restart.ps1"),
|
||||
],
|
||||
creationflags=subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP,
|
||||
)
|
||||
script = _SERVER_DIR / "restart.ps1"
|
||||
cmd = ["powershell", "-ExecutionPolicy", "Bypass", "-File", str(script)]
|
||||
popen_kwargs = {
|
||||
"creationflags": (
|
||||
subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP
|
||||
),
|
||||
}
|
||||
else:
|
||||
subprocess.Popen(
|
||||
["bash", str(_SERVER_DIR / "restart.sh")],
|
||||
start_new_session=True,
|
||||
script = _SERVER_DIR / "restart.sh"
|
||||
cmd = ["bash", str(script)]
|
||||
popen_kwargs = {"start_new_session": True}
|
||||
|
||||
if not script.is_file():
|
||||
logger.error("Restart script missing: %s", script)
|
||||
return
|
||||
|
||||
log_path = _SERVER_DIR / "restart.log"
|
||||
try:
|
||||
# Open in append mode so multiple restarts accumulate; the child
|
||||
# owns its own duped handle, so closing here in the parent is safe.
|
||||
with open(log_path, "ab") as log_file:
|
||||
log_file.write(
|
||||
f"\n--- restart spawned at {time.strftime('%Y-%m-%d %H:%M:%S')} ---\n".encode()
|
||||
)
|
||||
log_file.flush()
|
||||
proc = subprocess.Popen(
|
||||
cmd,
|
||||
stdout=log_file,
|
||||
stderr=subprocess.STDOUT,
|
||||
**popen_kwargs,
|
||||
)
|
||||
logger.info("Restart script launched: %s (PID %s, log %s)", cmd[0], proc.pid, log_path)
|
||||
except OSError as e:
|
||||
logger.error("Failed to launch restart script %s: %s", script, e, exc_info=True)
|
||||
except Exception as e:
|
||||
logger.error("Unexpected error launching restart script: %s", e, exc_info=True)
|
||||
|
||||
threading.Thread(target=_restart, daemon=True).start()
|
||||
|
||||
|
||||
@@ -84,6 +84,21 @@ class PlatformDetector:
|
||||
]
|
||||
user32.DefWindowProcW.restype = ctypes.c_ssize_t
|
||||
|
||||
# Pin the MSG pointer type so byref(msg) matches the prototype
|
||||
# (Python 3.13 ctypes rejects mismatched POINTER(MSG) caches).
|
||||
LPMSG = ctypes.POINTER(ctypes.wintypes.MSG)
|
||||
user32.GetMessageW.argtypes = [
|
||||
LPMSG,
|
||||
ctypes.wintypes.HWND,
|
||||
ctypes.c_uint,
|
||||
ctypes.c_uint,
|
||||
]
|
||||
user32.GetMessageW.restype = ctypes.c_int
|
||||
user32.TranslateMessage.argtypes = [LPMSG]
|
||||
user32.TranslateMessage.restype = ctypes.wintypes.BOOL
|
||||
user32.DispatchMessageW.argtypes = [LPMSG]
|
||||
user32.DispatchMessageW.restype = ctypes.c_ssize_t
|
||||
|
||||
def wnd_proc(hwnd, msg, wparam, lparam):
|
||||
if msg == WM_POWERBROADCAST and wparam == PBT_POWERSETTINGCHANGE:
|
||||
try:
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
"""Auto-backup engine — periodic SQLite snapshot backups."""
|
||||
"""Auto-backup engine — periodic SQLite + assets snapshot backups."""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import tempfile
|
||||
import zipfile
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
from typing import Iterable, List
|
||||
|
||||
from ledgrab.storage.database import Database
|
||||
from ledgrab.utils import get_logger
|
||||
@@ -20,19 +22,35 @@ DEFAULT_SETTINGS = {
|
||||
# Skip the immediate-on-start backup if a recent backup exists within this window.
|
||||
_STARTUP_BACKUP_COOLDOWN = timedelta(minutes=5)
|
||||
|
||||
_BACKUP_EXT = ".db"
|
||||
# Current write format. ``.db`` is still recognised on read so backups taken
|
||||
# by older versions remain listable, restorable, and prunable.
|
||||
_BACKUP_EXT = ".zip"
|
||||
_RECOGNISED_EXTS: tuple[str, ...] = (".zip", ".db")
|
||||
|
||||
# Soft warning threshold — large backups indicate an unbounded assets dir or
|
||||
# bloated DB. We don't refuse to write (user data is theirs), but log loudly
|
||||
# so the operator can investigate before disk fills up over many intervals.
|
||||
_BACKUP_SIZE_WARN_BYTES = 500 * 1024 * 1024 # 500 MB
|
||||
|
||||
|
||||
class AutoBackupEngine:
|
||||
"""Creates periodic SQLite snapshot backups of the database."""
|
||||
"""Creates periodic backups of the database and asset files.
|
||||
|
||||
Each backup is a ZIP archive containing ``ledgrab.db`` plus every file
|
||||
from ``assets_dir`` under ``assets/`` — matching the format produced by
|
||||
the manual ``GET /api/v1/system/backup`` download. The restore endpoint
|
||||
accepts either ``.zip`` or ``.db`` interchangeably.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
backup_dir: Path,
|
||||
db: Database,
|
||||
assets_dir: Path | None = None,
|
||||
):
|
||||
self._backup_dir = Path(backup_dir)
|
||||
self._db = db
|
||||
self._assets_dir = Path(assets_dir) if assets_dir else None
|
||||
self._task: asyncio.Task | None = None
|
||||
self._last_backup_time: datetime | None = None
|
||||
|
||||
@@ -82,9 +100,14 @@ class AutoBackupEngine:
|
||||
self._task.cancel()
|
||||
self._task = None
|
||||
|
||||
def _iter_backup_files(self) -> Iterable[Path]:
|
||||
"""Yield every backup file (both legacy ``.db`` and current ``.zip``)."""
|
||||
for ext in _RECOGNISED_EXTS:
|
||||
yield from self._backup_dir.glob(f"*{ext}")
|
||||
|
||||
def _most_recent_backup_age(self) -> timedelta | None:
|
||||
"""Return the age of the newest backup file, or None if no backups exist."""
|
||||
files = list(self._backup_dir.glob(f"*{_BACKUP_EXT}"))
|
||||
files = list(self._iter_backup_files())
|
||||
if not files:
|
||||
return None
|
||||
newest = max(files, key=lambda p: p.stat().st_mtime)
|
||||
@@ -124,15 +147,72 @@ class AutoBackupEngine:
|
||||
timestamp = now.strftime("%Y-%m-%dT%H%M%S")
|
||||
filename = f"ledgrab-backup-{timestamp}{_BACKUP_EXT}"
|
||||
file_path = self._backup_dir / filename
|
||||
# Stage the ZIP at <name>.partial then os.replace into place once it's
|
||||
# fully written. A crash mid-write leaves a .partial file (cleaned up
|
||||
# on the next backup) but never a half-written backup that would fool
|
||||
# ``_most_recent_backup_age`` / ``_prune_old_backups`` into trusting
|
||||
# corrupt data.
|
||||
partial_path = file_path.with_suffix(file_path.suffix + ".partial")
|
||||
|
||||
self._db.backup_to(file_path)
|
||||
# SQLite backup API → temp .db so we get a consistent snapshot
|
||||
# without holding the DB lock for the ZIP write.
|
||||
tmp = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
|
||||
tmp_path = Path(tmp.name)
|
||||
tmp.close()
|
||||
asset_count = 0
|
||||
try:
|
||||
self._db.backup_to(tmp_path)
|
||||
with zipfile.ZipFile(partial_path, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.write(tmp_path, "ledgrab.db")
|
||||
if self._assets_dir and self._assets_dir.is_dir():
|
||||
for asset_file in self._assets_dir.iterdir():
|
||||
# Skip symlinks: ``is_file()`` follows them and we
|
||||
# don't want to silently slurp a symlink target that
|
||||
# lives outside the assets dir into every backup.
|
||||
if asset_file.is_symlink():
|
||||
continue
|
||||
if asset_file.is_file():
|
||||
zf.write(asset_file, f"assets/{asset_file.name}")
|
||||
asset_count += 1
|
||||
os.replace(partial_path, file_path)
|
||||
except Exception:
|
||||
# Roll back the staged partial so it doesn't accumulate; the
|
||||
# finally block still removes the SQLite temp file. Re-raise so
|
||||
# the caller (``_backup_loop`` / ``trigger_backup``) sees + logs
|
||||
# the failure instead of silently emitting a missing backup.
|
||||
partial_path.unlink(missing_ok=True)
|
||||
raise
|
||||
finally:
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
|
||||
# Best-effort sweep of any older orphan .partial files left by a
|
||||
# crash on a previous run.
|
||||
for stale in self._backup_dir.glob("*.partial"):
|
||||
try:
|
||||
stale.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
size_bytes = file_path.stat().st_size
|
||||
self._last_backup_time = now
|
||||
logger.info(f"Backup created: {filename}")
|
||||
logger.info(
|
||||
"Backup created: %s (%d asset files, %.1f MB)",
|
||||
filename,
|
||||
asset_count,
|
||||
size_bytes / (1024 * 1024),
|
||||
)
|
||||
if size_bytes > _BACKUP_SIZE_WARN_BYTES:
|
||||
logger.warning(
|
||||
"Backup %s is %.1f MB — exceeds %d MB warning threshold; "
|
||||
"consider pruning the assets directory or lowering max_backups",
|
||||
filename,
|
||||
size_bytes / (1024 * 1024),
|
||||
_BACKUP_SIZE_WARN_BYTES // (1024 * 1024),
|
||||
)
|
||||
|
||||
def _prune_old_backups(self) -> None:
|
||||
max_backups = self._settings["max_backups"]
|
||||
files = sorted(self._backup_dir.glob(f"*{_BACKUP_EXT}"), key=lambda p: p.stat().st_mtime)
|
||||
files = sorted(self._iter_backup_files(), key=lambda p: p.stat().st_mtime)
|
||||
excess = len(files) - max_backups
|
||||
if excess > 0:
|
||||
for f in files[:excess]:
|
||||
@@ -179,9 +259,7 @@ class AutoBackupEngine:
|
||||
|
||||
def list_backups(self) -> List[dict]:
|
||||
backups = []
|
||||
for f in sorted(
|
||||
self._backup_dir.glob(f"*{_BACKUP_EXT}"), key=lambda p: p.stat().st_mtime, reverse=True
|
||||
):
|
||||
for f in sorted(self._iter_backup_files(), key=lambda p: p.stat().st_mtime, reverse=True):
|
||||
stat = f.stat()
|
||||
backups.append(
|
||||
{
|
||||
|
||||
@@ -283,6 +283,7 @@ async def lifespan(app: FastAPI):
|
||||
auto_backup_engine = AutoBackupEngine(
|
||||
backup_dir=_data_dir / "backups",
|
||||
db=db,
|
||||
assets_dir=Path(config.assets.assets_dir),
|
||||
)
|
||||
|
||||
# Create update service (checks for new releases)
|
||||
|
||||
|
Before Width: | Height: | Size: 1.8 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 4.3 KiB After Width: | Height: | Size: 72 KiB |
|
Before Width: | Height: | Size: 5.0 KiB After Width: | Height: | Size: 87 KiB |
|
After Width: | Height: | Size: 47 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 35 KiB |
@@ -180,6 +180,8 @@ class CSSEditorModal extends Modal {
|
||||
notification_default_color: getNotificationDefaultColorSnapshot(),
|
||||
notification_filter_mode: (document.getElementById('css-editor-notification-filter-mode') as HTMLInputElement).value,
|
||||
notification_filter_list: (document.getElementById('css-editor-notification-filter-list') as HTMLInputElement).value,
|
||||
notification_sound: (document.getElementById('css-editor-notification-sound') as HTMLSelectElement).value,
|
||||
notification_volume: getNotificationVolumeSnapshot(),
|
||||
notification_app_overrides: JSON.stringify(notificationGetRawAppOverrides()),
|
||||
clock_id: (document.getElementById('css-editor-clock') as HTMLInputElement).value,
|
||||
daylight_speed: (document.getElementById('css-editor-daylight-speed') as HTMLInputElement).value,
|
||||
|
||||
@@ -133,18 +133,23 @@ def _bind_winapi() -> None:
|
||||
]
|
||||
user32.DefWindowProcW.restype = LRESULT
|
||||
|
||||
# Pin the MSG pointer type once and reuse the same class object on all
|
||||
# three prototypes — Python 3.13 ctypes rejects mismatched POINTER(_MSG)
|
||||
# caches across argtypes, even though they look identical.
|
||||
LPMSG = ctypes.POINTER(_MSG)
|
||||
|
||||
user32.GetMessageW.argtypes = [
|
||||
ctypes.POINTER(_MSG),
|
||||
LPMSG,
|
||||
wintypes.HWND,
|
||||
wintypes.UINT,
|
||||
wintypes.UINT,
|
||||
]
|
||||
user32.GetMessageW.restype = wintypes.BOOL
|
||||
|
||||
user32.TranslateMessage.argtypes = [ctypes.POINTER(_MSG)]
|
||||
user32.TranslateMessage.argtypes = [LPMSG]
|
||||
user32.TranslateMessage.restype = wintypes.BOOL
|
||||
|
||||
user32.DispatchMessageW.argtypes = [ctypes.POINTER(_MSG)]
|
||||
user32.DispatchMessageW.argtypes = [LPMSG]
|
||||
user32.DispatchMessageW.restype = LRESULT
|
||||
|
||||
user32.PostMessageW.argtypes = [
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
"""Regression coverage for :class:`AutoBackupEngine` ZIP format.
|
||||
|
||||
Pins behavior added when auto-backups switched from raw ``.db`` snapshots to
|
||||
``.zip`` archives (DB + assets), and the partial-write hardening that goes
|
||||
with it (stage at ``<name>.partial`` then ``os.replace`` so a crash mid-write
|
||||
never leaves a corrupt file masquerading as a valid backup).
|
||||
"""
|
||||
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from ledgrab.core.backup.auto_backup import AutoBackupEngine
|
||||
from ledgrab.storage.database import Database
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def engine_with_assets(tmp_path):
|
||||
"""Engine wired to a small DB + assets dir with two sample files."""
|
||||
db_path = tmp_path / "ledgrab.db"
|
||||
assets_dir = tmp_path / "assets"
|
||||
assets_dir.mkdir()
|
||||
(assets_dir / "alert.wav").write_bytes(b"WAV_PAYLOAD_1")
|
||||
(assets_dir / "ping.wav").write_bytes(b"WAV_PAYLOAD_2")
|
||||
|
||||
db = Database(db_path)
|
||||
db.upsert("devices", "dev_1", "Living Room", {"id": "dev_1", "type": "mock"})
|
||||
|
||||
backup_dir = tmp_path / "backups"
|
||||
engine = AutoBackupEngine(backup_dir=backup_dir, db=db, assets_dir=assets_dir)
|
||||
yield engine
|
||||
db.close()
|
||||
|
||||
|
||||
def _only_backup(backup_dir: Path) -> Path:
|
||||
"""Return the single .zip backup in ``backup_dir``; assert exactly one."""
|
||||
zips = sorted(backup_dir.glob("*.zip"))
|
||||
assert len(zips) == 1, f"expected exactly one .zip, found {[p.name for p in zips]}"
|
||||
return zips[0]
|
||||
|
||||
|
||||
def test_backup_bundles_db_and_assets(engine_with_assets):
|
||||
"""Happy path: ZIP contains ``ledgrab.db`` plus every assets file."""
|
||||
engine_with_assets._perform_backup_sync()
|
||||
|
||||
backup = _only_backup(engine_with_assets._backup_dir)
|
||||
with zipfile.ZipFile(backup) as zf:
|
||||
names = sorted(zf.namelist())
|
||||
assert names == ["assets/alert.wav", "assets/ping.wav", "ledgrab.db"]
|
||||
|
||||
|
||||
def test_backup_preserves_asset_bytes(engine_with_assets):
|
||||
"""The asset binary inside the ZIP matches the source byte-for-byte."""
|
||||
engine_with_assets._perform_backup_sync()
|
||||
|
||||
backup = _only_backup(engine_with_assets._backup_dir)
|
||||
with zipfile.ZipFile(backup) as zf:
|
||||
assert zf.read("assets/alert.wav") == b"WAV_PAYLOAD_1"
|
||||
assert zf.read("assets/ping.wav") == b"WAV_PAYLOAD_2"
|
||||
|
||||
|
||||
def test_backup_no_partial_file_left_on_success(engine_with_assets):
|
||||
"""After a successful backup, the staging ``.partial`` is renamed away."""
|
||||
engine_with_assets._perform_backup_sync()
|
||||
|
||||
leftovers = list(engine_with_assets._backup_dir.glob("*.partial"))
|
||||
assert leftovers == []
|
||||
|
||||
|
||||
def test_backup_skips_assets_when_dir_none(tmp_path):
|
||||
"""``assets_dir=None`` produces a DB-only ZIP, no error."""
|
||||
db_path = tmp_path / "ledgrab.db"
|
||||
db = Database(db_path)
|
||||
db.upsert("devices", "dev_1", "A", {"id": "dev_1"})
|
||||
backup_dir = tmp_path / "backups"
|
||||
try:
|
||||
engine = AutoBackupEngine(backup_dir=backup_dir, db=db, assets_dir=None)
|
||||
engine._perform_backup_sync()
|
||||
|
||||
backup = _only_backup(backup_dir)
|
||||
with zipfile.ZipFile(backup) as zf:
|
||||
assert zf.namelist() == ["ledgrab.db"]
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_backup_skips_assets_when_dir_missing(tmp_path):
|
||||
"""A non-existent ``assets_dir`` is silently skipped, not an error."""
|
||||
db_path = tmp_path / "ledgrab.db"
|
||||
db = Database(db_path)
|
||||
backup_dir = tmp_path / "backups"
|
||||
try:
|
||||
engine = AutoBackupEngine(
|
||||
backup_dir=backup_dir, db=db, assets_dir=tmp_path / "does-not-exist"
|
||||
)
|
||||
engine._perform_backup_sync()
|
||||
|
||||
backup = _only_backup(backup_dir)
|
||||
with zipfile.ZipFile(backup) as zf:
|
||||
assert zf.namelist() == ["ledgrab.db"]
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_backup_failure_leaves_no_zip_or_partial(engine_with_assets):
|
||||
"""When ``db.backup_to`` raises, neither the final .zip nor the .partial
|
||||
survives. The exception propagates so the caller can log it.
|
||||
"""
|
||||
with patch.object(engine_with_assets._db, "backup_to", side_effect=RuntimeError("boom")):
|
||||
with pytest.raises(RuntimeError, match="boom"):
|
||||
engine_with_assets._perform_backup_sync()
|
||||
|
||||
assert list(engine_with_assets._backup_dir.glob("*.zip")) == []
|
||||
assert list(engine_with_assets._backup_dir.glob("*.partial")) == []
|
||||
|
||||
|
||||
def test_backup_cleans_stale_partial_from_previous_crash(engine_with_assets):
|
||||
"""A leftover ``.partial`` from a prior crash is swept on the next run."""
|
||||
stale = engine_with_assets._backup_dir
|
||||
stale.mkdir(parents=True, exist_ok=True)
|
||||
(stale / "ledgrab-backup-2025-01-01T000000.zip.partial").write_bytes(b"corrupt")
|
||||
|
||||
engine_with_assets._perform_backup_sync()
|
||||
|
||||
assert list(stale.glob("*.partial")) == []
|
||||
# The successful backup is also present.
|
||||
assert len(list(stale.glob("*.zip"))) == 1
|
||||
|
||||
|
||||
def test_backup_skips_symlinks_in_assets_dir(tmp_path):
|
||||
"""Symlinked files in ``assets_dir`` are not bundled (security guard)."""
|
||||
db_path = tmp_path / "ledgrab.db"
|
||||
assets_dir = tmp_path / "assets"
|
||||
assets_dir.mkdir()
|
||||
(assets_dir / "real.wav").write_bytes(b"REAL")
|
||||
|
||||
# Some test environments (e.g. unprivileged Windows) can't create
|
||||
# symlinks; skip rather than fail spuriously when that's the case.
|
||||
secret_target = tmp_path / "secret.bin"
|
||||
secret_target.write_bytes(b"PRIVATE")
|
||||
link_path = assets_dir / "linked.wav"
|
||||
try:
|
||||
link_path.symlink_to(secret_target)
|
||||
except (OSError, NotImplementedError):
|
||||
pytest.skip("symlinks not supported in this environment")
|
||||
|
||||
db = Database(db_path)
|
||||
try:
|
||||
backup_dir = tmp_path / "backups"
|
||||
engine = AutoBackupEngine(backup_dir=backup_dir, db=db, assets_dir=assets_dir)
|
||||
engine._perform_backup_sync()
|
||||
|
||||
backup = _only_backup(backup_dir)
|
||||
with zipfile.ZipFile(backup) as zf:
|
||||
names = zf.namelist()
|
||||
assert "assets/real.wav" in names
|
||||
assert "assets/linked.wav" not in names
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_list_backups_unions_legacy_db_and_new_zip(engine_with_assets):
|
||||
"""``list_backups`` returns both legacy ``.db`` and current ``.zip`` files
|
||||
so users can still restore from auto-backups written by older versions.
|
||||
"""
|
||||
backup_dir = engine_with_assets._backup_dir
|
||||
backup_dir.mkdir(parents=True, exist_ok=True)
|
||||
(backup_dir / "ledgrab-backup-2024-12-31T000000.db").write_bytes(b"legacy")
|
||||
|
||||
engine_with_assets._perform_backup_sync()
|
||||
|
||||
names = {b["filename"] for b in engine_with_assets.list_backups()}
|
||||
assert any(n.endswith(".db") for n in names)
|
||||
assert any(n.endswith(".zip") for n in names)
|
||||
|
||||
|
||||
def test_prune_honors_max_across_formats(tmp_path):
|
||||
"""``_prune_old_backups`` enforces ``max_backups`` across both extensions."""
|
||||
db_path = tmp_path / "ledgrab.db"
|
||||
db = Database(db_path)
|
||||
backup_dir = tmp_path / "backups"
|
||||
backup_dir.mkdir()
|
||||
|
||||
# Two legacy .db files (older), two .zip files (newer) — max=3 should
|
||||
# prune the single oldest .db and keep the rest.
|
||||
import time
|
||||
|
||||
(backup_dir / "ledgrab-backup-2024-01-01T000000.db").write_bytes(b"a")
|
||||
time.sleep(0.02)
|
||||
(backup_dir / "ledgrab-backup-2024-02-01T000000.db").write_bytes(b"b")
|
||||
time.sleep(0.02)
|
||||
(backup_dir / "ledgrab-backup-2024-03-01T000000.zip").write_bytes(b"c")
|
||||
time.sleep(0.02)
|
||||
(backup_dir / "ledgrab-backup-2024-04-01T000000.zip").write_bytes(b"d")
|
||||
|
||||
try:
|
||||
engine = AutoBackupEngine(backup_dir=backup_dir, db=db, assets_dir=None)
|
||||
engine._settings["max_backups"] = 3
|
||||
engine._prune_old_backups()
|
||||
|
||||
remaining = sorted(p.name for p in engine._iter_backup_files())
|
||||
assert remaining == [
|
||||
"ledgrab-backup-2024-02-01T000000.db",
|
||||
"ledgrab-backup-2024-03-01T000000.zip",
|
||||
"ledgrab-backup-2024-04-01T000000.zip",
|
||||
]
|
||||
finally:
|
||||
db.close()
|
||||