2e51f46dfd7f8f9bc6502f7d680d23a25df9288b
76 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 alexei.dolgolyov
|
2e51f46dfd |
feat(graph): make the visual editor a full wiring control surface
Lets users wire the system end-to-end from the graph, and fixes the core bug that made drag-to-wire silently fail. - Fix drag-to-wire 422s across 5 entity kinds: updateConnection() now echoes the target's discriminator (source_type/stream_type/target_type) into the partial PUT, so value/colour-strip/audio/picture sources and output targets all wire correctly. New contract test (54 cases) in test_graph_wiring_contract.py. - Re-wire composite layers / mapped zones from the graph (right-click a layer/zone source edge -> Re-wire). Whole-list write preserves every sibling layer/zone setting, with an optimistic-concurrency guard and undo. - Secret-safe /graph topology: project entities to id/name/subtype + reference roots so the endpoint cannot leak webhook tokens or other credentials. - Carry slot indices on list edges; node custom-icon + schema-drift refinements; rewire i18n keys (en/ru/zh); wiring-control roadmap (TODO.md). |
||
|
alexei.dolgolyov
|
a5effba553 |
feat: aggregated snapshot + wiring-graph APIs, MQTT device brokers
Backend
- snapshot: GET /api/v1/snapshot aggregates targets, devices, sources,
presets and system into one payload for the HA coordinator, collapsing
the prior ~2N+M request fan-out; per-section ?include= gating.
- graph: GET /api/v1/graph{,/schema,/dependents} backed by a pure,
unit-tested graph_schema engine — one authoritative connectable-field
registry so the editor no longer hard-codes topology in two places.
- devices: thread mqtt_source_id through DeviceCreate/Update/Response and
the routes for multi-broker MQTT; shared validate_mqtt_source_exists
(_mqtt_validation.py) reused by device + output-target routes; stop
update_device masking intentional 4xx as 500.
- shutdown: bound uvicorn graceful-shutdown via GRACEFUL_SHUTDOWN_TIMEOUT
(shared by __main__, android_entry, demo) so a lingering events WebSocket
can't strand LED targets or block process exit.
- access log: structured _access_log middleware attributing each request to
its authenticated token label (never the secret); uvicorn access_log off.
Frontend
- graph editor: generic schema-driven port/edge rendering, layout and
connection handling; service-worker refresh.
- device modals: MQTT broker EntitySelect for device_type=mqtt in add-device
and settings, wired into load/save/validate/dirty-check/clone.
- i18n: en/ru/zh keys.
Tests: graph routes + schema, snapshot routes, access log, mqtt_source_id
device regressions, bounded-shutdown entrypoint. 1614 passed.
|
||
|
alexei.dolgolyov
|
85da2e538d |
feat(backup): bundle assets in ZIP + partial-write hardening + restart log
Auto-backups now produce a ZIP containing ledgrab.db plus every file in the assets dir under assets/ — matching the manual GET /api/v1/system/backup format, so restore accepts either output interchangeably. Legacy .db backups remain listable, restorable, and prunable; both extensions count toward max_backups. Writes stage to <name>.partial then os.replace into place — a crash mid-ZIP never leaves a half-written backup that masquerades as valid. Stale .partials from prior crashes are swept on the next run. Symlinks inside the assets dir are skipped so a hostile link can't slurp a target outside the dir into every backup. Backups larger than 500 MB log a warning so operators notice unbounded asset growth before disk fills up. restart.py: redirect the spawned restart script's stdout/stderr to restart.log and bail out early if the script is missing — silent failures (PowerShell off PATH, restart.ps1 erroring) used to vanish into a detached child with no diagnostic trail. Tests cover happy path, asset bytes round-trip, partial cleanup, None/missing assets_dir, failure rollback, stale-partial sweep, symlink rejection, mixed legacy+new listing, and cross-format prune. |
||
|
alexei.dolgolyov
|
f591e258f7 |
fix(storage/database): reopen connection on lifespan restart
Database opened its sqlite3 connection eagerly in __init__ and closed it
in close(); the lifespan called close() on shutdown. In production this
is fine — the lifespan runs once per process. Under pytest the module-
level ``db`` singleton survives across every TestClient session, so the
second test file's lifespan startup hit
``sqlite3.ProgrammingError: Cannot operate on a closed database`` at
fixture-setup time (AutoBackupEngine.__init__ → db.get_setting("…")
was the first reader). 65 spurious "errors" on a full Windows pytest run.
- Database: extract _open() from __init__, add ensure_open() that
reopens iff _conn is None, and have close() null _conn after the
TRUNCATE checkpoint so re-close is idempotent.
- main.py lifespan startup: call db.ensure_open() before any setting
read, so subsequent TestClient sessions get a live connection.
- tests/storage/test_database_reopen.py: pin the four invariants —
close→ensure_open round-trips data, ensure_open is a no-op when
open, close is idempotent, and using the DB after close without
ensure_open raises (callers must opt in).
Full backend suite: 1551 pass / 1 skip / 0 errors. Ruff clean.
|
||
|
alexei.dolgolyov
|
888f8fd16e |
refactor(types): PEP-604 union sweep + UP007/UP045 enforcement
ruff --select UP007,UP045 --fix converted ~1760 sites across the backend: `Optional[T]` → `T | None`, `Union[X, Y]` → `X | Y`. The remaining module-level alias targets that ruff conservatively skips (BindableFloatInput, ColorList, DeviceConfig) were converted by hand earlier in the pass. black -formatted the result so the wider unions fit cleanly under the 100-char line budget. pyproject.toml now sets [tool.ruff.lint] extend-select = ["UP007", "UP045"] so future legacy imports fire CI on every push. The pre-commit ruff hook was bumped from v0.8.0 -> v0.15.12 to recognise UP045 (split off from UP007 in v0.13). |
||
|
alexei.dolgolyov
|
907bdaf043 |
test(url-scheme): WLED route-level integration + IPv6 regression
TestWLEDSchemeInference in test_devices_routes covers the POST/PUT create-and-update flow with a stubbed WLED provider so the infer_http_scheme integration hop has end-to-end coverage instead of just the unit tests. test_url_scheme grows public IPv6 (Cloudflare / Google / Quad9 DNS), bracketed-form, and ULA cases. Adds an explicit pin for the Python ipaddress documentation-prefix quirk (2001:db8::/32 is is_private, so it routes to http:// even though some audits colloquially call it "public"). |
||
|
alexei.dolgolyov
|
ddae5719cf |
chore(frontend-infra): inbound-event allowlist + storage/state touch-ups
events-ws gains an inbound-event allowlist matching the new server-side allowlist; test_events_ws_parity pins the two lists in sync. state + storage modules and the streams / integrations / z2m-light-targets / streams-*-templates editors absorb the closeIfPristine guard alongside small UX fixes. css-editor template picks up the new MiniSelect markup for the filter-kind picker. |
||
|
alexei.dolgolyov
|
898912f8b1 |
chore(backend): MQTT/WLED/devices/capture/utils + api routes hardening
Bundle the remaining backend touch-ups that the production review landed individually as small surgical edits across many modules: - MQTT runtime: fire-and-forget task tracking + drain resilience. - mqtt_source + store + storage/color_strip_source: secret_box encryption for credentials with auto-migration of plaintext fields. - devices/discovery_watcher: task tracking on watcher start/stop. - devices/wled_client + wled_provider: URL scheme inference helper applied at the create/update boundary so bare hostnames stay valid. - core/capture/screen_capture: hardened error paths. - core/processing (mapped/processed/processor_manager/video/wled_target): smaller follow-throughs from the registry refactor that landed earlier on the branch. - utils/safe_source + utils/file_ops + utils/__init__: shared URL + IP classification helpers + larger streaming upload size caps. - api/auth: WebSocket Origin allow-list + /docs auth-gate. - api/dependencies: register the new HTTP-endpoint store. - api/routes (assets, backup, webhooks): streaming-upload caps + asyncio.gather return_exceptions on broadcast loops. - tests/test_api + tests/e2e/test_backup_flow: cover the new caps and the Origin allow-list. |
||
|
alexei.dolgolyov
|
45d12b2811 |
feat(update-service): SSRF-validated redirects + restart hardening
update_service grows explicit URL validation on the redirect chain so a hostile mirror can't bounce the updater to a private IP. restart.ps1 gets stricter argument handling and clearer log lines. default_config.yaml exposes the new toggles. test_system_routes pins the new behaviour. |
||
|
alexei.dolgolyov
|
826e680f37 |
refactor(color-strip): rename static -> single + frontend follow-through
The "static" source kind always rendered a SINGLE color and the name confused new code paths. Rename the module + kind to "single". Storage keeps backward-compatible serialisation. Frontend color-strip cards / gradient / index / test modules and the affected tests follow the new name. |
||
|
alexei.dolgolyov
|
3fe66d80cb |
feat(automations): expand automation rules + UI + engine coverage
Storage model + Pydantic schema + route surface gain the new rule shapes the engine already supports. Frontend automations editor grows the matching inputs. New core/test_automation_engine.py pins the dispatch table rules behind ~285 lines of unit coverage. |
||
|
alexei.dolgolyov
|
d6cc80074d |
feat(http-endpoints): introduce HTTP endpoint output target stack
New output kind that POSTs the current strip frame to a user-configured HTTP endpoint, alongside WLED / MQTT / Hue. Stack mirrors the existing output-target shape end-to-end: storage model + store, FastAPI router + Pydantic schemas, JS feature module + modal template, router wiring in api/__init__.py and the modal include in index.html. Tests cover both the routes and the store. |
||
|
alexei.dolgolyov
|
2f15fbb752 |
refactor(output-targets): registry + coverage assertion for response builders
``_target_to_response`` in ``api/routes/output_targets.py`` used to be
an isinstance ladder over the three OutputTarget subclasses with a
silent fallback that fabricated a ``LedOutputTargetResponse`` for
unknown types (audit finding H3). The fallback masked exactly the
kind of bug we hit on the CSS side in Phase 1.1: a new target subclass
slipped past the ladder and got mis-shaped on the wire.
Replace the ladder with a ``_TARGET_RESPONSE_BUILDERS`` dict keyed by
the concrete subclass plus an import-time
``_assert_target_response_coverage()`` that requires the registry to
exactly match ``{WledOutputTarget, HALightOutputTarget,
Z2MLightOutputTarget}``. ``_target_to_response`` now raises
``RuntimeError`` instead of silently fabricating a LED response for an
unknown subclass — coverage is asserted at import so this branch is
unreachable in normal operation.
Tests: 5 new regression tests cover bijection between expected classes
and registered builders, callable shape, the rogue-target-raises
contract, and missing/extra entry rejection in the assertion. 24
existing output-target tests stay green; ruff clean.
|
||
|
alexei.dolgolyov
|
c1aa2ebec5 |
fix(value-source): preserve store contract for game_event + error precedence
Two HIGH issues surfaced by review of
|
||
|
alexei.dolgolyov
|
3b8f00e3f9 |
refactor(value-source): per-type factories for create / update dispatch
ValueSourceStore.create_source used to be a ~260-line if/elif chain
over 14 source_type strings; update_source did the same dance again
with 14 isinstance branches (audit finding C7 store-side). Each
branch duplicated the common-fields scaffold and the per-type
defaulting + validation logic.
Lift each per-type create / update body into a free function in a
new ``storage.value_source_factories`` module:
* ``CREATE_BUILDERS[source_type]`` — owns defaulting + per-type
validation (HA needs ha_source_id + entity_id; gradient_map
needs value_source_id; system_metrics validates against
VALID_SYSTEM_METRICS; http rejects interval_s < 1; the two
adaptive_* sub-modes route to the same AdaptiveValueSource
class with different source_type discriminators).
* ``UPDATE_APPLIERS[source_type]`` — mirrors the above on the
update side; ``resolve_ref`` is applied to cross-entity
references so empty-string clears keep working.
* ``build_source(...)`` / ``apply_update(source, **kwargs)`` are
the public entry points the store calls.
* ``_assert_factory_coverage()`` runs at module import and
requires BOTH registries to match storage's _VALUE_SOURCE_MAP
exactly.
The store's ``create_source`` shrinks from ~260 lines to ~25;
``update_source`` from ~200 lines to ~40.
Tests: 14 new tests cover registry coverage in both directions
plus drift assertions, representative builder paths (static /
adaptive_time / adaptive_scene / ha_entity / http / unknown),
the AdaptiveValueSource dual-source-type discriminator, and
several applier paths including ``**_`` swallowing unknown kwargs
and HTTP zero-interval rejection. 47 existing value-source store
tests stay green; 769 storage / core / api tests in aggregate.
Ruff clean.
|
||
|
alexei.dolgolyov
|
9f3f346543 |
refactor(value-source): MetricSpec registry for SystemMetricsValueStream
SystemMetricsValueStream used to dispatch on its ``self._metric`` string across three independent if/elif chains (audit finding M5): * priming in ``start()`` (cpu_percent seed, initial network counter); * raw reading in ``_read_metric_psutil`` plus ``_read_metric_fallback``; * normalisation in ``_normalize`` (percent / min-max range / max-rate). Adding a new metric meant editing all three chains plus the Android fallback — and forgetting one branch made the metric silently return 0. Lift each per-metric concern into a free function and register them as a ``MetricSpec(name, read_psutil, read_fallback, normalize, prime)`` in a new ``core.processing.metric_readers`` module. Shared normalisers (``_norm_percent`` / ``_norm_range`` / ``_norm_rate`` / ``_zero``) live once. The stream's ``start()`` / ``_read_metric()`` / ``_normalize()`` collapse to a single registry lookup + delegation. The stream still owns its mutable state (``_disk_path``, ``_sensor_label``, ``_gpu_unavailable``, ``_prev_net_bytes``, ``_prev_net_time``, etc.) — readers operate on the stream by parameter, not by inheritance, so the kitchen-sink class shrinks by ~140 lines without losing the per-stream cadence bookkeeping. Each spec function's docstring documents which fields it reads or mutates. Tests: 16 new tests cover the 10-metric coverage set, callable shape of every spec field, the three normaliser primitives' clamping + divide-by-zero behaviour, prime-hook presence (only the three metrics that need a baseline: cpu_load + network_rx + network_tx), and fallback-path expectations (desktop-only sensors -> _zero, cpu/ram -> real MetricsProvider). 754 existing core / storage / api tests stay green; ruff clean. |
||
|
alexei.dolgolyov
|
98fb61d932 |
refactor(automations): rule dispatch via class-level handler table
AutomationEngine._evaluate_rule used to rebuild a 9-entry dispatch
dict on EVERY rule evaluation (audit finding H2). Unknown rule types
silently returned False — adding a new Rule subclass without an entry
just made it inert forever.
Refactor:
* Per-rule-type bodies are now ``_handle_<kind>(self, rule, ctx)``
methods on AutomationEngine.
* A ``_RuleEvalContext`` frozen dataclass bundles all the
cross-cutting state (running_procs, topmost_proc,
topmost_fullscreen, fullscreen_procs, idle_seconds, display_state)
so adding a new handler does not require widening
``_evaluate_rule``'s parameter list.
* ``AutomationEngine._RULE_HANDLERS`` is bound once at module-import
time after the class is defined.
* ``_assert_rule_handler_coverage()`` runs at import: every Rule
subclass imported by the module must have an entry, and entries
keyed by an unknown class are also rejected.
Unknown-type fallback now logs a warning instead of silently returning
False, so a future Rule subclass missing from the registry surfaces in
operator logs rather than just behaving as if the automation were off.
The pure storage layer (storage/automation.py) is untouched — the
handler bodies stay on the engine where the cross-layer dependencies
(MQTT runtime, HA manager, HTTP endpoint store, webhook state) live.
Tests: 4 new tests cover the rule-type/handler bijection, callable
shape, missing-entry rejection, and unknown-class rejection. 44
existing automation engine tests stay green; ruff clean.
|
||
|
alexei.dolgolyov
|
5fec8db901 |
refactor(capture): lift duplicated edge-to-LED kernels into shared module
PixelMapper and AdvancedPixelMapper in calibration.py used to carry
byte-for-byte copies of two ~80-line numpy kernels (audit finding M4):
* the vectorised average-colour-per-LED path with its cumsum + take
scratch-buffer dance; and
* the per-LED fallback loop for median / dominant colour modes.
Lift both into a new ``core.capture.edge_interpolation`` module exposing
``average_edge_to_leds(edge_pixels, edge_name, led_count, cache,
cache_key)`` and ``fallback_edge_to_leds(edge_pixels, edge_name,
led_count, calc_color)``. The cache parameter is the caller-owned dict
(``self._edge_cache``) so allocations still happen once per
(edge_len, led_count) signature — the difference is that the
boundary-builder, the buffer set, and the inner numpy ops live in
exactly one place.
PixelMapper keys its cache by edge name (``"top"`` / ``"left"`` etc.);
AdvancedPixelMapper keys by line-index int (same dict, no collision).
Both mappers' ``_map_edge_average`` / ``_map_edge_fallback`` shrink to
single delegating lines.
Tests: 9 new kernel-level tests cover uint8 dtype + shape, the cache
reuse / rebuild contract, independent cache keying, a gradient input
producing a monotonic output, the calc_color callable contract for the
fallback path, and segment-position tracking for both axes. 30
existing calibration tests stay green; ruff clean.
|
||
|
alexei.dolgolyov
|
97dae2cd62 |
refactor(processing): replace inline effect dispatch with @_effect_renderer registry
EffectColorStripStream._animate_loop used to rebuild a 12-entry dict
``renderers = {"fire": self._render_fire, ...}`` on every frame, then
look up ``renderers.get(self._effect_type, self._render_fire)``. Two
audit smells (H1) at once: per-frame dict-rebuild churn and a silent
fallback to fire whenever ``self._effect_type`` was a typo or any
``_render_*`` method got renamed without updating the dict.
Fix:
* ``@_effect_renderer("fire")`` stamps an attribute on the unbound
method.
* ``@_collect_effect_renderers`` (applied to the class) walks
members at class-creation, gathers the marked ones into
``cls._RENDERERS``, and raises ``RuntimeError`` on duplicate
registration.
The loop now reads ``type(self)._RENDERERS`` once and calls the
unbound method with explicit ``self``. An unknown ``_effect_type``
logs a warning and skips the frame (sleep one frame_time) instead of
silently rendering fire — louder failure mode without crashing the
animation thread.
Tests: 5 new tests cover the 12-effect coverage set, callable shape,
class-level (not per-instance) dict identity, duplicate-name
rejection, and the marker stamp contract.
343 existing processing / storage / API tests stay green; ruff clean.
|
||
|
alexei.dolgolyov
|
29bdacf69a |
refactor(processing): dedupe HA/Z2M _swap_color_source via shared helper
HALightTargetProcessor and Z2MLightTargetProcessor used to carry
character-for-character identical _swap_color_source method bodies
(audit finding C5) — only the log prefix differed. Extract the body
into a free function ``swap_color_source(processor, new_kind,
new_color_vs_id, *, log_label)`` in a new ``light_target_helpers``
module. Each processor's _swap_color_source now delegates to the helper
and then clears its per-entity history (``_previous_colors`` /
``_previous_on``) — that bit stays on the processor because it's per-
target state, not colour-source state.
Scope deliberately narrower than the full BaseLightTargetProcessor ABC
the audit gestured at: the 76 read sites for the per-processor colour
state across the two files made a full state-composition refactor too
risky for the live LED control loop. The free-function helper is the
minimum-blast-radius way to delete the duplication while leaving WLED
(which has no value-stream-vs-CSS dispatch) untouched.
The helper standardises both warning messages on HA's original wording
("failed to acquire color VS stream" / "failed to re-acquire CSS
stream") so existing log alerts/grep patterns keep working.
A LightTargetSwapState Protocol under TYPE_CHECKING documents the
expected processor surface; no runtime enforcement (acceptable trade-
off vs a 76-site touchpoint).
Tests: 7 new tests cover the release+acquire ordering, the not-running
no-op path, the manager-error-swallowing behaviour, the empty-id
short-circuit, and the missing-manager (TargetContext(None, None))
fallback. 354 existing storage + API + e2e + processing tests stay
green; ruff clean.
|
||
|
alexei.dolgolyov
|
563cbac88c |
refactor(storage,processing): kind registries + versioned data migrations
Two CRITICAL data-safety bugs from the architecture audit and the two
worst parallel-change problems are fixed in one coherent pass.
Audit findings addressed:
- C2 silent CSS response fallback. The previous _RESPONSE_MAP fell
through to a fabricated PictureCSSResponse whenever a source
class lacked an entry; in particular game_event sources were
silently mis-shaped. Now: GameEventCSSResponse/Create/Update
schemas exist, _RESPONSE_MAP is re-keyed by source_type string,
an import-time _assert_response_map_coverage() requires symmetric
agreement with storage._SOURCE_TYPE_MAP, and the runtime path
raises instead of fabricating a response.
- C11 string-replace JSON migration. ColorStripStore used
blob.replace('"source_type": "static"', '"source_type":
"single_color"') which can corrupt unrelated substrings (e.g.
an animation type named "static_wave") and provides no audit,
no transaction, no idempotency. Replaced with
storage.data_migrations.MigrationRunner backed by a
data_migrations audit table. Each migration runs inside one
db.transaction() that covers the applied-check, the apply(),
and the audit-INSERT — partial failures roll back atomically.
StaticToSingleColorMigration parses each row with json.loads
and mutates only the source_type field. Frozen-write databases
skip with a warning.
- C3+C4 color-strip stream dispatch. The 7-branch elif in
ColorStripStreamManager.acquire() and the duplicate one in
ws_stream._create_stream() now share a single STREAM_BUILDERS
registry in core.processing.color_strip_kinds, keyed by
source.source_type. Both call sites populate a StreamDeps bag
and delegate to build_stream(). _assert_stream_kind_coverage()
asserts at import that STREAM_BUILDERS plus SHARABLE_KINDS
partitions storage._SOURCE_TYPE_MAP. ws_stream's preview path
wraps each FastAPI-DI getter in _safe() so non-audio previews
no longer crash when audio/CSPT stores are not wired.
- C6+C7 value stream dispatch. The 14-branch isinstance ladder in
ValueStreamManager._create_stream and its silent
StaticValueStream(value=1.0) fallback are replaced by
core.processing.value_kinds.STREAM_BUILDERS, keyed by
source_type string (so AdaptiveValueSource's adaptive_time and
adaptive_scene route to different builders correctly). The
manager retains only the SyncClockRuntime pre-acquisition step
for animated_color (kinds needing this are listed explicitly
in NEEDS_CLOCK_RUNTIME). Symmetric coverage assertion plus a
separate assertion that NEEDS_CLOCK_RUNTIME is a subset of the
registry.
Bundled in: the static->single_color rename plus the HTTPValueStream
/ http_endpoint introduction that were already in flight on this
branch share these files; the registry refactor naturally absorbs
both via the new "single_color" / "static" alias entries and the
_build_http builder.
Tests: 26 new tests cover response-map coverage drift, migration
runner audit-table mechanics + transactional rollback +
frozen-write skip, and the two stream-builder registries. 343
existing storage / API / e2e tests stay green. Ruff clean.
|
||
|
alexei.dolgolyov
|
e24f9d33cc |
fix(shutdown): survive PC restart with WAL fsync + Win32 session-end guard
Two bugs caused user data ('G502' target's color-strip ref, etc.) to
revert after PC restart while persisting fine across normal app
restarts:
1. SQLite was in WAL mode with synchronous=NORMAL and Database.close()
was never called. On graceful Python exit the sqlite3 finalizer
checkpoints the WAL, but on an unclean PC shutdown (power loss,
forced reboot, or Windows force-terminating pythonw.exe) the WAL
stayed in OS cache, never reached disk, and the next boot rolled the
DB back to the last checkpoint -- losing recent edits.
2. Nothing handled WM_QUERYENDSESSION / WM_ENDSESSION, so on PC
shutdown Windows force-killed pythonw.exe after ~5s and the FastAPI
lifespan never ran. The 'stop_targets' setting was silently ignored
and devices were left at their last frame.
Changes:
- Database: PRAGMA synchronous=FULL + wal_autocheckpoint=100, plus an
explicit wal_checkpoint(TRUNCATE) inside Database.close().
- New utils/win_shutdown.py: hidden top-level window in a daemon thread
with a ctypes WindowProc that catches WM_QUERYENDSESSION (calls
ShutdownBlockReasonCreate to extend Windows' 5s hung-app timeout up
to the ~20s GUI ceiling), fires the shutdown callback, then waits in
WM_ENDSESSION on a completion event before returning. Also raises
the process shutdown priority via SetProcessShutdownParameters. All
Win32 argtypes/restypes are bound once at import to avoid LPARAM
overflow on x64.
- New shutdown_state.py: leaf module owning the cross-thread Event so
__main__ does not import the heavy ledgrab.main at startup.
- main.py lifespan: per-step asyncio.wait_for budgets (8s for
processor_manager.stop_all, 1.5s each for HA/MQTT, etc.) so a hung
device cannot starve the DB checkpoint, then db.close() and
shutdown_complete.set() always run.
- __main__.py: install the Windows shutdown guard before tray start;
install SIGINT/SIGTERM/SIGBREAK handlers only on the tray path
(uvicorn overwrites them on no-tray); raise server_thread.join to 20s.
- Tests cover WM_QUERYENDSESSION (fires callback, returns TRUE,
idempotent), WM_ENDSESSION (waits on event, times out cleanly,
cancel-path returns instantly), signal handler installation, and
that main and shutdown_state share the same Event instance.
|
||
|
alexei.dolgolyov
|
0e3ae78de7 |
fix(devices): address pre-merge review findings
Closes the issues surfaced by the pre-merge code review of the expand-device-support branch. CRITICAL #2 -- update_device double-encrypts secrets in memory. storage/device_store.py round-tripped through device.to_dict() which encrypts hue_username / hue_client_key / ble_govee_key / nanoleaf_token via _enc(), but Device.__init__ does not decrypt. The cached self._items[device_id] thus held ciphertext where plaintext belonged, breaking runtime auth for paired devices on any update -- even an innocuous rename. Sourcing kwargs from vars(device) directly avoids the round-trip. Regression tests cover Nanoleaf and Hue. HIGH #3 -- secrets leaked in GET /api/v1/devices response. DeviceResponse previously returned nanoleaf_token / hue_username / hue_client_key in plaintext (decrypted server-side from storage), defeating the encryption-at-rest. Replaced with nanoleaf_paired and hue_paired booleans. ble_govee_key intentionally stays -- it's a user-managed value pasted from a third-party tool, must remain visible for edit. Frontend types.ts + the one nanoleaf_token reader updated to the boolean. HIGH #4 -- SSRF surface. validate_lan_host() added to net_classify.py; called from each new driver's validate_device (DDP / Yeelight / WiZ / LIFX / Govee / OPC / Nanoleaf) and from pair_device. Rejects literal public IPs with a descriptive ValueError; non-IP hostnames pass through (mDNS labels, bare hostnames). RFC6890 ranges (documentation, former class E) are accepted as LAN-like since Python's ipaddress.is_private treats them so -- correct policy for LedGrab. HIGH #5 -- decrypt failure deletes the device row. _dec() now catches the exception, logs an error, and returns "" instead of propagating. Without the fix, a regenerated data/.secret_key would silently make every Hue / Nanoleaf / BLE-Govee device disappear from the device list on next startup. Regression test asserts a corrupt envelope leaves the device hydratable. HIGH #6 -- update_device route does not rstrip("/") for non-WLED. Moved the trim before the WLED-specific scheme inference so every device type gets consistent URL normalization between create and update. MEDIUM #7 -- Govee discovery port 4002 collision. Added a lazily- initialized module-level asyncio.Lock that serializes concurrent discover_govee_devices() calls; the previous behavior had the second parallel scan silently return [] when the first still held port 4002. Error message also clarified to mention another Govee tool. MEDIUM #8 -- Nanoleaf discover() leaked browser tasks on cancellation. Moved the browser cancel loop into the finally block so an interrupted mDNS scan still tears them down. MEDIUM #9 -- pair endpoint logged user-supplied URL with exc_info=True. Added _sanitize_url_for_log() that strips userinfo + fragment, and demoted the log from exc_info to type(exc).__name__ + str(exc) so a hostile receiver's response body can't end up in the log file. LOW -- Nanoleaf was the only client without a .port property. Added one (returns NANOLEAF_PORT, fixed) for cross-driver symmetry. LOW -- no end-to-end pair-then-create coverage. Added TestPairThenCreateFlow.test_pair_then_create_persists_encrypted_token which exercises the full path: POST /api/v1/devices/pair returned fields, store.create_device, then asserts (a) in-memory plaintext, (b) to_config() plaintext, (c) persisted ciphertext, (d) API response strip + paired-boolean. Tests: 1379 pass (was 1358 -- 21 new regression tests added). ruff clean. TypeScript clean. |
||
|
alexei.dolgolyov
|
7736bc6f58 |
fix(utils): commit url_scheme + net_classify dependencies
The DDP commit (
|
||
|
alexei.dolgolyov
|
426484adf8 |
feat(devices): Nanoleaf OpenAPI target type + first pair-flow user
Adds support for Nanoleaf controllers (Light Panels / Canvas / Shapes /
Lines / Elements) via the documented HTTP REST API on port 16021.
First concrete consumer of the pair-UX scaffold from commit
|
||
|
alexei.dolgolyov
|
2f31680823 |
feat(devices): pairing-UX scaffold (Phase 2)
Lays the groundwork for device families that require a one-time
physical pairing action (Nanoleaf hold-power-button, Tuya local-key
extraction, Twinkly network-setup mode, Hue link-button). No driver
uses it yet -- Nanoleaf will be the first concrete consumer.
Phase 2 as originally written had three bullets; only this one was
genuinely missing work. The other two (generic NetworkDiscoveryService
fan-out, unified scan-network UI) were already solved at the route
level by the existing /api/v1/devices/discover handler running all
providers in parallel via asyncio.gather(return_exceptions=True).
Marked WONTDO in TODO.md with rationale.
Backend:
- LEDDeviceProvider gains an async pair_device(url) -> dict method.
Default raises NotImplementedError so missing implementations on a
requires_pairing provider fail loud at request time.
- New PairingNotReady exception, distinct from generic errors so the
route handler can return 409 (user must perform the physical action,
retry possible) instead of 500.
- POST /api/v1/devices/pair endpoint with PairDeviceRequest /
PairDeviceResponse schemas. Status-code mapping:
200 -> paired, fields returned for the subsequent create payload
400 -> unknown device type, or type doesn't support pairing
409 -> PairingNotReady (retryable from the UI)
422 -> invalid URL / device configuration (ValueError)
502 -> transport / network failure (other exceptions)
500 -> provider returned a non-dict (defensive)
- 8 route tests register a stub provider and exercise every
status-code path.
Frontend:
- New modals/pair-device.html with five state blocks (idle / pairing
/ not_ready / success / failed) toggled via data-pair-state, plus
a 30-second SVG progress ring with monospace countdown.
- New features/pairing-flow.ts exposing
runPairingFlow({deviceType, url, instructionsKey?}) ->
Promise<{fields: Record<string, unknown>>. Wires the modal to the
pair endpoint, maps response codes to UI states, AbortControllers
in-flight fetches on cancel. Exports a PairingCancelled sentinel
error class.
- Generic pairing.* i18n keys in en/ru/zh. Drivers will add their own
device.<type>.pair.instructions key that overrides the default.
Design decisions (per frontend-design skill):
- Single SVG ring + centered countdown (HomeKit-style)
- Instructions stay visible during pairing, dimmed to 60% via :has()
- Success state held 450 ms before auto-dismiss
- Cancel-X in the footer; primary action lives in the state block
- prefers-reduced-motion disables pulse/fade/ring transitions
Note: the components.css diff includes a pre-existing MiniSelect block
from the user's parallel work; pairing-specific styles are the second
hunk (lines ~1628+).
|
||
|
alexei.dolgolyov
|
31c6c3abb2 |
feat(devices): Open Pixel Control (OPC) target type
Adds support for Open Pixel Control receivers (Fadecandy boards, xLights/Falcon endpoints, OPC bridges, art-installation controllers, hobbyist LED driver software). OPC is a tiny TCP protocol on port 7890 with a 4-byte header [channel][cmd][len_hi][len_lo] + RGB body. Backend: - OPCClient opens one persistent TCP connection and streams frames as header+body byte pairs. Channel 0 broadcasts to every output on the OPC server; channels 1-255 address a specific channel on multi-output servers (Fadecandy with multiple Open Pixel chains). - supports_fast_send=True with a synchronous send_pixels_fast hot path. The fast path skips the async drain so the OS write-buffer flushes on its own schedule -- exactly what ambilight streaming wants. - Brightness applies client-side before the frame is sent (OPC has no reply channel for hardware-side brightness). - Health check opens a TCP connection and closes it. - OPCConfig joins the typed config union; storage gains an opc_channel field; full to_dict/from_dict/to_config wiring. - 36 unit tests cover URL parsing, header construction, send_pixels emitting header+body in order, brightness application, list and flat-array input shapes, drain behavior, connection lifecycle, provider validate/discover/capabilities, Device.to_config round-trip. Frontend: - 'opc' in DEVICE_TYPE_KEYS (next to 'ddp'), paper-plane icon -- same as DDP since both are open pixel-streaming protocols. - isOpcDevice predicate + per-type field show/hide. - Optional channel number input (default 0 = broadcast) with hint copy explaining the channel semantics. - Locale strings in en/ru/zh. No native discovery (OPC has no discovery protocol); users supply the receiver IP manually. |
||
|
alexei.dolgolyov
|
887131d4af |
feat(devices): Govee LAN target type
Adds support for Govee Wi-Fi smart bulbs and ambient-lighting kits via their LAN API (opened in 2023). Discovery is multicast UDP on 239.255.255.250:4001; control commands go unicast to the device's port 4003; responses arrive on port 4002. Each device requires "LAN Control" toggled ON in the Govee Home app (Device -> settings -> LAN Control). Devices with LAN Control disabled silently fail to appear in discovery and won't respond to commands; the UI hint copy reminds users. Backend: - GoveeClient is a single-pixel UDP adapter: averages the strip to one RGB triple and pushes a 'colorwc' command with colorTemInKelvin=0 to select pure RGB mode (non-zero kelvin would switch the bulb to CCT mode and ignore the RGB values). - Brightness folds into the RGB scaling so we burn one packet per frame instead of two. - supports_fast_send=True with a synchronous send_pixels_fast hot path. Default rate gate 50 ms (~20 Hz); UDP fire-and-forget tolerates it. - Multicast discovery: scan request to 239.255.255.250:4001, listen on port 4002, parse the inner data dict for IP + device-id + SKU + firmware version. Degrades to [] when port 4002 is already bound or network is unavailable. - Health check sends devStatus and waits 1.5s for any reply; the error message points at the LAN-Control toggle since that's the #1 root cause of silent failures. - GoveeConfig joins the typed config union; storage gains govee_min_interval_ms; full to_dict/from_dict/to_config wiring. - 40 unit tests cover URL parsing, scan-reply parsing (rejecting non-scan commands and malformed JSON), payload builders (colorwc with colorTemInKelvin=0, brightness clamping, power as 1/0 not true/false), strip averaging, rate limiting, fast-send hot path, provider validate/discover/health, Device.to_config round-trip. Frontend: - 'govee' in DEVICE_TYPE_KEYS (next to 'lifx'), lightbulb icon (deliberate smart-bulb family grouping). - isGoveeDevice predicate + per-type field show/hide. - Rate-limit number input (default 50 ms). - URL hint copy explicitly instructs users to enable LAN Control in the Govee Home app -- the #1 source of "why isn't my Govee responding?" support churn. - Locale strings in en/ru/zh. |
||
|
alexei.dolgolyov
|
8f9d490063 |
feat(devices): LIFX LAN target type
Adds support for LIFX smart bulbs and lightstrips that speak the LIFX binary UDP protocol on port 56700, with broadcast LAN discovery via the standard GetService/StateService probe. Backend: - LIFXClient is a single-pixel UDP adapter: averages the strip to one RGB triple, converts to LIFX HSBK (16-bit hue/saturation/brightness + kelvin), and pushes a tagged SetColor packet so all bulbs on the subnet act on it. Brightness folds into the HSBK brightness channel. - Hand-rolled packet builder: 36-byte LIFX header (frame + frame-address + protocol-header) + variable-length payload. Source ID 'LGGR' identifies LedGrab in protocol logs. - supports_fast_send=True with a synchronous send_pixels_fast hot path -- UDP costs nothing, so the default rate gate is 50 ms (~20 Hz) to match LIFX's documented <=20 cmd/sec recommendation. - Broadcast discovery sends GetService and parses StateService replies back into IP + MAC + service-port triples. Broadcast failures yield [] rather than raising. - Health check sends GetService and waits 1.5s for any reply on a one-shot UDP socket. - LIFXConfig joins the typed config union; Device storage gains a lifx_min_interval_ms field; full to_dict/from_dict/to_config wiring. - 47 unit tests cover URL parsing, RGB->HSBK conversion (red/green/ blue/white/black/clamping), packet construction (size, msg type, tagged flag, target MAC, sequence byte), SetColor and SetPower payload layouts, StateService reply parsing (including rejection of wrong msg types and runt payloads), strip averaging, rate limiting, fast-send hot path, provider validate/discover/health, and Device.to_config round-trip. Frontend: - 'lifx' in DEVICE_TYPE_KEYS (next to 'wiz'), lightbulb icon (deliberate smart-bulb family grouping with Hue + Yeelight + WiZ). - isLifxDevice predicate + per-type field show/hide in create and settings modals. - Rate-limit number input (default 50 ms) in both modals with hint text referencing LIFX's documented <=20 cmd/sec ceiling. - Locale strings in en/ru/zh. LIFX bulbs are reachable from the existing "Scan network" button -- no new discovery UI affordance was needed. No brightness_control capability exposed; LIFX brightness is folded into the HSBK on the wire. |
||
|
alexei.dolgolyov
|
ede627b4ac |
feat(devices): WiZ Connected LAN target type
Adds support for WiZ Connected (Philips' budget-tier) smart bulbs that accept JSON commands as UDP datagrams on port 38899 with broadcast LAN discovery on 255.255.255.255:38899. Backend: - WiZClient is a single-pixel UDP adapter: averages the incoming strip to one RGB triple and pushes it via setPilot with r/g/b params. Brightness folds into the RGB scaling so we burn one packet per frame instead of two. - UDP fire-and-forget tolerates high update rates with no ack overhead, so the default rate gate is 50 ms (~20 Hz) -- 10x faster than Yeelight. - supports_fast_send=True with a synchronous send_pixels_fast hot path. - Broadcast discovery sends the standard registration envelope; bulb replies are parsed for IP+MAC and surfaced as DiscoveredDevice entries. Broadcast failures (no network, firewall) yield [] rather than raising. - Health check sends getPilot and waits 1.5s for any reply on a one-shot UDP socket. - WiZConfig joins the typed config union; Device storage gains a wiz_min_interval_ms field; full to_dict/from_dict/to_config wiring. - 36 unit tests cover URL parsing, MAC extraction, strip averaging, rate limiting, fast-send hot path, provider validate/discover/health, and Device.to_config round-trip. Frontend: - 'wiz' in DEVICE_TYPE_KEYS (next to 'yeelight'), lightbulb icon (deliberate smart-bulb family grouping with Hue + Yeelight). - isWizDevice predicate + per-type field show/hide in create and settings modals. - Rate-limit number input (default 50 ms) in both modals with hint text noting the UDP fire-and-forget characteristic. - Locale strings in en/ru/zh. WiZ bulbs are reachable from the existing "Scan network" button -- no new discovery UI affordance was needed. |
||
|
alexei.dolgolyov
|
4b65005823 |
feat(devices): Yeelight LAN target type
Adds support for Xiaomi/Yeelight smart bulbs and lightstrips that speak the bulb-vendor's JSON-RPC protocol over TCP port 55443 with SSDP-style LAN discovery on 239.255.255.250:1982. Backend: - YeelightClient is a single-pixel adapter: it averages the incoming strip down to one RGB triple, packs it into the 24-bit color int the bulb expects, and pushes it via set_rgb with sudden+0ms effect. - Brightness folds into the RGB scaling on the wire so we burn one command per frame instead of two. - A configurable client-side rate gate (yeelight_min_interval_ms, default 500) keeps us under the bulb's ~1 cmd/sec cap. Frames that arrive inside the gate no-op without TX. Music mode (~60 Hz via reverse-TCP) is deferred -- the MVP caps at ~2 Hz and that's fine for a strip-to- single-pixel averaging device. - SSDP discovery scans 239.255.255.250:1982 with the bulb-specific ST: wifi_bulb header; replies are parsed into DiscoveredDevice entries. Multicast failures (no network, firewall) yield [] rather than raising -- discovery is best-effort. - Health check opens a TCP socket to the bulb and closes it. - YeelightConfig joins the typed config union; Device storage gains a yeelight_min_interval_ms field; full to_dict/from_dict/to_config wiring. - 34 unit tests cover URL parsing, RGB packing, strip averaging, rate limiting, SSDP response parsing, provider validate/discover/health, and Device.to_config round-trip. Frontend: - 'yeelight' in DEVICE_TYPE_KEYS (next to 'hue'), lightbulb icon (intentional family-grouping signal with Hue). - isYeelightDevice predicate + per-type field show/hide in create and settings modals. - Rate-limit number input (default 500 ms) in both modals with hint text explaining the trade-off. - Locale strings in en/ru/zh. - Drive-by: types.ts DeviceType union backfilled with 'ddp' and 'ble' for type-safety consistency. Yeelight bulbs are now reachable from the existing "Scan network" button -- no new discovery UI affordance was needed. |
||
|
alexei.dolgolyov
|
8f1140abad |
feat(devices): standalone DDP target type
Promotes the existing DDP packet layer (previously WLED-internal) to a
first-class device type so any DDP-speaking receiver (Pixelblaze,
ESPixelStick, xLights/Falcon endpoints, generic firmware) can be driven
directly without WLED in the path.
Backend:
- New DDPLEDClient wraps the DDPClient transport as a proper LEDClient
with supports_fast_send=True (synchronous UDP push on the hot loop).
- New DDPDeviceProvider — no native discovery, manual LED count,
capabilities = {manual_led_count, health_check}.
- DDPConfig joins the typed config union; Device storage gains
ddp_port / ddp_destination_id / ddp_color_order fields with safe
defaults (0/1/1 -> port 4048, destination 1=display, RGB byte order).
- URL scheme: ddp://host[:port] or bare host[:port] (default 4048).
- Health check resolves the host via async DNS; UDP has no reply
channel so reachability is best-effort by design.
- 29 new tests in test_ddp_led_client.py cover URL parsing, packet
hot path (brightness, list/numpy input shapes, fast vs async send),
provider validate/discover/capabilities, config round-trip via
Device.to_config() and to_dict/from_dict.
Frontend:
- 'ddp' in DEVICE_TYPE_KEYS (next to 'dmx'), paper-plane icon.
- isDdpDevice predicate + per-type field show/hide in the create &
settings modals.
- Color-order picker uses IconSelect (project rule bans plain select).
- Locale strings added in en/ru/zh.
Note: this commit also carries two pre-existing in-flight hunks that
were intermixed in the same files and could not be split out
non-interactively:
- api/routes/devices.py: URL-scheme inference for bare WLED hosts,
safer error messages, exception-isolated parallel discovery.
- storage/device_store.py: secret_box helpers + at-rest encryption of
Hue / BLE-Govee / MQTT credentials.
Both are independent of DDP and intentional per the user.
|
||
|
alexei.dolgolyov
|
530316c2c3 |
feat(mqtt): multi-broker MQTT + Zigbee2MQTT light target
- New Z2MLightOutputTarget storage, processor, editor and routes for Zigbee2MQTT light entities (shares the HA-Light editor UI via the new light-target-editor module) - Replace global MQTTService/MQTTConfig with per-source MQTTManager + MQTTRuntime; thread mqtt_source_id through Z2M targets, DIY MQTT devices, and the automation engine - Migrate legacy single-broker YAML/env config to a "Default Broker" MQTTSource on startup (core/mqtt/legacy_migration.py) and drop the obsolete core/mqtt/mqtt_service.py - Refresh /api/v1/system integration status to surface every MQTT source - Extract shared light-target editor and refactor OutputTargetStore + output_targets routes around typed factories / auto-registry - Modal CSS polish, locale strings, and storage/bindable test coverage |
||
|
alexei.dolgolyov
|
ad84b60ae4 |
fix(ha-light): apply brightness_scale once and respect boost multipliers
`_send_entity_color` was multiplying the per-mapping `brightness_scale` into the brightness payload twice when the effective scale was below 1, yielding a quartered output for a configured half-scale. Conversely, when the value-stream multiplier exceeded 1.0 with a default scale, the entire scaling step was skipped and the boost was lost. Compute brightness as `clamp(max(r,g,b) * bs * vs, 0, 255)` once and ship it directly, with regression tests pinning the half-scale, boost, and 255-clamp cases. |
||
|
alexei.dolgolyov
|
75ca487be1 |
feat(ui): per-surface card presentation modes (C/M/D/R)
Adds a comfortable/compact/dense/row toggle to every card grid in the
app. Each surface (LED devices, targets, automations, scenes, sources,
streams, dashboard subsections, etc.) remembers its mode independently.
Persistence mirrors dashboard-layout: localStorage cache for first paint,
debounced PUT to /api/v1/preferences/card-modes (new endpoint) for
cross-browser sync. Surface registry is open — any non-empty key
accepted server-side; modes validated against {comfortable, compact,
dense, row}.
CSS is token-driven: grid min-width and gap come from --card-grid-min /
--card-grid-gap / --card-grid-min-narrow / --card-grid-gap-narrow /
--templates-grid-min / --templates-grid-gap defined on :root, overridden
per [data-card-mode]. Dense/row also hide .mod-leds, collapse secondary
button labels, and tighten .mod-metrics; row collapses the grid to one
full-width column. Coexists with the existing per-section [data-density]
on the dashboard tab — different attribute, additive concern.
Toggle UI auto-mounts into every CardSection header (18+ surfaces) plus
the six dashboard subsections via post-render mount; teardown tracking
keeps the listener Set bounded across re-renders.
i18n: card_mode.{tooltip,comfortable,compact,dense,row} in en/ru/zh.
Tests: 9 new cases in tests/test_preferences_card_modes_api.py covering
defaults, round-trip, validation, open-registry keys, row mode, delete.
|
||
|
alexei.dolgolyov
|
a79f4bf73c |
feat(ha-light): broadcast a single Color Value Source to all entities
HALightOutputTarget gains a `source_kind` field with two modes: - `css` (existing): per-mapping LED segments averaged from a ColorStripSource. - `color_vs` (new): one colour from a colour-returning ValueSource pushed to every mapped entity (mapping LED ranges are ignored in this mode). Backend wiring: - Schema/route: add `source_kind` + `color_value_source_id` to create/update/ response payloads, with VS existence + return_type=color validation. - Storage: persist new fields, with defensive `or ""` coalesce so legacy rows written via resolve_ref with None survive the str-typed response schema. - Processor: ha_light_target_processor reworked to drive both source kinds (incl. update_target_settings hot-swap of source mode). New unit tests in tests/core/test_ha_light_target_processor.py and extended store tests. Frontend: - ha-light editor modal: collapsed Color Strip + Color VS into one "Color Source" picker with grouped headers; mappings list shows a mode-aware hint when broadcasting a single colour. - EntityPalette: support non-selectable header rows (with keyboard / filter handling) for grouped source pickers. Bundled UI polish (icon inheritance + cleanup): - Custom card icons now flow into more surfaces: command palette, dashboard target cards, scene-preset target picker, calibration test-device picker, and the LED-target device picker. LED targets inherit their device's icon when none is set on the target itself. - Empty mod-card icon plates render as a dashed "+" placeholder when an icon-picker hook is wired, so the action stays discoverable. - Icon picker: distinct "HA light target" eyebrow label and supports HA-light cards (data-ha-target-id) for channel-colour resolution. - Update banner: "View release" now opens the in-app Update settings tab instead of an external link; uses the sparkles icon. - Color-strip delete: cleaner toast on 409 conflict. |
||
|
alexei.dolgolyov
|
fdac26b9d9 |
feat: daylight tz, camera engine, value stream + modal/UI polish
- daylight: new daylight_settings module + daylight-tz frontend helper; expanded daylight_stream behavior - camera engine: capture path additions plus new test_camera_engine suite - value stream: schema + processing updates (~178 lines) - color strip: drop cycle effect (cycle.py / color-cycle.ts removed), tighten static path - modal CSS: large refactor (+883), components.css polish (+110) - templates: settings, css-editor, value-source-editor, test-template, display-picker, image-lightbox - frontend core: state, modal, icons, graph-nodes, app - frontend features: displays, streams, streams-capture-templates, value-sources, settings, color-strips/cards - locales: en/ru/zh - storage: color_strip, picture_source, value_source loaders touched - preferences/sync_clocks/picture_sources routes; home_assistant + templates schemas |
||
|
alexei.dolgolyov
|
797b806972 |
feat: LED hot-path perf, tutorials expansion, modal markup polish
Performance (LED hot path, allocation-free per-frame): - Adalight: dedicated single-worker tx executor (avoids asyncio.to_thread overhead), pre-allocated wire buffer + uint8 scratch, header struct precomputed. New tests cover header format, buffer reuse, non-contiguous input, and brightness scaling. - DDP: pre-built struct.Struct for the 10-byte header, allocation-free send buffer + memoryview emit path. New tests cover RGB/RGBW packets, sequence/PUSH semantics, and multi-packet fragmentation. - Calibration: precomputed Phase 3 skip-LED resampling (floor/ceil indices, fractional weights, take/blend scratch buffers) — per-frame work is now np.take + in-place blend, no allocations. - WLED target processor: matching hot-path tightening. Tutorials: - Sub-tab switching, breadcrumb header, and prepare/switchSubTab hooks so a tour can open/close the dashboard customize panel and resolve targets behind sub-tabs. - New steps for integrations tab, dashboard customize panel (presets, global, sections, perf cells), targets, scenes, sync-clocks. - en/ru/zh locales updated with the new tour strings. Dashboard layout: - Structural deep-equal so the "modified" indicator reflects truth after a user edits then reverts, instead of a stale flag. UI polish: - Mod-card / modal markup pass across ~33 modal templates and the tutorial overlay partial. - appearance.css, modal.css, tutorials.css refresh. Tooling: - Add .mcp.json with code-review-graph MCP server config so the graph tools are available to the team out of the box. |
||
|
alexei.dolgolyov
|
9d4a534ec6 |
feat(ui): release notes overlay v2 + settings/streams/dashboard polish
Release Notes overlay redesign (scoped via .release-notes-shell)
- Backend exposes release.assets (name/size/download_url) through
UpdateReleaseInfo so the frontend can render real download links.
- New masthead: eyebrow + display-font title + tag/published/pre-release
chip strip + close/external action buttons; opts out of layout.css's
global `header { height: 60px }` and `header::before` accent bar that
were leaking into the overlay's <header>.
- Markdown body: <code> filenames are wrapped in clickable <a> via fuzzy
asset match (exact basename, then same-extension token-overlap), with
per-asset description tooltip and a small download glyph.
- Per-asset description derived from filename pattern (Windows installer
/portable/msi, Linux tarball/AppImage/deb/rpm, macOS dmg/pkg, Android
apk/aab, iOS ipa, generic archives) with i18n keys in en/ru/zh.
- Hide checksum / signature side-files (.sha256/.sha512/.sig/.asc/...).
Settings modal & dashboard polish
- ds-section refresh, rail-channel routing, notif matrix updates.
- Dashboard customize panel + per-account layout updates.
- New docs/settings-modal-redesign.html design reference.
Streams / targets / color-strip
- Stream cards rewrite (cards.css, streams.css, streams.ts).
- Composite stream + metrics history adjustments.
- WLED target processor + color-strip pipeline refinements.
- Color-strip WS source streamer touch-ups.
Misc
- Perf charts overhaul; tabular game-integration / HA / MQTT / weather
source cards; donation/sync-clocks/scene-presets minor polish.
- New i18n keys across en/ru/zh.
Test infrastructure
- conftest pre-creates the test DB so main.py's legacy-data migration
doesn't shovel the user's production DB into the test temp dir.
- test_preferences_notifications wipes its own setting at the start of
the defaults test (was relying on isolation it never enforced).
Pre-commit gates: ruff clean, tsc clean, npm run build clean,
pytest 899/899 passing.
|
||
|
alexei.dolgolyov
|
8aa3a323d6 |
feat(notifications): device event notifications (snack + Web Notifications)
Surface device connection state changes (configured target online/offline) and discovery events (new WLED on LAN, new serial port, devices that disappear) through a configurable per-event channel matrix: none / snack / OS / both. - Backend: long-running mDNS browser + 10 s serial poller in core/devices/discovery_watcher.py, gated by user pref. Reuses the existing device_health_changed event for online/offline transitions. New GET/PUT /api/v1/preferences/notifications endpoint with Pydantic v2 schema (channel matrix + background-discovery flag + grace/debounce). 13 new tests, full suite still 899 passing. - Frontend: features/notifications-watcher.ts with startup-grace + flap-debounce + bulk-coalesce pipeline. Web Notifications API for the OS channel (no platform-specific code, works in PWA shell). New "Notifications" tab in Settings with 4 IconSelect rows + bg toggle + permission row + test button. en/ru/zh translations. Defaults: device_offline=both (urgent), online/discovered=snack, lost=none, background discovery on. Already-configured devices are filtered from discovery events to avoid double-notifications. |
||
|
alexei.dolgolyov
|
80f01d4813 |
chore: harden test isolation, gitignore stale src/data, mark shutdown action done
- ``tests/test_preferences_api.py`` no longer captures the auth API key at module-import time. The new ``client`` fixture resolves it inside its body and bakes the Bearer header into ``TestClient.headers``, so the e2e conftest swapping the global config singleton during collection cannot leave the test holding a stale 401-bound header. Same proven pattern as ``test_audio_processing_templates_api.py``. - ``.gitignore`` now anchors ``/server/src/data/`` defensively. If the server is launched from ``server/src/`` (uncommon but possible during ad-hoc debugging), its relative ``data/`` resolves there. Templates now live in SQLite (``capture_templates`` / ``pattern_templates`` / ``postprocessing_templates`` tables); any stale ``*.json`` that lands in that directory is a runtime export and must not be committed. - Three such stale exports were untracked at the start of the pre-merge audit and have been deleted from the working tree. - ``TODO.md`` flips the shutdown-action checklist to done and notes that real-hardware verification (WLED + serial after Ctrl+C) is still pending. |
||
|
alexei.dolgolyov
|
56853b7123 |
feat(dashboard): per-account customizable dashboard with slide-in panel
Open-registry section/perf-cell schema persisted server-side under
db.get_setting('dashboard_layout'); localStorage cache for instant
first-paint, server sync after auth. 5 built-in presets
(Studio/Operator/Showrunner/Diagnostics/TV); JSON export/import.
Slide-in Customize panel toggles section + perf-cell visibility,
reorders via hand-rolled HTML5 drag (with up/down buttons for
keyboard/TV-remote use), changes density per section, and exposes
global Width / Animations / Perf-mode / Window with per-cell Inherit
overrides.
Window setting now drives the actual sparkline slice (30s/1m/2m/5m at
configurable poll interval) instead of always rendering 120 fixed
samples. Perf-grid edits re-render in place — sparklines repaint from
persistent module-level history, value labels replay from cached
last-fetch payload, so there is no flicker frame and no zero-data
window between layout change and next poll. initPerfCharts now fires
an immediate fetch on init so reload no longer shows "—" until the
first interval tick.
Reset confirmation uses the project's themed showConfirm modal
instead of the browser dialog. Reserved registry keys (audio-meters,
alerts, led-preview, source-thumbs, pinned, flow) are forward-
compatible so v1.1 cards slot in without a schema bump.
Backend exposes GET/PUT/DELETE /api/v1/preferences/dashboard-layout
treating the body as opaque JSON with a numeric version gate; covered
by 6 round-trip / validation / unknown-field tests.
|
||
|
alexei.dolgolyov
|
e5a2af9821 |
feat(ui): dashboard polish, richer perf strip, transport-bar controls
Dashboard perf strip:
- Unified rack-module shell with hairline-divided cells (mockup parity)
replacing 3 separate perf cards. Cells auto-wrap to 2 rows of 4 on
widescreen; responsive breakpoints at 1100 / 760 / 480 px.
- Active Patches cell (first) shows running/total channel count plus up
to 4 live FPS readouts with channel-colored stripes; bottom-right
radial glow anchors the "live channel bank" corner.
- Total FPS cell — aggregate throughput across running targets, mono
"fps" unit suffix, session-peak-scaled sparkline with a 60 FPS floor.
- Devices cell — online/total count + per-device dot strip (green when
online with signal-glow, coral when offline, tooltip with name +
latency), fed from /devices/batch/states (added to the dashboard
batch poll).
- Value font uses clamp(1.8rem, 2.8vw, 2.8rem) + white-space: nowrap so
long readouts (RAM "18.9/31.8 GB", GPU "50% · 37°C") scale down
instead of wrapping.
- Sparklines anchor to the cell bottom via margin-top: auto so baselines
align across cells regardless of subtitle presence.
- App-load tag ("APP 3.1%") moved to a pinned top-right position per
card, accent-colored pill; replaces the subdued inline badge.
- Perf mode toggle (System / App / Both) triggers an immediate poll so
positioning updates without waiting for the next tick.
- Chart.js removed from perf-charts — inline SVG sparklines with
drop-shadow filter for the "lit instrument" feel. Chart.js still used
for per-target FPS charts via chart-utils (now owns the registration).
- Fixed history seed bug: app_ram is MB in the server history payload,
not percent — convert to percent using sample's ram_total before
pushing into _appHistory.ram. Skip seeding app_gpu_mem since the
history schema has no gpu_memory_total.
- Temperature card reveals with an explanatory hint when the backend
reports cpu_temp_hint_key (e.g. Windows without LibreHardwareMonitor)
instead of silently hiding; .perf-chart-card-hint neutralizes the big
display font so the message reads as plain body copy.
Transport bar:
- LED brand mark — 28 px, double-layer signal glow (0 22px + 0 8px),
brandPulse animation. Brand-stack wraps the title + version so
"LED GRAB" sits above "V0.3.0" on a single line each.
- Transport status chip — bigger (9/18 padding), mono uppercase,
inner+outer signal glow when .is-armed.
- Transport meta cells — Uptime (JS-local session ticker), CPU (app
CPU share), Mem (app RAM, G/M format) as stacked KEY/VALUE mono
readouts with hairline separators.
- New interactive Poll cell cycles through 1/2/5/10s presets on click;
replaces the range slider that used to live in the Dashboard toolbar
(it controlled the whole app, not just the Dashboard).
- Header icon buttons — hairline-bordered 30 px squares with channel-
glow on hover, replacing the pill container.
- Perf poll moved to global bootstrap so transport CPU / Mem stay live
across all tabs (was paused when leaving the Dashboard).
- Connection pip (#server-status) hidden; the brand mark itself turns
coral when offline via :has() selector on .header-title.
Dashboard cards:
- renderDashboardTarget now emits full rack-module markup with CH badge,
name, meta, LED cluster, 3-cell metric grid (FPS / Uptime / Errors),
and patch-label + stop button. Running cards get the signal-flow
strip at the bottom. data-fps-text / data-uptime-text / data-errors-
text hooks preserved so _updateRunningMetrics updates in place.
- LED count surfaced in the target card meta line (e.g. "LED · WLED ·
144 LED · GRADIENT") when the linked device reports led_count > 0.
- Integrations (HA + MQTT) picked up .mod-head markup — compact module
layout with online/offline patch indicator. Integration card stripe
uses the default signal color (not cyan or amber).
- Scene presets, sync clocks, automations gain the same compact module
treatment. Automations/scenes dropped into a dashboard-autostart-grid
so they share the visual language.
- Perf mode toggle, stream sub-tabs, cs-count / tree-count /
tab-badge / dashboard-section-count badges all use the mono
rectangular style with tabular-nums.
Command palette:
- Flat background (no gradient), channel-accent rule across the top,
mono placeholder / group headers / footer, active result gets a
channel-green left stripe.
Modals:
- Popover + backdrop get a stronger radial dim + 6 px blur.
- Per-modal-ID channel lanes (target→green, source→cyan, audio→magenta,
automation/scene→violet, settings→amber, confirm→coral) via --modal-ch
override.
- Modal header picks up a vertical channel stripe + hairline divider;
footer gets hairline top + subtle wash.
Components:
- Inputs use hairline borders + tabular-nums mono for number fields;
focus state has channel-green ring + soft glow.
- Buttons switch to mono-uppercase with signal-glow on primary,
coral-glow on danger, hairline border on secondary.
- Card background flattened — removed gradient wash in favor of solid
--lux-bg-1 for both dark (#0e1014) and light (#f6f8fb).
- Page background: pure black for dark, pure white for light.
Color-picker:
- Always detaches to <body> with fixed positioning when its swatch sits
inside an overflow: hidden / auto / clip ancestor (perf strip, modal
bodies, tree-dd panels). Prevents the popover getting clipped.
Settings modal:
- Remembers the last-opened tab via localStorage key
settings_active_tab; falls back to 'general' if the tab id no longer
exists. Explicit overrides (donation → about, update badge →
updates) still work because callers invoke switchSettingsTab after
openSettingsModal.
Microcopy:
- Sidebar / transport localization for en/ru/zh:
sidebar.workspaces · transport.meta.{uptime,cpu,mem,poll,poll_hint}
· transport.status.{ready,armed} · dashboard.perf.{active_patches,
total_fps,devices}
Backend (coordinated with frontend):
- /system/performance now returns cpu_temp_hint_key when no live CPU
temperature is available, so the Temperature card can render an
actionable explainer instead of being hidden. Frontend respects the
key via t() lookup.
Section headers:
- Underline switched from dashed to solid; channel-green accent rule
(40 px) on the left remains.
Build / tests:
- ruff clean on touched Python files.
- tsc --noEmit clean.
- Python metrics-provider tests: 18 passed.
- CSS bundle ~214 KB.
|
||
|
alexei.dolgolyov
|
45f93fd30e |
fix(devices): SP110E vendor handshake + Windows/bleak robustness
SP110E peripherals silently tear down the GATT link ~1s after connect unless a two-write vendor handshake (01 00 → FFE2, 01 B7 E3 D5 → FFE1) arrives immediately. Without it the first real write hangs 30s then reconnect-loops forever. Adds optional BLEProtocol.init_writes executed on connect, plumbs a per-write char_uuid through both transports, and fixes the SP110E color/power frames from an incorrect 5 bytes to the documented 4 bytes. Windows/WinRT robustness: - asyncio.wait_for hangs on bleak because WinRT IAsyncOperations refuse to cancel. _bounded_await() uses asyncio.wait() instead so timeouts actually return control even when the inner task is uncancellable. - BleakClient connect by raw MAC string times out when WinRT guesses address type wrong; switched to pre-scanning with BleakScanner and passing the resolved BLEDevice, which carries the address type. - Target-start fetch timeout bumped to 30s with retry disabled so the UI doesn't abort during the BLE pre-scan + connect + handshake path. UI: - Settings modal exposes Protocol Family (IconSelect grid, shared with add-device via parameterized ensureBleFamilyIconSelect) so users can fix a wrong family pick without recreating the device. Govee AES key row toggles on/off with family selection. Also turns LAN auth back on in default_config.yaml, logs start_processing requests on entry for easier diagnosis, and captures the full debug trail in docs/BLE_LED_CONTROLLERS.md for future BLE work. Refs the mbullington SP110E protocol gist for the handshake bytes. |
||
|
alexei.dolgolyov
|
2b5dac2c42 |
feat(devices): BLE LED controller support (SP110E/Triones/Zengge/Govee)
End-to-end BLE streaming: provider + client + per-protocol wire encoders with whole-strip averaging, desktop (bleak) and Android (Kotlin BleBridge via Chaquopy) transports, discovery with protocol-family detection that auto-fills the UI, throttled not-connected warning + 10 s reconnect cooldown so a dropped link no longer stalls the pipeline at ~30 s/frame, and an explicit asyncio.wait_for wrapper around bleak connect() since the WinRT backend doesn't always honor the timeout kwarg. Also rewrites server/restart.ps1 to be parameterized (-Port / -Module / -PythonVersion / timeouts / -Quiet), pick the right interpreter via the py launcher, pre-flight the target module, poll port readiness on both shutdown and startup, redirect child stdout/stderr so Start-Process doesn't hang on inherited Git-Bash handles, and return proper exit codes. Rolls in concurrent work: Android BLE permissions + launcher icons + ru/zh resources, Chaquopy-safe value_stream psutil fallback, setup-required modal, asset-store test coverage, and misc system/config touch-ups. |
||
|
alexei.dolgolyov
|
d3a6416a1d |
refactor(devices): per-provider typed configs (phases 1-4)
Phase 1 — DeviceConfig hierarchy (device_config.py): - 17 @dataclass(frozen=True) subclasses (WLEDConfig, AdalightConfig, …) sharing BaseDeviceConfig; DeviceConfig = Union[all 17] - Device.to_config() in device_store.py: single flat→typed dispatch point Phase 2+3 — Typed provider signatures + call-site migration: - ProviderDeps(device_store) frozen dataclass in led_client.py - LEDDeviceProvider.create_client(config, *, deps) abstract signature - create_led_client(config, *, deps) factory dispatches via config.device_type - All 17 providers narrowed to their specific config type; drop kwargs.get() - GroupLEDClient.connect() uses device.to_config() + create_led_client() - wled_target_processor: replaced 21-field DeviceInfo unpacking with to_config() + dataclasses.replace(config, use_ddp=…) for DDP override - device_test_mode: build typed config via to_config() + ProviderDeps - Deleted DeviceInfo dataclass, _get_device_info(), _DEVICE_FIELD_DEFAULTS - TargetContext: replaced get_device_info callback with is_test_mode_active Phase 4 — Test migration: - 47-case test suite in tests/core/devices/test_device_config.py (100% coverage) - test_group_device.py TestGroupLEDClient migrated to GroupConfig + ProviderDeps - Removed legacy keyword-arg init path from GroupLEDClient |
||
|
alexei.dolgolyov
|
123da1b5c4 |
fix: comprehensive security, stability, and code quality audit
Security: - Force API key auth for LAN (non-loopback) requests; remove shipped dev key - Block path-traversal in backup restore; require auth on backup endpoints - SSRF protection: DNS resolve + private/loopback/link-local IP rejection - AES-256-GCM encryption for HA tokens and MQTT passwords with auto-migration - WebSocket auth migrated from query-string to first-message protocol - Asset upload: extension allowlist, server-side mime, Content-Disposition - Update installer: SHA256 verification, tar/zip member validation - Tightened CORS (explicit methods/headers, no credentials) - ADB serial regex allowlist, webhook rate-limit key fix, log scrubbing Android: - Root-capture: ordered teardown, screenrecord respawn watchdog, child reaping - USB permission blocking API via CompletableDeferred - Python init crash guard with fatal-error screen - Moved root grant + QR generation off Main thread - Cached PyObject engine for per-frame bridge calls - Ordered ScreenCapture resource cleanup, allowBackup=false Python: - Replaced all asyncio.get_event_loop() with get_running_loop/to_thread - Split color_strip_sources.py (1683->5 files) and color_strip_stream.py (1324->7 files) into packages - Extracted FrameLimiter utility, migrated 9 stream loops - Provider base-class reuse, WLED state caching + URL normalization - Narrowed broad except-pass in WS routes, threading fixes in BaseStore Frontend: - XSS fix: escapeHtml on dynamic option labels, reconcile-based list renders - Typed DOM helpers, safe localStorage access, AbortController listener hygiene - openAuthedWs helper for first-message WS auth protocol - Migrated remaining plain <select>s to IconSelect/EntitySelect Design: - WCAG AA primary color on light theme (#2e7d32, 5.4:1 contrast) - Android TV 10-foot breakpoint (tv.css) - Consolidated z-index tokens, unified easing, card-running GPU hints |
||
|
alexei.dolgolyov
|
7fcb8dd346 |
feat(devices): Android USB-serial support for Adalight/AmbiLED controllers
Adds end-to-end support for driving USB-connected Adalight / AmbiLED LED controllers from Android TV boxes. Android's security model blocks direct USB access from Python, so writes route through a Kotlin UsbSerialBridge singleton via Chaquopy. Python side: - New SerialTransport Protocol (serial_transport.py) with open / write / flush / close. Desktop uses PySerialTransport (wraps pyserial), Android uses AndroidSerialTransport (wraps the Kotlin bridge). - list_serial_ports() factory returns desktop COM ports on desktop, USB devices on Android — callers don't branch. - URL scheme extended: existing COM3[:baud] and /dev/ttyUSB0[:baud] unchanged; new usb:VID:PID[:serial][@baud] for Android (@ is the baud separator since : is already used between VID and PID). - AdalightClient and SerialDeviceProvider refactored to go through the transport — no more direct pyserial imports in hot paths. - 17 new unit tests cover URL parsing, PySerial transport, factory selection, platform-branching discovery. Full suite 750 passing. Kotlin side: - UsbSerialBridge.kt singleton uses com.hoho.android.usbserial (mik3y) which ships drivers for CH340, CP2102, FTDI, Prolific, and CDC-ACM (Arduino). Exposes listDevices, open, write, close via @JvmStatic for Chaquopy. First open() attempt without permission triggers the system USB permission dialog; next call succeeds once user grants. - usb-serial-for-android is distributed via JitPack — added that repo in settings.gradle.kts and the dependency in app/build.gradle.kts. - AndroidManifest declares uses-feature android.hardware.usb.host (required=false so non-USB-host phones still install). - LedGrabApp.onCreate calls UsbSerialBridge.init(this) so the bridge resolves the UsbManager without needing an Activity ref. Verified: ./gradlew compileDebugKotlin succeeds; off-Android import of android_serial_transport works. Real-hardware smoke test on a TV box with a CH340/CP2102/FTDI adapter still pending. ESP-NOW (espnow_client / espnow_provider) still imports pyserial directly because it needs bidirectional reads — separate refactor to extend the transport with read() if that path ever needs Android USB support. |
||
|
alexei.dolgolyov
|
ecae05d00b |
feat(metrics): battery + thermal-zone readings with dashboard temp chart
Extends MetricsProvider with thermals() returning a ThermalSnapshot
(battery_percent, battery_temp_c, cpu_temp_c — all optional). Each
provider implements it independently:
- AndroidMetricsProvider reads /sys/class/power_supply/battery/{capacity,
temp} (battery temp is tenths of degC) and walks
/sys/class/thermal/thermal_zone*, filtering by zone type
(cpu/soc/tsens/core) so battery and skin sensors don't dominate the
reading. Rejects nonsense values like INT_MAX from buggy zones.
- PsutilMetricsProvider uses sensors_battery() and
sensors_temperatures() when present (Linux+laptops); no-ops on
Windows/macOS where psutil doesn't expose them.
- NullMetricsProvider returns the empty snapshot.
PerformanceResponse gains battery_percent / battery_temp_c / cpu_temp_c.
The metrics-history ring buffer also carries cpu_temp / battery_pct /
battery_temp per sample so the dashboard can graph them over time.
Frontend dashboard (perf-charts.ts) gets a new Temperature chart card,
hidden by default and revealed only after seed/poll confirms the
backend reports cpu_temp_c. Battery temperature shows inline as a
secondary badge. The GPU card now also hides entirely when the backend
reports gpu=null instead of showing an "unavailable" placeholder.
HOST_ONLY_KEYS prevents the System/App/Both toggle from flipping a
non-existent app dataset for temp.
Tests: 6 new for thermals (battery tenths-of-degC parsing, CPU zone
filtering, fallback when sensors absent, INT_MAX rejection); 18 metrics
tests total; full suite 733 passing.
|
||
|
alexei.dolgolyov
|
546b24d015 |
refactor(metrics): MetricsProvider abstraction with Android /proc backend
Moves direct psutil.* calls behind a MetricsProvider Protocol so the codebase no longer needs ad-hoc `if psutil is not None` guards at every call site. Each provider lives in its own module under utils/metrics/: PsutilMetricsProvider for desktop, NullMetricsProvider as a zeroed fallback, AndroidMetricsProvider that reads /proc/stat, /proc/meminfo, /proc/self/stat, and /proc/self/status directly (psutil isn't available under Chaquopy). The Android provider tracks the previous CPU sample so cpu_percent() returns delta-based percentages matching psutil's interval=None semantics, and degrades to zeros when any /proc file is unreadable instead of crashing the dashboard. Factory get_metrics_provider() in utils/metrics/__init__.py picks Android > psutil > Null. api/routes/system.py and core/processing/metrics_history.py now go through the factory; psutil import is confined to one place. 12 new unit tests cover paren-in-comm parsing of /proc/self/stat, delta CPU%, missing-file resilience, and factory selection order. Full suite: 727 passing. |