alexei.dolgolyov
2e51f46dfd
Lets users wire the system end-to-end from the graph, and fixes the core bug that made drag-to-wire silently fail. - Fix drag-to-wire 422s across 5 entity kinds: updateConnection() now echoes the target's discriminator (source_type/stream_type/target_type) into the partial PUT, so value/colour-strip/audio/picture sources and output targets all wire correctly. New contract test (54 cases) in test_graph_wiring_contract.py. - Re-wire composite layers / mapped zones from the graph (right-click a layer/zone source edge -> Re-wire). Whole-list write preserves every sibling layer/zone setting, with an optimistic-concurrency guard and undo. - Secret-safe /graph topology: project entities to id/name/subtype + reference roots so the endpoint cannot leak webhook tokens or other credentials. - Carry slot indices on list edges; node custom-icon + schema-drift refinements; rewire i18n keys (en/ru/zh); wiring-control roadmap (TODO.md).
1116 lines
69 KiB
Markdown
1116 lines
69 KiB
Markdown
# LedGrab TODO
|
||
|
||
## HTTP polling automation trigger
|
||
|
||
Goal: a new automation trigger that periodically polls an HTTP endpoint
|
||
and activates a scene when the response matches a condition. Split into
|
||
three single-responsibility entities so the endpoint can be reused
|
||
beyond automations (e.g. as a value-source driving brightness/color):
|
||
|
||
- `HTTPEndpoint` (storage/http_endpoint.py) — connection definition:
|
||
URL + auth + headers + timeout. NO polling cadence; NO extraction.
|
||
- `HTTPValueSource` (storage/value_source.py, source_type='http') —
|
||
references an endpoint + owns json_path + interval + min/max + EMA
|
||
smoothing. Backed by `HTTPValueStream` (core/processing/value_stream.py)
|
||
which lives under the existing `ValueStreamManager` (ref-counted,
|
||
one poll task per unique value source).
|
||
- `HTTPPollRule` (storage/automation.py) — thin: `{value_source_id,
|
||
operator, value}`. Reads `stream.get_raw_value()` from the value
|
||
source and compares with `_apply_operator`.
|
||
|
||
Pivoted from a 2-entity shape mid-build (was: HTTPSource+rule with
|
||
interval+json_path mushed). The 3-entity shape mirrors HA's pattern
|
||
(HomeAssistantSource → HAEntityValueSource → rule).
|
||
|
||
### Phase 1 — endpoint + value source + thin rule (backend) ✅
|
||
|
||
- [x] `storage/http_endpoint.py` — `HTTPEndpoint` dataclass with
|
||
secret_box auth_token encryption + `__post_init__` plaintext
|
||
invariant. NO `default_interval_s` (moved to value source).
|
||
- [x] `storage/http_endpoint_store.py` — `HTTPEndpointStore` with
|
||
`_migrate_plaintext_tokens()`. ID prefix `htep_`.
|
||
- [x] `storage/database.py` — `"http_endpoints"` in `_ENTITY_TABLES`
|
||
(replaces the old `"http_sources"`).
|
||
- [x] `storage/value_source.py` — added `HTTPValueSource` alongside
|
||
`HAEntityValueSource` (endpoint_id, json_path, interval_s,
|
||
min/max, smoothing). Registered in `_VALUE_SOURCE_MAP`.
|
||
- [x] `storage/value_source_store.py` — CRUD branch for `source_type =
|
||
"http"` + new kwargs on create/update.
|
||
- [x] `core/processing/value_stream.py` — `HTTPValueStream` with poll
|
||
task + `get_value()` (normalized 0-1) + `get_raw_value()` (raw
|
||
extracted value). Dispatched in `ValueStreamManager._create_stream`.
|
||
Manager now takes `http_endpoint_store` so the stream can resolve
|
||
endpoints at fetch time.
|
||
|
||
### Phase 2 — rule + engine wiring ✅
|
||
|
||
- [x] `storage/automation.py` — `HTTPPollRule` is now thin: just
|
||
`{value_source_id, operator, value}` (no http_source_id, no
|
||
json_path on the rule). Legacy keys silently dropped on load.
|
||
- [x] `core/automations/automation_engine.py` — drops the standalone
|
||
http_poll_manager; takes `value_stream_manager`. Engine
|
||
`_sync_value_stream_refs` acquires/releases value streams for
|
||
every enabled HTTPPollRule, mirroring the HA/MQTT sync pattern.
|
||
`_evaluate_http_poll` reads `stream.get_raw_value()` and applies
|
||
the operator. `_apply_operator` kept at module top.
|
||
- [x] `api/schemas/automations.py` — RuleSchema fields are now
|
||
`value_source_id + operator + value` (dropped http_source_id +
|
||
json_path).
|
||
- [x] `api/routes/automations.py` — `http_poll` factory updated.
|
||
|
||
### Phase 3 — CRUD endpoints + wiring ✅
|
||
|
||
- [x] `api/schemas/http_endpoints.py` — Create/Update/Response/List/Test
|
||
(no interval field; that's on the value source).
|
||
- [x] `api/routes/http_endpoints.py` — full CRUD + `/test` +
|
||
plaintext-http-token warning.
|
||
- [x] `api/schemas/value_sources.py` — `HTTPValueSource{Create,Update,Response}`
|
||
added to the discriminated unions.
|
||
- [x] `api/routes/value_sources.py` — `_RESPONSE_MAP` entry for
|
||
`HTTPValueSource`.
|
||
- [x] `api/__init__.py` — `http_endpoints_router` registered.
|
||
- [x] `api/dependencies.py` — `get_http_endpoint_store` (dropped the
|
||
http_poll_manager getter).
|
||
- [x] `main.py` — instantiate `HTTPEndpointStore`, pass it through
|
||
`ProcessorDependencies`, wire `value_stream_manager` +
|
||
`value_source_store` into `AutomationEngine`.
|
||
- [x] `core/processing/processor_manager.py` — `ProcessorDependencies`
|
||
gains `http_endpoint_store`; threaded into `ValueStreamManager`.
|
||
|
||
### Phase 4 — tests ✅
|
||
|
||
- [x] `tests/storage/test_http_endpoint_store.py` — 14 tests (CRUD +
|
||
auth_token encryption + headers + case-insensitive Authorization).
|
||
- [x] `tests/core/test_automation_engine.py` — `TestApplyOperator` +
|
||
`TestHTTPPollRuleEvaluation` (new shape: mock ValueStreamManager
|
||
with `_streams` dict) + `TestSyncValueStreamRefs` (acquire /
|
||
release / disabled-ignored) + `TestHTTPValueStreamExtraction`
|
||
(`_extract_simple_path` now lives in value_stream.py).
|
||
- [x] `tests/api/routes/test_http_endpoints_routes.py` — CRUD shape, no
|
||
auth_token leak in responses, schema-layer method allowlist,
|
||
CRLF / invalid header rejection, `/test` endpoint, LAN policy.
|
||
- [x] Removed: `tests/core/test_http_poll_manager.py` (manager deleted —
|
||
polling now lives inside `HTTPValueStream`).
|
||
- [x] Full suite: 1426 passed, ruff clean.
|
||
|
||
### Phase 5 — frontend ✅
|
||
|
||
- [x] `static/js/features/http-endpoints.ts` (new, ~540 LOC) — endpoint
|
||
CRUD, modal subclass with dirty-check, headers row editor, test
|
||
result rendering, card builder, event delegation. Mirrors
|
||
`home-assistant-sources.ts`.
|
||
- [x] `templates/modals/http-endpoint-editor.html` (new) — sectioned
|
||
rack-panel modal (Identity / Request / Headers / Notes) with
|
||
IconSelect method picker, password-toggle on auth token, inline
|
||
Test button + result block.
|
||
- [x] `static/js/features/value-sources.ts` — added `http` branch with
|
||
EntitySelect over `httpEndpointsCache`, edit-data/defaults,
|
||
`onValueSourceTypeChange` section toggle, save-payload assembly
|
||
+ required-field validation.
|
||
- [x] `templates/modals/value-source-editor.html` — new
|
||
`#value-source-http-section` with endpoint picker + json_path +
|
||
interval + min/max + smoothing.
|
||
- [x] `static/js/features/automations.ts` — `http_poll` rule type with
|
||
operator IconSelect + value-source EntitySelect; hides Value
|
||
field when operator is `exists`.
|
||
- [x] `static/js/features/integrations.ts` — `csHTTPEndpoints` section,
|
||
tree/tab entry, render + reconcile + delegation paths.
|
||
- [x] `static/js/types.ts` — `HTTPEndpoint`, `HTTPMethod`,
|
||
`HTTPEndpointListResponse`, `HTTPTestRequest/Response`,
|
||
`HTTPValueSource`, `HTTPPollOperator`; extended `RuleType` +
|
||
`AutomationRule`.
|
||
- [x] `static/js/core/state.ts` — `httpEndpointsCache` (`/http/endpoints`).
|
||
- [x] `static/js/core/icons.ts` — `http: P.globe` in
|
||
`_valueSourceTypeIcons`.
|
||
- [x] `templates/index.html` — includes
|
||
`modals/http-endpoint-editor.html`.
|
||
- [x] Locales: 77 new keys per file in `en.json` / `ru.json` /
|
||
`zh.json` (parity confirmed).
|
||
- [x] Verification: `npx tsc --noEmit` clean; `npm run build` clean
|
||
(app.bundle.css 366.6kb, app.bundle.js 2.7mb).
|
||
|
||
### Follow-ups (out of scope for initial PR)
|
||
|
||
- [ ] **Global concurrency cap / minimum interval.** Each
|
||
`HTTPValueStream` runs its own task at `interval_s` (min 1s); no
|
||
project-wide cap. Reviewer flagged: pick a min (e.g. 5s) + max
|
||
active runtimes (e.g. 32) + shared `httpx.AsyncClient` with
|
||
`limits=httpx.Limits(max_connections=N)`.
|
||
- [ ] **DNS-rebinding hardening.** `safe_request_bounded` validates
|
||
the URL hostname's resolved IPs once; httpx independently
|
||
re-resolves. The window is short but not zero. True fix: pin
|
||
to the validated IP + set Host header (and SNI for HTTPS). This
|
||
affects every outbound caller (`safe_fetch`, weather, image
|
||
sources) — handle as a project-wide hardening, not local to
|
||
this feature.
|
||
- [ ] **`delete_http_endpoint` orphan refs.** When an admin deletes an
|
||
endpoint referenced by N value sources, the value-stream task
|
||
keeps polling until its source is also deleted. Same shape as
|
||
the MQTT defect — fix both together (refuse-with-409 when in
|
||
use, or cascade value-source deletion).
|
||
- [ ] **Per-endpoint `connected` / last-poll status on the response**
|
||
(frontend agent flagged). `HTTPEndpointResponse` has no live
|
||
status, unlike HA/MQTT sources. Card LEDs default to "on".
|
||
Could aggregate `last_status_code` / `last_error` from all
|
||
`HTTPValueStream` instances referencing the endpoint and surface
|
||
on `GET /http/endpoints/{id}`.
|
||
- [x] **Per-endpoint live `/test` after save** — added `POST
|
||
/http/endpoints/{id}/test` (runs stored config server-side so the
|
||
auth token never round-trips) and wired a flask-icon test action
|
||
on the endpoint card (toasts the result). Custom-headers section
|
||
and inline test-result UI in the editor modal also restyled to
|
||
match the `.group-child-row` and result-card vocabulary.
|
||
- [ ] **Dedicated icon for HTTP value source / endpoint** (frontend
|
||
agent flagged). Both use `P.globe` — visually fine in practice
|
||
but adding a `cable`/`webhook` glyph in `icon-paths.ts` would
|
||
improve differentiation.
|
||
|
||
## Multi-broker MQTT refactor
|
||
|
||
Goal: drop the global `MQTTService` / `MQTTConfig`. Every MQTT consumer
|
||
references an `MQTTSource.id`; `MQTTManager` is the only entry point.
|
||
`MQTTManager` + `MQTTRuntime` already exist — the job is to migrate every
|
||
caller off the legacy path, then delete it.
|
||
|
||
### Phase 1 — `mqtt_source_id` on Z2M target
|
||
|
||
- [x] Field on `Z2MLightOutputTarget` storage dataclass (+ to/from_dict)
|
||
- [x] Field on Z2M create/update/response schemas
|
||
- [x] Validate referenced `MQTTSource` exists at create/update
|
||
- [x] Thread through `output_target_store.create_z2m_light_target` + update
|
||
- [x] Thread through `ProcessorManager.add_z2m_light_target`
|
||
- [x] Thread through `Z2MLightTargetProcessor` constructor
|
||
|
||
### Phase 2 — Z2M processor uses `MQTTManager`
|
||
|
||
- [x] Replace `_mqtt_service` with `_mqtt_runtime` acquired from manager
|
||
- [x] `start()` acquire / `stop()` release
|
||
- [x] `_publish_payload` → `self._mqtt_runtime.publish(...)`
|
||
- [x] `turn_off_lights` borrow-pattern via manager (mirror HA-light)
|
||
- [x] Add `mqtt_manager` to `ProcessorDependencies` / `TargetContext`
|
||
|
||
### Phase 3 — Z2M editor UI
|
||
|
||
- [x] Add MQTT broker `EntitySelect` in Routing
|
||
- [x] Reuse `mqttSourcesCache`
|
||
- [x] Wire `mqtt_source_id` into edit-load + save payload + validation
|
||
|
||
### Phase 4 — DIY MQTT device (`MQTTLEDClient`)
|
||
|
||
- [x] `mqtt_source_id` field on `Device` storage
|
||
- [x] Field on `device_config.MQTTConfig`
|
||
- [x] `MQTTLEDClient` acquires runtime in `connect()`, releases in `close()`
|
||
- [x] Provider threads `mqtt_manager` via `ProviderDeps`
|
||
- [x] Device editor: MQTT source picker shown for `device_type=mqtt`. Turned
|
||
out the API layer was *also* missing it (the TODO's "backend accepts the
|
||
field" was wrong — `mqtt_source_id` lived in `device_store` +
|
||
`device_config.MQTTConfig` but was dropped by `DeviceCreate/Update/Response`
|
||
and the routes). Added: schema fields + route threading + referenced-source
|
||
validation (`_validate_mqtt_source_exists`, mirrors output_targets) +
|
||
`except HTTPException: raise` guard in `update_device` (it was masking its
|
||
own 4xx as 500). Frontend: broker `EntitySelect` (reusing `mqttSourcesCache`)
|
||
in both the add-device (`device-discovery.ts`) and settings
|
||
(`devices.ts`) modals — shown for `device_type=mqtt`, wired into
|
||
load/save/validate/dirty-check/clone. Empty = "first available broker".
|
||
4 regression tests in `test_devices_routes.py::TestMqttSourceId`; full
|
||
suite 1567 passing; en/ru/zh keys added.
|
||
|
||
### Phase 5 — `AutomationEngine`
|
||
|
||
- [x] Drop `mqtt_service` ctor parameter
|
||
- [x] Drop legacy fallback in `_evaluate_mqtt` (rule must reference a source)
|
||
|
||
### Phase 6 — `api/routes/system.py`
|
||
|
||
- [x] Replace integration status with `mqtt_manager.get_all_sources_status()`
|
||
- [x] Update frontend dashboard payload (MQTT widget now expects a list of
|
||
sources instead of a single `enabled`/`connected` pair — surface in UI).
|
||
Done: `dashboard.ts` `_renderMQTTIntegrationCard` renders one card per
|
||
`mqttStatus.connections` entry; `_updateIntegrationsInPlace` iterates the
|
||
list.
|
||
|
||
### Phase 7 — Startup migration
|
||
|
||
- [x] Seed a "Default Broker" `MQTTSource` if legacy YAML / env had a
|
||
broker configured and the store is empty (`core.mqtt.legacy_migration`)
|
||
- [x] Deprecation warning logged on migration; YAML/env no longer read after
|
||
|
||
### Phase 8 — Remove legacy
|
||
|
||
- [x] Delete `core/mqtt/mqtt_service.py`
|
||
- [x] Delete `set_mqtt_service` / `get_mqtt_service` (mqtt_client.py)
|
||
- [x] Remove `MQTTService` from `main.py`
|
||
- [x] Remove `MQTTConfig` + `resolve_mqtt_password` from `config.py`
|
||
- [x] Remove `mqtt: MQTTConfig` from `Config` (with `extra="ignore"` so legacy
|
||
YAML still loads)
|
||
|
||
### Phase 9 — Verification
|
||
|
||
- [x] `pytest tests/ --no-cov -q` clean (973 passing; removed obsolete
|
||
`test_default_mqtt_disabled`)
|
||
- [x] `ruff check src/` clean
|
||
- [x] `tsc --noEmit` + `npm run build`
|
||
- [ ] Smoke test: Z2M target on a configured MQTT Source publishes to broker
|
||
(manual)
|
||
|
||
## Refactor: typed output-target factories + auto-registry
|
||
|
||
Replaced `target_type` string elif chains in `OutputTargetStore` and
|
||
`OutputTarget.from_dict` with: (1) `__init_subclass__` registry for
|
||
deserialization, (2) per-type typed `create_*_target` /
|
||
`update_*_target` methods called directly from the route layer's
|
||
`match data:` dispatch. API contract unchanged, no DB migration.
|
||
|
||
### Phase 1 — Registry on `OutputTarget`
|
||
|
||
- [x] Added `_registry` + `_type_key` ClassVars + `__init_subclass__(*, type_key)`
|
||
- [x] Rewrote `OutputTarget.from_dict` to dispatch via registry
|
||
- [x] Declared `type_key="led"` / `"ha_light"` / `"z2m_light"` on the three subclasses
|
||
|
||
### Phase 2 — Typed `create_*_target` methods
|
||
|
||
- [x] Extracted `_resolve_brightness`, `_resolve_transition`, `_check_unique_name`,
|
||
`_new_id_and_now`, `_finalize` helpers on the store
|
||
- [x] Added `create_wled_target` / `create_ha_light_target` / `create_z2m_light_target`
|
||
with per-type defaults (transition 0.5/0.3, update_rate 2.0/5.0) baked into
|
||
their signatures
|
||
|
||
### Phase 3 — Typed `update_*_target` methods
|
||
|
||
- [x] Added `update_wled_target` / `update_ha_light_target` / `update_z2m_light_target`
|
||
with `_begin_update` / `_commit_update` helpers
|
||
- [x] Each typed update method validates the target's class before mutating
|
||
|
||
### Phase 4 — Route migration
|
||
|
||
- [x] `create_target` route uses `match data:` to call typed store methods —
|
||
no more `getattr(data, "x", default)` pyramid
|
||
- [x] `update_target` route uses `match data:` and computes `settings_changed` /
|
||
`css_changed` / `brightness_changed` per-arm from typed fields
|
||
- [x] Helpers `_build_ha_mappings`, `_build_z2m_mappings`,
|
||
`_validate_device_exists`, `_resolve_effective_color_vs_id` extracted
|
||
|
||
### Phase 5 — Decision: keep both shims
|
||
|
||
After grepping for callers, `src/ledgrab/core/scenes/scene_activator.py:90`
|
||
calls `target_store.update_target(target_id, **changed)` with a dynamically
|
||
built dict — it legitimately doesn't know the target's type at the call site.
|
||
The shims are now ~30-line dispatchers that route to typed methods (no more
|
||
inline construction elif chains), so the original anti-pattern is gone while
|
||
the generic API remains available for "don't-know-the-type" callers like the
|
||
scene activator. Tests continue to use the shorthand `create_target("A", "led")`
|
||
form without churn.
|
||
|
||
### Phase 6 — Verify
|
||
|
||
- [x] `ruff check` clean on all modified files
|
||
- [x] `py -3.13 -m pytest tests/ --no-cov -q` — 974 passed (was 974 before)
|
||
- [ ] Manual smoke test in UI: create/edit/delete each of the three target types
|
||
|
||
## Custom card icons — extend to all card types
|
||
|
||
Migrate the existing icon-plate work (devices, LED targets, HA-light targets)
|
||
to all remaining card types. ~17 entity types. Branch: `feat/icons-everywhere`.
|
||
|
||
### Foundation
|
||
|
||
- [x] Refactor `icon-picker.ts` — replace hardcoded 2-entry `_adapters`
|
||
record with a `Map<EntityType, EntityTypeAdapter>` and expose
|
||
`registerIconEntityType()` for feature modules to register their
|
||
own. Added `makeSimpleIconAdapter()` helper that reduces a
|
||
registration to ~6 lines.
|
||
- [x] Generalised `bodyExtras` for discriminated routes (output-targets
|
||
`target_type` etc.) — now keyed off id, adapter does its own
|
||
lookup.
|
||
- [x] `_onDocumentClick` accepts any registered type instead of
|
||
hardcoded device/target check.
|
||
- [x] Locale entity-type labels added to en/ru/zh for 18 new types
|
||
(picture_source, audio_source, weather_source, value_source,
|
||
mqtt_source, ha_source, automation, scene_preset, sync_clock,
|
||
game_integration, audio_processing_template, pattern_template,
|
||
capture_template, pp_template, cspt, audio_template, gradient,
|
||
color_strip_source, asset).
|
||
|
||
### Backend (storage + schemas + routes per entity)
|
||
|
||
Recipe: add `icon: str = ""` + `icon_color: str = ""` to dataclass,
|
||
emit-when-truthy in `to_dict`, default `""` in `from_dict`; add 3
|
||
`Optional[str]` Field defs to Create/Response/Update schemas; thread
|
||
`getattr(entity, "icon", "") or ""` into the response builder.
|
||
SQLite JSON-blob storage means **no migration required**.
|
||
|
||
- [x] Integrations (6): weather_sources, value_sources, mqtt_source,
|
||
home_assistant_source, sync_clocks, game_integration
|
||
- [x] Streams (10): picture_source, audio_source, audio_template,
|
||
audio_processing_template, pattern_template, postprocessing_template,
|
||
color_strip_processing_template, color_strip_source, gradient,
|
||
capture_template (`storage/template.py` — was missed by initial pass)
|
||
- [x] Other (3): automation, scene_preset, asset
|
||
|
||
### Frontend (per feature module)
|
||
|
||
For each card render call:
|
||
|
||
- Use the new `core/card-icon.ts` helper:
|
||
`...makeCardIconFields('<type>', entity.id, entity)` spread into the
|
||
mod-card head — computes `iconHtml`/`iconColor`/`iconAttrs` in one go.
|
||
- Register the entity type in the feature module via
|
||
`registerIconEntityType('<type>', makeSimpleIconAdapter({ … }))`.
|
||
|
||
Modules wired:
|
||
|
||
- [x] streams.ts (7 cards: picture, capture, pp, cspt, audio source,
|
||
audio template, gradient — built-in gradients skip the plate)
|
||
- [x] automations.ts
|
||
- [x] scene-presets.ts
|
||
- [x] sync-clocks.ts
|
||
- [x] weather-sources.ts
|
||
- [x] value-sources.ts (bodyExtras propagates `source_type`)
|
||
- [x] mqtt-sources.ts
|
||
- [x] home-assistant-sources.ts
|
||
- [x] game-integration.ts
|
||
- [x] audio-processing-templates.ts
|
||
- [x] assets.ts
|
||
- [x] color-strips/cards.ts (bodyExtras propagates `source_type`)
|
||
- [WONTDO] pattern-templates.ts — uses legacy `wrapCard({content, actions})`
|
||
string API, not the mod-card system. Migration would be a separate
|
||
effort and the cards are tiny (name + rect count) so the value is low.
|
||
|
||
### Discriminated routes
|
||
|
||
Adapters provide `bodyExtras` to inject the discriminator field on PUT
|
||
so the Pydantic discriminated-union route validators don't reject the
|
||
icon-only update:
|
||
|
||
- output-targets → `target_type` (already wired before)
|
||
- color-strip-sources → `source_type`
|
||
- audio-sources → `source_type`
|
||
- value-sources → `source_type`
|
||
- picture-sources → `stream_type`
|
||
|
||
### Verification
|
||
|
||
- [x] `cd server && ruff check src/ tests/` clean
|
||
- [x] `cd server && npx tsc --noEmit` clean
|
||
- [x] `cd server && npm run build` produces 2.6 MB bundle
|
||
- [x] `cd server && py -3.13 -m pytest tests/ --no-cov -q` — 949 passed
|
||
- [ ] Manual: open picker on each card type, confirm save persists,
|
||
confirm channel-color preview matches the live card
|
||
|
||
## Device Event Notifications
|
||
|
||
Notify the user when LED devices come online/go offline (configured targets), and when new
|
||
WLED/serial devices are discovered or disappear from the LAN/USB. Each event class has a
|
||
configurable channel: `none` | `snack` | `os` | `both`. OS channel uses Web Notifications
|
||
(works in any browser tab and in the PWA shell — no platform-specific Python).
|
||
|
||
Branch: `feat/device-event-notifications`. Default ON.
|
||
|
||
### Backend
|
||
|
||
- [x] `core/devices/discovery_watcher.py` — long-running mDNS browser
|
||
(`AsyncServiceBrowser` kept alive for the process lifetime) + 10 s serial-port
|
||
poller. Fires `device_discovered`/`device_lost` via `processor_manager.fire_event`,
|
||
suppresses events for URLs already in `device_store`. Seeded ports do NOT generate
|
||
startup-time toasts.
|
||
- [x] Wired into `lifespan` (`main.py`). Gated by `notification_preferences.
|
||
background_discovery_enabled`. Default True. Stops before health monitor stop.
|
||
- [x] `api/schemas/preferences.py` — `NotificationPreferences` Pydantic v2 model with
|
||
the 4-event channel matrix, `background_discovery_enabled`, `startup_grace_sec`
|
||
(0..300), `flap_debounce_sec` (0..60).
|
||
- [x] `api/routes/preferences.py` — `GET/PUT /api/v1/preferences/notifications`,
|
||
persisted under `db.set_setting("notification_preferences", …)`. Corrupt stored
|
||
values fall back to defaults instead of 500.
|
||
- [x] Reuses existing `device_health_changed` event from `device_health.py` (already
|
||
fires online/offline transitions on the same event bus).
|
||
- [x] Tests: 7 in `tests/test_preferences_notifications_api.py`, 6 in
|
||
`tests/test_discovery_watcher.py`. Full pytest suite still 899 passing.
|
||
|
||
### Frontend
|
||
|
||
- [x] `js/features/notifications-watcher.ts` — listens to the three `server:*` DOM
|
||
events. Applies user prefs. Pipeline: startup grace → flap debounce → bulk
|
||
coalesce (≥3 events / 800 ms collapse to one summary).
|
||
- [x] Web Notification permission requested from the Settings → Notifications panel
|
||
via a user-gesture button. State chip reflects granted/denied/default.
|
||
- [x] Settings panel — new "Notifications" subtab between Backup and Appearance.
|
||
4 IconSelects (`none`/`snack`/`os`/`both`) + background-discovery toggle +
|
||
permission row + Test-notification button.
|
||
- [x] i18n: `settings.notifications.*` and `notifications.*` keys in en/ru/zh.
|
||
|
||
### Verification (notifications)
|
||
|
||
- [x] `npx tsc --noEmit` clean, `npm run build` produces 2.5 MB bundle.
|
||
- [x] `ruff check src/ tests/` clean. 899/899 pytest pass.
|
||
- [x] App import smoke-test (`from ledgrab.main import app`) loads 233 routes
|
||
without errors.
|
||
- [ ] Real-hardware test pending — verify on user's network:
|
||
(1) plug a fresh WLED in → snack toast appears, (2) configure it → next
|
||
offline transition fires both snack + OS toast, (3) Background-discovery
|
||
toggle off → no more discovered/lost events.
|
||
|
||
### Out of scope for v1
|
||
|
||
- Per-device-type granularity (we ship one matrix per event-type, no device-type split)
|
||
- Per-device mute list (deferred — user can globally toggle off if noisy)
|
||
- Native OS toast via Windows winrt API (Web Notifications cover the use case;
|
||
also avoids the `os_notification_listener` feedback loop)
|
||
- Notification history panel — could land later as the reserved `alerts` dashboard cell
|
||
|
||
## Server shutdown action
|
||
|
||
Let user choose what happens to LED targets on server shutdown.
|
||
|
||
- [x] Backend storage: `shutdown_action` in `db.settings` (`"stop_targets"` default | `"nothing"`)
|
||
- [x] Backend route: `GET/PUT /api/v1/system/shutdown-action` in `system_settings.py`
|
||
- [x] Backend schema: `ShutdownActionResponse/Request` in `schemas/system.py`
|
||
- [x] Backend wiring: lifespan shutdown in `main.py` reads action, passes `restore_devices` flag to `processor_manager.stop_all()`
|
||
- [x] `processor_manager.stop_all(restore_devices: bool = True)` — when False, calls public `proc.cancel_task()` (defined on `TargetProcessor`) which awaits cancellation without restoring device state; skips `_restore_device_idle_state` loop. No reach into private `_task` attribute.
|
||
- [x] Frontend: hidden `<select>` + IconSelect in `settings.html` General tab (icons via `ICON_SQUARE` / `ICON_CIRCLE` from `core/icons.ts`)
|
||
- [x] Frontend: load/save handlers in `features/settings.ts`, wired into `openSettingsModal()`
|
||
- [x] i18n: en / ru / zh keys for label, hint, item descriptions
|
||
- [ ] Real-hardware test pending — verify that "nothing" actually leaves a WLED + a serial device on the last frame after `Ctrl+C`/SIGTERM.
|
||
|
||
## WebUI Redesign — "Lumenworks" Studio-Console Aesthetic
|
||
|
||
Full-app UI/UX refresh. Design direction committed to by user 2026-04-24.
|
||
Mockup lives at [server/docs/ui-redesign-mockup.html](server/docs/ui-redesign-mockup.html).
|
||
Phases are independent and CSS-only where possible — backend untouched.
|
||
|
||
### Phase 1 — Design tokens & font embed
|
||
|
||
- [x] Embed variable fonts (`server/src/ledgrab/static/fonts/`):
|
||
Manrope (latin + latin-ext + cyrillic + cyrillic-ext),
|
||
JetBrains Mono (same 4 subsets),
|
||
Big Shoulders Display (latin + latin-ext). Total +201 KB gzipped,
|
||
served via `unicode-range` so only latin paints on first load.
|
||
- [x] `fonts.css` — declare `@font-face` entries for all new families with
|
||
proper `unicode-range` subsetting; keep DM Sans + Orbitron registered
|
||
for legacy-token callers during migration.
|
||
- [x] `base.css` — add additive Lumenworks tokens:
|
||
`--font-display/--font-brand/--font-body`, `--lux-r-*`, `--lux-hairline`,
|
||
`--lux-rule`. Both `[data-theme="dark"]` and `[data-theme="light"]`
|
||
define `--lux-bg-0…3`, `--lux-line/-bold`, `--lux-ink/-dim/-mute/-faint`,
|
||
`--ch-signal/-cyan/-magenta/-amber/-coral/-violet`, `--lux-signal-glow`,
|
||
`--lux-shadow-rack`. Existing tokens untouched — no visual regression.
|
||
|
||
### Phase 2 — Shell (header → transport bar + channel-strip sidebar)
|
||
|
||
- [x] `index.html` — `.tab-bar` moved out of `<header>` into a new
|
||
`<aside class="sidebar">`; wrapped content in `.app-body` 2-col grid
|
||
(sidebar | main). `.transport-center` section added between
|
||
`.header-title` and `.header-toolbar` with a placeholder `.transport-status`
|
||
chip ("Ready" → "Armed · N live" wired in Phase 3). All tab-button IDs,
|
||
`data-tab` attributes, and `onclick="switchTab(…)"` handlers preserved.
|
||
- [x] `layout.css` — `<header>` rebuilt as the transport bar: 3-column grid
|
||
(brand | center | toolbar), 60 px fixed height, sticky, gradient bottom
|
||
rule with channel-color wash. `.header-title::before/::after` render
|
||
the glowing LED brand mark; `#server-status` repositioned as the LED
|
||
core pip. `#server-version` restyled as a mono-type console badge.
|
||
- [x] `sidebar.css` (new) — vertical channel-strip navigation. Active tab
|
||
gets a glowing left stripe + radial tint. `.sidebar-foot` contains
|
||
a `.cpu-meter` plate with two live bars (Load, FPS) ready to be
|
||
JS-bound in Phase 3. Collapses to a 56 px icon rail at ≤1100 px;
|
||
hides entirely at ≤600 px via `display: contents` so `.tab-bar`
|
||
falls through to `mobile.css`'s fixed-bottom strip unchanged.
|
||
- [x] `all.css` — new sidebar import after layout.
|
||
- [x] `base.css` — body font-family switched to `var(--font-body)` which
|
||
resolves to Manrope (with DM Sans + system fallbacks). Added
|
||
`font-feature-settings` for stylistic set + alternate 1.
|
||
- [x] Locale additions: `sidebar.workspaces`, `sidebar.load`, `sidebar.fps`,
|
||
`transport.status.ready`, `transport.status.armed` in en/ru/zh.
|
||
- [x] Tutorial + auth selectors (`header .header-title`, `#tab-btn-*`,
|
||
`.tab-bar` querySelector, `a.header-link[href="/docs"]`, onclick
|
||
markers on theme/settings/search) all survive the move.
|
||
- [ ] JS: bind `.cpu-meter` + `.transport-status` chip to existing
|
||
`performance` WebSocket / poller. Done as part of Phase 3.
|
||
- [ ] Tablet-range visual polish pass once other phases render (some tabs
|
||
currently have their own internal sticky headers that may overlap
|
||
the transport bar on narrow viewports).
|
||
|
||
### Phase 3 — Dashboard hero + module redesign
|
||
|
||
- [x] `cards.css` — `.card` gets rack-module treatment: channel stripe on
|
||
left edge (color-coded via `data-card-type` + `.ch-*` utility classes),
|
||
`::after` corner bracket in top-right, mono-typed metric labels
|
||
planned for Phase 4. Running cards glow the stripe brighter + emit a
|
||
`signalFlow` keyframe strip along the bottom edge.
|
||
- [x] Removed the `@property --border-angle` rotating conic-gradient border
|
||
(retired the WebKit mask workaround + light-theme variant + fallback
|
||
for `@supports not (mask-composite: exclude)`). Replaced with the
|
||
signal-flow strip — one animated linear-gradient on a 2 px line, no
|
||
GPU layer compositing per card.
|
||
- [x] `dashboard.css` — `.dashboard-target` rows pick up the same channel
|
||
stripe + signal-flow treatment. Section headers now use mono caps
|
||
with a channel-green underline accent. Metric values use mono with
|
||
tabular numerics; labels use silkscreened micro-caps.
|
||
- [x] Skeleton-card rewritten: left hairline + corner bracket so it reads
|
||
as "loading module" instead of a generic flashing block.
|
||
`skeletonShimmer` gradient replaces the old opacity-pulse on
|
||
`--text-color`.
|
||
- [x] `_updateSidebarMeter` binds CPU% (Load) and app-CPU share (FPS)
|
||
to the sidebar meter plate on every perf poll.
|
||
- [x] `_updateTransportStatus` updates the transport chip ("Ready" →
|
||
"Armed · N live") whenever the dashboard's running-target set is
|
||
recomputed.
|
||
- [ ] `.hero` 4-cell readout row (Active Patches / Throughput / CPU /
|
||
Latency + inline sparklines) — CSS tokens + layout are ready; HTML
|
||
render deferred until the dashboard JS is refactored to emit it
|
||
(Phase 3b, non-blocking).
|
||
|
||
### Phase 4 — Other tabs adopt module language
|
||
|
||
- [x] `tree-nav.css` — trigger pill gets a channel stripe on its left edge
|
||
(glows + widens when open). Trigger title uses mono-uppercase with
|
||
wide letter-spacing. Dropdown panel has a gradient channel-accent
|
||
rule across its top edge. Group headers use silkscreened micro-caps
|
||
with a small square marker instead of the old bold-uppercase. Active
|
||
leaf has a pulsing LED pip on the left and a channel tint behind it.
|
||
Count badges switched to mono tabular-nums in 2-px-radius pills.
|
||
- [x] `.subtab-section-header` — channel-green underline accent + mono
|
||
micro-caps. Consistent with the dashboard-section pattern so the
|
||
whole app shares one section-header language.
|
||
- [x] `.stream-tab-btn` sub-tabs — mono uppercase with wide tracking,
|
||
active tab shows channel-green underline + glowing count badge.
|
||
- [x] `.perf-chart-card` — channel stripe on the left (replaces old
|
||
`border-top` accent). Per-metric accents swapped to channel palette
|
||
(`--ch-coral` for CPU, `--ch-violet` for RAM, `--ch-signal` for GPU,
|
||
`--ch-amber` for temp). Corner bracket added. Metric values pick up
|
||
`tabular-nums` + a soft glow.
|
||
- [x] `cards.css` — channel-color mapping extended to attributes the JS
|
||
already emits (`data-target-id` → green, `data-stream-id` → cyan,
|
||
`data-audio-source-id` → magenta, `data-automation-id` /
|
||
`data-scene-id` → violet). No JS changes required; cards pick up
|
||
their correct stripe automatically on the Targets/Sources/Automations
|
||
tabs.
|
||
- [x] Graph editor — toolbar gets a gradient background + hairline +
|
||
rack shadow + backdrop blur. Canvas and nodes untouched.
|
||
- [x] `.template-card` — Lumenworks treatment (channel stripe on left,
|
||
corner bracket top-right, hairline border, hover lift + stripe
|
||
glow). Brings Inputs (streams / capture / pp / cspt / pattern
|
||
templates) and Integrations (HA / MQTT / weather / value /
|
||
sync-clock / game-integration cards) up to the same visual
|
||
language as `.card` and `.dashboard-target`.
|
||
- [x] `cards.css` — channel mapping extended to `.template-card`.
|
||
Direct attr hooks for `data-stream-id`/`data-template-id`/`data-pp-template-id`
|
||
(cyan), `data-cspt-id`/`data-pattern-template-id` (signal),
|
||
`data-audio-template-id`/`data-apt-id` (magenta). Section-scoped
|
||
hooks via `[data-card-section="…"]` for cards that share a
|
||
generic `data-id` (HA / MQTT / weather / value → cyan;
|
||
game-integrations → amber; sync-clocks → violet; HA-light-targets
|
||
→ signal). No JS changes — uses the section markup `CardSection`
|
||
already emits.
|
||
- [x] Graph editor nodes — body fill `--lux-bg-1` with hairline stroke,
|
||
hover bold-line, selected/running stroke `--ch-signal` with
|
||
drop-shadow glow. Title font switched from DM Sans to
|
||
`--font-display`; subtitle to mono uppercase wide-tracking.
|
||
Port-drop-target glow recoloured to `--ch-signal`. Port labels
|
||
adopt the mono caption treatment. Grid dots use `--lux-line`.
|
||
Running gradient stops switched from `--primary-color`/`--success-color`
|
||
to channel palette (signal → cyan → signal).
|
||
|
||
### Phase 5 — Modal restyle
|
||
|
||
- [x] `modal.css` — backdrop gains a radial dim + 6 px blur for stronger
|
||
separation. `.modal-content` gets a gradient background + hairline +
|
||
deep rack shadow. Channel-accent rule across the top edge driven by
|
||
`--modal-ch` (per-modal override). Corner bracket bottom-right on
|
||
desktop. `.modal-header` gains a vertical channel-color stripe to
|
||
the left of the title; `.modal-footer` picks up a hairline divider.
|
||
- [x] Per-modal channel mapping by modal ID:
|
||
- Target editors → green
|
||
- Input/Source editors → cyan
|
||
- Audio editors → magenta
|
||
- Automation / Scene / Game editors → violet
|
||
- Settings / API key / Setup / Notifications → amber
|
||
- Confirm dialog → coral
|
||
- [x] `components.css` — inputs use hairline borders, tabular-nums mono
|
||
for `input[type="number"]`, channel-green focus ring + glow. Buttons
|
||
use mono-uppercase type, signal-glow on primary, coral-glow on
|
||
danger. `<select>` audit deferred (project already enforces via
|
||
CLAUDE.md rule + IconSelect/EntitySelect wrappers).
|
||
|
||
### Phase 6 — Mobile dedicated shell
|
||
|
||
- [x] `mobile.css` (existing file, not forked) — fixed-bottom `.tab-bar`
|
||
promoted to full Lumenworks treatment: gradient background + hairline
|
||
divider at top + channel-accent rule matching the transport-bar
|
||
bottom. Active tab gets an LED pip above the icon and a channel-tint
|
||
background. Tab labels + badges use mono uppercase to match the
|
||
rest of the app. Phone (≤600 px): modal corner-bracket hidden
|
||
(fullscreen modals), modal-header stripe slimmed to 18 px.
|
||
- [x] Phase 2's layout.css already strips the transport-center on phones
|
||
and collapses the sidebar via `display: contents`, so the mobile
|
||
shell automatically routes the tab-bar to the bottom without a
|
||
separate JS hook.
|
||
- [WONTDO] Fork into `mobile-shell.css` — keeping changes in `mobile.css`
|
||
since the cascade was already organized by viewport. A rename adds
|
||
churn without improving maintainability.
|
||
|
||
### Phase 7 — Microcopy + retire legacy
|
||
|
||
- [x] Locale rename: `targets.title` + `dashboard.section.targets` →
|
||
"Channels" (en) / "Каналы" (ru) / "通道" (zh);
|
||
`streams.title` → "Inputs" / "Входы" / "输入".
|
||
Automations kept as-is (Automations + Scenes is a meaningful
|
||
distinction; "Patches" would conflate them). Internal tab keys
|
||
(`dashboard` / `automations` / `targets` / `streams` / `integrations`
|
||
/ `graph`) unchanged so no JS or localStorage migration needed.
|
||
- [x] Ambient WebGL background — default is already `off`; kept the
|
||
toggle button and localStorage preference so users who want the
|
||
shader can turn it on. No entry-point change needed: `data-bg-anim`
|
||
is initialized from localStorage with `off` fallback.
|
||
- [DEFERRED] Delete DM Sans + legacy color tokens — would cascade through
|
||
every file that reads `--primary-color` / `--text-color` etc. Safer
|
||
as a separate cleanup PR after the new design has soaked.
|
||
- [WONTDO] Delete `mobile.css` — Phase 6 kept the filename.
|
||
|
||
## Dashboard Customization
|
||
|
||
Per-account dashboard layout — slide-in Customize panel lets users
|
||
toggle section / perf-cell visibility, reorder via drag, change density,
|
||
pick presets, and import/export the layout as JSON. Server-synced via
|
||
`db.get_setting('dashboard_layout')` so settings follow the user.
|
||
|
||
- [x] `js/features/dashboard-layout.ts` — schema (open registry of section
|
||
/ perf-cell keys so v1.1 cards slot in with no migration), defaults,
|
||
5 built-in presets (Studio/Operator/Showrunner/Diagnostics/TV),
|
||
localStorage cache + server sync, legacy-key migration from
|
||
`dashboard_collapsed`, `perfMetricsMode`, `perfChartColor_*`.
|
||
- [x] `api/routes/preferences.py` — `GET/PUT/DELETE
|
||
/api/v1/preferences/dashboard-layout`. Treats payload as opaque
|
||
(frontend owns the schema); validates only that body is an object
|
||
with a numeric `version`. 6 pytest tests in
|
||
`tests/test_preferences_api.py` cover round-trip, default-empty,
|
||
validation, delete, and unknown-field passthrough.
|
||
- [x] `js/features/dashboard.ts` — sections rendered into a fragment map,
|
||
then assembled in layout-driven order; perf section stays pinned
|
||
top (chart-persistence reasons) but its visibility is layout-
|
||
driven. Layout-change subscription invalidates the in-place-update
|
||
optimization so density / order / visibility changes always
|
||
rebuild section HTML.
|
||
- [x] `js/features/perf-charts.ts` — `renderPerfSection()` iterates
|
||
`getOrderedPerfCells()`; existing legacy `setPerfMode` writes
|
||
through to the layout so the global toggle and the customize
|
||
panel stay in sync.
|
||
- [x] `js/features/dashboard-customize.ts` + `css/dashboard-customize.css`
|
||
— slide-in panel, hand-rolled HTML5 drag-and-drop reorder, ↑/↓
|
||
buttons for keyboard / TV remote, debounced (300 ms) autosave,
|
||
live preview while open. Reset / export / import actions.
|
||
- [x] i18n keys for `dashboard.customize.*` in en/ru/zh.
|
||
- [ ] (v1.1) Audio meters section — peak / RMS / BPM bars per audio
|
||
source. Schema key `audio-meters` already reserved.
|
||
- [ ] (v1.1) Alerts section — quiet by default, loud on issues.
|
||
Reserved key `alerts`.
|
||
- [ ] (v1.1) Live LED preview strip per running device. Reserved
|
||
key `led-preview`.
|
||
- [ ] (v1.1) Source thumbnails grid (1 fps multiviewer). Reserved
|
||
key `source-thumbs`.
|
||
- [ ] (v1.2) Pinned section (user-curated mix of targets / scenes /
|
||
devices). Reserved key `pinned`.
|
||
- [ ] (v1.2) Patch/flow map — read-only mini graph of routing.
|
||
Reserved key `flow`.
|
||
|
||
## BLE LED Controller Support (SP110E / Triones / Zengge / Govee)
|
||
|
||
Add support for Bluetooth Low Energy LED controllers driven by mobile apps like "LED Hue", HappyLighting, iLightsIn. Whole-strip ambient-color output only — these protocols don't support per-pixel streaming.
|
||
|
||
- [x] Add `bleak>=0.22` as optional extra `[ble]` in `server/pyproject.toml` (desktop-only, NOT in android `build.gradle.kts`)
|
||
- [x] `core/devices/ble_transport.py` — bleak wrapper: scan, connect, write-with/without-response
|
||
- [x] `core/devices/ble_protocols/` package
|
||
- [x] `__init__.py` — `BLEProtocol` dataclass + registry (family → encoder)
|
||
- [x] `sp110e.py` — SP110E / SP108E (service FFE0, char FFE1, `RR GG BB 00 1E` static-color frame)
|
||
- [x] `triones.py` — Triones / HappyLighting / LEDnet (service FFE5, char FFE9, `7E 07 05 03 RR GG BB 10 EF`)
|
||
- [x] `zengge.py` — Zengge / iLightsIn (service FFE0, framing `56 RR GG BB 00 F0 AA`)
|
||
- [x] `govee.py` — Govee unencrypted framed protocol (AES keyed variants — marked experimental)
|
||
- [x] `core/devices/ble_client.py` — unified `BLEClient(LEDClient)` — picks protocol by `ble_family`, averages strip → one color, drops duplicate frames, rate-limits to BLE connection interval
|
||
- [x] `core/devices/ble_provider.py` — `BLEDeviceProvider` + discovery via `BleakScanner`
|
||
- [x] Register in `core/devices/led_client.py::_register_builtin_providers` (guarded `try/except ImportError`)
|
||
- [x] Storage: `ble_family`, `ble_govee_key` fields threaded through `Device.__init__`/`to_dict`/`from_dict`/`_UPDATABLE_FIELDS`/`create_device`
|
||
- [x] Schemas: BLE fields on `DeviceCreate`, `DeviceUpdate`, `DeviceResponse`
|
||
- [x] Routes: BLE fields propagated through create/update in `api/routes/devices.py` + `_device_to_response`
|
||
- [x] ProcessorManager: `ble_family`/`ble_govee_key` added to `_DEVICE_FIELD_DEFAULTS` and `DeviceInfo`; passed through `wled_target_processor.py` and `group_client.py` to `create_led_client`
|
||
- [x] Tests: 21 protocol encoder unit tests + 16 BLEClient fake-transport tests — all passing, 814 total tests still green
|
||
- [x] Frontend: BLE option in the device type picker with a bluetooth Lucide icon; add-device modal shows a 4-option `IconSelect` for protocol family (SP110E / Triones / Zengge / Govee) with a Govee-only AES key field that auto-hides for the other three families; URL label/placeholder/hint adapt to `ble://<address>` pattern; submit payload carries `ble_family` (+ optional `ble_govee_key`); clone flow pre-fills family and key; modal dirty-check snapshots the new fields; network scan button now also discovers BLE peripherals via the existing `/api/v1/devices/discover?device_type=ble` endpoint
|
||
- [x] Frontend: `isBleDevice` helper in `core/api.ts`; `ICON_BLUETOOTH` + `ICON_LIGHTBULB` constants in `core/icons.ts`; `bluetooth` path in `core/icon-paths.ts`; i18n keys in `en.json` / `ru.json` / `zh.json`; TypeScript compiles; esbuild bundle rebuilt
|
||
- [x] Android BLE via Kotlin bridge — `BleBridge.kt` singleton (scan/connect/write/disconnect); `android_ble_transport.py` Python wrapper; `make_transport()` factory in `ble_transport.py` auto-selects backend; `BleBridge.init()` called from `LedGrabApp.onCreate`; BLE permissions in `AndroidManifest.xml`
|
||
- [x] Govee per-model AES key — `_encrypt_govee_frame()` in `ble_client.py` uses AES-128-ECB from `cryptography`; key validated on `BLEClient` construction; applied to both `send_pixels` and `set_power`; 8 new AES unit tests
|
||
|
||
## Android — Restore Multi-ABI Wheels
|
||
|
||
During emulator testing, we switched the build to **x86 only** (see `android/app/build.gradle.kts` `abiFilters`) to avoid having to keep the arm64-v8a / x86_64 pydantic-core wheels current. Before shipping, restore all three ABIs:
|
||
|
||
- [x] Rebuild `pydantic-core` wheels for all three ABIs with the current SOABI + libpython linking settings (`android/build-scripts/build-pydantic-core.sh` — now supports `arm64`, `x86_64`, `x86` args; defaults to all three).
|
||
- [x] Verify wheels: all three now list `libpython3.11.so` in `NEEDED` (`llvm-readelf -d`), automated in the build script.
|
||
- [x] Restored `abiFilters += listOf("arm64-v8a", "x86_64", "x86")` in `build.gradle.kts`. Multi-ABI debug APK builds cleanly (~99 MB).
|
||
- [ ] Re-test on real ARM64 Android TV hardware (still pending — only emulator-verified build).
|
||
|
||
Build cache + scripts live in `android/build-scripts/` and `android/.build-cache/` (junction host + sysconfigdata for each ABI).
|
||
|
||
## Android CI Pipeline
|
||
|
||
Build the Android APK automatically on push/tag.
|
||
|
||
- [x] Generate Gradle wrapper (`gradlew`) and commit it
|
||
- [x] Create CI workflow (`.gitea/workflows/build-android.yml`)
|
||
- JDK 17 + Android SDK + NDK setup
|
||
- Python 3.11 for Chaquopy build
|
||
- Recreate the directory junction via `ln -s` on Linux CI
|
||
- `./gradlew assembleDebug` on master push, `assembleRelease` on `v*` tags (if signing secrets set)
|
||
- Uploads APK as CI artifact; attaches to Gitea release on tag push
|
||
- [x] Commit pre-built pydantic-core wheels to `android/wheels/` (arm64, x86, x86_64)
|
||
- [x] APK signing for release builds — conditional signing config reads keystore from env vars (`ANDROID_KEYSTORE_PATH/_PASSWORD/_ALIAS/_KEY_PASSWORD`), falls back to debug signing locally
|
||
- [ ] Provision a real keystore and add the four CI secrets:
|
||
- `ANDROID_KEYSTORE_BASE64` (base64-encoded .jks)
|
||
- `ANDROID_KEYSTORE_PASSWORD`
|
||
- `ANDROID_KEY_ALIAS`
|
||
- `ANDROID_KEY_PASSWORD`
|
||
- [ ] Add `LedGrab-{tag}-android-release.apk` row to the release description table in `.gitea/workflows/release.yml` → `create-release` job
|
||
- [ ] Verify the CI workflow passes end-to-end with the now-restored multi-ABI build (larger APK, longer Android build step)
|
||
|
||
## Android Root Capture (No Permission Dialog, No System Indicator)
|
||
|
||
MediaProjection shows a mandatory system overlay/indicator while capturing — unavoidable on stock Android. Many cheap Android TV boxes ship pre-rooted, so an alternative root-only path gives much better UX.
|
||
|
||
- [x] Root detection — `Root.kt` checks common `su` binary paths and, on demand, runs `su -c id` to actually prove UID 0. First call triggers Magisk's grant dialog; grant is cached per session. Exposed to Python via Chaquopy.
|
||
- [x] `RootScreenrecord.kt` — spawns `su -c screenrecord --output-format=h264 --size=WxH -`, feeds the H.264 stdout through a MediaCodec decoder whose output Surface is wired into an ImageReader (RGBA_8888, row-stride-aware). Decoded frames reach the Python pipeline via `PythonBridge.pushRootFrame`.
|
||
- [x] Python-side `RootScreenrecordEngine` (`core/capture_engines/root_screenrecord_engine.py`) mirrors `MediaProjectionEngine` with `ENGINE_PRIORITY=110` (> MediaProjection's 100) so the factory picks it automatically when available.
|
||
- [x] `MainActivity` tries `Root.requestGrant()` before launching the MediaProjection consent flow — on rooted devices the consent dialog is skipped entirely. `CaptureService` has a `createRootIntent()` entry point that bypasses the MediaProjection path.
|
||
- [x] Fallback: if `Root.requestGrant()` returns false (no root, user denied, or `su` timeout) the existing MediaProjection flow runs unchanged.
|
||
- [ ] Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) grant dialog appears once, (2) frames actually flow through MediaCodec without the Android 14 capture indicator showing, (3) stop/start cycle terminates the `su` process cleanly.
|
||
- [WONTDO] `SurfaceControl.screenshot()` via reflection — renamed/moved across API 28/29/30/33, hidden-API blocklist varies by release, even rooted apps hit it; days of maintenance for a marginal latency win over the screenrecord path. Not worth it.
|
||
- [WONTDO] `adb screencap` fallback — full-PNG-per-frame pipeline is slower than MediaProjection, no value as a last resort.
|
||
|
||
Known projects using the screenrecord approach for reference: scrcpy (over ADB), scrcpy-hidden-api, shizuku.
|
||
|
||
## Android Autostart on Boot
|
||
|
||
Boot-time, zero-interaction startup so LedGrab always has display capture and control on rooted TV boxes.
|
||
|
||
- [x] Manifest: declare `RECEIVE_BOOT_COMPLETED`, `REQUEST_IGNORE_BATTERY_OPTIMIZATIONS`, `WAKE_LOCK`; register `.BootReceiver` for `BOOT_COMPLETED` / `LOCKED_BOOT_COMPLETED` / `MY_PACKAGE_REPLACED`.
|
||
- [x] `BootReceiver.kt` — gated by `AutostartPrefs` + `Root.looksRooted()`; dispatches `CaptureService.createRootIntent()` via `ContextCompat.startForegroundService`. Unrooted devices are a no-op because MediaProjection consent cannot be bypassed silently.
|
||
- [x] `AutostartPrefs.kt` — thin SharedPreferences wrapper, defaults to enabled. Shown as a CheckBox on the stopped panel; greyed out on unrooted devices.
|
||
- [x] `CaptureService` returns `START_REDELIVER_INTENT` for root-mode intents so the OS can cleanly restart the service after being killed (token-free path). MediaProjection-mode keeps `START_NOT_STICKY` — restart is pointless with a dead consent token.
|
||
- [x] `isRunning` race: moved assignment to after `startForeground` succeeds, resets on exception; `onStartCommand` wraps `startForeground` in try/catch and stops the service cleanly if the FG transition fails.
|
||
- [x] Root-capture watchdog: coroutine on `serviceScope` checks `RootScreenrecord.framesDelivered` every 5s after a 5s grace. Respawns the pipeline (reusing the existing Python bridge) on stall, caps at 3 consecutive restarts before giving up.
|
||
- [x] `RootScreenrecord.framesDelivered` exposed as a property backed by `AtomicInteger` (was `@Volatile var framesDelivered = 0` with non-atomic `+= 1`).
|
||
- [x] `ScreenCapture` accepts `onProjectionStopped` lambda — `MediaProjection.Callback.onStop` now tears the whole service down instead of leaving a stale FG notification.
|
||
- [x] `MainActivity` wires the autostart toggle to `AutostartPrefs`; enabling it prompts `REQUEST_IGNORE_BATTERY_OPTIMIZATIONS` so Doze doesn't kill the FG service on phones.
|
||
- [x] `versionCode` derived from `git rev-list --count HEAD` (or `ANDROID_VERSION_CODE` env var in CI). Was stuck at 1 — sideload updates were silently refusing to install.
|
||
- [ ] Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) boot-time autostart dispatches the service without UI, (2) capture indicator still absent under root mode post-reboot, (3) watchdog respawns the pipeline when `screenrecord` is externally killed, (4) sideload upgrade installs cleanly after the versionCode bump.
|
||
- [ ] Optional follow-up: "kiosk" mode — add `<category android:name="android.intent.category.HOME" />` to `MainActivity` so power users can set LedGrab as the default TV launcher for truly always-running behavior.
|
||
|
||
## Android USB Serial Support
|
||
|
||
Drive USB LED controllers (APA102, WS2812) connected directly to the Android TV box via USB-to-serial adapters.
|
||
|
||
- [x] Added `com.github.mik3y:usb-serial-for-android:3.8.1` (via JitPack) to `android/app/build.gradle.kts`.
|
||
- [x] Kotlin `UsbSerialBridge` singleton (`android/app/src/main/java/com/ledgrab/android/UsbSerialBridge.kt`) — exposes `listDevices()`, `open(vid, pid, serial, baud)`, `write(handle, ByteArray)`, `close(handle)`. Permission request fires automatically from `open()` when the user hasn't granted access yet. Handles are opaque integers, port map is synchronized, so Python threads can share one bridge.
|
||
- [x] Python `AndroidSerialTransport` in `server/src/ledgrab/core/devices/android_serial_transport.py` drives the bridge through Chaquopy. `SerialTransport` Protocol + `PySerialTransport` + `list_serial_ports()` factory live in `serial_transport.py`; `AdalightClient` and `SerialDeviceProvider` now go through the abstraction instead of importing `pyserial` directly.
|
||
- [x] URL scheme extended: `usb:VID:PID[:serial][@baud]` on Android alongside the existing `COM3[:baud]` / `/dev/ttyUSB0[:baud]` desktop paths.
|
||
- [x] App initializes the bridge on startup (`LedGrabApp.onCreate` → `UsbSerialBridge.init(this)`); manifest declares `uses-feature android.hardware.usb.host`.
|
||
- [ ] Real-device test pending — no USB-serial hardware on dev machine. Need to verify on a TV box with CH340, CP2102, or FTDI adapter.
|
||
- [ ] Document supported USB LED controllers in README (once real-device test passes).
|
||
- [ ] Optional: auto-launch the app when a known USB-serial adapter is plugged in (intent-filter on `USB_DEVICE_ATTACHED` + `res/xml/device_filter.xml`). Skipped in v1 — users can just open LedGrab and hit "Discover".
|
||
- [x] ESP-NOW client (`espnow_client.py` / `espnow_provider.py`) now routes through `SerialTransport` — `open_transport()` for the gateway serial link, `list_serial_ports()` + `port_exists()` for discovery/validation. Works transparently with `usb:VID:PID` URLs on Android. (Gateway protocol is write-only, so no `read()` extension was needed after all.)
|
||
|
||
## Performance Metrics Abstraction
|
||
|
||
- [x] `MetricsProvider` protocol + dataclass DTOs (`MemorySnapshot`, `ProcessSnapshot`) live in `server/src/ledgrab/utils/metrics/types.py`. Each provider has its own module: `psutil_provider.py`, `null_provider.py`, `android_provider.py`.
|
||
- [x] Factory `get_metrics_provider()` in `utils/metrics/__init__.py` selects Android → psutil → Null. `psutil` import is now confined to one place.
|
||
- [x] `api/routes/system.py` and `core/processing/metrics_history.py` use the provider; no more `if psutil is not None` guards in the hot paths.
|
||
- [x] Android `/proc`-backed provider implemented (`/proc/stat`, `/proc/meminfo`, `/proc/self/stat`, `/proc/self/status`). Carries previous-sample state for delta-based CPU%; degrades to zeros if any `/proc` file is locked down. 12 unit tests cover both desktop and Android paths.
|
||
|
||
## Android Performance Metrics — Future Enhancements
|
||
|
||
Beyond the `/proc`-based AndroidMetricsProvider that's now in place:
|
||
|
||
- [x] Device battery + thermal-zone readings (`/sys/class/power_supply/battery/{capacity,temp}`, `/sys/class/thermal/thermal_zone*/temp` filtered by zone type). Surfaced through `MetricsProvider.thermals()`, `PerformanceResponse.{cpu_temp_c,battery_percent,battery_temp_c}`, the metrics-history snapshot, and a new dashboard temperature chart that hides itself when the backend reports null. GPU card now hides (no "unavailable" placeholder) when no GPU is present.
|
||
- [WONTDO] Optional: app-specific memory via `Debug.getMemoryInfo()` through a Kotlin → Python Chaquopy bridge (more accurate than `VmRSS` for split-app-process accounting)
|
||
- [WONTDO] Optional: GPU usage via `/sys/class/kgsl/kgsl-3d0/gpubusy` on Adreno, Mali-specific paths for Mali GPUs
|
||
|
||
## Refactor: Per-Provider Device Configs
|
||
|
||
Replace flat `DeviceInfo` + `**kwargs` provider contract with a discriminated union of typed per-provider config dataclasses.
|
||
|
||
- [x] Phase 1 — `DeviceConfig` hierarchy + `Device.to_config()` (non-breaking, additive only)
|
||
- [x] Phases 2+3 — narrow `LEDDeviceProvider.create_client` to typed configs; migrate 3 call sites; delete `DeviceInfo` + `_get_device_info` + `_DEVICE_FIELD_DEFAULTS` (single PR)
|
||
- [x] Phase 4 — migrate `tests/test_group_device.py` to `GroupConfig`/`ProviderDeps`; remove legacy `GroupLEDClient` init path; 47-test config suite with 100% coverage on `device_config.py`
|
||
- [ ] Phase 5 (separate PR, optional) — Pydantic v2 discriminated union in `api/schemas/devices.py`; scope frontend POST/PATCH payloads by `device_type`
|
||
|
||
## Expand device support (Phase 1: open protocols)
|
||
|
||
Branch: `feat/expand-device-support`.
|
||
|
||
Goal: maximize the universe of LED controllers LedGrab can drive by adding aggregator + open-protocol providers in roughly-this order. Each driver follows the established `LEDDeviceProvider` + `*Config` + tests pattern.
|
||
|
||
### Phase 1.1 — Standalone DDP target ✅ shipped (commit `8f1140a`)
|
||
|
||
DDP packet layer (previously WLED-internal) promoted to a first-class device
|
||
type. Pixelblaze, ESPixelStick, xLights/Falcon endpoints, and generic DDP
|
||
receivers are now drivable directly without WLED in the path.
|
||
|
||
### Phase 1.2 — Yeelight LAN
|
||
|
||
Xiaomi/Yeelight bulbs, port 55443 TCP JSON. Direct protocol (no
|
||
`python-yeelight` dependency — implementation is ~200 lines).
|
||
|
||
- [x] `YeelightConfig` dataclass with `yeelight_min_interval_ms` rate limit
|
||
- [x] `YeelightClient` in `core/devices/yeelight_client.py` — TCP JSON-RPC,
|
||
averaging single-pixel adapter, client-side rate gate
|
||
- [x] SSDP-style discovery (Yeelight's variant on `239.255.255.250:1982`)
|
||
- [x] `YeelightDeviceProvider` with validate/health/discover
|
||
- [x] Storage + API schemas + route handler wiring
|
||
- [x] 34 unit tests (URL parsing, RGB packing, averaging, rate limit, SSDP
|
||
parsing, provider validate/discover, Device.to_config round-trip)
|
||
- [ ] Frontend: Yeelight in device-type picker + edit form (spawned to a
|
||
`frontend-design` subagent)
|
||
- [ ] Locale strings (en/ru/zh)
|
||
- [ ] Music mode (~60 Hz updates via reverse-TCP) — follow-up, current
|
||
MVP caps at ~2 Hz via the client-side rate gate
|
||
|
||
### Phase 1.3 — WiZ Connected
|
||
|
||
Philips' UDP-local budget tier. Port 38899 JSON UDP.
|
||
|
||
- [x] `WiZConfig` + `WiZClient` + `WiZDeviceProvider`
|
||
- [x] UDP broadcast discovery on 255.255.255.255:38899 with the standard
|
||
`registration` envelope; replies parsed for IP+MAC.
|
||
- [x] Sync `send_pixels_fast` for the hot loop (UDP is fire-and-forget,
|
||
no async needed). 50 ms default min interval → ~20 Hz cap.
|
||
- [x] Health check sends `getPilot` and waits for any reply.
|
||
- [x] Storage + API schemas + route handler wiring
|
||
- [x] 36 unit tests
|
||
- [ ] Frontend: WiZ in device-type picker + edit form
|
||
- [ ] Locale strings (en/ru/zh)
|
||
|
||
### Phase 2 — Unified discovery + pairing UX layer
|
||
|
||
After phase 1 the codebase will have 3 fresh examples of "ping the LAN, listen for replies, present a list". Factor that out into a generic discovery scaffold + a "first-run pairing" UX component before adding Tuya/Govee/etc., which each need a one-time pairing dance.
|
||
|
||
- [WONTDO] Generic `NetworkDiscoveryService` — the existing
|
||
`/api/v1/devices/discover` route already runs all providers in parallel
|
||
via `asyncio.gather(return_exceptions=True)`. Extracting it would not
|
||
unlock anything; revisit only if discovery cadence/dedup becomes a
|
||
real complaint.
|
||
- [WONTDO] Unified scan UI — already exists; one "Scan network" button
|
||
triggers the cross-provider fan-out.
|
||
- [x] **Reusable pair-device scaffold** (the actually-needed piece).
|
||
Backend: `LEDDeviceProvider.pair_device(url)` abstract method with
|
||
`PairingNotReady` sentinel; `POST /api/v1/devices/pair` endpoint
|
||
with status-code mapping (200/400/409/422/502); 8 route tests
|
||
covering every outcome. Frontend: `templates/modals/pair-device.html`
|
||
five-state modal (idle / pairing / not-ready / success / failed)
|
||
with a 30-second SVG progress ring; reusable
|
||
`static/js/features/pairing-flow.ts` exposing
|
||
`runPairingFlow({deviceType, url}) → Promise<{fields}>` with
|
||
`PairingCancelled` sentinel; locale strings in en/ru/zh. No driver
|
||
uses it yet — Nanoleaf will be the first concrete consumer.
|
||
|
||
### Phase 3 — Big aggregator unlocks
|
||
|
||
- [ ] ESPHome native API (`aioesphomeapi`)
|
||
- [ ] Tuya Local (`tinytuya`) — biggest single market unlock; needs the pairing UX from Phase 2
|
||
- [ ] Matter over IP (forward-looking)
|
||
- [ ] Hyperion JSON downstream
|
||
|
||
### Phase 4 — Major consumer brands
|
||
|
||
- [x] **LIFX LAN** — UDP binary protocol on port 56700; RGB→HSBK 16-bit
|
||
conversion; broadcast discovery via GetService/StateService probe;
|
||
47 unit tests. Single-pixel adapter shape, identical to WiZ
|
||
structurally. Frontend wired via subagent.
|
||
- [x] **Govee LAN API** — UDP JSON on port 4003 (control) + 4002
|
||
(responses) + 4001 (multicast discovery on 239.255.255.250).
|
||
Single-pixel `colorwc` command with `colorTemInKelvin=0` for RGB
|
||
mode. **Per-device "LAN Control" toggle required in Govee Home
|
||
app.** 40 unit tests. Frontend wired via subagent.
|
||
- [x] **Nanoleaf OpenAPI** — Light Panels / Canvas / Shapes / Lines /
|
||
Elements via HTTP REST on port 16021. **First concrete user of
|
||
the pairing-UX scaffold from Phase 2.** mDNS discovery via
|
||
`_nanoleafapi._tcp`. Single-pixel adapter (averaged strip → HSB
|
||
`PUT /state`). Auth token encrypted at rest via `_enc`/`_dec`.
|
||
42 unit tests covering URL parsing, RGB→HSB conversion, pairing
|
||
handshake (200/403/500/missing-token/transport-error), state
|
||
mutations, brightness clamping, Device.to_config round-trip
|
||
including encrypted-token roundtrip.
|
||
- [ ] Twinkly — multi-pixel + login flow; deferred
|
||
- [WONTDO] Mi-Light / MiBoxer UDP gateway — the recommended path for
|
||
modern Mi-Light deployments is `esp8266_milight_hub` firmware → MQTT,
|
||
which LedGrab already supports through the existing MQTT device target
|
||
(commit `530316c`). Native V6 driver would be ~400 lines + finicky
|
||
session protocol + custom 1-byte hue table; the marginal benefit over
|
||
the MQTT path is small. Revisit if a user complaint surfaces.
|
||
|
||
### Phase 5 — Open pixel protocols (cheap completionism)
|
||
|
||
- [x] **OPC (Open Pixel Control)** — TCP, port 7890, 4-byte header
|
||
`[channel][cmd][len_hi][len_lo]` + RGB body. Channel 0 broadcasts.
|
||
Single-pixel-strip protocol, no discovery, no pairing. 36 unit
|
||
tests. Fadecandy + xLights + hobbyist receivers reachable.
|
||
- [ ] TPM2.net
|
||
|
||
### Phase 6 — PC gaming RGB completion
|
||
|
||
- [ ] Corsair iCUE SDK
|
||
- [ ] Logitech LIGHTSYNC
|
||
- [ ] ASUS Aura SDK
|
||
|
||
### Phase 7 — Proprietary USB HID ambient kits
|
||
|
||
- [ ] Generic HID-ambient framework + VID/PID registry
|
||
- [ ] First reverse-engineered target (probably Govee Immersion / DreamView)
|
||
|
||
### Cleanup + verification
|
||
|
||
- [x] **`_average_color` extraction** (commit `cc87fba`). Six identical
|
||
copies (Yeelight / WiZ / LIFX / Govee / Nanoleaf / BLE) collapsed
|
||
into `core/devices/pixel_reduce.average_color`. Net -76 lines.
|
||
Hue is out by design — its Entertainment API addresses up to seven
|
||
lights individually.
|
||
- [x] **Pre-merge verification pass.** 1358 pytest tests pass; ruff
|
||
clean across all device modules and tests; black clean against
|
||
the pre-commit-pinned 24.10.0; `npx tsc --noEmit` clean; bundle
|
||
compiles.
|
||
- [x] **Pre-merge code review (subagent)** — surfaced 2 CRITICAL +
|
||
4 HIGH + 3 MEDIUM + 3 LOW findings.
|
||
- [x] **All review findings fixed** (commits `7736bc6` + `0e3ae78`):
|
||
- CRITICAL #1: missing `url_scheme.py` / `net_classify.py`
|
||
committed (4 files / 557 lines).
|
||
- CRITICAL #2: `update_device` no longer re-encrypts secrets in
|
||
memory via the `to_dict()` round-trip (uses `vars()` directly).
|
||
- HIGH #3: `nanoleaf_token` / `hue_username` / `hue_client_key`
|
||
stripped from `DeviceResponse`; replaced with paired-flag
|
||
booleans. Frontend updated.
|
||
- HIGH #4: `validate_lan_host()` rejects literal public IPs at
|
||
each driver's `validate_device` + `pair_device`.
|
||
- HIGH #5: `_dec()` failures clear the field and log, not crash
|
||
the row.
|
||
- HIGH #6: update route now rstrip's URL for all device types.
|
||
- MEDIUM #7: Govee discovery serialized via `asyncio.Lock`.
|
||
- MEDIUM #8: Nanoleaf mDNS browser cleanup moved to `finally`.
|
||
- MEDIUM #9: pair endpoint sanitizes URL userinfo in logs.
|
||
- LOW: Nanoleaf `.port` property added; pair-then-create E2E test
|
||
added.
|
||
- Tests: 1379 pass (+21 regression tests).
|
||
|
||
## Graph editor — "full control of wiring via graph" (in progress)
|
||
|
||
Goal: make the visual graph a first-class wiring control surface, not just a
|
||
viewer. Driven by the ULTRA-DEEP review (findings A1–A5, B1–B6, C1–C6, D1–D6).
|
||
|
||
### Done (NOT yet committed — awaiting review/commit)
|
||
|
||
- [x] **A1** Undo/redo wired to connect/detach/move (was dead code); inverse ops
|
||
throw on failure so the stack can't silently desync.
|
||
- [x] **A2** Manual node layout persists to `localStorage` (`graph_node_positions`),
|
||
cleared on relayout.
|
||
- [x] **A3** Scene-preset disambiguation — deactivation scene now reachable via a
|
||
field picker (was always picking the first match).
|
||
- [x] **B6** Edge field labels (revealed on zoom ≥ 0.9).
|
||
- [x] **C3** Health overlay — broken refs (referrer exists, target missing),
|
||
dependency cycles, orphans; node warning badges + an issues toolbar button.
|
||
- [x] **D1** `GET /api/v1/graph/schema` — authoritative connectable-field registry
|
||
(`api/graph_schema.py`, pure + unit-tested).
|
||
- [x] **D2** `GET /api/v1/graph` (nodes+edges+validation) and
|
||
`GET /api/v1/graph/dependents/{kind}/{id}`.
|
||
- [x] **D4** `POST /api/v1/graph/validate-connection` — existence + source-kind +
|
||
cycle pre-flight; frontend validates before every write (fails open if the
|
||
endpoint is unreachable). List/double-nested fields rejected.
|
||
- [x] **B2** Drop-on-node connect — empty top-level slots are now wireable (drop a
|
||
source onto any compatible node body, not just an existing port).
|
||
- [x] **C4** Overwrite-occupied-slot confirm + delete-with-dependents warning
|
||
(single delete only; bulk keeps the batch confirm).
|
||
- [x] **D5** Create-and-connect — drag a port onto empty canvas → pick a compatible
|
||
new entity kind → it's created and auto-wired (kind-scoped watcher).
|
||
- [x] **D6 (read-only half)** "Export graph (JSON)" toolbar action.
|
||
- [x] Custom per-entity `icon` + `icon_color` now render on graph nodes (parity
|
||
with custom node colours; fallback to kind/subtype glyph).
|
||
- [x] **B1** Edit single-level **BindableFloat** value slots from the graph
|
||
(`brightness`, `smoothing`, `intensity`, `scale`, `speed`, … on
|
||
color_strip_source; `brightness`/`transition` on output_target). Subtype-safe
|
||
(only offers slots the target entity actually has). Writes the partial
|
||
`{ <slot>: { source_id } }` payload → backend `Bindable*.apply_update` merges,
|
||
preserving the static value. Verified data-safe (no `from_raw`/value-reset path).
|
||
- [x] Render the two functional value-source references `buildGraph` was missing —
|
||
`value_source.value_source_id` (gradient_map → inner value source) and
|
||
`value_source.color_strip_source_id` (css_extract → strip). Both are runtime-
|
||
resolved and already drag-editable; now visible/detachable in the graph.
|
||
- [x] **B4 foundation:** backend schema now authoritative about graph-editability
|
||
(`is_editable()` + `editable` flag in `/graph/schema`); `validate-connection`
|
||
hardened to reject non-editable fields (colour/list/double-nested), not just lists.
|
||
- [x] **B4 drift guard + gap fixes:** `checkSchemaDrift()` (graph-connections.ts) warns
|
||
once if the frontend `CONNECTION_MAP` editable set diverges from `/graph/schema`
|
||
(the automated "10-step checklist"). Surfacing it found 3 real gaps; fixed 2:
|
||
`color_strip_source.input_source_id` + `processing_template_id` are now drag-editable
|
||
(processed-strip wiring; `apply_update` is partial-safe). The 3rd —
|
||
`device.default_css_processing_template_id` — is intentionally NOT drag-editable
|
||
(the device PUT route isn't partial-safe; a one-field PUT could null the URL) and is
|
||
in the drift-check exclude set. Also broadened `_availableMatches` to hide any slot
|
||
the target entity doesn't expose (subtype-accurate; refs are always-emitted so empty
|
||
slots stay wireable). Review also caught a **dead `output_target.picture_source_id`
|
||
slot** (no output target stores it — not a field/schema, never emitted) — removed
|
||
from both registries + `buildGraph`.
|
||
- [x] **Comprehensive review pass (4 subagents: backend/frontend-core/orchestrator/security).**
|
||
Findings fixed:
|
||
- **CRITICAL (security):** `GET /api/v1/graph` leaked plaintext **webhook tokens**
|
||
(`asdict` recursed `Automation.rules[].token`, an auth-equivalent secret). Fixed with
|
||
**field-projection** — `serialize_entity_for_graph()` / `graph_field_roots()` project
|
||
each entity to only `{id, name, subtype, reference-roots}`; secrets can't survive.
|
||
Added a structural regression test asserting no projection root is secret-bearing for
|
||
any kind (drift-proof boundary) + a token-drop test.
|
||
- MEDIUM: added missing `value_source.clock_id` (AnimatedColorValueSource → sync_clock)
|
||
to the backend registry for topology/dependents completeness (drift-excluded on the
|
||
frontend — value-source PUT needs a `source_type` discriminator, so it's editor-only).
|
||
- MEDIUM/LOW: `CSS.escape` on the markIssues id selector; grouped/clarified
|
||
`_DRIFT_EXCLUDE`; fixed the stale `_availableMatches` JSDoc; documented the
|
||
`checkSchemaDrift` forward-reference. Orchestrator + frontend-core + security: APPROVE.
|
||
- Verification: `npm --prefix server run typecheck` + `run build` clean; ruff clean;
|
||
graph backend tests 35 pass; full backend suite green. ~8 code-review passes,
|
||
all CRITICAL/HIGH findings fixed.
|
||
|
||
### Left to do (deferred)
|
||
|
||
- [x] **BindableColor slots** — CHECKED, decision: keep read-only (won't fix).
|
||
Value sources are scalar-only (`ValueStream.get_value() -> float`) and every
|
||
colour consumer (`color_strip/single.py`, `effect_stream.py`) reads the static
|
||
RGB via `bcolor()`, ignoring `source_id`. So a value_source cannot drive a
|
||
colour — wiring `color`/`color_peak`/… would be a dead binding. Documented in
|
||
`api/graph_schema.py` next to the BindableColor entries. (Would only become
|
||
viable if a colour-producing value-source type is added.)
|
||
- [~] **B4 — delete the frontend `CONNECTION_MAP` duplication.**
|
||
- [x] **Foundation done:** the backend schema now carries an authoritative
|
||
`editable` flag per field (`is_editable()` in `api/graph_schema.py`, mirroring
|
||
the frontend `_isEditable`: top-level refs + single-level BindableFloat slots;
|
||
NOT colour/list/double-nested). `validate-connection` is hardened to reject any
|
||
non-editable field (was list-only). `editable` is surfaced in `/graph/schema`.
|
||
- [ ] **Remaining (the refactor):** frontend fetches `/graph/schema` on load and
|
||
derives connection metadata + edges from it (port the `extract_refs` dot-path/list
|
||
grammar to TS), keeping only a tiny `kind → {endpoint, cache}` write-routing table;
|
||
then delete the field-level `CONNECTION_MAP` + the `buildGraph` edge loops
|
||
(graph-connections.ts / graph-layout.ts). Removes the 10-step sync checklist in
|
||
`contexts/graph-editor.md`. **A backend apply-write endpoint is NOT required** —
|
||
keep the proven per-entity PUT. Risk: regressing drag-connect/bindable; keep a
|
||
dev drift-check (frontend editable set vs `/graph/schema`) during the transition.
|
||
Note: frontend `CONNECTION_MAP` also has inert `ha_source_id`/`gradient_id` entries
|
||
(no graph node kind) — drop them, the backend schema already omits them.
|
||
- [ ] **D6 — blueprint import/instantiate.** Export exists; the apply half (serialize
|
||
a selected subgraph's topology + entities, re-import with id remapping, conflict
|
||
handling) is large and data-integrity-sensitive (see Data Migration Policy in
|
||
CLAUDE.md). Scope as its own feature.
|
||
- [ ] **List-slot editing** (composite `layers[]`, mapped `zones[]`, scene preset
|
||
`targets[]`) — needs an element index in the write + validate paths
|
||
(`validate_connection` currently rejects list fields). Edit via entity modal
|
||
for now.
|
||
|
||
### Notes / decisions
|
||
|
||
- The backend `CONNECTION_SCHEMA` (`api/graph_schema.py`) is the authoritative
|
||
superset; it already declares the bindable + list + value_source-chain edges. The
|
||
frontend `CONNECTION_MAP` still owns write-routing (endpoint/cache) — that's the
|
||
only reason it survives (see B4).
|
||
- Bindable edges render dashed (`.graph-edge-nested`) but ARE editable — the dashed
|
||
style intentionally distinguishes value bindings from structural edges.
|
||
- `validate-connection` and `dependents` fail **open/safe** on the frontend so the
|
||
graph keeps working against an older server without these endpoints.
|