Compare commits

...

13 Commits

Author SHA1 Message Date
alexei.dolgolyov 4b2e8fc5ec docs(android): add audio-capture design + missing-functionality review
- android-audio-capture-plan.md — design behind the merged on-device audio
  capture feature (487259a).
- android-missing-functionality.md — Android missing-feature review notes.
2026-06-02 03:30:43 +03:00
alexei.dolgolyov 487259a96d Merge feature/android-audio-capture: Android on-device audio capture
Push-based AndroidAudioEngine (AudioPlaybackCapture, API 29+) reusing the
MediaProjection token, feeding the unchanged AudioAnalyzer. Build green
(assembleDebug), 1854+13 tests pass. Reviewed: READY/SECURE, 0 blockers.
2026-06-02 03:28:37 +03:00
alexei.dolgolyov fd62db1720 feat(audio): Android on-device system playback capture
Enable audio-reactive lighting on the Android-TV build. A push-based
AndroidAudioEngine captures system playback audio via AudioPlaybackCapture
(API 29+), reusing the existing MediaProjection token, and feeds PCM into
the unchanged AudioAnalyzer pipeline. No new Python deps; no Chaquopy/pip
changes (numpy already bundled).

- Python: android_audio_engine.py — module-level queue + configure/
  push_samples/shutdown mirroring mediaprojection_engine; AndroidAudioEngine
  (priority 100) registered behind a guarded import. push_samples copies and
  defensively trims/clamps each block so the analyzer can't crash on
  variable-length or non-frame-divisible PCM.
- Kotlin: AudioCapture.kt — AudioRecord + AudioPlaybackCaptureConfiguration,
  fixed chunk-size block framing, little-endian float32, mic fallback;
  reads back the actual negotiated channel/sample rate. PythonBridge gains
  configureAudio/pushAudio/shutdownAudio with a cached module handle.
- Wiring: CaptureService starts/stops AudioCapture in the MediaProjection
  path (gated on API>=29 + RECORD_AUDIO + live projection); MainActivity
  requests RECORD_AUDIO; manifest declares it. Degrades gracefully when
  denied; root path stays audio-less by design.
- Tests: 13 desktop-CI tests incl. an over-length/non-divisible regression
  guard that exercises the full read_chunk -> AudioAnalyzer.analyze path.
2026-06-02 03:28:22 +03:00
alexei.dolgolyov 669ae20824 feat(value-sources): optional normalization for magnitude sources
Add an opt-out `normalize` flag to the four magnitude value sources
(ha_entity, http, system_metrics, game_event). get_value() stays in
[0,1] for every source (the normalized scalar-bus invariant), so all
existing consumers — brightness sinks, gradient_map, template `name`
bindings, and color-strip bindable floats — are unaffected.

- normalize=False is a clamp-passthrough: skip the min/max rescale and
  clamp the raw reading into [0,1] (for sources already reporting a
  0..1 fraction). The un-normalized magnitude stays available via
  get_raw_value() / template raw[name] / automations.
- Add get_raw_value() + a raw channel to GameEventValueStream (it had
  none). game_event's flag is model/stream-level only (no CRUD schema).
- Finite-safe clamp01() util; harden the composite-layer brightness
  multiply (latent negative-wrap / >=1 skip) with it.
- Preview WebSocket: tolerate non-numeric raw, generalize raw-range.
- Frontend: settings-toggle slider per HA/system_metrics/http editor
  with min/max grey-out; toggle hidden for fixed-mapping percent
  metrics. en/ru/zh locale keys.
- Additive optional field (default True) — JSON round-trip, no migration.

Tests: store create/update round-trip, clamp-passthrough, live
normalize flip, game_event raw channel + build_stream forwarding,
and finite-safe clamp01.
2026-06-02 02:24:40 +03:00
alexei.dolgolyov 6de61b965e feat(value-sources): add sandboxed-Jinja template combinator
A new `template` value source evaluates a hardened, sandboxed Jinja
expression over the live values of other value sources — the system's
first float combinator.

Backend:
- Shared engine (utils/template_expr.py): ImmutableSandboxedEnvironment with
  filters/tests and auto-injected globals stripped; only min/max/abs/round/
  clamp exposed; rejects **, string/collection-literal repetition, attribute
  access and non-global calls; NaN/inf-safe result coercion.
- TemplateValueSource model + TemplateValueStream runtime: compile-once,
  primitives-only eval context, raw[name] exposure, eval_interval throttle,
  ref-counted input acquire/release, rename-safe hot-update.
- Validation: unbound-variable + reserved-name rejection, reference
  cycle/depth guards (depth-only at create, full cycle at update), runtime
  acquire() depth backstop, and delete referential-integrity.
- API: Create/Update/Response schemas + discriminated unions, _RESPONSE_MAP,
  and an advisory POST /value-sources/validate-template endpoint.
- Demo seed: a static source plus a template combinator example.

Frontend:
- Editor modal section: repeatable inputs list (EntitySelect rows), a
  zero-dependency Jinja syntax highlighter, a hints/reference panel, and a
  debounced live validator that gates Save (stale-response-safe).
- Graph editor: read-only template node with one edge per input.
- i18n (en/ru/zh), icon, and card rendering.

Tests: engine, stream, factory/cycle, validate endpoint, and demo seed.
2026-06-01 18:53:56 +03:00
alexei.dolgolyov 12b40e6071 docs: actualize README and API reference, embed screenshots
README: add free & open-source (MIT) framing; add a Platforms table (Windows/Linux/macOS/Docker supported, Android-TV build marked experimental, scrcpy phone-capture retained); expand the LED device list to ~20 protocols; correct storage to SQLite, the auth model, and resource names (Output Targets, Automations); add a prebuilt-download section; remove the Architecture and Acknowledgments sections; embed dashboard, channels, live-preview, and device-picker screenshots.

docs/API.md: full rewrite from the live route modules — all 253 endpoints across 34 modules, grouped by resource with REST + WebSocket tables, accurate auth model and WS handshake, worked examples, and sensitive-endpoint markers. Replaces the stale v0.1.0 stub.

server/CLAUDE.md: storage is now SQLite (BaseSqliteStore / ledgrab.db / LEDGRAB_DATA_DIR / data_migrations.py); fix the auth description (loopback anonymous, LAN rejected with 401 — not all endpoints open); router registration happens in api/__init__.py.
2026-05-29 14:35:45 +03:00
alexei.dolgolyov 498854f04d refactor(storage): gate clone() behind an opt-in allowlist; expand duplicate tests
Follow-up polish from the duplicate-subgraph review:

- BaseSqliteStore.clone() now requires an opt-in `_cloneable` flag (default
  off), so a new or secret-bearing store can never be cloned by accident —
  defence-in-depth on top of the endpoint's `_DUPLICABLE_KINDS` restriction.
  Only value-source and colour-strip stores are flagged.
- Fix `_check_name_unique` annotation (`str | None = None`).
- Add tests: `layers[].brightness_source_id` remap, the warnings safety net
  firing, the clone allowlist invariant, and clone() refusing a non-cloneable
  store.
2026-05-29 11:55:58 +03:00
alexei.dolgolyov 15cfb821d3 feat(graph): duplicate a selected subgraph server-side
A secret-safe equivalent of blueprint import: the graph editor's overflow menu
gains "Duplicate selection", which deep-clones the selected value and
colour-strip sources server-side (full config preserved, never crossing the
wire) and rewires references that point within the selection — shared
dependencies (devices, HA sources, …) stay shared.

- graph_schema.remap_refs: write-twin of extract_refs (same dot/list/bindable
  grammar) that rewrites only in-selection ids; 8 unit tests.
- BaseSqliteStore.clone(): faithful deep-copy clone (no schema round-trip, so no
  field is lost), prefix-preserving fresh id; reusable by any store.
- POST /api/v1/graph/duplicate: two-pass clone-then-rewire restricted to value /
  colour-strip sources (no inline secrets), with a safety net flagging any
  unremapped reference; 7 integration tests vs real stores.
- Frontend: duplicateSubgraph (+cache invalidation), graphDuplicateSelection
  (reload + reselect the new cluster), overflow-menu item, i18n (en/ru/zh).
2026-05-29 11:45:55 +03:00
alexei.dolgolyov 2e51f46dfd feat(graph): make the visual editor a full wiring control surface
Lets users wire the system end-to-end from the graph, and fixes the core
bug that made drag-to-wire silently fail.

- Fix drag-to-wire 422s across 5 entity kinds: updateConnection() now echoes
  the target's discriminator (source_type/stream_type/target_type) into the
  partial PUT, so value/colour-strip/audio/picture sources and output targets
  all wire correctly. New contract test (54 cases) in test_graph_wiring_contract.py.
- Re-wire composite layers / mapped zones from the graph (right-click a
  layer/zone source edge -> Re-wire). Whole-list write preserves every sibling
  layer/zone setting, with an optimistic-concurrency guard and undo.
- Secret-safe /graph topology: project entities to id/name/subtype + reference
  roots so the endpoint cannot leak webhook tokens or other credentials.
- Carry slot indices on list edges; node custom-icon + schema-drift refinements;
  rewire i18n keys (en/ru/zh); wiring-control roadmap (TODO.md).
2026-05-29 02:29:19 +03:00
alexei.dolgolyov 05cf121666 fix(installer): open WebUI once after "Launch LedGrab"
The finish-page launch action started the app and also opened
http://localhost:8080/ itself, but the app already auto-opens the
browser on a manual (non-autostart) launch, so the page appeared
twice. Drop the installer's redundant open and let the app handle it
(it waits on /health, which is more reliable than a fixed sleep).
2026-05-28 23:52:52 +03:00
alexei.dolgolyov d505388f0e docs: graph-editor wiring-control roadmap (review findings A1-D6) 2026-05-28 23:38:04 +03:00
alexei.dolgolyov 6aeda935f1 chore: release v0.8.1
Build Android APK / build-android (push) Failing after 11s
Build Release / create-release (push) Successful in 6s
Build Release / build-docker (push) Successful in 2m57s
Build Release / build-linux (push) Successful in 1m49s
Build Release / build-windows (push) Successful in 3m35s
Build Release / publish-release (push) Successful in 2s
2026-05-28 23:35:35 +03:00
alexei.dolgolyov a5effba553 feat: aggregated snapshot + wiring-graph APIs, MQTT device brokers
Backend
- snapshot: GET /api/v1/snapshot aggregates targets, devices, sources,
  presets and system into one payload for the HA coordinator, collapsing
  the prior ~2N+M request fan-out; per-section ?include= gating.
- graph: GET /api/v1/graph{,/schema,/dependents} backed by a pure,
  unit-tested graph_schema engine — one authoritative connectable-field
  registry so the editor no longer hard-codes topology in two places.
- devices: thread mqtt_source_id through DeviceCreate/Update/Response and
  the routes for multi-broker MQTT; shared validate_mqtt_source_exists
  (_mqtt_validation.py) reused by device + output-target routes; stop
  update_device masking intentional 4xx as 500.
- shutdown: bound uvicorn graceful-shutdown via GRACEFUL_SHUTDOWN_TIMEOUT
  (shared by __main__, android_entry, demo) so a lingering events WebSocket
  can't strand LED targets or block process exit.
- access log: structured _access_log middleware attributing each request to
  its authenticated token label (never the secret); uvicorn access_log off.

Frontend
- graph editor: generic schema-driven port/edge rendering, layout and
  connection handling; service-worker refresh.
- device modals: MQTT broker EntitySelect for device_type=mqtt in add-device
  and settings, wired into load/save/validate/dirty-check/clone.
- i18n: en/ru/zh keys.

Tests: graph routes + schema, snapshot routes, access log, mqtt_source_id
device regressions, bounded-shutdown entrypoint. 1614 passed.
2026-05-28 22:51:04 +03:00
90 changed files with 9920 additions and 642 deletions
@@ -0,0 +1,308 @@
# Plan: Android on-device audio capture
> Status: proposed plan (not yet approved). No code changes. Last updated 2026-06-01.
## Context
LedGrab's audio-reactive features (music analyzer, audio value sources, band filters)
depend on capturing an audio stream and running it through `AudioAnalyzer`
(`server/src/ledgrab/core/audio/analysis.py`). On desktop this is fed by **WASAPI**
(Windows) or **Sounddevice/PortAudio** (cross-platform). On the **experimental
Android-TV build** neither is available — `sounddevice` has no Chaquopy wheel and PortAudio
isn't bundled — so `core/audio/__init__.py` registers only `DemoAudioEngine`, and
audio-reactive lighting is effectively dead on Android.
Android does not need PortAudio: the platform exposes **`AudioPlaybackCapture`** (API 29+),
which captures system playback audio and **takes a `MediaProjection` token — the very token
the app already obtains for screen capture** (`ScreenCapture(projection, …)`). This plan adds
a push-based Android audio engine so the TV box can drive sound-reactive lighting from its own
media playback, at parity with how desktop audio feeds the analyzer.
The design mirrors the working screen-capture bridge
(`mediaprojection_engine.py``ScreenCapture.kt``PythonBridge`) and the existing audio
engine abstraction (`AudioCaptureEngine` / `AudioCaptureStreamBase` /
`AudioEngineRegistry`). **No new Python dependencies** (`numpy` is already bundled) → no
Chaquopy / `build.gradle.kts` `pip {}` changes.
---
## Approach
A new **push-based** audio engine registered in the existing `AudioEngineRegistry`:
- **Python:** `AndroidAudioEngine` + `AndroidAudioCaptureStream` mirroring `SounddeviceEngine`,
but `read_chunk()` pops PCM from a module-level queue that **Kotlin fills** (mirror of
`mediaprojection_engine.push_frame`). High `ENGINE_PRIORITY` so
`AudioEngineRegistry.get_best_available_engine()` selects it on Android. The existing
`ManagedAudioStream` capture loop and `AudioAnalyzer` consume `read_chunk()` unchanged.
- **Android:** an `AudioCapture` helper using `AudioRecord` + `AudioPlaybackCaptureConfiguration`
(reusing `CaptureService`'s `MediaProjection`), pushing float32 PCM to Python. Mic
(`AudioSource.MIC`) fallback. Wired into `CaptureService` next to `ScreenCapture`.
```
[media playback] → AudioRecord (AudioPlaybackCapture, reuses MediaProjection)
→ AudioCapture.kt → PythonBridge.pushAudio(pcmFloat32, frames, channels)
→ android_audio_engine.push_samples() [module-level queue]
→ AndroidAudioCaptureStream.read_chunk() → ManagedAudioStream → AudioAnalyzer [unchanged]
```
---
## Part A — Python (server)
**New file: `server/src/ledgrab/core/audio/android_audio_engine.py`** — mirror
`mediaprojection_engine.py` (queue + configure + push) and `sounddevice_engine.py` (engine/stream shape):
```python
import queue
import numpy as np
from typing import Any, Dict, List
from ledgrab.core.audio.base import AudioCaptureEngine, AudioCaptureStreamBase, AudioDeviceInfo
from ledgrab.utils import get_logger
logger = get_logger(__name__)
_pcm_queue: "queue.Queue[np.ndarray]" = queue.Queue(maxsize=8)
_sample_rate = 48000
_channels = 2
_chunk_size = 1024
_active = False
def configure(sample_rate: int, channels: int, chunk_size: int) -> None:
"""Called from Kotlin before audio frames start flowing. Drains stale PCM."""
global _sample_rate, _channels, _chunk_size, _active
while not _pcm_queue.empty():
try: _pcm_queue.get_nowait()
except queue.Empty: break
_sample_rate, _channels, _chunk_size = sample_rate, channels, chunk_size
_active = True
def push_samples(pcm_float32: bytes) -> None:
"""Push one interleaved float32 PCM chunk from Kotlin. Drops oldest if full."""
samples = np.frombuffer(pcm_float32, dtype=np.float32)
try:
_pcm_queue.put_nowait(samples)
except queue.Full:
try: _pcm_queue.get_nowait()
except queue.Empty: pass
try: _pcm_queue.put_nowait(samples)
except queue.Full: pass
def shutdown() -> None:
global _active
_active = False
class AndroidAudioCaptureStream(AudioCaptureStreamBase):
@property
def channels(self) -> int: return _channels
@property
def sample_rate(self) -> int: return _sample_rate
@property
def chunk_size(self) -> int: return _chunk_size
def initialize(self) -> None:
if not _active:
raise RuntimeError("Android audio engine not configured (only valid in-app).")
self._initialized = True
def cleanup(self) -> None:
self._initialized = False
def read_chunk(self) -> np.ndarray | None:
try:
return _pcm_queue.get(timeout=0.1) # 1-D float32 interleaved
except queue.Empty:
return None
class AndroidAudioEngine(AudioCaptureEngine):
ENGINE_TYPE = "android_playback"
ENGINE_PRIORITY = 100 # highest on Android (demo is lower)
@classmethod
def is_available(cls) -> bool:
from ledgrab.utils.platform import is_android
return is_android() and _active
@classmethod
def get_default_config(cls) -> Dict[str, Any]:
return {"sample_rate": _sample_rate, "channels": _channels, "chunk_size": _chunk_size}
@classmethod
def enumerate_devices(cls) -> List[AudioDeviceInfo]:
if not cls.is_available(): return []
return [AudioDeviceInfo(index=0, name="Android playback (system audio)",
is_input=True, is_loopback=True,
channels=_channels, default_samplerate=float(_sample_rate))]
@classmethod
def create_stream(cls, device_index, is_loopback, config) -> AndroidAudioCaptureStream:
return AndroidAudioCaptureStream(device_index, is_loopback, {**cls.get_default_config(), **config})
```
**Modify `server/src/ledgrab/core/audio/__init__.py`** — register behind a guarded import,
matching the existing `_has_wasapi` / `_has_sounddevice` pattern:
```python
try:
from ledgrab.core.audio.android_audio_engine import AndroidAudioEngine
_has_android_audio = True
except ImportError:
_has_android_audio = False
...
if _has_android_audio:
AudioEngineRegistry.register(AndroidAudioEngine)
```
**Reused, unchanged:** `AudioEngineRegistry.get_best_available_engine()` (picks by priority),
`ManagedAudioStream._capture_loop()` (`audio_capture.py`), `AudioAnalyzer`, the audio value
sources, and the device-enumeration endpoints. The Android engine appears as one loopback
device named "Android playback (system audio)".
---
## Part B — Android (Kotlin + manifest)
**New file: `android/app/src/main/java/com/ledgrab/android/AudioCapture.kt`**
Mirrors `ScreenCapture.kt`, taking the same `MediaProjection`:
```kotlin
class AudioCapture(
private val projection: MediaProjection,
private val bridge: PythonBridge,
private val sampleRate: Int = 48000,
private val channels: Int = 2,
private val chunkFrames: Int = 1024,
)
```
- `start()` (API 29+, MediaProjection mode):
- Build `AudioPlaybackCaptureConfiguration(projection)` adding usages
`USAGE_MEDIA`, `USAGE_GAME`, `USAGE_UNKNOWN` (the capturable set).
- `AudioRecord.Builder().setAudioPlaybackCaptureConfig(cfg)` with
`AudioFormat(ENCODING_PCM_FLOAT, sampleRate, CHANNEL_IN_STEREO)`.
- On a dedicated `HandlerThread`, loop `audioRecord.read(floatBuf, …, READ_BLOCKING)`
wrap into a little-endian float32 `ByteArray` (reusable buffer, like `ScreenCapture`'s
`frameBuffer`) → `bridge.pushAudio(bytes, framesRead, channels)`.
- `stop()`: stop/release `AudioRecord`, quit the thread.
- **Mic fallback** (`startMic()`): `AudioSource.MIC` for root mode (no MediaProjection) or
API < 29. Used only when playback capture is unavailable.
**Modify `android/app/src/main/java/com/ledgrab/android/PythonBridge.kt`** — add the audio
push path (same shape as `pushFrame`, with a cached PyObject handle):
```kotlin
@Volatile private var androidAudioEngine: PyObject? = null
fun configureAudio(sampleRate: Int, channels: Int, chunkFrames: Int) {
val engine = Python.getInstance().getModule("ledgrab.core.audio.android_audio_engine")
engine.callAttr("configure", sampleRate, channels, chunkFrames)
androidAudioEngine = engine
}
fun pushAudio(pcmFloat32: ByteArray, frames: Int, channels: Int) {
if (!running) return
androidAudioEngine?.let {
try { it.callAttr("push_samples", pcmFloat32) }
catch (e: Exception) { Log.w(TAG, "pushAudio failed: ${e.message}") }
}
}
```
**Modify `android/app/src/main/java/com/ledgrab/android/CaptureService.kt`** — in the
MediaProjection start path (where `ScreenCapture` is created with the projection), if
`RECORD_AUDIO` is granted and API ≥ 29, also `bridge.configureAudio(...)` and start an
`AudioCapture(projection, bridge)`. Stop/release it in `onDestroy` alongside `ScreenCapture`.
Root path → optional mic fallback (or skip; see Risks).
**Modify `android/app/src/main/AndroidManifest.xml`:**
```xml
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<!-- For mic-mode foreground capture on API 34+ (playback capture is covered by the
existing mediaProjection FGS type): -->
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
```
The existing `CaptureService` already declares `foregroundServiceType="mediaProjection|specialUse"`
and holds `FOREGROUND_SERVICE_MEDIA_PROJECTION`; add `microphone` to the type only if mic
fallback is implemented.
**Modify `MainActivity.kt`** — request `RECORD_AUDIO` at runtime alongside the existing
`ensureNotificationPermission()` (POST_NOTIFICATIONS) flow, before starting capture. Capture
proceeds without audio if denied (graceful degradation).
---
## Orchestration decision (the main trade-off)
Desktop starts audio capture **on demand** when an audio-reactive source is acquired
(`AudioCaptureManager.acquire`). On Android, PCM only flows if Kotlin has set up `AudioRecord`.
- **MVP (recommended):** start `AudioCapture` when `CaptureService` starts (if `RECORD_AUDIO`
granted + MediaProjection mode + API ≥ 29) and push continuously; the bounded queue drops
frames when no audio source consumes them. Simplest; modest extra CPU.
- **Future optimization:** on-demand start/stop signaled Python→Kotlin (Chaquopy can call
Kotlin, as `BleBridge`/`UsbSerialBridge` show) so `AudioRecord` runs only while an
audio-reactive source is active. Defer unless CPU/battery on low-end boxes warrants it.
---
## What does NOT change
- **Frontend / API** — audio engine + device selection, the music analyzer UI, and audio value
sources are engine-agnostic; the Android engine shows up via the existing device enumeration.
- **`build.gradle.kts` / Chaquopy pip block** — no new Python packages.
- **Audio analysis pipeline** — `AudioAnalyzer`, band filters, `ManagedAudioStream` untouched.
---
## Files
**Create**
- `server/src/ledgrab/core/audio/android_audio_engine.py`
- `android/app/src/main/java/com/ledgrab/android/AudioCapture.kt`
- `server/tests/core/audio/test_android_audio_engine.py`
**Modify**
- `server/src/ledgrab/core/audio/__init__.py` — guarded import + registry registration.
- `android/app/src/main/java/com/ledgrab/android/PythonBridge.kt``configureAudio` + `pushAudio`.
- `android/app/src/main/java/com/ledgrab/android/CaptureService.kt` — start/stop `AudioCapture`.
- `android/app/src/main/java/com/ledgrab/android/MainActivity.kt` — request `RECORD_AUDIO`.
- `android/app/src/main/AndroidManifest.xml``RECORD_AUDIO` (+ mic FGS if mic fallback).
---
## Tests (Python — run on desktop CI, no Android device needed)
New `server/tests/core/audio/test_android_audio_engine.py`:
- `configure()` then `push_samples()``read_chunk()` returns the same float32 samples;
queue drops oldest when full (push > maxsize).
- `AndroidAudioEngine.is_available()` is `False` until `configure()` and only on Android
(monkeypatch `ledgrab.utils.platform.is_android`); `True` after.
- `enumerate_devices()` returns exactly one loopback device when active, `[]` otherwise.
- Integration: with `is_android()` patched true + `configure()`, `get_best_available_engine()`
returns `"android_playback"` (priority beats demo), and a stream created via
`AudioEngineRegistry.create_stream("android_playback", 0, True, {})` yields pushed chunks.
- Registry isolation: use `AudioEngineRegistry.clear_registry()` / re-register in fixtures so
desktop engines aren't disturbed.
## Verification
1. **Python:** `py -3.13 -m pytest tests/core/audio/test_android_audio_engine.py --no-cov -q`
(from `server/`), then the full suite.
2. **Lint:** `ruff check src/ tests/ --fix` (from `server/`).
3. **Android build:** `./gradlew :app:assembleDebug` (from `android/`).
4. **On device/emulator (manual):** install APK → grant `RECORD_AUDIO` + screen-capture consent
→ start capture → play non-DRM media (e.g. a local video / YouTube web) → create an
audio-reactive value source bound to a strip → confirm the LEDs react to the audio, and the
Android playback device appears in audio device enumeration.
## Risks / notes
- **DRM opt-out:** Netflix/Disney+/etc. set audio as non-capturable; `AudioPlaybackCapture`
yields silence for them. Works for non-DRM media and the device's own audio. Document in UI.
- **API 29 minimum** for playback capture (minSdk is 24). API 2428 and root mode (no
MediaProjection) → mic fallback only, or audio unsupported. Gate cleanly + log.
- **`RECORD_AUDIO`** is a runtime "dangerous" permission — must be requested; capture must
degrade gracefully when denied.
- **Format:** request `ENCODING_PCM_FLOAT` so Kotlin pushes float32 matching
`read_chunk()`'s contract (1-D interleaved float32, length = frames × channels). If a device
rejects float, capture 16-bit PCM and convert (`/32768.0`) before pushing.
- **Latency/CPU:** small `chunkFrames` (e.g. 1024 @ 48 kHz ≈ 21 ms) keeps reactivity tight;
continuous capture (MVP) adds modest CPU on low-end boxes — see the orchestration trade-off.
- **R8/ProGuard:** minify is disabled and the Python module is resolved by string from Kotlin;
no new keep-rules needed.
@@ -0,0 +1,153 @@
# Android (TV) — Missing Functionality Assessment
> Status: review/feasibility document. No code changes. Last updated 2026-06-01.
## Context
LedGrab ships an **experimental on-device Android-TV build**: a Kotlin shell that
embeds the Python FastAPI server via **Chaquopy**, with Kotlin↔Python **bridges**
(`PythonBridge`, `BleBridge`, `UsbSerialBridge`). Several desktop features are
unavailable on this build because their Python backends rely on native libraries
that have no Android/Chaquopy wheels (`mss`, `dxcam`, `sounddevice`/PortAudio,
`opencv`, `nvidia-ml-py`, `winrt`, `dbus-next`), or on OS facilities Android
sandboxes differently.
The README "Feature support by OS" table now carries an Android column reflecting
this. This document assesses **whether each missing feature can be added**, how, and
whether it's worth it.
### The enabling pattern (why most of this is feasible)
Every desktop capability that's "missing" on Android is missing only because of a
*native dependency*, not because the capability is impossible. Android exposes the
same capability through a platform API, and the codebase already has the bridge
shape to plug it in:
> **Bridge pattern:** a Kotlin component captures an event/buffer → pushes it across
> the Chaquopy JNI boundary into a **module-level receiver** in a small Python engine
> → an existing engine/stream consumes it unchanged.
Reference implementation: `server/src/ledgrab/core/capture_engines/mediaprojection_engine.py`
(`configure()` + `push_frame()` + a bounded `queue.Queue`) ↔
`android/app/src/main/java/com/ledgrab/android/ScreenCapture.kt`
`PythonBridge.pushFrame()`. Screen capture already works on Android this exact way.
So for most missing features the work is: **add a Kotlin capture source + a thin
Python receiver engine mirroring that pattern.**
---
## Current Android capability matrix
| Feature | Desktop | Android (TV) today | Missing? |
| ------- | ------- | ------------------ | -------- |
| Screen capture | DXCam/WGC/MSS | ✅ MediaProjection + root `screenrecord` | No |
| LED transports (network/USB-serial/BLE) | ✅ | ✅ (USB via Android driver, BLE via Android bridge) | No |
| System metrics | psutil | ✅ CPU/RAM/battery/thermal via `/proc`, `/sys` (`AndroidMetricsProvider`) | No |
| **Audio capture** | WASAPI / Sounddevice | ❌ no PortAudio | **Yes** |
| **Notification capture** | WinRT / D-Bus | ❌ listener only Win/Linux | **Yes** |
| Webcam capture | OpenCV | ❌ no OpenCV wheel | Yes (niche) |
| GPU monitoring | NVML | ❌ no NVIDIA GPU | Marginal |
| Capture from *another* Android phone | scrcpy/ADB | ❌ | Skip (redundant) |
| Automation: window/process conditions | Windows ctypes | ❌ sandboxed | Partial |
| Monitor names / multi-display | WMI / generic | Single built-in display | Low value |
---
## Per-feature feasibility
### 🔊 Audio capture — **FEASIBLE, HIGH VALUE** ⭐ (detailed plan exists)
- **Blocker:** only `sounddevice`/PortAudio is missing — not the capability.
- **Android path:** `AudioPlaybackCapture` (API 29+) captures system playback audio and
**takes a `MediaProjection` token — which the app already obtains for screen capture.**
Kotlin `AudioRecord` → push PCM (float32) → a new push-based `AndroidAudioEngine`
mirroring `mediaprojection_engine.py`, registered in `core/audio/__init__.py`, feeding
the existing `AudioAnalyzer` unchanged. Mic (`AudioSource.MIC`) is the fallback.
- **Effort:** moderate. **Value:** high — music/sound-reactive lighting is a flagship use
on a TV box. **No new Python deps.**
- ⚠️ DRM-protected apps (Netflix etc.) opt out of playback capture; works for non-DRM
media and the device's own audio. Root mode (no MediaProjection) → mic-only.
- 📄 **See `android-audio-capture-plan.md`** for the full implementation plan.
### 🔔 Notification capture — **FEASIBLE, HIGH VALUE** ⭐ (planned)
- **Android is the *best* platform for this:** `NotificationListenerService` is the native,
event-push mechanism (no polling).
- **Path:** a `NotificationListenerService` resolves the posting app's display label and
pushes it via a module-level `push_notification()` into the existing
`os_notification_listener.py` pipeline (a new push-based `_AndroidBackend` alongside
`_WindowsBackend`/`_LinuxBackend`). Existing `NotificationColorStripSource` filters,
per-app colors/sounds, and the history endpoint all work unchanged. **No new Python deps.**
- **Permission:** user enables "Notification access" in Settings (`ACTION_NOTIFICATION_LISTENER_SETTINGS`);
no runtime-permission popup.
- **Effort:** moderate. **Value:** high.
- 📄 **Plan approved & detailed** — see `C:\Users\Alexei\.claude\plans\deep-enchanting-muffin.md`
(app-name parity; prompt-once permission UX).
### 📷 Webcam capture — **FEASIBLE, LOW VALUE**
- **Blocker** is `opencv-python-headless` (no Chaquopy cp311 wheel) — but capture doesn't
*need* OpenCV. Use **CameraX / Camera2** + `ImageReader` in Kotlin and push frames through
the same bridge as MediaProjection into a new `CameraBridgeEngine`.
- **Effort:** moderate. **Value:** low — TVs rarely have cameras; USB-UVC webcams need extra
device handling. Recommend deferring unless a concrete use case appears.
### 🎮 GPU monitoring — **MARGINAL, SKIP FOR NOW**
- NVML is desktop-NVIDIA only. Android GPU load lives in **vendor-specific sysfs**
(Adreno `/sys/class/kgsl/kgsl-3d0/gpubusy`, Mali `/sys/class/devfreq/*.mali/...`),
inconsistent and often root-only.
- CPU/RAM/battery/thermal are **already** covered by `AndroidMetricsProvider`. A best-effort
GPU-load reader could be added to that provider, but reliability is poor and value is low.
### 🪟 Automation: window/process conditions — **PARTIAL**
- Android forbids full window/process enumeration (`getRunningTasks` restricted since API 21+).
- **Obtainable:** the *current foreground app package* via `UsageStatsManager` (needs the
`PACKAGE_USAGE_STATS` special access) or an `AccessibilityService`.
- So "when <app> is in the foreground → scene X" is feasible (mirrors
`automations/platform_detector.py`, which currently returns empty off-Windows); full
window-title matching is **not**. **Effort:** moderate. **Value:** moderate (per-app scenes
on a TV box).
### 📱 Capture from *another* Android phone (scrcpy/ADB) — **SKIP**
- Impractical and redundant: no `adb` binary in Chaquopy, TV boxes can't reliably host an
adb server, and the device already captures its **own** screen via MediaProjection.
### 🖥️ Monitor names / multi-display — **LOW VALUE**
- `DisplayManager` can report a better display name and enumerate secondary (HDMI) displays,
but MediaProjection captures the default display; capturing a secondary display is more
involved and rarely useful on a single-screen box.
---
## Prioritization
| Priority | Feature | Effort | Value | New Python deps | Status |
| -------- | ------- | ------ | ----- | --------------- | ------ |
| 1 | Notification capture | Moderate | High | None | **Plan approved** |
| 2 | Audio capture | Moderate | High | None | **Plan written** (this folder) |
| 3 | Automation: foreground-app condition | Moderate | Moderate | None | Idea |
| 4 | Webcam capture (CameraX) | Moderate | Low | None | Idea |
| — | GPU load (vendor sysfs) | LowMed | Low | None | Not recommended |
| — | Capture from another phone | — | — | — | Won't do |
| — | Multi-display / monitor names | Low | Low | None | Not recommended |
**Recommended order:** ship notifications → ship audio → reassess. Both reuse existing
infrastructure (bridge pattern, the MediaProjection consent token, the audio/notification
pipelines) and add **zero** Python dependencies, so neither risks the Chaquopy
`--no-deps` build constraint documented in `CLAUDE.md`.
## Cross-cutting notes
- **No `build.gradle.kts` / Chaquopy pip impact** for notifications or audio — both use Android
platform APIs (Kotlin) + stdlib/`numpy` (already bundled) on the Python side.
- **Per-instance `PythonBridge`:** `PythonBridge` is created per `CaptureService` instance, so
system-bound services (e.g. a `NotificationListenerService`) call Python via the
process-global `Python.getInstance()` rather than borrowing that bridge.
- **Permissions are the recurring friction**, not the capture: audio needs `RECORD_AUDIO` +
(for playback capture) a MediaProjection token; notifications need the "Notification access"
settings toggle; foreground-app automation needs `PACKAGE_USAGE_STATS`.
+112 -102
View File
@@ -1,36 +1,58 @@
# LED Grab
Ambient lighting system that captures screen content and drives LED strips in real time. Supports WLED, Adalight, AmbileD, and DDP devices with audio-reactive effects, pattern generation, and automated profile switching.
Ambient lighting system that captures screen content and drives LED strips and smart lights in real time. Supports a wide range of devices — WLED, DDP, Adalight, smart bulbs, PC peripherals, Bluetooth strips, and more — with audio-reactive effects, pattern generation, and condition-based automation.
**Free and open source.** LedGrab is released under the [MIT license](LICENSE) — free to use, modify, and self-host, with no accounts, telemetry, or cloud dependency. Everything runs locally on your own machine and network.
## What It Does
The server captures pixels from a screen (or Android device via ADB), extracts border colors, applies post-processing filters, and streams the result to LED strips at up to 60 fps. A built-in web dashboard provides device management, calibration, live LED preview, and real-time metrics — no external UI required.
The server captures pixels from a screen (or from a connected Android phone via ADB), extracts border colors, applies a post-processing filter pipeline, and streams the result to your LED devices at up to 60 fps. A built-in web dashboard provides device management, calibration, a visual wiring editor, live LED preview, and real-time metrics — no external UI required.
A Home Assistant integration exposes devices as entities for smart home automation.
A separate Home Assistant integration exposes devices as entities for smart-home automation.
## Screenshots
![LedGrab dashboard](docs/screenshots/dashboard.PNG)
*Dashboard — live system performance, integrations, automations, and scene presets at a glance.*
![Channels view](docs/screenshots/dashboard-targets.PNG)
*Channels — start, stop, and monitor each source-to-device pipeline with live FPS.*
![Live capture preview](docs/screenshots/test-preview-capture.PNG)
*Live preview — inspect the processed capture output in real time before it reaches the LEDs.*
## Features
### Screen Capture
- Multi-monitor support with per-target display selection
- 6 capture engine backends MSS (cross-platform), DXCam, BetterCam, Windows Graphics Capture (Windows), Scrcpy (Android via ADB), Camera/Webcam (OpenCV)
- Capture engine backends: MSS (cross-platform), DXCam, BetterCam, Windows Graphics Capture (Windows only), and Camera/Webcam (OpenCV)
- Capture from a connected Android phone's screen via scrcpy (ADB) — the device is a *source*; LedGrab itself runs on your desktop
- Configurable capture regions, FPS, and border width
- Capture templates for reusable configurations
- Reusable capture templates
### LED Device Support
- WLED (HTTP/UDP) with mDNS auto-discovery
- Adalight (serial) — Arduino-compatible LED controllers
- AmbileD (serial)
- DDP (Distributed Display Protocol, UDP)
- OpenRGB — PC peripherals (keyboard, mouse, RAM, fans, LED strips)
- Serial port auto-detection and baud rate configuration
LedGrab speaks many protocols, so a single setup can drive everything from a DIY strip to off-the-shelf smart bulbs:
![Device type picker](docs/screenshots/devices.PNG)
- **Network LED controllers** — WLED (HTTP/UDP, with mDNS auto-discovery), DDP (Pixelblaze, ESPixelStick, Falcon), Open Pixel Control (OPC), Art-Net / sACN (E1.31), ESP-NOW, and generic WebSocket streaming
- **Serial / direct hardware** — Adalight (Arduino-compatible), AmbiLED, SPI-attached strips (e.g. WS2812B), and USB HID controllers
- **Smart bulbs & panels** — Philips Hue (Entertainment API), Nanoleaf, Yeelight, WiZ, LIFX, and Govee (Wi-Fi LAN)
- **Bluetooth LE strips** — SP110E, Triones / HappyLighting, Zengge, and Govee BLE
- **PC peripherals** — OpenRGB, Razer Chroma, and SteelSeries GameSense (keyboards, mice, RAM, fans, etc.)
- **Device groups** — combine multiple devices into one logical target
- Serial port auto-detection and baud-rate configuration
### Color Processing
- Post-processing filter pipeline: brightness, gamma, saturation, color correction, auto-crop, frame interpolation, pixelation, flip
- Post-processing filter pipeline: brightness, gamma, saturation, color correction, auto-crop, frame interpolation, pixelation, flip, and more
- Reusable post-processing templates
- Color strip sources: audio-reactive, pattern generator, composite layering, audio-to-color mapping
- Color strip sources: audio-reactive, pattern generator, gradients, composite layering, and audio-to-color mapping
- Pattern templates with customizable effects
### Audio Integration
@@ -38,17 +60,20 @@ A Home Assistant integration exposes devices as entities for smart home automati
- Multichannel audio capture from any system device (input or loopback)
- WASAPI engine on Windows, Sounddevice (PortAudio) engine on Linux/macOS
- Per-channel mono extraction
- Audio-reactive color strip sources driven by frequency analysis
- Audio filter / processing pipeline feeding audio-reactive color sources driven by frequency analysis
### Automation
- Profile engine with condition-based switching (time of day, active window, etc.)
- Dynamic brightness value sources (schedule-based, scene-aware)
- Key Colors (KC) targets with live WebSocket color streaming
- Automations engine with condition-based rules — switch targets, scenes, or brightness by time of day, active window/process, MQTT, webhooks, or game events
- Scene presets for one-click lighting changes
- Dynamic value sources for brightness and other parameters (schedule-based, weather-based, scene-aware)
- Weather sources, clock sync, webhooks, and inbound/outbound HTTP endpoints
- Game integration adapters (e.g. League of Legends)
### Dashboard
- Web UI at `http://localhost:8080` — no installation needed on the client side
- Web UI at `http://localhost:8080` — nothing to install on the client side
- Visual node-graph editor for wiring sources → processing → targets
- Progressive Web App (PWA) — installable on phones and tablets with offline caching
- Responsive mobile layout with bottom tab navigation
- Device management with auto-discovery wizard
@@ -59,32 +84,56 @@ A Home Assistant integration exposes devices as entities for smart home automati
### Home Assistant Integration
- HACS-compatible custom component
- HACS-compatible custom component (separate repository)
- Light, switch, sensor, and number entities per device
- Real-time metrics via data coordinator
- Real-time metrics via a data coordinator
- WebSocket-based live LED preview in HA
## Requirements
## Platforms
- Python 3.11+ (or Docker)
- A supported LED device on the local network or connected via USB
- Windows, Linux, or macOS — all core features work cross-platform
LedGrab runs as a desktop / server application:
### Platform Notes
| Platform | Status | Notes |
| -------- | ------ | ----- |
| Windows | ✅ Supported | Installer (`.exe`) and portable ZIP; all capture/audio backends |
| Linux | ✅ Supported | Tarball and Docker image; X11 capture (Wayland in-container capture not supported) |
| macOS | ✅ Supported | Runs from source / Docker; MSS capture |
| Docker | ✅ Supported | Multi-arch container image |
| Android (TV) | ⚠️ Experimental | An on-device Android-TV build exists (APK attached to releases) but is emulator-verified only and **not officially supported** |
> **There is no production Android app.** Android phones are only supported as a *capture source* (via scrcpy/ADB) from a desktop host. The on-device Android-TV build is experimental.
### Feature support by OS
| Feature | Windows | Linux / macOS |
| ------- | ------- | ------------- |
| Screen capture | DXCam, BetterCam, WGC, MSS | MSS |
| Webcam capture | OpenCV (DirectShow) | OpenCV (V4L2) |
| Audio capture | WASAPI, Sounddevice | Sounddevice (PulseAudio/PipeWire) |
| GPU monitoring | NVIDIA (pynvml) | NVIDIA (pynvml) |
| Android capture | Scrcpy (ADB) | Scrcpy (ADB) |
| GPU monitoring | NVIDIA (nvidia-ml-py) | NVIDIA (nvidia-ml-py) |
| Capture from Android phone | scrcpy (ADB) | scrcpy (ADB) |
| Notification capture | WinRT | dbus (Linux) |
| Monitor names | Friendly names (WMI) | Generic ("Display 0") |
| Profile conditions | Process/window detection | Not yet implemented |
| Automation: window/process conditions | Supported | Partial |
## Requirements
- Python 3.11+ (or Docker)
- A supported LED device on the local network, connected via USB/serial, or reachable over Bluetooth
- Windows, Linux, or macOS
## Quick Start
### Docker (recommended)
### Prebuilt downloads
Grab a ready-to-run build from the [Releases page](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/releases):
- **Windows** — `LedGrab-<version>-setup.exe` (installer, no admin required) or `LedGrab-<version>-win-x64.zip` (portable)
- **Linux** — `LedGrab-<version>-linux-x64.tar.gz`
- **Docker** — see below
- **Android TV** — `.apk` (experimental, see [Platforms](#platforms))
### Docker (recommended for servers)
```bash
git clone https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab.git
@@ -115,11 +164,11 @@ export PYTHONPATH=$(pwd)/src # Linux/Mac
uvicorn ledgrab.main:app --host 0.0.0.0 --port 8080
```
Open **http://localhost:8080** to access the dashboard.
Open <http://localhost:8080> to access the dashboard.
> **Important:** The default API key is `development-key-change-in-production`. Change it before exposing the server outside localhost. See [INSTALLATION.md](INSTALLATION.md) for details.
> **Network access:** By default, LedGrab allows anonymous access only from `localhost`. Any request from another machine on your LAN is rejected unless you configure an API key (`auth.api_keys`). Set a key before exposing the server on your network — see [INSTALLATION.md](INSTALLATION.md).
See [INSTALLATION.md](INSTALLATION.md) for the full installation guide, including configuration, Docker manual builds, and Home Assistant setup.
See [INSTALLATION.md](INSTALLATION.md) for the full installation guide, including configuration, Docker manual builds, and CORS setup.
## Demo Mode
@@ -133,50 +182,9 @@ docker compose run -e LEDGRAB_DEMO=true server
# Python
LEDGRAB_DEMO=true uvicorn ledgrab.main:app --host 0.0.0.0 --port 8081
# Windows (installed app)
set LEDGRAB_DEMO=true
LedGrab.bat
```
Demo mode uses port **8081**, config file `config/demo_config.yaml`, and stores data in `data/demo/` (separate from production data). It can run alongside the main server.
## Architecture
```text
ledgrab/
├── server/ # Python FastAPI backend
│ ├── src/ledgrab/
│ │ ├── main.py # Application entry point
│ │ ├── config.py # YAML + env var configuration
│ │ ├── api/
│ │ │ ├── routes/ # REST + WebSocket endpoints
│ │ │ └── schemas/ # Pydantic request/response models
│ │ ├── core/
│ │ │ ├── capture/ # Screen capture, calibration, pixel processing
│ │ │ ├── capture_engines/ # MSS, DXCam, BetterCam, WGC, Scrcpy, Camera backends
│ │ │ ├── devices/ # WLED, Adalight, AmbileD, DDP, OpenRGB clients
│ │ │ ├── audio/ # Audio capture engines
│ │ │ ├── filters/ # Post-processing filter pipeline
│ │ │ ├── processing/ # Stream orchestration and target processors
│ │ │ └── profiles/ # Condition-based profile automation
│ │ ├── storage/ # JSON-based persistence layer
│ │ ├── static/ # Web dashboard (vanilla JS, CSS, HTML)
│ │ │ ├── js/core/ # API client, state, i18n, modals, events
│ │ │ ├── js/features/ # Feature modules (devices, streams, targets, etc.)
│ │ │ ├── css/ # Stylesheets
│ │ │ └── locales/ # en.json, ru.json, zh.json
│ │ └── utils/ # Logging, monitor detection
│ ├── config/ # default_config.yaml
│ ├── tests/ # pytest suite
│ ├── Dockerfile
│ └── docker-compose.yml
├── docs/
│ ├── API.md # REST API reference
│ └── CALIBRATION.md # LED calibration guide
├── INSTALLATION.md
└── LICENSE # MIT
```
Demo mode uses port **8081**, config file `config/demo_config.yaml`, and stores data under `data/demo/` (separate from production data). It can run alongside the main server.
## Configuration
@@ -187,14 +195,15 @@ server:
host: "0.0.0.0"
port: 8080
log_level: "INFO"
cors_origins:
- "http://localhost:8080"
auth:
api_keys:
dev: "development-key-change-in-production"
storage:
devices_file: "data/devices.json"
templates_file: "data/capture_templates.json"
# Empty (default) → loopback-only anonymous access; LAN requests are rejected.
# Add a key to enable LAN/remote access (generate one with: openssl rand -hex 32).
api_keys: {}
# api_keys:
# dev: "your-secret-key-here"
logging:
format: "json"
@@ -202,25 +211,26 @@ logging:
max_size_mb: 100
```
Environment variable override example: `LEDGRAB_SERVER__PORT=9090`.
- Application data is stored in a SQLite database (`data/ledgrab.db` by default). Set `LEDGRAB_DATA_DIR` to relocate the data root (database + assets).
- Environment variable override example: `LEDGRAB_SERVER__PORT=9090`.
See [INSTALLATION.md](INSTALLATION.md) and [`server/.env.example`](server/.env.example) for the full configuration reference.
## API
The server exposes a REST API (with Swagger docs at `/docs`) covering:
The server exposes a REST API (with interactive Swagger docs at `/docs`) plus WebSocket endpoints. Resources include:
- **Devices** — CRUD, discovery, validation, state, metrics
- **Capture Templates** — Screen capture configurations
- **Picture Sources** — Screen capture stream definitions
- **Picture Targets** — LED target management, start/stop processing
- **Post-Processing Templates** — Filter pipeline configurations
- **Color Strip Sources** — Audio, pattern, composite, mapped sources
- **Audio Sources** — Multichannel and mono audio device configuration
- **Pattern Templates** — Effect pattern definitions
- **Value Sources** — Dynamic brightness/value providers
- **Key Colors Targets** — KC targets with WebSocket live color stream
- **Profiles** — Condition-based automation profiles
- **Capture Templates** & **Picture Sources** — screen capture configuration and stream definitions
- **Output Targets** — LED target management, start/stop processing, live color stream
- **Post-Processing Templates** — filter pipeline configurations
- **Color Strip Sources**, **Pattern Templates**, **Gradients** — color generation
- **Audio Sources / Templates / Filters** — audio capture and reactive processing
- **Value Sources**, **Weather Sources**, **Scene Presets** — dynamic parameters and presets
- **Automations**, **Webhooks**, **HTTP Endpoints**, **Game Integration** — triggers and rules
- **MQTT** & **Home Assistant**broker sources and HA integration
All endpoints require API key authentication via `X-API-Key` header or `?token=` query parameter.
Authentication uses a Bearer token (`Authorization: Bearer <api-key>`) when API keys are configured; loopback requests are anonymous by default. WebSocket connections authenticate via a first-message handshake.
See [docs/API.md](docs/API.md) for the full reference.
@@ -253,16 +263,16 @@ ruff check src/ tests/
Optional extras:
```bash
pip install -e ".[perf]" # High-performance capture engines (Windows)
pip install -e ".[camera]" # Webcam capture via OpenCV
pip install -e ".[perf]" # High-performance capture engines (Windows: DXCam, BetterCam, WGC)
pip install -e ".[notifications]" # OS notification capture (WinRT / dbus)
pip install -e ".[scrcpy]" # Capture from an Android phone via scrcpy
pip install -e ".[ble]" # Bluetooth LE LED controllers (desktop only)
```
## Contributing
Contributions are welcome. LedGrab is MIT-licensed, so you're free to fork, modify, and self-host. Please open an issue or pull request on the [repository](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab).
## License
MIT — see [LICENSE](LICENSE).
## Acknowledgments
- [WLED](https://github.com/Aircoookie/WLED) — LED control firmware
- [FastAPI](https://fastapi.tiangolo.com/) — Python web framework
- [MSS](https://python-mss.readthedocs.io/) — Cross-platform screen capture
MIT — see [LICENSE](LICENSE). Free and open source.
+22 -40
View File
@@ -1,72 +1,54 @@
## v0.8.0 (2026-05-28)
## v0.8.1 (2026-05-28)
### User-facing changes
#### Features
##### Android TV — production-readiness pass
##### Multi-broker MQTT devices
- **Security:** per-install random API key (persisted, threaded into the embedded server via env, embedded in the pairing QR as a URL-fragment so it never reaches HTTP logs); root-shell injection eliminated via POSIX-quoted `runAsRoot(argv)` overload; broadcast receivers locked to the app package; release builds refuse to silently sign with the debug keystore; crash log retention capped at 10 entries
- **Performance:** single reusable RGBA buffer in `ScreenCapture` / `RootScreenrecord` (eliminates ~15 MB/s GC churn at 30 fps); frame pacer switched to `elapsedRealtimeNanos` with catch-up accumulator (fixes ~30.3 fps drift); capture dimensions derived from source aspect ratio so non-16:9 displays aren't squashed; QR bitmap cached by URL
- **Compatibility:** compileSdk/targetSdk → 35 (Play Store requirement); armeabi-v7a build path; foreground service type declared as `mediaProjection|specialUse` with proper `ServiceCompat.startForeground` promotion; Ethernet > Wi-Fi > VPN > cellular selection in `NetworkUtils`; Android 15 predictive-back via `enableOnBackInvokedCallback`; splash screen API hides Chaquopy cold-start delay
- **UI/UX:** all hardcoded English strings localised across en/ru/zh; monochrome notification icon; 320×180 TV banner; ViewStub-based running panel; pulse animator on Running dot; "Starting…" button while probing root; autostart checkbox hidden on unrooted devices
- **Lifecycle hardening:** `processLock` serialises EOF respawn vs `stop()` to prevent orphaned screenrecord; publish-before-start under `@Synchronized` in `CaptureService.restartRootPipeline` closes the orphan window during watchdog restarts; watchdog give-up bound corrected ([ef1f9ea](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/ef1f9ea))
- The device editor now shows an MQTT **broker picker** for `device_type=mqtt` (in both the add-device and device-settings modals), wired into load / save / validate / dirty-check / clone. An empty selection means "first available broker"
- `mqtt_source_id` is now threaded end-to-end through `DeviceCreate` / `DeviceUpdate` / `DeviceResponse` and the device routes; the referenced broker is validated on create **and** update ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
##### Backup format — bundled DB + assets ZIP
##### Schema-driven wiring-graph editor
- Auto-backups now produce a `.zip` containing `ledgrab.db` plus every file from the assets directory under `assets/` — matching the manual `GET /api/v1/system/backup` download. Restore accepts both `.zip` and legacy `.db` interchangeably
- Partial-write hardening: writes stage to `<name>.partial` then `os.replace` into place — a crash mid-write never leaves a corrupt backup masquerading as valid. Stale `.partial` files from prior crashes are swept on the next run
- Symlinks inside the assets directory are skipped (security guard against link targets outside the dir)
- Backups over 500 MB log a warning so operators notice unbounded asset growth before disk fills up
- `restart.py` redirects spawned restart script stdout/stderr to `restart.log` and bails out early if the script is missing — silent failures used to vanish into a detached child ([85da2e5](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/85da2e5))
- The visual graph editor now renders ports and edges generically from a backend-served schema (`GET /api/v1/graph/schema`) instead of hard-coding the connectable-field topology in two places — so client and server can no longer drift
- New `GET /api/v1/graph` returns the full nodes + edges + validation topology, and `GET /api/v1/graph/dependents/{kind}/{id}` reports what references an entity ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
##### Spectrum-aperture icon set
##### Aggregated snapshot endpoint
- Regenerated icon family from a single Pillow script: rounded-square aperture traced by a continuous RGB color-wheel stroke over a vignette canvas with chromatic bloom. 4× supersampled then downsampled per output for crispness
- New 256 px transparent-background **tray variant** — taskbar icon reads cleanly against light themes instead of showing a dark tile
- `icon.ico` now embeds 16/24/32/48/64/128/256 frames sourced from the transparent master (fixes the dark-square halo on light Windows themes)
- Maskable 512 variant safe-area padded for PWA round-crops ([3645216](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/3645216))
- New `GET /api/v1/snapshot` returns all output targets (with processing state + metrics), devices (with brightness), the source / preset / clock lists, and the system block in a **single response** — collapsing the Home Assistant integration's previous ~2N+M request fan-out into one round trip
- `?include=` fetches only a subset of sections, and an excluded section also skips its server-side work (e.g. cold-cache hardware brightness probes or the blocking NVML performance query) ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
#### Bug Fixes
- **Notification sound dropdowns:** both the per-app override list and the main row now always render the EntitySelect (was silently inert before any sound assets were registered) and offer "no sound" as a first-class option via `allowNone` ([1f95993](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/1f95993))
- **CSS editor:** `notification_sound` and `notification_volume` are now persisted on save — they were silently dropped from the payload before ([66b85b0](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/66b85b0))
- **Python 3.13 ctypes:** Win32 message-pump prototypes (`GetMessageW` / `TranslateMessage` / `DispatchMessageW`) now share a single `LPMSG = POINTER(wintypes.MSG)` class across `WindowsShutdownGuard` and `PlatformDetector` — fixes the `expected LP_MSG instance instead of pointer to _MSG` error and the resulting shutdown-guard / display-power-monitor failure on 3.13 ([e4d24a0](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/e4d24a0), [0d840ad](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/0d840ad))
- **Graceful shutdown no longer hangs:** uvicorn's graceful-shutdown wait is now bounded (`GRACEFUL_SHUTDOWN_TIMEOUT`, shared by the desktop, Android, and demo launchers). A lingering events WebSocket (which the browser auto-reconnects) used to keep connections from draining, so the lifespan shutdown never ran — leaving LED targets lit and blocking process exit. Ctrl+C / OS shutdown with the UI open now reliably stops targets and checkpoints the DB ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
- **Device update error codes:** `update_device` no longer masks an intentional 4xx (e.g. an unknown `mqtt_source_id` or failed group validation) as a generic 500 ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
---
### Development / Internal
#### CI/Build
#### Backend
- `release.yml` now creates the Gitea release as a **draft** and only flips `draft=false` once every build job (Windows, Linux, Docker) has uploaded its artifacts and sha256 sidecars — users never see a release page that's missing assets, which would have broken the in-app updater ([bc42604](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/bc42604))
- **Wiring-graph schema engine** (`api/graph_schema.py`): a pure, unit-tested module that is the single source of truth for which reference fields connect which entity kinds; builds the topology and performs dependency lookup plus cycle / dangling-reference detection without booting the app or any store. The route layer only gathers serialized entities and delegates ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
- **Structured access log:** a new middleware emits one structured line per request, attributing it to the authenticated token's friendly label (the key name, **never** the secret) so traffic can be traced to a client (e.g. `homeassistant` vs `android`). uvicorn's own access log is disabled to avoid duplicate lines ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
- Shared `validate_mqtt_source_exists` (`_mqtt_validation.py`) deduplicates the MQTT-source existence check between the device and output-target routes ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
#### Refactoring
#### Frontend
- **Shared API client + automations registry (audit M7, H8):** new `core/api-client.ts` wraps `fetchWithAuth` with typed `apiGet` / `apiPost` / `apiPut` / `apiPatch` / `apiDelete`; 35 feature/core files migrated. FastAPI validation-array detail unwrap hardened. Automations editor's two hand-rolled `RuleType` dispatch ladders converted to `Record<RuleType, ...>` registries with an import-time exhaustiveness check ([bb3a316](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/bb3a316))
- **types.ts split (audit H6):** 1140 LOC `types.ts` split into 18 per-entity files under `types/`, original file kept as a pure re-export barrel — 102 type exports preserved with no import sites changed ([49c35a2](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/49c35a2))
- Service-worker refresh for the new bundle ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
#### Documentation
#### Tests
- `REVIEW_RECONCILE_NOTES.md` — design doc for the dashboard innerHTML reconciliation work: bug-class analysis, latent-site inventory, decision ladder (helper / hand-rolled cells / Lit), and recommendation to migrate polling-heavy modules to Lit with `entity-events.ts` tab reconciliation sequenced first ([10eb24b](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/10eb24b))
- New suites: graph routes + schema engine, snapshot routes, access-log middleware, `mqtt_source_id` device regressions, and the bounded-shutdown entrypoint. Full suite: **1614 passing** ([a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba))
---
<details>
<summary>All Commits (11)</summary>
<summary>All Commits (1)</summary>
| Hash | Message | Author |
| ---- | ------- | ------ |
| [0d840ad](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/0d840ad) | fix(ctypes): share wintypes.MSG with platform_detector to avoid argtype races | alexei.dolgolyov |
| [1f95993](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/1f95993) | fix(notification): allow clearing the sound on per-app overrides and main row | alexei.dolgolyov |
| [10eb24b](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/10eb24b) | docs: dashboard innerHTML reconciliation review notes | alexei.dolgolyov |
| [66b85b0](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/66b85b0) | fix(css-editor): persist notification_sound + notification_volume | alexei.dolgolyov |
| [bc42604](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/bc42604) | ci(release): publish release only after every build job uploads assets | alexei.dolgolyov |
| [3645216](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/3645216) | feat(icons): spectrum aperture icon set + dedicated tray variant | alexei.dolgolyov |
| [85da2e5](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/85da2e5) | feat(backup): bundle assets in ZIP + partial-write hardening + restart log | alexei.dolgolyov |
| [e4d24a0](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/e4d24a0) | fix(ctypes): pin LPMSG across MSG-pump prototypes for Python 3.13 | alexei.dolgolyov |
| [bb3a316](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/bb3a316) | refactor(frontend): shared API client + automations registry (audit M7, H8) | alexei.dolgolyov |
| [49c35a2](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/49c35a2) | refactor(frontend): split types.ts into 18 per-entity files (audit H6) | alexei.dolgolyov |
| [ef1f9ea](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/ef1f9ea) | feat(android): production-readiness pass — security, perf, compat, UI/UX | alexei.dolgolyov |
| [a5effba](https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab/commit/a5effba) | feat: aggregated snapshot + wiring-graph APIs, MQTT device brokers | alexei.dolgolyov |
</details>
+138 -5
View File
@@ -201,9 +201,19 @@ caller off the legacy path, then delete it.
- [x] Field on `device_config.MQTTConfig`
- [x] `MQTTLEDClient` acquires runtime in `connect()`, releases in `close()`
- [x] Provider threads `mqtt_manager` via `ProviderDeps`
- [ ] Device editor: MQTT source picker shown for `device_type=mqtt` *(UI still
pending — backend accepts the field, but the device-create form doesn't
expose it yet)*
- [x] Device editor: MQTT source picker shown for `device_type=mqtt`. Turned
out the API layer was *also* missing it (the TODO's "backend accepts the
field" was wrong — `mqtt_source_id` lived in `device_store` +
`device_config.MQTTConfig` but was dropped by `DeviceCreate/Update/Response`
and the routes). Added: schema fields + route threading + referenced-source
validation (`_validate_mqtt_source_exists`, mirrors output_targets) +
`except HTTPException: raise` guard in `update_device` (it was masking its
own 4xx as 500). Frontend: broker `EntitySelect` (reusing `mqttSourcesCache`)
in both the add-device (`device-discovery.ts`) and settings
(`devices.ts`) modals — shown for `device_type=mqtt`, wired into
load/save/validate/dirty-check/clone. Empty = "first available broker".
4 regression tests in `test_devices_routes.py::TestMqttSourceId`; full
suite 1567 passing; en/ru/zh keys added.
### Phase 5 — `AutomationEngine`
@@ -213,8 +223,11 @@ caller off the legacy path, then delete it.
### Phase 6 — `api/routes/system.py`
- [x] Replace integration status with `mqtt_manager.get_all_sources_status()`
- [ ] Update frontend dashboard payload (MQTT widget now expects a list of
sources instead of a single `enabled`/`connected` pair — surface in UI)
- [x] Update frontend dashboard payload (MQTT widget now expects a list of
sources instead of a single `enabled`/`connected` pair — surface in UI).
Done: `dashboard.ts` `_renderMQTTIntegrationCard` renders one card per
`mqttStatus.connections` entry; `_updateIntegrationsInPlace` iterates the
list.
### Phase 7 — Startup migration
@@ -980,3 +993,123 @@ After phase 1 the codebase will have 3 fresh examples of "ping the LAN, listen f
- LOW: Nanoleaf `.port` property added; pair-then-create E2E test
added.
- Tests: 1379 pass (+21 regression tests).
## Graph editor — "full control of wiring via graph" (in progress)
Goal: make the visual graph a first-class wiring control surface, not just a
viewer. Driven by the ULTRA-DEEP review (findings A1A5, B1B6, C1C6, D1D6).
### Done (NOT yet committed — awaiting review/commit)
- [x] **A1** Undo/redo wired to connect/detach/move (was dead code); inverse ops
throw on failure so the stack can't silently desync.
- [x] **A2** Manual node layout persists to `localStorage` (`graph_node_positions`),
cleared on relayout.
- [x] **A3** Scene-preset disambiguation — deactivation scene now reachable via a
field picker (was always picking the first match).
- [x] **B6** Edge field labels (revealed on zoom ≥ 0.9).
- [x] **C3** Health overlay — broken refs (referrer exists, target missing),
dependency cycles, orphans; node warning badges + an issues toolbar button.
- [x] **D1** `GET /api/v1/graph/schema` — authoritative connectable-field registry
(`api/graph_schema.py`, pure + unit-tested).
- [x] **D2** `GET /api/v1/graph` (nodes+edges+validation) and
`GET /api/v1/graph/dependents/{kind}/{id}`.
- [x] **D4** `POST /api/v1/graph/validate-connection` — existence + source-kind +
cycle pre-flight; frontend validates before every write (fails open if the
endpoint is unreachable). List/double-nested fields rejected.
- [x] **B2** Drop-on-node connect — empty top-level slots are now wireable (drop a
source onto any compatible node body, not just an existing port).
- [x] **C4** Overwrite-occupied-slot confirm + delete-with-dependents warning
(single delete only; bulk keeps the batch confirm).
- [x] **D5** Create-and-connect — drag a port onto empty canvas → pick a compatible
new entity kind → it's created and auto-wired (kind-scoped watcher).
- [x] **D6 (read-only half)** "Export graph (JSON)" toolbar action.
- [x] Custom per-entity `icon` + `icon_color` now render on graph nodes (parity
with custom node colours; fallback to kind/subtype glyph).
- [x] **B1** Edit single-level **BindableFloat** value slots from the graph
(`brightness`, `smoothing`, `intensity`, `scale`, `speed`, … on
color_strip_source; `brightness`/`transition` on output_target). Subtype-safe
(only offers slots the target entity actually has). Writes the partial
`{ <slot>: { source_id } }` payload → backend `Bindable*.apply_update` merges,
preserving the static value. Verified data-safe (no `from_raw`/value-reset path).
- [x] Render the two functional value-source references `buildGraph` was missing —
`value_source.value_source_id` (gradient_map → inner value source) and
`value_source.color_strip_source_id` (css_extract → strip). Both are runtime-
resolved and already drag-editable; now visible/detachable in the graph.
- [x] **B4 foundation:** backend schema now authoritative about graph-editability
(`is_editable()` + `editable` flag in `/graph/schema`); `validate-connection`
hardened to reject non-editable fields (colour/list/double-nested), not just lists.
- [x] **B4 drift guard + gap fixes:** `checkSchemaDrift()` (graph-connections.ts) warns
once if the frontend `CONNECTION_MAP` editable set diverges from `/graph/schema`
(the automated "10-step checklist"). Surfacing it found 3 real gaps; fixed 2:
`color_strip_source.input_source_id` + `processing_template_id` are now drag-editable
(processed-strip wiring; `apply_update` is partial-safe). The 3rd —
`device.default_css_processing_template_id` — is intentionally NOT drag-editable
(the device PUT route isn't partial-safe; a one-field PUT could null the URL) and is
in the drift-check exclude set. Also broadened `_availableMatches` to hide any slot
the target entity doesn't expose (subtype-accurate; refs are always-emitted so empty
slots stay wireable). Review also caught a **dead `output_target.picture_source_id`
slot** (no output target stores it — not a field/schema, never emitted) — removed
from both registries + `buildGraph`.
- [x] **Comprehensive review pass (4 subagents: backend/frontend-core/orchestrator/security).**
Findings fixed:
- **CRITICAL (security):** `GET /api/v1/graph` leaked plaintext **webhook tokens**
(`asdict` recursed `Automation.rules[].token`, an auth-equivalent secret). Fixed with
**field-projection** — `serialize_entity_for_graph()` / `graph_field_roots()` project
each entity to only `{id, name, subtype, reference-roots}`; secrets can't survive.
Added a structural regression test asserting no projection root is secret-bearing for
any kind (drift-proof boundary) + a token-drop test.
- MEDIUM: added missing `value_source.clock_id` (AnimatedColorValueSource → sync_clock)
to the backend registry for topology/dependents completeness (drift-excluded on the
frontend — value-source PUT needs a `source_type` discriminator, so it's editor-only).
- MEDIUM/LOW: `CSS.escape` on the markIssues id selector; grouped/clarified
`_DRIFT_EXCLUDE`; fixed the stale `_availableMatches` JSDoc; documented the
`checkSchemaDrift` forward-reference. Orchestrator + frontend-core + security: APPROVE.
- Verification: `npm --prefix server run typecheck` + `run build` clean; ruff clean;
graph backend tests 35 pass; full backend suite green. ~8 code-review passes,
all CRITICAL/HIGH findings fixed.
### Left to do (deferred)
- [x] **BindableColor slots** — CHECKED, decision: keep read-only (won't fix).
Value sources are scalar-only (`ValueStream.get_value() -> float`) and every
colour consumer (`color_strip/single.py`, `effect_stream.py`) reads the static
RGB via `bcolor()`, ignoring `source_id`. So a value_source cannot drive a
colour — wiring `color`/`color_peak`/… would be a dead binding. Documented in
`api/graph_schema.py` next to the BindableColor entries. (Would only become
viable if a colour-producing value-source type is added.)
- [~] **B4 — delete the frontend `CONNECTION_MAP` duplication.**
- [x] **Foundation done:** the backend schema now carries an authoritative
`editable` flag per field (`is_editable()` in `api/graph_schema.py`, mirroring
the frontend `_isEditable`: top-level refs + single-level BindableFloat slots;
NOT colour/list/double-nested). `validate-connection` is hardened to reject any
non-editable field (was list-only). `editable` is surfaced in `/graph/schema`.
- [ ] **Remaining (the refactor):** frontend fetches `/graph/schema` on load and
derives connection metadata + edges from it (port the `extract_refs` dot-path/list
grammar to TS), keeping only a tiny `kind → {endpoint, cache}` write-routing table;
then delete the field-level `CONNECTION_MAP` + the `buildGraph` edge loops
(graph-connections.ts / graph-layout.ts). Removes the 10-step sync checklist in
`contexts/graph-editor.md`. **A backend apply-write endpoint is NOT required** —
keep the proven per-entity PUT. Risk: regressing drag-connect/bindable; keep a
dev drift-check (frontend editable set vs `/graph/schema`) during the transition.
Note: frontend `CONNECTION_MAP` also has inert `ha_source_id`/`gradient_id` entries
(no graph node kind) — drop them, the backend schema already omits them.
- [ ] **D6 — blueprint import/instantiate.** Export exists; the apply half (serialize
a selected subgraph's topology + entities, re-import with id remapping, conflict
handling) is large and data-integrity-sensitive (see Data Migration Policy in
CLAUDE.md). Scope as its own feature.
- [ ] **List-slot editing** (composite `layers[]`, mapped `zones[]`, scene preset
`targets[]`) — needs an element index in the write + validate paths
(`validate_connection` currently rejects list fields). Edit via entity modal
for now.
### Notes / decisions
- The backend `CONNECTION_SCHEMA` (`api/graph_schema.py`) is the authoritative
superset; it already declares the bindable + list + value_source-chain edges. The
frontend `CONNECTION_MAP` still owns write-routing (endpoint/cache) — that's the
only reason it survives (see B4).
- Bindable edges render dashed (`.graph-edge-nested`) but ARE editable — the dashed
style intentionally distinguishes value bindings from structural edges.
- `validate-connection` and `dependents` fail **open/safe** on the frontend so the
graph keeps working against an older server without these endpoints.
+1 -1
View File
@@ -41,7 +41,7 @@ android {
// in CI). See ledgrabVersionCode above. Was stuck at 1 before —
// sideload updates silently refused to install.
versionCode = ledgrabVersionCode
versionName = "0.8.0"
versionName = "0.8.1"
// ABI selection. Detect armeabi-v7a wheel presence and opt the
// ABI in only when the matching pydantic-core wheel is on disk —
+8
View File
@@ -39,6 +39,14 @@
<!-- POST_NOTIFICATIONS for Android 13+ foreground service notification -->
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<!-- RECORD_AUDIO for on-device system-playback capture (AudioPlaybackCapture,
API 29+) feeding audio-reactive lighting. Runtime "dangerous" permission,
requested in MainActivity; capture degrades gracefully when denied.
Playback capture runs under the existing mediaProjection FGS type, so no
FOREGROUND_SERVICE_MICROPHONE / microphone FGS type is needed (that would
only be required if the mic-fallback path ran inside the service). -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<!-- Autostart on boot — BootReceiver spawns CaptureService in root
mode so capture resumes without the user touching the remote. -->
<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED" />
@@ -0,0 +1,234 @@
package com.ledgrab.android
import android.annotation.SuppressLint
import android.media.AudioAttributes
import android.media.AudioFormat
import android.media.AudioPlaybackCaptureConfiguration
import android.media.AudioRecord
import android.media.MediaRecorder
import android.media.projection.MediaProjection
import android.os.Build
import android.util.Log
import java.nio.ByteBuffer
import java.nio.ByteOrder
/**
* Captures audio with [AudioRecord] and pushes interleaved float32 PCM to
* the LedGrab Python server via [PythonBridge], where the
* `android_audio_engine` feeds it into the unchanged audio-analysis
* pipeline.
*
* Two sources:
* - [start] — system playback capture via `AudioPlaybackCapture` (API 29+),
* reusing the same [MediaProjection] token the app already holds for
* screen capture. This is the primary path on the consent flow.
* - [startMic] — microphone fallback (`AudioSource.MIC`) for paths with no
* MediaProjection (root mode) or API < 29.
*
* Mirrors [ScreenCapture]'s shape: a dedicated capture thread, a single
* reusable cross-JNI buffer (no per-block allocation → no GC churn on
* low-end TV boxes), and graceful teardown in [stop].
*
* The capture format is negotiated by [AudioRecord]; the **actual**
* channel count and sample rate are read back and forwarded to
* `configureAudio` so the Python analyzer's interleaving matches the bytes
* we push (e.g. a stereo request that the device satisfies as mono).
*/
class AudioCapture(
private val projection: MediaProjection?,
private val bridge: PythonBridge,
private val sampleRate: Int = 48000,
private val channels: Int = 2,
private val chunkFrames: Int = 1024,
) {
companion object {
private const val TAG = "AudioCapture"
private const val BYTES_PER_FLOAT = 4
}
private var audioRecord: AudioRecord? = null
private var captureThread: Thread? = null
@Volatile private var running = false
/**
* Start system playback capture (API 29+). Requires the app to hold
* RECORD_AUDIO and a valid [projection]. Returns true if capture began.
*/
@SuppressLint("MissingPermission")
fun start(): Boolean {
if (running) return true
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q) {
Log.i(TAG, "Playback capture needs API 29+; skipping (have ${Build.VERSION.SDK_INT})")
return false
}
val proj = projection
if (proj == null) {
Log.i(TAG, "No MediaProjection; playback capture unavailable")
return false
}
val config = AudioPlaybackCaptureConfiguration.Builder(proj)
.addMatchingUsage(AudioAttributes.USAGE_MEDIA)
.addMatchingUsage(AudioAttributes.USAGE_GAME)
.addMatchingUsage(AudioAttributes.USAGE_UNKNOWN)
.build()
val record = try {
AudioRecord.Builder()
.setAudioFormat(audioFormat())
.setBufferSizeInBytes(bufferBytes())
.setAudioPlaybackCaptureConfig(config)
.build()
} catch (e: Exception) {
Log.e(TAG, "Failed to build playback AudioRecord: ${e.message}")
return false
}
return begin(record, "playback")
}
/**
* Start microphone capture (fallback). Works on API 24+ and needs no
* MediaProjection. Requires RECORD_AUDIO. Returns true if capture began.
*
* ⚠️ SECURITY/POLICY: currently UNWIRED (no caller). Microphone capture is
* a materially different posture than playback capture — it records real
* room audio (bystander voices). Before wiring this into [CaptureService]:
* - add FOREGROUND_SERVICE_MICROPHONE permission + the `microphone` FGS
* type (on API 34+ the service is killed without it), and
* - add the Play Store privacy disclosure for microphone use,
* - re-trigger a security review.
* Do NOT call this from inside the foreground service without the above.
*/
@SuppressLint("MissingPermission")
fun startMic(): Boolean {
if (running) return true
val record = try {
AudioRecord.Builder()
.setAudioSource(MediaRecorder.AudioSource.MIC)
.setAudioFormat(audioFormat())
.setBufferSizeInBytes(bufferBytes())
.build()
} catch (e: Exception) {
Log.e(TAG, "Failed to build mic AudioRecord: ${e.message}")
return false
}
return begin(record, "mic")
}
/** Stop capturing and release all resources. Idempotent. */
fun stop() {
running = false
// AudioRecord.stop() unblocks a pending READ_BLOCKING read within
// milliseconds, so the loop sees running=false and returns well inside
// the 500ms join window — release() below won't race a live read.
// (Mirrors ScreenCapture's bounded join.)
runCatching { audioRecord?.stop() }
captureThread?.let { runCatching { it.join(500) } }
captureThread = null
runCatching { audioRecord?.release() }
audioRecord = null
runCatching { bridge.shutdownAudio() }
Log.i(TAG, "Audio capture stopped")
}
// ── internals ──────────────────────────────────────────────────────
private fun begin(record: AudioRecord, mode: String): Boolean {
if (record.state != AudioRecord.STATE_INITIALIZED) {
Log.e(TAG, "AudioRecord ($mode) failed to initialize")
runCatching { record.release() }
return false
}
val actualChannels = record.channelCount.coerceAtLeast(1)
val actualRate = record.sampleRate
// Confirm recording actually started before reporting success —
// startRecording() can throw (exclusive-capture contention) or
// leave the record in a non-recording state, in which case read()
// would only ever return errors.
val started = runCatching { record.startRecording() }.isSuccess &&
record.recordingState == AudioRecord.RECORDSTATE_RECORDING
if (!started) {
Log.e(TAG, "AudioRecord ($mode) failed to start recording")
runCatching { record.release() }
return false
}
// Recording confirmed — tell Python the real negotiated format
// before frames flow, so the analyzer's channel/sample-rate match
// the interleaving we push.
bridge.configureAudio(actualRate, actualChannels, chunkFrames)
audioRecord = record
running = true
captureThread = Thread(
{ captureLoop(record, actualChannels) },
"LedGrab-AudioCapture",
).also { it.start() }
Log.i(TAG, "Audio capture started ($mode, sr=$actualRate ch=$actualChannels chunk=$chunkFrames)")
return true
}
/**
* Blocking read loop. Accumulates into fixed `chunkFrames * channels`
* float blocks and pushes only COMPLETE blocks — [AudioRecord.read]
* returns a variable count, so partial reads are stitched here rather
* than handed to Python as ragged chunks (the analyzer requires
* whole-frame, ≤ chunk-size blocks).
*/
private fun captureLoop(record: AudioRecord, actualChannels: Int) {
val blockFloats = chunkFrames * actualChannels
val floatBuf = FloatArray(blockFloats)
// Reusable little-endian byte buffer — Python copies on push, so the
// same backing array is safe to overwrite next block. Default
// ByteBuffer order is BIG_ENDIAN, which would corrupt every sample;
// LITTLE_ENDIAN matches numpy's native float32 on all Android ABIs.
val byteBuf = ByteArray(blockFloats * BYTES_PER_FLOAT)
val floatView = ByteBuffer.wrap(byteBuf).order(ByteOrder.LITTLE_ENDIAN).asFloatBuffer()
var filled = 0
while (running) {
val n = record.read(floatBuf, filled, blockFloats - filled, AudioRecord.READ_BLOCKING)
if (n < 0) {
if (running) {
// A negative read (e.g. ERROR_DEAD_OBJECT after an audio-route
// change, ERROR_INVALID_OPERATION) means this AudioRecord is
// finished. Deactivate the Python engine so is_available() stops
// advertising a dead stream and the audio-reactive consumer isn't
// left polling an empty queue forever. We're on the capture thread,
// so we can't call stop() (it would self-join) — just flip running
// and shut the engine down; onDestroy's stop() releases the record.
Log.w(TAG, "AudioRecord.read error: $n — stopping audio capture")
running = false
runCatching { bridge.shutdownAudio() }
}
break
}
filled += n
if (filled < blockFloats) continue
floatView.clear()
floatView.put(floatBuf, 0, blockFloats)
bridge.pushAudio(byteBuf)
filled = 0
}
}
private fun channelMask(): Int =
if (channels >= 2) AudioFormat.CHANNEL_IN_STEREO else AudioFormat.CHANNEL_IN_MONO
private fun audioFormat(): AudioFormat =
AudioFormat.Builder()
.setEncoding(AudioFormat.ENCODING_PCM_FLOAT)
.setSampleRate(sampleRate)
.setChannelMask(channelMask())
.build()
private fun bufferBytes(): Int {
val minBuf = AudioRecord.getMinBufferSize(sampleRate, channelMask(), AudioFormat.ENCODING_PCM_FLOAT)
// A few blocks of headroom so a slow consumer doesn't overrun the
// hardware buffer between reads.
val want = chunkFrames * channels * BYTES_PER_FLOAT * 4
return if (minBuf > 0) maxOf(minBuf, want) else want
}
}
@@ -4,9 +4,11 @@ import android.app.Notification
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.Manifest
import android.app.Service
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import android.content.pm.ServiceInfo
import android.media.projection.MediaProjection
import android.media.projection.MediaProjectionManager
@@ -85,6 +87,7 @@ class CaptureService : Service() {
private var bridge: PythonBridge? = null
private var screenCapture: ScreenCapture? = null
private var rootCapture: RootScreenrecord? = null
private var audioCapture: AudioCapture? = null
private var mediaProjection: MediaProjection? = null
// Service-scoped coroutine scope for the root-capture watchdog.
@@ -338,6 +341,25 @@ class CaptureService : Service() {
onProjectionStopped = { stopSelf() },
).also { it.start() }
// Reuse the same projection to capture system playback audio so
// audio-reactive lighting works on-device (API 29+, RECORD_AUDIO
// granted). Best-effort: screen capture and the server keep running
// if audio is unavailable. Started AFTER ScreenCapture so the
// projection's callback is already registered.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q &&
checkSelfPermission(Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
) {
audioCapture = AudioCapture(projection, newBridge).also { ac ->
if (!ac.start()) {
Log.i(TAG, "Playback audio capture unavailable — continuing without audio")
audioCapture = null
}
}
} else {
Log.i(TAG, "RECORD_AUDIO not granted or API < 29 — audio-reactive capture disabled")
}
Log.i(TAG, "LedGrab service started (MediaProjection) — web UI at $url")
}
@@ -351,6 +373,10 @@ class CaptureService : Service() {
screenCapture?.stop()
screenCapture = null
// Stop audio before the server: stop() calls bridge.shutdownAudio().
audioCapture?.stop()
audioCapture = null
rootCapture?.stop()
rootCapture = null
@@ -53,6 +53,7 @@ class MainActivity : Activity() {
private const val SERVER_PORT = 8080
private const val REQUEST_MEDIA_PROJECTION = 1001
private const val REQUEST_POST_NOTIFICATIONS = 1002
private const val REQUEST_RECORD_AUDIO = 1003
private const val QR_SIZE_PX = 560
}
@@ -215,6 +216,7 @@ class MainActivity : Activity() {
private fun startCaptureService(resultCode: Int, resultData: Intent) {
ensureNotificationPermission()
ensureAudioPermission()
val intent = CaptureService.createIntent(this, resultCode, resultData)
ContextCompat.startForegroundService(this, intent)
updateUI()
@@ -471,4 +473,24 @@ class MainActivity : Activity() {
}
}
}
/**
* Request RECORD_AUDIO (API 29+) so the capture service can capture
* system playback audio for audio-reactive lighting. Fire-and-forget,
* like [ensureNotificationPermission]: capture still works without it
* (just no audio), so we don't block on the result. If first granted
* here, audio becomes available on the next Start.
*/
private fun ensureAudioPermission() {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q) return
if (checkSelfPermission(Manifest.permission.RECORD_AUDIO)
!= PackageManager.PERMISSION_GRANTED
) {
@Suppress("DEPRECATION")
requestPermissions(
arrayOf(Manifest.permission.RECORD_AUDIO),
REQUEST_RECORD_AUDIO,
)
}
}
}
@@ -28,6 +28,7 @@ class PythonBridge(private val context: Context) {
// single-writer/single-reader pattern we have here.
@Volatile private var mediaProjectionEngine: PyObject? = null
@Volatile private var rootEngine: PyObject? = null
@Volatile private var androidAudioEngine: PyObject? = null
/**
* Configure the MediaProjection engine with screen dimensions.
@@ -53,6 +54,49 @@ class PythonBridge(private val context: Context) {
Log.i(TAG, "Root screenrecord engine configured: ${width}x${height}")
}
/**
* Configure the Android playback-capture audio engine with the format
* actually negotiated by [AudioCapture]'s `AudioRecord`. Must be called
* before [pushAudio]. Caches the module handle for the per-block fast
* path (same pattern as [configureCapture]).
*/
fun configureAudio(sampleRate: Int, channels: Int, chunkFrames: Int) {
val py = Python.getInstance()
val engine = py.getModule("ledgrab.core.audio.android_audio_engine")
engine.callAttr("configure", sampleRate, channels, chunkFrames)
androidAudioEngine = engine
Log.i(TAG, "Android audio engine configured: sr=$sampleRate ch=$channels chunk=$chunkFrames")
}
/**
* Push one interleaved little-endian float32 PCM block to the Python
* audio engine. Called from [AudioCapture]'s capture thread. The byte
* array crosses the JNI boundary; Python copies it on receipt, so the
* caller may reuse the same buffer for the next block.
*/
fun pushAudio(pcmFloat32: ByteArray) {
if (!running) return
val engine = androidAudioEngine ?: return
try {
engine.callAttr("push_samples", pcmFloat32)
} catch (e: Exception) {
Log.w(TAG, "Failed to push audio: ${e.message}")
}
}
/**
* Deactivate the Python audio engine. Called from [AudioCapture.stop].
*/
fun shutdownAudio() {
val engine = androidAudioEngine ?: return
try {
engine.callAttr("shutdown")
} catch (e: Exception) {
Log.w(TAG, "Failed to shut down audio engine: ${e.message}")
}
androidAudioEngine = null
}
/**
* Start the LedGrab FastAPI server on a background thread.
*
+3 -2
View File
@@ -56,9 +56,10 @@ SetCompressor /SOLID lzma
; ── Functions ─────────────────────────────────────────────
Function LaunchApp
; Only launch the app — do NOT open the browser here. A manual launch (no
; --autostart) makes the app open the WebUI itself once /health responds,
; so opening the URL here too made the page appear twice.
ExecShell "open" "wscript.exe" '"$INSTDIR\scripts\${VBSNAME}"'
Sleep 2000
ExecShell "open" "http://localhost:8080/"
FunctionEnd
; Detect running instance before install (file lock check on python.exe)
+605 -289
View File
@@ -1,335 +1,651 @@
# LedGrab API Documentation
# LedGrab API Reference
Complete REST API reference for the LedGrab server.
Complete REST + WebSocket API reference for the LedGrab server.
**Base URL:** `http://localhost:8080`
**API Version:** v1
- **Base URL:** `http://localhost:8080`
- **API version:** `v1` (all REST paths are under `/api/v1`, except `/health`)
- **Interactive docs:** Swagger UI at [`/docs`](http://localhost:8080/docs), ReDoc at [`/redoc`](http://localhost:8080/redoc), raw schema at [`/openapi.json`](http://localhost:8080/openapi.json). The interactive docs are always the authoritative, up-to-date source for request/response schemas — this file is a hand-maintained overview.
> The application version is reported by `GET /api/v1/version`; this document is version-agnostic.
---
## Table of Contents
- [Health & Info](#health--info)
- [Device Management](#device-management)
- [Processing Control](#processing-control)
- [Settings Management](#settings-management)
- [Calibration](#calibration)
- [Metrics](#metrics)
- [Authentication](#authentication)
- [Conventions](#conventions)
- [WebSocket protocol](#websocket-protocol)
- [Worked examples](#worked-examples)
- **Endpoint reference**
- [Health & system info](#health--system-info)
- [System settings](#system-settings)
- [User preferences](#user-preferences)
- [Backup, restore & server control](#backup-restore--server-control)
- [Updates](#updates)
- [Snapshot](#snapshot)
- [Devices](#devices)
- [Capture templates, engines & filters](#capture-templates-engines--filters)
- [Picture sources](#picture-sources)
- [Post-processing templates](#post-processing-templates)
- [Output targets](#output-targets)
- [Output target control & live preview](#output-target-control--live-preview)
- [Color strip sources](#color-strip-sources)
- [Color strip processing templates](#color-strip-processing-templates)
- [Pattern templates](#pattern-templates)
- [Gradients](#gradients)
- [Audio devices](#audio-devices)
- [Audio sources](#audio-sources)
- [Audio templates & engines](#audio-templates--engines)
- [Audio processing templates](#audio-processing-templates)
- [Audio filters](#audio-filters)
- [Value sources](#value-sources)
- [Weather sources](#weather-sources)
- [Automations](#automations)
- [Scene presets](#scene-presets)
- [Sync clocks](#sync-clocks)
- [Webhooks](#webhooks)
- [HTTP endpoints](#http-endpoints)
- [Game integration](#game-integration)
- [Home Assistant](#home-assistant)
- [MQTT sources](#mqtt-sources)
- [Assets](#assets)
- [Graph wiring](#graph-wiring)
- [Web UI & PWA](#web-ui--pwa)
---
## Health & Info
## Authentication
### GET /health
LedGrab uses API-key authentication. The behavior depends on whether any keys are configured under `auth.api_keys` (see [INSTALLATION.md](../INSTALLATION.md)):
Health check endpoint.
| Situation | Loopback (`127.0.0.1` / `::1` / `localhost`) | LAN / remote |
| --------- | -------------------------------------------- | ------------ |
| **No keys configured** (default) | Allowed anonymously | **Rejected with `401`** |
| **Keys configured** | Valid Bearer token required | Valid Bearer token required |
Pass the key as a Bearer token:
```http
Authorization: Bearer <your-api-key>
```
A few **sensitive endpoints require a real API key even from localhost** (they reject the loopback-anonymous identity): the backup download/restore endpoints, and any endpoint that reveals stored secrets (e.g. `GET /api/v1/home-assistant/sources?include_secrets=true`). Configure a key to use those.
WebSocket endpoints authenticate with a [first-message handshake](#websocket-protocol) rather than the `Authorization` header.
---
## Conventions
- **Content type:** request and response bodies are JSON (`application/json`) unless noted (file uploads use `multipart/form-data`; some endpoints stream binary or file responses).
- **Errors:** failures return the standard FastAPI shape with an HTTP status code and a body of `{"detail": "<message>"}`. Validation errors return `422` with a structured `detail` array.
- **IDs:** entities are addressed by string IDs (e.g. `dev_…`, `ot_…`, `css_…`) generated on creation.
- **Common create/update fields:** most configurable entities accept `name`, `description`, `tags` (string array), and UI styling fields `icon` and `icon_color`.
- **Referential integrity:** deleting an entity that is still referenced (e.g. a device used by an output target) returns `409 Conflict`.
- **Timestamps:** ISO-8601 UTC strings.
---
## WebSocket protocol
All WebSocket endpoints share the same auth handshake:
1. The client connects. The server accepts the socket.
2. The client sends a JSON auth message as the **first** message, within ~3 seconds: `{"type": "auth", "token": "<your-api-key>"}`. On loopback with no keys configured, `token` may be `null` or the message omitted.
3. The server replies `{"type": "auth_ok"}` on success, or `{"type": "auth_error", "reason": "..."}` then closes (close code `4401`) on failure. A cross-site `Origin` is rejected with close code `4403`.
Browser clients must connect from an allowed `cors_origins` origin. After `auth_ok`, the stream payload depends on the endpoint (JSON event objects, JSON spectrum/metric frames, or binary RGB frames — see each endpoint's description).
The WebSocket endpoints are listed within their resource sections below (method `WS`).
---
## Worked examples
> Example values are illustrative.
**Health check**`GET /health` (no auth on loopback):
**Response:**
```json
{
"status": "healthy",
"timestamp": "2026-02-06T12:00:00Z",
"version": "0.1.0"
"timestamp": "2026-05-29T12:00:00Z",
"version": "0.8.1",
"demo_mode": false,
"auth_required": false,
"setup_required": false,
"uptime_seconds": 3600
}
```
### GET /api/v1/version
**Create a WLED device**`POST /api/v1/devices`:
Get version information.
**Response:**
```json
{
"version": "0.1.0",
"python_version": "3.11.0",
"api_version": "v1"
}
```
### GET /api/v1/config/displays
List available displays for screen capture.
**Response:**
```json
{
"displays": [
{
"index": 0,
"name": "Display 1",
"width": 1920,
"height": 1080,
"is_primary": true
}
],
"count": 1
}
```
---
## Device Management
### POST /api/v1/devices
Create and attach a new WLED device.
**Request:**
```json
{
"name": "Living Room TV",
"url": "http://192.168.1.100",
"device_type": "wled",
"led_count": 150
}
```
**Response:** `201 Created`
Response `201 Created` returns the stored device, including its generated `id`. (For Adalight, send `device_type: "adalight"`, the serial `url` like `COM3` or `/dev/ttyUSB0`, `led_count`, and `baud_rate`. Each device type accepts its own fields — see `/docs`.)
**Start / stop a target**`POST /api/v1/output-targets/{target_id}/start`:
```json
{
"id": "device_abc123",
"name": "Living Room TV",
"url": "http://192.168.1.100",
"led_count": 150,
"enabled": true,
"status": "disconnected",
"settings": {
"display_index": 0,
"fps": 30,
"border_width": 10
},
"calibration": {
"layout": "clockwise",
"start_position": "bottom_left",
"segments": [...]
},
"created_at": "2026-02-06T12:00:00Z",
"updated_at": "2026-02-06T12:00:00Z"
}
{ "status": "started", "target_id": "ot_abc123" }
```
### GET /api/v1/devices
**Authenticated request with a configured key:**
List all attached devices.
**Response:**
```json
{
"devices": [...],
"count": 2
}
```
### GET /api/v1/devices/{device_id}
Get device details.
**Response:** Same as POST response
### PUT /api/v1/devices/{device_id}
Update device information.
**Request:**
```json
{
"name": "Updated Name",
"enabled": true
}
```
### DELETE /api/v1/devices/{device_id}
Delete/detach a device.
**Response:** `204 No Content`
---
## Processing Control
### POST /api/v1/devices/{device_id}/start
Start screen processing for a device.
**Response:**
```json
{
"status": "started",
"device_id": "device_abc123"
}
```
### POST /api/v1/devices/{device_id}/stop
Stop screen processing.
**Response:**
```json
{
"status": "stopped",
"device_id": "device_abc123"
}
```
### GET /api/v1/devices/{device_id}/state
Get current processing state.
**Response:**
```json
{
"device_id": "device_abc123",
"processing": true,
"fps_actual": 29.8,
"fps_target": 30,
"display_index": 0,
"last_update": "2026-02-06T12:00:00Z",
"errors": []
}
```bash
curl -H "Authorization: Bearer your-api-key" \
http://localhost:8080/api/v1/devices
```
---
## Settings Management
## Endpoint reference
### GET /api/v1/devices/{device_id}/settings
## Health & system info
Get processing settings.
Health checks, version information, displays, system metrics, and integration status.
**Response:**
```json
{
"display_index": 0,
"fps": 30,
"brightness": 1.0,
"smoothing": 0.3,
"interpolation_mode": "average",
"standby_interval": 1.0,
"state_check_interval": 30
}
```
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/health` | Service health: status, version, uptime, and whether auth/setup is required. |
| GET | `/api/v1/version` | Application version, Python version, and API version. |
| GET | `/api/v1/tags` | All tags used across every entity in the system. |
| GET | `/api/v1/config/displays` | Available displays/monitors for screen capture (optional `engine_type` query, e.g. `scrcpy`). |
| GET | `/api/v1/system/processes` | Running process names, for use in automation conditions. |
| GET | `/api/v1/system/performance` | Current CPU, RAM, and GPU utilization metrics. |
| GET | `/api/v1/system/metrics-history` | Last ~2 minutes of system and per-target metrics for dashboard charts. |
| GET | `/api/v1/system/api-keys` | API-key labels with masked values (read-only; keys live in YAML config). |
| GET | `/api/v1/system/integrations-status` | Connection status for MQTT and Home Assistant integrations. |
### PUT /api/v1/devices/{device_id}/settings
## System settings
Update processing settings.
Server configuration: MQTT broker, external URL, shutdown action, log level, ADB connection, and live log streaming.
**Request:**
```json
{
"display_index": 1,
"fps": 60,
"brightness": 0.8
}
```
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/system/mqtt/settings` | Current MQTT broker settings (password masked). |
| PUT | `/api/v1/system/mqtt/settings` | Update MQTT broker settings (empty password preserves existing). |
| GET | `/api/v1/system/external-url` | Configured external base URL. |
| PUT | `/api/v1/system/external-url` | Set the external base URL for webhooks and user-visible links. |
| GET | `/api/v1/system/shutdown-action` | Configured server shutdown action (`stop_targets` or `nothing`). |
| PUT | `/api/v1/system/shutdown-action` | Set what happens to targets when the server shuts down. |
| WS | `/api/v1/system/logs/ws` | Live server log stream with a buffered backlog. |
| POST | `/api/v1/adb/connect` | Connect to a Wi-Fi ADB device by IP (auto-appends `:5555`). |
| POST | `/api/v1/adb/disconnect` | Disconnect a Wi-Fi ADB device. |
| GET | `/api/v1/system/log-level` | Current root logger level. |
| PUT | `/api/v1/system/log-level` | Change the log level at runtime without restart. |
## User preferences
Dashboard layout, notification settings, card display modes, and the global daylight timezone.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/preferences/dashboard-layout` | Read the saved dashboard layout (empty when unset). |
| PUT | `/api/v1/preferences/dashboard-layout` | Save the dashboard layout (opaque versioned JSON blob). |
| DELETE | `/api/v1/preferences/dashboard-layout` | Delete the saved layout; revert to default. |
| GET | `/api/v1/preferences/notifications` | Read notification preferences (server defaults when unset). |
| PUT | `/api/v1/preferences/notifications` | Persist notification preferences (channels, discovery, grace/debounce). |
| GET | `/api/v1/preferences/card-modes` | Read per-surface card-mode preferences. |
| PUT | `/api/v1/preferences/card-modes` | Save per-surface card modes (comfortable/compact/dense/row). |
| DELETE | `/api/v1/preferences/card-modes` | Delete card-mode preferences; revert to defaults. |
| GET | `/api/v1/preferences/daylight-timezone` | Read the global IANA timezone for daylight cycles. |
| PUT | `/api/v1/preferences/daylight-timezone` | Persist the daylight-cycle timezone (empty = server local). |
## Backup, restore & server control
Database backup/restore, server restart/shutdown, and auto-backup management.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/system/backup` | Download a full backup `.zip` (database + assets). 🔒 requires a key. |
| POST | `/api/v1/system/restore` | Upload a `.db`/`.zip` backup to restore config and trigger a restart. 🔒 requires a key. |
| POST | `/api/v1/system/restart` | Schedule a server restart and return immediately. |
| POST | `/api/v1/system/shutdown` | Gracefully shut down the server. |
| GET | `/api/v1/system/auto-backup/settings` | Auto-backup settings and status (enabled, interval, retention, last/next). |
| PUT | `/api/v1/system/auto-backup/settings` | Update auto-backup settings. |
| POST | `/api/v1/system/auto-backup/trigger` | Trigger a backup now and return its metadata. |
| GET | `/api/v1/system/backups` | List saved auto-backup files. |
| GET | `/api/v1/system/backups/{filename}` | Download a specific saved backup file. |
| DELETE | `/api/v1/system/backups/{filename}` | Delete a specific saved backup file. |
> 🔒 = requires a real API key even from localhost (rejects loopback-anonymous access).
## Updates
Auto-update management: check, apply, dismiss, and configure.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/system/update/status` | Current update status (available version, install type, capability). |
| POST | `/api/v1/system/update/check` | Trigger an immediate update check. |
| POST | `/api/v1/system/update/dismiss` | Dismiss the notification for a specific version. |
| POST | `/api/v1/system/update/apply` | Download and apply the available update, then shut down. |
| GET | `/api/v1/system/update/settings` | Update settings (enabled, interval, include prereleases). |
| PUT | `/api/v1/system/update/settings` | Change auto-update settings. |
## Snapshot
A single aggregated poll endpoint for low-overhead clients.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/snapshot` | Full poll payload (targets, states, metrics, devices, brightness, color/value sources, scene presets, sync clocks, system) in one response. Use `?include=` to request a subset; per-section fault isolation. |
## Devices
LED device CRUD, pairing, discovery, health checks, brightness/power control, and the WS pixel stream.
| Method | Path | Description |
| ------ | ---- | ----------- |
| POST | `/api/v1/devices` | Create/attach a new LED device (validates connectivity). |
| POST | `/api/v1/devices/pair` | Run a pairing handshake before creating a device. |
| GET | `/api/v1/devices` | List all attached devices. |
| GET | `/api/v1/devices/discover` | Scan the network for devices (optional `timeout`, `device_type`). |
| GET | `/api/v1/devices/openrgb-zones` | List zones on an OpenRGB device (`url` query). |
| GET | `/api/v1/devices/batch/states` | Health/connection state for all devices at once. |
| GET | `/api/v1/devices/{device_id}` | Get a device by ID. |
| PUT | `/api/v1/devices/{device_id}` | Update device configuration. |
| DELETE | `/api/v1/devices/{device_id}` | Delete/detach a device (`409` if referenced). |
| GET | `/api/v1/devices/{device_id}/state` | Get device health/connection state. |
| POST | `/api/v1/devices/{device_id}/ping` | Force an immediate health check. |
| GET | `/api/v1/devices/{device_id}/brightness` | Get current (cached) brightness. |
| PUT | `/api/v1/devices/{device_id}/brightness` | Set brightness (`0255`). |
| GET | `/api/v1/devices/{device_id}/power` | Get current power state. |
| PUT | `/api/v1/devices/{device_id}/power` | Turn the device on or off. |
| WS | `/api/v1/devices/{device_id}/ws` | Pixel stream for `ws` device type (`[brightness][R G B …]`). |
## Capture templates, engines & filters
Capture template CRUD/testing, capture engine discovery, and post-processing filter discovery.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/capture-templates` | List all capture templates. |
| POST | `/api/v1/capture-templates` | Create a capture template. |
| GET | `/api/v1/capture-templates/{template_id}` | Get a capture template by ID. |
| PUT | `/api/v1/capture-templates/{template_id}` | Update a capture template (partial). |
| DELETE | `/api/v1/capture-templates/{template_id}` | Delete a template (`409` if used by streams). |
| GET | `/api/v1/capture-engines` | List capture engines with platform availability. |
| POST | `/api/v1/capture-templates/test` | Test a capture config; returns FPS metrics + preview. |
| WS | `/api/v1/capture-templates/test/ws` | Real-time capture test with intermediate frame previews. |
| GET | `/api/v1/filters` | List post-processing filter types and option schemas. |
| GET | `/api/v1/strip-filters` | List filter types that support 1D LED-strip processing. |
## Picture sources
Screen captures, static images, video files, and processed streams used for color extraction.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/picture-sources` | List all picture sources. |
| POST | `/api/v1/picture-sources/validate-image` | Validate an image source and return a preview thumbnail. |
| GET | `/api/v1/picture-sources/full-image` | Serve a full-resolution image for lightbox preview (`source` query). |
| POST | `/api/v1/picture-sources` | Create a picture source (`raw`/`processed`/`static`/`video`). |
| GET | `/api/v1/picture-sources/{stream_id}` | Get a picture source by ID. |
| PUT | `/api/v1/picture-sources/{stream_id}` | Update a picture source. |
| DELETE | `/api/v1/picture-sources/{stream_id}` | Delete a picture source (`409` if referenced). |
| GET | `/api/v1/picture-sources/{stream_id}/thumbnail` | Thumbnail (first frame) for a video source. |
| POST | `/api/v1/picture-sources/{stream_id}/test` | Resolve the chain and run a capture test. |
| WS | `/api/v1/picture-sources/{stream_id}/test/ws` | Test stream with intermediate frame previews. |
## Post-processing templates
Reusable filter chains applied to picture sources.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/postprocessing-templates` | List all post-processing templates. |
| POST | `/api/v1/postprocessing-templates` | Create a template (name + filter list). |
| GET | `/api/v1/postprocessing-templates/{template_id}` | Get a template by ID. |
| PUT | `/api/v1/postprocessing-templates/{template_id}` | Update a template (partial). |
| DELETE | `/api/v1/postprocessing-templates/{template_id}` | Delete a template (`409` if referenced). |
| POST | `/api/v1/postprocessing-templates/{template_id}/test` | Capture from a source and apply the filters. |
| WS | `/api/v1/postprocessing-templates/{template_id}/test/ws` | Real-time test with intermediate frame previews. |
## Output targets
LED strips, Home Assistant light groups, and Zigbee2MQTT bulb groups.
| Method | Path | Description |
| ------ | ---- | ----------- |
| POST | `/api/v1/output-targets` | Create a target (`led` / `ha_light` / `z2m_light`). |
| GET | `/api/v1/output-targets` | List all output targets. |
| GET | `/api/v1/output-targets/batch/states` | Processing state for all targets at once. |
| GET | `/api/v1/output-targets/batch/metrics` | Metrics for all targets at once. |
| GET | `/api/v1/output-targets/{target_id}` | Get a single target. |
| PUT | `/api/v1/output-targets/{target_id}` | Update a target (partial, per type). |
| DELETE | `/api/v1/output-targets/{target_id}` | Delete a target (stops processing first). |
## Output target control & live preview
Start/stop processing, state & metrics, the calibration overlay, the global event stream, and live color/LED preview WebSockets.
| Method | Path | Description |
| ------ | ---- | ----------- |
| POST | `/api/v1/output-targets/bulk/start` | Start processing for multiple targets. |
| POST | `/api/v1/output-targets/bulk/stop` | Stop processing for multiple targets. |
| POST | `/api/v1/output-targets/{target_id}/start` | Start processing for one target. |
| POST | `/api/v1/output-targets/{target_id}/stop` | Stop processing for one target. |
| GET | `/api/v1/output-targets/{target_id}/state` | Current processing state (FPS, timing, device, errors). |
| GET | `/api/v1/output-targets/{target_id}/metrics` | Processing metrics (uptime, frames, error count). |
| WS | `/api/v1/events/ws` | Real-time state-change events across all targets. |
| POST | `/api/v1/output-targets/{target_id}/overlay/start` | Start the on-screen sampling/LED overlay. |
| POST | `/api/v1/output-targets/{target_id}/overlay/stop` | Stop the overlay. |
| GET | `/api/v1/output-targets/{target_id}/overlay/status` | Whether the overlay is active. |
| POST | `/api/v1/output-targets/{target_id}/ha-light/turn-off` | Turn off all HA light entities for the target. |
| WS | `/api/v1/output-targets/{target_id}/ha-light/ws` | Live HA light color preview. |
| POST | `/api/v1/output-targets/{target_id}/z2m-light/turn-off` | Publish OFF to all Zigbee2MQTT bulbs for the target. |
| WS | `/api/v1/output-targets/{target_id}/z2m-light/ws` | Live Zigbee2MQTT bulb color preview. |
| WS | `/api/v1/output-targets/{target_id}/led-preview/ws` | Live LED-strip preview (binary RGB frames). |
## Color strip sources
CRUD, calibration, raw color push, notifications, and preview streaming for color strip sources.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/color-strip-sources` | List all color strip sources. |
| POST | `/api/v1/color-strip-sources` | Create a color strip source (by `source_type`). |
| GET | `/api/v1/color-strip-sources/{source_id}` | Get a color strip source by ID. |
| PUT | `/api/v1/color-strip-sources/{source_id}` | Update a source; hot-reloads running streams. |
| DELETE | `/api/v1/color-strip-sources/{source_id}` | Delete a source (`409` if referenced). |
| POST | `/api/v1/color-strip-sources/{source_id}/overlay/start` | Start the screen overlay (picture-type, calibrated). |
| POST | `/api/v1/color-strip-sources/{source_id}/overlay/stop` | Stop the screen overlay. |
| GET | `/api/v1/color-strip-sources/{source_id}/overlay/status` | Whether the overlay is active. |
| POST | `/api/v1/color-strip-sources/{source_id}/colors` | Push raw LED colors to an `api_input` source. |
| POST | `/api/v1/color-strip-sources/{source_id}/notify` | Trigger a one-shot notification effect. |
| GET | `/api/v1/color-strip-sources/os-notifications/history` | Recent OS-notification capture history. |
| PUT | `/api/v1/color-strip-sources/{source_id}/calibration/test` | Light up LED edges to verify calibration. |
| POST | `/api/v1/color-strip-sources/{source_id}/key-colors/test` | Test a `key_colors` source (extract colors from rectangles). |
| WS | `/api/v1/color-strip-sources/{source_id}/key-colors/test/ws` | Real-time key-colors test preview. |
| WS | `/api/v1/color-strip-sources/preview/ws` | Transient ad-hoc source preview stream. |
| WS | `/api/v1/color-strip-sources/{source_id}/ws` | Push raw colors to an `api_input` source over WS. |
| WS | `/api/v1/color-strip-sources/{source_id}/test/ws` | Real-time source preview (binary RGB, optional JPEG). |
## Color strip processing templates
Reusable filter chains applied to color strips (1D LED data).
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/color-strip-processing-templates` | List all color-strip processing templates. |
| POST | `/api/v1/color-strip-processing-templates` | Create a template (name + filter list). |
| GET | `/api/v1/color-strip-processing-templates/{template_id}` | Get a template by ID. |
| PUT | `/api/v1/color-strip-processing-templates/{template_id}` | Update a template. |
| DELETE | `/api/v1/color-strip-processing-templates/{template_id}` | Delete a template (`409` if referenced). |
| WS | `/api/v1/color-strip-processing-templates/{template_id}/test/ws` | Real-time preview: apply the filter chain to an input source. |
## Pattern templates
Layout templates of named rectangles for LED device configuration.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/pattern-templates` | List all pattern templates. |
| POST | `/api/v1/pattern-templates` | Create a pattern template (named rectangles). |
| GET | `/api/v1/pattern-templates/{template_id}` | Get a pattern template by ID. |
| PUT | `/api/v1/pattern-templates/{template_id}` | Update a pattern template. |
| DELETE | `/api/v1/pattern-templates/{template_id}` | Delete a template (`409` if referenced by targets). |
## Gradients
Reusable gradient definitions (color stops). Built-in gradients are read-only but clonable.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/gradients` | List all gradients (built-in and user-created). |
| POST | `/api/v1/gradients` | Create a user-defined gradient. |
| GET | `/api/v1/gradients/{gradient_id}` | Get a gradient by ID. |
| PUT | `/api/v1/gradients/{gradient_id}` | Update a gradient (built-ins are read-only). |
| POST | `/api/v1/gradients/{gradient_id}/clone` | Clone a gradient into a customizable copy. |
| DELETE | `/api/v1/gradients/{gradient_id}` | Delete a gradient (`400` if built-in or referenced). |
## Audio devices
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/audio-devices` | List audio input/output devices (flat list + per-engine grouping). |
## Audio sources
Audio capture and processing sources for audio-reactive effects.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/audio-sources` | List all audio sources (optional `source_type`). |
| POST | `/api/v1/audio-sources` | Create an audio source (`capture` or `processed`). |
| GET | `/api/v1/audio-sources/{source_id}` | Get an audio source by ID. |
| PUT | `/api/v1/audio-sources/{source_id}` | Update an audio source (partial). |
| DELETE | `/api/v1/audio-sources/{source_id}` | Delete an audio source (`409` if referenced). |
| WS | `/api/v1/audio-sources/{source_id}/test/ws` | Real-time spectrum/RMS/peak/beat analysis (~20 Hz). |
## Audio templates & engines
Audio capture templates and engine discovery.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/audio-templates` | List all audio capture templates. |
| POST | `/api/v1/audio-templates` | Create an audio capture template. |
| GET | `/api/v1/audio-templates/{template_id}` | Get an audio template by ID. |
| PUT | `/api/v1/audio-templates/{template_id}` | Update an audio template. |
| DELETE | `/api/v1/audio-templates/{template_id}` | Delete a template (cascades to audio sources). |
| GET | `/api/v1/audio-engines` | List audio capture engines and availability. |
| WS | `/api/v1/audio-templates/{template_id}/test/ws` | Real-time spectrum test for a template + device. |
## Audio processing templates
Reusable audio filter chains.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/audio-processing-templates` | List all audio processing templates. |
| POST | `/api/v1/audio-processing-templates` | Create a template (name + filter list). |
| GET | `/api/v1/audio-processing-templates/{template_id}` | Get a template by ID. |
| PUT | `/api/v1/audio-processing-templates/{template_id}` | Update a template (hot-updates running streams). |
| DELETE | `/api/v1/audio-processing-templates/{template_id}` | Delete a template (`409` if referenced). |
## Audio filters
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/audio-filters` | List audio filter types and their option schemas. |
## Value sources
Dynamic data inputs (brightness and other parameters): static, animated, audio, adaptive, color, sensor, HTTP, Home Assistant, and `template` — a sandboxed-Jinja **combinator** that evaluates an expression over the live values of other value sources.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/value-sources` | List all value sources (optional `source_type`). |
| POST | `/api/v1/value-sources` | Create a value source (discriminated by `source_type`). |
| POST | `/api/v1/value-sources/validate-template` | Validate a template expression + inputs (advisory; always `200` with `{valid, error, errors, warnings, variables}`). |
| GET | `/api/v1/value-sources/{source_id}` | Get a value source by ID. |
| PUT | `/api/v1/value-sources/{source_id}` | Update a value source; hot-reloads running streams. |
| DELETE | `/api/v1/value-sources/{source_id}` | Delete a value source (`400` if referenced by a target or another value source). |
| WS | `/api/v1/value-sources/{source_id}/test/ws` | Real-time value output stream (~20 Hz). |
### Template value source (`source_type: "template"`)
A `float` combinator. Fields: `template` (a Jinja *expression*), `inputs` (`[{name, value_source_id}]` bindings to other value sources), `default_value` (fallback in `[0,1]` on any error), and `eval_interval` (optional re-eval throttle in seconds; `0`/null = every poll). At runtime each input is exposed by its `name` (the source's normalized `0..1` value) plus `raw[name]` (its un-normalized value, where available). Globals: `min`, `max`, `abs`, `round`, `clamp(x, lo=0, hi=1)`. The expression runs in a hardened `ImmutableSandboxedEnvironment` (no statements/blocks, filters, attribute access, `**`, or string repetition); results are coerced, NaN/inf-rejected, and clamped to `[0,1]`. Reference cycles and over-deep nesting are rejected at save time. For time-of-day logic, bind an `adaptive_time` or `daylight` source as an input.
## Weather sources
Weather data providers feeding weather-driven value sources.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/weather-sources` | List all weather sources. |
| POST | `/api/v1/weather-sources` | Create a weather source (provider, lat/lon, interval). |
| GET | `/api/v1/weather-sources/{source_id}` | Get a weather source by ID. |
| PUT | `/api/v1/weather-sources/{source_id}` | Update a weather source. |
| DELETE | `/api/v1/weather-sources/{source_id}` | Delete a weather source. |
| POST | `/api/v1/weather-sources/{source_id}/test` | Force-fetch current weather and return it. |
## Automations
Rules that trigger scene presets (time, display state, MQTT, webhooks, Home Assistant, HTTP polling, active window).
| Method | Path | Description |
| ------ | ---- | ----------- |
| POST | `/api/v1/automations` | Create an automation (rules + scene preset + deactivation). |
| GET | `/api/v1/automations` | List automations with current activity state. |
| GET | `/api/v1/automations/{automation_id}` | Get an automation by ID (includes webhook URL if any). |
| PUT | `/api/v1/automations/{automation_id}` | Update an automation (partial); re-evaluates if enabled. |
| DELETE | `/api/v1/automations/{automation_id}` | Delete and deactivate an automation. |
| POST | `/api/v1/automations/{automation_id}/enable` | Enable and immediately evaluate rules. |
| POST | `/api/v1/automations/{automation_id}/disable` | Disable and deactivate. |
## Scene presets
Captured snapshots of target state that can be restored.
| Method | Path | Description |
| ------ | ---- | ----------- |
| POST | `/api/v1/scene-presets` | Create a preset by capturing current target state. |
| GET | `/api/v1/scene-presets` | List all scene presets. |
| GET | `/api/v1/scene-presets/{preset_id}` | Get a scene preset by ID. |
| PUT | `/api/v1/scene-presets/{preset_id}` | Update metadata and optionally change targets. |
| DELETE | `/api/v1/scene-presets/{preset_id}` | Delete a scene preset. |
| POST | `/api/v1/scene-presets/{preset_id}/recapture` | Re-capture current state into the preset. |
| POST | `/api/v1/scene-presets/{preset_id}/activate` | Activate the preset (restore captured state). |
## Sync clocks
Shared clocks that drive linked animations with configurable speed.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/sync-clocks` | List all synchronization clocks. |
| POST | `/api/v1/sync-clocks` | Create a sync clock. |
| GET | `/api/v1/sync-clocks/{clock_id}` | Get a sync clock by ID. |
| PUT | `/api/v1/sync-clocks/{clock_id}` | Update a clock (speed changes hot-applied). |
| DELETE | `/api/v1/sync-clocks/{clock_id}` | Delete a clock (`409` if referenced). |
| POST | `/api/v1/sync-clocks/{clock_id}/pause` | Pause the clock (freeze linked animations). |
| POST | `/api/v1/sync-clocks/{clock_id}/resume` | Resume a paused clock. |
| POST | `/api/v1/sync-clocks/{clock_id}/reset` | Reset the clock to `t=0`. |
## Webhooks
Inbound trigger endpoint for external services.
| Method | Path | Description |
| ------ | ---- | ----------- |
| POST | `/api/v1/webhooks/{token}` | Trigger an automation by secret token (`activate`/`deactivate`; rate-limited 30/min/IP). |
## HTTP endpoints
Outbound HTTP polling endpoints for integrations. 🔒 These require a real API key even on loopback.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/http/endpoints` | List all HTTP polling endpoints. |
| POST | `/api/v1/http/endpoints` | Create an endpoint (URL, method, auth token, headers). |
| GET | `/api/v1/http/endpoints/{endpoint_id}` | Get an endpoint by ID. |
| PUT | `/api/v1/http/endpoints/{endpoint_id}` | Update an endpoint. |
| DELETE | `/api/v1/http/endpoints/{endpoint_id}` | Delete an endpoint. |
| POST | `/api/v1/http/endpoints/test` | One-shot test fetch to validate a config before saving. |
| POST | `/api/v1/http/endpoints/{endpoint_id}/test` | Test a stored endpoint without re-entering its token. |
## Game integration
Game event ingestion, adapter metadata, presets, and diagnostics.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/game-integrations/presets` | List built-in effect presets. |
| GET | `/api/v1/game-integrations` | List all game integration configs. |
| POST | `/api/v1/game-integrations` | Create a game integration config. |
| GET | `/api/v1/game-integrations/{integration_id}` | Get a config by ID. |
| PUT | `/api/v1/game-integrations/{integration_id}` | Update a config. |
| DELETE | `/api/v1/game-integrations/{integration_id}` | Delete a config. |
| POST | `/api/v1/game-integrations/{integration_id}/event` | Ingest a game event (adapter-level auth; 1664 Hz). |
| GET | `/api/v1/game-integrations/{integration_id}/status` | Runtime status (connected state, event counts). |
| GET | `/api/v1/game-integrations/{integration_id}/events` | Recent events for debugging (`limit`). |
| GET | `/api/v1/game-adapters` | List adapter types and supported events. |
| POST | `/api/v1/game-integrations/{integration_id}/apply-preset` | Apply a built-in preset (optionally replacing mappings). |
| POST | `/api/v1/game-integrations/{integration_id}/auto-setup` | Write game config files and generate an auth token. |
## Home Assistant
Home Assistant WebSocket sources, entity discovery, and live status.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/home-assistant/sources` | List HA sources with status and entity counts (`?include_secrets=true` 🔒). |
| POST | `/api/v1/home-assistant/sources` | Create an HA source (host, long-lived token, filters). |
| GET | `/api/v1/home-assistant/sources/{source_id}` | Get an HA source (`?include_secrets=true` 🔒). |
| PUT | `/api/v1/home-assistant/sources/{source_id}` | Update an HA source; refreshes the connection. |
| DELETE | `/api/v1/home-assistant/sources/{source_id}` | Delete an HA source and release its runtime. |
| GET | `/api/v1/home-assistant/sources/{source_id}/entities` | List available HA entities (live + cache fallback). |
| POST | `/api/v1/home-assistant/sources/{source_id}/test` | Test connection/auth and report HA version. |
| GET | `/api/v1/home-assistant/status` | Overall HA integration status per source. |
## MQTT sources
MQTT broker connections (sources) and status monitoring.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/mqtt/sources` | List MQTT sources with connection status. |
| POST | `/api/v1/mqtt/sources` | Create an MQTT source (broker connection). |
| GET | `/api/v1/mqtt/sources/{source_id}` | Get an MQTT source by ID. |
| PUT | `/api/v1/mqtt/sources/{source_id}` | Update a source; restarts the broker runtime. |
| DELETE | `/api/v1/mqtt/sources/{source_id}` | Delete a source and release its runtime. |
| POST | `/api/v1/mqtt/sources/{source_id}/test` | Test connection to the broker (10s timeout). |
| GET | `/api/v1/mqtt/status` | Overall MQTT integration status per source. |
## Assets
Media files (sounds, images, videos) used by effects and notifications.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/assets` | List assets (optional `asset_type` filter). |
| GET | `/api/v1/assets/{asset_id}` | Get asset metadata by ID. |
| POST | `/api/v1/assets` | Upload a new asset file (`multipart/form-data`). |
| PUT | `/api/v1/assets/{asset_id}` | Update asset metadata. |
| DELETE | `/api/v1/assets/{asset_id}` | Delete an asset (prebuilt assets are soft-deleted/restorable). |
| GET | `/api/v1/assets/{asset_id}/file` | Serve the asset file (download). |
| POST | `/api/v1/assets/restore-prebuilt` | Re-import any deleted prebuilt assets. |
## Graph wiring
The wiring-graph: schema registry, topology, dependents, validation, and subgraph duplication.
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/api/v1/graph/schema` | Registry of connectable reference fields. |
| GET | `/api/v1/graph` | Full wiring topology (nodes + edges) and validation report. |
| GET | `/api/v1/graph/dependents/{kind}/{entity_id}` | Every entity that references `(kind, entity_id)`. |
| POST | `/api/v1/graph/validate-connection` | Validate a proposed wiring edit (existence, kind, no cycle). |
| POST | `/api/v1/graph/duplicate` | Deep-clone selected value/color-strip sources with remapped wiring. |
## Web UI & PWA
App-level routes served by FastAPI (not under `/api/v1`).
| Method | Path | Description |
| ------ | ---- | ----------- |
| GET | `/` | The web dashboard UI. |
| GET | `/manifest.json` | PWA manifest (root scope). |
| GET | `/sw.js` | Service worker (root scope). |
| GET | `/openapi.json` | OpenAPI schema. |
| GET | `/docs` | Swagger UI (interactive API docs). |
| GET | `/redoc` | ReDoc API reference. |
---
## Calibration
## Next steps
### GET /api/v1/devices/{device_id}/calibration
Get calibration configuration.
**Response:**
```json
{
"layout": "clockwise",
"start_position": "bottom_left",
"segments": [
{
"edge": "bottom",
"led_start": 0,
"led_count": 40,
"reverse": false
},
{
"edge": "right",
"led_start": 40,
"led_count": 30,
"reverse": false
},
{
"edge": "top",
"led_start": 70,
"led_count": 40,
"reverse": true
},
{
"edge": "left",
"led_start": 110,
"led_count": 40,
"reverse": true
}
]
}
```
### PUT /api/v1/devices/{device_id}/calibration
Update calibration.
**Request:** Same as GET response
### POST /api/v1/devices/{device_id}/calibration/test
Test calibration by lighting up specific edge.
**Query Parameters:**
- `edge`: Edge to test (top, right, bottom, left)
- `color`: RGB color array (e.g., [255, 0, 0])
---
## Metrics
### GET /api/v1/devices/{device_id}/metrics
Get detailed processing metrics.
**Response:**
```json
{
"device_id": "device_abc123",
"processing": true,
"fps_actual": 29.8,
"fps_target": 30,
"uptime_seconds": 3600.5,
"frames_processed": 107415,
"errors_count": 2,
"last_error": null,
"last_update": "2026-02-06T12:00:00Z"
}
```
---
## Error Responses
All endpoints may return error responses in this format:
```json
{
"error": "ErrorType",
"message": "Human-readable error message",
"detail": {...},
"timestamp": "2026-02-06T12:00:00Z"
}
```
**Common HTTP Status Codes:**
- `200 OK` - Success
- `201 Created` - Resource created
- `204 No Content` - Success with no response body
- `400 Bad Request` - Invalid request
- `404 Not Found` - Resource not found
- `500 Internal Server Error` - Server error
---
## Interactive Documentation
The server provides interactive API documentation:
- **Swagger UI:** http://localhost:8080/docs
- **ReDoc:** http://localhost:8080/redoc
- **OpenAPI JSON:** http://localhost:8080/openapi.json
- [Installation Guide](../INSTALLATION.md)
- [Calibration Guide](CALIBRATION.md)
- Interactive, always-current schemas: [`/docs`](http://localhost:8080/docs)
Binary file not shown.

After

Width:  |  Height:  |  Size: 270 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 412 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 213 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 358 KiB

+12 -10
View File
@@ -6,33 +6,35 @@
- `src/ledgrab/api/routes/` — REST API endpoints (one file per entity)
- `src/ledgrab/api/schemas/` — Pydantic request/response models (one file per entity)
- `src/ledgrab/core/` — Core business logic (capture, devices, audio, processing, automations)
- `src/ledgrab/storage/` — Data models (dataclasses) and JSON persistence stores
- `src/ledgrab/storage/` — Data models (dataclasses) and SQLite-backed persistence stores (`BaseSqliteStore`)
- `src/ledgrab/utils/` — Utility functions (logging, monitor detection, SSRF validation, sound playback)
- `src/ledgrab/static/` — Frontend files (TypeScript, CSS, locales)
- `src/ledgrab/templates/` — Jinja2 HTML templates
- `config/` — Configuration files (YAML)
- `data/` — Runtime data (JSON stores, persisted state)
- `data/` — Runtime data: SQLite database (`ledgrab.db`) + assets. Relocate the root with `LEDGRAB_DATA_DIR`.
## Entity & Storage Pattern
Each entity follows: dataclass model (`storage/`) + JSON store (`storage/*_store.py`) + Pydantic schemas (`api/schemas/`) + routes (`api/routes/`).
Each entity follows: dataclass model (`storage/`) + SQLite store (`storage/*_store.py`, subclassing `BaseSqliteStore`) + Pydantic schemas (`api/schemas/`) + routes (`api/routes/`).
Stores keep an in-memory write-through cache over a per-entity SQLite table (the legacy `BaseJsonStore` still exists for reference but new stores use `BaseSqliteStore`). Schema/data shape changes go through `storage/data_migrations.py` — migrations are idempotent and tracked in a dedicated `data_migrations` audit table, so they run safely on every startup. **When renaming or restructuring stored fields, add a migration there** (see the Data Migration Policy in the root `CLAUDE.md`).
## Authentication
Server uses API key authentication via Bearer token in `Authorization` header.
API key authentication via Bearer token in the `Authorization` header (`Authorization: Bearer <key>`). WebSocket connections authenticate with a first-message handshake (`{"type":"auth","token":"<key>"}`). See `src/ledgrab/api/auth.py` for the canonical logic.
- Config: `config/default_config.yaml` under `auth.api_keys`
- Env var: `LEDGRAB_AUTH__API_KEYS`
- When `api_keys` is empty (default), auth is disabled — all endpoints are open
- To enable auth, add key entries (e.g. `dev: "your-secret-key"`)
- Config: `config/default_config.yaml` under `auth.api_keys`; env var `LEDGRAB_AUTH__API_KEYS`
- When `api_keys` is **empty** (default): **loopback** requests (`127.0.0.1` / `::1` / `localhost`) are allowed anonymously, but **LAN / remote** requests are rejected with `401`. Auth is *not* fully open.
- When `api_keys` is **set**: a valid Bearer token is required from every client (loopback included).
- `require_authenticated()` rejects even loopback-anonymous callers on sensitive endpoints (e.g. backup download, secret reveal).
## Common Tasks
### Adding a new API endpoint
1. Create route file in `api/routes/`
1. Create route file in `api/routes/` (define an `APIRouter(prefix="/api/v1/...")`)
2. Define request/response schemas in `api/schemas/`
3. Register the router in `main.py`
3. Register the router in `api/__init__.py` (it aggregates every route module into the single `router` that `main.py` mounts)
4. Restart the server
5. Test via `/docs` (Swagger UI)
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "ledgrab"
version = "0.8.0"
version = "0.8.1"
description = "Ambient lighting system that captures screen content and drives LED strips in real time"
authors = [
{name = "Alexei Dolgolyov", email = "dolgolyov.alexei@gmail.com"}
+1 -1
View File
@@ -9,7 +9,7 @@ from pathlib import Path
# In dev (running from source without `pip install -e .`) and on Android
# (Chaquopy embeds the source directly with no dist-info), we additionally
# read pyproject.toml so the version is always correct without manual sync.
_FALLBACK_VERSION = "0.8.0"
_FALLBACK_VERSION = "0.8.1"
def _read_pyproject_version() -> str | None:
+22 -8
View File
@@ -39,8 +39,9 @@ _fix_embedded_tcl_paths()
import uvicorn # noqa: E402
from ledgrab.config import get_config # noqa: E402
from ledgrab.config import Config, get_config # noqa: E402
from ledgrab.server_ref import set_server, set_tray # noqa: E402
from ledgrab.shutdown_state import GRACEFUL_SHUTDOWN_TIMEOUT # noqa: E402
from ledgrab.tray import PYSTRAY_AVAILABLE, TrayManager # noqa: E402
from ledgrab.utils import setup_logging, get_logger # noqa: E402
from ledgrab.utils.platform import is_windows # noqa: E402
@@ -108,17 +109,28 @@ def _check_port(host: str, port: int) -> None:
sys.exit(1)
def main() -> None:
config = get_config()
_check_port(config.server.host, config.server.port)
def _build_server(config: Config) -> uvicorn.Server:
"""Construct the uvicorn Server with a bounded graceful-shutdown timeout.
Extracted so the graceful-shutdown bound is unit-testable — leaving it
unset (the uvicorn default of ``None``) is the regression that strands
LED targets and prevents the process from exiting.
"""
uv_config = uvicorn.Config(
"ledgrab.main:app",
host=config.server.host,
port=config.server.port,
log_level=config.server.log_level.lower(),
timeout_graceful_shutdown=GRACEFUL_SHUTDOWN_TIMEOUT,
)
server = uvicorn.Server(uv_config)
return uvicorn.Server(uv_config)
def main() -> None:
config = get_config()
_check_port(config.server.host, config.server.port)
server = _build_server(config)
set_server(server)
# Wire the OS-shutdown safety net. The lifespan in ``ledgrab.main`` signals
@@ -165,9 +177,11 @@ def main() -> None:
tray.run()
# Tray exited — wait for server to finish its graceful shutdown.
# Use a longer join than the lifespan's own ~18 s budget so we don't
# cut the DB checkpoint short on a slow disk.
server_thread.join(timeout=20)
# Budget: the graceful-shutdown wait (GRACEFUL_SHUTDOWN_TIMEOUT) runs
# first, then the lifespan's own ~16 s shutdown (target restore + DB
# checkpoint). Join longer than their sum so a slow disk doesn't get
# the DB checkpoint cut short.
server_thread.join(timeout=25)
if guard is not None:
guard.stop()
else:
+5
View File
@@ -84,6 +84,8 @@ def start_server(data_dir: str, port: int = 8080, api_key: str | None = None) ->
"LEDGRAB_AUTH__API_KEYS."
)
from ledgrab.shutdown_state import GRACEFUL_SHUTDOWN_TIMEOUT
uv_config = uvicorn.Config(
"ledgrab.main:app",
host=config.server.host,
@@ -91,6 +93,9 @@ def start_server(data_dir: str, port: int = 8080, api_key: str | None = None) ->
log_level=config.server.log_level.lower(),
# No uvloop/httptools on Android — use pure-Python asyncio
loop="asyncio",
# Bound the graceful-shutdown wait so stop_server() can't hang forever
# on a lingering WebView events WebSocket — see shutdown_state for why.
timeout_graceful_shutdown=GRACEFUL_SHUTDOWN_TIMEOUT,
)
global _server, _loop
+4
View File
@@ -33,6 +33,8 @@ from .routes.audio_processing_templates import router as audio_processing_templa
from .routes.audio_filters import router as audio_filters_router
from .routes.pattern_templates import router as pattern_templates_router
from .routes.preferences import router as preferences_router
from .routes.snapshot import router as snapshot_router
from .routes.graph import router as graph_router
router = APIRouter()
router.include_router(system_router)
@@ -66,5 +68,7 @@ router.include_router(audio_processing_templates_router)
router.include_router(audio_filters_router)
router.include_router(pattern_templates_router)
router.include_router(preferences_router)
router.include_router(snapshot_router)
router.include_router(graph_router)
__all__ = ["router"]
+4
View File
@@ -80,6 +80,7 @@ def verify_api_key(
if not config.auth.api_keys:
# No keys configured — allow loopback only.
if _is_loopback(client_host):
request.state.auth_label = "anonymous"
return "anonymous"
# Allow caller to authenticate explicitly even without configured keys?
# No — there are no keys to compare against. Reject.
@@ -123,6 +124,9 @@ def verify_api_key(
# Log successful authentication
logger.debug(f"Authenticated as: {authenticated_as}")
# Stash the friendly label so the access-log middleware can attribute the
# request to a client without re-running the token comparison.
request.state.auth_label = authenticated_as
return authenticated_as
+609
View File
@@ -0,0 +1,609 @@
"""Authoritative wiring-graph schema and topology engine.
This module is the single source of truth for **which reference fields connect
which entity kinds**. The frontend graph editor historically hard-coded the same
information in two places (``graph-connections.ts`` ``CONNECTION_MAP`` and
``graph-layout.ts`` ``buildGraph``); the ``GET /api/v1/graph/schema`` endpoint
now serves this registry so the client can render ports and edges generically
and the two never drift.
This registry is a *superset* of the current frontend ``buildGraph``: it also
declares real references that ``buildGraph`` does not yet draw (e.g.
``value_source.value_source_id`` chaining and ``value_source.color_strip_source_id``).
The backend is authoritative; the client is expected to converge on it.
Everything in this module is pure (operates on plain dicts), so the topology
build, dependency lookup, cycle and dangling-reference detection are all unit
testable without booting the app or any store.
Field-path grammar (the ``field`` of a :class:`ConnectionField`):
* ``"device_id"`` — a top-level string id.
* ``"brightness.source_id"`` — a nested object; ``brightness`` may be a
plain number (unbound :class:`BindableFloat`) or ``{"value", "source_id"}``.
* ``"settings.pattern_template_id"`` — arbitrarily deep object access.
* ``"layers[].source_id"`` — ``layers`` is a list; read ``source_id``
from every element.
* ``"calibration.lines[].picture_source_id"`` — object → list → field.
"""
from __future__ import annotations
import logging
from dataclasses import asdict, dataclass, is_dataclass
from typing import Any
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class ConnectionField:
"""One connectable reference: ``target_kind.field`` points at ``source_kind``."""
target_kind: str
"""Entity kind that *holds* the reference (the consumer / referrer)."""
field: str
"""Dot-path to the reference value (see module docstring grammar)."""
source_kind: str
"""Entity kind being referenced (the producer / source)."""
edge_type: str
"""Edge category, used by the client for colour and port grouping."""
bindable: bool = False
"""True when the slot is a :class:`BindableFloat`/``BindableColor`` value binding."""
nested: bool = False
"""True when the field lives inside a nested object/list (dotted path)."""
@property
def is_list(self) -> bool:
"""True when any path segment iterates a list (``foo[]``)."""
return "[]" in self.field
# ── Entity kinds & their human "type" attribute ────────────────────────────
# Mirrors the frontend buildGraph(): kind → the serialized field that carries
# the entity's subtype (used only for the node label / icon).
NODE_TYPE_FIELD: dict[str, str] = {
"device": "device_type",
"capture_template": "engine_type",
"pp_template": "",
"audio_template": "engine_type",
"pattern_template": "",
"picture_source": "stream_type",
"audio_source": "source_type",
"value_source": "source_type",
"color_strip_source": "source_type",
"sync_clock": "",
"output_target": "target_type",
"scene_preset": "",
"automation": "",
"cspt": "",
}
ENTITY_KINDS: tuple[str, ...] = tuple(NODE_TYPE_FIELD.keys())
# ── The registry ───────────────────────────────────────────────────────────
# NOTE: ``gradient`` and ``ha_source`` reference fields are intentionally
# omitted — they are not first-class graph node kinds, so wiring them would
# only ever produce dangling-reference noise.
CONNECTION_SCHEMA: tuple[ConnectionField, ...] = (
# ── Picture sources ──
ConnectionField("picture_source", "capture_template_id", "capture_template", "template"),
ConnectionField("picture_source", "source_stream_id", "picture_source", "picture"),
ConnectionField("picture_source", "postprocessing_template_id", "pp_template", "template"),
# ── Audio sources ──
ConnectionField("audio_source", "audio_template_id", "audio_template", "audio"),
ConnectionField("audio_source", "audio_source_id", "audio_source", "audio"),
# ── Value sources ──
ConnectionField("value_source", "audio_source_id", "audio_source", "audio"),
ConnectionField("value_source", "picture_source_id", "picture_source", "picture"),
ConnectionField("value_source", "value_source_id", "value_source", "value"),
ConnectionField("value_source", "color_strip_source_id", "color_strip_source", "colorstrip"),
# AnimatedColorValueSource references a sync clock for shared timing.
ConnectionField("value_source", "clock_id", "sync_clock", "clock"),
# ── Color strip sources (top-level) ──
ConnectionField("color_strip_source", "picture_source_id", "picture_source", "picture"),
ConnectionField("color_strip_source", "audio_source_id", "audio_source", "audio"),
ConnectionField("color_strip_source", "clock_id", "sync_clock", "clock"),
ConnectionField("color_strip_source", "input_source_id", "color_strip_source", "colorstrip"),
ConnectionField("color_strip_source", "processing_template_id", "cspt", "template"),
# ── Color strip sources (BindableFloat value bindings) ──
*(
ConnectionField(
"color_strip_source",
f"{prop}.source_id",
"value_source",
"value",
bindable=True,
nested=True,
)
for prop in (
"smoothing",
"sensitivity",
"intensity",
"scale",
"speed",
"wind_strength",
"temperature_influence",
"sound_volume",
"timeout",
"brightness",
)
),
# ── Color strip sources (BindableColor value bindings) ──
# NOTE: `bindable` here is *structural* (these are BindableColor fields). They
# are NOT usefully wireable from the graph: a ValueStream yields a scalar
# (`get_value() -> float`) and every colour consumer reads the static RGB via
# `bcolor()` (source_id ignored at runtime). The graph editor keeps them
# read-only; do not enable them without a colour-producing value source.
*(
ConnectionField(
"color_strip_source",
f"{prop}.source_id",
"value_source",
"value",
bindable=True,
nested=True,
)
for prop in ("color", "color_peak", "fallback_color", "default_color")
),
# ── Color strip sources (composite layers / mapped zones / calibration) ──
ConnectionField(
"color_strip_source", "layers[].source_id", "color_strip_source", "colorstrip", nested=True
),
ConnectionField(
"color_strip_source",
"layers[].brightness_source_id",
"value_source",
"value",
bindable=True,
nested=True,
),
ConnectionField(
"color_strip_source", "layers[].processing_template_id", "cspt", "template", nested=True
),
ConnectionField(
"color_strip_source", "zones[].source_id", "color_strip_source", "colorstrip", nested=True
),
ConnectionField(
"color_strip_source",
"calibration.lines[].picture_source_id",
"picture_source",
"picture",
nested=True,
),
# ── Output targets ──
ConnectionField("output_target", "device_id", "device", "device"),
ConnectionField("output_target", "color_strip_source_id", "color_strip_source", "colorstrip"),
ConnectionField(
"output_target", "brightness.source_id", "value_source", "value", bindable=True, nested=True
),
ConnectionField(
"output_target", "transition.source_id", "value_source", "value", bindable=True, nested=True
),
ConnectionField(
"output_target", "settings.pattern_template_id", "pattern_template", "template", nested=True
),
ConnectionField(
"output_target",
"settings.brightness.source_id",
"value_source",
"value",
bindable=True,
nested=True,
),
# ── Scene presets ──
ConnectionField("scene_preset", "targets[].target_id", "output_target", "scene", nested=True),
# ── Automations ──
ConnectionField("automation", "scene_preset_id", "scene_preset", "scene"),
ConnectionField("automation", "deactivation_scene_preset_id", "scene_preset", "scene"),
# ── Devices ──
ConnectionField("device", "default_css_processing_template_id", "cspt", "template"),
)
def schema_for_kind(kind: str) -> list[ConnectionField]:
"""Every connectable field whose *referrer* is ``kind``."""
return [c for c in CONNECTION_SCHEMA if c.target_kind == kind]
# BindableColor slots are structurally bindable but NOT graph-editable: a
# ValueStream yields a scalar (``get_value() -> float``) and colour consumers
# read the static RGB via ``bcolor()`` (source_id ignored at runtime), so a
# value source cannot drive a colour.
_COLOR_BINDABLE_FIELDS: frozenset[str] = frozenset(
{
"color.source_id",
"color_peak.source_id",
"fallback_color.source_id",
"default_color.source_id",
}
)
def is_editable(cf: ConnectionField) -> bool:
"""Whether a field can be wired from the graph.
Editable = a top-level reference, or a single-level ``BindableFloat`` slot.
List slots (need an element index), double-nested fields, and the dead
colour bindings stay read-only.
"""
if cf.is_list:
return False
if not cf.nested:
return True
return cf.bindable and cf.field.count(".") == 1 and cf.field not in _COLOR_BINDABLE_FIELDS
def schema_as_dicts() -> list[dict[str, Any]]:
"""Serialize the registry for the ``/graph/schema`` endpoint."""
return [
{
"target_kind": c.target_kind,
"field": c.field,
"source_kind": c.source_kind,
"edge_type": c.edge_type,
"bindable": c.bindable,
"nested": c.nested,
"is_list": c.is_list,
"editable": is_editable(c),
}
for c in CONNECTION_SCHEMA
]
# ── Reference extraction ────────────────────────────────────────────────────
def extract_refs(entity: dict[str, Any], field_path: str) -> list[str]:
"""Resolve a (possibly nested/list) ``field_path`` to its referenced ids.
Returns only non-empty string ids. Tolerant of missing keys, ``None``
values and unbound bindables (a plain number where an object was expected).
"""
current: list[Any] = [entity]
for segment in field_path.split("."):
is_list = segment.endswith("[]")
key = segment[:-2] if is_list else segment
nxt: list[Any] = []
for obj in current:
if not isinstance(obj, dict):
continue
val = obj.get(key)
if is_list:
if isinstance(val, list):
nxt.extend(val)
elif val is not None:
nxt.append(val)
current = nxt
return [v for v in current if isinstance(v, str) and v]
def remap_refs(entity: dict[str, Any], field_path: str, id_map: dict[str, str]) -> int:
"""Rewrite referenced ids under ``field_path`` *in place*, using ``id_map``.
The write-twin of :func:`extract_refs`: it walks the same dot/list/bindable
grammar and replaces any leaf id present in ``id_map`` with its mapped value.
Ids absent from ``id_map`` (references to entities outside the remap set) are
left untouched, so a clone keeps sharing its un-cloned dependencies. Unbound
bindables (a plain number where an object was expected) and missing keys are
tolerated. Returns the number of ids rewritten.
"""
segments = field_path.split(".")
# Descend to the container(s) that hold the final key.
parents: list[Any] = [entity]
for segment in segments[:-1]:
is_list = segment.endswith("[]")
key = segment[:-2] if is_list else segment
nxt: list[Any] = []
for obj in parents:
if not isinstance(obj, dict):
continue
val = obj.get(key)
if is_list:
if isinstance(val, list):
nxt.extend(val)
elif isinstance(val, dict):
nxt.append(val)
parents = nxt
last = segments[-1]
last_is_list = last.endswith("[]")
key = last[:-2] if last_is_list else last
count = 0
for obj in parents:
if not isinstance(obj, dict):
continue
val = obj.get(key)
if last_is_list:
if isinstance(val, list):
for i, item in enumerate(val):
if isinstance(item, str) and item in id_map:
val[i] = id_map[item]
count += 1
elif isinstance(val, str) and val in id_map:
obj[key] = id_map[val]
count += 1
return count
def serialize_entity(model: Any) -> dict[str, Any]:
"""Best-effort serialize a storage model to a plain dict for graph use.
Prefers ``dataclasses.asdict`` (pure structural, recurses bindables/lists,
invokes no managers), falling back to ``to_dict()`` then ``{}``.
"""
if is_dataclass(model) and not isinstance(model, type):
try:
return asdict(model)
except Exception as exc: # noqa: BLE001 — defensive: never let one model break the graph
logger.debug("graph: asdict failed for %r: %s", type(model).__name__, exc)
to_dict = getattr(model, "to_dict", None)
if callable(to_dict):
try:
result = to_dict()
if isinstance(result, dict):
return result
except Exception as exc: # noqa: BLE001
logger.debug("graph: to_dict failed for %r: %s", type(model).__name__, exc)
logger.warning(
"graph: could not serialize model %r; excluding from graph", type(model).__name__
)
return {}
def graph_field_roots(kind: str) -> set[str]:
"""Top-level keys the graph needs for ``kind``: ``id``/``name``, the subtype
field, and the root segment of every reference path for that kind."""
roots: set[str] = {"id", "name"}
type_field = NODE_TYPE_FIELD.get(kind, "")
if type_field:
roots.add(type_field)
for cf in CONNECTION_SCHEMA:
if cf.target_kind == kind:
roots.add(cf.field.split(".", 1)[0].removesuffix("[]"))
return roots
def serialize_entity_for_graph(kind: str, model: Any) -> dict[str, Any]:
"""Serialize a model and project it to ONLY the keys the graph needs.
This projection is a **security boundary**: a full ``asdict``/``to_dict``
can carry secrets (webhook tokens, device/HA/MQTT credentials), so every
field except ``id``/``name``, the subtype field and reference-path roots is
dropped before the data reaches the graph API.
"""
full = serialize_entity(model)
roots = graph_field_roots(kind)
return {k: v for k, v in full.items() if k in roots}
# ── Topology / validation ───────────────────────────────────────────────────
def _node_from(kind: str, entity: dict[str, Any]) -> dict[str, Any] | None:
eid = entity.get("id")
if not isinstance(eid, str) or not eid:
return None
type_field = NODE_TYPE_FIELD.get(kind, "")
subtype = entity.get(type_field, "") if type_field else ""
return {
"id": eid,
"kind": kind,
"name": entity.get("name") or eid,
"type": subtype if isinstance(subtype, str) else "",
}
def build_topology(entities_by_kind: dict[str, list[dict[str, Any]]]) -> dict[str, Any]:
"""Build the full wiring graph + a validation report.
Args:
entities_by_kind: ``{kind: [serialized_entity_dict, ...]}``.
Returns a dict with ``nodes``, ``edges`` and ``issues`` (``orphans``,
``broken_refs``, ``cycles``).
"""
nodes: list[dict[str, Any]] = []
node_ids: set[str] = set()
for kind in ENTITY_KINDS:
for entity in entities_by_kind.get(kind, []):
node = _node_from(kind, entity)
if node and node["id"] not in node_ids:
node_ids.add(node["id"])
nodes.append(node)
edges: list[dict[str, Any]] = []
broken_refs: list[dict[str, str]] = []
for cf in CONNECTION_SCHEMA:
for entity in entities_by_kind.get(cf.target_kind, []):
referrer = entity.get("id")
if not isinstance(referrer, str) or not referrer:
continue
for ref in extract_refs(entity, cf.field):
if ref not in node_ids:
broken_refs.append({"ref": ref, "by": referrer, "field": cf.field})
continue
edges.append(
{
"from": ref,
"to": referrer,
"field": cf.field,
"edge_type": cf.edge_type,
"nested": cf.nested,
}
)
connected: set[str] = set()
for e in edges:
connected.add(e["from"])
connected.add(e["to"])
orphans = sorted(nid for nid in node_ids if nid not in connected)
cycles = sorted(detect_cycles(edges))
return {
"nodes": nodes,
"edges": edges,
"issues": {
"orphans": orphans,
"broken_refs": broken_refs,
"cycles": cycles,
},
}
def find_dependents(
entities_by_kind: dict[str, list[dict[str, Any]]], kind: str, entity_id: str
) -> list[dict[str, str]]:
"""Return every entity that references ``(kind, entity_id)``.
``kind`` is the kind of the *referenced* entity; matching schema entries are
those whose ``source_kind == kind``.
"""
name_by_id: dict[str, str] = {}
for k in ENTITY_KINDS:
for entity in entities_by_kind.get(k, []):
eid = entity.get("id")
if isinstance(eid, str):
name_by_id[eid] = entity.get("name") or eid
dependents: list[dict[str, str]] = []
seen: set[tuple[str, str]] = set()
for cf in CONNECTION_SCHEMA:
if cf.source_kind != kind:
continue
for entity in entities_by_kind.get(cf.target_kind, []):
referrer = entity.get("id")
if not isinstance(referrer, str):
continue
if entity_id in extract_refs(entity, cf.field):
key = (referrer, cf.field)
if key in seen:
continue
seen.add(key)
dependents.append(
{
"id": referrer,
"kind": cf.target_kind,
"name": name_by_id.get(referrer, referrer),
"field": cf.field,
}
)
return dependents
def detect_cycles(edges: list[dict[str, Any]]) -> set[str]:
"""Return every node id that participates in a directed cycle (from→to)."""
adj: dict[str, list[str]] = {}
for e in edges:
adj.setdefault(e["from"], []).append(e["to"])
WHITE, GRAY, BLACK = 0, 1, 2
color: dict[str, int] = {}
in_cycle: set[str] = set()
for start in list(adj.keys()):
if color.get(start, WHITE) != WHITE:
continue
stack: list[tuple[str, int]] = [(start, 0)]
path: list[str] = [start]
color[start] = GRAY
while stack:
node, idx = stack[-1]
neighbors = adj.get(node, [])
if idx < len(neighbors):
stack[-1] = (node, idx + 1)
nxt = neighbors[idx]
c = color.get(nxt, WHITE)
if c == GRAY:
if nxt in path:
i = path.index(nxt)
in_cycle.update(path[i:])
elif c == WHITE:
color[nxt] = GRAY
path.append(nxt)
stack.append((nxt, 0))
else:
color[node] = BLACK
if path and path[-1] == node:
path.pop()
stack.pop()
return in_cycle
def _reachable(edges: list[dict[str, Any]], start: str, goal: str) -> bool:
"""True if ``goal`` is reachable from ``start`` following from→to edges."""
if start == goal:
return True
adj: dict[str, list[str]] = {}
for e in edges:
adj.setdefault(e["from"], []).append(e["to"])
seen = {start}
queue = [start]
while queue:
cur = queue.pop()
for nxt in adj.get(cur, []):
if nxt == goal:
return True
if nxt not in seen:
seen.add(nxt)
queue.append(nxt)
return False
def would_create_cycle(edges: list[dict[str, Any]], source_id: str, target_id: str) -> bool:
"""Would wiring ``source_id`` into ``target_id`` (edge source→target) loop?
A cycle forms if ``source_id`` is already reachable from ``target_id`` via
the existing data-flow edges (so the new edge would close the loop), or the
two are the same node.
"""
if source_id == target_id:
return True
return _reachable(edges, target_id, source_id)
def _entity_exists(
entities_by_kind: dict[str, list[dict[str, Any]]], kind: str, entity_id: str
) -> bool:
return any(e.get("id") == entity_id for e in entities_by_kind.get(kind, []))
def validate_connection(
entities_by_kind: dict[str, list[dict[str, Any]]],
target_kind: str,
target_id: str,
field: str,
source_id: str,
) -> tuple[bool, str | None]:
"""Validate a proposed wiring edit before it is persisted.
Checks, in order: the field is a known connectable reference; the target
exists; (when not detaching) the source exists and is of the registry's
expected kind; and the edit would not create a dependency cycle. Returns
``(ok, error_message)``. Detaching (empty ``source_id``) is always allowed.
"""
cf = next(
(c for c in CONNECTION_SCHEMA if c.target_kind == target_kind and c.field == field),
None,
)
if cf is None:
return False, f"Unknown connection field: {target_kind}.{field}"
if not is_editable(cf):
# List slots (need an element index), double-nested fields, and dead
# colour bindings can't be wired from the graph — edit via the entity
# editor instead.
return False, f"Field '{field}' is not editable via the graph"
if not _entity_exists(entities_by_kind, target_kind, target_id):
return False, f"Target entity not found: {target_id}"
if not source_id:
return True, None # detaching a slot is always valid
if not _entity_exists(entities_by_kind, cf.source_kind, source_id):
return False, f"Source {cf.source_kind} not found: {source_id}"
# Cycle check: ignore the edge currently occupying this slot, since the
# write replaces it.
topo = build_topology(entities_by_kind)
edges = [e for e in topo["edges"] if not (e["to"] == target_id and e["field"] == field)]
if would_create_cycle(edges, source_id, target_id):
return False, "Connection would create a dependency cycle"
return True, None
@@ -0,0 +1,25 @@
"""Shared MQTT-source validation for route handlers.
Both the device routes and the output-target routes accept an
``mqtt_source_id`` that must reference an existing ``MQTTSource``. This module
is the single source of truth for that check so the two callers cannot drift.
"""
from fastapi import HTTPException
from ledgrab.storage.base_store import EntityNotFoundError
from ledgrab.storage.mqtt_source_store import MQTTSourceStore
def validate_mqtt_source_exists(mqtt_store: MQTTSourceStore, mqtt_source_id: str | None) -> None:
"""Ensure a referenced MQTT source exists.
Empty / ``None`` is allowed (unconfigured = "first available broker").
Raises ``HTTPException(422)`` if a non-empty id does not resolve.
"""
if not mqtt_source_id:
return
try:
mqtt_store.get(mqtt_source_id)
except (ValueError, EntityNotFoundError):
raise HTTPException(status_code=422, detail=f"MQTT source {mqtt_source_id} not found")
+41
View File
@@ -13,6 +13,7 @@ from ledgrab.core.devices.led_client import (
from ledgrab.api.dependencies import (
fire_entity_event,
get_device_store,
get_mqtt_store,
get_output_target_store,
get_processor_manager,
)
@@ -33,10 +34,13 @@ from ledgrab.api.schemas.devices import (
)
from ledgrab.core.processing.processor_manager import ProcessorManager
from ledgrab.storage import DeviceStore
from ledgrab.storage.mqtt_source_store import MQTTSourceStore
from ledgrab.storage.output_target_store import OutputTargetStore
from ledgrab.utils import get_logger
from ledgrab.utils.url_scheme import infer_http_scheme
from ._mqtt_validation import validate_mqtt_source_exists
logger = get_logger(__name__)
router = APIRouter()
@@ -105,6 +109,7 @@ def _device_to_response(device) -> DeviceResponse:
gamesense_device_type=device.gamesense_device_type,
ble_family=device.ble_family,
ble_govee_key=device.ble_govee_key,
mqtt_source_id=getattr(device, "mqtt_source_id", "") or "",
default_css_processing_template_id=device.default_css_processing_template_id,
group_device_ids=device.group_device_ids,
group_mode=device.group_mode,
@@ -124,11 +129,13 @@ async def create_device(
_auth: AuthRequired,
store: DeviceStore = Depends(get_device_store),
manager: ProcessorManager = Depends(get_processor_manager),
mqtt_store: MQTTSourceStore = Depends(get_mqtt_store),
):
"""Create and attach a new LED device."""
try:
device_type = device_data.device_type
logger.info(f"Creating {device_type} device: {device_data.name}")
validate_mqtt_source_exists(mqtt_store, device_data.mqtt_source_id)
# ── Group device: validate children + compute LED count ──
if device_type == "group":
@@ -287,6 +294,7 @@ async def create_device(
gamesense_device_type=device_data.gamesense_device_type or "keyboard",
ble_family=device_data.ble_family or "",
ble_govee_key=device_data.ble_govee_key or "",
mqtt_source_id=device_data.mqtt_source_id or "",
group_device_ids=group_device_ids,
group_mode=group_mode,
)
@@ -543,12 +551,14 @@ async def update_device(
_auth: AuthRequired,
store: DeviceStore = Depends(get_device_store),
manager: ProcessorManager = Depends(get_processor_manager),
mqtt_store: MQTTSourceStore = Depends(get_mqtt_store),
):
"""Update device information."""
try:
# Group-specific validation before applying update
existing = store.get_device(device_id)
is_group = existing.device_type == "group"
validate_mqtt_source_exists(mqtt_store, update_data.mqtt_source_id)
# Normalize URL the same way we do on create:
# * always rstrip trailing slashes (so PUT-with-trailing-/ matches
@@ -634,6 +644,7 @@ async def update_device(
gamesense_device_type=update_data.gamesense_device_type,
ble_family=update_data.ble_family,
ble_govee_key=update_data.ble_govee_key,
mqtt_source_id=update_data.mqtt_source_id,
group_device_ids=update_data.group_device_ids,
group_mode=update_data.group_mode,
icon=update_data.icon,
@@ -669,6 +680,10 @@ async def update_device(
fire_entity_event("device", "updated", device_id)
return _device_to_response(device)
except HTTPException:
# Intentional 4xx (e.g. unknown mqtt_source_id, group validation)
# must propagate unchanged — not be masked as a 500.
raise
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
except Exception as e:
@@ -777,6 +792,32 @@ async def ping_device(
# ===== WLED BRIGHTNESS ENDPOINTS =====
async def resolve_device_brightness(device, manager: ProcessorManager) -> int | None:
"""Resolve a device's current brightness for aggregate/batch reads.
Mirrors GET /brightness but degrades to ``None`` instead of raising, so one
unreachable device can't fail a whole snapshot. Reads the server-side cache
first and only touches hardware when the cache is cold, then populates it so
subsequent reads are I/O-free.
"""
if "brightness_control" not in get_device_capabilities(device.device_type):
return None
ds = manager.find_device_state(device.id)
if ds and ds.hardware_brightness is not None:
return ds.hardware_brightness
try:
provider = get_provider(device.device_type)
bri = await provider.get_brightness(device.url)
if ds:
ds.hardware_brightness = bri
return bri
except NotImplementedError:
return device.software_brightness
except Exception as e:
logger.warning("Failed to resolve brightness for device %s: %s", device.id, e)
return None
@router.get("/api/v1/devices/{device_id}/brightness", tags=["Settings"])
async def get_device_brightness(
device_id: str,
+249
View File
@@ -0,0 +1,249 @@
"""Wiring-graph endpoints: schema registry, full topology, and dependents.
These power the visual graph editor (and any other client) with a single
authoritative view of how entities are wired together:
* ``GET /api/v1/graph/schema`` — the connectable-field registry.
* ``GET /api/v1/graph`` — nodes + edges + validation.
* ``GET /api/v1/graph/dependents/{kind}/{id}`` — what references an entity.
All heavy logic lives in :mod:`ledgrab.api.graph_schema` (pure, unit-tested);
this layer only gathers serialized entities from the stores and delegates.
"""
from __future__ import annotations
import logging
from typing import Any, Callable
from fastapi import APIRouter, HTTPException
from fastapi.concurrency import run_in_threadpool
from pydantic import BaseModel, Field
from ledgrab.api import dependencies as deps
from ledgrab.api.auth import AuthRequired
from ledgrab.api.graph_schema import (
ENTITY_KINDS,
NODE_TYPE_FIELD,
build_topology,
extract_refs,
find_dependents,
remap_refs,
schema_as_dicts,
schema_for_kind,
serialize_entity,
serialize_entity_for_graph,
validate_connection,
)
logger = logging.getLogger(__name__)
class ConnectionValidationRequest(BaseModel):
"""A proposed wiring edit: set ``target_kind.field`` to ``source_id``."""
target_kind: str
target_id: str
field: str
source_id: str = Field(default="", description="Empty string detaches the slot.")
router = APIRouter()
# kind → dependency getter for the store that owns that entity kind.
_KIND_STORES: dict[str, Callable[[], Any]] = {
"device": deps.get_device_store,
"capture_template": deps.get_template_store,
"pp_template": deps.get_pp_template_store,
"audio_template": deps.get_audio_template_store,
"pattern_template": deps.get_pattern_template_store,
"picture_source": deps.get_picture_source_store,
"audio_source": deps.get_audio_source_store,
"value_source": deps.get_value_source_store,
"color_strip_source": deps.get_color_strip_store,
"sync_clock": deps.get_sync_clock_store,
"output_target": deps.get_output_target_store,
"scene_preset": deps.get_scene_preset_store,
"automation": deps.get_automation_store,
"cspt": deps.get_cspt_store,
}
def _gather_entities() -> dict[str, list[dict[str, Any]]]:
"""Serialize every entity, keyed by kind. Missing stores yield ``[]``."""
out: dict[str, list[dict[str, Any]]] = {}
for kind, getter in _KIND_STORES.items():
try:
store = getter()
models = store.get_all()
except (
Exception
) as exc: # noqa: BLE001 — an uninitialized/failing store must not 500 the graph
logger.warning("graph: store for kind %s unavailable: %s", kind, exc)
out[kind] = []
continue
out[kind] = [serialize_entity_for_graph(kind, m) for m in models]
return out
@router.get("/api/v1/graph/schema", tags=["Graph"])
async def get_graph_schema(_auth: AuthRequired) -> dict[str, Any]:
"""Return the authoritative registry of connectable reference fields."""
return {
"kinds": list(ENTITY_KINDS),
"node_type_field": NODE_TYPE_FIELD,
"connections": schema_as_dicts(),
}
@router.get("/api/v1/graph", tags=["Graph"])
async def get_graph(_auth: AuthRequired) -> dict[str, Any]:
"""Return the full wiring topology (nodes + edges) and a validation report."""
entities = await run_in_threadpool(_gather_entities)
return build_topology(entities)
@router.get("/api/v1/graph/dependents/{kind}/{entity_id}", tags=["Graph"])
async def get_graph_dependents(kind: str, entity_id: str, _auth: AuthRequired) -> dict[str, Any]:
"""Return every entity that references ``(kind, entity_id)``."""
if kind not in ENTITY_KINDS:
raise HTTPException(status_code=404, detail=f"Unknown entity kind: {kind}")
entities = await run_in_threadpool(_gather_entities)
return {"dependents": find_dependents(entities, kind, entity_id)}
@router.post("/api/v1/graph/validate-connection", tags=["Graph"])
async def validate_graph_connection(
body: ConnectionValidationRequest, _auth: AuthRequired
) -> dict[str, Any]:
"""Validate a proposed wiring edit (existence + source kind + no cycle).
The graph editor calls this before persisting a drag-connect so it can
refuse edits that would dangle a reference or create a dependency loop.
"""
entities = await run_in_threadpool(_gather_entities)
ok, error = validate_connection(
entities, body.target_kind, body.target_id, body.field, body.source_id
)
return {"ok": ok, "error": error}
# ── Subgraph duplication (server-side blueprint instantiate) ─────────────────
# Only these kinds are cloned. They carry no inline secrets — they *reference*
# shared secret-bearing entities (devices, HA sources, HTTP endpoints) by id,
# and those are NOT cloned — and they have no hardware identity to conflict
# over. Output targets, automations, devices and integrations are out of scope.
_DUPLICABLE_KINDS: tuple[str, ...] = ("value_source", "color_strip_source")
_MAX_DUPLICATE = 200
class DuplicateRequest(BaseModel):
"""Duplicate a selected subgraph of value / colour-strip sources."""
node_ids: list[str] = Field(..., min_length=1, max_length=_MAX_DUPLICATE)
name_suffix: str = Field(default=" (copy)", max_length=40)
def _unique_name(existing: set[str], desired: str) -> str:
"""A name not already in ``existing`` (appends ' 2', ' 3', … on collision)."""
if desired not in existing:
return desired
i = 2
while f"{desired} {i}" in existing:
i += 1
return f"{desired} {i}"
def _duplicate_subgraph(node_ids: list[str], name_suffix: str) -> dict[str, Any]:
"""Deep-clone selected value/colour-strip sources with new ids, rewiring
references that point *within* the selection (shared deps are left alone)."""
# Index every duplicable entity by id → (kind, store, model); track names.
index: dict[str, tuple[str, Any, Any]] = {}
existing_names: dict[str, set[str]] = {}
for kind in _DUPLICABLE_KINDS:
try:
store = _KIND_STORES[kind]()
models = store.get_all()
except Exception as exc: # noqa: BLE001 — a failing store must not 500 the request
logger.warning("graph.duplicate: store for %s unavailable: %s", kind, exc)
continue
names = existing_names.setdefault(kind, set())
for m in models:
mid = getattr(m, "id", None)
mname = getattr(m, "name", None)
if isinstance(mname, str):
names.add(mname)
if isinstance(mid, str) and mid:
index[mid] = (kind, store, m)
selected: list[str] = []
skipped: list[dict[str, str]] = []
for nid in dict.fromkeys(node_ids): # de-dupe, preserve order
if nid in index:
selected.append(nid)
else:
skipped.append(
{"id": nid, "reason": "only value and colour-strip sources can be duplicated"}
)
# Pass 1 — create clones; their refs still point at the originals (valid).
id_map: dict[str, str] = {}
created: list[dict[str, str]] = []
clones: list[tuple[str, Any, str]] = []
for old_id in selected:
kind, store, model = index[old_id]
base = (getattr(model, "name", None) or old_id) + name_suffix
name = _unique_name(existing_names[kind], base)
existing_names[kind].add(name)
try:
new = store.clone(old_id, name)
except Exception as exc: # noqa: BLE001
logger.warning("graph.duplicate: clone of %s %s failed: %s", kind, old_id, exc)
skipped.append({"id": old_id, "reason": f"clone failed: {exc}"})
continue
id_map[old_id] = new.id
created.append({"id": new.id, "kind": kind, "name": new.name})
clones.append((kind, store, new.id))
# Pass 2 — rewrite references that point within the cloned set.
warnings: list[dict[str, str]] = []
for kind, store, new_id in clones:
clone = serialize_entity(store.get(new_id))
changed_roots: set[str] = set()
for cf in schema_for_kind(kind):
if remap_refs(clone, cf.field, id_map):
changed_roots.add(cf.field.split(".", 1)[0].removesuffix("[]"))
if not changed_roots:
continue
# `clone` is the FULL serialized entity, so each changed root carries a
# complete, structurally-intact value (the whole `layers` list / bindable
# dict) that ``update_source`` replaces or merges wholesale. (Within the
# duplicable set the only roots that change are scalar ids, `layers` and
# bindable slots — never a partially-built nested object.)
updates = {root: clone[root] for root in changed_roots if root in clone}
try:
store.update_source(new_id, **updates)
except Exception as exc: # noqa: BLE001
logger.warning("graph.duplicate: ref remap of %s failed: %s", new_id, exc)
warnings.append({"id": new_id, "reason": f"reference remap failed: {exc}"})
# Safety net — a clone must never still reference an OLD (in-selection) id.
for kind, store, new_id in clones:
clone = serialize_entity(store.get(new_id))
for cf in schema_for_kind(kind):
if any(ref in id_map for ref in extract_refs(clone, cf.field)):
warnings.append({"id": new_id, "reason": f"unremapped reference at {cf.field}"})
return {"id_map": id_map, "created": created, "skipped": skipped, "warnings": warnings}
@router.post("/api/v1/graph/duplicate", tags=["Graph"])
async def duplicate_subgraph(body: DuplicateRequest, _auth: AuthRequired) -> dict[str, Any]:
"""Deep-clone the selected value/colour-strip sources (new ids, wiring remapped).
References that point *within* the selection are rewired to the new clones;
references to entities outside it (devices, HA sources, …) stay shared with
the originals. Only value and colour-strip sources are cloned — they carry no
inline secrets — so any other kind in the selection is reported in ``skipped``.
"""
return await run_in_threadpool(_duplicate_subgraph, body.node_ids, body.name_suffix)
@@ -49,6 +49,8 @@ from ledgrab.storage.value_source_store import ValueSourceStore
from ledgrab.utils import get_logger
from ledgrab.storage.base_store import EntityNotFoundError
from ._mqtt_validation import validate_mqtt_source_exists
logger = get_logger(__name__)
router = APIRouter()
@@ -270,16 +272,6 @@ def _validate_device_exists(device_store: DeviceStore, device_id: str) -> None:
raise HTTPException(status_code=422, detail=f"Device {device_id} not found")
def _validate_mqtt_source_exists(mqtt_store: MQTTSourceStore, mqtt_source_id: str) -> None:
"""Ensure the referenced MQTT source exists. Empty id is allowed (unconfigured)."""
if not mqtt_source_id:
return
try:
mqtt_store.get(mqtt_source_id)
except (ValueError, EntityNotFoundError):
raise HTTPException(status_code=422, detail=f"MQTT source {mqtt_source_id} not found")
@router.post(
"/api/v1/output-targets", response_model=OutputTargetResponse, tags=["Targets"], status_code=201
)
@@ -333,7 +325,7 @@ async def create_target(
case Z2MLightOutputTargetCreate():
if data.source_kind == "color_vs":
_validate_color_value_source(value_source_store, data.color_value_source_id)
_validate_mqtt_source_exists(mqtt_store, data.mqtt_source_id)
validate_mqtt_source_exists(mqtt_store, data.mqtt_source_id)
target = target_store.create_z2m_light_target(
name=data.name,
description=data.description,
@@ -540,7 +532,7 @@ async def update_target(
)
_validate_color_value_source(value_source_store, effective_id)
if data.mqtt_source_id:
_validate_mqtt_source_exists(mqtt_store, data.mqtt_source_id)
validate_mqtt_source_exists(mqtt_store, data.mqtt_source_id)
target = target_store.update_z2m_light_target(
target_id,
name=data.name,
+201
View File
@@ -0,0 +1,201 @@
"""Aggregated snapshot endpoint for low-overhead polling clients.
Returns, in a single response, everything the Home Assistant integration's
coordinator needs per poll: all output targets with processing state + metrics,
all devices with brightness, the color-strip / value-source / scene-preset /
sync-clock lists, and the system block (performance, health, update).
This collapses the integration's previous ~2N+M request fan-out (per-target
``/state`` + ``/metrics`` and per-device ``/brightness``) into one round trip.
The handler delegates to the existing list/batch route handlers so the response
sub-shapes stay byte-identical to the individual endpoints — no shaping logic is
duplicated here.
Callers that don't need the whole payload can pass ``?include=`` with a
comma-separated subset of section names (the response keys). Omitting it returns
every section. Gating is per section, so an excluded section also skips its
server-side work — dropping ``device_brightness`` avoids cold-cache hardware
probes, and dropping ``system`` skips the (blocking) NVML performance query.
"""
import asyncio
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from fastapi.concurrency import run_in_threadpool
from ledgrab.api.auth import AuthRequired
from ledgrab.api.dependencies import (
get_color_strip_store,
get_device_store,
get_output_target_store,
get_processor_manager,
get_scene_preset_store,
get_sync_clock_manager,
get_sync_clock_store,
get_update_service,
get_value_source_store,
)
from ledgrab.api.schemas.update import UpdateStatusResponse
from ledgrab.utils import get_logger
from .color_strip_sources.crud import list_color_strip_sources
from .devices import list_devices, resolve_device_brightness
from .output_targets import batch_target_metrics, batch_target_states, list_targets
from .scene_presets import list_scene_presets
from .sync_clocks import list_sync_clocks
from .system import get_system_performance, health_check
from .update import get_update_status
from .value_sources import list_value_sources
logger = get_logger(__name__)
router = APIRouter()
# Selectable snapshot sections — these are exactly the response top-level keys.
SNAPSHOT_SECTIONS = (
"targets",
"target_states",
"target_metrics",
"devices",
"device_brightness",
"css_sources",
"value_sources",
"scene_presets",
"sync_clocks",
"system",
)
_SECTION_SET = frozenset(SNAPSHOT_SECTIONS)
def _resolve_sections(include: str | None) -> frozenset[str]:
"""Validate the ``include`` query param into the set of sections to emit.
``None``/empty → every section. Unknown names are rejected with 422 so a
typo fails loudly instead of silently returning a smaller payload.
"""
if not include:
return _SECTION_SET
requested = {part.strip() for part in include.split(",") if part.strip()}
unknown = requested - _SECTION_SET
if unknown:
raise HTTPException(
status_code=422,
detail=(
f"Unknown snapshot section(s): {', '.join(sorted(unknown))}. "
f"Valid sections: {', '.join(SNAPSHOT_SECTIONS)}."
),
)
return frozenset(requested)
async def _safe_section(awaitable, label: str):
"""Await a section, degrading to ``None`` on failure instead of 500-ing.
The snapshot is a resilience-oriented poll surface: one failing section
(e.g. NVML performance probing) must not fail the whole response. This
preserves the per-section fault isolation the HA coordinator relied on
before these calls were merged into one request — the coordinator already
tolerates a ``None`` section.
"""
try:
return await awaitable
except Exception:
logger.warning("snapshot: section %r failed, returning null", label, exc_info=True)
return None
async def _update_status_model(_auth, update_service) -> UpdateStatusResponse:
"""Fetch update status and coerce it through the response model.
The standalone ``/system/update/status`` endpoint declares
``response_model=UpdateStatusResponse``; coercing here keeps the snapshot's
``system.update`` field identical to that endpoint rather than emitting the
service's raw dict unfiltered.
"""
raw = await get_update_status(_auth, update_service)
return UpdateStatusResponse.model_validate(raw)
@router.get("/api/v1/snapshot", tags=["Snapshot"])
async def get_snapshot(
request: Request,
_auth: AuthRequired,
include: str | None = Query(
None,
description=(
"Comma-separated subset of sections to include. Omit for all. "
"Valid: " + ", ".join(SNAPSHOT_SECTIONS)
),
),
manager=Depends(get_processor_manager),
target_store=Depends(get_output_target_store),
device_store=Depends(get_device_store),
css_store=Depends(get_color_strip_store),
value_store=Depends(get_value_source_store),
preset_store=Depends(get_scene_preset_store),
clock_store=Depends(get_sync_clock_store),
clock_manager=Depends(get_sync_clock_manager),
update_service=Depends(get_update_service),
) -> dict[str, Any]:
"""Return the full poll payload (or a requested subset) in one response.
Shape (a key is present only when its section is requested)::
{
"targets": [<OutputTargetResponse>, ...],
"target_states": {target_id: <state>, ...},
"target_metrics": {target_id: <metrics>, ...},
"devices": [<DeviceResponse>, ...],
"device_brightness": {device_id: int | null, ...},
"css_sources": [...],
"value_sources": [...],
"scene_presets": [...],
"sync_clocks": [...],
"system": {"performance": {...}, "health": {...}, "update": {...}}
}
"""
sections = _resolve_sections(include)
result: dict[str, Any] = {}
if "targets" in sections:
result["targets"] = (await list_targets(_auth, target_store)).targets
if "target_states" in sections:
result["target_states"] = (await batch_target_states(_auth, manager))["states"]
if "target_metrics" in sections:
result["target_metrics"] = (await batch_target_metrics(_auth, manager))["metrics"]
if "devices" in sections:
result["devices"] = (await list_devices(_auth, device_store)).devices
if "device_brightness" in sections:
device_models = device_store.get_all_devices()
brightness_values = await asyncio.gather(
*(resolve_device_brightness(d, manager) for d in device_models),
return_exceptions=True,
)
result["device_brightness"] = {
model.id: (None if isinstance(value, BaseException) else value)
for model, value in zip(device_models, brightness_values)
}
if "css_sources" in sections:
css = await list_color_strip_sources(_auth, css_store, manager)
result["css_sources"] = css.sources
if "value_sources" in sections:
result["value_sources"] = (await list_value_sources(_auth, None, value_store)).sources
if "scene_presets" in sections:
result["scene_presets"] = (await list_scene_presets(_auth, preset_store)).presets
if "sync_clocks" in sections:
clocks = await list_sync_clocks(_auth, clock_store, clock_manager)
result["sync_clocks"] = clocks.clocks
if "system" in sections:
result["system"] = {
"performance": await _safe_section(
run_in_threadpool(get_system_performance, _auth), "system.performance"
),
"health": await _safe_section(health_check(request), "system.health"),
"update": await _safe_section(
_update_status_model(_auth, update_service), "system.update"
),
}
return result
+161 -4
View File
@@ -4,6 +4,7 @@ import asyncio
from typing import Annotated
from fastapi import APIRouter, Body, Depends, HTTPException, Query, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, Field
from ledgrab.api.auth import AuthRequired
from ledgrab.api.dependencies import (
@@ -27,6 +28,8 @@ from ledgrab.api.schemas.value_sources import (
StaticColorValueSourceResponse,
StaticValueSourceResponse,
SystemMetricsValueSourceResponse,
TemplateInput,
TemplateValueSourceResponse,
ValueSourceCreate,
ValueSourceListResponse,
ValueSourceResponse,
@@ -46,6 +49,7 @@ from ledgrab.storage.value_source import (
StaticColorValueSource,
StaticValueSource,
SystemMetricsValueSource,
TemplateValueSource,
ValueSource,
)
from ledgrab.storage.value_source_store import ValueSourceStore
@@ -170,6 +174,7 @@ _RESPONSE_MAP = {
min_ha_value=s.min_ha_value,
max_ha_value=s.max_ha_value,
smoothing=s.smoothing,
normalize=s.normalize,
),
GradientMapValueSource: lambda s: GradientMapValueSourceResponse(
id=s.id,
@@ -214,6 +219,7 @@ _RESPONSE_MAP = {
sensor_label=s.sensor_label,
poll_interval=s.poll_interval,
smoothing=s.smoothing,
normalize=s.normalize,
),
HTTPValueSource: lambda s: HTTPValueSourceResponse(
id=s.id,
@@ -230,6 +236,23 @@ _RESPONSE_MAP = {
min_value=s.min_value,
max_value=s.max_value,
smoothing=s.smoothing,
normalize=s.normalize,
),
TemplateValueSource: lambda s: TemplateValueSourceResponse(
id=s.id,
name=s.name,
description=s.description,
tags=s.tags,
icon=getattr(s, "icon", "") or "",
icon_color=getattr(s, "icon_color", "") or "",
created_at=s.created_at,
updated_at=s.updated_at,
template=s.template,
inputs=[
TemplateInput(name=i["name"], value_source_id=i["value_source_id"]) for i in s.inputs
],
default_value=s.default_value,
eval_interval=s.eval_interval,
),
}
@@ -395,6 +418,13 @@ async def delete_value_source(
if getattr(target, "brightness_value_source_id", "") == source_id:
raise ValueError(f"Cannot delete: referenced by target '{target.name}'")
# Check if any other value source (template / gradient_map) references it.
referencing = store.find_referencing_sources(source_id)
if referencing:
raise ValueError(
"Cannot delete: referenced by value source(s) " + ", ".join(referencing)
)
store.delete_source(source_id)
fire_entity_event("value_source", "deleted", source_id)
except EntityNotFoundError as e:
@@ -404,6 +434,121 @@ async def delete_value_source(
raise HTTPException(status_code=400, detail=str(e))
class ValidateTemplateRequest(BaseModel):
"""Request body for the advisory template-validation endpoint."""
template: str = Field(description="Jinja2 expression to validate", max_length=2000)
inputs: list[TemplateInput] = Field(default_factory=list, description="Named input bindings")
id: str | None = Field(None, description="Source id when editing (enables cycle detection)")
@router.post("/api/v1/value-sources/validate-template", tags=["Value Sources"])
async def validate_template_value_source(
payload: ValidateTemplateRequest,
_auth: AuthRequired,
store: ValueSourceStore = Depends(get_value_source_store),
):
"""Validate a template expression + inputs without persisting anything.
Advisory: always returns HTTP 200 with ``{valid, error, errors, warnings,
variables}``. Powers the live editor validator (which must run before a
source exists), reusing the exact factory/store validation so the client and
server can never disagree. ``errors`` are blocking (save disabled);
``warnings`` are non-blocking (e.g. unknown/unbound inputs — create is
lenient about those).
"""
from ledgrab.utils.template_expr import (
TemplateValidationError,
extract_variables,
validate_input_name,
validate_template_expression,
)
errors: list[str] = []
warnings: list[str] = []
# 1) Expression compiles and is safe (cost-guarded).
try:
validate_template_expression(payload.template)
except TemplateValidationError as e:
errors.append(str(e))
# 2) Input names valid / unique / non-reserved (blocking).
seen: set[str] = set()
for inp in payload.inputs:
try:
validate_input_name(inp.name)
except TemplateValidationError as e:
errors.append(str(e))
continue
if inp.name in seen:
errors.append(f"duplicate input name: {inp.name}")
seen.add(inp.name)
# 3) Referenced sources exist (non-blocking warning — create is lenient).
missing = [
inp.value_source_id
for inp in payload.inputs
if inp.value_source_id and not _source_exists(store, inp.value_source_id)
]
if missing:
warnings.append("unknown value source(s): " + ", ".join(sorted(set(missing))))
# 4) Variables referenced in the expression but not bound to an input
# (blocking): at runtime they raise UndefinedError, so the template would
# silently always return default_value. This is almost always a typo, so
# flag it as an error rather than letting "valid" mislead the user.
used = set(extract_variables(payload.template))
undeclared = used - seen
if undeclared:
errors.append("unbound variable(s): " + ", ".join(sorted(undeclared)))
# 5) Cycle check when editing an existing source (blocking).
if payload.id:
child_ids = [i.value_source_id for i in payload.inputs if i.value_source_id]
try:
store.validate_nesting(payload.id, child_ids)
except ValueError as e:
errors.append(str(e))
return {
"valid": not errors,
"error": errors[0] if errors else None,
"errors": errors,
"warnings": warnings,
"variables": extract_variables(payload.template),
}
def _source_exists(store: ValueSourceStore, source_id: str) -> bool:
try:
store.get_source(source_id)
return True
except Exception:
return False
# Per-stream (min, max) attribute pairs for the normalization range, so the
# preview can show where the raw value maps. Attribute names differ per stream
# type (historical), so probe each pair rather than assume one.
_RAW_RANGE_ATTRS: tuple[tuple[str, str], ...] = (
("_min_ha", "_max_ha"), # HAEntityValueStream
("_min_value", "_max_value"), # HTTPValueStream
("_min_val", "_max_val"), # SystemMetricsValueStream
("_min_game", "_max_game"), # GameEventValueStream
)
def _stream_raw_range(stream) -> list | None:
"""Return ``[min, max]`` for the stream's normalization range, or None."""
for lo_attr, hi_attr in _RAW_RANGE_ATTRS:
lo = getattr(stream, lo_attr, None)
hi = getattr(stream, hi_attr, None)
if isinstance(lo, (int, float)) and isinstance(hi, (int, float)):
return [lo, hi]
return None
# ===== REAL-TIME VALUE SOURCE TEST WEBSOCKET =====
@@ -467,10 +612,22 @@ async def test_value_source_ws(
msg["input_value"] = round(stream.get_input_value(), 4)
if hasattr(stream, "get_raw_value"):
raw = stream.get_raw_value()
if raw is not None:
msg["raw_value"] = round(raw, 4)
if hasattr(stream, "_min_ha"):
msg["raw_range"] = [stream._min_ha, stream._max_ha]
if isinstance(raw, bool):
# bool is a subclass of int — send as-is (don't coerce/round).
msg["raw_value"] = raw
elif isinstance(raw, (int, float)):
msg["raw_value"] = round(float(raw), 4)
elif raw is not None:
# Non-numeric raw (e.g. an HTTP string payload) — send verbatim
# rather than crash the socket on round().
msg["raw_value"] = raw
rng = _stream_raw_range(stream)
if rng is not None:
msg["raw_range"] = rng
# Tell the client whether this source is currently normalizing, so the
# preview can render the value as a fraction vs a clamped passthrough.
if hasattr(stream, "_normalize_enabled"):
msg["normalized"] = bool(stream._normalize_enabled)
await websocket.send_json(msg)
await asyncio.sleep(0.05)
except WebSocketDisconnect:
+11
View File
@@ -131,6 +131,11 @@ class DeviceCreate(BaseModel):
None,
description="Govee AES key (hex) — required for encrypted Govee firmware",
)
# MQTT (multi-broker) field
mqtt_source_id: str | None = Field(
None,
description="MQTT source (broker) ID for device_type=mqtt. Empty = first available broker.",
)
default_css_processing_template_id: str | None = Field(
None, description="Default color strip processing template ID"
)
@@ -217,6 +222,9 @@ class DeviceUpdate(BaseModel):
ble_govee_key: str | None = Field(
None, description="Govee AES key (hex) — required for encrypted Govee firmware"
)
mqtt_source_id: str | None = Field(
None, description="MQTT source (broker) ID for device_type=mqtt"
)
default_css_processing_template_id: str | None = Field(
None, description="Default color strip processing template ID"
)
@@ -436,6 +444,9 @@ class DeviceResponse(BaseModel):
ble_govee_key: str = Field(
default="", description="Govee AES key (hex) — required for encrypted Govee firmware"
)
mqtt_source_id: str = Field(
default="", description="MQTT source (broker) ID for device_type=mqtt"
)
default_css_processing_template_id: str = Field(
default="", description="Default color strip processing template ID"
)
@@ -10,6 +10,17 @@ from pydantic import BaseModel, Discriminator, Field, Tag
# =====================================================================
class TemplateInput(BaseModel):
"""A single ``{name -> value_source_id}`` binding for a template source."""
name: str = Field(
description="Variable name used in the expression (valid identifier)",
min_length=1,
max_length=64,
)
value_source_id: str = Field("", description="Bound value source ID (empty = unbound)")
class _ValueSourceResponseBase(BaseModel):
"""Shared fields for all value source responses."""
@@ -120,6 +131,9 @@ class HAEntityValueSourceResponse(_ValueSourceResponseBase):
min_ha_value: float = Field(description="Raw HA value mapped to output 0.0")
max_ha_value: float = Field(description="Raw HA value mapped to output 1.0")
smoothing: float = Field(description="EMA smoothing factor (0.0-1.0)")
normalize: bool = Field(
description="Rescale raw value to [0,1] via min/max; false clamps the raw value as-is"
)
class GradientMapValueSourceResponse(_ValueSourceResponseBase):
@@ -149,6 +163,9 @@ class SystemMetricsValueSourceResponse(_ValueSourceResponseBase):
sensor_label: str = Field(description="Sensor label for cpu_temp/fan_speed")
poll_interval: float = Field(description="Seconds between reads")
smoothing: float = Field(description="EMA smoothing factor (0.0-1.0)")
normalize: bool = Field(
description="Rescale raw value to [0,1] via min/max; false clamps the raw value as-is"
)
class HTTPValueSourceResponse(_ValueSourceResponseBase):
@@ -160,6 +177,22 @@ class HTTPValueSourceResponse(_ValueSourceResponseBase):
min_value: float = Field(description="Raw value mapped to output 0.0")
max_value: float = Field(description="Raw value mapped to output 1.0")
smoothing: float = Field(description="EMA smoothing factor (0.0-1.0)")
normalize: bool = Field(
description="Rescale raw value to [0,1] via min/max; false clamps the raw value as-is"
)
class TemplateValueSourceResponse(_ValueSourceResponseBase):
source_type: Literal["template"] = "template"
return_type: Literal["float"] = "float"
template: str = Field(description="Jinja2 expression")
inputs: List[TemplateInput] = Field(
default_factory=list, description="Named value-source bindings"
)
default_value: float = Field(description="Fallback when the expression errors (0.0-1.0)")
eval_interval: float | None = Field(
None, description="Re-eval throttle in seconds (None/0 = every poll)"
)
ValueSourceResponse = Annotated[
@@ -176,7 +209,8 @@ ValueSourceResponse = Annotated[
| Annotated[GradientMapValueSourceResponse, Tag("gradient_map")]
| Annotated[CSSExtractValueSourceResponse, Tag("css_extract")]
| Annotated[SystemMetricsValueSourceResponse, Tag("system_metrics")]
| Annotated[HTTPValueSourceResponse, Tag("http")],
| Annotated[HTTPValueSourceResponse, Tag("http")]
| Annotated[TemplateValueSourceResponse, Tag("template")],
Discriminator("source_type"),
]
@@ -292,6 +326,9 @@ class HAEntityValueSourceCreate(_ValueSourceCreateBase):
min_ha_value: float = Field(0.0, description="Raw HA value mapped to output 0.0")
max_ha_value: float = Field(100.0, description="Raw HA value mapped to output 1.0")
smoothing: float = Field(0.0, description="EMA smoothing (0.0-1.0)", ge=0.0, le=1.0)
normalize: bool = Field(
True, description="Rescale raw value to [0,1] via min/max; false clamps the raw value as-is"
)
class GradientMapValueSourceCreate(_ValueSourceCreateBase):
@@ -318,6 +355,9 @@ class SystemMetricsValueSourceCreate(_ValueSourceCreateBase):
sensor_label: str = Field("", description="Sensor label for cpu_temp/fan_speed")
poll_interval: float = Field(1.0, description="Poll interval in seconds", ge=0.1, le=60.0)
smoothing: float = Field(0.0, description="EMA smoothing (0.0-1.0)", ge=0.0, le=1.0)
normalize: bool = Field(
True, description="Rescale raw value to [0,1] via min/max; false clamps the raw value as-is"
)
class HTTPValueSourceCreate(_ValueSourceCreateBase):
@@ -328,6 +368,30 @@ class HTTPValueSourceCreate(_ValueSourceCreateBase):
min_value: float = Field(0.0, description="Raw value mapped to output 0.0")
max_value: float = Field(100.0, description="Raw value mapped to output 1.0")
smoothing: float = Field(0.0, description="EMA smoothing (0.0-1.0)", ge=0.0, le=1.0)
normalize: bool = Field(
True, description="Rescale raw value to [0,1] via min/max; false clamps the raw value as-is"
)
class TemplateValueSourceCreate(_ValueSourceCreateBase):
source_type: Literal["template"] = "template"
template: str = Field(
description=(
"Jinja2 expression (no statements/blocks). Inputs are exposed by name and via "
"raw[name]; globals: min, max, abs, round, clamp(x, lo=0, hi=1)."
),
min_length=1,
max_length=2000,
)
inputs: List[TemplateInput] = Field(
default_factory=list, description="Named value-source bindings"
)
default_value: float = Field(
0.0, description="Fallback when the expression errors (0.0-1.0)", ge=0.0, le=1.0
)
eval_interval: float | None = Field(
None, description="Re-eval throttle in seconds (None/0 = every poll)", ge=0.0
)
ValueSourceCreate = Annotated[
@@ -344,7 +408,8 @@ ValueSourceCreate = Annotated[
| Annotated[GradientMapValueSourceCreate, Tag("gradient_map")]
| Annotated[CSSExtractValueSourceCreate, Tag("css_extract")]
| Annotated[SystemMetricsValueSourceCreate, Tag("system_metrics")]
| Annotated[HTTPValueSourceCreate, Tag("http")],
| Annotated[HTTPValueSourceCreate, Tag("http")]
| Annotated[TemplateValueSourceCreate, Tag("template")],
Discriminator("source_type"),
]
@@ -452,6 +517,9 @@ class HAEntityValueSourceUpdate(_ValueSourceUpdateBase):
min_ha_value: float | None = Field(None, description="Min HA value")
max_ha_value: float | None = Field(None, description="Max HA value")
smoothing: float | None = Field(None, description="EMA smoothing", ge=0.0, le=1.0)
normalize: bool | None = Field(
None, description="Rescale raw via min/max (false = clamp as-is)"
)
class GradientMapValueSourceUpdate(_ValueSourceUpdateBase):
@@ -478,6 +546,9 @@ class SystemMetricsValueSourceUpdate(_ValueSourceUpdateBase):
sensor_label: str | None = Field(None, description="Sensor label")
poll_interval: float | None = Field(None, description="Poll interval", ge=0.1, le=60.0)
smoothing: float | None = Field(None, description="EMA smoothing", ge=0.0, le=1.0)
normalize: bool | None = Field(
None, description="Rescale raw via min/max (false = clamp as-is)"
)
class HTTPValueSourceUpdate(_ValueSourceUpdateBase):
@@ -488,6 +559,23 @@ class HTTPValueSourceUpdate(_ValueSourceUpdateBase):
min_value: float | None = Field(None, description="Raw value mapped to 0.0")
max_value: float | None = Field(None, description="Raw value mapped to 1.0")
smoothing: float | None = Field(None, description="EMA smoothing", ge=0.0, le=1.0)
normalize: bool | None = Field(
None, description="Rescale raw via min/max (false = clamp as-is)"
)
class TemplateValueSourceUpdate(_ValueSourceUpdateBase):
source_type: Literal["template"] = "template"
template: str | None = Field(
None, description="Jinja2 expression", min_length=1, max_length=2000
)
inputs: List[TemplateInput] | None = Field(None, description="Named value-source bindings")
default_value: float | None = Field(
None, description="Fallback when the expression errors (0.0-1.0)", ge=0.0, le=1.0
)
eval_interval: float | None = Field(
None, description="Re-eval throttle in seconds (0 = every poll)", ge=0.0
)
ValueSourceUpdate = Annotated[
@@ -504,7 +592,8 @@ ValueSourceUpdate = Annotated[
| Annotated[GradientMapValueSourceUpdate, Tag("gradient_map")]
| Annotated[CSSExtractValueSourceUpdate, Tag("css_extract")]
| Annotated[SystemMetricsValueSourceUpdate, Tag("system_metrics")]
| Annotated[HTTPValueSourceUpdate, Tag("http")],
| Annotated[HTTPValueSourceUpdate, Tag("http")]
| Annotated[TemplateValueSourceUpdate, Tag("template")],
Discriminator("source_type"),
]
+17
View File
@@ -38,6 +38,19 @@ try:
except ImportError:
_has_sounddevice = False
# Android playback-capture engine — pure Python (numpy only), but the
# guard keeps the registration pattern uniform and tolerant of any future
# import-time dependency.
try:
from ledgrab.core.audio.android_audio_engine import (
AndroidAudioEngine,
AndroidAudioCaptureStream,
)
_has_android_audio = True
except ImportError:
_has_android_audio = False
from ledgrab.core.audio.demo_engine import DemoAudioEngine, DemoAudioCaptureStream
# Auto-register available engines
@@ -45,6 +58,8 @@ if _has_wasapi:
AudioEngineRegistry.register(WasapiEngine)
if _has_sounddevice:
AudioEngineRegistry.register(SounddeviceEngine)
if _has_android_audio:
AudioEngineRegistry.register(AndroidAudioEngine)
AudioEngineRegistry.register(DemoAudioEngine)
__all__ = [
@@ -65,3 +80,5 @@ if _has_wasapi:
__all__ += ["WasapiEngine", "WasapiCaptureStream"]
if _has_sounddevice:
__all__ += ["SounddeviceEngine", "SounddeviceCaptureStream"]
if _has_android_audio:
__all__ += ["AndroidAudioEngine", "AndroidAudioCaptureStream"]
@@ -0,0 +1,229 @@
"""Android playback-capture audio engine.
Receives PCM pushed from Kotlin (via Chaquopy) through a module-level
sample queue. The Kotlin layer captures system playback audio with
``AudioRecord`` + ``AudioPlaybackCaptureConfiguration`` (reusing the
app's ``MediaProjection`` token) and calls :func:`push_samples` with
interleaved float32 PCM for each fixed-size block.
Mirrors the screen-capture bridge
(``core/capture_engines/mediaprojection_engine.py``): a module-level
queue plus ``configure`` / ``push_samples`` / ``shutdown`` filled by
Kotlin, consumed through the standard :class:`AudioCaptureStreamBase`
interface so :class:`~ledgrab.core.audio.audio_capture.ManagedAudioStream`
and :class:`~ledgrab.core.audio.analysis.AudioAnalyzer` work unchanged.
This engine is only available when running inside the LedGrab Android
app, which has set up the sample queue via :func:`configure`.
"""
import queue
from typing import Any, Dict, List
import numpy as np
from ledgrab.core.audio.base import (
AudioCaptureEngine,
AudioCaptureStreamBase,
AudioDeviceInfo,
)
from ledgrab.utils import get_logger
from ledgrab.utils.platform import is_android
logger = get_logger(__name__)
# ---------------------------------------------------------------------------
# Sample queue — the bridge between Kotlin and Python
# ---------------------------------------------------------------------------
_pcm_queue: "queue.Queue[np.ndarray]" = queue.Queue(maxsize=8)
_sample_rate = 48000
_channels = 2
_chunk_size = 1024
_active = False
_frames_received = 0
def configure(sample_rate: int, channels: int, chunk_size: int) -> None:
"""Set the stream format. Called from Kotlin before frames flow.
Drains any stale PCM from a previous capture session so the first
chunk after a restart is actually current. ``channels`` /
``sample_rate`` should be the values the Kotlin ``AudioRecord``
actually negotiated (which can differ from the requested values,
e.g. a stereo request that falls back to mono) — the analyzer keys
off these, so they must match the interleaving of pushed samples.
"""
global _sample_rate, _channels, _chunk_size, _active, _frames_received
while not _pcm_queue.empty():
try:
_pcm_queue.get_nowait()
except queue.Empty:
break
_sample_rate = sample_rate
_channels = max(1, channels)
_chunk_size = max(1, chunk_size)
_frames_received = 0
_active = True
logger.info(
"Android audio engine configured: sr=%d channels=%d chunk=%d",
_sample_rate,
_channels,
_chunk_size,
)
def push_samples(pcm_float32: bytes) -> None:
"""Push one interleaved float32 PCM block from Kotlin.
The byte buffer is interpreted as native-endian float32 (Kotlin
packs little-endian; all Android ABIs are little-endian). Drops the
oldest queued block if the consumer is slow (non-blocking).
Defensive framing: the downstream :class:`AudioAnalyzer` reshapes to
``(-1, channels)`` and copies into ``chunk_size``-sized scratch
buffers, so it raises on a block whose length is not a whole number
of frames or that exceeds ``chunk_size`` frames. We trim to a whole
multiple of ``_channels`` and clamp to ``_chunk_size`` frames so a
malformed push can never crash the capture thread.
"""
global _frames_received
# np.frombuffer raises if the length isn't a whole number of float32s.
# Kotlin always pushes complete blocks, but guard so a malformed buffer is
# dropped here rather than surfacing as an exception across the JNI bridge.
if len(pcm_float32) % 4 != 0:
return
samples = np.frombuffer(pcm_float32, dtype=np.float32)
# Trim to whole frames, then clamp to chunk_size frames.
frames = len(samples) // _channels
if frames <= 0:
return
frames = min(frames, _chunk_size)
usable = frames * _channels
# Copy out of the read-only frombuffer view so the queued block owns its
# memory. This lets the Kotlin side push from a reusable buffer (low GC on
# low-end TV boxes) without the not-yet-consumed queued block aliasing
# bytes Kotlin is about to overwrite. Mirrors mediaprojection_engine's
# push_frame .copy().
block = samples[:usable].copy()
_frames_received += 1
if _frames_received == 1 or _frames_received % 100 == 0:
logger.info("Android audio: received %d blocks", _frames_received)
try:
_pcm_queue.put_nowait(block)
except queue.Full:
try:
_pcm_queue.get_nowait()
except queue.Empty:
pass
try:
_pcm_queue.put_nowait(block)
except queue.Full:
pass
def shutdown() -> None:
"""Deactivate the engine. Called when the Android app stops audio."""
global _active
_active = False
logger.info("Android audio engine shut down")
# ---------------------------------------------------------------------------
# CaptureStream
# ---------------------------------------------------------------------------
class AndroidAudioCaptureStream(AudioCaptureStreamBase):
"""Reads PCM blocks pushed by Kotlin from the module-level queue."""
@property
def channels(self) -> int:
return _channels
@property
def sample_rate(self) -> int:
return _sample_rate
@property
def chunk_size(self) -> int:
return _chunk_size
def initialize(self) -> None:
if self._initialized:
return
if not _active:
raise RuntimeError(
"Android audio engine not configured. "
"This engine is only available inside the Android app."
)
self._initialized = True
logger.info("Android audio capture stream initialized")
def cleanup(self) -> None:
self._initialized = False
logger.info("Android audio capture stream cleaned up")
def read_chunk(self) -> np.ndarray | None:
try:
return _pcm_queue.get(timeout=0.1) # 1-D float32 interleaved
except queue.Empty:
return None
# ---------------------------------------------------------------------------
# CaptureEngine
# ---------------------------------------------------------------------------
class AndroidAudioEngine(AudioCaptureEngine):
"""Android playback-capture audio engine.
Only available when running inside the LedGrab Android app, which
calls :func:`configure` once audio capture is set up. Exposes a
single loopback "device" representing the system audio mix.
"""
ENGINE_TYPE = "android_playback"
ENGINE_PRIORITY = 100 # highest on a real Android device (demo only wins in demo mode)
@classmethod
def is_available(cls) -> bool:
return is_android() and _active
@classmethod
def get_default_config(cls) -> Dict[str, Any]:
return {
"sample_rate": _sample_rate,
"channels": _channels,
"chunk_size": _chunk_size,
}
@classmethod
def enumerate_devices(cls) -> List[AudioDeviceInfo]:
if not cls.is_available():
return []
return [
AudioDeviceInfo(
index=0,
name="Android playback (system audio)",
is_input=True,
is_loopback=True,
channels=_channels,
default_samplerate=float(_sample_rate),
)
]
@classmethod
def create_stream(
cls,
device_index: int,
is_loopback: bool,
config: Dict[str, Any],
) -> AndroidAudioCaptureStream:
merged = {**cls.get_default_config(), **config}
return AndroidAudioCaptureStream(device_index, is_loopback, merged)
+40
View File
@@ -40,6 +40,11 @@ _AS_IDS = {
"system": "as_demo0001",
}
_VS_IDS = {
"level": "vs_demo0001",
"boost": "vs_demo0002",
}
_TPL_ID = "tpl_demo0001"
_SCENE_ID = "scene_demo0001"
@@ -86,6 +91,7 @@ def seed_demo_data(db: Database) -> None:
_insert_entities(db, "picture_sources", _build_picture_sources())
_insert_entities(db, "color_strip_sources", _build_color_strip_sources())
_insert_entities(db, "audio_sources", _build_audio_sources())
_insert_entities(db, "value_sources", _build_value_sources())
_insert_entities(db, "scene_presets", _build_scene_presets())
logger.info("Demo seed data complete")
@@ -334,6 +340,40 @@ def _build_audio_sources() -> dict:
}
# ── Value Sources ──────────────────────────────────────────────────
def _build_value_sources() -> dict:
"""A static float source plus a template combinator that references it,
so demo mode showcases the Jinja template value source out of the box."""
return {
_VS_IDS["level"]: {
"id": _VS_IDS["level"],
"name": "Base Level",
"source_type": "static",
"description": "A constant brightness level (demo input for the template below)",
"tags": ["demo"],
"value": 0.5,
"created_at": _NOW,
"updated_at": _NOW,
},
_VS_IDS["boost"]: {
"id": _VS_IDS["boost"],
"name": "Boosted Level (template)",
"source_type": "template",
"return_type": "float",
"description": "Jinja combinator: clamps 1.5x the Base Level into [0,1]",
"tags": ["demo"],
"template": "clamp(level * 1.5)",
"inputs": [{"name": "level", "value_source_id": _VS_IDS["level"]}],
"default_value": 0.0,
"eval_interval": None,
"created_at": _NOW,
"updated_at": _NOW,
},
}
# ── Scene Presets ──────────────────────────────────────────────────
@@ -8,7 +8,7 @@ import numpy as np
from ledgrab.core.processing.color_strip_stream import ColorStripStream
from ledgrab.storage.bindable import bfloat
from ledgrab.utils import get_logger
from ledgrab.utils import clamp01, get_logger
from ledgrab.utils.frame_limiter import FrameLimiter
logger = get_logger(__name__)
@@ -176,7 +176,7 @@ class CompositeColorStripStream(ColorStripStream):
if i in self._brightness_streams:
_vs_id, vs = self._brightness_streams[i]
try:
result.append(vs.get_value())
result.append(clamp01(vs.get_value()))
except Exception:
result.append(None)
else:
@@ -660,10 +660,14 @@ class CompositeColorStripStream(ColorStripStream):
if layer.get("reverse", False):
colors = colors[::-1].copy()
# Apply per-layer brightness from value source
# Apply per-layer brightness from value source.
# clamp01 is finite-safe: it rejects nan/inf (which would
# crash the int() cast) and pins out-of-range values into
# [0,1] so the uint16 fixed-point multiply can't wrap on a
# negative. bri == 1.0 correctly skips the scale (no-op).
if i in self._brightness_streams:
_vs_id, vs = self._brightness_streams[i]
bri = vs.get_value()
bri = clamp01(vs.get_value())
if bri < 1.0:
colors = (colors.astype(np.uint16) * int(bri * 256) >> 8).astype(
np.uint8
@@ -193,6 +193,7 @@ def _build_ha_entity(source, d: ValueStreamDeps):
min_ha_value=source.min_ha_value,
max_ha_value=source.max_ha_value,
smoothing=source.smoothing,
normalize=source.normalize,
ha_manager=d.ha_manager,
)
@@ -232,6 +233,7 @@ def _build_system_metrics(source, _d: ValueStreamDeps):
sensor_label=source.sensor_label,
poll_interval=source.poll_interval,
smoothing=source.smoothing,
normalize=source.normalize,
)
@@ -249,6 +251,7 @@ def _build_game_event(source, d: ValueStreamDeps):
smoothing=source.smoothing,
default_value=source.default_value,
timeout=source.timeout,
normalize=source.normalize,
event_bus=d.event_bus,
)
@@ -263,10 +266,25 @@ def _build_http(source, d: ValueStreamDeps):
min_value=source.min_value,
max_value=source.max_value,
smoothing=source.smoothing,
normalize=source.normalize,
http_endpoint_store=d.http_endpoint_store,
)
def _build_template(source, d: ValueStreamDeps):
# References other value sources via d.value_stream_manager (recursively
# acquired in start()), exactly like _build_gradient_map.
from ledgrab.core.processing.value_stream import TemplateValueStream
return TemplateValueStream(
template=source.template,
inputs=source.inputs,
default_value=source.default_value,
eval_interval=source.eval_interval,
value_stream_manager=d.value_stream_manager,
)
# ---------------------------------------------------------------------------
# Registry
# ---------------------------------------------------------------------------
@@ -290,6 +308,7 @@ STREAM_BUILDERS: dict[str, StreamBuilder] = {
"system_metrics": _build_system_metrics,
"game_event": _build_game_event,
"http": _build_http,
"template": _build_template,
}
@@ -33,7 +33,12 @@ import numpy as np
from ledgrab.core.processing import metric_readers as _metric_readers
from ledgrab.storage.base_store import EntityNotFoundError
from ledgrab.utils import get_logger
from ledgrab.utils import clamp01, get_logger
from ledgrab.utils.template_expr import (
TemplateValidationError,
compile_template,
finalize_result,
)
# Compiled once — used by ``_extract_simple_path`` on every poll.
_NAME_HEAD_RE = re.compile(r"^([^\[]*)")
@@ -53,6 +58,12 @@ if TYPE_CHECKING:
logger = get_logger(__name__)
# Runtime cap on recursive value-stream acquisition (referencing sources like
# template / gradient_map re-enter acquire() from start()). Higher than the
# storage-level MAX_VALUE_SOURCE_DEPTH (8) so legitimate chains never trip it;
# it only fires on a cycle that bypassed storage validation.
_MAX_ACQUIRE_DEPTH = 12
# ---------------------------------------------------------------------------
# Base class
@@ -904,6 +915,7 @@ class HAEntityValueStream(ValueStream):
min_ha_value: float = 0.0,
max_ha_value: float = 100.0,
smoothing: float = 0.0,
normalize: bool = True,
ha_manager: Any | None = None,
):
self._ha_source_id = ha_source_id
@@ -912,6 +924,7 @@ class HAEntityValueStream(ValueStream):
self._min_ha = min_ha_value
self._max_ha = max_ha_value
self._smoothing = smoothing
self._normalize_enabled = normalize
self._ha_manager = ha_manager
self._prev_value: float | None = None
self._raw_value: float | None = None
@@ -976,16 +989,23 @@ class HAEntityValueStream(ValueStream):
self._raw_value = raw
# Normalize to [0, 1]
ha_range = self._max_ha - self._min_ha
if abs(ha_range) < 1e-9:
normalized = 0.5
if self._normalize_enabled:
# Normalize to [0, 1] via the configured min/max range.
ha_range = self._max_ha - self._min_ha
if abs(ha_range) < 1e-9:
normalized = 0.5
else:
normalized = (raw - self._min_ha) / ha_range
normalized = max(0.0, min(1.0, normalized))
else:
normalized = (raw - self._min_ha) / ha_range
# Skip the rescale: treat the raw reading as already a 01 fraction
# (finite-safe clamp). The un-clamped magnitude stays on
# get_raw_value(); get_value() never leaves [0, 1] either way.
normalized = clamp01(raw)
normalized = max(0.0, min(1.0, normalized))
# EMA smoothing
# EMA smoothing — both branches produce a [0, 1] value, so _prev_value
# is always normalized and flipping ``normalize`` live never blends a
# raw magnitude against a fraction.
if self._smoothing > 0.0 and self._prev_value is not None:
normalized = self._smoothing * self._prev_value + (1.0 - self._smoothing) * normalized
@@ -1009,6 +1029,7 @@ class HAEntityValueStream(ValueStream):
self._min_ha = source.min_ha_value
self._max_ha = source.max_ha_value
self._smoothing = source.smoothing
self._normalize_enabled = source.normalize
# If HA source changed, swap runtime
if source.ha_source_id != old_ha_source and self._ha_manager:
@@ -1052,6 +1073,7 @@ class HTTPValueStream(ValueStream):
min_value: float,
max_value: float,
smoothing: float,
normalize: bool = True,
http_endpoint_store: "HTTPEndpointStore" | None = None,
) -> None:
self._endpoint_id = endpoint_id
@@ -1060,6 +1082,7 @@ class HTTPValueStream(ValueStream):
self._min_value = min_value
self._max_value = max_value
self._smoothing = smoothing
self._normalize_enabled = normalize
self._http_endpoint_store = http_endpoint_store
self._task: asyncio.Task | None = None
self._raw_value: Any = None
@@ -1099,12 +1122,19 @@ class HTTPValueStream(ValueStream):
except (TypeError, ValueError):
return self._prev_normalized if self._prev_normalized is not None else 0.0
rng = self._max_value - self._min_value
if abs(rng) < 1e-9:
normalized = 0.5
if self._normalize_enabled:
rng = self._max_value - self._min_value
if abs(rng) < 1e-9:
normalized = 0.5
else:
normalized = (numeric - self._min_value) / rng
normalized = max(0.0, min(1.0, normalized))
else:
normalized = (numeric - self._min_value) / rng
normalized = max(0.0, min(1.0, normalized))
# Skip the rescale: treat the extracted number as already a 01
# fraction (finite-safe clamp). The verbatim extracted value (which
# may be non-numeric) stays on get_raw_value(); get_value() is always
# a float in [0, 1].
normalized = clamp01(numeric)
if self._smoothing > 0.0 and self._prev_normalized is not None:
normalized = (
@@ -1128,6 +1158,7 @@ class HTTPValueStream(ValueStream):
self._min_value = source.min_value
self._max_value = source.max_value
self._smoothing = source.smoothing
self._normalize_enabled = source.normalize
async def _poll_loop(self) -> None:
from ledgrab.utils.safe_source import safe_request_bounded
@@ -1365,6 +1396,168 @@ class GradientMapValueStream(ValueStream):
self._inner_stream = None
# ---------------------------------------------------------------------------
# Template (Jinja expression combinator)
# ---------------------------------------------------------------------------
class TemplateValueStream(ValueStream):
"""Evaluates a hardened sandboxed-Jinja expression over the live values of
other value sources (the system's float combinator).
Acquires each referenced input stream from the manager on ``start()`` and
releases it on ``stop()`` — the same ref-counted protocol as
:class:`GradientMapValueStream`, but over a *set* of inputs. Acquisition is
tracked per unique ``value_source_id`` so two variables bound to the same
source share one ref. ``get_value()`` builds a primitives-only context
(each input's normalized ``get_value()`` plus a float-only ``raw`` dict),
evaluates the compiled expression, then coerces / NaN-guards / clamps the
result. Any error — or an uncompilable template — falls back to
``default_value``. An optional ``eval_interval`` caches the last result to
bound steady-state evaluation cost.
"""
def __init__(
self,
template: str,
inputs: List[dict],
default_value: float = 0.0,
eval_interval: float | None = None,
value_stream_manager: "ValueStreamManager" | None = None,
):
self._template = template
self._inputs = [dict(i) for i in (inputs or [])]
self._default = max(0.0, min(1.0, float(default_value)))
self._eval_interval = float(eval_interval) if eval_interval else 0.0
self._vsm = value_stream_manager
self._streams_by_id: Dict[str, ValueStream] = {} # value_source_id -> stream
self._expr = self._compile(template)
self._last_value: float = self._default
self._last_eval: float = 0.0
self._has_value = False
self._error_logged = False
@staticmethod
def _compile(template: str):
"""Compile once; return ``None`` (→ always default) on invalid template.
Creation should already have rejected invalid templates via the factory;
this is defense in depth so a bad row never crashes the engine.
"""
try:
return compile_template(template)
except TemplateValidationError as e:
logger.warning("TemplateValueStream: invalid template, using default (%s)", e)
return None
@staticmethod
def _unique_ids(inputs: List[dict]) -> set:
return {i["value_source_id"] for i in inputs if i.get("value_source_id")}
def start(self) -> None:
if not self._vsm:
return
for vs_id in self._unique_ids(self._inputs):
try:
self._streams_by_id[vs_id] = self._vsm.acquire(vs_id)
except Exception as e:
logger.warning("TemplateValueStream: failed to acquire input %s: %s", vs_id, e)
def stop(self) -> None:
if self._vsm:
for vs_id in list(self._streams_by_id):
try:
self._vsm.release(vs_id)
except Exception as e:
logger.debug("TemplateValueStream: release %s failed: %s", vs_id, e)
self._streams_by_id.clear()
self._has_value = False
def get_value(self) -> float:
if self._expr is None:
return self._default
if (
self._eval_interval > 0.0
and self._has_value
and (time.monotonic() - self._last_eval) < self._eval_interval
):
return self._last_value
try:
ctx: Dict[str, Any] = {}
raw: Dict[str, float] = {}
for inp in self._inputs:
name = inp.get("name")
vs_id = inp.get("value_source_id")
if not name or not vs_id:
continue
stream = self._streams_by_id.get(vs_id)
if stream is None:
continue
ctx[name] = float(stream.get_value())
getter = getattr(stream, "get_raw_value", None)
if getter is not None:
rv = getter()
if rv is not None:
try:
raw[name] = float(rv)
except (TypeError, ValueError):
# Non-numeric raw values never cross into the sandbox.
pass
ctx["raw"] = raw
# Globals (min/max/abs/round/clamp) resolve from SANDBOX_ENV.globals.
value = finalize_result(self._expr(**ctx), self._default)
except Exception as e:
if not self._error_logged:
logger.warning("TemplateValueStream eval error (using default): %s", e)
self._error_logged = True
value = self._default
self._last_value = value
self._last_eval = time.monotonic()
self._has_value = True
return value
def update_source(self, source: "ValueSource") -> None:
from ledgrab.storage.value_source import TemplateValueSource
if not isinstance(source, TemplateValueSource):
return
if source.template != self._template:
self._template = source.template
self._expr = self._compile(source.template)
self._error_logged = False
self._default = max(0.0, min(1.0, float(source.default_value)))
self._eval_interval = float(source.eval_interval) if source.eval_interval else 0.0
new_inputs = [dict(i) for i in (source.inputs or [])]
old_ids = set(self._streams_by_id)
new_ids = self._unique_ids(new_inputs)
if self._vsm:
# Release-before-acquire (mirrors GradientMapValueStream); safe under
# ref-counting. Unchanged ids keep their existing stream untouched.
for vs_id in old_ids - new_ids:
try:
self._vsm.release(vs_id)
except Exception as e:
logger.debug("TemplateValueStream: release %s failed: %s", vs_id, e)
self._streams_by_id.pop(vs_id, None)
for vs_id in new_ids - old_ids:
try:
self._streams_by_id[vs_id] = self._vsm.acquire(vs_id)
except Exception as e:
logger.warning("TemplateValueStream: acquire %s failed: %s", vs_id, e)
# Rebuild inputs (re-keys variable names on rename even when id unchanged,
# since get_value() maps name -> stream via value_source_id each tick).
self._inputs = new_inputs
self._has_value = False
# ---------------------------------------------------------------------------
# CSS Extract
# ---------------------------------------------------------------------------
@@ -1501,6 +1694,7 @@ class SystemMetricsValueStream(ValueStream):
sensor_label: str = "",
poll_interval: float = 1.0,
smoothing: float = 0.0,
normalize: bool = True,
):
self._metric = metric
self._min_val = min_value
@@ -1510,6 +1704,7 @@ class SystemMetricsValueStream(ValueStream):
self._sensor_label = sensor_label
self._poll_interval = max(0.1, poll_interval)
self._smoothing = smoothing
self._normalize_enabled = normalize
self._prev_value: float | None = None
self._raw_value: float | None = None
self._last_poll: float = 0.0
@@ -1548,10 +1743,16 @@ class SystemMetricsValueStream(ValueStream):
raw = self._read_metric()
self._raw_value = raw
# Normalize
normalized = self._normalize(raw)
if self._normalize_enabled:
normalized = self._normalize(raw)
else:
# Skip the rescale: treat the raw reading as already a 01 fraction
# (finite-safe clamp). The un-clamped reading stays on
# get_raw_value(); get_value() never leaves [0, 1].
normalized = clamp01(raw)
# EMA smoothing
# EMA smoothing — both branches output [0, 1], so _prev_value is always
# normalized and a live ``normalize`` flip never blends raw vs fraction.
if self._smoothing > 0.0 and self._prev_value is not None:
normalized = self._smoothing * self._prev_value + (1.0 - self._smoothing) * normalized
@@ -1599,6 +1800,7 @@ class SystemMetricsValueStream(ValueStream):
self._sensor_label = source.sensor_label
self._poll_interval = max(0.1, source.poll_interval)
self._smoothing = source.smoothing
self._normalize_enabled = source.normalize
# ---------------------------------------------------------------------------
@@ -1644,6 +1846,10 @@ class ValueStreamManager:
self._http_endpoint_store = http_endpoint_store
self._streams: Dict[str, ValueStream] = {} # vs_id → stream
self._ref_counts: Dict[str, int] = {} # vs_id → ref count
# Recursion-depth backstop for referencing sources (template / gradient
# map). A cycle that slipped past storage validation (e.g. a hand-edited
# DB or restored backup) would otherwise overflow the stack at acquire().
self._acquire_depth = 0
# Tracks which clock_id (if any) was acquired for each stream so we
# can release/swap it without re-querying the store at teardown time.
self._stream_clock_ids: Dict[str, str] = {} # vs_id → clock_id
@@ -1659,9 +1865,29 @@ class ValueStreamManager:
logger.info(f"Shared value stream {vs_id} (refs={self._ref_counts[vs_id]})")
return self._streams[vs_id]
if self._acquire_depth >= _MAX_ACQUIRE_DEPTH:
logger.warning(
"Value source acquire depth limit (%d) reached at %s; returning "
"static fallback (possible reference cycle)",
_MAX_ACQUIRE_DEPTH,
vs_id,
)
# The intermediate referencing streams built while descending a
# cyclic chain are not stop()'d here — but this only triggers on a
# stored cycle that storage validation already rejects (e.g. a
# hand-edited DB / corrupt restore), so those transient objects are
# simply garbage-collected. Normal graphs never reach this depth.
return StaticValueStream(0.5)
source = self._value_source_store.get_source(vs_id)
stream = self._create_stream(source, vs_id)
stream.start()
# Increment around create+start: a referencing stream (template /
# gradient_map) re-enters acquire() from its own start().
self._acquire_depth += 1
try:
stream = self._create_stream(source, vs_id)
stream.start()
finally:
self._acquire_depth -= 1
self._streams[vs_id] = stream
self._ref_counts[vs_id] = 1
logger.info(f"Acquired value stream {vs_id} (type={source.source_type})")
@@ -15,7 +15,7 @@ import time
from typing import TYPE_CHECKING
from ledgrab.core.processing.value_stream import ValueStream
from ledgrab.utils import get_logger
from ledgrab.utils import clamp01, get_logger
if TYPE_CHECKING:
from ledgrab.core.game_integration.event_bus import GameEventBus
@@ -41,6 +41,7 @@ class GameEventValueStream(ValueStream):
smoothing: float = 0.0,
default_value: float = 0.5,
timeout: float = 5.0,
normalize: bool = True,
event_bus: "GameEventBus" | None = None,
) -> None:
self._event_type = event_type
@@ -49,10 +50,15 @@ class GameEventValueStream(ValueStream):
self._smoothing = max(0.0, min(1.0, smoothing))
self._default_value = max(0.0, min(1.0, default_value))
self._timeout = max(0.0, timeout)
# When False, skip the min/max rescale: the [0,1] output clamps the raw
# value as-is. get_value() stays in [0,1] either way; the un-clamped
# value is exposed via get_raw_value() for templates/automations.
self._normalize_enabled = normalize
self._event_bus = event_bus
self._lock = threading.Lock()
self._current_value: float = self._default_value
self._current_raw: float | None = None
self._last_event_time: float | None = None
self._subscription_id: str | None = None
self._has_received_event: bool = False
@@ -82,11 +88,18 @@ class GameEventValueStream(ValueStream):
self._subscription_id = None
with self._lock:
self._current_value = self._default_value
self._current_raw = None
self._last_event_time = None
self._has_received_event = False
def get_value(self) -> float:
"""Return current normalized value (0.0-1.0), or default if timed out."""
"""Return current value in [0,1], or default_value if timed out.
Always in [0,1]: ``_current_value`` holds the smoothed output computed
at event time under the active ``normalize`` mode (rescaled, or the raw
value clamped into [0,1]). ``default_value`` is itself in [0,1], so the
timeout fallback is valid in both modes.
"""
with self._lock:
if not self._has_received_event:
return self._default_value
@@ -98,6 +111,15 @@ class GameEventValueStream(ValueStream):
return self._current_value
def get_raw_value(self) -> float | None:
"""Return the last raw game value before normalization.
``None`` until the first event arrives (mirrors HA/HTTP/SystemMetrics).
Exposes the un-clamped magnitude to template ``raw[]`` and automations.
"""
with self._lock:
return self._current_raw
def get_color(self) -> tuple:
"""Game event value source only provides scalars, not colors."""
raise NotImplementedError("GameEventValueStream does not produce colors")
@@ -115,6 +137,7 @@ class GameEventValueStream(ValueStream):
self._smoothing = max(0.0, min(1.0, source.smoothing))
self._default_value = max(0.0, min(1.0, source.default_value))
self._timeout = max(0.0, source.timeout)
self._normalize_enabled = source.normalize
def _on_event(self, event: "GameEvent") -> None:
"""EventBus callback — normalize and apply smoothing.
@@ -122,14 +145,17 @@ class GameEventValueStream(ValueStream):
Called from the publisher's thread; must be thread-safe.
"""
raw_value = event.value
normalized = self._normalize(raw_value)
# Output is always in [0,1]: rescale via min/max, or (normalize off)
# clamp the raw value as-is. The un-clamped raw is kept for get_raw_value().
out = self._normalize(raw_value) if self._normalize_enabled else clamp01(raw_value)
with self._lock:
self._current_raw = raw_value
if self._smoothing > 0.0 and self._has_received_event:
alpha = 1.0 - self._smoothing
normalized = alpha * normalized + self._smoothing * self._current_value
out = alpha * out + self._smoothing * self._current_value
self._current_value = normalized
self._current_value = out
self._last_event_time = time.monotonic()
self._has_received_event = True
+8
View File
@@ -15,6 +15,7 @@ def main():
import uvicorn
from ledgrab.config import get_config
from ledgrab.shutdown_state import GRACEFUL_SHUTDOWN_TIMEOUT
config = get_config()
uvicorn.run(
@@ -22,7 +23,14 @@ def main():
host=config.server.host,
port=config.server.port,
log_level=config.server.log_level.lower(),
# Access logging is handled by the _access_log middleware (with token
# attribution); disable uvicorn's to avoid duplicate lines.
access_log=False,
reload=False,
# Bound the graceful-shutdown wait so Ctrl+C with the UI open still
# runs the lifespan shutdown instead of hanging on a lingering events
# WebSocket — see shutdown_state.
timeout_graceful_shutdown=GRACEFUL_SHUTDOWN_TIMEOUT,
)
+37 -1
View File
@@ -2,6 +2,7 @@
import asyncio
import sys
import time
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Awaitable
@@ -74,7 +75,7 @@ config = get_config()
# The shutdown-complete signal is owned by a leaf module so ``__main__``
# can import it without dragging in this module's heavy global state.
from ledgrab.shutdown_state import shutdown_complete # noqa: E402
from ledgrab.shutdown_state import GRACEFUL_SHUTDOWN_TIMEOUT, shutdown_complete # noqa: E402
def _migrate_legacy_data_location() -> None:
@@ -577,6 +578,33 @@ async def _security_headers(request: Request, call_next):
return response
# Middleware: structured access log enriched with the authenticated token's
# friendly label (the key name from auth.api_keys), so requests can be
# attributed to a specific client (e.g. "homeassistant" vs "android"). The
# label is set onto request.state by verify_api_key; endpoints without auth
# (or failed auth) log "unauthenticated". Only the label is logged — never the
# token secret. Registered last so it runs outermost: it measures total
# handling time and always records the final status, even on error.
@app.middleware("http")
async def _access_log(request: Request, call_next):
start = time.perf_counter()
status_code = 500
try:
response = await call_next(request)
status_code = response.status_code
return response
finally:
logger.info(
"http_request",
method=request.method,
path=request.url.path,
status=status_code,
token=getattr(request.state, "auth_label", None) or "unauthenticated",
client=request.client.host if request.client else None,
duration_ms=round((time.perf_counter() - start) * 1000, 1),
)
# ── Auth-gated OpenAPI surface ────────────────────────────────────────────
# Re-add the docs endpoints we disabled above, now protected by the same
# Bearer auth as the rest of the API. When auth is unconfigured, loopback
@@ -645,5 +673,13 @@ if __name__ == "__main__":
host=config.server.host,
port=config.server.port,
log_level=config.server.log_level.lower(),
# Our _access_log middleware emits a richer structured line (incl. the
# authenticated token label), so suppress uvicorn's default access log
# to avoid two lines per request.
access_log=False,
reload=False, # Disabled due to watchfiles infinite reload loop
# Bound the graceful-shutdown wait so Ctrl+C with the UI open still
# runs the lifespan shutdown (stop targets + DB checkpoint) instead of
# hanging on a lingering events WebSocket — see shutdown_state.
timeout_graceful_shutdown=GRACEFUL_SHUTDOWN_TIMEOUT,
)
+12
View File
@@ -16,3 +16,15 @@ they release Windows / unblock only once cleanup is genuinely done.
import threading
shutdown_complete: threading.Event = threading.Event()
# Bound uvicorn's graceful-shutdown wait (``uvicorn.Config(timeout_graceful_shutdown=...)``).
# uvicorn defaults this to ``None`` — ``Server.shutdown()`` then waits *forever*
# for open connections (and their tasks) to drain before it runs the lifespan
# shutdown. The events WebSocket handler blocks on ``queue.get()`` and the
# browser auto-reconnects, so connections never drain on their own. Without a
# bound, the lifespan shutdown — which stops LED targets and checkpoints the
# DB — never runs: targets stay lit and the process can't exit (leftover
# processor threads). Shared by both the desktop (__main__) and Android
# (android_entry) launchers. Keep it small so OS-shutdown cleanup still fits
# Windows' ~20 s budget; it is spent BEFORE the lifespan's own ~16 s budget.
GRACEFUL_SHUTDOWN_TIMEOUT: int = 3
@@ -298,6 +298,214 @@ select.field-invalid {
line-height: 1.3;
}
.field-ok-msg {
display: flex;
align-items: center;
gap: 4px;
color: var(--success-color);
font-size: 0.78rem;
margin-top: 4px;
line-height: 1.3;
}
.field-ok-msg .icon { width: 14px; height: 14px; }
.field-error-msg .icon { width: 14px; height: 14px; vertical-align: -2px; margin-right: 3px; }
.field-warn-msg {
display: block;
color: var(--text-muted);
font-size: 0.76rem;
margin-top: 4px;
line-height: 1.35;
}
/* ── Jinja expression editor ─────────────────────────────────────
A transparent <textarea> layered over a synced highlight <pre>.
Both share identical type metrics so the colour layer aligns with
the typed glyphs. The shared box rules below MUST stay in sync. */
.jinja-editor {
position: relative;
display: block;
border: 1px solid var(--border-color);
border-radius: var(--radius-sm);
background: var(--surface-2, color-mix(in srgb, var(--text-color) 6%, var(--bg-color)));
overflow: hidden;
transition: border-color 0.15s ease, box-shadow 0.15s ease;
}
.jinja-editor:focus-within {
border-color: var(--primary-color);
box-shadow: 0 0 0 2px color-mix(in srgb, var(--primary-color) 18%, transparent);
}
.jinja-editor.field-invalid {
border-color: var(--danger-color);
box-shadow: 0 0 0 2px color-mix(in srgb, var(--danger-color) 15%, transparent);
}
/* Shared metrics — applied identically to both layers. */
.jinja-hl,
.jinja-input {
margin: 0;
padding: 10px 12px;
font-family: var(--font-mono);
font-size: 0.86rem;
line-height: 1.55;
letter-spacing: 0;
tab-size: 2;
white-space: pre;
word-wrap: normal;
overflow: auto;
box-sizing: border-box;
}
.jinja-hl {
position: absolute;
inset: 0;
pointer-events: none;
color: var(--text-color);
overflow: hidden; /* scroll is mirrored from the textarea */
user-select: none;
}
.jinja-input {
position: relative;
display: block;
width: 100%;
min-height: 4.6em; /* ~3 lines */
resize: vertical;
border: none;
background: transparent;
color: transparent; /* glyphs are painted by .jinja-hl underneath */
caret-color: var(--text-color);
outline: none;
}
/* The global `textarea:focus` rule (higher specificity than `.jinja-input`)
sets an opaque background; on this overlay editor that would cover the
`.jinja-hl` highlight layer and hide the transparent glyphs on focus. Keep
the textarea fully transparent — the focus ring is drawn by the wrapper's
`.jinja-editor:focus-within`. */
.jinja-input:focus {
background: transparent;
color: transparent;
box-shadow: none;
}
.jinja-input::selection { background: color-mix(in srgb, var(--primary-color) 30%, transparent); }
/* Token palette — restrained, three accents plus muted operators. */
.jinja-hl .tok-str { color: var(--success-color); }
.jinja-hl .tok-num { color: #d19a66; }
.jinja-hl .tok-fn { color: var(--primary-color); font-weight: 600; }
.jinja-hl .tok-raw { color: #c678dd; font-style: italic; }
.jinja-hl .tok-var { color: #61afef; }
.jinja-hl .tok-op { color: var(--text-muted); }
/* ── Template-input rows ─────────────────────────────────────── */
.template-inputs-list {
display: flex;
flex-direction: column;
gap: 8px;
margin-bottom: 8px;
}
.template-input-row {
display: grid;
grid-template-columns: minmax(96px, 0.42fr) minmax(0, 1fr) auto;
align-items: center;
gap: 8px;
}
.template-input-name {
font-family: var(--font-mono);
font-size: 0.84rem;
}
.template-input-row .entity-select-trigger { width: 100%; }
.template-inputs-empty {
color: var(--text-muted);
font-size: 0.82rem;
padding: 6px 2px;
}
/* ── Expression hints panel ──────────────────────────────────── */
.jinja-hints {
margin-top: 10px;
border: 1px solid var(--border-color);
border-radius: var(--radius-sm);
background: color-mix(in srgb, var(--text-color) 3%, transparent);
}
.jinja-hints > summary {
cursor: pointer;
list-style: none;
padding: 8px 12px;
font-size: 0.8rem;
font-weight: 600;
color: var(--text-secondary);
user-select: none;
}
.jinja-hints > summary::-webkit-details-marker { display: none; }
.jinja-hints > summary::before {
content: '';
display: inline-block;
margin-right: 6px;
transition: transform 0.15s ease;
color: var(--text-muted);
}
.jinja-hints[open] > summary::before { transform: rotate(90deg); }
.jinja-hints-body {
padding: 4px 14px 12px;
font-size: 0.8rem;
line-height: 1.5;
color: var(--text-secondary);
}
.jinja-hints-body code,
.jinja-hints-body .jinja-hints-vars code {
font-family: var(--font-mono);
font-size: 0.78rem;
padding: 1px 5px;
border-radius: var(--radius-sm);
background: color-mix(in srgb, var(--text-color) 8%, transparent);
color: var(--text-color);
}
.jinja-hints-section { margin-top: 8px; }
.jinja-hints-section-title {
display: block;
font-size: 0.72rem;
text-transform: uppercase;
letter-spacing: 0.06em;
color: var(--text-muted);
margin-bottom: 3px;
}
.jinja-hints-vars {
display: flex;
flex-wrap: wrap;
gap: 5px;
}
.jinja-hints-vars .tok-var-chip {
font-family: var(--font-mono);
font-size: 0.76rem;
padding: 1px 6px;
border-radius: var(--radius-pill);
background: color-mix(in srgb, #61afef 16%, transparent);
color: #61afef;
}
.jinja-hints-examples { margin: 4px 0 0; padding-left: 0; list-style: none; }
.jinja-hints-examples li { margin: 3px 0; }
.jinja-hints-empty { color: var(--text-muted); font-size: 0.76rem; font-style: italic; }
.jinja-hints-time { color: var(--text-muted); font-size: 0.76rem; }
/* Remove browser autofill styling */
input:-webkit-autofill,
input:-webkit-autofill:hover,
@@ -495,6 +495,18 @@ html:has(#tab-graph.active) {
opacity: 0.95;
}
/* Custom per-entity icon: the embedded SVG strokes with currentColor, so it is
tinted via `color` (default muted; the node's icon_color overrides inline). */
.graph-node-custom-icon {
color: var(--lux-ink-mute, var(--text-muted));
opacity: 0.9;
}
.graph-node.running .graph-node-custom-icon {
color: var(--ch-signal, var(--primary-color));
opacity: 1;
}
/* ── Running indicator (animated gradient border + signal-flow glow) ── */
.graph-node.running .graph-node-body {
@@ -588,6 +600,21 @@ html:has(#tab-graph.active) {
filter: drop-shadow(0 0 6px var(--ch-signal, var(--primary-color)));
}
/* Whole-node drop targets: a source can be dropped on any compatible node to
wire one of its slots — including empty slots that have no input port yet. */
.graph-svg.connecting .graph-node-compatible .graph-node-body {
stroke: var(--ch-signal, var(--primary-color));
stroke-dasharray: 4 3;
stroke-width: 1.5;
}
.graph-node-drop-target .graph-node-body {
stroke: var(--ch-signal, var(--primary-color)) !important;
stroke-width: 2.5 !important;
stroke-dasharray: none !important;
filter: drop-shadow(0 0 8px color-mix(in srgb, var(--ch-signal, var(--primary-color)) 55%, transparent));
}
/* ── Edges ── */
.graph-edge {
@@ -630,6 +657,25 @@ html:has(#tab-graph.active) {
opacity: 0.4;
}
/* Edge field labels — hidden until zoomed in enough to read them. */
.graph-edge-label {
font-size: 9px;
font-weight: 600;
font-family: var(--font-mono, monospace);
fill: var(--text-secondary);
paint-order: stroke;
stroke: var(--lux-bg-1, var(--card-bg));
stroke-width: 3px;
stroke-linejoin: round;
pointer-events: none;
opacity: 0;
transition: opacity 0.15s ease;
}
.graph-edges.show-labels .graph-edge-label {
opacity: 0.85;
}
/* Edge type colors */
.graph-edge-picture { stroke: #42A5F5; color: #42A5F5; }
.graph-edge-colorstrip { stroke: #66BB6A; color: #66BB6A; }
@@ -788,6 +834,17 @@ html:has(#tab-graph.active) {
stroke-dasharray: 4 3;
}
/* ── Health overlay: configuration issues (broken refs / cycles) ── */
.graph-node.has-issue .graph-node-body {
stroke: var(--danger-color);
stroke-width: 2;
stroke-dasharray: 5 3;
}
.graph-node-issue {
color: var(--danger-color);
}
/* ── Search highlight ── */
.graph-node.search-match .graph-node-body {
@@ -1001,6 +1058,33 @@ html:has(#tab-graph.active) {
display: inline-block;
}
/* Issues toolbar button + count badge */
.graph-issues-btn {
position: relative;
color: var(--danger-color);
}
.graph-issues-count {
position: absolute;
top: 0;
right: 0;
transform: translate(35%, -35%);
background: var(--danger-color);
color: #fff;
border-radius: 8px;
font-size: 0.6rem;
font-weight: 700;
min-width: 14px;
height: 14px;
line-height: 14px;
text-align: center;
padding: 0 3px;
}
.graph-issues-count:empty {
display: none;
}
.graph-filter-types-popover {
display: none;
flex-direction: column;
+8 -1
View File
@@ -184,9 +184,11 @@ import {
showValueSourceModal, closeValueSourceModal, saveValueSource,
editValueSource, cloneValueSource, deleteValueSource, onValueSourceTypeChange,
onDaylightVSRealTimeChange,
onValueSourceNormalizeChange,
addSchedulePoint,
addAnimatedColor, removeAnimatedColor,
addColorSchedulePoint, removeColorSchedulePoint,
addTemplateInput,
testValueSource, closeTestValueSourceModal,
} from './features/value-sources.ts';
@@ -207,7 +209,7 @@ import {
import {
loadGraphEditor,
toggleGraphLegend, toggleGraphMinimap, toggleGraphFilter, toggleGraphFilterTypes, toggleGraphHelp, graphUndo, graphRedo,
graphFitAll, graphZoomIn, graphZoomOut, graphRelayout,
graphFitAll, graphZoomIn, graphZoomOut, graphRelayout, graphShowIssues, graphExportTopology, graphDuplicateSelection,
graphToggleFullscreen, graphAddEntity, toggleToolbarOverflow, closeToolbarOverflow,
} from './features/graph-editor.ts';
@@ -579,11 +581,13 @@ Object.assign(window, {
deleteValueSource,
onValueSourceTypeChange,
onDaylightVSRealTimeChange,
onValueSourceNormalizeChange,
addSchedulePoint,
addAnimatedColor,
removeAnimatedColor,
addColorSchedulePoint,
removeColorSchedulePoint,
addTemplateInput,
testValueSource,
closeTestValueSourceModal,
@@ -625,6 +629,9 @@ Object.assign(window, {
graphZoomIn,
graphZoomOut,
graphRelayout,
graphShowIssues,
graphExportTopology,
graphDuplicateSelection,
graphToggleFullscreen,
graphAddEntity,
toggleToolbarOverflow,
@@ -3,11 +3,128 @@
* Supports creating, changing, and detaching connections via the graph editor.
*/
import { apiPut } from './api-client.ts';
import { apiPut, apiPost, apiGet } from './api-client.ts';
import {
streamsCache, colorStripSourcesCache, valueSourcesCache,
audioSourcesCache, outputTargetsCache, automationsCacheObj,
} from './state.ts';
import { logError } from './log.ts';
/** Result of the backend pre-write connection validator. */
export interface ConnectionValidation {
ok: boolean;
error?: string | null;
}
/**
* Ask the backend whether a proposed wiring edit is valid (target/source exist,
* source is the right kind, and it would not create a dependency cycle).
*
* Fails *open*: if the validation endpoint is unavailable we return ``ok`` so
* wiring still works against older servers — the per-entity PUT remains the
* source of truth, this is just an early, friendlier guard.
*/
export async function validateConnection(
targetKind: string, targetId: string, field: string, sourceId: string,
): Promise<ConnectionValidation> {
try {
return await apiPost<ConnectionValidation>('/graph/validate-connection', {
target_kind: targetKind,
target_id: targetId,
field,
source_id: sourceId,
});
} catch {
return { ok: true };
}
}
/** An entity that references another entity (one row of the dependents query). */
export interface GraphDependent {
id: string;
kind: string;
name: string;
field: string;
}
/**
* List every entity that references ``(kind, id)``. Used to warn before a
* delete would dangle other entities' references. Fails *safe* (empty list)
* if the endpoint is unavailable.
*/
export async function getDependents(kind: string, id: string): Promise<GraphDependent[]> {
try {
const res = await apiGet<{ dependents: GraphDependent[] }>(
`/graph/dependents/${encodeURIComponent(kind)}/${encodeURIComponent(id)}`,
);
return res.dependents || [];
} catch {
return [];
}
}
/* ── Schema drift guard (B4) ───────────────────────────────────── */
// Backend-declared reference fields the frontend intentionally does NOT drag-edit
// (the backend still lists them for topology/dependents completeness, so the drift
// check ignores them). Two categories:
// (a) the source kind is not a graph node — nothing to drag from.
// (b) the owning entity's PUT route is not safely partial-writable via a single
// dragged field, so it's edited through the entity editor instead.
const _DRIFT_EXCLUDE = new Set<string>([
// (a) no graph node for the source kind — nothing to drag from:
'value_source|ha_source_id',
'value_source|gradient_id',
// (b) not safely partial-PUT-able from a single dragged field:
'device|default_css_processing_template_id', // a one-field device PUT could null the URL
// (c) editable in principle but not surfaced as a graph edge yet:
'value_source|clock_id', // sync_clock → animated_colour value-source timing
]);
let _driftChecked = false;
interface SchemaConnection { target_kind: string; field: string; editable: boolean; }
/**
* Dev safety net for the B4 finding: warn once if the frontend CONNECTION_MAP's
* editable set diverges from the backend `/graph/schema` (the drift the manual
* "10-step checklist" guards against). Read-only — never affects the graph.
* No-op against older servers without the endpoint.
*
* Note: this references `CONNECTION_MAP` (const) and `_isEditable` (fn) declared
* later in the module — safe because it is only ever invoked at runtime (from
* `loadGraphEditor`), well after module initialization.
*/
export async function checkSchemaDrift(): Promise<void> {
if (_driftChecked) return;
_driftChecked = true;
let connections: SchemaConnection[];
try {
connections = (await apiGet<{ connections: SchemaConnection[] }>('/graph/schema')).connections || [];
} catch {
return; // endpoint unavailable — nothing to compare against
}
const k = (kind: string, field: string): string => `${kind}|${field}`;
const backend = new Set<string>();
for (const c of connections) {
if (c.editable && !_DRIFT_EXCLUDE.has(k(c.target_kind, c.field))) backend.add(k(c.target_kind, c.field));
}
const frontend = new Set<string>();
for (const c of CONNECTION_MAP) {
if (_isEditable(c) && !_DRIFT_EXCLUDE.has(k(c.targetKind, c.field))) frontend.add(k(c.targetKind, c.field));
}
const missingOnFrontend = [...backend].filter(key => !frontend.has(key));
const missingOnBackend = [...frontend].filter(key => !backend.has(key));
if (missingOnFrontend.length || missingOnBackend.length) {
logError('graph.schema_drift', new Error(
`CONNECTION_MAP drift vs /graph/schema — editable fields missing on frontend: ` +
`[${missingOnFrontend.join(', ')}]; missing on backend: [${missingOnBackend.join(', ')}]`,
));
}
}
/* ── Types ────────────────────────────────────────────────────── */
@@ -19,6 +136,13 @@ interface ConnectionEntry {
endpoint?: string;
cache?: { invalidate(): void };
nested?: boolean;
/**
* A single-level value-source binding (e.g. `brightness.source_id`). These
* are structurally nested but ARE drag-editable: the write goes through the
* entity's `BindableFloat.apply_update`, which merges `{source_id}` while
* preserving the static value. (List/double-nested fields stay read-only.)
*/
bindable?: boolean;
}
interface CompatibleInput {
@@ -52,35 +176,42 @@ const CONNECTION_MAP: ConnectionEntry[] = [
{ targetKind: 'value_source', field: 'value_source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/value-sources/{id}', cache: valueSourcesCache },
{ targetKind: 'value_source', field: 'gradient_id', sourceKind: 'gradient', edgeType: 'gradient', endpoint: '/value-sources/{id}', cache: valueSourcesCache },
{ targetKind: 'value_source', field: 'color_strip_source_id', sourceKind: 'color_strip_source', edgeType: 'colorstrip', endpoint: '/value-sources/{id}', cache: valueSourcesCache },
// TODO: template.inputs[] drag-wiring — template value sources reference one
// inner value source per bound input (field path inputs[<name>].value_source_id).
// These render as read-only 'value' edges in graph-layout for now; a list-aware
// CONNECTION_MAP entry (with list/index/ref slot metadata) would make them
// re-wirable from the graph the way composite layers / mapped zones are.
// Color strip sources
{ targetKind: 'color_strip_source', field: 'picture_source_id', sourceKind: 'picture_source', edgeType: 'picture', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache },
{ targetKind: 'color_strip_source', field: 'audio_source_id', sourceKind: 'audio_source', edgeType: 'audio', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache },
{ targetKind: 'color_strip_source', field: 'clock_id', sourceKind: 'sync_clock', edgeType: 'clock', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache },
// Processed strip: input source + processing template (apply_update is partial-safe)
{ targetKind: 'color_strip_source', field: 'input_source_id', sourceKind: 'color_strip_source', edgeType: 'colorstrip', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache },
{ targetKind: 'color_strip_source', field: 'processing_template_id', sourceKind: 'cspt', edgeType: 'template', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache },
// Output targets
{ targetKind: 'output_target', field: 'device_id', sourceKind: 'device', edgeType: 'device', endpoint: '/output-targets/{id}', cache: outputTargetsCache },
{ targetKind: 'output_target', field: 'color_strip_source_id', sourceKind: 'color_strip_source', edgeType: 'colorstrip', endpoint: '/output-targets/{id}', cache: outputTargetsCache },
{ targetKind: 'output_target', field: 'brightness.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/output-targets/{id}', cache: outputTargetsCache, nested: true },
{ targetKind: 'output_target', field: 'picture_source_id', sourceKind: 'picture_source', edgeType: 'picture', endpoint: '/output-targets/{id}', cache: outputTargetsCache },
{ targetKind: 'output_target', field: 'brightness.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/output-targets/{id}', cache: outputTargetsCache, nested: true, bindable: true },
// Automations
{ targetKind: 'automation', field: 'scene_preset_id', sourceKind: 'scene_preset', edgeType: 'scene', endpoint: '/automations/{id}', cache: automationsCacheObj },
{ targetKind: 'automation', field: 'deactivation_scene_preset_id', sourceKind: 'scene_preset', edgeType: 'scene', endpoint: '/automations/{id}', cache: automationsCacheObj },
// ── BindableFloat value source edges (CSS properties) ──
{ targetKind: 'color_strip_source', field: 'smoothing.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'sensitivity.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'intensity.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'scale.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'speed.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'wind_strength.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'temperature_influence.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'sound_volume.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'timeout.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'brightness.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
// ── BindableFloat value source edges (CSS properties) — drag-editable ──
{ targetKind: 'color_strip_source', field: 'smoothing.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'sensitivity.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'intensity.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'scale.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'speed.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'wind_strength.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'temperature_influence.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'sound_volume.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'timeout.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
{ targetKind: 'color_strip_source', field: 'brightness.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache, nested: true, bindable: true },
// HA light target transition binding
{ targetKind: 'output_target', field: 'transition.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'output_target', field: 'transition.source_id', sourceKind: 'value_source', edgeType: 'value', endpoint: '/output-targets/{id}', cache: outputTargetsCache, nested: true, bindable: true },
// ── BindableColor value source edges (CSS color properties) ──
{ targetKind: 'color_strip_source', field: 'color.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
{ targetKind: 'color_strip_source', field: 'color_peak.source_id', sourceKind: 'value_source', edgeType: 'value', nested: true },
@@ -97,12 +228,23 @@ const CONNECTION_MAP: ConnectionEntry[] = [
{ targetKind: 'scene_preset', field: 'target_id', sourceKind: 'output_target', edgeType: 'scene', nested: true },
];
/** Editable via the graph: top-level reference fields, plus single-level
* bindable value-source slots (list/double-nested fields stay read-only). */
function _isEditable(c: ConnectionEntry): boolean {
return !c.nested || !!c.bindable;
}
/** True when a field is a bindable slot (its parent is a `Bindable*`). */
export function isBindableField(targetKind: string, field: string): boolean {
return !!CONNECTION_MAP.find(c => c.targetKind === targetKind && c.field === field)?.bindable;
}
/**
* Check if an edge (by field name) is editable via drag-connect.
*/
export function isEditableEdge(field: string): boolean {
const entry = CONNECTION_MAP.find(c => c.field === field);
return entry ? !entry.nested : false;
return entry ? _isEditable(entry) : false;
}
/**
@@ -111,7 +253,7 @@ export function isEditableEdge(field: string): boolean {
*/
export function findConnection(targetKind: string, sourceKind: string, edgeType?: string): ConnectionEntry[] {
return CONNECTION_MAP.filter(c =>
!c.nested &&
_isEditable(c) &&
c.targetKind === targetKind &&
c.sourceKind === sourceKind &&
(!edgeType || c.edgeType === edgeType)
@@ -124,7 +266,7 @@ export function findConnection(targetKind: string, sourceKind: string, edgeType?
*/
export function getCompatibleInputs(sourceKind: string): CompatibleInput[] {
return CONNECTION_MAP
.filter(c => !c.nested && c.sourceKind === sourceKind)
.filter(c => _isEditable(c) && c.sourceKind === sourceKind)
.map(c => ({ targetKind: c.targetKind, field: c.field, edgeType: c.edgeType }));
}
@@ -132,9 +274,28 @@ export function getCompatibleInputs(sourceKind: string): CompatibleInput[] {
* Find the connection entry for a specific edge (by target kind and field).
*/
export function getConnectionByField(targetKind: string, field: string): ConnectionEntry | undefined {
return CONNECTION_MAP.find(c => c.targetKind === targetKind && c.field === field && !c.nested);
return CONNECTION_MAP.find(c => c.targetKind === targetKind && c.field === field && _isEditable(c));
}
/**
* Entity kinds whose per-entity PUT route validates the body against a Pydantic
* **discriminated union** (`Body(discriminator=...)`). Such a route 422s unless
* the body echoes the discriminator field, so a partial wiring write (just a
* reference field) is rejected outright. Maps the target kind → the
* discriminator's body-field name; the value is the target's *current* subtype,
* which we read back from the entity immediately before the write.
*
* Without this, drag-to-wire silently fails for nearly every source kind. Keep
* in sync with the backend `NODE_TYPE_FIELD` map in `api/graph_schema.py`.
*/
const _DISCRIMINATOR_FIELD: Readonly<Record<string, string>> = {
picture_source: 'stream_type',
audio_source: 'source_type',
value_source: 'source_type',
color_strip_source: 'source_type',
output_target: 'target_type',
};
/**
* Update a connection: set the reference field on the target entity.
* @param {string} targetId - The target entity's ID
@@ -144,11 +305,31 @@ export function getConnectionByField(targetKind: string, field: string): Connect
* @returns {Promise<boolean>} success
*/
export async function updateConnection(targetId: string, targetKind: string, field: string, newSourceId: string | null): Promise<boolean> {
const entry = CONNECTION_MAP.find(c => c.targetKind === targetKind && c.field === field && !c.nested);
if (!entry) return false;
const entry = CONNECTION_MAP.find(c => c.targetKind === targetKind && c.field === field && _isEditable(c));
if (!entry || !entry.endpoint) return false;
const url = entry.endpoint!.replace('{id}', targetId);
const body = { [field]: newSourceId };
const url = entry.endpoint.replace('{id}', targetId);
// For a bindable slot (`<parent>.source_id`) PUT `{ <parent>: { source_id } }`
// so the backend's `Bindable*.apply_update` merges and preserves the static
// value/colour. Top-level fields keep the flat `{ field: id }` shape.
const body: Record<string, unknown> = entry.bindable
? { [field.split('.')[0]]: { source_id: newSourceId || '' } }
: { [field]: newSourceId };
// Discriminated-union PUT routes reject a body without their discriminator.
// Echo the target's current subtype so a partial wiring write validates
// instead of 422-ing. Best-effort: a failed read leaves the PUT to fail as
// before — it never makes things worse.
const discrimField = _DISCRIMINATOR_FIELD[targetKind];
if (discrimField) {
try {
const current = await apiGet<Record<string, unknown>>(url);
const tag = current?.[discrimField];
if (typeof tag === 'string' && tag) body[discrimField] = tag;
} catch {
/* leave body as-is; the PUT below will surface any error */
}
}
try {
await apiPut(url, body);
@@ -167,4 +348,115 @@ export async function detachConnection(targetId: string, targetKind: string, fie
return updateConnection(targetId, targetKind, field, '');
}
/* ── List-element slots (composite layers / mapped zones) ──────────── */
/**
* Targets that hold *list* reference slots. Editing one element means
* re-PUTting the whole list, so we map the kind → its endpoint + cache.
* (Only color strip sources have list slots today: composite `layers`,
* mapped `zones`.)
*/
const _LIST_SLOT_TARGET: Readonly<Record<string, { endpoint: string; cache: { invalidate(): void } }>> = {
color_strip_source: { endpoint: '/color-strip-sources/{id}', cache: colorStripSourcesCache },
};
/**
* Re-wire a single element of a list reference slot — e.g. a composite
* `layers[index].source_id` or a mapped `zones[index].source_id`.
*
* The owning entity's PUT replaces the *entire* list, so this reads the entity
* back, copies every element verbatim, changes only `list[index][refField]`,
* and PUTs the full list (plus the discriminator). Echoing the existing element
* objects is what preserves each layer/zone's other settings (blend mode,
* opacity, LED range, per-layer brightness/template, …) — a naive partial write
* would silently drop that config.
*
* @param newSourceId New source id, or '' to clear (only valid for optional refs).
* @returns Promise<boolean> success
*/
export async function updateListSlotConnection(
targetId: string,
targetKind: string,
listField: string,
index: number,
refField: string,
newSourceId: string | null,
expectedCurrent?: string | null,
): Promise<boolean> {
const target = _LIST_SLOT_TARGET[targetKind];
if (!target || !Number.isInteger(index) || index < 0) return false;
const url = target.endpoint.replace('{id}', targetId);
try {
const current = await apiGet<Record<string, unknown>>(url);
const list = current?.[listField];
if (!Array.isArray(list) || index >= list.length) return false;
// Optimistic-concurrency guard: `index` is positional, so if the list was
// reordered/edited out-of-band (e.g. via the entity editor) between render
// and write — or between an action and its undo/redo — that index now points
// at a *different* element. Refuse rather than rewrite the wrong slot.
if (expectedCurrent != null) {
const el = list[index] as Record<string, unknown>;
const actual = typeof el?.[refField] === 'string' ? (el[refField] as string) : '';
if (actual !== expectedCurrent) return false;
}
// Copy every element; change only the one ref on the targeted element.
// (`|| ''` clears the ref — only valid for *optional* refs; the graph only
// re-wires the required `source_id`, so callers always pass a real id here.)
const nextList = list.map((el, i) =>
i === index
? { ...(el as Record<string, unknown>), [refField]: newSourceId || '' }
: { ...(el as Record<string, unknown>) },
);
const body: Record<string, unknown> = { [listField]: nextList };
// Discriminated-union PUT routes need the subtype echoed (see updateConnection).
const discrimField = _DISCRIMINATOR_FIELD[targetKind];
if (discrimField) {
const tag = current[discrimField];
if (typeof tag === 'string' && tag) body[discrimField] = tag;
}
await apiPut(url, body);
target.cache.invalidate();
return true;
} catch {
return false;
}
}
/* ── Subgraph duplication (D6) ─────────────────────────────────────── */
/** Result of `POST /graph/duplicate` (server-side clone of a selected subgraph). */
export interface DuplicateResult {
id_map: Record<string, string>;
created: Array<{ id: string; kind: string; name: string }>;
skipped: Array<{ id: string; reason: string }>;
warnings: Array<{ id: string; reason: string }>;
}
/**
* Server-side duplicate of a selected subgraph: the backend deep-clones the
* value / colour-strip sources among `nodeIds` with fresh ids and rewires
* references that point *within* the selection (shared deps stay shared).
* Returns the result, or `null` on failure. Invalidates the affected caches so
* a subsequent graph reload shows the clones.
*/
export async function duplicateSubgraph(
nodeIds: string[], nameSuffix?: string,
): Promise<DuplicateResult | null> {
try {
const body: Record<string, unknown> = { node_ids: nodeIds };
if (nameSuffix) body.name_suffix = nameSuffix;
const res = await apiPost<DuplicateResult>('/graph/duplicate', body);
valueSourcesCache.invalidate();
colorStripSourcesCache.invalidate();
return res;
} catch {
return null;
}
}
export { CONNECTION_MAP };
@@ -19,6 +19,9 @@ interface GraphEdge {
type: string;
field?: string;
editable?: boolean;
/** List-element reference (composite layer / mapped zone) — exposed as
* `data-slot-*` so the editor can re-wire just this slot. */
slot?: { list: string; index: number; ref: string };
points?: { x: number; y: number }[] | null;
fromNode?: GraphNodeRect;
toNode?: GraphNodeRect;
@@ -52,6 +55,43 @@ export function renderEdges(group: SVGGElement, edges: GraphEdge[]): void {
const path = _renderEdge(edge);
group.appendChild(path);
}
// Field labels rendered last so they sit above the paths. Hidden by
// default — revealed when zoomed in (`.show-labels`) or on highlight.
for (const edge of edges) {
const label = _renderEdgeLabel(edge);
if (label) group.appendChild(label);
}
}
/** Human-readable label for a reference field, e.g. `capture_template_id` → `capture template`. */
function _edgeFieldLabel(field: string): string {
return field.replace(/_id$/, '').replace(/\./g, ' ').replace(/_/g, ' ').trim();
}
/** Midpoint of the port-aware cubic bezier (its control points are horizontal
* offsets only, so the t=0.5 point is exactly the endpoint midpoint). */
function _edgeMidpoint(fromNode: GraphNodeRect, toNode: GraphNodeRect, fromPortY?: number, toPortY?: number): { x: number; y: number } {
const x1 = fromNode.x + fromNode.width;
const y1 = fromNode.y + (fromPortY ?? fromNode.height / 2);
const x2 = toNode.x;
const y2 = toNode.y + (toPortY ?? toNode.height / 2);
return { x: (x1 + x2) / 2, y: (y1 + y2) / 2 };
}
function _renderEdgeLabel(edge: GraphEdge): SVGElement | null {
if (!edge.field || !edge.fromNode || !edge.toNode) return null;
const mid = _edgeMidpoint(edge.fromNode, edge.toNode, edge.fromPortY, edge.toPortY);
const text = svgEl('text', {
class: `graph-edge-label graph-edge-label-${edge.type}`,
x: mid.x, y: mid.y - 4,
'text-anchor': 'middle',
'data-from': edge.from,
'data-to': edge.to,
'data-field': edge.field,
});
text.textContent = _edgeFieldLabel(edge.field);
return text;
}
function _createArrowMarker(type: string): SVGElement {
@@ -87,6 +127,12 @@ function _renderEdge(edge: GraphEdge): SVGElement {
'data-to': to,
'data-field': field || '',
});
// List-element reference: expose the slot so the editor can re-wire it.
if (edge.slot) {
path.setAttribute('data-slot-list', edge.slot.list);
path.setAttribute('data-slot-index', String(edge.slot.index));
path.setAttribute('data-slot-ref', edge.slot.ref);
}
// Tooltip
const title = svgEl('title');
@@ -263,6 +309,14 @@ export function updateEdgesForNode(group: SVGGElement, nodeId: string, nodeMap:
pathEl.setAttribute('d', d);
}
});
// Keep the field label pinned to the edge midpoint while dragging.
const mid = _edgeMidpoint(fromNode, toNode, edge.fromPortY, edge.toPortY);
group.querySelectorAll(`.graph-edge-label[data-from="${edge.from}"][data-to="${edge.to}"]`).forEach(lbl => {
if ((lbl.getAttribute('data-field') || '') === (edge.field || '')) {
lbl.setAttribute('x', String(mid.x));
lbl.setAttribute('y', String(mid.y - 4));
}
});
}
}
@@ -13,6 +13,8 @@ interface LayoutNode {
name: string;
subtype: string;
tags: string[];
icon?: string;
iconColor?: string;
running?: boolean;
x?: number;
y?: number;
@@ -27,12 +29,27 @@ interface LayoutEdge {
label: string;
type: string;
editable: boolean;
/** For list-element references (composite layers / mapped zones): which list,
* which element index, and the reference field on that element. Lets the
* editor re-wire one slot without disturbing its siblings. */
slot?: { list: string; index: number; ref: string };
}
interface LayoutResult {
nodes: Map<string, LayoutNode>;
edges: (LayoutEdge & { points: { x: number; y: number }[] | null; fromNode: LayoutNode; toNode: LayoutNode })[];
bounds: { x: number; y: number; width: number; height: number };
brokenRefs: BrokenRef[];
}
/** A reference field that points at an entity which no longer exists. */
export interface BrokenRef {
/** The missing (referenced) entity id. */
ref: string;
/** The id of the entity that still holds the dangling reference. */
by: string;
/** The reference field name on the referrer. */
field: string;
}
interface PortSet {
@@ -81,7 +98,7 @@ const ELK_OPTIONS = {
*/
export async function computeLayout(entities: EntitiesInput): Promise<LayoutResult> {
const elk = new ELK();
const { nodes: nodeList, edges: edgeList } = buildGraph(entities);
const { nodes: nodeList, edges: edgeList, brokenRefs } = buildGraph(entities);
const elkGraph = {
id: 'root',
@@ -151,7 +168,7 @@ export async function computeLayout(entities: EntitiesInput): Promise<LayoutResu
? { x: minX, y: minY, width: maxX - minX, height: maxY - minY }
: { x: 0, y: 0, width: 400, height: 300 };
return { nodes: nodeMap, edges, bounds };
return { nodes: nodeMap, edges, bounds, brokenRefs };
}
/* ── Entity color mapping ── */
@@ -207,97 +224,113 @@ function edgeType(fromKind: string, toKind: string, field: string): string {
/* ── Graph builder ── */
function buildGraph(e: EntitiesInput): { nodes: LayoutNode[]; edges: LayoutEdge[] } {
function buildGraph(e: EntitiesInput): { nodes: LayoutNode[]; edges: LayoutEdge[]; brokenRefs: BrokenRef[] } {
const nodes: LayoutNode[] = [];
const edges: LayoutEdge[] = [];
const brokenRefs: BrokenRef[] = [];
const nodeIds = new Set<string>();
// Index nodes by id so edge-building is O(1) instead of O(N) per edge.
const nodeByIdLocal = new Map<string, LayoutNode>();
function addNode(id: string, kind: string, name: string, subtype: string, extra: Record<string, any> = {}): void {
if (!id || nodeIds.has(id)) return;
nodeIds.add(id);
nodes.push({ id, kind, name: name || id, subtype: subtype || '', tags: extra.tags || [], ...extra });
const node = { id, kind, name: name || id, subtype: subtype || '', tags: extra.tags || [], ...extra };
nodes.push(node);
nodeByIdLocal.set(id, node);
}
function addEdge(from: string, to: string, field: string, label: string = ''): void {
if (!from || !to || !nodeIds.has(from) || !nodeIds.has(to)) return;
function addEdge(from: string, to: string, field: string, label: string = '', slot?: { list: string; index: number; ref: string }): void {
if (!from || !to) return;
// The referrer (`to`) is always a current entity in these loops; if the
// referenced entity (`from`) is missing, the reference is dangling —
// record it so the editor can surface a "broken reference" warning
// instead of silently dropping the edge (the old behaviour).
if (!nodeIds.has(from)) {
if (nodeIds.has(to)) brokenRefs.push({ ref: from, by: to, field });
return;
}
if (!nodeIds.has(to)) return;
const type = edgeType(
nodes.find(n => n.id === from)?.kind ?? '',
nodes.find(n => n.id === to)?.kind ?? '',
nodeByIdLocal.get(from)?.kind ?? '',
nodeByIdLocal.get(to)?.kind ?? '',
field
);
// Edges with dotted fields are nested (composite layers, zones, etc.) — not drag-editable
const editable = !field.includes('.');
edges.push({ from, to, field, label, type, editable });
edges.push({ from, to, field, label, type, editable, ...(slot ? { slot } : {}) });
}
// Every entity may carry a custom `icon` (+ `icon_color`); pass them through
// so node rendering can honour them (parity with custom node colours).
// 1. Devices
for (const d of e.devices || []) {
addNode(d.id, 'device', d.name, d.device_type, { tags: d.tags });
addNode(d.id, 'device', d.name, d.device_type, { tags: d.tags, icon: d.icon, iconColor: d.icon_color });
}
// 2. Capture templates
for (const t of e.captureTemplates || []) {
addNode(t.id, 'capture_template', t.name, t.engine_type, { tags: t.tags });
addNode(t.id, 'capture_template', t.name, t.engine_type, { tags: t.tags, icon: t.icon, iconColor: t.icon_color });
}
// 3. PP templates
for (const t of e.ppTemplates || []) {
addNode(t.id, 'pp_template', t.name, '', { tags: t.tags });
addNode(t.id, 'pp_template', t.name, '', { tags: t.tags, icon: t.icon, iconColor: t.icon_color });
}
// 4. Audio templates
for (const t of e.audioTemplates || []) {
addNode(t.id, 'audio_template', t.name, t.engine_type, { tags: t.tags });
addNode(t.id, 'audio_template', t.name, t.engine_type, { tags: t.tags, icon: t.icon, iconColor: t.icon_color });
}
// 5. Pattern templates
for (const t of e.patternTemplates || []) {
addNode(t.id, 'pattern_template', t.name, '', { tags: t.tags });
addNode(t.id, 'pattern_template', t.name, '', { tags: t.tags, icon: t.icon, iconColor: t.icon_color });
}
// 6. Sync clocks
for (const c of e.syncClocks || []) {
addNode(c.id, 'sync_clock', c.name, '', { running: c.is_running !== false, tags: c.tags });
addNode(c.id, 'sync_clock', c.name, '', { running: c.is_running !== false, tags: c.tags, icon: c.icon, iconColor: c.icon_color });
}
// 7. Picture sources
for (const s of e.pictureSources || []) {
addNode(s.id, 'picture_source', s.name, s.stream_type, { tags: s.tags });
addNode(s.id, 'picture_source', s.name, s.stream_type, { tags: s.tags, icon: s.icon, iconColor: s.icon_color });
}
// 8. Audio sources
for (const s of e.audioSources || []) {
addNode(s.id, 'audio_source', s.name, s.source_type, { tags: s.tags });
addNode(s.id, 'audio_source', s.name, s.source_type, { tags: s.tags, icon: s.icon, iconColor: s.icon_color });
}
// 9. Value sources
for (const s of e.valueSources || []) {
addNode(s.id, 'value_source', s.name, s.source_type, { tags: s.tags });
addNode(s.id, 'value_source', s.name, s.source_type, { tags: s.tags, icon: s.icon, iconColor: s.icon_color });
}
// 10. Color strip sources
for (const s of e.colorStripSources || []) {
addNode(s.id, 'color_strip_source', s.name, s.source_type, { tags: s.tags });
addNode(s.id, 'color_strip_source', s.name, s.source_type, { tags: s.tags, icon: s.icon, iconColor: s.icon_color });
}
// 11. Output targets
for (const t of e.outputTargets || []) {
addNode(t.id, 'output_target', t.name, t.target_type, { running: t.running || false, tags: t.tags });
addNode(t.id, 'output_target', t.name, t.target_type, { running: t.running || false, tags: t.tags, icon: t.icon, iconColor: t.icon_color });
}
// 12. Scene presets
for (const s of e.scenePresets || []) {
addNode(s.id, 'scene_preset', s.name, '', { tags: s.tags });
addNode(s.id, 'scene_preset', s.name, '', { tags: s.tags, icon: s.icon, iconColor: s.icon_color });
}
// 13. Automations
for (const a of e.automations || []) {
addNode(a.id, 'automation', a.name, '', { running: a.enabled || false, tags: a.tags });
addNode(a.id, 'automation', a.name, '', { running: a.enabled || false, tags: a.tags, icon: a.icon, iconColor: a.icon_color });
}
// 14. Color strip processing templates (CSPT)
for (const t of e.csptTemplates || []) {
addNode(t.id, 'cspt', t.name, '', { tags: t.tags });
addNode(t.id, 'cspt', t.name, '', { tags: t.tags, icon: t.icon, iconColor: t.icon_color });
}
// ── Edges ──
@@ -319,6 +352,21 @@ function buildGraph(e: EntitiesInput): { nodes: LayoutNode[]; edges: LayoutEdge[
for (const s of e.valueSources || []) {
if (s.audio_source_id) addEdge(s.audio_source_id, s.id, 'audio_source_id');
if (s.picture_source_id) addEdge(s.picture_source_id, s.id, 'picture_source_id');
// Derived value sources: gradient_map derives from an inner value source;
// css_extract derives from a color strip. Both are real, runtime-resolved
// references (and drag-editable) — render them so they're visible.
if (s.value_source_id) addEdge(s.value_source_id, s.id, 'value_source_id');
if (s.color_strip_source_id) addEdge(s.color_strip_source_id, s.id, 'color_strip_source_id');
// Template value sources reference one inner value source per bound input.
// Each `inputs[].value_source_id` is a real 'value' edge; the dotted field
// path marks it non-editable (drag-wiring deferred — see graph-connections).
if (s.source_type === 'template' && Array.isArray(s.inputs)) {
s.inputs.forEach((inp: any) => {
if (inp?.value_source_id) {
addEdge(inp.value_source_id, s.id, `inputs[${inp.name}].value_source_id`);
}
});
}
}
// Color strip source edges
@@ -331,19 +379,20 @@ function buildGraph(e: EntitiesInput): { nodes: LayoutNode[]; edges: LayoutEdge[
if (s.input_source_id) addEdge(s.input_source_id, s.id, 'input_source_id');
if (s.processing_template_id) addEdge(s.processing_template_id, s.id, 'processing_template_id');
// Composite layers
// Composite layers — carry the slot index so each `layer.source_id`
// edge can be re-wired individually from the graph (siblings untouched).
if (s.layers) {
for (const layer of s.layers) {
if (layer.source_id) addEdge(layer.source_id, s.id, 'layer.source_id');
s.layers.forEach((layer: any, i: number) => {
if (layer.source_id) addEdge(layer.source_id, s.id, 'layer.source_id', '', { list: 'layers', index: i, ref: 'source_id' });
if (layer.brightness_source_id) addEdge(layer.brightness_source_id, s.id, 'layer.brightness_source_id');
}
});
}
// Mapped zones
// Mapped zones — carry the slot index (re-wirable from the graph).
if (s.zones) {
for (const zone of s.zones) {
if (zone.source_id) addEdge(zone.source_id, s.id, 'zone.source_id');
}
s.zones.forEach((zone: any, i: number) => {
if (zone.source_id) addEdge(zone.source_id, s.id, 'zone.source_id', '', { list: 'zones', index: i, ref: 'source_id' });
});
}
// Advanced picture calibration lines
@@ -376,7 +425,6 @@ function buildGraph(e: EntitiesInput): { nodes: LayoutNode[]; edges: LayoutEdge[
if (bvsId) addEdge(bvsId, t.id, 'brightness.source_id');
const transVsId = bindableSourceId(t.transition);
if (transVsId) addEdge(transVsId, t.id, 'transition.source_id');
if (t.picture_source_id) addEdge(t.picture_source_id, t.id, 'picture_source_id');
// KC target settings
if (t.settings) {
if (t.settings.pattern_template_id) addEdge(t.settings.pattern_template_id, t.id, 'settings.pattern_template_id');
@@ -414,7 +462,7 @@ function buildGraph(e: EntitiesInput): { nodes: LayoutNode[]; edges: LayoutEdge[
if (d.default_css_processing_template_id) addEdge(d.default_css_processing_template_id, d.id, 'default_css_processing_template_id');
}
return { nodes, edges };
return { nodes, edges, brokenRefs };
}
/* ── Port computation ── */
@@ -6,6 +6,7 @@ import { ENTITY_COLORS, NODE_WIDTH, NODE_HEIGHT, computePorts } from './graph-la
import { EDGE_COLORS } from './graph-edges.ts';
import { createColorPicker, registerColorPicker, closeAllColorPickers } from './color-picker.ts';
import { getCardColor, setCardColor } from './card-colors.ts';
import { renderDeviceIconSvg } from './device-icons.ts';
import * as P from './icon-paths.ts';
const SVG_NS = 'http://www.w3.org/2000/svg';
@@ -22,6 +23,8 @@ interface GraphNode {
kind: string;
name: string;
subtype?: string;
icon?: string;
iconColor?: string;
x: number;
y: number;
width: number;
@@ -103,6 +106,7 @@ const SUBTYPE_ICONS = {
value_source: {
static: P.layoutDashboard, animated: P.refreshCw, audio: P.music,
adaptive_time: P.clock, adaptive_scene: P.cloudSun, daylight: P.sun,
template: P.code,
},
audio_source: { capture: P.volume2, processed: P.slidersHorizontal },
output_target: { led: P.lightbulb, wled: P.lightbulb, ha_light: P.lightbulb },
@@ -360,15 +364,29 @@ function renderNode(node: GraphNode, callbacks: NodeCallbacks): SVGElement {
}
}
// Entity icon (right side)
const iconPaths = (SUBTYPE_ICONS[kind]?.[subtype]) || KIND_ICONS[kind];
if (iconPaths) {
// Entity icon (right side). A custom per-entity icon wins over the
// kind/subtype default (parity with custom node colours); unknown icon ids
// yield '' so we fall back gracefully.
const customIconSvg = node.icon ? renderDeviceIconSvg(node.icon, { size: 16 }) : '';
if (customIconSvg) {
const iconG = svgEl('g', {
class: 'graph-node-icon',
transform: `translate(${width - 28}, ${height / 2 - 8}) scale(0.667)`,
class: 'graph-node-custom-icon',
transform: `translate(${width - 28}, ${height / 2 - 8})`,
});
iconG.innerHTML = iconPaths;
iconG.innerHTML = customIconSvg;
// The rendered SVG strokes with currentColor — tint via `color`.
if (node.iconColor) (iconG as unknown as SVGGElement).style.color = node.iconColor;
g.appendChild(iconG);
} else {
const iconPaths = (SUBTYPE_ICONS[kind]?.[subtype]) || KIND_ICONS[kind];
if (iconPaths) {
const iconG = svgEl('g', {
class: 'graph-node-icon',
transform: `translate(${width - 28}, ${height / 2 - 8}) scale(0.667)`,
});
iconG.innerHTML = iconPaths;
g.appendChild(iconG);
}
}
// Running dot
@@ -627,6 +645,39 @@ export function markOrphans(group: SVGGElement, nodeMap: Map<string, GraphNode>,
}
}
/**
* Mark nodes that have configuration issues (e.g. broken references, cycles).
* Adds a warning badge anchored to the node's top-left corner with a tooltip
* describing every problem. Call after `renderNodes`.
*/
export function markIssues(group: SVGGElement, issues: Map<string, string[]>): void {
// Clear previous markers so repeated calls don't stack badges.
group.querySelectorAll('.graph-node-issue').forEach(e => e.remove());
group.querySelectorAll('.graph-node.has-issue').forEach(n => n.classList.remove('has-issue'));
for (const [id, msgs] of issues) {
if (!msgs.length) continue;
const el = group.querySelector(`.graph-node[data-id="${CSS.escape(id)}"]`);
if (!el) continue;
el.classList.add('has-issue');
const badge = svgEl('g', { class: 'graph-node-issue' });
const icon = svgEl('g', { transform: 'translate(2, -9) scale(0.6)' });
icon.innerHTML = P.triangleAlert;
icon.setAttribute('fill', 'none');
icon.setAttribute('stroke', 'currentColor');
icon.setAttribute('stroke-width', '2.5');
icon.setAttribute('stroke-linecap', 'round');
icon.setAttribute('stroke-linejoin', 'round');
badge.appendChild(icon);
const tip = svgEl('title');
tip.textContent = msgs.join('\n');
badge.appendChild(tip);
el.appendChild(badge);
}
}
/**
* Update selection state on nodes.
*/
@@ -43,6 +43,7 @@ const _valueSourceTypeIcons = {
system_metrics: _svg(P.cpu),
game_event: _svg(P.gamepad2),
http: _svg(P.globe),
template: _svg(P.code),
};
const _audioSourceTypeIcons = { capture: _svg(P.volume2), processed: _svg(P.slidersHorizontal) };
const _deviceTypeIcons = {
@@ -0,0 +1,191 @@
/**
* Tiny zero-dependency Jinja-expression highlighter.
*
* A transparent <textarea> is layered over a synced <pre class="jinja-hl">.
* On every input the text is re-tokenised and painted into the <pre> so the
* caret and selection stay native while the colours live underneath. The two
* layers share identical font metrics (set in CSS via --font-mono) so the
* highlight aligns pixel-perfectly with the typed glyphs.
*
* Tokenised: strings, numbers, the sandbox globals (min|max|abs|round|clamp),
* the `raw` keyword, bound input variable names (supplied live via
* getInputNames), and operators. Everything else renders as plain text.
*
* Usage:
* const ed = create({ textarea, getInputNames: () => ['audio','temp'], onChange });
* ed.refresh(); // re-paint after the input list or value changes externally
* ed.destroy();
*/
/** Globals available inside the sandboxed expression (see backend contract). */
const JINJA_GLOBALS = new Set(['min', 'max', 'abs', 'round', 'clamp']);
const JINJA_RAW = 'raw';
export interface JinjaEditorOpts {
textarea: HTMLTextAreaElement;
getInputNames: () => string[];
onChange?: (value: string) => void;
}
export interface JinjaEditorHandle {
/** Re-paint the highlight layer (e.g. after the bound-input list changed). */
refresh: () => void;
/** Detach listeners and remove the highlight overlay. */
destroy: () => void;
}
function escapeHtml(s: string): string {
return s
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;');
}
type TokenKind = 'str' | 'num' | 'fn' | 'raw' | 'var' | 'op' | 'text';
interface Token {
kind: TokenKind;
text: string;
}
/**
* Tokenise a Jinja expression. Deliberately small — this is presentational
* only; the backend is the source of truth for validity.
*/
function tokenize(src: string, inputNames: Set<string>): Token[] {
const tokens: Token[] = [];
const n = src.length;
let i = 0;
const push = (kind: TokenKind, text: string) => {
// Coalesce consecutive plain-text runs to keep the DOM tiny.
const last = tokens[tokens.length - 1];
if (kind === 'text' && last && last.kind === 'text') last.text += text;
else tokens.push({ kind, text });
};
while (i < n) {
const ch = src[i];
// Strings — single or double quoted, with simple escape passthrough.
if (ch === '"' || ch === "'") {
const quote = ch;
let j = i + 1;
while (j < n && src[j] !== quote) {
if (src[j] === '\\' && j + 1 < n) j += 2;
else j += 1;
}
j = Math.min(j + 1, n); // include closing quote if present
push('str', src.slice(i, j));
i = j;
continue;
}
// Numbers — integer / float.
if (ch >= '0' && ch <= '9') {
let j = i + 1;
while (j < n && /[0-9._]/.test(src[j])) j += 1;
push('num', src.slice(i, j));
i = j;
continue;
}
// Identifiers — globals, the `raw` keyword, bound input names, or plain.
if (/[A-Za-z_]/.test(ch)) {
let j = i + 1;
while (j < n && /[A-Za-z0-9_]/.test(src[j])) j += 1;
const word = src.slice(i, j);
if (JINJA_GLOBALS.has(word)) push('fn', word);
else if (word === JINJA_RAW) push('raw', word);
else if (inputNames.has(word)) push('var', word);
else push('text', word);
i = j;
continue;
}
// Operators / punctuation.
if ('+-*/%()[]<>=!,&|?:'.includes(ch)) {
push('op', ch);
i += 1;
continue;
}
// Whitespace and everything else.
push('text', ch);
i += 1;
}
return tokens;
}
function render(src: string, inputNames: Set<string>): string {
// A trailing newline is swallowed by <pre>; pad it so the highlight box
// keeps the same height as the textarea while typing a fresh line.
const padded = src.endsWith('\n') ? src + ' ' : src;
const html = tokenize(padded, inputNames)
.map(tok =>
tok.kind === 'text'
? escapeHtml(tok.text)
: `<span class="tok-${tok.kind}">${escapeHtml(tok.text)}</span>`,
)
.join('');
return html;
}
export function create({ textarea, getInputNames, onChange }: JinjaEditorOpts): JinjaEditorHandle {
// Wrap the textarea so the highlight layer can sit directly behind it.
const wrap = document.createElement('div');
wrap.className = 'jinja-editor';
const pre = document.createElement('pre');
pre.className = 'jinja-hl';
pre.setAttribute('aria-hidden', 'true');
const parent = textarea.parentNode;
if (parent) {
parent.insertBefore(wrap, textarea);
wrap.appendChild(pre);
wrap.appendChild(textarea);
}
textarea.classList.add('jinja-input');
textarea.spellcheck = false;
textarea.setAttribute('autocomplete', 'off');
textarea.setAttribute('autocapitalize', 'off');
textarea.setAttribute('autocorrect', 'off');
textarea.setAttribute('wrap', 'off');
const paint = () => {
pre.innerHTML = render(textarea.value, new Set(getInputNames()));
// Keep the highlight scrolled in lock-step with the textarea.
pre.scrollTop = textarea.scrollTop;
pre.scrollLeft = textarea.scrollLeft;
};
const onInput = () => {
paint();
if (onChange) onChange(textarea.value);
};
const onScroll = () => {
pre.scrollTop = textarea.scrollTop;
pre.scrollLeft = textarea.scrollLeft;
};
textarea.addEventListener('input', onInput);
textarea.addEventListener('scroll', onScroll);
paint();
return {
refresh: paint,
destroy: () => {
textarea.removeEventListener('input', onInput);
textarea.removeEventListener('scroll', onScroll);
textarea.classList.remove('jinja-input');
// Restore the textarea to its original place, drop the overlay.
if (wrap.parentNode) {
wrap.parentNode.insertBefore(textarea, wrap);
wrap.remove();
}
},
};
}
@@ -5,7 +5,7 @@
import {
_discoveryScanRunning, set_discoveryScanRunning,
_discoveryCache, set_discoveryCache,
csptCache,
csptCache, mqttSourcesCache,
} from '../core/state.ts';
import { API_BASE, isSerialDevice, isMockDevice, isMqttDevice, isWsDevice, isOpenrgbDevice, isDmxDevice, isDdpDevice, isOpcDevice, isEspnowDevice, isHueDevice, isYeelightDevice, isWizDevice, isLifxDevice, isGoveeDevice, isNanoleafDevice, isBleDevice, isUsbhidDevice, isSpiDevice, isChromaDevice, isGameSenseDevice, isGroupDevice, escapeHtml } from '../core/api.ts';
import { apiGet, apiPost } from '../core/api-client.ts';
@@ -18,6 +18,7 @@ import { runPairingFlow, PairingCancelled } from './pairing-flow.ts';
import { getDeviceTypeIcon, ICON_RADIO, ICON_GLOBE, ICON_CPU, ICON_KEYBOARD, ICON_MOUSE, ICON_HEADPHONES, ICON_PLUG, ICON_TARGET_ICON, ICON_ACTIVITY, ICON_TEMPLATE, ICON_CHEVRON_UP, ICON_CHEVRON_DOWN, ICON_PLUS, ICON_TRASH, ICON_GIT_MERGE, ICON_COPY, ICON_BLUETOOTH, ICON_LIGHTBULB, ICON_SPARKLES, ICON_PALETTE } from '../core/icons.ts';
import { EntitySelect, EntityPalette } from '../core/entity-palette.ts';
import { IconSelect, showTypePicker } from '../core/icon-select.ts';
import type { MQTTSource } from '../types.ts';
class AddDeviceModal extends Modal {
constructor() { super('add-device-modal'); }
@@ -44,6 +45,7 @@ class AddDeviceModal extends Modal {
opcChannel: (document.getElementById('device-opc-channel') as HTMLInputElement)?.value || '0',
bleFamily: (document.getElementById('device-ble-family') as HTMLSelectElement)?.value || '',
bleGoveeKey: (document.getElementById('device-ble-govee-key') as HTMLInputElement)?.value || '',
mqttSource: (document.getElementById('device-mqtt-source') as HTMLSelectElement)?.value || '',
yeelightMinInterval: (document.getElementById('device-yeelight-min-interval') as HTMLInputElement)?.value || '500',
wizMinInterval: (document.getElementById('device-wiz-min-interval') as HTMLInputElement)?.value || '50',
lifxMinInterval: (document.getElementById('device-lifx-min-interval') as HTMLInputElement)?.value || '50',
@@ -72,6 +74,7 @@ function _buildDeviceTypeItems() {
let _deviceTypeIconSelect: any = null;
let _csptEntitySelect: any = null;
let _mqttSourceEntitySelect: any = null;
function _ensureDeviceTypeIconSelect() {
const sel = document.getElementById('device-type');
@@ -104,6 +107,36 @@ function _ensureCsptEntitySelect() {
}
}
// MQTT broker picker for device_type=mqtt. Empty value = first available broker.
function _ensureMqttSourceSelect() {
const sel = document.getElementById('device-mqtt-source') as HTMLSelectElement | null;
if (!sel) return;
const sources: MQTTSource[] = mqttSourcesCache.data || [];
const current = sel.value;
sel.innerHTML = `<option value="">${t('device.mqtt_source.none')}</option>` +
sources.map((s: MQTTSource) => `<option value="${escapeHtml(s.id)}">${escapeHtml(s.name)}</option>`).join('');
sel.value = current;
if (_mqttSourceEntitySelect) _mqttSourceEntitySelect.destroy();
_mqttSourceEntitySelect = new EntitySelect({
target: sel,
getItems: () => (mqttSourcesCache.data || []).map((s: MQTTSource) => ({
value: s.id,
label: s.name,
icon: ICON_PALETTE,
desc: `${s.broker_host}:${s.broker_port}`,
})),
placeholder: t('palette.search'),
allowNone: true,
noneLabel: t('device.mqtt_source.none'),
} as any);
}
function _showMqttSourceField(show: boolean) {
const group = document.getElementById('device-mqtt-source-group') as HTMLElement | null;
if (group) group.style.display = show ? '' : 'none';
if (show) mqttSourcesCache.fetch().then(() => _ensureMqttSourceSelect());
}
/* ── Icon-grid DMX protocol selector ─────────────────────────── */
function _buildDmxProtocolItems() {
@@ -297,6 +330,7 @@ export function onDeviceTypeChanged() {
_showGameSenseFields(false);
_showGroupFields(false);
_showOpcFields(false);
_showMqttSourceField(false);
if (isMqttDevice(deviceType)) {
// MQTT: show URL (topic), LED count; hide serial/baud/led-type/latency/discovery
@@ -314,6 +348,7 @@ export function onDeviceTypeChanged() {
if (urlLabel) urlLabel.textContent = t('device.mqtt_topic');
if (urlHint) urlHint.textContent = t('device.mqtt_topic.hint');
urlInput.placeholder = t('device.mqtt_topic.placeholder') || 'mqtt://ledgrab/device/living-room';
_showMqttSourceField(true);
} else if (isMockDevice(deviceType)) {
urlGroup.style.display = 'none';
urlInput.removeAttribute('required');
@@ -920,6 +955,14 @@ export function showAddDevice(presetType: any = null, cloneData: any = null) {
if (goveeKeyEl && cloneData.ble_govee_key) goveeKeyEl.value = cloneData.ble_govee_key;
_updateBleGoveeKeyVisibility();
}
// Prefill MQTT broker (after the source cache loads)
if (isMqttDevice(presetType) && cloneData.mqtt_source_id) {
mqttSourcesCache.fetch().then(() => {
const msEl = document.getElementById('device-mqtt-source') as HTMLSelectElement;
if (msEl) msEl.value = cloneData.mqtt_source_id;
_ensureMqttSourceSelect();
});
}
// Prefill DMX fields
if (isDmxDevice(presetType)) {
const dmxProto = document.getElementById('device-dmx-protocol') as HTMLSelectElement;
@@ -1217,6 +1260,10 @@ export async function handleAddDevice(event: any) {
const goveeKey = (document.getElementById('device-ble-govee-key') as HTMLInputElement)?.value?.trim();
if (goveeKey) body.ble_govee_key = goveeKey;
}
if (isMqttDevice(deviceType)) {
const mqttSource = (document.getElementById('device-mqtt-source') as HTMLSelectElement)?.value || '';
if (mqttSource) body.mqtt_source_id = mqttSource;
}
if (isSpiDevice(deviceType)) {
body.spi_speed_hz = parseInt((document.getElementById('device-spi-speed') as HTMLInputElement)?.value || '800000', 10);
body.spi_led_type = (document.getElementById('device-spi-led-type') as HTMLSelectElement)?.value || 'WS2812B';
@@ -4,7 +4,7 @@
import {
_deviceBrightnessCache, updateDeviceBrightness,
csptCache,
csptCache, mqttSourcesCache,
} from '../core/state.ts';
import { API_BASE, getHeaders, escapeHtml, isSerialDevice, isMockDevice, isMqttDevice, isWsDevice, isOpenrgbDevice, isDmxDevice, isDdpDevice, isOpcDevice, isYeelightDevice, isWizDevice, isLifxDevice, isGoveeDevice, isNanoleafDevice, isBleDevice, isGroupDevice } from '../core/api.ts';
import { apiGet, apiPost, apiPut, apiDelete } from '../core/api-client.ts';
@@ -13,18 +13,19 @@ import { _fetchOpenrgbZones, _getCheckedZones, _splitOpenrgbZone, _getZoneMode,
import { t } from '../core/i18n.ts';
import { showToast, showConfirm, desktopFocus } from '../core/ui.ts';
import { Modal } from '../core/modal.ts';
import { ICON_SETTINGS, ICON_STOP_PLAIN, ICON_LED, ICON_WEB, ICON_REFRESH, ICON_TEMPLATE } from '../core/icons.ts';
import { ICON_SETTINGS, ICON_STOP_PLAIN, ICON_LED, ICON_WEB, ICON_REFRESH, ICON_TEMPLATE, ICON_PALETTE } from '../core/icons.ts';
import { wrapCard } from '../core/card-colors.ts';
import type { ModCardOpts, LedState, ModMetricOpts, ModChipOpts, ModBtnOpts, ModMenuItemOpts } from '../core/mod-card.ts';
import { TagInput, renderTagChips } from '../core/tag-input.ts';
import { EntitySelect } from '../core/entity-palette.ts';
import { getBaseOrigin } from './settings.ts';
import type { Device } from '../types.ts';
import type { Device, MQTTSource } from '../types.ts';
import { renderDeviceIconSvg } from '../core/device-icons.ts';
import { ICON_EDIT } from '../core/icons.ts';
let _deviceTagsInput: any = null;
let _settingsCsptEntitySelect: any = null;
let _settingsMqttEntitySelect: any = null;
/* The General Settings modal groups its many conditional fields into
four `.ds-section` panels (Identity / Connection / Hardware / Behavior).
@@ -73,6 +74,29 @@ function _ensureSettingsCsptSelect() {
}
}
// MQTT broker picker for the settings modal (device_type=mqtt). Empty = first available.
function _ensureSettingsMqttSelect(selectedId: string = '') {
const sel = document.getElementById('settings-mqtt-source') as HTMLSelectElement | null;
if (!sel) return;
const sources: MQTTSource[] = mqttSourcesCache.data || [];
sel.innerHTML = `<option value="">${t('device.mqtt_source.none')}</option>` +
sources.map((s: MQTTSource) => `<option value="${escapeHtml(s.id)}">${escapeHtml(s.name)}</option>`).join('');
sel.value = selectedId || '';
if (_settingsMqttEntitySelect) _settingsMqttEntitySelect.destroy();
_settingsMqttEntitySelect = new EntitySelect({
target: sel,
getItems: () => (mqttSourcesCache.data || []).map((s: MQTTSource) => ({
value: s.id,
label: s.name,
icon: ICON_PALETTE,
desc: `${s.broker_host}:${s.broker_port}`,
})),
placeholder: t('palette.search'),
allowNone: true,
noneLabel: t('device.mqtt_source.none'),
} as any);
}
class DeviceSettingsModal extends Modal {
constructor() { super('device-settings-modal'); }
@@ -103,6 +127,7 @@ class DeviceSettingsModal extends Modal {
goveeMinInterval: (document.getElementById('settings-govee-min-interval') as HTMLInputElement | null)?.value || '50',
nanoleafMinInterval: (document.getElementById('settings-nanoleaf-min-interval') as HTMLInputElement | null)?.value || '100',
csptId: (document.getElementById('settings-css-processing-template') as HTMLSelectElement | null)?.value || '',
mqttSource: (document.getElementById('settings-mqtt-source') as HTMLSelectElement | null)?.value || '',
};
}
@@ -439,6 +464,8 @@ export async function showSettings(deviceId: any) {
const urlLabel = urlGroup.querySelector('label[for="settings-device-url"]') as HTMLElement | null;
const urlHint = urlGroup.querySelector('.input-hint') as HTMLElement | null;
const urlInput = document.getElementById('settings-device-url') as HTMLInputElement;
const mqttSourceGroup = document.getElementById('settings-mqtt-source-group') as HTMLElement | null;
if (mqttSourceGroup) mqttSourceGroup.style.display = 'none';
if (isMock || isWs || isGroup) {
urlGroup.style.display = 'none';
urlInput.removeAttribute('required');
@@ -458,6 +485,7 @@ export async function showSettings(deviceId: any) {
if (urlLabel) urlLabel.textContent = t('device.mqtt_topic');
if (urlHint) urlHint.textContent = t('device.mqtt_topic.hint');
urlInput.placeholder = t('device.mqtt_topic.placeholder') || 'mqtt://ledgrab/device/living-room';
if (mqttSourceGroup) mqttSourceGroup.style.display = '';
} else if (isOpenrgbDevice(device.device_type)) {
if (urlLabel) urlLabel.textContent = t('device.openrgb.url');
if (urlHint) urlHint.textContent = t('device.openrgb.url.hint');
@@ -805,6 +833,13 @@ export async function showSettings(deviceId: any) {
const csptSel = document.getElementById('settings-css-processing-template') as HTMLSelectElement | null;
if (csptSel) csptSel.value = device.default_css_processing_template_id || '';
// MQTT broker selector (mqtt devices only) — populated before snapshot
// so the dirty-check baseline matches the current broker.
if (isMqtt) {
await mqttSourcesCache.fetch();
_ensureSettingsMqttSelect(device.mqtt_source_id || '');
}
_updateSettingsSectionVisibility();
settingsModal.snapshot();
settingsModal.open();
@@ -819,7 +854,7 @@ export async function showSettings(deviceId: any) {
}
export function isSettingsDirty() { return settingsModal.isDirty(); }
export function forceCloseDeviceSettingsModal() { if (_deviceTagsInput) { _deviceTagsInput.destroy(); _deviceTagsInput = null; } if (_settingsCsptEntitySelect) { _settingsCsptEntitySelect.destroy(); _settingsCsptEntitySelect = null; } destroyBleFamilyIconSelect('settings-ble-family'); destroyDdpColorOrderIconSelect('settings-ddp-color-order'); destroyDmxProtocolIconSelect('settings-dmx-protocol'); settingsModal.forceClose(); }
export function forceCloseDeviceSettingsModal() { if (_deviceTagsInput) { _deviceTagsInput.destroy(); _deviceTagsInput = null; } if (_settingsCsptEntitySelect) { _settingsCsptEntitySelect.destroy(); _settingsCsptEntitySelect = null; } if (_settingsMqttEntitySelect) { _settingsMqttEntitySelect.destroy(); _settingsMqttEntitySelect = null; } destroyBleFamilyIconSelect('settings-ble-family'); destroyDdpColorOrderIconSelect('settings-ddp-color-order'); destroyDmxProtocolIconSelect('settings-dmx-protocol'); settingsModal.forceClose(); }
export function closeDeviceSettingsModal() { settingsModal.close(); }
export async function saveDeviceSettings() {
@@ -908,6 +943,9 @@ export async function saveDeviceSettings() {
const goveeKey = (document.getElementById('settings-ble-govee-key') as HTMLInputElement | null)?.value?.trim() || '';
body.ble_govee_key = goveeKey;
}
if (isMqttDevice(settingsModal.deviceType)) {
body.mqtt_source_id = (document.getElementById('settings-mqtt-source') as HTMLSelectElement | null)?.value || '';
}
if (isGroup) {
const childRows = document.querySelectorAll('#settings-group-children-list .group-child-row') as NodeListOf<HTMLElement>;
body.group_device_ids = Array.from(childRows).map(r => r.dataset.deviceId || '').filter(v => v !== '');
@@ -4,7 +4,8 @@
import { GraphCanvas } from '../core/graph-canvas.ts';
import { computeLayout, computePorts, ENTITY_COLORS, ENTITY_LABELS } from '../core/graph-layout.ts';
import { renderNodes, patchNodeRunning, highlightNode, markOrphans, updateSelection, getNodeDisplayColor } from '../core/graph-nodes.ts';
import type { BrokenRef } from '../core/graph-layout.ts';
import { renderNodes, patchNodeRunning, highlightNode, markOrphans, markIssues, updateSelection, getNodeDisplayColor } from '../core/graph-nodes.ts';
import { renderEdges, highlightChain, clearEdgeHighlights, updateEdgesForNode, renderFlowDots, updateFlowDotsForNode } from '../core/graph-edges.ts';
import {
devicesCache, captureTemplatesCache, ppTemplatesCache,
@@ -14,13 +15,14 @@ import {
automationsCacheObj, csptCache,
} from '../core/state.ts';
import { fetchWithAuth, fetchMetricsHistory } from '../core/api.ts';
import { apiGet } from '../core/api-client.ts';
import { showToast, showConfirm, formatUptime, formatCompact } from '../core/ui.ts';
import { createFpsSparkline } from '../core/chart-utils.ts';
import { t } from '../core/i18n.ts';
import { findConnection, getCompatibleInputs, getConnectionByField, updateConnection, detachConnection, isEditableEdge } from '../core/graph-connections.ts';
import { findConnection, getCompatibleInputs, getConnectionByField, updateConnection, detachConnection, updateListSlotConnection, duplicateSubgraph, isEditableEdge, validateConnection, getDependents, checkSchemaDrift } from '../core/graph-connections.ts';
import { showTypePicker } from '../core/icon-select.ts';
import * as P from '../core/icon-paths.ts';
import { readJson, isObject, isString, isNumber } from '../core/storage.ts';
import { readJson, writeJson, isObject, isString, isNumber } from '../core/storage.ts';
import { logError } from '../core/log.ts';
// Local type guard for AnchoredRect — `JSON.parse` returns unknown and the
@@ -122,8 +124,33 @@ let _dragState: DragState | null = null;
let _justDragged = false;
let _dragListenersAdded = false;
// Manual position overrides (persisted in memory; cleared on relayout)
let _manualPositions: Map<string, { x: number; y: number }> = new Map();
// Manual position overrides persisted to localStorage so a hand-arranged
// layout survives page reloads; cleared explicitly on relayout.
const _POS_KEY = 'graph_node_positions';
function _isPositionMap(v: unknown): v is Record<string, { x: number; y: number }> {
if (!isObject(v)) return false;
return Object.values(v).every(p => isObject(p) && isNumber((p as any).x) && isNumber((p as any).y));
}
function _loadManualPositions(): Map<string, { x: number; y: number }> {
const obj = readJson(_POS_KEY, _isPositionMap);
const map = new Map<string, { x: number; y: number }>();
if (obj) for (const [id, p] of Object.entries(obj)) map.set(id, { x: p.x, y: p.y });
return map;
}
function _saveManualPositions(): void {
const obj: Record<string, { x: number; y: number }> = {};
for (const [id, p] of _manualPositions) obj[id] = p;
writeJson(_POS_KEY, obj);
}
let _manualPositions: Map<string, { x: number; y: number }> = _loadManualPositions();
// Node IDs that currently have a configuration issue (broken ref / cycle).
let _issueIds: Set<string> = new Set();
// Dangling references discovered during the last layout build.
let _brokenRefs: BrokenRef[] = [];
// Raw fetched entities by id — lets drop-resolution check which bindable slots
// a target actually has (subtype-safe), without re-fetching.
let _entitiesById: Map<string, any> = new Map();
// Rubber-band selection state
interface RubberBandState { startGraph: { x: number; y: number }; startClient: { x: number; y: number }; active: boolean; }
@@ -331,9 +358,18 @@ export async function loadGraphEditor(): Promise<void> {
if (gc) gc.appendChild(overlay);
}
// Dev safety net: warn once if the frontend connection map drifts from the
// backend schema (fire-and-forget; never blocks the graph).
void checkSchemaDrift();
try {
const entities = await _fetchAllEntities();
const { nodes, edges, bounds } = await computeLayout(entities);
// Index raw entities by id for subtype-safe bindable-slot resolution.
_entitiesById = new Map();
for (const arr of Object.values(entities)) {
if (Array.isArray(arr)) for (const ent of arr) if (ent && ent.id) _entitiesById.set(ent.id, ent);
}
const { nodes, edges, bounds, brokenRefs } = await computeLayout(entities);
// Apply manual position overrides from previous drag operations
_applyManualPositions(nodes, edges);
@@ -341,6 +377,7 @@ export async function loadGraphEditor(): Promise<void> {
computePorts(nodes as any, edges);
_nodeMap = nodes as any;
_edges = edges;
_brokenRefs = brokenRefs;
_bounds = _calcBounds(nodes);
_renderGraph(container);
} finally {
@@ -661,9 +698,181 @@ export async function graphRelayout(): Promise<void> {
if (!ok) return;
}
_manualPositions.clear();
_saveManualPositions();
await loadGraphEditor();
}
/* ── Health overlay (broken references + dependency cycles) ── */
/** Humanize a reference field name for display (e.g. `capture_template_id` → `capture template`). */
function _humanField(field: string): string {
return field.replace(/_id$/, '').replace(/\./g, ' ').replace(/_/g, ' ').trim();
}
/**
* Detect every node that participates in a directed dependency cycle.
* Iterative DFS with an explicit path stack (no deep recursion).
*/
function _detectCycles(nodeMap: Map<string, any>, edges: any[]): Set<string> {
const adj = new Map<string, string[]>();
for (const e of edges) {
if (!adj.has(e.from)) adj.set(e.from, []);
adj.get(e.from)!.push(e.to);
}
const WHITE = 0, GRAY = 1, BLACK = 2;
const color = new Map<string, number>();
const inCycle = new Set<string>();
for (const start of nodeMap.keys()) {
if (color.get(start)) continue; // already GRAY/BLACK
const stack: Array<{ id: string; i: number }> = [{ id: start, i: 0 }];
const path: string[] = [start];
color.set(start, GRAY);
while (stack.length) {
const frame = stack[stack.length - 1];
const neighbors = adj.get(frame.id) || [];
if (frame.i < neighbors.length) {
const v = neighbors[frame.i++];
const c = color.get(v) ?? WHITE;
if (c === GRAY) {
// Back edge → mark the whole cycle from v to the current node.
const idx = path.indexOf(v);
if (idx >= 0) for (let k = idx; k < path.length; k++) inCycle.add(path[k]);
} else if (c === WHITE) {
color.set(v, GRAY);
path.push(v);
stack.push({ id: v, i: 0 });
}
} else {
color.set(frame.id, BLACK);
path.pop();
stack.pop();
}
}
}
return inCycle;
}
/** Build the per-node issue map, render badges, and update the toolbar button. */
function _computeAndMarkIssues(nodeGroup: SVGGElement): void {
const issues = new Map<string, string[]>();
const add = (id: string, msg: string): void => {
const list = issues.get(id);
if (list) list.push(msg); else issues.set(id, [msg]);
};
// Dangling references: the referrer still exists but its target is gone.
for (const br of _brokenRefs) {
add(br.by, t('graph.issue.broken_ref', { field: _humanField(br.field) }));
}
// Dependency cycles.
if (_nodeMap && _edges) {
for (const id of _detectCycles(_nodeMap, _edges)) add(id, t('graph.issue.cycle'));
}
_issueIds = new Set(issues.keys());
markIssues(nodeGroup, issues);
_updateIssuesButton();
}
function _updateIssuesButton(): void {
const btn = document.getElementById('graph-issues-btn');
if (!btn) return;
const count = _issueIds.size;
const countEl = btn.querySelector('.graph-issues-count');
if (countEl) countEl.textContent = count > 0 ? String(count) : '';
(btn as HTMLElement).style.display = count > 0 ? '' : 'none';
}
/**
* Export the full wiring topology (nodes + edges + validation report) as a
* downloadable JSON file. This is the read-only half of "wiring blueprints":
* a shareable, inspectable snapshot. Re-importing/instantiating a blueprint
* (with id remapping) is a separate, larger feature.
*/
export async function graphExportTopology(): Promise<void> {
try {
const topo = await apiGet<unknown>('/graph');
const blob = new Blob([JSON.stringify(topo, null, 2)], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = `ledgrab-graph-${new Date().toISOString().slice(0, 10)}.json`;
document.body.appendChild(a);
a.click();
a.remove();
URL.revokeObjectURL(url);
showToast(t('graph.export_done'), 'success');
} catch (e) {
showToast(e instanceof Error ? e.message : t('graph.export_failed'), 'error');
}
}
/**
* Duplicate the current node selection server-side. The backend clones the
* value / colour-strip sources in the selection (full config preserved, no
* secrets crossing the wire), rewires references that point *within* the
* selection, and shares everything else. The graph then reloads with the new
* cluster selected.
*/
export async function graphDuplicateSelection(): Promise<void> {
const ids = [..._selectedIds];
if (ids.length === 0) {
showToast(t('graph.duplicate_none'), 'info');
return;
}
const res = await duplicateSubgraph(ids);
if (!res) {
showToast(t('graph.duplicate_failed'), 'error');
return;
}
const newIds = Object.values(res.id_map || {});
if (newIds.length === 0) {
showToast(t('graph.duplicate_none_eligible'), 'info');
return;
}
await loadGraphEditor();
_selectedIds = new Set(newIds);
const nodeGroup = document.querySelector('.graph-nodes') as SVGGElement | null;
if (nodeGroup) updateSelection(nodeGroup, _selectedIds);
const hasWarn = !!res.warnings?.length;
showToast(
t(hasWarn ? 'graph.duplicate_done_warn' : 'graph.duplicate_done', { count: newIds.length }),
hasWarn ? 'info' : 'success',
);
}
/** Frame and highlight all nodes flagged with configuration issues. */
export function graphShowIssues(): void {
if (_issueIds.size === 0 || !_nodeMap || !_canvas) {
showToast(t('graph.issues_none'), 'info');
return;
}
const ng = document.querySelector('.graph-nodes') as SVGGElement | null;
const eg = document.querySelector('.graph-edges') as SVGGElement | null;
_selectedIds = new Set(_issueIds);
if (ng) {
updateSelection(ng, _selectedIds);
ng.querySelectorAll('.graph-node').forEach((n: any) => {
n.style.opacity = _issueIds.has(n.getAttribute('data-id')) ? '1' : '0.2';
});
}
if (eg) clearEdgeHighlights(eg);
// Fit the viewport to the bounding box of the flagged nodes.
let minX = Infinity, minY = Infinity, maxX = -Infinity, maxY = -Infinity;
for (const id of _issueIds) {
const n = _nodeMap.get(id);
if (!n) continue;
minX = Math.min(minX, n.x); minY = Math.min(minY, n.y);
maxX = Math.max(maxX, n.x + n.width); maxY = Math.max(maxY, n.y + n.height);
}
if (minX !== Infinity) {
_canvas.fitAll({ x: minX, y: minY, width: maxX - minX, height: maxY - minY }, true);
}
}
// Entity kind → window function to open add/create modal + icon path
const _ico = (d: string): string => `<svg class="icon" viewBox="0 0 24 24">${d}</svg>`;
const _w = window as any;
@@ -693,6 +902,25 @@ const ALL_CACHES = [
automationsCacheObj, csptCache,
];
// entity kind → its DataCache, so a kind-scoped watcher (create-and-connect)
// only reacts to new entities of the expected kind, never an unrelated one.
const KIND_CACHE: Record<string, { data?: any[]; subscribe(fn: (d: any) => void): void; unsubscribe(fn: (d: any) => void): void }> = {
device: devicesCache,
capture_template: captureTemplatesCache,
pp_template: ppTemplatesCache,
audio_template: audioTemplatesCache,
pattern_template: patternTemplatesCache,
picture_source: streamsCache,
audio_source: audioSourcesCache,
value_source: valueSourcesCache,
color_strip_source: colorStripSourcesCache,
sync_clock: syncClocksCache,
output_target: outputTargetsCache,
scene_preset: scenePresetsCache,
automation: automationsCacheObj,
cspt: csptCache,
};
export function graphAddEntity(): void {
const items = ADD_ENTITY_MAP.map(item => ({
value: item.kind,
@@ -712,16 +940,62 @@ export function graphAddEntity(): void {
});
}
/**
* Drag-from-port onto empty canvas → offer to create a compatible consumer
* entity and wire it to the source in one gesture (the classic node-editor
* "drag out a new node" flow).
*/
function _promptCreateAndConnect(sourceKind: string, sourceId: string): void {
// Kinds that can consume this source (non-nested) and have an add action.
const kinds = [...new Set(getCompatibleInputs(sourceKind).map(c => c.targetKind))]
.filter(k => ADD_ENTITY_MAP.some(e => e.kind === k));
if (kinds.length === 0) {
showToast(t('graph.no_compatible_connection'), 'info');
return;
}
const items = kinds.map(k => {
const entry = ADD_ENTITY_MAP.find(e => e.kind === k)!;
return { value: k, icon: entry.icon, label: ENTITY_LABELS[k] || k.replace(/_/g, ' ') };
});
showTypePicker({
title: t('graph.create_and_connect'),
items,
onPick: (kind) => _createAndConnect(kind, sourceKind, sourceId),
});
}
/** Open the add-entity modal for `targetKind`, then wire the new entity to `sourceId`. */
function _createAndConnect(targetKind: string, sourceKind: string, sourceId: string): void {
const entry = ADD_ENTITY_MAP.find(e => e.kind === targetKind);
if (!entry) return;
_watchForNewEntity((newId) => {
const matches = findConnection(targetKind, sourceKind);
if (matches.length === 1) {
_doConnect(newId, targetKind, matches[0].field, sourceId);
} else if (matches.length > 1) {
_promptConnectionField(matches, newId, targetKind, sourceId);
} else {
// No resolvable field — just refresh so the new node appears.
loadGraphEditor();
}
}, targetKind);
entry.fn();
}
// Watch for new entity creation after add-entity menu action
let _entityWatchCleanup: (() => void) | null = null;
function _watchForNewEntity(): void {
function _watchForNewEntity(onNew?: (newId: string) => void, expectKind?: string): void {
// Cleanup any previous watcher
if (_entityWatchCleanup) _entityWatchCleanup();
// Scope to the expected kind's cache when given (create-and-connect), so the
// callback never fires for an unrelated entity that happens to appear first.
const caches = (expectKind && KIND_CACHE[expectKind]) ? [KIND_CACHE[expectKind]] : ALL_CACHES;
// Snapshot all current IDs
const knownIds = new Set<string>();
for (const cache of ALL_CACHES) {
for (const cache of caches) {
for (const item of (cache.data || [])) {
if (item.id) knownIds.add(item.id);
}
@@ -731,9 +1005,12 @@ function _watchForNewEntity(): void {
if (!Array.isArray(data)) return;
for (const item of data) {
if (item.id && !knownIds.has(item.id)) {
// Found a new entity — reload graph and zoom to it
// Found a new entity.
const newId = item.id;
cleanup();
// Custom handler (e.g. create-and-connect) takes over.
if (onNew) { onNew(newId); return; }
// Default: reload graph and zoom to the new node.
loadGraphEditor().then(() => {
const node = _nodeMap?.get(newId);
if (node && _canvas) {
@@ -751,14 +1028,14 @@ function _watchForNewEntity(): void {
}
};
for (const cache of ALL_CACHES) cache.subscribe(handler);
for (const cache of caches) cache.subscribe(handler);
// Auto-cleanup after 2 minutes (user might cancel the modal)
const timeout = setTimeout(cleanup, 120_000);
function cleanup() {
clearTimeout(timeout);
for (const cache of ALL_CACHES) cache.unsubscribe(handler);
for (const cache of caches) cache.unsubscribe(handler);
_entityWatchCleanup = null;
}
@@ -829,6 +1106,9 @@ function _renderGraph(container: HTMLElement): void {
});
markOrphans(nodeGroup, _nodeMap!, _edges!);
// Health overlay: flag dangling references and dependency cycles.
_computeAndMarkIssues(nodeGroup);
// Animated flow dots for running nodes
const runningIds = new Set<string>();
for (const node of _nodeMap!.values()) {
@@ -845,6 +1125,8 @@ function _renderGraph(container: HTMLElement): void {
_canvas.onZoomChange = (z) => {
const label = container.querySelector('.graph-zoom-label');
if (label) label.textContent = `${Math.round(z * 100)}%`;
// Reveal edge field labels once zoomed in enough to read them.
edgeGroup.classList.toggle('show-labels', z >= 0.9);
};
_canvas.onViewChange = (vp) => {
@@ -993,6 +1275,9 @@ function _renderGraph(container: HTMLElement): void {
container.focus();
// Re-focus when clicking inside the graph
svgEl.addEventListener('pointerdown', () => container.focus());
// The toolbar markup hardcodes `disabled` on undo/redo; re-sync with the
// live stacks since this render may follow an undoable action.
_updateUndoRedoButtons();
_initialized = true;
}
@@ -1048,6 +1333,10 @@ function _graphHTML(): string {
<button class="btn-icon" onclick="graphRelayout()" title="${t('graph.relayout')}" data-collapse>
<svg class="icon" viewBox="0 0 24 24"><path d="M3 12a9 9 0 0 1 9-9 9.75 9.75 0 0 1 6.74 2.74L21 8"/><path d="M21 3v5h-5"/><path d="M21 12a9 9 0 0 1-9 9 9.75 9.75 0 0 1-6.74-2.74L3 16"/><path d="M8 16H3v5"/></svg>
</button>
<button class="btn-icon graph-issues-btn" id="graph-issues-btn" onclick="graphShowIssues()" title="${t('graph.issues')}" style="display:none" data-collapse>
<svg class="icon" viewBox="0 0 24 24"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3"/><path d="M12 9v4"/><path d="M12 17h.01"/></svg>
<span class="graph-issues-count"></span>
</button>
<button class="btn-icon" onclick="graphToggleFullscreen()" title="${t('graph.fullscreen')} (F11)" data-collapse>
<svg class="icon" viewBox="0 0 24 24"><path d="M8 3H5a2 2 0 0 0-2 2v3"/><path d="M21 8V5a2 2 0 0 0-2-2h-3"/><path d="M3 16v3a2 2 0 0 0 2 2h3"/><path d="M16 21h3a2 2 0 0 0 2-2v-3"/></svg>
</button>
@@ -1098,6 +1387,14 @@ function _graphHTML(): string {
<svg class="icon" viewBox="0 0 24 24"><path d="M8 3H5a2 2 0 0 0-2 2v3"/><path d="M21 8V5a2 2 0 0 0-2-2h-3"/><path d="M3 16v3a2 2 0 0 0 2 2h3"/><path d="M16 21h3a2 2 0 0 0 2-2v-3"/></svg>
<span>${t('graph.fullscreen')}</span>
</button>
<button onclick="graphExportTopology(); closeToolbarOverflow()">
<svg class="icon" viewBox="0 0 24 24"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><path d="M7 10l5 5 5-5"/><path d="M12 15V3"/></svg>
<span>${t('graph.export')}</span>
</button>
<button onclick="graphDuplicateSelection(); closeToolbarOverflow()">
<svg class="icon" viewBox="0 0 24 24"><rect x="9" y="9" width="13" height="13" rx="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>
<span>${t('graph.duplicate')}</span>
</button>
<div class="graph-overflow-sep"></div>
<button id="graph-overflow-help" onclick="toggleGraphHelp(); closeToolbarOverflow()">
<svg class="icon" viewBox="0 0 24 24"><circle cx="12" cy="12" r="10"/><path d="M9.09 9a3 3 0 0 1 5.83 1c0 2-3 3-3 3"/><path d="M12 17h.01"/></svg>
@@ -1525,7 +1822,8 @@ function _onEditNode(node: any) {
fnMap[node.kind]?.();
}
function _onDeleteNode(node: any) {
/** Dispatch the per-entity delete (each shows its own confirm + handles API/cache). */
function _deleteNodeRaw(node: any): void {
const fnMap: any = {
device: () => _w.removeDevice?.(node.id),
capture_template: () => _w.deleteTemplate?.(node.id),
@@ -1545,6 +1843,30 @@ function _onDeleteNode(node: any) {
fnMap[node.kind]?.();
}
// Guards the await gap (dependents fetch + confirm) against a double-fire from
// rapid Delete presses or trash-button + Delete on the same node.
const _deletingIds = new Set<string>();
/**
* Single-node delete from the graph: first warn if other entities reference
* this one (their wiring would break), then hand off to the per-entity delete.
*/
async function _onDeleteNode(node: any): Promise<void> {
if (_deletingIds.has(node.id)) return;
_deletingIds.add(node.id);
try {
const deps = await getDependents(node.kind, node.id);
if (deps.length > 0) {
const names = deps.slice(0, 5).map(d => d.name).join(', ') + (deps.length > 5 ? ', …' : '');
const ok = await showConfirm(t('graph.delete_with_dependents_confirm', { count: deps.length, names }));
if (!ok) return;
}
_deleteNodeRaw(node);
} finally {
_deletingIds.delete(node.id);
}
}
async function _bulkDeleteSelected(): Promise<void> {
const count = _selectedIds.size;
if (count < 2) return;
@@ -1552,9 +1874,11 @@ async function _bulkDeleteSelected(): Promise<void> {
(t('graph.bulk_delete_confirm') || 'Delete {count} selected entities?').replace('{count}', String(count))
);
if (!ok) return;
// Bulk uses the raw delegate — the single bulk confirm covers the batch, and
// per-node dependents prompts would be a dialog storm.
for (const id of _selectedIds) {
const node = _nodeMap?.get(id);
if (node) _onDeleteNode(node);
if (node) _deleteNodeRaw(node);
}
_selectedIds.clear();
}
@@ -1991,17 +2315,37 @@ function _onDragPointerUp(): void {
_justDragged = true;
requestAnimationFrame(() => { _justDragged = false; });
const moved: Array<{ id: string; oldX: number; oldY: number; newX: number; newY: number }> = [];
if (_dragState.multi) {
_dragState.nodes.forEach(n => {
if (n.el) n.el.classList.remove('dragging');
const node = _nodeMap!.get(n.id);
if (node) _manualPositions.set(n.id, { x: node.x, y: node.y });
if (node) {
_manualPositions.set(n.id, { x: node.x, y: node.y });
moved.push({ id: n.id, oldX: n.startX, oldY: n.startY, newX: node.x, newY: node.y });
}
});
} else {
const ds = _dragState as DragStateSingle;
ds.el.classList.remove('dragging');
const node = _nodeMap!.get(ds.nodeId);
if (node) _manualPositions.set(ds.nodeId, { x: node.x, y: node.y });
if (node) {
_manualPositions.set(ds.nodeId, { x: node.x, y: node.y });
moved.push({ id: ds.nodeId, oldX: ds.startNode.x, oldY: ds.startNode.y, newX: node.x, newY: node.y });
}
}
// Persist the hand-arranged layout so it survives page reloads.
_saveManualPositions();
// Record an undoable move (skip no-op drags below the dead zone).
if (moved.some(m => m.oldX !== m.newX || m.oldY !== m.newY)) {
pushUndoAction({
label: t('graph.action.move'),
undo: async () => { for (const m of moved) _manualPositions.set(m.id, { x: m.oldX, y: m.oldY }); _saveManualPositions(); },
redo: async () => { for (const m of moved) _manualPositions.set(m.id, { x: m.newX, y: m.newY }); _saveManualPositions(); },
});
}
_bounds = _calcBounds(_nodeMap);
@@ -2201,11 +2545,26 @@ function _initPortDrag(svgEl: SVGSVGElement, nodeGroup: SVGGElement, _edgeGroup:
p.classList.add('graph-port-incompatible');
}
});
// Also highlight whole compatible target NODES, so a slot with no
// existing edge (and therefore no input port yet) is still droppable —
// the user can drop anywhere on the node body to wire an empty slot.
const compatibleKinds = new Set(compatible.map(c => c.targetKind));
nodeGroup.querySelectorAll('.graph-node').forEach(n => {
const nKind = n.getAttribute('data-kind');
const nId = n.getAttribute('data-id');
if (nId !== sourceNodeId && nKind && compatibleKinds.has(nKind)) {
n.classList.add('graph-node-compatible');
}
});
}, true); // capture phase to beat node drag
if (!_connectListenersAdded) {
window.addEventListener('pointermove', _onConnectPointerMove);
window.addEventListener('pointerup', _onConnectPointerUp);
// pointercancel (touch interruption, capture loss) must also tear down
// the drag — otherwise the temp edge, node highlights and blockPan stick.
window.addEventListener('pointercancel', _onConnectPointerUp);
_connectListenersAdded = true;
}
}
@@ -2228,12 +2587,20 @@ function _onConnectPointerMove(e: PointerEvent): void {
svgEl.querySelectorAll('.graph-port-drop-target').forEach(p => p.classList.remove('graph-port-drop-target'));
if (port) port.classList.add('graph-port-drop-target');
// Highlight the compatible node under the cursor for drop-on-node wiring
// (only when not already hovering a specific port).
svgEl.querySelectorAll('.graph-node-drop-target').forEach(n => n.classList.remove('graph-node-drop-target'));
if (!port) {
const nodeUnder = elem?.closest?.('.graph-node-compatible');
if (nodeUnder) nodeUnder.classList.add('graph-node-drop-target');
}
}
function _onConnectPointerUp(e: PointerEvent): void {
if (!_connectState) return;
const { sourceNodeId, sourceKind, portType, dragPath } = _connectState;
const { sourceNodeId, sourceKind, startX, startY, dragPath } = _connectState;
// Clean up drag edge
dragPath.remove();
@@ -2241,48 +2608,138 @@ function _onConnectPointerUp(e: PointerEvent): void {
if (svgEl) svgEl.classList.remove('connecting');
if (_canvas) _canvas.blockPan = false;
// Clean up port highlights
// Clean up port + node highlights
const nodeGroup = document.querySelector('.graph-nodes') as SVGGElement | null;
if (nodeGroup) {
nodeGroup.querySelectorAll('.graph-port-compatible, .graph-port-incompatible, .graph-port-drop-target').forEach(p => {
p.classList.remove('graph-port-compatible', 'graph-port-incompatible', 'graph-port-drop-target');
});
nodeGroup.querySelectorAll('.graph-node-compatible, .graph-node-drop-target').forEach(n => {
n.classList.remove('graph-node-compatible', 'graph-node-drop-target');
});
}
// Check if dropped on a compatible input port
const elem = document.elementFromPoint(e.clientX, e.clientY);
const targetPort = elem?.closest?.('.graph-port-in');
if (targetPort) {
// Dropped on a specific input port — resolve by that port's edge type.
const targetNodeId = targetPort.getAttribute('data-node-id') ?? '';
const targetKind = targetPort.getAttribute('data-node-kind') ?? '';
const targetPortType = targetPort.getAttribute('data-port-type') ?? '';
if (targetNodeId !== sourceNodeId) {
// Find the matching connection
const matches = findConnection(targetKind, sourceKind, targetPortType);
const matches = _availableMatches(findConnection(targetKind, sourceKind, targetPortType), targetNodeId);
if (matches.length === 1) {
_doConnect(targetNodeId, targetKind, matches[0].field, sourceNodeId);
} else if (matches.length > 1) {
// Multiple possible fields (e.g., template → picture_source could be capture or pp template)
// Resolve by source kind
const exact = matches.find(m => m.sourceKind === sourceKind);
if (exact) {
_doConnect(targetNodeId, targetKind, exact.field, sourceNodeId);
// Genuinely ambiguous: the same source kind feeds two distinct
// fields (e.g. an automation's activation vs. deactivation scene).
_promptConnectionField(matches, targetNodeId, targetKind, sourceNodeId);
}
}
} else {
// Dropped on the node body (or an empty slot that has no port yet):
// resolve every connectable field for this source→target pair. This is
// what makes unconnected slots wireable from the graph.
const targetNode = elem?.closest?.('.graph-node');
if (targetNode) {
const targetNodeId = targetNode.getAttribute('data-id') ?? '';
const targetKind = targetNode.getAttribute('data-kind') ?? '';
if (targetNodeId && targetNodeId !== sourceNodeId) {
const matches = _availableMatches(findConnection(targetKind, sourceKind), targetNodeId);
if (matches.length === 1) {
_doConnect(targetNodeId, targetKind, matches[0].field, sourceNodeId);
} else if (matches.length > 1) {
_promptConnectionField(matches, targetNodeId, targetKind, sourceNodeId);
} else {
showToast(t('graph.no_compatible_connection'), 'info');
}
}
} else if (_canvas) {
// Dropped on empty canvas — offer to create a compatible consumer
// and wire it (skip when the gesture was effectively a click).
const CREATE_CONNECT_MIN_DRAG = 40; // graph units
const gp = _canvas.screenToGraph(e.clientX, e.clientY);
if (Math.hypot(gp.x - startX, gp.y - startY) > CREATE_CONNECT_MIN_DRAG) {
_promptCreateAndConnect(sourceKind, sourceNodeId);
}
}
}
_connectState = null;
}
/**
* Keep only the slots the target entity actually exposes (subtype-safe) — a
* field is offered iff its first path segment is a key on the serialized entity.
* E.g. an "effect" strip offers `intensity`/`scale`, a "picture" strip offers
* `smoothing`; a processed strip offers `input_source_id`. Applies to ALL
* matches (bindable and top-level alike); reference fields are always emitted by
* `to_dict` so empty slots stay wireable.
*/
function _availableMatches<T extends { field: string }>(matches: T[], targetId: string): T[] {
const ent = _entitiesById.get(targetId);
if (!ent) return matches; // no data (e.g. freshly created) → don't over-filter
// Offer a field only if the target entity actually exposes its slot (its
// first path segment is a key on the serialized entity). Reference fields
// are always emitted by `to_dict` even when empty, so empty slots stay
// wireable (B2); subtype-specific fields (e.g. processed-strip
// `input_source_id`) are correctly hidden on subtypes that lack them.
return matches.filter(m => m.field.split('.')[0] in ent);
}
/** Ask the user which field to wire when a source maps to multiple target fields. */
function _promptConnectionField(
matches: Array<{ field: string }>,
targetNodeId: string,
targetKind: string,
sourceNodeId: string,
): void {
showTypePicker({
title: t('graph.choose_connection'),
items: matches.map(m => ({ value: m.field, icon: _ico(P.link), label: _humanField(m.field) })),
onPick: (field) => { _doConnect(targetNodeId, targetKind, field, sourceNodeId); },
});
}
/** The id currently wired into (targetId, field), or '' if the slot is empty. */
function _currentSourceFor(targetId: string, field: string): string {
const edge = _edges?.find(e => e.to === targetId && e.field === field);
return edge ? edge.from : '';
}
async function _doConnect(targetId: string, targetKind: string, field: string, sourceId: string): Promise<void> {
const prevSourceId = _currentSourceFor(targetId, field);
if (prevSourceId === sourceId) return; // dropped onto the existing connection — no-op
// Pre-flight validation (existence + source kind + no dependency cycle).
const v = await validateConnection(targetKind, targetId, field, sourceId);
if (!v.ok) {
showToast(v.error || t('graph.connection_failed'), 'error');
return;
}
// Confirm before overwriting an already-occupied slot.
if (prevSourceId) {
const confirmed = await showConfirm(t('graph.replace_connection_confirm'));
if (!confirmed) return;
}
const ok = await updateConnection(targetId, targetKind, field, sourceId);
if (ok) {
showToast(t('graph.connection_updated') || 'Connection updated', 'success');
// Record an undoable action that restores the previous slot occupant.
// The inverse ops throw on failure so `_undo`/`_redo` can keep the
// action on its stack instead of silently desyncing (updateConnection
// returns false rather than throwing on API error).
pushUndoAction({
label: t('graph.action.connect'),
undo: async () => { if (!(await updateConnection(targetId, targetKind, field, prevSourceId))) throw new Error(t('graph.connection_failed')); },
redo: async () => { if (!(await updateConnection(targetId, targetKind, field, sourceId))) throw new Error(t('graph.connection_failed')); },
});
showToast(t('graph.connection_updated'), 'success');
await loadGraphEditor();
} else {
showToast(t('graph.connection_failed') || 'Failed to update connection', 'error');
showToast(t('graph.connection_failed'), 'error');
}
}
@@ -2311,31 +2768,35 @@ export async function graphUndo(): Promise<void> { await _undo(); }
export async function graphRedo(): Promise<void> { await _redo(); }
async function _undo(): Promise<void> {
if (_undoStack.length === 0) { showToast(t('graph.nothing_to_undo') || 'Nothing to undo', 'info'); return; }
if (_undoStack.length === 0) { showToast(t('graph.nothing_to_undo'), 'info'); return; }
const action = _undoStack.pop()!;
try {
await action.undo();
_redoStack.push(action);
showToast(t('graph.undone') || `Undone: ${action.label}`, 'info');
showToast(t('graph.undone'), 'info');
_updateUndoRedoButtons();
await loadGraphEditor();
} catch (e) {
showToast(e.message, 'error');
// The inverse op failed — keep the action on the undo stack so the
// user can retry, and surface the error instead of a false success.
_undoStack.push(action);
showToast(e instanceof Error ? e.message : String(e), 'error');
_updateUndoRedoButtons();
}
}
async function _redo(): Promise<void> {
if (_redoStack.length === 0) { showToast(t('graph.nothing_to_redo') || 'Nothing to redo', 'info'); return; }
if (_redoStack.length === 0) { showToast(t('graph.nothing_to_redo'), 'info'); return; }
const action = _redoStack.pop()!;
try {
await action.redo();
_undoStack.push(action);
showToast(t('graph.redone') || `Redone: ${action.label}`, 'info');
showToast(t('graph.redone'), 'info');
_updateUndoRedoButtons();
await loadGraphEditor();
} catch (e) {
showToast(e.message, 'error');
_redoStack.push(action);
showToast(e instanceof Error ? e.message : String(e), 'error');
_updateUndoRedoButtons();
}
}
@@ -2402,13 +2863,43 @@ export function toggleGraphHelp(): void {
/* ── Edge context menu (right-click to detach) ── */
/**
* Detach a connection and record an undoable action that restores it.
* @param prevSourceId the id previously wired into the slot (for undo)
*/
async function _doDetach(to: string, targetKind: string, field: string, prevSourceId: string): Promise<boolean> {
const ok = await detachConnection(to, targetKind, field);
if (ok) {
if (prevSourceId) {
pushUndoAction({
label: t('graph.action.disconnect'),
undo: async () => { if (!(await updateConnection(to, targetKind, field, prevSourceId))) throw new Error(t('graph.connection_failed')); },
redo: async () => { if (!(await detachConnection(to, targetKind, field))) throw new Error(t('graph.disconnect_failed')); },
});
}
showToast(t('graph.connection_removed'), 'success');
await loadGraphEditor();
} else {
showToast(t('graph.disconnect_failed'), 'error');
}
return ok;
}
function _onEdgeContextMenu(edgePath: Element, e: MouseEvent, container: HTMLElement): void {
_dismissEdgeContextMenu();
const field = edgePath.getAttribute('data-field') || '';
// List-slot edges (composite layers / mapped zones) aren't editable through
// the flat (to, field) path, but each carries its slot index so it can be
// re-wired individually.
if (edgePath.getAttribute('data-slot-list')) {
_showListSlotRewireMenu(edgePath, e, container);
return;
}
if (!isEditableEdge(field)) return; // nested fields can't be detached from graph
const toId = edgePath.getAttribute('data-to') ?? '';
const fromId = edgePath.getAttribute('data-from') ?? '';
const toNode = _nodeMap?.get(toId);
if (!toNode) return;
@@ -2419,19 +2910,13 @@ function _onEdgeContextMenu(edgePath: Element, e: MouseEvent, container: HTMLEle
const btn = document.createElement('button');
btn.className = 'graph-edge-menu-item danger';
btn.textContent = t('graph.disconnect') || 'Disconnect';
btn.textContent = t('graph.disconnect');
btn.addEventListener('click', async () => {
try {
_dismissEdgeContextMenu();
const ok = await detachConnection(toId, toNode.kind, field);
if (ok) {
showToast(t('graph.connection_removed') || 'Connection removed', 'success');
await loadGraphEditor();
} else {
showToast(t('graph.disconnect_failed') || 'Failed to disconnect', 'error');
}
await _doDetach(toId, toNode.kind, field, fromId);
} catch (err) {
showToast(`${t('graph.disconnect_failed') || 'Failed to disconnect'}: ${err instanceof Error ? err.message : String(err)}`, 'error');
showToast(`${t('graph.disconnect_failed')}: ${err instanceof Error ? err.message : String(err)}`, 'error');
}
});
menu.appendChild(btn);
@@ -2447,20 +2932,80 @@ function _dismissEdgeContextMenu(): void {
}
}
async function _detachSelectedEdge(): Promise<void> {
if (!_selectedEdge) return;
const { to, field, targetKind } = _selectedEdge;
_selectedEdge = null;
/**
* Re-wire menu for a list-slot edge (a composite layer or mapped zone source).
* Each such edge carries `data-slot-*`, so we can replace just that one
* element's reference and leave its siblings (and its own other settings)
* untouched. The picker lists every node of the source's kind; the backend
* rejects self-reference / cycles.
*/
function _showListSlotRewireMenu(edgePath: Element, e: MouseEvent, container: HTMLElement): void {
const toId = edgePath.getAttribute('data-to') ?? ''; // composite/mapped owner
const fromId = edgePath.getAttribute('data-from') ?? ''; // current source occupant
const listField = edgePath.getAttribute('data-slot-list') ?? '';
const index = parseInt(edgePath.getAttribute('data-slot-index') ?? '', 10);
const refField = edgePath.getAttribute('data-slot-ref') ?? '';
const toNode = _nodeMap?.get(toId);
if (!toNode || Number.isNaN(index)) return;
const ok = await detachConnection(to, targetKind, field);
const sourceKind = _nodeMap?.get(fromId)?.kind ?? 'color_strip_source';
const candidates = [...(_nodeMap?.values() ?? [])]
.filter((n: any) => n.kind === sourceKind && n.id !== toId && n.id !== fromId)
.sort((a: any, b: any) => (a.name || a.id).localeCompare(b.name || b.id));
const menu = document.createElement('div');
menu.className = 'graph-edge-menu';
menu.style.left = `${e.clientX - container.getBoundingClientRect().left}px`;
menu.style.top = `${e.clientY - container.getBoundingClientRect().top}px`;
const btn = document.createElement('button');
btn.className = 'graph-edge-menu-item';
btn.textContent = t('graph.rewire');
btn.addEventListener('click', () => {
_dismissEdgeContextMenu();
if (candidates.length === 0) { showToast(t('graph.no_compatible_connection'), 'info'); return; }
showTypePicker({
title: t('graph.rewire_choose_source'),
items: candidates.map((n: any) => ({ value: n.id, icon: _ico(P.link), label: n.name || n.id })),
onPick: (newSourceId: string) => { void _doListSlotRewire(toId, toNode.kind, listField, index, refField, newSourceId, fromId); },
});
});
menu.appendChild(btn);
container.querySelector('.graph-container')!.appendChild(menu);
_edgeContextMenu = menu;
}
/** Re-wire one list slot (composite layer / mapped zone) and record an undoable action. */
async function _doListSlotRewire(
targetId: string, targetKind: string, listField: string, index: number,
refField: string, newSourceId: string, prevSourceId: string,
): Promise<void> {
if (newSourceId === prevSourceId) return;
// Pass the expected current occupant so the write refuses if the slot was
// reordered/edited out-of-band (positional `index` could otherwise hit the
// wrong element). Each step expects the slot to still hold what it replaces.
const ok = await updateListSlotConnection(targetId, targetKind, listField, index, refField, newSourceId, prevSourceId);
if (ok) {
showToast(t('graph.connection_removed') || 'Connection removed', 'success');
pushUndoAction({
label: t('graph.action.rewire'),
undo: async () => { if (!(await updateListSlotConnection(targetId, targetKind, listField, index, refField, prevSourceId, newSourceId))) throw new Error(t('graph.connection_failed')); },
redo: async () => { if (!(await updateListSlotConnection(targetId, targetKind, listField, index, refField, newSourceId, prevSourceId))) throw new Error(t('graph.connection_failed')); },
});
showToast(t('graph.connection_updated'), 'success');
await loadGraphEditor();
} else {
showToast(t('graph.disconnect_failed') || 'Failed to disconnect', 'error');
showToast(t('graph.connection_failed'), 'error');
}
}
async function _detachSelectedEdge(): Promise<void> {
if (!_selectedEdge) return;
const { from, to, field, targetKind } = _selectedEdge;
_selectedEdge = null;
await _doDetach(to, targetKind, field, from);
}
/* ── Node hover FPS tooltip ── */
let _hoverTooltip: HTMLDivElement | null = null; // the <div> element, created once per graph render
@@ -28,6 +28,7 @@ import {
ICON_LED_PREVIEW, ICON_ACTIVITY, ICON_TIMER, ICON_MOVE_VERTICAL, ICON_CLOCK,
ICON_MUSIC, ICON_TRENDING_UP, ICON_MAP_PIN, ICON_MONITOR, ICON_REFRESH, ICON_TRASH,
ICON_HOME, ICON_RAINBOW, ICON_LINK, ICON_DROPLETS, ICON_GAMEPAD, ICON_X,
ICON_CHECK, ICON_FILE_TEXT,
} from '../core/icons.ts';
import { wrapCard } from '../core/card-colors.ts';
import type { ModCardOpts, ModChipOpts } from '../core/mod-card.ts';
@@ -56,9 +57,10 @@ registerIconEntityType('value_source', makeSimpleIconAdapter<any>({
import type { IconSelectItem } from '../core/icon-select.ts';
import * as P from '../core/icon-paths.ts';
import { EntitySelect } from '../core/entity-palette.ts';
import * as JinjaEditor from '../core/jinja-editor.ts';
import { loadPictureSources } from './streams.ts';
import { hexToRgbArray, rgbArrayToHex } from './css-gradient-editor.ts';
import type { ValueSource } from '../types.ts';
import type { ValueSource, TemplateInput } from '../types.ts';
export { getValueSourceIcon };
@@ -78,6 +80,14 @@ let _vsAnimColorClockEntitySelect: EntitySelect | null = null;
let _vsHTTPEndpointEntitySelect: EntitySelect | null = null;
let _vsTagsInput: TagInput | null = null;
// ── Template value source editor state ──
// (the bound inputs live in the DOM rows; read them via _readTemplateInputRows)
const _vsTemplateInputSelects = new Map<HTMLElement, EntitySelect>();
let _vsTemplateEditor: JinjaEditor.JinjaEditorHandle | null = null;
let _templateValidateTimer: ReturnType<typeof setTimeout> | null = null;
let _templateValidateGen = 0;
let _vsTemplateInputUid = 0;
class ValueSourceModal extends Modal {
constructor() { super('value-source-modal'); }
@@ -95,6 +105,13 @@ class ValueSourceModal extends Modal {
if (_vsAnimColorClockEntitySelect) { _vsAnimColorClockEntitySelect.destroy(); _vsAnimColorClockEntitySelect = null; }
if (_vsGameIntegrationEntitySelect) { _vsGameIntegrationEntitySelect.destroy(); _vsGameIntegrationEntitySelect = null; }
if (_vsHTTPEndpointEntitySelect) { _vsHTTPEndpointEntitySelect.destroy(); _vsHTTPEndpointEntitySelect = null; }
// Template editor: destroy all per-row EntitySelect portals + the highlighter
// and cancel any pending validation so stale responses are ignored.
_vsTemplateInputSelects.forEach(sel => sel.destroy());
_vsTemplateInputSelects.clear();
if (_vsTemplateEditor) { _vsTemplateEditor.destroy(); _vsTemplateEditor = null; }
if (_templateValidateTimer) { clearTimeout(_templateValidateTimer); _templateValidateTimer = null; }
_templateValidateGen++;
}
snapshotValues() {
@@ -146,6 +163,11 @@ class ValueSourceModal extends Modal {
httpMin: (document.getElementById('value-source-http-min') as HTMLInputElement | null)?.value || '',
httpMax: (document.getElementById('value-source-http-max') as HTMLInputElement | null)?.value || '',
httpSmoothing: (document.getElementById('value-source-http-smoothing') as HTMLInputElement | null)?.value || '',
// Template value source
template: (document.getElementById('value-source-template-expression') as HTMLTextAreaElement | null)?.value || '',
templateInputs: JSON.stringify(_readTemplateInputRows()),
templateDefault: (document.getElementById('value-source-template-default-value') as HTMLInputElement | null)?.value || '',
templateEvalInterval: (document.getElementById('value-source-template-eval-interval') as HTMLInputElement | null)?.value || '',
};
}
}
@@ -188,7 +210,7 @@ function _autoGenerateVSName() {
/* ── Icon-grid type selector ──────────────────────────────────── */
const VS_FLOAT_TYPE_KEYS = ['static', 'animated', 'audio', 'adaptive_time', 'adaptive_scene', 'daylight', 'ha_entity', 'system_metrics', 'game_event', 'http'];
const VS_FLOAT_TYPE_KEYS = ['static', 'animated', 'audio', 'adaptive_time', 'adaptive_scene', 'daylight', 'ha_entity', 'system_metrics', 'game_event', 'http', 'template'];
const VS_COLOR_TYPE_KEYS = ['static_color', 'animated_color', 'adaptive_time_color', 'gradient_map', 'css_extract'];
const VS_TYPE_KEYS = [...VS_FLOAT_TYPE_KEYS, ...VS_COLOR_TYPE_KEYS];
@@ -396,6 +418,16 @@ function _onMetricChange(metric: string) {
if (networkFields) networkFields.style.display = networkMetrics.includes(metric) ? '' : 'none';
if (diskFields) diskFields.style.display = metric === 'disk_usage' ? '' : 'none';
if (sensorFields) sensorFields.style.display = sensorMetrics.includes(metric) ? '' : 'none';
// The Normalize toggle only makes sense where a Min/Max range is shown
// (rangeMetrics). Percent/network/disk metrics have a fixed natural→0-1
// mapping in their reader, so "clamp the raw value as-is" would just
// saturate them; hide the toggle and force normalization on for those.
const normGroup = document.getElementById('value-source-sysmetric-normalize-group') as HTMLElement | null;
const normCb = document.getElementById('value-source-sysmetric-normalize') as HTMLInputElement | null;
const showNorm = rangeMetrics.includes(metric);
if (normGroup) normGroup.style.display = showNorm ? '' : 'none';
if (!showNorm && normCb) normCb.checked = true;
_syncVsNormalizeUI();
}
// ── Game Event Value Source helpers ──
@@ -597,6 +629,8 @@ export async function showValueSourceModal(editData: any, presetType: any = null
(document.getElementById('value-source-min-ha-value') as HTMLInputElement).value = String(editData.min_ha_value ?? 0);
(document.getElementById('value-source-max-ha-value') as HTMLInputElement).value = String(editData.max_ha_value ?? 100);
_setSlider('value-source-ha-smoothing', editData.smoothing ?? 0);
(document.getElementById('value-source-ha-normalize') as HTMLInputElement).checked = editData.normalize !== false;
_syncVsNormalizeUI();
} else if (editData.source_type === 'gradient_map') {
_populateGradientInputDropdown(editData.value_source_id || '');
_populateGradientEntityDropdown(editData.gradient_id || '');
@@ -616,7 +650,9 @@ export async function showValueSourceModal(editData: any, presetType: any = null
(document.getElementById('value-source-sensor-label') as HTMLInputElement).value = editData.sensor_label || '';
_setSlider('value-source-poll-interval', editData.poll_interval ?? 1.0);
_setSlider('value-source-sysmetric-smoothing', editData.smoothing ?? 0);
(document.getElementById('value-source-sysmetric-normalize') as HTMLInputElement).checked = editData.normalize !== false;
_onMetricChange(editData.metric || 'cpu_load');
_syncVsNormalizeUI();
} else if (editData.source_type === 'game_event') {
_populateVSGameIntegrationDropdown(editData.game_integration_id || '');
_populateVSGameEventTypeDropdown(editData.event_type || 'health');
@@ -633,6 +669,17 @@ export async function showValueSourceModal(editData: any, presetType: any = null
(document.getElementById('value-source-http-min') as HTMLInputElement).value = String(editData.min_value ?? 0);
(document.getElementById('value-source-http-max') as HTMLInputElement).value = String(editData.max_value ?? 100);
_setSlider('value-source-http-smoothing', editData.smoothing ?? 0);
(document.getElementById('value-source-http-normalize') as HTMLInputElement).checked = editData.normalize !== false;
_syncVsNormalizeUI();
} else if (editData.source_type === 'template') {
(document.getElementById('value-source-template-expression') as HTMLTextAreaElement).value = editData.template || '';
_setSlider('value-source-template-default-value', editData.default_value ?? 0.0);
(document.getElementById('value-source-template-default-value-display') as HTMLElement).textContent =
parseFloat(String(editData.default_value ?? 0.0)).toFixed(2);
(document.getElementById('value-source-template-eval-interval') as HTMLInputElement).value = String(editData.eval_interval ?? 0);
_ensureTemplateEditor();
_populateTemplateInputsUI(editData.inputs ?? []);
_runTemplateValidationDebounced();
}
} else {
(document.getElementById('value-source-name') as HTMLInputElement).value = '';
@@ -683,6 +730,7 @@ export async function showValueSourceModal(editData: any, presetType: any = null
(document.getElementById('value-source-min-ha-value') as HTMLInputElement).value = '0';
(document.getElementById('value-source-max-ha-value') as HTMLInputElement).value = '100';
_setSlider('value-source-ha-smoothing', 0);
(document.getElementById('value-source-ha-normalize') as HTMLInputElement).checked = true;
// Gradient map defaults
(document.getElementById('value-source-gradient-easing') as HTMLSelectElement).value = 'linear';
// CSS extract defaults
@@ -697,6 +745,7 @@ export async function showValueSourceModal(editData: any, presetType: any = null
(document.getElementById('value-source-sensor-label') as HTMLInputElement).value = '';
_setSlider('value-source-poll-interval', 1.0);
_setSlider('value-source-sysmetric-smoothing', 0);
(document.getElementById('value-source-sysmetric-normalize') as HTMLInputElement).checked = true;
// HTTP value source defaults
const httpJsonPath = document.getElementById('value-source-http-json-path') as HTMLInputElement | null;
if (httpJsonPath) httpJsonPath.value = '';
@@ -707,6 +756,24 @@ export async function showValueSourceModal(editData: any, presetType: any = null
const httpMax = document.getElementById('value-source-http-max') as HTMLInputElement | null;
if (httpMax) httpMax.value = '100';
_setSlider('value-source-http-smoothing', 0);
const httpNormalize = document.getElementById('value-source-http-normalize') as HTMLInputElement | null;
if (httpNormalize) httpNormalize.checked = true;
_syncVsNormalizeUI();
// Template value source defaults
const tmplExpr = document.getElementById('value-source-template-expression') as HTMLTextAreaElement | null;
if (tmplExpr) tmplExpr.value = '';
_setSlider('value-source-template-default-value', 0.0);
const tmplDefDisp = document.getElementById('value-source-template-default-value-display') as HTMLElement | null;
if (tmplDefDisp) tmplDefDisp.textContent = '0.00';
const tmplEval = document.getElementById('value-source-template-eval-interval') as HTMLInputElement | null;
if (tmplEval) tmplEval.value = '0';
if (presetType === 'template') {
_ensureTemplateEditor();
_populateTemplateInputsUI([]);
_runTemplateValidationDebounced();
} else {
_populateTemplateInputsUI([]);
}
_autoGenerateVSName();
}
@@ -763,6 +830,19 @@ export function onValueSourceTypeChange() {
// before the integrations tab has been visited.
httpEndpointsCache.fetch().then(() => _populateVSHTTPEndpointDropdown(''));
}
const templateSec = document.getElementById('value-source-template-section') as HTMLElement | null;
if (templateSec) templateSec.style.display = type === 'template' ? '' : 'none';
if (type === 'template') {
_ensureTemplateEditor();
_runTemplateValidationDebounced();
} else {
// Leaving template type — cancel any pending/in-flight validation so a
// stale response cannot re-disable save on a non-template form, then
// restore the button.
if (_templateValidateTimer) { clearTimeout(_templateValidateTimer); _templateValidateTimer = null; }
_templateValidateGen++;
_setVSSaveEnabled(true);
}
(document.getElementById('value-source-adaptive-range-section') as HTMLElement).style.display =
(type === 'adaptive_time' || type === 'adaptive_scene' || type === 'daylight') ? '' : 'none';
@@ -815,6 +895,35 @@ function _syncDaylightVSSpeedVisibility() {
(document.getElementById('value-source-daylight-speed-group') as HTMLElement).style.display = rt ? 'none' : '';
}
// ── Normalize toggle (magnitude sources: ha_entity / system_metrics / http) ──
export function onValueSourceNormalizeChange() {
_syncVsNormalizeUI();
}
// Grey out + disable the Min/Max range inputs when a magnitude source is in
// clamp-passthrough mode (normalize off) — they are inert there. Only the
// visible section's pair is relevant, but syncing all three is harmless.
function _syncVsNormalizeUI() {
const groups: Array<[string, string[]]> = [
['value-source-ha-normalize', ['value-source-min-ha-value', 'value-source-max-ha-value']],
['value-source-sysmetric-normalize', ['value-source-sysmetric-min', 'value-source-sysmetric-max']],
['value-source-http-normalize', ['value-source-http-min', 'value-source-http-max']],
];
for (const [cbId, inputIds] of groups) {
const cb = document.getElementById(cbId) as HTMLInputElement | null;
if (!cb) continue;
const on = cb.checked;
for (const id of inputIds) {
const inp = document.getElementById(id) as HTMLInputElement | null;
if (!inp) continue;
inp.disabled = !on;
const fg = inp.closest('.form-group') as HTMLElement | null;
if (fg) fg.style.opacity = on ? '' : '0.45';
}
}
}
// ── Save ──────────────────────────────────────────────────────
export async function saveValueSource() {
@@ -889,6 +998,7 @@ export async function saveValueSource() {
payload.min_ha_value = parseFloat((document.getElementById('value-source-min-ha-value') as HTMLInputElement).value);
payload.max_ha_value = parseFloat((document.getElementById('value-source-max-ha-value') as HTMLInputElement).value);
payload.smoothing = parseFloat((document.getElementById('value-source-ha-smoothing') as HTMLInputElement).value);
payload.normalize = (document.getElementById('value-source-ha-normalize') as HTMLInputElement).checked;
if (!payload.ha_source_id) {
errorEl.textContent = t('value_source.ha_source') + ' required';
errorEl.style.display = '';
@@ -932,6 +1042,7 @@ export async function saveValueSource() {
payload.sensor_label = (document.getElementById('value-source-sensor-label') as HTMLInputElement).value;
payload.poll_interval = parseFloat((document.getElementById('value-source-poll-interval') as HTMLInputElement).value) || 1.0;
payload.smoothing = parseFloat((document.getElementById('value-source-sysmetric-smoothing') as HTMLInputElement).value) || 0;
payload.normalize = (document.getElementById('value-source-sysmetric-normalize') as HTMLInputElement).checked;
} else if (sourceType === 'game_event') {
payload.game_integration_id = (document.getElementById('value-source-game-integration') as HTMLSelectElement).value;
payload.event_type = (document.getElementById('value-source-game-event-type') as HTMLSelectElement).value;
@@ -952,6 +1063,7 @@ export async function saveValueSource() {
payload.min_value = parseFloat((document.getElementById('value-source-http-min') as HTMLInputElement).value) || 0;
payload.max_value = parseFloat((document.getElementById('value-source-http-max') as HTMLInputElement).value) || 100;
payload.smoothing = parseFloat((document.getElementById('value-source-http-smoothing') as HTMLInputElement).value) || 0;
payload.normalize = (document.getElementById('value-source-http-normalize') as HTMLInputElement).checked;
if (!payload.http_endpoint_id) {
errorEl.textContent = t('value_source.http.endpoint_required');
errorEl.style.display = '';
@@ -962,6 +1074,26 @@ export async function saveValueSource() {
errorEl.style.display = '';
return;
}
} else if (sourceType === 'template') {
const inputs = _getTemplateInputsFromUI();
if (inputs === null) return; // toast already shown
payload.template = (document.getElementById('value-source-template-expression') as HTMLTextAreaElement).value;
payload.inputs = inputs;
payload.default_value = parseFloat((document.getElementById('value-source-template-default-value') as HTMLInputElement).value) || 0.0;
const evalRaw = (document.getElementById('value-source-template-eval-interval') as HTMLInputElement).value.trim();
payload.eval_interval = evalRaw === '' ? null : (parseFloat(evalRaw) || 0);
if (!payload.template.trim()) {
errorEl.textContent = t('value_source.template.error.invalid_expr');
errorEl.style.display = '';
return;
}
// Final server-side gate: block save when the expression is invalid.
const result = await _validateTemplateRequest(payload.template, inputs, id || undefined);
if (result && result.valid === false) {
errorEl.textContent = (result.errors && result.errors[0]) || result.error || t('value_source.template.error.invalid_expr');
errorEl.style.display = '';
return;
}
}
try {
@@ -1373,6 +1505,7 @@ const VALUE_BADGE: Record<string, string> = {
gradient_map: 'VALUE · MAP',
css_extract: 'VALUE · STRIP',
system_metrics: 'VALUE · SYS',
template: 'VALUE · EXPR',
};
function _valueSourceChipsAndExtras(src: ValueSource): { chips: ModChipOpts[]; metaText: string; extra: string } {
@@ -1503,6 +1636,16 @@ function _valueSourceChipsAndExtras(src: ValueSource): { chips: ModChipOpts[]; m
const metricLabel = t(`value_source.metric.${(src as any).metric}`) || (src as any).metric;
chips.push({ icon: ICON_ACTIVITY, text: metricLabel });
metaText = metricLabel;
} else if (src.source_type === 'template') {
const inputs = (src as any).inputs || [];
const nInputs = inputs.length;
const expr = ((src as any).template || '').trim();
chips.push({
icon: ICON_FILE_TEXT,
text: `${nInputs} ${nInputs === 1 ? t('value_source.template.input_count_one') : t('value_source.template.input_count')}`,
});
if (expr) chips.push({ icon: ICON_ACTIVITY, text: expr.length > 28 ? expr.slice(0, 27) + '…' : expr });
metaText = `${t('value_source.type.template')} · ${nInputs} ${t('value_source.template.input_count')}`;
}
return { chips, metaText, extra };
@@ -1970,3 +2113,302 @@ function _populateCSSSourceDropdown(selectedId: string) {
});
}
}
// ── Template (Jinja expression) helpers ────────────────────────
/** Reserved identifiers an input variable may NOT shadow. */
const TEMPLATE_RESERVED_NAMES = new Set([
'min', 'max', 'abs', 'round', 'clamp', 'raw', 'range', 'dict', 'namespace',
]);
const TEMPLATE_IDENT_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
/** Lazily attach the syntax-highlighting overlay to the expression textarea. */
function _ensureTemplateEditor() {
const ta = document.getElementById('value-source-template-expression') as HTMLTextAreaElement | null;
if (!ta) return;
if (_vsTemplateEditor) { _vsTemplateEditor.refresh(); return; }
_vsTemplateEditor = JinjaEditor.create({
textarea: ta,
getInputNames: () => _readTemplateInputRows().map(i => i.name).filter(Boolean),
onChange: () => _runTemplateValidationDebounced(),
});
}
/** Bound-input names that are currently valid identifiers (for the highlighter + hints). */
function _currentTemplateVarNames(): string[] {
return _readTemplateInputRows()
.map(i => i.name)
.filter(n => TEMPLATE_IDENT_RE.test(n));
}
/**
* Append one `.template-input-row` (a variable-name field + a value-source
* EntitySelect + a remove button). Invoked from the inline add button.
*/
export function addTemplateInput(name: string = '', vsId: string = '') {
const list = document.getElementById('value-source-template-inputs-list');
if (!list) return;
const row = document.createElement('div');
row.className = 'template-input-row';
const nameInput = document.createElement('input');
nameInput.type = 'text';
nameInput.className = 'template-input-name';
nameInput.placeholder = t('value_source.template.input_name');
nameInput.value = name;
nameInput.spellcheck = false;
nameInput.autocomplete = 'off';
const select = document.createElement('select');
select.id = `value-source-template-input-${++_vsTemplateInputUid}`;
const floatSources = _templateFloatSources();
select.innerHTML = `<option value=""></option>` + floatSources.map(s =>
`<option value="${escapeHtml(s.id)}"${s.id === vsId ? ' selected' : ''}>${escapeHtml(s.name)}</option>`
).join('');
select.value = vsId || '';
const removeBtn = document.createElement('button');
removeBtn.type = 'button';
removeBtn.className = 'btn btn-icon btn-danger btn-sm';
removeBtn.innerHTML = ICON_TRASH;
removeBtn.setAttribute('aria-label', t('common.remove') || 'Remove');
row.appendChild(nameInput);
row.appendChild(select);
row.appendChild(removeBtn);
// Remove any "no inputs yet" placeholder before adding a real row.
const empty = list.querySelector('.template-inputs-empty');
if (empty) empty.remove();
list.appendChild(row);
const es = new EntitySelect({
target: select,
getItems: () => _templateFloatSources().map(s => ({
value: s.id,
label: s.name,
icon: getValueSourceIcon(s.source_type),
desc: s.source_type,
})),
placeholder: t('palette.search'),
allowNone: true,
noneLabel: '—',
onChange: () => _runTemplateValidationDebounced(),
});
_vsTemplateInputSelects.set(select, es);
nameInput.addEventListener('input', () => {
_vsTemplateEditor?.refresh();
_runTemplateValidationDebounced();
});
removeBtn.addEventListener('click', () => {
const sel = _vsTemplateInputSelects.get(select);
if (sel) { sel.destroy(); _vsTemplateInputSelects.delete(select); }
row.remove();
if (!list.querySelector('.template-input-row')) _showTemplateInputsEmpty(list);
_vsTemplateEditor?.refresh();
_renderTemplateHintVars();
_runTemplateValidationDebounced();
});
_renderTemplateHintVars();
}
/** Float value sources eligible to bind, excluding the source being edited. */
function _templateFloatSources(): ValueSource[] {
const selfId = (document.getElementById('value-source-id') as HTMLInputElement).value || undefined;
return _cachedValueSources.filter(v => v.return_type === 'float' && v.id !== selfId);
}
function _showTemplateInputsEmpty(list: HTMLElement) {
const empty = document.createElement('div');
empty.className = 'template-inputs-empty';
empty.textContent = t('value_source.template.inputs.empty');
list.appendChild(empty);
}
/** Raw row read — no validation, no toast (used by snapshot + the live validator). */
function _readTemplateInputRows(): TemplateInput[] {
const rows = document.querySelectorAll('#value-source-template-inputs-list .template-input-row');
const out: TemplateInput[] = [];
rows.forEach(row => {
const name = (row.querySelector('.template-input-name') as HTMLInputElement)?.value.trim() ?? '';
const select = row.querySelector('select') as HTMLSelectElement | null;
out.push({ name, value_source_id: select?.value || '' });
});
return out;
}
/**
* Validated read for save: every name must be a unique, non-reserved, valid
* identifier. Shows a toast and returns null on the first failure.
*/
function _getTemplateInputsFromUI(): TemplateInput[] | null {
const rows = _readTemplateInputRows();
const seen = new Set<string>();
for (const inp of rows) {
if (!inp.name) {
showToast(t('value_source.template.error.missing_input'), 'error');
return null;
}
if (!TEMPLATE_IDENT_RE.test(inp.name)) {
showToast(`${t('value_source.template.error.invalid_name')}: ${inp.name}`, 'error');
return null;
}
if (TEMPLATE_RESERVED_NAMES.has(inp.name)) {
showToast(`${t('value_source.template.error.reserved_name')}: ${inp.name}`, 'error');
return null;
}
if (seen.has(inp.name)) {
showToast(`${t('value_source.template.error.duplicate_name')}: ${inp.name}`, 'error');
return null;
}
seen.add(inp.name);
}
return rows;
}
/** Rebuild the inputs list from scratch (destroying any leaked EntitySelects). */
function _populateTemplateInputsUI(inputs: TemplateInput[]) {
_vsTemplateInputSelects.forEach(sel => sel.destroy());
_vsTemplateInputSelects.clear();
const list = document.getElementById('value-source-template-inputs-list');
if (!list) return;
list.innerHTML = '';
if (inputs.length === 0) {
_showTemplateInputsEmpty(list);
} else {
inputs.forEach(i => addTemplateInput(i.name, i.value_source_id));
}
_vsTemplateEditor?.refresh();
_renderTemplateHintVars();
}
/** Paint the live list of bound variable chips into the hints panel. */
function _renderTemplateHintVars() {
const host = document.getElementById('value-source-template-hint-vars');
if (!host) return;
const names = _currentTemplateVarNames();
if (names.length === 0) {
host.innerHTML = `<span class="jinja-hints-empty">${escapeHtml(t('value_source.template.hints.no_inputs'))}</span>`;
return;
}
host.innerHTML = names
.map(n => `<span class="tok-var-chip">${escapeHtml(n)}</span>`)
.join('');
}
// ── Live validation (debounced + generation-tagged) ────────────
interface TemplateValidateResult {
valid: boolean;
error: string | null;
errors: string[];
warnings: string[];
variables: string[];
}
/** Enable/disable the value-source modal's primary (save) button. */
function _setVSSaveEnabled(enabled: boolean) {
const btn = document.querySelector('#value-source-modal .modal-footer .btn-primary') as HTMLButtonElement | null;
if (btn) btn.disabled = !enabled;
}
function _runTemplateValidationDebounced() {
if (_vsTemplateEditor) _vsTemplateEditor.refresh();
_renderTemplateHintVars();
if (_templateValidateTimer) clearTimeout(_templateValidateTimer);
_templateValidateTimer = setTimeout(() => { void _validateTemplate(); }, 300);
}
/** POST the current expression/inputs to the validate endpoint (no UI side effects). */
async function _validateTemplateRequest(
template: string, inputs: TemplateInput[], id?: string,
): Promise<TemplateValidateResult | null> {
try {
const resp = await fetchWithAuth('/value-sources/validate-template', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ template, inputs, ...(id ? { id } : {}) }),
});
if (!resp.ok) return null;
return await resp.json() as TemplateValidateResult;
} catch {
return null;
}
}
/** Live validator: paints inline error/ok/warn state and gates the save button. */
async function _validateTemplate() {
const editor = document.querySelector('#value-source-template-section .jinja-editor') as HTMLElement | null;
const errEl = document.getElementById('value-source-template-error') as HTMLElement | null;
const okEl = document.getElementById('value-source-template-ok') as HTMLElement | null;
const warnEl = document.getElementById('value-source-template-warn') as HTMLElement | null;
const ta = document.getElementById('value-source-template-expression') as HTMLTextAreaElement | null;
if (!ta) return;
// If the user switched away from template while this was pending, do nothing.
if ((document.getElementById('value-source-type') as HTMLSelectElement).value !== 'template') {
_setVSSaveEnabled(true);
return;
}
const template = ta.value;
// Drop blank-name rows: TemplateInput.name is min_length=1 server-side, so
// posting a half-typed row would 422 and silently blank the live validator.
const inputs = _readTemplateInputRows().filter(i => i.name.trim());
const id = (document.getElementById('value-source-id') as HTMLInputElement).value || undefined;
const gen = ++_templateValidateGen;
const result = await _validateTemplateRequest(template, inputs, id);
if (gen !== _templateValidateGen) return; // a newer request superseded this one
// The user may have switched away from template during the await — never
// touch a non-template form's save state.
if ((document.getElementById('value-source-type') as HTMLSelectElement).value !== 'template') {
_setVSSaveEnabled(true);
return;
}
const setSaveEnabled = _setVSSaveEnabled;
const clearMsgs = () => {
if (errEl) { errEl.style.display = 'none'; errEl.textContent = ''; }
if (okEl) { okEl.style.display = 'none'; okEl.innerHTML = ''; }
if (warnEl) { warnEl.style.display = 'none'; warnEl.textContent = ''; }
};
if (!result) {
// Network/endpoint failure — don't block the user; clear inline state.
editor?.classList.remove('field-invalid');
clearMsgs();
setSaveEnabled(true);
return;
}
clearMsgs();
if (result.valid === false) {
editor?.classList.add('field-invalid');
if (errEl) {
errEl.innerHTML = `${ICON_X}<span></span>`;
const span = errEl.querySelector('span');
if (span) span.textContent = (result.errors && result.errors[0]) || result.error || t('value_source.template.error.invalid_expr');
errEl.style.display = '';
}
setSaveEnabled(false);
return;
}
editor?.classList.remove('field-invalid');
setSaveEnabled(true);
if (okEl && template.trim()) {
okEl.innerHTML = `${ICON_CHECK}<span></span>`;
const span = okEl.querySelector('span');
if (span) span.textContent = t('value_source.template.valid');
okEl.style.display = '';
}
if (warnEl && result.warnings && result.warnings.length > 0) {
warnEl.textContent = result.warnings.join(' · ');
warnEl.style.display = '';
}
}
+5
View File
@@ -331,11 +331,13 @@ startTargetOverlay: (...args: any[]) => any;
deleteValueSource: (...args: any[]) => any;
onValueSourceTypeChange: (...args: any[]) => any;
onDaylightVSRealTimeChange: (...args: any[]) => any;
onValueSourceNormalizeChange: (...args: any[]) => any;
addSchedulePoint: (...args: any[]) => any;
addAnimatedColor: (...args: any[]) => any;
removeAnimatedColor: (...args: any[]) => any;
addColorSchedulePoint: (...args: any[]) => any;
removeColorSchedulePoint: (...args: any[]) => any;
addTemplateInput: (...args: any[]) => any;
testValueSource: (...args: any[]) => any;
closeTestValueSourceModal: (...args: any[]) => any;
@@ -377,6 +379,9 @@ startTargetOverlay: (...args: any[]) => any;
graphZoomIn: (...args: any[]) => any;
graphZoomOut: (...args: any[]) => any;
graphRelayout: (...args: any[]) => any;
graphShowIssues: (...args: any[]) => any;
graphExportTopology: (...args: any[]) => any;
graphDuplicateSelection: (...args: any[]) => any;
graphToggleFullscreen: (...args: any[]) => any;
graphAddEntity: (...args: any[]) => any;
+2
View File
@@ -76,6 +76,8 @@ export type {
SystemMetricsValueSource,
GameEventValueSource,
HTTPValueSource,
TemplateValueSource,
TemplateInput,
ValueSource,
ValueSourceListResponse,
} from './types/value-source.ts';
@@ -9,7 +9,12 @@ export type ValueSourceType =
| 'adaptive_time' | 'adaptive_scene' | 'daylight'
| 'static_color' | 'animated_color' | 'adaptive_time_color'
| 'ha_entity' | 'gradient_map' | 'css_extract'
| 'system_metrics' | 'game_event' | 'http';
| 'system_metrics' | 'game_event' | 'http' | 'template';
export interface TemplateInput {
name: string;
value_source_id: string;
}
export interface SchedulePoint {
time: string;
@@ -121,6 +126,7 @@ export interface HAEntityValueSource extends ValueSourceBase {
min_ha_value: number;
max_ha_value: number;
smoothing: number;
normalize: boolean;
}
export interface GradientMapValueSource extends ValueSourceBase {
@@ -150,6 +156,7 @@ export interface SystemMetricsValueSource extends ValueSourceBase {
sensor_label: string;
poll_interval: number;
smoothing: number;
normalize: boolean;
}
export interface GameEventValueSource extends ValueSourceBase {
@@ -173,6 +180,16 @@ export interface HTTPValueSource extends ValueSourceBase {
min_value: number;
max_value: number;
smoothing: number;
normalize: boolean;
}
export interface TemplateValueSource extends ValueSourceBase {
source_type: 'template';
return_type: 'float';
template: string;
inputs: TemplateInput[];
default_value: number;
eval_interval?: number | null;
}
export type ValueSource =
@@ -190,7 +207,8 @@ export type ValueSource =
| CSSExtractValueSource
| SystemMetricsValueSource
| GameEventValueSource
| HTTPValueSource;
| HTTPValueSource
| TemplateValueSource;
export interface ValueSourceListResponse {
sources: ValueSource[];
+64
View File
@@ -362,6 +362,9 @@
"device.mqtt_topic": "MQTT Topic:",
"device.mqtt_topic.hint": "MQTT topic path for publishing pixel data (e.g. mqtt://ledgrab/device/name)",
"device.mqtt_topic.placeholder": "mqtt://ledgrab/device/living-room",
"device.mqtt_source": "MQTT Broker:",
"device.mqtt_source.hint": "Which MQTT broker this device publishes to. Manage brokers under Integrations → MQTT Sources. Leave unset to use the first available broker.",
"device.mqtt_source.none": "— First available broker",
"device.ws_url": "Connection URL:",
"device.ws_url.hint": "WebSocket URL for clients to connect and receive LED data",
"device.openrgb.url": "OpenRGB URL:",
@@ -1982,6 +1985,8 @@
"value_source.daylight.speed.hint": "Cycle speed multiplier. 1.0 = full day/night cycle in ~4 minutes. Higher values cycle faster.",
"value_source.daylight.use_real_time": "Use Real Time:",
"value_source.daylight.use_real_time.hint": "When enabled, the value follows the actual time of day. Speed is ignored.",
"value_source.normalize": "Normalize to 01:",
"value_source.normalize.hint": "On: rescale the raw value to 01 using Min/Max. Off: the value is clamped to 01 as-is (for sources that already report a 01 fraction). The raw value stays available to templates (raw[name]) and automations.",
"value_source.daylight.enable_real_time": "Follow wall clock",
"value_source.daylight.latitude": "Latitude:",
"value_source.daylight.latitude.hint": "Your geographic latitude (-90 to 90). Steepens or flattens the sunrise/sunset edges of the cycle.",
@@ -2582,6 +2587,32 @@
"graph.tooltip.fps": "FPS",
"graph.tooltip.errors": "Errors",
"graph.tooltip.uptime": "Uptime",
"graph.undone": "Undone",
"graph.redone": "Redone",
"graph.action.connect": "Connect",
"graph.action.disconnect": "Disconnect",
"graph.action.move": "Move node",
"graph.action.rewire": "Re-wire slot",
"graph.choose_connection": "Choose connection",
"graph.rewire": "Re-wire…",
"graph.rewire_choose_source": "Choose a new source",
"graph.issues": "Issues",
"graph.issues_none": "No issues found",
"graph.issue.broken_ref": "Broken reference: {field}",
"graph.issue.cycle": "Part of a dependency cycle",
"graph.replace_connection_confirm": "Replace the existing connection?",
"graph.no_compatible_connection": "No compatible connection between these entities",
"graph.create_and_connect": "Create & connect…",
"graph.export": "Export graph (JSON)",
"graph.export_done": "Graph exported",
"graph.export_failed": "Failed to export graph",
"graph.duplicate": "Duplicate selection",
"graph.duplicate_none": "Select one or more nodes to duplicate",
"graph.duplicate_none_eligible": "Nothing duplicable in the selection (only value & colour-strip sources)",
"graph.duplicate_done": "Duplicated {count} source(s)",
"graph.duplicate_done_warn": "Duplicated {count} source(s) — some references could not be remapped",
"graph.duplicate_failed": "Failed to duplicate selection",
"graph.delete_with_dependents_confirm": "This entity is used by {count} other(s): {names}. Delete it and break those connections?",
"automation.enabled": "Automation enabled",
"automation.disabled": "Automation disabled",
"scene_preset.activated": "Preset activated",
@@ -3044,6 +3075,39 @@
"value_source.http.modulator.hint": "Only used when this source drives brightness or color. Automation rules read the raw extracted value and ignore these settings.",
"value_source.http.endpoint_required": "HTTP endpoint is required",
"value_source.http.interval_invalid": "Interval must be at least 1 second",
"value_source.type.template": "Jinja Template",
"value_source.type.template.desc": "Combine bound inputs with a sandboxed Jinja expression to compute a 0-1 value.",
"value_source.template.expression": "Expression:",
"value_source.template.expression.hint": "A sandboxed Jinja expression returning a number. Bound inputs are available by name; use raw[name] for the un-normalized value.",
"value_source.template.expression.placeholder": "clamp((temp - 18) / 10, 0, 1)",
"value_source.template.inputs": "Inputs:",
"value_source.template.inputs.hint": "Bind float value sources to variable names you reference in the expression.",
"value_source.template.inputs.empty": "No inputs yet. Click + to bind a value source.",
"value_source.template.add_input": "+ Add Input",
"value_source.template.input_name": "variable name",
"value_source.template.input_count": "inputs",
"value_source.template.input_count_one": "input",
"value_source.template.default_value": "Default Value:",
"value_source.template.default_value.hint": "Output used when the expression cannot be evaluated (e.g. an input is missing).",
"value_source.template.eval_interval": "Eval Interval (s):",
"value_source.template.eval_interval.hint": "How often to re-evaluate the expression. 0 = every poll (re-evaluate as fast as the inputs update).",
"value_source.template.valid": "Expression is valid",
"value_source.template.hints.title": "Expression help",
"value_source.template.hints.inputs_title": "Bound inputs",
"value_source.template.hints.no_inputs": "No inputs bound yet",
"value_source.template.hints.globals_title": "Globals",
"value_source.template.hints.globals": "min(a, b), max(a, b), abs(x), round(x), clamp(x, lo, hi)",
"value_source.template.hints.raw_title": "Raw values",
"value_source.template.hints.raw": "raw[name] gives the un-normalized value of an input that has one.",
"value_source.template.hints.examples_title": "Examples",
"value_source.template.hints.time": "Tip: for time-of-day logic, bind an Adaptive (Time) or Daylight source as an input.",
"value_source.template.error.invalid_expr": "Expression is invalid",
"value_source.template.error.cycle": "This expression would create a dependency cycle",
"value_source.template.error.missing_input": "Every input needs a variable name",
"value_source.template.error.invalid_name": "Invalid variable name",
"value_source.template.error.reserved_name": "Reserved name cannot be used as an input",
"value_source.template.error.duplicate_name": "Duplicate input name",
"value_source.template.error.unbound": "Expression references an unbound variable",
"automations.rule.http_poll": "HTTP Poll",
"automations.rule.http_poll.desc": "Activate when the latest extracted value from an HTTP value source matches.",
"automations.rule.http_poll.hint": "Compares the latest extracted value against your input. The value source decides what gets extracted (raw body or JSON path).",
+64
View File
@@ -417,6 +417,9 @@
"device.mqtt_topic": "MQTT Топик:",
"device.mqtt_topic.hint": "MQTT топик для публикации пиксельных данных (напр. mqtt://ledgrab/device/name)",
"device.mqtt_topic.placeholder": "mqtt://ledgrab/device/гостиная",
"device.mqtt_source": "MQTT Брокер:",
"device.mqtt_source.hint": "К какому MQTT-брокеру публикует это устройство. Брокеры настраиваются в разделе Интеграции → Источники MQTT. Оставьте пустым, чтобы использовать первый доступный брокер.",
"device.mqtt_source.none": "— Первый доступный брокер",
"device.ws_url": "URL подключения:",
"device.ws_url.hint": "WebSocket URL для подключения клиентов и получения LED данных",
"device.openrgb.url": "OpenRGB URL:",
@@ -1842,6 +1845,8 @@
"value_source.daylight.speed.hint": "Множитель скорости цикла. 1.0 = полный цикл день/ночь за ~4 минуты.",
"value_source.daylight.use_real_time": "Реальное время:",
"value_source.daylight.use_real_time.hint": "Значение следует за реальным временем суток. Скорость игнорируется.",
"value_source.normalize": "Нормализовать в 0–1:",
"value_source.normalize.hint": "Вкл.: масштабировать сырое значение в 0–1 по Min/Max. Выкл.: значение ограничивается диапазоном 0–1 как есть (для источников, уже выдающих долю 0–1). Сырое значение остаётся доступным в шаблонах (raw[name]) и автоматизациях.",
"value_source.daylight.enable_real_time": "Следовать за часами",
"value_source.daylight.latitude": "Широта:",
"value_source.daylight.latitude.hint": "Географическая широта (-90 до 90). Делает переходы рассвета и заката круче или плавнее.",
@@ -2264,6 +2269,32 @@
"graph.tooltip.fps": "FPS",
"graph.tooltip.errors": "Ошибки",
"graph.tooltip.uptime": "Время работы",
"graph.undone": "Отменено",
"graph.redone": "Повторено",
"graph.action.connect": "Соединить",
"graph.action.disconnect": "Отсоединить",
"graph.action.move": "Переместить узел",
"graph.action.rewire": "Переподключить слот",
"graph.choose_connection": "Выберите соединение",
"graph.rewire": "Переподключить…",
"graph.rewire_choose_source": "Выберите новый источник",
"graph.issues": "Проблемы",
"graph.issues_none": "Проблем не найдено",
"graph.issue.broken_ref": "Битая ссылка: {field}",
"graph.issue.cycle": "Входит в цикл зависимостей",
"graph.replace_connection_confirm": "Заменить существующее соединение?",
"graph.no_compatible_connection": "Нет совместимого соединения между этими объектами",
"graph.create_and_connect": "Создать и соединить…",
"graph.export": "Экспорт графа (JSON)",
"graph.export_done": "Граф экспортирован",
"graph.export_failed": "Не удалось экспортировать граф",
"graph.duplicate": "Дублировать выбранное",
"graph.duplicate_none": "Выберите один или несколько узлов для дублирования",
"graph.duplicate_none_eligible": "В выборе нечего дублировать (только источники значений и цветовых лент)",
"graph.duplicate_done": "Продублировано источников: {count}",
"graph.duplicate_done_warn": "Продублировано источников: {count} — часть ссылок не удалось перепривязать",
"graph.duplicate_failed": "Не удалось дублировать выбранное",
"graph.delete_with_dependents_confirm": "Этот объект используется {count} другими: {names}. Удалить и разорвать эти связи?",
"automation.enabled": "Автоматизация включена",
"automation.disabled": "Автоматизация выключена",
"scene_preset.activated": "Пресет активирован",
@@ -2726,6 +2757,39 @@
"value_source.http.modulator.hint": "Используется только когда этот источник управляет яркостью или цветом. Правила автоматизации читают извлечённое значение в исходном виде и игнорируют эти настройки.",
"value_source.http.endpoint_required": "Требуется HTTP-эндпоинт",
"value_source.http.interval_invalid": "Интервал должен быть не меньше 1 секунды",
"value_source.type.template": "Шаблон Jinja",
"value_source.type.template.desc": "Объедините привязанные входы в изолированном выражении Jinja для вычисления значения 0-1.",
"value_source.template.expression": "Выражение:",
"value_source.template.expression.hint": "Изолированное выражение Jinja, возвращающее число. Привязанные входы доступны по имени; используйте raw[name] для ненормализованного значения.",
"value_source.template.expression.placeholder": "clamp((temp - 18) / 10, 0, 1)",
"value_source.template.inputs": "Входы:",
"value_source.template.inputs.hint": "Привяжите числовые источники значений к именам переменных, используемым в выражении.",
"value_source.template.inputs.empty": "Пока нет входов. Нажмите +, чтобы привязать источник значений.",
"value_source.template.add_input": "+ Добавить вход",
"value_source.template.input_name": "имя переменной",
"value_source.template.input_count": "входов",
"value_source.template.input_count_one": "вход",
"value_source.template.default_value": "Значение по умолчанию:",
"value_source.template.default_value.hint": "Значение на выходе, когда выражение нельзя вычислить (например, отсутствует вход).",
"value_source.template.eval_interval": "Интервал вычисления (с):",
"value_source.template.eval_interval.hint": "Как часто пересчитывать выражение. 0 = при каждом опросе (так быстро, как обновляются входы).",
"value_source.template.valid": "Выражение корректно",
"value_source.template.hints.title": "Справка по выражению",
"value_source.template.hints.inputs_title": "Привязанные входы",
"value_source.template.hints.no_inputs": "Входы ещё не привязаны",
"value_source.template.hints.globals_title": "Глобальные функции",
"value_source.template.hints.globals": "min(a, b), max(a, b), abs(x), round(x), clamp(x, lo, hi)",
"value_source.template.hints.raw_title": "Исходные значения",
"value_source.template.hints.raw": "raw[name] даёт ненормализованное значение входа, если оно есть.",
"value_source.template.hints.examples_title": "Примеры",
"value_source.template.hints.time": "Совет: для логики времени суток привяжите источник «Адаптивный (время)» или «Дневной цикл» как вход.",
"value_source.template.error.invalid_expr": "Некорректное выражение",
"value_source.template.error.cycle": "Это выражение создало бы циклическую зависимость",
"value_source.template.error.missing_input": "Каждому входу нужно имя переменной",
"value_source.template.error.invalid_name": "Некорректное имя переменной",
"value_source.template.error.reserved_name": "Зарезервированное имя нельзя использовать как вход",
"value_source.template.error.duplicate_name": "Повторяющееся имя входа",
"value_source.template.error.unbound": "Выражение ссылается на непривязанную переменную",
"automations.rule.http_poll": "HTTP-опрос",
"automations.rule.http_poll.desc": "Срабатывает, когда последнее значение HTTP-источника соответствует условию.",
"automations.rule.http_poll.hint": "Сравнивает последнее извлечённое значение с вашим вводом. Что именно извлекается (тело или JSON-путь), задаётся в источнике-значении.",
+64
View File
@@ -415,6 +415,9 @@
"device.mqtt_topic": "MQTT 主题:",
"device.mqtt_topic.hint": "用于发布像素数据的 MQTT 主题路径(例如 mqtt://ledgrab/device/name",
"device.mqtt_topic.placeholder": "mqtt://ledgrab/device/客厅",
"device.mqtt_source": "MQTT 代理:",
"device.mqtt_source.hint": "此设备发布到哪个 MQTT 代理。在集成 → MQTT 源中管理代理。留空则使用第一个可用代理。",
"device.mqtt_source.none": "— 第一个可用代理",
"device.ws_url": "连接 URL",
"device.ws_url.hint": "客户端连接并接收 LED 数据的 WebSocket URL",
"device.openrgb.url": "OpenRGB URL",
@@ -1838,6 +1841,8 @@
"value_source.daylight.speed.hint": "周期速度倍率。1.0 = 完整日夜周期约4分钟。",
"value_source.daylight.use_real_time": "使用实时:",
"value_source.daylight.use_real_time.hint": "启用后,数值跟随实际时间。速度设置将被忽略。",
"value_source.normalize": "归一化到 01",
"value_source.normalize.hint": "开启:使用最小/最大值将原始值缩放到 0–1。关闭:直接将数值钳制到 0–1(适用于本身就输出 0–1 比例的来源)。原始值始终可在模板(raw[name])和自动化中使用。",
"value_source.daylight.enable_real_time": "跟随系统时钟",
"value_source.daylight.latitude": "纬度:",
"value_source.daylight.latitude.hint": "地理纬度(-90到90)。使日出/日落过渡更陡峭或更平缓。",
@@ -2260,6 +2265,32 @@
"graph.tooltip.fps": "帧率",
"graph.tooltip.errors": "错误",
"graph.tooltip.uptime": "运行时间",
"graph.undone": "已撤销",
"graph.redone": "已重做",
"graph.action.connect": "连接",
"graph.action.disconnect": "断开连接",
"graph.action.move": "移动节点",
"graph.action.rewire": "重新连接槽位",
"graph.choose_connection": "选择连接",
"graph.rewire": "重新连接…",
"graph.rewire_choose_source": "选择新的来源",
"graph.issues": "问题",
"graph.issues_none": "未发现问题",
"graph.issue.broken_ref": "无效引用:{field}",
"graph.issue.cycle": "属于依赖循环",
"graph.replace_connection_confirm": "替换现有连接?",
"graph.no_compatible_connection": "这些实体之间没有兼容的连接",
"graph.create_and_connect": "创建并连接…",
"graph.export": "导出图谱 (JSON)",
"graph.export_done": "图谱已导出",
"graph.export_failed": "导出图谱失败",
"graph.duplicate": "复制所选",
"graph.duplicate_none": "请选择一个或多个节点以复制",
"graph.duplicate_none_eligible": "所选内容中没有可复制的项(仅值源和色带源)",
"graph.duplicate_done": "已复制 {count} 个源",
"graph.duplicate_done_warn": "已复制 {count} 个源 — 部分引用无法重新映射",
"graph.duplicate_failed": "复制所选失败",
"graph.delete_with_dependents_confirm": "此实体被 {count} 个其他实体引用:{names}。删除并断开这些连接?",
"automation.enabled": "自动化已启用",
"automation.disabled": "自动化已禁用",
"scene_preset.activated": "预设已激活",
@@ -2720,6 +2751,39 @@
"value_source.http.modulator.hint": "仅当此源用于驱动亮度或颜色时使用。自动化规则会直接读取提取的原始值,并忽略这些设置。",
"value_source.http.endpoint_required": "需要 HTTP 端点",
"value_source.http.interval_invalid": "间隔至少为 1 秒",
"value_source.type.template": "Jinja 模板",
"value_source.type.template.desc": "使用沙盒化的 Jinja 表达式组合已绑定的输入,计算出 0-1 的值。",
"value_source.template.expression": "表达式:",
"value_source.template.expression.hint": "返回数字的沙盒化 Jinja 表达式。已绑定的输入可按名称使用;使用 raw[name] 获取未归一化的值。",
"value_source.template.expression.placeholder": "clamp((temp - 18) / 10, 0, 1)",
"value_source.template.inputs": "输入:",
"value_source.template.inputs.hint": "将浮点值源绑定到你在表达式中引用的变量名。",
"value_source.template.inputs.empty": "暂无输入。点击 + 绑定一个值源。",
"value_source.template.add_input": "+ 添加输入",
"value_source.template.input_name": "变量名",
"value_source.template.input_count": "个输入",
"value_source.template.input_count_one": "个输入",
"value_source.template.default_value": "默认值:",
"value_source.template.default_value.hint": "当表达式无法求值(例如缺少输入)时使用的输出值。",
"value_source.template.eval_interval": "求值间隔(秒):",
"value_source.template.eval_interval.hint": "重新求值表达式的频率。0 = 每次轮询(随输入更新一样快地重新求值)。",
"value_source.template.valid": "表达式有效",
"value_source.template.hints.title": "表达式帮助",
"value_source.template.hints.inputs_title": "已绑定的输入",
"value_source.template.hints.no_inputs": "尚未绑定输入",
"value_source.template.hints.globals_title": "全局函数",
"value_source.template.hints.globals": "min(a, b), max(a, b), abs(x), round(x), clamp(x, lo, hi)",
"value_source.template.hints.raw_title": "原始值",
"value_source.template.hints.raw": "raw[name] 给出某个输入的未归一化值(如果有)。",
"value_source.template.hints.examples_title": "示例",
"value_source.template.hints.time": "提示:要实现一天中的时间逻辑,请将“自适应(时间)”或“日光周期”源绑定为输入。",
"value_source.template.error.invalid_expr": "表达式无效",
"value_source.template.error.cycle": "该表达式会造成依赖循环",
"value_source.template.error.missing_input": "每个输入都需要一个变量名",
"value_source.template.error.invalid_name": "变量名无效",
"value_source.template.error.reserved_name": "保留名称不能用作输入",
"value_source.template.error.duplicate_name": "输入名称重复",
"value_source.template.error.unbound": "表达式引用了未绑定的变量",
"automations.rule.http_poll": "HTTP 轮询",
"automations.rule.http_poll.desc": "当 HTTP 值源的最新提取值匹配时激活。",
"automations.rule.http_poll.hint": "将最新的提取值与您的输入进行比较。提取的内容(原始响应体或 JSON 路径)由值源决定。",
+223 -10
View File
@@ -4,20 +4,233 @@
* Strategy:
* - Static assets (/static/): stale-while-revalidate
* - API / config requests: network-only (device control must be live)
* - Navigation: network-first with offline fallback
* - Navigation: network-first with branded offline fallback
*/
const CACHE_NAME = 'ledgrab-v34';
const CACHE_NAME = 'ledgrab-v35';
// Only pre-cache static assets (no auth required).
// Do NOT pre-cache '/' — it requires API key auth and would cache an error page.
// The Orbitron brand font is precached so the offline page renders on-brand
// even on a device that hasn't warmed the font cache yet.
const PRECACHE_URLS = [
'/static/dist/app.bundle.css',
'/static/dist/app.bundle.js',
'/static/icons/icon-192.png',
'/static/icons/icon-512.png',
'/static/fonts/orbitron-700-latin.woff2',
];
// Branded offline fallback shown when a navigation can't reach the server.
// Self-contained (no CDN, no app CSS) so it renders with zero live network.
// Mirrors the "Lumenworks" console aesthetic: pure-black panel, Orbitron brand
// mark, channel-coral for the offline/alarm state, signal-green for restore.
// A background probe self-heals the page — it reloads the instant the server
// answers again, so a restarting server no longer leaves a dead-end screen.
const OFFLINE_HTML = `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<meta name="color-scheme" content="dark light">
<title>LED Grab Signal Lost</title>
<style>
*{margin:0;padding:0;box-sizing:border-box}
:root{
--bg:#000;--line:#1c2027;
--ink:#eef2f7;--dim:#aeb7c4;--mute:#6b7480;
--coral:#ff5e5e;
--signal:#4caf50;--signal-hi:#6fd173;
--glow-coral:0 0 18px rgba(255,94,94,.55);
--glow-signal:0 0 10px rgba(76,175,80,.8);
}
@media (prefers-color-scheme: light){
:root{
--bg:#f6f8fb;--line:#dee3ea;
--ink:#0f1419;--dim:#41505f;--mute:#7b8694;
--coral:#d8392e;
--signal:#2e7d32;--signal-hi:#3d8b40;
--glow-coral:0 0 16px rgba(216,57,46,.30);
--glow-signal:0 0 10px rgba(46,125,50,.5);
}
}
@font-face{
font-family:'Orbitron';font-style:normal;font-weight:700;font-display:swap;
src:url('/static/fonts/orbitron-700-latin.woff2') format('woff2');
}
html,body{height:100%}
body{
background:var(--bg);color:var(--ink);
font-family:'Manrope','Segoe UI',system-ui,-apple-system,BlinkMacSystemFont,sans-serif;
-webkit-font-smoothing:antialiased;text-rendering:optimizeLegibility;
display:grid;place-items:center;min-height:100dvh;position:relative;overflow:hidden;
padding:calc(28px + env(safe-area-inset-top)) 24px calc(28px + env(safe-area-inset-bottom));
}
/* atmosphere: coral alarm vignette + faint signal floor */
body::before{
content:'';position:fixed;inset:0;z-index:0;pointer-events:none;
background:
radial-gradient(125% 80% at 50% -12%, rgba(255,94,94,.11), transparent 60%),
radial-gradient(100% 55% at 50% 118%, rgba(0,216,255,.045), transparent 60%);
}
/* fine equipment-panel scanlines */
body::after{
content:'';position:fixed;inset:0;z-index:0;pointer-events:none;opacity:.5;
mix-blend-mode:screen;
background:repeating-linear-gradient(0deg, rgba(255,255,255,.018) 0 1px, transparent 1px 3px);
}
@media (prefers-color-scheme: light){
body::after{opacity:.4;mix-blend-mode:multiply;
background:repeating-linear-gradient(0deg, rgba(0,0,0,.02) 0 1px, transparent 1px 3px)}
}
.panel{position:relative;z-index:1;width:min(460px,100%);
display:flex;flex-direction:column;align-items:center;text-align:center}
.panel>*{opacity:0;transform:translateY(10px);animation:rise .6s cubic-bezier(.16,1,.3,1) forwards}
.brand{animation-delay:.05s}.strip-wrap{animation-delay:.14s}.chip{animation-delay:.22s}
.headline{animation-delay:.30s}.copy{animation-delay:.38s}.btn{animation-delay:.46s}
.telemetry{animation-delay:.54s}.foot{animation-delay:.62s}
@keyframes rise{to{opacity:1;transform:none}}
.brand{display:flex;align-items:center;gap:11px;margin-bottom:32px}
.brand-dot{width:9px;height:9px;border-radius:50%;background:var(--coral);
box-shadow:var(--glow-coral);animation:beat 1.6s ease-in-out infinite}
.brand-name{font-family:'Orbitron','Segoe UI',sans-serif;font-weight:700;
font-size:.95rem;letter-spacing:.34em;text-indent:.34em;color:var(--ink)}
@keyframes beat{0%,100%{opacity:1}50%{opacity:.28}}
.strip-wrap{width:100%;margin-bottom:28px}
.strip{position:relative;display:flex;gap:7px;padding:15px 12px;overflow:hidden;
border:1px solid var(--line);border-radius:13px;
background:linear-gradient(180deg, rgba(255,255,255,.025), transparent)}
.strip i{flex:1 1 0;height:11px;border-radius:3px;background:var(--coral);opacity:.16;
transition:opacity .45s ease, background .45s ease, box-shadow .45s ease}
.strip::before{content:'';position:absolute;top:0;bottom:0;width:30%;left:-40%;
mix-blend-mode:screen;filter:blur(7px);
background:linear-gradient(90deg, transparent, var(--coral), transparent);
animation:sweep 2.4s linear infinite}
@keyframes sweep{0%{left:-42%}100%{left:112%}}
.chip{display:inline-flex;align-items:center;gap:9px;margin-bottom:24px;
font-family:'JetBrains Mono',ui-monospace,'Cascadia Code',monospace;
font-size:.66rem;font-weight:600;letter-spacing:.22em;text-transform:uppercase;
color:var(--coral);padding:5px 14px;border-radius:100px;
border:1px solid color-mix(in srgb, var(--coral) 38%, transparent)}
.chip b{width:7px;height:7px;border-radius:1px;background:var(--coral);
box-shadow:var(--glow-coral);animation:beat 1.6s ease-in-out infinite}
.headline{font-family:'Orbitron','Segoe UI',sans-serif;font-weight:700;
font-size:clamp(2.5rem,11.5vw,3.6rem);line-height:.92;letter-spacing:.015em;
text-transform:uppercase;color:var(--coral);text-shadow:0 0 26px rgba(255,94,94,.34);
margin-bottom:20px;animation:rise .6s cubic-bezier(.16,1,.3,1) forwards, flicker 5.5s 1.4s ease-in-out infinite}
.headline span{display:block;color:var(--ink);text-shadow:none}
@keyframes flicker{0%,93%,100%{opacity:1}94%{opacity:.55}96%{opacity:.85}97%{opacity:.4}}
.copy{max-width:35ch;color:var(--dim);font-size:.99rem;line-height:1.62;margin-bottom:32px}
.btn{position:relative;overflow:hidden;cursor:pointer;border:none;border-radius:11px;
padding:14px 36px;color:#04140a;background:var(--signal);
font-family:'JetBrains Mono',ui-monospace,monospace;font-weight:700;
font-size:.8rem;letter-spacing:.18em;text-transform:uppercase;
box-shadow:0 0 0 1px color-mix(in srgb, var(--signal) 55%, transparent), 0 10px 30px rgba(76,175,80,.28);
transition:transform .15s ease, box-shadow .25s ease, background .25s ease}
.btn:hover{background:var(--signal-hi);box-shadow:0 0 0 1px var(--signal), 0 14px 40px rgba(76,175,80,.42)}
.btn:active{transform:translateY(1px) scale(.99)}
.btn:focus-visible{outline:none;box-shadow:0 0 0 3px var(--bg), 0 0 0 5px var(--signal)}
.btn[disabled]{cursor:default;opacity:.7}
.btn::after{content:'';position:absolute;inset:0;transform:translateX(-130%);
background:linear-gradient(90deg, transparent, rgba(255,255,255,.38), transparent)}
.btn:not([disabled]):hover::after{animation:sheen .85s ease}
@keyframes sheen{to{transform:translateX(130%)}}
.telemetry{margin-top:18px;min-height:1.1em;
font-family:'JetBrains Mono',ui-monospace,monospace;
font-size:.7rem;letter-spacing:.12em;color:var(--mute)}
.foot{margin-top:34px;font-family:'JetBrains Mono',ui-monospace,monospace;
font-size:.6rem;letter-spacing:.3em;text-transform:uppercase;color:var(--mute);opacity:.65}
/* ── link-restored state ── */
body.online .strip i{opacity:1;background:var(--signal);box-shadow:var(--glow-signal)}
body.online .strip::before{display:none}
body.online .brand-dot,body.online .chip b{background:var(--signal);box-shadow:var(--glow-signal);animation:none}
body.online .chip{color:var(--signal);border-color:color-mix(in srgb, var(--signal) 45%, transparent)}
body.online .headline{animation:none}
@media (prefers-reduced-motion: reduce){
.panel>*,.headline{animation:none;opacity:1;transform:none}
.strip::before{display:none}.strip i{opacity:.5}
.brand-dot,.chip b{animation:none}.btn:hover::after{animation:none}
}
</style>
</head>
<body>
<main class="panel" role="alert" aria-live="assertive">
<div class="brand"><span class="brand-dot" aria-hidden="true"></span><span class="brand-name">LED&nbsp;GRAB</span></div>
<div class="strip-wrap"><div class="strip" aria-hidden="true"><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i></div></div>
<p class="chip"><b aria-hidden="true"></b><span id="chip-text">No Signal</span></p>
<h1 class="headline"><span>Signal</span>Lost</h1>
<p class="copy">Can&rsquo;t reach the LED&nbsp;Grab server. Make sure it&rsquo;s running and your device is on the same network.</p>
<button id="retry" class="btn" type="button"><span id="btn-label">Reconnect</span></button>
<p class="telemetry" id="telemetry" aria-live="polite">Listening for the server&hellip;</p>
<p class="foot">LED&nbsp;Grab &middot; Local Console</p>
</main>
<script>
(function(){
var RETRY = 3; // seconds between automatic probes
var attempts = 0, checking = false, done = false, timer = null;
var tel = document.getElementById('telemetry');
var chipText = document.getElementById('chip-text');
var btn = document.getElementById('retry');
var btnLabel = document.getElementById('btn-label');
function pad(n){ return n < 10 ? '0' + n : '' + n; }
function say(m){ if (tel) tel.textContent = m; }
function probe(){
if (checking || done) return;
checking = true; attempts++;
if (btn) btn.disabled = true;
if (btnLabel) btnLabel.textContent = 'Checking';
say('Probing for signal · attempt ' + pad(attempts));
// Any settled response (even 401/403) means the server is reachable.
// Cache-bust + no-store so a stale SW cache can't fake a recovery.
fetch('/?_swping=' + Date.now(), { method: 'HEAD', cache: 'no-store' })
.then(restored)
.catch(function(){
checking = false;
if (btn) btn.disabled = false;
if (btnLabel) btnLabel.textContent = 'Reconnect';
countdown(RETRY);
});
}
function restored(){
done = true;
document.body.classList.add('online');
if (chipText) chipText.textContent = 'Link Restored';
if (btnLabel) btnLabel.textContent = 'Reconnecting';
say('Signal acquired — reloading');
setTimeout(function(){ location.reload(); }, 750);
}
function countdown(s){
if (done) return;
if (s <= 0){ probe(); return; }
say('Retrying in ' + pad(s) + 's · attempt ' + pad(attempts + 1));
timer = setTimeout(function(){ countdown(s - 1); }, 1000);
}
function probeNow(){ if (timer){ clearTimeout(timer); timer = null; } probe(); }
if (btn) btn.addEventListener('click', probeNow);
window.addEventListener('online', probeNow);
document.addEventListener('visibilitychange', function(){ if (!document.hidden) probeNow(); });
probe();
})();
</script>
</body>
</html>`;
// Install: pre-cache core shell
self.addEventListener('install', (event) => {
event.waitUntil(
@@ -66,17 +279,17 @@ self.addEventListener('fetch', (event) => {
return;
}
// Navigation: network-only (page requires auth, no useful offline fallback)
// Navigation: network-first with branded, self-healing offline fallback
if (event.request.mode === 'navigate') {
event.respondWith(
fetch(event.request).catch(() =>
new Response(
'<html><body style="font-family:system-ui;text-align:center;padding:60px 20px;background:#1a1a1a;color:#ccc">' +
'<h2>LED Grab</h2><p>Cannot reach the server. Check that it is running and you are on the same network.</p>' +
'<button onclick="location.reload()" style="margin-top:20px;padding:10px 24px;border-radius:8px;border:none;background:#4CAF50;color:#fff;font-size:1rem;cursor:pointer">Retry</button>' +
'</body></html>',
{ status: 503, headers: { 'Content-Type': 'text/html' } }
)
new Response(OFFLINE_HTML, {
status: 503,
headers: {
'Content-Type': 'text/html; charset=utf-8',
'Cache-Control': 'no-store',
},
})
)
);
return;
@@ -6,7 +6,10 @@ writes go through to SQLite immediately (write-through cache).
"""
import asyncio
import copy
import threading
import uuid
from datetime import datetime, timezone
from typing import Callable, Dict, Generic, List, TypeVar
from ledgrab.storage.database import Database
@@ -26,6 +29,10 @@ class BaseSqliteStore(Generic[T]):
_table_name: str
_entity_name: str
# Opt-in allowlist for clone(): defaults off so a new (possibly
# secret-bearing) store is never cloneable by accident. Subclasses that hold
# no inline secrets and are safe to duplicate set this True.
_cloneable: bool = False
def __init__(self, db: Database, deserializer: Callable[[dict], T]):
self._db = db
@@ -136,13 +143,50 @@ class BaseSqliteStore(Generic[T]):
await asyncio.to_thread(self._delete_item, item_id)
logger.info(f"Deleted {self._entity_name}: {item_id}")
def clone(self, item_id: str, new_name: str) -> T:
"""Faithfully duplicate an entity under a new id and name.
Deep-copies every field of the original (no serialize/deserialize
round-trip, so no field can be silently lost to a schema/dataclass name
mismatch), mints a fresh id that preserves the original's id prefix,
applies ``new_name`` and resets timestamps. References *inside* the clone
still point at whatever the original referenced callers that want to
rewire intra-set references must do so after cloning.
SECURITY: this copies *every* field verbatim, including any secret a
model might hold. Callers must restrict cloning to non-secret-bearing
kinds (see ``_DUPLICABLE_KINDS`` in ``api/routes/graph.py``).
"""
if not self._cloneable:
raise NotImplementedError(f"{self._entity_name} store does not support cloning")
with self._lock:
if item_id not in self._items:
from ledgrab.storage.base_store import EntityNotFoundError
raise EntityNotFoundError(f"{self._entity_name} not found: {item_id}")
self._check_name_unique(new_name)
new = copy.deepcopy(self._items[item_id])
prefix = item_id.rsplit("_", 1)[0] if "_" in item_id else item_id
new_id = f"{prefix}_{uuid.uuid4().hex[:8]}"
now = datetime.now(timezone.utc)
new.id = new_id
new.name = new_name
if hasattr(new, "created_at"):
new.created_at = now
if hasattr(new, "updated_at"):
new.updated_at = now
self._items[new_id] = new
self._save_item(new_id, new)
logger.info("Cloned %s %s -> %s (%s)", self._entity_name, item_id, new_id, new_name)
return new
def count(self) -> int:
with self._lock:
return len(self._items)
# -- Helpers -------------------------------------------------------------
def _check_name_unique(self, name: str, exclude_id: str = None) -> None:
def _check_name_unique(self, name: str, exclude_id: str | None = None) -> None:
"""Raise ValueError if *name* is empty or already taken.
Must be called while holding ``self._lock``.
@@ -33,6 +33,7 @@ class ColorStripStore(BaseSqliteStore[ColorStripSource]):
_table_name = "color_strip_sources"
_entity_name = "Color strip source"
_cloneable = True # no inline secrets — only references shared entities by id
def __init__(self, db: Database):
super().__init__(db, ColorStripSource.from_dict)
@@ -66,6 +66,11 @@ class ValueSource:
"use_real_time": None,
"latitude": None,
"longitude": None,
# Template (Jinja expression combinator)
"template": None,
"inputs": None,
"default_value": None,
"eval_interval": None,
}
if self.icon:
d["icon"] = self.icon
@@ -391,6 +396,11 @@ class HAEntityValueSource(ValueSource):
min_ha_value: float = 0.0 # raw HA value mapped to output 0.0
max_ha_value: float = 100.0 # raw HA value mapped to output 1.0
smoothing: float = 0.0 # EMA smoothing factor (0.01.0)
# When False, skip the min/max rescale: get_value() clamps the raw value
# into [0,1] as-is (for entities that already report a 01 fraction). The
# un-clamped magnitude stays available via get_raw_value(). get_value() is
# always in [0,1] regardless, so the normalized scalar bus is preserved.
normalize: bool = True
def to_dict(self) -> dict:
d = super().to_dict()
@@ -400,6 +410,7 @@ class HAEntityValueSource(ValueSource):
d["min_ha_value"] = self.min_ha_value
d["max_ha_value"] = self.max_ha_value
d["smoothing"] = self.smoothing
d["normalize"] = self.normalize
return d
@classmethod
@@ -416,6 +427,7 @@ class HAEntityValueSource(ValueSource):
data.get("max_ha_value") if data.get("max_ha_value") is not None else 100.0
),
smoothing=float(data.get("smoothing") or 0.0),
normalize=bool(data.get("normalize", True)),
)
@@ -514,6 +526,13 @@ class GameEventValueSource(ValueSource):
smoothing: float = 0.0 # EMA smoothing factor (0.0-1.0)
default_value: float = 0.5 # value when timed out or no events
timeout: float = 5.0 # seconds before reverting to default
# When False, skip the min/max rescale: get_value() clamps the raw game
# value into [0,1] as-is. The un-clamped value stays available via
# get_raw_value(). get_value() is always in [0,1]. See HAEntityValueSource.
# NOTE: game_event has no value-source CRUD schema/API (constructed only via
# the game-integration path), so this flag is settable only there, not over
# POST/PUT /value-sources.
normalize: bool = True
def to_dict(self) -> dict:
d = super().to_dict()
@@ -524,6 +543,7 @@ class GameEventValueSource(ValueSource):
d["smoothing"] = self.smoothing
d["default_value"] = self.default_value
d["timeout"] = self.timeout
d["normalize"] = self.normalize
return d
@classmethod
@@ -543,6 +563,7 @@ class GameEventValueSource(ValueSource):
data.get("default_value") if data.get("default_value") is not None else 0.5
),
timeout=float(data.get("timeout") if data.get("timeout") is not None else 5.0),
normalize=bool(data.get("normalize", True)),
)
@@ -562,6 +583,10 @@ class SystemMetricsValueSource(ValueSource):
sensor_label: str = "" # for cpu_temp/fan_speed (empty = first available)
poll_interval: float = 1.0 # seconds between reads
smoothing: float = 0.0 # EMA smoothing factor
# When False, skip the min/max rescale: get_value() clamps the raw metric
# into [0,1] as-is. The un-clamped reading stays available via
# get_raw_value(). get_value() is always in [0,1]. See HAEntityValueSource.
normalize: bool = True
def to_dict(self) -> dict:
d = super().to_dict()
@@ -573,6 +598,7 @@ class SystemMetricsValueSource(ValueSource):
d["sensor_label"] = self.sensor_label
d["poll_interval"] = self.poll_interval
d["smoothing"] = self.smoothing
d["normalize"] = self.normalize
return d
@classmethod
@@ -591,6 +617,7 @@ class SystemMetricsValueSource(ValueSource):
sensor_label=data.get("sensor_label") or "",
poll_interval=float(data.get("poll_interval") or 1.0),
smoothing=float(data.get("smoothing") or 0.0),
normalize=bool(data.get("normalize", True)),
)
@@ -617,6 +644,11 @@ class HTTPValueSource(ValueSource):
min_value: float = 0.0 # raw value → 0.0
max_value: float = 100.0 # raw value → 1.0
smoothing: float = 0.0 # EMA smoothing on the normalized output
# When False, skip the min/max rescale: get_value() coerces the extracted
# value to float and clamps it into [0,1] as-is. The verbatim extracted
# value (which may be str/bool) stays available via get_raw_value().
# get_value() is always a float in [0,1]. See HAEntityValueSource.
normalize: bool = True
def to_dict(self) -> dict:
d = super().to_dict()
@@ -626,6 +658,7 @@ class HTTPValueSource(ValueSource):
d["min_value"] = self.min_value
d["max_value"] = self.max_value
d["smoothing"] = self.smoothing
d["normalize"] = self.normalize
return d
@classmethod
@@ -640,6 +673,70 @@ class HTTPValueSource(ValueSource):
min_value=float(data.get("min_value") or 0.0),
max_value=float(data.get("max_value") if data.get("max_value") is not None else 100.0),
smoothing=float(data.get("smoothing") or 0.0),
normalize=bool(data.get("normalize", True)),
)
def _coerce_float(value, fallback):
"""Best-effort float; returns ``fallback`` on None/non-numeric.
Keeps a tampered DB / buggy migration from dropping the whole row when
BaseSqliteStore's loader swallows a per-row deserialization error.
"""
try:
return float(value)
except (TypeError, ValueError):
return fallback
@dataclass
class TemplateValueSource(ValueSource):
"""Value source that evaluates a sandboxed Jinja expression over the live
values of other value sources (the system's float combinator).
``template`` is a Jinja *expression* (no statements/blocks) evaluated by the
hardened engine in :mod:`ledgrab.utils.template_expr`. Each entry in
``inputs`` binds a variable ``name`` to another value source by id; at
runtime the variable holds that source's normalized ``get_value()`` (0..1)
and ``raw[name]`` holds its un-normalized ``get_raw_value()`` (float) where
the stream exposes one. The callables ``min``/``max``/``abs``/``round``/
``clamp`` are available. The result is coerced to float, NaN/inf rejected,
and clamped to [0, 1]; any error falls back to ``default_value``.
"""
template: str = ""
inputs: List[dict] = field(default_factory=list) # [{name, value_source_id}]
default_value: float = 0.0 # fallback when the expression errors (0.0-1.0)
eval_interval: float | None = None # re-eval throttle (s); None/0 = every poll
def to_dict(self) -> dict:
d = super().to_dict()
d["template"] = self.template
d["inputs"] = [dict(i) for i in self.inputs]
d["default_value"] = self.default_value
d["eval_interval"] = self.eval_interval
d["return_type"] = "float"
return d
@classmethod
def from_dict(cls, data: dict) -> "TemplateValueSource":
common = _parse_common_fields(data)
raw_inputs = data.get("inputs") or []
inputs = [
{
"name": str(i.get("name", "")),
"value_source_id": str(i.get("value_source_id", "")),
}
for i in raw_inputs
if isinstance(i, dict)
]
return cls(
**common,
source_type="template",
template=data.get("template") or "",
inputs=inputs,
default_value=_coerce_float(data.get("default_value"), 0.0),
eval_interval=_coerce_float(data.get("eval_interval"), None),
)
@@ -661,4 +758,5 @@ _VALUE_SOURCE_MAP: Dict[str, Type[ValueSource]] = {
"system_metrics": SystemMetricsValueSource,
"game_event": GameEventValueSource,
"http": HTTPValueSource,
"template": TemplateValueSource,
}
@@ -40,6 +40,7 @@ from ledgrab.storage.value_source import (
StaticColorValueSource,
StaticValueSource,
SystemMetricsValueSource,
TemplateValueSource,
ValueSource,
_VALUE_SOURCE_MAP,
)
@@ -237,6 +238,7 @@ def _build_ha_entity(
min_ha_value: float | None = None,
max_ha_value: float | None = None,
smoothing: float | None = None,
normalize: bool | None = None,
**_,
) -> ValueSource:
if not ha_source_id:
@@ -251,6 +253,7 @@ def _build_ha_entity(
min_ha_value=min_ha_value if min_ha_value is not None else 0.0,
max_ha_value=max_ha_value if max_ha_value is not None else 100.0,
smoothing=smoothing if smoothing is not None else 0.0,
normalize=normalize if normalize is not None else True,
)
@@ -301,6 +304,7 @@ def _build_system_metrics(
sensor_label: str | None = None,
poll_interval: float | None = None,
smoothing: float | None = None,
normalize: bool | None = None,
**_,
) -> ValueSource:
m = metric or "cpu_load"
@@ -316,6 +320,7 @@ def _build_system_metrics(
sensor_label=sensor_label or "",
poll_interval=poll_interval if poll_interval is not None else 1.0,
smoothing=smoothing if smoothing is not None else 0.0,
normalize=normalize if normalize is not None else True,
)
@@ -347,6 +352,7 @@ def _build_http(
min_value: float | None = None,
max_value: float | None = None,
smoothing: float | None = None,
normalize: bool | None = None,
**_,
) -> ValueSource:
if not http_endpoint_id:
@@ -362,6 +368,80 @@ def _build_http(
min_value=min_value if min_value is not None else 0.0,
max_value=max_value if max_value is not None else 100.0,
smoothing=smoothing if smoothing is not None else 0.0,
normalize=normalize if normalize is not None else True,
)
def _validate_template_inputs(inputs: list | None) -> list:
"""Validate + normalize template input bindings.
Each input is ``{name, value_source_id}``; ``name`` must be a valid,
non-reserved identifier and unique. ``value_source_id`` is *not* required to
resolve (lenient, like gradient_map) a missing/unknown id just yields a
runtime fallback to ``default_value``.
"""
from ledgrab.utils.template_expr import validate_input_name
result: list = []
seen: set = set()
for item in inputs or []:
if not isinstance(item, dict):
raise ValueError("each template input must be an object with name and value_source_id")
name = str(item.get("name", "")).strip()
vs_id = str(item.get("value_source_id", "")).strip()
validate_input_name(name) # identifier + reserved-name check (raises ValueError)
if name in seen:
raise ValueError(f"duplicate template input name: {name!r}")
seen.add(name)
result.append({"name": name, "value_source_id": vs_id})
return result
def _reject_unbound_template_vars(template: str, inputs: list) -> None:
"""Reject expression variables that aren't bound to an input.
An unbound variable raises ``UndefinedError`` at runtime, so the template
would silently always return ``default_value`` almost always a typo. The
globals (min/max/abs/round/clamp) and ``raw`` are excluded by
``extract_variables``, so anything left over is genuinely unbound.
"""
from ledgrab.utils.template_expr import extract_variables
declared = {i["name"] for i in inputs}
undeclared = sorted(set(extract_variables(template)) - declared)
if undeclared:
raise ValueError("expression uses unbound variable(s): " + ", ".join(undeclared))
def _build_template(
*,
common: dict,
template: str | None = None,
inputs: list | None = None,
default_value: float | None = None,
eval_interval: float | None = None,
**_,
) -> ValueSource:
from ledgrab.utils.template_expr import validate_template_expression
tpl = (template or "").strip()
if not tpl:
raise ValueError("template expression is required for template type")
validate_template_expression(tpl) # raises ValueError on compile / cost-bomb
clean_inputs = _validate_template_inputs(inputs)
_reject_unbound_template_vars(tpl, clean_inputs)
dv = default_value if default_value is not None else 0.0
if not (0.0 <= dv <= 1.0):
raise ValueError("default_value must be between 0.0 and 1.0")
ei = float(eval_interval) if eval_interval is not None else None
if ei is not None and ei < 0.0:
raise ValueError("eval_interval must be >= 0")
return TemplateValueSource(
**common,
template=tpl,
inputs=clean_inputs,
default_value=dv,
eval_interval=ei,
)
@@ -381,6 +461,7 @@ CREATE_BUILDERS: Dict[str, CreateBuilder] = {
"system_metrics": _build_system_metrics,
"game_event": _build_game_event,
"http": _build_http,
"template": _build_template,
}
@@ -569,6 +650,7 @@ def _apply_ha_entity(
min_ha_value=None,
max_ha_value=None,
smoothing=None,
normalize=None,
**_,
) -> None:
if ha_source_id is not None:
@@ -583,6 +665,8 @@ def _apply_ha_entity(
source.max_ha_value = max_ha_value
if smoothing is not None:
source.smoothing = smoothing
if normalize is not None:
source.normalize = normalize
def _apply_gradient_map(
@@ -630,6 +714,7 @@ def _apply_system_metrics(
sensor_label=None,
poll_interval=None,
smoothing=None,
normalize=None,
**_,
) -> None:
if metric is not None:
@@ -650,6 +735,8 @@ def _apply_system_metrics(
source.poll_interval = poll_interval
if smoothing is not None:
source.smoothing = smoothing
if normalize is not None:
source.normalize = normalize
def _apply_game_event(source: GameEventValueSource, **_) -> None:
@@ -667,6 +754,7 @@ def _apply_http(
min_value=None,
max_value=None,
smoothing=None,
normalize=None,
**_,
) -> None:
if http_endpoint_id is not None:
@@ -683,6 +771,47 @@ def _apply_http(
source.max_value = max_value
if smoothing is not None:
source.smoothing = smoothing
if normalize is not None:
source.normalize = normalize
def _apply_template(
source: TemplateValueSource,
*,
template=None,
inputs=None,
default_value=None,
eval_interval=None,
**_,
) -> None:
from ledgrab.utils.template_expr import validate_template_expression
# Compute the prospective final state and validate it BEFORE mutating, so a
# rejected update never leaves the cached object half-applied. (inputs/
# template may each change independently; unbound vars are checked against
# the combined final state.)
final_template = template.strip() if template is not None else source.template
final_inputs = _validate_template_inputs(inputs) if inputs is not None else source.inputs
if template is not None:
if not final_template:
raise ValueError("template expression cannot be empty")
validate_template_expression(final_template)
if template is not None or inputs is not None:
_reject_unbound_template_vars(final_template, final_inputs)
if template is not None:
source.template = final_template
if inputs is not None:
source.inputs = final_inputs
if default_value is not None:
if not (0.0 <= default_value <= 1.0):
raise ValueError("default_value must be between 0.0 and 1.0")
source.default_value = default_value
if eval_interval is not None:
if eval_interval < 0.0:
raise ValueError("eval_interval must be >= 0")
source.eval_interval = eval_interval
UPDATE_APPLIERS: Dict[str, UpdateApplier] = {
@@ -701,6 +830,7 @@ UPDATE_APPLIERS: Dict[str, UpdateApplier] = {
"system_metrics": _apply_system_metrics,
"game_event": _apply_game_event,
"http": _apply_http,
"template": _apply_template,
}
@@ -12,7 +12,11 @@ from datetime import datetime, timezone
from ledgrab.storage.base_sqlite_store import BaseSqliteStore
from ledgrab.storage.database import Database
from ledgrab.storage.value_source import ValueSource
from ledgrab.storage.value_source import (
GradientMapValueSource,
TemplateValueSource,
ValueSource,
)
from ledgrab.storage.value_source_factories import (
apply_update as _apply_value_source_update,
build_source as _build_value_source,
@@ -21,12 +25,18 @@ from ledgrab.utils import get_logger
logger = get_logger(__name__)
# Storage-level cap on value-source reference nesting depth. The runtime acquire
# backstop (ValueStreamManager) uses a higher cap so legitimate chains never trip
# it. Color-strip sources cap at 4; value-source chains are flatter combinators.
MAX_VALUE_SOURCE_DEPTH = 8
class ValueSourceStore(BaseSqliteStore[ValueSource]):
"""Persistent storage for value sources."""
_table_name = "value_sources"
_entity_name = "Value source"
_cloneable = True # no inline secrets — only references shared entities by id
def __init__(self, db: Database):
super().__init__(db, ValueSource.from_dict)
@@ -66,6 +76,12 @@ class ValueSourceStore(BaseSqliteStore[ValueSource]):
**kwargs,
)
# Reject over-deep reference chains (cycles are impossible at create:
# the new id is not yet referenceable by anything).
child_ids = self._child_ids_of(source)
if child_ids:
self.validate_nesting(None, child_ids)
# Name-uniqueness happens last so we never burn a uuid on a source
# we end up rejecting AND so the user-facing error precedence
# (type errors before name errors) matches the old code's order.
@@ -87,6 +103,13 @@ class ValueSourceStore(BaseSqliteStore[ValueSource]):
"""
source = self.get(source_id)
# Cycle/depth guard FIRST — before any field mutation — so a rejection
# never leaves the cached object half-mutated. validate_nesting works
# off the prospective child ids derived from kwargs without applying them.
child_ids = self._prospective_child_ids(source, kwargs)
if child_ids:
self.validate_nesting(source_id, child_ids)
name = kwargs.pop("name", None)
if name is not None:
self._check_name_unique(name, exclude_id=source_id)
@@ -118,3 +141,110 @@ class ValueSourceStore(BaseSqliteStore[ValueSource]):
logger.info(f"Updated value source: {source_id}")
return source
# ── Reference graph (cycle / depth / referential integrity) ──────
#
# Value sources may reference other value sources: gradient_map via
# ``value_source_id`` and template via ``inputs[].value_source_id``. (Note:
# css_extract.color_strip_source_id points into the *color strip* store, a
# different graph, so it is intentionally not followed here.) Without a
# guard, a cycle would infinitely recurse in ValueStreamManager.acquire().
@staticmethod
def _child_ids_of(source: ValueSource) -> list[str]:
"""Value-source ids that ``source`` references (gradient_map / template)."""
if isinstance(source, TemplateValueSource):
return [
i["value_source_id"]
for i in source.inputs
if isinstance(i, dict) and i.get("value_source_id")
]
if isinstance(source, GradientMapValueSource):
return [source.value_source_id] if source.value_source_id else []
return []
def _prospective_child_ids(self, source: ValueSource, kwargs: dict) -> list[str]:
"""Child ids the source *would* reference after applying ``kwargs``.
Computed without mutating ``source`` so cycle/depth validation can run
before the update is applied (a raise must not leave a half-mutated
cached object).
"""
if isinstance(source, TemplateValueSource):
inputs = kwargs.get("inputs")
if inputs is None:
inputs = source.inputs
return [
i["value_source_id"]
for i in (inputs or [])
if isinstance(i, dict) and i.get("value_source_id")
]
if isinstance(source, GradientMapValueSource):
vs_id = kwargs.get("value_source_id")
if vs_id is None:
vs_id = source.value_source_id
return [vs_id] if vs_id else []
return []
def get_transitive_dependencies(self, source_id: str) -> set[str]:
"""All value-source ids reachable from ``source_id`` via reference edges."""
seen: set[str] = set()
stack = self._children_of_id(source_id)
while stack:
cid = stack.pop()
if cid in seen:
continue
seen.add(cid)
stack.extend(self._children_of_id(cid))
return seen
def _children_of_id(self, source_id: str) -> list[str]:
try:
src = self.get(source_id)
except Exception:
return []
return self._child_ids_of(src)
def _max_depth(self, ids: list[str], visiting: set[str]) -> int:
"""Longest reference chain length starting at any id in ``ids``."""
best = 0
for cid in ids:
if cid in visiting:
continue # cycle — handled separately; don't recurse forever
try:
src = self.get(cid)
except Exception:
depth = 1 # unresolved leaf still counts as one hop
else:
depth = 1 + self._max_depth(self._child_ids_of(src), visiting | {cid})
best = max(best, depth)
return best
def validate_nesting(self, parent_id: str | None, child_ids: list[str]) -> None:
"""Reject self-reference, circular dependencies, and over-deep nesting.
``parent_id`` is ``None`` at create time a brand-new node has no id, so
no existing source can reference it and a cycle through it is impossible;
only the depth check applies. At update time the cycle check runs too.
"""
if 1 + self._max_depth(child_ids, set()) > MAX_VALUE_SOURCE_DEPTH:
raise ValueError(
f"value source reference chain too deep (max {MAX_VALUE_SOURCE_DEPTH})"
)
if parent_id is None:
return
for cid in child_ids:
if cid == parent_id:
raise ValueError("a value source cannot reference itself")
if parent_id in self.get_transitive_dependencies(cid):
raise ValueError(f"input {cid!r} creates a circular value-source dependency")
def find_referencing_sources(self, source_id: str) -> list[str]:
"""Names of value sources that reference ``source_id`` (for delete-protection)."""
names: list[str] = []
for src in self.get_all():
if src.id == source_id:
continue
if source_id in self._child_ids_of(src):
names.append(src.name)
return names
@@ -84,6 +84,14 @@
<small class="input-hint" style="display:none" id="device-url-hint" data-i18n="device.url.hint">IP address or hostname of the device (e.g. http://192.168.1.100)</small>
<input type="text" id="device-url" data-i18n-placeholder="device.url.placeholder" placeholder="http://192.168.1.100" required>
</div>
<div class="form-group" id="device-mqtt-source-group" style="display: none;">
<div class="label-row">
<label for="device-mqtt-source" data-i18n="device.mqtt_source">MQTT Broker:</label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="device.mqtt_source.hint">Which MQTT broker this device publishes to. Manage brokers under Integrations → MQTT Sources. Leave unset to use the first available broker.</small>
<select id="device-mqtt-source"></select>
</div>
<div class="form-group" id="device-serial-port-group" style="display: none;">
<div class="label-row">
<label for="device-serial-port" id="device-serial-port-label" data-i18n="device.serial_port">Serial Port:</label>
@@ -53,6 +53,15 @@
<select id="settings-serial-port"></select>
</div>
<div class="form-group" id="settings-mqtt-source-group" style="display: none;">
<div class="label-row">
<label for="settings-mqtt-source" data-i18n="device.mqtt_source">MQTT Broker:</label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="device.mqtt_source.hint">Which MQTT broker this device publishes to. Manage brokers under Integrations → MQTT Sources. Leave unset to use the first available broker.</small>
<select id="settings-mqtt-source"></select>
</div>
<div class="form-group" id="settings-ble-family-group" style="display: none;">
<div class="label-row">
<label for="settings-ble-family" data-i18n="device.ble.family">Protocol Family:</label>
@@ -62,6 +62,7 @@
<option value="css_extract" data-i18n="value_source.type.css_extract">Strip Extract</option>
<option value="system_metrics" data-i18n="value_source.type.system_metrics">System Metrics</option>
<option value="http" data-i18n="value_source.type.http">HTTP Poll</option>
<option value="template" data-i18n="value_source.type.template">Jinja Template</option>
</select>
</div>
@@ -394,6 +395,18 @@
<input type="text" id="value-source-attribute" placeholder="">
</div>
<div class="form-group">
<div class="label-row">
<label for="value-source-ha-normalize" data-i18n="value_source.normalize">Normalize to 01:</label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?" data-i18n-aria-label="aria.hint">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="value_source.normalize.hint">On: rescale the raw value to 01 using Min/Max below. Off: the value is clamped to 01 as-is (for entities that already report a 01 fraction). The raw value is always available to templates (raw[name]) and automations.</small>
<label class="settings-toggle">
<input type="checkbox" id="value-source-ha-normalize" checked onchange="onValueSourceNormalizeChange()">
<span class="settings-toggle-slider"></span>
</label>
</div>
<div class="form-group">
<div class="label-row">
<label for="value-source-min-ha-value"><span data-i18n="value_source.min_ha_value">Min HA Value:</span> <span id="value-source-min-ha-value-display">0</span></label>
@@ -456,6 +469,82 @@
</div>
</div>
<!-- Template (Jinja expression) fields -->
<div id="value-source-template-section" style="display:none">
<div class="form-group">
<div class="label-row">
<label for="value-source-template-expression" data-i18n="value_source.template.expression">Expression:</label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?" data-i18n-aria-label="aria.hint">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="value_source.template.expression.hint">A sandboxed Jinja expression returning a number. Bound inputs are available by name; use raw[name] for the un-normalized value.</small>
<!-- The highlight overlay is injected around this textarea by jinja-editor.ts -->
<textarea id="value-source-template-expression" rows="3" spellcheck="false"
data-i18n-placeholder="value_source.template.expression.placeholder"
placeholder="clamp((temp - 18) / 10, 0, 1)"></textarea>
<div id="value-source-template-error" class="field-error-msg" style="display:none"></div>
<div id="value-source-template-ok" class="field-ok-msg" style="display:none"></div>
<div id="value-source-template-warn" class="field-warn-msg" style="display:none"></div>
<details class="jinja-hints">
<summary data-i18n="value_source.template.hints.title">Expression help</summary>
<div class="jinja-hints-body">
<div class="jinja-hints-section">
<span class="jinja-hints-section-title" data-i18n="value_source.template.hints.inputs_title">Bound inputs</span>
<div id="value-source-template-hint-vars" class="jinja-hints-vars"></div>
</div>
<div class="jinja-hints-section">
<span class="jinja-hints-section-title" data-i18n="value_source.template.hints.globals_title">Globals</span>
<div data-i18n="value_source.template.hints.globals">min(a, b), max(a, b), abs(x), round(x), clamp(x, lo, hi)</div>
</div>
<div class="jinja-hints-section">
<span class="jinja-hints-section-title" data-i18n="value_source.template.hints.raw_title">Raw values</span>
<div data-i18n="value_source.template.hints.raw">raw[name] gives the un-normalized value of an input that has one.</div>
</div>
<div class="jinja-hints-section">
<span class="jinja-hints-section-title" data-i18n="value_source.template.hints.examples_title">Examples</span>
<ul class="jinja-hints-examples">
<li><code>min(audio * 2, 1)</code></li>
<li><code>clamp((temp - 18) / 10, 0, 1)</code></li>
<li><code>(a + b) / 2</code></li>
</ul>
</div>
<div class="jinja-hints-section">
<small class="jinja-hints-time" data-i18n="value_source.template.hints.time">Tip: for time-of-day logic, bind an Adaptive (Time) or Daylight source as an input.</small>
</div>
</div>
</details>
</div>
<div class="form-group">
<div class="label-row">
<label data-i18n="value_source.template.inputs">Inputs:</label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?" data-i18n-aria-label="aria.hint">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="value_source.template.inputs.hint">Bind float value sources to variable names you reference in the expression.</small>
<div id="value-source-template-inputs-list" class="template-inputs-list"></div>
<button type="button" class="btn btn-secondary btn-sm" onclick="addTemplateInput()" data-i18n="value_source.template.add_input">+ Add Input</button>
</div>
<div class="form-group">
<div class="label-row">
<label for="value-source-template-default-value"><span data-i18n="value_source.template.default_value">Default Value:</span> <span id="value-source-template-default-value-display">0.00</span></label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?" data-i18n-aria-label="aria.hint">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="value_source.template.default_value.hint">Output used when the expression cannot be evaluated (e.g. an input is missing).</small>
<input type="range" id="value-source-template-default-value" min="0" max="1" step="0.01" value="0"
oninput="document.getElementById('value-source-template-default-value-display').textContent = parseFloat(this.value).toFixed(2)">
</div>
<div class="form-group">
<div class="label-row">
<label for="value-source-template-eval-interval" data-i18n="value_source.template.eval_interval">Eval Interval (s):</label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?" data-i18n-aria-label="aria.hint">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="value_source.template.eval_interval.hint">How often to re-evaluate the expression. 0 = every poll (re-evaluate as fast as the inputs update).</small>
<input type="number" id="value-source-template-eval-interval" min="0" step="0.1" value="0">
</div>
</div>
<!-- CSS Extract fields -->
<div id="value-source-css-extract-section" style="display:none">
<div class="form-group">
@@ -509,6 +598,18 @@
</select>
</div>
<div class="form-group" id="value-source-sysmetric-normalize-group">
<div class="label-row">
<label for="value-source-sysmetric-normalize" data-i18n="value_source.normalize">Normalize to 01:</label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?" data-i18n-aria-label="aria.hint">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="value_source.normalize.hint">On: rescale the raw value to 01 using Min/Max below. Off: the value is clamped to 01 as-is (for metrics that already report a 01 fraction). The raw value is always available to templates (raw[name]) and automations.</small>
<label class="settings-toggle">
<input type="checkbox" id="value-source-sysmetric-normalize" checked onchange="onValueSourceNormalizeChange()">
<span class="settings-toggle-slider"></span>
</label>
</div>
<div id="value-source-sysmetric-range" style="display:none">
<div class="form-group">
<label for="value-source-sysmetric-min"><span data-i18n="value_source.sysmetric.min">Min Value:</span></label>
@@ -670,6 +771,17 @@
<summary data-i18n="value_source.http.modulator.summary">Modulator mapping (optional)</summary>
<div class="form-collapse-body">
<small class="input-hint" data-i18n="value_source.http.modulator.hint">Only used when this source drives brightness or color. Automation rules read the raw extracted value and ignore these settings.</small>
<div class="form-group">
<div class="label-row">
<label for="value-source-http-normalize" data-i18n="value_source.normalize">Normalize to 01:</label>
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?" data-i18n-aria-label="aria.hint">?</button>
</div>
<small class="input-hint" style="display:none" data-i18n="value_source.normalize.hint">On: rescale the raw value to 01 using Min/Max below. Off: the value is clamped to 01 as-is (for endpoints that already return a 01 fraction). The raw value is always available to templates (raw[name]) and automations.</small>
<label class="settings-toggle">
<input type="checkbox" id="value-source-http-normalize" checked onchange="onValueSourceNormalizeChange()">
<span class="settings-toggle-slider"></span>
</label>
</div>
<div class="form-group">
<div class="label-row">
<label for="value-source-http-min" data-i18n="value_source.http.min_value">Min Value:</label>
+2
View File
@@ -3,6 +3,7 @@
from .file_ops import atomic_write_json, read_upload_capped
from .logger import setup_logging, get_logger
from .monitor_names import get_monitor_names, get_monitor_name, get_monitor_refresh_rates
from .numeric import clamp01
from .timer import high_resolution_timer
from .log_broadcaster import broadcaster as log_broadcaster, install_broadcast_handler
from .url_scheme import infer_http_scheme
@@ -15,6 +16,7 @@ __all__ = [
"get_monitor_names",
"get_monitor_name",
"get_monitor_refresh_rates",
"clamp01",
"high_resolution_timer",
"log_broadcaster",
"install_broadcast_handler",
+24
View File
@@ -0,0 +1,24 @@
"""Small numeric helpers shared across the processing layer."""
from __future__ import annotations
import math
def clamp01(x: float, default: float = 0.0) -> float:
"""Clamp ``x`` into the unit interval ``[0.0, 1.0]``, finite-safe.
NaN/inf are rejected to ``default`` *before* clamping they are valid
floats, so ``max/min`` alone would silently pass them through (and an
``int(base * inf)`` cast downstream raises OverflowError, ``int(nan)``
raises ValueError). Use this at any boundary that feeds a value into a
fixed-point / uint brightness multiply where a non-finite or out-of-range
value would corrupt or crash the math.
"""
if not math.isfinite(x):
return default
if x < 0.0:
return 0.0
if x > 1.0:
return 1.0
return x
+209
View File
@@ -0,0 +1,209 @@
"""Hardened sandboxed-Jinja expression engine for template value sources.
Single source of truth for compiling, validating, and evaluating user-authored
Jinja *expressions* that combine the live values of other value sources into a
single float in [0, 1]. Imported by the storage factory (create/update
validation), the runtime ``TemplateValueStream``, and the validate-template API
route, so the client and server can never disagree about what is valid.
Security model a user-authored expression is attacker-influenceable config
that runs server-side (LAN device, shareable backups), so we layer defenses:
* :class:`~jinja2.sandbox.ImmutableSandboxedEnvironment` blocks ``__class__`` /
``mro`` traversal, mutation, and unsafe attribute/method access.
* ALL default filters and tests are stripped (``|attr``, ``|pprint``, ``|map``,
``|format`` ) and the auto-injected globals (``range``, ``dict``,
``namespace``, ``cycler``, ``lipsum``, ``joiner``) are removed none are
needed for numeric math and several are escape/DoS amplifiers.
* Only five vetted numeric callables are exposed: ``min``, ``max``, ``abs``,
``round``, ``clamp``.
* The evaluation context contains ONLY primitive floats plus a flat dict of
floats (``raw``) never any application object so even a hypothetical
sandbox escape has nothing privileged to pivot to.
* Obvious cost bombs are rejected at validate time: the ``**`` (power) operator
and string/list ``*`` repetition can hang or OOM the interpreter, which
``try/except`` cannot catch.
* The result is coerced to float, NaN/inf are rejected (they are valid floats,
not exceptions, so clamping alone would silently keep them), then clamped.
"""
from __future__ import annotations
import math
import re
from typing import Any
from jinja2 import nodes
from jinja2.exceptions import TemplateError
from jinja2.sandbox import ImmutableSandboxedEnvironment
def clamp(x: float, lo: float = 0.0, hi: float = 1.0) -> float:
"""Clamp ``x`` into ``[lo, hi]`` (defaults to the unit interval)."""
return max(lo, min(hi, x))
# The five callables a template author may use. Kept as the *only* names in the
# environment globals so no other builtin/Jinja helper is reachable.
GLOBALS: dict[str, Any] = {
"min": min,
"max": max,
"abs": abs,
"round": round,
"clamp": clamp,
}
# Input variable names that would shadow a global or the ``raw`` dict (and thus
# silently break the expression) or that Jinja auto-injects. Rejected at save
# time so the user gets a clear error instead of a template that always falls
# back to ``default_value``.
RESERVED_NAMES: frozenset[str] = frozenset(
{*GLOBALS, "raw", "range", "dict", "namespace", "cycler", "lipsum", "joiner"}
)
_IDENT_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
class TemplateValidationError(ValueError):
"""Raised when a template expression (or an input name) is invalid/unsafe.
Subclasses :class:`ValueError` so existing route handlers that map
``ValueError -> HTTP 400`` surface a clean message automatically.
"""
def _build_env() -> ImmutableSandboxedEnvironment:
env = ImmutableSandboxedEnvironment(autoescape=False)
# Strip the entire default filter/test surface — numeric expressions need
# none of them, and |attr/|pprint/|map/|format are escape/DoS amplifiers.
env.filters.clear()
env.tests.clear()
# Replace the auto-injected globals (range/dict/namespace/cycler/...) with
# only our five vetted callables.
env.globals.clear()
env.globals.update(GLOBALS)
return env
# Module-level shared environment. Compiled expressions and evaluation are
# cheap; the environment itself is immutable config built once.
SANDBOX_ENV = _build_env()
def _clean_jinja_error(exc: TemplateError) -> str:
"""A user-facing one-line message from a Jinja compile error."""
msg = str(exc) or exc.__class__.__name__
return msg.strip()
def _guard_ast(template: str) -> None:
"""Reject cost-bomb / disallowed constructs via the Jinja AST.
Called only on already-compilable single expressions, so wrapping in an
output block is safe (a ``}}`` inside a string literal stays inside it).
"""
try:
tree = SANDBOX_ENV.parse("{{ (" + template + ") }}")
except TemplateError as exc: # pragma: no cover - compile already gated this
raise TemplateValidationError(_clean_jinja_error(exc)) from exc
if next(tree.find_all(nodes.Pow), None) is not None:
raise TemplateValidationError("the '**' (power) operator is not allowed")
if next(tree.find_all((nodes.Filter, nodes.Test)), None) is not None:
raise TemplateValidationError("filters and tests are not allowed")
# Collection literals have no use in a numeric expression and enable a
# memory-bomb via repetition ([0] * 10**8 allocates gigabytes). The only
# collection access we need is raw[...] subscript, which is a Getitem, not a
# literal. Forbid list/tuple/dict literals outright.
if next(tree.find_all((nodes.List, nodes.Tuple, nodes.Dict)), None) is not None:
raise TemplateValidationError("list/tuple/dict literals are not allowed")
# Attribute access (``a.b``) has no use in numeric expressions and is the
# classic sandbox-escape vector (``__class__``, ``.format`` …). Raw values
# are read by subscript (``raw['x']``), which stays allowed.
if next(tree.find_all(nodes.Getattr), None) is not None:
raise TemplateValidationError("attribute access ('.') is not allowed")
# Only the five vetted globals may be called — blocks dict()/namespace()/
# cycler()/etc. at validate time with a clear message (they would also fail
# at runtime since the environment globals are cleared, but failing early is
# better UX and defense in depth).
for call in tree.find_all(nodes.Call):
fn = call.node
if not (isinstance(fn, nodes.Name) and fn.name in GLOBALS):
name = getattr(fn, "name", None) or fn.__class__.__name__
raise TemplateValidationError(
f"only min/max/abs/round/clamp may be called (got {name!r})"
)
# float * float is fine; reject only string/list repetition (the OOM path).
for mul in tree.find_all(nodes.Mul):
for side in (mul.left, mul.right):
if isinstance(side, nodes.Const) and isinstance(side.value, (str, list, tuple)):
raise TemplateValidationError("string/list repetition is not allowed")
def compile_template(template: str):
"""Compile ``template`` to a reusable Jinja ``Expression``.
Raises :class:`TemplateValidationError` if empty, uncompilable, or it uses a
disallowed/cost-bomb construct. Globals (min/max/abs/round/clamp) resolve
from ``SANDBOX_ENV.globals`` at call time, so the returned expression is
invoked with only the data context: ``expr(**ctx)``.
"""
if not template or not template.strip():
raise TemplateValidationError("expression is empty")
try:
expr = SANDBOX_ENV.compile_expression(template)
except TemplateError as exc:
raise TemplateValidationError(_clean_jinja_error(exc)) from exc
_guard_ast(template)
return expr
def validate_template_expression(template: str) -> None:
"""Validate ``template`` (compile + guard); raise on any problem."""
compile_template(template)
def validate_input_name(name: str) -> None:
"""Validate a single template input variable name; raise on any problem."""
if not name or not _IDENT_RE.match(name):
raise TemplateValidationError(f"input name {name!r} is not a valid identifier")
if name in RESERVED_NAMES:
raise TemplateValidationError(f"input name {name!r} is reserved")
def extract_variables(template: str) -> list[str]:
"""Return the free input variables referenced by ``template``.
Excludes the globals and ``raw`` so the validate endpoint reports only the
names the author is expected to bind. Returns ``[]`` if unparsable.
"""
from jinja2 import meta
try:
tree = SANDBOX_ENV.parse("{{ (" + template + ") }}")
except TemplateError:
return []
undeclared = meta.find_undeclared_variables(tree)
return sorted(undeclared - set(GLOBALS) - {"raw"})
def finalize_result(value: Any, default: float, lo: float = 0.0, hi: float = 1.0) -> float:
"""Coerce an evaluated result to a safe float in ``[lo, hi]``.
Non-numeric ``default``; NaN/inf ``default`` (they are valid floats, so
clamping alone would silently keep them); otherwise clamp.
"""
try:
f = float(value)
except (TypeError, ValueError, OverflowError):
# OverflowError: float() of a multi-hundred-digit int (e.g. a chained
# big-int multiply). Treated as "not a usable number" → default.
return default
if math.isnan(f) or math.isinf(f):
return default
return clamp(f, lo, hi)
+103 -1
View File
@@ -47,6 +47,13 @@ def output_target_store(_route_db):
return OutputTargetStore(_route_db)
@pytest.fixture
def mqtt_source_store(_route_db):
from ledgrab.storage.mqtt_source_store import MQTTSourceStore
return MQTTSourceStore(_route_db)
@pytest.fixture
def processor_manager():
"""A mock ProcessorManager — avoids real hardware."""
@@ -60,7 +67,7 @@ def processor_manager():
@pytest.fixture
def client(device_store, output_target_store, processor_manager):
def client(device_store, output_target_store, processor_manager, mqtt_source_store):
app = _make_app()
# Override auth to always pass
@@ -72,6 +79,7 @@ def client(device_store, output_target_store, processor_manager):
app.dependency_overrides[deps.get_device_store] = lambda: device_store
app.dependency_overrides[deps.get_output_target_store] = lambda: output_target_store
app.dependency_overrides[deps.get_processor_manager] = lambda: processor_manager
app.dependency_overrides[deps.get_mqtt_store] = lambda: mqtt_source_store
return TestClient(app, raise_server_exceptions=False)
@@ -428,6 +436,100 @@ class TestWLEDSchemeInference:
assert device_store.get_device(existing.id).url == "http://10.0.0.5"
class TestMqttSourceId:
"""Regression coverage for the device ``mqtt_source_id`` field.
The store + ``device_config.MQTTConfig`` already carried the field, but
the API schema/route layer dropped it (DeviceCreate/Update/Response never
declared it, and the route never threaded it). These pin the create +
update round-trip and the referenced-source validation so it can't
silently regress.
"""
@pytest.fixture
def _stub_mqtt_validate(self, monkeypatch):
async def fake_validate(self, url): # noqa: ARG001 — provider self
return {"led_count": 60}
from ledgrab.core.devices.mqtt_provider import MQTTDeviceProvider
monkeypatch.setattr(MQTTDeviceProvider, "validate_device", fake_validate)
return fake_validate
def test_create_mqtt_device_persists_source_id(
self, client, device_store, mqtt_source_store, _stub_mqtt_validate
):
src = mqtt_source_store.create_source(name="Broker A", broker_host="192.168.1.10")
resp = client.post(
"/api/v1/devices",
json={
"name": "Living Room MQTT",
"device_type": "mqtt",
"url": "mqtt://ledgrab/device/living-room",
"led_count": 60,
"mqtt_source_id": src.id,
},
)
assert resp.status_code == 201, resp.text
body = resp.json()
assert body["mqtt_source_id"] == src.id
assert device_store.get_device(body["id"]).mqtt_source_id == src.id
def test_create_mqtt_device_rejects_unknown_source(self, client, _stub_mqtt_validate):
resp = client.post(
"/api/v1/devices",
json={
"name": "Bad Broker Ref",
"device_type": "mqtt",
"url": "mqtt://ledgrab/device/x",
"led_count": 60,
"mqtt_source_id": "mqs_doesnotexist",
},
)
assert resp.status_code == 422, resp.text
assert "not found" in resp.json()["detail"]
def test_update_device_sets_mqtt_source_id(self, client, device_store, mqtt_source_store):
src = mqtt_source_store.create_source(name="Broker B", broker_host="10.0.0.2")
dev = device_store.create_device(
name="MQTT dev",
url="mqtt://ledgrab/device/a",
led_count=10,
device_type="mqtt",
)
resp = client.put(f"/api/v1/devices/{dev.id}", json={"mqtt_source_id": src.id})
assert resp.status_code == 200, resp.text
assert resp.json()["mqtt_source_id"] == src.id
assert device_store.get_device(dev.id).mqtt_source_id == src.id
def test_update_device_can_clear_mqtt_source(self, client, device_store, mqtt_source_store):
"""An empty string unsets the broker (back to 'first available'). The
store's None-means-skip rule means '' is a real value that persists."""
src = mqtt_source_store.create_source(name="Broker C", broker_host="10.0.0.3")
dev = device_store.create_device(
name="MQTT dev3",
url="mqtt://ledgrab/device/c",
led_count=10,
device_type="mqtt",
mqtt_source_id=src.id,
)
resp = client.put(f"/api/v1/devices/{dev.id}", json={"mqtt_source_id": ""})
assert resp.status_code == 200, resp.text
assert resp.json()["mqtt_source_id"] == ""
assert device_store.get_device(dev.id).mqtt_source_id == ""
def test_update_device_rejects_unknown_mqtt_source(self, client, device_store):
dev = device_store.create_device(
name="MQTT dev2",
url="mqtt://ledgrab/device/b",
led_count=10,
device_type="mqtt",
)
resp = client.put(f"/api/v1/devices/{dev.id}", json={"mqtt_source_id": "mqs_nope"})
assert resp.status_code == 422, resp.text
assert "not found" in resp.json()["detail"]
class TestPairThenCreateFlow:
"""End-to-end coverage: pair, then persist; assert the token is
encrypted at rest and decrypted in to_config(), and that the API
@@ -0,0 +1,143 @@
"""Endpoint tests for the wiring-graph router (/api/v1/graph*).
The graph router resolves stores via the ``dependencies`` getters *directly*
(not FastAPI ``Depends``), so these tests populate the ``deps._deps`` registry
rather than using ``app.dependency_overrides``. Auth stays real (conftest pins a
test API key) so the rejection path is covered too.
"""
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from ledgrab.api import dependencies as deps
from ledgrab.api.routes.graph import router
_AUTH = {"Authorization": "Bearer test-api-key-12345"}
class _Ent:
"""Minimal stand-in for a storage model exposing ``to_dict``."""
def __init__(self, data: dict):
self._data = data
def to_dict(self) -> dict:
return self._data
def _store(*entities: dict):
class _S:
def get_all(self):
return [_Ent(e) for e in entities]
return _S()
@pytest.fixture
def client(test_config, monkeypatch):
import ledgrab.config as config_mod
monkeypatch.setattr(config_mod, "config", test_config)
# Populate only the kinds under test; the rest resolve to RuntimeError and
# are gracefully treated as empty by the route's _gather_entities.
monkeypatch.setattr(
deps,
"_deps",
{
"device_store": _store({"id": "dev_1", "name": "Strip", "device_type": "wled"}),
"picture_source_store": _store({"id": "ps_1", "name": "Cap", "stream_type": "raw"}),
"color_strip_store": _store(
{
"id": "css_1",
"name": "CSS",
"source_type": "picture",
"picture_source_id": "ps_1",
}
),
"output_target_store": _store(
{
"id": "ot_1",
"name": "TV",
"target_type": "led",
"device_id": "dev_1",
"color_strip_source_id": "css_1",
}
),
},
)
app = FastAPI()
app.include_router(router)
return TestClient(app, raise_server_exceptions=False)
def test_schema_endpoint_returns_registry(client):
resp = client.get("/api/v1/graph/schema", headers=_AUTH)
assert resp.status_code == 200
data = resp.json()
assert "device" in data["kinds"]
assert any(
c["target_kind"] == "output_target" and c["field"] == "device_id"
for c in data["connections"]
)
# Bindable + nested flags must be carried through.
assert any(c["bindable"] and c["nested"] for c in data["connections"])
def test_schema_endpoint_requires_auth(client):
assert client.get("/api/v1/graph/schema").status_code in (401, 403)
def test_graph_endpoint_builds_topology(client):
data = client.get("/api/v1/graph", headers=_AUTH).json()
assert {n["id"] for n in data["nodes"]} == {"dev_1", "ps_1", "css_1", "ot_1"}
edges = {(e["from"], e["to"]) for e in data["edges"]}
assert ("dev_1", "ot_1") in edges
assert ("css_1", "ot_1") in edges
assert ("ps_1", "css_1") in edges
assert data["issues"]["broken_refs"] == []
def test_dependents_endpoint(client):
data = client.get("/api/v1/graph/dependents/color_strip_source/css_1", headers=_AUTH).json()
assert data["dependents"] == [
{"id": "ot_1", "kind": "output_target", "name": "TV", "field": "color_strip_source_id"}
]
def test_dependents_endpoint_rejects_unknown_kind(client):
resp = client.get("/api/v1/graph/dependents/bogus/x", headers=_AUTH)
assert resp.status_code == 404
def test_validate_connection_endpoint_accepts_valid(client):
resp = client.post(
"/api/v1/graph/validate-connection",
json={
"target_kind": "output_target",
"target_id": "ot_1",
"field": "color_strip_source_id",
"source_id": "css_1",
},
headers=_AUTH,
)
assert resp.status_code == 200
assert resp.json() == {"ok": True, "error": None}
def test_validate_connection_endpoint_rejects_missing_source(client):
resp = client.post(
"/api/v1/graph/validate-connection",
json={
"target_kind": "output_target",
"target_id": "ot_1",
"field": "color_strip_source_id",
"source_id": "ghost",
},
headers=_AUTH,
)
data = resp.json()
assert data["ok"] is False
assert "not found" in data["error"]
@@ -0,0 +1,225 @@
"""Tests for the aggregated /api/v1/snapshot endpoint.
The snapshot collapses the integration's per-target/per-device poll fan-out
into one response. These tests build a minimal app with the snapshot router and
override the store/manager getters, mirroring tests/api/routes/test_devices_routes.py.
Auth is left real (the conftest patches a test API key) so the rejection path is
also covered. System metrics + health run for real they read module-level
providers and the patched config, no lifespan needed.
"""
import types
from unittest.mock import AsyncMock, MagicMock
import pytest
from fastapi import FastAPI, HTTPException
from fastapi.testclient import TestClient
from ledgrab.api import dependencies as deps
from ledgrab.api.routes import devices as devices_mod
from ledgrab.api.routes.devices import resolve_device_brightness
from ledgrab.api.routes.snapshot import SNAPSHOT_SECTIONS, _resolve_sections, router
from ledgrab.core.processing.processor_manager import ProcessorManager
from ledgrab.core.update.update_service import UpdateService
_AUTH = {"Authorization": "Bearer test-api-key-12345"}
_TOP_LEVEL_KEYS = (
"targets",
"target_states",
"target_metrics",
"devices",
"device_brightness",
"css_sources",
"value_sources",
"scene_presets",
"sync_clocks",
"system",
)
@pytest.fixture
def client(test_config, monkeypatch):
# Pin the global config (with its test API key) so auth is deterministic
# regardless of test ordering — other suites mutate the config singleton.
import ledgrab.config as config_mod
monkeypatch.setattr(config_mod, "config", test_config)
target_store = MagicMock()
target_store.get_all_targets.return_value = []
device_store = MagicMock()
device_store.get_all_devices.return_value = []
css_store = MagicMock()
css_store.get_all_sources.return_value = []
value_store = MagicMock()
value_store.get_all_sources.return_value = []
preset_store = MagicMock()
preset_store.get_all_presets.return_value = []
clock_store = MagicMock()
clock_store.get_all_clocks.return_value = []
clock_manager = MagicMock()
manager = MagicMock(spec=ProcessorManager)
manager.get_all_target_states.return_value = {}
manager.get_all_target_metrics.return_value = {}
update_service = MagicMock(spec=UpdateService)
update_service.get_status.return_value = {"has_update": False, "current_version": "test"}
app = FastAPI()
app.include_router(router)
app.dependency_overrides[deps.get_output_target_store] = lambda: target_store
app.dependency_overrides[deps.get_device_store] = lambda: device_store
app.dependency_overrides[deps.get_color_strip_store] = lambda: css_store
app.dependency_overrides[deps.get_value_source_store] = lambda: value_store
app.dependency_overrides[deps.get_scene_preset_store] = lambda: preset_store
app.dependency_overrides[deps.get_sync_clock_store] = lambda: clock_store
app.dependency_overrides[deps.get_sync_clock_manager] = lambda: clock_manager
app.dependency_overrides[deps.get_processor_manager] = lambda: manager
app.dependency_overrides[deps.get_update_service] = lambda: update_service
return TestClient(app, raise_server_exceptions=False)
def test_snapshot_returns_all_sections(client):
resp = client.get("/api/v1/snapshot", headers=_AUTH)
assert resp.status_code == 200
data = resp.json()
for key in _TOP_LEVEL_KEYS:
assert key in data, f"snapshot missing top-level key: {key}"
for list_key in (
"targets",
"devices",
"css_sources",
"value_sources",
"scene_presets",
"sync_clocks",
):
assert data[list_key] == []
for dict_key in ("target_states", "target_metrics", "device_brightness"):
assert data[dict_key] == {}
def test_snapshot_system_block_has_health_version(client):
data = client.get("/api/v1/snapshot", headers=_AUTH).json()
system = data["system"]
assert {"performance", "health", "update"}.issubset(system)
# health drives the coordinator's version + boot-time derivation
assert system["health"]["version"]
assert "uptime_seconds" in system["health"]
def test_snapshot_requires_auth(client):
resp = client.get("/api/v1/snapshot")
assert resp.status_code in (401, 403)
def test_snapshot_include_filters_to_requested_sections(client):
resp = client.get("/api/v1/snapshot", params={"include": "devices,system"}, headers=_AUTH)
assert resp.status_code == 200
# Only requested sections are present — excluded ones are omitted entirely.
assert set(resp.json().keys()) == {"devices", "system"}
def test_snapshot_include_rejects_unknown_section(client):
resp = client.get("/api/v1/snapshot", params={"include": "devices,bogus"}, headers=_AUTH)
assert resp.status_code == 422
assert "bogus" in resp.json()["detail"]
# ---------------------------------------------------------------------------
# _resolve_sections — query-param parsing edge cases
# ---------------------------------------------------------------------------
def test_resolve_sections_defaults_to_all_when_empty():
assert _resolve_sections(None) == frozenset(SNAPSHOT_SECTIONS)
assert _resolve_sections("") == frozenset(SNAPSHOT_SECTIONS)
def test_resolve_sections_strips_whitespace_and_dedupes():
assert _resolve_sections("devices, system ,devices") == frozenset({"devices", "system"})
def test_resolve_sections_ignores_empty_segments():
assert _resolve_sections("devices,,system,") == frozenset({"devices", "system"})
def test_resolve_sections_is_case_sensitive():
with pytest.raises(HTTPException) as exc:
_resolve_sections("Devices")
assert exc.value.status_code == 422
# ---------------------------------------------------------------------------
# resolve_device_brightness — cached / cold-fetch / graceful-degrade paths
# ---------------------------------------------------------------------------
def _fake_device(**kw):
return types.SimpleNamespace(
id=kw.get("id", "d1"),
device_type=kw.get("device_type", "wled"),
url=kw.get("url", "http://x"),
software_brightness=kw.get("software_brightness", 42),
)
async def test_brightness_none_without_capability(monkeypatch):
monkeypatch.setattr(devices_mod, "get_device_capabilities", lambda _dt: set())
manager = MagicMock()
assert await resolve_device_brightness(_fake_device(), manager) is None
manager.find_device_state.assert_not_called()
async def test_brightness_returns_cached(monkeypatch):
monkeypatch.setattr(devices_mod, "get_device_capabilities", lambda _dt: {"brightness_control"})
manager = MagicMock()
manager.find_device_state.return_value = types.SimpleNamespace(hardware_brightness=128)
assert await resolve_device_brightness(_fake_device(), manager) == 128
async def test_brightness_active_fetch_and_caches_on_cold_cache(monkeypatch):
monkeypatch.setattr(devices_mod, "get_device_capabilities", lambda _dt: {"brightness_control"})
provider = MagicMock()
provider.get_brightness = AsyncMock(return_value=200)
monkeypatch.setattr(devices_mod, "get_provider", lambda _dt: provider)
ds = types.SimpleNamespace(hardware_brightness=None)
manager = MagicMock()
manager.find_device_state.return_value = ds
assert await resolve_device_brightness(_fake_device(), manager) == 200
assert ds.hardware_brightness == 200 # cached so the next poll is I/O-free
async def test_brightness_degrades_to_none_on_provider_error(monkeypatch):
monkeypatch.setattr(devices_mod, "get_device_capabilities", lambda _dt: {"brightness_control"})
provider = MagicMock()
provider.get_brightness = AsyncMock(side_effect=OSError("unreachable"))
monkeypatch.setattr(devices_mod, "get_provider", lambda _dt: provider)
manager = MagicMock()
manager.find_device_state.return_value = types.SimpleNamespace(hardware_brightness=None)
# A single unreachable device must not raise — it degrades to None.
assert await resolve_device_brightness(_fake_device(), manager) is None
async def test_brightness_falls_back_to_software_when_unsupported(monkeypatch):
monkeypatch.setattr(devices_mod, "get_device_capabilities", lambda _dt: {"brightness_control"})
provider = MagicMock()
provider.get_brightness = AsyncMock(side_effect=NotImplementedError)
monkeypatch.setattr(devices_mod, "get_provider", lambda _dt: provider)
manager = MagicMock()
manager.find_device_state.return_value = types.SimpleNamespace(hardware_brightness=None)
assert await resolve_device_brightness(_fake_device(software_brightness=42), manager) == 42
@@ -0,0 +1,182 @@
"""Tests for template value source API: CRUD, validate-template, delete-protection."""
from unittest.mock import MagicMock
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from ledgrab.api import dependencies as deps
from ledgrab.api.routes.value_sources import router
from ledgrab.storage.value_source_store import ValueSourceStore
@pytest.fixture
def _route_db(tmp_path):
from ledgrab.storage.database import Database
db = Database(tmp_path / "test.db")
yield db
db.close()
@pytest.fixture
def store(_route_db):
return ValueSourceStore(_route_db)
@pytest.fixture
def client(store):
app = FastAPI()
app.include_router(router)
from ledgrab.api.auth import verify_api_key
app.dependency_overrides[verify_api_key] = lambda: "test-user"
app.dependency_overrides[deps.get_value_source_store] = lambda: store
app.dependency_overrides[deps.get_processor_manager] = lambda: MagicMock()
app.dependency_overrides[deps.get_output_target_store] = lambda: MagicMock(
get_all_targets=lambda: []
)
deps._deps["processor_manager"] = MagicMock()
return TestClient(app, raise_server_exceptions=False)
def _create(client, **over):
body = {
"source_type": "template",
"name": "Combo",
"template": "min(a * 2, 1)",
"inputs": [{"name": "a", "value_source_id": ""}],
"default_value": 0.2,
}
body.update(over)
return client.post("/api/v1/value-sources", json=body)
class TestCRUD:
def test_create_get_list_roundtrip(self, client):
r = _create(client)
assert r.status_code == 201, r.text
body = r.json()
assert body["source_type"] == "template"
assert body["return_type"] == "float"
assert body["template"] == "min(a * 2, 1)"
assert body["inputs"] == [{"name": "a", "value_source_id": ""}]
assert body["default_value"] == 0.2
sid = body["id"]
got = client.get(f"/api/v1/value-sources/{sid}").json()
assert got["template"] == "min(a * 2, 1)"
lst = client.get("/api/v1/value-sources").json()
assert any(s["id"] == sid and s["source_type"] == "template" for s in lst["sources"])
def test_update(self, client):
sid = _create(client).json()["id"]
r = client.put(
f"/api/v1/value-sources/{sid}",
json={"source_type": "template", "template": "clamp(a * 3)"},
)
assert r.status_code == 200, r.text
assert r.json()["template"] == "clamp(a * 3)"
def test_create_compile_error_returns_400(self, client):
r = _create(client, template="a +")
assert r.status_code == 400
def test_create_reserved_name_returns_400(self, client):
r = _create(client, inputs=[{"name": "min", "value_source_id": ""}])
assert r.status_code == 400
class TestDeleteProtection:
def test_delete_blocked_when_referenced(self, client):
base = client.post(
"/api/v1/value-sources",
json={"source_type": "static", "name": "Base", "value": 0.5},
).json()
_create(
client,
name="Uses",
template="b",
inputs=[{"name": "b", "value_source_id": base["id"]}],
)
r = client.delete(f"/api/v1/value-sources/{base['id']}")
assert r.status_code == 400
assert "referenced by" in r.json()["detail"]
class TestValidateEndpoint:
def _validate(self, client, **body):
return client.post("/api/v1/value-sources/validate-template", json=body)
def test_valid_expression(self, client):
r = self._validate(
client,
template="min(a, b)",
inputs=[{"name": "a", "value_source_id": ""}, {"name": "b", "value_source_id": ""}],
)
assert r.status_code == 200
data = r.json()
assert data["valid"] is True
assert set(data["variables"]) == {"a", "b"}
def test_compile_error(self, client):
r = self._validate(client, template="a +", inputs=[])
data = r.json()
assert data["valid"] is False
assert data["error"]
def test_reserved_name(self, client):
r = self._validate(
client, template="min(0,1)", inputs=[{"name": "raw", "value_source_id": ""}]
)
assert r.json()["valid"] is False
def test_missing_input_is_warning_not_error(self, client):
r = self._validate(
client, template="a", inputs=[{"name": "a", "value_source_id": "vs_nope"}]
)
data = r.json()
assert data["valid"] is True
assert data["warnings"]
def test_unbound_variable_is_error(self, client):
# Typo: expression uses 'ha_enti' but the input is named 'ha_entity'.
r = self._validate(
client, template="ha_enti", inputs=[{"name": "ha_entity", "value_source_id": ""}]
)
data = r.json()
assert data["valid"] is False
assert any("unbound" in e for e in data["errors"])
def test_cycle_detected_with_id(self, client):
t1 = _create(client, name="T1", template="clamp(0.5)", inputs=[]).json()
t2 = _create(
client,
name="T2",
template="x",
inputs=[{"name": "x", "value_source_id": t1["id"]}],
).json()
# Editing t1 to point at t2 would close a cycle.
r = self._validate(
client, template="x", inputs=[{"name": "x", "value_source_id": t2["id"]}], id=t1["id"]
)
assert r.json()["valid"] is False
class TestResponseMapCoverage:
def test_template_in_response_map(self):
from ledgrab.api.routes.value_sources import _RESPONSE_MAP
from ledgrab.storage.value_source import TemplateValueSource
assert TemplateValueSource in _RESPONSE_MAP
def test_template_in_all_unions(self):
from ledgrab.api.schemas import value_sources as sch
for union_name in ("ValueSourceResponse", "ValueSourceCreate", "ValueSourceUpdate"):
src = repr(getattr(sch, union_name))
assert "template" in src.lower() or "Template" in src
+171
View File
@@ -0,0 +1,171 @@
"""Integration tests for server-side subgraph duplication.
Exercises ``_duplicate_subgraph`` against the *real* value-source and
colour-strip stores (temp DB), asserting that references *within* the selection
are remapped to the clones while references to entities *outside* the selection
stay shared with the originals and that the deep-copy clone preserves config.
"""
import pytest
from ledgrab.api import dependencies as deps
from ledgrab.api.graph_schema import serialize_entity
from ledgrab.api.routes.graph import _duplicate_subgraph
from ledgrab.storage.color_strip_store import ColorStripStore
from ledgrab.storage.database import Database
from ledgrab.storage.value_source_store import ValueSourceStore
@pytest.fixture
def stores(tmp_path, monkeypatch):
db = Database(tmp_path / "dup.db")
css = ColorStripStore(db)
vss = ValueSourceStore(db)
monkeypatch.setattr(deps, "_deps", {"color_strip_store": css, "value_source_store": vss})
yield css, vss
db.close()
def _layer(sid: str) -> dict:
return {"source_id": sid, "blend_mode": "normal", "opacity": 1.0, "enabled": True}
def test_duplicate_composite_remaps_only_in_selection_refs(stores):
css, _vss = stores
b = css.create_source(name="B", source_type="single_color", colors=[[255, 0, 0]])
d = css.create_source(name="D", source_type="single_color", colors=[[0, 255, 0]])
a = css.create_source(name="A", source_type="composite", layers=[_layer(b.id), _layer(d.id)])
res = _duplicate_subgraph([a.id, b.id], " (copy)")
assert set(res["id_map"]) == {a.id, b.id}
assert res["skipped"] == []
assert res["warnings"] == []
a_new = serialize_entity(css.get(res["id_map"][a.id]))
layer_ids = [ly["source_id"] for ly in a_new["layers"]]
# B was in the selection -> remapped to the clone; D was not -> stays shared.
assert layer_ids == [res["id_map"][b.id], d.id]
assert a_new["name"] == "A (copy)"
# Original is untouched.
assert [ly["source_id"] for ly in serialize_entity(css.get(a.id))["layers"]] == [b.id, d.id]
def test_duplicate_preserves_clone_config(stores):
"""The deep-copy clone keeps every field except identity/name/timestamps —
guarding against the storage-vs-create-schema name mismatch (e.g. single
color's ``colors``) that a create-schema round-trip would silently drop."""
css, _vss = stores
src = css.create_source(name="Solid", source_type="single_color", colors=[[12, 34, 56]])
res = _duplicate_subgraph([src.id], " (copy)")
orig = serialize_entity(css.get(src.id))
clone = serialize_entity(css.get(res["id_map"][src.id]))
ignore = {"id", "name", "created_at", "updated_at"}
assert {k: v for k, v in clone.items() if k not in ignore} == {
k: v for k, v in orig.items() if k not in ignore
}
def test_duplicate_partial_selection_keeps_layer_refs_shared(stores):
css, _vss = stores
b = css.create_source(name="B", source_type="single_color", colors=[[255, 0, 0]])
a = css.create_source(name="A", source_type="composite", layers=[_layer(b.id)])
res = _duplicate_subgraph([a.id], " (copy)") # only the composite, not its layer
a_new = serialize_entity(css.get(res["id_map"][a.id]))
assert [ly["source_id"] for ly in a_new["layers"]] == [b.id] # shared with original
def test_duplicate_remaps_bindable_slot_to_cloned_value_source(stores):
"""Pass-2's dict round-trip path: a colour-strip bindable slot bound to a
value source that is *also* in the selection must point at the value clone
(not the original) after duplication."""
css, vss = stores
vs = vss.create_source(name="Pulse", source_type="static", value=0.5)
c = css.create_source(name="Candle", source_type="candlelight")
css.update_source(c.id, intensity={"source_id": vs.id}) # bind intensity -> vs
assert serialize_entity(css.get(c.id))["intensity"]["source_id"] == vs.id # binding took
res = _duplicate_subgraph([c.id, vs.id], " (copy)")
assert res["warnings"] == []
c_new = serialize_entity(css.get(res["id_map"][c.id]))
assert c_new["intensity"]["source_id"] == res["id_map"][vs.id]
def test_duplicate_remaps_layer_brightness_source(stores):
"""A composite layer's value-source brightness binding (list + value ref) is
remapped when that value source is also in the selection."""
css, vss = stores
vs = vss.create_source(name="Dim", source_type="static", value=0.3)
leaf = css.create_source(name="Leaf", source_type="single_color", colors=[[9, 9, 9]])
comp = css.create_source(
name="Comp",
source_type="composite",
layers=[{**_layer(leaf.id), "brightness_source_id": vs.id}],
)
res = _duplicate_subgraph([comp.id, leaf.id, vs.id], " (copy)")
layer = serialize_entity(css.get(res["id_map"][comp.id]))["layers"][0]
assert layer["source_id"] == res["id_map"][leaf.id]
assert layer["brightness_source_id"] == res["id_map"][vs.id]
def test_duplicate_safety_net_flags_unremapped_ref(stores, monkeypatch):
"""If a reference somehow isn't remapped, the post-clone safety net reports
it in `warnings` rather than silently leaving two pipelines sharing a node."""
css, _vss = stores
b = css.create_source(name="B", source_type="single_color", colors=[[1, 2, 3]])
a = css.create_source(name="A", source_type="composite", layers=[_layer(b.id)])
# Force remap to a no-op so the clone keeps pointing at the original in-set id.
monkeypatch.setattr("ledgrab.api.routes.graph.remap_refs", lambda *a, **k: 0)
res = _duplicate_subgraph([a.id, b.id], " (copy)")
assert any(w["id"] == res["id_map"][a.id] for w in res["warnings"])
def test_duplicate_value_source(stores):
_css, vss = stores
v = vss.create_source(name="V", source_type="static", value=0.5)
res = _duplicate_subgraph([v.id], " (copy)")
assert list(res["id_map"]) == [v.id]
new = vss.get(res["id_map"][v.id])
assert new.id != v.id
assert new.name == "V (copy)"
assert getattr(new, "value", None) == 0.5
def test_duplicate_skips_non_duplicable_ids(stores):
_css, vss = stores
v = vss.create_source(name="V", source_type="static", value=0.5)
res = _duplicate_subgraph([v.id, "dev_external", "bogus"], " (copy)")
assert list(res["id_map"]) == [v.id]
assert {s["id"] for s in res["skipped"]} == {"dev_external", "bogus"}
def test_duplicate_name_collision_is_suffixed(stores):
_css, vss = stores
v = vss.create_source(name="V", source_type="static", value=0.5)
vss.create_source(name="V (copy)", source_type="static", value=0.1) # occupy the obvious name
res = _duplicate_subgraph([v.id], " (copy)")
new = vss.get(res["id_map"][v.id])
assert new.name == "V (copy) 2"
def test_clone_allowlist_invariant():
"""Only explicitly-flagged (secret-free) stores are cloneable; the base
default is off so a new store is never cloneable by accident."""
from ledgrab.storage.base_sqlite_store import BaseSqliteStore
assert BaseSqliteStore._cloneable is False
assert ColorStripStore._cloneable is True
assert ValueSourceStore._cloneable is True
def test_clone_refuses_non_cloneable_store(stores, monkeypatch):
"""clone() refuses stores not on the allowlist (defence-in-depth even though
the duplicate endpoint already restricts to the safe kinds)."""
css, _vss = stores
src = css.create_source(name="X", source_type="single_color", colors=[[1, 1, 1]])
monkeypatch.setattr(type(css), "_cloneable", False)
with pytest.raises(NotImplementedError):
css.clone(src.id, "X (copy)")
+396
View File
@@ -0,0 +1,396 @@
"""Unit tests for the pure wiring-graph engine (ledgrab.api.graph_schema).
These exercise reference extraction, topology building, dependents, cycle and
dangling-reference detection without booting the app or any store.
"""
import json
import re
from dataclasses import dataclass, field
from ledgrab.api.graph_schema import (
CONNECTION_SCHEMA,
ENTITY_KINDS,
build_topology,
detect_cycles,
extract_refs,
find_dependents,
graph_field_roots,
is_editable,
remap_refs,
schema_for_kind,
serialize_entity_for_graph,
validate_connection,
would_create_cycle,
)
# ── extract_refs ────────────────────────────────────────────────────────────
def test_extract_refs_top_level():
assert extract_refs({"device_id": "dev_1"}, "device_id") == ["dev_1"]
def test_extract_refs_empty_and_missing_are_dropped():
assert extract_refs({"device_id": ""}, "device_id") == []
assert extract_refs({}, "device_id") == []
assert extract_refs({"device_id": None}, "device_id") == []
def test_extract_refs_bindable_bound_vs_unbound():
# Bound bindable serializes as a dict.
assert extract_refs(
{"brightness": {"value": 1.0, "source_id": "vs_1"}}, "brightness.source_id"
) == ["vs_1"]
# Unbound bindable serializes as a plain number → no reference.
assert extract_refs({"brightness": 0.5}, "brightness.source_id") == []
# Bound-but-empty source_id → no reference.
assert (
extract_refs({"brightness": {"value": 1.0, "source_id": ""}}, "brightness.source_id") == []
)
def test_extract_refs_list_field():
entity = {"layers": [{"source_id": "a"}, {"source_id": ""}, {"other": "x"}]}
assert extract_refs(entity, "layers[].source_id") == ["a"]
def test_extract_refs_deep_object_then_list():
entity = {"calibration": {"lines": [{"picture_source_id": "p1"}, {"picture_source_id": "p2"}]}}
assert extract_refs(entity, "calibration.lines[].picture_source_id") == ["p1", "p2"]
def test_extract_refs_nested_object_none_is_safe():
assert extract_refs({"settings": None}, "settings.pattern_template_id") == []
assert extract_refs(
{"settings": {"pattern_template_id": "pt_1"}}, "settings.pattern_template_id"
) == ["pt_1"]
# ── remap_refs (write-twin of extract_refs) ──────────────────────────────────
def test_remap_refs_top_level():
e = {"device_id": "a"}
assert remap_refs(e, "device_id", {"a": "b"}) == 1
assert e["device_id"] == "b"
def test_remap_refs_leaves_unmapped_ids_untouched():
e = {"device_id": "external"}
assert remap_refs(e, "device_id", {"a": "b"}) == 0
assert e["device_id"] == "external" # shared dependency outside the set
def test_remap_refs_bindable_bound():
e = {"brightness": {"value": 1.0, "source_id": "a"}}
assert remap_refs(e, "brightness.source_id", {"a": "b"}) == 1
assert e["brightness"] == {"value": 1.0, "source_id": "b"}
def test_remap_refs_unbound_bindable_is_safe():
e = {"brightness": 0.5} # plain number, no binding
assert remap_refs(e, "brightness.source_id", {"a": "b"}) == 0
assert e["brightness"] == 0.5
def test_remap_refs_list_field_only_mapped_elements():
e = {"layers": [{"source_id": "a"}, {"source_id": "external"}, {"source_id": "c"}]}
assert remap_refs(e, "layers[].source_id", {"a": "b", "c": "d"}) == 2
assert [layer["source_id"] for layer in e["layers"]] == ["b", "external", "d"]
def test_remap_refs_deep_object_then_list():
e = {"calibration": {"lines": [{"picture_source_id": "p1"}, {"picture_source_id": "p2"}]}}
assert remap_refs(e, "calibration.lines[].picture_source_id", {"p1": "q1"}) == 1
assert [ln["picture_source_id"] for ln in e["calibration"]["lines"]] == ["q1", "p2"]
def test_remap_refs_missing_keys_are_safe():
assert remap_refs({}, "layers[].source_id", {"a": "b"}) == 0
assert remap_refs({"layers": None}, "layers[].source_id", {"a": "b"}) == 0
def test_remap_refs_round_trips_with_extract_refs():
e = {"layers": [{"source_id": "a"}, {"source_id": "a"}]}
remap_refs(e, "layers[].source_id", {"a": "b"})
assert extract_refs(e, "layers[].source_id") == ["b", "b"]
# ── registry consistency ─────────────────────────────────────────────────────
def test_registry_kinds_are_known():
for cf in CONNECTION_SCHEMA:
assert cf.target_kind in ENTITY_KINDS, cf.target_kind
assert cf.source_kind in ENTITY_KINDS, cf.source_kind
def test_registry_has_no_duplicate_target_field_pairs():
pairs = [(cf.target_kind, cf.field) for cf in CONNECTION_SCHEMA]
assert len(pairs) == len(set(pairs)), "duplicate (target_kind, field) in CONNECTION_SCHEMA"
def test_schema_for_kind_filters_by_referrer():
fields = {cf.field for cf in schema_for_kind("automation")}
assert fields == {"scene_preset_id", "deactivation_scene_preset_id"}
# ── build_topology ───────────────────────────────────────────────────────────
def _sample_entities():
return {
"device": [{"id": "dev_1", "name": "Strip", "device_type": "wled"}],
"picture_source": [{"id": "ps_1", "name": "Cap", "stream_type": "raw"}],
"color_strip_source": [
{"id": "css_1", "name": "CSS", "source_type": "picture", "picture_source_id": "ps_1"}
],
"output_target": [
{
"id": "ot_1",
"name": "TV",
"target_type": "led",
"device_id": "dev_1",
"color_strip_source_id": "css_1",
}
],
}
def test_build_topology_nodes_and_edges():
topo = build_topology(_sample_entities())
assert {n["id"] for n in topo["nodes"]} == {"dev_1", "ps_1", "css_1", "ot_1"}
edge_set = {(e["from"], e["to"], e["field"]) for e in topo["edges"]}
assert ("ps_1", "css_1", "picture_source_id") in edge_set
assert ("dev_1", "ot_1", "device_id") in edge_set
assert ("css_1", "ot_1", "color_strip_source_id") in edge_set
assert topo["issues"]["orphans"] == []
assert topo["issues"]["broken_refs"] == []
assert topo["issues"]["cycles"] == []
def test_build_topology_flags_orphan_and_broken_ref():
entities = _sample_entities()
entities["sync_clock"] = [{"id": "clk_1", "name": "Lonely"}] # no edges → orphan
entities["color_strip_source"][0]["audio_source_id"] = "ghost" # dangling
topo = build_topology(entities)
assert "clk_1" in topo["issues"]["orphans"]
broken = topo["issues"]["broken_refs"]
assert {"ref": "ghost", "by": "css_1", "field": "audio_source_id"} in broken
# The dangling ref must NOT appear as a real edge.
assert all(e["from"] != "ghost" for e in topo["edges"])
def test_build_topology_detects_cycle():
entities = {
"value_source": [
{"id": "vs_1", "name": "A", "source_type": "static", "value_source_id": "vs_2"},
{"id": "vs_2", "name": "B", "source_type": "static", "value_source_id": "vs_1"},
]
}
topo = build_topology(entities)
assert set(topo["issues"]["cycles"]) == {"vs_1", "vs_2"}
# ── detect_cycles ────────────────────────────────────────────────────────────
def test_detect_cycles_no_false_positive_on_diamond():
# a→b, a→c, b→d, c→d (DAG)
edges = [
{"from": "a", "to": "b"},
{"from": "a", "to": "c"},
{"from": "b", "to": "d"},
{"from": "c", "to": "d"},
]
assert detect_cycles(edges) == set()
def test_detect_cycles_marks_only_cycle_members():
# x→a→b→a (a,b in cycle; x is not)
edges = [
{"from": "x", "to": "a"},
{"from": "a", "to": "b"},
{"from": "b", "to": "a"},
]
assert detect_cycles(edges) == {"a", "b"}
# ── would_create_cycle ───────────────────────────────────────────────────────
def test_would_create_cycle_detects_back_edge_and_self():
edges = [{"from": "a", "to": "b"}] # b depends on a (a→b)
# Wiring a to consume b (edge b→a) closes the a→b→a loop.
assert would_create_cycle(edges, source_id="b", target_id="a") is True
# Wiring an unrelated pair is fine.
assert would_create_cycle(edges, source_id="a", target_id="c") is False
# Self-reference is always a cycle.
assert would_create_cycle(edges, source_id="z", target_id="z") is True
# ── find_dependents ──────────────────────────────────────────────────────────
def test_find_dependents_returns_referrers():
deps = find_dependents(_sample_entities(), "color_strip_source", "css_1")
assert deps == [
{"id": "ot_1", "kind": "output_target", "name": "TV", "field": "color_strip_source_id"}
]
def test_find_dependents_empty_when_unreferenced():
assert find_dependents(_sample_entities(), "color_strip_source", "css_unused") == []
# ── validate_connection ──────────────────────────────────────────────────────
def test_validate_connection_accepts_valid_edit():
ok, err = validate_connection(_sample_entities(), "output_target", "ot_1", "device_id", "dev_1")
assert ok is True
assert err is None
def test_validate_connection_detach_always_ok():
ok, err = validate_connection(_sample_entities(), "output_target", "ot_1", "device_id", "")
assert ok is True and err is None
def test_validate_connection_rejects_unknown_field():
ok, err = validate_connection(_sample_entities(), "output_target", "ot_1", "nope_id", "dev_1")
assert ok is False
assert "Unknown connection field" in err
def test_validate_connection_rejects_missing_source():
ok, err = validate_connection(_sample_entities(), "output_target", "ot_1", "device_id", "ghost")
assert ok is False
assert "not found" in err
def test_validate_connection_accepts_bindable_single_field():
# Single-level bindable slots (e.g. brightness) ARE editable — only list
# fields are rejected.
entities = {
"color_strip_source": [
{"id": "css_1", "name": "X", "source_type": "effect", "brightness": 0.5}
],
"value_source": [{"id": "vs_1", "name": "V", "source_type": "static"}],
}
ok, err = validate_connection(
entities, "color_strip_source", "css_1", "brightness.source_id", "vs_1"
)
assert ok is True
assert err is None
def test_validate_connection_rejects_list_field():
# List slots can't be cycle-checked without an element index → rejected.
entities = {
"color_strip_source": [
{
"id": "css_1",
"name": "X",
"source_type": "composite",
"layers": [{"source_id": "css_2"}],
},
{"id": "css_2", "name": "Y", "source_type": "picture"},
]
}
ok, err = validate_connection(
entities, "color_strip_source", "css_1", "layers[].source_id", "css_2"
)
assert ok is False
assert "not editable" in err
def test_validate_connection_rejects_color_bindable():
# Colour bindings are structurally bindable but not graph-editable (a value
# source can't drive a colour).
entities = {
"color_strip_source": [
{"id": "css_1", "name": "X", "source_type": "single_color", "color": [255, 0, 0]}
],
"value_source": [{"id": "vs_1", "name": "V", "source_type": "static"}],
}
ok, err = validate_connection(
entities, "color_strip_source", "css_1", "color.source_id", "vs_1"
)
assert ok is False
assert "not editable" in err
@dataclass
class _FakeRule:
token: str = "SUPER_SECRET_WEBHOOK_TOKEN"
@dataclass
class _FakeAutomation:
id: str = "auto_1"
name: str = "A"
enabled: bool = True
scene_preset_id: str = "sp_1"
rules: list = field(default_factory=lambda: [_FakeRule()])
# Keys that must NEVER appear in the /graph projection allowlist.
_SECRET_KEY_RE = re.compile(
r"token|password|secret|credential|api[_-]?key|_key$|username|\burl\b|\bhost\b", re.I
)
def test_projection_roots_never_expose_secrets():
# The /graph projection (graph_field_roots) is the leak boundary. Assert no
# kind's allowlist contains a secret-bearing key — locks the boundary against
# future schema drift (a new reference field whose root looks like a secret).
for kind in ENTITY_KINDS:
for root in graph_field_roots(kind):
assert not _SECRET_KEY_RE.search(
root
), f"{kind}: projected root {root!r} looks secret-bearing"
def test_serialize_entity_for_graph_drops_secrets():
# The graph projection must strip everything except id/name/type + reference
# roots — a deep asdict would otherwise leak the webhook token (a real,
# auth-equivalent secret) in the /graph response.
projected = serialize_entity_for_graph("automation", _FakeAutomation())
assert projected["id"] == "auto_1"
assert projected["name"] == "A"
assert projected["scene_preset_id"] == "sp_1" # reference root kept
assert "rules" not in projected # non-reference field dropped
assert "enabled" not in projected
assert "SUPER_SECRET_WEBHOOK_TOKEN" not in json.dumps(projected)
def test_is_editable_classifies_fields():
by_field = {(c.target_kind, c.field): c for c in CONNECTION_SCHEMA}
# Top-level reference and single-level BindableFloat → editable.
assert is_editable(by_field[("output_target", "device_id")])
assert is_editable(by_field[("color_strip_source", "brightness.source_id")])
assert is_editable(by_field[("output_target", "transition.source_id")])
# Colour binding, list slot, and double-nested → NOT editable.
assert not is_editable(by_field[("color_strip_source", "color.source_id")])
assert not is_editable(by_field[("color_strip_source", "layers[].source_id")])
assert not is_editable(by_field[("output_target", "settings.brightness.source_id")])
def test_validate_connection_rejects_cycle():
entities = {
"value_source": [
{"id": "vs_1", "name": "A", "source_type": "static", "value_source_id": ""},
{"id": "vs_2", "name": "B", "source_type": "static", "value_source_id": "vs_1"},
]
}
# vs_2 already depends on vs_1 (edge vs_1→vs_2). Wiring vs_1 to consume vs_2
# would close the loop.
ok, err = validate_connection(entities, "value_source", "vs_1", "value_source_id", "vs_2")
assert ok is False
assert "cycle" in err.lower()
@@ -0,0 +1,159 @@
"""Contract tests for graph drag-to-wire writes.
The graph editor's ``updateConnection()`` performs a *partial* PUT for a single
dragged edge: it sends only the reference (or bindable) field being wired. Five
entity kinds have ``Body(discriminator=...)`` PUT routes, so such a partial body
is rejected with a 422 unless it also echoes the entity's subtype
(``source_type`` / ``stream_type`` / ``target_type``). ``updateConnection()`` now
reads the target's subtype back and includes it.
These tests lock in that contract from the backend side: each ``(kind, field)``
pair the frontend ``CONNECTION_MAP`` drag-edits must validate as a minimal
``{discriminator, field}`` body and must be rejected without the discriminator
(the exact failure that silently broke wiring). If a future schema change makes
one of these fields required-with-siblings, or drops a subtype, these tests fail
and flag that graph wiring will break.
"""
import pytest
from pydantic import TypeAdapter, ValidationError
from ledgrab.api.schemas.audio_sources import AudioSourceUpdate
from ledgrab.api.schemas.color_strip_sources import ColorStripSourceUpdate
from ledgrab.api.schemas.output_targets import OutputTargetUpdate
from ledgrab.api.schemas.picture_sources import PictureSourceUpdate
from ledgrab.api.schemas.value_sources import ValueSourceUpdate
# Each row mirrors one drag-editable (target_kind, field) pair from the frontend
# CONNECTION_MAP, paired with a real subtype tag that owns that field and the
# body shape updateConnection() sends: a flat ref id, or a bindable
# ``{parent: {source_id}}`` slot (here keyed by the parent field root).
# (kind, update_union, discriminator_field, subtype_tag, body_field, sample_value)
_WIRING_WRITES = [
("value_source", ValueSourceUpdate, "source_type", "audio", "audio_source_id", "as_1"),
(
"value_source",
ValueSourceUpdate,
"source_type",
"adaptive_scene",
"picture_source_id",
"ps_1",
),
("value_source", ValueSourceUpdate, "source_type", "gradient_map", "value_source_id", "vs_1"),
(
"value_source",
ValueSourceUpdate,
"source_type",
"css_extract",
"color_strip_source_id",
"css_1",
),
(
"color_strip_source",
ColorStripSourceUpdate,
"source_type",
"picture",
"picture_source_id",
"ps_1",
),
(
"color_strip_source",
ColorStripSourceUpdate,
"source_type",
"audio",
"audio_source_id",
"as_1",
),
(
"color_strip_source",
ColorStripSourceUpdate,
"source_type",
"processed",
"input_source_id",
"css_2",
),
(
"color_strip_source",
ColorStripSourceUpdate,
"source_type",
"processed",
"processing_template_id",
"cspt_1",
),
# bindable BindableFloat slot — body is {<parent>: {source_id}}
(
"color_strip_source",
ColorStripSourceUpdate,
"source_type",
"audio",
"smoothing",
{"source_id": "vs_1"},
),
("audio_source", AudioSourceUpdate, "source_type", "capture", "audio_template_id", "at_1"),
("audio_source", AudioSourceUpdate, "source_type", "processed", "audio_source_id", "as_2"),
("picture_source", PictureSourceUpdate, "stream_type", "raw", "capture_template_id", "ct_1"),
("picture_source", PictureSourceUpdate, "stream_type", "processed", "source_stream_id", "ps_2"),
(
"picture_source",
PictureSourceUpdate,
"stream_type",
"processed",
"postprocessing_template_id",
"ppt_1",
),
("output_target", OutputTargetUpdate, "target_type", "led", "device_id", "dev_1"),
("output_target", OutputTargetUpdate, "target_type", "led", "color_strip_source_id", "css_1"),
# bindable slots on output targets
(
"output_target",
OutputTargetUpdate,
"target_type",
"led",
"brightness",
{"source_id": "vs_1"},
),
(
"output_target",
OutputTargetUpdate,
"target_type",
"ha_light",
"transition",
{"source_id": "vs_1"},
),
]
_IDS = [f"{kind}.{field}[{tag}]" for kind, _u, _d, tag, field, _v in _WIRING_WRITES]
@pytest.mark.parametrize("kind,union,disc,tag,field,value", _WIRING_WRITES, ids=_IDS)
def test_partial_wiring_put_validates_with_discriminator(kind, union, disc, tag, field, value):
"""A single dragged wiring edit validates when the subtype is echoed.
This is exactly what updateConnection() now sends; if it fails, drag-to-wire
for ``kind.field`` is broken.
"""
model = TypeAdapter(union).validate_python({disc: tag, field: value})
assert getattr(model, disc) == tag
@pytest.mark.parametrize("kind,union,disc,tag,field,value", _WIRING_WRITES, ids=_IDS)
def test_partial_wiring_put_rejected_without_discriminator(kind, union, disc, tag, field, value):
"""Without the discriminator the same body is rejected.
This is the 422 that silently broke drag-to-wire before updateConnection()
echoed the subtype guarding against anyone "simplifying" that away.
"""
with pytest.raises(ValidationError):
TypeAdapter(union).validate_python({field: value})
@pytest.mark.parametrize("kind,union,disc,tag,field,value", _WIRING_WRITES, ids=_IDS)
def test_partial_wiring_detach_validates_with_discriminator(kind, union, disc, tag, field, value):
"""Detach (clearing a slot) goes through the same partial-PUT path and must
also validate with the subtype echoed: ``{field: ""}`` for a flat reference,
``{parent: {source_id: ""}}`` for a bindable slot exactly what
``detachConnection()`` -> ``updateConnection(..., "")`` sends.
"""
cleared = {"source_id": ""} if isinstance(value, dict) else ""
model = TypeAdapter(union).validate_python({disc: tag, field: cleared})
assert getattr(model, disc) == tag
@@ -0,0 +1,231 @@
"""Tests for TemplateValueStream (the Jinja combinator runtime)."""
from collections import defaultdict
from datetime import datetime, timezone
import pytest
from ledgrab.core.processing.value_stream import (
TemplateValueStream,
ValueStreamManager,
)
from ledgrab.storage.value_source import TemplateValueSource
# --- Fakes for precise control over input values / raw -----------------------
class _FakeStream:
_NO_RAW = object()
def __init__(self, value, raw=_NO_RAW):
self._value = value
self._raw = raw
def get_value(self):
return self._value
# get_raw_value only exists when a raw value was provided
def __getattr__(self, name):
if name == "get_raw_value" and self._raw is not _FakeStream._NO_RAW:
return lambda: self._raw
raise AttributeError(name)
class _FakeVSM:
def __init__(self, streams):
self._streams = streams # id -> _FakeStream
self.refcounts = defaultdict(int)
def acquire(self, vs_id):
self.refcounts[vs_id] += 1
return self._streams[vs_id]
def release(self, vs_id):
self.refcounts[vs_id] -= 1
def _inputs(*pairs):
return [{"name": n, "value_source_id": i} for n, i in pairs]
def _make(template, inputs, streams, default_value=0.0, eval_interval=None):
vsm = _FakeVSM(streams)
stream = TemplateValueStream(
template=template,
inputs=inputs,
default_value=default_value,
eval_interval=eval_interval,
value_stream_manager=vsm,
)
stream.start()
return stream, vsm
class TestEvaluation:
def test_eval_with_inputs(self):
stream, vsm = _make("min(a * 2, 1)", _inputs(("a", "vs_a")), {"vs_a": _FakeStream(0.3)})
assert vsm.refcounts["vs_a"] == 1
assert stream.get_value() == pytest.approx(0.6)
def test_clamps_out_of_range(self):
stream, _ = _make("a * 10", _inputs(("a", "vs_a")), {"vs_a": _FakeStream(0.5)})
assert stream.get_value() == 1.0 # 5.0 clamped
def test_two_inputs(self):
stream, _ = _make(
"(a + b) / 2",
_inputs(("a", "vs_a"), ("b", "vs_b")),
{"vs_a": _FakeStream(0.2), "vs_b": _FakeStream(0.8)},
)
assert stream.get_value() == pytest.approx(0.5)
def test_shared_id_single_ref(self):
# Two variables bound to the same source share one acquisition.
stream, vsm = _make(
"min(a + b, 1)",
_inputs(("a", "vs_x"), ("b", "vs_x")),
{"vs_x": _FakeStream(0.3)},
)
assert vsm.refcounts["vs_x"] == 1
assert stream.get_value() == pytest.approx(0.6)
class TestErrorHandling:
def test_div_by_zero_returns_default(self):
stream, _ = _make(
"a / 0", _inputs(("a", "vs_a")), {"vs_a": _FakeStream(0.5)}, default_value=0.25
)
assert stream.get_value() == 0.25
def test_missing_variable_returns_default(self):
# template references 'b' but only 'a' is bound
stream, _ = _make(
"a + b", _inputs(("a", "vs_a")), {"vs_a": _FakeStream(0.5)}, default_value=0.1
)
assert stream.get_value() == 0.1
def test_nan_returns_default(self):
stream, _ = _make(
"a - a", _inputs(("a", "vs_a")), {"vs_a": _FakeStream(float("inf"))}, default_value=0.3
)
# inf - inf = nan -> default
assert stream.get_value() == 0.3
def test_invalid_template_uses_default(self):
stream, _ = _make(
"a +", _inputs(("a", "vs_a")), {"vs_a": _FakeStream(0.5)}, default_value=0.42
)
assert stream.get_value() == 0.42
class TestRawExposure:
def test_raw_present_when_stream_exposes_it(self):
stream, _ = _make(
"raw['t'] / 100",
_inputs(("t", "vs_t")),
{"vs_t": _FakeStream(0.5, raw=42.0)},
)
assert stream.get_value() == pytest.approx(0.42)
def test_raw_absent_without_getter(self):
# input stream has no get_raw_value -> raw['t'] -> None -> error -> default
stream, _ = _make(
"raw['t'] / 100",
_inputs(("t", "vs_t")),
{"vs_t": _FakeStream(0.5)},
default_value=0.2,
)
assert stream.get_value() == 0.2
def test_non_numeric_raw_is_dropped(self):
# raw value is a string -> never crosses into sandbox -> raw['t'] absent
stream, _ = _make(
"raw['t'] / 100",
_inputs(("t", "vs_t")),
{"vs_t": _FakeStream(0.5, raw="playing")},
default_value=0.15,
)
assert stream.get_value() == 0.15
class TestLifecycle:
def test_stop_releases_all(self):
stream, vsm = _make(
"min(a + b, 1)",
_inputs(("a", "vs_a"), ("b", "vs_b")),
{"vs_a": _FakeStream(0.1), "vs_b": _FakeStream(0.2)},
)
stream.stop()
assert vsm.refcounts["vs_a"] == 0
assert vsm.refcounts["vs_b"] == 0
def test_eval_interval_caches(self):
backing = _FakeStream(0.2)
stream, _ = _make("a", _inputs(("a", "vs_a")), {"vs_a": backing}, eval_interval=3600.0)
first = stream.get_value()
backing._value = 0.9 # change the live input
# Cached within the interval -> still the first value.
assert stream.get_value() == pytest.approx(first)
class TestHotUpdate:
def test_swap_input_releases_old_acquires_new(self):
stream, vsm = _make(
"a", _inputs(("a", "vs_a")), {"vs_a": _FakeStream(0.1), "vs_b": _FakeStream(0.9)}
)
assert vsm.refcounts["vs_a"] == 1
new_src = TemplateValueSource(
id="t1",
name="t",
source_type="template",
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc),
template="a",
inputs=_inputs(("a", "vs_b")),
default_value=0.0,
)
stream.update_source(new_src)
assert vsm.refcounts["vs_a"] == 0 # old released
assert vsm.refcounts["vs_b"] == 1 # new acquired
assert stream.get_value() == pytest.approx(0.9)
def test_rename_keeps_same_source(self):
stream, vsm = _make("a", _inputs(("a", "vs_a")), {"vs_a": _FakeStream(0.7)})
renamed = TemplateValueSource(
id="t1",
name="t",
source_type="template",
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc),
template="b", # variable renamed a -> b, same source id
inputs=_inputs(("b", "vs_a")),
default_value=0.0,
)
stream.update_source(renamed)
assert vsm.refcounts["vs_a"] == 1 # not re-acquired (unchanged id)
assert stream.get_value() == pytest.approx(0.7)
class TestAcquireDepthBackstop:
def test_self_reference_does_not_overflow(self):
"""A cycle that bypassed storage validation must not stack-overflow."""
now = datetime.now(timezone.utc)
src = TemplateValueSource(
id="vs_cycle",
name="cycle",
source_type="template",
created_at=now,
updated_at=now,
template="x",
inputs=_inputs(("x", "vs_cycle")),
default_value=0.0,
)
class _CycleStore:
def get_source(self, vs_id):
return src
manager = ValueStreamManager(value_source_store=_CycleStore())
stream = manager.acquire("vs_cycle") # must terminate, not recurse forever
assert isinstance(stream.get_value(), float)
@@ -0,0 +1,380 @@
"""Tests for the optional-normalization feature on magnitude value sources.
Covers, for HA / HTTP / system_metrics / game_event:
* the ``normalize`` flag round-trips through to_dict/from_dict and defaults to
True for old rows that predate the field,
* ``normalize=False`` makes get_value() a finite-safe clamp passthrough while
get_value() stays in [0, 1] (the normalized scalar-bus contract), and
* the raw magnitude remains available via get_raw_value() (added to
game_event, which had no raw channel before).
Plus a unit test for the shared finite-safe ``clamp01`` helper.
"""
import math
import pytest
from ledgrab.core.game_integration.event_bus import GameEventBus
from ledgrab.core.game_integration.events import GameEvent
from ledgrab.core.processing.value_kinds import ValueStreamDeps, build_stream
from ledgrab.core.processing.value_stream import (
HAEntityValueStream,
HTTPValueStream,
SystemMetricsValueStream,
)
from ledgrab.core.value_sources.game_event_value_source import GameEventValueStream
from ledgrab.storage.value_source import ValueSource
from ledgrab.utils import clamp01
# ---------------------------------------------------------------------------
# clamp01 finite-safe helper
# ---------------------------------------------------------------------------
class TestClamp01:
def test_in_range_passthrough(self):
assert clamp01(0.0) == 0.0
assert clamp01(0.42) == pytest.approx(0.42)
assert clamp01(1.0) == 1.0
def test_out_of_range_clamped(self):
assert clamp01(-2.5) == 0.0
assert clamp01(5.0) == 1.0
def test_non_finite_rejected_to_default(self):
assert clamp01(float("nan")) == 0.0
assert clamp01(float("inf")) == 0.0
assert clamp01(float("-inf")) == 0.0
# custom default is honoured
assert clamp01(float("nan"), default=0.5) == 0.5
# ---------------------------------------------------------------------------
# Round-trip persistence of the normalize flag
# ---------------------------------------------------------------------------
def _base(source_type: str, **extra) -> dict:
return {
"id": f"vs_{source_type}",
"name": source_type,
"source_type": source_type,
"created_at": "2025-01-01T00:00:00+00:00",
"updated_at": "2025-01-01T00:00:00+00:00",
**extra,
}
class TestNormalizeRoundTrip:
@pytest.mark.parametrize(
"source_type,extra",
[
("ha_entity", {"ha_source_id": "ha1", "entity_id": "sensor.temp"}),
("http", {"http_endpoint_id": "ep1"}),
("system_metrics", {"metric": "cpu_load"}),
("game_event", {}),
],
)
def test_normalize_false_round_trips(self, source_type, extra):
src = ValueSource.from_dict(_base(source_type, normalize=False, **extra))
assert src.normalize is False
assert src.to_dict()["normalize"] is False
restored = ValueSource.from_dict(src.to_dict())
assert restored.normalize is False
@pytest.mark.parametrize(
"source_type,extra",
[
("ha_entity", {"ha_source_id": "ha1", "entity_id": "sensor.temp"}),
("http", {"http_endpoint_id": "ep1"}),
("system_metrics", {"metric": "cpu_load"}),
("game_event", {}),
],
)
def test_missing_key_defaults_true(self, source_type, extra):
"""An old row written before the field existed reads back normalize=True."""
src = ValueSource.from_dict(_base(source_type, **extra))
assert src.normalize is True
assert src.to_dict()["normalize"] is True
# ---------------------------------------------------------------------------
# HTTP stream — clamp passthrough + verbatim raw
# ---------------------------------------------------------------------------
def _http(normalize: bool, raw) -> HTTPValueStream:
s = HTTPValueStream(
endpoint_id="ep",
json_path="",
interval_s=60,
min_value=0.0,
max_value=100.0,
smoothing=0.0,
normalize=normalize,
)
s._raw_value = raw # bypass the poll loop; exercise get_value() directly
return s
class TestHttpNormalize:
def test_normalize_true_rescales(self):
assert _http(True, 50.0).get_value() == pytest.approx(0.5)
def test_normalize_false_clamps_in_range(self):
assert _http(False, 0.7).get_value() == pytest.approx(0.7)
def test_normalize_false_clamps_out_of_range(self):
assert _http(False, 5.0).get_value() == 1.0
assert _http(False, -3.0).get_value() == 0.0
def test_normalize_false_non_finite_safe(self):
assert _http(False, float("inf")).get_value() == 0.0
assert _http(False, float("nan")).get_value() == 0.0
def test_non_numeric_raw_keeps_get_value_numeric_and_raw_verbatim(self):
s = _http(False, "playing")
assert s.get_value() == 0.0 # float("playing") fails -> safe fallback
assert s.get_raw_value() == "playing" # verbatim for automations/templates
# ---------------------------------------------------------------------------
# HA stream — clamp passthrough via a minimal fake HA manager
# ---------------------------------------------------------------------------
class _FakeState:
def __init__(self, state, attributes=None):
self.state = state
self.attributes = attributes or {}
class _FakeHA:
def __init__(self, state):
self._state = state
def get_state(self, _source_id, _entity_id):
return self._state
def _ha(normalize: bool, state_value: str) -> HAEntityValueStream:
return HAEntityValueStream(
ha_source_id="ha1",
entity_id="sensor.temp",
attribute="",
min_ha_value=0.0,
max_ha_value=100.0,
smoothing=0.0,
normalize=normalize,
ha_manager=_FakeHA(_FakeState(state_value)),
)
class TestHaNormalize:
def test_normalize_true_rescales(self):
assert _ha(True, "50").get_value() == pytest.approx(0.5)
def test_normalize_false_clamps_and_keeps_raw(self):
s = _ha(False, "21.5") # e.g. °C — out of [0,1]
assert s.get_value() == 1.0 # clamped
assert s.get_raw_value() == pytest.approx(21.5) # un-clamped magnitude
def test_normalize_false_in_range_fraction(self):
s = _ha(False, "0.3")
assert s.get_value() == pytest.approx(0.3)
# ---------------------------------------------------------------------------
# system_metrics stream — clamp passthrough (raw reader monkeypatched)
# ---------------------------------------------------------------------------
class TestSystemMetricsNormalize:
def test_normalize_false_clamps_and_keeps_raw(self):
s = SystemMetricsValueStream(
metric="cpu_load", min_value=0.0, max_value=100.0, normalize=False
)
s._read_metric = lambda: 65.0 # type: ignore[method-assign]
assert s.get_value() == 1.0 # 65 clamped to 1.0
assert s.get_raw_value() == pytest.approx(65.0)
def test_normalize_false_in_range_fraction(self):
s = SystemMetricsValueStream(
metric="cpu_load", min_value=0.0, max_value=100.0, normalize=False
)
s._read_metric = lambda: 0.25 # type: ignore[method-assign]
assert s.get_value() == pytest.approx(0.25)
# ---------------------------------------------------------------------------
# game_event stream — new raw channel + normalize flag
# ---------------------------------------------------------------------------
def _evt(value: float) -> GameEvent:
return GameEvent(adapter_id="t", event_type="health", value=value)
class TestGameEventNormalizeAndRaw:
def test_raw_value_none_before_first_event(self):
bus = GameEventBus()
stream = GameEventValueStream(event_type="health", event_bus=bus)
stream.start()
assert stream.get_raw_value() is None
stream.stop()
def test_raw_value_exposed_after_event(self):
bus = GameEventBus()
stream = GameEventValueStream(
event_type="health", min_game_value=0.0, max_game_value=100.0, event_bus=bus
)
stream.start()
bus.publish(_evt(73.0))
assert stream.get_value() == pytest.approx(0.73) # normalized output
assert stream.get_raw_value() == pytest.approx(73.0) # un-clamped raw
stream.stop()
def test_normalize_false_clamps_output_keeps_raw(self):
bus = GameEventBus()
stream = GameEventValueStream(
event_type="health",
min_game_value=0.0,
max_game_value=100.0,
normalize=False,
event_bus=bus,
)
stream.start()
bus.publish(_evt(50.0)) # 50 is > 1 -> clamped to 1.0 in passthrough mode
assert stream.get_value() == 1.0
assert stream.get_raw_value() == pytest.approx(50.0)
stream.stop()
def test_normalize_false_in_range_fraction(self):
bus = GameEventBus()
stream = GameEventValueStream(event_type="health", normalize=False, event_bus=bus)
stream.start()
bus.publish(_evt(0.4))
assert stream.get_value() == pytest.approx(0.4)
stream.stop()
def test_get_value_always_finite_in_passthrough(self):
bus = GameEventBus()
stream = GameEventValueStream(event_type="health", normalize=False, event_bus=bus)
stream.start()
bus.publish(_evt(math.inf))
assert stream.get_value() == 0.0 # finite-safe clamp
stream.stop()
# ---------------------------------------------------------------------------
# Live update_source() flip of the normalize flag on a running stream
# ---------------------------------------------------------------------------
# The design's load-bearing invariant: flipping normalize live never blends a
# raw magnitude against a fraction in the EMA, because BOTH modes emit [0,1].
# These tests lock that in (a future regression dropping a stream's
# _normalize_enabled reassignment would otherwise pass every other test).
class TestLiveNormalizeFlip:
def test_http_flip_true_to_false_and_back(self):
s = _http(True, 50.0)
assert s.get_value() == pytest.approx(0.5) # rescaled
s.update_source(
ValueSource.from_dict(
_base(
"http", http_endpoint_id="ep", min_value=0.0, max_value=100.0, normalize=False
)
)
)
assert s.get_value() == 1.0 # now clamp-passthrough of 50.0
s.update_source(
ValueSource.from_dict(
_base("http", http_endpoint_id="ep", min_value=0.0, max_value=100.0, normalize=True)
)
)
assert s.get_value() == pytest.approx(0.5) # rescaled again, no stale-cache blend
def test_ha_flip_true_to_false(self):
s = _ha(True, "50")
assert s.get_value() == pytest.approx(0.5)
s.update_source(
ValueSource.from_dict(
_base(
"ha_entity",
ha_source_id="ha1",
entity_id="sensor.temp",
min_ha_value=0.0,
max_ha_value=100.0,
normalize=False,
)
)
)
assert s.get_value() == 1.0
def test_system_metrics_flip_true_to_false(self):
s = SystemMetricsValueStream(
metric="cpu_load", min_value=0.0, max_value=100.0, normalize=True
)
s._read_metric = lambda: 65.0 # type: ignore[method-assign]
first = s.get_value() # rescaled via the cpu_load spec
assert 0.0 <= first <= 1.0
s.update_source(
ValueSource.from_dict(
_base(
"system_metrics",
metric="cpu_load",
min_value=0.0,
max_value=100.0,
normalize=False,
)
)
)
s._read_metric = lambda: 65.0 # type: ignore[method-assign]
s._prev_value = None # force a fresh poll (bypass the poll-interval cache)
assert s.get_value() == 1.0 # clamp-passthrough of 65.0
def test_game_event_flip_true_to_false(self):
bus = GameEventBus()
stream = GameEventValueStream(
event_type="health",
min_game_value=0.0,
max_game_value=100.0,
normalize=True,
event_bus=bus,
)
stream.start()
bus.publish(_evt(50.0))
assert stream.get_value() == pytest.approx(0.5)
stream.update_source(
ValueSource.from_dict(
_base("game_event", min_game_value=0.0, max_game_value=100.0, normalize=False)
)
)
bus.publish(_evt(50.0))
assert stream.get_value() == 1.0 # clamp-passthrough
assert stream.get_raw_value() == pytest.approx(50.0)
stream.stop()
# ---------------------------------------------------------------------------
# build_stream() forwarding — game_event's SOLE wiring path (no CRUD/schema)
# ---------------------------------------------------------------------------
class TestBuildStreamForwarding:
def test_game_event_builder_forwards_normalize(self):
src = ValueSource.from_dict(
_base("game_event", min_game_value=0.0, max_game_value=100.0, normalize=False)
)
bus = GameEventBus()
deps = ValueStreamDeps(value_stream_manager=None, event_bus=bus)
stream = build_stream(src, deps)
assert isinstance(stream, GameEventValueStream)
assert stream._normalize_enabled is False # forwarded from the dataclass
stream.start()
bus.publish(_evt(50.0))
assert stream.get_value() == 1.0 # passthrough clamp, not 0.5 rescale
assert stream.get_raw_value() == pytest.approx(50.0)
stream.stop()
@@ -0,0 +1,253 @@
"""Tests for the Android playback-capture audio engine.
These run on desktop CI (no Android device needed): ``is_android`` is
monkeypatched and PCM is pushed directly into the module-level queue,
exactly as the Kotlin bridge would.
"""
import queue
import numpy as np
import pytest
# Importing the package triggers auto-registration of AndroidAudioEngine.
import ledgrab.core.audio # noqa: F401
from ledgrab.core.audio import android_audio_engine as eng
from ledgrab.core.audio.analysis import AudioAnalysis, AudioAnalyzer
from ledgrab.core.audio.audio_capture import AudioCaptureManager
from ledgrab.core.audio.factory import AudioEngineRegistry
ENGINE_MOD = "ledgrab.core.audio.android_audio_engine"
SAMPLE_RATE = 48000
CHANNELS = 2
CHUNK = 1024
# ---------------------------------------------------------------------------
# Helpers / fixtures
# ---------------------------------------------------------------------------
def _drain() -> None:
while not eng._pcm_queue.empty():
try:
eng._pcm_queue.get_nowait()
except queue.Empty:
break
def _block(marker: float = 0.0, frames: int = CHUNK, channels: int = CHANNELS) -> np.ndarray:
"""A float32 interleaved block whose first sample is ``marker``."""
data = np.zeros(frames * channels, dtype=np.float32)
data[0] = marker
return data
@pytest.fixture
def reset_engine():
"""Reset module-global engine state; snapshot/restore the registry.
The engine keeps its queue + format in module globals and the registry
is a class-level singleton both must be restored so this test file
never disturbs the desktop engines other tests rely on.
"""
saved_engines = dict(AudioEngineRegistry._engines)
eng.shutdown()
_drain()
eng._sample_rate = SAMPLE_RATE
eng._channels = CHANNELS
eng._chunk_size = CHUNK
eng._frames_received = 0
yield eng
eng.shutdown()
_drain()
AudioEngineRegistry._engines.clear()
AudioEngineRegistry._engines.update(saved_engines)
@pytest.fixture
def on_android(monkeypatch, reset_engine):
"""Engine fixture with ``is_android`` forced True and demo mode off."""
monkeypatch.setattr(f"{ENGINE_MOD}.is_android", lambda: True)
monkeypatch.setattr("ledgrab.core.audio.factory.is_demo_mode", lambda: False)
return reset_engine
# ---------------------------------------------------------------------------
# Queue / push contract
# ---------------------------------------------------------------------------
def test_configure_then_push_round_trips_samples(reset_engine):
# Arrange
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
samples = np.arange(CHUNK * CHANNELS, dtype=np.float32)
# Act
eng.push_samples(samples.tobytes())
stream = eng.AndroidAudioEngine.create_stream(0, True, {})
stream.initialize()
got = stream.read_chunk()
# Assert
assert got is not None
np.testing.assert_array_equal(got, samples)
def test_queue_drops_oldest_when_full(reset_engine):
# Arrange
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
maxsize = eng._pcm_queue.maxsize # 8
# Act — push more blocks than the queue can hold, each tagged 0..N-1
total = maxsize + 2
for i in range(total):
eng.push_samples(_block(marker=float(i)).tobytes())
drained = []
while True:
try:
drained.append(eng._pcm_queue.get_nowait())
except queue.Empty:
break
# Assert — only the newest `maxsize` blocks survived, oldest dropped
assert len(drained) == maxsize
markers = [int(b[0]) for b in drained]
assert markers == list(range(total - maxsize, total))
def test_initialize_raises_when_not_configured(reset_engine):
# Arrange — fixture left the engine inactive
stream = eng.AndroidAudioEngine.create_stream(0, True, {})
# Act / Assert
with pytest.raises(RuntimeError):
stream.initialize()
def test_read_chunk_returns_none_when_empty(reset_engine):
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
stream = eng.AndroidAudioEngine.create_stream(0, True, {})
stream.initialize()
assert stream.read_chunk() is None
# ---------------------------------------------------------------------------
# Availability / enumeration (platform-gated)
# ---------------------------------------------------------------------------
def test_is_available_requires_android_and_active(monkeypatch, reset_engine):
# Not configured yet → inactive → unavailable even on Android.
monkeypatch.setattr(f"{ENGINE_MOD}.is_android", lambda: True)
assert eng.AndroidAudioEngine.is_available() is False
# Configured → active + Android → available.
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
assert eng.AndroidAudioEngine.is_available() is True
# Active but not on Android → unavailable.
monkeypatch.setattr(f"{ENGINE_MOD}.is_android", lambda: False)
assert eng.AndroidAudioEngine.is_available() is False
def test_enumerate_devices(on_android):
# Inactive → no devices.
assert eng.AndroidAudioEngine.enumerate_devices() == []
# Active → exactly one loopback device.
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
devices = eng.AndroidAudioEngine.enumerate_devices()
assert len(devices) == 1
dev = devices[0]
assert dev.is_loopback is True
assert dev.is_input is True
assert "Android playback" in dev.name
assert dev.channels == CHANNELS
# ---------------------------------------------------------------------------
# Regression guard — the analyzer must never crash on a malformed block
# (over-length or non-frame-divisible). This is the on-device failure the
# plan review surfaced; the desktop suite must catch it.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"raw_floats",
[
(CHUNK + 100) * CHANNELS, # over-length (more frames than chunk_size)
CHUNK * CHANNELS + 1, # not a whole number of stereo frames
3, # tiny + odd
CHUNK * CHANNELS, # exact (control)
],
)
def test_pushed_block_never_crashes_analyzer(reset_engine, raw_floats):
# Arrange
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
pcm = np.random.default_rng(0).standard_normal(raw_floats).astype(np.float32)
analyzer = AudioAnalyzer(sample_rate=SAMPLE_RATE, chunk_size=CHUNK)
stream = eng.AndroidAudioEngine.create_stream(0, True, {})
stream.initialize()
# Act
eng.push_samples(pcm.tobytes())
chunk = stream.read_chunk()
# Assert — chunk is a safe shape and analyze() does not raise.
assert chunk is not None
assert len(chunk) % CHANNELS == 0
assert len(chunk) <= CHUNK * CHANNELS
analysis = analyzer.analyze(chunk, CHANNELS)
assert isinstance(analysis, AudioAnalysis)
# ---------------------------------------------------------------------------
# Registry integration
# ---------------------------------------------------------------------------
def test_best_available_engine_is_android_when_active(on_android):
# Arrange
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
# Act
best = AudioEngineRegistry.get_best_available_engine()
# Assert — priority 100 beats every desktop engine; demo only wins in demo mode.
assert best == "android_playback"
def test_stream_via_registry_yields_pushed_chunk(on_android):
# Arrange
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
samples = np.linspace(-1.0, 1.0, CHUNK * CHANNELS, dtype=np.float32)
# Act
stream = AudioEngineRegistry.create_stream("android_playback", 0, True, {})
stream.initialize()
eng.push_samples(samples.tobytes())
got = stream.read_chunk()
# Assert
assert stream.channels == CHANNELS
assert stream.sample_rate == SAMPLE_RATE
assert stream.chunk_size == CHUNK
np.testing.assert_array_equal(got, samples)
def test_device_surfaces_through_capture_manager(on_android):
# Arrange
eng.configure(SAMPLE_RATE, CHANNELS, CHUNK)
# Act
devices = AudioCaptureManager.enumerate_devices()
# Assert — the Android device is enumerated and tagged with its engine.
android = [d for d in devices if d["engine_type"] == "android_playback"]
assert len(android) == 1
assert android[0]["name"] == "Android playback (system audio)"
assert android[0]["is_loopback"] is True
+50
View File
@@ -0,0 +1,50 @@
"""Demo-seed regression tests (value sources, incl. the template combinator)."""
from ledgrab.core.demo_seed import seed_demo_data
from ledgrab.storage.database import Database
from ledgrab.storage.value_source import StaticValueSource, TemplateValueSource
from ledgrab.storage.value_source_store import ValueSourceStore
def _seed(tmp_path):
db = Database(tmp_path / "demo.db")
seed_demo_data(db)
return db
def test_demo_seeds_template_value_source(tmp_path):
db = _seed(tmp_path)
try:
store = ValueSourceStore(db)
by_id = {s.id: s for s in store.get_all_sources()}
base = by_id["vs_demo0001"]
boost = by_id["vs_demo0002"]
assert isinstance(base, StaticValueSource)
assert isinstance(boost, TemplateValueSource)
assert boost.template == "clamp(level * 1.5)"
assert boost.inputs == [{"name": "level", "value_source_id": "vs_demo0001"}]
# The reference graph is intact and consistent.
assert store.get_transitive_dependencies("vs_demo0002") == {"vs_demo0001"}
assert store.find_referencing_sources("vs_demo0001") == [boost.name]
finally:
db.close()
def test_demo_template_evaluates_through_manager(tmp_path):
"""The seeded template must actually evaluate over its seeded input."""
from ledgrab.core.processing.value_stream import ValueStreamManager
db = _seed(tmp_path)
try:
store = ValueSourceStore(db)
vsm = ValueStreamManager(value_source_store=store)
stream = vsm.acquire("vs_demo0002")
try:
# base level 0.5 -> clamp(0.5 * 1.5) = 0.75
assert abs(stream.get_value() - 0.75) < 1e-6
finally:
vsm.release("vs_demo0002")
finally:
db.close()
@@ -0,0 +1,231 @@
"""Tests for the template value source: model, factory, cycle/depth, refs."""
from datetime import datetime, timezone
import pytest
from ledgrab.storage.value_source import TemplateValueSource
class TestModelRoundTrip:
def _make(self, **over):
now = datetime.now(timezone.utc)
defaults = dict(
id="vs_t1",
name="Combo",
source_type="template",
created_at=now,
updated_at=now,
template="min(a * 2, 1)",
inputs=[{"name": "a", "value_source_id": "vs_a"}],
default_value=0.2,
eval_interval=1.5,
)
defaults.update(over)
return TemplateValueSource(**defaults)
def test_to_from_dict_idempotent(self):
src = self._make()
rebuilt = TemplateValueSource.from_dict(src.to_dict())
assert rebuilt.template == src.template
assert rebuilt.inputs == src.inputs
assert rebuilt.default_value == src.default_value
assert rebuilt.eval_interval == src.eval_interval
assert rebuilt.to_dict()["return_type"] == "float"
def test_old_row_deserializes_with_defaults(self):
"""A row written before template fields existed must load safely."""
now = datetime.now(timezone.utc).isoformat()
src = TemplateValueSource.from_dict(
{
"id": "vs_old",
"name": "Old",
"source_type": "template",
"created_at": now,
"updated_at": now,
}
)
assert src.template == ""
assert src.inputs == []
assert src.default_value == 0.0
assert src.eval_interval is None
def test_dirty_scalars_coerce_to_defaults(self):
"""Non-numeric stored scalars must not drop the whole row on load."""
src = TemplateValueSource.from_dict(
{
"id": "x",
"name": "n",
"source_type": "template",
"template": "a",
"default_value": "not-a-number",
"eval_interval": "bad",
}
)
assert src.default_value == 0.0
assert src.eval_interval is None
def test_inputs_normalized_from_dirty_data(self):
src = TemplateValueSource.from_dict(
{
"id": "x",
"name": "n",
"source_type": "template",
"inputs": [{"name": "a", "value_source_id": "vs_a"}, "junk", {"bad": 1}],
}
)
# non-dict entries dropped; dict entries coerced to {name, value_source_id}
assert src.inputs == [
{"name": "a", "value_source_id": "vs_a"},
{"name": "", "value_source_id": ""},
]
class TestFactoryCreate:
def test_create_valid(self, value_source_store):
src = value_source_store.create_source(
"Combo",
"template",
template="min(a * 2, 1)",
inputs=[{"name": "a", "value_source_id": ""}],
default_value=0.3,
)
assert isinstance(src, TemplateValueSource)
assert src.id.startswith("vs_")
assert src.default_value == 0.3
def test_empty_template_rejected(self, value_source_store):
with pytest.raises(ValueError):
value_source_store.create_source("X", "template", template=" ", inputs=[])
def test_compile_error_rejected(self, value_source_store):
with pytest.raises(ValueError):
value_source_store.create_source("X", "template", template="a +", inputs=[])
def test_cost_bomb_rejected(self, value_source_store):
with pytest.raises(ValueError):
value_source_store.create_source("X", "template", template="10 ** 10", inputs=[])
def test_reserved_input_name_rejected(self, value_source_store):
with pytest.raises(ValueError):
value_source_store.create_source(
"X",
"template",
template="min(0, 1)",
inputs=[{"name": "min", "value_source_id": "vs_a"}],
)
def test_duplicate_input_name_rejected(self, value_source_store):
with pytest.raises(ValueError):
value_source_store.create_source(
"X",
"template",
template="a",
inputs=[
{"name": "a", "value_source_id": "vs_a"},
{"name": "a", "value_source_id": "vs_b"},
],
)
def test_default_value_out_of_range_rejected(self, value_source_store):
with pytest.raises(ValueError):
value_source_store.create_source(
"X",
"template",
template="a",
inputs=[{"name": "a", "value_source_id": ""}],
default_value=5.0,
)
def test_unbound_variable_rejected(self, value_source_store):
# 'ha_enti' is referenced but only 'ha_entity' is bound (typo) → reject.
with pytest.raises(ValueError):
value_source_store.create_source(
"X",
"template",
template="ha_enti",
inputs=[{"name": "ha_entity", "value_source_id": ""}],
)
class TestFactoryUpdate:
def test_partial_update_template_only(self, value_source_store):
src = value_source_store.create_source(
"X",
"template",
template="a",
inputs=[{"name": "a", "value_source_id": ""}],
default_value=0.1,
)
updated = value_source_store.update_source(src.id, template="clamp(a * 3)")
assert updated.template == "clamp(a * 3)"
assert updated.default_value == 0.1 # unchanged
def test_update_invalid_template_rejected(self, value_source_store):
src = value_source_store.create_source("X", "template", template="clamp(0.5)", inputs=[])
with pytest.raises(ValueError):
value_source_store.update_source(src.id, template="a |")
class TestCycleAndDepth:
def test_self_reference_rejected(self, value_source_store):
t = value_source_store.create_source("T", "template", template="clamp(0.5)", inputs=[])
with pytest.raises(ValueError):
value_source_store.update_source(t.id, inputs=[{"name": "x", "value_source_id": t.id}])
def test_circular_reference_rejected(self, value_source_store):
t1 = value_source_store.create_source("T1", "template", template="clamp(0.5)", inputs=[])
t2 = value_source_store.create_source(
"T2",
"template",
template="x",
inputs=[{"name": "x", "value_source_id": t1.id}],
)
# t1 -> t2 -> t1 would be a cycle
with pytest.raises(ValueError):
value_source_store.update_source(
t1.id, inputs=[{"name": "x", "value_source_id": t2.id}]
)
def test_deep_chain_rejected(self, value_source_store):
prev = value_source_store.create_source("L0", "template", template="clamp(0.5)", inputs=[])
created = 1
with pytest.raises(ValueError):
for i in range(1, 12):
node = value_source_store.create_source(
f"L{i}",
"template",
template="x",
inputs=[{"name": "x", "value_source_id": prev.id}],
)
prev = node
created += 1
# Should have rejected before building an unbounded chain.
assert created <= 8
def test_get_transitive_dependencies(self, value_source_store):
leaf = value_source_store.create_source(
"leaf", "template", template="clamp(0.5)", inputs=[]
)
mid = value_source_store.create_source(
"mid", "template", template="x", inputs=[{"name": "x", "value_source_id": leaf.id}]
)
top = value_source_store.create_source(
"top", "template", template="x", inputs=[{"name": "x", "value_source_id": mid.id}]
)
deps = value_source_store.get_transitive_dependencies(top.id)
assert deps == {mid.id, leaf.id}
class TestReferencingSources:
def test_find_referencing_sources(self, value_source_store):
base = value_source_store.create_source("Base", "static", value=0.5)
tmpl = value_source_store.create_source(
"Uses",
"template",
template="b",
inputs=[{"name": "b", "value_source_id": base.id}],
)
refs = value_source_store.find_referencing_sources(base.id)
assert tmpl.name in refs
assert value_source_store.find_referencing_sources(tmpl.id) == []
@@ -7,7 +7,10 @@ from ledgrab.storage.value_source import (
AnimatedValueSource,
AudioValueSource,
DaylightValueSource,
HAEntityValueSource,
HTTPValueSource,
StaticValueSource,
SystemMetricsValueSource,
ValueSource,
)
from ledgrab.storage.value_source_store import ValueSourceStore
@@ -238,6 +241,50 @@ class TestValueSourceStoreCRUD:
assert updated.speed == 30.0
# ---------------------------------------------------------------------------
# normalize flag — full store create/update path (factory builder + applier)
# ---------------------------------------------------------------------------
class TestValueSourceNormalizeStoreRoundTrip:
"""The ``normalize`` flag must survive the real write path —
create_source -> CREATE_BUILDERS and update_source -> UPDATE_APPLIERS
not just dataclass from_dict/to_dict. A dropped builder/applier line would
silently revert/ignore the flag while every dataclass-level test stayed
green, so these exercise the factory layer end-to-end (game_event is
intentionally excluded it has no store/CRUD path).
"""
_CASES = [
("ha_entity", {"ha_source_id": "ha1", "entity_id": "sensor.temp"}, HAEntityValueSource),
("http", {"http_endpoint_id": "ep1"}, HTTPValueSource),
("system_metrics", {"metric": "cpu_load"}, SystemMetricsValueSource),
]
@pytest.mark.parametrize("source_type,kwargs,cls", _CASES)
def test_create_normalize_false_persists(self, store, source_type, kwargs, cls):
s = store.create_source(
name=f"{source_type}_n", source_type=source_type, normalize=False, **kwargs
)
assert isinstance(s, cls)
assert s.normalize is False
# Re-read exercises the SQLite blob persistence round-trip as well.
assert store.get_source(s.id).normalize is False
@pytest.mark.parametrize("source_type,kwargs,cls", _CASES)
def test_create_normalize_defaults_true(self, store, source_type, kwargs, cls):
s = store.create_source(name=f"{source_type}_d", source_type=source_type, **kwargs)
assert s.normalize is True
@pytest.mark.parametrize("source_type,kwargs,cls", _CASES)
def test_update_normalize_flip(self, store, source_type, kwargs, cls):
s = store.create_source(
name=f"{source_type}_u", source_type=source_type, normalize=False, **kwargs
)
assert store.update_source(s.id, normalize=True).normalize is True
assert store.update_source(s.id, normalize=False).normalize is False
# ---------------------------------------------------------------------------
# Name uniqueness
# ---------------------------------------------------------------------------
+78
View File
@@ -0,0 +1,78 @@
"""Tests for the request access-log middleware (token-label attribution).
The middleware emits one structured ``http_request`` line per request, tagged
with the friendly label of the API token used (from ``auth.api_keys``) so
traffic can be attributed to a specific client. These tests capture the
``main.logger`` calls to assert the label is recorded and that no-auth
endpoints fall back to ``"unauthenticated"``.
"""
import pytest
@pytest.fixture
def client_and_logs(authenticated_client, monkeypatch):
"""authenticated_client plus a captured list of (event, kwargs) log calls."""
import ledgrab.main as main_mod
calls: list[tuple[str, dict]] = []
class _FakeLogger:
def info(self, event, **kw):
calls.append((event, kw))
# main.logger is shared; keep other levels as harmless no-ops.
def debug(self, *a, **k):
pass
def warning(self, *a, **k):
pass
def error(self, *a, **k):
pass
monkeypatch.setattr(main_mod, "logger", _FakeLogger())
return authenticated_client, calls
def _http_request_logs(calls):
return [kw for event, kw in calls if event == "http_request"]
def test_access_log_records_token_label_for_authed_request(client_and_logs):
client, calls = client_and_logs
# /openapi.json requires auth but needs no initialized store, so it returns
# 200 even without app lifespan — and exercises the auth → label path.
resp = client.get("/openapi.json")
assert resp.status_code == 200
logs = _http_request_logs(calls)
assert logs, "no http_request access log emitted"
entry = logs[-1]
assert entry["method"] == "GET"
assert entry["path"] == "/openapi.json"
assert entry["status"] == 200
assert entry["token"] == "test" # label of the configured test API key
assert "duration_ms" in entry
def test_access_log_marks_unauthenticated_for_no_auth_endpoint(client_and_logs):
client, calls = client_and_logs
# /health has no auth dependency, so no label is set on request.state.
resp = client.get("/health")
assert resp.status_code == 200
logs = _http_request_logs(calls)
assert logs
assert logs[-1]["token"] == "unauthenticated"
def test_access_log_never_contains_the_token_secret(client_and_logs):
client, calls = client_and_logs
client.get("/openapi.json")
for _event, kw in calls:
assert "test-api-key-12345" not in repr(kw), "token secret leaked into logs"
+33 -1
View File
@@ -12,7 +12,12 @@ import signal
import threading
from types import SimpleNamespace
from ledgrab.__main__ import _install_signal_handlers, _request_shutdown
from ledgrab.__main__ import (
_build_server,
_install_signal_handlers,
_request_shutdown,
)
from ledgrab.shutdown_state import GRACEFUL_SHUTDOWN_TIMEOUT
def test_request_shutdown_sets_should_exit() -> None:
@@ -21,6 +26,33 @@ def test_request_shutdown_sets_should_exit() -> None:
assert server.should_exit is True
def test_build_server_bounds_graceful_shutdown() -> None:
"""uvicorn defaults ``timeout_graceful_shutdown`` to ``None`` (wait
forever). A lingering events WebSocket then blocks ``Server.shutdown()``
from ever reaching the lifespan shutdown, so LED targets are never
stopped and the process can't exit. The bound is what guarantees the
lifespan shutdown runs assert we never regress to the unbounded default.
"""
fake_config = SimpleNamespace(
server=SimpleNamespace(host="127.0.0.1", port=8080, log_level="INFO")
)
server = _build_server(fake_config) # type: ignore[arg-type]
assert server.config.timeout_graceful_shutdown == GRACEFUL_SHUTDOWN_TIMEOUT
assert server.config.timeout_graceful_shutdown is not None
assert server.config.timeout_graceful_shutdown > 0
def test_graceful_shutdown_timeout_fits_os_budget() -> None:
"""The graceful wait runs BEFORE the lifespan's own ~16 s shutdown budget,
and OS shutdown gives the whole process only ~20 s before a force-kill.
Keep the bound small so target restore + DB checkpoint still fit.
"""
assert isinstance(GRACEFUL_SHUTDOWN_TIMEOUT, int)
assert 0 < GRACEFUL_SHUTDOWN_TIMEOUT <= 5
def test_install_signal_handlers_installs_for_known_signals() -> None:
"""Tray path runs uvicorn on a background thread, so our handlers must
actually survive verify each catchable signal is replaced.
+136
View File
@@ -0,0 +1,136 @@
"""Tests for the hardened sandboxed-Jinja expression engine."""
import pytest
from ledgrab.utils.template_expr import (
GLOBALS,
RESERVED_NAMES,
TemplateValidationError,
clamp,
compile_template,
extract_variables,
finalize_result,
validate_input_name,
validate_template_expression,
)
class TestCompileAndEval:
def test_basic_eval(self):
assert compile_template("min(a * 2, 1)")(a=0.3, raw={}) == pytest.approx(0.6)
def test_clamp_global(self):
assert compile_template("clamp((t - 18) / 10)")(t=22.5, raw={}) == pytest.approx(0.45)
def test_raw_subscript(self):
assert compile_template("raw['t'] / 100")(raw={"t": 42.0}) == pytest.approx(0.42)
def test_ternary_and_comparison(self):
expr = compile_template("a if a > 0.5 else b")
assert expr(a=0.8, b=0.1, raw={}) == pytest.approx(0.8)
assert expr(a=0.2, b=0.1, raw={}) == pytest.approx(0.1)
def test_all_globals_callable(self):
for tpl in ("min(a, b)", "max(a, b)", "abs(a - b)", "round(a, 1)", "clamp(a)"):
compile_template(tpl)(a=0.4, b=0.6, raw={})
class TestRejections:
@pytest.mark.parametrize(
"tpl",
[
"",
" ",
"a +", # syntax error
"10 ** 3", # power bomb
"'a' * 1000", # string repetition
"a | pprint", # filter
"a is defined", # test
"a.__class__", # attribute access
"raw['s'].format(1)", # str gadget via attribute
"dict(x=1)", # non-global call
"namespace(x=1)",
"range(3)",
"cycler(1, 2)",
"[0] * 1000000", # list-literal repetition (memory bomb)
"(1,) * 1000000", # tuple-literal repetition (memory bomb)
"[1, 2, 3]", # bare list literal
"{1: 2}", # dict literal
],
)
def test_rejected(self, tpl):
with pytest.raises(TemplateValidationError):
validate_template_expression(tpl)
@pytest.mark.parametrize(
"tpl",
[
"min(a * 2, 1)",
"(a + b) / 2",
"clamp((t - 18) / 10, 0, 1)",
"raw['x'] / 100",
"a if a > b else b",
"abs(a - b)",
],
)
def test_accepted(self, tpl):
validate_template_expression(tpl) # must not raise
class TestFinalizeResult:
def test_nan_returns_default(self):
assert finalize_result(float("nan"), 0.25) == 0.25
def test_inf_returns_default(self):
assert finalize_result(float("inf"), 0.25) == 0.25
assert finalize_result(float("-inf"), 0.25) == 0.25
def test_non_numeric_returns_default(self):
assert finalize_result("nope", 0.25) == 0.25
assert finalize_result(None, 0.25) == 0.25
def test_overflow_returns_default(self):
# float() of a multi-hundred-digit int (chained big-int multiply) raises
# OverflowError, not ValueError — must still fall back, not propagate.
assert finalize_result(10**400, 0.25) == 0.25
def test_clamps_to_unit(self):
assert finalize_result(5.0, 0.0) == 1.0
assert finalize_result(-1.0, 0.0) == 0.0
assert finalize_result(0.5, 0.0) == pytest.approx(0.5)
def test_clamp_helper(self):
assert clamp(2.0) == 1.0
assert clamp(-2.0) == 0.0
assert clamp(5.0, 0.0, 10.0) == 5.0
class TestInputNames:
@pytest.mark.parametrize("name", ["audio", "cpu_load", "_x", "Temp2"])
def test_valid(self, name):
validate_input_name(name)
@pytest.mark.parametrize("name", ["", "1bad", "has space", "a-b", "a.b"])
def test_invalid_identifier(self, name):
with pytest.raises(TemplateValidationError):
validate_input_name(name)
@pytest.mark.parametrize("name", sorted(RESERVED_NAMES))
def test_reserved(self, name):
with pytest.raises(TemplateValidationError):
validate_input_name(name)
def test_globals_are_reserved(self):
assert set(GLOBALS).issubset(RESERVED_NAMES)
assert "raw" in RESERVED_NAMES
class TestExtractVariables:
def test_excludes_globals_and_raw(self):
assert extract_variables("min(a, raw['x']) + b") == ["a", "b"]
def test_empty_for_uncompilable(self):
assert extract_variables("a +") == []
def test_constant_expression(self):
assert extract_variables("clamp(0.5)") == []