ledgrab/TODO.md

LedGrab TODO

WebUI Redesign — "Lumenworks" Studio-Console Aesthetic

Full-app UI/UX refresh. Design direction committed to by user 2026-04-24. Mockup lives at server/docs/ui-redesign-mockup.html. Phases are independent and CSS-only where possible — backend untouched.

Phase 1 — Design tokens & font embed

• Embed variable fonts (server/src/ledgrab/static/fonts/): Manrope (latin + latin-ext + cyrillic + cyrillic-ext), JetBrains Mono (same 4 subsets), Big Shoulders Display (latin + latin-ext). Total +201 KB gzipped, served via unicode-range so only latin paints on first load.
• fonts.css — declare @font-face entries for all new families with proper unicode-range subsetting; keep DM Sans + Orbitron registered for legacy-token callers during migration.
• base.css — add additive Lumenworks tokens: --font-display/--font-brand/--font-body, --lux-r-*, --lux-hairline, --lux-rule. Both [data-theme="dark"] and [data-theme="light"] define --lux-bg-0…3, --lux-line/-bold, --lux-ink/-dim/-mute/-faint, --ch-signal/-cyan/-magenta/-amber/-coral/-violet, --lux-signal-glow, --lux-shadow-rack. Existing tokens untouched — no visual regression.

Phase 2 — Shell (header → transport bar + channel-strip sidebar)

• index.html — .tab-bar moved out of <header> into a new <aside class="sidebar">; wrapped content in .app-body 2-col grid (sidebar | main). .transport-center section added between .header-title and .header-toolbar with a placeholder .transport-status chip ("Ready" → "Armed · N live" wired in Phase 3). All tab-button IDs, data-tab attributes, and onclick="switchTab(…)" handlers preserved.
• layout.css — <header> rebuilt as the transport bar: 3-column grid (brand | center | toolbar), 60 px fixed height, sticky, gradient bottom rule with channel-color wash. .header-title::before/::after render the glowing LED brand mark; #server-status repositioned as the LED core pip. #server-version restyled as a mono-type console badge.
• sidebar.css (new) — vertical channel-strip navigation. Active tab gets a glowing left stripe + radial tint. .sidebar-foot contains a .cpu-meter plate with two live bars (Load, FPS) ready to be JS-bound in Phase 3. Collapses to a 56 px icon rail at ≤1100 px; hides entirely at ≤600 px via display: contents so .tab-bar falls through to mobile.css's fixed-bottom strip unchanged.
• all.css — new sidebar import after layout.
• base.css — body font-family switched to var(--font-body) which resolves to Manrope (with DM Sans + system fallbacks). Added font-feature-settings for stylistic set + alternate 1.
• Locale additions: sidebar.workspaces, sidebar.load, sidebar.fps, transport.status.ready, transport.status.armed in en/ru/zh.
• Tutorial + auth selectors (header .header-title, #tab-btn-*, .tab-bar querySelector, a.header-link[href="/docs"], onclick markers on theme/settings/search) all survive the move.
• JS: bind .cpu-meter + .transport-status chip to existing performance WebSocket / poller. Done as part of Phase 3.
• Tablet-range visual polish pass once other phases render (some tabs currently have their own internal sticky headers that may overlap the transport bar on narrow viewports).

Phase 3 — Dashboard hero + module redesign

• cards.css — .card gets rack-module treatment: channel stripe on left edge (color-coded via data-card-type + .ch-* utility classes), ::after corner bracket in top-right, mono-typed metric labels planned for Phase 4. Running cards glow the stripe brighter + emit a signalFlow keyframe strip along the bottom edge.
• Removed the @property --border-angle rotating conic-gradient border (retired the WebKit mask workaround + light-theme variant + fallback for @supports not (mask-composite: exclude)). Replaced with the signal-flow strip — one animated linear-gradient on a 2 px line, no GPU layer compositing per card.
• dashboard.css — .dashboard-target rows pick up the same channel stripe + signal-flow treatment. Section headers now use mono caps with a channel-green underline accent. Metric values use mono with tabular numerics; labels use silkscreened micro-caps.
• Skeleton-card rewritten: left hairline + corner bracket so it reads as "loading module" instead of a generic flashing block. skeletonShimmer gradient replaces the old opacity-pulse on --text-color.
• _updateSidebarMeter binds CPU% (Load) and app-CPU share (FPS) to the sidebar meter plate on every perf poll.
• _updateTransportStatus updates the transport chip ("Ready" → "Armed · N live") whenever the dashboard's running-target set is recomputed.
• .hero 4-cell readout row (Active Patches / Throughput / CPU / Latency + inline sparklines) — CSS tokens + layout are ready; HTML render deferred until the dashboard JS is refactored to emit it (Phase 3b, non-blocking).

Phase 4 — Other tabs adopt module language

• tree-nav.css — trigger pill gets a channel stripe on its left edge (glows + widens when open). Trigger title uses mono-uppercase with wide letter-spacing. Dropdown panel has a gradient channel-accent rule across its top edge. Group headers use silkscreened micro-caps with a small square marker instead of the old bold-uppercase. Active leaf has a pulsing LED pip on the left and a channel tint behind it. Count badges switched to mono tabular-nums in 2-px-radius pills.
• .subtab-section-header — channel-green underline accent + mono micro-caps. Consistent with the dashboard-section pattern so the whole app shares one section-header language.
• .stream-tab-btn sub-tabs — mono uppercase with wide tracking, active tab shows channel-green underline + glowing count badge.
• .perf-chart-card — channel stripe on the left (replaces old border-top accent). Per-metric accents swapped to channel palette (--ch-coral for CPU, --ch-violet for RAM, --ch-signal for GPU, --ch-amber for temp). Corner bracket added. Metric values pick up tabular-nums + a soft glow.
• cards.css — channel-color mapping extended to attributes the JS already emits (data-target-id → green, data-stream-id → cyan, data-audio-source-id → magenta, data-automation-id / data-scene-id → violet). No JS changes required; cards pick up their correct stripe automatically on the Targets/Sources/Automations tabs.
• Graph editor — toolbar gets a gradient background + hairline + rack shadow + backdrop blur. Canvas and nodes untouched.

Phase 5 — Modal restyle

• modal.css — backdrop gains a radial dim + 6 px blur for stronger separation. .modal-content gets a gradient background + hairline + deep rack shadow. Channel-accent rule across the top edge driven by --modal-ch (per-modal override). Corner bracket bottom-right on desktop. .modal-header gains a vertical channel-color stripe to the left of the title; .modal-footer picks up a hairline divider.
• Per-modal channel mapping by modal ID: - Target editors → green - Input/Source editors → cyan - Audio editors → magenta - Automation / Scene / Game editors → violet - Settings / API key / Setup / Notifications → amber - Confirm dialog → coral
• components.css — inputs use hairline borders, tabular-nums mono for input[type="number"], channel-green focus ring + glow. Buttons use mono-uppercase type, signal-glow on primary, coral-glow on danger. <select> audit deferred (project already enforces via CLAUDE.md rule + IconSelect/EntitySelect wrappers).

Phase 6 — Mobile dedicated shell

• mobile.css (existing file, not forked) — fixed-bottom .tab-bar promoted to full Lumenworks treatment: gradient background + hairline divider at top + channel-accent rule matching the transport-bar bottom. Active tab gets an LED pip above the icon and a channel-tint background. Tab labels + badges use mono uppercase to match the rest of the app. Phone (≤600 px): modal corner-bracket hidden (fullscreen modals), modal-header stripe slimmed to 18 px.
• Phase 2's layout.css already strips the transport-center on phones and collapses the sidebar via display: contents, so the mobile shell automatically routes the tab-bar to the bottom without a separate JS hook.
• [WONTDO] Fork into mobile-shell.css — keeping changes in mobile.css since the cascade was already organized by viewport. A rename adds churn without improving maintainability.

Phase 7 — Microcopy + retire legacy

• Locale rename: targets.title + dashboard.section.targets → "Channels" (en) / "Каналы" (ru) / "通道" (zh); streams.title → "Inputs" / "Входы" / "输入". Automations kept as-is (Automations + Scenes is a meaningful distinction; "Patches" would conflate them). Internal tab keys (dashboard / automations / targets / streams / integrations / graph) unchanged so no JS or localStorage migration needed.
• Ambient WebGL background — default is already off; kept the toggle button and localStorage preference so users who want the shader can turn it on. No entry-point change needed: data-bg-anim is initialized from localStorage with off fallback.
• [DEFERRED] Delete DM Sans + legacy color tokens — would cascade through every file that reads --primary-color / --text-color etc. Safer as a separate cleanup PR after the new design has soaked.
• [WONTDO] Delete mobile.css — Phase 6 kept the filename.

BLE LED Controller Support (SP110E / Triones / Zengge / Govee)

Add support for Bluetooth Low Energy LED controllers driven by mobile apps like "LED Hue", HappyLighting, iLightsIn. Whole-strip ambient-color output only — these protocols don't support per-pixel streaming.

• Add bleak>=0.22 as optional extra [ble] in server/pyproject.toml (desktop-only, NOT in android build.gradle.kts)
• core/devices/ble_transport.py — bleak wrapper: scan, connect, write-with/without-response
• core/devices/ble_protocols/ package
  • __init__.py — BLEProtocol dataclass + registry (family → encoder)
  • sp110e.py — SP110E / SP108E (service FFE0, char FFE1, RR GG BB 00 1E static-color frame)
  • triones.py — Triones / HappyLighting / LEDnet (service FFE5, char FFE9, 7E 07 05 03 RR GG BB 10 EF)
  • zengge.py — Zengge / iLightsIn (service FFE0, framing 56 RR GG BB 00 F0 AA)
  • govee.py — Govee unencrypted framed protocol (AES keyed variants — marked experimental)
• core/devices/ble_client.py — unified BLEClient(LEDClient) — picks protocol by ble_family, averages strip → one color, drops duplicate frames, rate-limits to BLE connection interval
• core/devices/ble_provider.py — BLEDeviceProvider + discovery via BleakScanner
• Register in core/devices/led_client.py::_register_builtin_providers (guarded try/except ImportError)
• Storage: ble_family, ble_govee_key fields threaded through Device.__init__/to_dict/from_dict/_UPDATABLE_FIELDS/create_device
• Schemas: BLE fields on DeviceCreate, DeviceUpdate, DeviceResponse
• Routes: BLE fields propagated through create/update in api/routes/devices.py + _device_to_response
• ProcessorManager: ble_family/ble_govee_key added to _DEVICE_FIELD_DEFAULTS and DeviceInfo; passed through wled_target_processor.py and group_client.py to create_led_client
• Tests: 21 protocol encoder unit tests + 16 BLEClient fake-transport tests — all passing, 814 total tests still green
• Frontend: BLE option in the device type picker with a bluetooth Lucide icon; add-device modal shows a 4-option IconSelect for protocol family (SP110E / Triones / Zengge / Govee) with a Govee-only AES key field that auto-hides for the other three families; URL label/placeholder/hint adapt to ble://<address> pattern; submit payload carries ble_family (+ optional ble_govee_key); clone flow pre-fills family and key; modal dirty-check snapshots the new fields; network scan button now also discovers BLE peripherals via the existing /api/v1/devices/discover?device_type=ble endpoint
• Frontend: isBleDevice helper in core/api.ts; ICON_BLUETOOTH + ICON_LIGHTBULB constants in core/icons.ts; bluetooth path in core/icon-paths.ts; i18n keys in en.json / ru.json / zh.json; TypeScript compiles; esbuild bundle rebuilt
• Android BLE via Kotlin bridge — BleBridge.kt singleton (scan/connect/write/disconnect); android_ble_transport.py Python wrapper; make_transport() factory in ble_transport.py auto-selects backend; BleBridge.init() called from LedGrabApp.onCreate; BLE permissions in AndroidManifest.xml
• Govee per-model AES key — _encrypt_govee_frame() in ble_client.py uses AES-128-ECB from cryptography; key validated on BLEClient construction; applied to both send_pixels and set_power; 8 new AES unit tests

Android — Restore Multi-ABI Wheels

During emulator testing, we switched the build to x86 only (see android/app/build.gradle.kts abiFilters) to avoid having to keep the arm64-v8a / x86_64 pydantic-core wheels current. Before shipping, restore all three ABIs:

• Rebuild pydantic-core wheels for all three ABIs with the current SOABI + libpython linking settings (android/build-scripts/build-pydantic-core.sh — now supports arm64, x86_64, x86 args; defaults to all three).
• Verify wheels: all three now list libpython3.11.so in NEEDED (llvm-readelf -d), automated in the build script.
• Restored abiFilters += listOf("arm64-v8a", "x86_64", "x86") in build.gradle.kts. Multi-ABI debug APK builds cleanly (~99 MB).
• Re-test on real ARM64 Android TV hardware (still pending — only emulator-verified build).

Build cache + scripts live in android/build-scripts/ and android/.build-cache/ (junction host + sysconfigdata for each ABI).

Android CI Pipeline

Build the Android APK automatically on push/tag.

• Generate Gradle wrapper (gradlew) and commit it
• Create CI workflow (.gitea/workflows/build-android.yml)
  • JDK 17 + Android SDK + NDK setup
  • Python 3.11 for Chaquopy build
  • Recreate the directory junction via ln -s on Linux CI
  • ./gradlew assembleDebug on master push, assembleRelease on v* tags (if signing secrets set)
  • Uploads APK as CI artifact; attaches to Gitea release on tag push
• Commit pre-built pydantic-core wheels to android/wheels/ (arm64, x86, x86_64)
• APK signing for release builds — conditional signing config reads keystore from env vars (ANDROID_KEYSTORE_PATH/_PASSWORD/_ALIAS/_KEY_PASSWORD), falls back to debug signing locally
• Provision a real keystore and add the four CI secrets:
  • ANDROID_KEYSTORE_BASE64 (base64-encoded .jks)
  • ANDROID_KEYSTORE_PASSWORD
  • ANDROID_KEY_ALIAS
  • ANDROID_KEY_PASSWORD
• Add LedGrab-{tag}-android-release.apk row to the release description table in .gitea/workflows/release.yml → create-release job
• Verify the CI workflow passes end-to-end with the now-restored multi-ABI build (larger APK, longer Android build step)

Android Root Capture (No Permission Dialog, No System Indicator)

MediaProjection shows a mandatory system overlay/indicator while capturing — unavoidable on stock Android. Many cheap Android TV boxes ship pre-rooted, so an alternative root-only path gives much better UX.

• Root detection — Root.kt checks common su binary paths and, on demand, runs su -c id to actually prove UID 0. First call triggers Magisk's grant dialog; grant is cached per session. Exposed to Python via Chaquopy.
• RootScreenrecord.kt — spawns su -c screenrecord --output-format=h264 --size=WxH -, feeds the H.264 stdout through a MediaCodec decoder whose output Surface is wired into an ImageReader (RGBA_8888, row-stride-aware). Decoded frames reach the Python pipeline via PythonBridge.pushRootFrame.
• Python-side RootScreenrecordEngine (core/capture_engines/root_screenrecord_engine.py) mirrors MediaProjectionEngine with ENGINE_PRIORITY=110 (> MediaProjection's 100) so the factory picks it automatically when available.
• MainActivity tries Root.requestGrant() before launching the MediaProjection consent flow — on rooted devices the consent dialog is skipped entirely. CaptureService has a createRootIntent() entry point that bypasses the MediaProjection path.
• Fallback: if Root.requestGrant() returns false (no root, user denied, or su timeout) the existing MediaProjection flow runs unchanged.
• Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) grant dialog appears once, (2) frames actually flow through MediaCodec without the Android 14 capture indicator showing, (3) stop/start cycle terminates the su process cleanly.
• [WONTDO] SurfaceControl.screenshot() via reflection — renamed/moved across API 28/29/30/33, hidden-API blocklist varies by release, even rooted apps hit it; days of maintenance for a marginal latency win over the screenrecord path. Not worth it.
• [WONTDO] adb screencap fallback — full-PNG-per-frame pipeline is slower than MediaProjection, no value as a last resort.

Known projects using the screenrecord approach for reference: scrcpy (over ADB), scrcpy-hidden-api, shizuku.

Android Autostart on Boot

Boot-time, zero-interaction startup so LedGrab always has display capture and control on rooted TV boxes.

• Manifest: declare RECEIVE_BOOT_COMPLETED, REQUEST_IGNORE_BATTERY_OPTIMIZATIONS, WAKE_LOCK; register .BootReceiver for BOOT_COMPLETED / LOCKED_BOOT_COMPLETED / MY_PACKAGE_REPLACED.
• BootReceiver.kt — gated by AutostartPrefs + Root.looksRooted(); dispatches CaptureService.createRootIntent() via ContextCompat.startForegroundService. Unrooted devices are a no-op because MediaProjection consent cannot be bypassed silently.
• AutostartPrefs.kt — thin SharedPreferences wrapper, defaults to enabled. Shown as a CheckBox on the stopped panel; greyed out on unrooted devices.
• CaptureService returns START_REDELIVER_INTENT for root-mode intents so the OS can cleanly restart the service after being killed (token-free path). MediaProjection-mode keeps START_NOT_STICKY — restart is pointless with a dead consent token.
• isRunning race: moved assignment to after startForeground succeeds, resets on exception; onStartCommand wraps startForeground in try/catch and stops the service cleanly if the FG transition fails.
• Root-capture watchdog: coroutine on serviceScope checks RootScreenrecord.framesDelivered every 5s after a 5s grace. Respawns the pipeline (reusing the existing Python bridge) on stall, caps at 3 consecutive restarts before giving up.
• RootScreenrecord.framesDelivered exposed as a property backed by AtomicInteger (was @Volatile var framesDelivered = 0 with non-atomic += 1).
• ScreenCapture accepts onProjectionStopped lambda — MediaProjection.Callback.onStop now tears the whole service down instead of leaving a stale FG notification.
• MainActivity wires the autostart toggle to AutostartPrefs; enabling it prompts REQUEST_IGNORE_BATTERY_OPTIMIZATIONS so Doze doesn't kill the FG service on phones.
• versionCode derived from git rev-list --count HEAD (or ANDROID_VERSION_CODE env var in CI). Was stuck at 1 — sideload updates were silently refusing to install.
• Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) boot-time autostart dispatches the service without UI, (2) capture indicator still absent under root mode post-reboot, (3) watchdog respawns the pipeline when screenrecord is externally killed, (4) sideload upgrade installs cleanly after the versionCode bump.
• Optional follow-up: "kiosk" mode — add <category android:name="android.intent.category.HOME" /> to MainActivity so power users can set LedGrab as the default TV launcher for truly always-running behavior.

Android USB Serial Support

Drive USB LED controllers (APA102, WS2812) connected directly to the Android TV box via USB-to-serial adapters.

• Added com.github.mik3y:usb-serial-for-android:3.8.1 (via JitPack) to android/app/build.gradle.kts.
• Kotlin UsbSerialBridge singleton (android/app/src/main/java/com/ledgrab/android/UsbSerialBridge.kt) — exposes listDevices(), open(vid, pid, serial, baud), write(handle, ByteArray), close(handle). Permission request fires automatically from open() when the user hasn't granted access yet. Handles are opaque integers, port map is synchronized, so Python threads can share one bridge.
• Python AndroidSerialTransport in server/src/ledgrab/core/devices/android_serial_transport.py drives the bridge through Chaquopy. SerialTransport Protocol + PySerialTransport + list_serial_ports() factory live in serial_transport.py; AdalightClient and SerialDeviceProvider now go through the abstraction instead of importing pyserial directly.
• URL scheme extended: usb:VID:PID[:serial][@baud] on Android alongside the existing COM3[:baud] / /dev/ttyUSB0[:baud] desktop paths.
• App initializes the bridge on startup (LedGrabApp.onCreate → UsbSerialBridge.init(this)); manifest declares uses-feature android.hardware.usb.host.
• Real-device test pending — no USB-serial hardware on dev machine. Need to verify on a TV box with CH340, CP2102, or FTDI adapter.
• Document supported USB LED controllers in README (once real-device test passes).
• Optional: auto-launch the app when a known USB-serial adapter is plugged in (intent-filter on USB_DEVICE_ATTACHED + res/xml/device_filter.xml). Skipped in v1 — users can just open LedGrab and hit "Discover".
• ESP-NOW client (espnow_client.py / espnow_provider.py) now routes through SerialTransport — open_transport() for the gateway serial link, list_serial_ports() + port_exists() for discovery/validation. Works transparently with usb:VID:PID URLs on Android. (Gateway protocol is write-only, so no read() extension was needed after all.)

Performance Metrics Abstraction

• MetricsProvider protocol + dataclass DTOs (MemorySnapshot, ProcessSnapshot) live in server/src/ledgrab/utils/metrics/types.py. Each provider has its own module: psutil_provider.py, null_provider.py, android_provider.py.
• Factory get_metrics_provider() in utils/metrics/__init__.py selects Android → psutil → Null. psutil import is now confined to one place.
• api/routes/system.py and core/processing/metrics_history.py use the provider; no more if psutil is not None guards in the hot paths.
• Android /proc-backed provider implemented (/proc/stat, /proc/meminfo, /proc/self/stat, /proc/self/status). Carries previous-sample state for delta-based CPU%; degrades to zeros if any /proc file is locked down. 12 unit tests cover both desktop and Android paths.

Android Performance Metrics — Future Enhancements

Beyond the /proc-based AndroidMetricsProvider that's now in place:

• Device battery + thermal-zone readings (/sys/class/power_supply/battery/{capacity,temp}, /sys/class/thermal/thermal_zone*/temp filtered by zone type). Surfaced through MetricsProvider.thermals(), PerformanceResponse.{cpu_temp_c,battery_percent,battery_temp_c}, the metrics-history snapshot, and a new dashboard temperature chart that hides itself when the backend reports null. GPU card now hides (no "unavailable" placeholder) when no GPU is present.
• [WONTDO] Optional: app-specific memory via Debug.getMemoryInfo() through a Kotlin → Python Chaquopy bridge (more accurate than VmRSS for split-app-process accounting)
• [WONTDO] Optional: GPU usage via /sys/class/kgsl/kgsl-3d0/gpubusy on Adreno, Mali-specific paths for Mali GPUs

Refactor: Per-Provider Device Configs

Replace flat DeviceInfo + **kwargs provider contract with a discriminated union of typed per-provider config dataclasses. Full plan: docs/plans/device-typed-configs.md.

• Phase 1 — DeviceConfig hierarchy + Device.to_config() (non-breaking, additive only)
• Phases 2+3 — narrow LEDDeviceProvider.create_client to typed configs; migrate 3 call sites; delete DeviceInfo + _get_device_info + _DEVICE_FIELD_DEFAULTS (single PR)
• Phase 4 — migrate tests/test_group_device.py to GroupConfig/ProviderDeps; remove legacy GroupLEDClient init path; 47-test config suite with 100% coverage on device_config.py
• Phase 5 (separate PR, optional) — Pydantic v2 discriminated union in api/schemas/devices.py; scope frontend POST/PATCH payloads by device_type