ledgrab/TODO.md

# LedGrab TODO

## BLE LED Controller Support (SP110E / Triones / Zengge / Govee)

Add support for Bluetooth Low Energy LED controllers driven by mobile apps like "LED Hue", HappyLighting, iLightsIn. Whole-strip ambient-color output only — these protocols don't support per-pixel streaming.

- [x] Add `bleak>=0.22` as optional extra `[ble]` in `server/pyproject.toml` (desktop-only, NOT in android `build.gradle.kts`)
- [x] `core/devices/ble_transport.py` — bleak wrapper: scan, connect, write-with/without-response
- [x] `core/devices/ble_protocols/` package
  - [x] `__init__.py` — `BLEProtocol` dataclass + registry (family → encoder)
  - [x] `sp110e.py` — SP110E / SP108E (service FFE0, char FFE1, `RR GG BB 00 1E` static-color frame)
  - [x] `triones.py` — Triones / HappyLighting / LEDnet (service FFE5, char FFE9, `7E 07 05 03 RR GG BB 10 EF`)
  - [x] `zengge.py` — Zengge / iLightsIn (service FFE0, framing `56 RR GG BB 00 F0 AA`)
  - [x] `govee.py` — Govee unencrypted framed protocol (AES keyed variants — marked experimental)
- [x] `core/devices/ble_client.py` — unified `BLEClient(LEDClient)` — picks protocol by `ble_family`, averages strip → one color, drops duplicate frames, rate-limits to BLE connection interval
- [x] `core/devices/ble_provider.py` — `BLEDeviceProvider` + discovery via `BleakScanner`
- [x] Register in `core/devices/led_client.py::_register_builtin_providers` (guarded `try/except ImportError`)
- [x] Storage: `ble_family`, `ble_govee_key` fields threaded through `Device.__init__`/`to_dict`/`from_dict`/`_UPDATABLE_FIELDS`/`create_device`
- [x] Schemas: BLE fields on `DeviceCreate`, `DeviceUpdate`, `DeviceResponse`
- [x] Routes: BLE fields propagated through create/update in `api/routes/devices.py` + `_device_to_response`
- [x] ProcessorManager: `ble_family`/`ble_govee_key` added to `_DEVICE_FIELD_DEFAULTS` and `DeviceInfo`; passed through `wled_target_processor.py` and `group_client.py` to `create_led_client`
- [x] Tests: 21 protocol encoder unit tests + 16 BLEClient fake-transport tests — all passing, 814 total tests still green
- [x] Frontend: BLE option in the device type picker with a bluetooth Lucide icon; add-device modal shows a 4-option `IconSelect` for protocol family (SP110E / Triones / Zengge / Govee) with a Govee-only AES key field that auto-hides for the other three families; URL label/placeholder/hint adapt to `ble://<address>` pattern; submit payload carries `ble_family` (+ optional `ble_govee_key`); clone flow pre-fills family and key; modal dirty-check snapshots the new fields; network scan button now also discovers BLE peripherals via the existing `/api/v1/devices/discover?device_type=ble` endpoint
- [x] Frontend: `isBleDevice` helper in `core/api.ts`; `ICON_BLUETOOTH` + `ICON_LIGHTBULB` constants in `core/icons.ts`; `bluetooth` path in `core/icon-paths.ts`; i18n keys in `en.json` / `ru.json` / `zh.json`; TypeScript compiles; esbuild bundle rebuilt
- [x] Android BLE via Kotlin bridge — `BleBridge.kt` singleton (scan/connect/write/disconnect); `android_ble_transport.py` Python wrapper; `make_transport()` factory in `ble_transport.py` auto-selects backend; `BleBridge.init()` called from `LedGrabApp.onCreate`; BLE permissions in `AndroidManifest.xml`
- [x] Govee per-model AES key — `_encrypt_govee_frame()` in `ble_client.py` uses AES-128-ECB from `cryptography`; key validated on `BLEClient` construction; applied to both `send_pixels` and `set_power`; 8 new AES unit tests

## Android — Restore Multi-ABI Wheels

During emulator testing, we switched the build to **x86 only** (see `android/app/build.gradle.kts` `abiFilters`) to avoid having to keep the arm64-v8a / x86_64 pydantic-core wheels current. Before shipping, restore all three ABIs:

- [x] Rebuild `pydantic-core` wheels for all three ABIs with the current SOABI + libpython linking settings (`android/build-scripts/build-pydantic-core.sh` — now supports `arm64`, `x86_64`, `x86` args; defaults to all three).
- [x] Verify wheels: all three now list `libpython3.11.so` in `NEEDED` (`llvm-readelf -d`), automated in the build script.
- [x] Restored `abiFilters += listOf("arm64-v8a", "x86_64", "x86")` in `build.gradle.kts`. Multi-ABI debug APK builds cleanly (~99 MB).
- [ ] Re-test on real ARM64 Android TV hardware (still pending — only emulator-verified build).

Build cache + scripts live in `android/build-scripts/` and `android/.build-cache/` (junction host + sysconfigdata for each ABI).

## Android CI Pipeline

Build the Android APK automatically on push/tag.

- [x] Generate Gradle wrapper (`gradlew`) and commit it
- [x] Create CI workflow (`.gitea/workflows/build-android.yml`)
  - JDK 17 + Android SDK + NDK setup
  - Python 3.11 for Chaquopy build
  - Recreate the directory junction via `ln -s` on Linux CI
  - `./gradlew assembleDebug` on master push, `assembleRelease` on `v*` tags (if signing secrets set)
  - Uploads APK as CI artifact; attaches to Gitea release on tag push
- [x] Commit pre-built pydantic-core wheels to `android/wheels/` (arm64, x86, x86_64)
- [x] APK signing for release builds — conditional signing config reads keystore from env vars (`ANDROID_KEYSTORE_PATH/_PASSWORD/_ALIAS/_KEY_PASSWORD`), falls back to debug signing locally
- [ ] Provision a real keystore and add the four CI secrets:
  - `ANDROID_KEYSTORE_BASE64` (base64-encoded .jks)
  - `ANDROID_KEYSTORE_PASSWORD`
  - `ANDROID_KEY_ALIAS`
  - `ANDROID_KEY_PASSWORD`
- [ ] Add `LedGrab-{tag}-android-release.apk` row to the release description table in `.gitea/workflows/release.yml` → `create-release` job
- [ ] Verify the CI workflow passes end-to-end with the now-restored multi-ABI build (larger APK, longer Android build step)

## Android Root Capture (No Permission Dialog, No System Indicator)

MediaProjection shows a mandatory system overlay/indicator while capturing — unavoidable on stock Android. Many cheap Android TV boxes ship pre-rooted, so an alternative root-only path gives much better UX.

- [x] Root detection — `Root.kt` checks common `su` binary paths and, on demand, runs `su -c id` to actually prove UID 0. First call triggers Magisk's grant dialog; grant is cached per session. Exposed to Python via Chaquopy.
- [x] `RootScreenrecord.kt` — spawns `su -c screenrecord --output-format=h264 --size=WxH -`, feeds the H.264 stdout through a MediaCodec decoder whose output Surface is wired into an ImageReader (RGBA_8888, row-stride-aware). Decoded frames reach the Python pipeline via `PythonBridge.pushRootFrame`.
- [x] Python-side `RootScreenrecordEngine` (`core/capture_engines/root_screenrecord_engine.py`) mirrors `MediaProjectionEngine` with `ENGINE_PRIORITY=110` (> MediaProjection's 100) so the factory picks it automatically when available.
- [x] `MainActivity` tries `Root.requestGrant()` before launching the MediaProjection consent flow — on rooted devices the consent dialog is skipped entirely. `CaptureService` has a `createRootIntent()` entry point that bypasses the MediaProjection path.
- [x] Fallback: if `Root.requestGrant()` returns false (no root, user denied, or `su` timeout) the existing MediaProjection flow runs unchanged.
- [ ] Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) grant dialog appears once, (2) frames actually flow through MediaCodec without the Android 14 capture indicator showing, (3) stop/start cycle terminates the `su` process cleanly.
- [WONTDO] `SurfaceControl.screenshot()` via reflection — renamed/moved across API 28/29/30/33, hidden-API blocklist varies by release, even rooted apps hit it; days of maintenance for a marginal latency win over the screenrecord path. Not worth it.
- [WONTDO] `adb screencap` fallback — full-PNG-per-frame pipeline is slower than MediaProjection, no value as a last resort.

Known projects using the screenrecord approach for reference: scrcpy (over ADB), scrcpy-hidden-api, shizuku.

## Android Autostart on Boot

Boot-time, zero-interaction startup so LedGrab always has display capture and control on rooted TV boxes.

- [x] Manifest: declare `RECEIVE_BOOT_COMPLETED`, `REQUEST_IGNORE_BATTERY_OPTIMIZATIONS`, `WAKE_LOCK`; register `.BootReceiver` for `BOOT_COMPLETED` / `LOCKED_BOOT_COMPLETED` / `MY_PACKAGE_REPLACED`.
- [x] `BootReceiver.kt` — gated by `AutostartPrefs` + `Root.looksRooted()`; dispatches `CaptureService.createRootIntent()` via `ContextCompat.startForegroundService`. Unrooted devices are a no-op because MediaProjection consent cannot be bypassed silently.
- [x] `AutostartPrefs.kt` — thin SharedPreferences wrapper, defaults to enabled. Shown as a CheckBox on the stopped panel; greyed out on unrooted devices.
- [x] `CaptureService` returns `START_REDELIVER_INTENT` for root-mode intents so the OS can cleanly restart the service after being killed (token-free path). MediaProjection-mode keeps `START_NOT_STICKY` — restart is pointless with a dead consent token.
- [x] `isRunning` race: moved assignment to after `startForeground` succeeds, resets on exception; `onStartCommand` wraps `startForeground` in try/catch and stops the service cleanly if the FG transition fails.
- [x] Root-capture watchdog: coroutine on `serviceScope` checks `RootScreenrecord.framesDelivered` every 5s after a 5s grace. Respawns the pipeline (reusing the existing Python bridge) on stall, caps at 3 consecutive restarts before giving up.
- [x] `RootScreenrecord.framesDelivered` exposed as a property backed by `AtomicInteger` (was `@Volatile var framesDelivered = 0` with non-atomic `+= 1`).
- [x] `ScreenCapture` accepts `onProjectionStopped` lambda — `MediaProjection.Callback.onStop` now tears the whole service down instead of leaving a stale FG notification.
- [x] `MainActivity` wires the autostart toggle to `AutostartPrefs`; enabling it prompts `REQUEST_IGNORE_BATTERY_OPTIMIZATIONS` so Doze doesn't kill the FG service on phones.
- [x] `versionCode` derived from `git rev-list --count HEAD` (or `ANDROID_VERSION_CODE` env var in CI). Was stuck at 1 — sideload updates were silently refusing to install.
- [ ] Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) boot-time autostart dispatches the service without UI, (2) capture indicator still absent under root mode post-reboot, (3) watchdog respawns the pipeline when `screenrecord` is externally killed, (4) sideload upgrade installs cleanly after the versionCode bump.
- [ ] Optional follow-up: "kiosk" mode — add `<category android:name="android.intent.category.HOME" />` to `MainActivity` so power users can set LedGrab as the default TV launcher for truly always-running behavior.

## Android USB Serial Support

Drive USB LED controllers (APA102, WS2812) connected directly to the Android TV box via USB-to-serial adapters.

- [x] Added `com.github.mik3y:usb-serial-for-android:3.8.1` (via JitPack) to `android/app/build.gradle.kts`.
- [x] Kotlin `UsbSerialBridge` singleton (`android/app/src/main/java/com/ledgrab/android/UsbSerialBridge.kt`) — exposes `listDevices()`, `open(vid, pid, serial, baud)`, `write(handle, ByteArray)`, `close(handle)`. Permission request fires automatically from `open()` when the user hasn't granted access yet. Handles are opaque integers, port map is synchronized, so Python threads can share one bridge.
- [x] Python `AndroidSerialTransport` in `server/src/ledgrab/core/devices/android_serial_transport.py` drives the bridge through Chaquopy. `SerialTransport` Protocol + `PySerialTransport` + `list_serial_ports()` factory live in `serial_transport.py`; `AdalightClient` and `SerialDeviceProvider` now go through the abstraction instead of importing `pyserial` directly.
- [x] URL scheme extended: `usb:VID:PID[:serial][@baud]` on Android alongside the existing `COM3[:baud]` / `/dev/ttyUSB0[:baud]` desktop paths.
- [x] App initializes the bridge on startup (`LedGrabApp.onCreate` → `UsbSerialBridge.init(this)`); manifest declares `uses-feature android.hardware.usb.host`.
- [ ] Real-device test pending — no USB-serial hardware on dev machine. Need to verify on a TV box with CH340, CP2102, or FTDI adapter.
- [ ] Document supported USB LED controllers in README (once real-device test passes).
- [ ] Optional: auto-launch the app when a known USB-serial adapter is plugged in (intent-filter on `USB_DEVICE_ATTACHED` + `res/xml/device_filter.xml`). Skipped in v1 — users can just open LedGrab and hit "Discover".
- [x] ESP-NOW client (`espnow_client.py` / `espnow_provider.py`) now routes through `SerialTransport` — `open_transport()` for the gateway serial link, `list_serial_ports()` + `port_exists()` for discovery/validation. Works transparently with `usb:VID:PID` URLs on Android. (Gateway protocol is write-only, so no `read()` extension was needed after all.)

## Performance Metrics Abstraction

- [x] `MetricsProvider` protocol + dataclass DTOs (`MemorySnapshot`, `ProcessSnapshot`) live in `server/src/ledgrab/utils/metrics/types.py`. Each provider has its own module: `psutil_provider.py`, `null_provider.py`, `android_provider.py`.
- [x] Factory `get_metrics_provider()` in `utils/metrics/__init__.py` selects Android → psutil → Null. `psutil` import is now confined to one place.
- [x] `api/routes/system.py` and `core/processing/metrics_history.py` use the provider; no more `if psutil is not None` guards in the hot paths.
- [x] Android `/proc`-backed provider implemented (`/proc/stat`, `/proc/meminfo`, `/proc/self/stat`, `/proc/self/status`). Carries previous-sample state for delta-based CPU%; degrades to zeros if any `/proc` file is locked down. 12 unit tests cover both desktop and Android paths.

## Android Performance Metrics — Future Enhancements

Beyond the `/proc`-based AndroidMetricsProvider that's now in place:

- [x] Device battery + thermal-zone readings (`/sys/class/power_supply/battery/{capacity,temp}`, `/sys/class/thermal/thermal_zone*/temp` filtered by zone type). Surfaced through `MetricsProvider.thermals()`, `PerformanceResponse.{cpu_temp_c,battery_percent,battery_temp_c}`, the metrics-history snapshot, and a new dashboard temperature chart that hides itself when the backend reports null. GPU card now hides (no "unavailable" placeholder) when no GPU is present.
- [WONTDO] Optional: app-specific memory via `Debug.getMemoryInfo()` through a Kotlin → Python Chaquopy bridge (more accurate than `VmRSS` for split-app-process accounting)
- [WONTDO] Optional: GPU usage via `/sys/class/kgsl/kgsl-3d0/gpubusy` on Adreno, Mali-specific paths for Mali GPUs

## Refactor: Per-Provider Device Configs

Replace flat `DeviceInfo` + `**kwargs` provider contract with a discriminated union of typed per-provider config dataclasses. Full plan: [docs/plans/device-typed-configs.md](docs/plans/device-typed-configs.md).

- [x] Phase 1 — `DeviceConfig` hierarchy + `Device.to_config()` (non-breaking, additive only)
- [x] Phases 2+3 — narrow `LEDDeviceProvider.create_client` to typed configs; migrate 3 call sites; delete `DeviceInfo` + `_get_device_info` + `_DEVICE_FIELD_DEFAULTS` (single PR)
- [x] Phase 4 — migrate `tests/test_group_device.py` to `GroupConfig`/`ProviderDeps`; remove legacy `GroupLEDClient` init path; 47-test config suite with 100% coverage on `device_config.py`
- [ ] Phase 5 (separate PR, optional) — Pydantic v2 discriminated union in `api/schemas/devices.py`; scope frontend POST/PATCH payloads by `device_type`