ledgrab/TODO.md

LedGrab TODO

BLE LED Controller Support (SP110E / Triones / Zengge / Govee)

Add support for Bluetooth Low Energy LED controllers driven by mobile apps like "LED Hue", HappyLighting, iLightsIn. Whole-strip ambient-color output only — these protocols don't support per-pixel streaming.

• Add bleak>=0.22 as optional extra [ble] in server/pyproject.toml (desktop-only, NOT in android build.gradle.kts)
• core/devices/ble_transport.py — bleak wrapper: scan, connect, write-with/without-response
• core/devices/ble_protocols/ package
  • __init__.py — BLEProtocol dataclass + registry (family → encoder)
  • sp110e.py — SP110E / SP108E (service FFE0, char FFE1, RR GG BB 00 1E static-color frame)
  • triones.py — Triones / HappyLighting / LEDnet (service FFE5, char FFE9, 7E 07 05 03 RR GG BB 10 EF)
  • zengge.py — Zengge / iLightsIn (service FFE0, framing 56 RR GG BB 00 F0 AA)
  • govee.py — Govee unencrypted framed protocol (AES keyed variants — marked experimental)
• core/devices/ble_client.py — unified BLEClient(LEDClient) — picks protocol by ble_family, averages strip → one color, drops duplicate frames, rate-limits to BLE connection interval
• core/devices/ble_provider.py — BLEDeviceProvider + discovery via BleakScanner
• Register in core/devices/led_client.py::_register_builtin_providers (guarded try/except ImportError)
• Storage: ble_family, ble_govee_key fields threaded through Device.__init__/to_dict/from_dict/_UPDATABLE_FIELDS/create_device
• Schemas: BLE fields on DeviceCreate, DeviceUpdate, DeviceResponse
• Routes: BLE fields propagated through create/update in api/routes/devices.py + _device_to_response
• ProcessorManager: ble_family/ble_govee_key added to _DEVICE_FIELD_DEFAULTS and DeviceInfo; passed through wled_target_processor.py and group_client.py to create_led_client
• Tests: 21 protocol encoder unit tests + 16 BLEClient fake-transport tests — all passing, 814 total tests still green
• Frontend: BLE option in the device type picker with a bluetooth Lucide icon; add-device modal shows a 4-option IconSelect for protocol family (SP110E / Triones / Zengge / Govee) with a Govee-only AES key field that auto-hides for the other three families; URL label/placeholder/hint adapt to ble://<address> pattern; submit payload carries ble_family (+ optional ble_govee_key); clone flow pre-fills family and key; modal dirty-check snapshots the new fields; network scan button now also discovers BLE peripherals via the existing /api/v1/devices/discover?device_type=ble endpoint
• Frontend: isBleDevice helper in core/api.ts; ICON_BLUETOOTH + ICON_LIGHTBULB constants in core/icons.ts; bluetooth path in core/icon-paths.ts; i18n keys in en.json / ru.json / zh.json; TypeScript compiles; esbuild bundle rebuilt
• Android BLE via Kotlin bridge — BleBridge.kt singleton (scan/connect/write/disconnect); android_ble_transport.py Python wrapper; make_transport() factory in ble_transport.py auto-selects backend; BleBridge.init() called from LedGrabApp.onCreate; BLE permissions in AndroidManifest.xml
• Govee per-model AES key — _encrypt_govee_frame() in ble_client.py uses AES-128-ECB from cryptography; key validated on BLEClient construction; applied to both send_pixels and set_power; 8 new AES unit tests

Android — Restore Multi-ABI Wheels

During emulator testing, we switched the build to x86 only (see android/app/build.gradle.kts abiFilters) to avoid having to keep the arm64-v8a / x86_64 pydantic-core wheels current. Before shipping, restore all three ABIs:

• Rebuild pydantic-core wheels for all three ABIs with the current SOABI + libpython linking settings (android/build-scripts/build-pydantic-core.sh — now supports arm64, x86_64, x86 args; defaults to all three).
• Verify wheels: all three now list libpython3.11.so in NEEDED (llvm-readelf -d), automated in the build script.
• Restored abiFilters += listOf("arm64-v8a", "x86_64", "x86") in build.gradle.kts. Multi-ABI debug APK builds cleanly (~99 MB).
• Re-test on real ARM64 Android TV hardware (still pending — only emulator-verified build).

Build cache + scripts live in android/build-scripts/ and android/.build-cache/ (junction host + sysconfigdata for each ABI).

Android CI Pipeline

Build the Android APK automatically on push/tag.

• Generate Gradle wrapper (gradlew) and commit it
• Create CI workflow (.gitea/workflows/build-android.yml)
  • JDK 17 + Android SDK + NDK setup
  • Python 3.11 for Chaquopy build
  • Recreate the directory junction via ln -s on Linux CI
  • ./gradlew assembleDebug on master push, assembleRelease on v* tags (if signing secrets set)
  • Uploads APK as CI artifact; attaches to Gitea release on tag push
• Commit pre-built pydantic-core wheels to android/wheels/ (arm64, x86, x86_64)
• APK signing for release builds — conditional signing config reads keystore from env vars (ANDROID_KEYSTORE_PATH/_PASSWORD/_ALIAS/_KEY_PASSWORD), falls back to debug signing locally
• Provision a real keystore and add the four CI secrets:
  • ANDROID_KEYSTORE_BASE64 (base64-encoded .jks)
  • ANDROID_KEYSTORE_PASSWORD
  • ANDROID_KEY_ALIAS
  • ANDROID_KEY_PASSWORD
• Add LedGrab-{tag}-android-release.apk row to the release description table in .gitea/workflows/release.yml → create-release job
• Verify the CI workflow passes end-to-end with the now-restored multi-ABI build (larger APK, longer Android build step)

Android Root Capture (No Permission Dialog, No System Indicator)

MediaProjection shows a mandatory system overlay/indicator while capturing — unavoidable on stock Android. Many cheap Android TV boxes ship pre-rooted, so an alternative root-only path gives much better UX.

• Root detection — Root.kt checks common su binary paths and, on demand, runs su -c id to actually prove UID 0. First call triggers Magisk's grant dialog; grant is cached per session. Exposed to Python via Chaquopy.
• RootScreenrecord.kt — spawns su -c screenrecord --output-format=h264 --size=WxH -, feeds the H.264 stdout through a MediaCodec decoder whose output Surface is wired into an ImageReader (RGBA_8888, row-stride-aware). Decoded frames reach the Python pipeline via PythonBridge.pushRootFrame.
• Python-side RootScreenrecordEngine (core/capture_engines/root_screenrecord_engine.py) mirrors MediaProjectionEngine with ENGINE_PRIORITY=110 (> MediaProjection's 100) so the factory picks it automatically when available.
• MainActivity tries Root.requestGrant() before launching the MediaProjection consent flow — on rooted devices the consent dialog is skipped entirely. CaptureService has a createRootIntent() entry point that bypasses the MediaProjection path.
• Fallback: if Root.requestGrant() returns false (no root, user denied, or su timeout) the existing MediaProjection flow runs unchanged.
• Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) grant dialog appears once, (2) frames actually flow through MediaCodec without the Android 14 capture indicator showing, (3) stop/start cycle terminates the su process cleanly.
• [WONTDO] SurfaceControl.screenshot() via reflection — renamed/moved across API 28/29/30/33, hidden-API blocklist varies by release, even rooted apps hit it; days of maintenance for a marginal latency win over the screenrecord path. Not worth it.
• [WONTDO] adb screencap fallback — full-PNG-per-frame pipeline is slower than MediaProjection, no value as a last resort.

Known projects using the screenrecord approach for reference: scrcpy (over ADB), scrcpy-hidden-api, shizuku.

Android USB Serial Support

Drive USB LED controllers (APA102, WS2812) connected directly to the Android TV box via USB-to-serial adapters.

• Added com.github.mik3y:usb-serial-for-android:3.8.1 (via JitPack) to android/app/build.gradle.kts.
• Kotlin UsbSerialBridge singleton (android/app/src/main/java/com/ledgrab/android/UsbSerialBridge.kt) — exposes listDevices(), open(vid, pid, serial, baud), write(handle, ByteArray), close(handle). Permission request fires automatically from open() when the user hasn't granted access yet. Handles are opaque integers, port map is synchronized, so Python threads can share one bridge.
• Python AndroidSerialTransport in server/src/ledgrab/core/devices/android_serial_transport.py drives the bridge through Chaquopy. SerialTransport Protocol + PySerialTransport + list_serial_ports() factory live in serial_transport.py; AdalightClient and SerialDeviceProvider now go through the abstraction instead of importing pyserial directly.
• URL scheme extended: usb:VID:PID[:serial][@baud] on Android alongside the existing COM3[:baud] / /dev/ttyUSB0[:baud] desktop paths.
• App initializes the bridge on startup (LedGrabApp.onCreate → UsbSerialBridge.init(this)); manifest declares uses-feature android.hardware.usb.host.
• Real-device test pending — no USB-serial hardware on dev machine. Need to verify on a TV box with CH340, CP2102, or FTDI adapter.
• Document supported USB LED controllers in README (once real-device test passes).
• Optional: auto-launch the app when a known USB-serial adapter is plugged in (intent-filter on USB_DEVICE_ATTACHED + res/xml/device_filter.xml). Skipped in v1 — users can just open LedGrab and hit "Discover".
• ESP-NOW client (espnow_client.py / espnow_provider.py) now routes through SerialTransport — open_transport() for the gateway serial link, list_serial_ports() + port_exists() for discovery/validation. Works transparently with usb:VID:PID URLs on Android. (Gateway protocol is write-only, so no read() extension was needed after all.)

Performance Metrics Abstraction

• MetricsProvider protocol + dataclass DTOs (MemorySnapshot, ProcessSnapshot) live in server/src/ledgrab/utils/metrics/types.py. Each provider has its own module: psutil_provider.py, null_provider.py, android_provider.py.
• Factory get_metrics_provider() in utils/metrics/__init__.py selects Android → psutil → Null. psutil import is now confined to one place.
• api/routes/system.py and core/processing/metrics_history.py use the provider; no more if psutil is not None guards in the hot paths.
• Android /proc-backed provider implemented (/proc/stat, /proc/meminfo, /proc/self/stat, /proc/self/status). Carries previous-sample state for delta-based CPU%; degrades to zeros if any /proc file is locked down. 12 unit tests cover both desktop and Android paths.

Android Performance Metrics — Future Enhancements

Beyond the /proc-based AndroidMetricsProvider that's now in place:

• Device battery + thermal-zone readings (/sys/class/power_supply/battery/{capacity,temp}, /sys/class/thermal/thermal_zone*/temp filtered by zone type). Surfaced through MetricsProvider.thermals(), PerformanceResponse.{cpu_temp_c,battery_percent,battery_temp_c}, the metrics-history snapshot, and a new dashboard temperature chart that hides itself when the backend reports null. GPU card now hides (no "unavailable" placeholder) when no GPU is present.
• [WONTDO] Optional: app-specific memory via Debug.getMemoryInfo() through a Kotlin → Python Chaquopy bridge (more accurate than VmRSS for split-app-process accounting)
• [WONTDO] Optional: GPU usage via /sys/class/kgsl/kgsl-3d0/gpubusy on Adreno, Mali-specific paths for Mali GPUs

Refactor: Per-Provider Device Configs

Replace flat DeviceInfo + **kwargs provider contract with a discriminated union of typed per-provider config dataclasses. Full plan: docs/plans/device-typed-configs.md.

• Phase 1 — DeviceConfig hierarchy + Device.to_config() (non-breaking, additive only)
• Phases 2+3 — narrow LEDDeviceProvider.create_client to typed configs; migrate 3 call sites; delete DeviceInfo + _get_device_info + _DEVICE_FIELD_DEFAULTS (single PR)
• Phase 4 — migrate tests/test_group_device.py to GroupConfig/ProviderDeps; remove legacy GroupLEDClient init path; 47-test config suite with 100% coverage on device_config.py
• Phase 5 (separate PR, optional) — Pydantic v2 discriminated union in api/schemas/devices.py; scope frontend POST/PATCH payloads by device_type