ledgrab/TODO.md

LedGrab TODO

Android — Restore Multi-ABI Wheels

During emulator testing, we switched the build to x86 only (see android/app/build.gradle.kts abiFilters) to avoid having to keep the arm64-v8a / x86_64 pydantic-core wheels current. Before shipping, restore all three ABIs:

• Rebuild pydantic-core wheels for all three ABIs with the current SOABI + libpython linking settings (android/build-scripts/build-pydantic-core.sh — now supports arm64, x86_64, x86 args; defaults to all three).
• Verify wheels: all three now list libpython3.11.so in NEEDED (llvm-readelf -d), automated in the build script.
• Restored abiFilters += listOf("arm64-v8a", "x86_64", "x86") in build.gradle.kts. Multi-ABI debug APK builds cleanly (~99 MB).
• Re-test on real ARM64 Android TV hardware (still pending — only emulator-verified build).

Build cache + scripts live in android/build-scripts/ and android/.build-cache/ (junction host + sysconfigdata for each ABI).

Android CI Pipeline

Build the Android APK automatically on push/tag.

• Generate Gradle wrapper (gradlew) and commit it
• Create CI workflow (.gitea/workflows/build-android.yml)
  • JDK 17 + Android SDK + NDK setup
  • Python 3.11 for Chaquopy build
  • Recreate the directory junction via ln -s on Linux CI
  • ./gradlew assembleDebug on master push, assembleRelease on v* tags (if signing secrets set)
  • Uploads APK as CI artifact; attaches to Gitea release on tag push
• Commit pre-built pydantic-core wheels to android/wheels/ (arm64, x86, x86_64)
• APK signing for release builds — conditional signing config reads keystore from env vars (ANDROID_KEYSTORE_PATH/_PASSWORD/_ALIAS/_KEY_PASSWORD), falls back to debug signing locally
• Provision a real keystore and add the four CI secrets:
  • ANDROID_KEYSTORE_BASE64 (base64-encoded .jks)
  • ANDROID_KEYSTORE_PASSWORD
  • ANDROID_KEY_ALIAS
  • ANDROID_KEY_PASSWORD
• Add LedGrab-{tag}-android-release.apk row to the release description table in .gitea/workflows/release.yml → create-release job
• Verify the CI workflow passes end-to-end with the now-restored multi-ABI build (larger APK, longer Android build step)

Android Root Capture (No Permission Dialog, No System Indicator)

MediaProjection shows a mandatory system overlay/indicator while capturing — unavoidable on stock Android. Many cheap Android TV boxes ship pre-rooted, so an alternative root-only path gives much better UX.

• Root detection — Root.kt checks common su binary paths and, on demand, runs su -c id to actually prove UID 0. First call triggers Magisk's grant dialog; grant is cached per session. Exposed to Python via Chaquopy.
• RootScreenrecord.kt — spawns su -c screenrecord --output-format=h264 --size=WxH -, feeds the H.264 stdout through a MediaCodec decoder whose output Surface is wired into an ImageReader (RGBA_8888, row-stride-aware). Decoded frames reach the Python pipeline via PythonBridge.pushRootFrame.
• Python-side RootScreenrecordEngine (core/capture_engines/root_screenrecord_engine.py) mirrors MediaProjectionEngine with ENGINE_PRIORITY=110 (> MediaProjection's 100) so the factory picks it automatically when available.
• MainActivity tries Root.requestGrant() before launching the MediaProjection consent flow — on rooted devices the consent dialog is skipped entirely. CaptureService has a createRootIntent() entry point that bypasses the MediaProjection path.
• Fallback: if Root.requestGrant() returns false (no root, user denied, or su timeout) the existing MediaProjection flow runs unchanged.
• Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) grant dialog appears once, (2) frames actually flow through MediaCodec without the Android 14 capture indicator showing, (3) stop/start cycle terminates the su process cleanly.
• [WONTDO] SurfaceControl.screenshot() via reflection — renamed/moved across API 28/29/30/33, hidden-API blocklist varies by release, even rooted apps hit it; days of maintenance for a marginal latency win over the screenrecord path. Not worth it.
• [WONTDO] adb screencap fallback — full-PNG-per-frame pipeline is slower than MediaProjection, no value as a last resort.

Known projects using the screenrecord approach for reference: scrcpy (over ADB), scrcpy-hidden-api, shizuku.

Android USB Serial Support

Drive USB LED controllers (APA102, WS2812) connected directly to the Android TV box via USB-to-serial adapters.

• Added com.github.mik3y:usb-serial-for-android:3.8.1 (via JitPack) to android/app/build.gradle.kts.
• Kotlin UsbSerialBridge singleton (android/app/src/main/java/com/ledgrab/android/UsbSerialBridge.kt) — exposes listDevices(), open(vid, pid, serial, baud), write(handle, ByteArray), close(handle). Permission request fires automatically from open() when the user hasn't granted access yet. Handles are opaque integers, port map is synchronized, so Python threads can share one bridge.
• Python AndroidSerialTransport in server/src/ledgrab/core/devices/android_serial_transport.py drives the bridge through Chaquopy. SerialTransport Protocol + PySerialTransport + list_serial_ports() factory live in serial_transport.py; AdalightClient and SerialDeviceProvider now go through the abstraction instead of importing pyserial directly.
• URL scheme extended: usb:VID:PID[:serial][@baud] on Android alongside the existing COM3[:baud] / /dev/ttyUSB0[:baud] desktop paths.
• App initializes the bridge on startup (LedGrabApp.onCreate → UsbSerialBridge.init(this)); manifest declares uses-feature android.hardware.usb.host.
• Real-device test pending — no USB-serial hardware on dev machine. Need to verify on a TV box with CH340, CP2102, or FTDI adapter.
• Document supported USB LED controllers in README (once real-device test passes).
• Optional: auto-launch the app when a known USB-serial adapter is plugged in (intent-filter on USB_DEVICE_ATTACHED + res/xml/device_filter.xml). Skipped in v1 — users can just open LedGrab and hit "Discover".
• ESP-NOW client (espnow_client.py / espnow_provider.py) now routes through SerialTransport — open_transport() for the gateway serial link, list_serial_ports() + port_exists() for discovery/validation. Works transparently with usb:VID:PID URLs on Android. (Gateway protocol is write-only, so no read() extension was needed after all.)

Performance Metrics Abstraction

• MetricsProvider protocol + dataclass DTOs (MemorySnapshot, ProcessSnapshot) live in server/src/ledgrab/utils/metrics/types.py. Each provider has its own module: psutil_provider.py, null_provider.py, android_provider.py.
• Factory get_metrics_provider() in utils/metrics/__init__.py selects Android → psutil → Null. psutil import is now confined to one place.
• api/routes/system.py and core/processing/metrics_history.py use the provider; no more if psutil is not None guards in the hot paths.
• Android /proc-backed provider implemented (/proc/stat, /proc/meminfo, /proc/self/stat, /proc/self/status). Carries previous-sample state for delta-based CPU%; degrades to zeros if any /proc file is locked down. 12 unit tests cover both desktop and Android paths.

Android Performance Metrics — Future Enhancements

Beyond the /proc-based AndroidMetricsProvider that's now in place:

• Device battery + thermal-zone readings (/sys/class/power_supply/battery/{capacity,temp}, /sys/class/thermal/thermal_zone*/temp filtered by zone type). Surfaced through MetricsProvider.thermals(), PerformanceResponse.{cpu_temp_c,battery_percent,battery_temp_c}, the metrics-history snapshot, and a new dashboard temperature chart that hides itself when the backend reports null. GPU card now hides (no "unavailable" placeholder) when no GPU is present.
• [WONTDO] Optional: app-specific memory via Debug.getMemoryInfo() through a Kotlin → Python Chaquopy bridge (more accurate than VmRSS for split-app-process accounting)
• [WONTDO] Optional: GPU usage via /sys/class/kgsl/kgsl-3d0/gpubusy on Adreno, Mali-specific paths for Mali GPUs