ledgrab/TODO.md

LedGrab TODO

Android — Restore Multi-ABI Wheels

During emulator testing, we switched the build to x86 only (see android/app/build.gradle.kts abiFilters) to avoid having to keep the arm64-v8a / x86_64 pydantic-core wheels current. Before shipping, restore all three ABIs:

• Rebuild pydantic-core wheels for all three ABIs with the current SOABI + libpython linking settings:
  • arm64-v8a (real TV boxes — the primary target)
  • x86_64 (modern emulators)
  • x86 (legacy emulators)
• Verify wheels with readelf -d: SONAME must be libpython3.11.so in NEEDED
• Restore abiFilters += listOf("arm64-v8a", "x86_64", "x86") in build.gradle.kts
• Re-test on real ARM64 Android TV hardware

Build cache + scripts live in android/build-scripts/ and android/.build-cache/ (junction host + sysconfigdata for each ABI).

Android CI Pipeline

Build the Android APK automatically on push/tag.

• Generate Gradle wrapper (gradlew) and commit it
• Create CI workflow (.gitea/workflows/build-android.yaml or .github/workflows/)
  • JDK 17 + Android SDK + NDK setup
  • Python 3.11 for Chaquopy build
  • Recreate the directory junction (ln -s on Linux, mklink /J on Windows)
  • ./gradlew assembleRelease
  • Upload APK as artifact
• Commit pre-built pydantic-core wheels to android/wheels/ (arm64, x86, x86_64)
• APK signing for release builds (keystore setup)
• Consider: publish APK to GitHub/Gitea releases on tag push

Android Root Capture (No Permission Dialog, No System Indicator)

MediaProjection shows a mandatory system overlay/indicator while capturing — unavoidable on stock Android. Many cheap Android TV boxes ship pre-rooted, so an alternative root-only path would give much better UX.

• Detect root at runtime: check for su binary, Superuser.apk, etc.
• Implement SurfaceControlCaptureEngine (new capture engine) using hidden SurfaceControl.screenshot() API via reflection
  • No permission dialog
  • No system capture indicator
  • Direct bitmap output (no encoder/decoder roundtrip)
• Engine priority: higher than MediaProjection when root detected
• Fallback chain: SurfaceControl (root) → MediaProjection (stock) → adb screencap (last resort)
• Handle Android version differences in SurfaceControl API surface (renamed/moved across API 29, 30, 33)
• Alternative: shell out to screenrecord --output-format=h264 - as root (same H.264 decode as scrcpy_client_engine, but local instead of remote ADB)

Known projects using this approach for reference: scrcpy-hidden-api, shizuku, commercial scrcpy-derived apps.

Android USB Serial Support

Drive USB LED controllers (APA102, WS2812) connected directly to the Android TV box via USB-to-serial adapters.

• Add usb-serial-for-android dependency to android/app/build.gradle.kts
• Create Kotlin UsbSerialBridge class that:
  • Enumerates USB serial devices via Android USB Host API
  • Requests user permission for USB device access
  • Opens a serial connection (baud rate configurable)
  • Exposes a write method callable from Python via Chaquopy
• Create Python AndroidSerialProvider in server/src/ledgrab/core/devices/ that:
  • Replaces pyserial on Android (which can't access USB ports)
  • Calls UsbSerialBridge via Chaquopy to send LED data
  • Registers as an alternative serial transport when is_android() is True
• Add USB device permission dialog to MainActivity (auto-triggered on device connect)
• Test with common USB-to-serial chips: CH340, CP2102, FTDI
• Document supported USB LED controllers in README

Android App — Known Issues

Issues discovered during first end-to-end test on emulator (2026-04-14):

• Stop/Start capture produces no frames on second try. First capture works, but after tapping Stop and Start again, no frames reach the Python pipeline. Likely causes:
  • Global MediaProjection engine state (_active, _frame_queue) not reset on service restart
  • ScreenCapture listener not properly detached from ImageReader on stop
  • Python uvicorn port reuse issue when server restarts
• Web UI tabs show persistent spinners on Dashboard, Automation, Sources, Integrations, Graph tabs (Targets tab works). All API calls return 200 OK, no console errors. Probably waiting on a specific WebSocket event that never fires when no output targets are configured/active.
• Dead keyboard/IME handling — password inputs lack autocomplete attributes (minor accessibility warning in browser console)

Performance Metrics Abstraction

The codebase has direct psutil.* calls scattered across api/routes/system.py and core/processing/metrics_history.py, with ad-hoc if psutil is not None guards sprinkled in to support Android. This couples Android platform handling to every call site.

• Refactor: introduce MetricsProvider protocol in utils/metrics.py with methods like cpu_percent(), memory_info(), process_info()
• Implement PsutilMetricsProvider (desktop) and NullMetricsProvider (fallback when psutil missing)
• Later: AndroidMetricsProvider reading from /proc (see section below)
• Replace all direct psutil.* calls with the provider; only one factory location knows about psutil availability

Android Performance Metrics

Currently psutil (used for CPU/RAM monitoring in the web UI) is not available on Android via Chaquopy. Metrics calls are guarded with if psutil is not None so they return no data on Android.

• Implement Android-native metrics collection:
  • CPU usage via /proc/self/stat + /proc/stat parsing (no psutil needed)
  • RAM usage via /proc/meminfo or ActivityManager.getMemoryInfo() through Chaquopy bridge
  • App-specific memory via Debug.getMemoryInfo() (Kotlin → Python)
• Create AndroidMetricsProvider in server/src/ledgrab/utils/ that implements the same interface as the psutil-based provider
• Wire into existing metrics endpoints (/api/v1/system/metrics) with platform detection
• Consider: device battery/temperature readings for TV boxes (some have thermal throttling)
• Optional: GPU usage via /sys/class/kgsl/kgsl-3d0/gpubusy on Adreno, Mali-specific paths for Mali GPUs