alexei.dolgolyov
928d626620
Drops the direct pyserial imports from espnow_client/espnow_provider in favor of open_transport/list_serial_ports/port_exists. The gateway protocol is write-only, so no read() extension was needed. ESP-NOW gateways are now reachable via usb:VID:PID URLs on Android.
78 lines
8.1 KiB
Markdown
78 lines
8.1 KiB
Markdown
# LedGrab TODO
|
|
|
|
## Android — Restore Multi-ABI Wheels
|
|
|
|
During emulator testing, we switched the build to **x86 only** (see `android/app/build.gradle.kts` `abiFilters`) to avoid having to keep the arm64-v8a / x86_64 pydantic-core wheels current. Before shipping, restore all three ABIs:
|
|
|
|
- [x] Rebuild `pydantic-core` wheels for all three ABIs with the current SOABI + libpython linking settings (`android/build-scripts/build-pydantic-core.sh` — now supports `arm64`, `x86_64`, `x86` args; defaults to all three).
|
|
- [x] Verify wheels: all three now list `libpython3.11.so` in `NEEDED` (`llvm-readelf -d`), automated in the build script.
|
|
- [x] Restored `abiFilters += listOf("arm64-v8a", "x86_64", "x86")` in `build.gradle.kts`. Multi-ABI debug APK builds cleanly (~99 MB).
|
|
- [ ] Re-test on real ARM64 Android TV hardware (still pending — only emulator-verified build).
|
|
|
|
Build cache + scripts live in `android/build-scripts/` and `android/.build-cache/` (junction host + sysconfigdata for each ABI).
|
|
|
|
## Android CI Pipeline
|
|
|
|
Build the Android APK automatically on push/tag.
|
|
|
|
- [x] Generate Gradle wrapper (`gradlew`) and commit it
|
|
- [x] Create CI workflow (`.gitea/workflows/build-android.yml`)
|
|
- JDK 17 + Android SDK + NDK setup
|
|
- Python 3.11 for Chaquopy build
|
|
- Recreate the directory junction via `ln -s` on Linux CI
|
|
- `./gradlew assembleDebug` on master push, `assembleRelease` on `v*` tags (if signing secrets set)
|
|
- Uploads APK as CI artifact; attaches to Gitea release on tag push
|
|
- [x] Commit pre-built pydantic-core wheels to `android/wheels/` (arm64, x86, x86_64)
|
|
- [x] APK signing for release builds — conditional signing config reads keystore from env vars (`ANDROID_KEYSTORE_PATH/_PASSWORD/_ALIAS/_KEY_PASSWORD`), falls back to debug signing locally
|
|
- [ ] Provision a real keystore and add the four CI secrets:
|
|
- `ANDROID_KEYSTORE_BASE64` (base64-encoded .jks)
|
|
- `ANDROID_KEYSTORE_PASSWORD`
|
|
- `ANDROID_KEY_ALIAS`
|
|
- `ANDROID_KEY_PASSWORD`
|
|
- [ ] Add `LedGrab-{tag}-android-release.apk` row to the release description table in `.gitea/workflows/release.yml` → `create-release` job
|
|
- [ ] Verify the CI workflow passes end-to-end with the now-restored multi-ABI build (larger APK, longer Android build step)
|
|
|
|
## Android Root Capture (No Permission Dialog, No System Indicator)
|
|
|
|
MediaProjection shows a mandatory system overlay/indicator while capturing — unavoidable on stock Android. Many cheap Android TV boxes ship pre-rooted, so an alternative root-only path gives much better UX.
|
|
|
|
- [x] Root detection — `Root.kt` checks common `su` binary paths and, on demand, runs `su -c id` to actually prove UID 0. First call triggers Magisk's grant dialog; grant is cached per session. Exposed to Python via Chaquopy.
|
|
- [x] `RootScreenrecord.kt` — spawns `su -c screenrecord --output-format=h264 --size=WxH -`, feeds the H.264 stdout through a MediaCodec decoder whose output Surface is wired into an ImageReader (RGBA_8888, row-stride-aware). Decoded frames reach the Python pipeline via `PythonBridge.pushRootFrame`.
|
|
- [x] Python-side `RootScreenrecordEngine` (`core/capture_engines/root_screenrecord_engine.py`) mirrors `MediaProjectionEngine` with `ENGINE_PRIORITY=110` (> MediaProjection's 100) so the factory picks it automatically when available.
|
|
- [x] `MainActivity` tries `Root.requestGrant()` before launching the MediaProjection consent flow — on rooted devices the consent dialog is skipped entirely. `CaptureService` has a `createRootIntent()` entry point that bypasses the MediaProjection path.
|
|
- [x] Fallback: if `Root.requestGrant()` returns false (no root, user denied, or `su` timeout) the existing MediaProjection flow runs unchanged.
|
|
- [ ] Real-hardware test pending — need to verify on the user's Magisk'd TV box that: (1) grant dialog appears once, (2) frames actually flow through MediaCodec without the Android 14 capture indicator showing, (3) stop/start cycle terminates the `su` process cleanly.
|
|
- [WONTDO] `SurfaceControl.screenshot()` via reflection — renamed/moved across API 28/29/30/33, hidden-API blocklist varies by release, even rooted apps hit it; days of maintenance for a marginal latency win over the screenrecord path. Not worth it.
|
|
- [WONTDO] `adb screencap` fallback — full-PNG-per-frame pipeline is slower than MediaProjection, no value as a last resort.
|
|
|
|
Known projects using the screenrecord approach for reference: scrcpy (over ADB), scrcpy-hidden-api, shizuku.
|
|
|
|
## Android USB Serial Support
|
|
|
|
Drive USB LED controllers (APA102, WS2812) connected directly to the Android TV box via USB-to-serial adapters.
|
|
|
|
- [x] Added `com.github.mik3y:usb-serial-for-android:3.8.1` (via JitPack) to `android/app/build.gradle.kts`.
|
|
- [x] Kotlin `UsbSerialBridge` singleton (`android/app/src/main/java/com/ledgrab/android/UsbSerialBridge.kt`) — exposes `listDevices()`, `open(vid, pid, serial, baud)`, `write(handle, ByteArray)`, `close(handle)`. Permission request fires automatically from `open()` when the user hasn't granted access yet. Handles are opaque integers, port map is synchronized, so Python threads can share one bridge.
|
|
- [x] Python `AndroidSerialTransport` in `server/src/ledgrab/core/devices/android_serial_transport.py` drives the bridge through Chaquopy. `SerialTransport` Protocol + `PySerialTransport` + `list_serial_ports()` factory live in `serial_transport.py`; `AdalightClient` and `SerialDeviceProvider` now go through the abstraction instead of importing `pyserial` directly.
|
|
- [x] URL scheme extended: `usb:VID:PID[:serial][@baud]` on Android alongside the existing `COM3[:baud]` / `/dev/ttyUSB0[:baud]` desktop paths.
|
|
- [x] App initializes the bridge on startup (`LedGrabApp.onCreate` → `UsbSerialBridge.init(this)`); manifest declares `uses-feature android.hardware.usb.host`.
|
|
- [ ] Real-device test pending — no USB-serial hardware on dev machine. Need to verify on a TV box with CH340, CP2102, or FTDI adapter.
|
|
- [ ] Document supported USB LED controllers in README (once real-device test passes).
|
|
- [ ] Optional: auto-launch the app when a known USB-serial adapter is plugged in (intent-filter on `USB_DEVICE_ATTACHED` + `res/xml/device_filter.xml`). Skipped in v1 — users can just open LedGrab and hit "Discover".
|
|
- [x] ESP-NOW client (`espnow_client.py` / `espnow_provider.py`) now routes through `SerialTransport` — `open_transport()` for the gateway serial link, `list_serial_ports()` + `port_exists()` for discovery/validation. Works transparently with `usb:VID:PID` URLs on Android. (Gateway protocol is write-only, so no `read()` extension was needed after all.)
|
|
|
|
## Performance Metrics Abstraction
|
|
|
|
- [x] `MetricsProvider` protocol + dataclass DTOs (`MemorySnapshot`, `ProcessSnapshot`) live in `server/src/ledgrab/utils/metrics/types.py`. Each provider has its own module: `psutil_provider.py`, `null_provider.py`, `android_provider.py`.
|
|
- [x] Factory `get_metrics_provider()` in `utils/metrics/__init__.py` selects Android → psutil → Null. `psutil` import is now confined to one place.
|
|
- [x] `api/routes/system.py` and `core/processing/metrics_history.py` use the provider; no more `if psutil is not None` guards in the hot paths.
|
|
- [x] Android `/proc`-backed provider implemented (`/proc/stat`, `/proc/meminfo`, `/proc/self/stat`, `/proc/self/status`). Carries previous-sample state for delta-based CPU%; degrades to zeros if any `/proc` file is locked down. 12 unit tests cover both desktop and Android paths.
|
|
|
|
## Android Performance Metrics — Future Enhancements
|
|
|
|
Beyond the `/proc`-based AndroidMetricsProvider that's now in place:
|
|
|
|
- [x] Device battery + thermal-zone readings (`/sys/class/power_supply/battery/{capacity,temp}`, `/sys/class/thermal/thermal_zone*/temp` filtered by zone type). Surfaced through `MetricsProvider.thermals()`, `PerformanceResponse.{cpu_temp_c,battery_percent,battery_temp_c}`, the metrics-history snapshot, and a new dashboard temperature chart that hides itself when the backend reports null. GPU card now hides (no "unavailable" placeholder) when no GPU is present.
|
|
- [WONTDO] Optional: app-specific memory via `Debug.getMemoryInfo()` through a Kotlin → Python Chaquopy bridge (more accurate than `VmRSS` for split-app-process accounting)
|
|
- [WONTDO] Optional: GPU usage via `/sys/class/kgsl/kgsl-3d0/gpubusy` on Adreno, Mali-specific paths for Mali GPUs
|