feat(android): foreground-app automation condition
Make the existing Application automation rule (foreground app -> activate scene) work on the Android-TV build. A Kotlin ForegroundAppBridge reads the foreground app via UsageStatsManager and lists launchable apps via LauncherApps; PlatformDetector bridges it in (ahead of the Windows-only ctypes guard) so the existing AutomationEngine / ApplicationRule / storage / deactivation modes are unchanged. New /system/installed-apps + /system/info endpoints feed an app picker that stores package names (vs process names on desktop); on Android the editor hides the match-type selector since the foreground app is the only obtainable signal. PACKAGE_USAGE_STATS is granted via an on-device button + a web-UI banner (no blanket prompt at capture start); detection degrades gracefully until granted. Zero new Python/Gradle deps (UsageStatsManager + LauncherApps are in-platform; matching only string-compares the package name, so no QUERY_ALL_PACKAGES). assembleDebug + 1897 pytest + ruff + tsc + npm build all green; independent final review (0 blockers) + security review (no critical issues).
This commit is contained in:
@@ -49,7 +49,7 @@ Python receiver engine mirroring that pattern.**
|
||||
| Webcam capture | OpenCV | ✅ Camera2 + on-demand bridge (`AndroidCameraEngine`) | No (implemented) |
|
||||
| GPU monitoring | NVML | ❌ no NVIDIA GPU | Marginal |
|
||||
| Capture from *another* Android phone | scrcpy/ADB | ❌ | Skip (redundant) |
|
||||
| Automation: window/process conditions | Windows ctypes | ❌ sandboxed | Partial |
|
||||
| Automation: foreground-app condition | Windows ctypes (running/topmost/fullscreen) | ✅ foreground app via UsageStatsManager (`ForegroundAppBridge`) | No (implemented) |
|
||||
| Monitor names / multi-display | WMI / generic | Single built-in display | Low value |
|
||||
|
||||
---
|
||||
@@ -127,15 +127,30 @@ Python receiver engine mirroring that pattern.**
|
||||
- CPU/RAM/battery/thermal are **already** covered by `AndroidMetricsProvider`. A best-effort
|
||||
GPU-load reader could be added to that provider, but reliability is poor and value is low.
|
||||
|
||||
### 🪟 Automation: window/process conditions — **PARTIAL**
|
||||
### 🪟 Automation: foreground-app condition — **IMPLEMENTED** ✅ (shipped)
|
||||
|
||||
- Android forbids full window/process enumeration (`getRunningTasks` restricted since API 21+).
|
||||
- **Obtainable:** the *current foreground app package* via `UsageStatsManager` (needs the
|
||||
`PACKAGE_USAGE_STATS` special access) or an `AccessibilityService`.
|
||||
- So "when <app> is in the foreground → scene X" is feasible (mirrors
|
||||
`automations/platform_detector.py`, which currently returns empty off-Windows); full
|
||||
window-title matching is **not**. **Effort:** moderate. **Value:** moderate (per-app scenes
|
||||
on a TV box).
|
||||
- Android forbids full window/process enumeration (`getRunningTasks` restricted since API 21+),
|
||||
but the *current foreground app package* is obtainable via `UsageStatsManager` (needs the
|
||||
`PACKAGE_USAGE_STATS` special access).
|
||||
- **Path:** a Kotlin `ForegroundAppBridge` (UsageStatsManager `queryEvents` over a ~10s trailing
|
||||
window + `LauncherApps` for the picker + an `AppOpsManager` access check) bridged into
|
||||
`automations/platform_detector.py` via the guarded-`jclass` pattern, ahead of the Windows-only
|
||||
ctypes path. The existing `ApplicationRule` / `AutomationEngine` / storage / deactivation modes
|
||||
are unchanged — only the detection + the picker's data source were filled in. **No new Python
|
||||
or Gradle deps** (UsageStatsManager + LauncherApps are in-platform; matching only string-compares
|
||||
the package name, so no `QUERY_ALL_PACKAGES` / package visibility is needed).
|
||||
- **UI:** the automation editor's app picker lists launchable apps by human label (storing the
|
||||
package name) via a new `GET /api/v1/system/installed-apps`; on Android the match-type selector
|
||||
is hidden and `match_type` is forced to `topmost` (the only obtainable signal), with a
|
||||
cross-platform value caveat — `apps` are **package names** on Android (`com.netflix.mediaclient`)
|
||||
vs **process names** on Windows (`chrome.exe`), so rules are not portable across platforms.
|
||||
- **Permission:** `PACKAGE_USAGE_STATS` is a special access (Settings deep-link via
|
||||
`ACTION_USAGE_ACCESS_SETTINGS`); the device shows a "Grant usage access" button when missing,
|
||||
and the web-UI rule editor shows a banner (driven by `/system/info`'s `usage_access_granted`).
|
||||
No blanket prompt at capture start. Detection degrades gracefully (rule never matches, warned
|
||||
once) until access is granted. **Effort:** moderate. **Value:** moderate (per-app scenes on a
|
||||
TV box). Full window-title matching remains out of scope (Android does not expose it).
|
||||
- 📄 **See `android-foreground-app-automation-plan.md`** for the full implementation notes.
|
||||
|
||||
### 📱 Capture from *another* Android phone (scrcpy/ADB) — **SKIP**
|
||||
|
||||
@@ -157,17 +172,17 @@ Python receiver engine mirroring that pattern.**
|
||||
| 1 | Notification capture | Moderate | High | None | **✅ Implemented** |
|
||||
| 2 | Audio capture | Moderate | High | None | **✅ Implemented** |
|
||||
| 4 | Webcam capture (Camera2) | Moderate | Low | None | **✅ Implemented** |
|
||||
| 3 | Automation: foreground-app condition | Moderate | Moderate | None | Idea (only remaining) |
|
||||
| 3 | Automation: foreground-app condition | Moderate | Moderate | None | **✅ Implemented** |
|
||||
| — | GPU load (vendor sysfs) | Low–Med | Low | None | Not recommended |
|
||||
| — | Capture from another phone | — | — | — | Won't do |
|
||||
| — | Multi-display / monitor names | Low | Low | None | Not recommended |
|
||||
|
||||
**Status:** notifications, audio, **and webcam** are all shipped — each reuses existing
|
||||
infrastructure (bridge pattern, the MediaProjection consent token / process-global
|
||||
`Python.getInstance()`, the capture/audio/notification pipelines) and adds **zero** Python
|
||||
dependencies, so none risks the Chaquopy `--no-deps` build constraint documented in
|
||||
`CLAUDE.md`. The only remaining idea is the **foreground-app automation condition** (moderate
|
||||
value); GPU load, another-phone capture, and multi-display remain not-recommended / won't-do.
|
||||
**Status:** notifications, audio, webcam, **and the foreground-app automation condition** are all
|
||||
shipped — each reuses existing infrastructure (the Kotlin↔Python bridge pattern, the
|
||||
MediaProjection consent token / process-global `Python.getInstance()`, the
|
||||
capture/audio/notification/automation pipelines) and adds **zero** Python dependencies, so none
|
||||
risks the Chaquopy `--no-deps` build constraint documented in `CLAUDE.md`. No prioritized ideas
|
||||
remain; GPU load, another-phone capture, and multi-display remain not-recommended / won't-do.
|
||||
|
||||
## Cross-cutting notes
|
||||
|
||||
|
||||
Reference in New Issue
Block a user