Merge feature/android-foreground-app-automation: Android foreground-app automation condition

Foreground-app -> scene automation on the Android-TV build via a Kotlin
ForegroundAppBridge (UsageStatsManager) bridged into PlatformDetector ahead of the
Windows-only ctypes path; LauncherApps-backed app picker (/system/installed-apps) +
platform signal (/system/info); PACKAGE_USAGE_STATS special-access UX (on-device
button + web-UI banner, graceful degradation). Reuses the existing automation engine
unchanged; zero new deps. assembleDebug + 1897 pytest + ruff + tsc + build green;
independent final + security reviews pass.
This commit is contained in:
2026-06-02 14:57:45 +03:00
24 changed files with 1044 additions and 63 deletions
@@ -0,0 +1,94 @@
# Android foreground-app automation condition — implementation notes
> Status: implemented on `feature/android-foreground-app-automation`. Last updated 2026-06-02.
## What & why
The desktop build has an **Application** automation rule (`ApplicationRule`): activate a scene
when given apps are running / foreground / fullscreen. It was already wired end-to-end on
Android (engine, storage, API, editor) but **silently never fired**, because the two
Windows-only ctypes paths return empty off-Windows:
1. **Detection**`PlatformDetector._get_topmost_process_sync()` (and the running/fullscreen
variants) returned `(None, False)` / `set()` on Android.
2. **The app picker** — populated from `GET /api/v1/system/processes`
`get_running_processes()`, also empty on Android, so users couldn't even choose an app.
This feature fills both holes using in-platform Android APIs and the established Kotlin↔Python
bridge pattern. **Zero new Python or Gradle dependencies.**
## Design decision: one implicit "foreground" mode on Android
Android exposes exactly one obtainable signal — the **current foreground app package**. The
desktop rule's four match types (`running` / `topmost` / `fullscreen` / `topmost_fullscreen`)
are either unobtainable (`running``getRunningTasks` is restricted) or identical (a foreground
TV app effectively *is* fullscreen). So on Android:
- The editor **hides the match-type selector** and the collector forces `match_type="topmost"`.
- `_get_topmost_process_sync()` returns `(package, True)`; the running/fullscreen detectors
return the foreground app as a best-effort single-element set so legacy rules still behave.
This avoided touching the existing plain `<select>` (forbidden for new UI) and removed a
misleading 4-way choice — a simplification surfaced by the pre-implementation plan review.
## Detection — `ForegroundAppBridge` (Kotlin) ↔ `platform_detector.py`
`android/app/src/main/java/com/ledgrab/android/ForegroundAppBridge.kt` (an `object` singleton,
mirroring `CameraBridge`, context bound in `LedGrabApp.onCreate`):
- `getForegroundPackage()``UsageStatsManager.queryEvents(now - 10s, now)`, returns the package
of the most recent `MOVE_TO_FOREGROUND` / `ACTIVITY_RESUMED` event (the two constants share a
value; the ~10s window absorbs event lag against the ~1s automation tick). `queryEvents` is the
right call — `queryUsageStats` gives aggregate durations, not "current app".
- `hasUsageAccess()``AppOpsManager` `OPSTR_GET_USAGE_STATS` check (`unsafeCheckOpNoThrow` on
API 29+, `checkOpNoThrow` below).
- `listLaunchableApps()``LauncherApps.getActivityList` → JSON `[{package,label}]` for the
picker. The sanctioned launchable-app API; **no `QUERY_ALL_PACKAGES`**.
`server/src/ledgrab/core/automations/platform_detector.py`:
- Module-level guarded wrappers `get_foreground_package()` / `has_usage_access()` /
`list_installed_apps()` resolve `jclass("com.ledgrab.android.ForegroundAppBridge").INSTANCE`
lazily (never at import — the module loads on desktop CI). These are the **test monkeypatch
surface**, mirroring `android_camera_engine`.
- The `is_android()` branch is placed **ahead of** the import-time `if not _IS_WINDOWS:`
early-return in each detector — the critical fix from plan review (a naive wiring would no-op
behind the Windows guard yet still pass tests). The Windows ctypes path is unchanged
(regression-tested).
- A one-time `logger.warning` fires when Usage Access is missing.
## App picker — `/system/installed-apps` + platform signal
- `GET /api/v1/system/installed-apps``{apps:[{package,label}], count}` (empty off-Android).
- `GET /api/v1/system/info``{is_android, app_match_kind, usage_access_granted}` — the editor
reads it to pick the app source + matching semantics and to show the Usage-Access banner.
- Frontend: the command-palette picker (`core/process-picker.ts`) gained label→value support; a
new `AppPalette` shows the human label and inserts the package name. On Android the app-rule
editor uses it (`attachAppPicker`) instead of the process picker, plus a package-name hint and
the Usage-Access banner.
## Value semantics (no migration)
`ApplicationRule.apps` are **package names** on Android (`com.netflix.mediaclient`) vs **process
names** on Windows (`chrome.exe`). Same field, same matching code — **no storage migration**
but rules are **not portable across platforms**. Documented in the model/schema docstrings and a
user-facing editor hint.
## Permission UX
`PACKAGE_USAGE_STATS` is a special access (can't be granted at runtime):
- Manifest declares it with `tools:ignore="ProtectedPermissions"`.
- MainActivity shows a passive **"Grant usage access"** button (opens
`ACTION_USAGE_ACCESS_SETTINGS`, with a generic-Settings fallback) only while access is missing.
**No blanket prompt at capture start** — most users have no foreground-app rule.
- The web-UI rule editor shows a banner when an Android Application rule lacks access.
## Limitations
- Foreground-app only; no full window-title or arbitrary process enumeration on Android.
- Detection rides the existing ~1s automation poll; `queryEvents` can lag a few seconds.
- Rules authored on desktop don't match on Android and vice-versa (package vs process names).
- The on-device "Grant usage access" button currently shows whenever access is missing (not
gated on whether an Android Application rule exists), to avoid Activity↔server coupling; the
web-UI banner provides the contextual guidance.
+31 -16
View File
@@ -49,7 +49,7 @@ Python receiver engine mirroring that pattern.**
| Webcam capture | OpenCV | ✅ Camera2 + on-demand bridge (`AndroidCameraEngine`) | No (implemented) |
| GPU monitoring | NVML | ❌ no NVIDIA GPU | Marginal |
| Capture from *another* Android phone | scrcpy/ADB | ❌ | Skip (redundant) |
| Automation: window/process conditions | Windows ctypes | ❌ sandboxed | Partial |
| Automation: foreground-app condition | Windows ctypes (running/topmost/fullscreen) | ✅ foreground app via UsageStatsManager (`ForegroundAppBridge`) | No (implemented) |
| Monitor names / multi-display | WMI / generic | Single built-in display | Low value |
---
@@ -127,15 +127,30 @@ Python receiver engine mirroring that pattern.**
- CPU/RAM/battery/thermal are **already** covered by `AndroidMetricsProvider`. A best-effort
GPU-load reader could be added to that provider, but reliability is poor and value is low.
### 🪟 Automation: window/process conditions — **PARTIAL**
### 🪟 Automation: foreground-app condition — **IMPLEMENTED** ✅ (shipped)
- Android forbids full window/process enumeration (`getRunningTasks` restricted since API 21+).
- **Obtainable:** the *current foreground app package* via `UsageStatsManager` (needs the
`PACKAGE_USAGE_STATS` special access) or an `AccessibilityService`.
- So "when <app> is in the foreground → scene X" is feasible (mirrors
`automations/platform_detector.py`, which currently returns empty off-Windows); full
window-title matching is **not**. **Effort:** moderate. **Value:** moderate (per-app scenes
on a TV box).
- Android forbids full window/process enumeration (`getRunningTasks` restricted since API 21+),
but the *current foreground app package* is obtainable via `UsageStatsManager` (needs the
`PACKAGE_USAGE_STATS` special access).
- **Path:** a Kotlin `ForegroundAppBridge` (UsageStatsManager `queryEvents` over a ~10s trailing
window + `LauncherApps` for the picker + an `AppOpsManager` access check) bridged into
`automations/platform_detector.py` via the guarded-`jclass` pattern, ahead of the Windows-only
ctypes path. The existing `ApplicationRule` / `AutomationEngine` / storage / deactivation modes
are unchanged — only the detection + the picker's data source were filled in. **No new Python
or Gradle deps** (UsageStatsManager + LauncherApps are in-platform; matching only string-compares
the package name, so no `QUERY_ALL_PACKAGES` / package visibility is needed).
- **UI:** the automation editor's app picker lists launchable apps by human label (storing the
package name) via a new `GET /api/v1/system/installed-apps`; on Android the match-type selector
is hidden and `match_type` is forced to `topmost` (the only obtainable signal), with a
cross-platform value caveat — `apps` are **package names** on Android (`com.netflix.mediaclient`)
vs **process names** on Windows (`chrome.exe`), so rules are not portable across platforms.
- **Permission:** `PACKAGE_USAGE_STATS` is a special access (Settings deep-link via
`ACTION_USAGE_ACCESS_SETTINGS`); the device shows a "Grant usage access" button when missing,
and the web-UI rule editor shows a banner (driven by `/system/info`'s `usage_access_granted`).
No blanket prompt at capture start. Detection degrades gracefully (rule never matches, warned
once) until access is granted. **Effort:** moderate. **Value:** moderate (per-app scenes on a
TV box). Full window-title matching remains out of scope (Android does not expose it).
- 📄 **See `android-foreground-app-automation-plan.md`** for the full implementation notes.
### 📱 Capture from *another* Android phone (scrcpy/ADB) — **SKIP**
@@ -157,17 +172,17 @@ Python receiver engine mirroring that pattern.**
| 1 | Notification capture | Moderate | High | None | **✅ Implemented** |
| 2 | Audio capture | Moderate | High | None | **✅ Implemented** |
| 4 | Webcam capture (Camera2) | Moderate | Low | None | **✅ Implemented** |
| 3 | Automation: foreground-app condition | Moderate | Moderate | None | Idea (only remaining) |
| 3 | Automation: foreground-app condition | Moderate | Moderate | None | **✅ Implemented** |
| — | GPU load (vendor sysfs) | LowMed | Low | None | Not recommended |
| — | Capture from another phone | — | — | — | Won't do |
| — | Multi-display / monitor names | Low | Low | None | Not recommended |
**Status:** notifications, audio, **and webcam** are all shipped — each reuses existing
infrastructure (bridge pattern, the MediaProjection consent token / process-global
`Python.getInstance()`, the capture/audio/notification pipelines) and adds **zero** Python
dependencies, so none risks the Chaquopy `--no-deps` build constraint documented in
`CLAUDE.md`. The only remaining idea is the **foreground-app automation condition** (moderate
value); GPU load, another-phone capture, and multi-display remain not-recommended / won't-do.
**Status:** notifications, audio, webcam, **and the foreground-app automation condition** are all
shipped — each reuses existing infrastructure (the Kotlin↔Python bridge pattern, the
MediaProjection consent token / process-global `Python.getInstance()`, the
capture/audio/notification/automation pipelines) and adds **zero** Python dependencies, so none
risks the Chaquopy `--no-deps` build constraint documented in `CLAUDE.md`. No prioritized ideas
remain; GPU load, another-phone capture, and multi-display remain not-recommended / won't-do.
## Cross-cutting notes
+1 -1
View File
@@ -115,7 +115,7 @@ LedGrab runs as a desktop / server application:
| Notification capture | WinRT | dbus (Linux) | NotificationListenerService |
| Monitor names | Friendly names (WMI) | Generic ("Display 0") | Single built-in display |
| LED transports | Network, USB-serial, BLE | Network, USB-serial, BLE | Network, USB-serial (Android driver), BLE (Android bridge) |
| Automation: window/process conditions | Supported | Partial | |
| Automation: window/process conditions | Supported | Partial | Foreground-app condition (UsageStatsManager) |
## Requirements
+12
View File
@@ -65,6 +65,18 @@
service start. -->
<uses-permission android:name="android.permission.CAMERA" />
<!-- PACKAGE_USAGE_STATS — read the foreground app for the "Application"
automation rule (foreground app -> activate scene) via UsageStatsManager.
A special-access permission: it can't be granted at runtime; the user
toggles it under Settings > Usage access (opened from MainActivity).
tools:ignore="ProtectedPermissions" silences the build warning that this
is a system/signature-level permission — it is honoured as a user-grantable
special access. NO QUERY_ALL_PACKAGES is needed: matching only compares the
foreground package NAME, and the app picker uses LauncherApps. -->
<uses-permission
android:name="android.permission.PACKAGE_USAGE_STATS"
tools:ignore="ProtectedPermissions" />
<!-- Autostart on boot — BootReceiver spawns CaptureService in root
mode so capture resumes without the user touching the remote. -->
<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED" />
@@ -0,0 +1,154 @@
package com.ledgrab.android
import android.app.AppOpsManager
import android.app.usage.UsageEvents
import android.app.usage.UsageStatsManager
import android.content.Context
import android.content.pm.LauncherApps
import android.os.Build
import android.os.Process
import android.util.Log
import org.json.JSONArray
import org.json.JSONObject
/**
* Foreground-app + installed-app bridge exposed to the Python server via Chaquopy.
*
* Backs the Android implementation of the "Application" automation rule
* (foreground app -> activate scene). Desktop detects the foreground process via
* Win32 ctypes in ``platform_detector.py``; Android has no such API, so this
* bridge wraps two in-platform services into synchronous calls a Python thread
* can invoke (Chaquopy proxy threads are real OS threads):
*
* - [getForegroundPackage] via [UsageStatsManager] (needs PACKAGE_USAGE_STATS,
* a special-access permission granted from Settings — see MainActivity).
* - [listLaunchableApps] via [LauncherApps] for the automation editor's app
* picker (no QUERY_ALL_PACKAGES needed — getActivityList is the sanctioned
* launchable-app enumeration API).
* - [hasUsageAccess] so the server / UI can detect the missing grant.
*
* Detection only ever string-compares the foreground *package name*, so no label
* resolution / package visibility is required at match time.
*
* Python callers access the singleton via
* `jclass("com.ledgrab.android.ForegroundAppBridge").INSTANCE` — see
* `server/src/ledgrab/core/automations/platform_detector.py`.
*/
object ForegroundAppBridge {
private const val TAG = "ForegroundAppBridge"
// Trailing window for queryEvents. queryEvents reports discrete foreground
// transitions (not "current app"), and events can lag a few seconds, so we
// look back far enough to reliably catch the latest MOVE_TO_FOREGROUND while
// staying recent enough not to report a stale app on the ~1s automation tick.
private const val WINDOW_MS = 10_000L
@Volatile private var appContext: Context? = null
/** Called once from [LedGrabApp.onCreate] to bind the application context. */
@JvmStatic
fun init(context: Context) {
appContext = context.applicationContext
}
/**
* Package name of the most recently foregrounded app, or null when none is
* found in the trailing window, Usage Access is not granted, or on any error.
* Never throws across the JNI boundary.
*/
@JvmStatic
fun getForegroundPackage(): String? {
val ctx = appContext ?: run {
Log.w(TAG, "getForegroundPackage: context not bound (init not called)")
return null
}
return try {
val usm = ctx.getSystemService(Context.USAGE_STATS_SERVICE) as? UsageStatsManager
?: return null
val end = System.currentTimeMillis()
val events = usm.queryEvents(end - WINDOW_MS, end)
val event = UsageEvents.Event()
var latestPkg: String? = null
var latestTs = Long.MIN_VALUE
while (events.hasNextEvent()) {
events.getNextEvent(event)
// ACTIVITY_RESUMED (API 29+) shares the value of the legacy
// MOVE_TO_FOREGROUND constant, so the single check covers both.
// >= (not >) so that on an exact-timestamp tie the later-iterated
// event wins — events arrive chronologically, so that is the most
// recent foreground transition.
if (event.eventType == UsageEvents.Event.MOVE_TO_FOREGROUND &&
event.timeStamp >= latestTs
) {
latestTs = event.timeStamp
latestPkg = event.packageName
}
}
latestPkg
} catch (e: Exception) {
// SecurityException when access is missing, plus any service error.
Log.w(TAG, "getForegroundPackage failed: ${e.message}")
null
}
}
/** Whether the user has granted Usage Access (PACKAGE_USAGE_STATS) to this app. */
@JvmStatic
fun hasUsageAccess(): Boolean {
val ctx = appContext ?: return false
return try {
val appOps = ctx.getSystemService(Context.APP_OPS_SERVICE) as? AppOpsManager
?: return false
val mode = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
appOps.unsafeCheckOpNoThrow(
AppOpsManager.OPSTR_GET_USAGE_STATS, Process.myUid(), ctx.packageName,
)
} else {
@Suppress("DEPRECATION")
appOps.checkOpNoThrow(
AppOpsManager.OPSTR_GET_USAGE_STATS, Process.myUid(), ctx.packageName,
)
}
mode == AppOpsManager.MODE_ALLOWED
} catch (e: Exception) {
Log.w(TAG, "hasUsageAccess failed: ${e.message}")
false
}
}
/**
* Launchable apps as a JSON array string the Python server parses:
* `[{"package":"com.netflix.mediaclient","label":"Netflix"}, ...]`
*
* Uses [LauncherApps.getActivityList] (launcher + leanback launchables) —
* no QUERY_ALL_PACKAGES. De-duplicated by package, sorted by label.
* Returns `[]` on any error.
*/
@JvmStatic
fun listLaunchableApps(): String {
val arr = JSONArray()
val ctx = appContext ?: run {
Log.w(TAG, "listLaunchableApps: context not bound (init not called)")
return arr.toString()
}
try {
val launcher = ctx.getSystemService(Context.LAUNCHER_APPS_SERVICE) as? LauncherApps
?: return arr.toString()
val seen = HashSet<String>()
val items = ArrayList<Pair<String, String>>()
for (info in launcher.getActivityList(null, Process.myUserHandle())) {
val pkg = info.applicationInfo?.packageName ?: continue
if (!seen.add(pkg)) continue
val label = info.label?.toString().takeUnless { it.isNullOrBlank() } ?: pkg
items.add(pkg to label)
}
items.sortBy { it.second.lowercase() }
for ((pkg, label) in items) {
arr.put(JSONObject().put("package", pkg).put("label", label))
}
} catch (e: Exception) {
Log.w(TAG, "listLaunchableApps failed: ${e.message}")
}
return arr.toString()
}
}
@@ -54,6 +54,10 @@ class LedGrabApp : Application() {
// Bind application context for the camera bridge so Python can
// enumerate cameras and open them on demand (webcam capture).
CameraBridge.init(this)
// Bind application context for the foreground-app bridge so Python can
// detect the foreground app (Application automation rule) and list
// launchable apps for the editor's picker.
ForegroundAppBridge.init(this)
// Pre-warm the API key on a background thread. First-launch
// generation does a SharedPreferences.commit() (synchronous
@@ -69,6 +69,7 @@ class MainActivity : Activity() {
private lateinit var autostartCheck: CheckBox
private lateinit var autostartPrefs: AutostartPrefs
private lateinit var grantNotificationButton: Button
private lateinit var grantUsageAccessButton: Button
// Running-state views (lazy-inflated via ViewStub).
private lateinit var runningPanelStub: ViewStub
@@ -113,6 +114,7 @@ class MainActivity : Activity() {
versionText = findViewById(R.id.version_text)
autostartCheck = findViewById(R.id.autostart_check)
grantNotificationButton = findViewById(R.id.grant_notification_button)
grantUsageAccessButton = findViewById(R.id.grant_usage_access_button)
val versionName = packageManager.getPackageInfo(packageName, 0).versionName
versionText.text = getString(R.string.version_prefix, versionName ?: "?")
@@ -134,9 +136,10 @@ class MainActivity : Activity() {
}
grantNotificationButton.setOnClickListener { openNotificationListenerSettings() }
grantUsageAccessButton.setOnClickListener { openUsageAccessSettings() }
toggleButton.setOnClickListener { startCapture() }
updateNotificationAccessUi()
updateStoppedPermissionButtons()
updateUI()
}
@@ -166,7 +169,7 @@ class MainActivity : Activity() {
if (CaptureService.isRunning) {
updateUI()
} else {
updateNotificationAccessUi()
updateStoppedPermissionButtons()
}
}
@@ -544,6 +547,26 @@ class MainActivity : Activity() {
}.onFailure { Log.w(TAG, "Notification-access settings unavailable: ${it.message}") }
}
/**
* Whether Usage Access (PACKAGE_USAGE_STATS) is granted — needed by the
* foreground-app automation rule. Delegates to the bridge's AppOps check.
*/
private fun isUsageAccessGranted(): Boolean = ForegroundAppBridge.hasUsageAccess()
/**
* Open the system Usage-Access screen so the user can grant LedGrab access
* for the foreground-app automation rule. Falls back to the generic Settings
* screen on TV-box OEM builds that strip the dedicated intent.
*/
private fun openUsageAccessSettings() {
runCatching {
startActivity(Intent(Settings.ACTION_USAGE_ACCESS_SETTINGS))
}.onFailure {
Log.w(TAG, "Usage-access settings unavailable: ${it.message}")
runCatching { startActivity(Intent(Settings.ACTION_SETTINGS)) }
}
}
/**
* Prompt-once-then-remember: the first time capture starts without
* notification-listener access, open the settings screen so the user can
@@ -559,20 +582,24 @@ class MainActivity : Activity() {
}
/**
* Show the "Grant notification access" button only while access is missing,
* then re-wire the D-pad focus chain. Called on create and on resume
* (access can change in Settings while we're backgrounded).
* Show each "Grant <permission> access" button only while that access is
* missing, then re-wire the D-pad focus chain. Called on create and on resume
* (access can change in Settings while we're backgrounded). The usage-access
* button is a passive affordance (no auto-prompt at capture start) — the
* primary guidance is the web-UI banner when an Android app rule needs it.
*/
private fun updateNotificationAccessUi() {
private fun updateStoppedPermissionButtons() {
if (!::grantNotificationButton.isInitialized) return
grantNotificationButton.visibility =
if (isNotificationAccessGranted()) View.GONE else View.VISIBLE
grantUsageAccessButton.visibility =
if (isUsageAccessGranted()) View.GONE else View.VISIBLE
wireStoppedFocusChain()
}
/**
* Link the visible stopped-panel controls into a single up/down D-pad chain.
* Both optional controls (the grant-access button and the root-only autostart
* The optional controls (the grant-access buttons and the root-only autostart
* checkbox) may be GONE, so the chain is computed from whatever is visible —
* a static nextFocus pointing at a GONE view would strand the focus on a TV
* remote.
@@ -581,6 +608,7 @@ class MainActivity : Activity() {
val chain = listOfNotNull(
toggleButton,
grantNotificationButton.takeIf { it.visibility == View.VISIBLE },
grantUsageAccessButton.takeIf { it.visibility == View.VISIBLE },
autostartCheck.takeIf { it.visibility == View.VISIBLE },
)
chain.forEachIndexed { i, view ->
@@ -81,6 +81,21 @@
android:focusableInTouchMode="true"
android:visibility="gone" />
<!-- Shown only while Usage Access is missing (needed by the foreground-app
automation rule). Like the grant-notification button, its D-pad focus
chain is wired at runtime (wireStoppedFocusChain). -->
<Button
android:id="@+id/grant_usage_access_button"
style="@style/Widget.LedGrab.Button.Secondary"
android:layout_width="320dp"
android:layout_height="56dp"
android:layout_marginTop="20dp"
android:text="@string/btn_grant_usage_access"
android:textSize="18sp"
android:focusable="true"
android:focusableInTouchMode="true"
android:visibility="gone" />
<CheckBox
android:id="@+id/autostart_check"
android:layout_width="wrap_content"
@@ -27,4 +27,5 @@
<string name="notification_text">Веб-интерфейс: %1$s</string>
<string name="notification_listener_label">Захват уведомлений LedGrab</string>
<string name="btn_grant_notification_access">Разрешить доступ к уведомлениям</string>
<string name="btn_grant_usage_access">Разрешить доступ к статистике использования</string>
</resources>
@@ -27,4 +27,5 @@
<string name="notification_text">Web界面:%1$s</string>
<string name="notification_listener_label">LedGrab 通知捕获</string>
<string name="btn_grant_notification_access">授予通知访问权限</string>
<string name="btn_grant_usage_access">授予使用情况访问权限</string>
</resources>
@@ -27,4 +27,5 @@
<string name="notification_text">Web UI: %1$s</string>
<string name="notification_listener_label">LedGrab notification capture</string>
<string name="btn_grant_notification_access">Grant notification access</string>
<string name="btn_grant_usage_access">Grant usage access</string>
</resources>
+49
View File
@@ -39,8 +39,11 @@ from ledgrab.api.schemas.system import (
DisplayListResponse,
GpuInfo,
HealthResponse,
InstalledAppItem,
InstalledAppsResponse,
PerformanceResponse,
ProcessListResponse,
SystemInfoResponse,
VersionResponse,
)
from ledgrab.config import get_config, is_demo_mode
@@ -278,6 +281,52 @@ async def get_running_processes(_: AuthRequired):
raise HTTPException(status_code=500, detail="Internal server error")
@router.get(
"/api/v1/system/installed-apps",
response_model=InstalledAppsResponse,
tags=["Config"],
)
def get_installed_apps(_: AuthRequired):
"""List launchable apps for the application-rule app picker (Android only).
Returns launchable apps (package + human label) on Android, where the
foreground-app automation rule matches package names. Returns an empty list
on desktop, where the process picker (``/system/processes``) is used instead.
Sync ``def`` so FastAPI runs the (potentially blocking) bridge call in a
thread pool.
"""
from ledgrab.core.automations import platform_detector as pd
try:
apps = pd.list_installed_apps()
items = [InstalledAppItem(package=a["package"], label=a["label"]) for a in apps]
return InstalledAppsResponse(apps=items, count=len(items))
except Exception as e:
logger.error("Failed to list installed apps: %s", e, exc_info=True)
raise HTTPException(status_code=500, detail="Internal server error")
@router.get("/api/v1/system/info", response_model=SystemInfoResponse, tags=["Info"])
def get_system_info(_: AuthRequired):
"""Platform capability signal for the automation editor.
Tells the frontend whether the server is on Android (so the application-rule
editor uses the launchable-app picker + package matching and surfaces the
Usage-Access banner) vs desktop (process picker + process names), and whether
Usage Access is currently granted. Sync ``def`` so the bridge call runs in a
thread pool.
"""
from ledgrab.core.automations import platform_detector as pd
from ledgrab.utils.platform import is_android
android = is_android()
return SystemInfoResponse(
is_android=android,
app_match_kind="package" if android else "process",
usage_access_granted=(pd.has_usage_access() if android else True),
)
@router.get(
"/api/v1/system/performance",
response_model=PerformanceResponse,
+14 -2
View File
@@ -11,9 +11,21 @@ class RuleSchema(BaseModel):
rule_type: str = Field(description="Rule type discriminator (e.g. 'application')")
# Application rule fields
apps: List[str] | None = Field(None, description="Process names (for application rule)")
apps: List[str] | None = Field(
None,
description=(
"App identifiers for the application rule. Platform-specific and not "
"portable: process names on Windows (e.g. 'chrome.exe'), package names "
"on Android (e.g. 'com.android.chrome'). Matched case-insensitively."
),
)
match_type: str | None = Field(
None, description="'running' or 'topmost' (for application rule)"
None,
description=(
"'running', 'topmost', 'fullscreen', or 'topmost_fullscreen' (application "
"rule). On Android only the foreground app is detectable, so all values "
"behave as 'foreground'."
),
)
# Time-of-day rule fields
start_time: str | None = Field(None, description="Start time HH:MM (for time_of_day rule)")
+36
View File
@@ -68,6 +68,42 @@ class ProcessListResponse(BaseModel):
count: int = Field(description="Number of unique processes")
class InstalledAppItem(BaseModel):
"""A launchable Android app, for the automation app picker."""
package: str = Field(description="Android package name, e.g. 'com.netflix.mediaclient'")
label: str = Field(description="Human-readable app label, e.g. 'Netflix'")
class InstalledAppsResponse(BaseModel):
"""Launchable apps for the application-rule picker (Android only; empty elsewhere)."""
apps: List[InstalledAppItem] = Field(description="Launchable apps, sorted by label")
count: int = Field(description="Number of apps")
class SystemInfoResponse(BaseModel):
"""Platform capability signal for the frontend (automation editor).
Lets the application-rule editor choose the right app source and matching
semantics per platform, and surface the Usage-Access permission state.
"""
is_android: bool = Field(description="True when the server runs on Android (Chaquopy)")
app_match_kind: Literal["process", "package"] = Field(
description=(
"What ApplicationRule.apps values represent: 'process' names on desktop, "
"'package' names on Android."
)
)
usage_access_granted: bool = Field(
description=(
"Android: whether PACKAGE_USAGE_STATS (Usage Access) is granted, gating "
"foreground-app detection. Always True (not applicable) off-Android."
)
)
class GpuInfo(BaseModel):
"""GPU performance information."""
@@ -6,12 +6,14 @@ Non-Windows: graceful degradation (returns empty results).
import asyncio
import ctypes
import json
import os
import sys
import threading
from typing import Set
from ledgrab.utils import get_logger
from ledgrab.utils.platform import is_android
logger = get_logger(__name__)
@@ -21,6 +23,105 @@ if _IS_WINDOWS:
import ctypes.wintypes
# ---------------------------------------------------------------------------
# Android ForegroundAppBridge interop — lazy + guarded (never at import time)
# ---------------------------------------------------------------------------
# Android reports ``sys.platform == "linux"`` so ``_IS_WINDOWS`` is False there;
# the foreground app is read via the Kotlin ``ForegroundAppBridge`` (UsageStats)
# instead of Win32 ctypes. These module-level wrappers are the monkeypatch
# surface used by tests (mirrors ``android_camera_engine``) — patch the module
# function, not the live ``jclass`` object.
# Emit the "Usage Access not granted" warning only once per process so the ~1s
# automation poll loop doesn't spam the log while access is missing.
_warned_no_usage_access = False
def _foreground_bridge():
"""Return the Kotlin ``ForegroundAppBridge`` singleton, or None off-Android.
The ``from java import jclass`` import only resolves inside the Chaquopy
runtime, so it must never run at module import time (this module is imported
on desktop CI too). Mirrors ``android_camera_engine._camera_bridge()``.
"""
if not is_android():
return None
try:
from java import jclass # type: ignore[import-not-found]
except ImportError as exc:
logger.debug("Chaquopy java interop not available: %s", exc)
return None
try:
return jclass("com.ledgrab.android.ForegroundAppBridge").INSTANCE
except Exception as exc: # pragma: no cover - Android-only path
logger.debug("ForegroundAppBridge singleton unavailable: %s", exc)
return None
def has_usage_access() -> bool:
"""Whether Usage Access (PACKAGE_USAGE_STATS) is granted. False off-Android."""
bridge = _foreground_bridge()
if bridge is None:
return False
try:
return bool(bridge.hasUsageAccess())
except Exception as exc: # pragma: no cover - Android-only path
logger.debug("ForegroundAppBridge.hasUsageAccess failed: %s", exc)
return False
def get_foreground_package() -> str | None:
"""Current foreground app package via the Kotlin bridge, or None.
None off-Android, when the bridge is unavailable, when Usage Access is
missing, or when no foreground event is found in the trailing window.
Monkeypatched in tests.
"""
bridge = _foreground_bridge()
if bridge is None:
return None
try:
pkg = bridge.getForegroundPackage()
except Exception as exc: # pragma: no cover - Android-only path
logger.warning("ForegroundAppBridge.getForegroundPackage failed: %s", exc)
return None
if pkg is None:
return None
s = str(pkg).strip()
return s or None
def list_installed_apps() -> list[dict]:
"""Launchable apps via the Kotlin bridge: ``[{"package": .., "label": ..}]``.
Returns ``[]`` off-Android, when the bridge is unavailable, on error, or on
invalid JSON. Sorted by label (the bridge sorts; order is preserved here).
Monkeypatched in tests.
"""
bridge = _foreground_bridge()
if bridge is None:
return []
try:
raw = bridge.listLaunchableApps() # JSON array string
except Exception as exc: # pragma: no cover - Android-only path
logger.warning("ForegroundAppBridge.listLaunchableApps failed: %s", exc)
return []
try:
parsed = json.loads(str(raw))
except (ValueError, TypeError) as exc: # pragma: no cover - Android-only path
logger.warning("ForegroundAppBridge.listLaunchableApps returned invalid JSON: %s", exc)
return []
apps: list[dict] = []
for entry in parsed if isinstance(parsed, list) else []:
if not isinstance(entry, dict):
continue
pkg = entry.get("package")
if not pkg:
continue
apps.append({"package": str(pkg), "label": str(entry.get("label") or pkg)})
return apps
class PlatformDetector:
"""Detect running processes and the foreground window's process."""
@@ -215,6 +316,31 @@ class PlatformDetector:
# ---- Process detection ----
def _get_android_foreground(self) -> tuple:
"""(package_lowercased, True) for the foreground app on Android.
Returns ``(None, False)`` when Usage Access is not granted (warned once)
or no foreground app is found. ``is_fullscreen`` is reported True because
a foreground TV app effectively covers the screen — so an Android rule's
``topmost``/``topmost_fullscreen``/``fullscreen`` match types all behave
as "this app is in front". Delegates to the module-level bridge wrappers
(the monkeypatch surface used by tests).
"""
global _warned_no_usage_access
if not has_usage_access():
if not _warned_no_usage_access:
logger.warning(
"Android 'Application' automation rules need Usage Access "
"(Settings > Usage access). Foreground-app rules will not match "
"until it is granted."
)
_warned_no_usage_access = True
return (None, False)
pkg = get_foreground_package()
if not pkg:
return (None, False)
return (pkg.lower(), True)
def _get_running_processes_sync(self) -> Set[str]:
"""Get set of lowercase process names via Win32 EnumProcesses.
@@ -222,7 +348,14 @@ class PlatformDetector:
which is ~300x faster than WMI (~8ms vs ~3s). System services
running under protected accounts are not visible, but all
user-facing applications are covered.
On Android there is no process enumeration API (getRunningTasks is
restricted); the foreground app is reported as the sole "running" entry
as a best-effort so ``match_type="running"`` rules still work.
"""
if is_android():
pkg, _ = self._get_android_foreground()
return {pkg} if pkg else set()
if not _IS_WINDOWS:
return set()
@@ -276,9 +409,13 @@ class PlatformDetector:
def _get_topmost_process_sync(self) -> tuple:
"""Get (process_name, is_fullscreen) of the foreground window.
Returns (None, False) when detection fails.
On Android the "foreground window" is the foreground app package (read
via the Kotlin ForegroundAppBridge); see ``_get_android_foreground``.
Returns (None, False) when detection fails / Usage Access is missing.
Blocking — call via executor.
"""
if is_android():
return self._get_android_foreground()
if not _IS_WINDOWS:
return (None, False)
@@ -369,7 +506,13 @@ class PlatformDetector:
Enumerates all top-level windows and checks each for fullscreen.
Returns process names (lowercase) whose window covers an entire monitor.
On Android the foreground app is treated as fullscreen, so it is the
sole entry (best-effort, mirrors ``_get_running_processes_sync``).
"""
if is_android():
pkg, _ = self._get_android_foreground()
return {pkg} if pkg else set()
if not _IS_WINDOWS:
return set()
@@ -74,6 +74,19 @@
font-size: 0.85rem;
}
/* Android-only: shown in the application rule when Usage Access is missing,
so the foreground-app rule can't fire until the user grants it on the TV. */
.rule-usage-warning {
margin-bottom: 10px;
padding: 8px 12px;
border-radius: 6px;
font-size: 0.85rem;
line-height: 1.35;
color: var(--warning-color, #ff9800);
background: color-mix(in srgb, var(--warning-color, #ff9800) 12%, transparent);
border: 1px solid color-mix(in srgb, var(--warning-color, #ff9800) 35%, transparent);
}
.btn-remove-rule {
background: none;
border: none;
@@ -2,13 +2,19 @@
* Command-palette style name picker — reusable UI for browsing a list of
* names fetched from any API endpoint. Mirrors the EntityPalette pattern.
*
* Two concrete pickers are exported:
* Three concrete pickers are exported:
*
* - **ProcessPalette** — picks from running OS processes (`/system/processes`)
* - **NotificationAppPalette** — picks from OS notification history apps
* - **AppPalette** — picks from Android launchable apps (`/system/installed-apps`),
* displaying the human label but inserting the package name
*
* Both support single-select (returns one value) and multi-select (appends to
* a textarea).
* Items may be plain strings (display == stored value) or `{ value, label }`
* pairs (display the label, store the value — used by AppPalette so the rule
* stores the package name while the user sees "Netflix").
*
* All support single-select (returns one value) and multi-select (appends the
* value to a textarea).
*
* Usage:
*
@@ -29,8 +35,16 @@ import { ICON_SEARCH } from './icons.ts';
/* ─── types ────────────────────────────────────────────────── */
interface PaletteItem {
name: string;
/** An item with a display label distinct from its stored value. */
interface AppItem {
value: string;
label: string;
}
/** Raw items a fetcher may return: bare strings or labelled pairs. */
type RawItem = string | AppItem;
interface PaletteEntry extends AppItem {
added: boolean;
}
@@ -44,7 +58,9 @@ interface PickMultiOpts {
placeholder?: string;
}
type FetchItemsFn = () => Promise<string[]>;
type FetchItemsFn = () => Promise<RawItem[]>;
const DEFAULT_EMPTY_KEY = 'automations.condition.application.no_processes';
/* ─── generic NamePalette (shared logic) ───────────────────── */
@@ -53,19 +69,21 @@ class NamePalette {
private _input: HTMLInputElement;
private _list: HTMLDivElement;
private _fetchItems: FetchItemsFn;
private _emptyKey: string;
private _resolveSingle: ((v: string | undefined) => void) | null = null;
private _multiTextarea: HTMLTextAreaElement | null = null;
private _items: string[] = [];
private _items: AppItem[] = [];
private _existing: Set<string> = new Set();
private _filtered: PaletteItem[] = [];
private _filtered: PaletteEntry[] = [];
private _highlightIdx = 0;
private _currentValue: string | undefined;
private _isMulti = false;
constructor(fetchItems: FetchItemsFn) {
constructor(fetchItems: FetchItemsFn, emptyKey: string = DEFAULT_EMPTY_KEY) {
this._fetchItems = fetchItems;
this._emptyKey = emptyKey;
this._overlay = document.createElement('div');
this._overlay.className = 'entity-palette-overlay process-palette-overlay';
@@ -107,14 +125,20 @@ class NamePalette {
this._isMulti = true;
this._multiTextarea = opts.textarea;
this._resolveSingle = resolve as any;
this._existing = new Set(
opts.textarea.value.split('\n').map(s => s.trim().toLowerCase()).filter(Boolean),
);
this._existing = this._textareaValues(opts.textarea);
this._currentValue = undefined;
this._open(opts.placeholder);
});
}
private _textareaValues(ta: HTMLTextAreaElement): Set<string> {
return new Set(ta.value.split('\n').map(s => s.trim().toLowerCase()).filter(Boolean));
}
private _normalize(raw: RawItem[]): AppItem[] {
return raw.map(r => (typeof r === 'string' ? { value: r, label: r } : r));
}
private async _open(placeholder?: string) {
this._input.placeholder = placeholder || '';
this._input.value = '';
@@ -123,15 +147,13 @@ class NamePalette {
requestAnimationFrame(() => this._input.focus());
try {
this._items = await this._fetchItems();
this._items = this._normalize(await this._fetchItems());
} catch {
this._items = [];
}
if (this._isMulti) {
this._existing = new Set(
this._multiTextarea!.value.split('\n').map(s => s.trim().toLowerCase()).filter(Boolean),
);
this._existing = this._textareaValues(this._multiTextarea!);
}
this._filter();
@@ -142,14 +164,11 @@ class NamePalette {
private _filter() {
const q = this._input.value.toLowerCase().trim();
this._filtered = this._items
.filter(p => !q || p.toLowerCase().includes(q))
.map(p => ({
name: p,
added: this._existing.has(p.toLowerCase()),
}));
.filter(p => !q || p.label.toLowerCase().includes(q) || p.value.toLowerCase().includes(q))
.map(p => ({ ...p, added: this._existing.has(p.value.toLowerCase()) }));
this._highlightIdx = this._filtered.findIndex(
i => i.name.toLowerCase() === (this._currentValue || '').toLowerCase(),
i => i.value.toLowerCase() === (this._currentValue || '').toLowerCase(),
);
if (this._highlightIdx === -1) this._highlightIdx = 0;
this._render();
@@ -158,9 +177,7 @@ class NamePalette {
private _render() {
if (this._filtered.length === 0) {
this._list.innerHTML = `<div class="entity-palette-empty">${
this._items.length === 0
? t('automations.condition.application.no_processes')
: '—'
this._items.length === 0 ? t(this._emptyKey) : '—'
}</div>`;
return;
}
@@ -170,12 +187,21 @@ class NamePalette {
'entity-palette-item',
i === this._highlightIdx ? 'ep-highlight' : '',
item.added ? 'ep-current' : '',
item.name.toLowerCase() === (this._currentValue || '').toLowerCase() ? 'ep-current' : '',
item.value.toLowerCase() === (this._currentValue || '').toLowerCase() ? 'ep-current' : '',
].filter(Boolean).join(' ');
// When the label differs from the stored value (e.g. "Netflix" vs
// "com.netflix.mediaclient"), show the value as a secondary line so
// users can see exactly what gets matched. Otherwise fall back to the
// ✓ added-marker.
const showValue = item.label !== item.value;
const trailing = showValue
? `<span class="ep-item-desc">${escapeHtml(item.value)}</span>`
: (item.added ? '<span class="ep-item-desc">✓</span>' : '');
return `<div class="${cls}" data-idx="${i}">
<span class="ep-item-label">${escapeHtml(item.name)}</span>
${item.added ? '<span class="ep-item-desc">\u2713</span>' : ''}
<span class="ep-item-label">${escapeHtml(item.label)}</span>
${trailing}
</div>`;
}).join('');
@@ -192,19 +218,19 @@ class NamePalette {
/* ── selection ──────────────────────────────────────────── */
private _selectItem(item: PaletteItem) {
private _selectItem(item: PaletteEntry) {
if (this._isMulti) {
if (!item.added) {
const ta = this._multiTextarea!;
const cur = ta.value.trim();
ta.value = cur ? cur + '\n' + item.name : item.name;
this._existing.add(item.name.toLowerCase());
ta.value = cur ? cur + '\n' + item.value : item.value;
this._existing.add(item.value.toLowerCase());
item.added = true;
this._render();
}
} else {
this._overlay.classList.remove('open');
if (this._resolveSingle) this._resolveSingle(item.name);
if (this._resolveSingle) this._resolveSingle(item.value);
this._resolveSingle = null;
}
}
@@ -269,6 +295,17 @@ async function _fetchNotificationApps(): Promise<string[]> {
return Array.from(seen.values()).sort((a, b) => a.localeCompare(b));
}
async function _fetchInstalledApps(): Promise<AppItem[]> {
try {
const data = await apiGet<{ apps?: Array<{ package: string; label: string }> }>(
'/system/installed-apps',
);
return (data.apps || []).map(a => ({ value: a.package, label: a.label || a.package }));
} catch {
return [];
}
}
/* ─── ProcessPalette (running processes) ───────────────────── */
let _processInst: NamePalette | null = null;
@@ -301,6 +338,22 @@ export class NotificationAppPalette {
}
}
/* ─── AppPalette (Android launchable apps) ─────────────────── */
let _appInst: NamePalette | null = null;
export class AppPalette {
static pick(opts: PickOpts): Promise<string | undefined> {
if (!_appInst) _appInst = new NamePalette(_fetchInstalledApps, 'automations.rule.application.no_apps');
return _appInst.pickSingle(opts);
}
static pickMulti(opts: PickMultiOpts): Promise<void> {
if (!_appInst) _appInst = new NamePalette(_fetchInstalledApps, 'automations.rule.application.no_apps');
return _appInst.pickMulti(opts);
}
}
/* ─── drop-in replacement for the old attachProcessPicker ─── */
/**
@@ -334,3 +387,19 @@ export function attachNotificationAppPicker(containerEl: HTMLElement, textareaEl
});
});
}
/**
* Wire up a `.btn-browse-apps` button to open the Android launchable-app palette
* (multi-select, feeding package names into a textarea while showing labels).
*/
export function attachAppPicker(containerEl: HTMLElement, textareaEl: HTMLTextAreaElement): void {
const browseBtn = containerEl.querySelector('.btn-browse-apps');
if (!browseBtn) return;
browseBtn.addEventListener('click', () => {
AppPalette.pickMulti({
textarea: textareaEl,
placeholder: t('automations.rule.application.search_apps') || 'Filter apps…',
});
});
}
@@ -29,7 +29,7 @@ import { getBaseOrigin } from './settings.ts';
import { IconSelect } from '../core/icon-select.ts';
import { EntitySelect } from '../core/entity-palette.ts';
import { enhanceMiniSelects } from '../core/mini-select.ts';
import { attachProcessPicker } from '../core/process-picker.ts';
import { attachProcessPicker, attachAppPicker } from '../core/process-picker.ts';
import { TreeNav } from '../core/tree-nav.ts';
import { csScenes, createSceneCard, initScenePresetDelegation } from './scene-presets.ts';
import type { Automation, RuleType } from '../types.ts';
@@ -215,6 +215,28 @@ document.addEventListener('server:automation_state_changed', () => {
if (apiKey && isActiveTab('automations')) loadAutomations();
});
/** Platform capability signal from `/system/info` — drives the application-rule
* editor (process picker + match types on desktop vs. app picker + foreground-only
* on Android) and the Usage-Access banner. Fetched once and cached. */
interface PlatformInfo {
is_android: boolean;
app_match_kind: 'process' | 'package';
usage_access_granted: boolean;
}
let _platformInfo: PlatformInfo | null = null;
async function ensurePlatformInfo(): Promise<PlatformInfo> {
if (_platformInfo) return _platformInfo;
try {
_platformInfo = await apiGet<PlatformInfo>('/system/info');
} catch {
// Default to desktop semantics if the signal can't be fetched.
_platformInfo = { is_android: false, app_match_kind: 'process', usage_access_granted: true };
}
return _platformInfo;
}
export async function loadAutomations() {
if (_automationsLoading) return;
set_automationsLoading(true);
@@ -222,6 +244,10 @@ export async function loadAutomations() {
if (!container) { set_automationsLoading(false); return; }
if (!csAutomations.isMounted()) setTabRefreshing('automations-content', true);
// Prime the platform signal so the editor renders the right app source +
// match semantics without an async hop when a rule row is expanded.
void ensurePlatformInfo();
try {
const [automations, scenes] = await Promise.all([
automationsCacheObj.fetch(),
@@ -559,6 +585,11 @@ export async function openAutomationEditor(automationId?: any, cloneData?: any)
errorEl.style.display = 'none';
ruleList!.innerHTML = '';
// Ensure the platform signal is loaded before rendering rule rows so the
// application rule picks the right app source + match semantics. The
// automations tab primes this, but the graph editor opens this directly.
await ensurePlatformInfo();
_ensureRuleLogicIconSelect();
_ensureDeactivationModeIconSelect();
@@ -1129,6 +1160,33 @@ function _renderWebhookFields(container: HTMLElement, data: any): void {
function _renderApplicationFields(container: HTMLElement, data: any): void {
const appsValue = (data.apps || []).join('\n');
// On Android there is exactly one obtainable signal — the foreground app —
// so the match-type selector is hidden (match_type is forced to "topmost" by
// the collector) and the app list comes from launchable apps (package names)
// rather than running processes (process names).
if (_platformInfo?.is_android) {
const banner = _platformInfo.usage_access_granted
? ''
: `<div class="rule-usage-warning">${t('automations.rule.application.usage_access_required')}</div>`;
container.innerHTML = `
<div class="rule-fields">
${banner}
<div class="rule-field">
<div class="rule-apps-header">
<label>${t('automations.rule.application.apps')}</label>
<button type="button" class="btn btn-icon btn-secondary btn-browse-apps" title="${t('automations.rule.application.browse')}">${ICON_SEARCH}</button>
</div>
<textarea class="rule-apps" rows="3" placeholder="com.netflix.mediaclient&#10;com.android.chrome">${escapeHtml(appsValue)}</textarea>
<small class="rule-hint-desc">${t('automations.rule.application.apps.hint_android')}</small>
</div>
</div>
`;
const textarea = container.querySelector('.rule-apps') as HTMLTextAreaElement;
attachAppPicker(container, textarea);
return;
}
const matchType = data.match_type || 'running';
container.innerHTML = `
<div class="rule-fields">
@@ -1299,7 +1357,10 @@ const RULE_COLLECTORS: Record<RuleType, RuleCollector> = {
return r;
},
application: (row) => {
const matchType = (row.querySelector('.rule-match-type') as HTMLSelectElement).value;
// On Android the match-type selector is hidden (only the foreground app is
// detectable), so default to "topmost" when the select isn't present.
const matchSel = row.querySelector('.rule-match-type') as HTMLSelectElement | null;
const matchType = matchSel ? matchSel.value : 'topmost';
const appsText = (row.querySelector('.rule-apps') as HTMLTextAreaElement).value.trim();
const apps = appsText ? appsText.split('\n').map(a => a.trim()).filter(Boolean) : [];
return { rule_type: 'application', apps, match_type: matchType };
@@ -1226,6 +1226,10 @@
"automations.rule.application.match_type.topmost_fullscreen.desc": "Foreground + fullscreen",
"automations.rule.application.match_type.fullscreen": "Fullscreen",
"automations.rule.application.match_type.fullscreen.desc": "Any fullscreen app",
"automations.rule.application.apps.hint_android": "Package names, one per line (e.g. com.netflix.mediaclient)",
"automations.rule.application.search_apps": "Filter apps...",
"automations.rule.application.no_apps": "No apps found",
"automations.rule.application.usage_access_required": "Needs Usage Access. On your LedGrab TV, open the app and tap 'Grant usage access'.",
"automations.rule.time_of_day": "Time of Day",
"automations.rule.time_of_day.desc": "Time range",
"automations.rule.time_of_day.start_time": "Start Time:",
@@ -1260,6 +1260,10 @@
"automations.rule.application.match_type.topmost_fullscreen.desc": "В фокусе + полный экран",
"automations.rule.application.match_type.fullscreen": "Полный экран",
"automations.rule.application.match_type.fullscreen.desc": "Любое полноэкранное",
"automations.rule.application.apps.hint_android": "Имена пакетов, по одному в строке (напр. com.netflix.mediaclient)",
"automations.rule.application.search_apps": "Поиск приложений...",
"automations.rule.application.no_apps": "Приложения не найдены",
"automations.rule.application.usage_access_required": "Требуется доступ к статистике использования. Откройте LedGrab на телевизоре и нажмите «Разрешить доступ к статистике использования».",
"automations.rule.time_of_day": "Время суток",
"automations.rule.time_of_day.desc": "Диапазон времени",
"automations.rule.time_of_day.start_time": "Время начала:",
@@ -1256,6 +1256,10 @@
"automations.rule.application.match_type.topmost_fullscreen.desc": "前台 + 全屏",
"automations.rule.application.match_type.fullscreen": "全屏",
"automations.rule.application.match_type.fullscreen.desc": "任意全屏应用",
"automations.rule.application.apps.hint_android": "包名,每行一个(例如 com.netflix.mediaclient",
"automations.rule.application.search_apps": "筛选应用…",
"automations.rule.application.no_apps": "未找到应用",
"automations.rule.application.usage_access_required": "需要使用情况访问权限。在您的 LedGrab 电视上打开应用并点按「授予使用情况访问权限」。",
"automations.rule.time_of_day": "时段",
"automations.rule.time_of_day.desc": "时间范围",
"automations.rule.time_of_day.start_time": "开始时间:",
+15 -2
View File
@@ -30,11 +30,24 @@ class Rule:
@dataclass
class ApplicationRule(Rule):
"""Activate when specified applications are running or topmost."""
"""Activate when specified applications are running or topmost.
``apps`` values are platform-specific and NOT portable across OSes:
on Windows they are **process names** (e.g. ``chrome.exe``); on Android
they are **package names** (e.g. ``com.android.chrome``). Matching is
exact and case-insensitive. The automation editor sources values from the
right place per platform (running processes on desktop, launchable apps on
Android), so a rule authored on one OS will simply not match on another.
``match_type`` is honoured on Windows for all four values below. On Android
only the foreground app is obtainable, so every match type collapses to
"this app is in the foreground" and the editor hides the selector.
"""
rule_type: str = "application"
apps: List[str] = field(default_factory=list)
match_type: str = "running" # "running" | "topmost"
# "running" | "topmost" | "fullscreen" | "topmost_fullscreen"
match_type: str = "running"
def to_dict(self) -> dict:
d = super().to_dict()
@@ -92,3 +92,57 @@ class TestRootEndpoint:
resp = client.get("/")
assert resp.status_code == 200
assert "text/html" in resp.headers["content-type"]
class TestInstalledAppsEndpoint:
def test_requires_auth(self, client):
resp = client.get("/api/v1/system/installed-apps")
assert resp.status_code == 401
def test_empty_off_android(self, client):
"""Desktop test host: is_android() is False, so the bridge wrapper
short-circuits to an empty list."""
resp = client.get("/api/v1/system/installed-apps", headers=_auth_headers())
assert resp.status_code == 200
assert resp.json() == {"apps": [], "count": 0}
def test_returns_apps_when_available(self, client, monkeypatch):
from ledgrab.core.automations import platform_detector as pd
monkeypatch.setattr(
pd,
"list_installed_apps",
lambda: [{"package": "com.netflix.mediaclient", "label": "Netflix"}],
)
resp = client.get("/api/v1/system/installed-apps", headers=_auth_headers())
assert resp.status_code == 200
data = resp.json()
assert data["count"] == 1
assert data["apps"][0] == {"package": "com.netflix.mediaclient", "label": "Netflix"}
class TestSystemInfoEndpoint:
def test_requires_auth(self, client):
resp = client.get("/api/v1/system/info")
assert resp.status_code == 401
def test_desktop_signal(self, client):
resp = client.get("/api/v1/system/info", headers=_auth_headers())
assert resp.status_code == 200
data = resp.json()
assert data["is_android"] is False
assert data["app_match_kind"] == "process"
assert data["usage_access_granted"] is True
def test_android_signal(self, client, monkeypatch):
import ledgrab.utils.platform as plat
from ledgrab.core.automations import platform_detector as pd
monkeypatch.setattr(plat, "is_android", lambda: True)
monkeypatch.setattr(pd, "has_usage_access", lambda: False)
resp = client.get("/api/v1/system/info", headers=_auth_headers())
assert resp.status_code == 200
data = resp.json()
assert data["is_android"] is True
assert data["app_match_kind"] == "package"
assert data["usage_access_granted"] is False
@@ -0,0 +1,194 @@
"""Tests for Android foreground-app detection in PlatformDetector.
These run on desktop CI (no Android device needed): ``is_android`` and the
Kotlin-bridge wrappers (``has_usage_access`` / ``get_foreground_package``) are
monkeypatched, exactly as the Kotlin ``ForegroundAppBridge`` would drive them on
device. The critical invariant under test is that the Android branch runs *ahead
of* the import-time ``_IS_WINDOWS`` guard, and that the Windows/desktop paths are
left untouched.
"""
import pytest
from ledgrab.core.automations import platform_detector as pd
from ledgrab.core.automations.platform_detector import PlatformDetector
@pytest.fixture
def detector(monkeypatch):
"""A PlatformDetector with the Windows display-power listener stubbed out.
``__init__`` otherwise spawns a thread that registers a global window class +
runs a ctypes message pump irrelevant here and noisy when many detectors are
constructed in one process.
"""
monkeypatch.setattr(PlatformDetector, "_display_power_listener", lambda self: None)
return PlatformDetector()
@pytest.fixture(autouse=True)
def _reset_warn():
"""Reset the process-global warn-once flag around every test."""
pd._warned_no_usage_access = False
yield
pd._warned_no_usage_access = False
# ---------------------------------------------------------------------------
# topmost (foreground) detection
# ---------------------------------------------------------------------------
def test_topmost_android_returns_lowercased_foreground_package(detector, monkeypatch):
monkeypatch.setattr(pd, "is_android", lambda: True)
monkeypatch.setattr(pd, "has_usage_access", lambda: True)
monkeypatch.setattr(pd, "get_foreground_package", lambda: "com.Netflix.MediaClient")
assert detector._get_topmost_process_sync() == ("com.netflix.mediaclient", True)
def test_topmost_android_no_access_returns_none_and_warns_once(detector, monkeypatch):
monkeypatch.setattr(pd, "is_android", lambda: True)
monkeypatch.setattr(pd, "has_usage_access", lambda: False)
fg_calls = []
monkeypatch.setattr(pd, "get_foreground_package", lambda: fg_calls.append(1) or "x")
warns = []
monkeypatch.setattr(pd.logger, "warning", lambda *a, **k: warns.append(a))
assert detector._get_topmost_process_sync() == (None, False)
assert detector._get_topmost_process_sync() == (None, False)
# Foreground is never queried when access is missing; warned exactly once.
assert fg_calls == []
assert len(warns) == 1
assert pd._warned_no_usage_access is True
def test_topmost_android_no_foreground_event_returns_none(detector, monkeypatch):
monkeypatch.setattr(pd, "is_android", lambda: True)
monkeypatch.setattr(pd, "has_usage_access", lambda: True)
monkeypatch.setattr(pd, "get_foreground_package", lambda: None)
assert detector._get_topmost_process_sync() == (None, False)
def test_android_branch_precedes_windows_guard(detector, monkeypatch):
"""Even with _IS_WINDOWS True, is_android() must win.
Proves the Android branch sits ahead of the ``if not _IS_WINDOWS`` early
return and never falls through to the Win32 ctypes path (the plan-review
critical gap: a naive wiring would no-op behind the Windows guard).
"""
monkeypatch.setattr(pd, "_IS_WINDOWS", True)
monkeypatch.setattr(pd, "is_android", lambda: True)
monkeypatch.setattr(pd, "has_usage_access", lambda: True)
monkeypatch.setattr(pd, "get_foreground_package", lambda: "com.App.X")
assert detector._get_topmost_process_sync() == ("com.app.x", True)
# ---------------------------------------------------------------------------
# running / fullscreen best-effort (foreground app as the sole entry)
# ---------------------------------------------------------------------------
def test_running_and_fullscreen_android_return_foreground_set(detector, monkeypatch):
monkeypatch.setattr(pd, "is_android", lambda: True)
monkeypatch.setattr(pd, "has_usage_access", lambda: True)
monkeypatch.setattr(pd, "get_foreground_package", lambda: "com.App.Y")
assert detector._get_running_processes_sync() == {"com.app.y"}
assert detector._get_fullscreen_processes_sync() == {"com.app.y"}
def test_running_and_fullscreen_android_empty_without_access(detector, monkeypatch):
monkeypatch.setattr(pd, "is_android", lambda: True)
monkeypatch.setattr(pd, "has_usage_access", lambda: False)
monkeypatch.setattr(pd, "get_foreground_package", lambda: "x")
assert detector._get_running_processes_sync() == set()
assert detector._get_fullscreen_processes_sync() == set()
# ---------------------------------------------------------------------------
# desktop paths untouched
# ---------------------------------------------------------------------------
def test_non_android_non_windows_skips_bridge(detector, monkeypatch):
"""Desktop Linux/mac: no Android branch, no Win32 path, empty results, and
the bridge wrappers are never consulted."""
monkeypatch.setattr(pd, "_IS_WINDOWS", False)
monkeypatch.setattr(pd, "is_android", lambda: False)
calls = []
monkeypatch.setattr(pd, "get_foreground_package", lambda: calls.append("fg"))
monkeypatch.setattr(pd, "has_usage_access", lambda: calls.append("acc") or True)
assert detector._get_topmost_process_sync() == (None, False)
assert detector._get_running_processes_sync() == set()
assert detector._get_fullscreen_processes_sync() == set()
assert calls == []
def test_wrappers_return_safe_defaults_off_android(monkeypatch):
"""is_android() False short-circuits the bridge accessor to None, so the
public wrappers return safe defaults without any java interop."""
monkeypatch.setattr(pd, "is_android", lambda: False)
assert pd._foreground_bridge() is None
assert pd.has_usage_access() is False
assert pd.get_foreground_package() is None
assert pd.list_installed_apps() == []
# ---------------------------------------------------------------------------
# bridge-response parsing wrappers (fed via a fake bridge object)
# ---------------------------------------------------------------------------
class _FakeBridge:
"""Stand-in for the Kotlin ForegroundAppBridge singleton."""
def __init__(self, fg=None, apps_json=None):
self._fg = fg
self._apps_json = apps_json
def getForegroundPackage(self):
return self._fg
def listLaunchableApps(self):
return self._apps_json
def test_get_foreground_package_strips_whitespace(monkeypatch):
# Stripped but NOT lowercased — the caller (_get_android_foreground) lowercases.
monkeypatch.setattr(pd, "_foreground_bridge", lambda: _FakeBridge(fg=" com.App.X "))
assert pd.get_foreground_package() == "com.App.X"
def test_get_foreground_package_blank_returns_none(monkeypatch):
monkeypatch.setattr(pd, "_foreground_bridge", lambda: _FakeBridge(fg=" "))
assert pd.get_foreground_package() is None
def test_list_installed_apps_parses_and_filters(monkeypatch):
import json
payload = json.dumps(
[
{"package": "com.a", "label": "A"},
{"package": "com.b", "label": ""}, # blank label -> falls back to package
{"label": "no package"}, # skipped: no package
"not a dict", # skipped: not an object
]
)
monkeypatch.setattr(pd, "_foreground_bridge", lambda: _FakeBridge(apps_json=payload))
assert pd.list_installed_apps() == [
{"package": "com.a", "label": "A"},
{"package": "com.b", "label": "com.b"},
]
def test_list_installed_apps_invalid_json_returns_empty(monkeypatch):
monkeypatch.setattr(pd, "_foreground_bridge", lambda: _FakeBridge(apps_json="not json{"))
assert pd.list_installed_apps() == []