From 0be3f833dfacc2b3e41187cbe201db2780300782 Mon Sep 17 00:00:00 2001 From: "alexei.dolgolyov" Date: Tue, 2 Jun 2026 11:47:13 +0300 Subject: [PATCH] feat(android): on-device OS notification capture (NotificationListenerService) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add an Android backend to os_notification_listener.py so notifications on the experimental Android-TV build drive the existing NotificationColorStripSource LED effects (flash/pulse/sweep, per-app colors + sounds) at app-name parity with the Windows/Linux backends. A Kotlin NotificationListenerService forwards the posting app's display label across the Chaquopy JNI boundary into a new push-based _AndroidBackend + module-level push_notification() receiver; the existing color-strip pipeline, per-app colors/filters, and history endpoint are reused unchanged. - Python: _AndroidBackend (probed first), push_notification() receiver, _LinuxBackend.probe() hardened with is_linux() to exclude Android (which also reports platform.system() == "Linux"). - Android: LedGrabNotificationListener NLS — serial single-thread executor, full crash isolation around Python.getInstance(), label-only forwarding (never notification title/body), ongoing/group-summary/self-package noise filtering. Manifest service exported + gated by BIND_NOTIFICATION_LISTENER_SERVICE (no new uses-permission). - UX: prompt-once notification-access + manual "Grant notification access" button wired into the D-pad focus chain (computed from visible controls); en/ru/zh strings. - Tests: 11 isolated unit tests — module-global + tmp_path history isolation, push routing contract, callback-exception swallowing, None app-name, and a desktop-regression lock on backend selection order. - Docs: README OS-support Android column (notification + audio cells), ANDROID-REVIEW status flipped to Implemented. Zero new Python deps; no build.gradle.kts / Chaquopy pip changes. --- .../android-missing-functionality.md | 15 +- README.md | 21 +- android/app/src/main/AndroidManifest.xml | 17 ++ .../android/LedGrabNotificationListener.kt | 97 +++++++ .../java/com/ledgrab/android/MainActivity.kt | 77 +++++- .../app/src/main/res/layout/activity_main.xml | 15 ++ .../app/src/main/res/values-ru/strings.xml | 2 + .../app/src/main/res/values-zh/strings.xml | 2 + android/app/src/main/res/values/strings.xml | 2 + .../processing/os_notification_listener.py | 70 +++++- .../test_os_notification_listener_android.py | 237 ++++++++++++++++++ 11 files changed, 532 insertions(+), 23 deletions(-) create mode 100644 android/app/src/main/java/com/ledgrab/android/LedGrabNotificationListener.kt create mode 100644 server/tests/core/processing/test_os_notification_listener_android.py diff --git a/ANDROID-REVIEW/android-missing-functionality.md b/ANDROID-REVIEW/android-missing-functionality.md index d382164..d5e8ccd 100644 --- a/ANDROID-REVIEW/android-missing-functionality.md +++ b/ANDROID-REVIEW/android-missing-functionality.md @@ -45,7 +45,7 @@ Python receiver engine mirroring that pattern.** | LED transports (network/USB-serial/BLE) | ✅ | ✅ (USB via Android driver, BLE via Android bridge) | No | | System metrics | psutil | ✅ CPU/RAM/battery/thermal via `/proc`, `/sys` (`AndroidMetricsProvider`) | No | | **Audio capture** | WASAPI / Sounddevice | ❌ no PortAudio | **Yes** | -| **Notification capture** | WinRT / D-Bus | ❌ listener only Win/Linux | **Yes** | +| Notification capture | WinRT / D-Bus | ✅ NotificationListenerService → `push_notification()` | No (implemented) | | Webcam capture | OpenCV | ❌ no OpenCV wheel | Yes (niche) | | GPU monitoring | NVML | ❌ no NVIDIA GPU | Marginal | | Capture from *another* Android phone | scrcpy/ADB | ❌ | Skip (redundant) | @@ -70,7 +70,7 @@ Python receiver engine mirroring that pattern.** media and the device's own audio. Root mode (no MediaProjection) → mic-only. - 📄 **See `android-audio-capture-plan.md`** for the full implementation plan. -### 🔔 Notification capture — **FEASIBLE, HIGH VALUE** ⭐ (planned) +### 🔔 Notification capture — **IMPLEMENTED** ✅ (shipped) - **Android is the *best* platform for this:** `NotificationListenerService` is the native, event-push mechanism (no polling). @@ -82,8 +82,13 @@ Python receiver engine mirroring that pattern.** - **Permission:** user enables "Notification access" in Settings (`ACTION_NOTIFICATION_LISTENER_SETTINGS`); no runtime-permission popup. - **Effort:** moderate. **Value:** high. -- 📄 **Plan approved & detailed** — see `C:\Users\Alexei\.claude\plans\deep-enchanting-muffin.md` - (app-name parity; prompt-once permission UX). +- ✅ **Implemented** on branch `feature/android-notification-capture`: a push-based + `_AndroidBackend` + module-level `push_notification()` in `os_notification_listener.py`, + a Kotlin `LedGrabNotificationListener` (NLS), and prompt-once permission UX. App-name + parity — only the resolved app label crosses the JNI boundary, never the notification + title/body. ⚠️ App labels can differ across OSes (Windows `display_name` / Linux D-Bus + `app_name` / Android `getApplicationLabel`), so desktop-configured per-app colors/filters + may need re-matching on Android. ### 📷 Webcam capture — **FEASIBLE, LOW VALUE** @@ -128,7 +133,7 @@ Python receiver engine mirroring that pattern.** | Priority | Feature | Effort | Value | New Python deps | Status | | -------- | ------- | ------ | ----- | --------------- | ------ | -| 1 | Notification capture | Moderate | High | None | **Plan approved** | +| 1 | Notification capture | Moderate | High | None | **✅ Implemented** | | 2 | Audio capture | Moderate | High | None | **Plan written** (this folder) | | 3 | Automation: foreground-app condition | Moderate | Moderate | None | Idea | | 4 | Webcam capture (CameraX) | Moderate | Low | None | Idea | diff --git a/README.md b/README.md index f2c7ef2..08135ed 100644 --- a/README.md +++ b/README.md @@ -105,16 +105,17 @@ LedGrab runs as a desktop / server application: ### Feature support by OS -| Feature | Windows | Linux / macOS | -| ------- | ------- | ------------- | -| Screen capture | DXCam, BetterCam, WGC, MSS | MSS | -| Webcam capture | OpenCV (DirectShow) | OpenCV (V4L2) | -| Audio capture | WASAPI, Sounddevice | Sounddevice (PulseAudio/PipeWire) | -| GPU monitoring | NVIDIA (nvidia-ml-py) | NVIDIA (nvidia-ml-py) | -| Capture from Android phone | scrcpy (ADB) | scrcpy (ADB) | -| Notification capture | WinRT | dbus (Linux) | -| Monitor names | Friendly names (WMI) | Generic ("Display 0") | -| Automation: window/process conditions | Supported | Partial | +| Feature | Windows | Linux / macOS | Android TV (experimental) | +| ------- | ------- | ------------- | ------------------------- | +| Screen capture | DXCam, BetterCam, WGC, MSS | MSS | MediaProjection; root `screenrecord` (rooted devices) | +| Webcam capture | OpenCV (DirectShow) | OpenCV (V4L2) | — (no OpenCV wheel) | +| Audio capture | WASAPI, Sounddevice | Sounddevice (PulseAudio/PipeWire) | AudioPlaybackCapture (API 29+) | +| GPU monitoring | NVIDIA (nvidia-ml-py) | NVIDIA (nvidia-ml-py) | — (CPU/RAM/battery/thermal via `/proc`) | +| Capture from Android phone | scrcpy (ADB) | scrcpy (ADB) | — (captures its own screen instead) | +| Notification capture | WinRT | dbus (Linux) | NotificationListenerService | +| Monitor names | Friendly names (WMI) | Generic ("Display 0") | Single built-in display | +| LED transports | Network, USB-serial, BLE | Network, USB-serial, BLE | Network, USB-serial (Android driver), BLE (Android bridge) | +| Automation: window/process conditions | Supported | Partial | — | ## Requirements diff --git a/android/app/src/main/AndroidManifest.xml b/android/app/src/main/AndroidManifest.xml index bb74f08..05778b4 100644 --- a/android/app/src/main/AndroidManifest.xml +++ b/android/app/src/main/AndroidManifest.xml @@ -110,6 +110,23 @@ android:value="Root-mode screen capture for ambient LED sync. Uses /system/bin/screenrecord on rooted devices to avoid MediaProjection's persistent capture indicator overlay, which is required for the always-on ambient-lighting use case." /> + + + + + + + +