diff --git a/.gitignore b/.gitignore index 3599186..5616374 100644 --- a/.gitignore +++ b/.gitignore @@ -25,6 +25,10 @@ parts/ sdist/ var/ wheels/ +# …but keep pre-built Android wheels (pydantic-core cross-compiled for +# arm64-v8a / x86_64 / x86, required by the Chaquopy build) +!android/wheels/ +!android/wheels/* *.egg-info/ .installed.cfg *.egg diff --git a/CLAUDE.md b/CLAUDE.md index 244ec8e..0f5a74c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,8 +28,21 @@ ast-index changed --base master # Show symbols changed in current bran ## Project Structure - `/server` — Python FastAPI backend (see [server/CLAUDE.md](server/CLAUDE.md)) +- `/android` — Android TV app (Kotlin shell + embedded Python via Chaquopy) - `/contexts` — Context files for Claude (frontend conventions, graph editor, Chrome tools, server ops, demo mode) +## Android Dependency Sync (CRITICAL) + +The Android app (`android/app/build.gradle.kts`) installs the server package with `--no-deps` and lists Android-compatible dependencies **explicitly** in the Chaquopy `pip {}` block. This is because `server/pyproject.toml` includes desktop-only packages (mss, psutil, sounddevice, etc.) that have no Android wheels. + +**When adding a new dependency to `server/pyproject.toml`:** + +1. If the package is **pure Python or has Chaquopy wheels** (check [Chaquopy PyPI](https://chaquo.com/pypi-13.1/)), also add it to `android/app/build.gradle.kts` in the `pip { install(...) }` block +2. If the package is **desktop-only** (native C/Rust extension without Android support), do NOT add it to `build.gradle.kts` — and guard its import with `try/except ImportError` in Python code +3. If unsure, check Chaquopy's package index first + +**Incident context:** Chaquopy's pip runs on the build machine (Windows), not on Android. Platform markers like `sys_platform != 'linux'` evaluate against the BUILD host, not the target device. `pip install --exclude` does not exist. The only reliable way to exclude packages is to not list them. + ## Context Files | File | When to read | diff --git a/TODO.md b/TODO.md index c37aef9..a42dd1b 100644 --- a/TODO.md +++ b/TODO.md @@ -1,34 +1,97 @@ -# Composite Nesting Support +# LedGrab TODO -## Phase 1: Store — Cycle & Depth Validation +## Android — Restore Multi-ABI Wheels -- [x] Add `get_transitive_dependencies()` to ColorStripStore -- [x] Add `validate_nesting()` with cycle detection + depth limit (MAX_DEPTH=4) +During emulator testing, we switched the build to **x86 only** (see `android/app/build.gradle.kts` `abiFilters`) to avoid having to keep the arm64-v8a / x86_64 pydantic-core wheels current. Before shipping, restore all three ABIs: -## Phase 2: API — Validation in Create/Update +- [ ] Rebuild `pydantic-core` wheels for all three ABIs with the current SOABI + libpython linking settings: + - `arm64-v8a` (real TV boxes — the primary target) + - `x86_64` (modern emulators) + - `x86` (legacy emulators) +- [ ] Verify wheels with `readelf -d`: SONAME must be `libpython3.11.so` in NEEDED +- [ ] Restore `abiFilters += listOf("arm64-v8a", "x86_64", "x86")` in `build.gradle.kts` +- [ ] Re-test on real ARM64 Android TV hardware -- [x] Call `validate_nesting()` in create handler for composite sources -- [x] Call `validate_nesting()` in update handler for composite sources +Build cache + scripts live in `android/build-scripts/` and `android/.build-cache/` (junction host + sysconfigdata for each ABI). -## Phase 3: Runtime — Depth Guard in Stream +## Android CI Pipeline -- [x] Add `depth` parameter to CompositeColorStripStream -- [x] Pass depth through ColorStripStreamManager.acquire() -- [x] Cap depth at runtime to prevent runaway nesting +Build the Android APK automatically on push/tag. -## Phase 4: Frontend — Allow Composites in Layer Dropdown +- [ ] Generate Gradle wrapper (`gradlew`) and commit it +- [ ] Create CI workflow (`.gitea/workflows/build-android.yaml` or `.github/workflows/`) + - JDK 17 + Android SDK + NDK setup + - Python 3.11 for Chaquopy build + - Recreate the directory junction (`ln -s` on Linux, `mklink /J` on Windows) + - `./gradlew assembleRelease` + - Upload APK as artifact +- [ ] Commit pre-built pydantic-core wheels to `android/wheels/` (arm64, x86, x86_64) +- [ ] APK signing for release builds (keystore setup) +- [ ] Consider: publish APK to GitHub/Gitea releases on tag push -- [x] Remove `source_type !== 'composite'` filter (keep self-exclusion) -- [x] Update docstring in composite_stream.py +## Android Root Capture (No Permission Dialog, No System Indicator) -## Phase 5: Tests +MediaProjection shows a mandatory system overlay/indicator while capturing — unavoidable on stock Android. Many cheap Android TV boxes ship pre-rooted, so an alternative root-only path would give much better UX. -- [x] Cycle detection tests (A→B→A, self-reference) -- [x] Depth limit tests (chain exceeding MAX_DEPTH) -- [x] Valid nesting tests (A→B both composite, no cycle) +- [ ] Detect root at runtime: check for `su` binary, `Superuser.apk`, etc. +- [ ] Implement `SurfaceControlCaptureEngine` (new capture engine) using hidden `SurfaceControl.screenshot()` API via reflection + - No permission dialog + - No system capture indicator + - Direct bitmap output (no encoder/decoder roundtrip) +- [ ] Engine priority: higher than MediaProjection when root detected +- [ ] Fallback chain: `SurfaceControl` (root) → `MediaProjection` (stock) → `adb screencap` (last resort) +- [ ] Handle Android version differences in `SurfaceControl` API surface (renamed/moved across API 29, 30, 33) +- [ ] Alternative: shell out to `screenrecord --output-format=h264 -` as root (same H.264 decode as scrcpy_client_engine, but local instead of remote ADB) -## Phase 6: Lint & Build +Known projects using this approach for reference: scrcpy-hidden-api, shizuku, commercial scrcpy-derived apps. -- [x] Ruff check passes -- [x] TypeScript build passes -- [x] All existing tests pass (715/715) +## Android USB Serial Support + +Drive USB LED controllers (APA102, WS2812) connected directly to the Android TV box via USB-to-serial adapters. + +- [ ] Add [usb-serial-for-android](https://github.com/mik3y/usb-serial-for-android) dependency to `android/app/build.gradle.kts` +- [ ] Create Kotlin `UsbSerialBridge` class that: + - Enumerates USB serial devices via Android USB Host API + - Requests user permission for USB device access + - Opens a serial connection (baud rate configurable) + - Exposes a write method callable from Python via Chaquopy +- [ ] Create Python `AndroidSerialProvider` in `server/src/ledgrab/core/devices/` that: + - Replaces `pyserial` on Android (which can't access USB ports) + - Calls `UsbSerialBridge` via Chaquopy to send LED data + - Registers as an alternative serial transport when `is_android()` is True +- [ ] Add USB device permission dialog to `MainActivity` (auto-triggered on device connect) +- [ ] Test with common USB-to-serial chips: CH340, CP2102, FTDI +- [ ] Document supported USB LED controllers in README + +## Android App — Known Issues + +Issues discovered during first end-to-end test on emulator (2026-04-14): + +- [ ] **Stop/Start capture produces no frames on second try.** First capture works, but after tapping Stop and Start again, no frames reach the Python pipeline. Likely causes: + - Global MediaProjection engine state (`_active`, `_frame_queue`) not reset on service restart + - ScreenCapture listener not properly detached from ImageReader on stop + - Python uvicorn port reuse issue when server restarts +- [ ] **Web UI tabs show persistent spinners** on Dashboard, Automation, Sources, Integrations, Graph tabs (Targets tab works). All API calls return 200 OK, no console errors. Probably waiting on a specific WebSocket event that never fires when no output targets are configured/active. +- [ ] **Dead keyboard/IME handling** — password inputs lack autocomplete attributes (minor accessibility warning in browser console) + +## Performance Metrics Abstraction + +The codebase has direct `psutil.*` calls scattered across `api/routes/system.py` and `core/processing/metrics_history.py`, with ad-hoc `if psutil is not None` guards sprinkled in to support Android. This couples Android platform handling to every call site. + +- [ ] Refactor: introduce `MetricsProvider` protocol in `utils/metrics.py` with methods like `cpu_percent()`, `memory_info()`, `process_info()` +- [ ] Implement `PsutilMetricsProvider` (desktop) and `NullMetricsProvider` (fallback when psutil missing) +- [ ] Later: `AndroidMetricsProvider` reading from `/proc` (see section below) +- [ ] Replace all direct `psutil.*` calls with the provider; only one factory location knows about psutil availability + +## Android Performance Metrics + +Currently `psutil` (used for CPU/RAM monitoring in the web UI) is not available on Android via Chaquopy. Metrics calls are guarded with `if psutil is not None` so they return no data on Android. + +- [ ] Implement Android-native metrics collection: + - CPU usage via `/proc/self/stat` + `/proc/stat` parsing (no psutil needed) + - RAM usage via `/proc/meminfo` or `ActivityManager.getMemoryInfo()` through Chaquopy bridge + - App-specific memory via `Debug.getMemoryInfo()` (Kotlin → Python) +- [ ] Create `AndroidMetricsProvider` in `server/src/ledgrab/utils/` that implements the same interface as the psutil-based provider +- [ ] Wire into existing metrics endpoints (`/api/v1/system/metrics`) with platform detection +- [ ] Consider: device battery/temperature readings for TV boxes (some have thermal throttling) +- [ ] Optional: GPU usage via `/sys/class/kgsl/kgsl-3d0/gpubusy` on Adreno, Mali-specific paths for Mali GPUs diff --git a/android/.gitignore b/android/.gitignore new file mode 100644 index 0000000..cda6433 --- /dev/null +++ b/android/.gitignore @@ -0,0 +1,17 @@ +*.iml +.gradle +/local.properties +/.idea +.DS_Store +/build +/app/build +/captures +.externalNativeBuild +.cxx +local.properties + +# Chaquopy build cache +.build-cache/ + +# Python source junction (points at ../server/src/ledgrab — do not commit) +/app/src/main/python/ledgrab diff --git a/android/app/build.gradle.kts b/android/app/build.gradle.kts new file mode 100644 index 0000000..3217dbd --- /dev/null +++ b/android/app/build.gradle.kts @@ -0,0 +1,92 @@ +plugins { + id("com.android.application") + id("org.jetbrains.kotlin.android") + id("com.chaquo.python") +} + +android { + namespace = "com.ledgrab.android" + compileSdk = 34 + + defaultConfig { + applicationId = "com.ledgrab.android" + minSdk = 24 // Android 7.0 — covers nearly all TV boxes + targetSdk = 34 + versionCode = 1 + versionName = "0.3.0" + + ndk { + // Temporarily x86 only for emulator testing with new wheel + abiFilters += listOf("x86") + } + } + + buildTypes { + release { + isMinifyEnabled = false + } + } + + compileOptions { + sourceCompatibility = JavaVersion.VERSION_17 + targetCompatibility = JavaVersion.VERSION_17 + } + + kotlinOptions { + jvmTarget = "17" + } +} + +chaquopy { + defaultConfig { + version = "3.11" + + pip { + // Pre-built wheels directory (for pydantic-core etc.) + // Must use absolute file:// URI with forward slashes + options("--find-links", "file:///${rootDir.absolutePath.replace("\\", "/")}/wheels/") + + // ── Android-compatible dependencies ───────────────── + // Listed explicitly because pyproject.toml includes + // desktop-only packages with no Android wheels. + // See CLAUDE.md "Android Dependency Sync" for policy. + install("fastapi") + install("uvicorn") // without [standard] — no uvloop/httptools + install("httpx") + install("numpy") + install("pydantic") // needs pydantic-core wheel in wheels/ + install("pydantic-settings") + install("PyYAML") + install("structlog") + install("python-json-logger") + install("python-dateutil") + install("python-multipart") + install("jinja2") + install("zeroconf") + install("aiomqtt") + install("openrgb-python") + // opencv-python-headless: no cp311 Android wheel on Chaquopy. + // LedGrab's cv2 usage is guarded with try/except ImportError + // and falls back to numpy/Pillow alternatives on Android. + install("Pillow") + install("websockets") + } + } +} + +// LedGrab Python source is included via a directory junction: +// android/app/src/main/python/ledgrab -> server/src/ledgrab +// This is the standard Chaquopy way to include local Python packages. +// Create the junction (run from repo root, no admin needed): +// cmd /c "mklink /J android\app\src\main\python\ledgrab server\src\ledgrab" + + +dependencies { + implementation("androidx.core:core-ktx:1.13.1") + implementation("androidx.appcompat:appcompat:1.7.0") + implementation("androidx.leanback:leanback:1.0.0") + implementation("com.google.android.material:material:1.12.0") + implementation("androidx.lifecycle:lifecycle-service:2.8.7") + // QR code generation for displaying server URL on TV + implementation("com.google.zxing:core:3.5.3") +} diff --git a/android/app/src/main/AndroidManifest.xml b/android/app/src/main/AndroidManifest.xml new file mode 100644 index 0000000..4e81310 --- /dev/null +++ b/android/app/src/main/AndroidManifest.xml @@ -0,0 +1,49 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/android/app/src/main/java/com/ledgrab/android/CaptureService.kt b/android/app/src/main/java/com/ledgrab/android/CaptureService.kt new file mode 100644 index 0000000..cb3fe4b --- /dev/null +++ b/android/app/src/main/java/com/ledgrab/android/CaptureService.kt @@ -0,0 +1,161 @@ +package com.ledgrab.android + +import android.app.Notification +import android.app.NotificationChannel +import android.app.NotificationManager +import android.app.Service +import android.content.Context +import android.content.Intent +import android.media.projection.MediaProjection +import android.media.projection.MediaProjectionManager +import android.os.Build +import android.os.IBinder +import android.util.DisplayMetrics +import android.util.Log +import android.view.WindowManager +import androidx.core.app.NotificationCompat + +/** + * Foreground service that runs the Python LedGrab server and captures + * the screen via MediaProjection. + */ +class CaptureService : Service() { + + companion object { + private const val TAG = "CaptureService" + private const val CHANNEL_ID = "ledgrab_capture" + private const val NOTIFICATION_ID = 1 + private const val EXTRA_RESULT_CODE = "result_code" + private const val EXTRA_RESULT_DATA = "result_data" + private const val SERVER_PORT = 8080 + + fun createIntent( + context: Context, + resultCode: Int, + resultData: Intent, + ): Intent { + return Intent(context, CaptureService::class.java).apply { + putExtra(EXTRA_RESULT_CODE, resultCode) + putExtra(EXTRA_RESULT_DATA, resultData) + } + } + } + + private var bridge: PythonBridge? = null + private var screenCapture: ScreenCapture? = null + private var mediaProjection: MediaProjection? = null + + override fun onBind(intent: Intent?): IBinder? = null + + override fun onCreate() { + super.onCreate() + createNotificationChannel() + } + + override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int { + // CRITICAL: startForeground must be called IMMEDIATELY — + // before any other work, especially before getMediaProjection(). + val localIp = NetworkUtils.getLocalIpAddress(this) ?: "unknown" + val url = "http://$localIp:$SERVER_PORT" + startForeground(NOTIFICATION_ID, buildNotification(url)) + + if (intent == null) { + Log.w(TAG, "Service restarted without intent — stopping") + stopSelf() + return START_NOT_STICKY + } + + val resultCode = intent.getIntExtra(EXTRA_RESULT_CODE, 0) + @Suppress("DEPRECATION") + val resultData = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) { + intent.getParcelableExtra(EXTRA_RESULT_DATA, Intent::class.java) + } else { + intent.getParcelableExtra(EXTRA_RESULT_DATA) + } + + if (resultData == null) { + Log.e(TAG, "No MediaProjection result data") + stopSelf() + return START_NOT_STICKY + } + + try { + // Create MediaProjection from the consent token + val projectionManager = + getSystemService(Context.MEDIA_PROJECTION_SERVICE) as MediaProjectionManager + mediaProjection = projectionManager.getMediaProjection(resultCode, resultData) + + if (mediaProjection == null) { + Log.e(TAG, "Failed to create MediaProjection") + stopSelf() + return START_NOT_STICKY + } + + // Get screen metrics + val windowManager = getSystemService(Context.WINDOW_SERVICE) as WindowManager + val metrics = DisplayMetrics() + @Suppress("DEPRECATION") + windowManager.defaultDisplay.getRealMetrics(metrics) + + // Initialize Python bridge and configure capture engine + bridge = PythonBridge(this).also { b -> + b.configureCapture(480, 270) // Downscaled for LED use + b.startServer(SERVER_PORT) + } + + // Start screen capture + screenCapture = ScreenCapture( + projection = mediaProjection!!, + metrics = metrics, + bridge = bridge!!, + targetWidth = 480, + targetHeight = 270, + targetFps = 30, + ).also { it.start() } + + Log.i(TAG, "LedGrab service started — web UI at $url") + } catch (e: Exception) { + Log.e(TAG, "Failed to start capture", e) + stopSelf() + } + + return START_NOT_STICKY + } + + override fun onDestroy() { + screenCapture?.stop() + screenCapture = null + + bridge?.stopServer() + bridge = null + + mediaProjection?.stop() + mediaProjection = null + + Log.i(TAG, "LedGrab service destroyed") + super.onDestroy() + } + + private fun createNotificationChannel() { + if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) { + val channel = NotificationChannel( + CHANNEL_ID, + "LedGrab Screen Capture", + NotificationManager.IMPORTANCE_LOW, + ).apply { + description = "Shows while LedGrab is capturing the screen" + } + val manager = getSystemService(NotificationManager::class.java) + manager.createNotificationChannel(channel) + } + } + + private fun buildNotification(url: String): Notification { + return NotificationCompat.Builder(this, CHANNEL_ID) + .setContentTitle("LedGrab Running") + .setContentText("Web UI: $url") + .setSmallIcon(R.drawable.ic_launcher) + .setOngoing(true) + .build() + } +} diff --git a/android/app/src/main/java/com/ledgrab/android/LedGrabApp.kt b/android/app/src/main/java/com/ledgrab/android/LedGrabApp.kt new file mode 100644 index 0000000..9288608 --- /dev/null +++ b/android/app/src/main/java/com/ledgrab/android/LedGrabApp.kt @@ -0,0 +1,22 @@ +package com.ledgrab.android + +import android.app.Application +import com.chaquo.python.Python +import com.chaquo.python.android.AndroidPlatform + +/** + * Application class — initializes the Chaquopy Python runtime. + * + * `Python.start()` must be called once before any Python code runs. + * It loads libpython, extracts stdlib + pip packages from APK assets + * (first launch only), and sets up `sys.path`. + */ +class LedGrabApp : Application() { + + override fun onCreate() { + super.onCreate() + if (!Python.isStarted()) { + Python.start(AndroidPlatform(this)) + } + } +} diff --git a/android/app/src/main/java/com/ledgrab/android/MainActivity.kt b/android/app/src/main/java/com/ledgrab/android/MainActivity.kt new file mode 100644 index 0000000..ba7375d --- /dev/null +++ b/android/app/src/main/java/com/ledgrab/android/MainActivity.kt @@ -0,0 +1,129 @@ +package com.ledgrab.android + +import android.app.Activity +import android.content.Intent +import android.graphics.Bitmap +import android.media.projection.MediaProjectionManager +import android.os.Bundle +import android.util.Log +import android.view.View +import android.widget.Button +import android.widget.ImageView +import android.widget.TextView +import com.google.zxing.BarcodeFormat +import com.google.zxing.qrcode.QRCodeWriter + +/** + * Main (and only) Activity for the Android TV app. + * + * Two-state UI: stopped (Start button) and running (URL + QR + Stop). + * Navigable with D-pad / USB controller. + */ +class MainActivity : Activity() { + + companion object { + private const val TAG = "MainActivity" + private const val SERVER_PORT = 8080 + private const val REQUEST_MEDIA_PROJECTION = 1001 + } + + private lateinit var stoppedPanel: View + private lateinit var runningPanel: View + private lateinit var statusText: TextView + private lateinit var urlText: TextView + private lateinit var qrImage: ImageView + private lateinit var toggleButton: Button + private lateinit var stopButtonRunning: Button + private lateinit var versionText: TextView + private var serviceRunning = false + + override fun onCreate(savedInstanceState: Bundle?) { + super.onCreate(savedInstanceState) + setContentView(R.layout.activity_main) + + stoppedPanel = findViewById(R.id.stopped_panel) + runningPanel = findViewById(R.id.running_panel) + statusText = findViewById(R.id.status_text) + urlText = findViewById(R.id.url_text) + qrImage = findViewById(R.id.qr_image) + toggleButton = findViewById(R.id.toggle_button) + stopButtonRunning = findViewById(R.id.stop_button_running) + versionText = findViewById(R.id.version_text) + + val versionName = packageManager + .getPackageInfo(packageName, 0).versionName + versionText.text = "v$versionName" + + toggleButton.setOnClickListener { requestMediaProjection() } + stopButtonRunning.setOnClickListener { stopCaptureService() } + + updateUI() + } + + private fun requestMediaProjection() { + val manager = getSystemService(MEDIA_PROJECTION_SERVICE) as MediaProjectionManager + @Suppress("DEPRECATION") + startActivityForResult(manager.createScreenCaptureIntent(), REQUEST_MEDIA_PROJECTION) + } + + @Deprecated("Using deprecated API for plain Activity compatibility") + override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) { + super.onActivityResult(requestCode, resultCode, data) + if (requestCode == REQUEST_MEDIA_PROJECTION) { + if (resultCode == RESULT_OK && data != null) { + startCaptureService(resultCode, data) + } else { + statusText.text = "Permission denied — screen capture requires authorization" + Log.w(TAG, "MediaProjection permission denied") + } + } + } + + private fun startCaptureService(resultCode: Int, resultData: Intent) { + val intent = CaptureService.createIntent(this, resultCode, resultData) + startForegroundService(intent) + serviceRunning = true + updateUI() + } + + private fun stopCaptureService() { + stopService(Intent(this, CaptureService::class.java)) + serviceRunning = false + updateUI() + } + + private fun updateUI() { + if (serviceRunning) { + val localIp = NetworkUtils.getLocalIpAddress(this) ?: "unknown" + val url = "http://$localIp:$SERVER_PORT" + + urlText.text = url + qrImage.setImageBitmap(generateQrCode(url)) + + stoppedPanel.visibility = View.GONE + versionText.visibility = View.GONE + runningPanel.visibility = View.VISIBLE + stopButtonRunning.requestFocus() + } else { + urlText.text = "" + qrImage.setImageBitmap(null) + + runningPanel.visibility = View.GONE + stoppedPanel.visibility = View.VISIBLE + versionText.visibility = View.VISIBLE + toggleButton.requestFocus() + } + } + + private fun generateQrCode(text: String): Bitmap { + val size = 560 + val bitMatrix = QRCodeWriter().encode(text, BarcodeFormat.QR_CODE, size, size) + val bitmap = Bitmap.createBitmap(size, size, Bitmap.Config.RGB_565) + for (x in 0 until size) { + for (y in 0 until size) { + bitmap.setPixel(x, y, if (bitMatrix[x, y]) 0xFF000000.toInt() else 0xFFFFFFFF.toInt()) + } + } + return bitmap + } +} diff --git a/android/app/src/main/java/com/ledgrab/android/NetworkUtils.kt b/android/app/src/main/java/com/ledgrab/android/NetworkUtils.kt new file mode 100644 index 0000000..1a76222 --- /dev/null +++ b/android/app/src/main/java/com/ledgrab/android/NetworkUtils.kt @@ -0,0 +1,28 @@ +package com.ledgrab.android + +import android.content.Context +import android.net.ConnectivityManager +import android.net.LinkProperties +import java.net.Inet4Address + +/** + * Network utilities for discovering the device's local IP address. + */ +object NetworkUtils { + + /** + * Return the device's local IPv4 address on the active network, + * or `null` if unavailable. + */ + fun getLocalIpAddress(context: Context): String? { + val cm = context.getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager + val network = cm.activeNetwork ?: return null + val props: LinkProperties = cm.getLinkProperties(network) ?: return null + + return props.linkAddresses + .map { it.address } + .filterIsInstance() + .firstOrNull { !it.isLoopbackAddress } + ?.hostAddress + } +} diff --git a/android/app/src/main/java/com/ledgrab/android/PythonBridge.kt b/android/app/src/main/java/com/ledgrab/android/PythonBridge.kt new file mode 100644 index 0000000..0798fbd --- /dev/null +++ b/android/app/src/main/java/com/ledgrab/android/PythonBridge.kt @@ -0,0 +1,102 @@ +package com.ledgrab.android + +import android.content.Context +import android.util.Log +import com.chaquo.python.Python + +/** + * Bridge between Kotlin and the LedGrab Python server. + * + * All Python calls go through Chaquopy's `Python.getInstance()`. + * Frame data crosses the JNI boundary as a `ByteArray`. + */ +class PythonBridge(private val context: Context) { + + companion object { + private const val TAG = "PythonBridge" + } + + private var serverThread: Thread? = null + @Volatile private var running = false + + /** + * Configure the MediaProjection engine with screen dimensions. + * Must be called before [startServer]. + */ + fun configureCapture(width: Int, height: Int) { + val py = Python.getInstance() + val engine = py.getModule("ledgrab.core.capture_engines.mediaprojection_engine") + engine.callAttr("configure", width, height) + Log.i(TAG, "MediaProjection engine configured: ${width}x${height}") + } + + /** + * Start the LedGrab FastAPI server on a background thread. + * + * This blocks until [stopServer] is called, so it runs in its own thread. + */ + fun startServer(port: Int = 8080) { + if (running) { + Log.w(TAG, "Server already running") + return + } + + running = true + val dataDir = context.filesDir.absolutePath + + serverThread = Thread({ + try { + Log.i(TAG, "Starting Python server (dataDir=$dataDir, port=$port)") + val py = Python.getInstance() + val entry = py.getModule("ledgrab.android_entry") + entry.callAttr("start_server", dataDir, port) + } catch (e: Exception) { + Log.e(TAG, "Python server error", e) + } finally { + running = false + } + }, "ledgrab-python-server") + serverThread?.start() + } + + /** + * Signal the Python server to shut down gracefully. + */ + fun stopServer() { + if (!running) return + + try { + val py = Python.getInstance() + val entry = py.getModule("ledgrab.android_entry") + entry.callAttr("stop_server") + } catch (e: Exception) { + Log.e(TAG, "Error stopping server", e) + } + + serverThread?.join(10_000) + serverThread = null + running = false + Log.i(TAG, "Server stopped") + } + + /** + * Push a captured RGBA frame to the Python MediaProjection engine. + * + * Called from [ScreenCapture] on the capture thread. The byte array + * crosses the JNI boundary — keep frames small (downscale to 480p + * before calling). + */ + fun pushFrame(rgbaBytes: ByteArray, width: Int, height: Int) { + if (!running) return + + try { + val py = Python.getInstance() + val engine = py.getModule("ledgrab.core.capture_engines.mediaprojection_engine") + engine.callAttr("push_frame", rgbaBytes, width, height) + } catch (e: Exception) { + Log.w(TAG, "Failed to push frame: ${e.message}") + } + } + + val isRunning: Boolean get() = running +} diff --git a/android/app/src/main/java/com/ledgrab/android/ScreenCapture.kt b/android/app/src/main/java/com/ledgrab/android/ScreenCapture.kt new file mode 100644 index 0000000..8c6023a --- /dev/null +++ b/android/app/src/main/java/com/ledgrab/android/ScreenCapture.kt @@ -0,0 +1,140 @@ +package com.ledgrab.android + +import android.graphics.Bitmap +import android.graphics.PixelFormat +import android.hardware.display.DisplayManager +import android.hardware.display.VirtualDisplay +import android.media.ImageReader +import android.media.projection.MediaProjection +import android.os.Handler +import android.os.HandlerThread +import android.util.DisplayMetrics +import android.util.Log +import java.nio.ByteBuffer + +/** + * Captures the Android screen via MediaProjection and feeds frames + * to [PythonBridge]. + * + * Frames are downscaled to [targetWidth] x [targetHeight] before + * crossing the JNI boundary to minimize overhead. For LED ambient + * lighting, even 480x270 contains far more data than needed. + */ +class ScreenCapture( + private val projection: MediaProjection, + private val metrics: DisplayMetrics, + private val bridge: PythonBridge, + private val targetWidth: Int = 480, + private val targetHeight: Int = 270, + private val targetFps: Int = 30, +) { + companion object { + private const val TAG = "ScreenCapture" + private const val VIRTUAL_DISPLAY_NAME = "LedGrabCapture" + } + + private var virtualDisplay: VirtualDisplay? = null + private var imageReader: ImageReader? = null + private var captureThread: HandlerThread? = null + private var captureHandler: Handler? = null + @Volatile private var running = false + private var lastFrameTimeMs = 0L + private val frameIntervalMs = 1000L / targetFps + + /** + * Start capturing the screen. + */ + fun start() { + if (running) return + running = true + + captureThread = HandlerThread("LedGrab-Capture").also { it.start() } + captureHandler = Handler(captureThread!!.looper) + + // Android 14+ requires registering a callback before createVirtualDisplay + projection.registerCallback(object : MediaProjection.Callback() { + override fun onStop() { + Log.i(TAG, "MediaProjection stopped") + stop() + } + }, captureHandler) + + imageReader = ImageReader.newInstance( + targetWidth, + targetHeight, + PixelFormat.RGBA_8888, + 2, // maxImages — double buffer + ) + + imageReader?.setOnImageAvailableListener({ reader -> + if (!running) return@setOnImageAvailableListener + + val now = System.currentTimeMillis() + if (now - lastFrameTimeMs < frameIntervalMs) { + // Skip frame to maintain target FPS + reader.acquireLatestImage()?.close() + return@setOnImageAvailableListener + } + + val image = reader.acquireLatestImage() ?: return@setOnImageAvailableListener + try { + val plane = image.planes[0] + val buffer = plane.buffer + val rowStride = plane.rowStride + val pixelStride = plane.pixelStride + + // Handle row padding: rowStride may be > width * pixelStride + val rgbaBytes = if (rowStride == targetWidth * pixelStride) { + // No padding — direct copy + val bytes = ByteArray(buffer.remaining()) + buffer.get(bytes) + bytes + } else { + // Strip row padding + val rowBytes = targetWidth * pixelStride + val bytes = ByteArray(targetWidth * targetHeight * 4) + for (row in 0 until targetHeight) { + buffer.position(row * rowStride) + buffer.get(bytes, row * rowBytes, rowBytes) + } + bytes + } + + bridge.pushFrame(rgbaBytes, targetWidth, targetHeight) + lastFrameTimeMs = now + } catch (e: Exception) { + Log.w(TAG, "Frame processing error: ${e.message}") + } finally { + image.close() + } + }, captureHandler) + + virtualDisplay = projection.createVirtualDisplay( + VIRTUAL_DISPLAY_NAME, + targetWidth, + targetHeight, + metrics.densityDpi, + DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR, + imageReader?.surface, + null, + captureHandler, + ) + + Log.i(TAG, "Screen capture started (${targetWidth}x${targetHeight} @ ${targetFps}fps)") + } + + /** + * Stop capturing and release all resources. + */ + fun stop() { + running = false + virtualDisplay?.release() + virtualDisplay = null + imageReader?.close() + imageReader = null + captureThread?.quitSafely() + captureThread = null + captureHandler = null + Log.i(TAG, "Screen capture stopped") + } +} diff --git a/android/app/src/main/res/color/btn_secondary_text.xml b/android/app/src/main/res/color/btn_secondary_text.xml new file mode 100644 index 0000000..85c75e2 --- /dev/null +++ b/android/app/src/main/res/color/btn_secondary_text.xml @@ -0,0 +1,5 @@ + + + + + diff --git a/android/app/src/main/res/drawable/bg_button_primary.xml b/android/app/src/main/res/drawable/bg_button_primary.xml new file mode 100644 index 0000000..1b9e53e --- /dev/null +++ b/android/app/src/main/res/drawable/bg_button_primary.xml @@ -0,0 +1,31 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/android/app/src/main/res/drawable/bg_button_secondary.xml b/android/app/src/main/res/drawable/bg_button_secondary.xml new file mode 100644 index 0000000..2c0a3e1 --- /dev/null +++ b/android/app/src/main/res/drawable/bg_button_secondary.xml @@ -0,0 +1,33 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/android/app/src/main/res/drawable/bg_main.xml b/android/app/src/main/res/drawable/bg_main.xml new file mode 100644 index 0000000..e64211e --- /dev/null +++ b/android/app/src/main/res/drawable/bg_main.xml @@ -0,0 +1,28 @@ + + + + + + + + + + + + + + + + diff --git a/android/app/src/main/res/drawable/bg_qr_container.xml b/android/app/src/main/res/drawable/bg_qr_container.xml new file mode 100644 index 0000000..c963b9a --- /dev/null +++ b/android/app/src/main/res/drawable/bg_qr_container.xml @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + diff --git a/android/app/src/main/res/drawable/bg_status_dot.xml b/android/app/src/main/res/drawable/bg_status_dot.xml new file mode 100644 index 0000000..7df4689 --- /dev/null +++ b/android/app/src/main/res/drawable/bg_status_dot.xml @@ -0,0 +1,5 @@ + + + + diff --git a/android/app/src/main/res/drawable/bg_url_chip.xml b/android/app/src/main/res/drawable/bg_url_chip.xml new file mode 100644 index 0000000..0782e26 --- /dev/null +++ b/android/app/src/main/res/drawable/bg_url_chip.xml @@ -0,0 +1,7 @@ + + + + + + diff --git a/android/app/src/main/res/drawable/ic_launcher.xml b/android/app/src/main/res/drawable/ic_launcher.xml new file mode 100644 index 0000000..300da25 --- /dev/null +++ b/android/app/src/main/res/drawable/ic_launcher.xml @@ -0,0 +1,52 @@ + + + + + + + + + + + + + + + + + + + + + + diff --git a/android/app/src/main/res/layout/activity_main.xml b/android/app/src/main/res/layout/activity_main.xml new file mode 100644 index 0000000..0b9d19e --- /dev/null +++ b/android/app/src/main/res/layout/activity_main.xml @@ -0,0 +1,179 @@ + + + + + + + + + + + + +