Replace slow PIL LANCZOS downscaler with OpenCV INTER_AREA (10-20x faster),
remove FPS throttling to maximize throughput, and add idle sleeps to prevent
CPU spinning. Also fix pixel mapping boundary clamping off-by-one error.
Changes:
- Downscaler filter: Use cv2.resize() with INTER_AREA instead of PIL LANCZOS
- Live streams: Remove FPS throttling, add 1ms sleep during idle/duplicate frames
- Processor manager: Remove FPS control sleep to process frames as fast as available
- Calibration: Fix boundary clamping to prevent index out of bounds crashes
Results: Processed stream FPS improved from 27 to ~42 FPS with lower CPU usage.
Parallel I2S network send verified at 0.1-0.2ms (can handle 200+ FPS).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements transparent on-screen overlay that displays LED calibration data
directly on the target display for easier setup and debugging. Overlay shows
border zones, LED position axes with tick labels, and calibration details.
Features:
- Tkinter-based transparent overlay window with click-through support
- Border zones highlighting pixel sampling areas (colored rectangles)
- LED position axes with numbered tick marks at regular intervals
- Calibration info box showing target name, LED counts, and configuration
- Toggle button (eye icon) in target cards for show/hide
- Localized UI strings (English and Russian)
Implementation:
- New screen_overlay.py module with OverlayWindow and OverlayManager classes
- Overlay runs in background thread with proper asyncio integration
- API endpoints for start/stop/status overlay control
- overlay_active state tracking in processor manager
Known limitation: tkinter threading cleanup causes server restart when overlay
is closed, but functionality works correctly while overlay is active.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Eliminate 5 numpy↔tuple conversions per frame in processing hot path:
map_border_to_leds returns ndarray, inline numpy smoothing with integer
math, send_pixels_fast accepts ndarray directly
- Fix numpy boolean bug in keepalive check (use `is not None`)
- Add per-stage pipeline timing (extract/map/smooth/send) to metrics API
and UI with color-coded breakdown bar
- Expose device_fps from WLED health check in API schemas
- Auto-sync LED count from WLED device: health check detects changes and
updates storage, calibration, and active targets automatically
- Use integer math for brightness scaling (uint16 * brightness >> 8)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three changes to eliminate periodic LED stutters on high-LED-count
WLED devices:
1. DDP PUSH flag: re-enable on the last packet of each frame so WLED
waits for the complete frame before rendering (prevents tearing
from partial multi-packet frames).
2. Health check: skip HTTP probe while a target is actively streaming
to the device — the device is clearly online and the HTTP request
to the ESP causes LED output to stutter.
3. Brightness polling: cache the value after first fetch and reuse it
on subsequent 2-second UI refreshes instead of hitting the ESP
every cycle.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Baud rate is now a first-class device field, passed through the full
stack: Device model → API schemas → routes → ProcessorManager →
AdalightClient. The frontend shows a baud rate dropdown (115200–2M)
for Adalight devices in both Add Device and Settings modals, with a
live "Max FPS ≈ N" hint computed from LED count and baud rate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements the second device provider (Adalight) for Arduino-based serial LED controllers,
validating the LEDDeviceProvider abstraction. Adds serial port auto-discovery, per-type
discovery caching with lazy-load, capability-driven UI (brightness control, manual LED count,
standby), and serial port combobox in both Add Device and General Settings modals.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate all device-type-specific logic into LEDDeviceProvider ABC
with provider registry. WLEDDeviceProvider handles client creation,
health checks, validation, mDNS discovery, and brightness control.
Routes now delegate to providers instead of using if/else type checks.
Add standby_required capability and expose device capabilities in API.
Target editor conditionally shows standby interval based on selected
device's capabilities.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Scan the local network for WLED devices advertising _wled._tcp.local.
and present them in the Add Device modal for one-click selection.
- New discovery.py: async mDNS browse + parallel /json/info enrichment
- GET /api/v1/devices/discover endpoint with already_added dedup
- Header scan button (magnifying glass icon) in add-device modal
- Discovered devices show name, IP, LED count, version; click to fill form
- en/ru locale strings for discovery UI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce abstract LEDClient base class with factory pattern so new
LED controller types can plug in alongside WLED. ProcessorManager is
now fully type-agnostic — all device-specific logic (health checks,
state snapshot/restore, fast send) lives behind the LEDClient interface.
- New led_client.py: LEDClient ABC, DeviceHealth, factory functions
- WLEDClient inherits LEDClient, encapsulates WLED health checks and state management
- device_type field on Device storage model (defaults to "wled")
- Rename target_type "wled" → "led" with backward-compat migration
- Frontend: "WLED" tab → "LED", device type badge, type selector in
add-device modal, device type shown in target device dropdown
- All wled_* API fields renamed to device_* for generic naming
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Skip LEDs at the start/end of the strip are blacked out while the full
screen perimeter is resampled onto the remaining active LEDs using linear
interpolation. Calibration canvas tick labels show per-edge display
ranges clipped to the active LED range. Moved LED offset control from
inline overlay to a dedicated form row alongside the new skip inputs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix KC fps_actual to use frame-to-frame timestamps (was inflated by measuring before sleep)
- Add fps_potential, fps_current, frames_skipped, frames_keepalive metrics to KC loop
- Add keepalive broadcast for static frames so WS clients stay in sync
- Expose all KC metrics in get_kc_target_state() and update UI card to show 7 metrics
- Add auto-refresh play/pause button to KC test lightbox (polls every ~1s)
- Fix WebSocket color swatches computing hex from r,g,b when hex field is absent
- Fix WebSocket auth crash by using get_config() instead of module-level config variable
- Fix lightbox closing when clicking auto-refresh button (event bubbling)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-frame image.copy() in ProcessedLiveStream with a 3-slot
ring buffer that reuses pre-allocated arrays. Use 256-byte LUTs for
brightness and gamma filters instead of full float32 conversion. Add
reusable float32 buffer to saturation filter. Use pool scratch buffer
for flip filter. Remove redundant .copy() calls from WGC capture engine.
Release intermediate filter outputs back to ImagePool.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Skip redundant processing/DDP sends when screen is static using object
identity comparison. Add configurable standby interval to periodically
resend last frame keeping WLED in live mode. Track frames skipped,
keepalive count, and current FPS (rolling 1-second send count). Always
use DDP regardless of LED count. Compact metrics grid with label-value
rows and remove Skipped from UI display.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add numpy-based DDP pixel packing (send_pixels_numpy) and fast send
path (send_pixels_fast) eliminating per-pixel Python loops
- Move ProcessedLiveStream filter processing to background thread so
get_latest_frame() returns pre-computed cached result instantly
- Vectorize map_border_to_leds for average interpolation using cumulative
sums instead of 934 individual np.mean calls (~16ms -> <1ms)
- Batch all CPU work into single asyncio.to_thread call per frame
- Fix FPS calculation to measure frame-to-frame interval (was measuring
processing time only, reporting inflated values)
- Add Potential FPS metric showing theoretical max without throttling
- Add FPS label to WLED target card properties
- Add fps_potential field to TargetProcessingState API schema
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce Pattern Template entity as a reusable rectangle layout that
Key Colors targets reference via pattern_template_id. This replaces
inline rectangle storage with a shared template system.
Backend:
- New PatternTemplate data model, store (JSON persistence), CRUD API
- KC targets now reference pattern_template_id instead of inline rectangles
- ProcessorManager resolves pattern template at KC processing start
- Picture source test endpoint supports capture_duration=0 for single frame
- Delete protection: 409 when template is referenced by a KC target
Frontend:
- Pattern Templates section in Key Colors sub-tab with card UI
- Visual canvas editor with drag-to-move, 8-point resize handles
- Background capture from any picture source for visual alignment
- Precise coordinate list synced bidirectionally with canvas
- Resizable editor container, viewport-constrained modal
- KC target editor uses pattern template dropdown instead of inline rects
- Localization (en/ru) for all new UI elements
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce a new "key_colors" target type alongside WLED targets, enabling
real-time color extraction from configurable screen rectangles with
average/median/dominant modes, temporal smoothing, and WebSocket streaming.
- Split WledPictureTarget into its own module, add KeyColorsPictureTarget
- Add KC target lifecycle to ProcessorManager (register, start/stop, processing loop)
- Extend API routes and schemas for KC targets (CRUD, settings, state, metrics, colors)
- Add WebSocket endpoint for real-time color updates with auth
- Add KC sub-tab in Targets UI with editor modal and live color swatches
- Add EN and RU translations for all key colors strings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add PictureTarget entity that bridges PictureSource to output device,
separating processing settings from device connection/calibration state.
This enables future target types (Art-Net, E1.31) and cleanly decouples
"what to stream" from "where to stream."
- Add PictureTarget/WledPictureTarget dataclasses and storage
- Split ProcessorManager into DeviceState (health) + TargetState (processing)
- Add /api/v1/picture-targets endpoints (CRUD, start/stop, settings, metrics)
- Simplify device API (remove processing/settings/metrics endpoints)
- Auto-migrate existing device settings to picture targets on first startup
- Add Targets tab to WebUI with target cards and editor modal
- Add en/ru locale keys for targets UI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add GET /picture-streams/full-image endpoint to serve full-res images
- Click static image preview thumbnail to open full-res lightbox
- Snapshot WLED state (on/off, lor, AudioReactive) before streaming
- Restore saved WLED state when streaming stops
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix device settings partial update using model_fields_set for true merge
- Add missing interpolation_mode and smoothing to all API responses
- Fix send_pixels race condition when wled_client is None during stop
- Allow LED segments to exceed edge pixel count (float stepping)
- Fix modal scroll lock using position:fixed to prevent layout shift
- Show loading state for brightness slider until real value is fetched
- Remove stream description from stream selector dialog
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New FlipFilter with horizontal/vertical bool options (np.fliplr/flipud)
- Add bool option type to filter system (base.py validate, app.js toggle UI)
- Rearrange calibration preview: direction toggle above LED count
- Normalize direction/offset control heights, brighten offset icon
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix WLED DDP config: use lor=0 (live overrides effects), remove
live flag (read-only, causes issues on 0.15.x), disable PUSH flag
which broke rendering on WLED 0.15.x
- Use dedicated single-thread executor for capture engine calls to
fix thread-local state issues with BetterCam/MSS/DXcam
- Sync processor state on stream/template change even when stopped,
preventing stale engine references on next start
- Add diagnostic logging for frame sends and DDP packets
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add PUSH flag (0x01) on last DDP packet to signal frame completion
- Align packet payload to multiples of 3 bytes to prevent splitting
pixel RGB channels across packet boundaries
- Add per-bus WLED configuration logging (pin, color order, LED range)
- Add BusConfig dataclass and per-bus color reorder infrastructure
for future use
- Remove old per-pixel color reordering (WLED handles internally)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add BetterCam capture engine (DXGI Desktop Duplication, priority 4)
- Fix missing picture_stream_id in get_device endpoint
- Fix template delete validation to check streams instead of devices
- Add description field to capture engine template UI
- Default template name changed to "Default" with descriptive text
- Display picker highlights selected display instead of primary
- Fix modals closing when dragging text selection outside dialog
- Rename "Engine Configuration" to "Configuration", hide when empty
- Rename "Run Test" to "Run" across all test buttons
- Always reserve space for vertical scrollbar
- Redesign Stream Settings info panel with pill-style props
- Fix processed stream showing internal ID instead of stream name
- Update en/ru locale files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Merge Capture Templates and Processing Templates main tabs into Picture
Streams sub-tabs (Screen Capture shows streams + engine templates,
Processed shows streams + filter templates)
- Rename "Capture Templates" to "Engine Templates" and "Processing
Templates" to "Filter Templates" across all locale strings
- Rename "Picture Streams" tab to "Streams" throughout UI and locales
- Rename "WLED Screen Controller" to "WLED Grab" across all files
- Add subtab section headers and styling for merged template views
- Remove add card labels, keeping only plus icon for cleaner UI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces a new "static_image" stream type that loads a frame from a URL
or local file path, enabling LED testing with known images or displaying
static content. Includes validate-image API endpoint, auto-validation on
blur/enter/paste with caching, capture template names on stream cards,
and conditional test stats display for single-frame results.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace hardcoded gamma/saturation/brightness fields with a flexible
filter pipeline architecture. Templates now contain an ordered list of
filter instances, each with its own options schema. Filters operate on
full images before border extraction.
- Add filter framework: base class, registry, image pool, filter instance
- Implement 6 built-in filters: brightness, saturation, gamma, downscaler, pixelate, auto crop
- Move smoothing from PP templates to device stream settings (temporal, not spatial)
- Add GET /api/v1/filters endpoint for available filter types
- Dynamic filter UI in template modal with add/remove/reorder/collapse
- Replace camera icon with display icon for screen capture streams
- Legacy migration: existing templates auto-convert flat fields to filter list
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce Picture Stream abstraction that separates the capture pipeline into
composable layers: raw streams (display + capture engine + FPS) and processed
streams (source stream + postprocessing template). Devices reference a picture
stream instead of managing individual capture settings.
- Add PictureStream and PostprocessingTemplate data models and stores
- Add CRUD API endpoints for picture streams and postprocessing templates
- Add stream chain resolution in ProcessorManager for start_processing
- Add picture stream test endpoint with postprocessing preview support
- Add Stream Settings modal with border_width and interpolation_mode controls
- Add stream test modal with capture preview and performance metrics
- Add full frontend: Picture Streams tab, Processing Templates tab, stream
selector on device cards, test buttons on stream cards
- Add localization keys for all new features (en, ru)
- Migrate existing devices to picture streams on startup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DXcam engine overhaul:
- Remove all user-facing config (device_idx, output_idx, output_color)
since these are auto-resolved or hardcoded to RGB
- Use one-shot grab() mode with retry for reliability
- Lazily create camera per display via _ensure_camera()
- Clear dxcam global factory cache to prevent stale DXGI state
UI improvements:
- Replace "Loading..." text with CSS spinner animations
- Fix template card header gap on default cards (scope padding-right
to cards with remove button only via :has selector)
- Add auto-restart server rule to CLAUDE.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Generate default templates (MSS, DXcam, WGC) in memory from EngineRegistry at startup
- Only persist user-created templates to JSON, skip defaults on load/save
- Add capture_template_id to Device model and DeviceCreate schema
- Remember last used template in localStorage, use it for new devices with fallback
- Split Device Settings dialog into General Settings and Capture Settings
- Add capture settings button (🎬) to device card
- Separate default and custom templates with visual separator in Templates tab
- Add capture engine integration to ProcessorManager
- Add CLAUDE.md with git commit/push policy and server restart instructions
- Add en/ru localization for all new UI elements
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- MSS and DXcam now auto-initialize on first capture_display() call, matching WGC behavior
- Remove engine.initialize() call from test endpoint to prevent WGC from initializing monitor 0 unnecessarily
- This fixes the issue where testing WGC with secondary monitor would show borders on both displays
Previously, calling engine.initialize() would initialize WGC's monitor 0 by default, then capture_display() would initialize the requested monitor, causing both to be active. Now all engines consistently auto-initialize only when needed.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Refactored WGC engine to maintain separate capture instances per monitor
- Each monitor gets dedicated instance, control, frame storage, and events
- Supports simultaneous capture from multiple monitors using same template
- Fixed template test endpoint to avoid redundant monitor 0 initialization
- Removed monitor_index from WGC template configuration (monitor-agnostic)
This enables using the same WGC template for multiple devices capturing
from different monitors without conflicts or unexpected borders.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The critical bug was not capturing the return value from start_free_threaded(),
which returns a CaptureControl object with the proper wait() method.
Changes:
- Store CaptureControl from start_free_threaded() return value
- Use CaptureControl.wait() to block until thread finishes (not timeout)
- Remove redundant capture_control storage from frame callback
- Cleanup now properly releases GPU resources and yellow border
Tested: Yellow border now disappears immediately after capture test ends.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add WGC engine with aggressive COM object cleanup (multiple GC passes)
- Persist capture test duration to localStorage
- Show only short error messages in snackbar (details in console)
Note: WGC cleanup still has issues with yellow border persistence - needs further investigation
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Allow LEDs to cover only a fraction of each screen edge via draggable
span bars in the calibration UI. Per-edge start/end (0.0-1.0) values
control which portion of the screen border is sampled for LED colors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Detect LED chip type (WS2812B, SK6812, etc.) from WLED /json/cfg
- Show RGBW/RGB channel indicator with colored squares on device card
- Move version, display index, LED count to subtitle row under device name
- Add remove button as × icon in top-right corner of card
- Hide latency from card display (still visible in health dot tooltip)
- Fix display layout overflow on narrow viewports
- Add wled_rgbw and wled_led_type to ProcessingState API schema
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace segment-based calibration with core parameters (leds_top/right/bottom/left);
segments are now derived at runtime via lookup tables
- Fix clockwise/counterclockwise edge traversal order for all 8 start_position/layout
combinations (e.g. bottom_left+clockwise now correctly goes up-left first)
- Add pixel layout preview overlay with color-coded edges, LED index labels,
direction arrows, and start position marker
- Move "Add New Device" form into a modal dialog triggered by "+" button
- Add display index selector to device settings modal
- Migrate from requirements.txt to pyproject.toml for dependency management
- Update Dockerfile and docs to use `pip install .`
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add background health checks (GET /json/info) with configurable interval per device
- Auto-detect LED count from WLED device on add (remove led_count from create API)
- Add calibration test mode: toggle edges on/off with colored LEDs via PUT endpoint
- Show WLED firmware version badge and LED count badge on device cards
- Add modal dirty tracking with discard confirmation on close/backdrop click
- Fix layout jump when modals open by compensating for scrollbar width
- Add state_check_interval to settings API and UI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add DDP client for LED strips >500 LEDs (UDP port 4048), with automatic
fallback from HTTP JSON API when LED count exceeds limit
- Wrap blocking operations (screen capture, image processing) in
asyncio.to_thread() to prevent event loop starvation
- Turn on WLED device and enable live mode when starting DDP streaming
- Add LED strip offset field to calibration (rotates color array to match
physical LED position vs start corner)
- Add server management scripts (start, stop, restart, background start)
- Fix WebUI auth error handling and auto-refresh loop
- Add development API key to default config
- Add i18n translations for offset field (en/ru)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Server fixes:
- Fix numpy uint8 JSON serialization by converting to Python int
- Change WLED payload to flat array format [r,g,b,r,g,b,...]
- Add payload debugging logs (size, LED count, sample data)
Web UI improvements:
- Add comprehensive console logging for device errors
- Log actual error messages from state.errors array
- Log device operations (start/stop/add) with details
- Fix password field form validation warning
The HTTP API may have payload size limitations for large LED counts.
Consider UDP protocols (DDP/E1.31) for better real-time performance.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Added get_monitor_refresh_rates() function in monitor_names.py using Windows ctypes/DEVMODE to detect monitor refresh rates
- Updated DisplayInfo dataclass and Pydantic schema to include refresh_rate field (in Hz)
- Modified get_available_displays() to detect and include refresh rates (defaults to 60Hz on non-Windows or if detection fails)
- Added refresh rate display in Web UI between Resolution and Position
- Added translations for refresh rate label (displays.refresh_rate) in English and Russian locales
- Cross-platform compatible: gracefully falls back to 60Hz default on non-Windows systems
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This is a complete WLED ambient lighting controller that captures screen border pixels
and sends them to WLED devices for immersive ambient lighting effects.
## Server Features:
- FastAPI-based REST API with 17+ endpoints
- Real-time screen capture with multi-monitor support
- Advanced LED calibration system with visual GUI
- API key authentication with labeled tokens
- Per-device brightness control (0-100%)
- Configurable FPS (1-60), border width, and color correction
- Persistent device storage (JSON-based)
- Comprehensive Web UI with dark/light themes
- Docker support with docker-compose
- Windows monitor name detection via WMI (shows "LG ULTRAWIDE" etc.)
## Web UI Features:
- Device management (add, configure, remove WLED devices)
- Real-time status monitoring with FPS metrics
- Settings modal for device configuration
- Visual calibration GUI with edge testing
- Brightness slider per device
- Display selection with friendly monitor names
- Token-based authentication with login/logout
- Responsive button layout
## Calibration System:
- Support for any LED strip layout (clockwise/counterclockwise)
- 4 starting position options (corners)
- Per-edge LED count configuration
- Visual preview with starting position indicator
- Test buttons to light up individual edges
- Smart LED ordering based on start position and direction
## Home Assistant Integration:
- Custom HACS integration
- Switch entities for processing control
- Sensor entities for status and FPS
- Select entities for display selection
- Config flow for easy setup
- Auto-discovery of devices from server
## Technical Stack:
- Python 3.11+
- FastAPI + uvicorn
- mss (screen capture)
- httpx (async WLED client)
- Pydantic (validation)
- WMI (Windows monitor detection)
- Structlog (logging)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
