# Phase 4: Runtime Integration **Status:** ✅ Done **Parent plan:** [PLAN.md](./PLAN.md) **Domain:** backend ## Objective Wire the audio filter pipeline into the runtime audio streaming system so that ProcessedAudioSources actually apply their filter chains to live audio data. ## Tasks - [x] Task 1: Create filter pipeline executor in `core/audio/filters/pipeline.py` - `AudioFilterPipeline` class: - `__init__(filter_instances: List[FilterInstance], registry: AudioFilterRegistry)` - Instantiates all filters from FilterInstance specs - `process(analysis: AudioAnalysis) -> AudioAnalysis` — runs analysis through all filters in order - `reset()` — resets all stateful filters - `close()` — cleanup resources - Handles stateful filter lifecycle (create on init, reset on demand, close on cleanup) - [x] Task 2: Update `AudioColorStripStream` in `core/processing/audio_stream.py` - On construction: if source is ProcessedAudioSource, resolve the full chain: - Walk to CaptureAudioSource for device info - Collect all AudioProcessingTemplates along the chain - Resolve all filter instances (with template expansion) - Create AudioFilterPipeline - In render loop: after getting AudioAnalysis from ManagedAudioStream, run it through the filter pipeline before visualization - Remove old inline channel selection and band filtering code (now handled by filters) - On stop: close the filter pipeline - [x] Task 3: Update `AudioValueStream` in `core/processing/value_stream.py` - Same pattern: resolve ProcessedAudioSource chain, create filter pipeline, apply in get_value() - Remove old inline channel/band handling - [x] Task 4: Hot-update support for filter templates - When an AudioProcessingTemplate is updated, running streams that use it should re-resolve their filter pipeline - Listen for template update events (or implement a refresh mechanism) - Re-create AudioFilterPipeline with updated filter instances - Reset stateful filter state on pipeline refresh - [x] Task 5: Update WebSocket test endpoint in `api/routes/audio_sources.py` - For ProcessedAudioSource: resolve chain, create pipeline, apply filters to test stream data - Return filtered analysis in real-time over WebSocket - [x] Task 6: Update any code that calls `AudioSourceStore.resolve_audio_source()` to handle the new return shape ## Files to Modify/Create - `core/audio/filters/pipeline.py` — **create** — AudioFilterPipeline - `core/processing/audio_stream.py` — **modify** — integrate filter pipeline - `core/processing/value_stream.py` — **modify** — integrate filter pipeline - `api/routes/audio_sources.py` — **modify** — update WebSocket test - Any other consumers of `resolve_audio_source()` — **modify** ## Acceptance Criteria - ProcessedAudioSource chains are resolved and filter pipelines created at stream start - AudioAnalysis passes through the filter chain before visualization/value extraction - Stateful filters maintain correct state across frames - Hot-update of templates refreshes running filter pipelines - WebSocket test endpoint works with processed sources - Old inline channel/band code removed from stream classes ## Notes - ⚠️ Temporary breakage: Removing inline channel/band code from AudioColorStripStream breaks existing MonoAudioSource/BandExtractAudioSource flows — but those types were already removed in Phase 3. - Filter pipeline must be thread-safe: AudioColorStripStream and AudioValueStream run in background threads. - For hot-update: consider using an event callback from the template store rather than polling. - The filter pipeline should produce a new AudioAnalysis each time (immutability), not mutate the shared snapshot from ManagedAudioStream. ## Review Checklist - [x] All tasks completed - [x] Code follows project conventions - [x] No unintended side effects - [ ] Build passes - [ ] Tests pass (new + existing) ## Handoff to Next Phase ### What was built - `AudioFilterPipeline` class in `core/audio/filters/pipeline.py` — thread-safe pipeline that instantiates, processes, resets, and closes audio filters - `build_pipeline_from_template_ids()` helper — resolves template IDs to FilterInstance lists and creates a pipeline - `AudioColorStripStream` now builds and applies filter pipeline: `_rebuild_filter_pipeline()` called on construction and source update; `_apply_filters()` replaces `_pick_channel()` in render loop; pipeline closed on stop - `AudioValueStream` now builds and applies filter pipeline: `_rebuild_filter_pipeline()` called on construction; filters applied in `_extract_raw()` before scalar extraction; pipeline closed on stop - Hot-update: `ProcessorManager.refresh_audio_filter_pipelines(template_id)` dispatches to both CSS and value stream managers; called from audio processing template update/delete routes - WebSocket test endpoint creates a filter pipeline from the resolved template chain and applies it to analysis before sending - Dependency injection: `audio_processing_template_store` threaded through `ProcessorDependencies` -> `ProcessorManager` -> `ColorStripStreamManager` / `ValueStreamManager` -> `AudioColorStripStream` / `AudioValueStream` ### What Phase 5 needs to know - The backend is fully wired: creating a ProcessedAudioSource that references templates with channel_extract, band_extract, gain, etc. filters will apply those filters to live audio data - The WebSocket test endpoint shows filtered analysis in real-time - Frontend needs to provide UI for creating/editing AudioProcessingTemplates and ProcessedAudioSources - Filter pipeline is per-stream-instance (not shared) — stateful filters maintain independent state ### Temporary breakages resolved - `_pick_channel()` removed from AudioColorStripStream — replaced by `_apply_filters()` which uses the filter pipeline - Channel/band handling in AudioValueStream now goes through filter pipeline - All "Phase 4 will wire..." comments resolved ### Known deviations from plan - `AudioFilterPipeline.__init__` takes `List[FilterInstance]` only (not `AudioFilterRegistry` — uses the class-level registry directly via `AudioFilterRegistry.create_instance()`) - Hot-update uses explicit method calls from the route handler rather than an event-listener pattern — simpler and avoids the need for a subscription system