feat(processed-audio-sources): phase 4 - runtime filter integration
Add AudioFilterPipeline for chained filter execution on AudioAnalysis. Wire filter pipelines into AudioColorStripStream, AudioValueStream, and WebSocket test endpoint. Add hot-update support via ProcessorManager.refresh_audio_filter_pipelines(). Thread AudioProcessingTemplateStore through dependency injection hierarchy.
This commit is contained in:
@@ -9,7 +9,7 @@
|
||||
- **Test:** `cd server && py -3.13 -m pytest tests/ --no-cov -q`
|
||||
|
||||
## Current State
|
||||
Phase 1 (Audio Filter Framework), Phase 2 (Audio Filters), and Phase 3 (Processed Audio Source Model) implemented.
|
||||
Phases 1-4 implemented. Phase 4 (Runtime Integration) wired the audio filter pipeline into the stream runtime.
|
||||
|
||||
Phase 1 framework:
|
||||
- `AudioFilter` base class, `AudioFilterRegistry`, `AudioFilterOptionDef` in `core/audio/filters/`
|
||||
@@ -93,7 +93,7 @@ _(none yet)_
|
||||
| Phase 1 | impl-agent | — | No | Tasks 7+8 skipped (SQLite migration made them obsolete) |
|
||||
| Phase 2 | impl-agent | — | No | All 11 filters implemented, no deviations |
|
||||
| Phase 3 | impl-agent | — | No | All 11 tasks done; channel/band logic deferred to Phase 4 |
|
||||
| Phase 4 | — | — | — | — |
|
||||
| Phase 4 | impl-agent | — | No | All 6 tasks done; dependency injection threaded through |
|
||||
| Phase 5 | — | — | — | — |
|
||||
| Phase 6 | — | — | — | — |
|
||||
| Phase 7 | — | — | — | — |
|
||||
|
||||
@@ -42,7 +42,7 @@ Clean-slate approach: no data migration for old source types.
|
||||
| Phase 1: Audio Filter Framework | backend | 🔨 In Progress | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 2: Audio Filters | backend | 🔨 In Progress | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 3: Processed Audio Source Model | backend | ✅ Done | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 4: Runtime Integration | backend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 4: Runtime Integration | backend | ✅ Done | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 5: Frontend — Audio Processing Templates | frontend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 6: Frontend — Source Types | frontend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 7: Testing & Polish | backend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Phase 4: Runtime Integration
|
||||
|
||||
**Status:** ⬜ Not Started
|
||||
**Status:** ✅ Done
|
||||
**Parent plan:** [PLAN.md](./PLAN.md)
|
||||
**Domain:** backend
|
||||
|
||||
@@ -9,7 +9,7 @@ Wire the audio filter pipeline into the runtime audio streaming system so that P
|
||||
|
||||
## Tasks
|
||||
|
||||
- [ ] Task 1: Create filter pipeline executor in `core/audio/filters/pipeline.py`
|
||||
- [x] Task 1: Create filter pipeline executor in `core/audio/filters/pipeline.py`
|
||||
- `AudioFilterPipeline` class:
|
||||
- `__init__(filter_instances: List[FilterInstance], registry: AudioFilterRegistry)`
|
||||
- Instantiates all filters from FilterInstance specs
|
||||
@@ -17,7 +17,7 @@ Wire the audio filter pipeline into the runtime audio streaming system so that P
|
||||
- `reset()` — resets all stateful filters
|
||||
- `close()` — cleanup resources
|
||||
- Handles stateful filter lifecycle (create on init, reset on demand, close on cleanup)
|
||||
- [ ] Task 2: Update `AudioColorStripStream` in `core/processing/audio_stream.py`
|
||||
- [x] Task 2: Update `AudioColorStripStream` in `core/processing/audio_stream.py`
|
||||
- On construction: if source is ProcessedAudioSource, resolve the full chain:
|
||||
- Walk to CaptureAudioSource for device info
|
||||
- Collect all AudioProcessingTemplates along the chain
|
||||
@@ -26,18 +26,18 @@ Wire the audio filter pipeline into the runtime audio streaming system so that P
|
||||
- In render loop: after getting AudioAnalysis from ManagedAudioStream, run it through the filter pipeline before visualization
|
||||
- Remove old inline channel selection and band filtering code (now handled by filters)
|
||||
- On stop: close the filter pipeline
|
||||
- [ ] Task 3: Update `AudioValueStream` in `core/processing/value_stream.py`
|
||||
- [x] Task 3: Update `AudioValueStream` in `core/processing/value_stream.py`
|
||||
- Same pattern: resolve ProcessedAudioSource chain, create filter pipeline, apply in get_value()
|
||||
- Remove old inline channel/band handling
|
||||
- [ ] Task 4: Hot-update support for filter templates
|
||||
- [x] Task 4: Hot-update support for filter templates
|
||||
- When an AudioProcessingTemplate is updated, running streams that use it should re-resolve their filter pipeline
|
||||
- Listen for template update events (or implement a refresh mechanism)
|
||||
- Re-create AudioFilterPipeline with updated filter instances
|
||||
- Reset stateful filter state on pipeline refresh
|
||||
- [ ] Task 5: Update WebSocket test endpoint in `api/routes/audio_sources.py`
|
||||
- [x] Task 5: Update WebSocket test endpoint in `api/routes/audio_sources.py`
|
||||
- For ProcessedAudioSource: resolve chain, create pipeline, apply filters to test stream data
|
||||
- Return filtered analysis in real-time over WebSocket
|
||||
- [ ] Task 6: Update any code that calls `AudioSourceStore.resolve_audio_source()` to handle the new return shape
|
||||
- [x] Task 6: Update any code that calls `AudioSourceStore.resolve_audio_source()` to handle the new return shape
|
||||
|
||||
## Files to Modify/Create
|
||||
- `core/audio/filters/pipeline.py` — **create** — AudioFilterPipeline
|
||||
@@ -61,11 +61,34 @@ Wire the audio filter pipeline into the runtime audio streaming system so that P
|
||||
- The filter pipeline should produce a new AudioAnalysis each time (immutability), not mutate the shared snapshot from ManagedAudioStream.
|
||||
|
||||
## Review Checklist
|
||||
- [ ] All tasks completed
|
||||
- [ ] Code follows project conventions
|
||||
- [ ] No unintended side effects
|
||||
- [x] All tasks completed
|
||||
- [x] Code follows project conventions
|
||||
- [x] No unintended side effects
|
||||
- [ ] Build passes
|
||||
- [ ] Tests pass (new + existing)
|
||||
|
||||
## Handoff to Next Phase
|
||||
<!-- Filled in by the implementation agent after completing this phase. -->
|
||||
|
||||
### What was built
|
||||
- `AudioFilterPipeline` class in `core/audio/filters/pipeline.py` — thread-safe pipeline that instantiates, processes, resets, and closes audio filters
|
||||
- `build_pipeline_from_template_ids()` helper — resolves template IDs to FilterInstance lists and creates a pipeline
|
||||
- `AudioColorStripStream` now builds and applies filter pipeline: `_rebuild_filter_pipeline()` called on construction and source update; `_apply_filters()` replaces `_pick_channel()` in render loop; pipeline closed on stop
|
||||
- `AudioValueStream` now builds and applies filter pipeline: `_rebuild_filter_pipeline()` called on construction; filters applied in `_extract_raw()` before scalar extraction; pipeline closed on stop
|
||||
- Hot-update: `ProcessorManager.refresh_audio_filter_pipelines(template_id)` dispatches to both CSS and value stream managers; called from audio processing template update/delete routes
|
||||
- WebSocket test endpoint creates a filter pipeline from the resolved template chain and applies it to analysis before sending
|
||||
- Dependency injection: `audio_processing_template_store` threaded through `ProcessorDependencies` -> `ProcessorManager` -> `ColorStripStreamManager` / `ValueStreamManager` -> `AudioColorStripStream` / `AudioValueStream`
|
||||
|
||||
### What Phase 5 needs to know
|
||||
- The backend is fully wired: creating a ProcessedAudioSource that references templates with channel_extract, band_extract, gain, etc. filters will apply those filters to live audio data
|
||||
- The WebSocket test endpoint shows filtered analysis in real-time
|
||||
- Frontend needs to provide UI for creating/editing AudioProcessingTemplates and ProcessedAudioSources
|
||||
- Filter pipeline is per-stream-instance (not shared) — stateful filters maintain independent state
|
||||
|
||||
### Temporary breakages resolved
|
||||
- `_pick_channel()` removed from AudioColorStripStream — replaced by `_apply_filters()` which uses the filter pipeline
|
||||
- Channel/band handling in AudioValueStream now goes through filter pipeline
|
||||
- All "Phase 4 will wire..." comments resolved
|
||||
|
||||
### Known deviations from plan
|
||||
- `AudioFilterPipeline.__init__` takes `List[FilterInstance]` only (not `AudioFilterRegistry` — uses the class-level registry directly via `AudioFilterRegistry.create_instance()`)
|
||||
- Hot-update uses explicit method calls from the route handler rather than an event-listener pattern — simpler and avoids the need for a subscription system
|
||||
|
||||
Reference in New Issue
Block a user