alexei.dolgolyov
86a9d344e6
Add the foundation for audio processing filters, mirroring the existing picture filter/postprocessing template system: - AudioFilter base class, AudioFilterRegistry, AudioFilterOptionDef - AudioProcessingTemplate dataclass + SQLite-backed store - audio_filter_template meta-filter with recursive resolution - Full REST API: CRUD templates + filter registry discovery - Dependency injection wired in dependencies.py and main.py
72 lines
4.0 KiB
Markdown
72 lines
4.0 KiB
Markdown
# Phase 4: Runtime Integration
|
|
|
|
**Status:** ⬜ Not Started
|
|
**Parent plan:** [PLAN.md](./PLAN.md)
|
|
**Domain:** backend
|
|
|
|
## Objective
|
|
Wire the audio filter pipeline into the runtime audio streaming system so that ProcessedAudioSources actually apply their filter chains to live audio data.
|
|
|
|
## Tasks
|
|
|
|
- [ ] Task 1: Create filter pipeline executor in `core/audio/filters/pipeline.py`
|
|
- `AudioFilterPipeline` class:
|
|
- `__init__(filter_instances: List[FilterInstance], registry: AudioFilterRegistry)`
|
|
- Instantiates all filters from FilterInstance specs
|
|
- `process(analysis: AudioAnalysis) -> AudioAnalysis` — runs analysis through all filters in order
|
|
- `reset()` — resets all stateful filters
|
|
- `close()` — cleanup resources
|
|
- Handles stateful filter lifecycle (create on init, reset on demand, close on cleanup)
|
|
- [ ] Task 2: Update `AudioColorStripStream` in `core/processing/audio_stream.py`
|
|
- On construction: if source is ProcessedAudioSource, resolve the full chain:
|
|
- Walk to CaptureAudioSource for device info
|
|
- Collect all AudioProcessingTemplates along the chain
|
|
- Resolve all filter instances (with template expansion)
|
|
- Create AudioFilterPipeline
|
|
- In render loop: after getting AudioAnalysis from ManagedAudioStream, run it through the filter pipeline before visualization
|
|
- Remove old inline channel selection and band filtering code (now handled by filters)
|
|
- On stop: close the filter pipeline
|
|
- [ ] Task 3: Update `AudioValueStream` in `core/processing/value_stream.py`
|
|
- Same pattern: resolve ProcessedAudioSource chain, create filter pipeline, apply in get_value()
|
|
- Remove old inline channel/band handling
|
|
- [ ] Task 4: Hot-update support for filter templates
|
|
- When an AudioProcessingTemplate is updated, running streams that use it should re-resolve their filter pipeline
|
|
- Listen for template update events (or implement a refresh mechanism)
|
|
- Re-create AudioFilterPipeline with updated filter instances
|
|
- Reset stateful filter state on pipeline refresh
|
|
- [ ] Task 5: Update WebSocket test endpoint in `api/routes/audio_sources.py`
|
|
- For ProcessedAudioSource: resolve chain, create pipeline, apply filters to test stream data
|
|
- Return filtered analysis in real-time over WebSocket
|
|
- [ ] Task 6: Update any code that calls `AudioSourceStore.resolve_audio_source()` to handle the new return shape
|
|
|
|
## Files to Modify/Create
|
|
- `core/audio/filters/pipeline.py` — **create** — AudioFilterPipeline
|
|
- `core/processing/audio_stream.py` — **modify** — integrate filter pipeline
|
|
- `core/processing/value_stream.py` — **modify** — integrate filter pipeline
|
|
- `api/routes/audio_sources.py` — **modify** — update WebSocket test
|
|
- Any other consumers of `resolve_audio_source()` — **modify**
|
|
|
|
## Acceptance Criteria
|
|
- ProcessedAudioSource chains are resolved and filter pipelines created at stream start
|
|
- AudioAnalysis passes through the filter chain before visualization/value extraction
|
|
- Stateful filters maintain correct state across frames
|
|
- Hot-update of templates refreshes running filter pipelines
|
|
- WebSocket test endpoint works with processed sources
|
|
- Old inline channel/band code removed from stream classes
|
|
|
|
## Notes
|
|
- ⚠️ Temporary breakage: Removing inline channel/band code from AudioColorStripStream breaks existing MonoAudioSource/BandExtractAudioSource flows — but those types were already removed in Phase 3.
|
|
- Filter pipeline must be thread-safe: AudioColorStripStream and AudioValueStream run in background threads.
|
|
- For hot-update: consider using an event callback from the template store rather than polling.
|
|
- The filter pipeline should produce a new AudioAnalysis each time (immutability), not mutate the shared snapshot from ManagedAudioStream.
|
|
|
|
## Review Checklist
|
|
- [ ] All tasks completed
|
|
- [ ] Code follows project conventions
|
|
- [ ] No unintended side effects
|
|
- [ ] Build passes
|
|
- [ ] Tests pass (new + existing)
|
|
|
|
## Handoff to Next Phase
|
|
<!-- Filled in by the implementation agent after completing this phase. -->
|