# Feature Context: Processed Audio Sources ## Configuration - **Development mode:** Automated - **Execution mode:** Orchestrator - **Strategy:** Big Bang - **Build (Python):** `cd server && ruff check src/ tests/ --fix` - **Build (TypeScript):** `cd server && npx tsc --noEmit && npm run build` - **Test:** `cd server && py -3.13 -m pytest tests/ --no-cov -q` ## Current State Phases 1-4 implemented. Phase 4 (Runtime Integration) wired the audio filter pipeline into the stream runtime. Phase 1 framework: - `AudioFilter` base class, `AudioFilterRegistry`, `AudioFilterOptionDef` in `core/audio/filters/` - `AudioProcessingTemplate` dataclass + `AudioProcessingTemplateStore` (SQLite-backed) in `storage/` - `audio_filter_template` meta-filter with recursive resolution - Full REST API: CRUD templates + filter registry discovery - Dependency injection wired in `dependencies.py` and `main.py` Phase 2 filters (12 total registered, 11 real + 1 meta): - Stateless: `channel_extract`, `band_extract`, `gain`, `inverter` - Stateful: `peak_hold`, `noise_gate`, `envelope_follower`, `spectral_smoothing`, `compressor`, `beat_gate`, `delay` - All produce new `AudioAnalysis` via `dataclasses.replace()` (immutability preserved) ## Key Architecture Reference ### Existing Pattern to Mirror: Processed Picture Sources - `ProcessedPictureSource` references `source_stream_id` + `postprocessing_template_id` - `PostprocessingTemplate` contains ordered `List[FilterInstance]` - `FilterInstance` = `filter_id` (string) + `options` (dict) - `FilterRegistry` handles registration, lookup, instantiation - `filter_template` meta-filter embeds one template inside another - `PostprocessingTemplateStore` has `resolve_filter_instances()` for recursive expansion - Picture filters transform images; audio filters will transform `AudioAnalysis` ### Current Audio Source Types (Phase 3 complete) - `CaptureAudioSource` (source_type="capture") — wraps a physical audio device - `ProcessedAudioSource` (source_type="processed") — references audio_source_id + audio_processing_template_id - `MonoAudioSource` — removed, replaced by channel_extract filter - `BandExtractAudioSource` — removed, replaced by band_extract filter ### AudioAnalysis Structure (filter input/output) ```python AudioAnalysis: timestamp: float rms: float # Overall RMS level peak: float # Peak amplitude spectrum: np.ndarray[64] # Log-spaced FFT bands beat: bool # Beat detected beat_intensity: float # 0-1 beat strength left_rms: float # Left channel RMS left_spectrum: np.ndarray # Left channel spectrum right_rms: float # Right channel RMS right_spectrum: np.ndarray # Right channel spectrum ``` ### Key Existing Files - `storage/audio_source.py` — current source dataclasses - `storage/audio_source_store.py` — CRUD + resolve_audio_source() - `core/audio/analysis.py` — AudioAnalyzer, AudioAnalysis - `core/audio/band_filter.py` — existing band filtering logic - `core/processing/audio_stream.py` — AudioColorStripStream - `core/processing/value_stream.py` — AudioValueStream - `core/filters/base.py` — PostprocessingFilter (picture filter base class) - `core/filters/registry.py` — FilterRegistry (picture filters) - `storage/postprocessing_template.py` — PostprocessingTemplate dataclass - `storage/postprocessing_template_store.py` — template store with resolve_filter_instances() ## Temporary Workarounds _(none yet)_ ## Cross-Phase Dependencies - Phase 2 depends on Phase 1 (filter framework) - Phase 3 depends on Phase 1 (template store for ProcessedAudioSource) - Phase 4 depends on Phases 1-3 (all backend pieces) - Phase 5 depends on Phase 1 (template API) - Phase 6 depends on Phase 3 (source type API) - Phase 7 depends on all prior phases ## Deferred Work _(none yet)_ ## Failed Approaches _(none yet)_ ## Review Findings Log _(none yet)_ ## Phase Execution Log | Phase | Agent Used | Test Writer | Parallel | Notes | |-------|-----------|-------------|----------|-------| | Phase 1 | impl-agent | — | No | Tasks 7+8 skipped (SQLite migration made them obsolete) | | Phase 2 | impl-agent | — | No | All 11 filters implemented, no deviations | | Phase 3 | impl-agent | — | No | All 11 tasks done; channel/band logic deferred to Phase 4 | | Phase 4 | impl-agent | — | No | All 6 tasks done; dependency injection threaded through | | Phase 5 | — | — | — | — | | Phase 6 | — | — | — | — | | Phase 7 | — | — | — | — | ## Environment & Runtime Notes - Platform: Windows 10 - Python: 3.13 - Server port: 8080 - Shell: bash (Git Bash on Windows) ## Implementation Notes - Clean-slate approach: no migration of existing MonoAudioSource/BandExtractAudioSource data - 7 of 11 filters are stateful (peak hold, noise gate, envelope follower, spectral smoothing, compressor, beat gate, delay) — need per-stream instance lifecycle - Audio filters operate on AudioAnalysis snapshots, not raw audio samples - Big Bang strategy: intermediate phases may break the build; only Phase 7 enforces build/tests