ledgrab/plans/processed-audio-sources/CONTEXT.md

Feature Context: Processed Audio Sources

Configuration

• Development mode: Automated
• Execution mode: Orchestrator
• Strategy: Big Bang
• Build (Python): cd server && ruff check src/ tests/ --fix
• Build (TypeScript): cd server && npx tsc --noEmit && npm run build
• Test: cd server && py -3.13 -m pytest tests/ --no-cov -q

Current State

Phases 1-6 implemented. Phase 6 (Frontend Source Types) cleaned up the modal HTML and i18n keys for the new capture/processed source types.

Phase 1 framework:

• AudioFilter base class, AudioFilterRegistry, AudioFilterOptionDef in core/audio/filters/
• AudioProcessingTemplate dataclass + AudioProcessingTemplateStore (SQLite-backed) in storage/
• audio_filter_template meta-filter with recursive resolution
• Full REST API: CRUD templates + filter registry discovery
• Dependency injection wired in dependencies.py and main.py

Phase 2 filters (12 total registered, 11 real + 1 meta):

• Stateless: channel_extract, band_extract, gain, inverter
• Stateful: peak_hold, noise_gate, envelope_follower, spectral_smoothing, compressor, beat_gate, delay
• All produce new AudioAnalysis via dataclasses.replace() (immutability preserved)

Key Architecture Reference

Existing Pattern to Mirror: Processed Picture Sources

• ProcessedPictureSource references source_stream_id + postprocessing_template_id
• PostprocessingTemplate contains ordered List[FilterInstance]
• FilterInstance = filter_id (string) + options (dict)
• FilterRegistry handles registration, lookup, instantiation
• filter_template meta-filter embeds one template inside another
• PostprocessingTemplateStore has resolve_filter_instances() for recursive expansion
• Picture filters transform images; audio filters will transform AudioAnalysis

Current Audio Source Types (Phase 3 complete)

• CaptureAudioSource (source_type="capture") — wraps a physical audio device
• ProcessedAudioSource (source_type="processed") — references audio_source_id + audio_processing_template_id
• MonoAudioSource — removed, replaced by channel_extract filter
• BandExtractAudioSource — removed, replaced by band_extract filter

AudioAnalysis Structure (filter input/output)

AudioAnalysis:
  timestamp: float
  rms: float                    # Overall RMS level
  peak: float                   # Peak amplitude
  spectrum: np.ndarray[64]      # Log-spaced FFT bands
  beat: bool                    # Beat detected
  beat_intensity: float         # 0-1 beat strength
  left_rms: float               # Left channel RMS
  left_spectrum: np.ndarray     # Left channel spectrum
  right_rms: float              # Right channel RMS
  right_spectrum: np.ndarray    # Right channel spectrum

Key Existing Files

• storage/audio_source.py — current source dataclasses
• storage/audio_source_store.py — CRUD + resolve_audio_source()
• core/audio/analysis.py — AudioAnalyzer, AudioAnalysis
• core/audio/band_filter.py — existing band filtering logic
• core/processing/audio_stream.py — AudioColorStripStream
• core/processing/value_stream.py — AudioValueStream
• core/filters/base.py — PostprocessingFilter (picture filter base class)
• core/filters/registry.py — FilterRegistry (picture filters)
• storage/postprocessing_template.py — PostprocessingTemplate dataclass
• storage/postprocessing_template_store.py — template store with resolve_filter_instances()

Temporary Workarounds

(none yet)

Cross-Phase Dependencies

• Phase 2 depends on Phase 1 (filter framework)
• Phase 3 depends on Phase 1 (template store for ProcessedAudioSource)
• Phase 4 depends on Phases 1-3 (all backend pieces)
• Phase 5 depends on Phase 1 (template API)
• Phase 6 depends on Phase 3 (source type API)
• Phase 7 depends on all prior phases

Deferred Work

(none yet)

Failed Approaches

(none yet)

Review Findings Log

(none yet)

Phase Execution Log

Phase	Agent Used	Test Writer	Parallel	Notes
Phase 1	impl-agent	—	No	Tasks 7+8 skipped (SQLite migration made them obsolete)
Phase 2	impl-agent	—	No	All 11 filters implemented, no deviations
Phase 3	impl-agent	—	No	All 11 tasks done; channel/band logic deferred to Phase 4
Phase 4	impl-agent	—	No	All 6 tasks done; dependency injection threaded through
Phase 5	impl-agent	—	No	6/7 tasks done; Task 4 (preview) deferred to Phase 7
Phase 6	impl-agent	—	No	Modal HTML + i18n cleanup; most tasks already done in Phases 3/5
Phase 7	—	—	—	—

Environment & Runtime Notes

• Platform: Windows 10
• Python: 3.13
• Server port: 8080
• Shell: bash (Git Bash on Windows)

Implementation Notes

• Clean-slate approach: no migration of existing MonoAudioSource/BandExtractAudioSource data
• 7 of 11 filters are stateful (peak hold, noise gate, envelope follower, spectral smoothing, compressor, beat gate, delay) — need per-stream instance lifecycle
• Audio filters operate on AudioAnalysis snapshots, not raw audio samples
• Big Bang strategy: intermediate phases may break the build; only Phase 7 enforces build/tests