ledgrab/plans/processed-audio-sources/CONTEXT.md

Feature Context: Processed Audio Sources

Configuration

• Development mode: Automated
• Execution mode: Orchestrator
• Strategy: Big Bang
• Build (Python): cd server && ruff check src/ tests/ --fix
• Build (TypeScript): cd server && npx tsc --noEmit && npm run build
• Test: cd server && py -3.13 -m pytest tests/ --no-cov -q

Current State

Phase 1 (Audio Filter Framework) implemented. Core framework is in place:

• AudioFilter base class, AudioFilterRegistry, AudioFilterOptionDef in core/audio/filters/
• AudioProcessingTemplate dataclass + AudioProcessingTemplateStore (SQLite-backed) in storage/
• audio_filter_template meta-filter with recursive resolution
• Full REST API: CRUD templates + filter registry discovery
• Dependency injection wired in dependencies.py and main.py

Key Architecture Reference

Existing Pattern to Mirror: Processed Picture Sources

• ProcessedPictureSource references source_stream_id + postprocessing_template_id
• PostprocessingTemplate contains ordered List[FilterInstance]
• FilterInstance = filter_id (string) + options (dict)
• FilterRegistry handles registration, lookup, instantiation
• filter_template meta-filter embeds one template inside another
• PostprocessingTemplateStore has resolve_filter_instances() for recursive expansion
• Picture filters transform images; audio filters will transform AudioAnalysis

Current Audio Source Types (to be replaced)

• MultichannelAudioSource → renamed to CaptureAudioSource
• MonoAudioSource → removed, replaced by channel_extract filter
• BandExtractAudioSource → removed, replaced by band_extract filter

AudioAnalysis Structure (filter input/output)

AudioAnalysis:
  timestamp: float
  rms: float                    # Overall RMS level
  peak: float                   # Peak amplitude
  spectrum: np.ndarray[64]      # Log-spaced FFT bands
  beat: bool                    # Beat detected
  beat_intensity: float         # 0-1 beat strength
  left_rms: float               # Left channel RMS
  left_spectrum: np.ndarray     # Left channel spectrum
  right_rms: float              # Right channel RMS
  right_spectrum: np.ndarray    # Right channel spectrum

Key Existing Files

• storage/audio_source.py — current source dataclasses
• storage/audio_source_store.py — CRUD + resolve_audio_source()
• core/audio/analysis.py — AudioAnalyzer, AudioAnalysis
• core/audio/band_filter.py — existing band filtering logic
• core/processing/audio_stream.py — AudioColorStripStream
• core/processing/value_stream.py — AudioValueStream
• core/filters/base.py — PostprocessingFilter (picture filter base class)
• core/filters/registry.py — FilterRegistry (picture filters)
• storage/postprocessing_template.py — PostprocessingTemplate dataclass
• storage/postprocessing_template_store.py — template store with resolve_filter_instances()

Temporary Workarounds

(none yet)

Cross-Phase Dependencies

• Phase 2 depends on Phase 1 (filter framework)
• Phase 3 depends on Phase 1 (template store for ProcessedAudioSource)
• Phase 4 depends on Phases 1-3 (all backend pieces)
• Phase 5 depends on Phase 1 (template API)
• Phase 6 depends on Phase 3 (source type API)
• Phase 7 depends on all prior phases

Deferred Work

(none yet)

Failed Approaches

(none yet)

Review Findings Log

(none yet)

Phase Execution Log

Phase	Agent Used	Test Writer	Parallel	Notes
Phase 1	impl-agent	—	No	Tasks 7+8 skipped (SQLite migration made them obsolete)
Phase 2	—	—	—	—
Phase 3	—	—	—	—
Phase 4	—	—	—	—
Phase 5	—	—	—	—
Phase 6	—	—	—	—
Phase 7	—	—	—	—

Environment & Runtime Notes

• Platform: Windows 10
• Python: 3.13
• Server port: 8080
• Shell: bash (Git Bash on Windows)

Implementation Notes

• Clean-slate approach: no migration of existing MonoAudioSource/BandExtractAudioSource data
• 5 of 11 filters are stateful (peak hold, envelope follower, spectral smoothing, compressor, delay) — need per-stream instance lifecycle
• Audio filters operate on AudioAnalysis snapshots, not raw audio samples
• Big Bang strategy: intermediate phases may break the build; only Phase 7 enforces build/tests