ledgrab/plans/processed-audio-sources/phase-4-runtime-integration.md

Phase 4: Runtime Integration

Status: ✅ Done Parent plan: PLAN.md Domain: backend

Objective

Wire the audio filter pipeline into the runtime audio streaming system so that ProcessedAudioSources actually apply their filter chains to live audio data.

Tasks

• Task 1: Create filter pipeline executor in core/audio/filters/pipeline.py
  • AudioFilterPipeline class:
    • __init__(filter_instances: List[FilterInstance], registry: AudioFilterRegistry)
    • Instantiates all filters from FilterInstance specs
    • process(analysis: AudioAnalysis) -> AudioAnalysis — runs analysis through all filters in order
    • reset() — resets all stateful filters
    • close() — cleanup resources
  • Handles stateful filter lifecycle (create on init, reset on demand, close on cleanup)
• Task 2: Update AudioColorStripStream in core/processing/audio_stream.py
  • On construction: if source is ProcessedAudioSource, resolve the full chain:
    • Walk to CaptureAudioSource for device info
    • Collect all AudioProcessingTemplates along the chain
    • Resolve all filter instances (with template expansion)
    • Create AudioFilterPipeline
  • In render loop: after getting AudioAnalysis from ManagedAudioStream, run it through the filter pipeline before visualization
  • Remove old inline channel selection and band filtering code (now handled by filters)
  • On stop: close the filter pipeline
• Task 3: Update AudioValueStream in core/processing/value_stream.py
  • Same pattern: resolve ProcessedAudioSource chain, create filter pipeline, apply in get_value()
  • Remove old inline channel/band handling
• Task 4: Hot-update support for filter templates
  • When an AudioProcessingTemplate is updated, running streams that use it should re-resolve their filter pipeline
  • Listen for template update events (or implement a refresh mechanism)
  • Re-create AudioFilterPipeline with updated filter instances
  • Reset stateful filter state on pipeline refresh
• Task 5: Update WebSocket test endpoint in api/routes/audio_sources.py
  • For ProcessedAudioSource: resolve chain, create pipeline, apply filters to test stream data
  • Return filtered analysis in real-time over WebSocket
• Task 6: Update any code that calls AudioSourceStore.resolve_audio_source() to handle the new return shape

Files to Modify/Create

• core/audio/filters/pipeline.py — create — AudioFilterPipeline
• core/processing/audio_stream.py — modify — integrate filter pipeline
• core/processing/value_stream.py — modify — integrate filter pipeline
• api/routes/audio_sources.py — modify — update WebSocket test
• Any other consumers of resolve_audio_source() — modify

Acceptance Criteria

• ProcessedAudioSource chains are resolved and filter pipelines created at stream start
• AudioAnalysis passes through the filter chain before visualization/value extraction
• Stateful filters maintain correct state across frames
• Hot-update of templates refreshes running filter pipelines
• WebSocket test endpoint works with processed sources
• Old inline channel/band code removed from stream classes

Notes

• ⚠️ Temporary breakage: Removing inline channel/band code from AudioColorStripStream breaks existing MonoAudioSource/BandExtractAudioSource flows — but those types were already removed in Phase 3.
• Filter pipeline must be thread-safe: AudioColorStripStream and AudioValueStream run in background threads.
• For hot-update: consider using an event callback from the template store rather than polling.
• The filter pipeline should produce a new AudioAnalysis each time (immutability), not mutate the shared snapshot from ManagedAudioStream.

Review Checklist

• All tasks completed
• Code follows project conventions
• No unintended side effects
• Build passes
• Tests pass (new + existing)

Handoff to Next Phase

What was built

• AudioFilterPipeline class in core/audio/filters/pipeline.py — thread-safe pipeline that instantiates, processes, resets, and closes audio filters
• build_pipeline_from_template_ids() helper — resolves template IDs to FilterInstance lists and creates a pipeline
• AudioColorStripStream now builds and applies filter pipeline: _rebuild_filter_pipeline() called on construction and source update; _apply_filters() replaces _pick_channel() in render loop; pipeline closed on stop
• AudioValueStream now builds and applies filter pipeline: _rebuild_filter_pipeline() called on construction; filters applied in _extract_raw() before scalar extraction; pipeline closed on stop
• Hot-update: ProcessorManager.refresh_audio_filter_pipelines(template_id) dispatches to both CSS and value stream managers; called from audio processing template update/delete routes
• WebSocket test endpoint creates a filter pipeline from the resolved template chain and applies it to analysis before sending
• Dependency injection: audio_processing_template_store threaded through ProcessorDependencies -> ProcessorManager -> ColorStripStreamManager / ValueStreamManager -> AudioColorStripStream / AudioValueStream

What Phase 5 needs to know

• The backend is fully wired: creating a ProcessedAudioSource that references templates with channel_extract, band_extract, gain, etc. filters will apply those filters to live audio data
• The WebSocket test endpoint shows filtered analysis in real-time
• Frontend needs to provide UI for creating/editing AudioProcessingTemplates and ProcessedAudioSources
• Filter pipeline is per-stream-instance (not shared) — stateful filters maintain independent state

Temporary breakages resolved

• _pick_channel() removed from AudioColorStripStream — replaced by _apply_filters() which uses the filter pipeline
• Channel/band handling in AudioValueStream now goes through filter pipeline
• All "Phase 4 will wire..." comments resolved

Known deviations from plan

• AudioFilterPipeline.__init__ takes List[FilterInstance] only (not AudioFilterRegistry — uses the class-level registry directly via AudioFilterRegistry.create_instance())
• Hot-update uses explicit method calls from the route handler rather than an event-listener pattern — simpler and avoids the need for a subscription system