feat(processed-audio-sources): phase 3 - processed audio source model

Replace MultichannelAudioSource with CaptureAudioSource, add
ProcessedAudioSource (audio_source_id + audio_processing_template_id),
remove MonoAudioSource and BandExtractAudioSource entirely.
Update store resolution to walk processed chains collecting template IDs.
Update all API schemas, routes, and frontend references.
This commit is contained in:
2026-03-31 19:01:46 +03:00
parent eb94066386
commit 353c090b42
23 changed files with 455 additions and 668 deletions
+7 -6
View File
@@ -9,7 +9,7 @@
- **Test:** `cd server && py -3.13 -m pytest tests/ --no-cov -q`
## Current State
Phase 1 (Audio Filter Framework) and Phase 2 (Audio Filters) implemented.
Phase 1 (Audio Filter Framework), Phase 2 (Audio Filters), and Phase 3 (Processed Audio Source Model) implemented.
Phase 1 framework:
- `AudioFilter` base class, `AudioFilterRegistry`, `AudioFilterOptionDef` in `core/audio/filters/`
@@ -34,10 +34,11 @@ Phase 2 filters (12 total registered, 11 real + 1 meta):
- `PostprocessingTemplateStore` has `resolve_filter_instances()` for recursive expansion
- Picture filters transform images; audio filters will transform `AudioAnalysis`
### Current Audio Source Types (to be replaced)
- `MultichannelAudioSource` → renamed to `CaptureAudioSource`
- `MonoAudioSource` → removed, replaced by channel_extract filter
- `BandExtractAudioSource` removed, replaced by band_extract filter
### Current Audio Source Types (Phase 3 complete)
- `CaptureAudioSource` (source_type="capture") — wraps a physical audio device
- `ProcessedAudioSource` (source_type="processed") — references audio_source_id + audio_processing_template_id
- `MonoAudioSource` removed, replaced by channel_extract filter
- `BandExtractAudioSource` — removed, replaced by band_extract filter
### AudioAnalysis Structure (filter input/output)
```python
@@ -91,7 +92,7 @@ _(none yet)_
|-------|-----------|-------------|----------|-------|
| Phase 1 | impl-agent | — | No | Tasks 7+8 skipped (SQLite migration made them obsolete) |
| Phase 2 | impl-agent | — | No | All 11 filters implemented, no deviations |
| Phase 3 | | — | | |
| Phase 3 | impl-agent | — | No | All 11 tasks done; channel/band logic deferred to Phase 4 |
| Phase 4 | — | — | — | — |
| Phase 5 | — | — | — | — |
| Phase 6 | — | — | — | — |
+1 -1
View File
@@ -41,7 +41,7 @@ Clean-slate approach: no data migration for old source types.
|-------|--------|--------|--------|-------|-----------|
| Phase 1: Audio Filter Framework | backend | 🔨 In Progress | ⬜ | ⬜ | ⬜ |
| Phase 2: Audio Filters | backend | 🔨 In Progress | ⬜ | ⬜ | ⬜ |
| Phase 3: Processed Audio Source Model | backend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
| Phase 3: Processed Audio Source Model | backend | ✅ Done | ⬜ | ⬜ | ⬜ |
| Phase 4: Runtime Integration | backend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
| Phase 5: Frontend — Audio Processing Templates | frontend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
| Phase 6: Frontend — Source Types | frontend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
@@ -1,6 +1,6 @@
# Phase 3: Processed Audio Source Model
**Status:** ⬜ Not Started
**Status:** ✅ Done
**Parent plan:** [PLAN.md](./PLAN.md)
**Domain:** backend
@@ -9,57 +9,69 @@ Add the `ProcessedAudioSource` type, rename `MultichannelAudioSource` to `Captur
## Tasks
- [ ] Task 1: Rename `MultichannelAudioSource``CaptureAudioSource` in `storage/audio_source.py`
- [x] Task 1: Rename `MultichannelAudioSource``CaptureAudioSource` in `storage/audio_source.py`
- Change class name, update `source_type` default to `"capture"`
- Same fields: `device_index`, `is_loopback`, `audio_template_id`
- [ ] Task 2: Add `ProcessedAudioSource` dataclass in `storage/audio_source.py`
- [x] Task 2: Add `ProcessedAudioSource` dataclass in `storage/audio_source.py`
- Fields: `audio_source_id: str` (input source), `audio_processing_template_id: str`
- `source_type` = `"processed"`
- Inherits standard base fields (id, name, description, tags, created_at, updated_at)
- [ ] Task 3: Remove `MonoAudioSource` class entirely
- [ ] Task 4: Remove `BandExtractAudioSource` class entirely
- [ ] Task 5: Update `create_audio_source()` factory function to handle new types
- [ ] Task 6: Update `AudioSourceStore` resolution logic:
- [x] Task 3: Remove `MonoAudioSource` class entirely
- [x] Task 4: Remove `BandExtractAudioSource` class entirely
- [x] Task 5: Update `create_audio_source()` factory function to handle new types
- [x] Task 6: Update `AudioSourceStore` resolution logic:
- `resolve_audio_source()` now returns: device info (from CaptureAudioSource at chain end) + ordered list of filter chains (from AudioProcessingTemplates along the chain)
- Walk chain: ProcessedAudioSource → ... → CaptureAudioSource
- Collect all audio_processing_template_ids in order
- Cycle detection for ProcessedAudioSource chains
- [ ] Task 7: Update `ResolvedAudioSource` dataclass:
- [x] Task 7: Update `ResolvedAudioSource` dataclass:
- Remove `channel` and `freq_low`/`freq_high` fields (handled by filters now)
- Add `filter_instances: List[FilterInstance]` — flattened, ordered list of all filters to apply
- Or add `template_ids: List[str]` and resolve at runtime
- [ ] Task 8: Update reference validation in store:
- Add `audio_processing_template_ids: List[str]` — ordered list of template IDs along the chain
- [x] Task 8: Update reference validation in store:
- `ProcessedAudioSource.audio_source_id` must reference an existing audio source
- `ProcessedAudioSource.audio_processing_template_id` must reference an existing template
- Delete checks: can't delete a source referenced by another ProcessedAudioSource
- Delete checks: can't delete a template referenced by a ProcessedAudioSource
- [ ] Task 9: Update API schemas in `api/schemas/audio_sources.py`
- Added `get_sources_referencing_template()` helper for template delete checks
- [x] Task 9: Update API schemas in `api/schemas/audio_sources.py`
- Remove `MonoAudioSourceCreate/Update/Response` schemas
- Remove `BandExtractAudioSourceCreate/Update/Response` schemas
- Add `CaptureAudioSourceCreate/Update/Response` (rename from Multichannel)
- Add `ProcessedAudioSourceCreate/Update/Response`
- Update discriminated union to use new type literals
- [ ] Task 10: Update API routes in `api/routes/audio_sources.py`
- [x] Task 10: Update API routes in `api/routes/audio_sources.py`
- Handle new source types in create/update endpoints
- Remove handling of old types
- Update WebSocket test endpoint to work with ProcessedAudioSource
- [ ] Task 11: Update any imports/references across the codebase that reference the old types
- Update WebSocket test endpoint to work with new resolution (no channel/band)
- [x] Task 11: Update any imports/references across the codebase that reference the old types
## Files to Modify/Create
- `storage/audio_source.py`**modify** — rename, add, remove dataclasses
- `storage/audio_source_store.py`**modify** — new resolution logic, validation
- `storage/audio_template_store.py`**modify** — CaptureAudioSource import
- `api/schemas/audio_sources.py`**modify** — new schemas
- `api/routes/audio_sources.py`**modify** — handle new types
- Any files importing `MultichannelAudioSource`, `MonoAudioSource`, `BandExtractAudioSource`**modify**
- `core/processing/audio_stream.py`**modify** — remove channel/band logic
- `core/processing/value_stream.py`**modify** — remove channel logic
- `core/demo_seed.py`**modify** — update demo data to new types
- `storage/color_strip_source.py`**modify** — update comment
- `storage/value_source.py`**modify** — update comment
- `static/js/types.ts`**modify** — new TS interfaces
- `static/js/core/icons.ts`**modify** — new icon mapping
- `static/js/core/graph-nodes.ts`**modify** — new icon mapping
- `static/js/features/audio-sources.ts`**modify** — new source types
- `static/js/features/streams.ts`**modify** — new card sections
- `static/js/features/value-sources.ts`**modify** — badge text
- `static/js/features/color-strips.ts`**modify** — badge text, navigation
- `static/js/core/command-palette.ts`**modify** — navigation mapping
## Acceptance Criteria
- `CaptureAudioSource` replaces `MultichannelAudioSource` (same behavior, new name/type)
- `ProcessedAudioSource` can be created referencing a source + template
- `MonoAudioSource` and `BandExtractAudioSource` are fully removed
- Chain resolution walks ProcessedAudioSource → ... → CaptureAudioSource correctly
- Cycle detection prevents circular source references
- Reference validation prevents dangling references
- API accepts/returns new type discriminators
- [x] `CaptureAudioSource` replaces `MultichannelAudioSource` (same behavior, new name/type)
- [x] `ProcessedAudioSource` can be created referencing a source + template
- [x] `MonoAudioSource` and `BandExtractAudioSource` are fully removed
- [x] Chain resolution walks ProcessedAudioSource → ... → CaptureAudioSource correctly
- [x] Cycle detection prevents circular source references
- [x] Reference validation prevents dangling references
- [x] API accepts/returns new type discriminators
## Notes
- Clean-slate: no migration of existing data. Old source type records will be lost.
@@ -68,11 +80,37 @@ Add the `ProcessedAudioSource` type, rename `MultichannelAudioSource` to `Captur
- Template reference checks in the store need coordination with `AudioProcessingTemplateStore` — may need to pass it as a dependency.
## Review Checklist
- [ ] All tasks completed
- [ ] Code follows project conventions
- [ ] No unintended side effects
- [x] All tasks completed
- [x] Code follows project conventions
- [x] No unintended side effects
- [ ] Build passes
- [ ] Tests pass (new + existing)
## Handoff to Next Phase
<!-- Filled in by the implementation agent after completing this phase. -->
### What was built
- `CaptureAudioSource` replaces `MultichannelAudioSource` (class + source_type "capture")
- `ProcessedAudioSource` added with `audio_source_id` + `audio_processing_template_id` fields
- `MonoAudioSource` and `BandExtractAudioSource` fully removed from model, store, schemas, routes, and all frontend references
- `ResolvedAudioSource` now returns `audio_processing_template_ids: List[str]` instead of `channel`/`freq_low`/`freq_high`
- Chain resolution walks ProcessedAudioSource → ... → CaptureAudioSource, collecting template IDs in order (outermost first)
- Cycle detection for both create and update operations
- `get_sources_referencing_template()` helper added for template delete checks
- All frontend TS files updated: types, icons, card sections, navigation, command palette
### What Phase 4 needs to know
- `ResolvedAudioSource` now has `audio_processing_template_ids` field — Phase 4 must resolve these to `FilterInstance` lists and instantiate/apply them in the stream runtime
- `AudioColorStripStream._pick_channel()` currently returns raw `analysis.spectrum, analysis.rms` — Phase 4 must wire filter processing here
- `AudioValueStream._pick_rms()` and `_pick_peak()` currently return raw analysis values — Phase 4 must apply filter chain
- Both streams store `self._audio_processing_template_ids` for use by Phase 4
- The WebSocket test endpoint also needs filter application wired in Phase 4
### Temporary breakages (resolved in Phase 4)
- Channel selection removed from `AudioColorStripStream._pick_channel()` — always uses mono mix
- Channel selection removed from `AudioValueStream._pick_rms()` and `_pick_peak()` — always uses mono
- These were previously handled by MonoAudioSource/BandExtractAudioSource; now handled by channel_extract/band_extract filters in ProcessedAudioSource chains
### Known deviations from plan
- Task 7: Used `audio_processing_template_ids: List[str]` (template IDs) rather than `filter_instances: List[FilterInstance]` — runtime resolution deferred to Phase 4
- Task 8: Template reference validation at create time not implemented (would require injecting AudioProcessingTemplateStore as dependency) — deferred to Phase 4 or Phase 7
- Frontend was also updated comprehensively (not just backend) to avoid broken UI
@@ -19,15 +19,13 @@ from wled_controller.api.schemas.audio_sources import (
AudioSourceListResponse,
AudioSourceResponse,
AudioSourceUpdate,
BandExtractAudioSourceResponse,
MonoAudioSourceResponse,
MultichannelAudioSourceResponse,
CaptureAudioSourceResponse,
ProcessedAudioSourceResponse,
)
from wled_controller.storage.audio_source import (
AudioSource,
BandExtractAudioSource,
MonoAudioSource,
MultichannelAudioSource,
CaptureAudioSource,
ProcessedAudioSource,
)
from wled_controller.storage.audio_source_store import AudioSourceStore
from wled_controller.storage.color_strip_store import ColorStripStore
@@ -40,7 +38,7 @@ router = APIRouter()
_RESPONSE_MAP = {
MultichannelAudioSource: lambda s: MultichannelAudioSourceResponse(
CaptureAudioSource: lambda s: CaptureAudioSourceResponse(
id=s.id,
name=s.name,
description=s.description,
@@ -51,7 +49,7 @@ _RESPONSE_MAP = {
is_loopback=s.is_loopback,
audio_template_id=s.audio_template_id,
),
MonoAudioSource: lambda s: MonoAudioSourceResponse(
ProcessedAudioSource: lambda s: ProcessedAudioSourceResponse(
id=s.id,
name=s.name,
description=s.description,
@@ -59,19 +57,7 @@ _RESPONSE_MAP = {
created_at=s.created_at,
updated_at=s.updated_at,
audio_source_id=s.audio_source_id,
channel=s.channel,
),
BandExtractAudioSource: lambda s: BandExtractAudioSourceResponse(
id=s.id,
name=s.name,
description=s.description,
tags=s.tags,
created_at=s.created_at,
updated_at=s.updated_at,
audio_source_id=s.audio_source_id,
band=s.band,
freq_low=s.freq_low,
freq_high=s.freq_high,
audio_processing_template_id=s.audio_processing_template_id,
),
}
@@ -80,8 +66,8 @@ def _to_response(source: AudioSource) -> AudioSourceResponse:
"""Convert an AudioSource dataclass to the matching response schema."""
builder = _RESPONSE_MAP.get(type(source))
if builder is None:
# Fallback for unknown types — return as multichannel
return MultichannelAudioSourceResponse(
# Fallback for unknown types — return as capture
return CaptureAudioSourceResponse(
id=source.id,
name=source.name,
description=source.description,
@@ -99,7 +85,7 @@ def _to_response(source: AudioSource) -> AudioSourceResponse:
async def list_audio_sources(
_auth: AuthRequired,
source_type: Optional[str] = Query(
None, description="Filter by source_type: multichannel, mono, or band_extract"
None, description="Filter by source_type: capture or processed"
),
store: AudioSourceStore = Depends(get_audio_source_store),
):
@@ -220,9 +206,13 @@ async def test_audio_source_ws(
):
"""WebSocket for real-time audio spectrum analysis. Auth via ?token=<api_key>.
Resolves the audio source to its device, acquires a ManagedAudioStream
(ref-counted — shares with running targets), and streams AudioAnalysis
snapshots as JSON at ~20 Hz.
Resolves the audio source to its device and template chain, acquires a
ManagedAudioStream (ref-counted — shares with running targets), and streams
AudioAnalysis snapshots as JSON at ~20 Hz.
NOTE: Audio processing filters from the template chain are NOT applied in
this WebSocket yet — that will be wired in Phase 4 when the stream runtime
integrates filter instances.
"""
from wled_controller.api.auth import verify_ws_token
@@ -230,7 +220,7 @@ async def test_audio_source_ws(
await websocket.close(code=4001, reason="Unauthorized")
return
# Resolve source → device info + optional band filter
# Resolve source → device info + processing template chain
store = get_audio_source_store()
template_store = get_audio_template_store()
manager = get_processor_manager()
@@ -243,17 +233,9 @@ async def test_audio_source_ws(
device_index = resolved.device_index
is_loopback = resolved.is_loopback
channel = resolved.channel
audio_template_id = resolved.audio_template_id
# Precompute band mask if this is a band_extract source
band_mask = None
if resolved.freq_low is not None and resolved.freq_high is not None:
from wled_controller.core.audio.band_filter import compute_band_mask
band_mask = compute_band_mask(resolved.freq_low, resolved.freq_high)
# Resolve template → engine_type + config
# Resolve capture template → engine_type + config
engine_type = None
engine_config = None
if audio_template_id:
@@ -283,27 +265,11 @@ async def test_audio_source_ws(
if analysis is not None and analysis.timestamp != last_ts:
last_ts = analysis.timestamp
# Select channel-specific data
if channel == "left":
spectrum = analysis.left_spectrum
rms = analysis.left_rms
elif channel == "right":
spectrum = analysis.right_spectrum
rms = analysis.right_rms
else:
spectrum = analysis.spectrum
rms = analysis.rms
# Apply band filter if present
if band_mask is not None:
from wled_controller.core.audio.band_filter import apply_band_filter
spectrum, rms = apply_band_filter(spectrum, rms, band_mask)
# Send raw analysis — filter processing will be added in Phase 4
await websocket.send_json(
{
"spectrum": spectrum.tolist(),
"rms": round(rms, 4),
"spectrum": analysis.spectrum.tolist(),
"rms": round(analysis.rms, 4),
"peak": round(analysis.peak, 4),
"beat": analysis.beat,
"beat_intensity": round(analysis.beat_intensity, 4),
@@ -21,32 +21,23 @@ class _AudioSourceResponseBase(BaseModel):
updated_at: datetime = Field(description="Last update timestamp")
class MultichannelAudioSourceResponse(_AudioSourceResponseBase):
source_type: Literal["multichannel"] = "multichannel"
class CaptureAudioSourceResponse(_AudioSourceResponseBase):
source_type: Literal["capture"] = "capture"
device_index: int = Field(description="Audio device index (-1 = default)")
is_loopback: bool = Field(description="WASAPI loopback mode")
audio_template_id: Optional[str] = Field(None, description="Audio capture template ID")
class MonoAudioSourceResponse(_AudioSourceResponseBase):
source_type: Literal["mono"] = "mono"
audio_source_id: str = Field(description="Parent audio source ID")
channel: str = Field(description="Channel: mono|left|right")
class BandExtractAudioSourceResponse(_AudioSourceResponseBase):
source_type: Literal["band_extract"] = "band_extract"
audio_source_id: str = Field(description="Parent audio source ID")
band: str = Field(description="Band preset: bass|mid|treble|custom")
freq_low: float = Field(description="Low frequency bound (Hz)")
freq_high: float = Field(description="High frequency bound (Hz)")
class ProcessedAudioSourceResponse(_AudioSourceResponseBase):
source_type: Literal["processed"] = "processed"
audio_source_id: str = Field(description="Input audio source ID")
audio_processing_template_id: str = Field(description="Audio processing template ID")
AudioSourceResponse = Annotated[
Union[
Annotated[MultichannelAudioSourceResponse, Tag("multichannel")],
Annotated[MonoAudioSourceResponse, Tag("mono")],
Annotated[BandExtractAudioSourceResponse, Tag("band_extract")],
Annotated[CaptureAudioSourceResponse, Tag("capture")],
Annotated[ProcessedAudioSourceResponse, Tag("processed")],
],
Discriminator("source_type"),
]
@@ -64,32 +55,23 @@ class _AudioSourceCreateBase(BaseModel):
tags: List[str] = Field(default_factory=list, description="User-defined tags")
class MultichannelAudioSourceCreate(_AudioSourceCreateBase):
source_type: Literal["multichannel"] = "multichannel"
class CaptureAudioSourceCreate(_AudioSourceCreateBase):
source_type: Literal["capture"] = "capture"
device_index: int = Field(-1, description="Audio device index (-1 = default)")
is_loopback: bool = Field(True, description="True for system audio (WASAPI loopback)")
audio_template_id: Optional[str] = Field(None, description="Audio capture template ID")
class MonoAudioSourceCreate(_AudioSourceCreateBase):
source_type: Literal["mono"] = "mono"
audio_source_id: str = Field("", description="Parent audio source ID")
channel: str = Field("mono", description="Channel: mono|left|right")
class BandExtractAudioSourceCreate(_AudioSourceCreateBase):
source_type: Literal["band_extract"] = "band_extract"
audio_source_id: str = Field("", description="Parent audio source ID")
band: str = Field("bass", description="Band preset: bass|mid|treble|custom")
freq_low: float = Field(20.0, description="Low frequency bound (Hz)", ge=20, le=20000)
freq_high: float = Field(250.0, description="High frequency bound (Hz)", ge=20, le=20000)
class ProcessedAudioSourceCreate(_AudioSourceCreateBase):
source_type: Literal["processed"] = "processed"
audio_source_id: str = Field(description="Input audio source ID")
audio_processing_template_id: str = Field(description="Audio processing template ID")
AudioSourceCreate = Annotated[
Union[
Annotated[MultichannelAudioSourceCreate, Tag("multichannel")],
Annotated[MonoAudioSourceCreate, Tag("mono")],
Annotated[BandExtractAudioSourceCreate, Tag("band_extract")],
Annotated[CaptureAudioSourceCreate, Tag("capture")],
Annotated[ProcessedAudioSourceCreate, Tag("processed")],
],
Discriminator("source_type"),
]
@@ -107,34 +89,25 @@ class _AudioSourceUpdateBase(BaseModel):
tags: Optional[List[str]] = None
class MultichannelAudioSourceUpdate(_AudioSourceUpdateBase):
source_type: Literal["multichannel"] = "multichannel"
class CaptureAudioSourceUpdate(_AudioSourceUpdateBase):
source_type: Literal["capture"] = "capture"
device_index: Optional[int] = Field(None, description="Audio device index (-1 = default)")
is_loopback: Optional[bool] = Field(None, description="True for system audio (WASAPI loopback)")
audio_template_id: Optional[str] = Field(None, description="Audio capture template ID")
class MonoAudioSourceUpdate(_AudioSourceUpdateBase):
source_type: Literal["mono"] = "mono"
audio_source_id: Optional[str] = Field(None, description="Parent audio source ID")
channel: Optional[str] = Field(None, description="Channel: mono|left|right")
class BandExtractAudioSourceUpdate(_AudioSourceUpdateBase):
source_type: Literal["band_extract"] = "band_extract"
audio_source_id: Optional[str] = Field(None, description="Parent audio source ID")
band: Optional[str] = Field(None, description="Band preset: bass|mid|treble|custom")
freq_low: Optional[float] = Field(None, description="Low frequency bound (Hz)", ge=20, le=20000)
freq_high: Optional[float] = Field(
None, description="High frequency bound (Hz)", ge=20, le=20000
class ProcessedAudioSourceUpdate(_AudioSourceUpdateBase):
source_type: Literal["processed"] = "processed"
audio_source_id: Optional[str] = Field(None, description="Input audio source ID")
audio_processing_template_id: Optional[str] = Field(
None, description="Audio processing template ID"
)
AudioSourceUpdate = Annotated[
Union[
Annotated[MultichannelAudioSourceUpdate, Tag("multichannel")],
Annotated[MonoAudioSourceUpdate, Tag("mono")],
Annotated[BandExtractAudioSourceUpdate, Tag("band_extract")],
Annotated[CaptureAudioSourceUpdate, Tag("capture")],
Annotated[ProcessedAudioSourceUpdate, Tag("processed")],
],
Discriminator("source_type"),
]
+3 -18
View File
@@ -39,7 +39,6 @@ _CSS_IDS = {
_AS_IDS = {
"system": "as_demo0001",
"mono": "as_demo0002",
}
_TPL_ID = "tpl_demo0001"
@@ -316,7 +315,7 @@ def _build_color_strip_sources() -> dict:
"clock_id": None,
"tags": ["demo"],
"visualization_mode": "spectrum",
"audio_source_id": _AS_IDS["mono"],
"audio_source_id": _AS_IDS["system"],
"sensitivity": 1.0,
"smoothing": 0.3,
"palette": "rainbow",
@@ -338,7 +337,7 @@ def _build_audio_sources() -> dict:
_AS_IDS["system"]: {
"id": _AS_IDS["system"],
"name": "Demo System Audio",
"source_type": "multichannel",
"source_type": "capture",
"device_index": 1,
"is_loopback": True,
"audio_template_id": None,
@@ -347,21 +346,7 @@ def _build_audio_sources() -> dict:
"created_at": _NOW,
"updated_at": _NOW,
"audio_source_id": None,
"channel": None,
},
_AS_IDS["mono"]: {
"id": _AS_IDS["mono"],
"name": "Demo Audio — Mono",
"source_type": "mono",
"audio_source_id": _AS_IDS["system"],
"channel": "mono",
"description": "Mono mix of demo system audio",
"tags": ["demo"],
"created_at": _NOW,
"updated_at": _NOW,
"device_index": None,
"is_loopback": None,
"audio_template_id": None,
"audio_processing_template_id": None,
},
}
@@ -17,7 +17,6 @@ import numpy as np
from wled_controller.core.audio.analysis import NUM_BANDS
from wled_controller.core.audio.audio_capture import AudioCaptureManager
from wled_controller.core.audio.band_filter import apply_band_filter, compute_band_mask
from wled_controller.core.processing.color_strip_stream import ColorStripStream
from wled_controller.core.processing.effect_stream import _build_palette_lut
from wled_controller.utils import get_logger
@@ -104,20 +103,21 @@ class AudioColorStripStream(ColorStripStream):
self._led_count = source.led_count if source.led_count and source.led_count > 0 else 1
self._mirror = bool(getattr(source, "mirror", False))
# Resolve audio device/channel/template via audio_source_id
# Resolve audio device/template via audio_source_id
# NOTE: channel selection and band filtering are now handled by audio
# processing filters (channel_extract, band_extract) applied via
# ProcessedAudioSource chains. Filter application will be wired in Phase 4.
audio_source_id = getattr(source, "audio_source_id", "")
self._audio_source_id = audio_source_id
self._audio_engine_type = None
self._audio_engine_config = None
self._band_mask = None # precomputed band filter mask (None = full range)
self._audio_processing_template_ids: list = []
if audio_source_id and self._audio_source_store:
try:
resolved = self._audio_source_store.resolve_audio_source(audio_source_id)
self._audio_device_index = resolved.device_index
self._audio_loopback = resolved.is_loopback
self._audio_channel = resolved.channel
if resolved.freq_low is not None and resolved.freq_high is not None:
self._band_mask = compute_band_mask(resolved.freq_low, resolved.freq_high)
self._audio_processing_template_ids = list(resolved.audio_processing_template_ids)
if resolved.audio_template_id and self._audio_template_store:
try:
tpl = self._audio_template_store.get_template(resolved.audio_template_id)
@@ -134,11 +134,9 @@ class AudioColorStripStream(ColorStripStream):
logger.warning(f"Failed to resolve audio source {audio_source_id}: {e}")
self._audio_device_index = -1
self._audio_loopback = True
self._audio_channel = "mono"
else:
self._audio_device_index = -1
self._audio_loopback = True
self._audio_channel = "mono"
with self._colors_lock:
self._colors: Optional[np.ndarray] = None
@@ -343,16 +341,14 @@ class AudioColorStripStream(ColorStripStream):
# ── Channel selection ─────────────────────────────────────────
def _pick_channel(self, analysis):
"""Return (spectrum, rms) for the configured audio channel, with band filtering."""
if self._audio_channel == "left":
spectrum, rms = analysis.left_spectrum, analysis.left_rms
elif self._audio_channel == "right":
spectrum, rms = analysis.right_spectrum, analysis.right_rms
else:
spectrum, rms = analysis.spectrum, analysis.rms
if self._band_mask is not None:
spectrum, rms = apply_band_filter(spectrum, rms, self._band_mask)
return spectrum, rms
"""Return (spectrum, rms) from the analysis.
Channel selection and band filtering are now handled by audio processing
filters applied via ProcessedAudioSource chains (Phase 4 will wire filter
instances into the stream runtime).
"""
# ⚠️ Temporary breakage: channel/band filtering removed — resolved in Phase 4
return analysis.spectrum, analysis.rms
# ── Spectrum Analyzer ──────────────────────────────────────────
@@ -195,9 +195,9 @@ class AudioValueStream(ValueStream):
# Resolved audio device params
self._audio_device_index = -1
self._audio_loopback = True
self._audio_channel = "mono"
self._audio_engine_type = None
self._audio_engine_config = None
self._audio_processing_template_ids: list = []
self._audio_stream = None
self._prev_value = 0.0
@@ -206,24 +206,27 @@ class AudioValueStream(ValueStream):
self._resolve_audio_source()
def _resolve_audio_source(self) -> None:
"""Resolve audio source to device index / channel / engine info."""
"""Resolve audio source to device index / engine info / processing template IDs.
Channel selection and band filtering are now handled by audio processing
filters applied via ProcessedAudioSource chains (Phase 4 will wire filter
instances into the stream runtime).
"""
if self._audio_source_id and self._audio_source_store:
try:
device_index, is_loopback, channel, template_id = (
self._audio_source_store.resolve_audio_source(self._audio_source_id)
)
self._audio_device_index = device_index
self._audio_loopback = is_loopback
self._audio_channel = channel
if template_id and self._audio_template_store:
resolved = self._audio_source_store.resolve_audio_source(self._audio_source_id)
self._audio_device_index = resolved.device_index
self._audio_loopback = resolved.is_loopback
self._audio_processing_template_ids = list(resolved.audio_processing_template_ids)
if resolved.audio_template_id and self._audio_template_store:
try:
tpl = self._audio_template_store.get_template(template_id)
tpl = self._audio_template_store.get_template(resolved.audio_template_id)
self._audio_engine_type = tpl.engine_type
self._audio_engine_config = tpl.engine_config
except ValueError as e:
logger.warning(
"Audio template %s not found for value stream, using default engine: %s",
template_id,
resolved.audio_template_id,
e,
)
pass
@@ -291,17 +294,13 @@ class AudioValueStream(ValueStream):
return self._pick_rms(analysis)
def _pick_rms(self, analysis) -> float:
if self._audio_channel == "left":
return getattr(analysis, "left_rms", 0.0)
if self._audio_channel == "right":
return getattr(analysis, "right_rms", 0.0)
# ⚠️ Temporary breakage: channel selection removed — resolved in Phase 4
# Channel selection is now handled by audio processing filters
# (channel_extract) applied via ProcessedAudioSource chains.
return getattr(analysis, "rms", 0.0)
def _pick_peak(self, analysis) -> float:
if self._audio_channel == "left":
return getattr(analysis, "left_peak", 0.0)
if self._audio_channel == "right":
return getattr(analysis, "right_peak", 0.0)
# ⚠️ Temporary breakage: channel selection removed — resolved in Phase 4
return getattr(analysis, "peak", 0.0)
def _compute_beat(self, analysis) -> float:
@@ -335,6 +335,13 @@
min-width: 48px;
}
/* ── Integrations grid ── */
.dashboard-integrations-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(250px, 1fr));
gap: 4px;
}
.dashboard-autostart-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(250px, 1fr));
@@ -127,8 +127,8 @@ function _buildItems(results: any[], states: any = {}) {
}));
_mapEntities(audioSrc, a => {
const section = a.source_type === 'mono' ? 'audio-mono' : a.source_type === 'band_extract' ? 'audio-band-extract' : 'audio-multi';
const tab = a.source_type === 'mono' ? 'audio_mono' : a.source_type === 'band_extract' ? 'audio_band_extract' : 'audio_multi';
const section = a.source_type === 'processed' ? 'audio-processed' : 'audio-capture';
const tab = a.source_type === 'processed' ? 'audio_processed' : 'audio_capture';
items.push({
name: a.name, detail: a.source_type, group: 'audio', icon: getAudioSourceIcon(a.source_type),
nav: ['streams', tab, section, 'data-id', a.id],
@@ -104,7 +104,7 @@ const SUBTYPE_ICONS = {
static: P.layoutDashboard, animated: P.refreshCw, audio: P.music,
adaptive_time: P.clock, adaptive_scene: P.cloudSun, daylight: P.sun,
},
audio_source: { mono: P.mic, multichannel: P.volume2 },
audio_source: { capture: P.volume2, processed: P.slidersHorizontal },
output_target: { led: P.lightbulb, wled: P.lightbulb, ha_light: P.lightbulb },
};
@@ -42,7 +42,7 @@ const _valueSourceTypeIcons = {
system_metrics: _svg(P.cpu),
game_event: _svg(P.gamepad2),
};
const _audioSourceTypeIcons = { mono: _svg(P.mic), multichannel: _svg(P.volume2), band_extract: _svg(P.activity) };
const _audioSourceTypeIcons = { capture: _svg(P.volume2), processed: _svg(P.slidersHorizontal) };
const _deviceTypeIcons = {
wled: _svg(P.wifi), adalight: _svg(P.usb), ambiled: _svg(P.usb),
mqtt: _svg(P.send), ws: _svg(P.globe), openrgb: _svg(P.palette),
@@ -1,10 +1,9 @@
/**
* Audio Sources — CRUD for multichannel, mono, and band extract audio sources.
* Audio Sources — CRUD for capture and processed audio sources.
*
* Audio sources are managed entities that encapsulate audio device
* configuration. Multichannel sources represent physical audio devices;
* mono sources extract a single channel from a multichannel source;
* band extract sources filter a parent source to a frequency band.
* configuration. Capture sources represent physical audio devices;
* processed sources apply audio processing filters to another source.
* CSS audio type references an audio source by ID.
*
* Card rendering is handled by streams.js (Audio tab).
@@ -30,8 +29,6 @@ class AudioSourceModal extends Modal {
onForceClose() {
if (_audioSourceTagsInput) { _audioSourceTagsInput.destroy(); _audioSourceTagsInput = null; }
if (_asChannelIconSelect) { _asChannelIconSelect.destroy(); _asChannelIconSelect = null; }
if (_asBandIconSelect) { _asBandIconSelect.destroy(); _asBandIconSelect = null; }
}
snapshotValues() {
@@ -41,12 +38,8 @@ class AudioSourceModal extends Modal {
type: (document.getElementById('audio-source-type') as HTMLSelectElement).value,
device: (document.getElementById('audio-source-device') as HTMLSelectElement).value,
audioTemplate: (document.getElementById('audio-source-audio-template') as HTMLSelectElement).value,
parent: (document.getElementById('audio-source-parent') as HTMLSelectElement).value,
channel: (document.getElementById('audio-source-channel') as HTMLSelectElement).value,
bandParent: (document.getElementById('audio-source-band-parent') as HTMLSelectElement).value,
band: (document.getElementById('audio-source-band') as HTMLSelectElement).value,
freqLow: (document.getElementById('audio-source-freq-low') as HTMLInputElement).value,
freqHigh: (document.getElementById('audio-source-freq-high') as HTMLInputElement).value,
parentSource: (document.getElementById('audio-source-parent') as HTMLSelectElement).value,
processingTemplate: (document.getElementById('audio-source-processing-template') as HTMLSelectElement).value,
tags: JSON.stringify(_audioSourceTagsInput ? _audioSourceTagsInput.getValue() : []),
};
}
@@ -54,25 +47,14 @@ class AudioSourceModal extends Modal {
const audioSourceModal = new AudioSourceModal();
// ── EntitySelect / IconSelect instances for audio source editor ──
// ── EntitySelect instances for audio source editor ──
let _asTemplateEntitySelect: EntitySelect | null = null;
let _asDeviceEntitySelect: EntitySelect | null = null;
let _asParentEntitySelect: EntitySelect | null = null;
let _asBandParentEntitySelect: EntitySelect | null = null;
let _asBandIconSelect: IconSelect | null = null;
let _asChannelIconSelect: IconSelect | null = null;
let _asProcessingTemplateEntitySelect: EntitySelect | null = null;
const _svg = (d: string): string => `<svg class="icon" viewBox="0 0 24 24">${d}</svg>`;
function _buildBandItems() {
return [
{ value: 'bass', icon: _svg(P.volume2), label: t('audio_source.band.bass'), desc: '20250 Hz' },
{ value: 'mid', icon: _svg(P.music), label: t('audio_source.band.mid'), desc: '2504000 Hz' },
{ value: 'treble', icon: _svg(P.zap), label: t('audio_source.band.treble'), desc: '4k20k Hz' },
{ value: 'custom', icon: _svg(P.slidersHorizontal), label: t('audio_source.band.custom') },
];
}
// ── Auto-name generation ──────────────────────────────────────
let _asNameManuallyEdited = false;
@@ -82,24 +64,14 @@ function _autoGenerateAudioSourceName() {
if ((document.getElementById('audio-source-id') as HTMLInputElement).value) return;
const type = (document.getElementById('audio-source-type') as HTMLSelectElement).value;
let name = '';
if (type === 'multichannel') {
if (type === 'capture') {
const devSel = document.getElementById('audio-source-device') as HTMLSelectElement | null;
const devName = devSel?.selectedOptions[0]?.textContent?.trim();
name = devName || t('audio_source.type.multichannel');
} else if (type === 'mono') {
name = devName || t('audio_source.type.capture');
} else if (type === 'processed') {
const parentSel = document.getElementById('audio-source-parent') as HTMLSelectElement | null;
const parentName = parentSel?.selectedOptions[0]?.textContent?.trim() || '';
const ch = (document.getElementById('audio-source-channel') as HTMLSelectElement).value;
const chLabel = ch === 'left' ? 'L' : ch === 'right' ? 'R' : 'M';
name = parentName ? `${parentName} · ${chLabel}` : t('audio_source.type.mono');
} else if (type === 'band_extract') {
const parentSel = document.getElementById('audio-source-band-parent') as HTMLSelectElement | null;
const parentName = parentSel?.selectedOptions[0]?.textContent?.trim() || '';
const band = (document.getElementById('audio-source-band') as HTMLSelectElement).value;
const bandLabel = band === 'custom'
? `${(document.getElementById('audio-source-freq-low') as HTMLInputElement).value}${(document.getElementById('audio-source-freq-high') as HTMLInputElement).value} Hz`
: t(`audio_source.band.${band}`);
name = parentName ? `${parentName} · ${bandLabel}` : bandLabel;
name = parentName ? `${parentName} (processed)` : t('audio_source.type.processed');
}
(document.getElementById('audio-source-name') as HTMLInputElement).value = name;
}
@@ -107,15 +79,14 @@ function _autoGenerateAudioSourceName() {
// ── Modal ─────────────────────────────────────────────────────
const _titleKeys: Record<string, Record<string, string>> = {
multichannel: { add: 'audio_source.add.multichannel', edit: 'audio_source.edit.multichannel' },
mono: { add: 'audio_source.add.mono', edit: 'audio_source.edit.mono' },
band_extract: { add: 'audio_source.add.band_extract', edit: 'audio_source.edit.band_extract' },
capture: { add: 'audio_source.add.capture', edit: 'audio_source.edit.capture' },
processed: { add: 'audio_source.add.processed', edit: 'audio_source.edit.processed' },
};
export async function showAudioSourceModal(sourceType: any, editData?: any) {
const isEdit = !!editData;
const st = isEdit ? editData.source_type : sourceType;
const titleKey = _titleKeys[st]?.[isEdit ? 'edit' : 'add'] || _titleKeys.multichannel.add;
const titleKey = _titleKeys[st]?.[isEdit ? 'edit' : 'add'] || _titleKeys.capture.add;
document.getElementById('audio-source-modal-title')!.innerHTML = `${ICON_MUSIC} ${t(titleKey)}`;
(document.getElementById('audio-source-id') as HTMLInputElement).value = isEdit ? editData.id : '';
@@ -131,41 +102,26 @@ export async function showAudioSourceModal(sourceType: any, editData?: any) {
(document.getElementById('audio-source-name') as HTMLInputElement).value = editData.name || '';
(document.getElementById('audio-source-description') as HTMLInputElement).value = editData.description || '';
if (editData.source_type === 'multichannel') {
if (editData.source_type === 'capture') {
_loadAudioTemplates(editData.audio_template_id);
(document.getElementById('audio-source-audio-template') as HTMLSelectElement).onchange = () => { _filterDevicesBySelectedTemplate(); _autoGenerateAudioSourceName(); };
await _loadAudioDevices();
_selectAudioDevice(editData.device_index, editData.is_loopback);
} else if (editData.source_type === 'mono') {
_loadMultichannelSources(editData.audio_source_id);
(document.getElementById('audio-source-channel') as HTMLSelectElement).value = editData.channel || 'mono';
_ensureChannelIconSelect();
} else if (editData.source_type === 'band_extract') {
_loadBandParentSources(editData.audio_source_id);
(document.getElementById('audio-source-band') as HTMLSelectElement).value = editData.band || 'bass';
_ensureBandIconSelect();
(document.getElementById('audio-source-freq-low') as HTMLInputElement).value = String(editData.freq_low ?? 20);
(document.getElementById('audio-source-freq-high') as HTMLInputElement).value = String(editData.freq_high ?? 20000);
onBandPresetChange();
} else if (editData.source_type === 'processed') {
_loadParentSources(editData.audio_source_id);
_loadProcessingTemplates(editData.audio_processing_template_id);
}
} else {
(document.getElementById('audio-source-name') as HTMLInputElement).value = '';
(document.getElementById('audio-source-description') as HTMLInputElement).value = '';
if (sourceType === 'multichannel') {
if (sourceType === 'capture') {
_loadAudioTemplates();
(document.getElementById('audio-source-audio-template') as HTMLSelectElement).onchange = () => { _filterDevicesBySelectedTemplate(); _autoGenerateAudioSourceName(); };
await _loadAudioDevices();
} else if (sourceType === 'mono') {
_loadMultichannelSources();
_ensureChannelIconSelect();
} else if (sourceType === 'band_extract') {
_loadBandParentSources();
(document.getElementById('audio-source-band') as HTMLSelectElement).value = 'bass';
_ensureBandIconSelect();
(document.getElementById('audio-source-freq-low') as HTMLInputElement).value = '20';
(document.getElementById('audio-source-freq-high') as HTMLInputElement).value = '20000';
onBandPresetChange();
} else if (sourceType === 'processed') {
_loadParentSources();
_loadProcessingTemplates();
}
}
@@ -189,14 +145,17 @@ export async function closeAudioSourceModal() {
export function onAudioSourceTypeChange() {
const type = (document.getElementById('audio-source-type') as HTMLSelectElement).value;
(document.getElementById('audio-source-multichannel-section') as HTMLElement).style.display = type === 'multichannel' ? '' : 'none';
(document.getElementById('audio-source-mono-section') as HTMLElement).style.display = type === 'mono' ? '' : 'none';
(document.getElementById('audio-source-band-extract-section') as HTMLElement).style.display = type === 'band_extract' ? '' : 'none';
}
export function onBandPresetChange() {
const band = (document.getElementById('audio-source-band') as HTMLSelectElement).value;
(document.getElementById('audio-source-custom-freq') as HTMLElement).style.display = band === 'custom' ? '' : 'none';
const captureSection = document.getElementById('audio-source-capture-section');
const processedSection = document.getElementById('audio-source-processed-section');
if (captureSection) captureSection.style.display = type === 'capture' ? '' : 'none';
if (processedSection) processedSection.style.display = type === 'processed' ? '' : 'none';
// Legacy sections — hide if present
const legacyMulti = document.getElementById('audio-source-multichannel-section');
const legacyMono = document.getElementById('audio-source-mono-section');
const legacyBand = document.getElementById('audio-source-band-extract-section');
if (legacyMulti) legacyMulti.style.display = 'none';
if (legacyMono) legacyMono.style.display = 'none';
if (legacyBand) legacyBand.style.display = 'none';
}
// ── Save ──────────────────────────────────────────────────────
@@ -216,22 +175,15 @@ export async function saveAudioSource() {
const payload: any = { name, source_type: sourceType, description, tags: _audioSourceTagsInput ? _audioSourceTagsInput.getValue() : [] };
if (sourceType === 'multichannel') {
if (sourceType === 'capture') {
const deviceVal = (document.getElementById('audio-source-device') as HTMLSelectElement).value || '-1:1';
const [devIdx, devLoop] = deviceVal.split(':');
payload.device_index = parseInt(devIdx) || -1;
payload.is_loopback = devLoop !== '0';
payload.audio_template_id = (document.getElementById('audio-source-audio-template') as HTMLSelectElement).value || null;
} else if (sourceType === 'mono') {
} else if (sourceType === 'processed') {
payload.audio_source_id = (document.getElementById('audio-source-parent') as HTMLSelectElement).value;
payload.channel = (document.getElementById('audio-source-channel') as HTMLSelectElement).value;
} else if (sourceType === 'band_extract') {
payload.audio_source_id = (document.getElementById('audio-source-band-parent') as HTMLSelectElement).value;
payload.band = (document.getElementById('audio-source-band') as HTMLSelectElement).value;
if (payload.band === 'custom') {
payload.freq_low = parseFloat((document.getElementById('audio-source-freq-low') as HTMLInputElement).value) || 20;
payload.freq_high = parseFloat((document.getElementById('audio-source-freq-high') as HTMLInputElement).value) || 20000;
}
payload.audio_processing_template_id = (document.getElementById('audio-source-processing-template') as HTMLSelectElement).value;
}
try {
@@ -392,77 +344,18 @@ function _selectAudioDevice(deviceIndex: any, isLoopback: any) {
if (opt) select.value = val;
}
function _loadMultichannelSources(selectedId?: any) {
function _loadParentSources(selectedId?: any) {
const select = document.getElementById('audio-source-parent') as HTMLSelectElement | null;
if (!select) return;
const multichannel = _cachedAudioSources.filter(s => s.source_type === 'multichannel');
select.innerHTML = multichannel.map(s =>
`<option value="${s.id}"${s.id === selectedId ? ' selected' : ''}>${escapeHtml(s.name)}</option>`
).join('');
if (_asParentEntitySelect) _asParentEntitySelect.destroy();
if (multichannel.length > 0) {
_asParentEntitySelect = new EntitySelect({
target: select,
getItems: () => multichannel.map((s: any) => ({
value: s.id,
label: s.name,
icon: getAudioSourceIcon('multichannel'),
})),
placeholder: t('palette.search'),
} as any);
}
}
function _ensureBandIconSelect() {
const sel = document.getElementById('audio-source-band') as HTMLSelectElement | null;
if (!sel) return;
if (_asBandIconSelect) {
_asBandIconSelect.updateItems(_buildBandItems());
return;
}
_asBandIconSelect = new IconSelect({
target: sel,
items: _buildBandItems(),
columns: 2,
onChange: () => { onBandPresetChange(); _autoGenerateAudioSourceName(); },
});
}
const _icon = (d: string) => `<svg class="icon" viewBox="0 0 24 24">${d}</svg>`;
function _ensureChannelIconSelect() {
const sel = document.getElementById('audio-source-channel') as HTMLSelectElement | null;
if (!sel) return;
const items = [
{ value: 'mono', icon: _icon(P.headphones), label: t('audio_source.channel.mono'), desc: t('audio_source.channel.mono.desc') },
{ value: 'left', icon: _icon(P.volume2), label: t('audio_source.channel.left'), desc: t('audio_source.channel.left.desc') },
{ value: 'right', icon: _icon(P.volume2), label: t('audio_source.channel.right'), desc: t('audio_source.channel.right.desc') },
];
if (_asChannelIconSelect) {
_asChannelIconSelect.updateItems(items);
return;
}
_asChannelIconSelect = new IconSelect({
target: sel,
items,
columns: 3,
onChange: () => _autoGenerateAudioSourceName(),
});
}
function _loadBandParentSources(selectedId?: any) {
const select = document.getElementById('audio-source-band-parent') as HTMLSelectElement | null;
if (!select) return;
// Band extract can reference any audio source type
// Processed sources can reference any audio source type
const sources = _cachedAudioSources;
select.innerHTML = sources.map(s =>
`<option value="${s.id}"${s.id === selectedId ? ' selected' : ''}>${escapeHtml(s.name)}</option>`
).join('');
if (_asBandParentEntitySelect) _asBandParentEntitySelect.destroy();
if (_asParentEntitySelect) _asParentEntitySelect.destroy();
if (sources.length > 0) {
_asBandParentEntitySelect = new EntitySelect({
_asParentEntitySelect = new EntitySelect({
target: select,
getItems: () => sources.map((s: any) => ({
value: s.id,
@@ -475,6 +368,34 @@ function _loadBandParentSources(selectedId?: any) {
}
}
function _loadProcessingTemplates(selectedId?: any) {
const select = document.getElementById('audio-source-processing-template') as HTMLSelectElement | null;
if (!select) return;
// TODO: Load audio processing templates from cache/API when available in Phase 6
// For now, populate from existing audio processing templates endpoint
fetchWithAuth('/audio-processing-templates').then(async resp => {
if (!resp.ok) return;
const data = await resp.json();
const templates = data.templates || [];
select.innerHTML = templates.map((t: any) =>
`<option value="${t.id}"${t.id === selectedId ? ' selected' : ''}>${escapeHtml(t.name)}</option>`
).join('');
if (_asProcessingTemplateEntitySelect) _asProcessingTemplateEntitySelect.destroy();
if (templates.length > 0) {
_asProcessingTemplateEntitySelect = new EntitySelect({
target: select,
getItems: () => templates.map((tmpl: any) => ({
value: tmpl.id,
label: tmpl.name,
icon: ICON_AUDIO_TEMPLATE,
})),
placeholder: t('palette.search'),
} as any);
}
}).catch(() => {});
}
function _loadAudioTemplates(selectedId?: any) {
const select = document.getElementById('audio-source-audio-template') as HTMLSelectElement | null;
if (!select) return;
@@ -623,7 +544,7 @@ export function initAudioSourceDelegation(container: HTMLElement): void {
const handler = _audioSourceActions[action];
if (handler) {
// Verify we're inside an audio source section
const section = btn.closest<HTMLElement>('[data-card-section="audio-multi"], [data-card-section="audio-mono"], [data-card-section="audio-band-extract"]');
const section = btn.closest<HTMLElement>('[data-card-section="audio-capture"], [data-card-section="audio-processed"]');
if (!section) return;
const card = btn.closest<HTMLElement>('[data-id]');
const id = card?.getAttribute('data-id');
@@ -695,3 +616,11 @@ function _renderAudioSpectrum() {
beatDot!.classList.remove('active');
}
}
// ── Removed types ─────────────────────────────────────────────
// MonoAudioSource and BandExtractAudioSource have been removed.
// Channel selection is now handled by the channel_extract audio filter.
// Band filtering is now handled by the band_extract audio filter.
// These are applied via ProcessedAudioSource referencing an AudioProcessingTemplate.
// Exported stubs for backward compatibility (no-op):
export function onBandPresetChange() { /* removed */ }
@@ -1544,7 +1544,7 @@ async function _loadAudioSources() {
try {
const sources: any[] = await audioSourcesCache.fetch();
select.innerHTML = sources.map(s => {
const badge = s.source_type === 'multichannel' ? ' [multichannel]' : s.source_type === 'band_extract' ? ' [band]' : ' [mono]';
const badge = s.source_type === 'capture' ? ' [capture]' : ' [processed]';
return `<option value="${s.id}">${escapeHtml(s.name)}${badge}</option>`;
}).join('');
if (sources.length === 0) {
@@ -1693,8 +1693,8 @@ const CSS_CARD_RENDERERS: Record<string, CardPropsRenderer> = {
${source.audio_source_id ? (() => {
const as = audioSourceMap && audioSourceMap[source.audio_source_id];
const asName = as ? as.name : source.audio_source_id;
const asSection = as ? (as.source_type === 'mono' ? 'audio-mono' : as.source_type === 'band_extract' ? 'audio-band-extract' : 'audio-multi') : 'audio-multi';
const asTab = as ? (as.source_type === 'mono' ? 'audio_mono' : as.source_type === 'band_extract' ? 'audio_band_extract' : 'audio_multi') : 'audio_multi';
const asSection = as ? (as.source_type === 'processed' ? 'audio-processed' : 'audio-capture') : 'audio-capture';
const asTab = as ? (as.source_type === 'processed' ? 'audio_processed' : 'audio_capture') : 'audio_capture';
return `<span class="stream-card-prop${as ? ' stream-card-link' : ''}" title="${t('color_strip.audio.source')}"${as ? ` onclick="event.stopPropagation(); navigateToCard('streams','${asTab}','${asSection}','data-id','${source.audio_source_id}')"` : ''}>${ICON_AUDIO_LOOPBACK} ${escapeHtml(asName)}</span>`;
})() : ''}
${source.mirror ? `<span class="stream-card-prop">🪞</span>` : ''}
@@ -4,7 +4,7 @@
*/
import { t } from '../core/i18n.ts';
import { ICON_HEART, ICON_EXTERNAL_LINK, ICON_X, ICON_GITHUB } from '../core/icons.ts';
import { ICON_HEART, ICON_EXTERNAL_LINK, ICON_X, ICON_GITHUB, ICON_HELP } from '../core/icons.ts';
// ─── Config ─────────────────────────────────────────────────
@@ -114,13 +114,10 @@ function _showBanner(): void {
</a>`;
}
if (_repoUrl) {
actions += `<a href="${_repoUrl}" target="_blank" rel="noopener"
class="btn btn-icon donation-banner-action"
title="${t('donation.view_source')}">
${ICON_GITHUB}
</a>`;
}
actions += `<button class="btn btn-icon donation-banner-action"
onclick="openSettingsModal(); switchSettingsTab('about')" title="${t('donation.about')}">
${ICON_HELP}
</button>`;
actions += `<button class="btn btn-icon donation-banner-action"
onclick="snoozeDonation()" title="${t('donation.later')}">
@@ -64,7 +64,7 @@ import {
getEngineIcon, getAudioEngineIcon, getPictureSourceIcon, getAudioSourceIcon, getColorStripIcon,
ICON_TEMPLATE, ICON_CLONE, ICON_EDIT, ICON_TEST, ICON_LINK_SOURCE,
ICON_FPS, ICON_WEB, ICON_VALUE_SOURCE, ICON_CLOCK, ICON_AUDIO_LOOPBACK, ICON_AUDIO_INPUT,
ICON_AUDIO_TEMPLATE, ICON_MONITOR, ICON_WRENCH, ICON_RADIO, ICON_ACTIVITY,
ICON_AUDIO_TEMPLATE, ICON_MONITOR, ICON_WRENCH,
ICON_CAPTURE_TEMPLATE, ICON_PP_TEMPLATE, ICON_CSPT, ICON_HELP, ICON_TRASH, ICON_PALETTE, ICON_ASSET,
ICON_GAMEPAD,
getAssetTypeIcon,
@@ -168,9 +168,8 @@ const csRawStreams = new CardSection('raw-streams', { titleKey: 'streams.section
const csRawTemplates = new CardSection('raw-templates', { titleKey: 'templates.title', gridClass: 'templates-grid', addCardOnclick: "showAddTemplateModal()", keyAttr: 'data-template-id', emptyKey: 'section.empty.capture_templates', bulkActions: _captureTemplateDeleteAction });
const csProcStreams = new CardSection('proc-streams', { titleKey: 'streams.section.streams', gridClass: 'templates-grid', addCardOnclick: "showAddStreamModal('processed')", keyAttr: 'data-stream-id', emptyKey: 'section.empty.picture_sources', bulkActions: _streamDeleteAction });
const csProcTemplates = new CardSection('proc-templates', { titleKey: 'postprocessing.title', gridClass: 'templates-grid', addCardOnclick: "showAddPPTemplateModal()", keyAttr: 'data-pp-template-id', emptyKey: 'section.empty.pp_templates', bulkActions: _ppTemplateDeleteAction });
const csAudioMulti = new CardSection('audio-multi', { titleKey: 'audio_source.group.multichannel', gridClass: 'templates-grid', addCardOnclick: "showAudioSourceModal('multichannel')", keyAttr: 'data-id', emptyKey: 'section.empty.audio_sources', bulkActions: _audioSourceDeleteAction });
const csAudioMono = new CardSection('audio-mono', { titleKey: 'audio_source.group.mono', gridClass: 'templates-grid', addCardOnclick: "showAudioSourceModal('mono')", keyAttr: 'data-id', emptyKey: 'section.empty.audio_sources', bulkActions: _audioSourceDeleteAction });
const csAudioBandExtract = new CardSection('audio-band-extract', { titleKey: 'audio_source.group.band_extract', gridClass: 'templates-grid', addCardOnclick: "showAudioSourceModal('band_extract')", keyAttr: 'data-id', emptyKey: 'section.empty.audio_sources', bulkActions: _audioSourceDeleteAction });
const csAudioCapture = new CardSection('audio-capture', { titleKey: 'audio_source.group.capture', gridClass: 'templates-grid', addCardOnclick: "showAudioSourceModal('capture')", keyAttr: 'data-id', emptyKey: 'section.empty.audio_sources', bulkActions: _audioSourceDeleteAction });
const csAudioProcessed = new CardSection('audio-processed', { titleKey: 'audio_source.group.processed', gridClass: 'templates-grid', addCardOnclick: "showAudioSourceModal('processed')", keyAttr: 'data-id', emptyKey: 'section.empty.audio_sources', bulkActions: _audioSourceDeleteAction });
const csStaticStreams = new CardSection('static-streams', { titleKey: 'streams.group.static_image', gridClass: 'templates-grid', addCardOnclick: "showAddStreamModal('static_image')", keyAttr: 'data-stream-id', emptyKey: 'section.empty.picture_sources', bulkActions: _streamDeleteAction });
const csVideoStreams = new CardSection('video-streams', { titleKey: 'streams.group.video', gridClass: 'templates-grid', addCardOnclick: "showAddStreamModal('video')", keyAttr: 'data-stream-id', emptyKey: 'section.empty.picture_sources', bulkActions: _streamDeleteAction });
const csAudioTemplates = new CardSection('audio-templates', { titleKey: 'audio_template.title', gridClass: 'templates-grid', addCardOnclick: "showAddAudioTemplateModal()", keyAttr: 'data-audio-template-id', emptyKey: 'section.empty.audio_templates', bulkActions: _audioTemplateDeleteAction });
@@ -349,9 +348,8 @@ const _streamSectionMap = {
proc_templates: [csProcTemplates],
css_processing: [csCSPTemplates],
color_strip: [csColorStrips],
audio_multi: [csAudioMulti],
audio_mono: [csAudioMono],
audio_band_extract: [csAudioBandExtract],
audio_capture: [csAudioCapture],
audio_processed: [csAudioProcessed],
audio_templates: [csAudioTemplates],
value: [csValueSources],
sync: [csSyncClocks],
@@ -552,9 +550,8 @@ function renderPictureSourcesList(streams: any) {
const staticImageStreams = streams.filter(s => s.stream_type === 'static_image');
const videoStreams = streams.filter(s => s.stream_type === 'video');
const multichannelSources = _cachedAudioSources.filter(s => s.source_type === 'multichannel');
const monoSources = _cachedAudioSources.filter(s => s.source_type === 'mono');
const bandExtractSources = _cachedAudioSources.filter(s => s.source_type === 'band_extract');
const captureSources = _cachedAudioSources.filter(s => s.source_type === 'capture');
const processedAudioSources = _cachedAudioSources.filter(s => s.source_type === 'processed');
// CSPT templates
const csptTemplates = csptCache.data;
@@ -578,9 +575,8 @@ function renderPictureSourcesList(streams: any) {
{ key: 'css_processing', icon: ICON_CSPT, titleKey: 'streams.group.css_processing', count: csptTemplates.length },
{ key: 'color_strip', icon: getColorStripIcon('static'), titleKey: 'streams.group.color_strip', count: colorStrips.length },
{ key: 'gradients', icon: ICON_PALETTE, titleKey: 'streams.group.gradients', count: gradients.length },
{ key: 'audio_multi', icon: getAudioSourceIcon('multichannel'), titleKey: 'audio_source.group.multichannel', count: multichannelSources.length },
{ key: 'audio_mono', icon: getAudioSourceIcon('mono'), titleKey: 'audio_source.group.mono', count: monoSources.length },
{ key: 'audio_band_extract', icon: getAudioSourceIcon('band_extract'), titleKey: 'audio_source.group.band_extract', count: bandExtractSources.length },
{ key: 'audio_capture', icon: getAudioSourceIcon('capture'), titleKey: 'audio_source.group.capture', count: captureSources.length },
{ key: 'audio_processed', icon: getAudioSourceIcon('processed'), titleKey: 'audio_source.group.processed', count: processedAudioSources.length },
{ key: 'audio_templates', icon: ICON_AUDIO_TEMPLATE, titleKey: 'streams.group.audio_templates', count: _cachedAudioTemplates.length },
{ key: 'value', icon: ICON_VALUE_SOURCE, titleKey: 'streams.group.value', count: _cachedValueSources.length },
{ key: 'sync', icon: ICON_CLOCK, titleKey: 'streams.group.sync', count: _cachedSyncClocks.length },
@@ -628,11 +624,10 @@ function renderPictureSourcesList(streams: any) {
]
},
{
key: 'audio_group', icon: getAudioSourceIcon('multichannel'), titleKey: 'tree.group.audio',
key: 'audio_group', icon: getAudioSourceIcon('capture'), titleKey: 'tree.group.audio',
children: [
{ key: 'audio_multi', titleKey: 'audio_source.group.multichannel', icon: getAudioSourceIcon('multichannel'), count: multichannelSources.length },
{ key: 'audio_mono', titleKey: 'audio_source.group.mono', icon: getAudioSourceIcon('mono'), count: monoSources.length },
{ key: 'audio_band_extract', titleKey: 'audio_source.group.band_extract', icon: getAudioSourceIcon('band_extract'), count: bandExtractSources.length },
{ key: 'audio_capture', titleKey: 'audio_source.group.capture', icon: getAudioSourceIcon('capture'), count: captureSources.length },
{ key: 'audio_processed', titleKey: 'audio_source.group.processed', icon: getAudioSourceIcon('processed'), count: processedAudioSources.length },
{ key: 'audio_templates', titleKey: 'tree.leaf.templates', icon: ICON_AUDIO_TEMPLATE, count: _cachedAudioTemplates.length },
]
},
@@ -655,51 +650,34 @@ function renderPictureSourcesList(streams: any) {
}
];
const _bandLabels: Record<string, string> = { bass: 'Bass', mid: 'Mid', treble: 'Treble', custom: 'Custom' };
const _getSectionForSource = (sourceType: string): string => {
if (sourceType === 'multichannel') return 'audio-multi';
if (sourceType === 'mono') return 'audio-mono';
return 'audio-band-extract';
if (sourceType === 'capture') return 'audio-capture';
return 'audio-processed';
};
const _getTabForSource = (sourceType: string): string => {
if (sourceType === 'multichannel') return 'audio_multi';
if (sourceType === 'mono') return 'audio_mono';
return 'audio_band_extract';
if (sourceType === 'capture') return 'audio_capture';
return 'audio_processed';
};
const renderAudioSourceCard = (src: any) => {
const icon = getAudioSourceIcon(src.source_type);
let propsHtml = '';
if (src.source_type === 'mono') {
if (src.source_type === 'processed') {
const parent = _cachedAudioSources.find(s => s.id === src.audio_source_id);
const parentName = parent ? parent.name : src.audio_source_id;
const chLabel = src.channel === 'left' ? 'L' : src.channel === 'right' ? 'R' : 'M';
const parentSection = parent ? _getSectionForSource(parent.source_type) : 'audio-capture';
const parentTab = parent ? _getTabForSource(parent.source_type) : 'audio_capture';
const parentBadge = parent
? `<span class="stream-card-prop stream-card-link" title="${escapeHtml(t('audio_source.parent'))}" onclick="event.stopPropagation(); navigateToCard('streams','audio_multi','audio-multi','data-id','${src.audio_source_id}')">${ICON_AUDIO_LOOPBACK} ${escapeHtml(parentName)}</span>`
? `<span class="stream-card-prop stream-card-link" title="${escapeHtml(t('audio_source.parent'))}" onclick="event.stopPropagation(); navigateToCard('streams','${parentTab}','${parentSection}','data-id','${src.audio_source_id}')">${getAudioSourceIcon(parent.source_type)} ${escapeHtml(parentName)}</span>`
: `<span class="stream-card-prop" title="${escapeHtml(t('audio_source.parent'))}">${ICON_AUDIO_LOOPBACK} ${escapeHtml(parentName)}</span>`;
propsHtml = `
${parentBadge}
<span class="stream-card-prop" title="${escapeHtml(t('audio_source.channel'))}">${ICON_RADIO} ${chLabel}</span>
`;
} else if (src.source_type === 'band_extract') {
const parent = _cachedAudioSources.find(s => s.id === src.audio_source_id);
const parentName = parent ? parent.name : src.audio_source_id;
const parentSection = parent ? _getSectionForSource(parent.source_type) : 'audio-multi';
const parentTab = parent ? _getTabForSource(parent.source_type) : 'audio_multi';
const parentBadge = parent
? `<span class="stream-card-prop stream-card-link" title="${escapeHtml(t('audio_source.band_parent'))}" onclick="event.stopPropagation(); navigateToCard('streams','${parentTab}','${parentSection}','data-id','${src.audio_source_id}')">${getAudioSourceIcon(parent.source_type)} ${escapeHtml(parentName)}</span>`
: `<span class="stream-card-prop" title="${escapeHtml(t('audio_source.band_parent'))}">${ICON_ACTIVITY} ${escapeHtml(parentName)}</span>`;
const bandLabel = _bandLabels[src.band] || src.band;
const freqRange = `${Math.round(src.freq_low)}${Math.round(src.freq_high)} Hz`;
propsHtml = `
${parentBadge}
<span class="stream-card-prop" title="${escapeHtml(t('audio_source.band'))}">${ICON_ACTIVITY} ${bandLabel}</span>
<span class="stream-card-prop" title="${escapeHtml(t('audio_source.freq_range'))}">${freqRange}</span>
`;
propsHtml = `${parentBadge}`;
if (src.audio_processing_template_id) {
propsHtml += `<span class="stream-card-prop">${ICON_AUDIO_TEMPLATE} ${escapeHtml(src.audio_processing_template_id)}</span>`;
}
} else {
// Capture source
const devIdx = src.device_index ?? -1;
const loopback = src.is_loopback !== false;
const devLabel = loopback ? `${ICON_AUDIO_LOOPBACK} Loopback` : `${ICON_AUDIO_INPUT} Input`;
@@ -800,9 +778,8 @@ function renderPictureSourcesList(streams: any) {
const rawTemplateItems = csRawTemplates.applySortOrder(_cachedCaptureTemplates.map(t => ({ key: t.id, html: renderCaptureTemplateCard(t) })));
const procStreamItems = csProcStreams.applySortOrder(processedStreams.map(s => ({ key: s.id, html: renderStreamCard(s) })));
const procTemplateItems = csProcTemplates.applySortOrder(_cachedPPTemplates.map(t => ({ key: t.id, html: renderPPTemplateCard(t) })));
const multiItems = csAudioMulti.applySortOrder(multichannelSources.map(s => ({ key: s.id, html: renderAudioSourceCard(s) })));
const monoItems = csAudioMono.applySortOrder(monoSources.map(s => ({ key: s.id, html: renderAudioSourceCard(s) })));
const bandExtractItems = csAudioBandExtract.applySortOrder(bandExtractSources.map(s => ({ key: s.id, html: renderAudioSourceCard(s) })));
const captureItems = csAudioCapture.applySortOrder(captureSources.map(s => ({ key: s.id, html: renderAudioSourceCard(s) })));
const processedAudioItems = csAudioProcessed.applySortOrder(processedAudioSources.map(s => ({ key: s.id, html: renderAudioSourceCard(s) })));
const audioTemplateItems = csAudioTemplates.applySortOrder(_cachedAudioTemplates.map(t => ({ key: t.id, html: renderAudioTemplateCard(t) })));
const staticItems = csStaticStreams.applySortOrder(staticImageStreams.map(s => ({ key: s.id, html: renderStreamCard(s) })));
const videoItems = csVideoStreams.applySortOrder(videoStreams.map(s => ({ key: s.id, html: renderStreamCard(s) })));
@@ -846,9 +823,8 @@ function renderPictureSourcesList(streams: any) {
csCSPTemplates.reconcile(csptItems);
csColorStrips.reconcile(colorStripItems);
csGradients.reconcile(gradientItems);
csAudioMulti.reconcile(multiItems);
csAudioMono.reconcile(monoItems);
csAudioBandExtract.reconcile(bandExtractItems);
csAudioCapture.reconcile(captureItems);
csAudioProcessed.reconcile(processedAudioItems);
csAudioTemplates.reconcile(audioTemplateItems);
csStaticStreams.reconcile(staticItems);
csVideoStreams.reconcile(videoItems);
@@ -870,9 +846,8 @@ function renderPictureSourcesList(streams: any) {
else if (tab.key === 'css_processing') panelContent = csCSPTemplates.render(csptItems);
else if (tab.key === 'color_strip') panelContent = csColorStrips.render(colorStripItems);
else if (tab.key === 'gradients') panelContent = csGradients.render(gradientItems);
else if (tab.key === 'audio_multi') panelContent = csAudioMulti.render(multiItems);
else if (tab.key === 'audio_mono') panelContent = csAudioMono.render(monoItems);
else if (tab.key === 'audio_band_extract') panelContent = csAudioBandExtract.render(bandExtractItems);
else if (tab.key === 'audio_capture') panelContent = csAudioCapture.render(captureItems);
else if (tab.key === 'audio_processed') panelContent = csAudioProcessed.render(processedAudioItems);
else if (tab.key === 'audio_templates') panelContent = csAudioTemplates.render(audioTemplateItems);
else if (tab.key === 'value') panelContent = csValueSources.render(valueItems);
else if (tab.key === 'sync') panelContent = csSyncClocks.render(syncClockItems);
@@ -887,7 +862,7 @@ function renderPictureSourcesList(streams: any) {
}).join('');
container.innerHTML = panels;
CardSection.bindAll([csRawStreams, csRawTemplates, csProcStreams, csProcTemplates, csCSPTemplates, csColorStrips, csGradients, csAudioMulti, csAudioMono, csAudioBandExtract, csAudioTemplates, csStaticStreams, csVideoStreams, csValueSources, csSyncClocks, csWeatherSources, csHASources, csMQTTSources, csAssets, csGameIntegrations]);
CardSection.bindAll([csRawStreams, csRawTemplates, csProcStreams, csProcTemplates, csCSPTemplates, csColorStrips, csGradients, csAudioCapture, csAudioProcessed, csAudioTemplates, csStaticStreams, csVideoStreams, csValueSources, csSyncClocks, csWeatherSources, csHASources, csMQTTSources, csAssets, csGameIntegrations]);
// Event delegation for card actions (replaces inline onclick handlers)
initSyncClockDelegation(container);
@@ -908,7 +883,7 @@ function renderPictureSourcesList(streams: any) {
'css-proc-templates': 'css_processing',
'color-strips': 'color_strip',
'gradients': 'gradients',
'audio-multi': 'audio_multi', 'audio-mono': 'audio_mono', 'audio-band-extract': 'audio_band_extract',
'audio-capture': 'audio_capture', 'audio-processed': 'audio_processed',
'audio-templates': 'audio_templates',
'value-sources': 'value',
'sync-clocks': 'sync',
@@ -1245,8 +1245,8 @@ export function createValueSourceCard(src: ValueSource) {
} else if (src.source_type === 'audio') {
const audioSrc = _cachedAudioSources.find(a => a.id === src.audio_source_id);
const audioName = audioSrc ? audioSrc.name : (src.audio_source_id || '-');
const audioSection = audioSrc ? (audioSrc.source_type === 'mono' ? 'audio-mono' : audioSrc.source_type === 'band_extract' ? 'audio-band-extract' : 'audio-multi') : 'audio-multi';
const audioTab = audioSrc ? (audioSrc.source_type === 'mono' ? 'audio_mono' : audioSrc.source_type === 'band_extract' ? 'audio_band_extract' : 'audio_multi') : 'audio_multi';
const audioSection = audioSrc ? (audioSrc.source_type === 'processed' ? 'audio-processed' : 'audio-capture') : 'audio-capture';
const audioTab = audioSrc ? (audioSrc.source_type === 'processed' ? 'audio_processed' : 'audio_capture') : 'audio_capture';
const modeLabel = src.mode || 'rms';
const audioBadge = audioSrc
? `<span class="stream-card-prop stream-card-link" title="${escapeHtml(t('value_source.audio_source'))}" onclick="event.stopPropagation(); navigateToCard('streams','${audioTab}','${audioSection}','data-id','${src.audio_source_id}')">${ICON_MUSIC} ${escapeHtml(audioName)}</span>`
@@ -1384,7 +1384,7 @@ function _populateAudioSourceDropdown(selectedId: any) {
const select = document.getElementById('value-source-audio-source') as HTMLSelectElement;
if (!select) return;
select.innerHTML = _cachedAudioSources.map((s: any) => {
const badge = s.source_type === 'multichannel' ? ' [multichannel]' : s.source_type === 'band_extract' ? ' [band]' : ' [mono]';
const badge = s.source_type === 'capture' ? ' [capture]' : ' [processed]';
return `<option value="${s.id}"${s.id === selectedId ? ' selected' : ''}>${escapeHtml(s.name)}${badge}</option>`;
}).join('');
+8 -17
View File
@@ -486,7 +486,7 @@ export type ValueSource =
// ── Audio Source ───────────────────────────────────────────────
export type AudioSourceType = 'multichannel' | 'mono' | 'band_extract';
export type AudioSourceType = 'capture' | 'processed';
interface AudioSourceBase {
id: string;
@@ -498,31 +498,22 @@ interface AudioSourceBase {
updated_at: string;
}
export interface MultichannelAudioSource extends AudioSourceBase {
source_type: 'multichannel';
export interface CaptureAudioSource extends AudioSourceBase {
source_type: 'capture';
device_index: number;
is_loopback: boolean;
audio_template_id?: string;
}
export interface MonoAudioSource extends AudioSourceBase {
source_type: 'mono';
export interface ProcessedAudioSource extends AudioSourceBase {
source_type: 'processed';
audio_source_id: string;
channel: string;
}
export interface BandExtractAudioSource extends AudioSourceBase {
source_type: 'band_extract';
audio_source_id: string;
band: string;
freq_low: number;
freq_high: number;
audio_processing_template_id: string;
}
export type AudioSource =
| MultichannelAudioSource
| MonoAudioSource
| BandExtractAudioSource;
| CaptureAudioSource
| ProcessedAudioSource;
// ── Picture Source ─────────────────────────────────────────────
@@ -1,21 +1,13 @@
"""Audio source data model with inheritance-based source types.
An AudioSource represents a reusable audio input configuration:
MultichannelAudioSource wraps a physical audio device (index + loopback flag)
MonoAudioSource extracts a single channel from a multichannel source
BandExtractAudioSource filters a parent source to a frequency band (bass/mid/treble/custom)
CaptureAudioSource wraps a physical audio device (index + loopback flag)
ProcessedAudioSource applies audio processing filters to another audio source
"""
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Dict, List, Optional, Tuple
# Frequency band presets: band name → (freq_low_hz, freq_high_hz)
BAND_PRESETS: Dict[str, Tuple[float, float]] = {
"bass": (20.0, 250.0),
"mid": (250.0, 4000.0),
"treble": (4000.0, 20000.0),
}
from typing import Dict, List, Optional, Type
@dataclass
@@ -24,7 +16,7 @@ class AudioSource:
id: str
name: str
source_type: str # "multichannel" | "mono" | "band_extract"
source_type: str # "capture" | "processed"
created_at: datetime
updated_at: datetime
description: Optional[str] = None
@@ -45,80 +37,54 @@ class AudioSource:
"is_loopback": None,
"audio_template_id": None,
"audio_source_id": None,
"channel": None,
"band": None,
"freq_low": None,
"freq_high": None,
"audio_processing_template_id": None,
}
@staticmethod
def from_dict(data: dict) -> "AudioSource":
"""Factory: dispatch to the correct subclass based on source_type."""
source_type: str = data.get("source_type", "multichannel") or "multichannel"
sid: str = data["id"]
name: str = data["name"]
description: str | None = data.get("description")
tags: list = data.get("tags", [])
source_type = data.get("source_type", "capture") or "capture"
subcls = _AUDIO_SOURCE_MAP.get(source_type)
if subcls is None:
# Fall back to capture for unknown types
subcls = CaptureAudioSource
return subcls.from_dict(data)
raw_created = data.get("created_at")
created_at: datetime = (
datetime.fromisoformat(raw_created)
if isinstance(raw_created, str)
else raw_created if isinstance(raw_created, datetime)
else datetime.now(timezone.utc)
)
raw_updated = data.get("updated_at")
updated_at: datetime = (
datetime.fromisoformat(raw_updated)
if isinstance(raw_updated, str)
else raw_updated if isinstance(raw_updated, datetime)
else datetime.now(timezone.utc)
)
if source_type == "mono":
return MonoAudioSource(
id=sid, name=name, source_type="mono",
created_at=created_at, updated_at=updated_at, description=description, tags=tags,
audio_source_id=data.get("audio_source_id") or "",
channel=data.get("channel") or "mono",
)
if source_type == "band_extract":
band = data.get("band") or "bass"
if band in BAND_PRESETS:
freq_low, freq_high = BAND_PRESETS[band]
else:
freq_low = float(data.get("freq_low") or 20.0)
freq_high = float(data.get("freq_high") or 20000.0)
return BandExtractAudioSource(
id=sid, name=name, source_type="band_extract",
created_at=created_at, updated_at=updated_at, description=description, tags=tags,
audio_source_id=data.get("audio_source_id") or "",
band=band,
freq_low=freq_low,
freq_high=freq_high,
)
# Default: multichannel
return MultichannelAudioSource(
id=sid, name=name, source_type="multichannel",
created_at=created_at, updated_at=updated_at, description=description, tags=tags,
device_index=int(data.get("device_index", -1)),
is_loopback=bool(data.get("is_loopback", True)),
audio_template_id=data.get("audio_template_id"),
)
def _parse_common_fields(data: dict) -> dict:
"""Extract common fields shared by all audio source types."""
raw_created = data.get("created_at")
created_at = (
datetime.fromisoformat(raw_created)
if isinstance(raw_created, str)
else raw_created if isinstance(raw_created, datetime) else datetime.now(timezone.utc)
)
raw_updated = data.get("updated_at")
updated_at = (
datetime.fromisoformat(raw_updated)
if isinstance(raw_updated, str)
else raw_updated if isinstance(raw_updated, datetime) else datetime.now(timezone.utc)
)
return dict(
id=data["id"],
name=data["name"],
description=data.get("description"),
tags=data.get("tags", []),
created_at=created_at,
updated_at=updated_at,
)
@dataclass
class MultichannelAudioSource(AudioSource):
class CaptureAudioSource(AudioSource):
"""Audio source wrapping a physical audio device.
Captures all channels from the device. For WASAPI loopback devices
(system audio output), set is_loopback=True.
"""
device_index: int = -1 # -1 = default device
is_loopback: bool = True # True = WASAPI loopback (system audio)
device_index: int = -1 # -1 = default device
is_loopback: bool = True # True = WASAPI loopback (system audio)
audio_template_id: Optional[str] = None # references AudioCaptureTemplate
def to_dict(self) -> dict:
@@ -128,43 +94,48 @@ class MultichannelAudioSource(AudioSource):
d["audio_template_id"] = self.audio_template_id
return d
@classmethod
def from_dict(cls, data: dict) -> "CaptureAudioSource":
common = _parse_common_fields(data)
return cls(
**common,
source_type="capture",
device_index=int(data.get("device_index", -1)),
is_loopback=bool(data.get("is_loopback", True)),
audio_template_id=data.get("audio_template_id"),
)
@dataclass
class MonoAudioSource(AudioSource):
"""Audio source that extracts a single channel from a multichannel source.
class ProcessedAudioSource(AudioSource):
"""Audio source that applies processing filters to another audio source.
References a MultichannelAudioSource and selects which channel to use:
mono (L+R mix), left, or right.
References an existing audio source (capture or processed) and an
AudioProcessingTemplate containing the filter chain to apply.
"""
audio_source_id: str = "" # references a MultichannelAudioSource
channel: str = "mono" # mono | left | right
audio_source_id: str = "" # references any AudioSource
audio_processing_template_id: str = "" # references AudioProcessingTemplate
def to_dict(self) -> dict:
d = super().to_dict()
d["audio_source_id"] = self.audio_source_id
d["channel"] = self.channel
d["audio_processing_template_id"] = self.audio_processing_template_id
return d
@classmethod
def from_dict(cls, data: dict) -> "ProcessedAudioSource":
common = _parse_common_fields(data)
return cls(
**common,
source_type="processed",
audio_source_id=data.get("audio_source_id") or "",
audio_processing_template_id=data.get("audio_processing_template_id") or "",
)
@dataclass
class BandExtractAudioSource(AudioSource):
"""Audio source that filters a parent source to a specific frequency band.
References any AudioSource and extracts only the specified frequency range.
Preset bands: bass (20-250 Hz), mid (250-4000 Hz), treble (4000-20000 Hz).
Custom band allows user-specified freq_low/freq_high.
"""
audio_source_id: str = "" # references any AudioSource
band: str = "bass" # bass | mid | treble | custom
freq_low: float = 20.0 # lower frequency bound (Hz)
freq_high: float = 250.0 # upper frequency bound (Hz)
def to_dict(self) -> dict:
d = super().to_dict()
d["audio_source_id"] = self.audio_source_id
d["band"] = self.band
d["freq_low"] = self.freq_low
d["freq_high"] = self.freq_high
return d
# -- Source type registry --
_AUDIO_SOURCE_MAP: Dict[str, Type[AudioSource]] = {
"capture": CaptureAudioSource,
"processed": ProcessedAudioSource,
}
@@ -5,11 +5,9 @@ from datetime import datetime, timezone
from typing import List, NamedTuple, Optional, Set
from wled_controller.storage.audio_source import (
BAND_PRESETS,
AudioSource,
BandExtractAudioSource,
MonoAudioSource,
MultichannelAudioSource,
CaptureAudioSource,
ProcessedAudioSource,
)
from wled_controller.storage.base_sqlite_store import BaseSqliteStore
from wled_controller.storage.database import Database
@@ -20,14 +18,16 @@ logger = get_logger(__name__)
class ResolvedAudioSource(NamedTuple):
"""Result of resolving an audio source to its physical device + band info."""
"""Result of resolving an audio source chain to its physical device + filter templates.
Walking the chain from a ProcessedAudioSource to the terminal CaptureAudioSource
collects audio_processing_template_ids in chain order (outermost first).
"""
device_index: int
is_loopback: bool
channel: str
audio_template_id: Optional[str]
freq_low: Optional[float] = None # None = full range (no band filtering)
freq_high: Optional[float] = None
audio_processing_template_ids: List[str] # ordered list of template IDs along the chain
class AudioSourceStore(BaseSqliteStore[AudioSource]):
@@ -43,10 +43,6 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
get_all_sources = BaseSqliteStore.get_all
get_source = BaseSqliteStore.get
def get_mono_sources(self) -> List[MonoAudioSource]:
"""Return only mono audio sources (for CSS dropdown)."""
return [s for s in self._items.values() if isinstance(s, MonoAudioSource)]
def create_source(
self,
name: str,
@@ -54,58 +50,51 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
device_index: Optional[int] = None,
is_loopback: Optional[bool] = None,
audio_source_id: Optional[str] = None,
channel: Optional[str] = None,
description: Optional[str] = None,
audio_template_id: Optional[str] = None,
tags: Optional[List[str]] = None,
band: Optional[str] = None,
freq_low: Optional[float] = None,
freq_high: Optional[float] = None,
audio_processing_template_id: Optional[str] = None,
) -> AudioSource:
self._check_name_unique(name)
if source_type not in ("multichannel", "mono", "band_extract"):
if source_type not in ("capture", "processed"):
raise ValueError(f"Invalid source type: {source_type}")
sid = f"as_{uuid.uuid4().hex[:8]}"
now = datetime.now(timezone.utc)
if source_type == "mono":
if source_type == "processed":
if not audio_source_id:
raise ValueError("Mono sources require audio_source_id")
# Validate parent exists and is multichannel
raise ValueError("Processed sources require audio_source_id")
if not audio_processing_template_id:
raise ValueError("Processed sources require audio_processing_template_id")
# Validate parent source exists
parent = self._items.get(audio_source_id)
if not parent:
raise ValueError(f"Parent audio source not found: {audio_source_id}")
if not isinstance(parent, MultichannelAudioSource):
raise ValueError("Mono sources must reference a multichannel source")
# Check for cycles (new source doesn't exist yet, just validate parent chain)
self._check_no_cycle_for_new(audio_source_id)
source: AudioSource = MonoAudioSource(
id=sid, name=name, source_type="mono",
created_at=now, updated_at=now, description=description, tags=tags or [],
source: AudioSource = ProcessedAudioSource(
id=sid,
name=name,
source_type="processed",
created_at=now,
updated_at=now,
description=description,
tags=tags or [],
audio_source_id=audio_source_id,
channel=channel or "mono",
)
elif source_type == "band_extract":
if not audio_source_id:
raise ValueError("Band extract sources require audio_source_id")
parent = self._items.get(audio_source_id)
if not parent:
raise ValueError(f"Parent audio source not found: {audio_source_id}")
band_val = band or "bass"
fl, fh = _resolve_band_freqs(band_val, freq_low, freq_high)
source = BandExtractAudioSource(
id=sid, name=name, source_type="band_extract",
created_at=now, updated_at=now, description=description, tags=tags or [],
audio_source_id=audio_source_id,
band=band_val, freq_low=fl, freq_high=fh,
audio_processing_template_id=audio_processing_template_id,
)
else:
source = MultichannelAudioSource(
id=sid, name=name, source_type="multichannel",
created_at=now, updated_at=now, description=description, tags=tags or [],
source = CaptureAudioSource(
id=sid,
name=name,
source_type="capture",
created_at=now,
updated_at=now,
description=description,
tags=tags or [],
device_index=device_index if device_index is not None else -1,
is_loopback=bool(is_loopback) if is_loopback is not None else True,
audio_template_id=audio_template_id,
@@ -124,13 +113,10 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
device_index: Optional[int] = None,
is_loopback: Optional[bool] = None,
audio_source_id: Optional[str] = None,
channel: Optional[str] = None,
description: Optional[str] = None,
audio_template_id: Optional[str] = None,
tags: Optional[List[str]] = None,
band: Optional[str] = None,
freq_low: Optional[float] = None,
freq_high: Optional[float] = None,
audio_processing_template_id: Optional[str] = None,
) -> AudioSource:
source = self.get(source_id)
@@ -143,27 +129,14 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
if tags is not None:
source.tags = tags
if isinstance(source, MultichannelAudioSource):
if isinstance(source, CaptureAudioSource):
if device_index is not None:
source.device_index = device_index
if is_loopback is not None:
source.is_loopback = bool(is_loopback)
if audio_template_id is not None:
source.audio_template_id = resolve_ref(audio_template_id, source.audio_template_id)
elif isinstance(source, MonoAudioSource):
if audio_source_id is not None:
resolved = resolve_ref(audio_source_id, source.audio_source_id)
if resolved is not None:
# Validate parent exists and is multichannel
parent = self._items.get(resolved)
if not parent:
raise ValueError(f"Parent audio source not found: {resolved}")
if not isinstance(parent, MultichannelAudioSource):
raise ValueError("Mono sources must reference a multichannel source")
source.audio_source_id = resolved
if channel is not None:
source.channel = channel
elif isinstance(source, BandExtractAudioSource):
elif isinstance(source, ProcessedAudioSource):
if audio_source_id is not None:
resolved = resolve_ref(audio_source_id, source.audio_source_id)
if resolved is not None:
@@ -173,17 +146,10 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
# Check for cycles
self._check_no_cycle(source_id, resolved)
source.audio_source_id = resolved
if band is not None:
fl, fh = _resolve_band_freqs(band, freq_low, freq_high)
source.band = band
source.freq_low = fl
source.freq_high = fh
elif freq_low is not None or freq_high is not None:
# Update custom freq range without changing band preset
if freq_low is not None:
source.freq_low = freq_low
if freq_high is not None:
source.freq_high = freq_high
if audio_processing_template_id is not None:
source.audio_processing_template_id = resolve_ref(
audio_processing_template_id, source.audio_processing_template_id
)
source.updated_at = datetime.now(timezone.utc)
self._save_item(source_id, source)
@@ -197,14 +163,13 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
source = self._items[source_id]
# Prevent deleting sources referenced by children (mono or band_extract)
# Prevent deleting sources referenced by ProcessedAudioSource children
for other in self._items.values():
if other.id == source_id:
continue
parent_ref = getattr(other, "audio_source_id", None)
if parent_ref == source_id:
if isinstance(other, ProcessedAudioSource) and other.audio_source_id == source_id:
raise ValueError(
f"Cannot delete '{source.name}': referenced by {other.source_type} source '{other.name}'"
f"Cannot delete '{source.name}': referenced by processed source '{other.name}'"
)
del self._items[source_id]
@@ -215,10 +180,10 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
# ── Resolution ───────────────────────────────────────────────────
def resolve_audio_source(self, source_id: str) -> ResolvedAudioSource:
"""Resolve any audio source to its physical device, channel, and band info.
"""Resolve any audio source to its physical device + ordered processing template IDs.
Follows the reference chain: band_extract mono/multichannel device.
For band_extract sources, intersects frequency ranges when chained.
Walks the chain: ProcessedAudioSource ... CaptureAudioSource.
Collects audio_processing_template_ids in chain order (outermost first).
Raises:
ValueError: If source not found, chain is broken, or cycle detected
@@ -232,43 +197,30 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
source = self.get_source(source_id)
if isinstance(source, MultichannelAudioSource):
if isinstance(source, CaptureAudioSource):
return ResolvedAudioSource(
source.device_index, source.is_loopback, "mono", source.audio_template_id,
device_index=source.device_index,
is_loopback=source.is_loopback,
audio_template_id=source.audio_template_id,
audio_processing_template_ids=[],
)
if isinstance(source, MonoAudioSource):
parent = self.get_source(source.audio_source_id)
if not isinstance(parent, MultichannelAudioSource):
raise ValueError(
f"Mono source {source_id} references non-multichannel source {source.audio_source_id}"
)
return ResolvedAudioSource(
parent.device_index, parent.is_loopback, source.channel, parent.audio_template_id,
)
if isinstance(source, BandExtractAudioSource):
if isinstance(source, ProcessedAudioSource):
if not source.audio_source_id:
raise ValueError(f"Processed source {source_id} has no audio_source_id")
parent_resolved = self._resolve(source.audio_source_id, visited)
# Intersect frequency ranges if parent also has band filtering
fl = source.freq_low
fh = source.freq_high
if parent_resolved.freq_low is not None:
fl = max(fl, parent_resolved.freq_low)
if parent_resolved.freq_high is not None:
fh = min(fh, parent_resolved.freq_high)
if fl >= fh:
raise ValueError(
f"Band extract '{source.name}' has empty frequency intersection: {fl}-{fh} Hz"
)
# Prepend this source's template to the list (outermost first)
template_ids = [source.audio_processing_template_id] + list(
parent_resolved.audio_processing_template_ids
)
return ResolvedAudioSource(
parent_resolved.device_index,
parent_resolved.is_loopback,
parent_resolved.channel,
parent_resolved.audio_template_id,
fl, fh,
device_index=parent_resolved.device_index,
is_loopback=parent_resolved.is_loopback,
audio_template_id=parent_resolved.audio_template_id,
audio_processing_template_ids=template_ids,
)
raise ValueError(f"Audio source {source_id} is not a valid audio source")
raise ValueError(f"Audio source {source_id} has unknown type: {source.source_type}")
def _check_no_cycle(self, source_id: str, new_parent_id: str) -> None:
"""Ensure setting new_parent_id as parent of source_id won't create a cycle."""
@@ -283,19 +235,25 @@ class AudioSourceStore(BaseSqliteStore[AudioSource]):
break
current = getattr(item, "audio_source_id", None)
def _check_no_cycle_for_new(self, parent_id: str) -> None:
"""Validate that parent chain doesn't already contain a cycle (for new source creation)."""
visited: Set[str] = set()
current = parent_id
while current:
if current in visited:
raise ValueError("Cycle detected in parent audio source chain")
visited.add(current)
item = self._items.get(current)
if item is None:
break
current = getattr(item, "audio_source_id", None)
def _resolve_band_freqs(
band: str,
freq_low: Optional[float],
freq_high: Optional[float],
) -> tuple[float, float]:
"""Resolve band preset or custom range to (freq_low, freq_high)."""
if band in BAND_PRESETS:
return BAND_PRESETS[band]
if band != "custom":
raise ValueError(f"Invalid band: {band}. Must be one of: bass, mid, treble, custom")
fl = float(freq_low) if freq_low is not None else 20.0
fh = float(freq_high) if freq_high is not None else 20000.0
if not (20.0 <= fl < fh <= 20000.0):
raise ValueError(f"Invalid frequency range: {fl}-{fh} Hz (must be 20-20000, low < high)")
return fl, fh
# ── Reference query helpers ──────────────────────────────────────
def get_sources_referencing_template(self, template_id: str) -> List[ProcessedAudioSource]:
"""Return all ProcessedAudioSources that reference a given audio processing template."""
return [
s
for s in self._items.values()
if isinstance(s, ProcessedAudioSource) and s.audio_processing_template_id == template_id
]
@@ -139,12 +139,13 @@ class AudioTemplateStore(BaseSqliteStore[AudioCaptureTemplate]):
if template_id not in self._items:
raise ValueError(f"{self._entity_name} not found: {template_id}")
# Check if any multichannel audio source references this template
# Check if any capture audio source references this template
if audio_source_store is not None:
from wled_controller.storage.audio_source import MultichannelAudioSource
from wled_controller.storage.audio_source import CaptureAudioSource
for source in audio_source_store.get_all_sources():
if (
isinstance(source, MultichannelAudioSource)
isinstance(source, CaptureAudioSource)
and getattr(source, "audio_template_id", None) == template_id
):
raise ValueError(
@@ -713,7 +713,7 @@ class AudioColorStripSource(ColorStripSource):
"""
visualization_mode: str = "spectrum" # spectrum | beat_pulse | vu_meter
audio_source_id: str = "" # references a MonoAudioSource
audio_source_id: str = "" # references an AudioSource (capture or processed)
sensitivity: BindableFloat = field(default_factory=lambda: BindableFloat(1.0))
smoothing: BindableFloat = field(default_factory=lambda: BindableFloat(0.3))
palette: str = "rainbow" # legacy palette name (kept for migration)
@@ -163,7 +163,7 @@ class AudioValueSource(ValueSource):
into a scalar value for brightness modulation.
"""
audio_source_id: str = "" # references an audio source (mono or multichannel)
audio_source_id: str = "" # references an AudioSource (capture or processed)
mode: str = "rms" # rms | peak | beat
sensitivity: float = 1.0 # gain multiplier (0.120.0)
smoothing: float = 0.3 # temporal smoothing (0.01.0)