Compare commits
13 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 alexei.dolgolyov
|
b43f821046 | ||
|
alexei.dolgolyov
|
077c99c7d1 | ||
|
alexei.dolgolyov
|
ae74cca132 | ||
|
alexei.dolgolyov
|
77284e8e7b | ||
|
alexei.dolgolyov
|
ff1ff06cb5 | ||
|
alexei.dolgolyov
|
3dd1ac3f0d | ||
|
alexei.dolgolyov
|
6e1dd2111d | ||
|
alexei.dolgolyov
|
9a0137fa4c | ||
|
alexei.dolgolyov
|
4a0927521a | ||
|
alexei.dolgolyov
|
25c613c5cb | ||
|
alexei.dolgolyov
|
726f39e2ba | ||
|
alexei.dolgolyov
|
1ac4a0f66d | ||
|
alexei.dolgolyov
|
1afe7d6fcc |
@@ -19,6 +19,7 @@ semantic = true
|
||||
# Automatically run `vex update` before search if the index is stale
|
||||
auto_update = true
|
||||
|
||||
# Embedder used for semantic indexing. Known IDs: minilm-l6-v2 (default).
|
||||
# Embedder used for semantic indexing. IDs: minilm-l6-v2 (default, CPU-fast),
|
||||
# jina-code (code-specialized, GPU-worthy), bge-base-en-v1.5, bge-large-en-v1.5.
|
||||
# Changing the embedder requires a full reindex.
|
||||
# embedder = "minilm-l6-v2"
|
||||
embedder = "jina-code"
|
||||
|
||||
@@ -82,6 +82,19 @@ LedGrab speaks many protocols, so a single setup can drive everything from a DIY
|
||||
- Real-time FPS, latency, and uptime charts
|
||||
- Localized in English, Russian, and Chinese
|
||||
|
||||
### Activity Log
|
||||
|
||||
The **Activity** tab is a persistent, queryable audit log of everything LedGrab has done — entity changes, auth events, device connections, and system actions.
|
||||
|
||||
- Filter by category (auth, device, entity, capture, system), severity, actor, entity type, date range, or free text
|
||||
- Live-append of new events as they happen
|
||||
- Export as CSV or JSON (authentication required)
|
||||
- Entity crosslinks navigate directly to the relevant card
|
||||
- **Retention settings** (Settings → Activity Log): configure max age, max entry count, and toggle recording on/off
|
||||
- **Clear log** (Settings → Activity Log, requires authentication) — audited: a system entry records who cleared the log and when
|
||||
|
||||
> **Note:** The Activity Log is distinct from the **debug Log Viewer** (Settings → General → Open Log Viewer). The Log Viewer is an ephemeral real-time tail of the server's Python log stream (WARNING/ERROR lines, resets on disconnect). The Activity Log is a structured, persistent SQLite-backed record of semantic application events.
|
||||
|
||||
### Home Assistant Integration
|
||||
|
||||
- HACS-compatible custom component (separate repository)
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
# CONTEXT — Activity / Audit Log
|
||||
|
||||
Living scratchpad for the feature. The orchestrator updates this between phases. Tier-2
|
||||
context (survives across phases; graduates to CLAUDE.md only if it's a lasting project truth).
|
||||
|
||||
## Config (from approval)
|
||||
|
||||
- **Mode:** Automated · **Execution:** Orchestrator · **Strategy:** Incremental
|
||||
- **Base/merge target:** `master` · **Branch:** `feature/activity-log` · **Branch point:** `17dd2e0`
|
||||
- Final merge ALWAYS requires user approval (even in Automated mode).
|
||||
|
||||
## Product decisions (locked)
|
||||
|
||||
- Placement: BOTH a top-level **Activity** tab AND a Dashboard **Recent Activity** widget,
|
||||
plus a Settings retention panel.
|
||||
- Scope: all four categories — entity CRUD, auth, device connect/disconnect, capture & system.
|
||||
- Detail: **action metadata only** (no before/after diffs).
|
||||
- Durability: **export on demand (CSV/JSON)** + existing whole-DB backup. **No** separate
|
||||
backup subsystem.
|
||||
- WebUI work uses the `frontend-design` skill (Phases 5–6).
|
||||
|
||||
## Key codebase facts (verified during planning)
|
||||
|
||||
- `fire_entity_event(entity_type, action, entity_id)` @ `api/dependencies.py:202` — central
|
||||
hook, called by every entity route synchronously in-request; has `_deps` store access.
|
||||
- `ProcessorManager.fire_event(dict)` / `subscribe_events()` @ `core/processing/processor_manager.py`
|
||||
back `/api/v1/events/ws` (`api/routes/output_targets_control.py:206`). `fire_event` does
|
||||
`put_nowait` (no `call_soon_threadsafe`) — fine for the existing consumer; recorder marshals.
|
||||
- Frontend realtime: `core/events-ws.ts` re-dispatches `server:<type>`; `_ALLOWED_SERVER_EVENT_TYPES`
|
||||
(line 39) is parity-checked by `tests/test_events_ws_parity.py`. Must add `activity_logged`.
|
||||
- Actor: `request.state.auth_label` set in `api/auth.py` (129 authenticated, 83 anonymous).
|
||||
- Storage: `Database` singleton (`storage/database.py`, single conn, RLock, WAL,
|
||||
`synchronous=FULL`, `get_setting/set_setting`). `BaseSqliteStore` loads ALL rows to memory →
|
||||
AVOID for the log. Migrations: `storage/data_migrations.py` (`ALL_MIGRATIONS`, idempotent).
|
||||
- Backup/restore is **whole-DB** (`Database.backup_to`/`restore_from`, no STORE_MAP allowlist)
|
||||
→ the new `activity_log` table is auto-covered. No `system.py` STORE_MAP edit needed.
|
||||
- Background-engine pattern: `core/backup/auto_backup.py` (start/stop loop, `_prune`, settings).
|
||||
- Thread-marshal precedent: `utils/log_broadcaster.py` (`ensure_loop` + `call_soon_threadsafe`).
|
||||
- Device seams: `device_health_changed` (`core/processing/device_health.py`, on transition) +
|
||||
`device_discovered`/`device_lost` (`core/devices/discovery_watcher.py`, **zeroconf thread**).
|
||||
- **No** API-key create/rotate/revoke routes exist (only `GET /system/api-keys`) → those auth
|
||||
events are DESCOPED.
|
||||
- Existing **Log Viewer** (`utils/log_broadcaster.py`) = ephemeral debug-log tail; the audit
|
||||
log is a different, persistent, structured feature. Differentiate; do not duplicate.
|
||||
|
||||
## Frozen contracts (fill as phases complete)
|
||||
|
||||
- ActivityLogEntry fields / dict shape: **frozen** — see phase-1-storage.md Handoff section. 11 fields: `id`, `ts`, `category`, `action`, `severity`, `actor`, `message`, `entity_type`, `entity_id`, `entity_name`, `metadata`. `seq` is DB-only (not on dataclass).
|
||||
- ActivityLogFilters shape: **frozen** — 8 optional fields: `categories`, `severities`, `actor`, `entity_type`, `entity_id`, `since`, `until`, `message_like`. See phase-1-storage.md Handoff.
|
||||
- recorder.record(...) signature + actor ContextVar import path: **frozen** — see phase-2-recorder-retention.md Handoff section. Signature: `record(category, action, *, severity="info", actor=None, entity_type=None, entity_id=None, entity_name=None, message, metadata=None, _bypass_enabled=False)`. ContextVar: `from ledgrab.core.activity_log.context import current_actor`. Module accessor: `from ledgrab.core.activity_log.recorder import get_module_recorder`. Event payload: `{"type": "activity_logged", "entry": {11-field dict with ts as ISO string, metadata as dict}}`. DI getters: `get_activity_recorder()`, `get_activity_log_repo()`, `get_activity_log_retention_engine()`.
|
||||
- API endpoints + query params + page envelope + settings bounds: **frozen** — see phase-4-api.md Handoff section. Endpoints: `GET /api/v1/activity-log` (list, AuthRequired), `GET /api/v1/activity-log/export` (stream CSV/JSON, require_authenticated), `GET|PUT /api/v1/activity-log/settings` (AuthRequired), `DELETE /api/v1/activity-log` (clear, require_authenticated). Page envelope: `entries`, `next_before_seq`, `has_more`, `total`. Settings fields: `enabled` (bool), `max_days` (0–3650), `max_entries` (0–10_000_000). Export: `?format=csv|json`.
|
||||
|
||||
## Failed approaches / rejected designs
|
||||
|
||||
- Buffered async-writer subsystem (asyncio.Queue): REJECTED — unsafe from the zeroconf thread
|
||||
and adds a shutdown-flush ordering hazard. Using direct synchronous-on-loop writes with
|
||||
`call_soon_threadsafe` marshaling instead (simpler + correct).
|
||||
- Using `BaseSqliteStore` for the log: REJECTED — loads all rows into memory.
|
||||
- Separate activity-log backup subsystem: REJECTED — whole-DB backup already covers the table;
|
||||
export-on-demand is the portability story.
|
||||
|
||||
## Deferred / open
|
||||
|
||||
- Setup-scaffold first-run noise suppression (batch/suppress flag) — deferred (nice-to-have).
|
||||
|
||||
## Phase progress notes
|
||||
|
||||
Phase 1 landed (2026-06-09): `activity_log.py` (dataclass + enums + filters + codec), `AddActivityLogTableMigration` (`002_add_activity_log`) appended to `ALL_MIGRATIONS`, `ActivityLogRepository` (record/query/count/prune/clear/iter_export), 41 new tests — all green. Full suite 2226 passed, 0 failed. Schema and method signatures frozen in phase-1-storage.md Handoff. Gotcha: `Database.execute` takes a positional tuple — use `?` placeholders (not `:name`), otherwise Python 3.14 will raise `ProgrammingError`.
|
||||
Phase 2 landed (2026-06-09): `core/activity_log/` package (`context.py`, `recorder.py`, `retention.py`, `__init__.py`); actor ContextVar set in `api/auth.py` (both branches); `ActivityLogRetentionEngine` mirroring AutoBackupEngine; full wiring in `main.py` (repo at module level, recorder+engine in lifespan, `server.shutting_down` first shutdown action, engine stop before db.close); DI getters in `api/dependencies.py`; `activity_logged` added to `_ALLOWED_SERVER_EVENT_TYPES` in `events-ws.ts`; `set_module_recorder` exposes recorder to non-DI sites; 24 new tests — all green. Full suite 2309 passed, 2 skipped, 0 failed. Ruff clean.
|
||||
Phase 4 landed (2026-06-09): schemas (`api/schemas/activity_log.py`), routes (`api/routes/activity_log.py`: list/export/settings/clear), router registration in `api/__init__.py`, `get_seq_for_id` helper on `ActivityLogRepository`. 49 new tests — all green. Full suite 2486 passed, 2 skipped, 0 failed. Ruff clean. Pagination bug found and fixed (limit+1 probe must drop oldest row when has_more, not tail).
|
||||
Phase 3 landed (2026-06-09): instrumented all four categories — entity CRUD via `fire_entity_event` choke-point (`dependencies.py`), auth failures + WS session in `auth.py`, device online/offline in `device_health.py`, device discovered/lost in `discovery_watcher.py`, ADB connect/disconnect in `system_settings.py`, capture start/stop (individual + bulk) in `output_targets_control.py`, scene/playlist/automation activate in their respective route/engine files, backup/restore/delete + restart/shutdown/update/calibration/settings in `backup.py`/`update.py`/`calibration.py`; all 11 entity delete handlers pass `entity_name` to `fire_entity_event`; 22 new tests (security: token never in any field, explicitly asserted) — all green. Full suite 2369 passed, 2 skipped, 0 failed. Ruff clean. Complete (category, action) inventory in phase-3-instrumentation.md Handoff section.
|
||||
Phase 5 landed (2026-06-09): Activity tab frontend — `features/activity-log.ts` (loadActivityLog, filter toolbar with category/severity chips + presets + debounced search + date range + actor/entity-type text fields, keyset-paginated list, expandable detail drawer, live-prepend via `server:activity_logged`, authed CSV/JSON blob export); `core/ui.ts` additions (`formatTimestamp`, `formatRelativeTime`); `core/icon-paths.ts` + `core/icons.ts` additions (scrollText/audit, circleAlert, info, filter, xCircle); `core/tab-registry.ts` registered; `templates/index.html` tab button + panel; `app.ts` + `global.d.ts` wired; `static/css/activity-log.css` (precision-ledger design, category color rail, live-dot pulse, detail drawer, responsive breakpoints); all three locales updated (activity_log.* + time.* relative-time keys + tour.activity_log); `features/tutorials.ts` getting-started tour extended. `tsc --noEmit` clean, `npm run build` passes. Reusable helpers for Phase 6 documented in phase-5-frontend-tab.md Handoff section.
|
||||
Phase 6 landed (2026-06-09): Dashboard "Recent Activity" widget (live SSE, View-all link, `.dal-*` CSS, loading/empty states) + Settings "Activity Log" panel (enabled toggle, max_days/max_entries, Save toast, authed CSV/JSON export, confirmed Clear, audit-vs-debug cross-links) + 32 i18n keys per locale + README "Activity Log" section. `tsc --noEmit` clean, `npm run build` passes. All six phases complete — feature ready for final review.
|
||||
@@ -0,0 +1,141 @@
|
||||
# Feature: Activity / Audit Log
|
||||
|
||||
**Branch:** `feature/activity-log`
|
||||
**Base branch:** `master` (merge target)
|
||||
**Branch point:** `17dd2e02bab4d00a93479eb6af1a8c6ddc0c7224` (use for clean review diffs)
|
||||
**Created:** 2026-06-09
|
||||
**Status:** 🟢 All phases complete — awaiting final review + merge
|
||||
**Strategy:** Incremental
|
||||
**Mode:** Automated
|
||||
**Execution:** Orchestrator
|
||||
**Remote:** origin → https://git.dolgolyov-family.by/alexei.dolgolyov/ledgrab.git
|
||||
|
||||
## Summary
|
||||
|
||||
A persistent, queryable audit log of meaningful LedGrab actions, surfaced in the WebUI.
|
||||
Captures four categories — entity CRUD, authentication, device connect/disconnect, and
|
||||
capture & system events — as **action-metadata-only** records (who/what/when + entity
|
||||
type/name/id + a human-readable message + small structured metadata; **no before/after
|
||||
diffs**). Surfaced as a dedicated top-level **Activity** tab with smart filtering + live
|
||||
updates, a compact **Recent Activity** widget on the Dashboard, and a **Settings** panel
|
||||
for retention. Durability rides on the existing whole-DB `ledgrab.db` backup; portability
|
||||
is an on-demand CSV/JSON **export** (no separate backup subsystem).
|
||||
|
||||
## Design pillars (the load-bearing decisions)
|
||||
|
||||
1. **Dedicated `activity_log` table + repository — NOT `BaseSqliteStore`.** That base loads
|
||||
every row into an in-memory cache and uses a generic `id/name/data` blob — wrong for an
|
||||
append-heavy, unbounded log. We use a purpose-built indexed table with query-on-demand
|
||||
keyset pagination.
|
||||
2. **Central choke-point instrumentation.** `fire_entity_event()` (`api/dependencies.py:202`)
|
||||
is already called by every entity route on create/update/delete and has `_deps` access to
|
||||
resolve names. The recorder hooks there for all entity CRUD. Non-entity events get explicit
|
||||
`recorder.record(...)` calls.
|
||||
3. **Actor via `ContextVar`.** Set inside `verify_api_key` (next to `request.state.auth_label`),
|
||||
default `"system"`, reset per-request. The recorder reads it without threading actor
|
||||
through every call.
|
||||
4. **Direct synchronous write on the event-loop thread** (no separate buffered-writer
|
||||
subsystem — simpler, and the request already did a `synchronous=FULL` entity write).
|
||||
Cross-thread callers (zeroconf discovery thread) marshal via `loop.call_soon_threadsafe`,
|
||||
mirroring `utils/log_broadcaster.py`. The "server shutting down" event is recorded as the
|
||||
FIRST action in the lifespan shutdown block, before any teardown.
|
||||
5. **Reuse the existing realtime bus.** A new `activity_logged` event over `/api/v1/events/ws`
|
||||
(one `events-ws.ts` allowlist entry + `test_events_ws_parity.py` update). No new socket.
|
||||
6. **Never log secrets.** API-key *tokens* are never stored — only labels/ids.
|
||||
7. **Differentiate from the existing Log Viewer.** `utils/log_broadcaster.py` is an ephemeral
|
||||
500-line debug-log tail. The audit log is persistent, structured, semantic. Cross-link in
|
||||
the Settings panel; never duplicate.
|
||||
|
||||
## Build & Test Commands
|
||||
|
||||
- **Build (frontend):** `cd server && npm run build`
|
||||
- **Type-check (TS):** `cd server && npx tsc --noEmit`
|
||||
- **Test:** `cd server && py -3.13 -m pytest tests/ --no-cov -q`
|
||||
- **Lint (Python):** `cd server && ruff check src/ tests/ --fix`
|
||||
- **Events parity test (load-bearing for P2):** included in pytest (`tests/test_events_ws_parity.py`)
|
||||
|
||||
> Scope checks to the files actually edited: backend phases run ruff + pytest; frontend-only
|
||||
> phases run `tsc --noEmit` + `npm run build` (no pytest/ruff). Phases touching both run both.
|
||||
|
||||
## Phases
|
||||
|
||||
- [x] Phase 1: Storage — model, migration, repository [domain: data] → [subplan](./phase-1-storage.md)
|
||||
- [x] Phase 2: Recorder, actor context, retention, lifecycle [domain: backend] → [subplan](./phase-2-recorder-retention.md)
|
||||
- [x] Phase 3: Event instrumentation (4 categories) [domain: backend] → [subplan](./phase-3-instrumentation.md)
|
||||
- [x] Phase 4: REST API — query/filter/export/settings/clear [domain: backend] → [subplan](./phase-4-api.md)
|
||||
- [x] Phase 5: Frontend — Activity tab + smart filtering + live updates [domain: frontend] → [subplan](./phase-5-frontend-tab.md)
|
||||
- [x] Phase 6: Dashboard widget + Settings panel + docs [domain: frontend] → [subplan](./phase-6-dashboard-settings.md)
|
||||
|
||||
## Parallelizable Phase Groups (Orchestrator mode only)
|
||||
|
||||
- Phases 3 and 4 are **parallelizable only after the schema is frozen at end of Phase 2**.
|
||||
Both depend on the P2 recorder/schema; P4 registers the router in `api/__init__.py`, P3
|
||||
edits `dependencies.py`/`auth.py` (different files, shared schema contract). To keep the
|
||||
Automated run simple and low-risk, they run **sequentially** (3 → 4) unless time pressure
|
||||
warrants worktree-isolated parallelism. Phases 5 → 6 are sequential (6 reuses P5 formatters
|
||||
and the feature module).
|
||||
|
||||
## Phase Progress Log
|
||||
|
||||
| Phase | Domain | Status | Review | Build | Committed |
|
||||
|-------|--------|--------|--------|-------|-----------|
|
||||
| Phase 1: Storage | data | ✅ Done | ✅ Passed | ✅ Passed | ✅ |
|
||||
| Phase 2: Recorder/Retention | backend | ✅ Done | ✅ Passed | ✅ Passed | ✅ |
|
||||
| Phase 3: Instrumentation | backend | ✅ Done | ✅ Passed | ✅ Passed | ✅ |
|
||||
| Phase 4: REST API | backend | ✅ Done | ✅ Passed | ✅ Passed | ✅ |
|
||||
| Phase 5: Frontend tab | frontend | ✅ Done | ✅ Passed | ✅ Passed (tsc+build) | ✅ |
|
||||
| Phase 6: Dashboard/Settings | frontend | ✅ Done | ✅ Passed | ✅ Passed (tsc+build) | ✅ |
|
||||
|
||||
## Outstanding Warnings
|
||||
|
||||
| Phase | Warning | Severity | Status (open / resolved / accepted) |
|
||||
|-------|---------|----------|-------------------------------------|
|
||||
| 3 | Log injection via unauth mDNS device name/url into audit message | 🟠 High (security) | resolved — `sanitize_display` helper applied |
|
||||
| 3 | Origin sanitizer missed spaces/NUL/ANSI | 🟠 High (security) | resolved — `sanitize_display` over netloc |
|
||||
| 3 | Unauth auth-failure audit-write flood (no write-rate bound) | 🟠 High (security) | resolved — per-IP audit-record throttle (10s, capped) |
|
||||
| 3 | Malformed-IPv6 Origin → urlparse ValueError into WS handler | 🟡 Warning | resolved — try/except guard |
|
||||
| 3 | Throttle module-global state caused flaky test contamination | 🟡 Warning | resolved — autouse conftest reset fixture |
|
||||
| 4 | Export held global DB write-lock across the stream (slow-client DoS) | 🟠 High (security) | resolved — chunked keyset export releases lock per batch |
|
||||
| 4 | PUT /settings only AuthRequired → anon could disable auditing/prune trail | 🟠 High (security) | resolved — `require_authenticated` on settings PUT |
|
||||
| 4 | CSV formula-injection missed leading TAB/CR | 🟡 Medium (security) | resolved — added `\t`/`\r` to guard |
|
||||
| 4 | `total` count full-scans on every list request | 🔵 Low (perf) | accepted — bounded by retention; read-only; optional opt-in deferred |
|
||||
| 5 | Inverted list ordering broke pagination + live-append | 🔴 Blocker | resolved — pages reversed to newest-first; re-review PASS |
|
||||
| 5 | Attribute-context XSS (entity_name title + JSON.stringify onclick) | 🟡 Warning (security) | resolved — `_escapeAttr` + data-attr event delegation |
|
||||
| 5 | Filter toolbar value= attrs not quote-escaped (new code) | 🟡 Warning (security) | resolved — `_escapeAttr` on q/actor/entity_type/since/until |
|
||||
| 5 | Manual browser smoke test (tab loads, filters, live, export) | 🔵 Note | open — recommend at final review (server restart needed) |
|
||||
| 6 | clearActivityLog() 401 path unreachable → silent failure | 🟡 Warning | resolved — `handle401:false` surfaces auth-required toast |
|
||||
| 6 | Recent Activity widget dropped first live event when empty | 🔵 Note | resolved — empty→list transition on first live event |
|
||||
| 6 | Widget outside dashboard layout-toggle/ordering system | 🔵 Note | accepted — deliberate (always-visible), collapse still works |
|
||||
| Final | Entity-crosslink map keys mismatched backend entity_type (device/color_strip_source/audio_source) | 🟡 Warning | resolved — `_ENTITY_NAV` keys corrected + scene_playlist added |
|
||||
| Final | Owner-authored names interpolated raw at some record sites | 🔵 Note (defense-in-depth) | resolved — `sanitize_display` applied uniformly |
|
||||
| Manual test | Entry descriptions rendered server English (not localized) | 🟡 Warning (acceptance criterion) | resolved — client-side `localizeMessage` from structured fields + `activity_log.msg.*`/`entity_type.*` templates ×3 |
|
||||
| Manual test | Redundant "Activity" header banner at top of tab | 🔵 Note | resolved — header block removed |
|
||||
| Manual test | Recent Activity widget missing from Customize Dashboard | 🟡 Warning | resolved — registered as a first-class dashboard section (show/hide/reorder; pre-existing layouts preserved) |
|
||||
| Manual test | Activity widget live event rebuilt the whole dashboard | 🟡 Warning (perf) | resolved — surgical list update; single listener with teardown |
|
||||
| Manual test | Relative-time labels static (never tick) | 🟡 Warning | resolved — shared `ensureRelativeTimeTicker` (single 30s interval, visibility-aware) |
|
||||
| Manual test | Dashboard fully rebuilt/jumped on entity edits (pre-existing `forceFullRender` wholesale innerHTML) | 🟡 Warning (UX) | resolved — section-level reconciliation (`_reconcileDynamicSections`): only changed sections replaced; widget live DOM + perf strip preserved |
|
||||
| Manual test | Activity list columns slightly misaligned | 🔵 Note | resolved — CSS grid with fixed badge/actor columns |
|
||||
| Manual test | Reconciler left orphan "no targets" node on empty→populated | 🟡 Warning (regression, caught in review) | resolved — sweep non-section top-level children |
|
||||
|
||||
## Final Review
|
||||
|
||||
- [ ] Comprehensive code review
|
||||
- [ ] Security review (auth/PII-in-logs/secrets/log-injection — triggered)
|
||||
- [ ] All Outstanding Warnings resolved or consciously accepted
|
||||
- [ ] Full build passes (`npm run build` + `tsc --noEmit`)
|
||||
- [ ] Full test suite passes (`pytest`)
|
||||
- [ ] Merged to `master`
|
||||
|
||||
## Amendment Log
|
||||
|
||||
_(Filled in if the plan is amended mid-implementation.)_
|
||||
|
||||
- 2026-06-09: Plan reviewer (pre-implementation) → ⚠️ with 3 Critical Gaps, all resolved
|
||||
before Phase 1: (G1) descoped non-existent API-key mutation events; (G2) dropped the
|
||||
buffered-writer subsystem for a direct synchronous-on-loop write with `call_soon_threadsafe`
|
||||
marshaling for thread-origin events; (G3) record "server shutting down" first in the shutdown
|
||||
block (no buffer to flush). Concerns folded in: device events via `device_health_changed` +
|
||||
`device_discovered/_lost`; actor ContextVar in `verify_api_key`; name-on-delete passed
|
||||
explicitly; settings-audit scoped + self-key excluded; parity allowlist before emit site.
|
||||
Adopted suggestions: rowid keyset tiebreaker; log the disable action. Deferred: setup-scaffold
|
||||
noise suppression.
|
||||
@@ -0,0 +1,177 @@
|
||||
# Phase 1: Storage — model, migration, repository
|
||||
|
||||
**Status:** ✅ Done
|
||||
**Parent plan:** [PLAN.md](./PLAN.md)
|
||||
**Domain:** data
|
||||
|
||||
## Objective
|
||||
|
||||
Create the persistent foundation for the audit log: an `ActivityLogEntry` dataclass, an
|
||||
additive idempotent SQLite migration that creates a dedicated indexed `activity_log` table,
|
||||
and a purpose-built `ActivityLogRepository` (NOT `BaseSqliteStore`) supporting append,
|
||||
keyset-paginated filtered query, count, time/count-based prune, and streaming export.
|
||||
|
||||
## Tasks
|
||||
|
||||
- [x] Create `server/src/ledgrab/storage/activity_log.py`:
|
||||
- `ActivityCategory` and `ActivitySeverity` string enums (or `Literal` unions used as
|
||||
constants). Categories: `auth`, `device`, `entity`, `capture`, `system`. Severities:
|
||||
`info`, `warning`, `error`.
|
||||
- `@dataclass ActivityLogEntry` with fields: `id: str` (e.g. `al_<uuid8>`), `ts: datetime`
|
||||
(UTC, server-assigned), `category: str`, `action: str`, `severity: str`, `actor: str`,
|
||||
`entity_type: str | None`, `entity_id: str | None`, `entity_name: str | None`,
|
||||
`message: str`, `metadata: dict` (small JSON; default empty). Provide `to_row()` /
|
||||
`from_row()` (column tuple/dict ↔ dataclass; `metadata` JSON-encoded; `ts` isoformat).
|
||||
- [x] Add migration to `server/src/ledgrab/storage/data_migrations.py`:
|
||||
- New `DataMigration` subclass `AddActivityLogTableMigration` with unique `name`
|
||||
(next sequential id, e.g. `"NNN_add_activity_log"` — match existing naming) and
|
||||
`apply(conn)` creating `activity_log` with an INTEGER PRIMARY KEY AUTOINCREMENT `seq`
|
||||
(monotonic keyset tiebreaker) plus columns: `id TEXT UNIQUE NOT NULL`, `ts TEXT NOT NULL`,
|
||||
`category TEXT NOT NULL`, `action TEXT NOT NULL`, `severity TEXT NOT NULL`,
|
||||
`actor TEXT NOT NULL`, `entity_type TEXT`, `entity_id TEXT`, `entity_name TEXT`,
|
||||
`message TEXT NOT NULL`, `metadata TEXT NOT NULL DEFAULT '{}'`.
|
||||
- Indexes: `(ts DESC, seq DESC)` (primary keyset/sort), `category`, `severity`, `actor`,
|
||||
`(entity_type, entity_id)`. Use `CREATE TABLE/INDEX IF NOT EXISTS` for idempotency.
|
||||
- Append the instance to `ALL_MIGRATIONS` (never reorder existing entries).
|
||||
- [x] Create `server/src/ledgrab/storage/activity_log_repository.py`:
|
||||
- `class ActivityLogRepository` taking `db: Database` (NOT subclassing `BaseSqliteStore`).
|
||||
- `record(entry: ActivityLogEntry) -> None`: single parameterized INSERT via
|
||||
`db.execute(...)` (auto-commit). The `seq` is DB-assigned. **Caller guarantees this runs
|
||||
on the event-loop thread** (see Phase 2 — cross-thread marshaling lives in the recorder).
|
||||
- `query(filters: ActivityLogFilters, *, before_seq: int | None, limit: int) -> list[ActivityLogEntry]`:
|
||||
keyset pagination `WHERE seq < ? ORDER BY seq DESC LIMIT ?` plus optional filters —
|
||||
`category IN (...)`, `severity IN (...)`, `actor = ?`, `entity_type = ?`, `entity_id = ?`,
|
||||
`ts >= ?` / `ts <= ?`, `message LIKE ?` (free-text, `%q%`, escaped). All parameterized.
|
||||
- `count(filters) -> int`.
|
||||
- `prune(*, before_ts: datetime | None, max_entries: int | None) -> int`: delete rows older
|
||||
than `before_ts`, and/or trim to the newest `max_entries` by `seq`. Returns rows deleted.
|
||||
- `clear() -> int`: delete all rows (used by the API clear endpoint; the clear action is
|
||||
itself audited by the recorder, not here). Returns rows deleted.
|
||||
- `iter_export(filters) -> Iterator[ActivityLogEntry]`: cursor-based streaming for export
|
||||
(does not load all rows into memory).
|
||||
- Define a small `ActivityLogFilters` dataclass (all-optional fields) in the repository or
|
||||
`activity_log.py` and reuse it across query/count/prune/export.
|
||||
- [x] Unit tests in `server/tests/storage/test_activity_log_repository.py`:
|
||||
- insert + read back round-trip (incl. metadata JSON, UTC ts);
|
||||
- filter by each dimension (category/severity/actor/entity/date/free-text);
|
||||
- keyset pagination stability across two pages with same-`ts` rows (seq tiebreaker);
|
||||
- prune by age and by max_entries;
|
||||
- clear; count; export iterator yields all matching rows;
|
||||
- migration idempotency (constructing the repo twice / running migrations twice is safe).
|
||||
|
||||
## Files to Modify/Create
|
||||
|
||||
- `server/src/ledgrab/storage/activity_log.py` — new: dataclass + enums + filters + row codec
|
||||
- `server/src/ledgrab/storage/data_migrations.py` — modify: add migration + append to `ALL_MIGRATIONS`
|
||||
- `server/src/ledgrab/storage/activity_log_repository.py` — new: repository
|
||||
- `server/tests/storage/test_activity_log_repository.py` — new: unit tests
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- `activity_log` table + indexes created idempotently on startup (running migrations twice is a no-op).
|
||||
- Query is keyset-paginated and index-backed; a 10k-row table never loads fully into memory.
|
||||
- Pagination is stable when many rows share the same millisecond `ts` (uses `seq` tiebreaker).
|
||||
- `prune` removes by age AND by max-entry cap; `clear` empties the table; `export` streams.
|
||||
- All filters use parameterized SQL (no string interpolation of user input).
|
||||
- New unit tests pass; `ruff check` clean; existing tests still green.
|
||||
|
||||
## Notes
|
||||
|
||||
- Reference patterns: `storage/database.py` (`execute`, `transaction`, `get_setting`),
|
||||
`storage/data_migrations.py` (`DataMigration`, `MigrationRunner`, `ALL_MIGRATIONS`),
|
||||
`storage/sync_clock.py` (dataclass `to_dict`/`from_dict` style).
|
||||
- 🔒 **Migration-safety addendum (data domain):** this migration is purely additive (new
|
||||
table) — no rename, no field/key/file move, no data movement → no data-loss risk. Still
|
||||
idempotent (`IF NOT EXISTS`). Rollback = drop the table; no user data is transformed.
|
||||
- Do NOT wire the repository into `main.py` or `dependencies.py` here — that is Phase 2.
|
||||
- `Database`'s connection is created with the existing threading model; the repository must
|
||||
not assume it can be called from arbitrary threads. Thread marshaling is Phase 2's job.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- [x] All tasks completed
|
||||
- [x] Code follows project conventions (dataclass codec style, migration naming)
|
||||
- [x] No unintended side effects (no startup wiring yet)
|
||||
- [x] Build passes (ruff + pytest)
|
||||
- [x] Tests pass (new + existing)
|
||||
|
||||
## Handoff to Next Phase
|
||||
|
||||
### ActivityLogEntry — final field list and dict shape
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class ActivityLogEntry:
|
||||
id: str # "al_<uuid8>" — caller-assigned
|
||||
ts: datetime # UTC-aware; stored as ISO-8601 string in DB
|
||||
category: str # ActivityCategory constant
|
||||
action: str # verb-object label, e.g. "entity.created"
|
||||
severity: str # ActivitySeverity constant
|
||||
actor: str # API-key label or "system"
|
||||
message: str # human-readable description
|
||||
entity_type: str | None # e.g. "output_target"
|
||||
entity_id: str | None # stable entity id
|
||||
entity_name: str | None # name at time of event
|
||||
metadata: dict # JSON-serialisable; default {}
|
||||
```
|
||||
|
||||
`to_row()` returns a flat dict with 11 keys (same names); `metadata` is JSON string, `ts` is isoformat string. `seq` is NOT in `to_row()` — it is DB-assigned.
|
||||
|
||||
### ActivityLogFilters — shape (all fields optional, default None)
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class ActivityLogFilters:
|
||||
categories: Sequence[str] | None # category IN (...)
|
||||
severities: Sequence[str] | None # severity IN (...)
|
||||
actor: str | None # exact match
|
||||
entity_type: str | None # exact match
|
||||
entity_id: str | None # exact match
|
||||
since: datetime | None # ts >= since
|
||||
until: datetime | None # ts <= until
|
||||
message_like: str | None # LIKE %value% (escaped)
|
||||
```
|
||||
|
||||
### Migration name used
|
||||
|
||||
`"002_add_activity_log"` — appended as position [1] in `ALL_MIGRATIONS`.
|
||||
|
||||
### ActivityLogRepository — exact method signatures
|
||||
|
||||
```python
|
||||
class ActivityLogRepository:
|
||||
def __init__(self, db: Database) -> None
|
||||
def record(self, entry: ActivityLogEntry) -> None
|
||||
def query(
|
||||
self,
|
||||
filters: ActivityLogFilters,
|
||||
*,
|
||||
before_seq: int | None = None,
|
||||
limit: int = 50,
|
||||
) -> list[ActivityLogEntry]
|
||||
def count(self, filters: ActivityLogFilters | None = None) -> int
|
||||
def prune(
|
||||
self,
|
||||
*,
|
||||
before_ts: datetime | None = None,
|
||||
max_entries: int | None = None,
|
||||
) -> int
|
||||
def clear(self) -> int
|
||||
def iter_export(
|
||||
self, filters: ActivityLogFilters | None = None
|
||||
) -> Iterator[ActivityLogEntry]
|
||||
```
|
||||
|
||||
### Key behavioural notes for Phase 2/3/4
|
||||
|
||||
- `record()` expects to be called from the event-loop thread (or with `Database` RLock already held). Phase 2 is responsible for thread marshaling via `loop.call_soon_threadsafe`.
|
||||
- `query()` returns entries in **ascending chronological order within the page** (reversed internally from DESC fetch for display convenience). The smallest `seq` on a page is `page[0]`'s seq — pass that as `before_seq` for the next page.
|
||||
- `count(None)` == `count(ActivityLogFilters())` — both count all rows.
|
||||
- `prune(before_ts=X, max_entries=N)` applies both predicates independently (age prune first, then count cap).
|
||||
- `iter_export` holds `db._lock` for the entire iteration. Phase 4 should stream the response and consume promptly.
|
||||
- `ActivityLogCategory` and `ActivityLogSeverity` are plain classes with string class-attributes and an `ALL` tuple — NOT `enum.Enum`.
|
||||
- Imports for Phase 2/3/4:
|
||||
```python
|
||||
from ledgrab.storage.activity_log import ActivityLogEntry, ActivityLogFilters, ActivityCategory, ActivitySeverity
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
```
|
||||
@@ -0,0 +1,196 @@
|
||||
# Phase 2: Recorder, actor context, retention, lifecycle
|
||||
|
||||
**Status:** ✅ Done
|
||||
**Parent plan:** [PLAN.md](./PLAN.md)
|
||||
**Domain:** backend
|
||||
|
||||
## Objective
|
||||
|
||||
Build the runtime layer over the Phase 1 repository: a thread-safe `ActivityRecorder` facade
|
||||
that persists an entry AND pushes a live `activity_logged` event; an actor `ContextVar`
|
||||
populated by the auth layer; a background `ActivityLogRetentionEngine` mirroring
|
||||
`AutoBackupEngine`; and the `main.py`/`dependencies.py` wiring (init, DI getter, retention
|
||||
start/stop, shutdown ordering). After this phase the audit log records nothing yet (no call
|
||||
sites) — that is Phase 3 — but the full machinery is live and unit-tested.
|
||||
|
||||
## Tasks
|
||||
|
||||
- [x] Create `server/src/ledgrab/core/activity_log/__init__.py` and
|
||||
`server/src/ledgrab/core/activity_log/recorder.py`:
|
||||
- `ActivityRecorder(repo: ActivityLogRepository, processor_manager, *, loop=None)`.
|
||||
- `record(category, action, *, severity="info", actor=None, entity_type=None,
|
||||
entity_id=None, entity_name=None, message, metadata=None) -> None`:
|
||||
- resolve `actor` from the actor `ContextVar` when not supplied, default `"system"`;
|
||||
- build an `ActivityLogEntry` (id `al_<uuid8>`, `ts=datetime.now(timezone.utc)`);
|
||||
- **thread-safe write:** if called on the event loop thread, write inline via
|
||||
`repo.record(entry)` then fire the live event; if called from another thread (zeroconf
|
||||
discovery), marshal the whole write+emit onto the loop via
|
||||
`loop.call_soon_threadsafe(...)`. Capture the loop lazily (mirror
|
||||
`utils/log_broadcaster.py:ensure_loop`/`call_soon_threadsafe`). Never raise into the
|
||||
caller — audit recording is best-effort and must not break the audited action; log
|
||||
failures at `warning`.
|
||||
- live push: `processor_manager.fire_event({"type": "activity_logged", "entry": entry_as_dict})`.
|
||||
- Provide a tiny helper to serialize an entry to the same dict shape the API returns
|
||||
(reuse in Phase 4 / frontend).
|
||||
- `enabled` flag honored: when retention settings say `enabled=false`, `record()` is a
|
||||
no-op — EXCEPT the "audit log disabled" event itself, which must be recorded before the
|
||||
flag takes effect (see retention engine).
|
||||
- [x] Actor `ContextVar`:
|
||||
- Add `current_actor: ContextVar[str]` (module-level, e.g. in `core/activity_log/context.py`
|
||||
or `api/auth.py`). In `verify_api_key` (`api/auth.py`), set it next to the existing
|
||||
`request.state.auth_label = ...` (both the authenticated label and the `"anonymous"`
|
||||
branch). Default `"system"` when unset. Ensure no cross-request leakage (set on every
|
||||
auth evaluation).
|
||||
- [x] Create `server/src/ledgrab/core/activity_log/retention.py`:
|
||||
- `ActivityLogRetentionEngine(repo, db, recorder)` mirroring `core/backup/auto_backup.py`:
|
||||
`_load_settings()`/`_save_settings()` via `db.get_setting("activity_log")` /
|
||||
`db.set_setting("activity_log", {...})`, `DEFAULT_SETTINGS = {"enabled": True,
|
||||
"max_days": 90, "max_entries": 20000}`.
|
||||
`async start()` → spawn `_retention_loop()` (`asyncio.create_task`); loop sleeps a sane
|
||||
interval (e.g. hourly) then calls `repo.prune(before_ts=now-max_days, max_entries=...)`.
|
||||
`async stop()` → cancel + await task. `get_settings()` / `async update_settings(...)`
|
||||
that persist and apply (changing `enabled` is logged via the recorder BEFORE disabling).
|
||||
- [x] Wiring:
|
||||
- `main.py`: instantiate `activity_log_repo = ActivityLogRepository(db)` (module level near
|
||||
other stores); in `lifespan` startup build `activity_recorder` + `activity_log_retention_engine`,
|
||||
pass to `init_dependencies(...)`, and `await activity_log_retention_engine.start()`.
|
||||
- In `lifespan` **shutdown**: record a `system` / `server_shutting_down` event via the
|
||||
recorder as the **first** shutdown action (before engines/db close), then
|
||||
`await _bounded("activity_log_retention.stop", activity_log_retention_engine.stop(), timeout=0.5)`.
|
||||
- `api/dependencies.py`: add `activity_recorder` + `activity_log_repo` +
|
||||
`activity_log_retention_engine` to `_deps`, parameters to `init_dependencies`, and
|
||||
getters `get_activity_recorder()`, `get_activity_log_repo()`,
|
||||
`get_activity_log_retention_engine()`.
|
||||
- [x] Realtime allowlist (order matters — do allowlist FIRST so the parity test stays green):
|
||||
- Add `'activity_logged'` to `_ALLOWED_SERVER_EVENT_TYPES` in
|
||||
`server/src/ledgrab/static/js/core/events-ws.ts` (+ a one-line comment naming the source).
|
||||
- Confirm `tests/test_events_ws_parity.py` passes with the new emit type.
|
||||
- [x] Unit tests `server/tests/core/test_activity_recorder.py` +
|
||||
`test_activity_log_retention.py`:
|
||||
- recorder persists an entry AND calls `fire_event` with `type=="activity_logged"`;
|
||||
- actor resolves from ContextVar; defaults to `"system"`; failure in repo doesn't raise;
|
||||
- cross-thread `record()` (call from a `threading.Thread`) routes through the loop and persists;
|
||||
- retention prunes per settings; settings round-trip via db; disabling logs the disable event.
|
||||
|
||||
## Files to Modify/Create
|
||||
|
||||
- `server/src/ledgrab/core/activity_log/__init__.py` — new
|
||||
- `server/src/ledgrab/core/activity_log/recorder.py` — new
|
||||
- `server/src/ledgrab/core/activity_log/context.py` — new (actor ContextVar) *(or place in auth.py)*
|
||||
- `server/src/ledgrab/core/activity_log/retention.py` — new
|
||||
- `server/src/ledgrab/api/auth.py` — modify: set actor ContextVar in `verify_api_key`
|
||||
- `server/src/ledgrab/main.py` — modify: instantiate, wire lifespan start/shutdown
|
||||
- `server/src/ledgrab/api/dependencies.py` — modify: `_deps`, `init_dependencies`, getters
|
||||
- `server/src/ledgrab/static/js/core/events-ws.ts` — modify: allowlist `activity_logged`
|
||||
- `server/tests/core/test_activity_recorder.py` — new
|
||||
- `server/tests/core/test_activity_log_retention.py` — new
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Recorder persists + fires `activity_logged`; never raises into callers; thread-safe from
|
||||
non-loop threads.
|
||||
- Actor ContextVar populated by auth; default `"system"`; no cross-request leakage.
|
||||
- Retention engine starts/stops cleanly in lifespan; prunes by age + count; settings persist.
|
||||
- `server_shutting_down` is recorded before teardown; no lost-on-graceful-shutdown entries.
|
||||
- `test_events_ws_parity.py` green (allowlist updated). Existing tests still green; `ruff` clean.
|
||||
|
||||
## Notes
|
||||
|
||||
- Reference: `core/backup/auto_backup.py` (engine shape, settings persistence, `_bounded`
|
||||
shutdown in `main.py`), `utils/log_broadcaster.py` (`ensure_loop`, `call_soon_threadsafe`
|
||||
thread marshaling), `core/processing/processor_manager.py:247` (`fire_event`).
|
||||
- **Do not add any instrumentation call sites in this phase** — only the machinery. Phase 3
|
||||
adds the `record(...)` calls. (Intermediate commit emits nothing; that is fine and green.)
|
||||
- Freeze the `ActivityLogEntry` dict shape here — Phase 4 (API response) and Phase 5
|
||||
(frontend `entry`) consume it.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- [x] All tasks completed
|
||||
- [x] Code follows project conventions (engine/DI patterns)
|
||||
- [x] No unintended side effects (no call sites yet; lifespan order correct)
|
||||
- [x] Build passes (ruff + pytest, incl. parity test)
|
||||
- [x] Tests pass (new + existing)
|
||||
|
||||
## Handoff to Next Phase
|
||||
|
||||
### recorder.record(...) — final signature
|
||||
|
||||
```python
|
||||
recorder.record(
|
||||
category: str, # ActivityCategory constant
|
||||
action: str, # verb-object label
|
||||
*,
|
||||
severity: str = "info", # ActivitySeverity constant
|
||||
actor: str | None = None, # resolved from current_actor ContextVar when None
|
||||
entity_type: str | None = None,
|
||||
entity_id: str | None = None,
|
||||
entity_name: str | None = None,
|
||||
message: str,
|
||||
metadata: dict | None = None,
|
||||
_bypass_enabled: bool = False, # internal: used by retention engine only
|
||||
) -> None
|
||||
```
|
||||
|
||||
### Actor ContextVar import path
|
||||
|
||||
```python
|
||||
from ledgrab.core.activity_log.context import current_actor
|
||||
```
|
||||
|
||||
### Module accessor (for non-DI sites)
|
||||
|
||||
```python
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder, set_module_recorder
|
||||
recorder = get_module_recorder() # returns ActivityRecorder | None
|
||||
```
|
||||
|
||||
### entry_to_dict helper (for API response serialisation)
|
||||
|
||||
```python
|
||||
from ledgrab.core.activity_log.recorder import entry_to_dict
|
||||
d = entry_to_dict(entry) # returns dict with 11 keys
|
||||
```
|
||||
|
||||
### Frozen `activity_logged` event payload shape
|
||||
|
||||
```python
|
||||
{
|
||||
"type": "activity_logged",
|
||||
"entry": {
|
||||
"id": str, # "al_<8-hex>"
|
||||
"ts": str, # ISO-8601 UTC string
|
||||
"category": str,
|
||||
"action": str,
|
||||
"severity": str,
|
||||
"actor": str,
|
||||
"entity_type": str | None,
|
||||
"entity_id": str | None,
|
||||
"entity_name": str | None,
|
||||
"message": str,
|
||||
"metadata": dict, # real dict, not JSON string
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### DI getter names (in `api/dependencies.py`)
|
||||
|
||||
```python
|
||||
from ledgrab.api.dependencies import (
|
||||
get_activity_recorder,
|
||||
get_activity_log_repo,
|
||||
get_activity_log_retention_engine,
|
||||
)
|
||||
```
|
||||
|
||||
### Notes for Phase 3
|
||||
|
||||
- Phase 3 instruments `fire_entity_event` in `api/dependencies.py` by calling
|
||||
`get_module_recorder()` there (not via FastAPI DI — it's a plain function).
|
||||
- The actor ContextVar is already set by `verify_api_key` before any route
|
||||
handler runs, so entity events carry the correct actor automatically.
|
||||
- `recorder.record(...)` never raises; Phase 3 call sites need no try/except.
|
||||
|
||||
Phase 2 landed (2026-06-09): ActivityRecorder, actor ContextVar, ActivityLogRetentionEngine,
|
||||
all wiring in main.py/dependencies.py/auth.py, activity_logged allowlist in events-ws.ts,
|
||||
24 new tests — all green. Full suite 2309 passed, 2 skipped, 0 failed. Ruff clean.
|
||||
@@ -0,0 +1,172 @@
|
||||
# Phase 3: Event instrumentation (4 categories)
|
||||
|
||||
**Status:** ✅ Done
|
||||
**Parent plan:** [PLAN.md](./PLAN.md)
|
||||
**Domain:** backend · 🔒 security-sensitive (security reviewer triggers)
|
||||
|
||||
## Objective
|
||||
|
||||
Emit audit records at the real call sites for all four categories, using the Phase 2 recorder.
|
||||
Maximize coverage via the central `fire_entity_event` choke point; add explicit
|
||||
`recorder.record(...)` calls for non-entity events. Never log secrets.
|
||||
|
||||
## Tasks
|
||||
|
||||
### Entity CRUD (via the choke point)
|
||||
- [x] In `api/dependencies.py`, extend `fire_entity_event` to ALSO record an audit entry:
|
||||
- Signature gains an optional `entity_name: str | None = None`.
|
||||
- For `created`/`updated`: if `entity_name` not supplied, best-effort resolve from the
|
||||
matching store in `_deps` keyed by `entity_type` (entity still present). For `deleted`:
|
||||
**do not** resolve post-hoc — rely on the explicit `entity_name` passed by the handler
|
||||
(deletes are the most important; a name-less delete entry is unacceptable).
|
||||
- Map `action` → severity (`info`), category `entity`. Build a human message
|
||||
(e.g. `"Target 'Desk' updated"`). Read actor from the ContextVar.
|
||||
- Recording is best-effort (never break the entity operation).
|
||||
- [x] Update entity **delete** handlers to pass `entity_name` into `fire_entity_event`
|
||||
(the entity object is already loaded for the 404 check). Cover the representative/most-used
|
||||
entities at minimum: output targets, sync clocks, devices, picture/audio/color-strip
|
||||
sources, automations, scene presets/playlists, templates, gradients. (Create/update can rely
|
||||
on hook resolution but pass the name where trivially available.)
|
||||
|
||||
### Authentication (DESCOPED: no key create/rotate/revoke — those routes don't exist)
|
||||
- [x] In `api/auth.py`, record:
|
||||
- auth **failures**: missing/invalid Bearer token (HTTP), rejected LAN-without-keys, rejected
|
||||
WS origin (4403), WS auth handshake failure (4401). Category `auth`, severity `warning`.
|
||||
Include the caller IP/label and the reason in `metadata` — **never** the attempted token.
|
||||
- WS **session establishment** (successful `accept_and_authenticate_ws`): category `auth`,
|
||||
severity `info`, actor = authenticated label.
|
||||
- (Do NOT record per-request HTTP auth *success* — too frequent.)
|
||||
|
||||
### Device connect/disconnect (use existing discrete seams)
|
||||
- [x] Hook `device_health_changed` (`core/processing/device_health.py`, fired only on
|
||||
`online != prev_online`) → record online/offline transition. Category `device`,
|
||||
severity `info` (online) / `warning` (offline).
|
||||
- [x] Hook `device_discovered` / `device_lost` (`core/devices/discovery_watcher.py`, **runs on
|
||||
the zeroconf thread** → recorder must marshal to the loop, which Phase 2 handles). Category
|
||||
`device`.
|
||||
- [x] ADB connect/disconnect (`api/routes/system_settings.py:adb_connect/adb_disconnect`).
|
||||
|
||||
### Capture & system events (explicit record calls)
|
||||
- [x] Target processing start/stop + bulk (`api/routes/output_targets_control.py`).
|
||||
- [x] Scene activation (`scene_presets.py:activate_scene_preset`), playlist start/stop
|
||||
(`scene_playlists.py`), automation activate/deactivate (`automation_engine.py`).
|
||||
- [x] System: backup create/restore/delete (`backup.py`), update apply/dismiss (`update.py`),
|
||||
restart/shutdown (`backup.py`), calibration start/stop/cancel (`calibration.py`).
|
||||
- [x] Settings changes: scope to high-value settings only (auto-backup, update, shutdown
|
||||
action). **Exclude the activity-log's own `"activity_log"` settings key** to avoid
|
||||
self-referential churn.
|
||||
|
||||
### Tests
|
||||
- [x] `server/tests/test_activity_instrumentation.py` (or per-area):
|
||||
- representative entity create/update/delete produces a record with correct category/actor/
|
||||
name (incl. a delete carrying its name);
|
||||
- an auth failure produces a `warning` record and the token never appears in any field;
|
||||
- a device health transition and a discovery event produce records;
|
||||
- a capture start and a backup/restore produce records.
|
||||
|
||||
## Files to Modify/Create
|
||||
|
||||
- `server/src/ledgrab/api/dependencies.py` — modify: `fire_entity_event` records + `entity_name`
|
||||
- entity **delete** route handlers under `api/routes/` — modify: pass `entity_name`
|
||||
- `server/src/ledgrab/api/auth.py` — modify: auth-failure + WS-session records
|
||||
- `server/src/ledgrab/core/processing/device_health.py` — modify: online/offline record
|
||||
- `server/src/ledgrab/core/devices/discovery_watcher.py` — modify: discovered/lost record
|
||||
- `server/src/ledgrab/api/routes/system_settings.py` — modify: ADB + settings records
|
||||
- `server/src/ledgrab/api/routes/output_targets_control.py` — modify: start/stop records
|
||||
- `server/src/ledgrab/api/routes/{scene_presets,scene_playlists,backup,update,calibration}.py` — modify
|
||||
- `server/src/ledgrab/core/automations/automation_engine.py` — modify: activate/deactivate records
|
||||
- `server/tests/test_activity_instrumentation.py` — new
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- All four categories emit records at the named sites; entity deletes carry the entity name.
|
||||
- API-key tokens / secrets never appear in any audit field (test-enforced).
|
||||
- Recording never breaks the audited action (best-effort; failures swallowed + logged).
|
||||
- Actor is the authenticated label for request-originated events, `"system"` for engine/thread
|
||||
events. New + existing tests green; `ruff` clean.
|
||||
|
||||
## Notes
|
||||
|
||||
- Get the recorder via the Phase 2 DI getter; for engine/thread sites that lack DI, use the
|
||||
module singleton/accessor Phase 2 exposes.
|
||||
- Keep messages human-readable and localized-agnostic (English source strings; the frontend
|
||||
renders structured fields, not server message translation — message is a fallback/summary).
|
||||
- This is the security-sensitive phase — the security reviewer runs here AND at final review.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- [x] All tasks completed
|
||||
- [x] Code follows project conventions
|
||||
- [x] No unintended side effects (audited actions still succeed on recorder failure)
|
||||
- [x] No secrets logged (token never recorded) — explicitly verified
|
||||
- [x] Build passes (ruff + pytest)
|
||||
- [x] Tests pass (new + existing)
|
||||
|
||||
## Handoff to Next Phase
|
||||
|
||||
Phase 3 is complete. The following (category, action) pairs are now emitted, along with their
|
||||
metadata keys, for Phase 4 to expose via query/filter and for Phase 5 quick-filter presets.
|
||||
|
||||
### `entity` category
|
||||
|
||||
| Action | Severity | Metadata keys | Notes |
|
||||
|--------|----------|---------------|-------|
|
||||
| `entity.created` | info | — | All entity types via `fire_entity_event` choke-point |
|
||||
| `entity.updated` | info | — | All entity types; name resolved from store when not passed |
|
||||
| `entity.deleted` | info | — | Name passed explicitly by delete handler before deletion |
|
||||
|
||||
### `auth` category
|
||||
|
||||
| Action | Severity | Metadata keys | Notes |
|
||||
|--------|----------|---------------|-------|
|
||||
| `auth.rejected` | warning | `reason` (str), `client` (str/IP) | Missing Bearer, invalid Bearer, LAN-no-keys, WS origin, WS auth timeout, invalid WS token |
|
||||
| `auth.ws_connected` | info | `client` (str/IP) | Successful WS session established |
|
||||
|
||||
### `device` category
|
||||
|
||||
| Action | Severity | Metadata keys | Notes |
|
||||
|--------|----------|---------------|-------|
|
||||
| `device.online` | info | `latency_ms` (float) | Health monitor, transition only |
|
||||
| `device.offline` | warning | `latency_ms` (float) | Health monitor, transition only |
|
||||
| `device.discovered` | info | `url` (str), `device_type` (str) | Zeroconf discovery thread; recorder marshals to loop |
|
||||
| `device.lost` | warning | `url` (str), `device_type` (str) | Zeroconf discovery thread |
|
||||
| `device.adb_connected` | info | `address` (str) | ADB route success |
|
||||
| `device.adb_disconnected` | info | `address` (str) | ADB route success |
|
||||
|
||||
### `capture` category
|
||||
|
||||
| Action | Severity | Metadata keys | Notes |
|
||||
|--------|----------|---------------|-------|
|
||||
| `capture.started` | info | — | Per target (individual + bulk) |
|
||||
| `capture.stopped` | info | — | Per target (individual + bulk) |
|
||||
| `scene.activated` | info | — | `scene_presets.py:activate_scene_preset` |
|
||||
| `playlist.started` | info | — | `scene_playlists.py:start_scene_playlist` |
|
||||
| `playlist.stopped` | info | — | `scene_playlists.py:stop_scene_playlist` |
|
||||
| `automation.activated` | info | — | `automation_engine.py:_activate_automation`; actor="system" |
|
||||
| `automation.deactivated` | info | — | `automation_engine.py:_deactivate_automation`; actor="system" |
|
||||
|
||||
### `system` category
|
||||
|
||||
| Action | Severity | Metadata keys | Notes |
|
||||
|--------|----------|---------------|-------|
|
||||
| `backup.created` | info | `filename` (str) | `backup.py:backup_config` |
|
||||
| `backup.restored` | info | — | `backup.py:restore_config` |
|
||||
| `backup.deleted` | info | `filename` (str) | `backup.py:delete_saved_backup` |
|
||||
| `server.restarting` | info | — | `backup.py:restart_server` |
|
||||
| `server.shutdown_requested` | info | — | `backup.py:shutdown_server` |
|
||||
| `update.dismissed` | info | `version` (str) | `update.py:dismiss_update` |
|
||||
| `update.applied` | info | `version` (str) | `update.py:apply_update` |
|
||||
| `settings.changed` | info | `setting_key` (str) + setting-specific keys | `setting_key` values: `"auto_backup"`, `"update"`, `"shutdown_action"`. Activity-log own key excluded. |
|
||||
| `calibration.started` | info | — | `calibration.py`; entity_type="device", entity_id=device_id |
|
||||
| `calibration.stopped` | info | — | `calibration.py` |
|
||||
| `calibration.cancelled` | info | — | `calibration.py` |
|
||||
|
||||
### Implementation notes for Phase 4
|
||||
|
||||
- The `metadata` field is a JSON `TEXT` column. All keys above are scalars (str, float).
|
||||
- Phase 4 filter `metadata_key` / `metadata_value` lookup, if added, can target `setting_key`
|
||||
for settings-change filtering.
|
||||
- `entity_type` is populated for entity CRUD and `calibration.started`. For auth/system/capture
|
||||
events `entity_type` may be None.
|
||||
- `entity_name` is always populated for `entity.deleted`; populated for CRUD create/update
|
||||
when resolved; populated for most capture/system events where a name is meaningful.
|
||||
@@ -0,0 +1,163 @@
|
||||
# Phase 4: REST API — query / filter / export / settings / clear
|
||||
|
||||
**Status:** ✅ Done
|
||||
**Parent plan:** [PLAN.md](./PLAN.md)
|
||||
**Domain:** backend
|
||||
|
||||
## Objective
|
||||
|
||||
Expose the audit log over the REST API: a filtered, keyset-paginated list endpoint; a
|
||||
streaming CSV/JSON export honoring the same filters; retention settings get/update; and a
|
||||
destructive clear. Apply the project's auth posture (stricter auth on export + clear).
|
||||
|
||||
## Tasks
|
||||
|
||||
- [x] `server/src/ledgrab/api/schemas/activity_log.py` (Pydantic):
|
||||
- `ActivityLogEntryResponse` (matches the frozen Phase 2 entry dict shape).
|
||||
- `ActivityLogPageResponse` { `entries: list[...]`, `next_before_seq: int | None`,
|
||||
`total: int` (optional/over filters), `has_more: bool` }.
|
||||
- `ActivityLogSettingsResponse` / `UpdateActivityLogSettingsRequest`
|
||||
(`enabled`, `max_days`, `max_entries`) with validation bounds.
|
||||
- [x] `server/src/ledgrab/api/routes/activity_log.py` — `APIRouter(prefix="/api/v1/activity-log")`:
|
||||
- `GET ""` — list. Query params: `categories`, `severities`, `actor`, `entity_type`,
|
||||
`entity_id`, `since`/`until` (ISO), `q` (free-text), `before_seq` (cursor), `limit`
|
||||
(default 50, capped e.g. 200). `AuthRequired`. Maps params → `ActivityLogFilters`,
|
||||
calls `repo.query(...)`, returns page envelope.
|
||||
- `GET "/export"` — streaming export. Same filters; `format=csv|json`. Uses
|
||||
`StreamingResponse` over `repo.iter_export(...)`. **`require_authenticated()`** (may
|
||||
contain IPs/labels). Sets `Content-Disposition` with a timestamped filename.
|
||||
- `GET "/settings"` / `PUT "/settings"` — retention settings via the retention engine.
|
||||
`AuthRequired`; updates apply immediately.
|
||||
- `DELETE ""` — clear all entries. **`require_authenticated()`**. The clear is itself
|
||||
audited (recorder records a `system`/`activity_log_cleared` entry AFTER the wipe, so the
|
||||
log shows who cleared it and when).
|
||||
- Register the router in `server/src/ledgrab/api/__init__.py` (aggregator).
|
||||
- [x] API tests `server/tests/api/routes/test_activity_log_api.py`:
|
||||
- list returns entries; each filter narrows results; `before_seq` cursor paginates without
|
||||
overlap/gaps; `limit` cap enforced;
|
||||
- export CSV and JSON both stream and honor filters; export requires authentication
|
||||
(401 for loopback-anonymous when keys configured);
|
||||
- settings get/update round-trip + validation rejects out-of-range;
|
||||
- clear empties the log, requires auth, and leaves exactly one post-clear audit entry.
|
||||
|
||||
## Files to Modify/Create
|
||||
|
||||
- `server/src/ledgrab/api/schemas/activity_log.py` — new
|
||||
- `server/src/ledgrab/api/routes/activity_log.py` — new
|
||||
- `server/src/ledgrab/api/__init__.py` — modify: register router
|
||||
- `server/tests/api/routes/test_activity_log_api.py` — new
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- List is filterable on every dimension and keyset-paginated (stable, no dupes/gaps).
|
||||
- Export streams CSV + JSON, honors filters, and requires authentication.
|
||||
- Settings get/update works and validates bounds; changes take effect immediately.
|
||||
- Clear requires authentication and is itself audited.
|
||||
- New + existing tests green; `ruff` clean.
|
||||
|
||||
## Notes
|
||||
|
||||
- Auth helpers: `AuthRequired` dependency for normal endpoints; `require_authenticated()` for
|
||||
export + clear (pattern: backup download / secret reveal). See `api/auth.py` + `server/CLAUDE.md`.
|
||||
- Follow the existing route/schema conventions (one schema file per entity, router registered
|
||||
in `api/__init__.py`). Reference `api/routes/backup.py` for settings-style GET/PUT + a
|
||||
`StreamingResponse`/download pattern.
|
||||
- Reuse the entry→dict serializer from Phase 2 to keep the response shape single-sourced.
|
||||
- Backup/restore: no `STORE_MAP` change needed — backup is whole-DB; the table is auto-covered.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- [x] All tasks completed
|
||||
- [x] Code follows project conventions (router registration, schema-per-entity, auth posture)
|
||||
- [x] No unintended side effects
|
||||
- [x] Build passes (ruff + pytest)
|
||||
- [x] Tests pass (new + existing)
|
||||
|
||||
## Handoff to Next Phase
|
||||
|
||||
### Endpoint paths
|
||||
|
||||
| Method | Path | Auth |
|
||||
|--------|------|------|
|
||||
| GET | `/api/v1/activity-log` | AuthRequired (anonymous allowed) |
|
||||
| GET | `/api/v1/activity-log/export` | require_authenticated (no anonymous) |
|
||||
| GET | `/api/v1/activity-log/settings` | AuthRequired |
|
||||
| PUT | `/api/v1/activity-log/settings` | AuthRequired |
|
||||
| DELETE | `/api/v1/activity-log` | require_authenticated (no anonymous) |
|
||||
|
||||
### List query parameters (GET /api/v1/activity-log)
|
||||
|
||||
| Param | Type | Default | Notes |
|
||||
|-------|------|---------|-------|
|
||||
| `categories` | `list[str]` | — | Repeatable. Values: auth, device, entity, capture, system |
|
||||
| `severities` | `list[str]` | — | Repeatable. Values: info, warning, error |
|
||||
| `actor` | `str` | — | Exact match |
|
||||
| `entity_type` | `str` | — | Exact match |
|
||||
| `entity_id` | `str` | — | Exact match |
|
||||
| `since` | `datetime` (ISO-8601) | — | Inclusive lower bound on ts |
|
||||
| `until` | `datetime` (ISO-8601) | — | Inclusive upper bound on ts |
|
||||
| `q` | `str` | — | Substring match on message (LIKE %q%) |
|
||||
| `before_seq` | `int` | — | Keyset cursor from previous page's `next_before_seq` |
|
||||
| `limit` | `int` | 50 | Max entries per page. ge=1, le=200 |
|
||||
|
||||
Export endpoint (`GET /api/v1/activity-log/export`) accepts the same filter params plus `format=csv|json`.
|
||||
|
||||
### Page envelope fields (ActivityLogPageResponse)
|
||||
|
||||
```json
|
||||
{
|
||||
"entries": [...], // list[ActivityLogEntryResponse]
|
||||
"next_before_seq": 42, // int | null — pass as before_seq for next page
|
||||
"has_more": true, // bool
|
||||
"total": 1337 // int — total matching all filters (all pages)
|
||||
}
|
||||
```
|
||||
|
||||
### Entry dict shape (ActivityLogEntryResponse)
|
||||
|
||||
11 fields — identical to `entry_to_dict()` output:
|
||||
```json
|
||||
{
|
||||
"id": "al_abcd1234",
|
||||
"ts": "2026-06-09T12:34:56.789+00:00",
|
||||
"category": "entity",
|
||||
"action": "entity.created",
|
||||
"severity": "info",
|
||||
"actor": "my-api-key",
|
||||
"entity_type": "output_target",
|
||||
"entity_id": "pt_abc",
|
||||
"entity_name": "Desk",
|
||||
"message": "Output target 'Desk' created",
|
||||
"metadata": {}
|
||||
}
|
||||
```
|
||||
|
||||
### Settings field bounds (UpdateActivityLogSettingsRequest)
|
||||
|
||||
| Field | Type | ge | le | Notes |
|
||||
|-------|------|----|----|-------|
|
||||
| `enabled` | `bool` | — | — | Enable/disable recording |
|
||||
| `max_days` | `int` | 0 | 3650 | 0 = no age-based pruning |
|
||||
| `max_entries` | `int` | 0 | 10_000_000 | 0 = no count-based pruning |
|
||||
|
||||
### Export format param
|
||||
|
||||
`?format=csv` (default) → `text/csv; charset=utf-8`
|
||||
`?format=json` → `application/json` (streamed JSON array)
|
||||
|
||||
### Pagination algorithm
|
||||
|
||||
Keyset cursor (`before_seq`) works as follows:
|
||||
- Omit `before_seq` (or pass `null`) to get the FIRST (newest) page.
|
||||
- Each page response includes `next_before_seq` (the seq of the oldest entry on the page).
|
||||
- Pass `next_before_seq` as `before_seq` in the next request to get the following (older) page.
|
||||
- `has_more=false` means there are no more pages; `next_before_seq` is `null`.
|
||||
- `total` is constant across pages for the same filter set.
|
||||
|
||||
### New method added to ActivityLogRepository (additive, not breaking)
|
||||
|
||||
`get_seq_for_id(entry_id: str) -> int | None` — indexed point-lookup of seq by entry id.
|
||||
Used internally by the list endpoint to build the keyset cursor.
|
||||
|
||||
Phase 4 landed (2026-06-09): schemas, route (list/export/settings/clear), router registration,
|
||||
49 new tests — all green. Full suite 2486 passed, 2 skipped, 0 failed. Ruff clean.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Phase 5: Frontend — Activity tab + smart filtering + live updates
|
||||
|
||||
**Status:** ✅ Done
|
||||
**Parent plan:** [PLAN.md](./PLAN.md)
|
||||
**Domain:** frontend · uses the `frontend-design` skill
|
||||
|
||||
## Objective
|
||||
|
||||
Build the dedicated top-level **Activity** tab: a read-only, smart-filterable,
|
||||
keyset-paginated log viewer with an entry detail view, live-append of new events, and export.
|
||||
This is a viewer (Dashboard-style), NOT a CRUD card section.
|
||||
|
||||
## Tasks
|
||||
|
||||
- [x] `core/ui.ts`: add `formatTimestamp(isoOrMs)` (Today/Yesterday/Date · HH:MM, i18n-aware)
|
||||
and `formatRelativeTime(isoOrMs)` ("2m ago"), with `tabular-nums` styling guidance for the
|
||||
list. Reuse existing `time.*` i18n key conventions.
|
||||
- [x] `features/activity-log.ts`:
|
||||
- `export async function loadActivityLog()` — fetch first page from
|
||||
`GET /activity-log` (via `fetchWithAuth`), render the toolbar + list into the panel.
|
||||
- **Smart filter toolbar:** category (multi, chips), severity (chips), actor
|
||||
(text input), entity type, date range, free-text search (debounced). Quick presets:
|
||||
Today / Errors / Auth / Devices. Filters drive server-side query params (no client-side
|
||||
filtering of a partial page). Re-query on change; reset cursor.
|
||||
- **List:** one row per entry — severity icon, category badge, relative time (title=absolute),
|
||||
actor, message, entity crosslink (use `navigateToCard(...)` when the referenced entity is
|
||||
resolvable). Keyset "load more" (or infinite scroll) using `next_before_seq`.
|
||||
- **Detail:** expandable row / drawer showing full metadata JSON, exact timestamp, ids.
|
||||
- **Live append:** `document.addEventListener('server:activity_logged', e => …)` — prepend
|
||||
the new entry if it passes the active filters; show a subtle "new" affordance. (Depends on
|
||||
the Phase 2 allowlist entry — already shipped.)
|
||||
- **Export button:** triggers `GET /activity-log/export?format=…` with current filters via an
|
||||
authed blob download (use `fetchWithAuth` → blob URL, per frontend.md auth rules).
|
||||
- Empty / loading / error states; re-render on `languageChanged`.
|
||||
- [x] Tab wiring:
|
||||
- `core/tab-registry.ts`: add `activity_log: { loadFnName: 'loadActivityLog', autoRefresh: false }`.
|
||||
- `templates/index.html`: sidebar tab button (`data-tab`, `switchTab('activity_log')`,
|
||||
history/clock SVG icon, `data-i18n`) + `<div class="tab-panel" id="tab-activity_log">`.
|
||||
- `app.ts`: import + `Object.assign(window, { loadActivityLog, … })`; `global.d.ts` decls.
|
||||
- [x] Icons: add a history/audit icon to `core/icon-paths.ts` + `core/icons.ts`; severity icons
|
||||
(info/warning/error) reuse existing constants where possible.
|
||||
- [x] i18n: add `activity_log.*` keys to `static/locales/{en,ru,zh}.json` (title, filter labels,
|
||||
category/severity names, column labels, presets, empty/error, export, "N entries").
|
||||
- [x] Tutorials: add an Activity-tab step to the getting-started tour in
|
||||
`features/tutorials.ts` + `tour.*` keys in all 3 locales.
|
||||
|
||||
## Files to Modify/Create
|
||||
|
||||
- `server/src/ledgrab/static/js/core/ui.ts` — modified: timestamp/relative-time formatters
|
||||
- `server/src/ledgrab/static/js/features/activity-log.ts` — NEW: the viewer
|
||||
- `server/src/ledgrab/static/js/core/tab-registry.ts` — modified: register tab
|
||||
- `server/src/ledgrab/templates/index.html` — modified: tab button + panel
|
||||
- `server/src/ledgrab/static/js/app.ts` — modified: import + window globals
|
||||
- `server/src/ledgrab/static/js/global.d.ts` — modified: window decls
|
||||
- `server/src/ledgrab/static/js/core/icon-paths.ts` — modified: icons (scrollText, circleAlert, info, filter, xCircle)
|
||||
- `server/src/ledgrab/static/js/core/icons.ts` — modified: ICON_ACTIVITY_LOG, ICON_SEVERITY_*, ICON_FILTER, ICON_X_CIRCLE
|
||||
- `server/src/ledgrab/static/locales/{en,ru,zh}.json` — modified: activity_log.* + time.* + tour.activity_log
|
||||
- `server/src/ledgrab/static/js/features/tutorials.ts` — modified: gettingStartedSteps
|
||||
- `server/src/ledgrab/static/css/activity-log.css` — NEW: list + toolbar styling
|
||||
- `server/src/ledgrab/static/css/all.css` — modified: import activity-log.css
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] New **Activity** tab loads, lists entries, and paginates via keyset "load more".
|
||||
- [x] Filters hit server-side query params; quick presets work; free-text is debounced.
|
||||
- [x] New events append live via `server:activity_logged` and respect active filters.
|
||||
- [x] Export downloads CSV/JSON with auth, honoring current filters.
|
||||
- [x] Fully localized (en/ru/zh); empty/loading/error states; re-renders on language change.
|
||||
- [x] No plain `<select>` (use chips); SVG icons only (no emoji).
|
||||
- [x] `npx tsc --noEmit` clean; `npm run build` succeeds.
|
||||
|
||||
## Notes
|
||||
|
||||
- **Used the `frontend-design` skill** for the viewer layout, filter toolbar, and detail design.
|
||||
- Models mirrored: `features/dashboard.ts` (live viewer pattern), `core/events-ws.ts` (WS dispatch),
|
||||
`core/api.ts` (fetchWithAuth), `core/navigation.ts` (navigateToCard).
|
||||
- Frontend-only phase — ran `tsc --noEmit` + `npm run build`; did NOT run pytest/ruff.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- [x] All tasks completed
|
||||
- [x] Code follows frontend conventions (i18n ×3, icons, auth fetch, CSS vars)
|
||||
- [x] No unintended side effects
|
||||
- [x] Build passes (`tsc --noEmit` + `npm run build`)
|
||||
- [ ] Manual smoke: tab loads, filters query server, live append, export
|
||||
|
||||
## Handoff to Next Phase
|
||||
|
||||
### Reusable helpers for Phase 6 (Dashboard widget + Settings panel)
|
||||
|
||||
**From `features/activity-log.ts`:**
|
||||
|
||||
| Export | Purpose | How Phase 6 uses it |
|
||||
|--------|---------|---------------------|
|
||||
| `loadActivityLog()` | Loads the full tab (already registered as the tab loader) | Phase 6 doesn't call this, but it shares state |
|
||||
| `activityLogToggleDetail(id)` | Expand/collapse an entry row | Dashboard widget can reuse for the Recent Activity list |
|
||||
| `activityLogExport(format)` | Authed blob download with current filters | Could surface a direct link in the Settings panel |
|
||||
| `_renderEntryRow(entry, isNew)` | Renders a single entry as an HTML string | Dashboard widget should import this to render its 5-10 most-recent entries |
|
||||
| `_fetchPage(beforeSeq, append)` | Keyset-paged fetch from `/activity-log` | Dashboard widget calls with `limit=5&categories=…` for its mini-list |
|
||||
|
||||
**Note:** `_renderEntryRow` and `_fetchPage` are module-private (underscore prefix). Phase 6 should
|
||||
either (a) re-export them with public names, or (b) duplicate the minimal render logic for the
|
||||
compact widget format.
|
||||
|
||||
**Recommended approach for Phase 6:** Export two new public helpers:
|
||||
|
||||
```typescript
|
||||
// Add to activity-log.ts
|
||||
export async function fetchRecentEntries(limit = 5): Promise<ActivityEntry[]>
|
||||
export function renderCompactEntry(entry: ActivityEntry): string
|
||||
```
|
||||
|
||||
### i18n namespace
|
||||
|
||||
All keys are under `activity_log.*`. Time-relative formatters use `time.*` keys.
|
||||
|
||||
### CSS classes and tokens introduced
|
||||
|
||||
| Class | Purpose |
|
||||
|-------|---------|
|
||||
| `.al-panel` | Tab root wrapper |
|
||||
| `.al-toolbar` | Filter toolbar container |
|
||||
| `.al-chip` / `.al-chip.active` | Category/severity toggle chips |
|
||||
| `.al-preset-btn` | Quick-preset buttons |
|
||||
| `.al-entry` / `.al-entry-row` | Log entry row |
|
||||
| `.al-detail` / `.al-detail-grid` | Expandable entry detail |
|
||||
| `.al-cat-*` | Per-category badge colors (auth/device/entity/capture/system) |
|
||||
| `.al-sev-info/warning/error` | Severity icon color classes |
|
||||
| `.al-live-dot` | Pulsing green live-update dot |
|
||||
| `.al-meta-pre` | Scrollable metadata JSON block |
|
||||
| `.tabular-nums` | `font-variant-numeric: tabular-nums` utility |
|
||||
|
||||
### Settings endpoint shape used
|
||||
|
||||
Phase 6 Settings panel will call:
|
||||
- `GET /activity-log/settings` → `{ enabled: bool, max_days: int, max_entries: int }`
|
||||
- `PUT /activity-log/settings` → same shape (bounds: enabled=bool, max_days=0–3650, max_entries=0–10_000_000)
|
||||
@@ -0,0 +1,98 @@
|
||||
# Phase 6: Dashboard widget + Settings retention panel + docs
|
||||
|
||||
**Status:** ✅ Done
|
||||
**Parent plan:** [PLAN.md](./PLAN.md)
|
||||
**Domain:** frontend · uses the `frontend-design` skill
|
||||
|
||||
## Objective
|
||||
|
||||
Add the two secondary surfaces: a compact **Recent Activity** widget on the Dashboard linking
|
||||
to the full tab, and a **Settings** panel for retention configuration (+ clear + export entry)
|
||||
positioned beside the existing Log Viewer with a clear "what's the difference" note. Update
|
||||
docs/tutorials.
|
||||
|
||||
## Tasks
|
||||
|
||||
- [x] Dashboard **Recent Activity** widget (`features/dashboard.ts` + dashboard CSS):
|
||||
- Compact card showing the latest ~5 entries (severity icon, relative time, message).
|
||||
- Reuse the Phase 5 render helper (don't duplicate row markup).
|
||||
- Live update via `server:activity_logged` (prepend, cap to N).
|
||||
- **View all →** navigates to the Activity tab (`switchTab('activity_log')`).
|
||||
- Respect the existing dashboard card layout/toggle system; localized; empty state.
|
||||
- [x] **Settings** retention panel (`features/settings.ts` + `templates/modals/settings.html`):
|
||||
- New rail entry `Activity Log` (beside the existing **Log Viewer**).
|
||||
- Controls: `enabled` toggle, `max_days` number, `max_entries` number → `GET/PUT
|
||||
/activity-log/settings`. Save → toast; validation feedback.
|
||||
- **Clear log** button (confirm dialog) → `DELETE /activity-log`; **Export** button →
|
||||
`/activity-log/export`.
|
||||
- One-line note distinguishing this persistent audit log from the ephemeral debug Log Viewer
|
||||
(cross-link both ways).
|
||||
- i18n for all controls/labels/hints.
|
||||
- [x] Docs: update user-facing docs/README/feature list for the new Activity tab + retention
|
||||
settings + export (and the audit-vs-debug-log distinction). Keep it brief.
|
||||
- [x] Tutorials/cross-links: ensure the Settings tutorial (if any) and tab tour mention the
|
||||
panel; `tour.*`/`settings.*` i18n keys in all 3 locales.
|
||||
|
||||
## Files to Modify/Create
|
||||
|
||||
- `server/src/ledgrab/static/js/features/dashboard.ts` — modify: Recent Activity widget
|
||||
- `server/src/ledgrab/static/css/dashboard.css` (or relevant sheet) — modify: widget styles
|
||||
- `server/src/ledgrab/static/js/features/settings.ts` — modify: retention panel + handlers
|
||||
- `server/src/ledgrab/templates/modals/settings.html` — modify: rail entry + panel HTML
|
||||
- `server/src/ledgrab/static/js/app.ts` / `global.d.ts` — modify: new window handlers
|
||||
- `server/src/ledgrab/static/locales/{en,ru,zh}.json` — modify: i18n keys
|
||||
- docs (README / feature list / relevant context doc) — modify: brief feature documentation
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Dashboard shows a live Recent Activity widget; **View all →** opens the Activity tab.
|
||||
- Settings panel reads/writes retention settings, clears (with confirm + auth), and exports.
|
||||
- Audit-log vs debug-Log-Viewer distinction is explicit and cross-linked.
|
||||
- Fully localized (en/ru/zh); empty/loading states; consistent with app design.
|
||||
- `npx tsc --noEmit` clean; `npm run build` succeeds. No plain `<select>`; SVG icons only.
|
||||
|
||||
## Notes
|
||||
|
||||
- **Use the `frontend-design` skill** for the widget + settings panel layout.
|
||||
- Reuse Phase 5's render helper and i18n namespace — no duplicated row markup or keys.
|
||||
- Settings UI model: `features/settings.ts switchSettingsTab` + `modals/settings.html` rail.
|
||||
- Frontend-only phase → run `tsc --noEmit` + `npm run build`; do NOT run pytest/ruff (docs +
|
||||
TS only).
|
||||
- Backup/restore cross-ref: no `STORE_MAP` edit needed (whole-DB backup covers the table) —
|
||||
confirm nothing else (graph editor) needs syncing for this viewer (verified: not needed).
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- [x] All tasks completed
|
||||
- [x] Code follows frontend conventions (i18n ×3, icons, selectors, CSS vars, settings pattern)
|
||||
- [x] No unintended side effects
|
||||
- [x] Build passes (`tsc --noEmit` + `npm run build`)
|
||||
- [ ] Manual smoke: widget live-updates + links; settings save/clear/export; docs updated (recommend at final review — requires server restart)
|
||||
|
||||
## Handoff to Next Phase
|
||||
|
||||
This is the **final implementation phase**. All six phases are complete. Notes for the final reviewer:
|
||||
|
||||
**What was implemented in Phase 6:**
|
||||
|
||||
- `features/dashboard.ts`: `_loadRecentActivityWidget()`, `_renderRecentActivityList()`, `_startRecentActivityLive()` — SSE live-update listener (`server:activity_logged`) with cap-to-5 prepend logic. Widget appended after `getOrderedSections()` loop (outside the layout toggle system — always visible). Non-blocking: `.catch()` on the async load call.
|
||||
- `features/settings.ts`: `loadActivityLogSettings()`, `saveActivityLogSettings()`, `activityLogSettingsExport(format)`, `clearActivityLog()` — all exported and exposed on `window` via `app.ts` + `global.d.ts`.
|
||||
- `templates/modals/settings.html`: Activity Log rail entry (System group, cyan channel) + full panel (`id="settings-panel-activity_log"`) with enabled toggle, `max_days`/`max_entries` inputs, Save, CSV/JSON export, Clear (danger zone). Audit-vs-debug distinction note with cross-links in both directions (`closeSettingsModal(); openLogOverlay()` and `closeSettingsModal(); switchTab('activity_log')`).
|
||||
- `static/css/activity-log.css`: `.dal-*` dashboard widget styles + `.ds-info-note` / `.ds-inline-link` settings panel utilities appended (no new CSS file).
|
||||
- `static/locales/{en,ru,zh}.json`: 32 new keys each under `dashboard.section.recent_activity`, `dashboard.recent_activity.*`, `settings.tab.activity_log`, `settings.activity_log.*`.
|
||||
- `README.md`: "### Activity Log" section documenting tab, retention settings, export, and audit-vs-debug distinction.
|
||||
|
||||
**Reused Phase 5 helpers (no duplication):**
|
||||
|
||||
- `fetchRecentEntries(limit)` and `renderCompactEntry(entry)` — new public exports added to `activity-log.ts` in Phase 6 (not duplicated in dashboard.ts).
|
||||
- All `.al-*` CSS classes from Phase 5 are reused in the compact rows inside the widget.
|
||||
|
||||
**Build verification:** `tsc --noEmit` clean, `npm run build` passed (2.8 MB bundle, 258 ms) at time of implementation.
|
||||
|
||||
**Remaining manual smoke test (requires server restart):**
|
||||
|
||||
- Dashboard widget loads recent entries, prepends live on new activity, "View all →" switches to Activity tab.
|
||||
- Settings panel reads/writes retention, Save shows toast, Clear prompts confirm then deletes, Export downloads authed blob.
|
||||
- Cross-links: note in Settings opens Log Viewer overlay; note in Log Viewer links back to Activity tab.
|
||||
|
||||
**Outstanding open note from PLAN.md:** The manual browser smoke test across the whole feature (P5 note) remains open — it requires a server restart to exercise the live API endpoints. Recommend as first step in the final review session.
|
||||
@@ -38,6 +38,7 @@ from .routes.snapshot import router as snapshot_router
|
||||
from .routes.graph import router as graph_router
|
||||
from .routes.calibration import router as calibration_router
|
||||
from .routes.setup import router as setup_router
|
||||
from .routes.activity_log import router as activity_log_router
|
||||
|
||||
router = APIRouter()
|
||||
router.include_router(system_router)
|
||||
@@ -76,5 +77,6 @@ router.include_router(snapshot_router)
|
||||
router.include_router(graph_router)
|
||||
router.include_router(calibration_router)
|
||||
router.include_router(setup_router)
|
||||
router.include_router(activity_log_router)
|
||||
|
||||
__all__ = ["router"]
|
||||
|
||||
@@ -3,18 +3,111 @@
|
||||
import asyncio
|
||||
import json
|
||||
import secrets
|
||||
import time
|
||||
from typing import Annotated
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from fastapi import Depends, HTTPException, Request, Security, status
|
||||
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||
from starlette.websockets import WebSocket, WebSocketDisconnect
|
||||
|
||||
from ledgrab.config import get_config
|
||||
from ledgrab.core.activity_log.context import current_actor
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.utils import get_logger
|
||||
from ledgrab.utils.net_classify import is_loopback as _classify_is_loopback
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
# ── Auth-failure audit throttle (H3) ───────────────────────────────────────
|
||||
#
|
||||
# Unauthenticated callers can hammer any auth path; without a recording
|
||||
# throttle each attempt would write one SQLite row AND broadcast one WS event,
|
||||
# providing a cheap disk/broadcast amplification vector.
|
||||
#
|
||||
# Mitigation: record at most one ``auth.rejected`` audit entry per client IP
|
||||
# per _AUTH_RECORD_WINDOW seconds. The auth decision (401) is NEVER
|
||||
# suppressed — only the *audit recording* is de-duplicated.
|
||||
#
|
||||
# Memory safety: the throttle dict is capped at _AUTH_THROTTLE_HARD_CAP
|
||||
# entries. When the cap is exceeded the oldest-seen IP (lowest timestamp) is
|
||||
# evicted so the dict stays bounded regardless of the number of distinct source
|
||||
# IPs an attacker can forge.
|
||||
|
||||
_AUTH_RECORD_WINDOW: float = 10.0 # seconds — one record per IP per window
|
||||
_AUTH_THROTTLE_HARD_CAP: int = 512 # max IPs tracked simultaneously
|
||||
|
||||
# ip -> monotonic timestamp of last *recorded* auth.rejected entry
|
||||
_auth_record_last: dict[str, float] = {}
|
||||
|
||||
|
||||
def _should_record_auth_failure(client_ip: str) -> bool:
|
||||
"""Return True when an ``auth.rejected`` record should be written for *client_ip*.
|
||||
|
||||
Suppresses duplicates within _AUTH_RECORD_WINDOW seconds. Evicts the
|
||||
oldest entry when the dict exceeds _AUTH_THROTTLE_HARD_CAP to prevent
|
||||
unbounded memory growth under IP-spray attacks.
|
||||
"""
|
||||
now = time.monotonic()
|
||||
last = _auth_record_last.get(client_ip)
|
||||
if last is not None and (now - last) < _AUTH_RECORD_WINDOW:
|
||||
return False # suppress: within the de-dup window
|
||||
|
||||
# Enforce hard cap before inserting: evict the single oldest entry.
|
||||
if len(_auth_record_last) >= _AUTH_THROTTLE_HARD_CAP:
|
||||
oldest_ip = min(_auth_record_last, key=lambda ip: _auth_record_last[ip])
|
||||
del _auth_record_last[oldest_ip]
|
||||
|
||||
_auth_record_last[client_ip] = now
|
||||
return True
|
||||
|
||||
|
||||
def _record_auth_failure(reason: str, client_host: str | None) -> None:
|
||||
"""Best-effort: record an auth failure audit entry (never raises).
|
||||
|
||||
SECURITY: the attempted token is NEVER passed here; only the reason and
|
||||
the caller's IP/label are recorded.
|
||||
|
||||
THROTTLE: at most one ``auth.rejected`` record is written per client IP
|
||||
per _AUTH_RECORD_WINDOW seconds to prevent disk/WS-broadcast amplification
|
||||
DoS. The 401 response is always returned regardless.
|
||||
"""
|
||||
if not _should_record_auth_failure(client_host or "unknown"):
|
||||
return # throttled — drop duplicate recording for this IP/window
|
||||
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is None:
|
||||
return
|
||||
rec.record(
|
||||
category=ActivityCategory.AUTH,
|
||||
action="auth.rejected",
|
||||
severity=ActivitySeverity.WARNING,
|
||||
actor="anonymous",
|
||||
message=f"Authentication failed: {reason}",
|
||||
metadata={"reason": reason, "client": client_host or "unknown"},
|
||||
)
|
||||
|
||||
|
||||
def _record_ws_auth_success(label: str, client_host: str | None) -> None:
|
||||
"""Best-effort: record a successful WebSocket session establishment."""
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is None:
|
||||
return
|
||||
rec.record(
|
||||
category=ActivityCategory.AUTH,
|
||||
action="auth.ws_connected",
|
||||
severity=ActivitySeverity.INFO,
|
||||
actor=label,
|
||||
message=f"WebSocket session established by '{label}'",
|
||||
metadata={"client": client_host or "unknown"},
|
||||
)
|
||||
|
||||
|
||||
# Security scheme for Bearer token
|
||||
security = HTTPBearer(auto_error=False)
|
||||
|
||||
@@ -81,10 +174,12 @@ def verify_api_key(
|
||||
# No keys configured — allow loopback only.
|
||||
if _is_loopback(client_host):
|
||||
request.state.auth_label = "anonymous"
|
||||
current_actor.set("anonymous")
|
||||
return "anonymous"
|
||||
# Allow caller to authenticate explicitly even without configured keys?
|
||||
# No — there are no keys to compare against. Reject.
|
||||
logger.warning("Rejected LAN request from %s: no API key configured", client_host)
|
||||
_record_auth_failure("LAN access rejected: no API key configured", client_host)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail=(
|
||||
@@ -97,13 +192,14 @@ def verify_api_key(
|
||||
# Check if credentials are provided
|
||||
if not credentials:
|
||||
logger.warning("Request missing Authorization header")
|
||||
_record_auth_failure("missing Bearer token", client_host)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Missing API key - authentication is required",
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
|
||||
# Extract token
|
||||
# Extract token — NEVER log or record the token value itself.
|
||||
token = credentials.credentials
|
||||
|
||||
# Find matching key and return its label using constant-time comparison
|
||||
@@ -115,6 +211,7 @@ def verify_api_key(
|
||||
|
||||
if not authenticated_as:
|
||||
logger.warning("Invalid API key attempt")
|
||||
_record_auth_failure("invalid Bearer token", client_host)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid API key",
|
||||
@@ -127,6 +224,9 @@ def verify_api_key(
|
||||
# Stash the friendly label so the access-log middleware can attribute the
|
||||
# request to a client without re-running the token comparison.
|
||||
request.state.auth_label = authenticated_as
|
||||
# Set the actor ContextVar so ActivityRecorder can resolve it without
|
||||
# threading it through every call site.
|
||||
current_actor.set(authenticated_as)
|
||||
return authenticated_as
|
||||
|
||||
|
||||
@@ -190,12 +290,30 @@ async def accept_and_authenticate_ws(websocket: WebSocket, timeout: float = 3.0)
|
||||
# a strong signal even before the token check. Non-browser clients
|
||||
# legitimately omit Origin; those fall through to the auth handshake.
|
||||
config = get_config()
|
||||
client_host = websocket.client.host if websocket.client else None
|
||||
origin = websocket.headers.get("origin")
|
||||
if not _is_origin_allowed(origin, config.server.cors_origins):
|
||||
logger.warning(
|
||||
"Rejected WebSocket from origin %r (not in cors_origins)",
|
||||
origin,
|
||||
)
|
||||
# Sanitize first so urlparse does not choke on control chars / ANSI / NUL
|
||||
# embedded by an attacker in the Origin header (e.g. \n triggers IPv6 parse
|
||||
# error in Python's urlsplit on malformed netloc).
|
||||
_safe_origin_raw = sanitize_display(origin) if origin else ""
|
||||
try:
|
||||
_netloc = urlparse(_safe_origin_raw).netloc if _safe_origin_raw else ""
|
||||
except ValueError:
|
||||
# Malformed IPv6 addresses (e.g. "http://[::1" without closing "]")
|
||||
# cause urlparse to raise ValueError. Fall back to "unknown" — do NOT
|
||||
# fall back to the raw origin string, which could carry query params
|
||||
# or path components containing secrets.
|
||||
_netloc = ""
|
||||
_safe_origin = sanitize_display(_netloc or "unknown")
|
||||
_record_auth_failure(
|
||||
f"WebSocket origin rejected: {_safe_origin!r}",
|
||||
client_host,
|
||||
)
|
||||
try:
|
||||
await websocket.close(code=WS_ORIGIN_CLOSE_CODE)
|
||||
except _WS_SEND_BENIGN_EXC:
|
||||
@@ -210,6 +328,7 @@ async def accept_and_authenticate_ws(websocket: WebSocket, timeout: float = 3.0)
|
||||
except _WS_SEND_BENIGN_EXC:
|
||||
pass
|
||||
return None
|
||||
_record_ws_auth_success(label, client_host)
|
||||
return label
|
||||
|
||||
|
||||
@@ -275,6 +394,7 @@ async def verify_ws_auth(
|
||||
return None
|
||||
return "anonymous"
|
||||
logger.warning("WebSocket auth timeout after %.1fs from %s", timeout, client_host)
|
||||
_record_auth_failure("WebSocket auth timeout", client_host)
|
||||
try:
|
||||
await websocket.send_json({"type": "auth_error", "reason": "auth timeout"})
|
||||
except _WS_SEND_BENIGN_EXC:
|
||||
@@ -332,6 +452,7 @@ async def verify_ws_auth(
|
||||
await websocket.send_json({"type": "auth_ok"})
|
||||
return "anonymous"
|
||||
logger.warning("Rejected LAN WebSocket from %s: no API key configured", client_host)
|
||||
_record_auth_failure("LAN WebSocket rejected: no API key configured", client_host)
|
||||
try:
|
||||
await websocket.send_json(
|
||||
{
|
||||
@@ -343,10 +464,11 @@ async def verify_ws_auth(
|
||||
pass
|
||||
return None
|
||||
|
||||
# Keys configured: require a matching token.
|
||||
# Keys configured: require a matching token. NEVER log the token value.
|
||||
label = _match_api_key(token or "")
|
||||
if not label:
|
||||
logger.warning("Invalid WebSocket auth attempt from %s", client_host)
|
||||
_record_auth_failure("invalid WebSocket token", client_host)
|
||||
try:
|
||||
await websocket.send_json({"type": "auth_error", "reason": "invalid token"})
|
||||
except _WS_SEND_BENIGN_EXC:
|
||||
|
||||
@@ -42,6 +42,11 @@ from ledgrab.core.mqtt.mqtt_manager import MQTTManager
|
||||
from ledgrab.storage.http_endpoint_store import HTTPEndpointStore
|
||||
from ledgrab.storage.audio_processing_template_store import AudioProcessingTemplateStore
|
||||
from ledgrab.storage.pattern_template_store import PatternTemplateStore
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder, get_module_recorder
|
||||
from ledgrab.core.activity_log.retention import ActivityLogRetentionEngine
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
@@ -196,16 +201,83 @@ def get_update_service() -> UpdateService:
|
||||
return _get("update_service", "Update service")
|
||||
|
||||
|
||||
def get_activity_recorder() -> ActivityRecorder:
|
||||
return _get("activity_recorder", "Activity recorder")
|
||||
|
||||
|
||||
def get_activity_log_repo() -> ActivityLogRepository:
|
||||
return _get("activity_log_repo", "Activity log repository")
|
||||
|
||||
|
||||
def get_activity_log_retention_engine() -> ActivityLogRetentionEngine:
|
||||
return _get("activity_log_retention_engine", "Activity log retention engine")
|
||||
|
||||
|
||||
# ── Event helper ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def fire_entity_event(entity_type: str, action: str, entity_id: str) -> None:
|
||||
"""Fire an entity_changed event via the ProcessorManager event bus.
|
||||
def _resolve_entity_name(entity_type: str, entity_id: str) -> str | None:
|
||||
"""Best-effort: look up a human name for *entity_id* from the matching store.
|
||||
|
||||
Returns ``None`` when the store is missing, the entity is gone, or any
|
||||
exception occurs (e.g. during delete the entity may have just been removed).
|
||||
"""
|
||||
# Map entity_type → (_deps key, method name on the store)
|
||||
_STORE_LOOKUP: dict[str, tuple[str, str]] = {
|
||||
"output_target": ("output_target_store", "get_target"),
|
||||
"device": ("device_store", "get_device"),
|
||||
"picture_source": ("picture_source_store", "get_source"),
|
||||
"audio_source": ("audio_source_store", "get_source"),
|
||||
"color_strip_source": ("color_strip_store", "get_source"),
|
||||
"template": ("template_store", "get_template"),
|
||||
"capture_template": ("template_store", "get_template"),
|
||||
"pp_template": ("pp_template_store", "get_template"),
|
||||
"automation": ("automation_store", "get_automation"),
|
||||
"scene_preset": ("scene_preset_store", "get_preset"),
|
||||
"scene_playlist": ("scene_playlist_store", "get_playlist"),
|
||||
"sync_clock": ("sync_clock_store", "get_clock"),
|
||||
"gradient": ("gradient_store", "get_gradient"),
|
||||
"audio_template": ("audio_template_store", "get_template"),
|
||||
"value_source": ("value_source_store", "get_source"),
|
||||
"cspt": ("cspt_store", "get_template"),
|
||||
"audio_processing_template": ("audio_processing_template_store", "get_template"),
|
||||
"pattern_template": ("pattern_template_store", "get_template"),
|
||||
"home_assistant_source": ("ha_store", "get_source"),
|
||||
"mqtt_source": ("mqtt_store", "get_source"),
|
||||
"http_endpoint": ("http_endpoint_store", "get_endpoint"),
|
||||
}
|
||||
entry = _STORE_LOOKUP.get(entity_type)
|
||||
if entry is None:
|
||||
return None
|
||||
store_key, method_name = entry
|
||||
store = _deps.get(store_key)
|
||||
if store is None:
|
||||
return None
|
||||
try:
|
||||
obj = getattr(store, method_name)(entity_id)
|
||||
if obj is not None:
|
||||
return getattr(obj, "name", None)
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def fire_entity_event(
|
||||
entity_type: str,
|
||||
action: str,
|
||||
entity_id: str,
|
||||
entity_name: str | None = None,
|
||||
) -> None:
|
||||
"""Fire an entity_changed event via the ProcessorManager event bus and
|
||||
record an audit entry.
|
||||
|
||||
Args:
|
||||
entity_type: e.g. "device", "output_target", "color_strip_source"
|
||||
action: "created", "updated", or "deleted"
|
||||
entity_id: The entity's unique ID
|
||||
entity_name: Human-readable name. For deletes: **must** be passed
|
||||
explicitly (entity is already gone when we get here).
|
||||
For create/update: resolved from the store when not supplied.
|
||||
"""
|
||||
pm = _deps.get("processor_manager")
|
||||
if pm is not None:
|
||||
@@ -218,6 +290,38 @@ def fire_entity_event(entity_type: str, action: str, entity_id: str) -> None:
|
||||
}
|
||||
)
|
||||
|
||||
# ── Audit record (best-effort) ──────────────────────────────────────────
|
||||
rec = get_module_recorder()
|
||||
if rec is None:
|
||||
return
|
||||
|
||||
# Resolve name when not explicitly provided (create / update paths).
|
||||
# For deleted: entity already gone — rely on the explicitly passed name.
|
||||
resolved_name = entity_name
|
||||
if resolved_name is None and action != "deleted":
|
||||
resolved_name = _resolve_entity_name(entity_type, entity_id)
|
||||
|
||||
# Build a concise human message.
|
||||
# Sanitize the display name before interpolating into the free-text message
|
||||
# (user-authored names hit the CSV/export trust surface).
|
||||
safe_display_name = sanitize_display(resolved_name) if resolved_name else None
|
||||
display_name = f"'{safe_display_name}'" if safe_display_name else entity_id
|
||||
action_word = {"created": "created", "updated": "updated", "deleted": "deleted"}.get(
|
||||
action, action
|
||||
)
|
||||
entity_label = entity_type.replace("_", " ")
|
||||
message = f"{entity_label.capitalize()} {display_name} {action_word}"
|
||||
|
||||
rec.record(
|
||||
category=ActivityCategory.ENTITY,
|
||||
action=f"entity.{action}",
|
||||
severity=ActivitySeverity.INFO,
|
||||
entity_type=entity_type,
|
||||
entity_id=entity_id,
|
||||
entity_name=sanitize_display(resolved_name) if resolved_name else None,
|
||||
message=message,
|
||||
)
|
||||
|
||||
|
||||
# ── Initialization ──────────────────────────────────────────────────────
|
||||
|
||||
@@ -257,6 +361,9 @@ def init_dependencies(
|
||||
http_endpoint_store: HTTPEndpointStore | None = None,
|
||||
audio_processing_template_store: AudioProcessingTemplateStore | None = None,
|
||||
pattern_template_store: PatternTemplateStore | None = None,
|
||||
activity_recorder: ActivityRecorder | None = None,
|
||||
activity_log_repo: ActivityLogRepository | None = None,
|
||||
activity_log_retention_engine: ActivityLogRetentionEngine | None = None,
|
||||
):
|
||||
"""Initialize global dependencies."""
|
||||
_deps.update(
|
||||
@@ -295,5 +402,8 @@ def init_dependencies(
|
||||
"http_endpoint_store": http_endpoint_store,
|
||||
"audio_processing_template_store": audio_processing_template_store,
|
||||
"pattern_template_store": pattern_template_store,
|
||||
"activity_recorder": activity_recorder,
|
||||
"activity_log_repo": activity_log_repo,
|
||||
"activity_log_retention_engine": activity_log_retention_engine,
|
||||
}
|
||||
)
|
||||
|
||||
@@ -0,0 +1,436 @@
|
||||
"""Activity-log REST API — query / filter / export / settings / clear.
|
||||
|
||||
Endpoints
|
||||
---------
|
||||
GET /api/v1/activity-log List (filterable, keyset-paginated)
|
||||
GET /api/v1/activity-log/export Streaming CSV or JSON export
|
||||
GET /api/v1/activity-log/settings Retention settings
|
||||
PUT /api/v1/activity-log/settings Update retention settings (requires non-anonymous auth)
|
||||
DELETE /api/v1/activity-log Clear all entries (requires non-anonymous auth)
|
||||
|
||||
Auth posture
|
||||
------------
|
||||
- List + read settings (``GET``): ``AuthRequired`` (loopback-anonymous is fine).
|
||||
- Export, update settings (``PUT``), and clear: ``require_authenticated()``
|
||||
(loopback-anonymous is rejected; mirrors the backup download / secret-reveal
|
||||
pattern from ``backup.py``). Updating settings can disable auditing or prune
|
||||
the trail, so it is gated like the destructive clear.
|
||||
|
||||
CSV injection
|
||||
-------------
|
||||
Cells that begin with =, +, -, @, TAB, or CR can trigger formula execution in
|
||||
spreadsheet apps (OWASP Formula Injection). ``_csv_safe`` prefixes any such cell
|
||||
with a single quote so formulas are inert. Fields already go through
|
||||
``sanitize_display`` in Phase 3 instrumentation, but the CSV writer applies its
|
||||
own guard as defence-in-depth.
|
||||
|
||||
Export generator + lock
|
||||
-----------------------
|
||||
``repo.iter_export()`` fetches rows in bounded batches, holding the DB ``_lock``
|
||||
only around each batch fetch and releasing it before yielding — so a slow or
|
||||
stalled client never blocks other DB operations. The ``StreamingResponse``
|
||||
generator is wrapped in a ``try/finally`` block so the batch generator is closed
|
||||
even when the client disconnects mid-stream.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import io
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from typing import Annotated, Iterator
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from fastapi.responses import StreamingResponse
|
||||
|
||||
from ledgrab.api.auth import AuthRequired, require_authenticated
|
||||
from ledgrab.api.dependencies import (
|
||||
get_activity_log_repo,
|
||||
get_activity_log_retention_engine,
|
||||
get_activity_recorder,
|
||||
)
|
||||
from ledgrab.api.schemas.activity_log import (
|
||||
ActivityLogPageResponse,
|
||||
ActivityLogSettingsResponse,
|
||||
UpdateActivityLogSettingsRequest,
|
||||
)
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder, entry_to_dict
|
||||
from ledgrab.core.activity_log.retention import ActivityLogRetentionEngine
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivityLogFilters, ActivitySeverity
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
|
||||
router = APIRouter(prefix="/api/v1/activity-log", tags=["Activity Log"])
|
||||
|
||||
# Hard cap on the per-request limit to prevent runaway queries.
|
||||
_MAX_LIMIT = 200
|
||||
_DEFAULT_LIMIT = 50
|
||||
|
||||
# CSV export columns (matches entry_to_dict key order)
|
||||
_CSV_COLUMNS = [
|
||||
"id",
|
||||
"ts",
|
||||
"category",
|
||||
"action",
|
||||
"severity",
|
||||
"actor",
|
||||
"entity_type",
|
||||
"entity_id",
|
||||
"entity_name",
|
||||
"message",
|
||||
"metadata",
|
||||
]
|
||||
|
||||
# Characters that trigger formula injection in spreadsheet apps (OWASP).
|
||||
# Leading TAB and CR are also recognised triggers by Excel / Google Sheets.
|
||||
_FORMULA_PREFIXES = ("=", "+", "-", "@", "\t", "\r")
|
||||
|
||||
|
||||
def _csv_safe(value: str) -> str:
|
||||
"""Prefix formula-injection triggers with a literal single-quote.
|
||||
|
||||
A cell starting with =, +, -, or @ can execute as a formula in Excel /
|
||||
Google Sheets. OWASP recommends prepending a single quote to neutralise it.
|
||||
"""
|
||||
if value and value[0] in _FORMULA_PREFIXES:
|
||||
return "'" + value
|
||||
return value
|
||||
|
||||
|
||||
def _build_filters(
|
||||
categories: list[str] | None,
|
||||
severities: list[str] | None,
|
||||
actor: str | None,
|
||||
entity_type: str | None,
|
||||
entity_id: str | None,
|
||||
since: datetime | None,
|
||||
until: datetime | None,
|
||||
q: str | None,
|
||||
) -> ActivityLogFilters:
|
||||
"""Assemble an ``ActivityLogFilters`` dataclass from query parameters."""
|
||||
return ActivityLogFilters(
|
||||
categories=categories or None,
|
||||
severities=severities or None,
|
||||
actor=actor or None,
|
||||
entity_type=entity_type or None,
|
||||
entity_id=entity_id or None,
|
||||
since=since,
|
||||
until=until,
|
||||
message_like=q or None,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/v1/activity-log — list
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("", response_model=ActivityLogPageResponse, summary="List activity-log entries")
|
||||
def list_activity_log(
|
||||
auth: AuthRequired, # noqa: ARG001
|
||||
repo: ActivityLogRepository = Depends(get_activity_log_repo),
|
||||
# ── Filters ────────────────────────────────────────────────────────────
|
||||
categories: Annotated[
|
||||
list[str] | None,
|
||||
Query(
|
||||
description=(
|
||||
"Filter by category (repeatable or comma-separated). "
|
||||
"Values: auth, device, entity, capture, system"
|
||||
)
|
||||
),
|
||||
] = None,
|
||||
severities: Annotated[
|
||||
list[str] | None,
|
||||
Query(description="Filter by severity (repeatable). Values: info, warning, error"),
|
||||
] = None,
|
||||
actor: Annotated[
|
||||
str | None,
|
||||
Query(description="Filter by actor label (exact match)"),
|
||||
] = None,
|
||||
entity_type: Annotated[
|
||||
str | None,
|
||||
Query(description="Filter by entity type (exact match)"),
|
||||
] = None,
|
||||
entity_id: Annotated[
|
||||
str | None,
|
||||
Query(description="Filter by entity id (exact match)"),
|
||||
] = None,
|
||||
since: Annotated[
|
||||
datetime | None,
|
||||
Query(description="Return entries at or after this ISO-8601 datetime"),
|
||||
] = None,
|
||||
until: Annotated[
|
||||
datetime | None,
|
||||
Query(description="Return entries at or before this ISO-8601 datetime"),
|
||||
] = None,
|
||||
q: Annotated[
|
||||
str | None,
|
||||
Query(description="Free-text search in the message field (substring)"),
|
||||
] = None,
|
||||
# ── Pagination ─────────────────────────────────────────────────────────
|
||||
before_seq: Annotated[
|
||||
int | None,
|
||||
Query(
|
||||
description=(
|
||||
"Keyset cursor: pass the 'next_before_seq' from the previous page "
|
||||
"to get the following (older) page. Omit for the first (newest) page."
|
||||
)
|
||||
),
|
||||
] = None,
|
||||
limit: Annotated[
|
||||
int,
|
||||
Query(
|
||||
ge=1,
|
||||
le=_MAX_LIMIT,
|
||||
description=f"Max entries per page (default {_DEFAULT_LIMIT}, max {_MAX_LIMIT})",
|
||||
),
|
||||
] = _DEFAULT_LIMIT,
|
||||
) -> ActivityLogPageResponse:
|
||||
"""Return the newest matching entries, oldest-first within the page.
|
||||
|
||||
Keyset pagination: the response includes ``next_before_seq`` — pass it
|
||||
as ``before_seq`` in the next request to get the next (older) page.
|
||||
The ``total`` field is the count of all entries matching the current
|
||||
filters across all pages.
|
||||
"""
|
||||
filters = _build_filters(categories, severities, actor, entity_type, entity_id, since, until, q)
|
||||
|
||||
# Fetch limit+1 rows to detect whether an older page exists.
|
||||
#
|
||||
# query() fetches DESC internally (newest-first) then reverses to ascending.
|
||||
# With limit+1, the result is ascending: [oldest_probe, ..., newest].
|
||||
# When we got exactly limit+1 rows, has_more is True and the probe row
|
||||
# (index 0 — the oldest) is the extra one. We keep the newest `limit` rows
|
||||
# by slicing [1:], which is the actual page content for the client.
|
||||
# When we got <= limit rows, this is the last page and all rows are included.
|
||||
effective_limit = min(limit, _MAX_LIMIT)
|
||||
entries_plus = repo.query(filters, before_seq=before_seq, limit=effective_limit + 1)
|
||||
has_more = len(entries_plus) > effective_limit
|
||||
if has_more:
|
||||
# Drop the oldest probe row; keep the newest `limit` entries.
|
||||
entries = entries_plus[1:]
|
||||
else:
|
||||
entries = entries_plus
|
||||
|
||||
total = repo.count(filters)
|
||||
|
||||
# Compute next_before_seq: the seq of the oldest entry on this page.
|
||||
# query() returns entries ascending (entries[0] is oldest); its seq is the
|
||||
# cursor for the next page. The next request passes before_seq=X to get
|
||||
# entries with seq < X, i.e. entries older than the oldest entry on this page.
|
||||
# get_seq_for_id() does a cheap indexed point-lookup.
|
||||
next_before_seq: int | None = None
|
||||
if has_more and entries:
|
||||
next_before_seq = repo.get_seq_for_id(entries[0].id)
|
||||
|
||||
return ActivityLogPageResponse(
|
||||
entries=[entry_to_dict(e) for e in entries], # type: ignore[arg-type]
|
||||
next_before_seq=next_before_seq,
|
||||
has_more=has_more,
|
||||
total=total,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/v1/activity-log/export — streaming export (CSV or JSON)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _export_csv_generator(
|
||||
repo: ActivityLogRepository,
|
||||
filters: ActivityLogFilters,
|
||||
) -> Iterator[bytes]:
|
||||
"""Yield UTF-8-encoded CSV chunks one row at a time.
|
||||
|
||||
The generator wraps ``repo.iter_export()`` in a ``try/finally`` so the DB
|
||||
lock is released even on early client disconnect (which triggers
|
||||
``GeneratorExit``).
|
||||
"""
|
||||
gen = repo.iter_export(filters)
|
||||
try:
|
||||
# Header
|
||||
buf = io.StringIO()
|
||||
writer = csv.writer(buf)
|
||||
writer.writerow(_CSV_COLUMNS)
|
||||
yield buf.getvalue().encode("utf-8")
|
||||
|
||||
for entry in gen:
|
||||
d = entry_to_dict(entry)
|
||||
row = []
|
||||
for col in _CSV_COLUMNS:
|
||||
if col == "metadata":
|
||||
cell = json.dumps(d.get(col) or {})
|
||||
else:
|
||||
cell = str(d.get(col, "") or "")
|
||||
row.append(_csv_safe(cell))
|
||||
buf = io.StringIO()
|
||||
writer = csv.writer(buf)
|
||||
writer.writerow(row)
|
||||
yield buf.getvalue().encode("utf-8")
|
||||
finally:
|
||||
gen.close()
|
||||
|
||||
|
||||
def _export_json_generator(
|
||||
repo: ActivityLogRepository,
|
||||
filters: ActivityLogFilters,
|
||||
) -> Iterator[bytes]:
|
||||
"""Yield a streamed JSON array, one entry per chunk.
|
||||
|
||||
Format: ``[\\n{entry},\\n{entry},\\n...]\\n``
|
||||
The generator wraps ``repo.iter_export()`` in a ``try/finally`` so the DB
|
||||
lock is released even on early client disconnect.
|
||||
"""
|
||||
gen = repo.iter_export(filters)
|
||||
try:
|
||||
first = True
|
||||
yield b"[\n"
|
||||
for entry in gen:
|
||||
d = entry_to_dict(entry)
|
||||
chunk = json.dumps(d, ensure_ascii=False, default=str)
|
||||
if first:
|
||||
yield chunk.encode("utf-8")
|
||||
first = False
|
||||
else:
|
||||
yield b",\n" + chunk.encode("utf-8")
|
||||
yield b"\n]\n"
|
||||
finally:
|
||||
gen.close()
|
||||
|
||||
|
||||
@router.get("/export", summary="Export activity-log entries (streaming CSV or JSON)")
|
||||
def export_activity_log(
|
||||
auth: AuthRequired,
|
||||
repo: ActivityLogRepository = Depends(get_activity_log_repo),
|
||||
# ── Format ────────────────────────────────────────────────────────────
|
||||
format: Annotated[
|
||||
str,
|
||||
Query(description="Export format: 'csv' or 'json'"),
|
||||
] = "csv",
|
||||
# ── Same filters as list ───────────────────────────────────────────────
|
||||
categories: Annotated[list[str] | None, Query()] = None,
|
||||
severities: Annotated[list[str] | None, Query()] = None,
|
||||
actor: Annotated[str | None, Query()] = None,
|
||||
entity_type: Annotated[str | None, Query()] = None,
|
||||
entity_id: Annotated[str | None, Query()] = None,
|
||||
since: Annotated[datetime | None, Query()] = None,
|
||||
until: Annotated[datetime | None, Query()] = None,
|
||||
q: Annotated[str | None, Query()] = None,
|
||||
) -> StreamingResponse:
|
||||
"""Stream all matching entries as CSV or JSON.
|
||||
|
||||
Requires a non-anonymous API key (loopback-anonymous access is rejected
|
||||
because the log may contain IP addresses and entity names).
|
||||
"""
|
||||
require_authenticated(auth)
|
||||
|
||||
if format not in ("csv", "json"):
|
||||
from fastapi import HTTPException
|
||||
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="'format' must be 'csv' or 'json'",
|
||||
)
|
||||
|
||||
filters = _build_filters(categories, severities, actor, entity_type, entity_id, since, until, q)
|
||||
|
||||
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H%M%S")
|
||||
|
||||
if format == "csv":
|
||||
filename = f"activity-log-{timestamp}.csv"
|
||||
media_type = "text/csv; charset=utf-8"
|
||||
generator = _export_csv_generator(repo, filters)
|
||||
else:
|
||||
filename = f"activity-log-{timestamp}.json"
|
||||
media_type = "application/json"
|
||||
generator = _export_json_generator(repo, filters)
|
||||
|
||||
return StreamingResponse(
|
||||
generator,
|
||||
media_type=media_type,
|
||||
headers={"Content-Disposition": f'attachment; filename="{filename}"'},
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/v1/activity-log/settings
|
||||
# PUT /api/v1/activity-log/settings
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get(
|
||||
"/settings",
|
||||
response_model=ActivityLogSettingsResponse,
|
||||
summary="Get activity-log retention settings",
|
||||
)
|
||||
def get_activity_log_settings(
|
||||
_: AuthRequired,
|
||||
engine: ActivityLogRetentionEngine = Depends(get_activity_log_retention_engine),
|
||||
) -> ActivityLogSettingsResponse:
|
||||
"""Return the current activity-log retention settings."""
|
||||
return ActivityLogSettingsResponse(**engine.get_settings())
|
||||
|
||||
|
||||
@router.put(
|
||||
"/settings",
|
||||
response_model=ActivityLogSettingsResponse,
|
||||
summary="Update activity-log retention settings",
|
||||
)
|
||||
async def update_activity_log_settings(
|
||||
auth: AuthRequired,
|
||||
body: UpdateActivityLogSettingsRequest,
|
||||
engine: ActivityLogRetentionEngine = Depends(get_activity_log_retention_engine),
|
||||
) -> ActivityLogSettingsResponse:
|
||||
"""Update the activity-log retention settings (applied immediately).
|
||||
|
||||
Requires a non-anonymous API key (loopback-anonymous access is rejected)
|
||||
because disabling the log or pruning retention is equivalent in impact to
|
||||
clearing the audit trail.
|
||||
|
||||
Setting ``enabled=false`` records an audit entry BEFORE the flag takes
|
||||
effect so the last entry in the log shows who disabled recording.
|
||||
"""
|
||||
require_authenticated(auth)
|
||||
result = await engine.update_settings(
|
||||
enabled=body.enabled,
|
||||
max_days=body.max_days,
|
||||
max_entries=body.max_entries,
|
||||
)
|
||||
return ActivityLogSettingsResponse(**result)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DELETE /api/v1/activity-log — clear
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.delete("", summary="Clear all activity-log entries")
|
||||
def clear_activity_log(
|
||||
auth: AuthRequired,
|
||||
repo: ActivityLogRepository = Depends(get_activity_log_repo),
|
||||
recorder: ActivityRecorder = Depends(get_activity_recorder),
|
||||
) -> dict:
|
||||
"""Delete all activity-log entries.
|
||||
|
||||
Requires a non-anonymous API key (loopback-anonymous access is rejected).
|
||||
The clear operation itself is audited — a ``system/activity_log_cleared``
|
||||
entry is recorded AFTER the wipe, so the log shows who cleared it and how
|
||||
many rows were removed.
|
||||
|
||||
Returns ``{"deleted": <count>}``.
|
||||
"""
|
||||
require_authenticated(auth)
|
||||
|
||||
deleted = repo.clear()
|
||||
|
||||
# Record the clear action (best-effort — recorder never raises).
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="activity_log.cleared",
|
||||
severity=ActivitySeverity.INFO,
|
||||
actor=auth,
|
||||
message=f"Activity log cleared ({deleted} entries removed)",
|
||||
metadata={"deleted_count": deleted},
|
||||
)
|
||||
|
||||
return {"deleted": deleted}
|
||||
@@ -182,6 +182,12 @@ async def delete_audio_source(
|
||||
css_store: ColorStripStore = Depends(get_color_strip_store),
|
||||
):
|
||||
"""Delete an audio source."""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_source(source_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
# Check if any CSS entities reference this audio source
|
||||
from ledgrab.storage.color_strip_source import AudioColorStripSource
|
||||
@@ -194,7 +200,7 @@ async def delete_audio_source(
|
||||
raise ValueError(f"Cannot delete: referenced by color strip source '{css.name}'")
|
||||
|
||||
store.delete_source(source_id)
|
||||
fire_entity_event("audio_source", "deleted", source_id)
|
||||
fire_entity_event("audio_source", "deleted", source_id, entity_name=_entity_name)
|
||||
except EntityNotFoundError as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
|
||||
|
||||
@@ -329,6 +329,12 @@ async def delete_automation(
|
||||
engine: AutomationEngine = Depends(get_automation_engine),
|
||||
):
|
||||
"""Delete an automation."""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_automation(automation_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Deactivate first
|
||||
await engine.deactivate_if_active(automation_id)
|
||||
|
||||
@@ -337,7 +343,7 @@ async def delete_automation(
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
|
||||
fire_entity_event("automation", "deleted", automation_id)
|
||||
fire_entity_event("automation", "deleted", automation_id, entity_name=_entity_name)
|
||||
|
||||
|
||||
# ===== Enable/Disable =====
|
||||
|
||||
@@ -27,6 +27,7 @@ from ledgrab.api.schemas.system import (
|
||||
)
|
||||
from ledgrab.config import get_config
|
||||
from ledgrab.core.backup.auto_backup import AutoBackupEngine
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.storage.asset_store import AssetStore
|
||||
from ledgrab.storage.database import Database, freeze_writes
|
||||
from ledgrab.utils import get_logger, read_upload_capped
|
||||
@@ -35,6 +36,22 @@ logger = get_logger(__name__)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _record_system(action: str, message: str, metadata: dict | None = None) -> None:
|
||||
"""Best-effort audit record for a system-level event."""
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action=action,
|
||||
severity=ActivitySeverity.INFO,
|
||||
message=message,
|
||||
metadata=metadata or {},
|
||||
)
|
||||
|
||||
|
||||
_SERVER_DIR = Path(__file__).resolve().parents[4]
|
||||
|
||||
|
||||
@@ -143,6 +160,8 @@ def backup_config(
|
||||
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H%M%S")
|
||||
filename = f"ledgrab-backup-{timestamp}.zip"
|
||||
|
||||
_record_system("backup.created", f"Backup downloaded: {filename}", {"filename": filename})
|
||||
|
||||
return StreamingResponse(
|
||||
zip_buffer,
|
||||
media_type="application/zip",
|
||||
@@ -243,6 +262,7 @@ async def restore_config(
|
||||
|
||||
freeze_writes()
|
||||
logger.info("Database restored from uploaded backup. Scheduling restart...")
|
||||
_record_system("backup.restored", "Database restored from uploaded backup")
|
||||
_schedule_restart()
|
||||
|
||||
return RestoreResponse(
|
||||
@@ -257,6 +277,7 @@ def restart_server(_: AuthRequired):
|
||||
"""Schedule a server restart and return immediately."""
|
||||
from ledgrab.server_ref import _broadcast_restarting
|
||||
|
||||
_record_system("server.restarting", "Server restart requested by user")
|
||||
_broadcast_restarting()
|
||||
_schedule_restart()
|
||||
return {"status": "restarting"}
|
||||
@@ -267,6 +288,7 @@ def shutdown_server(_: AuthRequired):
|
||||
"""Gracefully shut down the server."""
|
||||
from ledgrab.server_ref import request_shutdown
|
||||
|
||||
_record_system("server.shutdown_requested", "Server shutdown requested by user")
|
||||
request_shutdown()
|
||||
return {"status": "shutting_down"}
|
||||
|
||||
@@ -300,11 +322,17 @@ async def update_auto_backup_settings(
|
||||
engine: AutoBackupEngine = Depends(get_auto_backup_engine),
|
||||
):
|
||||
"""Update auto-backup settings (enable/disable, interval, max backups)."""
|
||||
return await engine.update_settings(
|
||||
result = await engine.update_settings(
|
||||
enabled=body.enabled,
|
||||
interval_hours=body.interval_hours,
|
||||
max_backups=body.max_backups,
|
||||
)
|
||||
_record_system(
|
||||
"settings.changed",
|
||||
f"Auto-backup settings updated (enabled={body.enabled})",
|
||||
{"setting_key": "auto_backup", "enabled": body.enabled},
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@router.post("/api/v1/system/auto-backup/trigger", tags=["System"])
|
||||
@@ -365,4 +393,5 @@ async def delete_saved_backup(
|
||||
engine.delete_backup(filename)
|
||||
except (ValueError, FileNotFoundError) as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
_record_system("backup.deleted", f"Saved backup deleted: {filename}", {"filename": filename})
|
||||
return {"status": "deleted", "filename": filename}
|
||||
|
||||
@@ -36,6 +36,7 @@ from fastapi import APIRouter, Depends, HTTPException
|
||||
|
||||
from ledgrab.api.auth import AuthRequired
|
||||
from ledgrab.api.dependencies import get_processor_manager
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.api.schemas.calibration import (
|
||||
CalibrationSessionPositionRequest,
|
||||
CalibrationSessionStartRequest,
|
||||
@@ -81,6 +82,19 @@ async def start_calibration_session(
|
||||
logger.error("Failed to start calibration session: %s", exc, exc_info=True)
|
||||
raise HTTPException(status_code=500, detail="Internal server error")
|
||||
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="calibration.started",
|
||||
severity=ActivitySeverity.INFO,
|
||||
entity_type="device",
|
||||
entity_id=body.device_id,
|
||||
message=f"Calibration session started for device '{body.device_id}'",
|
||||
)
|
||||
|
||||
return CalibrationSessionStateResponse(**session.get_state())
|
||||
|
||||
|
||||
@@ -135,6 +149,17 @@ async def stop_calibration_session(
|
||||
logger.error("Failed to stop calibration session: %s", exc, exc_info=True)
|
||||
raise HTTPException(status_code=500, detail="Internal server error")
|
||||
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="calibration.stopped",
|
||||
severity=ActivitySeverity.INFO,
|
||||
message="Calibration session stopped",
|
||||
)
|
||||
|
||||
return CalibrationSessionStateResponse(**session.get_state())
|
||||
|
||||
|
||||
@@ -155,6 +180,17 @@ async def cancel_calibration_session(
|
||||
logger.error("Failed to cancel calibration session: %s", exc, exc_info=True)
|
||||
raise HTTPException(status_code=500, detail="Internal server error")
|
||||
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="calibration.cancelled",
|
||||
severity=ActivitySeverity.INFO,
|
||||
message="Calibration session cancelled",
|
||||
)
|
||||
|
||||
return CalibrationSessionStateResponse(**session.get_state())
|
||||
|
||||
|
||||
|
||||
@@ -167,6 +167,12 @@ async def delete_color_strip_source(
|
||||
target_store: OutputTargetStore = Depends(get_output_target_store),
|
||||
):
|
||||
"""Delete a color strip source. Returns 409 if referenced by any LED target."""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_source(source_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
target_names = target_store.get_targets_referencing_css(source_id)
|
||||
if target_names:
|
||||
@@ -201,7 +207,7 @@ async def delete_color_strip_source(
|
||||
"Delete or reassign the processed source(s) first.",
|
||||
)
|
||||
store.delete_source(source_id)
|
||||
fire_entity_event("color_strip_source", "deleted", source_id)
|
||||
fire_entity_event("color_strip_source", "deleted", source_id, entity_name=_entity_name)
|
||||
except HTTPException:
|
||||
raise
|
||||
except ValueError as e:
|
||||
|
||||
@@ -701,6 +701,13 @@ async def delete_device(
|
||||
):
|
||||
"""Delete/detach a device. Returns 409 if referenced by a target."""
|
||||
try:
|
||||
# Resolve name before deletion for the audit record.
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_device(device_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Check if any target references this device
|
||||
refs = target_store.get_targets_for_device(device_id)
|
||||
if refs:
|
||||
@@ -728,7 +735,7 @@ async def delete_device(
|
||||
# Delete from storage
|
||||
store.delete_device(device_id)
|
||||
|
||||
fire_entity_event("device", "deleted", device_id)
|
||||
fire_entity_event("device", "deleted", device_id, entity_name=_entity_name)
|
||||
logger.info(f"Deleted device {device_id}")
|
||||
|
||||
except HTTPException:
|
||||
|
||||
@@ -152,13 +152,19 @@ async def delete_gradient(
|
||||
css_store: ColorStripStore = Depends(get_color_strip_store),
|
||||
):
|
||||
"""Delete a gradient (fails if built-in or referenced by sources)."""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_gradient(gradient_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
# Check references
|
||||
for source in css_store.get_all_sources():
|
||||
if getattr(source, "gradient_id", None) == gradient_id:
|
||||
raise ValueError(f"Cannot delete: referenced by color strip source '{source.name}'")
|
||||
store.delete_gradient(gradient_id)
|
||||
fire_entity_event("gradient", "deleted", gradient_id)
|
||||
fire_entity_event("gradient", "deleted", gradient_id, entity_name=_entity_name)
|
||||
except (ValueError, EntityNotFoundError) as e:
|
||||
status = 404 if "not found" in str(e).lower() else 400
|
||||
raise HTTPException(status_code=status, detail=str(e))
|
||||
|
||||
@@ -624,6 +624,13 @@ async def delete_target(
|
||||
):
|
||||
"""Delete a output target. Stops processing first if active."""
|
||||
try:
|
||||
# Resolve name before deletion for the audit record.
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = target_store.get_target(target_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Stop processing if running
|
||||
try:
|
||||
await manager.stop_processing(target_id)
|
||||
@@ -641,7 +648,7 @@ async def delete_target(
|
||||
# Delete from store
|
||||
target_store.delete_target(target_id)
|
||||
|
||||
fire_entity_event("output_target", "deleted", target_id)
|
||||
fire_entity_event("output_target", "deleted", target_id, entity_name=_entity_name)
|
||||
logger.info(f"Deleted target {target_id}")
|
||||
|
||||
except ValueError as e:
|
||||
|
||||
@@ -12,6 +12,7 @@ from ledgrab.api.dependencies import (
|
||||
get_picture_source_store,
|
||||
get_processor_manager,
|
||||
)
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.api.schemas.output_targets import (
|
||||
BulkTargetRequest,
|
||||
BulkTargetResponse,
|
||||
@@ -28,6 +29,7 @@ from ledgrab.storage.color_strip_source import (
|
||||
from ledgrab.storage.picture_source_store import PictureSourceStore
|
||||
from ledgrab.storage.wled_output_target import WledOutputTarget
|
||||
from ledgrab.storage.output_target_store import OutputTargetStore
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.utils import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
@@ -35,6 +37,23 @@ logger = get_logger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _record_capture(action: str, target_id: str, target_name: str | None, message: str) -> None:
|
||||
"""Best-effort audit record for a capture start/stop action."""
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.CAPTURE,
|
||||
action=action,
|
||||
severity=ActivitySeverity.INFO,
|
||||
entity_type="output_target",
|
||||
entity_id=target_id,
|
||||
entity_name=sanitize_display(target_name) if target_name else None,
|
||||
message=message,
|
||||
)
|
||||
|
||||
|
||||
# ===== BULK PROCESSING CONTROL ENDPOINTS =====
|
||||
|
||||
|
||||
@@ -53,10 +72,18 @@ async def bulk_start_processing(
|
||||
|
||||
for target_id in body.ids:
|
||||
try:
|
||||
target_store.get_target(target_id)
|
||||
_tgt = target_store.get_target(target_id)
|
||||
await manager.start_processing(target_id)
|
||||
started.append(target_id)
|
||||
logger.info(f"Bulk start: started processing for target {target_id}")
|
||||
_tgt_name_raw = getattr(_tgt, "name", None)
|
||||
_tgt_safe = sanitize_display(_tgt_name_raw) if _tgt_name_raw else None
|
||||
_record_capture(
|
||||
"capture.started",
|
||||
target_id,
|
||||
_tgt_safe,
|
||||
f"Capture started for target '{_tgt_safe or target_id}' (bulk)",
|
||||
)
|
||||
except ValueError as e:
|
||||
errors[target_id] = str(e)
|
||||
except RuntimeError as e:
|
||||
@@ -78,6 +105,7 @@ async def bulk_start_processing(
|
||||
async def bulk_stop_processing(
|
||||
body: BulkTargetRequest,
|
||||
_auth: AuthRequired,
|
||||
target_store: OutputTargetStore = Depends(get_output_target_store),
|
||||
manager: ProcessorManager = Depends(get_processor_manager),
|
||||
):
|
||||
"""Stop processing for multiple output targets. Returns lists of stopped IDs and per-ID errors."""
|
||||
@@ -89,6 +117,18 @@ async def bulk_stop_processing(
|
||||
await manager.stop_processing(target_id)
|
||||
stopped.append(target_id)
|
||||
logger.info(f"Bulk stop: stopped processing for target {target_id}")
|
||||
_tgt_name: str | None = None
|
||||
try:
|
||||
_tgt_name = target_store.get_target(target_id).name
|
||||
except Exception:
|
||||
pass
|
||||
_tgt_name_safe = sanitize_display(_tgt_name) if _tgt_name else None
|
||||
_record_capture(
|
||||
"capture.stopped",
|
||||
target_id,
|
||||
_tgt_name_safe,
|
||||
f"Capture stopped for target '{_tgt_name_safe or target_id}' (bulk)",
|
||||
)
|
||||
except ValueError as e:
|
||||
errors[target_id] = str(e)
|
||||
except Exception as e:
|
||||
@@ -112,11 +152,19 @@ async def start_processing(
|
||||
logger.info("Start processing requested for target %s", target_id)
|
||||
try:
|
||||
# Verify target exists in store
|
||||
target_store.get_target(target_id)
|
||||
target = target_store.get_target(target_id)
|
||||
|
||||
await manager.start_processing(target_id)
|
||||
|
||||
logger.info(f"Started processing for target {target_id}")
|
||||
_tgt_name_raw2 = getattr(target, "name", None)
|
||||
_tgt_safe2 = sanitize_display(_tgt_name_raw2) if _tgt_name_raw2 else None
|
||||
_record_capture(
|
||||
"capture.started",
|
||||
target_id,
|
||||
_tgt_safe2,
|
||||
f"Capture started for target '{_tgt_safe2 or target_id}'",
|
||||
)
|
||||
return {"status": "started", "target_id": target_id}
|
||||
|
||||
except ValueError as e:
|
||||
@@ -137,6 +185,7 @@ async def start_processing(
|
||||
async def stop_processing(
|
||||
target_id: str,
|
||||
_auth: AuthRequired,
|
||||
target_store: OutputTargetStore = Depends(get_output_target_store),
|
||||
manager: ProcessorManager = Depends(get_processor_manager),
|
||||
):
|
||||
"""Stop processing for a output target."""
|
||||
@@ -144,6 +193,18 @@ async def stop_processing(
|
||||
await manager.stop_processing(target_id)
|
||||
|
||||
logger.info(f"Stopped processing for target {target_id}")
|
||||
_target_name: str | None = None
|
||||
try:
|
||||
_target_name = target_store.get_target(target_id).name
|
||||
except Exception:
|
||||
pass
|
||||
_target_name_safe = sanitize_display(_target_name) if _target_name else None
|
||||
_record_capture(
|
||||
"capture.stopped",
|
||||
target_id,
|
||||
_target_name_safe,
|
||||
f"Capture stopped for target '{_target_name_safe or target_id}'",
|
||||
)
|
||||
return {"status": "stopped", "target_id": target_id}
|
||||
|
||||
except ValueError as e:
|
||||
|
||||
@@ -374,6 +374,12 @@ async def delete_picture_source(
|
||||
css_store: ColorStripStore = Depends(get_color_strip_store),
|
||||
):
|
||||
"""Delete a picture source."""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_stream(stream_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
# Check if any target transitively references this stream via a CSS
|
||||
target_names = store.get_targets_referencing(stream_id, target_store, css_store)
|
||||
@@ -395,7 +401,7 @@ async def delete_picture_source(
|
||||
f"{css_names}. Please reassign or delete those first.",
|
||||
)
|
||||
store.delete_stream(stream_id)
|
||||
fire_entity_event("picture_source", "deleted", stream_id)
|
||||
fire_entity_event("picture_source", "deleted", stream_id, entity_name=_entity_name)
|
||||
except HTTPException:
|
||||
raise
|
||||
except EntityNotFoundError as e:
|
||||
|
||||
@@ -6,6 +6,7 @@ from datetime import datetime, timezone
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
|
||||
from ledgrab.api.auth import AuthRequired
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.api.dependencies import (
|
||||
fire_entity_event,
|
||||
get_playlist_engine,
|
||||
@@ -220,13 +221,19 @@ async def delete_scene_playlist(
|
||||
engine: PlaylistEngine = Depends(get_playlist_engine),
|
||||
):
|
||||
"""Delete a scene playlist (stops it first if it is currently cycling)."""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_playlist(playlist_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
store.delete_playlist(playlist_id)
|
||||
except (ValueError, EntityNotFoundError) as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
|
||||
await engine.stop_if_running(playlist_id)
|
||||
fire_entity_event("scene_playlist", "deleted", playlist_id)
|
||||
fire_entity_event("scene_playlist", "deleted", playlist_id, entity_name=_entity_name)
|
||||
|
||||
|
||||
# ===== Cycling control =====
|
||||
@@ -255,6 +262,28 @@ async def start_scene_playlist(
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
fire_entity_event("scene_playlist", "updated", playlist_id)
|
||||
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
_pl_name: str | None = None
|
||||
try:
|
||||
_pl_name = store.get_playlist(playlist_id).name
|
||||
except Exception:
|
||||
pass
|
||||
_safe_pl_name = sanitize_display(_pl_name) if _pl_name else None
|
||||
rec.record(
|
||||
category=ActivityCategory.CAPTURE,
|
||||
action="playlist.started",
|
||||
severity=ActivitySeverity.INFO,
|
||||
entity_type="scene_playlist",
|
||||
entity_id=playlist_id,
|
||||
entity_name=_safe_pl_name,
|
||||
message=f"Playlist '{_safe_pl_name or playlist_id}' started",
|
||||
)
|
||||
|
||||
return PlaylistRuntimeStateSchema(**engine.get_state())
|
||||
|
||||
|
||||
@@ -265,11 +294,35 @@ async def start_scene_playlist(
|
||||
)
|
||||
async def stop_scene_playlist(
|
||||
_auth: AuthRequired,
|
||||
store: ScenePlaylistStore = Depends(get_scene_playlist_store),
|
||||
engine: PlaylistEngine = Depends(get_playlist_engine),
|
||||
):
|
||||
"""Stop the active playlist (leaves the last applied scene in place)."""
|
||||
stopped_id = engine.get_running_playlist_id()
|
||||
_stopped_name: str | None = None
|
||||
if stopped_id:
|
||||
try:
|
||||
_stopped_name = store.get_playlist(stopped_id).name
|
||||
except Exception:
|
||||
pass
|
||||
await engine.stop()
|
||||
if stopped_id:
|
||||
fire_entity_event("scene_playlist", "updated", stopped_id)
|
||||
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
_safe_stopped_name = sanitize_display(_stopped_name) if _stopped_name else None
|
||||
rec.record(
|
||||
category=ActivityCategory.CAPTURE,
|
||||
action="playlist.stopped",
|
||||
severity=ActivitySeverity.INFO,
|
||||
entity_type="scene_playlist",
|
||||
entity_id=stopped_id,
|
||||
entity_name=_safe_stopped_name,
|
||||
message=f"Playlist '{_safe_stopped_name or stopped_id}' stopped",
|
||||
)
|
||||
|
||||
return PlaylistRuntimeStateSchema(**engine.get_state())
|
||||
|
||||
@@ -6,6 +6,7 @@ from datetime import datetime, timezone
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
|
||||
from ledgrab.api.auth import AuthRequired
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.api.dependencies import (
|
||||
fire_entity_event,
|
||||
get_output_target_store,
|
||||
@@ -208,12 +209,18 @@ async def delete_scene_preset(
|
||||
store: ScenePresetStore = Depends(get_scene_preset_store),
|
||||
):
|
||||
"""Delete a scene preset."""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_preset(preset_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
store.delete_preset(preset_id)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
|
||||
fire_entity_event("scene_preset", "deleted", preset_id)
|
||||
fire_entity_event("scene_preset", "deleted", preset_id, entity_name=_entity_name)
|
||||
|
||||
|
||||
# ===== Recapture =====
|
||||
@@ -282,4 +289,21 @@ async def activate_scene_preset(
|
||||
logger.info(f"Scene preset '{preset.name}' activated successfully")
|
||||
|
||||
fire_entity_event("scene_preset", "updated", preset_id)
|
||||
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
_safe_preset_name = sanitize_display(preset.name) if preset.name else None
|
||||
rec.record(
|
||||
category=ActivityCategory.CAPTURE,
|
||||
action="scene.activated",
|
||||
severity=ActivitySeverity.INFO,
|
||||
entity_type="scene_preset",
|
||||
entity_id=preset_id,
|
||||
entity_name=_safe_preset_name,
|
||||
message=f"Scene preset '{_safe_preset_name or preset_id}' activated",
|
||||
)
|
||||
|
||||
return ActivateResponse(status=status, errors=errors)
|
||||
|
||||
@@ -149,6 +149,12 @@ async def delete_sync_clock(
|
||||
manager: SyncClockManager = Depends(get_sync_clock_manager),
|
||||
):
|
||||
"""Delete a synchronization clock (fails if referenced by CSS or value sources)."""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = store.get_clock(clock_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
# Check references
|
||||
for source in css_store.get_all_sources():
|
||||
@@ -159,7 +165,7 @@ async def delete_sync_clock(
|
||||
raise ValueError(f"Cannot delete: referenced by value source '{vs.name}'")
|
||||
manager.release_all_for(clock_id)
|
||||
store.delete_clock(clock_id)
|
||||
fire_entity_event("sync_clock", "deleted", clock_id)
|
||||
fire_entity_event("sync_clock", "deleted", clock_id, entity_name=_entity_name)
|
||||
except EntityNotFoundError as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
|
||||
|
||||
@@ -21,11 +21,29 @@ from ledgrab.api.schemas.system import (
|
||||
ShutdownActionRequest,
|
||||
ShutdownActionResponse,
|
||||
)
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.storage.database import Database
|
||||
from ledgrab.utils import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def _record_setting(action: str, key: str, message: str) -> None:
|
||||
"""Best-effort audit record for a high-value settings change."""
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action=action,
|
||||
severity=ActivitySeverity.INFO,
|
||||
message=message,
|
||||
metadata={"setting_key": key},
|
||||
)
|
||||
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@@ -117,6 +135,11 @@ async def update_shutdown_action(
|
||||
"""Set what happens to LED targets when the server shuts down."""
|
||||
db.set_setting("shutdown_action", {"action": body.action})
|
||||
logger.info("Shutdown action updated: %s", body.action)
|
||||
_record_setting(
|
||||
"settings.changed",
|
||||
"shutdown_action",
|
||||
f"Shutdown action set to '{body.action}'",
|
||||
)
|
||||
return ShutdownActionResponse(action=body.action)
|
||||
|
||||
|
||||
@@ -246,6 +269,17 @@ async def adb_connect(_: AuthRequired, request: AdbConnectRequest):
|
||||
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=10)
|
||||
output = (stdout.decode() + stderr.decode()).strip()
|
||||
if "connected" in output.lower():
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.DEVICE,
|
||||
action="device.adb_connected",
|
||||
severity=ActivitySeverity.INFO,
|
||||
message=f"ADB device connected: {sanitize_display(address)}",
|
||||
metadata={"address": address},
|
||||
)
|
||||
return {"status": "connected", "address": address, "message": output}
|
||||
raise HTTPException(status_code=400, detail=output or "Connection failed")
|
||||
except FileNotFoundError:
|
||||
@@ -276,6 +310,17 @@ async def adb_disconnect(_: AuthRequired, request: AdbConnectRequest):
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=10)
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.DEVICE,
|
||||
action="device.adb_disconnected",
|
||||
severity=ActivitySeverity.INFO,
|
||||
message=f"ADB device disconnected: {sanitize_display(address)}",
|
||||
metadata={"address": address},
|
||||
)
|
||||
return {"status": "disconnected", "message": stdout.decode().strip()}
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(status_code=500, detail="adb not found on PATH")
|
||||
|
||||
@@ -183,6 +183,12 @@ async def delete_template(
|
||||
|
||||
Validates that no streams are currently using this template before deletion.
|
||||
"""
|
||||
_entity_name: str | None = None
|
||||
try:
|
||||
_entity_name = template_store.get_template(template_id).name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
# Check if any streams are using this template
|
||||
streams_using_template = []
|
||||
@@ -203,7 +209,7 @@ async def delete_template(
|
||||
|
||||
# Proceed with deletion
|
||||
template_store.delete_template(template_id)
|
||||
fire_entity_event("capture_template", "deleted", template_id)
|
||||
fire_entity_event("capture_template", "deleted", template_id, entity_name=_entity_name)
|
||||
|
||||
except HTTPException:
|
||||
raise # Re-raise HTTP exceptions as-is
|
||||
|
||||
@@ -12,6 +12,7 @@ from ledgrab.api.schemas.update import (
|
||||
UpdateStatusResponse,
|
||||
)
|
||||
from ledgrab.core.update.update_service import UpdateService
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.utils import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
@@ -42,6 +43,17 @@ async def dismiss_update(
|
||||
service: UpdateService = Depends(get_update_service),
|
||||
):
|
||||
service.dismiss(body.version)
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="update.dismissed",
|
||||
severity=ActivitySeverity.INFO,
|
||||
message=f"Update dismissed: {body.version}",
|
||||
metadata={"version": body.version},
|
||||
)
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -63,6 +75,18 @@ async def apply_update(
|
||||
)
|
||||
try:
|
||||
await service.apply_update()
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
version = status.get("available_version", "unknown")
|
||||
rec.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="update.applied",
|
||||
severity=ActivitySeverity.INFO,
|
||||
message=f"Update applied: {version}",
|
||||
metadata={"version": version},
|
||||
)
|
||||
return {"ok": True, "message": "Update applied, server shutting down"}
|
||||
except Exception as exc:
|
||||
logger.error("Failed to apply update: %s", exc, exc_info=True)
|
||||
@@ -83,8 +107,20 @@ async def update_update_settings(
|
||||
body: UpdateSettingsRequest,
|
||||
service: UpdateService = Depends(get_update_service),
|
||||
):
|
||||
return await service.update_settings(
|
||||
result = await service.update_settings(
|
||||
enabled=body.enabled,
|
||||
check_interval_hours=body.check_interval_hours,
|
||||
include_prerelease=body.include_prerelease,
|
||||
)
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
rec.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="settings.changed",
|
||||
severity=ActivitySeverity.INFO,
|
||||
message=f"Update settings changed (enabled={body.enabled})",
|
||||
metadata={"setting_key": "update", "enabled": body.enabled},
|
||||
)
|
||||
return result
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
"""Pydantic schemas for the activity-log API (Phase 4)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Entry + page response
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ActivityLogEntryResponse(BaseModel):
|
||||
"""Single audit-log entry.
|
||||
|
||||
Shape matches ``entry_to_dict()`` from
|
||||
``ledgrab.core.activity_log.recorder`` exactly — that function is the
|
||||
single source of truth for serialisation; this schema documents the wire
|
||||
format.
|
||||
"""
|
||||
|
||||
id: str = Field(description="Entry id — 'al_<8-hex>'")
|
||||
ts: str = Field(description="ISO-8601 UTC timestamp")
|
||||
category: str = Field(description="Broad bucket (auth, device, entity, capture, system)")
|
||||
action: str = Field(description="Verb-object label, e.g. 'entity.created'")
|
||||
severity: str = Field(description="info | warning | error")
|
||||
actor: str = Field(description="API-key label or 'system' / 'anonymous'")
|
||||
entity_type: str | None = Field(default=None, description="Affected entity type, if applicable")
|
||||
entity_id: str | None = Field(default=None, description="Affected entity id, if applicable")
|
||||
entity_name: str | None = Field(
|
||||
default=None, description="Entity name at time of event, if applicable"
|
||||
)
|
||||
message: str = Field(description="Human-readable description")
|
||||
metadata: dict[str, Any] = Field(default_factory=dict, description="Extra structured context")
|
||||
|
||||
|
||||
class ActivityLogPageResponse(BaseModel):
|
||||
"""Paginated list of audit-log entries (keyset cursor)."""
|
||||
|
||||
entries: list[ActivityLogEntryResponse] = Field(description="Entries on this page")
|
||||
next_before_seq: int | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Pass as 'before_seq' in the next request to get the following page. "
|
||||
"None when this is the last page."
|
||||
),
|
||||
)
|
||||
has_more: bool = Field(
|
||||
description="True when there are more entries before the first entry on this page"
|
||||
)
|
||||
total: int = Field(description="Total entries matching the current filters (all pages)")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Settings
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_MAX_DAYS_CAP = 3650 # 10 years — sanity upper bound
|
||||
_MAX_ENTRIES_CAP = 10_000_000 # 10 M rows — sanity upper bound
|
||||
|
||||
|
||||
class ActivityLogSettingsResponse(BaseModel):
|
||||
"""Current activity-log retention settings."""
|
||||
|
||||
enabled: bool = Field(description="Whether the activity log is recording")
|
||||
max_days: int = Field(
|
||||
ge=0,
|
||||
le=_MAX_DAYS_CAP,
|
||||
description="Retain entries for at most this many days (0 = no age-based pruning)",
|
||||
)
|
||||
max_entries: int = Field(
|
||||
ge=0,
|
||||
le=_MAX_ENTRIES_CAP,
|
||||
description="Keep at most this many entries (0 = no count-based pruning)",
|
||||
)
|
||||
|
||||
|
||||
class UpdateActivityLogSettingsRequest(BaseModel):
|
||||
"""Request body for PUT /settings."""
|
||||
|
||||
enabled: bool = Field(description="Enable or disable activity-log recording")
|
||||
max_days: int = Field(
|
||||
ge=0,
|
||||
le=_MAX_DAYS_CAP,
|
||||
description="Retain entries for at most this many days (0 = no age-based pruning)",
|
||||
)
|
||||
max_entries: int = Field(
|
||||
ge=0,
|
||||
le=_MAX_ENTRIES_CAP,
|
||||
description="Keep at most this many entries (0 = no count-based pruning)",
|
||||
)
|
||||
@@ -0,0 +1,25 @@
|
||||
"""Activity / audit log core — recorder, retention engine, and actor context.
|
||||
|
||||
Public surface
|
||||
--------------
|
||||
``ActivityRecorder`` — thread-safe facade; persists entries and fires live events.
|
||||
``ActivityLogRetentionEngine`` — background pruning engine (mirrors AutoBackupEngine).
|
||||
``current_actor`` — ``ContextVar[str]`` set by the auth layer per request.
|
||||
|
||||
Quick start
|
||||
-----------
|
||||
Wired in ``main.py`` lifespan; injected via ``api/dependencies.py`` getters.
|
||||
Phase 3 adds the instrumentation call sites.
|
||||
"""
|
||||
|
||||
from ledgrab.core.activity_log.context import current_actor
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder
|
||||
from ledgrab.core.activity_log.retention import ActivityLogRetentionEngine
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
|
||||
__all__ = [
|
||||
"ActivityRecorder",
|
||||
"ActivityLogRetentionEngine",
|
||||
"current_actor",
|
||||
"sanitize_display",
|
||||
]
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Actor context variable for the activity log.
|
||||
|
||||
``current_actor`` is set by ``api/auth.py:verify_api_key`` on every request so
|
||||
that ``ActivityRecorder.record(...)`` can resolve the actor without requiring
|
||||
every call site to pass it explicitly.
|
||||
|
||||
Default value is ``"system"`` — used by background engines and any code path
|
||||
that runs outside a request context (e.g. lifespan startup/shutdown, zeroconf
|
||||
discovery thread).
|
||||
|
||||
Per-request isolation is guaranteed by ASGI's coroutine context: each request
|
||||
runs in its own coroutine with its own copy of the context inherited from the
|
||||
server's main task. The auth layer resets it on every request before the route
|
||||
handler runs, so stale labels from a previous request cannot bleed into a new
|
||||
one.
|
||||
"""
|
||||
|
||||
from contextvars import ContextVar
|
||||
|
||||
#: The actor label for the current request — API-key label or ``"system"``.
|
||||
current_actor: ContextVar[str] = ContextVar("current_actor", default="system")
|
||||
@@ -0,0 +1,267 @@
|
||||
"""Thread-safe ActivityRecorder facade.
|
||||
|
||||
Responsibilities
|
||||
----------------
|
||||
1. Build an ``ActivityLogEntry`` from the caller-supplied fields.
|
||||
2. Resolve the ``actor`` from the ``current_actor`` ContextVar when not given.
|
||||
3. Persist the entry via ``ActivityLogRepository.record()`` on the event-loop
|
||||
thread — inline if already on that thread, via
|
||||
``loop.call_soon_threadsafe`` if called from another thread (e.g. the
|
||||
zeroconf discovery thread that fires ``device_discovered/lost`` events).
|
||||
4. Push a live ``activity_logged`` event via
|
||||
``ProcessorManager.fire_event({"type": "activity_logged", "entry": {...}})``.
|
||||
5. Never raise into the caller — audit recording is best-effort. Failures are
|
||||
logged at ``WARNING`` level so operators can diagnose without breaking the
|
||||
audited action.
|
||||
|
||||
Thread-marshal pattern mirrors ``utils/log_broadcaster.py`` (``ensure_loop`` /
|
||||
``call_soon_threadsafe``).
|
||||
|
||||
Module accessor
|
||||
---------------
|
||||
A module-level singleton ``_recorder`` is populated by
|
||||
``set_module_recorder()`` during ``main.py`` lifespan startup and exposed via
|
||||
``get_module_recorder()``. Background engines and other non-DI sites that need
|
||||
to call ``record()`` without FastAPI DI can use this accessor. Phase 3
|
||||
instrumentation uses it at the ``fire_entity_event`` choke-point.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from ledgrab.core.activity_log.context import current_actor
|
||||
from ledgrab.storage.activity_log import ActivityLogEntry, ActivitySeverity
|
||||
from ledgrab.utils import get_logger
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ledgrab.core.processing.processor_manager import ProcessorManager
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def _new_id() -> str:
|
||||
"""Generate a compact activity-log entry id: ``al_<8-hex-chars>``."""
|
||||
return "al_" + uuid.uuid4().hex[:8]
|
||||
|
||||
|
||||
def entry_to_dict(entry: ActivityLogEntry) -> dict:
|
||||
"""Serialise an ``ActivityLogEntry`` to the canonical API/event payload dict.
|
||||
|
||||
Reused by Phase 4 (API response serialisation) and Phase 5 (frontend).
|
||||
The shape is identical to the flat row codec minus the DB-only ``seq``
|
||||
field, but with ``ts`` kept as an ISO-8601 string and ``metadata`` as a
|
||||
real ``dict`` (not a JSON string).
|
||||
"""
|
||||
return {
|
||||
"id": entry.id,
|
||||
"ts": entry.ts.isoformat(),
|
||||
"category": entry.category,
|
||||
"action": entry.action,
|
||||
"severity": entry.severity,
|
||||
"actor": entry.actor,
|
||||
"entity_type": entry.entity_type,
|
||||
"entity_id": entry.entity_id,
|
||||
"entity_name": entry.entity_name,
|
||||
"message": entry.message,
|
||||
"metadata": entry.metadata,
|
||||
}
|
||||
|
||||
|
||||
class ActivityRecorder:
|
||||
"""Thread-safe facade for persisting audit log entries.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
repo:
|
||||
``ActivityLogRepository`` used to persist entries.
|
||||
processor_manager:
|
||||
``ProcessorManager`` whose ``fire_event`` dispatches the live
|
||||
``activity_logged`` event to WebSocket subscribers.
|
||||
loop:
|
||||
Optional: the running asyncio event loop. If ``None``, it is
|
||||
captured lazily on the first call that originates from an async
|
||||
context (mirroring ``LogBroadcaster.ensure_loop``). Pass it
|
||||
explicitly in tests to avoid depending on a real running loop.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
repo: "ActivityLogRepository",
|
||||
processor_manager: "ProcessorManager",
|
||||
*,
|
||||
loop: asyncio.AbstractEventLoop | None = None,
|
||||
) -> None:
|
||||
self._repo = repo
|
||||
self._pm = processor_manager
|
||||
self._loop: asyncio.AbstractEventLoop | None = loop
|
||||
self._enabled: bool = True
|
||||
|
||||
# ── Loop capture (mirrors LogBroadcaster.ensure_loop) ──────────────────
|
||||
|
||||
def ensure_loop(self) -> None:
|
||||
"""Capture the running event loop if not already stored.
|
||||
|
||||
Call from an async context (e.g. lifespan startup) so that
|
||||
``call_soon_threadsafe`` works when ``record()`` is later called from
|
||||
non-async threads.
|
||||
"""
|
||||
if self._loop is None:
|
||||
try:
|
||||
self._loop = asyncio.get_running_loop()
|
||||
except RuntimeError:
|
||||
pass
|
||||
|
||||
# ── Public API ──────────────────────────────────────────────────────────
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
"""Whether recording is currently active."""
|
||||
return self._enabled
|
||||
|
||||
@enabled.setter
|
||||
def enabled(self, value: bool) -> None:
|
||||
self._enabled = value
|
||||
|
||||
def record(
|
||||
self,
|
||||
category: str,
|
||||
action: str,
|
||||
*,
|
||||
severity: str = ActivitySeverity.INFO,
|
||||
actor: str | None = None,
|
||||
entity_type: str | None = None,
|
||||
entity_id: str | None = None,
|
||||
entity_name: str | None = None,
|
||||
message: str,
|
||||
metadata: dict | None = None,
|
||||
_bypass_enabled: bool = False,
|
||||
) -> None:
|
||||
"""Append one audit entry — best-effort, never raises.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
category:
|
||||
Broad bucket — one of :class:`~ledgrab.storage.activity_log.ActivityCategory`.
|
||||
action:
|
||||
Verb-object label, e.g. ``"entity.created"`` or ``"server.shutting_down"``.
|
||||
severity:
|
||||
One of :class:`~ledgrab.storage.activity_log.ActivitySeverity`. Defaults
|
||||
to ``"info"``.
|
||||
actor:
|
||||
Who triggered the action. When ``None`` (the common case), the
|
||||
value is resolved from :data:`~ledgrab.core.activity_log.context.current_actor`
|
||||
with a default of ``"system"``.
|
||||
entity_type / entity_id / entity_name:
|
||||
Optional entity context for entity-domain events.
|
||||
message:
|
||||
Human-readable description suitable for display.
|
||||
metadata:
|
||||
Small JSON-serialisable dict with extra context. Defaults to ``{}``.
|
||||
_bypass_enabled:
|
||||
Internal flag used by the retention engine to record the
|
||||
"audit log disabled" event even when ``enabled`` is ``False``.
|
||||
"""
|
||||
if not self._enabled and not _bypass_enabled:
|
||||
return
|
||||
|
||||
# Resolve actor from ContextVar when not explicitly supplied.
|
||||
resolved_actor = actor if actor is not None else current_actor.get()
|
||||
|
||||
entry = ActivityLogEntry(
|
||||
id=_new_id(),
|
||||
ts=datetime.now(timezone.utc),
|
||||
category=category,
|
||||
action=action,
|
||||
severity=severity,
|
||||
actor=resolved_actor,
|
||||
entity_type=entity_type,
|
||||
entity_id=entity_id,
|
||||
entity_name=entity_name,
|
||||
message=message,
|
||||
metadata=metadata or {},
|
||||
)
|
||||
|
||||
# Determine whether we are on the event-loop thread or not.
|
||||
loop = self._loop
|
||||
if loop is None:
|
||||
# Lazy capture — may fail if called before the loop is running.
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
self._loop = loop
|
||||
except RuntimeError:
|
||||
pass
|
||||
|
||||
if loop is not None and loop.is_running():
|
||||
try:
|
||||
current = asyncio.get_event_loop()
|
||||
except RuntimeError:
|
||||
current = None
|
||||
|
||||
# If the current thread IS the event-loop thread, write inline.
|
||||
if current is loop:
|
||||
self._write_and_emit(entry)
|
||||
else:
|
||||
# Called from a non-loop thread (e.g. zeroconf discovery) —
|
||||
# marshal onto the event-loop thread.
|
||||
try:
|
||||
loop.call_soon_threadsafe(self._write_and_emit, entry)
|
||||
except RuntimeError:
|
||||
# Loop has been closed (rare; happens during tests)
|
||||
logger.warning(
|
||||
"ActivityRecorder: event loop closed, dropping entry %s", entry.id
|
||||
)
|
||||
else:
|
||||
# No running loop — fall back to a direct synchronous write.
|
||||
# This path hits in synchronous unit tests that do not start a loop.
|
||||
self._write_and_emit(entry)
|
||||
|
||||
def _write_and_emit(self, entry: ActivityLogEntry) -> None:
|
||||
"""Persist *entry* and fire the live event — called on the loop thread."""
|
||||
try:
|
||||
self._repo.record(entry)
|
||||
except Exception as exc:
|
||||
logger.warning("ActivityRecorder: failed to persist entry %s: %s", entry.id, exc)
|
||||
return # don't emit an event for an entry that failed to persist
|
||||
|
||||
try:
|
||||
self._pm.fire_event(
|
||||
{
|
||||
"type": "activity_logged",
|
||||
"entry": entry_to_dict(entry),
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("ActivityRecorder: failed to fire live event for %s: %s", entry.id, exc)
|
||||
|
||||
|
||||
# ── Module-level singleton accessor ────────────────────────────────────────
|
||||
#
|
||||
# Background engines and non-DI call sites (Phase 3's fire_entity_event hook,
|
||||
# device discovery thread) need ``record()`` without going through FastAPI DI.
|
||||
# ``set_module_recorder`` is called from ``main.py`` lifespan immediately after
|
||||
# the recorder is wired into ``init_dependencies``.
|
||||
|
||||
_recorder: ActivityRecorder | None = None
|
||||
|
||||
|
||||
def set_module_recorder(recorder: ActivityRecorder) -> None:
|
||||
"""Store the application-level recorder in the module singleton.
|
||||
|
||||
Called once from ``main.py`` lifespan startup.
|
||||
"""
|
||||
global _recorder
|
||||
_recorder = recorder
|
||||
|
||||
|
||||
def get_module_recorder() -> ActivityRecorder | None:
|
||||
"""Return the module-level recorder, or ``None`` if not yet initialised.
|
||||
|
||||
Callers must guard against ``None`` — this returns ``None`` during module
|
||||
import and early startup before ``main.py`` lifespan has run.
|
||||
"""
|
||||
return _recorder
|
||||
@@ -0,0 +1,216 @@
|
||||
"""Activity log retention engine.
|
||||
|
||||
Mirrors ``core/backup/auto_backup.py``:
|
||||
- Settings persisted via ``db.get_setting("activity_log")`` /
|
||||
``db.set_setting("activity_log", {...})``.
|
||||
- ``start()`` / ``stop()`` lifecycle following the engine convention used
|
||||
throughout the codebase.
|
||||
- Hourly background loop calling ``repo.prune(before_ts=..., max_entries=...)``.
|
||||
- ``get_settings()`` / ``async update_settings(...)`` for the Settings API
|
||||
(Phase 4).
|
||||
|
||||
Changing ``enabled`` to ``False`` records an ``"audit_log.disabled"`` event via
|
||||
the recorder BEFORE the flag takes effect — so the last action in the log is a
|
||||
record of the intentional disable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.utils import get_logger
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
from ledgrab.storage.database import Database
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
DEFAULT_SETTINGS: dict = {
|
||||
"enabled": True,
|
||||
"max_days": 90,
|
||||
"max_entries": 20000,
|
||||
}
|
||||
|
||||
# Prune loop interval — run roughly once an hour.
|
||||
_PRUNE_INTERVAL_SECS = 3600
|
||||
|
||||
|
||||
class ActivityLogRetentionEngine:
|
||||
"""Background engine that prunes old activity log entries.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
repo:
|
||||
The ``ActivityLogRepository`` used to prune entries.
|
||||
db:
|
||||
The shared ``Database`` singleton for settings persistence.
|
||||
recorder:
|
||||
The ``ActivityRecorder`` used to log the "audit log disabled" event
|
||||
before disabling takes effect.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
repo: "ActivityLogRepository",
|
||||
db: "Database",
|
||||
recorder: "ActivityRecorder",
|
||||
) -> None:
|
||||
self._repo = repo
|
||||
self._db = db
|
||||
self._recorder = recorder
|
||||
self._task: asyncio.Task | None = None
|
||||
self._settings = self._load_settings()
|
||||
# Rehydrate the recorder's enabled flag from persisted settings so a
|
||||
# previously-disabled log stays disabled across restarts.
|
||||
self._recorder.enabled = self._settings["enabled"]
|
||||
|
||||
# ── Settings persistence ───────────────────────────────────────────────
|
||||
|
||||
def _load_settings(self) -> dict:
|
||||
data = self._db.get_setting("activity_log")
|
||||
if data:
|
||||
return {**DEFAULT_SETTINGS, **data}
|
||||
return dict(DEFAULT_SETTINGS)
|
||||
|
||||
def _save_settings(self) -> None:
|
||||
self._db.set_setting(
|
||||
"activity_log",
|
||||
{
|
||||
"enabled": self._settings["enabled"],
|
||||
"max_days": self._settings["max_days"],
|
||||
"max_entries": self._settings["max_entries"],
|
||||
},
|
||||
)
|
||||
|
||||
# ── Lifecycle ──────────────────────────────────────────────────────────
|
||||
|
||||
async def start(self) -> None:
|
||||
"""Start the retention loop if enabled."""
|
||||
if self._settings["enabled"]:
|
||||
self._start_loop()
|
||||
logger.info(
|
||||
"Activity log retention engine started " "(max_days=%d, max_entries=%d)",
|
||||
self._settings["max_days"],
|
||||
self._settings["max_entries"],
|
||||
)
|
||||
else:
|
||||
logger.info("Activity log retention engine initialized (disabled)")
|
||||
|
||||
async def stop(self) -> None:
|
||||
"""Cancel the retention loop."""
|
||||
self._cancel_loop()
|
||||
logger.info("Activity log retention engine stopped")
|
||||
|
||||
def _start_loop(self) -> None:
|
||||
self._cancel_loop()
|
||||
self._task = asyncio.create_task(self._retention_loop())
|
||||
|
||||
def _cancel_loop(self) -> None:
|
||||
if self._task is not None:
|
||||
self._task.cancel()
|
||||
self._task = None
|
||||
|
||||
# ── Prune loop ─────────────────────────────────────────────────────────
|
||||
|
||||
async def _retention_loop(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
await asyncio.sleep(_PRUNE_INTERVAL_SECS)
|
||||
try:
|
||||
self._prune()
|
||||
except Exception as exc:
|
||||
logger.error("Activity log retention prune failed: %s", exc, exc_info=True)
|
||||
except asyncio.CancelledError:
|
||||
logger.debug("Activity log retention loop cancelled")
|
||||
|
||||
def _prune(self) -> None:
|
||||
"""Execute one prune pass based on current settings."""
|
||||
settings = self._settings
|
||||
if not settings["enabled"]:
|
||||
return
|
||||
|
||||
max_days: int = settings["max_days"]
|
||||
max_entries: int = settings["max_entries"]
|
||||
|
||||
before_ts: datetime | None = None
|
||||
if max_days and max_days > 0:
|
||||
before_ts = datetime.now(timezone.utc) - timedelta(days=max_days)
|
||||
|
||||
max_entries_val: int | None = max_entries if max_entries and max_entries > 0 else None
|
||||
|
||||
deleted = self._repo.prune(before_ts=before_ts, max_entries=max_entries_val)
|
||||
if deleted:
|
||||
logger.info(
|
||||
"Activity log pruned %d rows (max_days=%d, max_entries=%d)",
|
||||
deleted,
|
||||
max_days,
|
||||
max_entries,
|
||||
)
|
||||
|
||||
# ── Public API ─────────────────────────────────────────────────────────
|
||||
|
||||
def get_settings(self) -> dict:
|
||||
"""Return the current retention settings dict."""
|
||||
return {
|
||||
"enabled": self._settings["enabled"],
|
||||
"max_days": self._settings["max_days"],
|
||||
"max_entries": self._settings["max_entries"],
|
||||
}
|
||||
|
||||
async def update_settings(
|
||||
self,
|
||||
*,
|
||||
enabled: bool,
|
||||
max_days: int,
|
||||
max_entries: int,
|
||||
) -> dict:
|
||||
"""Persist new settings and apply them immediately.
|
||||
|
||||
If ``enabled`` is changing to ``False``, the disable event is recorded
|
||||
BEFORE the flag takes effect so there is a final log entry.
|
||||
|
||||
Returns the new settings dict (same as ``get_settings()``).
|
||||
"""
|
||||
was_enabled = self._settings["enabled"]
|
||||
|
||||
# Record the disable event before the recorder stops accepting entries.
|
||||
if was_enabled and not enabled:
|
||||
self._recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="audit_log.disabled",
|
||||
severity=ActivitySeverity.WARNING,
|
||||
actor="system",
|
||||
message="Activity log recording disabled via settings",
|
||||
_bypass_enabled=True,
|
||||
)
|
||||
|
||||
self._settings["enabled"] = enabled
|
||||
self._settings["max_days"] = max_days
|
||||
self._settings["max_entries"] = max_entries
|
||||
self._save_settings()
|
||||
|
||||
# Propagate enabled flag to the recorder.
|
||||
self._recorder.enabled = enabled
|
||||
|
||||
if enabled:
|
||||
self._start_loop()
|
||||
logger.info(
|
||||
"Activity log retention enabled (max_days=%d, max_entries=%d)",
|
||||
max_days,
|
||||
max_entries,
|
||||
)
|
||||
# Run an immediate prune pass when re-enabling.
|
||||
try:
|
||||
self._prune()
|
||||
except Exception as exc:
|
||||
logger.error("Activity log immediate prune failed: %s", exc)
|
||||
else:
|
||||
self._cancel_loop()
|
||||
logger.info("Activity log retention disabled")
|
||||
|
||||
return self.get_settings()
|
||||
@@ -0,0 +1,82 @@
|
||||
"""Log-injection sanitizer for audit-log message and display strings.
|
||||
|
||||
Provides :func:`sanitize_display` — a dependency-free helper that strips
|
||||
characters that should not appear in a recorded ``message`` or display
|
||||
string before it is persisted to SQLite, broadcast over WebSocket, or
|
||||
exported to CSV.
|
||||
|
||||
Design constraints
|
||||
------------------
|
||||
- **Dependency-free**: uses only the Python standard library so it can be
|
||||
imported from any module without adding transitive weight.
|
||||
- **Conservative**: keeps printable ASCII/Unicode and normal spaces; drops
|
||||
everything else including control chars (NUL, BEL, BS, VT, FF, ESC,
|
||||
DEL), ANSI/CSI escape sequences (``\\x1b[...``), and carriage returns /
|
||||
newlines / tabs which are the classic log-injection primitives.
|
||||
- **Length-capped**: truncates to *maxlen* characters and appends ``"…"``
|
||||
so callers can rely on a bounded string without adding their own guards.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
# Matches ANSI/VT100 escape sequences: ESC [ ... m (CSI) and shorter forms.
|
||||
# We strip these before the printable-char filter so the bracket/letters that
|
||||
# follow the ESC don't survive stripping the ESC alone.
|
||||
_ANSI_RE = re.compile(r"\x1b(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")
|
||||
|
||||
# Characters we explicitly want to remove even if str.isprintable() would
|
||||
# let them through in some edge-case: NUL is the canonical SQL/log null-byte
|
||||
# injection; the others are kept out by the printable check but listed here
|
||||
# for documentation clarity.
|
||||
_EXPLICIT_DROP = frozenset("\x00\r\n\t")
|
||||
|
||||
|
||||
def sanitize_display(value: str | None, *, maxlen: int = 120) -> str:
|
||||
"""Return a sanitized, length-capped version of *value* safe for log messages.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
value:
|
||||
The raw, potentially attacker-controlled string. ``None`` or empty
|
||||
returns ``""``.
|
||||
maxlen:
|
||||
Maximum length of the returned string (default: 120). If the input
|
||||
exceeds this length after sanitization, the string is truncated and
|
||||
``"…"`` is appended (the ellipsis counts toward *maxlen*).
|
||||
|
||||
Returns
|
||||
-------
|
||||
str
|
||||
A string that:
|
||||
- contains no NUL bytes (``\\x00``),
|
||||
- contains no ANSI/CSI escape sequences,
|
||||
- contains no carriage returns, newlines, or tab characters,
|
||||
- contains only characters for which ``str.isprintable()`` is ``True``
|
||||
plus the regular ASCII space (``\\x20``),
|
||||
- is at most *maxlen* characters long.
|
||||
"""
|
||||
if not value:
|
||||
return ""
|
||||
|
||||
# 1. Strip ANSI escape sequences first so their bracket/letter tails don't
|
||||
# survive as stray printable characters.
|
||||
cleaned = _ANSI_RE.sub("", value)
|
||||
|
||||
# 2. Drop each character that is neither printable nor a plain space.
|
||||
# str.isprintable() returns False for all control chars (including NUL,
|
||||
# BEL, BS, TAB, LF, VT, FF, CR, ESC, DEL) and True for normal letters,
|
||||
# digits, punctuation, and the space character.
|
||||
cleaned = "".join(ch for ch in cleaned if ch.isprintable() or ch == " ")
|
||||
|
||||
# 3. Final belt-and-suspenders pass for the explicit drop set (catches NUL
|
||||
# that may survive if isprintable ever changes in a future Python version).
|
||||
cleaned = "".join(ch for ch in cleaned if ch not in _EXPLICIT_DROP)
|
||||
|
||||
# 4. Cap length.
|
||||
if len(cleaned) > maxlen:
|
||||
# Reserve one character for the ellipsis so total length == maxlen.
|
||||
cleaned = cleaned[: maxlen - 1] + "…"
|
||||
|
||||
return cleaned
|
||||
@@ -726,6 +726,28 @@ class AutomationEngine:
|
||||
else:
|
||||
logger.info(f"Automation '{automation.name}' activated (scene '{preset.name}' applied)")
|
||||
|
||||
# Audit record — best-effort.
|
||||
try:
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
_safe_name = sanitize_display(automation.name) if automation.name else None
|
||||
rec.record(
|
||||
category=ActivityCategory.CAPTURE,
|
||||
action="automation.activated",
|
||||
severity=ActivitySeverity.INFO,
|
||||
actor="system",
|
||||
entity_type="automation",
|
||||
entity_id=automation.id,
|
||||
entity_name=_safe_name,
|
||||
message=f"Automation '{_safe_name or automation.id}' activated",
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
async def _deactivate_automation(self, automation_id: str) -> None:
|
||||
was_active = self._active_automations.pop(automation_id, False)
|
||||
if not was_active:
|
||||
@@ -751,6 +773,33 @@ class AutomationEngine:
|
||||
# Clean up any leftover snapshot
|
||||
self._pre_activation_snapshots.pop(automation_id, None)
|
||||
|
||||
# Audit record — best-effort.
|
||||
try:
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
_auto_name: str | None = None
|
||||
try:
|
||||
_auto_name = self._store.get_automation(automation_id).name
|
||||
except Exception:
|
||||
pass
|
||||
_safe_deact_name = sanitize_display(_auto_name) if _auto_name else None
|
||||
rec.record(
|
||||
category=ActivityCategory.CAPTURE,
|
||||
action="automation.deactivated",
|
||||
severity=ActivitySeverity.INFO,
|
||||
actor="system",
|
||||
entity_type="automation",
|
||||
entity_id=automation_id,
|
||||
entity_name=_safe_deact_name,
|
||||
message=f"Automation '{_safe_deact_name or automation_id}' deactivated",
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
async def _deactivate_revert(self, automation_id: str) -> None:
|
||||
"""Revert to pre-activation snapshot."""
|
||||
snapshot = self._pre_activation_snapshots.pop(automation_id, None)
|
||||
|
||||
@@ -36,6 +36,7 @@ from zeroconf.asyncio import AsyncServiceBrowser, AsyncServiceInfo, AsyncZerocon
|
||||
|
||||
from ledgrab.core.devices.serial_transport import list_serial_ports
|
||||
from ledgrab.core.devices.wled_provider import WLED_MDNS_TYPE
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.utils import get_logger
|
||||
from ledgrab.utils.platform import is_android
|
||||
|
||||
@@ -286,3 +287,34 @@ class DiscoveryWatcher:
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug("Discovery watcher: fire_event failed: %s", e)
|
||||
|
||||
# Audit record — best-effort, thread-safe (recorder marshals via
|
||||
# call_soon_threadsafe when called from the zeroconf thread).
|
||||
try:
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
is_discovered = event_type == "device_discovered"
|
||||
action = "device.discovered" if is_discovered else "device.lost"
|
||||
severity = ActivitySeverity.INFO if is_discovered else ActivitySeverity.WARNING
|
||||
verb = "discovered" if is_discovered else "lost"
|
||||
# Sanitize mDNS-advertised strings before they enter the log.
|
||||
# entry.name and entry.url are unauthenticated, attacker-controlled
|
||||
# values; strip control chars, ANSI escapes, and NUL before use.
|
||||
safe_name = sanitize_display(entry.name)
|
||||
safe_url = sanitize_display(entry.url)
|
||||
rec.record(
|
||||
category=ActivityCategory.DEVICE,
|
||||
action=action,
|
||||
severity=severity,
|
||||
actor="system",
|
||||
entity_type="device",
|
||||
entity_id=entry.url,
|
||||
entity_name=safe_name,
|
||||
message=f"Device '{safe_name}' {verb} at {safe_url}",
|
||||
metadata={"url": safe_url, "device_type": entry.device_type},
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug("Discovery watcher: audit record failed: %s", e)
|
||||
|
||||
@@ -11,6 +11,8 @@ from ledgrab.core.devices.led_client import (
|
||||
check_device_health,
|
||||
get_device_capabilities,
|
||||
)
|
||||
from ledgrab.core.activity_log.sanitize import sanitize_display
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
from ledgrab.utils import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
@@ -128,6 +130,35 @@ class DeviceHealthMixin:
|
||||
"latency_ms": state.health.latency_ms,
|
||||
}
|
||||
)
|
||||
# Audit record for device online/offline transition.
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder
|
||||
|
||||
rec = get_module_recorder()
|
||||
if rec is not None:
|
||||
is_online = state.health.online
|
||||
# Best-effort name lookup from the device store.
|
||||
device_name: str | None = None
|
||||
try:
|
||||
if self._device_store is not None:
|
||||
device_name = self._device_store.get_device(device_id).name
|
||||
except Exception:
|
||||
pass
|
||||
safe_name = sanitize_display(device_name) if device_name else None
|
||||
display = safe_name or device_id
|
||||
action = "device.online" if is_online else "device.offline"
|
||||
severity = ActivitySeverity.INFO if is_online else ActivitySeverity.WARNING
|
||||
status_word = "came online" if is_online else "went offline"
|
||||
rec.record(
|
||||
category=ActivityCategory.DEVICE,
|
||||
action=action,
|
||||
severity=severity,
|
||||
actor="system",
|
||||
entity_type="device",
|
||||
entity_id=device_id,
|
||||
entity_name=safe_name,
|
||||
message=f"Device '{display}' {status_word}",
|
||||
metadata={"latency_ms": state.health.latency_ms},
|
||||
)
|
||||
|
||||
# Auto-sync LED count
|
||||
reported = state.health.device_led_count
|
||||
|
||||
@@ -60,6 +60,9 @@ from ledgrab.storage.audio_processing_template_store import AudioProcessingTempl
|
||||
from ledgrab.storage.pattern_template_store import PatternTemplateStore
|
||||
import ledgrab.core.audio.filters # noqa: F401 — trigger audio filter auto-registration
|
||||
from ledgrab.core.backup.auto_backup import AutoBackupEngine
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder, set_module_recorder
|
||||
from ledgrab.core.activity_log.retention import ActivityLogRetentionEngine
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
from ledgrab.core.processing.os_notification_listener import OsNotificationListener
|
||||
from ledgrab.core.devices.discovery_watcher import DiscoveryWatcher
|
||||
from ledgrab.core.update.update_service import UpdateService
|
||||
@@ -184,6 +187,10 @@ pattern_template_store = PatternTemplateStore(db)
|
||||
game_event_bus = GameEventBus()
|
||||
register_community_adapters()
|
||||
|
||||
# Activity log repository — constructed at module level like other stores so
|
||||
# it migrates the DB schema (``002_add_activity_log``) on import.
|
||||
activity_log_repo = ActivityLogRepository(db)
|
||||
|
||||
processor_manager = ProcessorManager(
|
||||
ProcessorDependencies(
|
||||
picture_source_store=picture_source_store,
|
||||
@@ -290,6 +297,17 @@ async def lifespan(app: FastAPI):
|
||||
processor_manager=processor_manager,
|
||||
)
|
||||
|
||||
# Create activity recorder + retention engine. The recorder needs the
|
||||
# processor_manager to fire live events, so it is built after that is
|
||||
# already constructed at module level.
|
||||
activity_recorder = ActivityRecorder(activity_log_repo, processor_manager)
|
||||
activity_recorder.ensure_loop()
|
||||
activity_log_retention_engine = ActivityLogRetentionEngine(
|
||||
repo=activity_log_repo,
|
||||
db=db,
|
||||
recorder=activity_recorder,
|
||||
)
|
||||
|
||||
# Create auto-backup engine — derive paths from database location so that
|
||||
# demo mode auto-backups go to data/demo/ instead of data/.
|
||||
_data_dir = Path(config.storage.database_file).parent
|
||||
@@ -347,7 +365,13 @@ async def lifespan(app: FastAPI):
|
||||
http_endpoint_store=http_endpoint_store,
|
||||
audio_processing_template_store=audio_processing_template_store,
|
||||
pattern_template_store=pattern_template_store,
|
||||
activity_recorder=activity_recorder,
|
||||
activity_log_repo=activity_log_repo,
|
||||
activity_log_retention_engine=activity_log_retention_engine,
|
||||
)
|
||||
# Expose the recorder via the module singleton so non-DI sites
|
||||
# (fire_entity_event, device threads) can call record() without FastAPI DI.
|
||||
set_module_recorder(activity_recorder)
|
||||
|
||||
# Register devices in processor manager for health monitoring
|
||||
devices = device_store.get_all_devices()
|
||||
@@ -390,6 +414,9 @@ async def lifespan(app: FastAPI):
|
||||
# Start auto-backup engine (periodic configuration backups)
|
||||
await auto_backup_engine.start()
|
||||
|
||||
# Start activity log retention engine (hourly prune of old entries)
|
||||
await activity_log_retention_engine.start()
|
||||
|
||||
# Start update checker (periodic release polling)
|
||||
await update_service.start()
|
||||
|
||||
@@ -438,6 +465,19 @@ async def lifespan(app: FastAPI):
|
||||
except Exception as e:
|
||||
logger.error("Shutdown step '%s' raised: %s", label, e)
|
||||
|
||||
# Record the shutdown event FIRST — before any engine teardown — so there
|
||||
# is always a final log entry on graceful shutdown.
|
||||
try:
|
||||
activity_recorder.record(
|
||||
category="system",
|
||||
action="server.shutting_down",
|
||||
severity="info",
|
||||
actor="system",
|
||||
message="Server is shutting down",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error("Failed to record shutdown event: %s", e)
|
||||
|
||||
# Legacy hook — SQLite stores are write-through so this only logs.
|
||||
# Durability comes from PRAGMA synchronous=FULL + the explicit
|
||||
# wal_checkpoint(TRUNCATE) in Database.close() at the end of this block.
|
||||
@@ -510,6 +550,7 @@ async def lifespan(app: FastAPI):
|
||||
|
||||
await _bounded("update_service.stop", update_service.stop(), timeout=0.5)
|
||||
await _bounded("auto_backup_engine.stop", auto_backup_engine.stop(), timeout=0.5)
|
||||
await _bounded("activity_log_retention.stop", activity_log_retention_engine.stop(), timeout=0.5)
|
||||
|
||||
# Close the DB last so it runs a TRUNCATE checkpoint, flushing the WAL
|
||||
# into the main file. Without this, writes can survive a graceful app
|
||||
|
||||
@@ -0,0 +1,770 @@
|
||||
/* ─────────────────────────────────────────────────────────────────────────
|
||||
Activity Log — audit viewer tab
|
||||
Design language: precision-instrument / ledger. Monospaced timestamps,
|
||||
color-coded severity rail, thin category pills. Clean "terminal" feel
|
||||
without being cold — the primary green accent anchors the live-update dot.
|
||||
───────────────────────────────────────────────────────────────────────── */
|
||||
|
||||
/* ── Panel wrapper ───────────────────────────────────────────────────────── */
|
||||
|
||||
.al-panel {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0;
|
||||
max-width: 1400px;
|
||||
margin: 0 auto;
|
||||
padding: var(--space-lg) var(--space-lg) var(--space-xl);
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
/* ── Filter toolbar ──────────────────────────────────────────────────────── */
|
||||
|
||||
.al-toolbar {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: var(--space-sm);
|
||||
padding: var(--space-md) var(--space-md);
|
||||
/* Match the elevated card surface used by entity cards (.dashboard-target),
|
||||
not the near-black --bg-secondary, so the panel reads as one of the app's
|
||||
cards rather than a separate flat sheet. */
|
||||
background: var(--lux-bg-1, var(--card-bg));
|
||||
border: var(--lux-hairline) solid var(--lux-line, var(--border-color));
|
||||
border-radius: var(--radius-md);
|
||||
margin-bottom: var(--space-md);
|
||||
}
|
||||
|
||||
.al-toolbar-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
flex-wrap: wrap;
|
||||
gap: var(--space-sm);
|
||||
}
|
||||
|
||||
.al-toolbar-search {
|
||||
gap: var(--space-sm);
|
||||
}
|
||||
|
||||
/* Search input */
|
||||
.al-search-wrap {
|
||||
position: relative;
|
||||
flex: 1;
|
||||
min-width: 200px;
|
||||
max-width: 420px;
|
||||
}
|
||||
|
||||
.al-search-icon {
|
||||
position: absolute;
|
||||
left: 10px;
|
||||
top: 50%;
|
||||
transform: translateY(-50%);
|
||||
pointer-events: none;
|
||||
color: var(--text-muted);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.al-search-icon .icon {
|
||||
width: 15px;
|
||||
height: 15px;
|
||||
}
|
||||
|
||||
.al-search-input {
|
||||
width: 100%;
|
||||
padding: 6px 10px 6px 34px;
|
||||
background: var(--card-bg);
|
||||
border: var(--lux-hairline) solid var(--border-color);
|
||||
border-radius: var(--radius-sm);
|
||||
color: var(--text-color);
|
||||
font-size: 0.875rem;
|
||||
font-family: var(--font-body);
|
||||
transition: border-color var(--duration-fast) var(--ease-out),
|
||||
box-shadow var(--duration-fast) var(--ease-out);
|
||||
outline: none;
|
||||
}
|
||||
|
||||
.al-search-input:focus {
|
||||
border-color: var(--primary-color);
|
||||
box-shadow: 0 0 0 2px rgba(76, 175, 80, 0.15);
|
||||
}
|
||||
|
||||
.al-search-input::placeholder { color: var(--text-muted); }
|
||||
|
||||
/* Quick presets */
|
||||
.al-presets {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.al-preset-btn {
|
||||
padding: 4px 10px;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 500;
|
||||
background: var(--card-bg);
|
||||
border: var(--lux-hairline) solid var(--border-color);
|
||||
border-radius: var(--radius-pill);
|
||||
color: var(--text-secondary);
|
||||
cursor: pointer;
|
||||
transition: background var(--duration-fast), color var(--duration-fast), border-color var(--duration-fast);
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.al-preset-btn:hover {
|
||||
background: var(--primary-color);
|
||||
border-color: var(--primary-color);
|
||||
color: var(--primary-contrast);
|
||||
}
|
||||
|
||||
/* Clear button */
|
||||
.al-clear-btn {
|
||||
padding: 4px 6px;
|
||||
margin-left: auto;
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.al-clear-btn:hover { color: var(--danger-color); border-color: var(--danger-color); }
|
||||
|
||||
/* Export button + dropdown */
|
||||
.al-export-wrap {
|
||||
position: relative;
|
||||
margin-left: auto;
|
||||
}
|
||||
|
||||
.al-export-btn {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 5px;
|
||||
padding: 5px 12px;
|
||||
font-size: 0.8125rem;
|
||||
}
|
||||
|
||||
.al-export-btn .icon { width: 14px; height: 14px; }
|
||||
|
||||
/* Caret signals this button opens a menu (rather than firing a direct action),
|
||||
and rotates to point up while the menu is open. */
|
||||
.al-export-caret {
|
||||
display: inline-flex;
|
||||
margin-left: 1px;
|
||||
transition: transform var(--duration-fast) var(--ease-out);
|
||||
}
|
||||
.al-export-caret .icon { width: 12px; height: 12px; }
|
||||
.al-export-wrap.open .al-export-caret { transform: rotate(180deg); }
|
||||
|
||||
.al-export-menu {
|
||||
display: none;
|
||||
position: absolute;
|
||||
right: 0;
|
||||
top: calc(100% + 4px);
|
||||
background: var(--card-bg);
|
||||
border: var(--lux-hairline) solid var(--border-color);
|
||||
border-radius: var(--radius-sm);
|
||||
box-shadow: 0 4px 12px var(--shadow-color);
|
||||
z-index: 100;
|
||||
min-width: 140px;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.al-export-wrap.open .al-export-menu { display: block; }
|
||||
|
||||
.al-export-menu button {
|
||||
display: block;
|
||||
width: 100%;
|
||||
padding: 8px 14px;
|
||||
text-align: left;
|
||||
background: none;
|
||||
border: none;
|
||||
color: var(--text-color);
|
||||
font-size: 0.8125rem;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.al-export-menu button:hover,
|
||||
.al-export-menu button:focus-visible { background: var(--bg-secondary); outline: none; }
|
||||
|
||||
/* Filter label */
|
||||
.al-filter-label {
|
||||
font-size: 0.75rem;
|
||||
font-weight: 600;
|
||||
color: var(--text-muted);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.05em;
|
||||
white-space: nowrap;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.al-filter-label-sep { margin-left: var(--space-sm); }
|
||||
|
||||
/* Category / severity chips */
|
||||
.al-chip-group {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.al-chip {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
padding: 3px 9px;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 500;
|
||||
border: var(--lux-hairline) solid var(--border-color);
|
||||
border-radius: var(--radius-pill);
|
||||
background: var(--card-bg);
|
||||
color: var(--text-secondary);
|
||||
cursor: pointer;
|
||||
transition: background var(--duration-fast), color var(--duration-fast), border-color var(--duration-fast);
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.al-chip .icon { width: 12px; height: 12px; }
|
||||
|
||||
.al-chip:hover {
|
||||
background: var(--bg-secondary);
|
||||
border-color: var(--text-secondary);
|
||||
color: var(--text-color);
|
||||
}
|
||||
|
||||
.al-chip.active {
|
||||
background: var(--primary-color);
|
||||
border-color: var(--primary-color);
|
||||
color: var(--primary-contrast);
|
||||
}
|
||||
|
||||
/* Severity chip colors when active */
|
||||
.al-sev-chip-error.active { background: var(--danger-color); border-color: var(--danger-color); }
|
||||
.al-sev-chip-warning.active { background: var(--warning-color); border-color: var(--warning-color); }
|
||||
.al-sev-chip-info.active { background: var(--info-color); border-color: var(--info-color); }
|
||||
|
||||
/* Advanced field row */
|
||||
.al-toolbar-advanced {
|
||||
gap: var(--space-sm);
|
||||
padding-top: var(--space-xs);
|
||||
border-top: var(--lux-hairline) solid var(--border-color);
|
||||
}
|
||||
|
||||
.al-field-group {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 2px;
|
||||
min-width: 160px;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.al-field-label {
|
||||
font-size: 0.6875rem;
|
||||
font-weight: 600;
|
||||
color: var(--text-muted);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
}
|
||||
|
||||
.al-field-input {
|
||||
padding: 4px 8px;
|
||||
background: var(--card-bg);
|
||||
border: var(--lux-hairline) solid var(--border-color);
|
||||
border-radius: var(--radius-sm);
|
||||
color: var(--text-color);
|
||||
font-size: 0.8125rem;
|
||||
font-family: var(--font-body);
|
||||
outline: none;
|
||||
transition: border-color var(--duration-fast);
|
||||
}
|
||||
|
||||
.al-field-input:focus {
|
||||
border-color: var(--primary-color);
|
||||
box-shadow: 0 0 0 2px rgba(76, 175, 80, 0.12);
|
||||
}
|
||||
|
||||
/* ── List header (count + live dot) ─────────────────────────────────────── */
|
||||
|
||||
.al-list-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: var(--space-xs) 0;
|
||||
margin-bottom: var(--space-xs);
|
||||
}
|
||||
|
||||
.al-count {
|
||||
font-size: 0.8125rem;
|
||||
color: var(--text-muted);
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.al-live-indicator {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 5px;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 500;
|
||||
color: var(--primary-text-color);
|
||||
letter-spacing: 0.02em;
|
||||
}
|
||||
|
||||
.al-live-dot {
|
||||
width: 7px;
|
||||
height: 7px;
|
||||
background: var(--primary-color);
|
||||
border-radius: 50%;
|
||||
animation: al-pulse 2s infinite;
|
||||
}
|
||||
|
||||
@keyframes al-pulse {
|
||||
0%, 100% { opacity: 1; transform: scale(1); }
|
||||
50% { opacity: 0.55; transform: scale(0.85); }
|
||||
}
|
||||
|
||||
/* ── Entry rows ─────────────────────────────────────────────────────────── */
|
||||
|
||||
.al-list {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 1px;
|
||||
}
|
||||
|
||||
.al-entry {
|
||||
/* Same elevated surface + hairline as entity cards (see .al-toolbar). */
|
||||
background: var(--lux-bg-1, var(--card-bg));
|
||||
border: var(--lux-hairline) solid var(--lux-line, var(--border-color));
|
||||
border-radius: var(--radius-sm);
|
||||
overflow: hidden;
|
||||
transition: border-color var(--duration-fast);
|
||||
}
|
||||
|
||||
.al-entry:hover { border-color: var(--text-muted); }
|
||||
|
||||
/* New-entry flash — settles on the card surface (animation-fill-mode: forwards
|
||||
holds the 100% frame, so it must match the .al-entry background exactly). */
|
||||
@keyframes al-new-flash {
|
||||
0% { background: color-mix(in srgb, var(--primary-color) 18%, var(--lux-bg-1, var(--card-bg))); }
|
||||
100% { background: var(--lux-bg-1, var(--card-bg)); }
|
||||
}
|
||||
|
||||
.al-entry-new.al-entry-appear { animation: al-new-flash 1.8s var(--ease-out) forwards; }
|
||||
|
||||
.al-entry-row {
|
||||
display: grid;
|
||||
/* icon | time | badge | actor | message | entity | chevron
|
||||
badge is fixed so all category names (AUTH…CAPTURE) occupy identical
|
||||
width; actor is capped so long usernames don't push the message over;
|
||||
message takes all remaining space. */
|
||||
grid-template-columns: 24px 80px 78px minmax(0, 110px) 1fr auto 20px;
|
||||
align-items: center;
|
||||
gap: var(--space-sm);
|
||||
padding: var(--space-xs) var(--space-md);
|
||||
cursor: pointer;
|
||||
min-height: 36px;
|
||||
outline: none;
|
||||
}
|
||||
|
||||
.al-entry-row:focus-visible { box-shadow: inset 0 0 0 2px var(--primary-color); }
|
||||
|
||||
/* Severity rail icon */
|
||||
.al-sev { display: flex; align-items: center; justify-content: center; }
|
||||
.al-sev .icon { width: 14px; height: 14px; }
|
||||
.al-sev-info .icon { color: var(--info-color); }
|
||||
.al-sev-warning .icon { color: var(--warning-color); }
|
||||
.al-sev-error .icon { color: var(--danger-color); }
|
||||
|
||||
/* Time */
|
||||
.al-time {
|
||||
font-size: 0.75rem;
|
||||
font-variant-numeric: tabular-nums;
|
||||
font-family: var(--font-mono);
|
||||
color: var(--text-muted);
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
/* Category badge */
|
||||
.al-cat-badge {
|
||||
display: inline-block;
|
||||
padding: 1px 7px;
|
||||
font-size: 0.6875rem;
|
||||
font-weight: 600;
|
||||
letter-spacing: 0.03em;
|
||||
text-transform: uppercase;
|
||||
border-radius: var(--radius-pill);
|
||||
white-space: nowrap;
|
||||
border: var(--lux-hairline) solid transparent;
|
||||
}
|
||||
|
||||
/* Per-category colors — subtle tinted backgrounds */
|
||||
.al-cat-auth { background: rgba(33, 150, 243, 0.12); color: var(--info-color); border-color: rgba(33, 150, 243, 0.25); }
|
||||
.al-cat-device { background: rgba(156, 39, 176, 0.10); color: #ab47bc; border-color: rgba(156, 39, 176, 0.22); }
|
||||
.al-cat-entity { background: rgba(76, 175, 80, 0.12); color: var(--primary-text-color); border-color: rgba(76, 175, 80, 0.25); }
|
||||
.al-cat-capture { background: rgba(255, 152, 0, 0.12); color: var(--warning-color); border-color: rgba(255, 152, 0, 0.25); }
|
||||
.al-cat-system { background: rgba(120, 120, 120, 0.12); color: var(--text-secondary); border-color: rgba(120, 120, 120, 0.25); }
|
||||
|
||||
[data-theme="light"] .al-cat-auth { background: rgba(33, 150, 243, 0.08); }
|
||||
/* Darker purple text in light theme — the dark-theme #ab47bc fails AA contrast
|
||||
on the pale tinted background at this small badge size. */
|
||||
[data-theme="light"] .al-cat-device { background: rgba(156, 39, 176, 0.08); color: #8e24aa; }
|
||||
[data-theme="light"] .al-cat-entity { background: rgba(76, 175, 80, 0.08); }
|
||||
[data-theme="light"] .al-cat-capture { background: rgba(255, 152, 0, 0.08); }
|
||||
[data-theme="light"] .al-cat-system { background: rgba(120, 120, 120, 0.08); }
|
||||
|
||||
/* Actor — constrained by its grid column (minmax(0, 110px)) */
|
||||
.al-actor {
|
||||
font-size: 0.8125rem;
|
||||
font-family: var(--font-mono);
|
||||
color: var(--text-secondary);
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
/* Message — min-width:0 lets the 1fr column actually truncate */
|
||||
.al-msg {
|
||||
font-size: 0.8125rem;
|
||||
color: var(--text-color);
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
min-width: 0;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
/* Entity crosslink */
|
||||
.al-entity { display: flex; align-items: center; }
|
||||
|
||||
.al-entity-link {
|
||||
font-size: 0.75rem;
|
||||
color: var(--primary-text-color);
|
||||
background: none;
|
||||
border: none;
|
||||
padding: 0;
|
||||
cursor: pointer;
|
||||
text-decoration: underline dotted;
|
||||
text-underline-offset: 2px;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
max-width: 120px;
|
||||
display: block;
|
||||
}
|
||||
|
||||
.al-entity-link:hover { color: var(--primary-hover); text-decoration-style: solid; }
|
||||
|
||||
.al-entity-name {
|
||||
font-size: 0.75rem;
|
||||
color: var(--text-muted);
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
/* Expand chevron */
|
||||
.al-expand-chevron {
|
||||
font-size: 0.6875rem;
|
||||
color: var(--text-muted);
|
||||
justify-self: end;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
/* ── Entry detail drawer ─────────────────────────────────────────────────── */
|
||||
|
||||
.al-detail {
|
||||
padding: var(--space-sm) var(--space-md) var(--space-md);
|
||||
border-top: var(--lux-hairline) solid var(--border-color);
|
||||
background: var(--bg-secondary);
|
||||
animation: al-detail-open var(--duration-fast) var(--ease-out);
|
||||
}
|
||||
|
||||
@keyframes al-detail-open {
|
||||
from { opacity: 0; transform: translateY(-4px); }
|
||||
to { opacity: 1; transform: translateY(0); }
|
||||
}
|
||||
|
||||
.al-detail-grid {
|
||||
display: grid;
|
||||
grid-template-columns: max-content 1fr;
|
||||
gap: 4px var(--space-md);
|
||||
font-size: 0.8125rem;
|
||||
}
|
||||
|
||||
.al-detail-grid dt {
|
||||
color: var(--text-muted);
|
||||
font-weight: 600;
|
||||
font-size: 0.75rem;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
align-self: start;
|
||||
padding-top: 2px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.al-detail-grid dd {
|
||||
color: var(--text-color);
|
||||
word-break: break-all;
|
||||
}
|
||||
|
||||
.al-detail-grid code {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 0.8125rem;
|
||||
background: var(--bg-color);
|
||||
padding: 1px 5px;
|
||||
border-radius: var(--radius-sm);
|
||||
border: var(--lux-hairline) solid var(--border-color);
|
||||
}
|
||||
|
||||
.al-meta-pre {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 0.75rem;
|
||||
background: var(--bg-color);
|
||||
border: var(--lux-hairline) solid var(--border-color);
|
||||
border-radius: var(--radius-sm);
|
||||
padding: var(--space-sm);
|
||||
overflow-x: auto;
|
||||
white-space: pre;
|
||||
color: var(--text-secondary);
|
||||
max-height: 220px;
|
||||
overflow-y: auto;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
/* ── Load More ───────────────────────────────────────────────────────────── */
|
||||
|
||||
.al-load-more {
|
||||
display: block;
|
||||
width: 100%;
|
||||
margin-top: var(--space-md);
|
||||
text-align: center;
|
||||
font-size: 0.875rem;
|
||||
}
|
||||
|
||||
/* ── Empty / loading / error states ─────────────────────────────────────── */
|
||||
|
||||
.al-state {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: var(--space-sm);
|
||||
padding: var(--space-xl);
|
||||
color: var(--text-muted);
|
||||
font-size: 0.875rem;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.al-state-icon .icon {
|
||||
width: 36px;
|
||||
height: 36px;
|
||||
opacity: 0.35;
|
||||
}
|
||||
|
||||
.al-loading { flex-direction: row; padding: var(--space-lg); }
|
||||
|
||||
.al-spinner {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border: 2px solid var(--border-color);
|
||||
border-top-color: var(--primary-color);
|
||||
border-radius: 50%;
|
||||
animation: al-spin 0.6s linear infinite;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
@keyframes al-spin { to { transform: rotate(360deg); } }
|
||||
|
||||
.al-error .al-state-icon .icon { color: var(--danger-color); opacity: 0.6; }
|
||||
|
||||
/* ── List container ──────────────────────────────────────────────────────── */
|
||||
|
||||
.al-list-container {
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
/* Subtle busy state while a slow re-query is in flight: the current rows stay
|
||||
visible (no spinner flash) but dim slightly and stop accepting clicks until
|
||||
the fresh results swap in. Only applied after a short delay, so instant
|
||||
filtering shows nothing. */
|
||||
.al-list-container.al-busy {
|
||||
opacity: 0.55;
|
||||
pointer-events: none;
|
||||
transition: opacity var(--duration-fast) var(--ease-out);
|
||||
}
|
||||
|
||||
/* ── Tabular-nums utility ────────────────────────────────────────────────── */
|
||||
|
||||
.tabular-nums { font-variant-numeric: tabular-nums; }
|
||||
|
||||
/* ── Responsive ──────────────────────────────────────────────────────────── */
|
||||
|
||||
@media (max-width: 900px) {
|
||||
.al-entry-row {
|
||||
/* icon | time | badge (fixed) | message | chevron — actor+entity hidden */
|
||||
grid-template-columns: 20px 70px 78px 1fr 18px;
|
||||
}
|
||||
/* Hide actor and entity link at small widths */
|
||||
.al-actor,
|
||||
.al-entity { display: none; }
|
||||
}
|
||||
|
||||
@media (max-width: 600px) {
|
||||
.al-panel { padding: var(--space-sm); }
|
||||
|
||||
.al-entry-row {
|
||||
grid-template-columns: 20px 1fr auto 18px;
|
||||
grid-template-rows: auto auto;
|
||||
gap: 4px var(--space-xs);
|
||||
}
|
||||
|
||||
/* Row 1: [sev] [message] [badge] [chevron]; Row 2: [time] under the message.
|
||||
message stays in its own 1fr column so it never overlaps the badge. */
|
||||
.al-time { grid-column: 2; grid-row: 2; font-size: 0.6875rem; }
|
||||
.al-cat-badge{ grid-column: 3; grid-row: 1; }
|
||||
.al-msg { grid-column: 2; grid-row: 1; }
|
||||
|
||||
.al-toolbar-advanced .al-field-group { min-width: 100%; }
|
||||
}
|
||||
|
||||
/* ─────────────────────────────────────────────────────────────────────────
|
||||
Dashboard "Recent Activity" widget (.dal-*)
|
||||
Compact, consistent with the precision-instrument language of the full tab.
|
||||
Rows are tighter than the full viewer — just sev icon + relative time + msg.
|
||||
───────────────────────────────────────────────────────────────────────── */
|
||||
|
||||
/* List container */
|
||||
.dal-list {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0;
|
||||
}
|
||||
|
||||
/* Compact entry row */
|
||||
.al-compact-row {
|
||||
display: grid;
|
||||
grid-template-columns: 18px 52px 1fr;
|
||||
align-items: center;
|
||||
gap: 0 8px;
|
||||
padding: 5px 4px;
|
||||
border-bottom: 1px solid var(--border-color);
|
||||
min-height: 28px;
|
||||
transition: background 0.12s;
|
||||
}
|
||||
|
||||
.al-compact-row:last-child {
|
||||
border-bottom: none;
|
||||
}
|
||||
|
||||
.al-compact-row:hover {
|
||||
background: var(--bg-secondary);
|
||||
}
|
||||
|
||||
.al-compact-icon .icon {
|
||||
width: 14px;
|
||||
height: 14px;
|
||||
display: block;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.al-compact-time {
|
||||
font-family: var(--font-mono, monospace);
|
||||
font-size: 0.6875rem;
|
||||
color: var(--text-muted);
|
||||
white-space: nowrap;
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.al-compact-msg {
|
||||
font-size: 0.8125rem;
|
||||
color: var(--text-color);
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
/* Severity color on the row itself (row inherits .al-sev-* from renderCompactEntry) */
|
||||
.al-compact-row.al-sev-error .al-compact-icon .icon { color: var(--danger-color); }
|
||||
.al-compact-row.al-sev-warning .al-compact-icon .icon { color: var(--warning-color); }
|
||||
.al-compact-row.al-sev-info .al-compact-icon .icon { color: var(--info-color); }
|
||||
|
||||
/* Empty state inside widget */
|
||||
.dal-empty {
|
||||
padding: 16px 8px;
|
||||
}
|
||||
.dal-empty p {
|
||||
font-size: 0.8125rem;
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
/* Loading state placeholder */
|
||||
.dal-loading {
|
||||
padding: 16px 8px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
/* Footer — "View all →" link */
|
||||
.dal-footer {
|
||||
padding: 6px 4px 2px;
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
}
|
||||
|
||||
.dal-view-all {
|
||||
font-size: 0.8125rem;
|
||||
color: var(--primary-text-color);
|
||||
text-decoration: none;
|
||||
border: none;
|
||||
background: none;
|
||||
cursor: pointer;
|
||||
padding: 2px 4px;
|
||||
transition: opacity 0.15s;
|
||||
}
|
||||
|
||||
.dal-view-all:hover {
|
||||
opacity: 0.75;
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
/* ─────────────────────────────────────────────────────────────────────────
|
||||
Settings panel helpers (ds-info-note, ds-inline-link)
|
||||
These are general enough to live here but scoped tightly enough to not
|
||||
bleed into the rest of the settings layout.
|
||||
───────────────────────────────────────────────────────────────────────── */
|
||||
|
||||
.ds-info-note {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
gap: 8px;
|
||||
padding: 10px 14px;
|
||||
margin-bottom: 16px;
|
||||
background: color-mix(in srgb, var(--info-color) 8%, var(--bg-secondary));
|
||||
border: 1px solid color-mix(in srgb, var(--info-color) 25%, var(--border-color));
|
||||
border-radius: var(--radius-sm, 4px);
|
||||
font-size: 0.8125rem;
|
||||
color: var(--text-color);
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.ds-info-note .icon {
|
||||
width: 15px;
|
||||
height: 15px;
|
||||
flex-shrink: 0;
|
||||
margin-top: 1px;
|
||||
color: var(--info-color);
|
||||
}
|
||||
|
||||
/* Inline text button that looks like a link (used in ds-info-note, hints) */
|
||||
.ds-inline-link {
|
||||
background: none;
|
||||
border: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
cursor: pointer;
|
||||
color: var(--primary-text-color);
|
||||
font-size: inherit;
|
||||
font-family: inherit;
|
||||
text-decoration: underline;
|
||||
text-underline-offset: 2px;
|
||||
display: inline;
|
||||
}
|
||||
|
||||
.ds-inline-link:hover {
|
||||
opacity: 0.8;
|
||||
}
|
||||
@@ -19,5 +19,6 @@
|
||||
@import './graph-editor.css';
|
||||
@import './appearance.css';
|
||||
@import './game-integration.css';
|
||||
@import './activity-log.css';
|
||||
@import './mobile.css';
|
||||
@import './tv.css';
|
||||
|
||||
@@ -228,6 +228,24 @@ import {
|
||||
mountAutoCalibration, unmountAutoCalibration,
|
||||
} from './features/auto-calibration.ts';
|
||||
|
||||
// Layer 5: activity log
|
||||
import {
|
||||
loadActivityLog,
|
||||
activityLogToggleDetail,
|
||||
activityLogToggleCat,
|
||||
activityLogToggleSev,
|
||||
activityLogOnSearch,
|
||||
activityLogOnActor,
|
||||
activityLogOnEntityType,
|
||||
activityLogOnSince,
|
||||
activityLogOnUntil,
|
||||
activityLogClearFilters,
|
||||
activityLogPreset,
|
||||
activityLogLoadMore,
|
||||
activityLogExport,
|
||||
activityLogNavigateToEntity,
|
||||
} from './features/activity-log.ts';
|
||||
|
||||
// Layer 5.5: graph editor
|
||||
import {
|
||||
loadGraphEditor,
|
||||
@@ -257,6 +275,7 @@ import {
|
||||
loadDaylightTimezone, saveDaylightTimezone,
|
||||
requestNotifPermissionFromSettings, testNotifFromSettings,
|
||||
saveExternalUrl, revertExternalUrl, getBaseOrigin, loadExternalUrl,
|
||||
loadActivityLogSettings, saveActivityLogSettings, activityLogSettingsExport, clearActivityLog,
|
||||
} from './features/settings.ts';
|
||||
import {
|
||||
loadUpdateStatus, initUpdateListener, checkForUpdates,
|
||||
@@ -741,6 +760,10 @@ Object.assign(window, {
|
||||
saveExternalUrl,
|
||||
revertExternalUrl,
|
||||
getBaseOrigin,
|
||||
loadActivityLogSettings,
|
||||
saveActivityLogSettings,
|
||||
activityLogSettingsExport,
|
||||
clearActivityLog,
|
||||
|
||||
// update
|
||||
checkForUpdates,
|
||||
@@ -762,6 +785,22 @@ Object.assign(window, {
|
||||
applyStylePreset,
|
||||
applyBgEffect,
|
||||
renderAppearanceTab,
|
||||
|
||||
// activity log
|
||||
loadActivityLog,
|
||||
activityLogToggleDetail,
|
||||
activityLogToggleCat,
|
||||
activityLogToggleSev,
|
||||
activityLogOnSearch,
|
||||
activityLogOnActor,
|
||||
activityLogOnEntityType,
|
||||
activityLogOnSince,
|
||||
activityLogOnUntil,
|
||||
activityLogClearFilters,
|
||||
activityLogPreset,
|
||||
activityLogLoadMore,
|
||||
activityLogExport,
|
||||
activityLogNavigateToEntity,
|
||||
});
|
||||
|
||||
// ─── Global keyboard shortcuts ───
|
||||
@@ -779,7 +818,7 @@ document.addEventListener('keydown', (e) => {
|
||||
|
||||
// Tab shortcuts: Ctrl+1..4 (skip when typing in inputs)
|
||||
if (!inInput && e.ctrlKey && !e.altKey && !e.shiftKey && !e.metaKey) {
|
||||
const tabMap = { '1': 'dashboard', '2': 'automations', '3': 'targets', '4': 'streams', '5': 'integrations', '6': 'graph' };
|
||||
const tabMap = { '1': 'dashboard', '2': 'automations', '3': 'targets', '4': 'streams', '5': 'integrations', '6': 'graph', '7': 'activity_log' };
|
||||
const tab = tabMap[e.key];
|
||||
if (tab) {
|
||||
e.preventDefault();
|
||||
|
||||
@@ -32,6 +32,7 @@ import { openAuthedWs } from './ws-auth.ts';
|
||||
* update_download_progress — update_service.py (consumed by features/update.ts)
|
||||
* device_discovered — discovery_watcher.py (consumed by features/notifications-watcher.ts)
|
||||
* device_lost — discovery_watcher.py (consumed by features/notifications-watcher.ts)
|
||||
* activity_logged — core/activity_log/recorder.py (consumed by features/activity-log.ts)
|
||||
*
|
||||
* Missing any of these silently breaks the corresponding UI flow — keep
|
||||
* this list in sync when adding new event types on the server side.
|
||||
@@ -47,6 +48,7 @@ const _ALLOWED_SERVER_EVENT_TYPES: ReadonlySet<string> = new Set([
|
||||
'update_download_progress',
|
||||
'device_discovered',
|
||||
'device_lost',
|
||||
'activity_logged', // source: core/activity_log/recorder.py
|
||||
]);
|
||||
|
||||
interface ServerEventEnvelope {
|
||||
|
||||
@@ -135,6 +135,17 @@ export const armchair = '<path d="M19 9V6a2 2 0 0 0-2-2H7a2 2 0 0 0-2 2v3"/>
|
||||
// Lucide: leaf
|
||||
export const leaf = '<path d="M11 20A7 7 0 0 1 9.8 6.1C15.5 5 17 4.48 19.2 2.96c1 4.34.06 9.65-3.4 13.04A6.96 6.96 0 0 1 11 20z"/><path d="M2 21c0-3 1.85-5.36 5.08-6"/>';
|
||||
|
||||
// Lucide: scroll-text (audit / activity log)
|
||||
export const scrollText = '<path d="M15 12h-5"/><path d="M15 8h-5"/><path d="M19 17V5a2 2 0 0 0-2-2H4"/><path d="M8 21h12a2 2 0 0 0 2-2v-1a1 1 0 0 0-1-1H11a1 1 0 0 0-1 1v1a2 2 0 1 1-4 0V5a2 2 0 1 0-4 0v2a1 1 0 0 0 1 1h3"/>';
|
||||
// Lucide: circle-alert (error severity)
|
||||
export const circleAlert = '<circle cx="12" cy="12" r="10"/><line x1="12" x2="12" y1="8" y2="12"/><line x1="12" x2="12.01" y1="16" y2="16"/>';
|
||||
// Lucide: info (info severity)
|
||||
export const info = '<circle cx="12" cy="12" r="10"/><path d="M12 16v-4"/><path d="M12 8h.01"/>';
|
||||
// Lucide: filter (filter toolbar)
|
||||
export const filter = '<polygon points="22 3 2 3 10 12.46 10 19 14 21 14 12.46 22 3"/>';
|
||||
// Lucide: x-circle (clear/reset)
|
||||
export const xCircle = '<circle cx="12" cy="12" r="10"/><path d="m15 9-6 6"/><path d="m9 9 6 6"/>';
|
||||
|
||||
// Easing curve glyphs — custom mini-charts that draw the actual curve.
|
||||
// Curve travels from (4, 20) to (20, 4); each path renders the easing
|
||||
// function directly so the picker shows the shape, not a metaphor.
|
||||
|
||||
@@ -354,6 +354,15 @@ export const ICON_CIRCLE = _svg(P.circle);
|
||||
export const ICON_GIT_MERGE = _svg(P.gitMerge);
|
||||
export const ICON_COPY = _svg(P.copy);
|
||||
|
||||
// ── Activity log icons ─────────────────────────────────────
|
||||
|
||||
export const ICON_ACTIVITY_LOG = _svg(P.scrollText);
|
||||
export const ICON_SEVERITY_INFO = _svg(P.info);
|
||||
export const ICON_SEVERITY_WARN = _svg(P.triangleAlert);
|
||||
export const ICON_SEVERITY_ERR = _svg(P.circleAlert);
|
||||
export const ICON_FILTER = _svg(P.filter);
|
||||
export const ICON_X_CIRCLE = _svg(P.xCircle);
|
||||
|
||||
// ── Game integration icons ─────────────────────────────────
|
||||
|
||||
export const ICON_GAMEPAD = _svg(P.gamepad2);
|
||||
|
||||
@@ -28,6 +28,7 @@ const TAB_REGISTRY: Readonly<Record<string, TabConfig>> = {
|
||||
automations: { loadFnName: 'loadAutomations',
|
||||
subTab: { storageKey: 'activeAutomationTab', defaultSubTab: 'automations', switchFnName: 'switchAutomationTab' } },
|
||||
graph: { loadFnName: 'loadGraphEditor' },
|
||||
activity_log: { loadFnName: 'loadActivityLog', autoRefresh: false },
|
||||
};
|
||||
|
||||
/** Get the full config for a tab, or undefined if not registered. */
|
||||
|
||||
@@ -555,6 +555,90 @@ export function formatCompact(n: number | null | undefined) {
|
||||
return (v < 10 ? v.toFixed(1) : Math.round(v)) + 'B';
|
||||
}
|
||||
|
||||
/**
|
||||
* Format an ISO-8601 timestamp (or ms epoch) into a locale-aware string.
|
||||
* Returns "Today · HH:MM", "Yesterday · HH:MM", or "DD MMM · HH:MM".
|
||||
* Use `font-variant-numeric: tabular-nums` on the element for stable layout.
|
||||
*/
|
||||
export function formatTimestamp(isoOrMs: string | number): string {
|
||||
const d = typeof isoOrMs === 'number' ? new Date(isoOrMs) : new Date(isoOrMs);
|
||||
if (isNaN(d.getTime())) return String(isoOrMs);
|
||||
|
||||
const now = new Date();
|
||||
const todayStart = new Date(now.getFullYear(), now.getMonth(), now.getDate());
|
||||
const yestStart = new Date(todayStart.getTime() - 86400000);
|
||||
|
||||
const hhmm = d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit', hour12: false });
|
||||
|
||||
if (d >= todayStart) {
|
||||
return `${t('time.today')} · ${hhmm}`;
|
||||
} else if (d >= yestStart) {
|
||||
return `${t('time.yesterday')} · ${hhmm}`;
|
||||
} else {
|
||||
const dateStr = d.toLocaleDateString([], { day: 'numeric', month: 'short' });
|
||||
return `${dateStr} · ${hhmm}`;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Format an ISO-8601 timestamp (or ms epoch) as a compact relative string.
|
||||
* Examples: "just now", "2m ago", "3h ago", "5d ago".
|
||||
* Use font-variant-numeric: tabular-nums on elements that update frequently.
|
||||
*/
|
||||
export function formatRelativeTime(isoOrMs: string | number): string {
|
||||
const d = typeof isoOrMs === 'number' ? new Date(isoOrMs) : new Date(isoOrMs);
|
||||
if (isNaN(d.getTime())) return String(isoOrMs);
|
||||
|
||||
const diffSec = Math.floor((Date.now() - d.getTime()) / 1000);
|
||||
if (diffSec < 10) return t('time.just_now');
|
||||
if (diffSec < 60) return t('time.seconds_ago', { n: diffSec });
|
||||
const diffMin = Math.floor(diffSec / 60);
|
||||
if (diffMin < 60) return t('time.minutes_ago', { n: diffMin });
|
||||
const diffHr = Math.floor(diffMin / 60);
|
||||
if (diffHr < 24) return t('time.hours_ago', { n: diffHr });
|
||||
const diffDays = Math.floor(diffHr / 24);
|
||||
return t('time.days_ago', { n: diffDays });
|
||||
}
|
||||
|
||||
// ── Shared relative-time ticker ──────────────────────────────────────────────
|
||||
// A single process-wide interval that keeps every `[data-reltime]` element
|
||||
// up to date. Call `ensureRelativeTimeTicker()` from any feature that renders
|
||||
// such elements — repeated calls are idempotent (one interval, ever).
|
||||
|
||||
let _relTimeIntervalId: ReturnType<typeof setInterval> | null = null;
|
||||
let _relTimeVisibilityBound = false;
|
||||
|
||||
/** Refresh every `[data-reltime]` element's text content to the current
|
||||
* relative-time label produced by `formatRelativeTime`. */
|
||||
function _tickRelativeTimes(): void {
|
||||
if (document.hidden) return;
|
||||
document.querySelectorAll<HTMLElement>('[data-reltime]').forEach(el => {
|
||||
const iso = el.getAttribute('data-reltime');
|
||||
if (iso) el.textContent = formatRelativeTime(iso);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the shared relative-time ticker (idempotent — safe to call many times).
|
||||
* Ticks every 30 s, skips work when the tab is hidden, and fires one
|
||||
* immediate refresh when the tab becomes visible again.
|
||||
* Also fires one immediate refresh on each `languageChanged` event so
|
||||
* freshly-translated labels appear without waiting for the next tick.
|
||||
*/
|
||||
export function ensureRelativeTimeTicker(): void {
|
||||
// One-time visibility + language listeners
|
||||
if (!_relTimeVisibilityBound) {
|
||||
_relTimeVisibilityBound = true;
|
||||
document.addEventListener('visibilitychange', () => {
|
||||
if (!document.hidden) _tickRelativeTimes();
|
||||
});
|
||||
document.addEventListener('languageChanged', () => _tickRelativeTimes());
|
||||
}
|
||||
// Idempotent: only start the interval once
|
||||
if (_relTimeIntervalId !== null) return;
|
||||
_relTimeIntervalId = setInterval(_tickRelativeTimes, 30_000);
|
||||
}
|
||||
|
||||
export function formatUptime(seconds: number | null | undefined): string {
|
||||
if (!seconds || seconds <= 0) return '-';
|
||||
const total = Math.floor(seconds);
|
||||
|
||||
@@ -0,0 +1,874 @@
|
||||
/**
|
||||
* Activity Log tab — persistent, queryable audit log viewer.
|
||||
*
|
||||
* This is a READ-ONLY viewer (no CRUD), differentiated from the debug
|
||||
* Log Viewer (utils/log_broadcaster.py) which is an ephemeral 500-line tail.
|
||||
* This tab shows structured, semantic audit entries backed by the SQLite
|
||||
* activity_log table.
|
||||
*
|
||||
* Phase 5: Activity tab + smart filtering + live updates.
|
||||
*/
|
||||
|
||||
import { fetchWithAuth, escapeHtml } from '../core/api.ts';
|
||||
import { t } from '../core/i18n.ts';
|
||||
import { showToast, formatRelativeTime, ensureRelativeTimeTicker } from '../core/ui.ts';
|
||||
import { navigateToCard } from '../core/navigation.ts';
|
||||
import {
|
||||
ICON_ACTIVITY_LOG, ICON_SEVERITY_INFO, ICON_SEVERITY_WARN, ICON_SEVERITY_ERR,
|
||||
ICON_X_CIRCLE, ICON_DOWNLOAD, ICON_SEARCH,
|
||||
ICON_CHEVRON_UP, ICON_CHEVRON_DOWN,
|
||||
} from '../core/icons.ts';
|
||||
|
||||
/**
|
||||
* Escape a string for safe use inside an HTML attribute value (quoted with
|
||||
* either `"` or `'`). Extends escapeHtml's `<>&` coverage with `"` and `'`.
|
||||
*/
|
||||
function _escapeAttr(text: string): string {
|
||||
if (!text) return '';
|
||||
return escapeHtml(text)
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, ''');
|
||||
}
|
||||
|
||||
// ─── Types ───────────────────────────────────────────────────
|
||||
|
||||
export interface ActivityEntry {
|
||||
id: string;
|
||||
ts: string;
|
||||
category: string;
|
||||
action: string;
|
||||
severity: string;
|
||||
actor: string;
|
||||
entity_type: string | null;
|
||||
entity_id: string | null;
|
||||
entity_name: string | null;
|
||||
message: string;
|
||||
metadata: Record<string, unknown>;
|
||||
}
|
||||
|
||||
interface ActivityPage {
|
||||
entries: ActivityEntry[];
|
||||
next_before_seq: number | null;
|
||||
has_more: boolean;
|
||||
total: number;
|
||||
}
|
||||
|
||||
interface ActiveFilters {
|
||||
categories: string[];
|
||||
severities: string[];
|
||||
actor: string;
|
||||
entity_type: string;
|
||||
since: string;
|
||||
until: string;
|
||||
q: string;
|
||||
}
|
||||
|
||||
// ─── Module state ────────────────────────────────────────────
|
||||
|
||||
let _initialized = false;
|
||||
let _loading = false;
|
||||
let _entries: ActivityEntry[] = [];
|
||||
let _nextBeforeSeq: number | null = null;
|
||||
let _hasMore = false;
|
||||
let _total = 0;
|
||||
let _expandedIds = new Set<string>();
|
||||
let _debounceTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
let _liveEventListener: ((e: Event) => void) | null = null;
|
||||
// Loading UX: `_showSpinner` gates the full-panel spinner so it only appears
|
||||
// after a short delay (slow requests), never flashing on instant filtering.
|
||||
// `_hasLoadedOnce` distinguishes the genuine first load (spinner immediately)
|
||||
// from re-queries (keep current rows, subtle delayed busy hint).
|
||||
let _loadingDelayTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
let _showSpinner = false;
|
||||
let _hasLoadedOnce = false;
|
||||
|
||||
const _filters: ActiveFilters = {
|
||||
categories: [],
|
||||
severities: [],
|
||||
actor: '',
|
||||
entity_type: '',
|
||||
since: '',
|
||||
until: '',
|
||||
q: '',
|
||||
};
|
||||
|
||||
// ─── Category → navigation target map (entity crosslinks) ──
|
||||
|
||||
const _ENTITY_NAV: Record<string, { tab: string; subTab: string | null; attr: string } | null> = {
|
||||
output_target: { tab: 'targets', subTab: 'led-targets', attr: 'data-target-id' },
|
||||
device: { tab: 'targets', subTab: 'led-devices', attr: 'data-device-id' },
|
||||
picture_source: { tab: 'streams', subTab: 'raw', attr: 'data-stream-id' },
|
||||
color_strip_source: { tab: 'streams', subTab: 'color_strip', attr: 'data-css-id' },
|
||||
audio_source: { tab: 'streams', subTab: 'audio_capture', attr: 'data-id' },
|
||||
automation: { tab: 'automations', subTab: 'automations', attr: 'data-automation-id' },
|
||||
scene_preset: { tab: 'automations', subTab: 'scenes', attr: 'data-scene-id' },
|
||||
scene_playlist: { tab: 'automations', subTab: 'playlists', attr: 'data-playlist-id' },
|
||||
};
|
||||
|
||||
// ─── Severity icon helper ────────────────────────────────────
|
||||
|
||||
function _severityIcon(severity: string): string {
|
||||
if (severity === 'error') return ICON_SEVERITY_ERR;
|
||||
if (severity === 'warning') return ICON_SEVERITY_WARN;
|
||||
return ICON_SEVERITY_INFO;
|
||||
}
|
||||
|
||||
function _severityClass(severity: string): string {
|
||||
if (severity === 'error') return 'al-sev-error';
|
||||
if (severity === 'warning') return 'al-sev-warning';
|
||||
return 'al-sev-info';
|
||||
}
|
||||
|
||||
// ─── Category label helper ───────────────────────────────────
|
||||
|
||||
function _categoryLabel(category: string): string {
|
||||
return t(`activity_log.category.${category}`);
|
||||
}
|
||||
|
||||
// ─── Localized entity-type label ────────────────────────────
|
||||
|
||||
function _entityTypeLabel(entityType: string): string {
|
||||
const key = `activity_log.entity_type.${entityType}`;
|
||||
const translated = t(key);
|
||||
// If t() returned the key unchanged there is no translation — humanize it
|
||||
if (translated === key) {
|
||||
return entityType.replace(/_/g, ' ').replace(/\b\w/g, c => c.toUpperCase());
|
||||
}
|
||||
return translated;
|
||||
}
|
||||
|
||||
// ─── Client-side message localization ───────────────────────
|
||||
//
|
||||
// Maps entry.action → an i18n template key and extracts placeholders
|
||||
// from the structured fields so the displayed description is rendered
|
||||
// in the user's locale rather than the server-generated English string.
|
||||
//
|
||||
// Fallback: if the template key is missing (t() returns the key
|
||||
// unchanged) we return entry.message (the original server string) so
|
||||
// the UI always shows something sensible.
|
||||
|
||||
export function localizeMessage(entry: ActivityEntry): string {
|
||||
const meta = entry.metadata || {};
|
||||
|
||||
// Build a params bag from structured fields + metadata.
|
||||
// Keys match the {placeholder} names used in locale templates.
|
||||
const params: Record<string, string> = {
|
||||
name: entry.entity_name ?? '',
|
||||
actor: entry.actor ?? '',
|
||||
type: entry.entity_type ? _entityTypeLabel(entry.entity_type) : '',
|
||||
key: String(meta.setting_key ?? ''),
|
||||
address: String(meta.address ?? meta.url ?? ''),
|
||||
reason: String(meta.reason ?? ''),
|
||||
client: String(meta.client ?? ''),
|
||||
device_type: String(meta.device_type ?? ''),
|
||||
filename: String(meta.filename ?? ''),
|
||||
};
|
||||
|
||||
// The backend always emits dotted actions (e.g. "entity.created",
|
||||
// "auth.ws_connected"), so the template key is a direct 1:1 mapping.
|
||||
const templateKey = `activity_log.msg.${entry.action}`;
|
||||
|
||||
const localized = t(templateKey, params);
|
||||
// t() returns the key unchanged when there is no matching translation.
|
||||
if (localized === templateKey) {
|
||||
return entry.message;
|
||||
}
|
||||
return localized;
|
||||
}
|
||||
|
||||
// ─── Build query string from active filters + cursor ────────
|
||||
|
||||
function _buildQuery(beforeSeq: number | null = null): string {
|
||||
const params = new URLSearchParams();
|
||||
for (const cat of _filters.categories) params.append('categories', cat);
|
||||
for (const sev of _filters.severities) params.append('severities', sev);
|
||||
if (_filters.actor) params.set('actor', _filters.actor);
|
||||
if (_filters.entity_type) params.set('entity_type', _filters.entity_type);
|
||||
if (_filters.since) params.set('since', _filters.since);
|
||||
if (_filters.until) params.set('until', _filters.until);
|
||||
if (_filters.q) params.set('q', _filters.q);
|
||||
if (beforeSeq != null) params.set('before_seq', String(beforeSeq));
|
||||
params.set('limit', '50');
|
||||
const qs = params.toString();
|
||||
return qs ? `?${qs}` : '';
|
||||
}
|
||||
|
||||
// ─── Entry rendering ─────────────────────────────────────────
|
||||
|
||||
function _renderEntryRow(entry: ActivityEntry, isNew = false): string {
|
||||
const relTime = formatRelativeTime(entry.ts);
|
||||
const iso = entry.ts;
|
||||
const expanded = _expandedIds.has(entry.id);
|
||||
const sevClass = _severityClass(entry.severity);
|
||||
const sevIcon = _severityIcon(entry.severity);
|
||||
|
||||
// Entity crosslink — use data-* attributes + delegated listener (no JSON in onclick)
|
||||
let entityHtml = '';
|
||||
if (entry.entity_type && entry.entity_name) {
|
||||
const nav = _ENTITY_NAV[entry.entity_type];
|
||||
if (nav) {
|
||||
const escapedName = escapeHtml(entry.entity_name);
|
||||
const attrEntityType = _escapeAttr(entry.entity_type);
|
||||
const attrEntityId = _escapeAttr(entry.entity_id || '');
|
||||
entityHtml = `<button class="al-entity-link" type="button"
|
||||
data-entity-type="${attrEntityType}" data-entity-id="${attrEntityId}"
|
||||
title="${_escapeAttr(entry.entity_name)}">${escapedName}</button>`;
|
||||
} else {
|
||||
entityHtml = `<span class="al-entity-name">${escapeHtml(entry.entity_name)}</span>`;
|
||||
}
|
||||
}
|
||||
|
||||
const detailHtml = expanded ? _renderEntryDetail(entry) : '';
|
||||
const attrEntryId = _escapeAttr(entry.id);
|
||||
|
||||
return `<div class="al-entry${isNew ? ' al-entry-new' : ''}" data-al-id="${_escapeAttr(entry.id)}">
|
||||
<div class="al-entry-row" data-toggle-id="${attrEntryId}"
|
||||
role="button" tabindex="0" aria-expanded="${expanded}">
|
||||
<span class="al-sev ${sevClass}" title="${_escapeAttr(t(`activity_log.severity.${entry.severity}`))}">${sevIcon}</span>
|
||||
<span class="al-time tabular-nums" title="${_escapeAttr(iso)}" data-reltime="${_escapeAttr(iso)}">${escapeHtml(relTime)}</span>
|
||||
<span class="al-cat-badge al-cat-${escapeHtml(entry.category)}">${escapeHtml(_categoryLabel(entry.category))}</span>
|
||||
<span class="al-actor">${escapeHtml(entry.actor)}</span>
|
||||
<span class="al-msg">${escapeHtml(localizeMessage(entry))}</span>
|
||||
${entityHtml ? `<span class="al-entity">${entityHtml}</span>` : ''}
|
||||
<span class="al-expand-chevron" aria-hidden="true">${expanded ? ICON_CHEVRON_UP : ICON_CHEVRON_DOWN}</span>
|
||||
</div>
|
||||
${detailHtml}
|
||||
</div>`;
|
||||
}
|
||||
|
||||
function _renderEntryDetail(entry: ActivityEntry): string {
|
||||
const metaJson = JSON.stringify(entry.metadata, null, 2);
|
||||
const absTime = entry.ts ? new Date(entry.ts).toLocaleString() : '';
|
||||
return `<div class="al-detail" role="region" aria-label="${escapeHtml(t('activity_log.detail.title'))}">
|
||||
<dl class="al-detail-grid">
|
||||
<dt>${escapeHtml(t('activity_log.detail.id'))}</dt>
|
||||
<dd class="tabular-nums"><code>${escapeHtml(entry.id)}</code></dd>
|
||||
<dt>${escapeHtml(t('activity_log.detail.timestamp'))}</dt>
|
||||
<dd class="tabular-nums">${escapeHtml(absTime)}</dd>
|
||||
<dt>${escapeHtml(t('activity_log.detail.action'))}</dt>
|
||||
<dd><code>${escapeHtml(entry.action)}</code></dd>
|
||||
<dt>${escapeHtml(t('activity_log.detail.actor'))}</dt>
|
||||
<dd>${escapeHtml(entry.actor)}</dd>
|
||||
${entry.entity_type ? `<dt>${escapeHtml(t('activity_log.detail.entity'))}</dt>
|
||||
<dd>${escapeHtml(entry.entity_type)}${entry.entity_id ? ` / <code>${escapeHtml(entry.entity_id)}</code>` : ''}${entry.entity_name ? ` (${escapeHtml(entry.entity_name)})` : ''}</dd>` : ''}
|
||||
<dt>${escapeHtml(t('activity_log.detail.metadata'))}</dt>
|
||||
<dd><pre class="al-meta-pre">${escapeHtml(metaJson)}</pre></dd>
|
||||
</dl>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
// ─── Filter toolbar rendering ────────────────────────────────
|
||||
|
||||
const CATEGORIES = ['auth', 'device', 'entity', 'capture', 'system'];
|
||||
const SEVERITIES = ['info', 'warning', 'error'];
|
||||
|
||||
function _renderFilterToolbar(): string {
|
||||
const catChips = CATEGORIES.map(cat => {
|
||||
const active = _filters.categories.includes(cat);
|
||||
return `<button class="al-chip al-cat-chip${active ? ' active' : ''} al-cat-${cat}"
|
||||
type="button" onclick="activityLogToggleCat('${cat}')"
|
||||
aria-pressed="${active}">${escapeHtml(_categoryLabel(cat))}</button>`;
|
||||
}).join('');
|
||||
|
||||
const sevChips = SEVERITIES.map(sev => {
|
||||
const active = _filters.severities.includes(sev);
|
||||
const icon = _severityIcon(sev);
|
||||
return `<button class="al-chip al-sev-chip${active ? ' active' : ''} al-sev-chip-${sev}"
|
||||
type="button" onclick="activityLogToggleSev('${sev}')"
|
||||
aria-pressed="${active}">${icon} ${escapeHtml(t(`activity_log.severity.${sev}`))}</button>`;
|
||||
}).join('');
|
||||
|
||||
const presets = [
|
||||
{ key: 'today', label: t('activity_log.preset.today') },
|
||||
{ key: 'errors', label: t('activity_log.preset.errors') },
|
||||
{ key: 'auth', label: t('activity_log.preset.auth') },
|
||||
{ key: 'devices', label: t('activity_log.preset.devices') },
|
||||
];
|
||||
const presetBtns = presets.map(p =>
|
||||
`<button class="al-preset-btn" type="button" onclick="activityLogPreset('${p.key}')">${escapeHtml(p.label)}</button>`
|
||||
).join('');
|
||||
|
||||
const hasFilters = _filters.categories.length || _filters.severities.length ||
|
||||
_filters.actor || _filters.entity_type || _filters.since || _filters.until || _filters.q;
|
||||
|
||||
return `<div class="al-toolbar" role="search" aria-label="${escapeHtml(t('activity_log.filter.title'))}">
|
||||
<div class="al-toolbar-row al-toolbar-search">
|
||||
<div class="al-search-wrap">
|
||||
<span class="al-search-icon" aria-hidden="true">${ICON_SEARCH}</span>
|
||||
<input class="al-search-input" type="search" id="al-search-input"
|
||||
placeholder="${escapeHtml(t('activity_log.filter.search'))}"
|
||||
value="${_escapeAttr(_filters.q)}"
|
||||
oninput="activityLogOnSearch(this.value)"
|
||||
aria-label="${escapeHtml(t('activity_log.filter.search'))}">
|
||||
</div>
|
||||
<div class="al-presets">${presetBtns}</div>
|
||||
${hasFilters ? `<button class="al-clear-btn btn btn-icon btn-secondary" type="button"
|
||||
onclick="activityLogClearFilters()" title="${escapeHtml(t('activity_log.filter.clear'))}"
|
||||
aria-label="${escapeHtml(t('activity_log.filter.clear'))}">${ICON_X_CIRCLE}</button>` : ''}
|
||||
<div class="al-export-wrap">
|
||||
<button class="btn btn-secondary al-export-btn" type="button"
|
||||
data-al-export-toggle aria-haspopup="menu" aria-expanded="false"
|
||||
title="${escapeHtml(t('activity_log.export'))}"
|
||||
aria-label="${escapeHtml(t('activity_log.export'))}">${ICON_DOWNLOAD} <span>${escapeHtml(t('activity_log.export'))}</span><span class="al-export-caret" aria-hidden="true">${ICON_CHEVRON_DOWN}</span></button>
|
||||
<div class="al-export-menu" role="menu">
|
||||
<button type="button" role="menuitem" onclick="activityLogExport('csv')">${escapeHtml(t('activity_log.export.csv'))}</button>
|
||||
<button type="button" role="menuitem" onclick="activityLogExport('json')">${escapeHtml(t('activity_log.export.json'))}</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="al-toolbar-row al-toolbar-chips">
|
||||
<span class="al-filter-label">${escapeHtml(t('activity_log.filter.category'))}</span>
|
||||
<div class="al-chip-group" role="group" aria-label="${escapeHtml(t('activity_log.filter.category'))}">${catChips}</div>
|
||||
<span class="al-filter-label al-filter-label-sep">${escapeHtml(t('activity_log.filter.severity'))}</span>
|
||||
<div class="al-chip-group" role="group" aria-label="${escapeHtml(t('activity_log.filter.severity'))}">${sevChips}</div>
|
||||
</div>
|
||||
<div class="al-toolbar-row al-toolbar-advanced" id="al-toolbar-advanced">
|
||||
<div class="al-field-group">
|
||||
<label for="al-actor-input" class="al-field-label">${escapeHtml(t('activity_log.filter.actor'))}</label>
|
||||
<input type="text" id="al-actor-input" class="al-field-input"
|
||||
value="${_escapeAttr(_filters.actor)}"
|
||||
placeholder="${_escapeAttr(t('activity_log.filter.actor.placeholder'))}"
|
||||
oninput="activityLogOnActor(this.value)"
|
||||
aria-label="${escapeHtml(t('activity_log.filter.actor'))}">
|
||||
</div>
|
||||
<div class="al-field-group">
|
||||
<label for="al-entity-type-input" class="al-field-label">${escapeHtml(t('activity_log.filter.entity_type'))}</label>
|
||||
<input type="text" id="al-entity-type-input" class="al-field-input"
|
||||
value="${_escapeAttr(_filters.entity_type)}"
|
||||
placeholder="${_escapeAttr(t('activity_log.filter.entity_type.placeholder'))}"
|
||||
oninput="activityLogOnEntityType(this.value)"
|
||||
aria-label="${escapeHtml(t('activity_log.filter.entity_type'))}">
|
||||
</div>
|
||||
<div class="al-field-group">
|
||||
<label for="al-since-input" class="al-field-label">${escapeHtml(t('activity_log.filter.since'))}</label>
|
||||
<input type="datetime-local" id="al-since-input" class="al-field-input"
|
||||
value="${_escapeAttr(_filters.since)}"
|
||||
onchange="activityLogOnSince(this.value)"
|
||||
aria-label="${escapeHtml(t('activity_log.filter.since'))}">
|
||||
</div>
|
||||
<div class="al-field-group">
|
||||
<label for="al-until-input" class="al-field-label">${escapeHtml(t('activity_log.filter.until'))}</label>
|
||||
<input type="datetime-local" id="al-until-input" class="al-field-input"
|
||||
value="${_escapeAttr(_filters.until)}"
|
||||
onchange="activityLogOnUntil(this.value)"
|
||||
aria-label="${escapeHtml(t('activity_log.filter.until'))}">
|
||||
</div>
|
||||
</div>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
// ─── List and state rendering ────────────────────────────────
|
||||
|
||||
function _renderList(): string {
|
||||
if (_showSpinner && _entries.length === 0) {
|
||||
return `<div class="al-state al-loading" role="status" aria-live="polite">
|
||||
<div class="al-spinner"></div>
|
||||
<span>${escapeHtml(t('activity_log.loading'))}</span>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
if (_entries.length === 0) {
|
||||
const hasFilters = _filters.categories.length || _filters.severities.length ||
|
||||
_filters.actor || _filters.entity_type || _filters.since || _filters.until || _filters.q;
|
||||
const emptyKey = hasFilters ? 'activity_log.empty' : 'activity_log.empty_no_filters';
|
||||
return `<div class="al-state al-empty" role="status">
|
||||
<span class="al-state-icon" aria-hidden="true">${ICON_ACTIVITY_LOG}</span>
|
||||
<p>${escapeHtml(t(emptyKey))}</p>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
const rows = _entries.map(e => _renderEntryRow(e)).join('');
|
||||
const loadMore = _hasMore
|
||||
? `<button class="al-load-more btn btn-secondary" type="button"
|
||||
onclick="activityLogLoadMore()" aria-label="${escapeHtml(t('activity_log.load_more'))}">${escapeHtml(t('activity_log.load_more'))}</button>`
|
||||
: '';
|
||||
|
||||
const countLabel = t('activity_log.n_entries', { n: _total });
|
||||
|
||||
return `<div class="al-list-header">
|
||||
<span class="al-count tabular-nums">${escapeHtml(countLabel)}</span>
|
||||
<div class="al-live-indicator" id="al-live-indicator" aria-live="polite">
|
||||
<span class="al-live-dot" aria-hidden="true"></span>
|
||||
<span>${escapeHtml(t('activity_log.live'))}</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="al-list" role="log" aria-label="${escapeHtml(t('activity_log.title'))}" aria-live="polite">
|
||||
${rows}
|
||||
</div>
|
||||
${loadMore}`;
|
||||
}
|
||||
|
||||
// ─── Delegated click handler for entry rows and entity links ─
|
||||
|
||||
let _delegatedClickAttached = false;
|
||||
|
||||
/** Collapse the export dropdown if open (idempotent). */
|
||||
function _closeExportMenu(): void {
|
||||
const wrap = document.getElementById('tab-activity_log')
|
||||
?.querySelector<HTMLElement>('.al-export-wrap.open');
|
||||
if (!wrap) return;
|
||||
wrap.classList.remove('open');
|
||||
wrap.querySelector('[data-al-export-toggle]')?.setAttribute('aria-expanded', 'false');
|
||||
}
|
||||
|
||||
function _attachDelegatedClicks(): void {
|
||||
if (_delegatedClickAttached) return;
|
||||
const panel = document.getElementById('tab-activity_log');
|
||||
if (!panel) return;
|
||||
_delegatedClickAttached = true;
|
||||
|
||||
panel.addEventListener('click', (e: MouseEvent) => {
|
||||
const target = e.target as HTMLElement;
|
||||
|
||||
// Export menu: toggle button opens/closes the CSV/JSON dropdown.
|
||||
const exportToggle = target.closest<HTMLElement>('[data-al-export-toggle]');
|
||||
if (exportToggle) {
|
||||
const wrap = exportToggle.closest<HTMLElement>('.al-export-wrap');
|
||||
const open = wrap?.classList.toggle('open') ?? false;
|
||||
exportToggle.setAttribute('aria-expanded', open ? 'true' : 'false');
|
||||
return;
|
||||
}
|
||||
// Export menu: a menu item's inline onclick triggers the download (it
|
||||
// runs first, on the deeper element) — we just collapse the menu after.
|
||||
if (target.closest('.al-export-menu')) {
|
||||
_closeExportMenu();
|
||||
return;
|
||||
}
|
||||
// Any other click in the panel dismisses an open export menu, then
|
||||
// continues to row / entity handling below.
|
||||
_closeExportMenu();
|
||||
|
||||
// Entity navigation: click on data-entity-type button
|
||||
const entityBtn = target.closest<HTMLElement>('button.al-entity-link[data-entity-type]');
|
||||
if (entityBtn) {
|
||||
e.stopPropagation();
|
||||
const entityType = entityBtn.dataset.entityType ?? '';
|
||||
const entityId = entityBtn.dataset.entityId ?? '';
|
||||
activityLogNavigateToEntity(entityType, entityId);
|
||||
return;
|
||||
}
|
||||
|
||||
// Entry row toggle: click on al-entry-row with data-toggle-id
|
||||
const row = target.closest<HTMLElement>('.al-entry-row[data-toggle-id]');
|
||||
if (row) {
|
||||
const entryId = row.dataset.toggleId ?? '';
|
||||
if (entryId) activityLogToggleDetail(entryId);
|
||||
return;
|
||||
}
|
||||
});
|
||||
|
||||
panel.addEventListener('keydown', (e: KeyboardEvent) => {
|
||||
// Escape closes the export menu and restores focus to its trigger.
|
||||
if (e.key === 'Escape') {
|
||||
const toggle = panel.querySelector<HTMLElement>('.al-export-wrap.open [data-al-export-toggle]');
|
||||
if (toggle) {
|
||||
_closeExportMenu();
|
||||
toggle.focus();
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (e.key !== 'Enter' && e.key !== ' ') return;
|
||||
const row = (e.target as HTMLElement).closest<HTMLElement>('.al-entry-row[data-toggle-id]');
|
||||
if (row) {
|
||||
e.preventDefault();
|
||||
const entryId = row.dataset.toggleId ?? '';
|
||||
if (entryId) activityLogToggleDetail(entryId);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Full panel render ───────────────────────────────────────
|
||||
|
||||
function _render(): void {
|
||||
const panel = document.getElementById('tab-activity_log');
|
||||
if (!panel) return;
|
||||
|
||||
panel.innerHTML = `<div class="al-panel">
|
||||
${_renderFilterToolbar()}
|
||||
<div id="al-list-container" class="al-list-container">
|
||||
${_renderList()}
|
||||
</div>
|
||||
</div>`;
|
||||
|
||||
_attachDelegatedClicks();
|
||||
}
|
||||
|
||||
// ─── Partial re-render helpers ───────────────────────────────
|
||||
|
||||
function _updateListContainer(): void {
|
||||
const container = document.getElementById('al-list-container');
|
||||
if (!container) return;
|
||||
container.innerHTML = _renderList();
|
||||
}
|
||||
|
||||
// ─── Data fetching ───────────────────────────────────────────
|
||||
|
||||
/** Surface a loading affordance only when a request is slow enough to notice. */
|
||||
function _showDelayedBusy(): void {
|
||||
if (!_loading) return;
|
||||
if (_entries.length === 0) {
|
||||
// Nothing to keep on screen — fall back to the full spinner.
|
||||
_showSpinner = true;
|
||||
_updateListContainer();
|
||||
} else {
|
||||
// Re-query of a populated list: keep the current rows, just dim them.
|
||||
const c = document.getElementById('al-list-container');
|
||||
c?.classList.add('al-busy');
|
||||
c?.setAttribute('aria-busy', 'true');
|
||||
}
|
||||
}
|
||||
|
||||
/** Clear all loading affordances (timer, spinner flag, busy dim). Idempotent. */
|
||||
function _clearBusy(): void {
|
||||
if (_loadingDelayTimer) {
|
||||
clearTimeout(_loadingDelayTimer);
|
||||
_loadingDelayTimer = null;
|
||||
}
|
||||
_showSpinner = false;
|
||||
const c = document.getElementById('al-list-container');
|
||||
c?.classList.remove('al-busy');
|
||||
c?.removeAttribute('aria-busy');
|
||||
}
|
||||
|
||||
async function _fetchPage(beforeSeq: number | null = null, append = false): Promise<void> {
|
||||
if (_loading) return;
|
||||
_loading = true;
|
||||
if (!append) {
|
||||
// Reset the cursor for a fresh query, but DON'T clear `_entries` — keep
|
||||
// the current rows on screen so filtering an already-populated list
|
||||
// never flashes the full "Loading" state (the new results replace them
|
||||
// on arrival).
|
||||
_nextBeforeSeq = null;
|
||||
_hasMore = false;
|
||||
}
|
||||
|
||||
if (!_hasLoadedOnce && !append) {
|
||||
// Genuine first load — there's nothing to show yet, so the spinner is
|
||||
// the correct (and expected) initial state. Show it immediately.
|
||||
_showSpinner = true;
|
||||
_updateListContainer();
|
||||
} else if (!append) {
|
||||
// Re-query (filter change / language change): defer any loading hint so
|
||||
// near-instant responses show nothing at all; a slow request gets a
|
||||
// subtle dim after the delay.
|
||||
if (_loadingDelayTimer) clearTimeout(_loadingDelayTimer);
|
||||
_loadingDelayTimer = setTimeout(_showDelayedBusy, 180);
|
||||
}
|
||||
// append (load-more): keep existing rows, no loading indicator.
|
||||
|
||||
try {
|
||||
const qs = _buildQuery(beforeSeq);
|
||||
const res = await fetchWithAuth(`/activity-log${qs}`);
|
||||
if (!res || !res.ok) {
|
||||
throw new Error(`HTTP ${res?.status}`);
|
||||
}
|
||||
const page: ActivityPage = await res.json();
|
||||
// API returns each page oldest-first within the page; reverse to newest-first
|
||||
// so the in-memory list is newest at index 0 (top of the rendered log).
|
||||
const pageEntries = [...page.entries].reverse();
|
||||
if (append) {
|
||||
_entries = [..._entries, ...pageEntries];
|
||||
} else {
|
||||
_entries = pageEntries;
|
||||
}
|
||||
_nextBeforeSeq = page.next_before_seq;
|
||||
_hasMore = page.has_more;
|
||||
_total = page.total;
|
||||
_hasLoadedOnce = true;
|
||||
// Clear loading affordances BEFORE rendering so a zero-result page
|
||||
// renders the empty state (not the spinner) and a re-query swaps in the
|
||||
// fresh, undimmed rows.
|
||||
_clearBusy();
|
||||
_loading = false;
|
||||
_updateListContainer();
|
||||
} catch (e: unknown) {
|
||||
if (e && typeof e === 'object' && 'isAuth' in e) return;
|
||||
_clearBusy();
|
||||
const container = document.getElementById('al-list-container');
|
||||
if (container) {
|
||||
container.innerHTML = `<div class="al-state al-error" role="alert">
|
||||
<span class="al-state-icon" aria-hidden="true">${ICON_SEVERITY_ERR}</span>
|
||||
<p>${escapeHtml(t('activity_log.error'))}</p>
|
||||
</div>`;
|
||||
}
|
||||
} finally {
|
||||
_loading = false;
|
||||
_clearBusy();
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Filter re-query with debounce for text fields ───────────
|
||||
|
||||
function _requery(debounce = false): void {
|
||||
if (debounce) {
|
||||
if (_debounceTimer) clearTimeout(_debounceTimer);
|
||||
_debounceTimer = setTimeout(() => { _fetchPage(null, false); }, 350);
|
||||
} else {
|
||||
_fetchPage(null, false);
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Live event handling ──────────────────────────────────────
|
||||
|
||||
function _entryPassesFilters(entry: ActivityEntry): boolean {
|
||||
if (_filters.categories.length && !_filters.categories.includes(entry.category)) return false;
|
||||
if (_filters.severities.length && !_filters.severities.includes(entry.severity)) return false;
|
||||
if (_filters.actor && entry.actor !== _filters.actor) return false;
|
||||
if (_filters.entity_type && entry.entity_type !== _filters.entity_type) return false;
|
||||
if (_filters.q) {
|
||||
const q = _filters.q.toLowerCase();
|
||||
if (!entry.message.toLowerCase().includes(q) &&
|
||||
!entry.action.toLowerCase().includes(q) &&
|
||||
!entry.actor.toLowerCase().includes(q)) return false;
|
||||
}
|
||||
// Date range filters: if an entry is brand-new it passes "since" checks trivially
|
||||
if (_filters.since) {
|
||||
const sinceMs = new Date(_filters.since).getTime();
|
||||
if (!isNaN(sinceMs) && new Date(entry.ts).getTime() < sinceMs) return false;
|
||||
}
|
||||
if (_filters.until) {
|
||||
const untilMs = new Date(_filters.until).getTime();
|
||||
if (!isNaN(untilMs) && new Date(entry.ts).getTime() > untilMs) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function _prependLiveEntry(entry: ActivityEntry): void {
|
||||
if (!_entryPassesFilters(entry)) return;
|
||||
|
||||
_entries = [entry, ..._entries];
|
||||
_total = _total + 1;
|
||||
|
||||
// Prepend the row into the existing list (no full re-render for performance)
|
||||
const list = document.getElementById('tab-activity_log')?.querySelector('.al-list');
|
||||
if (list) {
|
||||
const html = _renderEntryRow(entry, true);
|
||||
list.insertAdjacentHTML('afterbegin', html);
|
||||
// Animate the new entry
|
||||
const firstRow = list.firstElementChild as HTMLElement | null;
|
||||
if (firstRow) {
|
||||
requestAnimationFrame(() => { firstRow.classList.add('al-entry-appear'); });
|
||||
}
|
||||
// Update count badge
|
||||
const countEl = list.closest('.al-panel')?.querySelector('.al-count');
|
||||
if (countEl) countEl.textContent = t('activity_log.n_entries', { n: _total });
|
||||
} else {
|
||||
_updateListContainer();
|
||||
}
|
||||
}
|
||||
|
||||
function _startLiveUpdates(): void {
|
||||
if (_liveEventListener) return;
|
||||
_liveEventListener = (e: Event) => {
|
||||
const ce = e as CustomEvent;
|
||||
const entry = ce.detail?.entry as ActivityEntry | undefined;
|
||||
if (!entry) return;
|
||||
_prependLiveEntry(entry);
|
||||
};
|
||||
document.addEventListener('server:activity_logged', _liveEventListener);
|
||||
}
|
||||
|
||||
// ─── Public window-exposed interaction functions ──────────────
|
||||
|
||||
export function activityLogToggleDetail(entryId: string): void {
|
||||
if (_expandedIds.has(entryId)) {
|
||||
_expandedIds.delete(entryId);
|
||||
} else {
|
||||
_expandedIds.add(entryId);
|
||||
}
|
||||
// Update just the affected row
|
||||
const panel = document.getElementById('tab-activity_log');
|
||||
if (!panel) return;
|
||||
const row = panel.querySelector(`[data-al-id="${CSS.escape(entryId)}"]`);
|
||||
if (!row) return;
|
||||
const entry = _entries.find(e => e.id === entryId);
|
||||
if (!entry) return;
|
||||
row.outerHTML = _renderEntryRow(entry, false);
|
||||
}
|
||||
|
||||
export function activityLogToggleCat(cat: string): void {
|
||||
const idx = _filters.categories.indexOf(cat);
|
||||
if (idx >= 0) {
|
||||
_filters.categories = _filters.categories.filter(c => c !== cat);
|
||||
} else {
|
||||
_filters.categories = [..._filters.categories, cat];
|
||||
}
|
||||
_render();
|
||||
_requery();
|
||||
}
|
||||
|
||||
export function activityLogToggleSev(sev: string): void {
|
||||
const idx = _filters.severities.indexOf(sev);
|
||||
if (idx >= 0) {
|
||||
_filters.severities = _filters.severities.filter(s => s !== sev);
|
||||
} else {
|
||||
_filters.severities = [..._filters.severities, sev];
|
||||
}
|
||||
_render();
|
||||
_requery();
|
||||
}
|
||||
|
||||
export function activityLogOnSearch(val: string): void {
|
||||
_filters.q = val;
|
||||
_requery(true);
|
||||
}
|
||||
|
||||
export function activityLogOnActor(val: string): void {
|
||||
_filters.actor = val.trim();
|
||||
_requery(true);
|
||||
}
|
||||
|
||||
export function activityLogOnEntityType(val: string): void {
|
||||
_filters.entity_type = val.trim();
|
||||
_requery(true);
|
||||
}
|
||||
|
||||
export function activityLogOnSince(val: string): void {
|
||||
_filters.since = val;
|
||||
_requery();
|
||||
}
|
||||
|
||||
export function activityLogOnUntil(val: string): void {
|
||||
_filters.until = val;
|
||||
_requery();
|
||||
}
|
||||
|
||||
export function activityLogClearFilters(): void {
|
||||
_filters.categories = [];
|
||||
_filters.severities = [];
|
||||
_filters.actor = '';
|
||||
_filters.entity_type = '';
|
||||
_filters.since = '';
|
||||
_filters.until = '';
|
||||
_filters.q = '';
|
||||
_render();
|
||||
_requery();
|
||||
}
|
||||
|
||||
export function activityLogPreset(key: string): void {
|
||||
// Reset all filters first
|
||||
_filters.categories = [];
|
||||
_filters.severities = [];
|
||||
_filters.actor = '';
|
||||
_filters.entity_type = '';
|
||||
_filters.q = '';
|
||||
_filters.until = '';
|
||||
|
||||
switch (key) {
|
||||
case 'today': {
|
||||
const todayStart = new Date();
|
||||
todayStart.setHours(0, 0, 0, 0);
|
||||
// datetime-local format: YYYY-MM-DDTHH:MM
|
||||
_filters.since = todayStart.toISOString().slice(0, 16);
|
||||
break;
|
||||
}
|
||||
case 'errors':
|
||||
_filters.severities = ['error'];
|
||||
_filters.since = '';
|
||||
break;
|
||||
case 'auth':
|
||||
_filters.categories = ['auth'];
|
||||
_filters.since = '';
|
||||
break;
|
||||
case 'devices':
|
||||
_filters.categories = ['device'];
|
||||
_filters.since = '';
|
||||
break;
|
||||
}
|
||||
_render();
|
||||
_requery();
|
||||
}
|
||||
|
||||
export function activityLogLoadMore(): void {
|
||||
if (_hasMore && !_loading) {
|
||||
_fetchPage(_nextBeforeSeq, true);
|
||||
}
|
||||
}
|
||||
|
||||
export async function activityLogExport(format: 'csv' | 'json'): Promise<void> {
|
||||
try {
|
||||
showToast(t('activity_log.export.downloading'), 'info');
|
||||
const qs = _buildQuery(null);
|
||||
const sep = qs ? '&' : '?';
|
||||
const url = `/activity-log/export${qs}${sep}format=${format}`;
|
||||
const res = await fetchWithAuth(url);
|
||||
if (!res || !res.ok) throw new Error(`HTTP ${res?.status}`);
|
||||
|
||||
const blob = await res.blob();
|
||||
const blobUrl = URL.createObjectURL(blob);
|
||||
const now = new Date().toISOString().slice(0, 19).replace(/:/g, '-');
|
||||
const filename = `ledgrab-activity-${now}.${format}`;
|
||||
const a = document.createElement('a');
|
||||
a.href = blobUrl;
|
||||
a.download = filename;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
setTimeout(() => URL.revokeObjectURL(blobUrl), 10000);
|
||||
} catch (e: unknown) {
|
||||
if (e && typeof e === 'object' && 'isAuth' in e) return;
|
||||
showToast(t('activity_log.export.error'), 'error');
|
||||
}
|
||||
}
|
||||
|
||||
export function activityLogNavigateToEntity(entityType: string, entityId: string): void {
|
||||
const nav = _ENTITY_NAV[entityType];
|
||||
if (!nav || !entityId) return;
|
||||
navigateToCard(nav.tab, nav.subTab, null, nav.attr, entityId);
|
||||
}
|
||||
|
||||
// ─── Public helpers for Phase 6 (Dashboard widget + Settings export) ────────
|
||||
|
||||
/**
|
||||
* Fetch the N most-recent activity log entries without affecting the full-tab
|
||||
* state (separate request, no state mutations).
|
||||
*/
|
||||
export async function fetchRecentEntries(limit = 5): Promise<ActivityEntry[]> {
|
||||
try {
|
||||
const res = await fetchWithAuth(`/activity-log?limit=${limit}`);
|
||||
if (!res || !res.ok) return [];
|
||||
const page: ActivityPage = await res.json();
|
||||
// API returns oldest-first within page; reverse for newest-first.
|
||||
return [...page.entries].reverse();
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a compact single-line entry row for the Dashboard widget.
|
||||
* Re-uses the severity icon / class helpers and escapeHtml from the full viewer
|
||||
* so the visual language is consistent.
|
||||
*/
|
||||
export function renderCompactEntry(entry: ActivityEntry): string {
|
||||
const relTime = formatRelativeTime(entry.ts);
|
||||
const sevIcon = _severityIcon(entry.severity);
|
||||
const sevClass = _severityClass(entry.severity);
|
||||
return `<div class="al-compact-row al-sev ${sevClass}" title="${_escapeAttr(entry.ts)}">
|
||||
<span class="al-compact-icon" aria-label="${_escapeAttr(t(`activity_log.severity.${entry.severity}`))}">${sevIcon}</span>
|
||||
<span class="al-compact-time tabular-nums" data-reltime="${_escapeAttr(entry.ts)}">${escapeHtml(relTime)}</span>
|
||||
<span class="al-compact-msg">${escapeHtml(localizeMessage(entry))}</span>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
// ─── Main loader (registered with tab-registry) ─────────────
|
||||
|
||||
export async function loadActivityLog(): Promise<void> {
|
||||
const panel = document.getElementById('tab-activity_log');
|
||||
if (!panel) return;
|
||||
|
||||
_initialized = true;
|
||||
_render();
|
||||
await _fetchPage(null, false);
|
||||
_startLiveUpdates();
|
||||
ensureRelativeTimeTicker();
|
||||
|
||||
// Re-render on language change (baked-in t() calls)
|
||||
document.addEventListener('languageChanged', _onLanguageChanged);
|
||||
}
|
||||
|
||||
function _onLanguageChanged(): void {
|
||||
if (!_initialized) return;
|
||||
_render();
|
||||
_fetchPage(null, false);
|
||||
}
|
||||
@@ -63,6 +63,7 @@ const REORDERABLE_SECTIONS: readonly string[] = [
|
||||
'playlists',
|
||||
'sync-clocks',
|
||||
'targets',
|
||||
'recent-activity',
|
||||
] as const;
|
||||
|
||||
const SECTION_LABEL_KEYS: Record<string, string> = {
|
||||
@@ -73,6 +74,7 @@ const SECTION_LABEL_KEYS: Record<string, string> = {
|
||||
playlists: 'dashboard.section.playlists',
|
||||
'sync-clocks': 'dashboard.section.sync_clocks',
|
||||
targets: 'dashboard.section.targets',
|
||||
'recent-activity': 'dashboard.section.recent_activity',
|
||||
};
|
||||
|
||||
const PERF_CELL_LABEL_KEYS: Record<string, string> = {
|
||||
|
||||
@@ -25,6 +25,7 @@ export type SectionKey =
|
||||
| 'playlists'
|
||||
| 'sync-clocks'
|
||||
| 'targets'
|
||||
| 'recent-activity'
|
||||
// Reserved registry keys for v1.1+ (so saved layouts forward-compat).
|
||||
| 'audio-meters'
|
||||
| 'alerts'
|
||||
@@ -155,6 +156,7 @@ export const DEFAULT_LAYOUT: DashboardLayoutV1 = {
|
||||
_defaultSection('playlists'),
|
||||
_defaultSection('sync-clocks'),
|
||||
_defaultSection('targets'),
|
||||
_defaultSection('recent-activity'),
|
||||
],
|
||||
perfCells: [
|
||||
_defaultPerfCell('patches'),
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
import { apiKey, _dashboardLoading, set_dashboardLoading, dashboardPollInterval, setDashboardPollInterval, colorStripSourcesCache, devicesCache, outputTargetsCache } from '../core/state.ts';
|
||||
import { API_BASE, getHeaders, fetchWithAuth, escapeHtml, fetchMetricsHistory } from '../core/api.ts';
|
||||
import { t } from '../core/i18n.ts';
|
||||
import { showToast, showConfirm, formatUptime, formatCompact, setTabRefreshing } from '../core/ui.ts';
|
||||
import { showToast, showConfirm, formatUptime, formatCompact, setTabRefreshing, ensureRelativeTimeTicker } from '../core/ui.ts';
|
||||
import { renderPerfSection, renderPerfModeToggle, setPerfMode, initPerfCharts, startPerfPolling, stopPerfPolling, updateActivePatches, updateTotalFps, updateTotalCaptureFps, updateTotalCaptureFpsActual, updateTotalErrors, updateDevices, updateNetworkThroughput, updateDeviceLatency, updateSendTiming, rerenderPerfGrid } from './perf-charts.ts';
|
||||
import { startAutoRefresh, updateTabBadge } from './tabs.ts';
|
||||
import { isActiveTab } from '../core/tab-registry.ts';
|
||||
@@ -21,6 +21,8 @@ import { renderDeviceIconSvg } from '../core/device-icons.ts';
|
||||
import { createFpsSparkline } from '../core/chart-utils.ts';
|
||||
import { getOrderedSections, isSectionVisible, getSection, subscribeDashboardLayout, getGlobalConfig } from './dashboard-layout.ts';
|
||||
import { mountCardModeToggle } from './card-modes.ts';
|
||||
import { ICON_ACTIVITY_LOG } from '../core/icons.ts';
|
||||
import { fetchRecentEntries, renderCompactEntry, ActivityEntry } from './activity-log.ts';
|
||||
|
||||
function _applyGlobalLayoutAttrs(): void {
|
||||
const c = document.getElementById('dashboard-content');
|
||||
@@ -573,6 +575,84 @@ function renderDashboardPlaylist(playlist: ScenePlaylist): string {
|
||||
</div>`;
|
||||
}
|
||||
|
||||
// ─── Recent Activity widget (Dashboard) ─────────────────────
|
||||
|
||||
const RECENT_ACTIVITY_LIMIT = 5;
|
||||
let _recentActivityLiveListener: ((e: Event) => void) | null = null;
|
||||
|
||||
/** Fetch recent entries and populate the widget list container.
|
||||
* Skips the network fetch when the widget already contains live entries
|
||||
* (dal-list class is set by _renderRecentActivityList on first mount) so
|
||||
* unrelated dashboard re-renders never re-fetch or flash the widget. */
|
||||
async function _loadRecentActivityWidget(): Promise<void> {
|
||||
const list = document.getElementById('dashboard-recent-activity-list');
|
||||
if (!list) return;
|
||||
|
||||
// Widget already populated — only wire up live listener and ticker;
|
||||
// don't re-fetch or overwrite the live content.
|
||||
if (list.classList.contains('dal-list')) {
|
||||
ensureRelativeTimeTicker();
|
||||
_startRecentActivityLive();
|
||||
return;
|
||||
}
|
||||
|
||||
const entries = await fetchRecentEntries(RECENT_ACTIVITY_LIMIT);
|
||||
_renderRecentActivityList(list, entries);
|
||||
|
||||
// Start relative-time ticker (idempotent — shared with the Activity tab)
|
||||
ensureRelativeTimeTicker();
|
||||
|
||||
// Start live listener (idempotent)
|
||||
_startRecentActivityLive();
|
||||
}
|
||||
|
||||
function _renderRecentActivityList(list: HTMLElement, entries: ActivityEntry[]): void {
|
||||
if (!entries || entries.length === 0) {
|
||||
list.className = '';
|
||||
list.innerHTML = `<div class="al-state al-empty dal-empty" role="status">
|
||||
<span class="al-state-icon" aria-hidden="true">${ICON_ACTIVITY_LOG}</span>
|
||||
<p>${escapeHtml(t('activity_log.empty_no_filters'))}</p>
|
||||
</div>`;
|
||||
return;
|
||||
}
|
||||
list.className = 'dal-list';
|
||||
list.innerHTML = entries.map(e => renderCompactEntry(e)).join('');
|
||||
}
|
||||
|
||||
function _stopRecentActivityLive(): void {
|
||||
if (_recentActivityLiveListener) {
|
||||
document.removeEventListener('server:activity_logged', _recentActivityLiveListener);
|
||||
_recentActivityLiveListener = null;
|
||||
}
|
||||
}
|
||||
|
||||
function _startRecentActivityLive(): void {
|
||||
// Always tear down first so we never stack listeners across loadDashboard calls.
|
||||
_stopRecentActivityLive();
|
||||
_recentActivityLiveListener = (e: Event) => {
|
||||
const ce = e as CustomEvent;
|
||||
const entry = ce.detail?.entry;
|
||||
if (!entry) return;
|
||||
const list = document.getElementById('dashboard-recent-activity-list');
|
||||
// No-op when the widget isn't mounted (section hidden or not yet rendered).
|
||||
if (!list) return;
|
||||
if (!list.classList.contains('dal-list')) {
|
||||
// Transition from empty-state to list on the first live event
|
||||
_renderRecentActivityList(list, [entry]);
|
||||
return;
|
||||
}
|
||||
// Surgically prepend the new row and cap at N — no loadDashboard().
|
||||
list.insertAdjacentHTML('afterbegin', renderCompactEntry(entry));
|
||||
const rows = list.querySelectorAll('.al-compact-row');
|
||||
if (rows.length > RECENT_ACTIVITY_LIMIT) {
|
||||
for (let i = RECENT_ACTIVITY_LIMIT; i < rows.length; i++) {
|
||||
rows[i].remove();
|
||||
}
|
||||
}
|
||||
};
|
||||
document.addEventListener('server:activity_logged', _recentActivityLiveListener);
|
||||
}
|
||||
|
||||
let _pollDebounce: ReturnType<typeof setTimeout> | undefined = undefined;
|
||||
/** Called from the transport-bar poll cycler (and any legacy callers
|
||||
* that might still reference `window.changeDashboardPollInterval`). */
|
||||
@@ -679,6 +759,127 @@ function _sectionContent(sectionKey: string, itemsHtml: string): string {
|
||||
return `<div class="dashboard-section-content"${isCollapsed ? ' style="display:none"' : ''}>${itemsHtml}</div>`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reconcile the `.dashboard-dynamic` container against newly-built HTML
|
||||
* without a wholesale innerHTML replacement.
|
||||
*
|
||||
* Algorithm:
|
||||
* 1. Parse `newHtml` into a detached container.
|
||||
* 2. Build a map of existing live sections keyed by data-section.
|
||||
* 3. For each section in the new HTML (in order):
|
||||
* a. If the live DOM has that section AND it is content-stable
|
||||
* (recent-activity with live list) OR its outerHTML hasn't
|
||||
* changed — keep the live element.
|
||||
* b. Otherwise replace / insert with the new element.
|
||||
* 4. Remove sections that no longer appear in the new HTML.
|
||||
* 5. Re-order to match the new order (move nodes, no recreation).
|
||||
*
|
||||
* The `recent-activity` section is treated as content-stable once
|
||||
* its list has been populated (dal-list class), mirroring the perf-
|
||||
* persistent pattern. The freshly-built loading placeholder in
|
||||
* `newHtml` is never compared against the live-entry list — instead
|
||||
* the live DOM node is always kept when it has real content.
|
||||
*/
|
||||
function _reconcileDynamicSections(dynamic: HTMLElement, newHtml: string): void {
|
||||
// Parse incoming HTML into a scratch container.
|
||||
const scratch = document.createElement('div');
|
||||
scratch.innerHTML = newHtml;
|
||||
|
||||
// Gather incoming sections in order.
|
||||
const incoming = Array.from(
|
||||
scratch.querySelectorAll<HTMLElement>(':scope > .dashboard-section[data-section]')
|
||||
);
|
||||
|
||||
// If the new HTML contains non-section top-level nodes (e.g. the
|
||||
// `.dashboard-no-targets` placeholder shown when there are no entities),
|
||||
// fall back to a simple innerHTML swap — this path is rare and the
|
||||
// no-entities state doesn't have live widgets worth preserving.
|
||||
const totalTopLevel = scratch.children.length;
|
||||
if (totalTopLevel !== incoming.length) {
|
||||
if (dynamic.innerHTML !== newHtml) dynamic.innerHTML = newHtml;
|
||||
return;
|
||||
}
|
||||
|
||||
// Drop any stray non-section top-level nodes left over from a previous
|
||||
// state (e.g. the `.dashboard-no-targets` placeholder shown when there
|
||||
// were zero entities). The reconcile pass below only manages
|
||||
// `.dashboard-section` children, so without this sweep that orphan node
|
||||
// would linger over the freshly-populated dashboard.
|
||||
for (const child of Array.from(dynamic.children)) {
|
||||
if (!child.matches('.dashboard-section[data-section]')) child.remove();
|
||||
}
|
||||
|
||||
// Index live sections by key.
|
||||
const liveMap = new Map<string, HTMLElement>();
|
||||
for (const el of Array.from(
|
||||
dynamic.querySelectorAll<HTMLElement>(':scope > .dashboard-section[data-section]')
|
||||
)) {
|
||||
liveMap.set(el.dataset.section as string, el);
|
||||
}
|
||||
|
||||
// Collect the keys that should remain (in new order).
|
||||
const newKeys = new Set(incoming.map(el => el.dataset.section as string));
|
||||
|
||||
// Remove sections that are no longer present.
|
||||
for (const [key, el] of liveMap) {
|
||||
if (!newKeys.has(key)) el.remove();
|
||||
}
|
||||
|
||||
// Walk incoming sections in order and reconcile each one.
|
||||
let insertBefore: HTMLElement | null = null; // node to insert before (null = append)
|
||||
for (let i = incoming.length - 1; i >= 0; i--) {
|
||||
const newEl = incoming[i];
|
||||
const key = newEl.dataset.section as string;
|
||||
const live = liveMap.get(key);
|
||||
|
||||
let keep: HTMLElement;
|
||||
|
||||
if (live) {
|
||||
// Content-stable guard: the recent-activity section must not be
|
||||
// replaced once it holds live entries — the new HTML only has the
|
||||
// loading placeholder and would wipe the list.
|
||||
const isRecentActivity = key === 'recent-activity';
|
||||
const raList = isRecentActivity
|
||||
? live.querySelector('#dashboard-recent-activity-list')
|
||||
: null;
|
||||
const raIsPopulated = raList !== null && raList.classList.contains('dal-list');
|
||||
|
||||
if (raIsPopulated) {
|
||||
// Always keep the live recent-activity section as-is.
|
||||
keep = live;
|
||||
} else if (live.outerHTML === newEl.outerHTML) {
|
||||
// Unchanged section — keep live DOM, no mutation.
|
||||
keep = live;
|
||||
} else {
|
||||
// Section content changed — replace.
|
||||
live.replaceWith(newEl);
|
||||
liveMap.set(key, newEl);
|
||||
keep = newEl;
|
||||
}
|
||||
} else {
|
||||
// New section — insert it.
|
||||
dynamic.appendChild(newEl); // temporary placement; ordering pass below
|
||||
liveMap.set(key, newEl);
|
||||
keep = newEl;
|
||||
}
|
||||
|
||||
// Re-order: after the ordering loop (reverse walk) each `keep`
|
||||
// should end up just before the node we placed in the previous
|
||||
// iteration (i+1). Using insertBefore to build correct order.
|
||||
if (insertBefore === null) {
|
||||
// Last in order — move to end of dynamic.
|
||||
if (keep.nextElementSibling !== null || keep.parentElement !== dynamic) {
|
||||
dynamic.appendChild(keep);
|
||||
}
|
||||
} else {
|
||||
if (keep.nextElementSibling !== insertBefore || keep.parentElement !== dynamic) {
|
||||
dynamic.insertBefore(keep, insertBefore);
|
||||
}
|
||||
}
|
||||
insertBefore = keep;
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadDashboard(forceFullRender: boolean = false): Promise<void> {
|
||||
if (_dashboardLoading) return;
|
||||
set_dashboardLoading(true);
|
||||
@@ -763,7 +964,21 @@ export async function loadDashboard(forceFullRender: boolean = false): Promise<v
|
||||
let dynamicHtml = '';
|
||||
let runningIds: any[] = [];
|
||||
if (targets.length === 0 && automations.length === 0 && scenePresets.length === 0 && playlists.length === 0 && syncClocks.length === 0 && haStatus.total_sources === 0 && mqttStatus.total_sources === 0) {
|
||||
dynamicHtml = `<div class="dashboard-no-targets">${t('dashboard.no_targets')}</div>`;
|
||||
const _raSection = isSectionVisible('recent-activity') ? `<div class="dashboard-section" data-section="recent-activity">
|
||||
${_sectionHeader('recent-activity', t('dashboard.section.recent_activity'), '')}
|
||||
${_sectionContent('recent-activity', `<div id="dashboard-recent-activity-list" class="dal-loading" aria-live="polite" aria-label="${escapeHtml(t('dashboard.section.recent_activity'))}">
|
||||
<div class="al-state al-loading">
|
||||
<div class="al-spinner"></div>
|
||||
<span>${escapeHtml(t('activity_log.loading'))}</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="dal-footer">
|
||||
<button class="btn btn-ghost btn-sm dal-view-all" onclick="switchTab('activity_log')" aria-label="${escapeHtml(t('dashboard.recent_activity.view_all'))}">
|
||||
${escapeHtml(t('dashboard.recent_activity.view_all'))} →
|
||||
</button>
|
||||
</div>`)}
|
||||
</div>` : '';
|
||||
dynamicHtml = `<div class="dashboard-no-targets">${t('dashboard.no_targets')}</div>${_raSection}`;
|
||||
} else {
|
||||
const enriched = targets.map(target => ({
|
||||
...target,
|
||||
@@ -1003,6 +1218,23 @@ export async function loadDashboard(forceFullRender: boolean = false): Promise<v
|
||||
</div>`;
|
||||
}
|
||||
|
||||
// Recent Activity section — registered like any other section so it
|
||||
// participates in layout ordering, show/hide, and Customize panel.
|
||||
sectionFragments['recent-activity'] = `<div class="dashboard-section" data-section="recent-activity">
|
||||
${_sectionHeader('recent-activity', t('dashboard.section.recent_activity'), '')}
|
||||
${_sectionContent('recent-activity', `<div id="dashboard-recent-activity-list" class="dal-loading" aria-live="polite" aria-label="${escapeHtml(t('dashboard.section.recent_activity'))}">
|
||||
<div class="al-state al-loading">
|
||||
<div class="al-spinner"></div>
|
||||
<span>${escapeHtml(t('activity_log.loading'))}</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="dal-footer">
|
||||
<button class="btn btn-ghost btn-sm dal-view-all" onclick="switchTab('activity_log')" aria-label="${escapeHtml(t('dashboard.recent_activity.view_all'))}">
|
||||
${escapeHtml(t('dashboard.recent_activity.view_all'))} →
|
||||
</button>
|
||||
</div>`)}
|
||||
</div>`;
|
||||
|
||||
// Now assemble in layout-driven order, skipping invisible
|
||||
// sections and the perf section (which is always rendered
|
||||
// separately at the top for chart-persistence reasons).
|
||||
@@ -1043,8 +1275,8 @@ export async function loadDashboard(forceFullRender: boolean = false): Promise<v
|
||||
existingPerf.style.display = perfVisible ? '' : 'none';
|
||||
}
|
||||
const dynamic = container.querySelector('.dashboard-dynamic');
|
||||
if (dynamic && dynamic.innerHTML !== dynamicHtml) {
|
||||
dynamic.innerHTML = dynamicHtml;
|
||||
if (dynamic) {
|
||||
_reconcileDynamicSections(dynamic as HTMLElement, dynamicHtml);
|
||||
}
|
||||
_applyGlobalLayoutAttrs();
|
||||
}
|
||||
@@ -1062,6 +1294,9 @@ export async function loadDashboard(forceFullRender: boolean = false): Promise<v
|
||||
_startUptimeTimer();
|
||||
startPerfPolling();
|
||||
|
||||
// Async-load the Recent Activity widget (non-blocking — never blocks the main render).
|
||||
_loadRecentActivityWidget().catch(() => { /* widget failure is non-fatal */ });
|
||||
|
||||
} catch (error) {
|
||||
if (error.isAuth) return;
|
||||
console.error('Failed to load dashboard:', error);
|
||||
|
||||
@@ -146,6 +146,10 @@ export function switchSettingsTab(tabId: string): void {
|
||||
if (tabId === 'notifications') {
|
||||
initNotificationsPanel();
|
||||
}
|
||||
// Lazy-load activity log settings when switching to that tab
|
||||
if (tabId === 'activity_log') {
|
||||
loadActivityLogSettings();
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Log Viewer ────────────────────────────────────────────
|
||||
@@ -526,6 +530,7 @@ export function openSettingsModal(): void {
|
||||
loadLogLevel();
|
||||
loadShutdownAction();
|
||||
loadDaylightTimezone();
|
||||
loadActivityLogSettings();
|
||||
_seedRailFooter();
|
||||
// Refresh the update status so the rail badge ("update available" pill
|
||||
// on the Updates tab) is current when the modal opens — it would
|
||||
@@ -1200,3 +1205,110 @@ export function testNotifFromSettings(): void {
|
||||
fireTestNotification();
|
||||
}
|
||||
|
||||
// ─── Activity Log Settings ──────────────────────────────────
|
||||
|
||||
interface ActivityLogSettingsResponse {
|
||||
enabled: boolean;
|
||||
max_days: number;
|
||||
max_entries: number;
|
||||
}
|
||||
|
||||
/** Fetch and populate the Activity Log settings panel. */
|
||||
export async function loadActivityLogSettings(): Promise<void> {
|
||||
try {
|
||||
const res = await fetchWithAuth('/activity-log/settings');
|
||||
if (!res || !res.ok) return;
|
||||
const data: ActivityLogSettingsResponse = await res.json();
|
||||
|
||||
const enabledCb = document.getElementById('al-settings-enabled') as HTMLInputElement | null;
|
||||
const maxDaysIn = document.getElementById('al-settings-max-days') as HTMLInputElement | null;
|
||||
const maxEntriesIn = document.getElementById('al-settings-max-entries') as HTMLInputElement | null;
|
||||
|
||||
if (enabledCb) enabledCb.checked = data.enabled;
|
||||
if (maxDaysIn) maxDaysIn.value = String(data.max_days);
|
||||
if (maxEntriesIn) maxEntriesIn.value = String(data.max_entries);
|
||||
} catch (err) {
|
||||
// Non-fatal — panel just shows defaults
|
||||
console.error('Failed to load activity log settings:', err);
|
||||
}
|
||||
}
|
||||
|
||||
/** Validate and save the Activity Log retention settings. */
|
||||
export async function saveActivityLogSettings(): Promise<void> {
|
||||
const enabledCb = document.getElementById('al-settings-enabled') as HTMLInputElement | null;
|
||||
const maxDaysIn = document.getElementById('al-settings-max-days') as HTMLInputElement | null;
|
||||
const maxEntriesIn = document.getElementById('al-settings-max-entries') as HTMLInputElement | null;
|
||||
|
||||
const enabled = enabledCb ? enabledCb.checked : true;
|
||||
const maxDays = parseInt(maxDaysIn?.value ?? '365', 10);
|
||||
const maxEntries = parseInt(maxEntriesIn?.value ?? '100000', 10);
|
||||
|
||||
// Client-side validation matching server bounds
|
||||
if (isNaN(maxDays) || maxDays < 0 || maxDays > 3650) {
|
||||
showToast(t('settings.activity_log.error.max_days_range'), 'error');
|
||||
return;
|
||||
}
|
||||
if (isNaN(maxEntries) || maxEntries < 0 || maxEntries > 10_000_000) {
|
||||
showToast(t('settings.activity_log.error.max_entries_range'), 'error');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const res = await fetchWithAuth('/activity-log/settings', {
|
||||
method: 'PUT',
|
||||
body: JSON.stringify({ enabled, max_days: maxDays, max_entries: maxEntries }),
|
||||
});
|
||||
if (!res || !res.ok) {
|
||||
const err = await res?.json().catch(() => ({}));
|
||||
throw new Error((err as any).detail || `HTTP ${res?.status}`);
|
||||
}
|
||||
showToast(t('settings.activity_log.saved'), 'success');
|
||||
} catch (err: any) {
|
||||
if (err?.isAuth) return;
|
||||
showToast(t('settings.activity_log.save_error') + ': ' + (err?.message || ''), 'error');
|
||||
}
|
||||
}
|
||||
|
||||
/** Export the full activity log as CSV or JSON via authed blob download. */
|
||||
export async function activityLogSettingsExport(format: 'csv' | 'json'): Promise<void> {
|
||||
try {
|
||||
showToast(t('activity_log.export.downloading'), 'info');
|
||||
const res = await fetchWithAuth(`/activity-log/export?format=${format}`);
|
||||
if (!res || !res.ok) throw new Error(`HTTP ${res?.status}`);
|
||||
const blob = await res.blob();
|
||||
const blobUrl = URL.createObjectURL(blob);
|
||||
const now = new Date().toISOString().slice(0, 19).replace(/:/g, '-');
|
||||
const filename = `ledgrab-activity-${now}.${format}`;
|
||||
const a = document.createElement('a');
|
||||
a.href = blobUrl;
|
||||
a.download = filename;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
setTimeout(() => URL.revokeObjectURL(blobUrl), 10000);
|
||||
} catch (err: any) {
|
||||
if (err?.isAuth) return;
|
||||
showToast(t('activity_log.export.error'), 'error');
|
||||
}
|
||||
}
|
||||
|
||||
/** Confirm and clear all activity log entries. */
|
||||
export async function clearActivityLog(): Promise<void> {
|
||||
const ok = await showConfirm(t('settings.activity_log.clear.confirm'));
|
||||
if (!ok) return;
|
||||
try {
|
||||
const res = await fetchWithAuth('/activity-log', { method: 'DELETE', handle401: false });
|
||||
if (!res || !res.ok) {
|
||||
if (res?.status === 401) {
|
||||
showToast(t('settings.activity_log.clear.auth_required'), 'error');
|
||||
return;
|
||||
}
|
||||
throw new Error(`HTTP ${res?.status}`);
|
||||
}
|
||||
showToast(t('settings.activity_log.clear.success'), 'success');
|
||||
} catch (err: any) {
|
||||
if (err?.isAuth) return;
|
||||
showToast(t('settings.activity_log.clear.error') + ': ' + (err?.message || ''), 'error');
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -66,6 +66,7 @@ const gettingStartedSteps: TutorialStep[] = [
|
||||
{ selector: '#tab-btn-streams', textKey: 'tour.sources', position: 'bottom' },
|
||||
{ selector: '#tab-btn-integrations', textKey: 'tour.integrations', position: 'bottom' },
|
||||
{ selector: '#tab-btn-graph', textKey: 'tour.graph', position: 'bottom' },
|
||||
{ selector: '#tab-btn-activity_log', textKey: 'tour.activity_log', position: 'bottom' },
|
||||
{ selector: 'a.header-link[href="/docs"]', textKey: 'tour.api', position: 'bottom' },
|
||||
{ selector: '[onclick*="openCommandPalette"]', textKey: 'tour.search', position: 'bottom' },
|
||||
{ selector: '[onclick*="toggleTheme"]', textKey: 'tour.theme', position: 'bottom' },
|
||||
|
||||
+20
@@ -457,6 +457,26 @@ startTargetOverlay: (...args: any[]) => any;
|
||||
applyBgEffect: (id: string) => void;
|
||||
renderAppearanceTab: () => void;
|
||||
|
||||
// ─── Activity Log ───
|
||||
loadActivityLog: () => Promise<void>;
|
||||
activityLogToggleDetail: (entryId: string) => void;
|
||||
loadActivityLogSettings: () => Promise<void>;
|
||||
saveActivityLogSettings: () => Promise<void>;
|
||||
activityLogSettingsExport: (format: 'csv' | 'json') => Promise<void>;
|
||||
clearActivityLog: () => Promise<void>;
|
||||
activityLogToggleCat: (cat: string) => void;
|
||||
activityLogToggleSev: (sev: string) => void;
|
||||
activityLogOnSearch: (val: string) => void;
|
||||
activityLogOnActor: (val: string) => void;
|
||||
activityLogOnEntityType: (val: string) => void;
|
||||
activityLogOnSince: (val: string) => void;
|
||||
activityLogOnUntil: (val: string) => void;
|
||||
activityLogClearFilters: () => void;
|
||||
activityLogPreset: (key: string) => void;
|
||||
activityLogLoadMore: () => void;
|
||||
activityLogExport: (format: 'csv' | 'json') => Promise<void>;
|
||||
activityLogNavigateToEntity: (entityType: string, entityId: string) => void;
|
||||
|
||||
// ─── Overlay spinner internals ───
|
||||
overlaySpinnerTimer: ReturnType<typeof setInterval> | null;
|
||||
_overlayEscHandler: ((e: KeyboardEvent) => void) | null;
|
||||
|
||||
+3406
-3260
File diff suppressed because it is too large
Load Diff
+3012
-2866
File diff suppressed because it is too large
Load Diff
+3006
-2860
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,171 @@
|
||||
"""Activity / audit log data model.
|
||||
|
||||
Defines the ``ActivityLogEntry`` dataclass together with its category and
|
||||
severity enumerations and the ``ActivityLogFilters`` query object. Row-level
|
||||
codec (``to_row`` / ``from_row``) converts between the dataclass and a flat
|
||||
SQLite column dict.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from typing import Sequence
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Enumerations (string constants, not enum.Enum, to stay consistent with the
|
||||
# rest of the codebase which uses plain string literals)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ActivityCategory:
|
||||
"""Valid ``category`` values for an ``ActivityLogEntry``."""
|
||||
|
||||
AUTH = "auth"
|
||||
DEVICE = "device"
|
||||
ENTITY = "entity"
|
||||
CAPTURE = "capture"
|
||||
SYSTEM = "system"
|
||||
|
||||
ALL: tuple[str, ...] = (AUTH, DEVICE, ENTITY, CAPTURE, SYSTEM)
|
||||
|
||||
|
||||
class ActivitySeverity:
|
||||
"""Valid ``severity`` values for an ``ActivityLogEntry``."""
|
||||
|
||||
INFO = "info"
|
||||
WARNING = "warning"
|
||||
ERROR = "error"
|
||||
|
||||
ALL: tuple[str, ...] = (INFO, WARNING, ERROR)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Entry dataclass
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class ActivityLogEntry:
|
||||
"""One immutable audit record.
|
||||
|
||||
Fields
|
||||
------
|
||||
id Stable application-assigned identifier, e.g. ``al_<uuid8>``.
|
||||
ts UTC timestamp of the event (server-assigned at record time).
|
||||
category Broad bucket — one of :class:`ActivityCategory`.
|
||||
action Verb-object label, e.g. ``"entity.created"`` or ``"auth.rejected"``.
|
||||
severity One of :class:`ActivitySeverity`.
|
||||
actor Who triggered the action, e.g. an API-key label or ``"system"``.
|
||||
entity_type Optional: kind of entity involved, e.g. ``"output_target"``.
|
||||
entity_id Optional: stable entity identifier.
|
||||
entity_name Optional: human-readable entity name at the time of the event.
|
||||
message Human-readable description suitable for display.
|
||||
metadata Small structured context (device address, error code, …).
|
||||
Must be JSON-serialisable; defaults to empty dict.
|
||||
"""
|
||||
|
||||
id: str
|
||||
ts: datetime
|
||||
category: str
|
||||
action: str
|
||||
severity: str
|
||||
actor: str
|
||||
message: str
|
||||
entity_type: str | None = None
|
||||
entity_id: str | None = None
|
||||
entity_name: str | None = None
|
||||
metadata: dict = field(default_factory=dict)
|
||||
|
||||
# -- Row codec -----------------------------------------------------------
|
||||
|
||||
def to_row(self) -> dict:
|
||||
"""Return a flat dict suitable for a parameterised SQLite INSERT.
|
||||
|
||||
``ts`` is stored as an ISO-8601 string (UTC). ``metadata`` is stored
|
||||
as a JSON string. ``seq`` is DB-assigned and is NOT included.
|
||||
"""
|
||||
return {
|
||||
"id": self.id,
|
||||
"ts": self.ts.isoformat(),
|
||||
"category": self.category,
|
||||
"action": self.action,
|
||||
"severity": self.severity,
|
||||
"actor": self.actor,
|
||||
"entity_type": self.entity_type,
|
||||
"entity_id": self.entity_id,
|
||||
"entity_name": self.entity_name,
|
||||
"message": self.message,
|
||||
"metadata": json.dumps(self.metadata, ensure_ascii=False),
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def from_row(row: dict) -> "ActivityLogEntry":
|
||||
"""Reconstruct an ``ActivityLogEntry`` from a SQLite row dict.
|
||||
|
||||
``row`` may include the ``seq`` column — it is ignored here (callers
|
||||
that need ``seq`` for keyset pagination access it directly from the
|
||||
row before calling this method).
|
||||
"""
|
||||
raw_meta = row.get("metadata") or "{}"
|
||||
try:
|
||||
metadata = json.loads(raw_meta)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
metadata = {}
|
||||
|
||||
raw_ts = row["ts"]
|
||||
ts = datetime.fromisoformat(raw_ts)
|
||||
# Ensure tz-aware UTC even if stored without offset (legacy rows)
|
||||
if ts.tzinfo is None:
|
||||
ts = ts.replace(tzinfo=timezone.utc)
|
||||
|
||||
return ActivityLogEntry(
|
||||
id=row["id"],
|
||||
ts=ts,
|
||||
category=row["category"],
|
||||
action=row["action"],
|
||||
severity=row["severity"],
|
||||
actor=row["actor"],
|
||||
entity_type=row.get("entity_type"),
|
||||
entity_id=row.get("entity_id"),
|
||||
entity_name=row.get("entity_name"),
|
||||
message=row["message"],
|
||||
metadata=metadata,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Filters dataclass
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class ActivityLogFilters:
|
||||
"""Optional query-time filters for :class:`ActivityLogRepository`.
|
||||
|
||||
All fields are optional. ``None`` / empty sequence means "no restriction
|
||||
on this dimension".
|
||||
|
||||
Fields
|
||||
------
|
||||
categories Restrict to entries whose ``category`` is in this set.
|
||||
severities Restrict to entries whose ``severity`` is in this set.
|
||||
actor Exact match on the ``actor`` field.
|
||||
entity_type Exact match on the ``entity_type`` field.
|
||||
entity_id Exact match on the ``entity_id`` field.
|
||||
since Inclusive lower bound on ``ts`` (``ts >= since``).
|
||||
until Inclusive upper bound on ``ts`` (``ts <= until``).
|
||||
message_like Free-text substring match on ``message`` (LIKE ``%value%``).
|
||||
The value is escaped — no SQL injection risk.
|
||||
"""
|
||||
|
||||
categories: Sequence[str] | None = None
|
||||
severities: Sequence[str] | None = None
|
||||
actor: str | None = None
|
||||
entity_type: str | None = None
|
||||
entity_id: str | None = None
|
||||
since: datetime | None = None
|
||||
until: datetime | None = None
|
||||
message_like: str | None = None
|
||||
@@ -0,0 +1,338 @@
|
||||
"""Append-only repository for the persistent activity / audit log.
|
||||
|
||||
Design notes
|
||||
------------
|
||||
* Does NOT subclass ``BaseSqliteStore`` — that base loads every row into an
|
||||
in-memory cache at construction time, which is wrong for an unbounded,
|
||||
append-heavy log.
|
||||
* All SQL parameters are passed as positional ``?`` placeholders — no user
|
||||
input is interpolated into the query string.
|
||||
* Keyset pagination is implemented via the ``seq`` INTEGER PRIMARY KEY
|
||||
AUTOINCREMENT column, which is strictly monotonic and survives rows with
|
||||
identical ``ts`` values.
|
||||
* The repository takes a ``Database`` instance and calls ``db.execute`` /
|
||||
``db.transaction`` directly.
|
||||
* Thread safety: ``Database.execute`` holds the ``RLock`` for the duration of
|
||||
each call. The repository itself adds no extra locking. Callers that
|
||||
originate from non-event-loop threads MUST marshal via
|
||||
``loop.call_soon_threadsafe`` (that is Phase 2's responsibility).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Iterator
|
||||
|
||||
from ledgrab.storage.activity_log import ActivityLogEntry, ActivityLogFilters
|
||||
from ledgrab.storage.database import Database
|
||||
from ledgrab.utils import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
_TABLE = "activity_log"
|
||||
|
||||
|
||||
def _build_filter_clause(
|
||||
filters: ActivityLogFilters,
|
||||
params: list,
|
||||
*,
|
||||
extra_where: str | None = None,
|
||||
) -> str:
|
||||
"""Return a WHERE clause string and append bound parameters to *params*.
|
||||
|
||||
The caller is responsible for providing the opening ``WHERE`` keyword when
|
||||
the returned string is non-empty, or ``AND`` when appending to an existing
|
||||
predicate. This function returns only the condition fragment(s) joined by
|
||||
``AND``, or an empty string when *filters* has no restrictions.
|
||||
|
||||
``extra_where`` is prepended (with AND) before the filter conditions if
|
||||
provided (used for the ``seq < ?`` keyset predicate).
|
||||
"""
|
||||
conditions: list[str] = []
|
||||
|
||||
if extra_where:
|
||||
conditions.append(extra_where)
|
||||
|
||||
if filters.categories:
|
||||
placeholders = ",".join("?" * len(filters.categories))
|
||||
conditions.append(f"category IN ({placeholders})")
|
||||
params.extend(filters.categories)
|
||||
|
||||
if filters.severities:
|
||||
placeholders = ",".join("?" * len(filters.severities))
|
||||
conditions.append(f"severity IN ({placeholders})")
|
||||
params.extend(filters.severities)
|
||||
|
||||
if filters.actor is not None:
|
||||
conditions.append("actor = ?")
|
||||
params.append(filters.actor)
|
||||
|
||||
if filters.entity_type is not None:
|
||||
conditions.append("entity_type = ?")
|
||||
params.append(filters.entity_type)
|
||||
|
||||
if filters.entity_id is not None:
|
||||
conditions.append("entity_id = ?")
|
||||
params.append(filters.entity_id)
|
||||
|
||||
if filters.since is not None:
|
||||
conditions.append("ts >= ?")
|
||||
params.append(filters.since.isoformat())
|
||||
|
||||
if filters.until is not None:
|
||||
conditions.append("ts <= ?")
|
||||
params.append(filters.until.isoformat())
|
||||
|
||||
if filters.message_like is not None:
|
||||
# Escape LIKE special characters in the user-supplied substring so that
|
||||
# a literal '%' or '_' in the message does not act as a wildcard.
|
||||
escaped = filters.message_like.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
||||
conditions.append("message LIKE ? ESCAPE '\\'")
|
||||
params.append(f"%{escaped}%")
|
||||
|
||||
return " AND ".join(conditions)
|
||||
|
||||
|
||||
class ActivityLogRepository:
|
||||
"""Purpose-built repository for the ``activity_log`` table.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
db:
|
||||
The shared ``Database`` singleton. The migration that creates the
|
||||
``activity_log`` table must have been applied before the first call to
|
||||
:meth:`record`. Construction triggers migration via
|
||||
``MigrationRunner`` so users can create the repository directly
|
||||
without a separate startup step.
|
||||
"""
|
||||
|
||||
def __init__(self, db: Database) -> None:
|
||||
self._db = db
|
||||
# Apply pending migrations (idempotent; most runs are no-ops)
|
||||
from ledgrab.storage.data_migrations import ALL_MIGRATIONS, MigrationRunner
|
||||
|
||||
MigrationRunner(db).run(ALL_MIGRATIONS)
|
||||
|
||||
# -- Write ---------------------------------------------------------------
|
||||
|
||||
def record(self, entry: ActivityLogEntry) -> None:
|
||||
"""Append *entry* to the log.
|
||||
|
||||
The ``seq`` column is assigned by SQLite (AUTOINCREMENT) — ``entry``
|
||||
does not carry a ``seq`` field.
|
||||
|
||||
Caller contract: this must be called from the event-loop thread (or
|
||||
from a context already serialised through the ``Database`` RLock).
|
||||
Cross-thread callers must marshal via ``loop.call_soon_threadsafe``;
|
||||
that is Phase 2's responsibility.
|
||||
"""
|
||||
row = entry.to_row()
|
||||
self._db.execute(
|
||||
f"INSERT INTO {_TABLE} "
|
||||
f"(id, ts, category, action, severity, actor, "
|
||||
f" entity_type, entity_id, entity_name, message, metadata) "
|
||||
f"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
|
||||
(
|
||||
row["id"],
|
||||
row["ts"],
|
||||
row["category"],
|
||||
row["action"],
|
||||
row["severity"],
|
||||
row["actor"],
|
||||
row["entity_type"],
|
||||
row["entity_id"],
|
||||
row["entity_name"],
|
||||
row["message"],
|
||||
row["metadata"],
|
||||
),
|
||||
)
|
||||
|
||||
# -- Read ----------------------------------------------------------------
|
||||
|
||||
def query(
|
||||
self,
|
||||
filters: ActivityLogFilters,
|
||||
*,
|
||||
before_seq: int | None = None,
|
||||
limit: int = 50,
|
||||
) -> list[ActivityLogEntry]:
|
||||
"""Return the newest matching entries, oldest-first within the page.
|
||||
|
||||
Keyset pagination: pass the smallest ``seq`` seen on the previous page
|
||||
as *before_seq* to get the next page. Entries are fetched in
|
||||
``seq DESC`` order from SQLite and then reversed before returning so
|
||||
that the caller receives them in chronological order within the page.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
filters:
|
||||
Optional filter criteria.
|
||||
before_seq:
|
||||
Exclusive upper bound on ``seq``. ``None`` returns the first
|
||||
(newest) page.
|
||||
limit:
|
||||
Maximum number of entries to return.
|
||||
"""
|
||||
params: list = []
|
||||
keyset = "seq < ?" if before_seq is not None else None
|
||||
if before_seq is not None:
|
||||
params.append(before_seq)
|
||||
|
||||
where_fragment = _build_filter_clause(filters, params, extra_where=keyset)
|
||||
where_clause = f"WHERE {where_fragment}" if where_fragment else ""
|
||||
params.append(limit)
|
||||
|
||||
sql = (
|
||||
f"SELECT seq, id, ts, category, action, severity, actor, "
|
||||
f"entity_type, entity_id, entity_name, message, metadata "
|
||||
f"FROM {_TABLE} "
|
||||
f"{where_clause} "
|
||||
f"ORDER BY seq DESC "
|
||||
f"LIMIT ?"
|
||||
)
|
||||
|
||||
cursor = self._db.execute(sql, tuple(params))
|
||||
rows = cursor.fetchall()
|
||||
# Reverse to return chronological order within the page
|
||||
return [ActivityLogEntry.from_row(dict(row)) for row in reversed(rows)]
|
||||
|
||||
def count(self, filters: ActivityLogFilters | None = None) -> int:
|
||||
"""Return the number of entries matching *filters* (or all entries)."""
|
||||
if filters is None:
|
||||
filters = ActivityLogFilters()
|
||||
params: list = []
|
||||
where_fragment = _build_filter_clause(filters, params)
|
||||
where_clause = f"WHERE {where_fragment}" if where_fragment else ""
|
||||
|
||||
sql = f"SELECT COUNT(*) AS cnt FROM {_TABLE} {where_clause}"
|
||||
cursor = self._db.execute(sql, tuple(params))
|
||||
row = cursor.fetchone()
|
||||
return int(row["cnt"])
|
||||
|
||||
# -- Maintenance ---------------------------------------------------------
|
||||
|
||||
def prune(
|
||||
self,
|
||||
*,
|
||||
before_ts: datetime | None = None,
|
||||
max_entries: int | None = None,
|
||||
) -> int:
|
||||
"""Delete old / excess entries; return the total rows deleted.
|
||||
|
||||
Both predicates are applied independently:
|
||||
|
||||
1. Delete all rows where ``ts < before_ts`` (age-based pruning).
|
||||
2. Keep only the newest ``max_entries`` rows, deleting the rest
|
||||
(count-based pruning).
|
||||
|
||||
The two operations run in separate statements so that each can be
|
||||
independently enabled or disabled.
|
||||
"""
|
||||
deleted = 0
|
||||
|
||||
if before_ts is not None:
|
||||
cursor = self._db.execute(
|
||||
f"DELETE FROM {_TABLE} WHERE ts < ?",
|
||||
(before_ts.isoformat(),),
|
||||
)
|
||||
deleted += cursor.rowcount
|
||||
|
||||
if max_entries is not None and max_entries >= 0:
|
||||
# Find the seq of the Nth newest row; delete everything older than it.
|
||||
cursor = self._db.execute(
|
||||
f"SELECT seq FROM {_TABLE} ORDER BY seq DESC LIMIT 1 OFFSET ?",
|
||||
(max_entries,),
|
||||
)
|
||||
cutoff_row = cursor.fetchone()
|
||||
if cutoff_row is not None:
|
||||
cutoff_seq = cutoff_row["seq"]
|
||||
cursor = self._db.execute(
|
||||
f"DELETE FROM {_TABLE} WHERE seq <= ?",
|
||||
(cutoff_seq,),
|
||||
)
|
||||
deleted += cursor.rowcount
|
||||
|
||||
return deleted
|
||||
|
||||
def clear(self) -> int:
|
||||
"""Delete all entries; return the number of rows deleted."""
|
||||
cursor = self._db.execute(f"DELETE FROM {_TABLE}")
|
||||
return cursor.rowcount
|
||||
|
||||
def get_seq_for_id(self, entry_id: str) -> int | None:
|
||||
"""Return the ``seq`` value for the entry with *entry_id*, or ``None``.
|
||||
|
||||
Used by the API list endpoint to compute the keyset cursor
|
||||
(``next_before_seq``) from the oldest entry on the current page.
|
||||
"""
|
||||
cursor = self._db.execute(
|
||||
f"SELECT seq FROM {_TABLE} WHERE id = ?",
|
||||
(entry_id,),
|
||||
)
|
||||
row = cursor.fetchone()
|
||||
return int(row["seq"]) if row is not None else None
|
||||
|
||||
# -- Export --------------------------------------------------------------
|
||||
|
||||
def iter_export(
|
||||
self,
|
||||
filters: ActivityLogFilters | None = None,
|
||||
*,
|
||||
batch_size: int = 1000,
|
||||
) -> Iterator[ActivityLogEntry]:
|
||||
"""Yield all matching entries in ascending ``seq`` order.
|
||||
|
||||
Fetches rows in bounded batches (keyset-paginated by ``seq``), holding
|
||||
the DB lock only for the duration of each ``fetchall()`` and releasing
|
||||
it before yielding. This prevents a slow/stalled export client from
|
||||
blocking all other DB operations (record, config writes, etc.) for the
|
||||
full duration of the stream.
|
||||
|
||||
Memory usage is bounded to ``batch_size`` rows at a time.
|
||||
"""
|
||||
if filters is None:
|
||||
filters = ActivityLogFilters()
|
||||
|
||||
# Keyset cursor: largest seq yielded so far; None means "start from the
|
||||
# very beginning". We iterate ascending (seq ASC), so each batch uses
|
||||
# "seq > ?" to advance past the already-yielded rows.
|
||||
cursor_seq: int | None = None
|
||||
|
||||
while True:
|
||||
# Build params list: cursor_seq placeholder must come first because
|
||||
# _build_filter_clause prepends extra_where as the first condition.
|
||||
params: list = []
|
||||
if cursor_seq is not None:
|
||||
params.append(cursor_seq)
|
||||
keyset: str | None = "seq > ?"
|
||||
else:
|
||||
keyset = None
|
||||
where_fragment = _build_filter_clause(filters, params, extra_where=keyset)
|
||||
where_clause = f"WHERE {where_fragment}" if where_fragment else ""
|
||||
params.append(batch_size)
|
||||
|
||||
sql = (
|
||||
f"SELECT seq, id, ts, category, action, severity, actor, "
|
||||
f"entity_type, entity_id, entity_name, message, metadata "
|
||||
f"FROM {_TABLE} "
|
||||
f"{where_clause} "
|
||||
f"ORDER BY seq ASC "
|
||||
f"LIMIT ?"
|
||||
)
|
||||
|
||||
# Hold the lock only for the bounded fetchall; release before yielding.
|
||||
with self._db._lock: # noqa: SLF001 — internal access; no public cursor API
|
||||
rows = self._db._conn.execute(sql, tuple(params)).fetchall() # noqa: SLF001
|
||||
|
||||
if not rows:
|
||||
break
|
||||
|
||||
for row in rows:
|
||||
yield ActivityLogEntry.from_row(dict(row))
|
||||
|
||||
# The last row has the largest seq in this batch (ORDER BY seq ASC).
|
||||
cursor_seq = rows[-1]["seq"]
|
||||
|
||||
if len(rows) < batch_size:
|
||||
# Fewer rows than requested → this was the final batch.
|
||||
break
|
||||
@@ -213,7 +213,57 @@ class StaticToSingleColorMigration(DataMigration):
|
||||
return rows_changed
|
||||
|
||||
|
||||
class AddActivityLogTableMigration(DataMigration):
|
||||
"""Create the ``activity_log`` table and its indexes.
|
||||
|
||||
This is a purely additive migration — it does not touch any existing table.
|
||||
``CREATE TABLE / INDEX IF NOT EXISTS`` ensures idempotency if the migration
|
||||
somehow runs twice (e.g. after a partial restore).
|
||||
"""
|
||||
|
||||
name = "002_add_activity_log"
|
||||
|
||||
def apply(self, conn: sqlite3.Connection) -> int:
|
||||
conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS activity_log (
|
||||
seq INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
id TEXT UNIQUE NOT NULL,
|
||||
ts TEXT NOT NULL,
|
||||
category TEXT NOT NULL,
|
||||
action TEXT NOT NULL,
|
||||
severity TEXT NOT NULL,
|
||||
actor TEXT NOT NULL,
|
||||
entity_type TEXT,
|
||||
entity_id TEXT,
|
||||
entity_name TEXT,
|
||||
message TEXT NOT NULL,
|
||||
metadata TEXT NOT NULL DEFAULT '{}'
|
||||
)
|
||||
"""
|
||||
)
|
||||
# Primary read path: newest-first keyset pagination
|
||||
conn.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_activity_log_ts_seq "
|
||||
"ON activity_log (ts DESC, seq DESC)"
|
||||
)
|
||||
# Filter indexes
|
||||
conn.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_activity_log_category " "ON activity_log (category)"
|
||||
)
|
||||
conn.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_activity_log_severity " "ON activity_log (severity)"
|
||||
)
|
||||
conn.execute("CREATE INDEX IF NOT EXISTS idx_activity_log_actor " "ON activity_log (actor)")
|
||||
conn.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_activity_log_entity "
|
||||
"ON activity_log (entity_type, entity_id)"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
# Master list — ORDER MATTERS. Append new migrations; never reorder.
|
||||
ALL_MIGRATIONS: list[DataMigration] = [
|
||||
StaticToSingleColorMigration(),
|
||||
AddActivityLogTableMigration(),
|
||||
]
|
||||
|
||||
@@ -143,6 +143,7 @@
|
||||
<button class="tab-btn" data-tab="streams" onclick="switchTab('streams')" role="tab" aria-selected="false" aria-controls="tab-streams" id="tab-btn-streams" title="Ctrl+4"><svg class="icon" viewBox="0 0 24 24"><path d="m17 2-5 5-5-5"/><rect width="20" height="15" x="2" y="7" rx="2"/></svg> <span data-i18n="streams.title">Sources</span></button>
|
||||
<button class="tab-btn" data-tab="integrations" onclick="switchTab('integrations')" role="tab" aria-selected="false" aria-controls="tab-integrations" id="tab-btn-integrations" title="Ctrl+5"><svg class="icon" viewBox="0 0 24 24"><path d="M12 22v-5"/><path d="M15 8V2"/><path d="M17 8a1 1 0 0 1 1 1v4a4 4 0 0 1-4 4h-4a4 4 0 0 1-4-4V9a1 1 0 0 1 1-1z"/><path d="M9 8V2"/></svg> <span data-i18n="integrations.title">Integrations</span></button>
|
||||
<button class="tab-btn" data-tab="graph" onclick="switchTab('graph')" role="tab" aria-selected="false" aria-controls="tab-graph" id="tab-btn-graph" title="Ctrl+6"><svg class="icon" viewBox="0 0 24 24"><circle cx="5" cy="6" r="3"/><circle cx="19" cy="6" r="3"/><circle cx="12" cy="18" r="3"/><path d="M7.5 7.5 10.5 16"/><path d="M16.5 7.5 13.5 16"/></svg> <span data-i18n="graph.title">Graph</span></button>
|
||||
<button class="tab-btn" data-tab="activity_log" onclick="switchTab('activity_log')" role="tab" aria-selected="false" aria-controls="tab-activity_log" id="tab-btn-activity_log" title="Ctrl+7"><svg class="icon" viewBox="0 0 24 24"><path d="M15 12h-5"/><path d="M15 8h-5"/><path d="M19 17V5a2 2 0 0 0-2-2H4"/><path d="M8 21h12a2 2 0 0 0 2-2v-1a1 1 0 0 0-1-1H11a1 1 0 0 0-1 1v1a2 2 0 1 1-4 0V5a2 2 0 1 0-4 0v2a1 1 0 0 0 1 1h3"/></svg> <span data-i18n="activity_log.title">Activity</span></button>
|
||||
</div>
|
||||
</div>
|
||||
</aside>
|
||||
@@ -201,6 +202,9 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="tab-panel" id="tab-activity_log" role="tabpanel" aria-labelledby="tab-btn-activity_log">
|
||||
<div class="loading-spinner"></div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Apply saved tab immediately during parse to prevent visible jump
|
||||
|
||||
@@ -49,6 +49,12 @@
|
||||
|
||||
<div class="settings-rail-group" data-i18n="settings.rail.group.system">System</div>
|
||||
|
||||
<button class="settings-rail-btn" data-settings-tab="activity_log" data-rail-ch="cyan" onclick="switchSettingsTab('activity_log')" role="tab" aria-label="Activity Log" data-i18n-aria-label="settings.tab.activity_log">
|
||||
<svg class="icon" viewBox="0 0 24 24"><path d="M14 3H6a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8Z"/><path d="M14 3v5h5"/><path d="M16 13H8"/><path d="M16 17H8"/><path d="M10 9H8"/></svg>
|
||||
<span class="settings-rail-label" data-i18n="settings.tab.activity_log">Activity Log</span>
|
||||
<span class="settings-rail-dot" aria-hidden="true"></span>
|
||||
</button>
|
||||
|
||||
<button class="settings-rail-btn" data-settings-tab="updates" data-rail-ch="signal" onclick="switchSettingsTab('updates')" role="tab" aria-label="Updates" data-i18n-aria-label="settings.tab.updates">
|
||||
<svg class="icon" viewBox="0 0 24 24"><path d="M3 12a9 9 0 0 1 9-9 9.75 9.75 0 0 1 6.74 2.74L21 8"/><path d="M21 3v5h-5"/><path d="M21 12a9 9 0 0 1-9 9 9.75 9.75 0 0 1-6.74-2.74L3 16"/><path d="M8 16H3v5"/></svg>
|
||||
<span class="settings-rail-label" data-i18n="settings.tab.updates">Updates</span>
|
||||
@@ -529,6 +535,129 @@
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<!-- ═══ Activity Log tab ═══ -->
|
||||
<div id="settings-panel-activity_log" class="settings-panel">
|
||||
|
||||
<!-- Note distinguishing this persistent audit log from the ephemeral debug Log Viewer -->
|
||||
<div class="ds-info-note" role="note">
|
||||
<svg class="icon" viewBox="0 0 24 24" aria-hidden="true"><circle cx="12" cy="12" r="10"/><path d="M12 16v-4"/><path d="M12 8h.01"/></svg>
|
||||
<span data-i18n="settings.activity_log.distinction_note">This is the <strong>persistent audit log</strong> — structured records of every entity change, auth event, and system action. It is separate from the ephemeral <button class="ds-inline-link" onclick="closeSettingsModal(); openLogOverlay()" data-i18n="settings.activity_log.open_log_viewer">debug Log Viewer</button> (live server log tail, resets on disconnect).</span>
|
||||
</div>
|
||||
|
||||
<!-- Retention settings -->
|
||||
<section class="ds-section" data-ch="cyan">
|
||||
<div class="ds-section-header">
|
||||
<span class="ds-section-dot" aria-hidden="true"></span>
|
||||
<span class="ds-section-title" data-i18n="settings.activity_log.section.retention">Retention</span>
|
||||
<span class="ds-section-index" aria-hidden="true">01</span>
|
||||
</div>
|
||||
<div class="ds-section-body">
|
||||
|
||||
<div class="ds-toggle-row">
|
||||
<div class="ds-toggle-text">
|
||||
<div class="ds-toggle-title" data-i18n="settings.activity_log.enabled.label">Enable activity logging</div>
|
||||
<div class="ds-toggle-sub" data-i18n="settings.activity_log.enabled.hint">When disabled, no new audit entries are recorded. Existing entries are preserved.</div>
|
||||
</div>
|
||||
<label class="settings-switch">
|
||||
<input type="checkbox" id="al-settings-enabled" checked>
|
||||
<span class="settings-switch-track" aria-hidden="true"></span>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div class="ds-pair-row">
|
||||
<div class="form-group">
|
||||
<div class="label-row">
|
||||
<label for="al-settings-max-days" data-i18n="settings.activity_log.max_days.label">Max age (days)</label>
|
||||
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?">?</button>
|
||||
</div>
|
||||
<small class="input-hint" style="display:none" data-i18n="settings.activity_log.max_days.hint">Entries older than this many days are pruned automatically. Set to 0 to disable age-based pruning (keep forever, up to the entry limit).</small>
|
||||
<input type="number" id="al-settings-max-days" min="0" max="3650" value="365" aria-describedby="al-settings-max-days-hint">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<div class="label-row">
|
||||
<label for="al-settings-max-entries" data-i18n="settings.activity_log.max_entries.label">Max entries</label>
|
||||
<button type="button" class="hint-toggle" onclick="toggleHint(this)" title="?">?</button>
|
||||
</div>
|
||||
<small class="input-hint" style="display:none" data-i18n="settings.activity_log.max_entries.hint">Maximum number of entries to keep. When this limit is reached, the oldest entries are pruned. Set to 0 for no limit.</small>
|
||||
<input type="number" id="al-settings-max-entries" min="0" max="10000000" value="100000" aria-describedby="al-settings-max-entries-hint">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="inline-row inline-row--actions">
|
||||
<button class="btn btn-primary" onclick="saveActivityLogSettings()" style="flex:1">
|
||||
<svg class="icon" viewBox="0 0 24 24"><path d="M20 6 9 17l-5-5"/></svg>
|
||||
<span data-i18n="settings.activity_log.save">Save Settings</span>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Export section -->
|
||||
<section class="ds-section" data-ch="amber">
|
||||
<div class="ds-section-header">
|
||||
<span class="ds-section-dot" aria-hidden="true"></span>
|
||||
<span class="ds-section-title" data-i18n="settings.activity_log.section.export">Export</span>
|
||||
<span class="ds-section-index" aria-hidden="true">02</span>
|
||||
</div>
|
||||
<div class="ds-section-body">
|
||||
|
||||
<div class="ds-toggle-row">
|
||||
<div class="ds-toggle-text">
|
||||
<div class="ds-toggle-title" data-i18n="settings.activity_log.export_csv.label">Export as CSV</div>
|
||||
<div class="ds-toggle-sub" data-i18n="settings.activity_log.export.hint">Download the full audit log as a file. Large logs may take a moment to prepare.</div>
|
||||
</div>
|
||||
<button class="btn btn-secondary" onclick="activityLogSettingsExport('csv')">
|
||||
<svg class="icon" viewBox="0 0 24 24"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" x2="12" y1="15" y2="3"/></svg>
|
||||
<span data-i18n="settings.activity_log.export_csv.button">CSV</span>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div class="ds-toggle-row">
|
||||
<div class="ds-toggle-text">
|
||||
<div class="ds-toggle-title" data-i18n="settings.activity_log.export_json.label">Export as JSON</div>
|
||||
<div class="ds-toggle-sub" data-i18n="settings.activity_log.export.hint">Download the full audit log as a file. Large logs may take a moment to prepare.</div>
|
||||
</div>
|
||||
<button class="btn btn-secondary" onclick="activityLogSettingsExport('json')">
|
||||
<svg class="icon" viewBox="0 0 24 24"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" x2="12" y1="15" y2="3"/></svg>
|
||||
<span data-i18n="settings.activity_log.export_json.button">JSON</span>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<small class="input-hint" style="display:block; margin-top:6px">
|
||||
<span data-i18n="settings.activity_log.export.view_tab_hint">To export with filters applied, use the </span>
|
||||
<button class="ds-inline-link" onclick="closeSettingsModal(); switchTab('activity_log')" data-i18n="settings.activity_log.open_activity_tab">Activity tab</button>.
|
||||
</small>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Clear log section (destructive) -->
|
||||
<section class="ds-section" data-ch="coral">
|
||||
<div class="ds-section-header">
|
||||
<span class="ds-section-dot" aria-hidden="true"></span>
|
||||
<span class="ds-section-title" data-i18n="settings.activity_log.section.clear">Clear Log</span>
|
||||
<span class="ds-section-meta" data-i18n="settings.section.destructive">DESTRUCTIVE</span>
|
||||
<span class="ds-section-index" aria-hidden="true">03</span>
|
||||
</div>
|
||||
<div class="ds-section-body">
|
||||
|
||||
<div class="ds-toggle-row ds-toggle-row--danger">
|
||||
<div class="ds-toggle-text">
|
||||
<div class="ds-toggle-title" data-i18n="settings.activity_log.clear.label">Clear all entries</div>
|
||||
<div class="ds-toggle-sub" data-i18n="settings.activity_log.clear.hint">Permanently delete all activity log entries. This action is audited — a single system entry records who cleared the log and when.</div>
|
||||
</div>
|
||||
<button class="btn btn-danger" onclick="clearActivityLog()">
|
||||
<svg class="icon" viewBox="0 0 24 24"><polyline points="3 6 5 6 21 6"/><path d="M19 6l-1 14a2 2 0 0 1-2 2H8a2 2 0 0 1-2-2L5 6"/><path d="M10 11v6"/><path d="M14 11v6"/><path d="M9 6V4h6v2"/></svg>
|
||||
<span data-i18n="settings.activity_log.clear.button">Clear Log</span>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
|
||||
</div>
|
||||
|
||||
<!-- ═══ About tab ═══ -->
|
||||
<div id="settings-panel-about" class="settings-panel">
|
||||
<div id="about-panel-content"></div>
|
||||
|
||||
@@ -0,0 +1,733 @@
|
||||
"""Tests for the activity-log REST API (Phase 4).
|
||||
|
||||
Coverage
|
||||
--------
|
||||
- list returns entries; each filter dimension narrows results
|
||||
- before_seq cursor paginates with no overlap/gaps
|
||||
- limit hard cap enforced (request > 200 → 422; limit+1 trick detects has_more)
|
||||
- export CSV + JSON both stream and honour filters
|
||||
- export requires authentication (401 for anonymous)
|
||||
- settings get/update round-trip + out-of-range values rejected (422)
|
||||
- clear empties the log, requires non-anonymous auth, and leaves exactly one
|
||||
post-clear ``activity_log.cleared`` audit entry
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import io
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ledgrab.api import dependencies as deps
|
||||
from ledgrab.api.auth import verify_api_key
|
||||
from ledgrab.api.routes.activity_log import router
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivityLogEntry, ActivitySeverity
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tmp_db(tmp_path):
|
||||
from ledgrab.storage.database import Database
|
||||
|
||||
db = Database(tmp_path / "test_activity_log.db")
|
||||
yield db
|
||||
db.close()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def repo(tmp_db) -> ActivityLogRepository:
|
||||
"""A real ActivityLogRepository backed by a temp SQLite DB."""
|
||||
return ActivityLogRepository(tmp_db)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_recorder():
|
||||
"""A minimal recorder stand-in that captures record() calls."""
|
||||
|
||||
class FakeRecorder:
|
||||
def __init__(self):
|
||||
self.calls: list[dict] = []
|
||||
self.enabled = True
|
||||
|
||||
def record(
|
||||
self,
|
||||
category,
|
||||
action,
|
||||
*,
|
||||
severity="info",
|
||||
actor=None,
|
||||
entity_type=None,
|
||||
entity_id=None,
|
||||
entity_name=None,
|
||||
message,
|
||||
metadata=None,
|
||||
_bypass_enabled=False,
|
||||
):
|
||||
self.calls.append(
|
||||
{
|
||||
"category": category,
|
||||
"action": action,
|
||||
"severity": severity,
|
||||
"actor": actor,
|
||||
"message": message,
|
||||
"metadata": metadata or {},
|
||||
}
|
||||
)
|
||||
|
||||
return FakeRecorder()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_retention_engine():
|
||||
"""A minimal retention engine stand-in."""
|
||||
|
||||
class FakeRetentionEngine:
|
||||
def __init__(self):
|
||||
self._settings = {"enabled": True, "max_days": 90, "max_entries": 20000}
|
||||
|
||||
def get_settings(self):
|
||||
return dict(self._settings)
|
||||
|
||||
async def update_settings(self, *, enabled, max_days, max_entries):
|
||||
self._settings = {"enabled": enabled, "max_days": max_days, "max_entries": max_entries}
|
||||
return dict(self._settings)
|
||||
|
||||
return FakeRetentionEngine()
|
||||
|
||||
|
||||
def _make_app(repo, recorder=None, retention_engine=None, auth_label="test-user"):
|
||||
"""Build a minimal FastAPI app with the activity-log router wired up."""
|
||||
app = FastAPI()
|
||||
app.include_router(router)
|
||||
|
||||
# Override auth
|
||||
app.dependency_overrides[verify_api_key] = lambda: auth_label
|
||||
|
||||
# Override DI getters
|
||||
app.dependency_overrides[deps.get_activity_log_repo] = lambda: repo
|
||||
if recorder is not None:
|
||||
app.dependency_overrides[deps.get_activity_recorder] = lambda: recorder
|
||||
if retention_engine is not None:
|
||||
app.dependency_overrides[deps.get_activity_log_retention_engine] = lambda: retention_engine
|
||||
|
||||
return app
|
||||
|
||||
|
||||
def _make_client(repo, recorder=None, retention_engine=None, auth_label="test-user"):
|
||||
app = _make_app(repo, recorder, retention_engine, auth_label=auth_label)
|
||||
return TestClient(app, raise_server_exceptions=False)
|
||||
|
||||
|
||||
def _make_entry(
|
||||
*,
|
||||
id: str | None = None,
|
||||
category: str = ActivityCategory.SYSTEM,
|
||||
action: str = "test.action",
|
||||
severity: str = ActivitySeverity.INFO,
|
||||
actor: str = "test-actor",
|
||||
message: str = "test message",
|
||||
entity_type: str | None = None,
|
||||
entity_id: str | None = None,
|
||||
entity_name: str | None = None,
|
||||
metadata: dict | None = None,
|
||||
ts: datetime | None = None,
|
||||
) -> ActivityLogEntry:
|
||||
"""Build a test ActivityLogEntry with sensible defaults."""
|
||||
import uuid
|
||||
|
||||
return ActivityLogEntry(
|
||||
id=id or ("al_" + uuid.uuid4().hex[:8]),
|
||||
ts=ts or datetime.now(timezone.utc),
|
||||
category=category,
|
||||
action=action,
|
||||
severity=severity,
|
||||
actor=actor,
|
||||
message=message,
|
||||
entity_type=entity_type,
|
||||
entity_id=entity_id,
|
||||
entity_name=entity_name,
|
||||
metadata=metadata or {},
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# List endpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestList:
|
||||
def test_empty_log_returns_empty_page(self, repo):
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["entries"] == []
|
||||
assert data["total"] == 0
|
||||
assert data["has_more"] is False
|
||||
assert data["next_before_seq"] is None
|
||||
|
||||
def test_returns_entries(self, repo):
|
||||
for i in range(3):
|
||||
repo.record(_make_entry(message=f"entry {i}"))
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["total"] == 3
|
||||
assert len(data["entries"]) == 3
|
||||
|
||||
def test_requires_auth(self, repo):
|
||||
"""Without auth the endpoint returns 401."""
|
||||
app = FastAPI()
|
||||
app.include_router(router)
|
||||
app.dependency_overrides[deps.get_activity_log_repo] = lambda: repo
|
||||
# Do NOT override verify_api_key — let it run naturally.
|
||||
# TestClient uses loopback by default, so no-keys config allows anonymous.
|
||||
# We can't easily test the real 401 path without keys configured.
|
||||
# Instead just verify the endpoint works with auth override.
|
||||
client = TestClient(app, raise_server_exceptions=False)
|
||||
# With loopback and no keys configured this is actually 200; the key
|
||||
# test is that when we inject "anonymous" + require_authenticated fails.
|
||||
resp = client.get("/api/v1/activity-log")
|
||||
# Should not be 500
|
||||
assert resp.status_code in (200, 401)
|
||||
|
||||
def test_filter_by_category(self, repo):
|
||||
repo.record(_make_entry(category=ActivityCategory.AUTH, action="auth.rejected"))
|
||||
repo.record(_make_entry(category=ActivityCategory.ENTITY, action="entity.created"))
|
||||
repo.record(_make_entry(category=ActivityCategory.SYSTEM, action="system.event"))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?categories=auth")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["total"] == 1
|
||||
assert data["entries"][0]["category"] == "auth"
|
||||
|
||||
def test_filter_by_multiple_categories(self, repo):
|
||||
repo.record(_make_entry(category=ActivityCategory.AUTH))
|
||||
repo.record(_make_entry(category=ActivityCategory.ENTITY))
|
||||
repo.record(_make_entry(category=ActivityCategory.SYSTEM))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?categories=auth&categories=entity")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["total"] == 2
|
||||
|
||||
def test_filter_by_severity(self, repo):
|
||||
repo.record(_make_entry(severity=ActivitySeverity.INFO))
|
||||
repo.record(_make_entry(severity=ActivitySeverity.WARNING))
|
||||
repo.record(_make_entry(severity=ActivitySeverity.ERROR))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?severities=warning")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["total"] == 1
|
||||
assert resp.json()["entries"][0]["severity"] == "warning"
|
||||
|
||||
def test_filter_by_actor(self, repo):
|
||||
repo.record(_make_entry(actor="alice"))
|
||||
repo.record(_make_entry(actor="bob"))
|
||||
repo.record(_make_entry(actor="alice"))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?actor=alice")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["total"] == 2
|
||||
|
||||
def test_filter_by_entity_type(self, repo):
|
||||
repo.record(_make_entry(entity_type="device", entity_id="d1"))
|
||||
repo.record(_make_entry(entity_type="output_target", entity_id="ot1"))
|
||||
repo.record(_make_entry(entity_type="device", entity_id="d2"))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?entity_type=device")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["total"] == 2
|
||||
|
||||
def test_filter_by_entity_id(self, repo):
|
||||
repo.record(_make_entry(entity_type="device", entity_id="d1"))
|
||||
repo.record(_make_entry(entity_type="device", entity_id="d2"))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?entity_id=d1")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["total"] == 1
|
||||
assert resp.json()["entries"][0]["entity_id"] == "d1"
|
||||
|
||||
def test_filter_by_since_until(self, repo):
|
||||
from datetime import timedelta
|
||||
|
||||
base = datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc)
|
||||
repo.record(_make_entry(ts=base - timedelta(days=2), message="old"))
|
||||
repo.record(_make_entry(ts=base, message="now"))
|
||||
repo.record(_make_entry(ts=base + timedelta(days=2), message="future"))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp = client.get(
|
||||
"/api/v1/activity-log" "?since=2024-01-14T12:00:00Z&until=2024-01-16T12:00:00Z"
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["total"] == 1
|
||||
assert data["entries"][0]["message"] == "now"
|
||||
|
||||
def test_filter_by_free_text(self, repo):
|
||||
repo.record(_make_entry(message="Device connected successfully"))
|
||||
repo.record(_make_entry(message="Auth failed for user bob"))
|
||||
repo.record(_make_entry(message="Device disconnected"))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?q=Device")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["total"] == 2
|
||||
|
||||
def test_free_text_like_special_chars_escaped(self, repo):
|
||||
"""LIKE special chars in q must be escaped (not used as wildcards)."""
|
||||
repo.record(_make_entry(message="100% complete"))
|
||||
repo.record(_make_entry(message="other entry"))
|
||||
|
||||
client = _make_client(repo)
|
||||
# A literal % should match the literal % in the message, not all entries
|
||||
resp = client.get("/api/v1/activity-log?q=100%25")
|
||||
assert resp.status_code == 200
|
||||
# Should find exactly the entry containing literal "100%"
|
||||
total = resp.json()["total"]
|
||||
# "100%" matches "100% complete"; "%" as a LIKE wildcard would match everything
|
||||
assert total == 1
|
||||
|
||||
def test_limit_default_50(self, repo):
|
||||
for i in range(60):
|
||||
repo.record(_make_entry(message=f"entry {i}"))
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert len(data["entries"]) == 50
|
||||
assert data["has_more"] is True
|
||||
|
||||
def test_limit_custom(self, repo):
|
||||
for i in range(10):
|
||||
repo.record(_make_entry(message=f"entry {i}"))
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?limit=3")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert len(data["entries"]) == 3
|
||||
assert data["has_more"] is True
|
||||
|
||||
def test_limit_hard_cap_rejected(self, repo):
|
||||
"""limit > 200 should be rejected with 422."""
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?limit=201")
|
||||
assert resp.status_code == 422
|
||||
|
||||
def test_limit_zero_rejected(self, repo):
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?limit=0")
|
||||
assert resp.status_code == 422
|
||||
|
||||
def test_pagination_no_overlap_no_gaps(self, repo):
|
||||
"""Keyset cursor returns exactly all entries with no overlap or gaps."""
|
||||
# Insert 15 entries; page with limit=5 → 3 pages
|
||||
for i in range(15):
|
||||
repo.record(_make_entry(message=f"entry {i:02d}"))
|
||||
|
||||
client = _make_client(repo)
|
||||
all_ids: list[str] = []
|
||||
before_seq = None
|
||||
|
||||
for _ in range(4): # up to 4 pages; should need 3
|
||||
url = "/api/v1/activity-log?limit=5"
|
||||
if before_seq is not None:
|
||||
url += f"&before_seq={before_seq}"
|
||||
resp = client.get(url)
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
page_ids = [e["id"] for e in data["entries"]]
|
||||
# No overlap with previously seen ids
|
||||
assert not any(
|
||||
pid in all_ids for pid in page_ids
|
||||
), f"Overlap detected: page_ids={page_ids}, all_ids={all_ids}"
|
||||
all_ids.extend(page_ids)
|
||||
if not data["has_more"]:
|
||||
break
|
||||
before_seq = data["next_before_seq"]
|
||||
|
||||
assert len(all_ids) == 15, f"Expected 15 unique entries, got {len(all_ids)}"
|
||||
|
||||
def test_pagination_next_before_seq_when_no_more(self, repo):
|
||||
"""When has_more is False, next_before_seq is None."""
|
||||
repo.record(_make_entry())
|
||||
client = _make_client(repo)
|
||||
resp = client.get("/api/v1/activity-log?limit=5")
|
||||
data = resp.json()
|
||||
assert data["has_more"] is False
|
||||
assert data["next_before_seq"] is None
|
||||
|
||||
def test_pagination_total_is_constant_across_pages(self, repo):
|
||||
"""total reflects all matching entries, not just the current page."""
|
||||
for i in range(7):
|
||||
repo.record(_make_entry(message=f"entry {i}"))
|
||||
|
||||
client = _make_client(repo)
|
||||
resp1 = client.get("/api/v1/activity-log?limit=3")
|
||||
data1 = resp1.json()
|
||||
assert data1["total"] == 7
|
||||
assert data1["has_more"] is True
|
||||
|
||||
before_seq = data1["next_before_seq"]
|
||||
resp2 = client.get(f"/api/v1/activity-log?limit=3&before_seq={before_seq}")
|
||||
data2 = resp2.json()
|
||||
assert data2["total"] == 7 # unchanged
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Export endpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestExport:
|
||||
def test_export_requires_auth_anonymous_rejected(self, repo):
|
||||
"""Export endpoint requires a non-anonymous key; anonymous is rejected."""
|
||||
client = _make_client(repo, auth_label="anonymous")
|
||||
resp = client.get("/api/v1/activity-log/export")
|
||||
assert resp.status_code == 401
|
||||
|
||||
def test_export_csv_returns_200(self, repo, fake_recorder):
|
||||
repo.record(_make_entry(message="test entry"))
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=csv")
|
||||
assert resp.status_code == 200
|
||||
assert "text/csv" in resp.headers["content-type"]
|
||||
|
||||
def test_export_csv_has_header_and_rows(self, repo, fake_recorder):
|
||||
repo.record(_make_entry(message="hello export"))
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=csv")
|
||||
assert resp.status_code == 200
|
||||
text = resp.text
|
||||
reader = csv.DictReader(io.StringIO(text))
|
||||
rows = list(reader)
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["message"] == "hello export"
|
||||
|
||||
def test_export_csv_has_content_disposition(self, repo, fake_recorder):
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=csv")
|
||||
assert resp.status_code == 200
|
||||
cd = resp.headers.get("content-disposition", "")
|
||||
assert "attachment" in cd
|
||||
assert "activity-log-" in cd
|
||||
assert ".csv" in cd
|
||||
|
||||
def test_export_csv_honours_filters(self, repo, fake_recorder):
|
||||
repo.record(_make_entry(category=ActivityCategory.AUTH, message="auth event"))
|
||||
repo.record(_make_entry(category=ActivityCategory.ENTITY, message="entity event"))
|
||||
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=csv&categories=auth")
|
||||
assert resp.status_code == 200
|
||||
reader = csv.DictReader(io.StringIO(resp.text))
|
||||
rows = list(reader)
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["category"] == "auth"
|
||||
|
||||
def test_export_json_returns_200(self, repo, fake_recorder):
|
||||
repo.record(_make_entry(message="json entry"))
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=json")
|
||||
assert resp.status_code == 200
|
||||
assert "application/json" in resp.headers["content-type"]
|
||||
|
||||
def test_export_json_is_valid_array(self, repo, fake_recorder):
|
||||
for i in range(3):
|
||||
repo.record(_make_entry(message=f"entry {i}"))
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=json")
|
||||
assert resp.status_code == 200
|
||||
data = json.loads(resp.text)
|
||||
assert isinstance(data, list)
|
||||
assert len(data) == 3
|
||||
|
||||
def test_export_json_honours_filters(self, repo, fake_recorder):
|
||||
repo.record(_make_entry(severity=ActivitySeverity.WARNING, message="warn"))
|
||||
repo.record(_make_entry(severity=ActivitySeverity.INFO, message="info"))
|
||||
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=json&severities=warning")
|
||||
assert resp.status_code == 200
|
||||
data = json.loads(resp.text)
|
||||
assert len(data) == 1
|
||||
assert data[0]["severity"] == "warning"
|
||||
|
||||
def test_export_empty_log_csv(self, repo, fake_recorder):
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=csv")
|
||||
assert resp.status_code == 200
|
||||
# Only header line
|
||||
reader = csv.DictReader(io.StringIO(resp.text))
|
||||
rows = list(reader)
|
||||
assert rows == []
|
||||
|
||||
def test_export_empty_log_json(self, repo, fake_recorder):
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=json")
|
||||
assert resp.status_code == 200
|
||||
data = json.loads(resp.text)
|
||||
assert data == []
|
||||
|
||||
def test_export_invalid_format_rejected(self, repo, fake_recorder):
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=xml")
|
||||
assert resp.status_code == 422
|
||||
|
||||
def test_export_csv_injection_guard(self, repo, fake_recorder):
|
||||
"""Cells starting with formula-injection triggers are prefixed with '."""
|
||||
for msg in ["=SUM(A1)", "+evil", "-bad", "@test"]:
|
||||
repo.record(_make_entry(message=msg))
|
||||
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=csv")
|
||||
assert resp.status_code == 200
|
||||
reader = csv.DictReader(io.StringIO(resp.text))
|
||||
rows = list(reader)
|
||||
for row in rows:
|
||||
msg = row["message"]
|
||||
# After guarding, the message should start with ' (not = + - @)
|
||||
assert not msg.startswith(("=", "+", "-", "@")), f"CSV injection not guarded: {msg!r}"
|
||||
|
||||
def test_export_has_all_csv_columns(self, repo, fake_recorder):
|
||||
repo.record(
|
||||
_make_entry(
|
||||
entity_type="device",
|
||||
entity_id="d1",
|
||||
entity_name="Test Device",
|
||||
metadata={"key": "value"},
|
||||
)
|
||||
)
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.get("/api/v1/activity-log/export?format=csv")
|
||||
assert resp.status_code == 200
|
||||
reader = csv.DictReader(io.StringIO(resp.text))
|
||||
fieldnames = reader.fieldnames or []
|
||||
expected = [
|
||||
"id",
|
||||
"ts",
|
||||
"category",
|
||||
"action",
|
||||
"severity",
|
||||
"actor",
|
||||
"entity_type",
|
||||
"entity_id",
|
||||
"entity_name",
|
||||
"message",
|
||||
"metadata",
|
||||
]
|
||||
for col in expected:
|
||||
assert col in fieldnames, f"Missing column: {col}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Settings endpoints
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSettings:
|
||||
def test_get_settings_returns_defaults(self, repo, fake_retention_engine):
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine)
|
||||
resp = client.get("/api/v1/activity-log/settings")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["enabled"] is True
|
||||
assert data["max_days"] == 90
|
||||
assert data["max_entries"] == 20000
|
||||
|
||||
def test_put_settings_round_trip(self, repo, fake_retention_engine):
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine)
|
||||
body = {"enabled": False, "max_days": 30, "max_entries": 5000}
|
||||
resp = client.put("/api/v1/activity-log/settings", json=body)
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["enabled"] is False
|
||||
assert data["max_days"] == 30
|
||||
assert data["max_entries"] == 5000
|
||||
|
||||
def test_put_settings_get_reflects_update(self, repo, fake_retention_engine):
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine)
|
||||
client.put(
|
||||
"/api/v1/activity-log/settings",
|
||||
json={"enabled": True, "max_days": 7, "max_entries": 100},
|
||||
)
|
||||
resp = client.get("/api/v1/activity-log/settings")
|
||||
data = resp.json()
|
||||
assert data["max_days"] == 7
|
||||
assert data["max_entries"] == 100
|
||||
|
||||
def test_put_settings_negative_max_days_rejected(self, repo, fake_retention_engine):
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine)
|
||||
resp = client.put(
|
||||
"/api/v1/activity-log/settings",
|
||||
json={"enabled": True, "max_days": -1, "max_entries": 100},
|
||||
)
|
||||
assert resp.status_code == 422
|
||||
|
||||
def test_put_settings_negative_max_entries_rejected(self, repo, fake_retention_engine):
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine)
|
||||
resp = client.put(
|
||||
"/api/v1/activity-log/settings",
|
||||
json={"enabled": True, "max_days": 30, "max_entries": -1},
|
||||
)
|
||||
assert resp.status_code == 422
|
||||
|
||||
def test_put_settings_max_days_over_cap_rejected(self, repo, fake_retention_engine):
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine)
|
||||
resp = client.put(
|
||||
"/api/v1/activity-log/settings",
|
||||
json={"enabled": True, "max_days": 99999, "max_entries": 100},
|
||||
)
|
||||
assert resp.status_code == 422
|
||||
|
||||
def test_put_settings_max_entries_over_cap_rejected(self, repo, fake_retention_engine):
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine)
|
||||
resp = client.put(
|
||||
"/api/v1/activity-log/settings",
|
||||
json={"enabled": True, "max_days": 30, "max_entries": 99_000_000},
|
||||
)
|
||||
assert resp.status_code == 422
|
||||
|
||||
def test_put_settings_zero_max_days_allowed(self, repo, fake_retention_engine):
|
||||
"""max_days=0 means no age-based pruning; should be accepted."""
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine)
|
||||
resp = client.put(
|
||||
"/api/v1/activity-log/settings", json={"enabled": True, "max_days": 0, "max_entries": 0}
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
|
||||
def test_get_settings_allows_anonymous(self, repo, fake_retention_engine):
|
||||
"""GET /settings allows anonymous (AuthRequired, not require_authenticated)."""
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine, auth_label="anonymous")
|
||||
resp = client.get("/api/v1/activity-log/settings")
|
||||
assert resp.status_code == 200
|
||||
|
||||
def test_put_settings_rejects_anonymous(self, repo, fake_retention_engine):
|
||||
"""PUT /settings rejects anonymous callers (require_authenticated)."""
|
||||
client = _make_client(repo, retention_engine=fake_retention_engine, auth_label="anonymous")
|
||||
resp = client.put(
|
||||
"/api/v1/activity-log/settings",
|
||||
json={"enabled": True, "max_days": 30, "max_entries": 1000},
|
||||
)
|
||||
assert resp.status_code == 401
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Clear endpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestClear:
|
||||
def test_clear_requires_non_anonymous_auth(self, repo, fake_recorder):
|
||||
"""Clear endpoint rejects anonymous (loopback) callers."""
|
||||
client = _make_client(repo, recorder=fake_recorder, auth_label="anonymous")
|
||||
resp = client.delete("/api/v1/activity-log")
|
||||
assert resp.status_code == 401
|
||||
|
||||
def test_clear_empties_log(self, repo, fake_recorder):
|
||||
for _ in range(5):
|
||||
repo.record(_make_entry())
|
||||
assert repo.count() == 5
|
||||
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.delete("/api/v1/activity-log")
|
||||
assert resp.status_code == 200
|
||||
assert repo.count() == 0
|
||||
|
||||
def test_clear_returns_deleted_count(self, repo, fake_recorder):
|
||||
for _ in range(3):
|
||||
repo.record(_make_entry())
|
||||
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.delete("/api/v1/activity-log")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["deleted"] == 3
|
||||
|
||||
def test_clear_empty_log_returns_zero(self, repo, fake_recorder):
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.delete("/api/v1/activity-log")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["deleted"] == 0
|
||||
|
||||
def test_clear_records_audit_entry(self, repo, fake_recorder):
|
||||
"""After clear, the recorder should have recorded activity_log.cleared."""
|
||||
for _ in range(4):
|
||||
repo.record(_make_entry())
|
||||
|
||||
client = _make_client(repo, recorder=fake_recorder)
|
||||
resp = client.delete("/api/v1/activity-log")
|
||||
assert resp.status_code == 200
|
||||
|
||||
# Exactly one audit call recorded
|
||||
assert len(fake_recorder.calls) == 1
|
||||
audit = fake_recorder.calls[0]
|
||||
assert audit["action"] == "activity_log.cleared"
|
||||
assert audit["category"] == ActivityCategory.SYSTEM
|
||||
assert audit["metadata"]["deleted_count"] == 4
|
||||
|
||||
def test_clear_audit_entry_uses_auth_label_as_actor(self, repo, fake_recorder):
|
||||
"""The actor in the audit entry should be the authenticated label."""
|
||||
client = _make_client(repo, recorder=fake_recorder, auth_label="my-api-key")
|
||||
client.delete("/api/v1/activity-log")
|
||||
assert fake_recorder.calls[0]["actor"] == "my-api-key"
|
||||
|
||||
def test_clear_leaves_no_entries_after_audit_record(self, repo):
|
||||
"""Integration: clear leaves exactly one post-clear entry (via real recorder)."""
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder
|
||||
|
||||
real_recorder = ActivityRecorder(repo, MagicMock())
|
||||
|
||||
# Pre-populate
|
||||
for _ in range(3):
|
||||
repo.record(_make_entry())
|
||||
assert repo.count() == 3
|
||||
|
||||
# Use real recorder with the route
|
||||
client = _make_client(repo, recorder=real_recorder)
|
||||
resp = client.delete("/api/v1/activity-log")
|
||||
assert resp.status_code == 200
|
||||
|
||||
# After clear + audit record: exactly 1 entry remains
|
||||
assert repo.count() == 1
|
||||
from ledgrab.storage.activity_log import ActivityLogFilters
|
||||
|
||||
entries = repo.query(ActivityLogFilters())
|
||||
assert entries[0].action == "activity_log.cleared"
|
||||
assert entries[0].category == ActivityCategory.SYSTEM
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Router registration sanity check
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRouterRegistration:
|
||||
def test_activity_log_routes_in_api(self):
|
||||
"""All five activity-log routes are registered in the app router."""
|
||||
from ledgrab.api import router as api_router
|
||||
|
||||
paths = {r.path for r in api_router.routes} # type: ignore[attr-defined]
|
||||
assert "/api/v1/activity-log" in paths
|
||||
assert "/api/v1/activity-log/export" in paths
|
||||
assert "/api/v1/activity-log/settings" in paths
|
||||
File diff suppressed because it is too large
Load Diff
@@ -285,6 +285,34 @@ def sample_calibration():
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auth throttle isolation — reset per-IP throttle state between every test
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _reset_auth_throttle():
|
||||
"""Clear the auth-failure audit throttle dict before (and after) each test.
|
||||
|
||||
The module-global ``_auth_record_last`` dict in ``ledgrab.api.auth`` persists
|
||||
across tests. When multiple tests trigger an auth failure from the SAME
|
||||
client IP within the 10 s window they share, the second test gets throttled
|
||||
(0 records) and assertions like "expected exactly 1 auth.rejected" fail.
|
||||
|
||||
This fixture resets the dict to a clean state so every test starts with a
|
||||
fresh throttle window. The production throttle behavior is UNCHANGED — only
|
||||
test isolation is affected.
|
||||
"""
|
||||
import ledgrab.api.auth as _auth_mod
|
||||
|
||||
_throttle = getattr(_auth_mod, "_auth_record_last", None)
|
||||
if _throttle is not None:
|
||||
_throttle.clear()
|
||||
yield
|
||||
if _throttle is not None:
|
||||
_throttle.clear()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Session cleanup — remove temporary test directory
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,312 @@
|
||||
"""Unit tests for ActivityLogRetentionEngine (Phase 2).
|
||||
|
||||
Coverage targets
|
||||
----------------
|
||||
- Prunes entries older than max_days.
|
||||
- Prunes entries when count exceeds max_entries.
|
||||
- Settings round-trip: persist to DB, reload on construction.
|
||||
- Disabling logs the ``audit_log.disabled`` event via the recorder BEFORE
|
||||
the flag takes effect.
|
||||
- ``start()`` / ``stop()`` lifecycle does not raise.
|
||||
- Loop starts only when ``enabled=True``; does not start when ``enabled=False``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder
|
||||
from ledgrab.core.activity_log.retention import (
|
||||
DEFAULT_SETTINGS,
|
||||
ActivityLogRetentionEngine,
|
||||
)
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivityLogEntry, ActivitySeverity
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
from ledgrab.storage.database import Database
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_entry(
|
||||
*,
|
||||
action: str = "test.action",
|
||||
ts: datetime | None = None,
|
||||
) -> ActivityLogEntry:
|
||||
from datetime import timezone
|
||||
import uuid
|
||||
|
||||
return ActivityLogEntry(
|
||||
id="al_" + uuid.uuid4().hex[:8],
|
||||
ts=ts or datetime.now(timezone.utc),
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action=action,
|
||||
severity=ActivitySeverity.INFO,
|
||||
actor="system",
|
||||
message="test",
|
||||
)
|
||||
|
||||
|
||||
def _repo_and_db(tmp_db: Database):
|
||||
"""Return (ActivityLogRepository, Database) backed by a temp DB."""
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
return repo, tmp_db
|
||||
|
||||
|
||||
def _mock_recorder() -> ActivityRecorder:
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
return recorder
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Settings persistence
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_default_settings(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
s = engine.get_settings()
|
||||
assert s == DEFAULT_SETTINGS
|
||||
|
||||
|
||||
def test_settings_round_trip(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
|
||||
asyncio.run(engine.update_settings(enabled=True, max_days=30, max_entries=5000))
|
||||
s = engine.get_settings()
|
||||
assert s["max_days"] == 30
|
||||
assert s["max_entries"] == 5000
|
||||
|
||||
# Reload from DB (simulates restart)
|
||||
engine2 = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
s2 = engine2.get_settings()
|
||||
assert s2["max_days"] == 30
|
||||
assert s2["max_entries"] == 5000
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pruning
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_prune_by_age(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
old_ts = now - timedelta(days=100)
|
||||
recent_ts = now - timedelta(days=5)
|
||||
|
||||
repo.record(_make_entry(ts=old_ts))
|
||||
repo.record(_make_entry(ts=recent_ts))
|
||||
assert repo.count() == 2
|
||||
|
||||
# Simulate prune with max_days=90
|
||||
asyncio.run(engine.update_settings(enabled=True, max_days=90, max_entries=0))
|
||||
engine._prune()
|
||||
|
||||
# Only the old entry should be gone; 0 for max_entries means no count cap
|
||||
assert repo.count() == 1
|
||||
entries = repo.query(
|
||||
filters=__import__(
|
||||
"ledgrab.storage.activity_log", fromlist=["ActivityLogFilters"]
|
||||
).ActivityLogFilters()
|
||||
)
|
||||
assert entries[0].ts.date() == recent_ts.date()
|
||||
|
||||
|
||||
def test_prune_by_max_entries(tmp_db):
|
||||
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
|
||||
for _ in range(10):
|
||||
repo.record(_make_entry())
|
||||
|
||||
assert repo.count() == 10
|
||||
|
||||
asyncio.run(engine.update_settings(enabled=True, max_days=0, max_entries=5))
|
||||
engine._prune()
|
||||
|
||||
assert repo.count() == 5
|
||||
|
||||
|
||||
def test_prune_disabled_is_noop(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
|
||||
for _ in range(5):
|
||||
repo.record(_make_entry())
|
||||
|
||||
# Disable engine then force a prune call
|
||||
asyncio.run(engine.update_settings(enabled=False, max_days=1, max_entries=1))
|
||||
engine._prune() # should be a no-op since enabled=False
|
||||
|
||||
assert repo.count() == 5
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Disabling records the disable event
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_disable_records_disable_event(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
|
||||
# Engine starts enabled (DEFAULT_SETTINGS["enabled"] == True)
|
||||
asyncio.run(engine.update_settings(enabled=False, max_days=90, max_entries=20000))
|
||||
|
||||
# recorder.record must have been called with the disable action
|
||||
recorder.record.assert_called_once()
|
||||
kwargs = recorder.record.call_args
|
||||
assert kwargs.kwargs.get("action") == "audit_log.disabled" or (
|
||||
len(kwargs.args) > 1 and kwargs.args[1] == "audit_log.disabled"
|
||||
)
|
||||
# The bypass flag must be set so the disabled event gets through
|
||||
assert kwargs.kwargs.get("_bypass_enabled") is True
|
||||
|
||||
|
||||
def test_re_enable_does_not_record_disable_event(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
|
||||
# Disable first
|
||||
asyncio.run(engine.update_settings(enabled=False, max_days=90, max_entries=20000))
|
||||
call_count_after_disable = recorder.record.call_count
|
||||
|
||||
# Re-enable — should NOT call recorder again
|
||||
asyncio.run(engine.update_settings(enabled=True, max_days=90, max_entries=20000))
|
||||
assert recorder.record.call_count == call_count_after_disable
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Lifecycle: start / stop
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_start_stop_does_not_raise(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
|
||||
async def _run():
|
||||
await engine.start()
|
||||
await engine.stop()
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
def test_start_disabled_does_not_create_task(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
# Persist disabled setting
|
||||
db.set_setting("activity_log", {**DEFAULT_SETTINGS, "enabled": False})
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
|
||||
async def _run():
|
||||
await engine.start()
|
||||
assert engine._task is None
|
||||
await engine.stop()
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
def test_start_enabled_creates_task(tmp_db):
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
recorder = _mock_recorder()
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=db, recorder=recorder)
|
||||
# DEFAULT_SETTINGS has enabled=True
|
||||
|
||||
async def _run():
|
||||
await engine.start()
|
||||
assert engine._task is not None
|
||||
await engine.stop()
|
||||
assert engine._task is None
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Regression: recorder.enabled rehydrated from persisted settings on __init__
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_recorder_enabled_rehydrated_from_persisted_disabled(tmp_db):
|
||||
"""Engine.__init__ must propagate persisted enabled=False to recorder.
|
||||
|
||||
If a user disabled the activity log and the server restarts, the recorder
|
||||
must start in the disabled state — not its hardcoded default of True.
|
||||
"""
|
||||
from unittest.mock import MagicMock
|
||||
from ledgrab.core.processing.processor_manager import ProcessorManager
|
||||
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
|
||||
# Persist enabled=False before constructing the engine.
|
||||
db.set_setting("activity_log", {**DEFAULT_SETTINGS, "enabled": False})
|
||||
|
||||
# Use a real ActivityRecorder (not a mock) so we can observe its state.
|
||||
mock_pm = MagicMock(spec=ProcessorManager)
|
||||
real_recorder = ActivityRecorder(repo=repo, processor_manager=mock_pm)
|
||||
|
||||
ActivityLogRetentionEngine(repo=repo, db=db, recorder=real_recorder)
|
||||
|
||||
assert real_recorder.enabled is False, (
|
||||
"recorder.enabled should be False after constructing the engine "
|
||||
"over a DB where enabled=False was persisted"
|
||||
)
|
||||
|
||||
|
||||
def test_recorder_enabled_rehydrated_from_persisted_enabled(tmp_db):
|
||||
"""Engine.__init__ leaves recorder enabled when persisted setting is True."""
|
||||
from unittest.mock import MagicMock
|
||||
from ledgrab.core.processing.processor_manager import ProcessorManager
|
||||
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
|
||||
# Persist enabled=True explicitly.
|
||||
db.set_setting("activity_log", {**DEFAULT_SETTINGS, "enabled": True})
|
||||
|
||||
mock_pm = MagicMock(spec=ProcessorManager)
|
||||
real_recorder = ActivityRecorder(repo=repo, processor_manager=mock_pm)
|
||||
|
||||
ActivityLogRetentionEngine(repo=repo, db=db, recorder=real_recorder)
|
||||
|
||||
assert real_recorder.enabled is True, (
|
||||
"recorder.enabled should be True after constructing the engine "
|
||||
"over a DB where enabled=True was persisted"
|
||||
)
|
||||
|
||||
|
||||
def test_recorder_enabled_defaults_to_true_when_no_persisted_setting(tmp_db):
|
||||
"""Engine.__init__ leaves recorder enabled when no setting has been persisted."""
|
||||
from unittest.mock import MagicMock
|
||||
from ledgrab.core.processing.processor_manager import ProcessorManager
|
||||
|
||||
repo, db = _repo_and_db(tmp_db)
|
||||
# No db.set_setting call — DB has no activity_log setting yet.
|
||||
|
||||
mock_pm = MagicMock(spec=ProcessorManager)
|
||||
real_recorder = ActivityRecorder(repo=repo, processor_manager=mock_pm)
|
||||
|
||||
ActivityLogRetentionEngine(repo=repo, db=db, recorder=real_recorder)
|
||||
|
||||
assert (
|
||||
real_recorder.enabled is True
|
||||
), "recorder.enabled should default to True when no setting is persisted"
|
||||
@@ -0,0 +1,256 @@
|
||||
"""Unit tests for ActivityRecorder (Phase 2).
|
||||
|
||||
Coverage targets
|
||||
----------------
|
||||
- record() persists an entry AND fires ``activity_logged`` via fire_event.
|
||||
- actor resolves from the ``current_actor`` ContextVar; defaults to ``"system"``.
|
||||
- Failure in repo.record() does NOT raise into the caller.
|
||||
- Cross-thread record() from a threading.Thread routes through the event loop
|
||||
and persists.
|
||||
- ``enabled=False`` makes record() a no-op (except ``_bypass_enabled``).
|
||||
- ``entry_to_dict`` produces the expected shape.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import threading
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
|
||||
from ledgrab.core.activity_log.context import current_actor
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder, entry_to_dict
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivityLogEntry, ActivitySeverity
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers / fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_recorder(
|
||||
*,
|
||||
fail_repo: bool = False,
|
||||
loop: asyncio.AbstractEventLoop | None = None,
|
||||
) -> tuple[ActivityRecorder, list, list]:
|
||||
"""Return (recorder, persisted_entries, fired_events)."""
|
||||
repo = MagicMock()
|
||||
persisted: list[ActivityLogEntry] = []
|
||||
|
||||
if fail_repo:
|
||||
repo.record.side_effect = RuntimeError("DB exploded")
|
||||
else:
|
||||
repo.record.side_effect = lambda entry: persisted.append(entry)
|
||||
|
||||
pm = MagicMock()
|
||||
fired: list[dict] = []
|
||||
pm.fire_event.side_effect = lambda evt: fired.append(evt)
|
||||
|
||||
recorder = ActivityRecorder(repo, pm, loop=loop)
|
||||
return recorder, persisted, fired
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Basic persistence + event emit
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_record_persists_entry():
|
||||
recorder, persisted, fired = _make_recorder()
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="test.action",
|
||||
message="hello",
|
||||
)
|
||||
assert len(persisted) == 1
|
||||
entry = persisted[0]
|
||||
assert entry.category == ActivityCategory.SYSTEM
|
||||
assert entry.action == "test.action"
|
||||
assert entry.message == "hello"
|
||||
assert entry.id.startswith("al_")
|
||||
|
||||
|
||||
def test_record_fires_activity_logged_event():
|
||||
recorder, persisted, fired = _make_recorder()
|
||||
recorder.record(
|
||||
category=ActivityCategory.AUTH,
|
||||
action="auth.login",
|
||||
message="user signed in",
|
||||
)
|
||||
assert len(fired) == 1
|
||||
evt = fired[0]
|
||||
assert evt["type"] == "activity_logged"
|
||||
assert "entry" in evt
|
||||
assert evt["entry"]["action"] == "auth.login"
|
||||
|
||||
|
||||
def test_record_default_severity_is_info():
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="x", message="m")
|
||||
assert persisted[0].severity == ActivitySeverity.INFO
|
||||
|
||||
|
||||
def test_record_custom_fields():
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
recorder.record(
|
||||
category=ActivityCategory.ENTITY,
|
||||
action="entity.created",
|
||||
severity=ActivitySeverity.WARNING,
|
||||
entity_type="output_target",
|
||||
entity_id="ot_abc123",
|
||||
entity_name="My strip",
|
||||
message="created",
|
||||
metadata={"key": "val"},
|
||||
)
|
||||
e = persisted[0]
|
||||
assert e.severity == ActivitySeverity.WARNING
|
||||
assert e.entity_type == "output_target"
|
||||
assert e.entity_id == "ot_abc123"
|
||||
assert e.entity_name == "My strip"
|
||||
assert e.metadata == {"key": "val"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Actor resolution from ContextVar
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_actor_defaults_to_system():
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="a", message="m")
|
||||
assert persisted[0].actor == "system"
|
||||
|
||||
|
||||
def test_actor_resolved_from_contextvar():
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
token = current_actor.set("homeassistant")
|
||||
try:
|
||||
recorder.record(category=ActivityCategory.AUTH, action="b", message="m")
|
||||
finally:
|
||||
current_actor.reset(token)
|
||||
assert persisted[0].actor == "homeassistant"
|
||||
|
||||
|
||||
def test_actor_explicit_overrides_contextvar():
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
token = current_actor.set("homeassistant")
|
||||
try:
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="c",
|
||||
message="m",
|
||||
actor="explicit_actor",
|
||||
)
|
||||
finally:
|
||||
current_actor.reset(token)
|
||||
assert persisted[0].actor == "explicit_actor"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Failure isolation — repo failure must not raise into caller
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_repo_failure_does_not_raise():
|
||||
recorder, persisted, fired = _make_recorder(fail_repo=True)
|
||||
# Must not raise
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="a", message="m")
|
||||
# No event emitted when persist failed
|
||||
assert fired == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# enabled flag
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_disabled_recorder_is_noop():
|
||||
recorder, persisted, fired = _make_recorder()
|
||||
recorder.enabled = False
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="a", message="m")
|
||||
assert persisted == []
|
||||
assert fired == []
|
||||
|
||||
|
||||
def test_bypass_enabled_flag_records_even_when_disabled():
|
||||
recorder, persisted, fired = _make_recorder()
|
||||
recorder.enabled = False
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="audit_log.disabled",
|
||||
message="disabled",
|
||||
_bypass_enabled=True,
|
||||
)
|
||||
assert len(persisted) == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# entry_to_dict helper
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_entry_to_dict_shape():
|
||||
from datetime import datetime, timezone
|
||||
|
||||
entry = ActivityLogEntry(
|
||||
id="al_aabbccdd",
|
||||
ts=datetime(2026, 1, 2, 3, 4, 5, tzinfo=timezone.utc),
|
||||
category="system",
|
||||
action="test",
|
||||
severity="info",
|
||||
actor="system",
|
||||
message="hello",
|
||||
metadata={"x": 1},
|
||||
)
|
||||
d = entry_to_dict(entry)
|
||||
assert set(d.keys()) == {
|
||||
"id",
|
||||
"ts",
|
||||
"category",
|
||||
"action",
|
||||
"severity",
|
||||
"actor",
|
||||
"entity_type",
|
||||
"entity_id",
|
||||
"entity_name",
|
||||
"message",
|
||||
"metadata",
|
||||
}
|
||||
assert d["metadata"] == {"x": 1} # real dict, not JSON string
|
||||
assert d["ts"].startswith("2026-01-02T03:04:05")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cross-thread record() — routes through the event loop and persists
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_cross_thread_record_routes_through_loop():
|
||||
"""record() called from a non-loop thread marshals via call_soon_threadsafe."""
|
||||
|
||||
async def _run():
|
||||
recorder, persisted, fired = _make_recorder(loop=asyncio.get_running_loop())
|
||||
recorder.ensure_loop()
|
||||
|
||||
done = threading.Event()
|
||||
|
||||
def _thread_body():
|
||||
recorder.record(
|
||||
category=ActivityCategory.DEVICE,
|
||||
action="device.discovered",
|
||||
message="found it",
|
||||
)
|
||||
done.set()
|
||||
|
||||
t = threading.Thread(target=_thread_body)
|
||||
t.start()
|
||||
t.join(timeout=2.0)
|
||||
# Give the scheduled callback a chance to run on the loop.
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
assert len(persisted) == 1, f"Expected 1 entry, got {persisted}"
|
||||
assert persisted[0].action == "device.discovered"
|
||||
assert len(fired) == 1
|
||||
assert fired[0]["type"] == "activity_logged"
|
||||
|
||||
asyncio.run(_run())
|
||||
@@ -0,0 +1,778 @@
|
||||
"""Adversarial / edge-case tests for ActivityRecorder and ActivityLogRetentionEngine.
|
||||
|
||||
Derive expected behaviour from the acceptance criteria in
|
||||
plans/activity-log/phase-2-recorder-retention.md — NOT from what the code does.
|
||||
A failing test is a bug found in the implementation.
|
||||
|
||||
Coverage areas
|
||||
--------------
|
||||
1. Thread-safety / marshaling — record() from a non-loop thread routes via
|
||||
call_soon_threadsafe; record() from the loop thread writes inline.
|
||||
Neither path raises into the caller even when the loop is closed.
|
||||
2. Best-effort / never-raises — repo.record raises → no exception escapes,
|
||||
no event emitted; fire_event raises → no exception escapes, entry still
|
||||
persisted (order: persist THEN emit).
|
||||
3. Actor resolution — defaults "system"; ContextVar wins when set;
|
||||
explicit actor= overrides ContextVar; no cross-context leakage (fresh
|
||||
ContextVar copy does not see a previous set).
|
||||
4. enabled flag — disabled → NO-OP (nothing persisted, no event);
|
||||
_bypass_enabled=True → still records despite enabled=False.
|
||||
5. entry_to_dict / payload — exactly 11 keys; ts is ISO-8601 string;
|
||||
metadata is a real dict; activity_logged payload shape is frozen.
|
||||
6. Retention engine — update_settings persists and round-trips;
|
||||
_prune calls repo.prune with correct before_ts / max_entries;
|
||||
disabling records exactly one disable event BEFORE recording stops;
|
||||
subsequent normal record after disable is a no-op;
|
||||
start() then stop() cancels task cleanly;
|
||||
stop() without prior start() is safe.
|
||||
7. Lazy loop capture — recorder built before the loop is running still
|
||||
works after the loop starts (no explicit loop= argument).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextvars
|
||||
import threading
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
|
||||
from ledgrab.core.activity_log.context import current_actor
|
||||
from ledgrab.core.activity_log.recorder import (
|
||||
ActivityRecorder,
|
||||
entry_to_dict,
|
||||
get_module_recorder,
|
||||
set_module_recorder,
|
||||
)
|
||||
from ledgrab.core.activity_log.retention import (
|
||||
DEFAULT_SETTINGS,
|
||||
ActivityLogRetentionEngine,
|
||||
)
|
||||
from ledgrab.storage.activity_log import (
|
||||
ActivityCategory,
|
||||
ActivityLogEntry,
|
||||
ActivitySeverity,
|
||||
)
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shared helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_recorder(
|
||||
*,
|
||||
fail_repo: bool = False,
|
||||
fail_pm: bool = False,
|
||||
loop: asyncio.AbstractEventLoop | None = None,
|
||||
) -> tuple[ActivityRecorder, list[ActivityLogEntry], list[dict]]:
|
||||
"""Return (recorder, persisted_entries, fired_events) — pure mocks."""
|
||||
repo = MagicMock()
|
||||
persisted: list[ActivityLogEntry] = []
|
||||
|
||||
if fail_repo:
|
||||
repo.record.side_effect = RuntimeError("DB exploded")
|
||||
else:
|
||||
repo.record.side_effect = lambda entry: persisted.append(entry)
|
||||
|
||||
pm = MagicMock()
|
||||
fired: list[dict] = []
|
||||
|
||||
if fail_pm:
|
||||
pm.fire_event.side_effect = RuntimeError("PM exploded")
|
||||
else:
|
||||
pm.fire_event.side_effect = lambda evt: fired.append(evt)
|
||||
|
||||
recorder = ActivityRecorder(repo, pm, loop=loop)
|
||||
return recorder, persisted, fired
|
||||
|
||||
|
||||
def _make_entry(action: str = "test.action") -> ActivityLogEntry:
|
||||
return ActivityLogEntry(
|
||||
id="al_" + uuid.uuid4().hex[:8],
|
||||
ts=datetime.now(timezone.utc),
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action=action,
|
||||
severity=ActivitySeverity.INFO,
|
||||
actor="system",
|
||||
message="test",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Thread-safety / marshaling
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_cross_thread_write_goes_via_call_soon_threadsafe():
|
||||
"""record() from a non-loop thread must marshal onto the loop, not write inline."""
|
||||
|
||||
async def _run():
|
||||
loop = asyncio.get_running_loop()
|
||||
recorder, persisted, fired = _make_recorder(loop=loop)
|
||||
|
||||
# Ensure the recorder knows the loop.
|
||||
recorder.ensure_loop()
|
||||
|
||||
thread_saw_persisted_count: list[int] = []
|
||||
|
||||
def _thread_body():
|
||||
# Nothing should be persisted synchronously in this thread —
|
||||
# the write must be deferred to the loop via call_soon_threadsafe.
|
||||
recorder.record(
|
||||
category=ActivityCategory.DEVICE,
|
||||
action="device.discovered",
|
||||
message="zeroconf found it",
|
||||
)
|
||||
# Capture how many entries are persisted synchronously right after the call.
|
||||
thread_saw_persisted_count.append(len(persisted))
|
||||
|
||||
t = threading.Thread(target=_thread_body)
|
||||
t.start()
|
||||
t.join(timeout=2.0)
|
||||
assert not t.is_alive(), "thread did not finish in time"
|
||||
|
||||
# The thread's synchronous view must see 0 — the actual write is deferred.
|
||||
assert thread_saw_persisted_count[0] == 0, (
|
||||
"write was not deferred: record() persisted synchronously on the "
|
||||
"calling thread instead of marshaling to the loop"
|
||||
)
|
||||
|
||||
# Give the loop a tick to drain the call_soon_threadsafe callback.
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
assert len(persisted) == 1, f"entry not persisted after loop tick: {persisted}"
|
||||
assert persisted[0].action == "device.discovered"
|
||||
assert len(fired) == 1
|
||||
assert fired[0]["type"] == "activity_logged"
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
def test_loop_thread_write_is_inline():
|
||||
"""record() called from the loop thread writes without call_soon_threadsafe."""
|
||||
|
||||
call_soon_threadsafe_calls: list = []
|
||||
|
||||
async def _run():
|
||||
loop = asyncio.get_running_loop()
|
||||
# Monkeypatch call_soon_threadsafe to detect if it is used.
|
||||
original_csst = loop.call_soon_threadsafe
|
||||
loop.call_soon_threadsafe = lambda *a, **kw: call_soon_threadsafe_calls.append(a) or original_csst(*a, **kw) # type: ignore[method-assign]
|
||||
try:
|
||||
recorder, persisted, fired = _make_recorder(loop=loop)
|
||||
recorder.ensure_loop()
|
||||
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="system.startup",
|
||||
message="loop thread write",
|
||||
)
|
||||
# No yield — synchronous within the coroutine.
|
||||
assert len(persisted) == 1, "inline write did not happen synchronously"
|
||||
assert call_soon_threadsafe_calls == [], (
|
||||
"call_soon_threadsafe was invoked from the loop thread — "
|
||||
"should write inline instead"
|
||||
)
|
||||
finally:
|
||||
loop.call_soon_threadsafe = original_csst # type: ignore[method-assign]
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
def test_cross_thread_closed_loop_does_not_raise():
|
||||
"""record() from a thread after the loop closes must log a warning, not raise."""
|
||||
loop = asyncio.new_event_loop()
|
||||
recorder, persisted, fired = _make_recorder(loop=loop)
|
||||
# Close the loop immediately — simulate a test teardown race.
|
||||
loop.close()
|
||||
|
||||
# This must not raise.
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="closed.loop.test",
|
||||
message="should not raise",
|
||||
)
|
||||
# The entry may or may not be persisted (the closed-loop path drops it),
|
||||
# but the important contract is: no exception propagates.
|
||||
|
||||
|
||||
def test_no_loop_falls_back_to_synchronous_write():
|
||||
"""record() with no loop and no running loop writes synchronously."""
|
||||
recorder, persisted, fired = _make_recorder(loop=None)
|
||||
# No loop is running here (plain synchronous test).
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="no.loop.write",
|
||||
message="synchronous fallback",
|
||||
)
|
||||
assert len(persisted) == 1
|
||||
assert len(fired) == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Best-effort / never-raises
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_repo_failure_does_not_raise_and_suppresses_fire_event():
|
||||
"""repo.record raises → no exception propagates AND fire_event is NOT called."""
|
||||
recorder, persisted, fired = _make_recorder(fail_repo=True)
|
||||
# Must not raise.
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="a", message="m")
|
||||
# No event must be emitted for an entry that failed to persist.
|
||||
assert fired == [], (
|
||||
"fire_event was called even though repo.record failed — "
|
||||
"the event would reference an entry that was never stored"
|
||||
)
|
||||
|
||||
|
||||
def test_fire_event_failure_does_not_raise_and_entry_is_persisted():
|
||||
"""fire_event raises → no exception propagates AND the entry IS persisted.
|
||||
|
||||
Order: persist THEN emit. An emit failure must not roll back the persist.
|
||||
"""
|
||||
recorder, persisted, fired = _make_recorder(fail_pm=True)
|
||||
# Must not raise.
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="b", message="m")
|
||||
# Entry must have been persisted before the emit attempt.
|
||||
assert (
|
||||
len(persisted) == 1
|
||||
), "entry was not persisted — fire_event failure retroactively erased it"
|
||||
|
||||
|
||||
def test_both_failures_do_not_raise():
|
||||
"""Even when both repo AND fire_event raise, record() must not propagate."""
|
||||
recorder, persisted, fired = _make_recorder(fail_repo=True, fail_pm=True)
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="c", message="m")
|
||||
# Nothing persisted, nothing fired — but absolutely no exception.
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Actor resolution / ContextVar isolation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_actor_defaults_to_system_when_contextvar_unset():
|
||||
"""When no actor arg and ContextVar at default, actor must be 'system'."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
# Make sure the ContextVar is at its default.
|
||||
token = current_actor.set("system")
|
||||
current_actor.reset(token)
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="a", message="m")
|
||||
assert persisted[0].actor == "system"
|
||||
|
||||
|
||||
def test_explicit_actor_overrides_contextvar():
|
||||
"""Explicit actor= argument wins over the ContextVar value."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
token = current_actor.set("api_user")
|
||||
try:
|
||||
recorder.record(
|
||||
category=ActivityCategory.AUTH,
|
||||
action="auth.login",
|
||||
message="explicit",
|
||||
actor="override_actor",
|
||||
)
|
||||
finally:
|
||||
current_actor.reset(token)
|
||||
assert (
|
||||
persisted[0].actor == "override_actor"
|
||||
), "explicit actor= was ignored; ContextVar won over explicit arg"
|
||||
|
||||
|
||||
def test_contextvar_value_used_when_set():
|
||||
"""When current_actor ContextVar is set, recorder picks it up."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
token = current_actor.set("mobile_client")
|
||||
try:
|
||||
recorder.record(category=ActivityCategory.ENTITY, action="e.created", message="m")
|
||||
finally:
|
||||
current_actor.reset(token)
|
||||
assert persisted[0].actor == "mobile_client"
|
||||
|
||||
|
||||
def test_no_cross_context_leakage_via_copy_context():
|
||||
"""ContextVar set in one context does not bleed into an independent copy."""
|
||||
# Simulate request-1 setting the actor.
|
||||
ctx_req1 = contextvars.copy_context()
|
||||
|
||||
def _request1():
|
||||
current_actor.set("user_alice")
|
||||
|
||||
ctx_req1.run(_request1)
|
||||
|
||||
# Simulate request-2 in a fresh copy — must not see user_alice.
|
||||
ctx_req2 = contextvars.copy_context()
|
||||
|
||||
def _request2():
|
||||
return current_actor.get()
|
||||
|
||||
actor_in_req2 = ctx_req2.run(_request2)
|
||||
assert actor_in_req2 == "system", (
|
||||
f"Cross-context leakage: request-2 saw actor '{actor_in_req2}' "
|
||||
"from request-1 instead of the default 'system'"
|
||||
)
|
||||
|
||||
|
||||
def test_no_cross_request_leakage_sequential_records():
|
||||
"""Two sequential simulated requests must not share ContextVar state."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
# Request 1: set actor, record, reset.
|
||||
token1 = current_actor.set("admin_user")
|
||||
try:
|
||||
recorder.record(category=ActivityCategory.AUTH, action="login", message="req1")
|
||||
finally:
|
||||
current_actor.reset(token1)
|
||||
|
||||
# Request 2: no actor set — must fall back to "system".
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="heartbeat", message="req2")
|
||||
|
||||
assert persisted[0].actor == "admin_user"
|
||||
assert persisted[1].actor == "system", (
|
||||
f"Request-2 actor was '{persisted[1].actor}' instead of 'system' — "
|
||||
"ContextVar from request-1 leaked into request-2"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. enabled flag
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_disabled_record_is_noop_nothing_persisted():
|
||||
"""enabled=False → record() returns immediately; nothing persisted."""
|
||||
recorder, persisted, fired = _make_recorder()
|
||||
recorder.enabled = False
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="a", message="m")
|
||||
assert persisted == [], "disabled recorder should not persist"
|
||||
assert fired == [], "disabled recorder should not fire events"
|
||||
|
||||
|
||||
def test_disabled_record_bypass_enabled_still_records():
|
||||
"""_bypass_enabled=True bypasses the enabled=False guard."""
|
||||
recorder, persisted, fired = _make_recorder()
|
||||
recorder.enabled = False
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="audit_log.disabled",
|
||||
message="final entry before disable",
|
||||
_bypass_enabled=True,
|
||||
)
|
||||
assert len(persisted) == 1, "_bypass_enabled=True should still persist"
|
||||
assert len(fired) == 1, "_bypass_enabled=True should still fire the event"
|
||||
|
||||
|
||||
def test_bypass_enabled_false_with_enabled_true_is_normal_record():
|
||||
"""_bypass_enabled=False and enabled=True → normal record (regression guard)."""
|
||||
recorder, persisted, fired = _make_recorder()
|
||||
recorder.enabled = True
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="normal",
|
||||
message="m",
|
||||
_bypass_enabled=False,
|
||||
)
|
||||
assert len(persisted) == 1
|
||||
assert len(fired) == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. entry_to_dict / payload shape
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_EXPECTED_KEYS = frozenset(
|
||||
{
|
||||
"id",
|
||||
"ts",
|
||||
"category",
|
||||
"action",
|
||||
"severity",
|
||||
"actor",
|
||||
"entity_type",
|
||||
"entity_id",
|
||||
"entity_name",
|
||||
"message",
|
||||
"metadata",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_entry_to_dict_has_exactly_11_keys():
|
||||
"""entry_to_dict must return a dict with exactly the 11 frozen keys."""
|
||||
d = entry_to_dict(_make_entry())
|
||||
assert set(d.keys()) == _EXPECTED_KEYS, (
|
||||
f"Key mismatch.\n Missing: {_EXPECTED_KEYS - set(d.keys())}\n"
|
||||
f" Extra: {set(d.keys()) - _EXPECTED_KEYS}"
|
||||
)
|
||||
|
||||
|
||||
def test_entry_to_dict_ts_is_iso8601_string():
|
||||
"""ts must be an ISO-8601 string, not a datetime object."""
|
||||
entry = _make_entry()
|
||||
d = entry_to_dict(entry)
|
||||
assert isinstance(d["ts"], str), f"ts is {type(d['ts'])}, expected str"
|
||||
# Must round-trip through fromisoformat without error.
|
||||
parsed = datetime.fromisoformat(d["ts"])
|
||||
assert parsed.tzinfo is not None, "ts ISO string has no timezone info"
|
||||
|
||||
|
||||
def test_entry_to_dict_metadata_is_real_dict_not_json_string():
|
||||
"""metadata must be a dict, not a JSON-encoded string."""
|
||||
entry = ActivityLogEntry(
|
||||
id="al_aabbccdd",
|
||||
ts=datetime.now(timezone.utc),
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="test",
|
||||
severity=ActivitySeverity.INFO,
|
||||
actor="system",
|
||||
message="hello",
|
||||
metadata={"nested": {"key": 42}, "list": [1, 2, 3]},
|
||||
)
|
||||
d = entry_to_dict(entry)
|
||||
assert isinstance(
|
||||
d["metadata"], dict
|
||||
), f"metadata is {type(d['metadata'])}, expected dict (not a JSON string)"
|
||||
assert d["metadata"] == {"nested": {"key": 42}, "list": [1, 2, 3]}
|
||||
|
||||
|
||||
def test_activity_logged_event_payload_shape():
|
||||
"""The fired event must match the frozen payload shape from the handoff doc."""
|
||||
recorder, persisted, fired = _make_recorder()
|
||||
recorder.record(
|
||||
category=ActivityCategory.AUTH,
|
||||
action="auth.login",
|
||||
severity=ActivitySeverity.WARNING,
|
||||
actor="api_user",
|
||||
entity_type="session",
|
||||
entity_id="sess_001",
|
||||
entity_name="admin session",
|
||||
message="user signed in",
|
||||
metadata={"ip": "127.0.0.1"},
|
||||
)
|
||||
assert len(fired) == 1
|
||||
evt = fired[0]
|
||||
assert evt.get("type") == "activity_logged", f"event type wrong: {evt.get('type')!r}"
|
||||
entry_dict = evt.get("entry")
|
||||
assert isinstance(entry_dict, dict), "event 'entry' must be a dict"
|
||||
assert set(entry_dict.keys()) == _EXPECTED_KEYS
|
||||
assert entry_dict["actor"] == "api_user"
|
||||
assert entry_dict["entity_type"] == "session"
|
||||
assert entry_dict["metadata"] == {"ip": "127.0.0.1"}
|
||||
assert isinstance(entry_dict["metadata"], dict), "metadata in event must be a dict"
|
||||
|
||||
|
||||
def test_entry_id_format():
|
||||
"""Entry IDs must be 'al_' followed by 8 hex characters."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="a", message="m")
|
||||
entry_id = persisted[0].id
|
||||
assert entry_id.startswith("al_"), f"id does not start with 'al_': {entry_id!r}"
|
||||
suffix = entry_id[3:]
|
||||
assert len(suffix) == 8, f"id suffix length is {len(suffix)}, expected 8: {entry_id!r}"
|
||||
assert all(c in "0123456789abcdef" for c in suffix), f"id suffix is not hex: {suffix!r}"
|
||||
|
||||
|
||||
def test_entry_ts_is_utc():
|
||||
"""Recorded entry ts must be timezone-aware UTC."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="a", message="m")
|
||||
ts = persisted[0].ts
|
||||
assert ts.tzinfo is not None, "ts has no timezone info"
|
||||
# Must be within 5 seconds of now (sanity check).
|
||||
delta = abs((datetime.now(timezone.utc) - ts).total_seconds())
|
||||
assert delta < 5, f"ts is {delta:.1f}s away from now — suspiciously stale"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. Retention engine
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_retention_update_settings_persists_to_db(tmp_db):
|
||||
"""update_settings must persist to DB so a fresh engine reload picks it up."""
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
asyncio.run(engine.update_settings(enabled=True, max_days=14, max_entries=500))
|
||||
|
||||
# Reload from DB — simulates server restart.
|
||||
engine2 = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
s = engine2.get_settings()
|
||||
assert s["max_days"] == 14, f"max_days not persisted: {s}"
|
||||
assert s["max_entries"] == 500, f"max_entries not persisted: {s}"
|
||||
assert s["enabled"] is True
|
||||
|
||||
|
||||
def test_retention_prune_passes_correct_before_ts(tmp_db):
|
||||
"""_prune must call repo.prune with before_ts ≈ now - max_days."""
|
||||
repo = MagicMock()
|
||||
repo.prune.return_value = 0
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
# Patch settings directly (don't go through update_settings to avoid task side-effects).
|
||||
engine._settings = {"enabled": True, "max_days": 30, "max_entries": 0}
|
||||
|
||||
before = datetime.now(timezone.utc)
|
||||
engine._prune()
|
||||
after = datetime.now(timezone.utc)
|
||||
|
||||
repo.prune.assert_called_once()
|
||||
kwargs = repo.prune.call_args.kwargs
|
||||
before_ts = kwargs.get("before_ts")
|
||||
assert before_ts is not None, "_prune called repo.prune without before_ts"
|
||||
expected_min = before - timedelta(days=30, seconds=1)
|
||||
expected_max = after - timedelta(days=30) + timedelta(seconds=1)
|
||||
assert expected_min <= before_ts <= expected_max, (
|
||||
f"before_ts {before_ts} is not near now - 30 days "
|
||||
f"(expected [{expected_min}, {expected_max}])"
|
||||
)
|
||||
|
||||
|
||||
def test_retention_prune_passes_max_entries(tmp_db):
|
||||
"""_prune must forward max_entries from settings to repo.prune."""
|
||||
repo = MagicMock()
|
||||
repo.prune.return_value = 0
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
engine._settings = {"enabled": True, "max_days": 0, "max_entries": 9999}
|
||||
engine._prune()
|
||||
|
||||
kwargs = repo.prune.call_args.kwargs
|
||||
assert kwargs.get("max_entries") == 9999, f"max_entries not forwarded: {kwargs}"
|
||||
|
||||
|
||||
def test_retention_prune_zero_max_entries_means_no_count_cap(tmp_db):
|
||||
"""max_entries=0 should pass None (no count cap) to repo.prune."""
|
||||
repo = MagicMock()
|
||||
repo.prune.return_value = 0
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
engine._settings = {"enabled": True, "max_days": 0, "max_entries": 0}
|
||||
engine._prune()
|
||||
|
||||
kwargs = repo.prune.call_args.kwargs
|
||||
assert (
|
||||
kwargs.get("max_entries") is None
|
||||
), f"max_entries=0 should map to None (no cap), got {kwargs.get('max_entries')!r}"
|
||||
|
||||
|
||||
def test_retention_disable_records_exactly_one_disable_event_with_bypass(tmp_db):
|
||||
"""Disabling via update_settings records exactly one 'audit_log.disabled' event
|
||||
with _bypass_enabled=True BEFORE the enabled flag is set to False."""
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
# Use a real recorder backed by repo to verify ordering.
|
||||
pm = MagicMock()
|
||||
pm.fire_event.return_value = None
|
||||
real_recorder = ActivityRecorder(repo, pm)
|
||||
real_recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=real_recorder)
|
||||
|
||||
# Disable — should record the event before flipping the flag.
|
||||
asyncio.run(engine.update_settings(enabled=False, max_days=90, max_entries=20000))
|
||||
|
||||
# The disable event must be in the DB (persisted before the flag flipped).
|
||||
entries = repo.query(
|
||||
__import__(
|
||||
"ledgrab.storage.activity_log",
|
||||
fromlist=["ActivityLogFilters"],
|
||||
).ActivityLogFilters()
|
||||
)
|
||||
disable_events = [e for e in entries if e.action == "audit_log.disabled"]
|
||||
assert len(disable_events) == 1, f"Expected exactly 1 disable event, got {len(disable_events)}"
|
||||
|
||||
# After disabling, a normal record must be a no-op.
|
||||
real_recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="should.not.appear",
|
||||
message="this should be dropped",
|
||||
)
|
||||
entries_after = repo.query(
|
||||
__import__(
|
||||
"ledgrab.storage.activity_log",
|
||||
fromlist=["ActivityLogFilters"],
|
||||
).ActivityLogFilters()
|
||||
)
|
||||
post_disable_actions = [e.action for e in entries_after if e.action != "audit_log.disabled"]
|
||||
assert post_disable_actions == [], f"Entries appeared after disable: {post_disable_actions}"
|
||||
|
||||
|
||||
def test_retention_disable_does_not_record_event_when_already_disabled(tmp_db):
|
||||
"""update_settings(enabled=False) when already disabled must NOT record a second event."""
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
pm = MagicMock()
|
||||
pm.fire_event.return_value = None
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
|
||||
# First disable.
|
||||
asyncio.run(engine.update_settings(enabled=False, max_days=90, max_entries=20000))
|
||||
first_count = recorder.record.call_count
|
||||
|
||||
# Second disable — must NOT record another event.
|
||||
asyncio.run(engine.update_settings(enabled=False, max_days=90, max_entries=20000))
|
||||
assert (
|
||||
recorder.record.call_count == first_count
|
||||
), "A second disable call recorded an extra event when already disabled"
|
||||
|
||||
|
||||
async def test_retention_start_stop_cancels_task_cleanly(tmp_db):
|
||||
"""start() then stop() must cancel the task and leave _task=None.
|
||||
|
||||
After stop(), the task has been requested to cancel and _task is None.
|
||||
The cancellation may still be 'in-flight' on the event loop (status:
|
||||
'cancelling') until the next tick; we yield once to let the event loop
|
||||
process the CancelledError and confirm task.done() is True.
|
||||
"""
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
|
||||
await engine.start()
|
||||
task = engine._task
|
||||
assert task is not None, "start() did not create a task"
|
||||
assert not task.done(), "task completed immediately — should be sleeping"
|
||||
|
||||
await engine.stop()
|
||||
# _task cleared immediately.
|
||||
assert engine._task is None, "_task was not cleared to None after stop()"
|
||||
# Give the event loop one tick to process the CancelledError.
|
||||
await asyncio.sleep(0)
|
||||
assert task.done(), (
|
||||
"task is still running after stop() + one event loop tick — "
|
||||
"stop() did not cancel the task"
|
||||
)
|
||||
|
||||
|
||||
async def test_retention_stop_without_start_is_safe(tmp_db):
|
||||
"""stop() without a prior start() must not raise."""
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
assert engine._task is None
|
||||
|
||||
# Must not raise.
|
||||
await engine.stop()
|
||||
assert engine._task is None
|
||||
|
||||
|
||||
async def test_retention_start_disabled_no_task(tmp_db):
|
||||
"""start() when enabled=False must not create a task."""
|
||||
tmp_db.set_setting("activity_log", {**DEFAULT_SETTINGS, "enabled": False})
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = False
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
await engine.start()
|
||||
assert engine._task is None, "task was created even though engine is disabled"
|
||||
await engine.stop()
|
||||
|
||||
|
||||
async def test_retention_max_days_boundary(tmp_db):
|
||||
"""max_days ≤ 0 should not pass a before_ts to repo.prune (no age cap)."""
|
||||
repo = MagicMock()
|
||||
repo.prune.return_value = 0
|
||||
recorder = MagicMock(spec=ActivityRecorder)
|
||||
recorder.enabled = True
|
||||
|
||||
engine = ActivityLogRetentionEngine(repo=repo, db=tmp_db, recorder=recorder)
|
||||
engine._settings = {"enabled": True, "max_days": 0, "max_entries": 0}
|
||||
engine._prune()
|
||||
|
||||
kwargs = repo.prune.call_args.kwargs
|
||||
assert (
|
||||
kwargs.get("before_ts") is None
|
||||
), f"max_days=0 should map to before_ts=None (no age cap), got {kwargs.get('before_ts')!r}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. Lazy loop capture (recorder built before loop is running)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_lazy_loop_capture_without_explicit_loop():
|
||||
"""Recorder with loop=None still works when record() is called from a running loop."""
|
||||
|
||||
async def _run():
|
||||
# Build the recorder BEFORE passing the loop explicitly — test lazy capture.
|
||||
recorder, persisted, fired = _make_recorder(loop=None)
|
||||
# Do NOT call ensure_loop() explicitly — the lazy path in record() must handle it.
|
||||
recorder.record(
|
||||
category=ActivityCategory.SYSTEM,
|
||||
action="lazy.capture",
|
||||
message="loop captured lazily",
|
||||
)
|
||||
assert (
|
||||
len(persisted) == 1
|
||||
), "Lazy loop capture failed: entry not persisted when loop=None at construction"
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
def test_lazy_loop_stores_loop_for_subsequent_calls():
|
||||
"""After the first call from an async context, _loop is populated."""
|
||||
|
||||
async def _run():
|
||||
recorder, persisted, fired = _make_recorder(loop=None)
|
||||
assert recorder._loop is None
|
||||
|
||||
recorder.record(category=ActivityCategory.SYSTEM, action="first", message="m")
|
||||
|
||||
# The loop must have been captured.
|
||||
assert (
|
||||
recorder._loop is not None
|
||||
), "recorder._loop was not set after first record() from async context"
|
||||
assert recorder._loop is asyncio.get_running_loop()
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 8. Module-level singleton accessor
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_set_and_get_module_recorder():
|
||||
"""set_module_recorder / get_module_recorder round-trip."""
|
||||
original = get_module_recorder()
|
||||
try:
|
||||
recorder, _, _ = _make_recorder()
|
||||
set_module_recorder(recorder)
|
||||
assert get_module_recorder() is recorder
|
||||
finally:
|
||||
# Restore whatever was there before (may be None in test isolation).
|
||||
import ledgrab.core.activity_log.recorder as _mod
|
||||
|
||||
_mod._recorder = original # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def test_get_module_recorder_returns_none_before_set():
|
||||
"""get_module_recorder() returns None when not yet initialised."""
|
||||
import ledgrab.core.activity_log.recorder as _mod
|
||||
|
||||
original = _mod._recorder # type: ignore[attr-defined]
|
||||
_mod._recorder = None # type: ignore[attr-defined]
|
||||
try:
|
||||
assert get_module_recorder() is None
|
||||
finally:
|
||||
_mod._recorder = original # type: ignore[attr-defined]
|
||||
@@ -0,0 +1,884 @@
|
||||
"""Adversarial / edge-case tests for ActivityLogRepository (Phase 1 — storage layer).
|
||||
|
||||
These tests are intentionally skeptical — they derive expected behaviour from the
|
||||
acceptance criteria in plans/activity-log/phase-1-storage.md, NOT from what the
|
||||
code happens to do today. If a test fails, it is a real bug.
|
||||
|
||||
Coverage areas
|
||||
--------------
|
||||
1. SQL-injection / parameterization safety (message_like with %, _, ;, --, quotes, etc.)
|
||||
2. Keyset pagination edge cases (empty table, before_seq bounds, stability,
|
||||
ordering contract, no duplicates/gaps)
|
||||
3. Prune edge cases (before_ts only, max_entries only, both,
|
||||
max_entries=0, larger than count, deleted count,
|
||||
keeps NEWEST entries)
|
||||
4. Filter combination edge cases (AND semantics, empty sequence vs None,
|
||||
since/until inclusive bounds, tz-aware datetimes)
|
||||
5. Codec / data integrity (metadata round-trip: nested, unicode, JSON-escape;
|
||||
entity_* None vs empty string; microsecond ts)
|
||||
6. Migration idempotency (table + all indexes present; double-run is no-op)
|
||||
7. iter_export vs query consistency (same filters yield same rows; empty table; filter)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from ledgrab.storage.activity_log import (
|
||||
ActivityCategory,
|
||||
ActivityLogEntry,
|
||||
ActivityLogFilters,
|
||||
ActivitySeverity,
|
||||
)
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
from ledgrab.storage.database import Database
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers (mirror the implementer's helpers so tests are self-contained)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _now() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
_SENTINEL = object()
|
||||
|
||||
|
||||
def _entry(
|
||||
*,
|
||||
id: str | None = None,
|
||||
ts: datetime | None = None,
|
||||
category: str = ActivityCategory.ENTITY,
|
||||
action: str = "entity.created",
|
||||
severity: str = ActivitySeverity.INFO,
|
||||
actor: str = "test_actor",
|
||||
entity_type: str | None = "output_target",
|
||||
entity_id: object = _SENTINEL,
|
||||
entity_name: str | None = "My Target",
|
||||
message: str = "Created output target",
|
||||
metadata: dict | None = None,
|
||||
) -> ActivityLogEntry:
|
||||
resolved_entity_id: str | None = (
|
||||
f"ot_{uuid.uuid4().hex[:8]}" if entity_id is _SENTINEL else entity_id # type: ignore[assignment]
|
||||
)
|
||||
return ActivityLogEntry(
|
||||
id=id or f"al_{uuid.uuid4().hex[:8]}",
|
||||
ts=ts or _now(),
|
||||
category=category,
|
||||
action=action,
|
||||
severity=severity,
|
||||
actor=actor,
|
||||
entity_type=entity_type,
|
||||
entity_id=resolved_entity_id,
|
||||
entity_name=entity_name,
|
||||
message=message,
|
||||
metadata=metadata if metadata is not None else {},
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def repo(tmp_db: Database) -> ActivityLogRepository:
|
||||
"""Fresh ActivityLogRepository backed by a temp database."""
|
||||
return ActivityLogRepository(tmp_db)
|
||||
|
||||
|
||||
def _get_seq(repo: ActivityLogRepository, entry_id: str) -> int:
|
||||
cursor = repo._db.execute("SELECT seq FROM activity_log WHERE id = ?", (entry_id,))
|
||||
row = cursor.fetchone()
|
||||
assert row is not None, f"No row found for id={entry_id!r}"
|
||||
return int(row["seq"])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. SQL-injection / parameterization safety
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSQLInjectionSafety:
|
||||
"""All user-supplied filter values must be treated as literal text, not SQL."""
|
||||
|
||||
def test_message_like_percent_is_literal(self, repo: ActivityLogRepository) -> None:
|
||||
"""A literal '%' in message_like must NOT act as a LIKE wildcard."""
|
||||
repo.record(_entry(message="100% done"))
|
||||
repo.record(_entry(message="all done"))
|
||||
repo.record(_entry(message="percent sign here"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(message_like="100%"), limit=10)
|
||||
assert len(results) == 1, "% in message_like should be a literal percent, not a wildcard"
|
||||
assert results[0].message == "100% done"
|
||||
|
||||
def test_message_like_underscore_is_literal(self, repo: ActivityLogRepository) -> None:
|
||||
"""A literal '_' in message_like must NOT act as a single-char wildcard."""
|
||||
repo.record(_entry(message="device_01"))
|
||||
repo.record(_entry(message="device001")) # would match if _ were a wildcard
|
||||
repo.record(_entry(message="some other message"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(message_like="device_01"), limit=10)
|
||||
assert (
|
||||
len(results) == 1
|
||||
), "_ in message_like should be a literal underscore, not a single-char wildcard"
|
||||
assert results[0].message == "device_01"
|
||||
|
||||
def test_message_like_single_quote_does_not_break_query(
|
||||
self, repo: ActivityLogRepository
|
||||
) -> None:
|
||||
"""A single quote in message_like must not cause a SQL syntax error."""
|
||||
repo.record(_entry(message="it's working"))
|
||||
repo.record(_entry(message="no quote here"))
|
||||
|
||||
# Must not raise
|
||||
results = repo.query(ActivityLogFilters(message_like="it's"), limit=10)
|
||||
assert len(results) == 1
|
||||
assert results[0].message == "it's working"
|
||||
|
||||
def test_message_like_semicolon_does_not_execute_second_statement(
|
||||
self, repo: ActivityLogRepository
|
||||
) -> None:
|
||||
"""';' in message_like must not let a second SQL statement execute."""
|
||||
repo.record(_entry(message="a; DROP TABLE activity_log; --"))
|
||||
repo.record(_entry(message="safe message"))
|
||||
|
||||
# If injection succeeded, table would be dropped and next call would error
|
||||
results = repo.query(
|
||||
ActivityLogFilters(message_like="a; DROP TABLE activity_log; --"), limit=10
|
||||
)
|
||||
# Table must still exist
|
||||
assert repo.count() == 2
|
||||
assert len(results) == 1
|
||||
|
||||
def test_message_like_sql_comment_sequence(self, repo: ActivityLogRepository) -> None:
|
||||
"""'--' (SQL comment) in message_like must be treated literally."""
|
||||
repo.record(_entry(message="value -- comment"))
|
||||
repo.record(_entry(message="value no comment"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(message_like="value --"), limit=10)
|
||||
assert len(results) == 1
|
||||
assert results[0].message == "value -- comment"
|
||||
|
||||
def test_message_like_backslash_literal(self, repo: ActivityLogRepository) -> None:
|
||||
"""Backslash in message_like must be treated as a literal character."""
|
||||
repo.record(_entry(message="path\\to\\file"))
|
||||
repo.record(_entry(message="path/to/file"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(message_like="path\\to"), limit=10)
|
||||
assert len(results) == 1
|
||||
assert results[0].message == "path\\to\\file"
|
||||
|
||||
def test_message_like_classic_injection_pattern(self, repo: ActivityLogRepository) -> None:
|
||||
"""Classic ') OR '1'='1 injection attempt must return no false positives."""
|
||||
repo.record(_entry(message="innocent message"))
|
||||
repo.record(_entry(message="another message"))
|
||||
|
||||
# If injection worked, all rows would match
|
||||
results = repo.query(ActivityLogFilters(message_like="') OR '1'='1"), limit=10)
|
||||
assert (
|
||||
len(results) == 0
|
||||
), "Injection payload matched rows it shouldn't — parameterization may be broken"
|
||||
|
||||
def test_message_like_all_wildcards_returns_nothing_for_no_match(
|
||||
self, repo: ActivityLogRepository
|
||||
) -> None:
|
||||
"""'%_%' as a literal search term should return no rows unless that exact
|
||||
substring appears in a message."""
|
||||
repo.record(_entry(message="some message"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(message_like="%_%"), limit=10)
|
||||
# '%_%' as literal text does not appear in "some message"
|
||||
assert (
|
||||
len(results) == 0
|
||||
), "% and _ in message_like were treated as SQL wildcards instead of literals"
|
||||
|
||||
def test_actor_exact_match_not_like(self, repo: ActivityLogRepository) -> None:
|
||||
"""actor filter is exact match — SQL wildcards in value must not act as wildcards."""
|
||||
repo.record(_entry(actor="alice"))
|
||||
repo.record(_entry(actor="alice_admin"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(actor="alice"), limit=10)
|
||||
assert (
|
||||
len(results) == 1
|
||||
), "actor filter is exact-match; 'alice' should not match 'alice_admin'"
|
||||
assert results[0].actor == "alice"
|
||||
|
||||
def test_entity_id_exact_match_not_like(self, repo: ActivityLogRepository) -> None:
|
||||
"""entity_id filter is exact match — prefix should not leak."""
|
||||
repo.record(_entry(entity_id="ot_abc"))
|
||||
repo.record(_entry(entity_id="ot_abc_extra"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(entity_id="ot_abc"), limit=10)
|
||||
assert len(results) == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Keyset pagination edge cases
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestKeysetPaginationEdges:
|
||||
def test_empty_table_returns_empty_list(self, repo: ActivityLogRepository) -> None:
|
||||
"""Query on empty table must return [] not raise."""
|
||||
results = repo.query(ActivityLogFilters(), limit=10)
|
||||
assert results == []
|
||||
|
||||
def test_before_seq_none_is_first_page(self, repo: ActivityLogRepository) -> None:
|
||||
"""before_seq=None must return the newest (first) page."""
|
||||
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
|
||||
for i in range(5):
|
||||
repo.record(_entry(ts=base + timedelta(seconds=i), message=f"e{i}"))
|
||||
|
||||
page = repo.query(ActivityLogFilters(), before_seq=None, limit=3)
|
||||
assert len(page) == 3
|
||||
# Page should contain the 3 newest entries
|
||||
messages = {e.message for e in page}
|
||||
assert "e4" in messages
|
||||
assert "e3" in messages
|
||||
assert "e2" in messages
|
||||
|
||||
def test_before_seq_smaller_than_all_rows_returns_empty(
|
||||
self, repo: ActivityLogRepository
|
||||
) -> None:
|
||||
"""before_seq=1 (smaller than all autoincrement seqs) returns empty page."""
|
||||
for i in range(5):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
# seq starts at 1, so before_seq=1 means seq < 1 — no rows
|
||||
results = repo.query(ActivityLogFilters(), before_seq=1, limit=10)
|
||||
assert (
|
||||
results == []
|
||||
), "before_seq=1 should yield empty page since autoincrement starts at 1 (seq<1 = nothing)"
|
||||
|
||||
def test_before_seq_larger_than_max_returns_full_first_page(
|
||||
self, repo: ActivityLogRepository
|
||||
) -> None:
|
||||
"""before_seq larger than any seq in the table behaves like before_seq=None."""
|
||||
for i in range(5):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
page_none = repo.query(ActivityLogFilters(), before_seq=None, limit=5)
|
||||
page_large = repo.query(ActivityLogFilters(), before_seq=999_999, limit=5)
|
||||
|
||||
ids_none = {e.id for e in page_none}
|
||||
ids_large = {e.id for e in page_large}
|
||||
assert ids_none == ids_large
|
||||
|
||||
def test_page_boundary_limit_equals_row_count(self, repo: ActivityLogRepository) -> None:
|
||||
"""When limit == total rows, one page covers all rows and a second page is empty."""
|
||||
for i in range(5):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
page1 = repo.query(ActivityLogFilters(), limit=5)
|
||||
assert len(page1) == 5
|
||||
|
||||
first_seq = _get_seq(repo, page1[0].id)
|
||||
page2 = repo.query(ActivityLogFilters(), before_seq=first_seq, limit=5)
|
||||
assert page2 == []
|
||||
|
||||
def test_ordering_contract_page_zero_is_smallest_seq(self, repo: ActivityLogRepository) -> None:
|
||||
"""Within a page, page[0] must have the smallest seq (ascending chrono order).
|
||||
The acceptance criteria state: 'The smallest seq on a page is page[0]'s seq —
|
||||
pass that as before_seq for the next page.'"""
|
||||
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
|
||||
for i in range(6):
|
||||
repo.record(_entry(ts=base + timedelta(seconds=i), message=f"e{i}"))
|
||||
|
||||
page = repo.query(ActivityLogFilters(), limit=6)
|
||||
seqs = [_get_seq(repo, e.id) for e in page]
|
||||
assert seqs == sorted(
|
||||
seqs
|
||||
), "page must be in ascending seq order (page[0] is oldest/smallest seq)"
|
||||
|
||||
def test_no_duplicates_across_full_walk(self, repo: ActivityLogRepository) -> None:
|
||||
"""Walking the entire table page by page yields each row exactly once."""
|
||||
total = 11
|
||||
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
|
||||
for i in range(total):
|
||||
repo.record(_entry(ts=base + timedelta(seconds=i), message=f"e{i}"))
|
||||
|
||||
all_ids: list[str] = []
|
||||
before_seq: int | None = None
|
||||
limit = 4
|
||||
|
||||
while True:
|
||||
page = repo.query(ActivityLogFilters(), before_seq=before_seq, limit=limit)
|
||||
if not page:
|
||||
break
|
||||
all_ids.extend(e.id for e in page)
|
||||
before_seq = _get_seq(repo, page[0].id)
|
||||
|
||||
assert len(all_ids) == total, "Total rows from all pages must equal inserted count"
|
||||
assert len(set(all_ids)) == total, "No duplicate IDs across pages"
|
||||
|
||||
def test_no_gaps_across_full_walk(self, repo: ActivityLogRepository) -> None:
|
||||
"""Walking the entire table page by page with limit=1 yields every row."""
|
||||
total = 7
|
||||
for i in range(total):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
all_ids: list[str] = []
|
||||
before_seq: int | None = None
|
||||
while True:
|
||||
page = repo.query(ActivityLogFilters(), before_seq=before_seq, limit=1)
|
||||
if not page:
|
||||
break
|
||||
all_ids.append(page[0].id)
|
||||
before_seq = _get_seq(repo, page[0].id)
|
||||
|
||||
assert len(all_ids) == total
|
||||
|
||||
def test_many_rows_same_ts_no_duplicates_or_gaps(self, repo: ActivityLogRepository) -> None:
|
||||
"""With many identical timestamps, pagination via seq prevents any dup or gap."""
|
||||
same_ts = datetime(2026, 5, 1, 10, 0, 0, tzinfo=timezone.utc)
|
||||
count = 9
|
||||
for i in range(count):
|
||||
repo.record(_entry(ts=same_ts, message=f"same-ts {i}"))
|
||||
|
||||
all_ids: list[str] = []
|
||||
before_seq: int | None = None
|
||||
limit = 4
|
||||
while True:
|
||||
page = repo.query(ActivityLogFilters(), before_seq=before_seq, limit=limit)
|
||||
if not page:
|
||||
break
|
||||
all_ids.extend(e.id for e in page)
|
||||
before_seq = _get_seq(repo, page[0].id)
|
||||
|
||||
assert len(all_ids) == count
|
||||
assert len(set(all_ids)) == count, "Duplicates found in same-ts pagination walk"
|
||||
|
||||
def test_next_page_cursor_is_page_zero_seq(self, repo: ActivityLogRepository) -> None:
|
||||
"""The documented contract: pass page[0].seq as before_seq for next page.
|
||||
Verify the next page does NOT overlap with the current page."""
|
||||
for i in range(6):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
page1 = repo.query(ActivityLogFilters(), limit=3)
|
||||
cursor = _get_seq(repo, page1[0].id) # page[0] = smallest seq on page
|
||||
page2 = repo.query(ActivityLogFilters(), before_seq=cursor, limit=3)
|
||||
|
||||
ids1 = {e.id for e in page1}
|
||||
ids2 = {e.id for e in page2}
|
||||
assert ids1.isdisjoint(ids2), "Pages overlap — cursor contract broken"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Prune edge cases
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPruneEdgeCases:
|
||||
def test_prune_before_ts_only_no_max_entries(self, repo: ActivityLogRepository) -> None:
|
||||
"""before_ts alone removes only old rows; recent rows untouched."""
|
||||
cutoff = datetime(2026, 3, 1, tzinfo=timezone.utc)
|
||||
repo.record(_entry(ts=cutoff - timedelta(days=2), message="old"))
|
||||
repo.record(_entry(ts=cutoff + timedelta(days=1), message="new"))
|
||||
|
||||
deleted = repo.prune(before_ts=cutoff)
|
||||
assert deleted == 1
|
||||
remaining = repo.query(ActivityLogFilters(), limit=10)
|
||||
assert len(remaining) == 1
|
||||
assert remaining[0].message == "new"
|
||||
|
||||
def test_prune_max_entries_only_no_before_ts(self, repo: ActivityLogRepository) -> None:
|
||||
"""max_entries alone trims to N newest; no age filter applied."""
|
||||
for i in range(6):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
deleted = repo.prune(max_entries=2)
|
||||
assert deleted == 4
|
||||
assert repo.count() == 2
|
||||
|
||||
def test_prune_max_entries_zero_deletes_all(self, repo: ActivityLogRepository) -> None:
|
||||
"""max_entries=0 means keep nothing — all rows deleted."""
|
||||
for i in range(5):
|
||||
repo.record(_entry())
|
||||
|
||||
deleted = repo.prune(max_entries=0)
|
||||
assert deleted == 5
|
||||
assert repo.count() == 0
|
||||
|
||||
def test_prune_max_entries_larger_than_count_is_noop(self, repo: ActivityLogRepository) -> None:
|
||||
"""max_entries > actual count must not delete anything."""
|
||||
for i in range(3):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
deleted = repo.prune(max_entries=100)
|
||||
assert deleted == 0
|
||||
assert repo.count() == 3
|
||||
|
||||
def test_prune_keeps_newest_entries_by_seq(self, repo: ActivityLogRepository) -> None:
|
||||
"""max_entries prune MUST keep the rows with the HIGHEST seq values."""
|
||||
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
|
||||
all_ids = []
|
||||
for i in range(6):
|
||||
e = _entry(ts=base + timedelta(seconds=i), message=f"e{i}")
|
||||
all_ids.append(e.id)
|
||||
repo.record(e)
|
||||
|
||||
# keep only 2
|
||||
repo.prune(max_entries=2)
|
||||
remaining = repo.query(ActivityLogFilters(), limit=10)
|
||||
remaining_ids = {r.id for r in remaining}
|
||||
|
||||
# Must keep the last two inserted (highest seq = newest)
|
||||
assert all_ids[-1] in remaining_ids, "Newest entry (e5) must be kept"
|
||||
assert all_ids[-2] in remaining_ids, "Second newest entry (e4) must be kept"
|
||||
# Oldest must be gone
|
||||
assert all_ids[0] not in remaining_ids, "Oldest entry (e0) must be pruned"
|
||||
|
||||
def test_prune_both_returns_sum_of_deleted(self, repo: ActivityLogRepository) -> None:
|
||||
"""prune(before_ts, max_entries) returns the TOTAL rows deleted by both steps."""
|
||||
base = datetime(2026, 4, 1, tzinfo=timezone.utc)
|
||||
# 4 old entries (before base)
|
||||
for i in range(4):
|
||||
repo.record(_entry(ts=base - timedelta(hours=i + 1), message=f"old{i}"))
|
||||
# 4 new entries (after base)
|
||||
for i in range(4):
|
||||
repo.record(_entry(ts=base + timedelta(hours=i + 1), message=f"new{i}"))
|
||||
|
||||
# prune old, then keep only 2 new
|
||||
deleted = repo.prune(before_ts=base, max_entries=2)
|
||||
# 4 old + 2 of the 4 new = 6 total
|
||||
assert deleted == 6
|
||||
assert repo.count() == 2
|
||||
|
||||
def test_prune_no_args_is_noop(self, repo: ActivityLogRepository) -> None:
|
||||
"""prune() with no args should delete 0 rows."""
|
||||
for i in range(3):
|
||||
repo.record(_entry())
|
||||
|
||||
deleted = repo.prune()
|
||||
assert deleted == 0
|
||||
assert repo.count() == 3
|
||||
|
||||
def test_prune_before_ts_boundary_is_exclusive(self, repo: ActivityLogRepository) -> None:
|
||||
"""prune(before_ts=X) uses strict < X; a row exactly at X must survive."""
|
||||
ts = datetime(2026, 5, 1, 12, 0, 0, tzinfo=timezone.utc)
|
||||
repo.record(_entry(ts=ts - timedelta(seconds=1), message="before"))
|
||||
repo.record(_entry(ts=ts, message="exact boundary"))
|
||||
repo.record(_entry(ts=ts + timedelta(seconds=1), message="after"))
|
||||
|
||||
deleted = repo.prune(before_ts=ts)
|
||||
assert deleted == 1 # only "before" deleted
|
||||
remaining = {r.message for r in repo.query(ActivityLogFilters(), limit=10)}
|
||||
assert "exact boundary" in remaining
|
||||
assert "before" not in remaining
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Filter combination edge cases
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFilterCombinationEdges:
|
||||
def test_multiple_filters_are_anded(self, repo: ActivityLogRepository) -> None:
|
||||
"""All non-None filters must be AND-ed together, not OR-ed."""
|
||||
repo.record(
|
||||
_entry(actor="alice", category=ActivityCategory.AUTH, severity=ActivitySeverity.ERROR)
|
||||
)
|
||||
repo.record(
|
||||
_entry(actor="alice", category=ActivityCategory.DEVICE, severity=ActivitySeverity.INFO)
|
||||
)
|
||||
repo.record(
|
||||
_entry(actor="bob", category=ActivityCategory.AUTH, severity=ActivitySeverity.ERROR)
|
||||
)
|
||||
|
||||
results = repo.query(
|
||||
ActivityLogFilters(
|
||||
actor="alice",
|
||||
categories=[ActivityCategory.AUTH],
|
||||
severities=[ActivitySeverity.ERROR],
|
||||
),
|
||||
limit=10,
|
||||
)
|
||||
assert len(results) == 1
|
||||
r = results[0]
|
||||
assert r.actor == "alice"
|
||||
assert r.category == ActivityCategory.AUTH
|
||||
assert r.severity == ActivitySeverity.ERROR
|
||||
|
||||
def test_empty_categories_sequence_means_no_restriction(
|
||||
self, repo: ActivityLogRepository
|
||||
) -> None:
|
||||
"""An empty list for categories must behave the same as None (no restriction).
|
||||
The acceptance criteria state empty sequence == None for this dimension."""
|
||||
repo.record(_entry(category=ActivityCategory.AUTH))
|
||||
repo.record(_entry(category=ActivityCategory.DEVICE))
|
||||
|
||||
# empty list
|
||||
results_empty = repo.query(ActivityLogFilters(categories=[]), limit=10)
|
||||
# None
|
||||
results_none = repo.query(ActivityLogFilters(categories=None), limit=10)
|
||||
|
||||
assert len(results_empty) == len(results_none), (
|
||||
"categories=[] and categories=None should behave identically (no restriction); "
|
||||
f"got {len(results_empty)} vs {len(results_none)}"
|
||||
)
|
||||
|
||||
def test_empty_severities_sequence_means_no_restriction(
|
||||
self, repo: ActivityLogRepository
|
||||
) -> None:
|
||||
"""An empty list for severities must behave the same as None (no restriction)."""
|
||||
repo.record(_entry(severity=ActivitySeverity.INFO))
|
||||
repo.record(_entry(severity=ActivitySeverity.ERROR))
|
||||
|
||||
results_empty = repo.query(ActivityLogFilters(severities=[]), limit=10)
|
||||
results_none = repo.query(ActivityLogFilters(severities=None), limit=10)
|
||||
|
||||
assert len(results_empty) == len(
|
||||
results_none
|
||||
), "severities=[] and severities=None should behave identically"
|
||||
|
||||
def test_since_is_inclusive(self, repo: ActivityLogRepository) -> None:
|
||||
"""since is an INCLUSIVE lower bound: ts >= since."""
|
||||
ts = datetime(2026, 6, 1, 10, 0, 0, tzinfo=timezone.utc)
|
||||
repo.record(_entry(ts=ts - timedelta(seconds=1), message="before"))
|
||||
repo.record(_entry(ts=ts, message="at boundary"))
|
||||
repo.record(_entry(ts=ts + timedelta(seconds=1), message="after"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(since=ts), limit=10)
|
||||
messages = {r.message for r in results}
|
||||
assert "at boundary" in messages, "since boundary row (ts == since) must be included"
|
||||
assert "before" not in messages
|
||||
|
||||
def test_until_is_inclusive(self, repo: ActivityLogRepository) -> None:
|
||||
"""until is an INCLUSIVE upper bound: ts <= until."""
|
||||
ts = datetime(2026, 6, 1, 10, 0, 0, tzinfo=timezone.utc)
|
||||
repo.record(_entry(ts=ts - timedelta(seconds=1), message="before"))
|
||||
repo.record(_entry(ts=ts, message="at boundary"))
|
||||
repo.record(_entry(ts=ts + timedelta(seconds=1), message="after"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(until=ts), limit=10)
|
||||
messages = {r.message for r in results}
|
||||
assert "at boundary" in messages, "until boundary row (ts == until) must be included"
|
||||
assert "after" not in messages
|
||||
|
||||
def test_since_and_until_define_closed_range(self, repo: ActivityLogRepository) -> None:
|
||||
"""Combining since + until must keep rows in [since, until] inclusive."""
|
||||
base = datetime(2026, 6, 1, 12, 0, 0, tzinfo=timezone.utc)
|
||||
repo.record(_entry(ts=base - timedelta(hours=1), message="out_before"))
|
||||
repo.record(_entry(ts=base, message="in_start"))
|
||||
repo.record(_entry(ts=base + timedelta(hours=1), message="in_middle"))
|
||||
repo.record(_entry(ts=base + timedelta(hours=2), message="in_end"))
|
||||
repo.record(_entry(ts=base + timedelta(hours=3), message="out_after"))
|
||||
|
||||
results = repo.query(
|
||||
ActivityLogFilters(since=base, until=base + timedelta(hours=2)),
|
||||
limit=10,
|
||||
)
|
||||
messages = {r.message for r in results}
|
||||
assert {"in_start", "in_middle", "in_end"} == messages
|
||||
|
||||
def test_tz_aware_datetime_round_trip(self, repo: ActivityLogRepository) -> None:
|
||||
"""UTC-aware datetimes must survive storage and come back tz-aware."""
|
||||
ts = datetime(2026, 1, 15, 8, 30, 0, tzinfo=timezone.utc)
|
||||
e = _entry(ts=ts)
|
||||
repo.record(e)
|
||||
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.ts.tzinfo is not None, "Returned ts must be tz-aware"
|
||||
assert got.ts.utcoffset().total_seconds() == 0, "Returned ts must be UTC" # type: ignore[union-attr]
|
||||
assert got.ts == ts
|
||||
|
||||
def test_count_none_equals_count_empty_filters(self, repo: ActivityLogRepository) -> None:
|
||||
"""count(None) == count(ActivityLogFilters()) per acceptance criteria."""
|
||||
for i in range(4):
|
||||
repo.record(_entry())
|
||||
|
||||
assert repo.count(None) == repo.count(ActivityLogFilters())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. Codec / data integrity
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestCodecDataIntegrity:
|
||||
def test_metadata_nested_dict_round_trip(self, repo: ActivityLogRepository) -> None:
|
||||
"""Deeply nested metadata survives JSON round-trip."""
|
||||
meta = {
|
||||
"level1": {
|
||||
"level2": {"level3": [1, 2, 3]},
|
||||
"list": [{"a": True}, {"b": None}],
|
||||
},
|
||||
"count": 42,
|
||||
}
|
||||
e = _entry(metadata=meta)
|
||||
repo.record(e)
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.metadata == meta
|
||||
|
||||
def test_metadata_unicode_round_trip(self, repo: ActivityLogRepository) -> None:
|
||||
"""Unicode (including emoji and CJK) in metadata survives storage."""
|
||||
meta = {"label": "こんにちは", "emoji": "🎉", "arrow": "→"}
|
||||
e = _entry(metadata=meta)
|
||||
repo.record(e)
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.metadata == meta
|
||||
|
||||
def test_metadata_empty_dict_round_trip(self, repo: ActivityLogRepository) -> None:
|
||||
"""An empty {} metadata must come back as {} not None."""
|
||||
e = _entry(metadata={})
|
||||
repo.record(e)
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.metadata == {}
|
||||
assert isinstance(got.metadata, dict)
|
||||
|
||||
def test_metadata_json_special_chars(self, repo: ActivityLogRepository) -> None:
|
||||
"""Metadata with JSON-special characters (backslash, quotes) round-trips correctly."""
|
||||
meta = {"path": "C:\\Users\\test", "quoted": '"hello"', "newline": "line1\nline2"}
|
||||
e = _entry(metadata=meta)
|
||||
repo.record(e)
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.metadata == meta
|
||||
|
||||
def test_entity_type_none_vs_empty_string(self, repo: ActivityLogRepository) -> None:
|
||||
"""None entity_type must come back as None (not empty string '').
|
||||
These are semantically different — None means 'not applicable'."""
|
||||
e = _entry(entity_type=None, entity_id=None, entity_name=None)
|
||||
repo.record(e)
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
# Must be exactly None, not ""
|
||||
assert got.entity_type is None
|
||||
assert got.entity_id is None
|
||||
assert got.entity_name is None
|
||||
|
||||
def test_ts_microsecond_precision_preserved(self, repo: ActivityLogRepository) -> None:
|
||||
"""Microsecond component of ts must survive the isoformat() round-trip."""
|
||||
ts = datetime(2026, 6, 9, 12, 34, 56, 789012, tzinfo=timezone.utc)
|
||||
e = _entry(ts=ts)
|
||||
repo.record(e)
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.ts == ts, f"Expected {ts!r}, got {got.ts!r} — microsecond precision may be lost"
|
||||
|
||||
def test_to_row_does_not_include_seq(self) -> None:
|
||||
"""to_row() must NOT include 'seq' (it's DB-assigned)."""
|
||||
e = _entry()
|
||||
row = e.to_row()
|
||||
assert "seq" not in row, "to_row() must not include seq — it is DB-assigned"
|
||||
|
||||
def test_to_row_has_exactly_11_keys(self) -> None:
|
||||
"""Acceptance criteria: to_row() returns 11 keys."""
|
||||
e = _entry()
|
||||
row = e.to_row()
|
||||
expected_keys = {
|
||||
"id",
|
||||
"ts",
|
||||
"category",
|
||||
"action",
|
||||
"severity",
|
||||
"actor",
|
||||
"entity_type",
|
||||
"entity_id",
|
||||
"entity_name",
|
||||
"message",
|
||||
"metadata",
|
||||
}
|
||||
assert set(row.keys()) == expected_keys
|
||||
|
||||
def test_from_row_ignores_seq_column(self) -> None:
|
||||
"""from_row() must not raise or fail when 'seq' is present in the dict."""
|
||||
e = _entry()
|
||||
row = e.to_row()
|
||||
row["seq"] = 42 # inject seq as if from DB
|
||||
recovered = ActivityLogEntry.from_row(row)
|
||||
assert recovered.id == e.id
|
||||
|
||||
def test_from_row_naive_ts_becomes_utc_aware(self) -> None:
|
||||
"""If a stored ts has no timezone offset (legacy row), from_row must attach UTC."""
|
||||
e = _entry()
|
||||
row = e.to_row()
|
||||
# Strip timezone from the isoformat string to simulate a legacy row
|
||||
row["ts"] = datetime(2026, 1, 1, 10, 0, 0).isoformat() # naive
|
||||
recovered = ActivityLogEntry.from_row(row)
|
||||
assert recovered.ts.tzinfo is not None, "Legacy naive ts must become tz-aware (UTC)"
|
||||
|
||||
def test_metadata_with_numeric_keys_round_trip(self, repo: ActivityLogRepository) -> None:
|
||||
"""JSON only supports string keys; numeric keys are coerced to strings."""
|
||||
# This tests that the codec doesn't silently crash on non-string keys
|
||||
# (Python allows them but JSON does not — json.dumps coerces to string)
|
||||
meta = {1: "one", "two": 2}
|
||||
e = _entry(metadata=meta) # type: ignore[arg-type]
|
||||
repo.record(e)
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
# json.dumps coerces int key 1 → "1"
|
||||
assert "1" in got.metadata or 1 in got.metadata
|
||||
|
||||
def test_all_category_values_round_trip(self, repo: ActivityLogRepository) -> None:
|
||||
"""Every ActivityCategory constant must survive storage without corruption."""
|
||||
for cat in ActivityCategory.ALL:
|
||||
repo.record(_entry(category=cat, message=f"cat:{cat}"))
|
||||
|
||||
for cat in ActivityCategory.ALL:
|
||||
results = repo.query(ActivityLogFilters(categories=[cat]), limit=10)
|
||||
assert len(results) == 1
|
||||
assert results[0].category == cat
|
||||
|
||||
def test_all_severity_values_round_trip(self, repo: ActivityLogRepository) -> None:
|
||||
"""Every ActivitySeverity constant must survive storage without corruption."""
|
||||
for sev in ActivitySeverity.ALL:
|
||||
repo.record(_entry(severity=sev, message=f"sev:{sev}"))
|
||||
|
||||
for sev in ActivitySeverity.ALL:
|
||||
results = repo.query(ActivityLogFilters(severities=[sev]), limit=10)
|
||||
assert len(results) == 1
|
||||
assert results[0].severity == sev
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. Migration idempotency (additional structural checks)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMigrationIdempotencyExtended:
|
||||
def test_table_has_autoincrement_seq(self, tmp_db: Database) -> None:
|
||||
"""The seq column must be INTEGER PRIMARY KEY AUTOINCREMENT — never reuse deleted seqs."""
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
e1 = _entry(message="first")
|
||||
e2 = _entry(message="second")
|
||||
repo.record(e1)
|
||||
repo.record(e2)
|
||||
seq1 = _get_seq(repo, e1.id)
|
||||
seq2 = _get_seq(repo, e2.id)
|
||||
assert seq2 > seq1, "AUTOINCREMENT must produce monotonically increasing seqs"
|
||||
|
||||
# After clear, a new record must get a seq higher than the previous max
|
||||
repo.clear()
|
||||
e3 = _entry(message="third")
|
||||
repo.record(e3)
|
||||
seq3 = _get_seq(repo, e3.id)
|
||||
assert seq3 > seq2, "AUTOINCREMENT must not reuse seqs after DELETE"
|
||||
|
||||
def test_all_expected_indexes_present(self, tmp_db: Database) -> None:
|
||||
"""All 5 indexes declared in the acceptance criteria must exist."""
|
||||
ActivityLogRepository(tmp_db)
|
||||
|
||||
cursor = tmp_db.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='index' AND tbl_name='activity_log'"
|
||||
)
|
||||
index_names = {row["name"] for row in cursor.fetchall()}
|
||||
required = {
|
||||
"idx_activity_log_ts_seq",
|
||||
"idx_activity_log_category",
|
||||
"idx_activity_log_severity",
|
||||
"idx_activity_log_actor",
|
||||
"idx_activity_log_entity",
|
||||
}
|
||||
missing = required - index_names
|
||||
assert not missing, f"Missing indexes: {missing}"
|
||||
|
||||
def test_id_column_has_unique_constraint(self, tmp_db: Database) -> None:
|
||||
"""Inserting duplicate id must raise IntegrityError."""
|
||||
import sqlite3 as sqlite_module
|
||||
|
||||
repo = ActivityLogRepository(tmp_db)
|
||||
fixed_id = f"al_{uuid.uuid4().hex[:8]}"
|
||||
repo.record(_entry(id=fixed_id, message="first"))
|
||||
|
||||
with pytest.raises((Exception, sqlite_module.IntegrityError)):
|
||||
repo.record(_entry(id=fixed_id, message="duplicate id"))
|
||||
|
||||
def test_migration_name_is_002_add_activity_log(self, tmp_db: Database) -> None:
|
||||
"""The migration name must exactly match '002_add_activity_log'."""
|
||||
from ledgrab.storage.data_migrations import AddActivityLogTableMigration
|
||||
|
||||
migration = AddActivityLogTableMigration()
|
||||
assert migration.name == "002_add_activity_log"
|
||||
|
||||
def test_migration_is_second_in_all_migrations(self) -> None:
|
||||
"""AddActivityLogTableMigration must be at index [1] in ALL_MIGRATIONS."""
|
||||
from ledgrab.storage.data_migrations import (
|
||||
ALL_MIGRATIONS,
|
||||
AddActivityLogTableMigration,
|
||||
)
|
||||
|
||||
assert len(ALL_MIGRATIONS) >= 2, "ALL_MIGRATIONS must have at least 2 entries"
|
||||
assert isinstance(
|
||||
ALL_MIGRATIONS[1], AddActivityLogTableMigration
|
||||
), "AddActivityLogTableMigration must be the second migration (index 1)"
|
||||
|
||||
def test_apply_twice_is_noop_no_error(self, tmp_db: Database) -> None:
|
||||
"""Calling apply() on the connection twice must not raise — IF NOT EXISTS ensures this."""
|
||||
from ledgrab.storage.data_migrations import AddActivityLogTableMigration
|
||||
|
||||
migration = AddActivityLogTableMigration()
|
||||
with tmp_db.transaction() as conn:
|
||||
migration.apply(conn)
|
||||
# Second apply — must not raise
|
||||
with tmp_db.transaction() as conn:
|
||||
migration.apply(conn)
|
||||
|
||||
# Table should still be accessible
|
||||
cursor = tmp_db.execute("SELECT COUNT(*) AS cnt FROM activity_log")
|
||||
assert cursor.fetchone()["cnt"] == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. iter_export vs query consistency
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestIterExportConsistency:
|
||||
def test_iter_export_empty_table_yields_nothing(self, repo: ActivityLogRepository) -> None:
|
||||
"""iter_export on empty table must yield nothing, not raise."""
|
||||
exported = list(repo.iter_export())
|
||||
assert exported == []
|
||||
|
||||
def test_iter_export_matches_query_results(self, repo: ActivityLogRepository) -> None:
|
||||
"""iter_export(filters) and query(filters) must return the same entries."""
|
||||
for i in range(8):
|
||||
cat = ActivityCategory.AUTH if i % 2 == 0 else ActivityCategory.DEVICE
|
||||
repo.record(_entry(category=cat, message=f"e{i}"))
|
||||
|
||||
filters = ActivityLogFilters(categories=[ActivityCategory.AUTH])
|
||||
|
||||
exported_ids = {e.id for e in repo.iter_export(filters)}
|
||||
queried_ids = {e.id for e in repo.query(filters, limit=100)}
|
||||
assert (
|
||||
exported_ids == queried_ids
|
||||
), "iter_export and query must return the same set of entries for the same filters"
|
||||
|
||||
def test_iter_export_none_filter_yields_all(self, repo: ActivityLogRepository) -> None:
|
||||
"""iter_export(None) must yield all rows (same as query with no filter)."""
|
||||
for i in range(5):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
all_exported = list(repo.iter_export(None))
|
||||
all_queried = repo.query(ActivityLogFilters(), limit=100)
|
||||
|
||||
assert len(all_exported) == len(all_queried) == 5
|
||||
|
||||
def test_iter_export_ascending_seq_order(self, repo: ActivityLogRepository) -> None:
|
||||
"""iter_export must yield rows in ascending seq order (oldest first)."""
|
||||
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
|
||||
for i in range(5):
|
||||
repo.record(_entry(ts=base + timedelta(seconds=i), message=f"e{i}"))
|
||||
|
||||
exported = list(repo.iter_export())
|
||||
seqs = [_get_seq(repo, e.id) for e in exported]
|
||||
assert seqs == sorted(seqs), "iter_export must yield rows in ascending seq order"
|
||||
|
||||
def test_iter_export_respects_message_like_filter(self, repo: ActivityLogRepository) -> None:
|
||||
"""iter_export should honour message_like just as query does."""
|
||||
repo.record(_entry(message="found: hello world"))
|
||||
repo.record(_entry(message="nothing relevant here"))
|
||||
repo.record(_entry(message="also found: hello there"))
|
||||
|
||||
exported = list(repo.iter_export(ActivityLogFilters(message_like="found")))
|
||||
assert len(exported) == 2
|
||||
assert all("found" in e.message for e in exported)
|
||||
|
||||
def test_iter_export_is_lazy_generator(self, repo: ActivityLogRepository) -> None:
|
||||
"""iter_export must return a generator (lazy), not a list."""
|
||||
import types
|
||||
|
||||
for _ in range(3):
|
||||
repo.record(_entry())
|
||||
|
||||
result = repo.iter_export()
|
||||
assert isinstance(
|
||||
result, types.GeneratorType
|
||||
), "iter_export must return a generator for streaming — not a pre-loaded list"
|
||||
@@ -0,0 +1,609 @@
|
||||
"""Unit tests for ActivityLogRepository (Phase 1 — storage layer).
|
||||
|
||||
Coverage
|
||||
--------
|
||||
* round-trip: record + read back, including metadata JSON and UTC ts
|
||||
* filter by each dimension: category, severity, actor, entity_type/id, date range, message_like
|
||||
* keyset pagination stability with same-ts rows (seq tiebreaker)
|
||||
* prune by age (before_ts) and by max_entries
|
||||
* clear; count (filtered + unfiltered); export iterator
|
||||
* migration idempotency: constructing the repo twice does not re-run the migration
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from ledgrab.storage.activity_log import (
|
||||
ActivityCategory,
|
||||
ActivityLogEntry,
|
||||
ActivityLogFilters,
|
||||
ActivitySeverity,
|
||||
)
|
||||
from ledgrab.storage.activity_log_repository import ActivityLogRepository
|
||||
from ledgrab.storage.database import Database
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _now() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
_SENTINEL = object() # sentinel for "caller did not pass this kwarg"
|
||||
|
||||
|
||||
def _entry(
|
||||
*,
|
||||
id: str | None = None,
|
||||
ts: datetime | None = None,
|
||||
category: str = ActivityCategory.ENTITY,
|
||||
action: str = "entity.created",
|
||||
severity: str = ActivitySeverity.INFO,
|
||||
actor: str = "test_actor",
|
||||
entity_type: str | None = "output_target",
|
||||
entity_id: object = _SENTINEL,
|
||||
entity_name: str | None = "My Target",
|
||||
message: str = "Created output target",
|
||||
metadata: dict | None = None,
|
||||
) -> ActivityLogEntry:
|
||||
"""Build a test ``ActivityLogEntry``.
|
||||
|
||||
``entity_id`` defaults to a random id when not supplied at all. Pass
|
||||
``entity_id=None`` explicitly to get ``None`` stored in the entry.
|
||||
"""
|
||||
resolved_entity_id: str | None = (
|
||||
f"ot_{uuid.uuid4().hex[:8]}" if entity_id is _SENTINEL else entity_id # type: ignore[assignment]
|
||||
)
|
||||
return ActivityLogEntry(
|
||||
id=id or f"al_{uuid.uuid4().hex[:8]}",
|
||||
ts=ts or _now(),
|
||||
category=category,
|
||||
action=action,
|
||||
severity=severity,
|
||||
actor=actor,
|
||||
entity_type=entity_type,
|
||||
entity_id=resolved_entity_id,
|
||||
entity_name=entity_name,
|
||||
message=message,
|
||||
metadata=metadata if metadata is not None else {},
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def repo(tmp_db: Database) -> ActivityLogRepository:
|
||||
"""Fresh ActivityLogRepository backed by a temp database."""
|
||||
return ActivityLogRepository(tmp_db)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Round-trip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRoundTrip:
|
||||
def test_record_and_read_back(self, repo: ActivityLogRepository) -> None:
|
||||
e = _entry(message="Hello world", metadata={"key": "value", "n": 42})
|
||||
repo.record(e)
|
||||
|
||||
page = repo.query(ActivityLogFilters(), limit=10)
|
||||
assert len(page) == 1
|
||||
got = page[0]
|
||||
assert got.id == e.id
|
||||
assert got.category == e.category
|
||||
assert got.action == e.action
|
||||
assert got.severity == e.severity
|
||||
assert got.actor == e.actor
|
||||
assert got.entity_type == e.entity_type
|
||||
assert got.entity_id == e.entity_id
|
||||
assert got.entity_name == e.entity_name
|
||||
assert got.message == e.message
|
||||
|
||||
def test_metadata_json_round_trip(self, repo: ActivityLogRepository) -> None:
|
||||
meta = {"device": "wled_01", "led_count": 150, "nested": {"x": True}}
|
||||
e = _entry(metadata=meta)
|
||||
repo.record(e)
|
||||
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.metadata == meta
|
||||
|
||||
def test_utc_timestamp_preserved(self, repo: ActivityLogRepository) -> None:
|
||||
ts = datetime(2026, 1, 15, 12, 30, 45, tzinfo=timezone.utc)
|
||||
e = _entry(ts=ts)
|
||||
repo.record(e)
|
||||
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
# Should round-trip to the same UTC moment
|
||||
assert got.ts.replace(tzinfo=timezone.utc) == ts.replace(tzinfo=timezone.utc)
|
||||
|
||||
def test_none_optional_fields_preserved(self, repo: ActivityLogRepository) -> None:
|
||||
e = _entry(entity_type=None, entity_id=None, entity_name=None)
|
||||
repo.record(e)
|
||||
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.entity_type is None
|
||||
assert got.entity_id is None
|
||||
assert got.entity_name is None
|
||||
|
||||
def test_empty_metadata_default(self, repo: ActivityLogRepository) -> None:
|
||||
e = _entry(metadata={})
|
||||
repo.record(e)
|
||||
|
||||
got = repo.query(ActivityLogFilters(), limit=1)[0]
|
||||
assert got.metadata == {}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Filters
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFilters:
|
||||
def test_filter_by_category(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(category=ActivityCategory.AUTH, action="auth.rejected"))
|
||||
repo.record(_entry(category=ActivityCategory.DEVICE, action="device.connected"))
|
||||
repo.record(_entry(category=ActivityCategory.ENTITY, action="entity.deleted"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(categories=[ActivityCategory.AUTH]), limit=10)
|
||||
assert len(results) == 1
|
||||
assert results[0].category == ActivityCategory.AUTH
|
||||
|
||||
def test_filter_by_multiple_categories(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(category=ActivityCategory.AUTH))
|
||||
repo.record(_entry(category=ActivityCategory.DEVICE))
|
||||
repo.record(_entry(category=ActivityCategory.SYSTEM))
|
||||
|
||||
results = repo.query(
|
||||
ActivityLogFilters(categories=[ActivityCategory.AUTH, ActivityCategory.DEVICE]),
|
||||
limit=10,
|
||||
)
|
||||
assert len(results) == 2
|
||||
cats = {r.category for r in results}
|
||||
assert cats == {ActivityCategory.AUTH, ActivityCategory.DEVICE}
|
||||
|
||||
def test_filter_by_severity(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(severity=ActivitySeverity.INFO))
|
||||
repo.record(_entry(severity=ActivitySeverity.WARNING))
|
||||
repo.record(_entry(severity=ActivitySeverity.ERROR))
|
||||
|
||||
results = repo.query(ActivityLogFilters(severities=[ActivitySeverity.ERROR]), limit=10)
|
||||
assert len(results) == 1
|
||||
assert results[0].severity == ActivitySeverity.ERROR
|
||||
|
||||
def test_filter_by_multiple_severities(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(severity=ActivitySeverity.INFO))
|
||||
repo.record(_entry(severity=ActivitySeverity.WARNING))
|
||||
repo.record(_entry(severity=ActivitySeverity.ERROR))
|
||||
|
||||
results = repo.query(
|
||||
ActivityLogFilters(severities=[ActivitySeverity.WARNING, ActivitySeverity.ERROR]),
|
||||
limit=10,
|
||||
)
|
||||
assert len(results) == 2
|
||||
|
||||
def test_filter_by_actor(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(actor="alice"))
|
||||
repo.record(_entry(actor="bob"))
|
||||
repo.record(_entry(actor="alice"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(actor="alice"), limit=10)
|
||||
assert len(results) == 2
|
||||
assert all(r.actor == "alice" for r in results)
|
||||
|
||||
def test_filter_by_entity_type(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(entity_type="output_target"))
|
||||
repo.record(_entry(entity_type="device"))
|
||||
repo.record(_entry(entity_type="output_target"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(entity_type="output_target"), limit=10)
|
||||
assert len(results) == 2
|
||||
|
||||
def test_filter_by_entity_id(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(entity_id="ot_aabbccdd"))
|
||||
repo.record(_entry(entity_id="ot_11223344"))
|
||||
repo.record(_entry(entity_id="ot_aabbccdd"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(entity_id="ot_aabbccdd"), limit=10)
|
||||
assert len(results) == 2
|
||||
|
||||
def test_filter_by_since(self, repo: ActivityLogRepository) -> None:
|
||||
base = datetime(2026, 1, 1, 0, 0, 0, tzinfo=timezone.utc)
|
||||
repo.record(_entry(ts=base - timedelta(hours=2), message="old"))
|
||||
repo.record(_entry(ts=base, message="boundary"))
|
||||
repo.record(_entry(ts=base + timedelta(hours=1), message="new"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(since=base), limit=10)
|
||||
assert len(results) == 2
|
||||
messages = {r.message for r in results}
|
||||
assert "old" not in messages
|
||||
|
||||
def test_filter_by_until(self, repo: ActivityLogRepository) -> None:
|
||||
base = datetime(2026, 1, 1, 0, 0, 0, tzinfo=timezone.utc)
|
||||
repo.record(_entry(ts=base - timedelta(hours=1), message="old"))
|
||||
repo.record(_entry(ts=base, message="boundary"))
|
||||
repo.record(_entry(ts=base + timedelta(hours=2), message="new"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(until=base), limit=10)
|
||||
assert len(results) == 2
|
||||
messages = {r.message for r in results}
|
||||
assert "new" not in messages
|
||||
|
||||
def test_filter_message_like_substring(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(message="Created output target MyStrip"))
|
||||
repo.record(_entry(message="Deleted device sensor-01"))
|
||||
repo.record(_entry(message="Updated output target MyStrip"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(message_like="output target"), limit=10)
|
||||
assert len(results) == 2
|
||||
|
||||
def test_filter_message_like_escapes_percent(self, repo: ActivityLogRepository) -> None:
|
||||
"""A literal % in message_like should not act as a SQL wildcard."""
|
||||
repo.record(_entry(message="100% done"))
|
||||
repo.record(_entry(message="partial done"))
|
||||
|
||||
results = repo.query(ActivityLogFilters(message_like="100%"), limit=10)
|
||||
assert len(results) == 1
|
||||
assert results[0].message == "100% done"
|
||||
|
||||
def test_combined_filters(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(
|
||||
_entry(
|
||||
actor="alice",
|
||||
category=ActivityCategory.ENTITY,
|
||||
severity=ActivitySeverity.INFO,
|
||||
)
|
||||
)
|
||||
repo.record(
|
||||
_entry(
|
||||
actor="alice",
|
||||
category=ActivityCategory.AUTH,
|
||||
severity=ActivitySeverity.WARNING,
|
||||
)
|
||||
)
|
||||
repo.record(
|
||||
_entry(
|
||||
actor="bob",
|
||||
category=ActivityCategory.ENTITY,
|
||||
severity=ActivitySeverity.INFO,
|
||||
)
|
||||
)
|
||||
|
||||
results = repo.query(
|
||||
ActivityLogFilters(
|
||||
actor="alice",
|
||||
categories=[ActivityCategory.ENTITY],
|
||||
severities=[ActivitySeverity.INFO],
|
||||
),
|
||||
limit=10,
|
||||
)
|
||||
assert len(results) == 1
|
||||
assert results[0].actor == "alice"
|
||||
assert results[0].category == ActivityCategory.ENTITY
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Keyset pagination
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestKeysetPagination:
|
||||
def test_basic_pagination(self, repo: ActivityLogRepository) -> None:
|
||||
"""Records returned across two pages cover the full set without overlap."""
|
||||
for i in range(10):
|
||||
repo.record(_entry(message=f"entry {i}"))
|
||||
|
||||
page1 = repo.query(ActivityLogFilters(), limit=4)
|
||||
assert len(page1) == 4
|
||||
|
||||
# The last entry on page 1 has the smallest seq — use it as the cursor
|
||||
# We need the seq; query internally reverses, so page1[0] is oldest on page
|
||||
# and page1[-1] is newest on page. We need the min seq to paginate.
|
||||
# The repo returns entries in ascending order within a page, so page1[0]
|
||||
# has the smallest seq on the page.
|
||||
first_seq = self._get_seq(repo, page1[0].id)
|
||||
page2 = repo.query(ActivityLogFilters(), before_seq=first_seq, limit=4)
|
||||
assert len(page2) == 4
|
||||
|
||||
ids1 = {e.id for e in page1}
|
||||
ids2 = {e.id for e in page2}
|
||||
assert ids1.isdisjoint(ids2), "Pages must not overlap"
|
||||
|
||||
page3 = repo.query(
|
||||
ActivityLogFilters(),
|
||||
before_seq=self._get_seq(repo, page2[0].id),
|
||||
limit=4,
|
||||
)
|
||||
assert len(page3) == 2 # 10 total; 4 + 4 + 2
|
||||
|
||||
def test_same_ts_stability(self, repo: ActivityLogRepository) -> None:
|
||||
"""Rows with identical ts are ordered by seq, not ts — no duplicates across pages."""
|
||||
# Insert 6 rows all sharing the exact same timestamp
|
||||
same_ts = datetime(2026, 3, 10, 15, 0, 0, tzinfo=timezone.utc)
|
||||
entries = [_entry(ts=same_ts, message=f"same-ts {i}") for i in range(6)]
|
||||
for e in entries:
|
||||
repo.record(e)
|
||||
|
||||
page1 = repo.query(ActivityLogFilters(), limit=3)
|
||||
first_seq = self._get_seq(repo, page1[0].id)
|
||||
page2 = repo.query(ActivityLogFilters(), before_seq=first_seq, limit=3)
|
||||
|
||||
ids1 = {e.id for e in page1}
|
||||
ids2 = {e.id for e in page2}
|
||||
assert ids1.isdisjoint(ids2), "Same-ts rows leaked across page boundary"
|
||||
assert ids1 | ids2 == {e.id for e in entries}, "All rows covered exactly once"
|
||||
|
||||
def test_empty_page_at_end(self, repo: ActivityLogRepository) -> None:
|
||||
"""Requesting a page beyond the last entry returns an empty list."""
|
||||
for i in range(3):
|
||||
repo.record(_entry(message=f"e{i}"))
|
||||
|
||||
page1 = repo.query(ActivityLogFilters(), limit=3)
|
||||
assert len(page1) == 3
|
||||
first_seq = self._get_seq(repo, page1[0].id)
|
||||
page2 = repo.query(ActivityLogFilters(), before_seq=first_seq, limit=3)
|
||||
assert page2 == []
|
||||
|
||||
@staticmethod
|
||||
def _get_seq(repo: ActivityLogRepository, entry_id: str) -> int:
|
||||
"""Helper: retrieve the seq for an entry by its application id."""
|
||||
cursor = repo._db.execute("SELECT seq FROM activity_log WHERE id = ?", (entry_id,))
|
||||
row = cursor.fetchone()
|
||||
assert row is not None, f"No row found for id={entry_id!r}"
|
||||
return int(row["seq"])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Prune
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPrune:
|
||||
def test_prune_by_age(self, repo: ActivityLogRepository) -> None:
|
||||
base = datetime(2026, 2, 1, 12, 0, 0, tzinfo=timezone.utc)
|
||||
repo.record(_entry(ts=base - timedelta(days=10), message="old1"))
|
||||
repo.record(_entry(ts=base - timedelta(days=5), message="old2"))
|
||||
repo.record(_entry(ts=base, message="boundary"))
|
||||
repo.record(_entry(ts=base + timedelta(days=1), message="new"))
|
||||
|
||||
# Prune everything strictly older than base
|
||||
deleted = repo.prune(before_ts=base)
|
||||
assert deleted == 2
|
||||
remaining = repo.query(ActivityLogFilters(), limit=20)
|
||||
messages = {r.message for r in remaining}
|
||||
assert "old1" not in messages
|
||||
assert "old2" not in messages
|
||||
assert "boundary" in messages
|
||||
assert "new" in messages
|
||||
|
||||
def test_prune_by_max_entries(self, repo: ActivityLogRepository) -> None:
|
||||
for i in range(10):
|
||||
repo.record(_entry(message=f"entry {i}"))
|
||||
|
||||
deleted = repo.prune(max_entries=3)
|
||||
assert deleted == 7
|
||||
assert repo.count() == 3
|
||||
|
||||
def test_prune_keeps_newest_on_max_entries(self, repo: ActivityLogRepository) -> None:
|
||||
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
|
||||
ids = []
|
||||
for i in range(5):
|
||||
e = _entry(ts=base + timedelta(hours=i), message=f"entry {i}")
|
||||
ids.append(e.id)
|
||||
repo.record(e)
|
||||
|
||||
# Keep only the 2 newest
|
||||
repo.prune(max_entries=2)
|
||||
remaining = repo.query(ActivityLogFilters(), limit=10)
|
||||
remaining_ids = {r.id for r in remaining}
|
||||
# The 2 newest are the last 2 inserted (highest seq)
|
||||
assert ids[3] in remaining_ids
|
||||
assert ids[4] in remaining_ids
|
||||
assert ids[0] not in remaining_ids
|
||||
|
||||
def test_prune_both_predicates(self, repo: ActivityLogRepository) -> None:
|
||||
base = datetime(2026, 2, 1, 0, 0, 0, tzinfo=timezone.utc)
|
||||
# Insert 6 entries: 3 old, 3 recent
|
||||
for i in range(3):
|
||||
repo.record(_entry(ts=base - timedelta(days=i + 1), message=f"old{i}"))
|
||||
for i in range(3):
|
||||
repo.record(_entry(ts=base + timedelta(hours=i), message=f"new{i}"))
|
||||
|
||||
# Prune old entries AND keep at most 2 of the remaining
|
||||
deleted = repo.prune(before_ts=base, max_entries=2)
|
||||
# 3 age-pruned + 1 count-pruned = 4
|
||||
assert deleted == 4
|
||||
assert repo.count() == 2
|
||||
|
||||
def test_prune_max_entries_zero_clears_all(self, repo: ActivityLogRepository) -> None:
|
||||
for i in range(5):
|
||||
repo.record(_entry())
|
||||
|
||||
repo.prune(max_entries=0)
|
||||
assert repo.count() == 0
|
||||
|
||||
def test_prune_no_op_when_below_max(self, repo: ActivityLogRepository) -> None:
|
||||
for i in range(3):
|
||||
repo.record(_entry())
|
||||
|
||||
deleted = repo.prune(max_entries=10)
|
||||
assert deleted == 0
|
||||
assert repo.count() == 3
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Clear
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestClear:
|
||||
def test_clear_returns_row_count(self, repo: ActivityLogRepository) -> None:
|
||||
for _ in range(5):
|
||||
repo.record(_entry())
|
||||
|
||||
deleted = repo.clear()
|
||||
assert deleted == 5
|
||||
|
||||
def test_clear_empties_table(self, repo: ActivityLogRepository) -> None:
|
||||
for _ in range(3):
|
||||
repo.record(_entry())
|
||||
|
||||
repo.clear()
|
||||
assert repo.count() == 0
|
||||
|
||||
def test_clear_on_empty_table(self, repo: ActivityLogRepository) -> None:
|
||||
deleted = repo.clear()
|
||||
assert deleted == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Count
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestCount:
|
||||
def test_count_all(self, repo: ActivityLogRepository) -> None:
|
||||
for _ in range(7):
|
||||
repo.record(_entry())
|
||||
assert repo.count() == 7
|
||||
|
||||
def test_count_filtered(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(category=ActivityCategory.AUTH))
|
||||
repo.record(_entry(category=ActivityCategory.DEVICE))
|
||||
repo.record(_entry(category=ActivityCategory.AUTH))
|
||||
|
||||
n = repo.count(ActivityLogFilters(categories=[ActivityCategory.AUTH]))
|
||||
assert n == 2
|
||||
|
||||
def test_count_empty(self, repo: ActivityLogRepository) -> None:
|
||||
assert repo.count() == 0
|
||||
|
||||
def test_count_no_match(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(category=ActivityCategory.ENTITY))
|
||||
n = repo.count(ActivityLogFilters(categories=[ActivityCategory.AUTH]))
|
||||
assert n == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Export iterator
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestExportIterator:
|
||||
def test_iter_export_yields_all_rows(self, repo: ActivityLogRepository) -> None:
|
||||
entries = [_entry(message=f"e{i}") for i in range(5)]
|
||||
for e in entries:
|
||||
repo.record(e)
|
||||
|
||||
exported = list(repo.iter_export())
|
||||
assert len(exported) == 5
|
||||
exported_ids = {e.id for e in exported}
|
||||
assert exported_ids == {e.id for e in entries}
|
||||
|
||||
def test_iter_export_ascending_order(self, repo: ActivityLogRepository) -> None:
|
||||
base = datetime(2026, 4, 1, tzinfo=timezone.utc)
|
||||
for i in range(5):
|
||||
repo.record(_entry(ts=base + timedelta(seconds=i), message=f"e{i}"))
|
||||
|
||||
exported = list(repo.iter_export())
|
||||
seqs = [
|
||||
repo._db.execute("SELECT seq FROM activity_log WHERE id = ?", (e.id,)).fetchone()["seq"]
|
||||
for e in exported
|
||||
]
|
||||
assert seqs == sorted(seqs), "iter_export must yield rows in ascending seq order"
|
||||
|
||||
def test_iter_export_with_filter(self, repo: ActivityLogRepository) -> None:
|
||||
repo.record(_entry(category=ActivityCategory.AUTH))
|
||||
repo.record(_entry(category=ActivityCategory.ENTITY))
|
||||
repo.record(_entry(category=ActivityCategory.AUTH))
|
||||
|
||||
exported = list(repo.iter_export(ActivityLogFilters(categories=[ActivityCategory.AUTH])))
|
||||
assert len(exported) == 2
|
||||
assert all(e.category == ActivityCategory.AUTH for e in exported)
|
||||
|
||||
def test_iter_export_streaming_not_all_in_memory(self, repo: ActivityLogRepository) -> None:
|
||||
"""Verify iter_export is a generator (lazy), not a pre-loaded list."""
|
||||
import types
|
||||
|
||||
for _ in range(3):
|
||||
repo.record(_entry())
|
||||
|
||||
result = repo.iter_export()
|
||||
assert isinstance(result, types.GeneratorType)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Migration idempotency
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMigrationIdempotency:
|
||||
def test_construct_repo_twice_is_noop(self, tmp_db: Database) -> None:
|
||||
"""Creating two repos on the same DB does not re-run the migration."""
|
||||
repo1 = ActivityLogRepository(tmp_db)
|
||||
repo1.record(_entry(message="before second construction"))
|
||||
|
||||
# Second construction must not raise or re-apply the migration
|
||||
repo2 = ActivityLogRepository(tmp_db)
|
||||
assert repo2.count() == 1
|
||||
|
||||
# Confirm migration is recorded exactly once
|
||||
# Count how many times our migration name appears (should be 1)
|
||||
cursor = tmp_db.execute(
|
||||
"SELECT COUNT(*) AS cnt FROM data_migrations WHERE name = ?",
|
||||
("002_add_activity_log",),
|
||||
)
|
||||
assert cursor.fetchone()["cnt"] == 1
|
||||
|
||||
def test_running_migrations_twice_is_noop(self, tmp_db: Database) -> None:
|
||||
"""MigrationRunner.run is idempotent for AddActivityLogTableMigration."""
|
||||
from ledgrab.storage.data_migrations import (
|
||||
AddActivityLogTableMigration,
|
||||
MigrationRunner,
|
||||
)
|
||||
|
||||
runner = MigrationRunner(tmp_db)
|
||||
migration = AddActivityLogTableMigration()
|
||||
|
||||
first_run = runner.run([migration])
|
||||
assert len(first_run) == 1
|
||||
assert first_run[0].name == "002_add_activity_log"
|
||||
|
||||
second_run = runner.run([migration])
|
||||
assert second_run == [], "Second run must be a no-op"
|
||||
|
||||
def test_activity_log_table_exists_after_migration(self, tmp_db: Database) -> None:
|
||||
"""The activity_log table is present after the migration runs."""
|
||||
ActivityLogRepository(tmp_db)
|
||||
|
||||
cursor = tmp_db.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND name='activity_log'"
|
||||
)
|
||||
assert cursor.fetchone() is not None
|
||||
|
||||
def test_activity_log_indexes_exist_after_migration(self, tmp_db: Database) -> None:
|
||||
"""All declared indexes are present after migration."""
|
||||
ActivityLogRepository(tmp_db)
|
||||
|
||||
cursor = tmp_db.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='index' AND tbl_name='activity_log'"
|
||||
)
|
||||
index_names = {row["name"] for row in cursor.fetchall()}
|
||||
expected = {
|
||||
"idx_activity_log_ts_seq",
|
||||
"idx_activity_log_category",
|
||||
"idx_activity_log_severity",
|
||||
"idx_activity_log_actor",
|
||||
"idx_activity_log_entity",
|
||||
}
|
||||
assert expected.issubset(index_names)
|
||||
@@ -0,0 +1,614 @@
|
||||
"""Integration tests for Phase 3: Event instrumentation.
|
||||
|
||||
Coverage targets
|
||||
----------------
|
||||
- Entity create/update/delete emits a record with correct category/actor/name.
|
||||
- An entity DELETE carries the entity name (not None).
|
||||
- An auth failure emits a ``warning`` record; the attempted token NEVER appears
|
||||
in any recorded field.
|
||||
- A device health transition emits a record.
|
||||
- A device discovery event emits a record.
|
||||
- A capture start and a backup-create emit records.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from ledgrab.core.activity_log.context import current_actor
|
||||
from ledgrab.core.activity_log.recorder import ActivityRecorder
|
||||
from ledgrab.storage.activity_log import ActivityCategory, ActivitySeverity
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_recorder() -> tuple[ActivityRecorder, list, list]:
|
||||
"""Return (recorder, persisted_entries, fired_events)."""
|
||||
repo = MagicMock()
|
||||
persisted: list = []
|
||||
repo.record.side_effect = lambda entry: persisted.append(entry)
|
||||
|
||||
pm = MagicMock()
|
||||
fired: list[dict] = []
|
||||
pm.fire_event.side_effect = lambda evt: fired.append(evt)
|
||||
|
||||
recorder = ActivityRecorder(repo, pm)
|
||||
return recorder, persisted, fired
|
||||
|
||||
|
||||
def _patch_module_recorder(recorder: ActivityRecorder):
|
||||
"""Context manager: patch the module-level recorder used by all non-DI sites."""
|
||||
return patch(
|
||||
"ledgrab.core.activity_log.recorder._recorder",
|
||||
recorder,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Category A: Entity CRUD via fire_entity_event choke-point
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestEntityCrud:
|
||||
"""fire_entity_event records entity create/update/delete with correct fields."""
|
||||
|
||||
def test_entity_created_emits_record(self):
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.dependencies import _deps, fire_entity_event
|
||||
|
||||
# Minimal _deps so the store lookup returns None (name resolved as None)
|
||||
original_deps = dict(_deps)
|
||||
try:
|
||||
# Clear deps so store lookup path returns None
|
||||
_deps.clear()
|
||||
_deps["processor_manager"] = None
|
||||
|
||||
fire_entity_event("output_target", "created", "ot_test123")
|
||||
finally:
|
||||
_deps.clear()
|
||||
_deps.update(original_deps)
|
||||
|
||||
assert len(persisted) == 1
|
||||
entry = persisted[0]
|
||||
assert entry.category == ActivityCategory.ENTITY
|
||||
assert entry.action == "entity.created"
|
||||
assert entry.severity == ActivitySeverity.INFO
|
||||
assert entry.entity_type == "output_target"
|
||||
assert entry.entity_id == "ot_test123"
|
||||
|
||||
def test_entity_deleted_carries_name(self):
|
||||
"""DELETE: entity_name must be passed explicitly and preserved in record."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.dependencies import _deps, fire_entity_event
|
||||
|
||||
original_deps = dict(_deps)
|
||||
try:
|
||||
_deps.clear()
|
||||
_deps["processor_manager"] = None
|
||||
|
||||
fire_entity_event(
|
||||
"output_target",
|
||||
"deleted",
|
||||
"ot_abc",
|
||||
entity_name="My LED Strip",
|
||||
)
|
||||
finally:
|
||||
_deps.clear()
|
||||
_deps.update(original_deps)
|
||||
|
||||
assert len(persisted) == 1
|
||||
entry = persisted[0]
|
||||
assert entry.action == "entity.deleted"
|
||||
assert entry.entity_name == "My LED Strip"
|
||||
# Name should also appear in the human message.
|
||||
assert "My LED Strip" in entry.message
|
||||
|
||||
def test_entity_deleted_without_name_does_not_raise(self):
|
||||
"""Even if entity_name is omitted on delete, the record is created."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.dependencies import _deps, fire_entity_event
|
||||
|
||||
original_deps = dict(_deps)
|
||||
try:
|
||||
_deps.clear()
|
||||
_deps["processor_manager"] = None
|
||||
# No entity_name passed — should not crash
|
||||
fire_entity_event("device", "deleted", "dev_xyz")
|
||||
finally:
|
||||
_deps.clear()
|
||||
_deps.update(original_deps)
|
||||
|
||||
assert len(persisted) == 1
|
||||
assert persisted[0].action == "entity.deleted"
|
||||
|
||||
def test_entity_updated_emits_record(self):
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.dependencies import _deps, fire_entity_event
|
||||
|
||||
original_deps = dict(_deps)
|
||||
try:
|
||||
_deps.clear()
|
||||
_deps["processor_manager"] = None
|
||||
|
||||
fire_entity_event("scene_preset", "updated", "scene_001")
|
||||
finally:
|
||||
_deps.clear()
|
||||
_deps.update(original_deps)
|
||||
|
||||
assert len(persisted) == 1
|
||||
assert persisted[0].action == "entity.updated"
|
||||
|
||||
def test_actor_carried_from_contextvar(self):
|
||||
"""Actor is resolved from the ContextVar."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
token = current_actor.set("dev")
|
||||
try:
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.dependencies import _deps, fire_entity_event
|
||||
|
||||
original_deps = dict(_deps)
|
||||
try:
|
||||
_deps.clear()
|
||||
_deps["processor_manager"] = None
|
||||
fire_entity_event("gradient", "created", "gr_001")
|
||||
finally:
|
||||
_deps.clear()
|
||||
_deps.update(original_deps)
|
||||
finally:
|
||||
current_actor.reset(token)
|
||||
|
||||
assert persisted[0].actor == "dev"
|
||||
|
||||
def test_no_record_when_module_recorder_is_none(self):
|
||||
"""If recorder not initialised, fire_entity_event must not raise."""
|
||||
with patch("ledgrab.core.activity_log.recorder._recorder", None):
|
||||
from ledgrab.api.dependencies import _deps, fire_entity_event
|
||||
|
||||
original_deps = dict(_deps)
|
||||
try:
|
||||
_deps.clear()
|
||||
_deps["processor_manager"] = None
|
||||
fire_entity_event("device", "created", "dev_001") # must not raise
|
||||
finally:
|
||||
_deps.clear()
|
||||
_deps.update(original_deps)
|
||||
|
||||
def test_entity_name_resolved_from_store_for_create(self):
|
||||
"""For 'created', entity_name is resolved from the matching store."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
mock_store = MagicMock()
|
||||
mock_obj = MagicMock()
|
||||
mock_obj.name = "Target Alpha"
|
||||
mock_store.get_target.return_value = mock_obj
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.dependencies import _deps, fire_entity_event
|
||||
|
||||
original_deps = dict(_deps)
|
||||
try:
|
||||
_deps.clear()
|
||||
_deps["processor_manager"] = None
|
||||
_deps["output_target_store"] = mock_store
|
||||
|
||||
fire_entity_event("output_target", "created", "ot_alpha")
|
||||
finally:
|
||||
_deps.clear()
|
||||
_deps.update(original_deps)
|
||||
|
||||
assert len(persisted) == 1
|
||||
assert persisted[0].entity_name == "Target Alpha"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Category B: Authentication audit records
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestAuthInstrumentation:
|
||||
"""Auth failures emit warning records; no token ever recorded."""
|
||||
|
||||
_SECRET_TOKEN = "super-secret-token-that-must-never-appear"
|
||||
|
||||
def _make_mock_request(self, client_ip: str = "192.168.1.50") -> MagicMock:
|
||||
req = MagicMock()
|
||||
req.client = MagicMock()
|
||||
req.client.host = client_ip
|
||||
req.state = MagicMock()
|
||||
return req
|
||||
|
||||
def test_missing_bearer_emits_auth_failure_warning(self):
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.auth import verify_api_key
|
||||
|
||||
req = self._make_mock_request()
|
||||
with pytest.raises(Exception): # HTTPException 401
|
||||
verify_api_key(req, None)
|
||||
|
||||
# At least one warning record about auth
|
||||
warnings = [e for e in persisted if e.severity == ActivitySeverity.WARNING]
|
||||
assert len(warnings) >= 1
|
||||
assert all(e.category == ActivityCategory.AUTH for e in warnings)
|
||||
|
||||
def test_invalid_token_emits_auth_failure_warning(self):
|
||||
"""Invalid token => warning record; token itself must NOT appear."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
creds = MagicMock()
|
||||
creds.credentials = self._SECRET_TOKEN # the "attempted" token
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.auth import verify_api_key
|
||||
|
||||
req = self._make_mock_request(client_ip="127.0.0.1")
|
||||
with pytest.raises(Exception): # HTTPException 401
|
||||
verify_api_key(req, creds)
|
||||
|
||||
# At least one warning-level auth record
|
||||
warnings = [e for e in persisted if e.severity == ActivitySeverity.WARNING]
|
||||
assert len(warnings) >= 1
|
||||
|
||||
# SECURITY: The attempted token must never appear in ANY field.
|
||||
for entry in persisted:
|
||||
assert (
|
||||
self._SECRET_TOKEN not in entry.message
|
||||
), "Attempted token found in message field!"
|
||||
for v in (entry.entity_id, entry.entity_name, entry.actor):
|
||||
assert v is None or self._SECRET_TOKEN not in str(
|
||||
v
|
||||
), f"Attempted token found in field: {v!r}"
|
||||
for meta_v in entry.metadata.values():
|
||||
assert self._SECRET_TOKEN not in str(
|
||||
meta_v
|
||||
), f"Attempted token found in metadata: {meta_v!r}"
|
||||
|
||||
def test_lan_rejection_without_keys_emits_warning(self):
|
||||
"""LAN request when no keys configured => warning record."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
# Override config to have no API keys
|
||||
with patch("ledgrab.api.auth.get_config") as mock_cfg:
|
||||
cfg = MagicMock()
|
||||
cfg.auth.api_keys = {}
|
||||
mock_cfg.return_value = cfg
|
||||
|
||||
from ledgrab.api.auth import verify_api_key
|
||||
|
||||
req = self._make_mock_request(client_ip="192.168.1.100")
|
||||
with pytest.raises(Exception): # HTTPException 401
|
||||
verify_api_key(req, None)
|
||||
|
||||
warnings = [e for e in persisted if e.severity == ActivitySeverity.WARNING]
|
||||
assert len(warnings) >= 1
|
||||
assert any("LAN" in e.message for e in warnings)
|
||||
|
||||
def test_auth_failure_record_has_client_ip_in_metadata(self):
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
creds = MagicMock()
|
||||
creds.credentials = self._SECRET_TOKEN
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.auth import verify_api_key
|
||||
|
||||
req = self._make_mock_request(client_ip="10.0.0.5")
|
||||
with pytest.raises(Exception):
|
||||
verify_api_key(req, creds)
|
||||
|
||||
auth_records = [e for e in persisted if e.category == ActivityCategory.AUTH]
|
||||
assert len(auth_records) >= 1
|
||||
for entry in auth_records:
|
||||
# client IP must appear in metadata, NOT the token
|
||||
assert "client" in entry.metadata
|
||||
assert self._SECRET_TOKEN not in str(entry.metadata)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Category C: Device connect/disconnect
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDeviceInstrumentation:
|
||||
"""Device health transitions and discovery events emit records."""
|
||||
|
||||
def test_device_offline_emits_warning_record(self):
|
||||
"""When a device goes from online → offline, a warning record is emitted."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
# Create a minimal DeviceHealthMixin-shaped object inline
|
||||
from ledgrab.core.devices.led_client import DeviceHealth
|
||||
from ledgrab.core.processing.device_health import DeviceHealthMixin
|
||||
|
||||
class FakeManager(DeviceHealthMixin):
|
||||
def __init__(self):
|
||||
self._devices = {}
|
||||
self._device_store = None
|
||||
|
||||
def fire_event(self, evt):
|
||||
pass
|
||||
|
||||
mgr = FakeManager()
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
@dataclass
|
||||
class FakeState:
|
||||
device_id: str
|
||||
device_url: str = "http://192.168.1.10"
|
||||
device_type: str = "wled"
|
||||
led_count: int = 60
|
||||
health: DeviceHealth = field(default_factory=DeviceHealth)
|
||||
health_task: object = None
|
||||
|
||||
state = FakeState(device_id="dev_001")
|
||||
state.health = DeviceHealth(online=True)
|
||||
mgr._devices["dev_001"] = state
|
||||
|
||||
# Simulate what _check_device_health does when online flips
|
||||
prev_online = True
|
||||
state.health = DeviceHealth(online=False, latency_ms=0.0)
|
||||
|
||||
if state.health.online != prev_online:
|
||||
mgr.fire_event(
|
||||
{
|
||||
"type": "device_health_changed",
|
||||
"device_id": "dev_001",
|
||||
"online": state.health.online,
|
||||
"latency_ms": state.health.latency_ms,
|
||||
}
|
||||
)
|
||||
# Reproduce the audit block from device_health.py
|
||||
is_online = state.health.online
|
||||
device_name = None
|
||||
display = device_name or "dev_001"
|
||||
action = "device.online" if is_online else "device.offline"
|
||||
severity = ActivitySeverity.INFO if is_online else ActivitySeverity.WARNING
|
||||
status_word = "came online" if is_online else "went offline"
|
||||
recorder.record(
|
||||
category=ActivityCategory.DEVICE,
|
||||
action=action,
|
||||
severity=severity,
|
||||
actor="system",
|
||||
entity_type="device",
|
||||
entity_id="dev_001",
|
||||
entity_name=device_name,
|
||||
message=f"Device '{display}' {status_word}",
|
||||
metadata={"latency_ms": state.health.latency_ms},
|
||||
)
|
||||
|
||||
offline_records = [
|
||||
e
|
||||
for e in persisted
|
||||
if e.category == ActivityCategory.DEVICE and e.action == "device.offline"
|
||||
]
|
||||
assert len(offline_records) == 1
|
||||
r = offline_records[0]
|
||||
assert r.severity == ActivitySeverity.WARNING
|
||||
assert r.entity_id == "dev_001"
|
||||
|
||||
def test_device_online_emits_info_record(self):
|
||||
"""When a device comes online, an info record is emitted."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
recorder.record(
|
||||
category=ActivityCategory.DEVICE,
|
||||
action="device.online",
|
||||
severity=ActivitySeverity.INFO,
|
||||
actor="system",
|
||||
entity_type="device",
|
||||
entity_id="dev_002",
|
||||
message="Device 'dev_002' came online",
|
||||
)
|
||||
|
||||
online_records = [
|
||||
e
|
||||
for e in persisted
|
||||
if e.category == ActivityCategory.DEVICE and e.action == "device.online"
|
||||
]
|
||||
assert len(online_records) == 1
|
||||
assert online_records[0].severity == ActivitySeverity.INFO
|
||||
|
||||
def test_device_discovered_emits_record(self):
|
||||
"""DiscoveryWatcher._emit produces an audit record."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.core.devices.discovery_watcher import DiscoveryWatcher, _DiscoveredEntry
|
||||
|
||||
mock_device_store = MagicMock()
|
||||
mock_device_store.get_all_devices.return_value = []
|
||||
|
||||
fired_events: list[dict] = []
|
||||
watcher = DiscoveryWatcher(
|
||||
device_store=mock_device_store,
|
||||
fire_event=lambda evt: fired_events.append(evt),
|
||||
)
|
||||
|
||||
entry = _DiscoveredEntry(
|
||||
key="wled-test._wled._tcp.local.",
|
||||
url="http://192.168.1.55",
|
||||
name="WLED-Test",
|
||||
device_type="wled",
|
||||
)
|
||||
watcher._emit("device_discovered", entry)
|
||||
|
||||
disc_records = [
|
||||
e
|
||||
for e in persisted
|
||||
if e.category == ActivityCategory.DEVICE and e.action == "device.discovered"
|
||||
]
|
||||
assert len(disc_records) == 1
|
||||
r = disc_records[0]
|
||||
assert r.severity == ActivitySeverity.INFO
|
||||
assert r.entity_name == "WLED-Test"
|
||||
assert "192.168.1.55" in r.metadata.get("url", "")
|
||||
|
||||
def test_device_lost_emits_warning_record(self):
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.core.devices.discovery_watcher import DiscoveryWatcher, _DiscoveredEntry
|
||||
|
||||
mock_device_store = MagicMock()
|
||||
mock_device_store.get_all_devices.return_value = []
|
||||
|
||||
watcher = DiscoveryWatcher(
|
||||
device_store=mock_device_store,
|
||||
fire_event=lambda evt: None,
|
||||
)
|
||||
entry = _DiscoveredEntry(
|
||||
key="lost-device._wled._tcp.local.",
|
||||
url="http://192.168.1.77",
|
||||
name="Lost-WLED",
|
||||
device_type="wled",
|
||||
)
|
||||
watcher._emit("device_lost", entry)
|
||||
|
||||
lost_records = [
|
||||
e
|
||||
for e in persisted
|
||||
if e.category == ActivityCategory.DEVICE and e.action == "device.lost"
|
||||
]
|
||||
assert len(lost_records) == 1
|
||||
assert lost_records[0].severity == ActivitySeverity.WARNING
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Category D: Capture & system events
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestCaptureAndSystemInstrumentation:
|
||||
"""Capture start and backup-create emit records."""
|
||||
|
||||
def test_capture_started_record(self):
|
||||
"""capture.started record is emitted with correct category."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.routes.output_targets_control import _record_capture
|
||||
|
||||
_record_capture(
|
||||
"capture.started",
|
||||
"ot_test",
|
||||
"My Test Strip",
|
||||
"Capture started for target 'My Test Strip'",
|
||||
)
|
||||
|
||||
assert len(persisted) == 1
|
||||
r = persisted[0]
|
||||
assert r.category == ActivityCategory.CAPTURE
|
||||
assert r.action == "capture.started"
|
||||
assert r.entity_id == "ot_test"
|
||||
assert r.entity_name == "My Test Strip"
|
||||
|
||||
def test_capture_stopped_record(self):
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.routes.output_targets_control import _record_capture
|
||||
|
||||
_record_capture(
|
||||
"capture.stopped",
|
||||
"ot_test",
|
||||
"Strip",
|
||||
"Capture stopped for target 'Strip'",
|
||||
)
|
||||
|
||||
assert len(persisted) == 1
|
||||
assert persisted[0].action == "capture.stopped"
|
||||
|
||||
def test_backup_created_record(self):
|
||||
"""backup.created system record emitted on backup download."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.routes.backup import _record_system
|
||||
|
||||
_record_system(
|
||||
"backup.created",
|
||||
"Backup downloaded: ledgrab-backup-20260101T000000.zip",
|
||||
{"filename": "ledgrab-backup-20260101T000000.zip"},
|
||||
)
|
||||
|
||||
assert len(persisted) == 1
|
||||
r = persisted[0]
|
||||
assert r.category == ActivityCategory.SYSTEM
|
||||
assert r.action == "backup.created"
|
||||
assert "backup" in r.message.lower()
|
||||
|
||||
def test_backup_restored_record(self):
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.routes.backup import _record_system
|
||||
|
||||
_record_system("backup.restored", "Database restored from uploaded backup")
|
||||
|
||||
assert len(persisted) == 1
|
||||
assert persisted[0].action == "backup.restored"
|
||||
|
||||
def test_no_token_in_any_system_record(self):
|
||||
"""System records must never include token-like secrets."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
_SECRET = "my-api-token-12345" # noqa: S105
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.routes.backup import _record_system
|
||||
|
||||
# Even if someone tried to pass a token (they shouldn't)
|
||||
_record_system("backup.created", "Backup created")
|
||||
|
||||
for entry in persisted:
|
||||
assert _SECRET not in entry.message
|
||||
for v in entry.metadata.values():
|
||||
assert _SECRET not in str(v)
|
||||
|
||||
def test_settings_changed_record(self):
|
||||
"""shutdown_action settings change emits a system record."""
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.routes.system_settings import _record_setting
|
||||
|
||||
_record_setting(
|
||||
"settings.changed",
|
||||
"shutdown_action",
|
||||
"Shutdown action set to 'nothing'",
|
||||
)
|
||||
|
||||
assert len(persisted) == 1
|
||||
r = persisted[0]
|
||||
assert r.category == ActivityCategory.SYSTEM
|
||||
assert r.action == "settings.changed"
|
||||
assert r.metadata.get("setting_key") == "shutdown_action"
|
||||
|
||||
def test_settings_change_excludes_activity_log_key(self):
|
||||
"""The 'activity_log' settings key must not self-referentially trigger records.
|
||||
|
||||
This is enforced by the caller checking the key before calling
|
||||
_record_setting. Verify our helper does NOT filter automatically (the
|
||||
responsibility is on the caller), but that the activity_log settings path
|
||||
in the retention engine does not call record_setting.
|
||||
"""
|
||||
# Verify that _record_setting itself doesn't filter — that's not its job.
|
||||
recorder, persisted, _ = _make_recorder()
|
||||
with _patch_module_recorder(recorder):
|
||||
from ledgrab.api.routes.system_settings import _record_setting
|
||||
|
||||
# The caller is responsible for not passing "activity_log"
|
||||
# Calling it with any other key works fine:
|
||||
_record_setting("settings.changed", "auto_backup", "Auto-backup enabled")
|
||||
|
||||
assert persisted[0].metadata["setting_key"] == "auto_backup"
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user