alexei.dolgolyov
726f39e2ba
- ActivityRecorder: thread-safe record() (inline on loop, call_soon_threadsafe off-loop), best-effort, fires activity_logged event - current_actor ContextVar set in verify_api_key (both branches), default system - ActivityLogRetentionEngine: prune loop (max_days+max_entries), settings persistence, rehydrates recorder.enabled on startup - lifespan wiring: server.shutting_down recorded first on shutdown, retention stop before db.close - events-ws.ts allowlist + parity; DI getters + module accessor; 62 new tests
70 lines
5.9 KiB
Markdown
70 lines
5.9 KiB
Markdown
# CONTEXT — Activity / Audit Log
|
||
|
||
Living scratchpad for the feature. The orchestrator updates this between phases. Tier-2
|
||
context (survives across phases; graduates to CLAUDE.md only if it's a lasting project truth).
|
||
|
||
## Config (from approval)
|
||
|
||
- **Mode:** Automated · **Execution:** Orchestrator · **Strategy:** Incremental
|
||
- **Base/merge target:** `master` · **Branch:** `feature/activity-log` · **Branch point:** `17dd2e0`
|
||
- Final merge ALWAYS requires user approval (even in Automated mode).
|
||
|
||
## Product decisions (locked)
|
||
|
||
- Placement: BOTH a top-level **Activity** tab AND a Dashboard **Recent Activity** widget,
|
||
plus a Settings retention panel.
|
||
- Scope: all four categories — entity CRUD, auth, device connect/disconnect, capture & system.
|
||
- Detail: **action metadata only** (no before/after diffs).
|
||
- Durability: **export on demand (CSV/JSON)** + existing whole-DB backup. **No** separate
|
||
backup subsystem.
|
||
- WebUI work uses the `frontend-design` skill (Phases 5–6).
|
||
|
||
## Key codebase facts (verified during planning)
|
||
|
||
- `fire_entity_event(entity_type, action, entity_id)` @ `api/dependencies.py:202` — central
|
||
hook, called by every entity route synchronously in-request; has `_deps` store access.
|
||
- `ProcessorManager.fire_event(dict)` / `subscribe_events()` @ `core/processing/processor_manager.py`
|
||
back `/api/v1/events/ws` (`api/routes/output_targets_control.py:206`). `fire_event` does
|
||
`put_nowait` (no `call_soon_threadsafe`) — fine for the existing consumer; recorder marshals.
|
||
- Frontend realtime: `core/events-ws.ts` re-dispatches `server:<type>`; `_ALLOWED_SERVER_EVENT_TYPES`
|
||
(line 39) is parity-checked by `tests/test_events_ws_parity.py`. Must add `activity_logged`.
|
||
- Actor: `request.state.auth_label` set in `api/auth.py` (129 authenticated, 83 anonymous).
|
||
- Storage: `Database` singleton (`storage/database.py`, single conn, RLock, WAL,
|
||
`synchronous=FULL`, `get_setting/set_setting`). `BaseSqliteStore` loads ALL rows to memory →
|
||
AVOID for the log. Migrations: `storage/data_migrations.py` (`ALL_MIGRATIONS`, idempotent).
|
||
- Backup/restore is **whole-DB** (`Database.backup_to`/`restore_from`, no STORE_MAP allowlist)
|
||
→ the new `activity_log` table is auto-covered. No `system.py` STORE_MAP edit needed.
|
||
- Background-engine pattern: `core/backup/auto_backup.py` (start/stop loop, `_prune`, settings).
|
||
- Thread-marshal precedent: `utils/log_broadcaster.py` (`ensure_loop` + `call_soon_threadsafe`).
|
||
- Device seams: `device_health_changed` (`core/processing/device_health.py`, on transition) +
|
||
`device_discovered`/`device_lost` (`core/devices/discovery_watcher.py`, **zeroconf thread**).
|
||
- **No** API-key create/rotate/revoke routes exist (only `GET /system/api-keys`) → those auth
|
||
events are DESCOPED.
|
||
- Existing **Log Viewer** (`utils/log_broadcaster.py`) = ephemeral debug-log tail; the audit
|
||
log is a different, persistent, structured feature. Differentiate; do not duplicate.
|
||
|
||
## Frozen contracts (fill as phases complete)
|
||
|
||
- ActivityLogEntry fields / dict shape: **frozen** — see phase-1-storage.md Handoff section. 11 fields: `id`, `ts`, `category`, `action`, `severity`, `actor`, `message`, `entity_type`, `entity_id`, `entity_name`, `metadata`. `seq` is DB-only (not on dataclass).
|
||
- ActivityLogFilters shape: **frozen** — 8 optional fields: `categories`, `severities`, `actor`, `entity_type`, `entity_id`, `since`, `until`, `message_like`. See phase-1-storage.md Handoff.
|
||
- recorder.record(...) signature + actor ContextVar import path: **frozen** — see phase-2-recorder-retention.md Handoff section. Signature: `record(category, action, *, severity="info", actor=None, entity_type=None, entity_id=None, entity_name=None, message, metadata=None, _bypass_enabled=False)`. ContextVar: `from ledgrab.core.activity_log.context import current_actor`. Module accessor: `from ledgrab.core.activity_log.recorder import get_module_recorder`. Event payload: `{"type": "activity_logged", "entry": {11-field dict with ts as ISO string, metadata as dict}}`. DI getters: `get_activity_recorder()`, `get_activity_log_repo()`, `get_activity_log_retention_engine()`.
|
||
- API endpoints + query params + page envelope + settings bounds: _(Phase 4 handoff)_
|
||
|
||
## Failed approaches / rejected designs
|
||
|
||
- Buffered async-writer subsystem (asyncio.Queue): REJECTED — unsafe from the zeroconf thread
|
||
and adds a shutdown-flush ordering hazard. Using direct synchronous-on-loop writes with
|
||
`call_soon_threadsafe` marshaling instead (simpler + correct).
|
||
- Using `BaseSqliteStore` for the log: REJECTED — loads all rows into memory.
|
||
- Separate activity-log backup subsystem: REJECTED — whole-DB backup already covers the table;
|
||
export-on-demand is the portability story.
|
||
|
||
## Deferred / open
|
||
|
||
- Setup-scaffold first-run noise suppression (batch/suppress flag) — deferred (nice-to-have).
|
||
|
||
## Phase progress notes
|
||
|
||
Phase 1 landed (2026-06-09): `activity_log.py` (dataclass + enums + filters + codec), `AddActivityLogTableMigration` (`002_add_activity_log`) appended to `ALL_MIGRATIONS`, `ActivityLogRepository` (record/query/count/prune/clear/iter_export), 41 new tests — all green. Full suite 2226 passed, 0 failed. Schema and method signatures frozen in phase-1-storage.md Handoff. Gotcha: `Database.execute` takes a positional tuple — use `?` placeholders (not `:name`), otherwise Python 3.14 will raise `ProgrammingError`.
|
||
Phase 2 landed (2026-06-09): `core/activity_log/` package (`context.py`, `recorder.py`, `retention.py`, `__init__.py`); actor ContextVar set in `api/auth.py` (both branches); `ActivityLogRetentionEngine` mirroring AutoBackupEngine; full wiring in `main.py` (repo at module level, recorder+engine in lifespan, `server.shutting_down` first shutdown action, engine stop before db.close); DI getters in `api/dependencies.py`; `activity_logged` added to `_ALLOWED_SERVER_EVENT_TYPES` in `events-ws.ts`; `set_module_recorder` exposes recorder to non-DI sites; 24 new tests — all green. Full suite 2309 passed, 2 skipped, 0 failed. Ruff clean.
|