alexei.dolgolyov
6e1dd2111d
- Dashboard 'Recent Activity' widget: latest 5 entries, live prepend, 'View all' -> Activity tab - Settings 'Activity Log' panel: retention (enabled/max_days/max_entries) GET/PUT, clear (confirm + auth-required toast), CSV/JSON export - audit-log vs ephemeral debug Log Viewer distinction note + cross-links - public helpers fetchRecentEntries/renderCompactEntry on activity-log.ts (reused, no dup markup) - README Activity Log section; i18n across en/ru/zh - review fixes: clear 401 surfaces toast; empty widget transitions on first live event
74 lines
9.0 KiB
Markdown
74 lines
9.0 KiB
Markdown
# CONTEXT — Activity / Audit Log
|
||
|
||
Living scratchpad for the feature. The orchestrator updates this between phases. Tier-2
|
||
context (survives across phases; graduates to CLAUDE.md only if it's a lasting project truth).
|
||
|
||
## Config (from approval)
|
||
|
||
- **Mode:** Automated · **Execution:** Orchestrator · **Strategy:** Incremental
|
||
- **Base/merge target:** `master` · **Branch:** `feature/activity-log` · **Branch point:** `17dd2e0`
|
||
- Final merge ALWAYS requires user approval (even in Automated mode).
|
||
|
||
## Product decisions (locked)
|
||
|
||
- Placement: BOTH a top-level **Activity** tab AND a Dashboard **Recent Activity** widget,
|
||
plus a Settings retention panel.
|
||
- Scope: all four categories — entity CRUD, auth, device connect/disconnect, capture & system.
|
||
- Detail: **action metadata only** (no before/after diffs).
|
||
- Durability: **export on demand (CSV/JSON)** + existing whole-DB backup. **No** separate
|
||
backup subsystem.
|
||
- WebUI work uses the `frontend-design` skill (Phases 5–6).
|
||
|
||
## Key codebase facts (verified during planning)
|
||
|
||
- `fire_entity_event(entity_type, action, entity_id)` @ `api/dependencies.py:202` — central
|
||
hook, called by every entity route synchronously in-request; has `_deps` store access.
|
||
- `ProcessorManager.fire_event(dict)` / `subscribe_events()` @ `core/processing/processor_manager.py`
|
||
back `/api/v1/events/ws` (`api/routes/output_targets_control.py:206`). `fire_event` does
|
||
`put_nowait` (no `call_soon_threadsafe`) — fine for the existing consumer; recorder marshals.
|
||
- Frontend realtime: `core/events-ws.ts` re-dispatches `server:<type>`; `_ALLOWED_SERVER_EVENT_TYPES`
|
||
(line 39) is parity-checked by `tests/test_events_ws_parity.py`. Must add `activity_logged`.
|
||
- Actor: `request.state.auth_label` set in `api/auth.py` (129 authenticated, 83 anonymous).
|
||
- Storage: `Database` singleton (`storage/database.py`, single conn, RLock, WAL,
|
||
`synchronous=FULL`, `get_setting/set_setting`). `BaseSqliteStore` loads ALL rows to memory →
|
||
AVOID for the log. Migrations: `storage/data_migrations.py` (`ALL_MIGRATIONS`, idempotent).
|
||
- Backup/restore is **whole-DB** (`Database.backup_to`/`restore_from`, no STORE_MAP allowlist)
|
||
→ the new `activity_log` table is auto-covered. No `system.py` STORE_MAP edit needed.
|
||
- Background-engine pattern: `core/backup/auto_backup.py` (start/stop loop, `_prune`, settings).
|
||
- Thread-marshal precedent: `utils/log_broadcaster.py` (`ensure_loop` + `call_soon_threadsafe`).
|
||
- Device seams: `device_health_changed` (`core/processing/device_health.py`, on transition) +
|
||
`device_discovered`/`device_lost` (`core/devices/discovery_watcher.py`, **zeroconf thread**).
|
||
- **No** API-key create/rotate/revoke routes exist (only `GET /system/api-keys`) → those auth
|
||
events are DESCOPED.
|
||
- Existing **Log Viewer** (`utils/log_broadcaster.py`) = ephemeral debug-log tail; the audit
|
||
log is a different, persistent, structured feature. Differentiate; do not duplicate.
|
||
|
||
## Frozen contracts (fill as phases complete)
|
||
|
||
- ActivityLogEntry fields / dict shape: **frozen** — see phase-1-storage.md Handoff section. 11 fields: `id`, `ts`, `category`, `action`, `severity`, `actor`, `message`, `entity_type`, `entity_id`, `entity_name`, `metadata`. `seq` is DB-only (not on dataclass).
|
||
- ActivityLogFilters shape: **frozen** — 8 optional fields: `categories`, `severities`, `actor`, `entity_type`, `entity_id`, `since`, `until`, `message_like`. See phase-1-storage.md Handoff.
|
||
- recorder.record(...) signature + actor ContextVar import path: **frozen** — see phase-2-recorder-retention.md Handoff section. Signature: `record(category, action, *, severity="info", actor=None, entity_type=None, entity_id=None, entity_name=None, message, metadata=None, _bypass_enabled=False)`. ContextVar: `from ledgrab.core.activity_log.context import current_actor`. Module accessor: `from ledgrab.core.activity_log.recorder import get_module_recorder`. Event payload: `{"type": "activity_logged", "entry": {11-field dict with ts as ISO string, metadata as dict}}`. DI getters: `get_activity_recorder()`, `get_activity_log_repo()`, `get_activity_log_retention_engine()`.
|
||
- API endpoints + query params + page envelope + settings bounds: **frozen** — see phase-4-api.md Handoff section. Endpoints: `GET /api/v1/activity-log` (list, AuthRequired), `GET /api/v1/activity-log/export` (stream CSV/JSON, require_authenticated), `GET|PUT /api/v1/activity-log/settings` (AuthRequired), `DELETE /api/v1/activity-log` (clear, require_authenticated). Page envelope: `entries`, `next_before_seq`, `has_more`, `total`. Settings fields: `enabled` (bool), `max_days` (0–3650), `max_entries` (0–10_000_000). Export: `?format=csv|json`.
|
||
|
||
## Failed approaches / rejected designs
|
||
|
||
- Buffered async-writer subsystem (asyncio.Queue): REJECTED — unsafe from the zeroconf thread
|
||
and adds a shutdown-flush ordering hazard. Using direct synchronous-on-loop writes with
|
||
`call_soon_threadsafe` marshaling instead (simpler + correct).
|
||
- Using `BaseSqliteStore` for the log: REJECTED — loads all rows into memory.
|
||
- Separate activity-log backup subsystem: REJECTED — whole-DB backup already covers the table;
|
||
export-on-demand is the portability story.
|
||
|
||
## Deferred / open
|
||
|
||
- Setup-scaffold first-run noise suppression (batch/suppress flag) — deferred (nice-to-have).
|
||
|
||
## Phase progress notes
|
||
|
||
Phase 1 landed (2026-06-09): `activity_log.py` (dataclass + enums + filters + codec), `AddActivityLogTableMigration` (`002_add_activity_log`) appended to `ALL_MIGRATIONS`, `ActivityLogRepository` (record/query/count/prune/clear/iter_export), 41 new tests — all green. Full suite 2226 passed, 0 failed. Schema and method signatures frozen in phase-1-storage.md Handoff. Gotcha: `Database.execute` takes a positional tuple — use `?` placeholders (not `:name`), otherwise Python 3.14 will raise `ProgrammingError`.
|
||
Phase 2 landed (2026-06-09): `core/activity_log/` package (`context.py`, `recorder.py`, `retention.py`, `__init__.py`); actor ContextVar set in `api/auth.py` (both branches); `ActivityLogRetentionEngine` mirroring AutoBackupEngine; full wiring in `main.py` (repo at module level, recorder+engine in lifespan, `server.shutting_down` first shutdown action, engine stop before db.close); DI getters in `api/dependencies.py`; `activity_logged` added to `_ALLOWED_SERVER_EVENT_TYPES` in `events-ws.ts`; `set_module_recorder` exposes recorder to non-DI sites; 24 new tests — all green. Full suite 2309 passed, 2 skipped, 0 failed. Ruff clean.
|
||
Phase 4 landed (2026-06-09): schemas (`api/schemas/activity_log.py`), routes (`api/routes/activity_log.py`: list/export/settings/clear), router registration in `api/__init__.py`, `get_seq_for_id` helper on `ActivityLogRepository`. 49 new tests — all green. Full suite 2486 passed, 2 skipped, 0 failed. Ruff clean. Pagination bug found and fixed (limit+1 probe must drop oldest row when has_more, not tail).
|
||
Phase 3 landed (2026-06-09): instrumented all four categories — entity CRUD via `fire_entity_event` choke-point (`dependencies.py`), auth failures + WS session in `auth.py`, device online/offline in `device_health.py`, device discovered/lost in `discovery_watcher.py`, ADB connect/disconnect in `system_settings.py`, capture start/stop (individual + bulk) in `output_targets_control.py`, scene/playlist/automation activate in their respective route/engine files, backup/restore/delete + restart/shutdown/update/calibration/settings in `backup.py`/`update.py`/`calibration.py`; all 11 entity delete handlers pass `entity_name` to `fire_entity_event`; 22 new tests (security: token never in any field, explicitly asserted) — all green. Full suite 2369 passed, 2 skipped, 0 failed. Ruff clean. Complete (category, action) inventory in phase-3-instrumentation.md Handoff section.
|
||
Phase 5 landed (2026-06-09): Activity tab frontend — `features/activity-log.ts` (loadActivityLog, filter toolbar with category/severity chips + presets + debounced search + date range + actor/entity-type text fields, keyset-paginated list, expandable detail drawer, live-prepend via `server:activity_logged`, authed CSV/JSON blob export); `core/ui.ts` additions (`formatTimestamp`, `formatRelativeTime`); `core/icon-paths.ts` + `core/icons.ts` additions (scrollText/audit, circleAlert, info, filter, xCircle); `core/tab-registry.ts` registered; `templates/index.html` tab button + panel; `app.ts` + `global.d.ts` wired; `static/css/activity-log.css` (precision-ledger design, category color rail, live-dot pulse, detail drawer, responsive breakpoints); all three locales updated (activity_log.* + time.* relative-time keys + tour.activity_log); `features/tutorials.ts` getting-started tour extended. `tsc --noEmit` clean, `npm run build` passes. Reusable helpers for Phase 6 documented in phase-5-frontend-tab.md Handoff section.
|
||
Phase 6 landed (2026-06-09): Dashboard "Recent Activity" widget (live SSE, View-all link, `.dal-*` CSS, loading/empty states) + Settings "Activity Log" panel (enabled toggle, max_days/max_entries, Save toast, authed CSV/JSON export, confirmed Clear, audit-vs-debug cross-links) + 32 i18n keys per locale + README "Activity Log" section. `tsc --noEmit` clean, `npm run build` passes. All six phases complete — feature ready for final review.
|