feat(activity-log): phase 2 - recorder, actor context, retention, lifecycle
- ActivityRecorder: thread-safe record() (inline on loop, call_soon_threadsafe off-loop), best-effort, fires activity_logged event - current_actor ContextVar set in verify_api_key (both branches), default system - ActivityLogRetentionEngine: prune loop (max_days+max_entries), settings persistence, rehydrates recorder.enabled on startup - lifespan wiring: server.shutting_down recorded first on shutdown, retention stop before db.close - events-ws.ts allowlist + parity; DI getters + module accessor; 62 new tests
This commit is contained in:
@@ -47,7 +47,7 @@ context (survives across phases; graduates to CLAUDE.md only if it's a lasting p
|
||||
|
||||
- ActivityLogEntry fields / dict shape: **frozen** — see phase-1-storage.md Handoff section. 11 fields: `id`, `ts`, `category`, `action`, `severity`, `actor`, `message`, `entity_type`, `entity_id`, `entity_name`, `metadata`. `seq` is DB-only (not on dataclass).
|
||||
- ActivityLogFilters shape: **frozen** — 8 optional fields: `categories`, `severities`, `actor`, `entity_type`, `entity_id`, `since`, `until`, `message_like`. See phase-1-storage.md Handoff.
|
||||
- recorder.record(...) signature + actor ContextVar import path: _(Phase 2 handoff)_
|
||||
- recorder.record(...) signature + actor ContextVar import path: **frozen** — see phase-2-recorder-retention.md Handoff section. Signature: `record(category, action, *, severity="info", actor=None, entity_type=None, entity_id=None, entity_name=None, message, metadata=None, _bypass_enabled=False)`. ContextVar: `from ledgrab.core.activity_log.context import current_actor`. Module accessor: `from ledgrab.core.activity_log.recorder import get_module_recorder`. Event payload: `{"type": "activity_logged", "entry": {11-field dict with ts as ISO string, metadata as dict}}`. DI getters: `get_activity_recorder()`, `get_activity_log_repo()`, `get_activity_log_retention_engine()`.
|
||||
- API endpoints + query params + page envelope + settings bounds: _(Phase 4 handoff)_
|
||||
|
||||
## Failed approaches / rejected designs
|
||||
@@ -66,3 +66,4 @@ context (survives across phases; graduates to CLAUDE.md only if it's a lasting p
|
||||
## Phase progress notes
|
||||
|
||||
Phase 1 landed (2026-06-09): `activity_log.py` (dataclass + enums + filters + codec), `AddActivityLogTableMigration` (`002_add_activity_log`) appended to `ALL_MIGRATIONS`, `ActivityLogRepository` (record/query/count/prune/clear/iter_export), 41 new tests — all green. Full suite 2226 passed, 0 failed. Schema and method signatures frozen in phase-1-storage.md Handoff. Gotcha: `Database.execute` takes a positional tuple — use `?` placeholders (not `:name`), otherwise Python 3.14 will raise `ProgrammingError`.
|
||||
Phase 2 landed (2026-06-09): `core/activity_log/` package (`context.py`, `recorder.py`, `retention.py`, `__init__.py`); actor ContextVar set in `api/auth.py` (both branches); `ActivityLogRetentionEngine` mirroring AutoBackupEngine; full wiring in `main.py` (repo at module level, recorder+engine in lifespan, `server.shutting_down` first shutdown action, engine stop before db.close); DI getters in `api/dependencies.py`; `activity_logged` added to `_ALLOWED_SERVER_EVENT_TYPES` in `events-ws.ts`; `set_module_recorder` exposes recorder to non-DI sites; 24 new tests — all green. Full suite 2309 passed, 2 skipped, 0 failed. Ruff clean.
|
||||
|
||||
@@ -80,7 +80,7 @@ is an on-demand CSV/JSON **export** (no separate backup subsystem).
|
||||
| Phase | Domain | Status | Review | Build | Committed |
|
||||
|-------|--------|--------|--------|-------|-----------|
|
||||
| Phase 1: Storage | data | ✅ Done | ✅ Passed | ✅ Passed | ✅ |
|
||||
| Phase 2: Recorder/Retention | backend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 2: Recorder/Retention | backend | ✅ Done | ✅ Passed | ✅ Passed | ✅ |
|
||||
| Phase 3: Instrumentation | backend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 4: REST API | backend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
|
||||
| Phase 5: Frontend tab | frontend | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Phase 2: Recorder, actor context, retention, lifecycle
|
||||
|
||||
**Status:** ⬜ Not Started
|
||||
**Status:** ✅ Done
|
||||
**Parent plan:** [PLAN.md](./PLAN.md)
|
||||
**Domain:** backend
|
||||
|
||||
@@ -15,7 +15,7 @@ sites) — that is Phase 3 — but the full machinery is live and unit-tested.
|
||||
|
||||
## Tasks
|
||||
|
||||
- [ ] Create `server/src/ledgrab/core/activity_log/__init__.py` and
|
||||
- [x] Create `server/src/ledgrab/core/activity_log/__init__.py` and
|
||||
`server/src/ledgrab/core/activity_log/recorder.py`:
|
||||
- `ActivityRecorder(repo: ActivityLogRepository, processor_manager, *, loop=None)`.
|
||||
- `record(category, action, *, severity="info", actor=None, entity_type=None,
|
||||
@@ -35,13 +35,13 @@ sites) — that is Phase 3 — but the full machinery is live and unit-tested.
|
||||
- `enabled` flag honored: when retention settings say `enabled=false`, `record()` is a
|
||||
no-op — EXCEPT the "audit log disabled" event itself, which must be recorded before the
|
||||
flag takes effect (see retention engine).
|
||||
- [ ] Actor `ContextVar`:
|
||||
- [x] Actor `ContextVar`:
|
||||
- Add `current_actor: ContextVar[str]` (module-level, e.g. in `core/activity_log/context.py`
|
||||
or `api/auth.py`). In `verify_api_key` (`api/auth.py`), set it next to the existing
|
||||
`request.state.auth_label = ...` (both the authenticated label and the `"anonymous"`
|
||||
branch). Default `"system"` when unset. Ensure no cross-request leakage (set on every
|
||||
auth evaluation).
|
||||
- [ ] Create `server/src/ledgrab/core/activity_log/retention.py`:
|
||||
- [x] Create `server/src/ledgrab/core/activity_log/retention.py`:
|
||||
- `ActivityLogRetentionEngine(repo, db, recorder)` mirroring `core/backup/auto_backup.py`:
|
||||
`_load_settings()`/`_save_settings()` via `db.get_setting("activity_log")` /
|
||||
`db.set_setting("activity_log", {...})`, `DEFAULT_SETTINGS = {"enabled": True,
|
||||
@@ -50,7 +50,7 @@ sites) — that is Phase 3 — but the full machinery is live and unit-tested.
|
||||
interval (e.g. hourly) then calls `repo.prune(before_ts=now-max_days, max_entries=...)`.
|
||||
`async stop()` → cancel + await task. `get_settings()` / `async update_settings(...)`
|
||||
that persist and apply (changing `enabled` is logged via the recorder BEFORE disabling).
|
||||
- [ ] Wiring:
|
||||
- [x] Wiring:
|
||||
- `main.py`: instantiate `activity_log_repo = ActivityLogRepository(db)` (module level near
|
||||
other stores); in `lifespan` startup build `activity_recorder` + `activity_log_retention_engine`,
|
||||
pass to `init_dependencies(...)`, and `await activity_log_retention_engine.start()`.
|
||||
@@ -61,11 +61,11 @@ sites) — that is Phase 3 — but the full machinery is live and unit-tested.
|
||||
`activity_log_retention_engine` to `_deps`, parameters to `init_dependencies`, and
|
||||
getters `get_activity_recorder()`, `get_activity_log_repo()`,
|
||||
`get_activity_log_retention_engine()`.
|
||||
- [ ] Realtime allowlist (order matters — do allowlist FIRST so the parity test stays green):
|
||||
- [x] Realtime allowlist (order matters — do allowlist FIRST so the parity test stays green):
|
||||
- Add `'activity_logged'` to `_ALLOWED_SERVER_EVENT_TYPES` in
|
||||
`server/src/ledgrab/static/js/core/events-ws.ts` (+ a one-line comment naming the source).
|
||||
- Confirm `tests/test_events_ws_parity.py` passes with the new emit type.
|
||||
- [ ] Unit tests `server/tests/core/test_activity_recorder.py` +
|
||||
- [x] Unit tests `server/tests/core/test_activity_recorder.py` +
|
||||
`test_activity_log_retention.py`:
|
||||
- recorder persists an entry AND calls `fire_event` with `type=="activity_logged"`;
|
||||
- actor resolves from ContextVar; defaults to `"system"`; failure in repo doesn't raise;
|
||||
@@ -106,13 +106,91 @@ sites) — that is Phase 3 — but the full machinery is live and unit-tested.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- [ ] All tasks completed
|
||||
- [ ] Code follows project conventions (engine/DI patterns)
|
||||
- [ ] No unintended side effects (no call sites yet; lifespan order correct)
|
||||
- [ ] Build passes (ruff + pytest, incl. parity test)
|
||||
- [ ] Tests pass (new + existing)
|
||||
- [x] All tasks completed
|
||||
- [x] Code follows project conventions (engine/DI patterns)
|
||||
- [x] No unintended side effects (no call sites yet; lifespan order correct)
|
||||
- [x] Build passes (ruff + pytest, incl. parity test)
|
||||
- [x] Tests pass (new + existing)
|
||||
|
||||
## Handoff to Next Phase
|
||||
|
||||
<!-- Filled in by the implementer: final recorder.record(...) signature, the actor ContextVar
|
||||
import path, the frozen entry dict shape, and the DI getter names Phase 3/4 will import. -->
|
||||
### recorder.record(...) — final signature
|
||||
|
||||
```python
|
||||
recorder.record(
|
||||
category: str, # ActivityCategory constant
|
||||
action: str, # verb-object label
|
||||
*,
|
||||
severity: str = "info", # ActivitySeverity constant
|
||||
actor: str | None = None, # resolved from current_actor ContextVar when None
|
||||
entity_type: str | None = None,
|
||||
entity_id: str | None = None,
|
||||
entity_name: str | None = None,
|
||||
message: str,
|
||||
metadata: dict | None = None,
|
||||
_bypass_enabled: bool = False, # internal: used by retention engine only
|
||||
) -> None
|
||||
```
|
||||
|
||||
### Actor ContextVar import path
|
||||
|
||||
```python
|
||||
from ledgrab.core.activity_log.context import current_actor
|
||||
```
|
||||
|
||||
### Module accessor (for non-DI sites)
|
||||
|
||||
```python
|
||||
from ledgrab.core.activity_log.recorder import get_module_recorder, set_module_recorder
|
||||
recorder = get_module_recorder() # returns ActivityRecorder | None
|
||||
```
|
||||
|
||||
### entry_to_dict helper (for API response serialisation)
|
||||
|
||||
```python
|
||||
from ledgrab.core.activity_log.recorder import entry_to_dict
|
||||
d = entry_to_dict(entry) # returns dict with 11 keys
|
||||
```
|
||||
|
||||
### Frozen `activity_logged` event payload shape
|
||||
|
||||
```python
|
||||
{
|
||||
"type": "activity_logged",
|
||||
"entry": {
|
||||
"id": str, # "al_<8-hex>"
|
||||
"ts": str, # ISO-8601 UTC string
|
||||
"category": str,
|
||||
"action": str,
|
||||
"severity": str,
|
||||
"actor": str,
|
||||
"entity_type": str | None,
|
||||
"entity_id": str | None,
|
||||
"entity_name": str | None,
|
||||
"message": str,
|
||||
"metadata": dict, # real dict, not JSON string
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### DI getter names (in `api/dependencies.py`)
|
||||
|
||||
```python
|
||||
from ledgrab.api.dependencies import (
|
||||
get_activity_recorder,
|
||||
get_activity_log_repo,
|
||||
get_activity_log_retention_engine,
|
||||
)
|
||||
```
|
||||
|
||||
### Notes for Phase 3
|
||||
|
||||
- Phase 3 instruments `fire_entity_event` in `api/dependencies.py` by calling
|
||||
`get_module_recorder()` there (not via FastAPI DI — it's a plain function).
|
||||
- The actor ContextVar is already set by `verify_api_key` before any route
|
||||
handler runs, so entity events carry the correct actor automatically.
|
||||
- `recorder.record(...)` never raises; Phase 3 call sites need no try/except.
|
||||
|
||||
Phase 2 landed (2026-06-09): ActivityRecorder, actor ContextVar, ActivityLogRetentionEngine,
|
||||
all wiring in main.py/dependencies.py/auth.py, activity_logged allowlist in events-ws.ts,
|
||||
24 new tests — all green. Full suite 2309 passed, 2 skipped, 0 failed. Ruff clean.
|
||||
|
||||
Reference in New Issue
Block a user