feat(phase-7-backend): implement anomaly detection — SuspensionFlip detector, use case, poller, and tests

- AnomalyDetector (pure domain): detects odds-flip pattern from live snapshot
  timelines using implied-probability vectors (p=1/rate, normalised), flip score
  = max(|p_post−p_pre|), gated by both threshold AND favourite-changed test
- SuspensionInterval record: typed pair of (pre, post) OddsSnapshot bracketing a gap
- AnomalyOptions POCO (Application layer): bound to Anomaly:* config section with
  four fields (SuspensionGapSeconds=60, OddsFlipThreshold=0.30, MinSnapshotCount=3,
  DetectionIntervalSeconds=60)
- DetectAnomaliesUseCase: iterates all events, loads last-24h live snapshots, runs
  detector, persists new anomalies with 1-minute dedup window
- AnomalyDetectionPoller: BackgroundService polling every DetectionIntervalSeconds,
  gated by WorkerOptions.AnomalyDetectionEnabled (default true)
- DI wiring: DetectAnomaliesUseCase registered Scoped in ApplicationModule;
  AnomalyOptions bound + AnomalyDetectionPoller hosted in InfrastructureModule
- WorkerOptions.AnomalyDetectionEnabled added; appsettings.json updated
- 13 domain tests + 4 application tests; total 245/245 passing (no regression)
This commit is contained in:
2026-05-05 13:15:50 +03:00
parent d915667da1
commit a6ff368015
14 changed files with 1411 additions and 34 deletions
+38
View File
@@ -101,6 +101,44 @@ with scraping research, no implementation.
## Implementation Notes
### Phase 7 Backend (Anomaly detection, 2026-05-05)
- **`AnomalyDetector` is pure domain — no I/O, no DI.** Constructor takes three ints/decimals
from `AnomalyOptions`; the caller (use case) materialises it per cycle.
The UI evidence panel can reconstruct the same probabilities from `EvidenceJson` without
needing to re-invoke the detector.
- **Implied probability formula:** `p_i = 1 / rate_i`, then normalise so all `p_i` sum to 1
(divorcé of the bookmaker's margin). This is the standard European odds conversion.
- **Flip score** = `max(|p_post[i] p_pre[i]|)` over Match-Win sides (p1, pDraw?, p2).
Score is clamped to `[0, 1]` before constructing `Anomaly` (domain invariant enforces ≤1).
- **Two-part gate** — an anomaly requires BOTH: (a) `flipScore ≥ OddsFlipThreshold` AND
(b) `argmax(p_pre) != argmax(p_post)`. This prevents spurious detections when one side's
probability shifts a lot but it was never the favourite.
- **Tennis / 2-way markets** — `pDraw` is `null` when no `BetType.Draw` bet is present.
The detector and `EvidenceJson` gracefully handle this (JSON field is omitted when null via
`DefaultIgnoreCondition.WhenWritingNull`).
- **`EvidenceJson` uses `System.Text.Json` with custom `JsonPropertyName` attributes** on
sealed nested records (`EvidencePayload`, `SnapshotEvidence`). Source generation was not
used at this scale — the payload is small and created infrequently.
- **`DetectAnomaliesUseCase` loads all events + last-24-h live snapshots per cycle.**
This is a deliberate simplification; a future optimisation is to track `last_run_at` per
event. Documented as 🟡 in the handoff.
- **Dedup strategy:** two anomalies are considered duplicates if they share `EventId`, `Kind`,
and their `DetectedAt` values fall within a 1-minute window. This prevents the same
suspension triggering re-insertion on consecutive detection cycles while the gap snapshot
pair remains in the 24-hour window.
- **`AnomalyOptions` placed in `Marathon.Application/Configuration/`** (not Infrastructure).
The `AnomalyDetector` itself is in `Marathon.Domain/AnomalyDetection/` but requires no
options binding — it takes plain constructor parameters.
- **`AnomalyDetectionPoller` reads `IOptionsMonitor<AnomalyOptions>` per cycle** so that
hot-reload of `DetectionIntervalSeconds` takes effect without a restart. Same pattern as
`LiveOddsPoller` reading `WorkerOptions`.
- **`Workers:AnomalyDetectionEnabled`** added to `WorkerOptions` (default `true`) and
`appsettings.json`. UI agent must add a Settings toggle for this flag.
- **New test count: +17** (13 domain + 4 application). Total: 245/245 passing.
- **Test note:** rates 1.5/2.5 produce a flip score of ~0.25 — BELOW the 0.30 threshold.
Always use 1.3/4.0 (flip score ~0.51) or steeper to guarantee detection in tests.
### Phase 6 (Event browsing UI, 2026-05-05)
- **Plotly.Blazor pinned to 5.4.1.** v7.x exists but introduces breaking changes;
+1 -1
View File
@@ -69,7 +69,7 @@ parameter configurable.
| Phase 4: Application + Workers | backend | ✅ Done | ⚠️ Pass with notes (Sonnet) | ✅ Build OK + 202/202 tests | ✅ 2acbaa5 |
| Phase 5: Host + Theme + i18n | frontend | ✅ Done | ⚠️ Pass with notes (Sonnet, combined batch) | ✅ Build OK + 11/11 UI tests | ✅ batch (e4d8476…686550d…+) |
| Phase 6: Event browsing UI | frontend | ✅ Done | ⚠️ Pass with notes (Sonnet) | ✅ Build OK + 228/228 tests | ✅ 553db2b |
| Phase 7: Anomaly detection | fullstack | ⬜ Not Started | ⬜ | ⏭️ Big Bang | ⬜ |
| Phase 7: Anomaly detection | fullstack | 🔨 Backend done | ⬜ | ✅ Build OK + 245/245 tests | ⬜ |
| Phase 8: Results loader | fullstack | ⬜ Not Started | ⬜ | ⏭️ Big Bang | ⬜ |
| Phase 9: Packaging + polish | fullstack | ⬜ Not Started | ⬜ | ⬜ | ⬜ |
@@ -1,6 +1,6 @@
# Phase 7: Anomaly Detection (Suspension + Flip)
**Status:** ⬜ Not Started
**Status:** 🔨 Backend Done — Awaiting Frontend (Opus)
**Parent plan:** [PLAN.md](./PLAN.md)
**Domain:** fullstack
**Implementer:** Sonnet (backend portion) + Opus (UI portion, with frontend-design)
@@ -14,42 +14,63 @@ and surface them in a dedicated UI feed page so the user can act on them.
## Tasks
### Backend (Sonnet)
### Backend (Sonnet) ✅ COMPLETE
- [ ] Implement `Marathon.Domain/AnomalyDetection/AnomalyDetector.cs`:
- [x] Implement `Marathon.Domain/AnomalyDetection/AnomalyDetector.cs`:
- Pure domain logic — takes `IReadOnlyList<OddsSnapshot>` for an event, returns
`IReadOnlyList<Anomaly>`
- Detect suspension intervals: gaps between snapshots > `SuspensionGapSeconds`
(configurable)
- For each suspension, compute pre-suspension and post-suspension implied
probability vectors `(p1, pDraw, p2)` from Win-1/Draw/Win-2 rates
- Compute flip score: `max(|p_post[i] - p_pre[i]|)` across i ∈ {1, draw, 2}
- If flip score ≥ `OddsFlipThreshold` AND the favorite changed (argmax differs),
- Compute flip score: `max(|p_post[i] p_pre[i]|)` across i ∈ {1, draw, 2}
- If flip score ≥ `OddsFlipThreshold` AND the favourite changed (argmax differs),
emit an `Anomaly(Kind=SuspensionFlip, Score, EvidenceJson)` where `EvidenceJson`
contains the snapshots bracketing the suspension
- [ ] Add `AnomalyOptions` POCO bound to `Anomaly:*`:
- [x] Add `AnomalyOptions` POCO bound to `Anomaly:*` (in `Marathon.Application/Configuration/`):
```csharp
public sealed class AnomalyOptions {
public int SuspensionGapSeconds { get; init; } = 60;
public decimal OddsFlipThreshold { get; init; } = 0.30m;
public int MinSnapshotCount { get; init; } = 3;
public int DetectionIntervalSeconds { get; init; } = 60;
}
```
- [ ] Implement `DetectAnomaliesUseCase` in `Marathon.Application/UseCases/`:
- Iterate over events with new snapshots since last detection run
- [x] Implement `DetectAnomaliesUseCase` in `Marathon.Application/UseCases/`:
- Iterate over all events and load snapshots from last 24 h
- Invoke `AnomalyDetector` per event
- Persist new anomalies via `IAnomalyRepository`
- [ ] Implement `AnomalyDetectionPoller : BackgroundService` in
- Persist new anomalies via `IAnomalyRepository` with dedup logic
- [x] Implement `AnomalyDetectionPoller : BackgroundService` in
`Marathon.Infrastructure/Workers/`:
- Runs every `Anomaly:DetectionIntervalSeconds` (default 60s)
- Calls `DetectAnomaliesUseCase`
- [ ] Backend tests in `Marathon.Domain.Tests/AnomalyDetection/`:
- Synthetic snapshot timeline with no flip → 0 anomalies
- Snapshot timeline with suspension + small odds shift → 0 anomalies (below threshold)
- Snapshot timeline with suspension + large flip (favorite ↔ underdog) → 1 anomaly
- Score calculation matches expected value
- Gated by `Workers:AnomalyDetectionEnabled` (default `true`)
- [x] Add `WorkerOptions.AnomalyDetectionEnabled` (default `true`)
- [x] Register `DetectAnomaliesUseCase` as Scoped in `ApplicationModule`
- [x] Bind `AnomalyOptions` and register `AnomalyDetectionPoller` in `InfrastructureModule`
- [x] Update `appsettings.json` — add `Workers:AnomalyDetectionEnabled: true`
(all 4 `Anomaly:*` keys already existed from Phase 5)
- [x] Backend tests in `Marathon.Domain.Tests/AnomalyDetection/AnomalyDetectorTests.cs` (10 tests):
- Empty snapshot list → 0 anomalies ✓
- Below `minSnapshotCount` → 0 anomalies ✓
- Pre-match-only snapshots → 0 anomalies ✓
- No suspension (regular intervals) → 0 anomalies ✓
- Suspension but odds shift below threshold → 0 anomalies ✓
- Suspension + favourite flip (2-way) → 1 anomaly ✓
- Score calculation correct for known inputs ✓
- Tennis (no draw) → 1 anomaly ✓
- Multiple suspensions → multiple anomalies ✓
- EvidenceJson contains pre/post probability vectors and rates ✓
- Determinism: same input → same output ✓
- 3-way market flip (draw becomes favourite) → 1 anomaly ✓
- Mixed pre-match + live snapshots → only live analysed ✓
- [x] Application tests in `Marathon.Application.Tests/UseCases/DetectAnomaliesUseCaseTests.cs` (4 tests):
- Iterates events, calls detector, persists new anomalies ✓
- Skips already-persisted anomalies (dedup logic) ✓
- Tolerates per-event failures (one event throwing doesn't abort the cycle) ✓
- Returns count of new anomalies ✓
### Frontend (Opus + frontend-design)
### Frontend (Opus + frontend-design) ⬜ NOT STARTED
- [ ] Create `Marathon.UI/Pages/Anomalies/AnomalyFeed.razor`:
- List of anomalies sorted by `DetectedAt` descending
@@ -62,30 +83,49 @@ and surface them in a dedicated UI feed page so the user can act on them.
information hierarchy.
- [ ] Add navigation entry to `MainLayout` drawer with notification badge showing
unread anomaly count.
- [ ] Localize all strings in RU + EN.
- [ ] Create `Marathon.UI/Services/IAnomalyBrowsingService.cs` + `AnomalyBrowsingService.cs`
+ `AnomalyBrowsingState.cs` + `AnomalyViewModels.cs`
- [ ] Append localization keys to `SharedResource.ru.resx` and `SharedResource.en.resx`
- [ ] Add Settings UI binding for `AnomalyDetectionEnabled` worker flag (see handoff)
- [ ] Frontend tests in `Marathon.UI.Tests/Pages/Anomalies/`:
- bUnit: anomaly card renders evidence timeline
- bUnit: filter narrows the list correctly
## Files to Modify/Create
- `src/Marathon.Domain/AnomalyDetection/AnomalyDetector.cs`
- `src/Marathon.Domain/AnomalyDetection/SuspensionInterval.cs`
- `src/Marathon.Application/UseCases/DetectAnomaliesUseCase.cs`
- `src/Marathon.Application/Configuration/AnomalyOptions.cs` (or in Infra)
- `src/Marathon.Infrastructure/Workers/AnomalyDetectionPoller.cs`
### Backend (done)
- `src/Marathon.Domain/AnomalyDetection/AnomalyDetector.cs` ✅ created
- `src/Marathon.Domain/AnomalyDetection/SuspensionInterval.cs` ✅ created
- `src/Marathon.Application/Configuration/AnomalyOptions.cs` ✅ created
- `src/Marathon.Application/UseCases/DetectAnomaliesUseCase.cs` ✅ created
- `src/Marathon.Application/ApplicationModule.cs` ✅ modified (added `DetectAnomaliesUseCase` registration)
- `src/Marathon.Infrastructure/Workers/AnomalyDetectionPoller.cs` ✅ created
- `src/Marathon.Infrastructure/Configuration/WorkerOptions.cs` ✅ modified (added `AnomalyDetectionEnabled`)
- `src/Marathon.Infrastructure/InfrastructureModule.cs` ✅ modified (added `AnomalyOptions` binding + poller)
- `src/Marathon.Hosts.WpfBlazor/appsettings.json` ✅ modified (added `Workers:AnomalyDetectionEnabled`)
- `tests/Marathon.Domain.Tests/AnomalyDetection/AnomalyDetectorTests.cs` ✅ created (13 tests)
- `tests/Marathon.Application.Tests/UseCases/DetectAnomaliesUseCaseTests.cs` ✅ created (4 tests)
### Frontend (UI agent owns)
- `src/Marathon.UI/Pages/Anomalies/AnomalyFeed.razor`
- `src/Marathon.UI/Components/AnomalyCard.razor`
- `tests/Marathon.Domain.Tests/AnomalyDetection/AnomalyDetectorTests.cs`
- `src/Marathon.UI/Services/IAnomalyBrowsingService.cs`
- `src/Marathon.UI/Services/AnomalyBrowsingService.cs`
- `src/Marathon.UI/Services/AnomalyBrowsingState.cs`
- `src/Marathon.UI/Services/AnomalyViewModels.cs`
- `src/Marathon.UI/Resources/SharedResource.ru.resx` (append new keys)
- `src/Marathon.UI/Resources/SharedResource.en.resx` (append new keys)
- `src/Marathon.UI/MainLayout.razor` or `NavBody.razor` (anomaly nav entry)
- `tests/Marathon.UI.Tests/Pages/Anomalies/**`
## Acceptance Criteria
- Compiles (Big Bang).
- `AnomalyDetector` is a pure function — no I/O, no DI dependencies.
- Configurable thresholds via `appsettings.json` (visible in Settings page).
- UI clearly distinguishes high/medium/low severity anomalies.
- Evidence timeline shows the actual snapshots that triggered the detection.
- [x] Compiles (Big Bang).
- [x] `AnomalyDetector` is a pure function — no I/O, no DI dependencies.
- [x] Configurable thresholds via `appsettings.json`.
- [ ] Visible in Settings page (UI agent must wire `AnomalyDetectionEnabled`).
- [ ] UI clearly distinguishes high/medium/low severity anomalies.
- [ ] Evidence timeline shows the actual snapshots that triggered the detection.
## Notes
@@ -96,12 +136,144 @@ and surface them in a dedicated UI feed page so the user can act on them.
## Review Checklist
- [ ] Detector is deterministic and pure
- [ ] Score calculation correct (verify against hand-computed example)
- [ ] No false positives on synthetic "normal" timelines
- [x] Detector is deterministic and pure
- [x] Score calculation correct (verified against hand-computed example in test comments)
- [x] No false positives on synthetic "normal" timelines
- [ ] UI evidence timeline matches stored `EvidenceJson`
- [ ] All strings localized
## Handoff to Next Phase
<!-- Filled by Phase 7 implementer. -->
### Handoff to Phase 7 Frontend (UI) Agent
> **Read this section first.** The backend is fully implemented. You own all `Marathon.UI`
> files listed above. Do NOT touch any backend files.
---
#### What the backend provides
**`DetectAnomaliesUseCase.ExecuteAsync(CancellationToken)`**
- Returns `Task<int>` (count of new anomalies persisted this cycle).
- Called automatically by `AnomalyDetectionPoller` every 60 s (default).
- You do NOT call this from the UI — it is worker-driven.
- The UI only reads from `IAnomalyRepository`.
**`AnomalyDetector` — detection formula (for rendering evidence)**
- Implied probability: `p_i = (1 / rate_i)` for each win side.
- Normalisation: divide each `p_i` by the sum of all raw `p_i` values → they sum to 1.
- Flip score: `max(|p_post[i] p_pre[i]|)` over i ∈ {p1, pDraw?, p2}.
- Favourite-changed test: `argmax(p_pre) != argmax(p_post)`.
- An anomaly is emitted only if BOTH conditions hold: score ≥ threshold AND favourite changed.
**`IAnomalyRepository`** — the UI service should call:
- `ListAsync(CancellationToken)` — all anomalies for the feed page (paginate client-side).
- `GetAsync(Guid id, CancellationToken)` — single anomaly for a detail view.
- There is no `ListByEventAsync` on `IAnomalyRepository` (only on `ISnapshotRepository`).
If you need anomalies for a specific event, filter the full list by `EventId`.
**`Anomaly` entity** — fields available to the UI:
```csharp
Guid Id // GUID primary key
EventId EventId // bookmaker event code (e.g. "26456117")
DateTimeOffset DetectedAt // Moscow TZ (UTC+3)
AnomalyKind Kind // currently always SuspensionFlip
decimal Score // normalised [0, 1] — the largest implied-prob delta
string EvidenceJson // see shape below
```
**`Anomaly.EvidenceJson` shape:**
```json
{
"suspensionGapSeconds": 90,
"preSuspension": {
"capturedAt": "2026-05-10T18:00:00+03:00",
"p1": 0.755,
"pDraw": null,
"p2": 0.245,
"rate1": 1.3,
"rateDraw": null,
"rate2": 4.0
},
"postSuspension": {
"capturedAt": "2026-05-10T18:01:30+03:00",
"p1": 0.245,
"pDraw": null,
"p2": 0.755,
"rate1": 4.0,
"rateDraw": null,
"rate2": 1.3
}
}
```
- `pDraw` / `rateDraw` are `null` for 2-way markets (tennis, etc.).
- Use `System.Text.Json.JsonDocument.Parse(anomaly.EvidenceJson)` to deserialise in the UI.
Or define a `EvidenceDto` record in `AnomalyViewModels.cs` and use `JsonSerializer.Deserialize<EvidenceDto>`.
**Recommended severity buckets** (for color-coding):
| Severity | Score range | MudBlazor color suggestion |
|----------|-------------|---------------------------|
| Low | 0.300.45 | `Color.Warning` |
| Medium | 0.450.60 | `Color.Error` |
| High | 0.60+ | deep red / `Color.Error` + pulsing badge |
---
#### Settings page addition (UI agent must wire)
`Workers:AnomalyDetectionEnabled` (`bool`, default `true`) was added to `WorkerOptions`
and `appsettings.json`. The Phase 5 Settings page needs a toggle for it.
The existing pattern is the same as `LivePollerEnabled` and `UpcomingPollerEnabled`.
---
#### Localization keys to add
Append these to both `SharedResource.ru.resx` and `SharedResource.en.resx`:
| Key | EN value | RU value |
|------------------------------|------------------------------|------------------------------------|
| `Anomaly.Title` | Anomaly Feed | Лента аномалий |
| `Anomaly.Severity.Low` | Low | Низкая |
| `Anomaly.Severity.Medium` | Medium | Средняя |
| `Anomaly.Severity.High` | High | Высокая |
| `Anomaly.Card.DetectedAt` | Detected at | Обнаружено |
| `Anomaly.Card.Score` | Score | Оценка |
| `Anomaly.Card.Kind.SuspensionFlip` | Suspension Flip | Переворот после паузы |
| `Anomaly.Card.GapSeconds` | Suspension gap | Длительность паузы |
| `Anomaly.Evidence.PreSuspension` | Before suspension | До паузы |
| `Anomaly.Evidence.PostSuspension` | After suspension | После паузы |
| `Anomaly.Evidence.Probability` | Implied prob. | Вероятность |
| `Anomaly.Evidence.Rate` | Rate | Коэффициент |
| `Anomaly.Filter.Severity` | Min severity | Минимальная важность |
| `Anomaly.Filter.Sport` | Sport | Вид спорта |
| `Anomaly.Filter.DateRange` | Date range | Диапазон дат |
| `Anomaly.Empty` | No anomalies detected yet. | Аномалии пока не обнаружены. |
| `Settings.AnomalyDetection` | Anomaly detection | Обнаружение аномалий |
| `Settings.AnomalyDetectionEnabled` | Enable anomaly detection | Включить обнаружение аномалий |
---
#### Integration pattern for the UI service
Follow the same split as `EventBrowsingService` (Scoped) + `EventBrowsingState` (Singleton)
documented in CONTEXT.md Phase 6 notes. Specifically:
- `AnomalyBrowsingState` (Singleton): holds current filter settings + fires `OnChange`.
- `AnomalyBrowsingService` (Scoped): resolves `IAnomalyRepository` from the DI scope,
loads anomalies, and maps to view-models (`AnomalyListItem`, `AnomalyDetail`).
- `AnomalyListItem` view-model should include `Severity` (computed from `Score`),
pre-rendered display strings, and the parsed `EvidenceDto`.
---
#### 🟡 Known gaps / deferred items
- **No "last detection run" tracking.** The use case currently loads the last 24 h of
snapshots for ALL events on every cycle. A Phase 8/9 optimisation: track last-run
timestamp per event to limit the snapshot window. Flag this in the UI as "best-effort
coverage window: last 24 h".
- **`Settings.razor` AnomalyDetectionEnabled toggle** — backend option exists, UI wiring
is the UI agent's responsibility.
- **No read API for "unread anomaly count"** — the nav badge will need to read from
the full list and maintain a "last seen" timestamp in `AnomalyBrowsingState`.
Consider using `LocalStorage` via Blazor interop (same as any SPA pattern).