fix(anomaly): exclude non-directional kinds from grading and backtest

Review follow-up (HIGH): the three detectors fed the same evaluator/backtest, but
SuspensionFreeze is non-directional (favourite unchanged) — grading it as "favourite
won" polluted the hit-rate with the base favourite-win rate, and its high frozen-ness
score always cleared the backtest threshold.

- Add AnomalyKind.IsDirectional() (flip + steam = true, freeze = false).
- AnomalyOutcomeEvaluator returns Unresolved for non-directional kinds (favourites
  still surfaced for display) so they don't distort calibration.
- RunBacktestUseCase skips non-directional anomalies when building candidates.
- Tests for the classification, the evaluator path, and the backtest skip.
This commit is contained in:
2026-05-29 01:25:16 +03:00
parent e307a54bec
commit c9eee9f907
6 changed files with 105 additions and 0 deletions
@@ -3,6 +3,7 @@ using Marathon.Application.Storage;
using Marathon.Domain.AnomalyDetection;
using Marathon.Domain.Backtesting;
using Marathon.Domain.Entities;
using Marathon.Domain.Enums;
using Microsoft.Extensions.Logging;
using DomainEventId = Marathon.Domain.ValueObjects.EventId;
@@ -95,6 +96,11 @@ public sealed class RunBacktestUseCase
{
ct.ThrowIfCancellationRequested();
// Only directional kinds are betting signals; SuspensionFreeze (favourite
// unchanged) is informational and must not be staked or it would skew ROI.
if (!anomaly.Kind.IsDirectional())
continue;
// Cannot simulate a bet whose event hasn't been graded yet.
if (!resultLookup.TryGetValue(anomaly.EventId, out var result))
continue;
@@ -63,6 +63,25 @@ public static class AnomalyOutcomeEvaluator
var preFav = data.PreSuspension.Favourite;
var postFav = data.PostSuspension.Favourite;
// Non-directional kinds (e.g. SuspensionFreeze — the favourite did NOT change)
// make no side prediction. Grading them as "favourite won" would just measure the
// base favourite-win rate, polluting the hit-rate and score-bin calibration, so we
// leave them Unresolved (the favourites are still surfaced for display).
if (!anomaly.Kind.IsDirectional())
{
return new ResolvedAnomaly(
AnomalyId: anomaly.Id,
EventId: anomaly.EventId,
DetectedAt: anomaly.DetectedAt,
Score: anomaly.Score,
Kind: anomaly.Kind,
Sport: sport,
PreFlipFavourite: preFav,
PostFlipFavourite: postFav,
ActualWinner: result?.WinnerSide,
Outcome: AnomalyOutcomeKind.Unresolved);
}
if (result is null)
{
return new ResolvedAnomaly(
@@ -0,0 +1,24 @@
namespace Marathon.Domain.Enums;
/// <summary>Semantic classification of anomaly kinds.</summary>
public static class AnomalyKindExtensions
{
/// <summary>
/// Whether the kind makes a <i>directional</i> prediction — a specific side/favourite
/// expected to win — that can be graded against the result and bet on in a backtest.
/// </summary>
/// <remarks>
/// <see cref="AnomalyKind.SuspensionFlip"/> and <see cref="AnomalyKind.SteamMove"/> are
/// directional (they point at a favourite). <see cref="AnomalyKind.SuspensionFreeze"/> is
/// informational — the line did NOT move — so "predicting" the unchanged favourite would
/// merely measure the base favourite-win rate; it is excluded from outcome grading and
/// from backtest staking so it does not distort detector calibration.
/// </remarks>
public static bool IsDirectional(this AnomalyKind kind) => kind switch
{
AnomalyKind.SuspensionFlip => true,
AnomalyKind.SteamMove => true,
AnomalyKind.SuspensionFreeze => false,
_ => false,
};
}