The Pull*UseCase implementations issued one HTTP request at a time despite
Scraping:MaxConcurrentRequests=4. With 30–80 live events and ~1s per
fetch, a 5–10s live cadence target was unreachable; cycles overflowed
the configured interval.
* New Marathon.Application.Configuration.ScrapingThrottle bound from the
shared Scraping:* section. Exposes only MaxConcurrentRequests so the
Application layer doesn't pull in the Infrastructure-side ScrapingOptions.
* PullLiveOddsUseCase + PullUpcomingEventsUseCase split into two phases:
- Phase 1 — Parallel.ForEachAsync over the event list with
MaxDegreeOfParallelism = throttle.MaxConcurrentRequests. The scraper's
Polly rate limiter still throttles to RequestsPerSecond underneath
this fan-out, so spikes are smoothed before they hit the bookmaker.
- Phase 2 — sequential foreach over the (Event, Snapshot) tuples
captured in Phase 1, doing event upsert + snapshot insert. EF Core
DbContext is not thread-safe so all DB writes stay on a single thread.
* InfrastructureModule binds ScrapingThrottle alongside AnomalyOptions.
* Failed snapshot scrapes in Phase 1 mean the event row is also NOT
persisted in Phase 2 — previously we'd persist the row even when the
snapshot scrape failed, leaving an orphan event with no odds. Updated
the regression test accordingly.
* Test fixture exposes TestFixtures.Throttle(maxConcurrentRequests=1) for
deterministic sequential test runs.
* One existing NSubstitute setup that chained Arg.Is<>() across two
configurations was rewritten to use a single Arg.Any<>() with inline
branching — chained matchers were leaking and returning wrong results.
Implements Phase 8 Amendment 1: marathonbet.by has no public results archive
endpoint, so results must be harvested per-event by re-fetching the event
detail page until eventJsonInfo.matchIsComplete=true.
Backend changes:
* IOddsScraper:
- ScrapeResultsAsync(DateRange) replaced with ScrapeEventResultAsync(Event)
returning a nullable EventResult — null when match still in progress.
- ScrapeEventOddsAsync now takes the full Event (so EventPath drives URL
construction) instead of bare EventId.
- New ScrapeLiveAsync() for the /su/live listing.
* Domain:
- Event gains EventPath (nullable string) — the data-event-path attribute
captured during scraping; required for reliable URL construction.
* Infrastructure:
- New migration 20260506000000_AddEventPath adds the column.
- EventEntity / EventConfiguration / Mapping / model-snapshot updated.
- MarathonbetScraper: new ScrapeLiveAsync + ScrapeEventResultAsync; URL
builder prefers EventPath, falls back to numeric ID for legacy rows.
- EventListingParserBase extracts data-event-path on every listing row.
* Application:
- PullResultsUseCase: branches on selection vs date-range, emits IProgress<
PullResultsProgress>, returns ResultLoadOutcome (Loaded / AlreadyLoaded /
NotYetComplete / Failed); idempotent (skips events whose result already
exists).
- PullLiveOddsUseCase now drives off the live listing (auto-discovers
events that go live without ever appearing in the upcoming list) and
backfills EventPath on legacy rows.
- PullUpcomingEventsUseCase wires EventPath on persisted events.
* Workers: UpcomingEventsPoller updates persistence path accordingly.
* Tests: 17 net-new tests across Application + Infrastructure + Domain;
all 293 still pass.
Combined-batch reviewer flagged three real blockers + two test-infra
issues across the parallel P2/P3/P5 batch. All resolved:
PHASE 3 — DateTimeOffset UTC-kind constructor (3 sites)
EventListingParserBase.cs:39, EventOddsParser.cs:72, ResultsParser.cs:104
Replaced `new DateTimeOffset(DateTimeOffset.UtcNow.UtcDateTime, MoscowOffset)`
(throws ArgumentException because UtcDateTime has Kind=Utc) with
`DateTimeOffset.UtcNow.ToOffset(MoscowOffset)`.
PHASE 2 — EF string.Compare not translatable (3 sites)
EventRepository.cs:34, SnapshotRepository.cs:46, ExcelExporter.cs:35
Replaced `string.Compare(col, str, StringComparison.Ordinal)` with
`col.CompareTo(str)` so EF Core's SQLite provider can translate the
expression. Semantics unchanged (SQLite default collation = BINARY = ordinal).
PHASE 3 — ServerTimeProvider regex misses JSON-quoted key
Regex `serverTime\s*:\s*"..."` only matched bare-key form. Updated to
`"?serverTime"?\s*:\s*"..."` so the JSON-quoted form (the actual
marathonbet.by production format) is matched.
PHASE 3 — fixture: orphan <td> elements stripped by HTML5 parser
tests/.../Fixtures/marathonbet/event-football-sample.html — wrapped
the <td> blocks in a proper <table><tbody><tr> hierarchy so AngleSharp
preserves them and `td.Closest("td")` succeeds in the parser.
PHASE 2 — InMemoryDbFixture shared state across parallel tests
All fixture instances used `Data Source=marathon_tests` causing xUnit's
parallel-within-class runs to contaminate each other's data. Each fixture
now uses a Guid-suffixed unique data source name.
PLAN.md — P2/P3/P5 rows updated to ✅ Done with batch commit reference.
Test status:
Domain.Tests: 96/96 ✅
Application.Tests: 1/1 ✅
Infrastructure.Tests: 77/77 ✅
UI.Tests: 11/11 ✅
TOTAL: 185/185 ✅
Build: 0 warnings, 0 errors.
Deferred to later phases (per reviewer 🟡 / 🔵 notes):
- SnapshotRepository.GetAsync(Guid) uses lossy GetHashCode workaround;
Phase 4 to fix or remove from interface.
- Excel Sport name column writes string.Empty (need lookup join in Phase 6).
- PeriodScopeMapper football n>2 falls through to "Quarter" token;
guarded by MaxPeriods today, but defensive cleanup at Phase 9.
- Settings.razor duplicate m-rise-5 class on Localization section.
Three minimal fixes to make Marathon.sln build with 0/0:
1. Marathon.Infrastructure.csproj — add InternalsVisibleTo for
Marathon.Infrastructure.Tests so test code can reference internal
repository and exporter classes (Phase 2 issue blocking Phase 3 tests).
2. EventOddsParserTests.cs — add 'using Marathon.Domain.ValueObjects' so
MatchScope/PeriodScope resolve.
3. RoundTripTests.cs — add 'using Microsoft.EntityFrameworkCore' so the
ExecuteSqlRawAsync extension method on DatabaseFacade resolves.
Phase 5's anticipated LocalizationOptions / Serilog issues were already
resolved by its agent before being killed — no changes needed there.
Build status: 0 warnings, 0 errors.
Test status: Domain 96/96, UI 11/11, Infrastructure 42/77 (35 failing —
parser fixture issues + a real DateTimeOffset bug; reviewer will assess).
Creates the 9-project .NET 8 solution (5 src + 4 test) with Marathon.Domain
fully implemented: value objects (SportCode, EventId, OddsRate, OddsValue,
BetScope hierarchy), enums (Side, BetType, OddsSource, AnomalyKind), and
entities (Sport, Country, League, Event, Bet, OddsSnapshot, EventResult,
Anomaly) with all invariants enforced in constructors. 96 domain tests pass
(FluentAssertions + xUnit). Directory.Build.props and Directory.Packages.props
centralise build settings and NuGet versions. Both Marathon.sln and Marathon.slnx
are committed; dotnet build Marathon.sln succeeds with 0 warnings/errors.
alexei.dolgolyov