070e34b911
Anonymous scraping confirmed feasible for marathonbet.by — site is fully SSR (nginx), no Cloudflare or JS challenge. HttpClient + AngleSharp + Polly v8 is sufficient; Playwright not required (kept as a future-flag). Spike outputs: - spike/SCRAPE_FINDINGS.md — page rendering, URL templates, anti-bot, rate limits, recommended scraping strategy for Phase 3. - spike/SCHEMA_DRAFT.md — customer-spec field → DOM selector mapping for Match + Period-N scope across football/basketball/tennis (hockey TBD). Phase 1+ handoff captured in subplan + CLAUDE.md. Critical Phase 8 finding: no public results endpoint at /su/results — phase 8 must switch to polling event-detail until eventJsonInfo.matchIsComplete=true (deviation flagged). Reviewer notes addressed: - Period market outcome codes corrected to RN_H/RN_D/RN_A (not 1/draw/3) and market name vocabulary clarified per-sport in SCHEMA_DRAFT §3.1. - results-page.html capture added to file list with caveat about live-landing score-state and unsampled hockey selectors.
7.8 KiB
7.8 KiB
Feature Context: Initial Implementation
Configuration
- Development mode: Automated
- Execution mode: Orchestrator
- Strategy: Big Bang
- Build:
dotnet build Marathon.sln - Test:
dotnet test Marathon.sln - Lint:
dotnet format Marathon.sln --verify-no-changes - Run:
dotnet run --project src/Marathon.Hosts.WpfBlazor - Implementer models: Sonnet 4.6 (backend), Opus (frontend)
- Reviewer model: Sonnet 4.6
Customer Constraints
- Source: marathonbet.by — anonymous scraping (no login). ToS risk acknowledged by customer.
- Output: Excel files matching customer's wide-column spec (
Bet_Match_Win_1,Bet_Period-1_Win_Fora_2_Value, etc.) with date-range filenames. - Storage: customer accepted SQLite-with-Excel-export instead of Excel-as-database (decided 2026-05-05).
- UI tech: Blazor Hybrid (changed from initial WPF assumption — better for web migration).
- Locale: RU + EN.
- Scope: analyze-only initially; design
IBetPlacerextension point for future betting. - Configurability: every variable parameter (polling, concurrency, retry, UA, retention,
thresholds, locale) goes in
appsettings.json+ Settings UI page.
Current State
Repo just initialized. Single main commit with .gitignore + README.md + CLAUDE.md.
Working on feature/initial-implementation branch. No source code yet — Phase 0 starts
with scraping research, no implementation.
Temporary Workarounds
(none yet)
Cross-Phase Dependencies
- Phase 1 (Domain) is the foundation; all later phases reference domain types.
- Phase 2 (Storage) & Phase 3 (Scraping) depend only on Phase 1 — can run in parallel.
- Phase 4 (Application + Workers) depends on Phase 2 + Phase 3.
- Phase 5 (UI Shell) depends on Phase 1 only — can run in parallel with 2/3.
- Phase 6 (Event Browsing UI) depends on Phase 4 + Phase 5.
- Phase 7 (Anomaly) depends on Phase 4 (snapshot storage) + Phase 6 (UI patterns).
- Phase 8 (Results) depends on Phase 6.
- Phase 9 (Packaging) is final — runs full build + test suite.
Deferred Work
- Bet placing (explicit out-of-scope; design extension point only).
- Authenticated scraping (anonymous now;
IOddsScraperimpl is swappable). - Multi-bookmaker support (only marathonbet.by; abstraction allows future expansion).
- PostgreSQL backend (SQLite for now;
IRepository<T>abstraction allows swap).
Failed Approaches
- Public results / archive endpoint — does NOT exist. Tested
https://www.marathonbet.by/su/results,/su/results/,/su/results.htm— all return HTTP 404. No/archive,/historylinks anywhere in the public HTML either. Phase 8 deviation: the Results loader cannot back-fill from an archive — it must poll each event detail page untileventJsonInfo.matchIsComplete=trueand snapshotresultDescriptionat that moment. Phase 8 implementer must revise the subplan accordingly. - JSONP
/su/liveupdate/popular/endpoint — exposes only refresh signals ({"modified":[{"type":"refreshPage"}],"updated":<ts>}), not actual odds. Cannot be used as a JSON odds source. Use it only as a "something changed" hint to trigger a full event-detail re-scrape. - Anonymous WebSocket (STOMP) at
/su/websocket/endpointis documented ininitData.stompbut appears to require an authenticated session (PUNTER-SESSION-HASHcookie); we did not test it but the customer's anonymous scraping constraint makes it unsuitable anyway.
Review Findings Log
(populated by reviewers)
Phase Execution Log
| Phase | Agent | Model | Test Writer | Parallel | Notes |
|---|---|---|---|---|---|
| Phase 0 | phase-implementer | Opus | ⏭️ Skipped (research only) | — | ✅ Done 2026-05-05. Outputs: spike/SCRAPE_FINDINGS.md + spike/SCHEMA_DRAFT.md + 7 local fixtures. Anonymous scraping confirmed feasible; HttpClient+AngleSharp recommended; no Playwright needed; no public results page found (Phase 8 deviation noted). |
| Phase 1 | phase-implementer | Sonnet 4.6 | ⏭️ Skipped (Big Bang) | — | — |
| Phase 2 | phase-implementer | Sonnet 4.6 | ⏭️ Skipped (Big Bang) | ✅ With 3 + 5 | — |
| Phase 3 | phase-implementer | Sonnet 4.6 | ⏭️ Skipped (Big Bang) | ✅ With 2 + 5 | — |
| Phase 4 | phase-implementer | Sonnet 4.6 | ⏭️ Skipped (Big Bang) | — | — |
| Phase 5 | phase-implementer-frontend | Opus | ⏭️ Skipped (Big Bang) | ✅ With 2 + 3 | Uses frontend-design skill |
| Phase 6 | phase-implementer-frontend | Opus | ⏭️ Skipped (Big Bang) | — | Uses frontend-design skill |
| Phase 7 | phase-implementer (split if needed) | Sonnet/Opus | ⏭️ Skipped (Big Bang) | — | UI portion uses Opus |
| Phase 8 | phase-implementer (split if needed) | Sonnet/Opus | ⏭️ Skipped (Big Bang) | — | UI portion uses Opus |
| Phase 9 | phase-implementer | Sonnet 4.6 | ✅ Final phase tests | — | Full build + test enforced |
Environment & Runtime Notes
- Windows 10, PowerShell 5.1 default shell, Bash also available.
gitconfigured globally; remoteorigin=https://git.dolgolyov-family.by/alexei.dolgolyov/maraphon-app.git.- Note: home directory (
C:\Users\Alexei) is itself a git repo (likely accidental). The maraphon-app local.gitoverrides it for this directory tree. - .NET SDK assumed installed; if Phase 1 fails on
dotnet --version, install or document in CONTEXT.md.
Implementation Notes
Phase 0 (Scraping spike, 2026-05-05)
- Anonymous scraping is feasible from a non-Belarus IP. No Cloudflare, no JS
challenge, no UA filtering observed.
Server: nginx. Standard cookies only. - Site is fully SSR. All needed data (event grid, full odds, breadcrumbs, period markets) is in the raw HTML. No SPA hydration required.
- Recommended scraper stack: HttpClient + AngleSharp + Polly v8. Playwright is
not required for read-only scraping — keep it as an optional fallback flag
(
Scraping:UsePlaywright) for future-proofing only. - Polling cadence: site itself polls live updates every 3 s; for our analyzer, pre-match 30 s and live 5–10 s is sufficient.
- Rate-limit: 5 sequential requests at 1 req/s pacing all returned 200 in <1 s,
no throttling. Recommend default
RequestsPerSecond=1,MaxConcurrent=4. - Sport ID semantics: customer's "Sport_Code = 6" (Basketball) maps to
data-sport-treeId="6"in the breadcrumb-canonical sport listing (/su/betting/Basketball+-+6). Some sports also have a separate "category tree ID" used inside the live grouping (e.g., 45356 for Basketball-live) — ignore those, use only the canonical breadcrumb ID. - Selection key format:
<eventId>@<MarketName>{LineIndex?}.<Outcome>. The market name is sport-specific (Match_Result,1st_Half_Result,Total_Goals,Total_Points,Total_Games,To_Win_Match_With_Handicap, etc.). Total thresholds are encoded in the outcome (Under_3.5,Over_213.5). Handicap values are NOT in the key — they're in<span class="middle-simple">text. - Tennis has no Draw outcome — domain
Bet_Match_Drawmust be nullable. - Date display ambiguity: listing shows
HH:MM(today) orDD <ru-month> HH:MM(future). Anchor the parser oninitData.serverTime(Moscow TZ, formatYYYY,MM,DD,HH,MM,SS). - No public results page (
/su/results→ 404). Final scores are exposed only on the event detail page itself viaeventJsonInfoJSON (matchIsComplete,resultDescription). Phase 8 must poll until completion; cannot back-fill from an archive endpoint. - Probe environment: Windows 10 + curl, geo-routed as Poland (
countryCode: PL). Customer in Belarus may see slightly different KYC overlays — parser must be defensive (treat missing markets as null, never throw). - Captures saved locally at
spike/captures/*.html(gitignored): 7 fixtures for offline parser development in Phase 3.