15e5b186cd831a79b3d87a70c28f34d1d6c80be2
175 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 alexei.dolgolyov
|
15e5b186cd |
feat(secrets): scoped shared secrets rule-management UI (Phase 2)
Completes scoped shared secrets end-to-end: /shared-secrets list/new/edit routes (mirroring metric-alert-rules) with an env-key name, a WRITE-ONLY value (password input; never pre-filled — the API returns only has_value; omitted on PATCH to keep the stored secret, provided to rotate; cleared after save), an encrypted toggle (flipping it requires re-entering the value, matching the server's 400 guard), a global|app scope with an App-grouping picker (listApps), description, and enabled. 409 conflicts surface a friendly message. New "System" nav entry (IconKey) + api.ts client + full sharedsecrets.* i18n (en/ru parity). Reviewed: typescript APPROVE (0 CRITICAL/HIGH). |
||
|
alexei.dolgolyov
|
fa6d5bd3ba |
feat(secrets): scoped shared secrets — backend + API (Phase 1)
Secrets defined once and applied to many workloads by scope (global or per-app), encrypted at rest and resolved into container env as a low-precedence default layer: global-shared < app-shared < image cfg.Env < workload_env. A workload with no applicable shared secrets is byte-identical to the prior workload_env-only behavior. - store: shared_secrets table + CRUD + ListApplicableSharedSecrets (enabled global + app, global-first), UNIQUE(scope,app_id,name). - plugin.ResolveSharedSecrets + integration into BuildWorkloadEnv (static/dockerfile) and image buildEnv; best-effort — a shared-secret store/decrypt error never fails a deploy, and values are never logged. - REST CRUD at /api/shared-secrets (reads authed, mutations AdminOnly); values encrypted at the boundary via crypto.Encrypt and never returned (only a has_value flag), mirroring workload_env. UNIQUE collisions 409. Compose is out of scope (YAML-defined env). Frontend rule UI is Phase 2. Reviewed: go + security APPROVE (0 CRITICAL/HIGH); two MEDIUMs fixed (translateSQLError -> 409, no driver-message leak). Deferred defense-in- depth: json:"-" on the model value + a description length cap. |
||
|
alexei.dolgolyov
|
bd7a11d4e7 |
refactor(source): dedup shared helpers across static + dockerfile plugins
Extract the verbatim-duplicated helpers into shared homes: - buildEnv -> plugin.BuildWorkloadEnv (base plugin pkg; a sourceName param preserves each plugin's slog prefix / log-scraper text) - idShort -> plugin.IDShort - commitStatusReporter -> staticsite.CommitStatusReporter, re-parameterized on primitives (owner/repo/sha/targetURL/enabled) so staticsite needs no dependency on the plugin package; reporter tests ported to staticsite (plus a new nil-provider case) containerNameFor/imageTagFor are intentionally left per-plugin: their prefixes differ (dw-site- vs tf-build-) and name real Docker resources, so merging them would risk mis-routing. Behavior-preserving; the static/dockerfile test suites pass unchanged. Reviewed: go APPROVE (0 CRITICAL/HIGH). |
||
|
alexei.dolgolyov
|
7576f54e76 |
feat(alerts): metric-alert rule-management UI (Phase 2)
Completes metric-threshold alerting end-to-end: /metric-alert-rules list/new/edit routes (mirroring log-scan-rules) with metric/comparator/ threshold fields, the workload scope picker, ToggleSwitch, and a ConfirmDialog delete flow; an api.ts MetricAlertRule CRUD client; an "Observe" nav entry; and a full metricalert.* i18n namespace (en/ru parity). Create-form cooldown defaults to 300s to match the server. Rules are now manageable in the WebUI; breaches already surface in the per-app activity timeline and fire any configured event-trigger webhook. Reviewed: typescript APPROVE (0 CRITICAL/HIGH). |
||
|
alexei.dolgolyov
|
2e26f555c5 |
fix(dashboard): count Recent workloads by source_kind, not raw rows
The Recent-workloads badge and empty-state guard used workloads.length (which includes legacy kind:site rows with empty source_kind) while the list renders pluginWorkloads (source_kind != ''). With one legacy row and no real workloads the badge showed "1" over an empty list and the empty state never appeared. Both now use pluginWorkloads, matching the list and the headline Total count. |
||
|
alexei.dolgolyov
|
cdb9fd57d1 |
feat(alerts): metric-threshold alerting (backend + API)
Operators can define metric-threshold alert rules (cpu_percent, memory_percent, memory_bytes; gt/lt) per-workload or global via /api/metric-alert-rules. A periodic evaluator (internal/metricalert, 30s tick) checks the freshest container stats sample per container against enabled rules and, on breach (per-rule-per-workload cooldown), emits into the existing event_log + bus pipeline (source "metric_alert", workload_id set). Alerts therefore surface on the global events page, the per-app activity timeline, and any configured event-trigger webhook -- no new notification plumbing. Mirrors the log_scan_rules store/API/route patterns and the stats.Collector lifecycle. Rule CRUD reads are authed, mutations AdminOnly. Frontend rule-config UI is a follow-up phase. Reviewed: go APPROVE (0 CRITICAL/HIGH). |
||
|
alexei.dolgolyov
|
5c17885197 |
perf(reconciler): batch workloads per tick, drop redundant image inspect
Load every workload once per tick into a map instead of a per-container GetWorkloadByID (N+1) in the upsert loop plus a second ListWorkloads in the plugin pass: one query per tick, zero GetWorkloadByID. The ListWorkloads error path returns before the missing-sweep so a failed load can't flip live container rows to 'missing'. image.Reconcile is now a no-op: the generic upsert+markMissing pass already syncs every labeled container's state from the single ListAllForReconciler (docker ps -a) snapshot earlier in the same tick, so the former per-container IsContainerRunning loop was N redundant Docker calls/tick. (Its no-op body sits in image.go, which landed with the preceding commit; the tests are here.) compose/static reconcile do non-redundant work and are intentionally untouched. Reviewed: go APPROVE. |
||
|
alexei.dolgolyov
|
93b6911b34 |
feat(apps): per-app deploy/activity timeline
Every deploy across all four source kinds now writes a workload-scoped
event via a shared plugin.EmitDeployEvent helper (replacing the inline
emit duplicated in static/dockerfile, standardizing static's metadata
key site_id->workload_id, and adding emission to image+compose which
were silent). New indexed event_log.workload_id column, EventLogFilter
.WorkloadID, and GET /api/workloads/{id}/events (id pinned from path).
Frontend: a forge "Activity" panel on /apps/[id] reusing EventLogEntry,
live SSE prepend filtered by workload_id, load-more pagination, an
All/Errors severity filter, and a shared toEventLogEntry mapper. en/ru
i18n parity.
Security: compose's failure status emits a generic reason instead of raw
`docker compose up` output, which can echo app secrets and egresses to
operator webhooks (NotificationURL + event-trigger actions); full detail
stays only in the returned error. Rune-safe 256-rune status cap.
Reviewed: go + typescript APPROVE; security HIGH fixed.
|
||
|
alexei.dolgolyov
|
3071cda512 |
feat(deploy): commit-status reporting to Git providers
Report deploy status back to the Git provider as a commit status (pending/success/failure) for git-sourced workloads (static + dockerfile). - GitProvider.SetCommitStatus on gitea/github/gitlab over the existing SSRF-safe client; fixed "tinyforge" context so redeploys update one row. postJSON returns status-code-only errors (never echoes the upstream body, which a hostile provider could use to reflect the auth token into the best-effort log line). - Best-effort deploy hook: pending on deploy start, success/failure on outcome, gated on a per-workload report_commit_status flag. Never fails or blocks a deploy; emits nothing on the unchanged-SHA short-circuit. - UI ToggleSwitch (create + edit) + reportCommitStatus in sourceForms.ts + en/ru i18n. - Tests: per-provider state mapping + request shape; reporter gating (enabled/disabled/empty-SHA/nil/error-swallow). Reviewed via go-reviewer + security-reviewer (0 CRITICAL/HIGH; one MEDIUM body-echo log-leak fixed). |
||
|
alexei.dolgolyov
|
410a131cec |
feat(apps): stepped creation wizard, branch previews, and app-creation fixes
This session (frontend focus):
- Rebuild /apps/new as a 4-step wizard (Basics → Configure → Trigger → Review):
WizardRail, SourceKindPicker card grid, AppManifest review, per-step validation,
ConfirmDialog-based unsaved-changes guard.
- Extract lib/workload/sourceForms.ts (single source of truth for source_config)
+ {Image,Compose,Static,Dockerfile}SourceForm + StaticDiscoveryWizard; fold the
/apps/[id] edit form onto the same components (removes the duplication). Add
vitest + sourceForms unit tests.
- Branch preview environments UI: /chain is_preview/preview_branch + a Preview
environments panel on /apps/[id] (per-branch URLs, ConfirmDialog teardown, armed
state); RegistryImagePicker on the registry trigger and the image source.
- Fixes: image-inspect 404 -> admin-gated POST /api/discovery/image/inspect;
conflict-panel blur flicker; friendly localized discovery errors; CPU/Memory
label hints; dashboard + /apps "Total workloads" count only source_kind workloads
(drop stale trigger_kind gate); NPM cert/access-list name cache; EntityPicker
empty-list guard.
- Update CLAUDE.md frontend conventions + add a Build & Test section.
Also captures pre-existing in-progress platform work (not from this session):
workload notifications, Prometheus metrics export, store lockfile, health probes,
backup hardening, and related store/webhook/scheduler changes.
|
||
|
alexei.dolgolyov
|
956943edbb |
feat(proxies): per-row Triggers deep-link to /apps/[id]#bindings
The proxies page now exposes the trigger bindings for each routed
workload via a per-row action chip. Resolves the explicit "what's
next" call-out in WORKLOAD_REFACTOR_TODO under Priority 3 polish.
- Added id="bindings" to the existing trigger bindings <section> on
/apps/[id]/+page.svelte so URL fragments resolve to the panel.
- New triggersHref(route) helper in /proxies that builds
/apps/{workload_id}#bindings; code-pointer comment explains the
back-compat naming (ProxyRoute.project_id is actually the workload
ID — see internal/store/models.go:110-113), so a future contributor
doesn't trip on the mismatch and rip the helper out.
- New right-aligned "Actions" column with a button-shaped link;
defensive — falls back to — when project_id is absent.
- Three new i18n keys under proxies.* (actions, viewTriggers,
viewTriggersTitle) mirrored across EN + RU. Key parity now 1512
each.
No backend change needed; ListProxyRoutes already selects w.id into
ProxyRoute.project_id. Workload-aware batch endpoints (showing
trigger counts inline) were deliberately out of scope for this
half-turn — flagged as a future enhancement only if users want
inline counts.
Verification: svelte-check 0 errors + 3 pre-existing warnings in
TagCombobox; go build + go test ./... all green across 20 packages.
|
||
|
alexei.dolgolyov
|
279f373f80 |
docs(extra_json): policy doc for containers.extra_json evolution
New CODEMAPS/container-extra-json.md documents the contract every source plugin must follow when reading or writing containers.extra_json. Closes the open architectural question that was tracked in WORKLOAD_REFACTOR_TODO.md. Covers: - Schema position (column default, four write-path normalization sites) and ownership model (per-source row keys, current writers). - Reader rules: tolerate unknown keys via default json.Unmarshal, tolerate decode failure where first-class columns suffice. - Writer patterns: wholesale-overwrite (image source, single-writer short-lived rows) vs preserve-unknown-keys (static source, RMW with generic-map round-trip). Preserve-unknown-keys is the recommended default for new sources. - Concurrency: SetMaxOpenConns(1) + WAL gives atomic per-row writes and consistent reader snapshots, but does NOT serialize multi- goroutine RMW — a per-workload sync.Mutex is required for that (fenced by TestSaveState_ConcurrentWritesDoNotLoseUpdates). - What extra_json is NOT for (workload config, cross-source state, queryable data, secrets) and a checklist for adding a new field. - Pointers to every example in tree: image's containerExtra writer/ reader, static's saveState round-trip, workload_runtime.go's decode-and-tolerate consumer. WORKLOAD_REFACTOR_TODO Container.extra_json question flipped to DONE. CODEMAPS/INDEX bumped + entry linked. Reviewer pass (code-reviewer subagent) caught one HIGH factual error (wrong cross-source consumer claim) and several MEDIUM/LOW drifts; all addressed inline before commit. |
||
|
alexei.dolgolyov
|
ea55d31177 |
feat(discovery+runtime): restore static-site wizard discovery + close /sites/[id] feature parity
Build / build (push) Successful in 10m43s
Two-stage feature arc closing the gaps left by the hard legacy cutover.
The static-site creation wizard regains its auto-discovery + connection-test
flow; /apps/[id] grows the runtime/storage/lifecycle surface the legacy
/sites/[id] page used to expose.
Backend (Go)
- internal/api/discovery.go: six admin-gated endpoints wrapping
staticsite.GitProvider — POST /api/discovery/git/{detect-provider,
test-connection,repos,branches,tree} + GET /api/discovery/image/conflicts.
Identifier validation (validateGitIdent / validateGitBranch) at the
boundary so provider URL interpolation cannot be hijacked via `..`.
Upstream errors scrubbed: detailed slog on the server, generic 502 to
the client (mitigates token-reflection-in-error-page).
- internal/api/workload_runtime.go: four endpoints —
GET /api/workloads/{id}/runtime-state decodes containers.extra_json for
static workloads; GET /api/workloads/{id}/storage execs `du -sb /app/data`
with a 30s in-process cache (storageProbeCache) so polling can't turn
into per-request execs; POST /api/workloads/{id}/{stop,start} iterate
ListContainersByWorkload and call docker.StopContainer / StartContainer,
returning 200 / 409 (nothing to act on) / 502 (all failed).
- internal/staticsite/safehttp.go: NewSafeHTTPClient + ValidateBaseURL +
blockReason. DialContext re-resolves hostnames and refuses loopback /
link-local / multicast / unspecified addresses. RFC1918 + ULA explicitly
allowed (self-hosted Gitea on LAN is the dominant deployment).
Replaced four raw &http.Client{} constructions in the provider files.
- internal/staticsite/gitlab_provider.go: url.PathEscape each segment in
the raw-file URL builder for parity with projectPath().
- Test coverage: 26 cases in discovery_test.go (image-tag stripping,
source-config decoding, conflict scenarios, validator boundaries,
scheme rejection), 14 in workload_runtime_test.go (404 / 409 / nil-docker
/ probe-cache), 16 in safehttp_test.go (URL validation + block-reason
policy matrix + live dial against loopback + AWS metadata literals).
Frontend (Svelte 5 + runes)
- web/src/lib/api.ts: typed wrappers for every endpoint, AbortSignal
threaded through post(); ApiError exported so callers can narrow on
e.status; new DetectedGitProvider narrow union.
- web/src/routes/apps/new/+page.svelte: static-form discovery controls
(auto-detect provider, test connection, repo / branch / folder
EntityPickers, Deno auto-detect); image-form conflict panel with
debounced lookup + double-click submit guard ("Forge anyway") + Inspect
button that pre-fills port/healthcheck; English error fallbacks routed
through apps.new.errors.* (en + ru).
- web/src/routes/apps/[id]/+page.svelte: runtime-state panel + storage
panel + Stop / Start / Open-site toolbar; universal live-state badge
in the hero lede for image/compose/static (RUNNING / TRANSITIONING /
STOPPED / NOT DEPLOYED / MIXED · n/m RUNNING); ContainerStats panel
per row (auto-collapsing native <details> when N > 2); read-only
webhook bindings summary card; responsive toolbar overflow with native
<details> at <640px (z-index 100 above sticky nav).
- web/src/app.css: project-wide .forge-btn-ghost:focus-visible outline.
Hardening from go-reviewer + security-reviewer + typescript-reviewer +
frontend-design UI/UX subagents (0 CRITICAL, all HIGH/BLOCKER addressed
inline, IMPORTANT applied before commit):
- AbortController + per-call sequence tokens on every long-running
fetch (loadRuntimeState / loadStorage / loadTriggerMeta / inspectImage /
listImageConflicts) plus onDestroy cleanup so late resolves cannot
mutate dead component state.
- doStop / doStart snapshot and restore `error` across the finally-block
reload so a load()-cleared message doesn't hide a real failure.
- triggersById refreshed after inline trigger creation so the webhook
card doesn't silently exclude the just-created trigger.
- Live-state badge wraps in role=status / aria-live=polite (no redundant
aria-label).
- Webhook row has a single click target (was two pointing at the same URL).
- Empty webhook section hides entirely.
- Dropped role=menu / role=menuitem from the overflow menu (they would
promise arrow-key nav we don't wire; native Tab + ESC carry it).
Doc
- docs/CODEMAPS/INDEX.md + new docs/CODEMAPS/discovery-and-runtime.md
map the endpoint surface, security posture, frontend integration
patterns, and an "add a new probe" recipe.
Verification
- svelte-check: 0 errors, 3 pre-existing a11y warnings.
- go build + go vet + go test ./...: all green.
- i18n parity: en + ru at 1413 keys each.
- Live smoke against :8090: 404 / 409 / 502 envelopes correct, discovery
sanity passes, ProbeError surfaces on no-container path.
|
||
|
alexei.dolgolyov
|
ef62a41fc0 |
test(static-plugin): cover pure helpers, build helpers, and state/env paths
Build / build (push) Successful in 11m3s
Bring the previously-untested internal/workload/plugin/source/static/ package from 0% to 23.6% coverage with three new test files: helpers_test.go (20 cases) - idShort/containerNameFor/imageTagFor/ siteVolumeKey shape + same-name-workload collision avoidance; sanitizeError newline collapse, empty-token no-op, 240-byte cap, and multi-byte UTF-8 validity at the cap; containerRowID determinism; lockFor map semantics (same lock for same workload, distinct locks for different workloads, real serialization under contention, safe concurrent insertion); runtimeStateKeys exactly equals the JSON-tag key set. build_test.go (8 cases) - copyDir copies files + subdirs and preserves modes on Unix; verifyDownloadInsideRoot accepts clean trees and surfaces ErrNotExist for missing roots; both functions reject symlinks (skipped cleanly on Windows non-admin where the SeCreateSymbolicLink privilege is absent); prepareStaticBuild writes the Dockerfile even for an empty source. state_integration_test.go (12 cases) - loadState/saveState round- trip on an in-memory SQLite store, including: unknown extra_json keys (future writers) survive a save; clearing a typed field drops the key; malformed extra_json is recovered from rather than panicked on; concurrent writers exercise the per-workload mutex by accumulating into state.LastError - the test verified to fail loudly (15+ lost markers) when the mutex is disabled. buildEnv returns plain values, decrypts encrypted ones, skips rows that fail to decrypt without leaking ciphertext, and returns empty on store failure without panicking. Review followups from go-reviewer pass applied inline: H1 rewrite to exercise actual lost-update race (verified against disabled mutex), H2 workload-ID scoping by t.Name() so the package-global saveLocks map cannot bleed across tests or -count=N runs, set- based env-assertions, JSON tag-set equality check, multi-byte truncation case, valid-JSON-on-recovery assertion, unique-keys in concurrent map test, double-close cleanup. |
||
|
alexei.dolgolyov
|
5e78f13e06 |
refactor(triggers): review followups — fire-now, dedupe trigger pages, hardening
Build / build (push) Failing after 34s
Follow-ups on commit
|
||
|
alexei.dolgolyov
|
39e1e36510 |
feat(triggers): add schedule trigger kind + internal scheduler
Build / build (push) Successful in 10m42s
Fourth trigger kind alongside registry/git/manual. Recurring time-interval fires driven by a new internal/scheduler tick loop (default 30s, clamped to 5m). Goes through the same webhook.Handler.FanOutForTrigger seam as inbound HTTP webhooks, so per-binding concurrency, outcome accounting, and config-merge semantics are identical. Schema: triggers.last_fired_at TEXT column (additive ALTER for existing DBs). Scheduler persists last_fired_at BEFORE dispatch so a panicking Match cannot wedge a tight loop; failed deploys wait one full interval before retry — correct trade-off for a periodic refresh trigger. Frontend: TriggerKindForm + /triggers/new + /triggers/[id] gain the schedule kind (4-col card grid, preset chips Hourly/Daily/Weekly, custom interval input matched to Go time.ParseDuration syntax, optional pinned reference). /triggers/[id] surfaces "last fired" on schedule rows. EN+RU i18n in parity. Review fixes from go-reviewer / security-reviewer / typescript-reviewer: - Scheduler Start/Stop wrapped in sync.Once (no goroutine leak / double- cancel panic on shutdown re-entry). - shouldFire rejects sub-MinInterval as defense-in-depth against hand-inserted rows that bypassed Validate. - fire() asserts trigger Kind=="schedule" before dispatching. - Aligned isValidInterval regex across all three frontend sites; reject the unsupported "d" unit (Go time.ParseDuration doesn't accept it). - formatLastFired falls back to lastFiredNever on malformed timestamps rather than leaking raw bytes into the UI. - main.go scheduler closure logs per-fire deployed/errored counts. |
||
|
alexei.dolgolyov
|
e3c7b13d58 |
chore(workload): close the workload-first arc — apps i18n + codemap + tests
Build / build (push) Successful in 10m36s
Closes the workload-first refactor by landing the Priority 3 polish items and the Priority 4 test gap. Net: ~2,400 lines added, ~350 lines modified across 13 files. Priority 3 — polish - apps.* i18n namespace: 276 new keys across apps.list.* (27), apps.new.* (91, sibling of existing apps.new.triggers.*), and apps.detail.* (158, sibling of existing apps.detail.bindings.*). EN+RU at 1314 keys each, perfectly in sync. /apps, /apps/new, /apps/[id] now render entirely from i18n. - New codemap docs/CODEMAPS/workload-plugin.md (238 lines): Source × Trigger contract, dispatch seam, webhook fan-out path, recipes for adding a new Source or Trigger kind. Plus docs/CODEMAPS/INDEX.md gateway. Priority 4 — tests - internal/api/workloads_test.go (new, ~30 subtests): /api/workloads CRUD + deploy + delete + env + volumes + chain + promote-from + triggers list/inline-bind + auth gating + standalone /api/triggers CRUD (create / dup-409 / kind filter / delete). Uses real POST handlers via httptest.NewServer + a fake plugin source registered under "testfakesource". - internal/deployer/dispatch_test.go (new, 11 tests): DispatchPlugin / DispatchTeardown / DispatchReconcile happy + unknown-kind + propagated-error each; PluginDeps wiring; a real 2s-bounded RWMutex deadlock probe on PluginDeps vs SetDNSProvider. - internal/workload/plugin/source/compose/compose_test.go (new, ~26 subtests): composeProjectName sanitization, writeYAML / writeYAMLIfChanged hash short-circuit, Validate happy + bad inputs, Kind / SchemaSample. Coverage delta on the workload-plugin path: - internal/api: 1.1% → 16.0% - internal/deployer: 0% → 54.1% - internal/workload/plugin/source/compose: 0% → 38.5% - Trigger plugins already at 87-95% from the trigger-split work. Production fix surfaced by the tests - store.CreateWorkload now self-references RefID = ID when caller leaves RefID empty (the typical plugin-native path). The api layer's broken backfill loop (called UpdateWorkload, which deliberately omits ref_id) is gone. Multiple sibling plugin workloads can now coexist under the UNIQUE(kind, ref_id) constraint. Review fixes addressed before commit - CRITICAL: deadlock-detect test gained a real 2s time.After (was selecting on context.Background().Done() which never fires). - HIGH: happy-path test now hard-asserts RefID = ID (was a t.Logf that would silently pass after a production fix). - HIGH: standalone /api/triggers CRUD coverage added (was bypassed by the workload-side bind flow). - HIGH: seedWorkload bypass deleted; tests now go through the real POST /api/workloads handler. - MEDIUM: withTempDir restore is a no-op (t.Setenv auto-restores); dead `old := os.Getenv(...)` capture removed. - MEDIUM: list-workloads test now asserts ID membership, not just count. Doc - WORKLOAD_REFACTOR_TODO: all three Priority 1 items, Priority 3 polish, and Priority 4 tests marked DONE. The workload-first arc is closed. |
||
|
alexei.dolgolyov
|
739b67856a |
feat(cutover): hard legacy cutover — drop projects/stacks/sites/deploys
Build / build (push) Successful in 10m39s
The clean-break delete that closes the workload-first refactor arc.
Net diff: ~30 backend files deleted, ~20 modified, ~12k LOC removed
on the Go side; entire /projects /stacks /sites /deploy frontend
trees gone; ~6.7k LOC removed on the Svelte/TypeScript side.
Backend
- API handlers gone: internal/api/{projects,stages,stage_env,stacks,
static_sites,deploys,instances,volume_browser}.go
- Store CRUD + tests gone: internal/store/{projects,stages,stage_env,
stacks,static_sites,static_site_secrets,deploys,poll_state,volumes,
workload_sync}.go (+ _test.go siblings)
- Legacy deployer pipeline gone: internal/deployer/{bluegreen,promote,
rollback,subdomain,resolver_test}.go; deployer.go trimmed to just the
dispatch surface used by the plugin pipeline
- internal/staticsite/{manager,healthcheck}.go and
internal/stack/manager.go gone (the rest of those packages stay as
helpers imported by the static + compose plugins)
- internal/registry/poller.go gone (legacy registry poller)
- internal/volume.ResolvePath gone; ResolveWorkloadPath stays
- internal/webhook: handleWebhook (project) + handleSiteWebhook (site)
gone; only POST /api/webhook/triggers/{secret} remains
- workload-side webhook URL handlers (getWorkloadWebhook +
regenerateWorkloadWebhook + EnsureWorkloadWebhookSecret +
SetWorkloadWebhookSecret + GetWorkloadByWebhookSecret) gone — they
minted URLs that would 404 against the new trigger-only ingress
- cmd/server/main.go: dropped staticsite.Manager, stack.Manager,
staticsite.HealthChecker, registry poller, SetSiteSyncTriggerer,
SetStaticSiteManager, SetStackManager, wireStaticBackend
- store/store.go: idempotent DROP TABLE IF EXISTS for every legacy
table (projects, stages, stage_env, volumes, deploys, deploy_logs,
poll_states, stacks, stack_revisions, stack_deploys, static_sites,
static_site_secrets); FK order children-then-parents
- store/models.go: dropped Project, Stage, Deploy, DeployLog, StageEnv,
Volume, StaticSite, StaticSiteSecret, Stack, StackRevision,
StackDeploy types; kept WorkloadKind constants as documented strings
- internal/store/helpers.go (new): BoolToInt, rowScanner,
GenerateWebhookSecret extracted from deleted CRUD files
- internal/api/secrets.go (new): forwards to store.GenerateWebhookSecret
so api + store paths share one secret-generation impl (no
panic-vs-UUID-fallback divergence)
- internal/reconciler/reconciler.go: dropped legacy stack-by-compose
+ static-site label paths; only canonical tinyforge.workload.id
dispatch remains
- providers (gitea_content/github_provider/gitlab_provider) gained
path-traversal rejection on every tree entry
- internal/webhook ParsedImage / ParseImageRef demoted to package-
private (no external callers)
Frontend
- /projects /stacks /sites /deploy routes deleted (entire trees)
- ProjectCard / InstanceCard / StaleContainerCard components deleted
- api.ts: dropped every project/stage/stack/site/deploy/instance
helper + types (Project, Stage, Stack, StaticSite, Deploy,
Instance, Volume, etc.); kept Workload, Container, App, Settings,
Registry, EventTrigger, LogScanRule, webhook envelopes
- WorkloadWebhook type + getWorkloadWebhook/regenerateWorkloadWebhook
api functions gone (mirror of the backend deletion above)
- web/src/routes/+layout.svelte: dropped /projects /sites /stacks
/deploy nav entries, trimmed quick-nav keymap
- web/src/routes/+page.svelte: dashboard rewrite — reads
listWorkloads + listContainers only; 4-card stat grid
(workloads/running/failed/stale) + recent workloads strip
- navCounts.ts, SystemHealthCard.svelte, ContainerLogs.svelte,
ContainerStats.svelte, StatusBadge.svelte, TagCombobox.svelte,
proxies/+page.svelte, containers/+page.svelte all rewired to the
workload-first surface
- AbortController plumbing on dashboard, nav-counts, stale page,
SystemHealthCard so navigation doesn't leave dangling fetches
- i18n: dropped projects.*, projectDetail.*, envEditor.*,
volumeEditor.*, volumeBrowser.*, quickDeploy.*, sites.*, stacks.*,
instance.*, confirm.* namespaces; en/ru parity preserved (1042
keys each)
Hardening from go-reviewer + security-reviewer + typescript-reviewer
subagent passes (0 CRITICAL across all three; 1 HIGH + ~12 MEDIUM
addressed inline before commit):
- Sec H1: dead-end workload webhook URL handlers (would mint URLs
that 404 the new trigger-only ingress) deleted across backend +
frontend
- Go M1: IsTerminalDeployStatus dropped (no production callers)
- Go M2: ParsedImage/ParseImageRef lowercased (in-package only)
- Go M6: generateWebhookSecret unified — api shim forwards to
store.GenerateWebhookSecret
- Doc/comment freshness: stage_id (no longer FK), ProxyRoute legacy
field names, workloadIDRow rationale, webhook_deliveries.target_type
enum, WebhookDeliveryLog component header
Doc
- WORKLOAD_REFACTOR_TODO: cutover marked DONE; all three Priority 1
items are now shipped. Next focus is Priority 3 polish (apps.* i18n
+ codemap entries) and Priority 4 tests.
Behavioral notes for operators upgrading from a pre-cutover build
- Existing rows in the dropped tables disappear on first boot.
- Legacy webhook URLs at /api/webhook/{secret} and
/api/webhook/sites/{secret} return 404; CI configs must repoint to
/api/webhook/triggers/{secret} (the trigger-split boot backfill
lifted any embedded workload secret onto a Trigger row, so the
secret value itself carries over).
- Frontend routes /projects /stacks /sites /deploy are gone; nav
links replaced with /apps and /triggers.
|
||
|
alexei.dolgolyov
|
234c3c711e |
feat(static): inline static-source plugin; drop phantom-row adapter
Build / build (push) Successful in 10m43s
Lift the static-site deploy pipeline from internal/staticsite/manager.go into internal/workload/plugin/source/static/ so plugin-native static workloads operate directly on plugin.Workload + the containers table + workload_env. The cmd/server/static_backend.go phantom-row adapter is gone; the legacy static_sites table is no longer touched on plugin deploys. Backend - new state.go: runtimeState (last_commit_sha, last_sync_at, last_error, status) persisted in containers.extra_json under the deterministic row id <workloadID>:site - per-workload sync.Mutex serializes saveState read-modify-write so parallel deploys for the same workload can't race container_id / proxy_route_id writes - extra_json round-trips through map[string]json.RawMessage so unknown keys survive — typed runtimeStateKeys are stripped before merge so clearing a typed field actually drops the key - new env.go reads workload_env (replaces static_site_secrets for plugin-native sites); decrypt-failure logs and skips one entry rather than failing the whole deploy - new build.go ports prepareDenoBuild + prepareStaticBuild + copyDir; copyDir uses filepath.WalkDir + Lstat to refuse symlinks and non-regular files - new deploy.go is the ~300-line core; intent.Reason gates force vs skip-if-no-changes; success-path saveState failure rolls back container + proxy route and writes "failed" state (no orphans) - new teardown.go combines Remove + Stop; idempotent on never-deployed workloads - new reconcile.go refreshes container state from Docker; flips runtimeState.Status to failed when the container is missing/crashed Hardening (from go-reviewer + security-reviewer subagent passes; 1 CRITICAL + 5 HIGH + 3 MEDIUM addressed before merge) - path-traversal defense in all 3 providers (gitea_content, github_provider, gitlab_provider): reject tree entries whose resolved local path escapes destDir - verifyDownloadInsideRoot walks the build dir post-download as a second line of defense - sanitizeError redacts the access token, collapses to one line, and clamps to 240 bytes before persisting to extra_json or fanning out to the notification webhook - container/image/volume names suffixed with workload-id short prefix (workload name is not UNIQUE in schema) - primaryDomain reads settings.Domain to complete a bare subdomain face into a full FQDN (matches legacy Manager behavior) - ctx-aware health-check sleep - json.Marshal for event metadata (was fmt.Sprintf JSON template) - strings.HasPrefix for failed-status detection (was brittle slice expression) Wire-up - cmd/server/main.go: removed wireStaticBackend(...) call; existing blank import on _ ".../source/static" drives init() registration - cmd/server/static_backend.go deleted Doc - WORKLOAD_REFACTOR_TODO: static port marked DONE; next focus is the hard legacy cutover (drop /api/projects, /api/stacks, /api/sites, /api/stages + their tables, internal/stack + internal/staticsite packages, frontend /projects /stacks /sites) Behavior notes for operators - plugin-native static workloads no longer write to static_sites; legacy /api/sites/* still serves original rows unchanged - legacy tinyforge.static-site / .static-site-name container labels dropped on plugin deploys; canonical tinyforge.workload.id / .kind cover ownership - container/image/volume names gained an 8-char ID suffix (e.g. dw-site-mysite-a1b2c3d4); legacy-deployed sites keep the old shape until redeployed through the plugin path |
||
|
alexei.dolgolyov
|
2aff22f565 |
feat(triggers): first-class triggers + bindings with fan-out webhook
Build / build (push) Successful in 10m39s
Promote triggers from embedded workload fields to standalone records
joined to workloads via workload_trigger_bindings. One trigger (webhook,
registry watcher, git push, manual) now fans out to many workloads with
per-binding config overrides (top-level JSON merge, binding wins).
Backend
- new triggers + workload_trigger_bindings tables with ON DELETE CASCADE
- boot-time backfill of embedded trigger config inside per-workload tx
- store.ErrUnique sentinel translates SQLite UNIQUE at store boundary
- /api/triggers CRUD + /api/triggers/{id}/{webhook,bindings}
- /api/bindings/{id} update/delete; /api/workloads/{id}/triggers list+bind
- bindTriggerToWorkload accepts trigger_id or inline {kind,name,config}
- inline-create uses CreateTriggerWithBindingTx (no orphan triggers)
- validateBindingConfig enforces 8 KiB cap + plugin Validate on merged
- ListTriggersWithBindingCount + ListBindings*WithNames remove N+1
- POST /api/webhook/triggers/{secret} resolves trigger then fans out
- bounded worker pool (4) per request; per-binding error isolation
- outcome accounting: deployed / skipped / no-match / errored
- legacy /api/webhook/workloads/{secret} route removed (clean break;
backfill keeps secrets resolvable at the new /triggers/{secret} path)
- reconciler gate dropped from (Source && Trigger) to Source only
- MergeJSONConfig returns freshly allocated slices (no fan-out aliasing)
- WithEffectiveTrigger lets existing Trigger.Match contract stay unchanged
Frontend
- /triggers list, new wizard, [id] detail (bindings, webhook rotate)
- workload create wizard: NEW / PICK / SKIP trigger modes
- workload detail: bindings panel + Add-trigger modal (inline / pick)
- per-binding override editor with merged-preview + 8 KiB guard
- "OVERRIDES n FIELDS" row badge when binding_config is non-empty
- shared TriggerKindForm component (registry / git / manual + JSON)
- 3 raw <input type=checkbox> replaced with <ToggleSwitch>
- full EN + RU i18n: redeployTriggers.*, apps.detail.bindings.*,
apps.new.triggers.*, nav.triggers; event-triggers nav disambiguated
Doc
- WORKLOAD_REFACTOR_TODO: trigger-split marked DONE; next focus is
the static-source inline port + hard legacy cutover (Priority 1)
|
||
|
alexei.dolgolyov
|
30133bc1eb |
docs: workload refactor + observability progress
Build / build (push) Successful in 10m40s
Two design + handoff docs: - docs/WORKLOAD_REFACTOR_TODO.md — status-at-a-glance table showing what's done (volume scopes, kind-aware editors, vendor webhook parsing, chain-panel CSS, Log Rules panel) and what's still pending (static source inline port + the hard legacy cutover gated on it; codemap entries; /apps page-level i18n; Priority 4 integration tests). - docs/LOGSCAN_AND_TRIGGERS_TODO.md — companion design + status doc for the two Observability features. Records the loop-prevention invariant (event_log = system observing itself, webhook_deliveries = system talking to outside) so the next contributor doesn't accidentally break it by adding a new EventLog subscriber that re-publishes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
alexei.dolgolyov
|
4707db1c3b |
feat(observability): event-triggers + log-scan-rules UI + i18n
Operator-facing surfaces for the two backend features:
- /event-triggers — list (filter summary, status pill),
/event-triggers/new (form with regex validation), and
/event-triggers/[id] (edit + Send-test + delete) with
CONFIGURED secret badge + clear-to-rotate flow, ConfirmDialog
for delete, aria-live regions on async result slots.
- /log-scan-rules — list with scope filter chips and stats panel
(active tails, RATE-LIMITED, COOLED DOWN, COMPILE ERRORS),
/log-scan-rules/new (with EntityPicker for workload scope and
inline RegexTestBox), /log-scan-rules/[id] (edit + server-side
/test + delete + live RegexTestBox panel).
- web/src/lib/components/RegexTestBox.svelte — reusable
client-side regex test with sample input + captures display.
- web/src/lib/api.ts — typed wrappers for EventTrigger and
LogScanRule CRUD + /test + getLogScanStats +
getEffectiveLogScanRules.
- web/src/routes/+layout.svelte — nav entries for both surfaces.
- web/src/lib/i18n/{en,ru}.json — ~90 keys under observability.*,
triggers.*, logscan.* namespaces; Russian translations cover
every key.
Design + a11y polish per a frontend-design review pass: all
boolean inputs use ToggleSwitch, all destructive actions use
ConfirmDialog with confirmVariant="danger" / onconfirm /
oncancel, hand-rolled .btn-primary replaced with global
forge-btn classes, hex colors replaced with var(--*) tokens,
role="alert" on error banners, aria-invalid + aria-describedby
on invalid-regex inputs, aria-busy on async forms, mobile
breakpoints (hide-md columns, .row.three collapsing 3→2→1,
.table-wrap overflow-x).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
alexei.dolgolyov
|
7a9ff7ad54 |
feat(observability): event triggers + log scanner backend
Two paired backends sharing the events.Bus seam:
Event triggers (consumer-side):
- internal/store/event_triggers.go — CRUD with action_secret
redaction on read (placeholder echo treated as "no change" on
PATCH so secrets aren't accidentally wiped).
- internal/events/dispatcher.go — bus subscriber, AND-composed
filters (severity CSV, source CSV, message regex with memoized
compile cache). Structural loop-prevention: never writes to
event_log. Sends via notifier.SendPayload.
- internal/notify: SendPayload + SendSyncForTestPayload methods,
TierEventTrigger constant, doSendRaw shared with the legacy
Event-shaped path.
- internal/api/event_triggers.go — admin-gated CRUD + /test
sending the real TriggerWebhookPayload shape. SSRF guard
rejects loopback / link-local / unspecified targets. PATCH
uses pointer-typed DTO for partial updates.
Log scanner (producer-side):
- internal/logscanner/ — engine (per-rule cooldown +
per-container token bucket, atomic drop counters), tail
(multiplexed docker frame demuxer with TTY fallback + 16 MiB
payload cap + 1 MiB reassembly cap + RFC3339Nano-validated
timestamp strip + UTF-8-safe message truncation), manager
(5s container polling, atomic.Pointer[Snapshot] hot-reload,
HitEmitter writes event_log + publishes EventLog so the
trigger dispatcher picks them up immediately).
- internal/docker/container.go — ContainerLogsOpts exposes
stream selection for stderr-only / stdout-only rules.
- internal/store: log_scan_rules table + CRUD with
EffectiveLogScanRules resolver (globals minus per-workload
overrides plus workload-only additions). Transactional
cascade-delete of overrides when a global rule is removed.
- internal/api/log_scan_rules.go — admin-gated CRUD + /test
(sample_line → matched/captures) + /stats (drop counters +
active tail count + last-snapshot compile errors) +
GET /api/workloads/{id}/effective-rules.
cmd/server/main.go wires both subsystems next to the existing
RegisterPersistentLogger. Coverage spans engine cooldown / bucket
counter tests, snapshot effective-set semantics, manager compile-
error capture, dispatcher matching, store validation +
cascade-delete, API URL validator + secret redaction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
alexei.dolgolyov
|
82d32181ba |
feat(webhook): vendor-specific event parsing (Gitea / GitHub / GitLab)
The /api/webhook/workloads/{secret} ingress now short-circuits on a
recognized X-*-Event header before falling back to the generic
simple-body parser. Vendor parsers populate fields the generic
parser cannot (image digest, GitEvent.Vendor, registry host).
internal/webhook/vendor_parsers.go covers:
- Gitea package events (X-Gitea-Event: package, container type)
- GitHub registry_package + package events (CONTAINER package_type)
- GitHub / Gitea push events with vendor stamping
- GitLab Push Hook + Tag Push Hook with path_with_namespace mapping
When a vendor parser claims a request (ok=true), it's authoritative
— a malformed Gitea package payload surfaces as an error rather
than silently re-parsing as generic. The generic {image} /
{ref + repository.full_name} fallback stays in place for legacy
CIs that send those shapes.
Coverage: internal/webhook/vendor_parsers_test.go +
inbound_event_test.go (round-trip through buildInboundEvent).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
alexei.dolgolyov
|
8d6a527a2b |
refactor(workload): plugin architecture wave + apps UI + volume scopes
Completes the workload-first refactor's plugin layer:
- internal/workload/plugin/ — Source/Trigger plugin contract,
registry, types (Workload, DeploymentIntent, InboundEvent,
PublicFace). Self-registering init() pattern + blank-import
in cmd/server/main.go.
- Source plugins: image (blue-green with multi-face proxy routing),
compose, static. Trigger plugins: registry, git, manual.
- internal/deployer/dispatch.go — DispatchPlugin/Teardown/Reconcile
seam routing the legacy deployer through plugins.
- internal/api/workload_*.go — REST surface: workloads, env,
volumes, chain (parent/children), promote-from. hooks.go
serves /api/hooks/kinds/{kind}/schema for the wizard.
- internal/store: workload_env (encrypt-at-rest secrets) and
workload_volumes tables, keyed on workload_id.
- cmd/server/static_backend.go — phantom-row adapter delegating
the static source plugin to the legacy staticsite.Manager
(deleted at hard cutover once the static inline port lands).
- web/src/routes/apps/ — /apps list + /apps/new wizard +
/apps/[id] detail with kind-aware compose / image / static
forms (Advanced JSON toggle), env panel, volumes panel,
webhook panel, chain panel, manual deploy.
Volume scope generalization (v2 resolver):
- internal/volume.ResolveWorkloadPath (workload-keyed, sits
next to legacy ResolvePath). Honors all VolumeScope values:
absolute, ephemeral, instance, stage, project, project_named,
named. internal/workload/plugin/source/image/image.go
computeMounts wires settings + imageTag through. Coverage in
internal/volume/resolver_test.go (portable Linux/Windows via
t.TempDir).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
alexei.dolgolyov
|
f42b21a2b9 |
fix(sites): show Secrets card before Resources on site detail
Build / build (push) Successful in 10m30s
Secrets is the more frequently edited section than Resources/logs; reorder so it appears right after Site Info on /sites/[id]. |
||
|
alexei.dolgolyov
|
cba2149aa9 |
refactor(workload): finalize containers index + post-review hardening
Wraps up the workload refactor with the fixes that came out of the multi-agent code review (see docs/plans/workload-refactor.md "What actually shipped"). Backend: - store.ReconcileContainer: separate write path so the 30s reconciler tick no longer overwrites deployer-owned fields (subdomain, proxy_route_id, npm_proxy_id, image_tag). - Container.stage_id column + index; ListProxyRoutes / ListContainersByStageID join via stage_id (survives stage rename), with legacy fallback to (project_id, role=stage_name). - Reconciler: workload-existence check (rejects forged tinyforge.workload.id labels), skips inventing project-kind rows, child-context cancel before wg.Wait() on shutdown. - Transactional CRUD across projects / stacks / static_sites: parent UPDATE and workload sync land in one transaction so secret rotations are durable. - Webhook routing reads exclusively through workloads.webhook_secret; legacy GetProjectByWebhookSecret / GetStaticSiteByWebhookSecret fallback removed. - store.GetStackByComposeProjectName + indexed lookup (no more full-table stack scan per compose container per tick). - store.ListMissingSweepRows: filtered query for the missing-sweep. - /api/instances/* handlers verify (workload_id, role) match URL (project_id, stage_name) before mutating — closes the cross-project hijack the security review flagged. - extra_json no longer referenced from Go (column kept on disk for now). Frontend: - WorkloadContainers.svelte: generic detail-page panel reusable by stack and site detail pages. - Containers page polish: client-side kind/state filters over an unfiltered fetch, URL-synced filters, race-safe loads via sequence number, EN+RU i18n, sidebar counter via navCounts.containers. Misc: - scripts/dev-server.sh: tolerate empty netstat grep result. - .gitignore: ignore docker-watcher binaries, .claude/worktrees/, .facts-sync.json. |
||
|
alexei.dolgolyov
|
d8ab22876f |
refactor(workload): extract Instance entirely; Container is canonical
Build / build (push) Successful in 10m41s
End-to-end extraction of the Instance concept. After this commit:
* internal/store/instances.go — DELETED
* internal/store/models.go — Instance struct gone, ProxyRoute moved here
* containers table is the single source of truth for project/stack/site
container state. instances table is dropped via DROP TABLE migration
(idempotent; re-runnable on every boot).
* Legacy tinyforge.project / tinyforge.stage / tinyforge.instance-id
Docker labels are no longer emitted; only tinyforge.workload.{id,kind},
tinyforge.role, and tinyforge.managed are stamped on new containers.
Backend rewrites:
- internal/deployer: executeDeploy + blueGreenDeploy + rollback +
promote use store.Container natively. New
removeContainer() replaces removeInstance().
enforceMaxInstances reads via
ListContainersByStageID.
- internal/reconciler: legacy tinyforge.instance-id dispatch removed;
upsertByWorkloadLabel now finds existing rows
by docker container ID first and falls back to
the deterministic workloadID:role key.
- internal/stale/scanner: Scan + new FindStaleContainers walk the
containers table; emit StaleContainer JSON.
- internal/stats/collector: ListContainers replaces ListAllInstances.
- internal/webhook/handler: workload-secret lookup tried first; falls back
to project / static_site secret column.
- internal/api: instances.go, stale.go, stats.go, stats_history.go,
projects.go, settings.go, docker.go, dns.go all read /
write through Container.
Docker layer:
- ManagedContainer exposes WorkloadID/Kind/Role from the canonical labels.
- ListContainers filters by tinyforge.managed=true.
- Network creation uses LabelManaged instead of LabelProject.
Frontend:
- Instance type is now a Container alias; .status → .state,
.last_alive_at → .last_seen_at.
- InstanceCard takes stageId as a prop (no longer derived from Instance).
- StaleContainer JSON shape rewritten: { container, workload_name, role,
days_stale }. StaleContainerCard + /containers/stale page updated.
- ProjectCard / homepage / SystemHealthCard filter by .state.
The migration loop now tolerates "no such table" alongside "duplicate
column" / "already exists" so obsolete ALTER TABLE entries targeting the
dropped instances table no-op cleanly on first boot.
Tests: store + deployer + reconciler + webhook + staticsite + notify all
still pass. Frontend svelte-check: zero errors.
|
||
|
alexei.dolgolyov
|
d516462750 |
feat(workload): switch ListProxyRoutes to containers index
The Proxies page consumer (and the secondary callers in internal/api/health.go and internal/api/settings.go) now read from the normalized containers index instead of the instances table. Stage ID is recovered through a (project_id, role=stage_name) join — uniquely-indexed via the existing UNIQUE(project_id, name) constraint on stages. Source field stays "instance" for back-compat with the Proxies page filter (the frontend keys off the literal string). Three new tests pin the join shape, verify the npm_proxy_id-only WHERE branch survives, and check that an orphan-role row falls out of the join cleanly (catches a regression to LEFT JOIN). |
||
|
alexei.dolgolyov
|
3e28588f10 |
feat(workload): global Containers tab + frontend client
Adds the user-visible piece of the Workload refactor:
- web/src/lib/types.ts — Workload, Container, ContainerView,
App, WorkloadKind, ContainerState
- web/src/lib/api.ts — listWorkloads, getWorkload,
listWorkloadContainers, setWorkloadAppID,
listContainers (with filter),
CRUD for apps
- web/src/lib/i18n/{en,ru}.json — nav.containers
- web/src/routes/+layout.svelte — "Containers" nav item between Stacks
and Deploy, IconContainer
- web/src/routes/containers/+page.svelte — global Containers table:
* filter chips for kind (project/stack/site) and state
* client-side search across workload name / role / image /
subdomain / container ID prefix
* Workload column links to the kind-specific detail page,
resolved through a one-time /api/workloads call to map
workload_id → ref_id
* existing /containers/stale route untouched
The page renders against the live database now — boot backfill
populated workload rows from existing projects/stacks/sites,
the deployer dual-writes containers on every deploy, and the
30s reconciler keeps the index in sync with `docker ps`.
|
||
|
alexei.dolgolyov
|
0acbcda084 |
feat(workload): /api/workloads /api/containers /api/apps endpoints
Adds the read API surface that the global Containers view (and
the per-workload container panel on project/stack/site detail
pages) consume.
- GET /api/workloads (?kind=) → workload list
- GET /api/workloads/{id} → single workload
- GET /api/workloads/{id}/containers → workload's containers
- PATCH /api/workloads/{id}/app → assign/clear app_id (admin)
- GET /api/containers (?workload_id=&kind=&state=&app_id=)
→ global index, decorated
with workload + app name
so the table renders
without N+1 fetches
- GET /api/containers/{id} → single container row
- GET /api/apps → list
- GET /api/apps/{id} → single
- POST /api/apps → create (admin)
- PUT /api/apps/{id} → update (admin)
- DELETE /api/apps/{id} → delete (admin) — clears
app_id on owning workloads
but leaves them assigned-to-none
Mutations on projects/stacks/sites still go through the existing
kind-specific endpoints; the new surface is read-only at the
workload layer.
|
||
|
alexei.dolgolyov
|
7f2d1bdae1 |
feat(workload): switch buildActiveImagesSet to containers index
First consumer migration off the instances table. The image prune logic now walks the normalized containers.image_ref column directly — one DB pass against a single table instead of joining instances against projects to reconstruct the full "image:tag" string. Demonstrates the consumer-switch pattern the remaining read sites (proxies, stale scanner, webhook matcher) will follow. The legacy `projects []store.Project` parameter is kept on the function signature for now so call sites don't change in this commit; the underscore-discard in the body makes it explicit that it's no longer load-bearing. |
||
|
alexei.dolgolyov
|
af82be3fb8 |
feat(workload): container index reconciler
Background worker that keeps the containers table in sync with docker ps. Runs one boot pass and ticks every 30s. Dispatch precedence per container: 1. tinyforge.workload.id label (canonical, new) 2. tinyforge.instance-id label (legacy project — joins via instances) 3. tinyforge.static-site label (legacy site) 4. com.docker.compose.project (stacks — joins via ComposeProjectName) Rows whose Docker container ID is no longer present are flipped to state='missing'. Placeholder rows (empty container_id, e.g. a deploy mid-flight) are left alone so a tick that races a deploy doesn't mark them as missing. DockerLister interface lets tests substitute a fake daemon — 6 unit tests cover the dispatch matrix, missing-sweep, and state normalization. Wired into cmd/server/main.go between docker.New and the existing startup chain. Boot pass populates the containers table from any pre-refactor running containers. |
||
|
alexei.dolgolyov
|
b6f20599d7 |
feat(workload): wire stack + static-site into containers index
Stack manager now upserts a Container row per compose service after every deploy (deterministic ID = workloadID + service so re-deploys update in place). Stop/Start bulk-flip the state field. Compose containers don't yet carry the new tinyforge.* labels — the reconciler will join via com.docker.compose.project when it lands. Static site manager passes WorkloadID/Kind to ContainerConfig so the new labels are stamped, and upserts a single Container row per site (deterministic ID = workloadID + ":site"). Stop/ Start flip state. Delete cascades through the store layer. Now every Tinyforge-managed container — project, stack service, or static site — has a row in the containers index, ready for the reconciler + global view in the next batches. |
||
|
alexei.dolgolyov
|
abb1da903f |
feat(workload): emit workload labels + dual-write containers from deployer
Project deploys (both standard and blue-green) now stamp the new
workload labels on every container and dual-write a row into the
containers index alongside the existing instances row. The legacy
project/stage/instance-id labels stay for now so operator runbooks
don't break — they will be removed after the migration soaks.
New labels:
- tinyforge.managed (every Tinyforge container)
- tinyforge.workload.id (workload row primary key)
- tinyforge.workload.kind ('project' | 'stack' | 'site')
- tinyforge.role (stage name for projects)
ContainerConfig grows WorkloadID/WorkloadKind/Role fields. The
deployer resolves the project's workload row (guaranteed to exist
by boot-time backfill) and passes the IDs through. Container row
ID matches instance ID by construction so removeInstance can drop
both records together.
Stack and static-site managers still need the same treatment;
those land in the next commit.
|
||
|
alexei.dolgolyov
|
db235c1412 |
feat(workload): write-through workload sync + boot-time backfill
CRUD on Project / Stack / StaticSite now keeps a paired Workload row in sync. Secret setters (webhook secret, signing secret, require-signature toggle, notification secret) all re-sync after mutating the source-of-truth row so the workload row always reflects the canonical state. Delete cascades: DeleteProject/Stack/StaticSite now drop the matching workload row plus any container index entries owned by it, so global views don't show ghost rows. Boot-time BackfillWorkloads scans every project/stack/site and ensures each has a workload row. Idempotent — safe to run on every restart, recovers from a deleted/missing workload row. Behavior unchanged for existing call sites; the workloads table just starts being populated. Deployer / reconciler / consumer switchover land in the next commit. |
||
|
alexei.dolgolyov
|
f54a6ecee3 |
feat(workload): add Workload/Container/App store foundation
Introduces the data layer for the Workload refactor (see docs/plans/workload-refactor.md): three new tables and store methods, no behavior changes elsewhere yet. - workloads: unifying primitive over Project/Stack/StaticSite, paired via UNIQUE(kind, ref_id). Notification + webhook config hosted here so it lives in one place across kinds. - containers: normalized index of every Tinyforge-managed container with first-class subdomain/proxy_route_id/npm_proxy_id columns (heavily queried by ListProxyRoutes / stale detection). - apps: optional grouping of workloads; schema only, no UI in v1. Foundation only — deployer surgery, reconciler, and consumer switchover land in the next commit. |
||
|
alexei.dolgolyov
|
0f60a7a5db |
feat(webhook): inbound delivery audit log
Build / build (push) Successful in 10m35s
Persists every inbound webhook hit (project + site) so users can debug
"why didn't my deploy fire?" without grepping daemon logs. Surfaces a
14-day rolling history under the WebhookPanel on each project + site
detail page; refreshes every 30s while open. Daily cron prunes records
older than 14 days alongside the existing event log prune.
Schema:
- webhook_deliveries(id, target_type, target_id, target_name, received_at,
source_ip, signature_state, status_code, outcome, detail, body_size)
- indexes on (target_type,target_id,received_at) and (received_at)
Backend:
- store: WebhookDelivery model + Insert/List/Prune helpers
- webhook/handler: deferred recordDelivery() captures the final outcome
on every return path including HMAC rejects, image mismatch, no-stage,
auto_deploy=false, and successful deploys; signatureStateFor()
classifies "unconfigured" vs "missing" vs "invalid" vs "valid"
- api: GET /api/{projects,sites}/{id}/webhook/deliveries with
parseLimit() helper (default 50, max 200)
- main: daily prune cron retains the last 14 days
Frontend:
- WebhookDeliveryLog.svelte: panel with refresh button, status code +
outcome + signature badges, relative time tooltip-on-hover for
absolute time, source IP column
- Mounted below WebhookPanel on project + site detail pages
- en/ru i18n strings for outcome/signature enums and column labels
|
||
|
alexei.dolgolyov
|
831b5c1a43 |
feat(webhook): HMAC-SHA256 signature verification on inbound webhooks
Adds an opt-in inbound HMAC scheme so a leaked URL alone is not enough
to forge deploy/sync requests — the caller must also know a separate
signing secret. Header format is X-Hub-Signature-256, matching the
Gitea/GitHub/GitLab convention so existing CI integrations work without
custom code.
Behaviour:
- per-project / per-site signing_secret is independent of the URL secret
- require_signature flag does a hard 401 on missing/invalid signatures
- even when require_signature is off, an *invalid* submitted signature
returns 401 — surfaces CI misconfiguration instead of silently passing
- comparison uses subtle/hmac.Equal (constant time)
Backend:
- store: webhook_signing_secret + webhook_require_signature columns on
projects + static_sites; scanProject helper, scan helpers updated; new
Set* helpers for both fields
- webhook/handler: verifyHMAC helper, body read once, integrated into
both project and site handlers
- api: per-entity signing-secret rotate / disable / require-toggle
endpoints under /api/{projects,sites}/{id}/webhook/...
Frontend:
- WebhookPanel gains optional signing handlers (no breaking change for
existing callers; signing UI hides when handlers aren't wired)
- one-shot reveal of the issued secret with copy + dismiss
- ToggleSwitch for require-signature, disabled until a secret is issued
- en/ru i18n strings
Tests:
- HMACRequiredAndValid (200 + deploy fires)
- HMACRequiredButMissing (401, no deploy)
- HMACPresentButWrong (401 even when require_signature=false)
- HMACOptionalUnsignedAccepted (200 when neither configured)
|
||
|
alexei.dolgolyov
|
793570f4a1 |
feat(stats): inline 30-min sparklines on container CPU + memory bars
Always-visible trend lines next to the existing bars on every running instance and static-site card so the user can spot a slow drift or recent spike at a glance, without expanding the full history chart. Implemented as a tiny pure-SVG component (no ECharts on hot list paths) — values are fetched once per 30s alongside the current snapshot via a 30m history query that the collector already serves. - Sparkline.svelte: pure-SVG polyline + optional area fill, normalises values to [0,100] and clamps out-of-range points - ContainerStats: parallel fetch of stats + 30m history, two new $derived arrays for CPU% and memory%, sparklines slotted between the bar and the numeric readout |
||
|
alexei.dolgolyov
|
2c109913bd |
fix(types): resolve pre-existing svelte-check errors
stale-containers page referenced container.id but StaleContainer wraps an inner instance object — switch to container.instance.id at the three call sites. env editor derived projectId from $page.params.id (string | undefined); coalesce to empty string so call sites that pass it to API helpers don't trip the strictNullChecks gate. Pre-existing errors flagged by `svelte-check`; not introduced by recent feature work but blocking the green check. |
||
|
alexei.dolgolyov
|
8b886ddf2b |
feat(backup): take Tinyforge DB snapshot before every deploy
Adds an opt-in "auto_backup_before_deploy" setting that triggers a "pre-deploy" backup at the start of every project deploy via the deploy pipeline (covers both the async HTTP path and the sync poller/webhook path). Failures are logged to the deploy log but do not abort — missing a backup is preferable to refusing to ship a fix. - store: settings.auto_backup_before_deploy column + scan/update wiring - backup: accept "pre-deploy" as a valid backup_type - deployer: small PreDeployBackuper interface, hooked into runDeploy right after settings load and before any state-mutating work - api: settings request/response surface the new flag - web: ToggleSwitch on the backup settings page; "Pre-deploy" badge variant in the backup list (badge-warning so it stands out) - i18n: en/ru strings for the toggle, help text, and badge label |
||
|
alexei.dolgolyov
|
0405ecd9ce |
feat(notify): HMAC-signed outgoing webhooks with per-tier secrets and test sender
Build / build (push) Successful in 10m36s
Outgoing notifications were bare POSTs with no auth and no way to verify
they came from Tinyforge. They also went out from one global URL only,
even though stages had a notification_url field, and static-site sync
emitted no events at all.
Schema: add notification_url + notification_secret (lazy-generated) to
settings, projects, stages and static_sites. Migrations are additive.
Notifier: SendSigned computes HMAC-SHA256 over the exact body bytes and
sends X-Hub-Signature-256 (GitHub-compatible — receivers built for
GitHub/Gitea/Forgejo verify out of the box). Aux headers
X-Tinyforge-Event/Delivery/Timestamp/Tier are advisory and not signed.
Empty secret => unsigned send for back-compat.
Resolution: deploys fall through stage > project > settings, sites fall
through site > settings. The secret travels with the URL that sourced
it, so any tier can sign even when its parents are unsigned. Site sync
events now actually emit (site_sync_success / site_sync_failure).
API: 12 new endpoints — {GET secret, POST regenerate, POST disable,
POST test} for each of the 4 tiers. SendSyncForTest returns
status_code/latency_ms/signature_sent/delivery_id/response_snippet so
the UI surfaces receiver feedback inline.
UI: shared OutgoingWebhookPanel.svelte fits the existing card aesthetic.
Signing-state pill, secret reveal-on-demand, regenerate/disable behind
ConfirmDialog modals (not inline strips — too easy to misclick), send-
test result card with colour-coded status. Wired into Settings →
Integrations, project edit form, per-stage edit, and per-site detail.
EN + RU i18n.
Tests: round-trip (sender signs, receiver verifies), tampered-body and
wrong-secret rejection, unsigned-send omits header, send-test surfaces
4xx, concurrent fan-out via Drain. Resolver precedence locked for both
deploy and site paths.
Docs: docs/webhooks.md with header reference, verifier snippets in
Node/Python/Go, and a recipe for the service-to-notification-bridge
generic webhook provider.
|
||
|
alexei.dolgolyov
|
134fe22fde |
refactor(ui): standardize boolean inputs on ToggleSwitch
Build / build (push) Successful in 11m22s
Replace raw <input type="checkbox"> with the ToggleSwitch component on sites/[id] (encrypt-secret), sites/new (render-markdown, enable-storage), and stacks/new (deploy-immediately). Document the convention in CLAUDE.md so future forms keep the same control instead of mixing styles. |
||
|
alexei.dolgolyov
|
a4362b842d |
fix: harden security, fix concurrency bugs, and address review findings
Build / build (push) Successful in 11m42s
Security: - rate limit /api/webhook routes per-IP and cap concurrent site syncs - global SSE connection cap (256) with new sse_gate - validate ?tail= and cap JSON log responses at 4 MiB - strip ANSI/CSI/OSC and control bytes from streamed log lines - redact webhook secret from request log middleware - scrub host details from /api/health for non-admin viewers - drop container_id from /api/system/stats/top for non-admins - generate webhook secrets via crypto/rand; require >=32 chars on insert - verify iid path consistency in streamContainerLogs - LimitReader on site webhook body; reject malformed non-empty bodies Concurrency / correctness: - stats collector: Stop() no longer hangs without Start(), semaphore acquired in parent loop so ctx cancellation short-circuits the queue, in-flight tick cancellable via shared base context, zero-ts guard - webhook handler: replace fire-and-forget goroutine with WaitGroup-tracked workers + Drain() wired into graceful shutdown - $derived(() => ...) mis-idiom fixed in ContainerStats / InstanceCard / ProjectCard (returned function instead of value) - SystemResourcesCard: rename `window` and `t` locals to avoid shadowing globalThis.window and the i18n `t` import Quality / performance: - replace O(n^2) insertion sort with sort.Slice in stats top - runMigrations only swallows duplicate-column / already-exists errors - PruneStatsSamplesBefore wrapped in a transaction - collapse N+1 in unusedImageStats / pruneImages to one ListAllInstances pass; surface DB errors instead of silently treating them as inactive - run Docker Info + DiskUsage in parallel via errgroup - container log SSE emits `: ping` heartbeat every 20 s - imageMatches case-insensitive on registry host (RFC behaviour) - log warning on invalid stage tag pattern instead of silent skip - reject malformed non-empty site webhook payloads Frontend / i18n: - shared formatBytes utility replaces three local copies - statsInterval store drives dynamic "no samples / collection disabled" copy across ContainerStats and SystemResourcesCard - top consumers row now shows owner_name (project/stage or site name) - drop seven `as any` casts on the Settings type; add cloudflare_api_token write-only field - move "Service status", "Docker daemon", "Docker unreachable", "Proxy unreachable", "reachable", and "Docker daemon is not reachable." strings into en/ru i18n bundles |
||
|
alexei.dolgolyov
|
05440a5f92 |
feat(stats): resource metrics dashboard + sites logs/stats
Build / build (push) Successful in 10m50s
Background collector samples CPU/memory/network/block I/O for every
instance and site on a configurable interval (default 15s, range
5-300s), persists samples to SQLite with a configurable retention
window (default 2h, range 0-24h), and skips ticks gracefully when
the Docker daemon is unreachable. Settings are reloadable without
a restart — each tick re-reads them.
New API endpoints:
- GET /api/system/stats (host snapshot: info + df)
- GET /api/system/stats/history
- GET /api/system/stats/top?by=cpu|memory
- GET /api/projects/{id}/stages/{s}/instances/{iid}/stats/history
- GET /api/sites/{id}/stats[/history]
- GET /api/sites/{id}/logs (SSE + JSON, reuses instance log streamer)
Frontend:
- ECharts added with tree-shaken imports (~180KB gzip) for
future-proof time-series/gantt/graph visualizations
- CollapsibleSection wraps all dashboard sections (system health,
daemons, system resources, static sites, projects) with
localStorage-persisted open state
- SystemResourcesCard shows capacity tiles, workload utilization
chart with 30m/2h/6h/24h window picker, disk breakdown with
reclaimable callouts, and top 5 consumers
- ContainerStats and ContainerLogs take a source discriminated union
so sites reuse the same components as instances; sites detail page
embeds both for Deno backend debugging
- Settings › Maintenance exposes collection interval + retention
- Docker-unavailable state returns 503 and renders an amber banner
instead of a generic 500
Full i18n coverage (en + ru) for all new strings.
|
||
|
alexei.dolgolyov
|
0632f512e6 |
feat(webhook): per-project and per-site webhook URLs
Build / build (push) Successful in 10m25s
Replace the single global webhook secret with entity-scoped secrets stored
on each project and static site. Webhook-driven project autocreate is
removed — projects must exist before their URL can trigger deploys.
Also wires static-site webhooks (sync_trigger=push|tag), turning the
previously inert "push" trigger into a functional one: POST the site's
webhook URL from a Git provider and Tinyforge re-syncs on matching refs.
- Adds webhook_secret columns + unique indexes to projects and static_sites
- Per-entity GET/regenerate endpoints under /api/projects/{id}/webhook
and /api/sites/{id}/webhook (admin-only)
- Removes /api/settings/webhook-url and the global webhook panel
- Reusable WebhookPanel Svelte component on both detail pages, i18n in en/ru
- Tests for matcher (siteRefMatches, ParseImageRef) and handler (project
match/mismatch/404 and site push/manual/branch-skip)
|
||
|
alexei.dolgolyov
|
e08acf5c0e |
refactor(settings): split General into focused pages
Build / build (push) Successful in 10m38s
General was a 547-line catch-all mixing seven concerns, destructive
actions (image prune) inches away from form fields, and Cloudflare DNS
buried under four unrelated cards. A single "Save" committed everything
at once — one invalid field blocked valid edits elsewhere.
Splits:
- /settings Overview: timezone, core infra, proxy choice
- /settings/integrations outgoing notification URL + incoming webhook
- /settings/dns wildcard + Cloudflare provider
- /settings/maintenance stale threshold, prune threshold, prune action
(in a dedicated "Danger zone" card)
- /settings/credentials removed (was an 18-line redirect stub)
Sidebar is grouped (Overview / Routing / System / Security) with
section headers; NPM & Traefik items remain conditional on the
proxy-provider choice. Each page loads settings and PUTs only its own
subset, so mistakes on one page can't block edits on another.
No backend changes — the API already accepts Partial<Settings>.
|
||
|
alexei.dolgolyov
|
03d58a072c |
fix: treat naive backend timestamps as UTC for relative labels
Build / build (push) Successful in 10m35s
Backend emits store.Now() as "2026-04-23 14:05:32" — UTC but without a Z marker or T separator. JavaScript parses such strings as local time, so every "N ago" label is skewed by the browser's UTC offset (3h off in UTC+3, causing the visible mismatch with the tooltip). Normalise bare/space-separated timestamps to UTC inside toDate() so every $fmt.* call, including relative, matches the absolute timestamp shown in the tooltip. InstanceCard drops its own parser and delegates to $fmt.relative for consistency. |
||
|
alexei.dolgolyov
|
90e6e59d9e |
feat: daemon health panel, brand-rail status chips, user timezone selector
Build / build (push) Successful in 10m35s
- Health API now surfaces Docker /info + /version (version, platform, kernel, container/image counts, storage driver, memory, latency) and NPM aggregates (proxy host total, managed-by-Tinyforge count, access lists, certificates, endpoint URL). - Docker/NPM indicators moved out of the sidebar footer and into a compact mono-styled rail directly under the Tinyforge brand title, with pulse/fault animations and click-to-expand error hints. - New SystemDaemonsCard on the dashboard: two terminal-styled panels (Docker Engine + Proxy) with a running/paused/stopped stacked bar, key-value diagnostics, and a total-vs-managed proportion meter on the proxy-hosts tile. - Shared health store so the sidebar and dashboard share a single 30 s poll instead of duplicating traffic. - User-facing timezone preference with auto-detect fallback; all dates across projects, sites, stacks, settings, backup, event log and stale containers now render through \$fmt.date / \$fmt.datetime. - en/ru translations for both features. |