12 Commits

Author	SHA1	Message	Date
alexei.dolgolyov	bd7a11d4e7	refactor(source): dedup shared helpers across static + dockerfile plugins ... Extract the verbatim-duplicated helpers into shared homes: - buildEnv -> plugin.BuildWorkloadEnv (base plugin pkg; a sourceName param preserves each plugin's slog prefix / log-scraper text) - idShort -> plugin.IDShort - commitStatusReporter -> staticsite.CommitStatusReporter, re-parameterized on primitives (owner/repo/sha/targetURL/enabled) so staticsite needs no dependency on the plugin package; reporter tests ported to staticsite (plus a new nil-provider case) containerNameFor/imageTagFor are intentionally left per-plugin: their prefixes differ (dw-site- vs tf-build-) and name real Docker resources, so merging them would risk mis-routing. Behavior-preserving; the static/dockerfile test suites pass unchanged. Reviewed: go APPROVE (0 CRITICAL/HIGH).	2026-05-29 14:57:30 +03:00
alexei.dolgolyov	3071cda512	feat(deploy): commit-status reporting to Git providers ... Report deploy status back to the Git provider as a commit status (pending/success/failure) for git-sourced workloads (static + dockerfile). - GitProvider.SetCommitStatus on gitea/github/gitlab over the existing SSRF-safe client; fixed "tinyforge" context so redeploys update one row. postJSON returns status-code-only errors (never echoes the upstream body, which a hostile provider could use to reflect the auth token into the best-effort log line). - Best-effort deploy hook: pending on deploy start, success/failure on outcome, gated on a per-workload report_commit_status flag. Never fails or blocks a deploy; emits nothing on the unchanged-SHA short-circuit. - UI ToggleSwitch (create + edit) + reportCommitStatus in sourceForms.ts + en/ru i18n. - Tests: per-provider state mapping + request shape; reporter gating (enabled/disabled/empty-SHA/nil/error-swallow). Reviewed via go-reviewer + security-reviewer (0 CRITICAL/HIGH; one MEDIUM body-echo log-leak fixed).	2026-05-29 11:37:56 +03:00
alexei.dolgolyov	410a131cec	feat(apps): stepped creation wizard, branch previews, and app-creation fixes ... This session (frontend focus): - Rebuild /apps/new as a 4-step wizard (Basics → Configure → Trigger → Review): WizardRail, SourceKindPicker card grid, AppManifest review, per-step validation, ConfirmDialog-based unsaved-changes guard. - Extract lib/workload/sourceForms.ts (single source of truth for source_config) + {Image,Compose,Static,Dockerfile}SourceForm + StaticDiscoveryWizard; fold the /apps/[id] edit form onto the same components (removes the duplication). Add vitest + sourceForms unit tests. - Branch preview environments UI: /chain is_preview/preview_branch + a Preview environments panel on /apps/[id] (per-branch URLs, ConfirmDialog teardown, armed state); RegistryImagePicker on the registry trigger and the image source. - Fixes: image-inspect 404 -> admin-gated POST /api/discovery/image/inspect; conflict-panel blur flicker; friendly localized discovery errors; CPU/Memory label hints; dashboard + /apps "Total workloads" count only source_kind workloads (drop stale trigger_kind gate); NPM cert/access-list name cache; EntityPicker empty-list guard. - Update CLAUDE.md frontend conventions + add a Build & Test section. Also captures pre-existing in-progress platform work (not from this session): workload notifications, Prometheus metrics export, store lockfile, health probes, backup hardening, and related store/webhook/scheduler changes.	2026-05-29 02:09:54 +03:00
alexei.dolgolyov	ea55d31177	feat(discovery+runtime): restore static-site wizard discovery + close /sites/[id] feature parity ... Two-stage feature arc closing the gaps left by the hard legacy cutover. The static-site creation wizard regains its auto-discovery + connection-test flow; /apps/[id] grows the runtime/storage/lifecycle surface the legacy /sites/[id] page used to expose. Backend (Go) - internal/api/discovery.go: six admin-gated endpoints wrapping staticsite.GitProvider — POST /api/discovery/git/{detect-provider, test-connection,repos,branches,tree} + GET /api/discovery/image/conflicts. Identifier validation (validateGitIdent / validateGitBranch) at the boundary so provider URL interpolation cannot be hijacked via `..`. Upstream errors scrubbed: detailed slog on the server, generic 502 to the client (mitigates token-reflection-in-error-page). - internal/api/workload_runtime.go: four endpoints — GET /api/workloads/{id}/runtime-state decodes containers.extra_json for static workloads; GET /api/workloads/{id}/storage execs `du -sb /app/data` with a 30s in-process cache (storageProbeCache) so polling can't turn into per-request execs; POST /api/workloads/{id}/{stop,start} iterate ListContainersByWorkload and call docker.StopContainer / StartContainer, returning 200 / 409 (nothing to act on) / 502 (all failed). - internal/staticsite/safehttp.go: NewSafeHTTPClient + ValidateBaseURL + blockReason. DialContext re-resolves hostnames and refuses loopback / link-local / multicast / unspecified addresses. RFC1918 + ULA explicitly allowed (self-hosted Gitea on LAN is the dominant deployment). Replaced four raw &http.Client{} constructions in the provider files. - internal/staticsite/gitlab_provider.go: url.PathEscape each segment in the raw-file URL builder for parity with projectPath(). - Test coverage: 26 cases in discovery_test.go (image-tag stripping, source-config decoding, conflict scenarios, validator boundaries, scheme rejection), 14 in workload_runtime_test.go (404 / 409 / nil-docker / probe-cache), 16 in safehttp_test.go (URL validation + block-reason policy matrix + live dial against loopback + AWS metadata literals). Frontend (Svelte 5 + runes) - web/src/lib/api.ts: typed wrappers for every endpoint, AbortSignal threaded through post(); ApiError exported so callers can narrow on e.status; new DetectedGitProvider narrow union. - web/src/routes/apps/new/+page.svelte: static-form discovery controls (auto-detect provider, test connection, repo / branch / folder EntityPickers, Deno auto-detect); image-form conflict panel with debounced lookup + double-click submit guard ("Forge anyway") + Inspect button that pre-fills port/healthcheck; English error fallbacks routed through apps.new.errors.* (en + ru). - web/src/routes/apps/[id]/+page.svelte: runtime-state panel + storage panel + Stop / Start / Open-site toolbar; universal live-state badge in the hero lede for image/compose/static (RUNNING / TRANSITIONING / STOPPED / NOT DEPLOYED / MIXED · n/m RUNNING); ContainerStats panel per row (auto-collapsing native <details> when N > 2); read-only webhook bindings summary card; responsive toolbar overflow with native <details> at <640px (z-index 100 above sticky nav). - web/src/app.css: project-wide .forge-btn-ghost:focus-visible outline. Hardening from go-reviewer + security-reviewer + typescript-reviewer + frontend-design UI/UX subagents (0 CRITICAL, all HIGH/BLOCKER addressed inline, IMPORTANT applied before commit): - AbortController + per-call sequence tokens on every long-running fetch (loadRuntimeState / loadStorage / loadTriggerMeta / inspectImage / listImageConflicts) plus onDestroy cleanup so late resolves cannot mutate dead component state. - doStop / doStart snapshot and restore `error` across the finally-block reload so a load()-cleared message doesn't hide a real failure. - triggersById refreshed after inline trigger creation so the webhook card doesn't silently exclude the just-created trigger. - Live-state badge wraps in role=status / aria-live=polite (no redundant aria-label). - Webhook row has a single click target (was two pointing at the same URL). - Empty webhook section hides entirely. - Dropped role=menu / role=menuitem from the overflow menu (they would promise arrow-key nav we don't wire; native Tab + ESC carry it). Doc - docs/CODEMAPS/INDEX.md + new docs/CODEMAPS/discovery-and-runtime.md map the endpoint surface, security posture, frontend integration patterns, and an "add a new probe" recipe. Verification - svelte-check: 0 errors, 3 pre-existing a11y warnings. - go build + go vet + go test ./...: all green. - i18n parity: en + ru at 1413 keys each. - Live smoke against :8090: 404 / 409 / 502 envelopes correct, discovery sanity passes, ProbeError surfaces on no-container path.	2026-05-16 21:35:51 +03:00
alexei.dolgolyov	739b67856a	feat(cutover): hard legacy cutover — drop projects/stacks/sites/deploys ... The clean-break delete that closes the workload-first refactor arc. Net diff: ~30 backend files deleted, ~20 modified, ~12k LOC removed on the Go side; entire /projects /stacks /sites /deploy frontend trees gone; ~6.7k LOC removed on the Svelte/TypeScript side. Backend - API handlers gone: internal/api/{projects,stages,stage_env,stacks, static_sites,deploys,instances,volume_browser}.go - Store CRUD + tests gone: internal/store/{projects,stages,stage_env, stacks,static_sites,static_site_secrets,deploys,poll_state,volumes, workload_sync}.go (+ _test.go siblings) - Legacy deployer pipeline gone: internal/deployer/{bluegreen,promote, rollback,subdomain,resolver_test}.go; deployer.go trimmed to just the dispatch surface used by the plugin pipeline - internal/staticsite/{manager,healthcheck}.go and internal/stack/manager.go gone (the rest of those packages stay as helpers imported by the static + compose plugins) - internal/registry/poller.go gone (legacy registry poller) - internal/volume.ResolvePath gone; ResolveWorkloadPath stays - internal/webhook: handleWebhook (project) + handleSiteWebhook (site) gone; only POST /api/webhook/triggers/{secret} remains - workload-side webhook URL handlers (getWorkloadWebhook + regenerateWorkloadWebhook + EnsureWorkloadWebhookSecret + SetWorkloadWebhookSecret + GetWorkloadByWebhookSecret) gone — they minted URLs that would 404 against the new trigger-only ingress - cmd/server/main.go: dropped staticsite.Manager, stack.Manager, staticsite.HealthChecker, registry poller, SetSiteSyncTriggerer, SetStaticSiteManager, SetStackManager, wireStaticBackend - store/store.go: idempotent DROP TABLE IF EXISTS for every legacy table (projects, stages, stage_env, volumes, deploys, deploy_logs, poll_states, stacks, stack_revisions, stack_deploys, static_sites, static_site_secrets); FK order children-then-parents - store/models.go: dropped Project, Stage, Deploy, DeployLog, StageEnv, Volume, StaticSite, StaticSiteSecret, Stack, StackRevision, StackDeploy types; kept WorkloadKind constants as documented strings - internal/store/helpers.go (new): BoolToInt, rowScanner, GenerateWebhookSecret extracted from deleted CRUD files - internal/api/secrets.go (new): forwards to store.GenerateWebhookSecret so api + store paths share one secret-generation impl (no panic-vs-UUID-fallback divergence) - internal/reconciler/reconciler.go: dropped legacy stack-by-compose + static-site label paths; only canonical tinyforge.workload.id dispatch remains - providers (gitea_content/github_provider/gitlab_provider) gained path-traversal rejection on every tree entry - internal/webhook ParsedImage / ParseImageRef demoted to package- private (no external callers) Frontend - /projects /stacks /sites /deploy routes deleted (entire trees) - ProjectCard / InstanceCard / StaleContainerCard components deleted - api.ts: dropped every project/stage/stack/site/deploy/instance helper + types (Project, Stage, Stack, StaticSite, Deploy, Instance, Volume, etc.); kept Workload, Container, App, Settings, Registry, EventTrigger, LogScanRule, webhook envelopes - WorkloadWebhook type + getWorkloadWebhook/regenerateWorkloadWebhook api functions gone (mirror of the backend deletion above) - web/src/routes/+layout.svelte: dropped /projects /sites /stacks /deploy nav entries, trimmed quick-nav keymap - web/src/routes/+page.svelte: dashboard rewrite — reads listWorkloads + listContainers only; 4-card stat grid (workloads/running/failed/stale) + recent workloads strip - navCounts.ts, SystemHealthCard.svelte, ContainerLogs.svelte, ContainerStats.svelte, StatusBadge.svelte, TagCombobox.svelte, proxies/+page.svelte, containers/+page.svelte all rewired to the workload-first surface - AbortController plumbing on dashboard, nav-counts, stale page, SystemHealthCard so navigation doesn't leave dangling fetches - i18n: dropped projects.*, projectDetail.*, envEditor.*, volumeEditor.*, volumeBrowser.*, quickDeploy.*, sites.*, stacks.*, instance.*, confirm.* namespaces; en/ru parity preserved (1042 keys each) Hardening from go-reviewer + security-reviewer + typescript-reviewer subagent passes (0 CRITICAL across all three; 1 HIGH + ~12 MEDIUM addressed inline before commit): - Sec H1: dead-end workload webhook URL handlers (would mint URLs that 404 the new trigger-only ingress) deleted across backend + frontend - Go M1: IsTerminalDeployStatus dropped (no production callers) - Go M2: ParsedImage/ParseImageRef lowercased (in-package only) - Go M6: generateWebhookSecret unified — api shim forwards to store.GenerateWebhookSecret - Doc/comment freshness: stage_id (no longer FK), ProxyRoute legacy field names, workloadIDRow rationale, webhook_deliveries.target_type enum, WebhookDeliveryLog component header Doc - WORKLOAD_REFACTOR_TODO: cutover marked DONE; all three Priority 1 items are now shipped. Next focus is Priority 3 polish (apps.* i18n + codemap entries) and Priority 4 tests. Behavioral notes for operators upgrading from a pre-cutover build - Existing rows in the dropped tables disappear on first boot. - Legacy webhook URLs at /api/webhook/{secret} and /api/webhook/sites/{secret} return 404; CI configs must repoint to /api/webhook/triggers/{secret} (the trigger-split boot backfill lifted any embedded workload secret onto a Trigger row, so the secret value itself carries over). - Frontend routes /projects /stacks /sites /deploy are gone; nav links replaced with /apps and /triggers.	2026-05-16 06:00:21 +03:00
alexei.dolgolyov	234c3c711e	feat(static): inline static-source plugin; drop phantom-row adapter ... Lift the static-site deploy pipeline from internal/staticsite/manager.go into internal/workload/plugin/source/static/ so plugin-native static workloads operate directly on plugin.Workload + the containers table + workload_env. The cmd/server/static_backend.go phantom-row adapter is gone; the legacy static_sites table is no longer touched on plugin deploys. Backend - new state.go: runtimeState (last_commit_sha, last_sync_at, last_error, status) persisted in containers.extra_json under the deterministic row id <workloadID>:site - per-workload sync.Mutex serializes saveState read-modify-write so parallel deploys for the same workload can't race container_id / proxy_route_id writes - extra_json round-trips through map[string]json.RawMessage so unknown keys survive — typed runtimeStateKeys are stripped before merge so clearing a typed field actually drops the key - new env.go reads workload_env (replaces static_site_secrets for plugin-native sites); decrypt-failure logs and skips one entry rather than failing the whole deploy - new build.go ports prepareDenoBuild + prepareStaticBuild + copyDir; copyDir uses filepath.WalkDir + Lstat to refuse symlinks and non-regular files - new deploy.go is the ~300-line core; intent.Reason gates force vs skip-if-no-changes; success-path saveState failure rolls back container + proxy route and writes "failed" state (no orphans) - new teardown.go combines Remove + Stop; idempotent on never-deployed workloads - new reconcile.go refreshes container state from Docker; flips runtimeState.Status to failed when the container is missing/crashed Hardening (from go-reviewer + security-reviewer subagent passes; 1 CRITICAL + 5 HIGH + 3 MEDIUM addressed before merge) - path-traversal defense in all 3 providers (gitea_content, github_provider, gitlab_provider): reject tree entries whose resolved local path escapes destDir - verifyDownloadInsideRoot walks the build dir post-download as a second line of defense - sanitizeError redacts the access token, collapses to one line, and clamps to 240 bytes before persisting to extra_json or fanning out to the notification webhook - container/image/volume names suffixed with workload-id short prefix (workload name is not UNIQUE in schema) - primaryDomain reads settings.Domain to complete a bare subdomain face into a full FQDN (matches legacy Manager behavior) - ctx-aware health-check sleep - json.Marshal for event metadata (was fmt.Sprintf JSON template) - strings.HasPrefix for failed-status detection (was brittle slice expression) Wire-up - cmd/server/main.go: removed wireStaticBackend(...) call; existing blank import on _ ".../source/static" drives init() registration - cmd/server/static_backend.go deleted Doc - WORKLOAD_REFACTOR_TODO: static port marked DONE; next focus is the hard legacy cutover (drop /api/projects, /api/stacks, /api/sites, /api/stages + their tables, internal/stack + internal/staticsite packages, frontend /projects /stacks /sites) Behavior notes for operators - plugin-native static workloads no longer write to static_sites; legacy /api/sites/* still serves original rows unchanged - legacy tinyforge.static-site / .static-site-name container labels dropped on plugin deploys; canonical tinyforge.workload.id / .kind cover ownership - container/image/volume names gained an 8-char ID suffix (e.g. dw-site-mysite-a1b2c3d4); legacy-deployed sites keep the old shape until redeployed through the plugin path	2026-05-16 02:56:23 +03:00
alexei.dolgolyov	d8ab22876f	refactor(workload): extract Instance entirely; Container is canonical ... End-to-end extraction of the Instance concept. After this commit: * internal/store/instances.go — DELETED * internal/store/models.go — Instance struct gone, ProxyRoute moved here * containers table is the single source of truth for project/stack/site container state. instances table is dropped via DROP TABLE migration (idempotent; re-runnable on every boot). * Legacy tinyforge.project / tinyforge.stage / tinyforge.instance-id Docker labels are no longer emitted; only tinyforge.workload.{id,kind}, tinyforge.role, and tinyforge.managed are stamped on new containers. Backend rewrites: - internal/deployer: executeDeploy + blueGreenDeploy + rollback + promote use store.Container natively. New removeContainer() replaces removeInstance(). enforceMaxInstances reads via ListContainersByStageID. - internal/reconciler: legacy tinyforge.instance-id dispatch removed; upsertByWorkloadLabel now finds existing rows by docker container ID first and falls back to the deterministic workloadID:role key. - internal/stale/scanner: Scan + new FindStaleContainers walk the containers table; emit StaleContainer JSON. - internal/stats/collector: ListContainers replaces ListAllInstances. - internal/webhook/handler: workload-secret lookup tried first; falls back to project / static_site secret column. - internal/api: instances.go, stale.go, stats.go, stats_history.go, projects.go, settings.go, docker.go, dns.go all read / write through Container. Docker layer: - ManagedContainer exposes WorkloadID/Kind/Role from the canonical labels. - ListContainers filters by tinyforge.managed=true. - Network creation uses LabelManaged instead of LabelProject. Frontend: - Instance type is now a Container alias; .status → .state, .last_alive_at → .last_seen_at. - InstanceCard takes stageId as a prop (no longer derived from Instance). - StaleContainer JSON shape rewritten: { container, workload_name, role, days_stale }. StaleContainerCard + /containers/stale page updated. - ProjectCard / homepage / SystemHealthCard filter by .state. The migration loop now tolerates "no such table" alongside "duplicate column" / "already exists" so obsolete ALTER TABLE entries targeting the dropped instances table no-op cleanly on first boot. Tests: store + deployer + reconciler + webhook + staticsite + notify all still pass. Frontend svelte-check: zero errors.	2026-05-09 14:43:12 +03:00
alexei.dolgolyov	b6f20599d7	feat(workload): wire stack + static-site into containers index ... Stack manager now upserts a Container row per compose service after every deploy (deterministic ID = workloadID + service so re-deploys update in place). Stop/Start bulk-flip the state field. Compose containers don't yet carry the new tinyforge.* labels — the reconciler will join via com.docker.compose.project when it lands. Static site manager passes WorkloadID/Kind to ContainerConfig so the new labels are stamped, and upserts a single Container row per site (deterministic ID = workloadID + ":site"). Stop/ Start flip state. Delete cascades through the store layer. Now every Tinyforge-managed container — project, stack service, or static site — has a row in the containers index, ready for the reconciler + global view in the next batches.	2026-05-09 13:41:03 +03:00
alexei.dolgolyov	0405ecd9ce	feat(notify): HMAC-signed outgoing webhooks with per-tier secrets and test sender ... Outgoing notifications were bare POSTs with no auth and no way to verify they came from Tinyforge. They also went out from one global URL only, even though stages had a notification_url field, and static-site sync emitted no events at all. Schema: add notification_url + notification_secret (lazy-generated) to settings, projects, stages and static_sites. Migrations are additive. Notifier: SendSigned computes HMAC-SHA256 over the exact body bytes and sends X-Hub-Signature-256 (GitHub-compatible — receivers built for GitHub/Gitea/Forgejo verify out of the box). Aux headers X-Tinyforge-Event/Delivery/Timestamp/Tier are advisory and not signed. Empty secret => unsigned send for back-compat. Resolution: deploys fall through stage > project > settings, sites fall through site > settings. The secret travels with the URL that sourced it, so any tier can sign even when its parents are unsigned. Site sync events now actually emit (site_sync_success / site_sync_failure). API: 12 new endpoints — {GET secret, POST regenerate, POST disable, POST test} for each of the 4 tiers. SendSyncForTest returns status_code/latency_ms/signature_sent/delivery_id/response_snippet so the UI surfaces receiver feedback inline. UI: shared OutgoingWebhookPanel.svelte fits the existing card aesthetic. Signing-state pill, secret reveal-on-demand, regenerate/disable behind ConfirmDialog modals (not inline strips — too easy to misclick), send- test result card with colour-coded status. Wired into Settings → Integrations, project edit form, per-stage edit, and per-site detail. EN + RU i18n. Tests: round-trip (sender signs, receiver verifies), tampered-body and wrong-secret rejection, unsigned-send omits header, send-test surfaces 4xx, concurrent fan-out via Drain. Resolver precedence locked for both deploy and site paths. Docs: docs/webhooks.md with header reference, verifier snippets in Node/Python/Go, and a recipe for the service-to-notification-bridge generic webhook provider.	2026-05-07 02:03:32 +03:00
alexei.dolgolyov	b622384774	feat: persistent storage for Deno static sites ... - Add storage_enabled and storage_limit_mb columns to static_sites. - Create/attach Docker volumes (tinyforge-site-{name}-data) for Deno sites with storage enabled, mounted at /app/data. - Grant --allow-write=/app/data in Deno container CMD. - Add storage usage API endpoint (GET /api/sites/{id}/storage). - Show storage section in site detail page with usage bar. - Add storage toggle and limit field to new site wizard. - Use ConfirmDialog for secret deletion instead of inline delete.	2026-04-13 00:12:51 +03:00
alexei.dolgolyov	791cd4d6af	feat: rename Docker Watcher to Tinyforge ... Rebrand the project as Tinyforge to reflect its evolution from a Docker container watcher into a self-hosted mini CI/deployment platform. Rename covers: Go module path, Docker labels, DB/config filenames, JWT issuer, Dockerfile binary, docker-compose, CI workflows, frontend i18n, README with static sites docs, and all code comments.	2026-04-12 21:30:39 +03:00
alexei.dolgolyov	8d2c5a063b	feat: static sites feature with Gitea/GitHub/GitLab support and Deno backend ... Deploy static content from Git repository folders with optional server-side API endpoints. Supports Gitea/Forgejo/Gogs, GitHub, and GitLab with provider autodetection. - New Sites entity with CRUD, encrypted secrets, and manual/push/tag sync triggers - Pluggable GitProvider interface with three implementations - Deno container mode: auto-generates router from API_{method}_{name} exports - Static container mode: nginx serving files with optional markdown rendering - Wizard UI with provider selector, repo picker, branch/folder tree pickers - Deploy pipeline builds fresh image, starts container, configures NPM proxy - Stop/Start buttons, force redeploy on manual trigger - Periodic health checker detects crashed containers - Proxy route existence check during auto-sync	2026-04-11 03:35:57 +03:00