feat(static): inline static-source plugin; drop phantom-row adapter
Build / build (push) Successful in 10m43s
Build / build (push) Successful in 10m43s
Lift the static-site deploy pipeline from internal/staticsite/manager.go into internal/workload/plugin/source/static/ so plugin-native static workloads operate directly on plugin.Workload + the containers table + workload_env. The cmd/server/static_backend.go phantom-row adapter is gone; the legacy static_sites table is no longer touched on plugin deploys. Backend - new state.go: runtimeState (last_commit_sha, last_sync_at, last_error, status) persisted in containers.extra_json under the deterministic row id <workloadID>:site - per-workload sync.Mutex serializes saveState read-modify-write so parallel deploys for the same workload can't race container_id / proxy_route_id writes - extra_json round-trips through map[string]json.RawMessage so unknown keys survive — typed runtimeStateKeys are stripped before merge so clearing a typed field actually drops the key - new env.go reads workload_env (replaces static_site_secrets for plugin-native sites); decrypt-failure logs and skips one entry rather than failing the whole deploy - new build.go ports prepareDenoBuild + prepareStaticBuild + copyDir; copyDir uses filepath.WalkDir + Lstat to refuse symlinks and non-regular files - new deploy.go is the ~300-line core; intent.Reason gates force vs skip-if-no-changes; success-path saveState failure rolls back container + proxy route and writes "failed" state (no orphans) - new teardown.go combines Remove + Stop; idempotent on never-deployed workloads - new reconcile.go refreshes container state from Docker; flips runtimeState.Status to failed when the container is missing/crashed Hardening (from go-reviewer + security-reviewer subagent passes; 1 CRITICAL + 5 HIGH + 3 MEDIUM addressed before merge) - path-traversal defense in all 3 providers (gitea_content, github_provider, gitlab_provider): reject tree entries whose resolved local path escapes destDir - verifyDownloadInsideRoot walks the build dir post-download as a second line of defense - sanitizeError redacts the access token, collapses to one line, and clamps to 240 bytes before persisting to extra_json or fanning out to the notification webhook - container/image/volume names suffixed with workload-id short prefix (workload name is not UNIQUE in schema) - primaryDomain reads settings.Domain to complete a bare subdomain face into a full FQDN (matches legacy Manager behavior) - ctx-aware health-check sleep - json.Marshal for event metadata (was fmt.Sprintf JSON template) - strings.HasPrefix for failed-status detection (was brittle slice expression) Wire-up - cmd/server/main.go: removed wireStaticBackend(...) call; existing blank import on _ ".../source/static" drives init() registration - cmd/server/static_backend.go deleted Doc - WORKLOAD_REFACTOR_TODO: static port marked DONE; next focus is the hard legacy cutover (drop /api/projects, /api/stacks, /api/sites, /api/stages + their tables, internal/stack + internal/staticsite packages, frontend /projects /stacks /sites) Behavior notes for operators - plugin-native static workloads no longer write to static_sites; legacy /api/sites/* still serves original rows unchanged - legacy tinyforge.static-site / .static-site-name container labels dropped on plugin deploys; canonical tinyforge.workload.id / .kind cover ownership - container/image/volume names gained an 8-char ID suffix (e.g. dw-site-mysite-a1b2c3d4); legacy-deployed sites keep the old shape until redeployed through the plugin path
This commit is contained in:
+103
-39
@@ -9,26 +9,30 @@ order.
|
||||
|
||||
> ## Current focus (read this first)
|
||||
>
|
||||
> **Triggers as first-class reusable entities — DONE** (2026-05-16). The
|
||||
> trigger-split arc shipped end-to-end: `triggers` + `workload_trigger_bindings`
|
||||
> tables, boot-time backfill, fan-out webhook handler at
|
||||
> `/api/webhook/triggers/{secret}` with bounded concurrency, `/api/triggers`
|
||||
> CRUD + `/api/bindings/{id}` + workload-side bind endpoints, full `/triggers`
|
||||
> frontend (list, new, detail), workload-page bindings panel + per-binding
|
||||
> override editor, i18n EN+RU.
|
||||
> **Triggers as first-class reusable entities — DONE** (2026-05-16) and
|
||||
> **Static source inline port — DONE** (2026-05-16). The phantom-row
|
||||
> adapter (`cmd/server/static_backend.go`) is gone; the static plugin
|
||||
> now operates directly on `plugin.Workload` + `containers` +
|
||||
> `workload_env`, with runtime state (`last_commit_sha`, `last_sync_at`,
|
||||
> `last_error`, `status`) carried in `containers.extra_json`. Provider
|
||||
> downloads enforce path-traversal rejection, error strings are
|
||||
> sanitized before persistence, and Docker resource names are suffixed
|
||||
> with the workload ID short prefix to dodge name collisions.
|
||||
>
|
||||
> **Next on Priority 1** is the **static source inline port** (~2150 LOC
|
||||
> across 8 files; details in the section below). After that, the
|
||||
> **hard legacy cutover** (drop `/api/projects`, `/api/stacks`, `/api/sites`,
|
||||
> `/api/stages` + their tables and frontends) clears the deck.
|
||||
> **Next on Priority 1** is the **hard legacy cutover** — drop
|
||||
> `/api/projects`, `/api/stacks`, `/api/sites`, `/api/stages` handlers,
|
||||
> drop their tables, delete `internal/stack/` + `internal/staticsite/`
|
||||
> packages, delete frontend `/projects` / `/stacks` / `/sites` routes.
|
||||
> The `internal/staticsite` package stays alive only for the legacy
|
||||
> `/api/sites/*` HTTP routes — once those drop, it dies with them.
|
||||
|
||||
## Status at a glance
|
||||
|
||||
| Item | Priority | Status |
|
||||
| ---- | -------- | ------ |
|
||||
| Triggers as first-class reusable entities | 1 | **DONE** (2026-05-16) |
|
||||
| Static source inline port | 1 | **PENDING — current focus** |
|
||||
| Hard legacy cutover | 1 | **PENDING** — gated by static port (volume scopes blocker is resolved) |
|
||||
| Static source inline port | 1 | **DONE** (2026-05-16) |
|
||||
| Hard legacy cutover | 1 | **PENDING — current focus** |
|
||||
| Generalized volume scopes | 2 | DONE |
|
||||
| Kind-aware editors (compose / image / static) | 2 | DONE |
|
||||
| Vendor-specific webhook parsing | 2 | DONE |
|
||||
@@ -148,39 +152,99 @@ replaced with `<ToggleSwitch>` to honor the project rule.
|
||||
typescript-reviewer subagents** — 0 CRITICAL; 5 HIGH and 4 MEDIUM
|
||||
findings addressed inline before merge.
|
||||
|
||||
### Static source inline port — ~2150 LOC across 8 files
|
||||
### ~~Static source inline port~~ — DONE (2026-05-16)
|
||||
|
||||
The current `internal/workload/plugin/source/static/` delegates to
|
||||
`staticsite.Manager` via a phantom-row adapter
|
||||
(`cmd/server/static_backend.go`) that keeps a synthetic row in the legacy
|
||||
`static_sites` table per workload. This works but blocks the hard cutover —
|
||||
you can't drop `static_sites` until the adapter is gone.
|
||||
The phantom-row adapter (`cmd/server/static_backend.go`) is deleted; the
|
||||
static plugin now operates directly on `plugin.Workload`, the `containers`
|
||||
table, and `workload_env`. The deploy pipeline body lives inline in
|
||||
`internal/workload/plugin/source/static/{deploy,teardown,reconcile,
|
||||
state,env,build,naming,static}.go`.
|
||||
|
||||
To port inline, the deploy pipeline body has to move into
|
||||
`internal/workload/plugin/source/static/`:
|
||||
**State migration:** the legacy `static_sites` columns
|
||||
(`last_commit_sha`, `last_sync_at`, `last_error`, `status`,
|
||||
`container_id`, `proxy_route_id`) are now persisted on the container
|
||||
row keyed `<workloadID>:site` — deterministic ID, single row per
|
||||
workload. First-class fields (`container_id`, `proxy_route_id`,
|
||||
`subdomain`, `state`, `port`, `image_ref`) move into their dedicated
|
||||
columns on the `containers` table; the rest live in
|
||||
`containers.extra_json` via a typed `runtimeState` struct that
|
||||
preserves unknown keys on round-trip (so future writers can extend
|
||||
`extra_json` without forcing this struct to grow). `workload_env`
|
||||
replaces `static_site_secrets` for plugin-native workloads.
|
||||
|
||||
| Source file | Lines | What to keep / port |
|
||||
| --- | --- | --- |
|
||||
| `internal/staticsite/manager.go` | 834 | Deploy / Stop / status pipeline. State should move to `containers` rows + `workload_env` instead of `static_sites`. |
|
||||
| `internal/staticsite/gitea_content.go` | 360 | Keep as helper — Gitea content download/listing. |
|
||||
| `internal/staticsite/github_provider.go` | 276 | Keep as helper. |
|
||||
| `internal/staticsite/gitlab_provider.go` | 254 | Keep as helper. |
|
||||
| `internal/staticsite/healthcheck.go` | 111 | Convert to plugin Reconcile body. |
|
||||
| `internal/staticsite/markdown.go` | 83 | Keep as helper. |
|
||||
| `internal/staticsite/provider.go` | 171 | Keep — provider abstraction. |
|
||||
| `internal/staticsite/deno/` | (sub-pkg) | Keep — Dockerfile + router.ts codegen. |
|
||||
**Reused helpers:** `internal/staticsite/{provider,gitea_content,
|
||||
github_provider,gitlab_provider,markdown,deno}` stay alive (and
|
||||
exported) as helpers — providers are still imported via
|
||||
`staticsite.NewGitProvider`. The `staticsite.Manager` itself stays
|
||||
alive only to service the legacy `/api/sites/*` HTTP routes; once
|
||||
those drop in the cutover the package can be deleted entirely.
|
||||
|
||||
Estimated as its own dedicated turn (or two). Strategy: keep the provider
|
||||
abstraction + helpers exported; rewrite only `Manager.Deploy` body into a new
|
||||
`source/static/deploy.go` that operates against `plugin.Workload` directly and
|
||||
writes container rows + workload_env rather than the `static_sites` table.
|
||||
**Hardening landed alongside the port** (from `go-reviewer` +
|
||||
`security-reviewer` subagent passes — 1 CRITICAL, 5 HIGH, 3 MEDIUM
|
||||
addressed before merge):
|
||||
|
||||
- **Path-traversal defense:** providers (`gitea_content.go`,
|
||||
`github_provider.go`, `gitlab_provider.go`) reject any tree entry
|
||||
whose resolved local path escapes `destDir`; the static plugin's
|
||||
`verifyDownloadInsideRoot` walks the build dir post-download as a
|
||||
second line of defense; `copyDir` uses `filepath.WalkDir` + `Lstat`
|
||||
to refuse symlinks and non-regular files.
|
||||
- **Error sanitization:** a `sanitizeError` helper redacts the
|
||||
decrypted access token, collapses to one line, and clamps to 240
|
||||
bytes before any error string lands in `runtimeState.LastError`
|
||||
(persisted in `extra_json`) or fans out to the notification
|
||||
webhook.
|
||||
- **Resource naming with workload-ID short suffix:** container,
|
||||
image, and storage volume names all carry `idShort(w)` so two
|
||||
workloads sharing a name can't clobber each other's resources
|
||||
(workload `name` is not UNIQUE in the schema).
|
||||
- **Per-workload mutex on `saveState`:** serializes the read-modify-
|
||||
write of `containers.extra_json` so two parallel deploys for the
|
||||
same workload can't race to clobber each other's
|
||||
`container_id` / `proxy_route_id`.
|
||||
- **`saveState` failure on the success path is fatal:** rolls back
|
||||
the just-created container + proxy route and writes a "failed"
|
||||
state, so we don't leak a running container with no row pointing
|
||||
at it.
|
||||
- **`primaryDomain` reads `settings.Domain`** to complete a bare
|
||||
subdomain face into a full FQDN (matches legacy Manager behavior).
|
||||
- **`time.Sleep` honors `ctx.Done()`** during the post-start health
|
||||
window.
|
||||
- **`json.Marshal` for event metadata + `strings.HasPrefix` for
|
||||
failed-status detection** — replaces the prior fmt.Sprintf JSON
|
||||
template + brittle slice expression.
|
||||
|
||||
**Touch points (final):**
|
||||
|
||||
- `internal/workload/plugin/source/static/{static,deploy,teardown,
|
||||
reconcile,state,env,build,naming}.go` — the inline plugin.
|
||||
- `internal/staticsite/{gitea_content,github_provider,
|
||||
gitlab_provider}.go` — added the path-traversal guards.
|
||||
- `cmd/server/main.go` — `wireStaticBackend(...)` call removed; the
|
||||
existing blank import on `_ "internal/workload/plugin/source/
|
||||
static"` now drives `init()` registration.
|
||||
- `cmd/server/static_backend.go` — deleted.
|
||||
|
||||
**Behavioral notes for operators:**
|
||||
|
||||
- Plugin-native static workloads no longer write to the `static_sites`
|
||||
table at all — anything querying that table for plugin-native
|
||||
workloads (operator dashboards, ad-hoc SQL) sees stale or absent
|
||||
values. The legacy `/api/sites/*` routes still serve original rows
|
||||
unchanged.
|
||||
- Container labels `tinyforge.static-site` / `tinyforge.static-site-name`
|
||||
are no longer set on plugin-native deploys; the canonical
|
||||
`tinyforge.workload.id` / `.kind` labels (added by
|
||||
`docker.ContainerConfig`) cover ownership.
|
||||
- Container, image, and volume names all gained an 8-char ID suffix
|
||||
(e.g. `dw-site-mysite-a1b2c3d4`). Existing legacy-deployed sites
|
||||
keep their old `dw-site-mysite` shape until they're redeployed
|
||||
through the plugin path.
|
||||
|
||||
### Hard legacy cutover
|
||||
|
||||
Sole remaining blocker is the static source inline port above. The
|
||||
generalized-volume-scopes blocker is resolved (legacy `ResolvePath`
|
||||
stays in place for legacy callers and dies with the cutover). When the
|
||||
static port lands:
|
||||
The static-source inline port (above) is now complete; the cutover is
|
||||
unblocked. Proceeding with the cutover means:
|
||||
|
||||
- Delete `/api/projects`, `/api/stacks`, `/api/sites`, `/api/stages` handlers.
|
||||
- Drop tables: `projects`, `stages`, `stacks`, `stack_revisions`,
|
||||
|
||||
Reference in New Issue
Block a user