feat(static): inline static-source plugin; drop phantom-row adapter
Build / build (push) Successful in 10m43s
Build / build (push) Successful in 10m43s
Lift the static-site deploy pipeline from internal/staticsite/manager.go into internal/workload/plugin/source/static/ so plugin-native static workloads operate directly on plugin.Workload + the containers table + workload_env. The cmd/server/static_backend.go phantom-row adapter is gone; the legacy static_sites table is no longer touched on plugin deploys. Backend - new state.go: runtimeState (last_commit_sha, last_sync_at, last_error, status) persisted in containers.extra_json under the deterministic row id <workloadID>:site - per-workload sync.Mutex serializes saveState read-modify-write so parallel deploys for the same workload can't race container_id / proxy_route_id writes - extra_json round-trips through map[string]json.RawMessage so unknown keys survive — typed runtimeStateKeys are stripped before merge so clearing a typed field actually drops the key - new env.go reads workload_env (replaces static_site_secrets for plugin-native sites); decrypt-failure logs and skips one entry rather than failing the whole deploy - new build.go ports prepareDenoBuild + prepareStaticBuild + copyDir; copyDir uses filepath.WalkDir + Lstat to refuse symlinks and non-regular files - new deploy.go is the ~300-line core; intent.Reason gates force vs skip-if-no-changes; success-path saveState failure rolls back container + proxy route and writes "failed" state (no orphans) - new teardown.go combines Remove + Stop; idempotent on never-deployed workloads - new reconcile.go refreshes container state from Docker; flips runtimeState.Status to failed when the container is missing/crashed Hardening (from go-reviewer + security-reviewer subagent passes; 1 CRITICAL + 5 HIGH + 3 MEDIUM addressed before merge) - path-traversal defense in all 3 providers (gitea_content, github_provider, gitlab_provider): reject tree entries whose resolved local path escapes destDir - verifyDownloadInsideRoot walks the build dir post-download as a second line of defense - sanitizeError redacts the access token, collapses to one line, and clamps to 240 bytes before persisting to extra_json or fanning out to the notification webhook - container/image/volume names suffixed with workload-id short prefix (workload name is not UNIQUE in schema) - primaryDomain reads settings.Domain to complete a bare subdomain face into a full FQDN (matches legacy Manager behavior) - ctx-aware health-check sleep - json.Marshal for event metadata (was fmt.Sprintf JSON template) - strings.HasPrefix for failed-status detection (was brittle slice expression) Wire-up - cmd/server/main.go: removed wireStaticBackend(...) call; existing blank import on _ ".../source/static" drives init() registration - cmd/server/static_backend.go deleted Doc - WORKLOAD_REFACTOR_TODO: static port marked DONE; next focus is the hard legacy cutover (drop /api/projects, /api/stacks, /api/sites, /api/stages + their tables, internal/stack + internal/staticsite packages, frontend /projects /stacks /sites) Behavior notes for operators - plugin-native static workloads no longer write to static_sites; legacy /api/sites/* still serves original rows unchanged - legacy tinyforge.static-site / .static-site-name container labels dropped on plugin deploys; canonical tinyforge.workload.id / .kind cover ownership - container/image/volume names gained an 8-char ID suffix (e.g. dw-site-mysite-a1b2c3d4); legacy-deployed sites keep the old shape until redeployed through the plugin path
This commit is contained in:
@@ -0,0 +1,180 @@
|
||||
package static
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"sync"
|
||||
|
||||
"github.com/alexei/tinyforge/internal/store"
|
||||
"github.com/alexei/tinyforge/internal/workload/plugin"
|
||||
)
|
||||
|
||||
// runtimeState is the per-workload state that the legacy static_sites
|
||||
// table used to track on its own row (last_commit_sha, last_sync_at,
|
||||
// status/error). With the cutover off the synthetic-row adapter these
|
||||
// fields live inside the container row's extra_json blob keyed by the
|
||||
// deterministic row ID `<workloadID>:site`.
|
||||
//
|
||||
// Unknown keys in extra_json are preserved across read+write so future
|
||||
// writers (e.g. per-face route maps) can extend the blob without
|
||||
// forcing this struct to grow. Decoding into a typed wrapper on its
|
||||
// own would silently drop them; saveState round-trips through a
|
||||
// generic map first, then merges the typed fields.
|
||||
type runtimeState struct {
|
||||
LastCommitSHA string `json:"last_commit_sha,omitempty"`
|
||||
LastSyncAt string `json:"last_sync_at,omitempty"`
|
||||
LastError string `json:"last_error,omitempty"`
|
||||
// Status mirrors the legacy static_sites.status column ("syncing",
|
||||
// "deployed", "failed", "stopped"). Kept in extra_json so callers
|
||||
// can still answer "is this site healthy?" without a Docker probe.
|
||||
Status string `json:"status,omitempty"`
|
||||
}
|
||||
|
||||
// runtimeStateKeys lists every JSON field name owned by runtimeState.
|
||||
// saveState strips these from the generic map before re-emitting so
|
||||
// the typed values don't double-write under both their JSON tag and
|
||||
// any subsequent extension's tag — and so that clearing a field
|
||||
// (LastError → "") actually removes the key instead of being shadowed
|
||||
// by a stale carry-over.
|
||||
var runtimeStateKeys = []string{"last_commit_sha", "last_sync_at", "last_error", "status"}
|
||||
|
||||
// containerRowID is the deterministic ID for the single container row
|
||||
// owned by a static workload. Stable across redeploys so saveState can
|
||||
// upsert in place.
|
||||
func containerRowID(w plugin.Workload) string {
|
||||
return w.ID + ":site"
|
||||
}
|
||||
|
||||
// loadState returns the persisted runtime state plus the underlying
|
||||
// container row. Both values are zero on first deploy (no row yet);
|
||||
// callers must tolerate a nil container without treating it as an
|
||||
// error.
|
||||
func loadState(deps plugin.Deps, w plugin.Workload) (runtimeState, *store.Container, error) {
|
||||
row, err := deps.Store.GetContainerByID(containerRowID(w))
|
||||
if err != nil {
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return runtimeState{}, nil, nil
|
||||
}
|
||||
return runtimeState{}, nil, fmt.Errorf("static source: load state: %w", err)
|
||||
}
|
||||
st := runtimeState{}
|
||||
if row.ExtraJSON != "" && row.ExtraJSON != "{}" {
|
||||
if err := json.Unmarshal([]byte(row.ExtraJSON), &st); err != nil {
|
||||
// The row is the source of truth for container_id /
|
||||
// proxy_route_id; only the optional state blob is at risk
|
||||
// and we fall back to zero state. Log so this is debuggable
|
||||
// after the fact.
|
||||
slog.Debug("static source: decode extra_json", "workload", w.ID, "error", err)
|
||||
}
|
||||
}
|
||||
return st, &row, nil
|
||||
}
|
||||
|
||||
// saveLocks serializes per-workload read-modify-write of the container
|
||||
// row inside saveState. With SQLite's MaxOpenConns=1, two parallel
|
||||
// deploys for the same workload would still race the read+write
|
||||
// against each other (the DB serializes statements but not the
|
||||
// caller's intent), letting the loser's write stomp the winner's
|
||||
// container_id / proxy_route_id and orphaning Docker resources. The
|
||||
// mutex caps the concurrency at 1 per workload; cross-workload
|
||||
// parallelism is unaffected.
|
||||
var saveLocks struct {
|
||||
mu sync.Mutex
|
||||
locks map[string]*sync.Mutex
|
||||
}
|
||||
|
||||
// lockFor returns the per-workload mutex, creating it on first use.
|
||||
// The outer mutex is held only briefly during map lookup; the returned
|
||||
// per-workload lock is what callers actually contend on.
|
||||
func lockFor(workloadID string) *sync.Mutex {
|
||||
saveLocks.mu.Lock()
|
||||
defer saveLocks.mu.Unlock()
|
||||
if saveLocks.locks == nil {
|
||||
saveLocks.locks = map[string]*sync.Mutex{}
|
||||
}
|
||||
m, ok := saveLocks.locks[workloadID]
|
||||
if !ok {
|
||||
m = &sync.Mutex{}
|
||||
saveLocks.locks[workloadID] = m
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// saveState upserts the container row, calling mutate so callers can
|
||||
// adjust both the runtime state blob (extra_json) and the row's
|
||||
// first-class fields (container_id, proxy_route_id, state, etc.) in a
|
||||
// single transaction.
|
||||
//
|
||||
// The mutate callback receives a pointer to a runtimeState seeded from
|
||||
// the existing extra_json; on save the typed fields are merged back on
|
||||
// top of any unknown keys so future-writer values (e.g. per-face
|
||||
// route maps) survive the round-trip.
|
||||
//
|
||||
// Per-workload mutex serializes concurrent callers so two parallel
|
||||
// Deploys can't read the same prior state and race their writes.
|
||||
func saveState(deps plugin.Deps, w plugin.Workload, mutate func(*runtimeState, *store.Container)) error {
|
||||
lk := lockFor(w.ID)
|
||||
lk.Lock()
|
||||
defer lk.Unlock()
|
||||
|
||||
prev, prevRow, err := loadState(deps, w)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
row := store.Container{
|
||||
ID: containerRowID(w),
|
||||
WorkloadID: w.ID,
|
||||
WorkloadKind: string(store.WorkloadKindSite),
|
||||
Host: "local",
|
||||
}
|
||||
if prevRow != nil {
|
||||
row = *prevRow
|
||||
}
|
||||
|
||||
// Round-trip extra_json through a generic map so unknown keys
|
||||
// survive. Strip the typed-state keys before the merge so that
|
||||
// a typed field cleared to its zero value (e.g. LastError = "")
|
||||
// actually drops the key instead of being re-introduced by the
|
||||
// generic decode.
|
||||
generic := map[string]json.RawMessage{}
|
||||
if row.ExtraJSON != "" && row.ExtraJSON != "{}" {
|
||||
if err := json.Unmarshal([]byte(row.ExtraJSON), &generic); err != nil {
|
||||
slog.Debug("static source: decode extra_json (generic)", "workload", w.ID, "error", err)
|
||||
}
|
||||
}
|
||||
for _, k := range runtimeStateKeys {
|
||||
delete(generic, k)
|
||||
}
|
||||
|
||||
state := prev
|
||||
mutate(&state, &row)
|
||||
|
||||
// Re-emit typed fields into the generic map so they win over any
|
||||
// historical key with the same name.
|
||||
typedBytes, err := json.Marshal(state)
|
||||
if err != nil {
|
||||
return fmt.Errorf("static source: marshal state: %w", err)
|
||||
}
|
||||
typedMap := map[string]json.RawMessage{}
|
||||
if err := json.Unmarshal(typedBytes, &typedMap); err != nil {
|
||||
return fmt.Errorf("static source: re-decode typed state: %w", err)
|
||||
}
|
||||
for k, v := range typedMap {
|
||||
generic[k] = v
|
||||
}
|
||||
|
||||
merged, err := json.Marshal(generic)
|
||||
if err != nil {
|
||||
return fmt.Errorf("static source: marshal merged state: %w", err)
|
||||
}
|
||||
row.ExtraJSON = string(merged)
|
||||
row.LastSeenAt = store.Now()
|
||||
|
||||
if err := deps.Store.UpsertContainer(row); err != nil {
|
||||
return fmt.Errorf("static source: upsert container row: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user