feat(cutover): hard legacy cutover — drop projects/stacks/sites/deploys
Build / build (push) Successful in 10m39s
Build / build (push) Successful in 10m39s
The clean-break delete that closes the workload-first refactor arc.
Net diff: ~30 backend files deleted, ~20 modified, ~12k LOC removed
on the Go side; entire /projects /stacks /sites /deploy frontend
trees gone; ~6.7k LOC removed on the Svelte/TypeScript side.
Backend
- API handlers gone: internal/api/{projects,stages,stage_env,stacks,
static_sites,deploys,instances,volume_browser}.go
- Store CRUD + tests gone: internal/store/{projects,stages,stage_env,
stacks,static_sites,static_site_secrets,deploys,poll_state,volumes,
workload_sync}.go (+ _test.go siblings)
- Legacy deployer pipeline gone: internal/deployer/{bluegreen,promote,
rollback,subdomain,resolver_test}.go; deployer.go trimmed to just the
dispatch surface used by the plugin pipeline
- internal/staticsite/{manager,healthcheck}.go and
internal/stack/manager.go gone (the rest of those packages stay as
helpers imported by the static + compose plugins)
- internal/registry/poller.go gone (legacy registry poller)
- internal/volume.ResolvePath gone; ResolveWorkloadPath stays
- internal/webhook: handleWebhook (project) + handleSiteWebhook (site)
gone; only POST /api/webhook/triggers/{secret} remains
- workload-side webhook URL handlers (getWorkloadWebhook +
regenerateWorkloadWebhook + EnsureWorkloadWebhookSecret +
SetWorkloadWebhookSecret + GetWorkloadByWebhookSecret) gone — they
minted URLs that would 404 against the new trigger-only ingress
- cmd/server/main.go: dropped staticsite.Manager, stack.Manager,
staticsite.HealthChecker, registry poller, SetSiteSyncTriggerer,
SetStaticSiteManager, SetStackManager, wireStaticBackend
- store/store.go: idempotent DROP TABLE IF EXISTS for every legacy
table (projects, stages, stage_env, volumes, deploys, deploy_logs,
poll_states, stacks, stack_revisions, stack_deploys, static_sites,
static_site_secrets); FK order children-then-parents
- store/models.go: dropped Project, Stage, Deploy, DeployLog, StageEnv,
Volume, StaticSite, StaticSiteSecret, Stack, StackRevision,
StackDeploy types; kept WorkloadKind constants as documented strings
- internal/store/helpers.go (new): BoolToInt, rowScanner,
GenerateWebhookSecret extracted from deleted CRUD files
- internal/api/secrets.go (new): forwards to store.GenerateWebhookSecret
so api + store paths share one secret-generation impl (no
panic-vs-UUID-fallback divergence)
- internal/reconciler/reconciler.go: dropped legacy stack-by-compose
+ static-site label paths; only canonical tinyforge.workload.id
dispatch remains
- providers (gitea_content/github_provider/gitlab_provider) gained
path-traversal rejection on every tree entry
- internal/webhook ParsedImage / ParseImageRef demoted to package-
private (no external callers)
Frontend
- /projects /stacks /sites /deploy routes deleted (entire trees)
- ProjectCard / InstanceCard / StaleContainerCard components deleted
- api.ts: dropped every project/stage/stack/site/deploy/instance
helper + types (Project, Stage, Stack, StaticSite, Deploy,
Instance, Volume, etc.); kept Workload, Container, App, Settings,
Registry, EventTrigger, LogScanRule, webhook envelopes
- WorkloadWebhook type + getWorkloadWebhook/regenerateWorkloadWebhook
api functions gone (mirror of the backend deletion above)
- web/src/routes/+layout.svelte: dropped /projects /sites /stacks
/deploy nav entries, trimmed quick-nav keymap
- web/src/routes/+page.svelte: dashboard rewrite — reads
listWorkloads + listContainers only; 4-card stat grid
(workloads/running/failed/stale) + recent workloads strip
- navCounts.ts, SystemHealthCard.svelte, ContainerLogs.svelte,
ContainerStats.svelte, StatusBadge.svelte, TagCombobox.svelte,
proxies/+page.svelte, containers/+page.svelte all rewired to the
workload-first surface
- AbortController plumbing on dashboard, nav-counts, stale page,
SystemHealthCard so navigation doesn't leave dangling fetches
- i18n: dropped projects.*, projectDetail.*, envEditor.*,
volumeEditor.*, volumeBrowser.*, quickDeploy.*, sites.*, stacks.*,
instance.*, confirm.* namespaces; en/ru parity preserved (1042
keys each)
Hardening from go-reviewer + security-reviewer + typescript-reviewer
subagent passes (0 CRITICAL across all three; 1 HIGH + ~12 MEDIUM
addressed inline before commit):
- Sec H1: dead-end workload webhook URL handlers (would mint URLs
that 404 the new trigger-only ingress) deleted across backend +
frontend
- Go M1: IsTerminalDeployStatus dropped (no production callers)
- Go M2: ParsedImage/ParseImageRef lowercased (in-package only)
- Go M6: generateWebhookSecret unified — api shim forwards to
store.GenerateWebhookSecret
- Doc/comment freshness: stage_id (no longer FK), ProxyRoute legacy
field names, workloadIDRow rationale, webhook_deliveries.target_type
enum, WebhookDeliveryLog component header
Doc
- WORKLOAD_REFACTOR_TODO: cutover marked DONE; all three Priority 1
items are now shipped. Next focus is Priority 3 polish (apps.* i18n
+ codemap entries) and Priority 4 tests.
Behavioral notes for operators upgrading from a pre-cutover build
- Existing rows in the dropped tables disappear on first boot.
- Legacy webhook URLs at /api/webhook/{secret} and
/api/webhook/sites/{secret} return 404; CI configs must repoint to
/api/webhook/triggers/{secret} (the trigger-split boot backfill
lifted any embedded workload secret onto a Trigger row, so the
secret value itself carries over).
- Frontend routes /projects /stacks /sites /deploy are gone; nav
links replaced with /apps and /triggers.
This commit is contained in:
@@ -1,21 +1,16 @@
|
||||
// Package reconciler keeps the normalized containers index in sync with the
|
||||
// Docker daemon. It runs on a tick (and one-shot at boot) — for every
|
||||
// Tinyforge-managed container in `docker ps`, it dispatches to a workload by
|
||||
// labels and writes a Container row through ReconcileContainer (which only
|
||||
// touches Docker-derived fields on conflict, never deployer-owned columns
|
||||
// like subdomain / proxy_route_id / npm_proxy_id / image_tag / stage_id).
|
||||
// Rows whose Docker container ID is no longer present are flipped to
|
||||
// state='missing'.
|
||||
// Tinyforge-managed container in `docker ps`, it resolves a workload by the
|
||||
// canonical workload-id label and writes a Container row through
|
||||
// ReconcileContainer (which only touches Docker-derived fields on conflict,
|
||||
// never deployer-owned columns like subdomain / proxy_route_id /
|
||||
// npm_proxy_id / image_tag / stage_id). Rows whose Docker container ID is no
|
||||
// longer present are flipped to state='missing'.
|
||||
//
|
||||
// Dispatch precedence (a container with multiple matching labels is dispatched
|
||||
// by the first match in this order):
|
||||
// 1. tinyforge.workload.id label (canonical, new)
|
||||
// 2. tinyforge.static-site label (legacy site — joins via static_sites)
|
||||
// 3. com.docker.compose.project (stack — joins via Stack.ComposeProjectName)
|
||||
//
|
||||
// The legacy tinyforge.instance-id path was removed when the deployer was
|
||||
// rewritten to use Container natively — every Tinyforge-managed project
|
||||
// container now carries the workload labels at create time.
|
||||
// Only the tinyforge.workload.id label is honored after the hard cutover —
|
||||
// every Source plugin labels its containers with the workload identity at
|
||||
// create time. The legacy tinyforge.static-site / compose-project paths
|
||||
// were dropped along with the static_sites / stacks tables.
|
||||
package reconciler
|
||||
|
||||
import (
|
||||
@@ -23,7 +18,6 @@ import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -118,12 +112,8 @@ func (r *Reconciler) ReconcileOnce(ctx context.Context) error {
|
||||
}
|
||||
seen := make(map[string]struct{}, len(items)) // container row IDs we touched
|
||||
|
||||
// Build a per-pass cache of compose project name → stack ID so we don't
|
||||
// hit the DB once per compose container.
|
||||
stackByCompose := map[string]store.Stack{}
|
||||
|
||||
for _, item := range items {
|
||||
rowID := r.upsertFromItem(item, stackByCompose)
|
||||
rowID := r.upsertFromItem(item)
|
||||
if rowID != "" {
|
||||
seen[rowID] = struct{}{}
|
||||
}
|
||||
@@ -221,16 +211,13 @@ func (r *Reconciler) loop(ctx context.Context) {
|
||||
|
||||
// upsertFromItem dispatches one container to its workload and writes the
|
||||
// Container row. Returns the row ID on success or "" if no dispatch matched.
|
||||
func (r *Reconciler) upsertFromItem(item docker.ReconcileItem, stackCache map[string]store.Stack) string {
|
||||
// After the hard cutover only the canonical tinyforge.workload.id label
|
||||
// path is honored — every Source plugin labels its containers with the
|
||||
// workload identity at create time.
|
||||
func (r *Reconciler) upsertFromItem(item docker.ReconcileItem) string {
|
||||
if id := item.Labels[docker.LabelWorkloadID]; id != "" {
|
||||
return r.upsertByWorkloadLabel(item, id)
|
||||
}
|
||||
if siteID := item.Labels["tinyforge.static-site"]; siteID != "" {
|
||||
return r.upsertBySiteLabel(item, siteID)
|
||||
}
|
||||
if cp := item.Labels["com.docker.compose.project"]; cp != "" && strings.HasPrefix(cp, "tinyforge-") {
|
||||
return r.upsertByComposeProject(item, cp, stackCache)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
@@ -300,9 +287,11 @@ func (r *Reconciler) upsertByWorkloadLabel(item docker.ReconcileItem, workloadID
|
||||
return ""
|
||||
}
|
||||
|
||||
// Site/stack reach this branch only when their kind-specific dispatcher
|
||||
// hasn't run yet (e.g. boot tick before site row is registered). The
|
||||
// site/stack dispatchers below own their own deterministic IDs.
|
||||
// Site/stack reach this branch only when their plugin hasn't yet
|
||||
// upserted the row (e.g. a boot tick that races the first deploy).
|
||||
// The deterministic ID computed here matches what the static and
|
||||
// compose plugins write in their state-save paths, so a subsequent
|
||||
// plugin write upserts in place rather than creating a sibling row.
|
||||
rowID := workloadIDRow(workloadID, kind, role, item.ID)
|
||||
port := 0
|
||||
if len(item.Ports) > 0 {
|
||||
@@ -326,79 +315,6 @@ func (r *Reconciler) upsertByWorkloadLabel(item docker.ReconcileItem, workloadID
|
||||
return rowID
|
||||
}
|
||||
|
||||
func (r *Reconciler) upsertBySiteLabel(item docker.ReconcileItem, siteID string) string {
|
||||
w, err := r.store.GetWorkloadByRef(store.WorkloadKindSite, siteID)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
rowID := w.ID + ":site"
|
||||
port := 0
|
||||
if len(item.Ports) > 0 {
|
||||
port = int(item.Ports[0])
|
||||
}
|
||||
if err := r.store.ReconcileContainer(store.Container{
|
||||
ID: rowID,
|
||||
WorkloadID: w.ID,
|
||||
WorkloadKind: string(store.WorkloadKindSite),
|
||||
Role: "",
|
||||
ContainerID: item.ID,
|
||||
ImageRef: item.Image,
|
||||
Host: "local",
|
||||
State: normalizeState(item.State),
|
||||
Port: port,
|
||||
LastSeenAt: store.Now(),
|
||||
}); err != nil {
|
||||
slog.Warn("reconciler: reconcile by site label", "container_id", item.ID, "error", err)
|
||||
return ""
|
||||
}
|
||||
return rowID
|
||||
}
|
||||
|
||||
func (r *Reconciler) upsertByComposeProject(item docker.ReconcileItem, composeProject string, cache map[string]store.Stack) string {
|
||||
stack, ok := cache[composeProject]
|
||||
if !ok {
|
||||
st, err := r.store.GetStackByComposeProjectName(composeProject)
|
||||
if err != nil {
|
||||
cache[composeProject] = store.Stack{} // negative cache for the rest of the pass
|
||||
return ""
|
||||
}
|
||||
stack = st
|
||||
cache[composeProject] = st
|
||||
}
|
||||
if stack.ID == "" {
|
||||
return ""
|
||||
}
|
||||
w, err := r.store.GetWorkloadByRef(store.WorkloadKindStack, stack.ID)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
role := item.Labels["com.docker.compose.service"]
|
||||
if role == "" {
|
||||
role = item.Name
|
||||
}
|
||||
rowID := w.ID + ":" + role
|
||||
port := 0
|
||||
if len(item.Ports) > 0 {
|
||||
port = int(item.Ports[0])
|
||||
}
|
||||
if err := r.store.ReconcileContainer(store.Container{
|
||||
ID: rowID,
|
||||
WorkloadID: w.ID,
|
||||
WorkloadKind: string(store.WorkloadKindStack),
|
||||
Role: role,
|
||||
ContainerID: item.ID,
|
||||
ImageRef: item.Image,
|
||||
Host: "local",
|
||||
State: normalizeState(item.State),
|
||||
Port: port,
|
||||
LastSeenAt: store.Now(),
|
||||
}); err != nil {
|
||||
slog.Warn("reconciler: reconcile by compose project", "container_id", item.ID, "error", err)
|
||||
return ""
|
||||
}
|
||||
return rowID
|
||||
}
|
||||
|
||||
// markMissingRows flips state to 'missing' for any container row whose Docker
|
||||
// container ID was not seen in this pass. Uses ListMissingSweepRows to scan
|
||||
// only rows that are bound to a real container and not already missing.
|
||||
@@ -419,9 +335,11 @@ func (r *Reconciler) markMissingRows(seen map[string]struct{}) {
|
||||
}
|
||||
|
||||
// workloadIDRow picks the row ID for a non-project workload-labelled
|
||||
// container that has no existing row. Stack rows use workloadID:role; sites
|
||||
// use workloadID:site. Project rows are never invented here — see
|
||||
// upsertByWorkloadLabel for the rationale.
|
||||
// container that has no existing row. Sites use `<workloadID>:site`
|
||||
// (matches the static plugin's `containerRowID` helper). Stack
|
||||
// services use `<workloadID>:<service-role>` (matches the compose
|
||||
// plugin). Project rows are never invented here — the deployer
|
||||
// pre-creates per-instance UUID rows so the reconciler must wait.
|
||||
func workloadIDRow(workloadID, kind, role, containerID string) string {
|
||||
if kind == string(store.WorkloadKindSite) {
|
||||
return workloadID + ":site"
|
||||
|
||||
@@ -28,17 +28,23 @@ func newTestStore(t *testing.T) *store.Store {
|
||||
return s
|
||||
}
|
||||
|
||||
// makeWorkload inserts a workload row with the given kind so reconciler
|
||||
// dispatch can resolve it by ID.
|
||||
func makeWorkload(t *testing.T, st *store.Store, name, kind string) store.Workload {
|
||||
t.Helper()
|
||||
w, err := st.CreateWorkload(store.Workload{
|
||||
Kind: kind, RefID: name + "-ref", Name: name,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateWorkload: %v", err)
|
||||
}
|
||||
return w
|
||||
}
|
||||
|
||||
func TestReconcileWorkloadLabelledStackContainer(t *testing.T) {
|
||||
st := newTestStore(t)
|
||||
|
||||
// Set up a stack workload (no project/site interaction).
|
||||
stack, err := st.CreateStack(store.Stack{
|
||||
Name: "wf-stack", ComposeProjectName: "tinyforge-wf-stack",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateStack: %v", err)
|
||||
}
|
||||
w, _ := st.GetWorkloadByRef(store.WorkloadKindStack, stack.ID)
|
||||
w := makeWorkload(t, st, "wf-stack", "stack")
|
||||
|
||||
// One container with the canonical workload labels stamped.
|
||||
fake := &fakeDocker{items: []docker.ReconcileItem{{
|
||||
@@ -76,51 +82,10 @@ func TestReconcileWorkloadLabelledStackContainer(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileComposeOnlyStackContainer(t *testing.T) {
|
||||
st := newTestStore(t)
|
||||
|
||||
stack, _ := st.CreateStack(store.Stack{
|
||||
Name: "compose-stack", ComposeProjectName: "tinyforge-compose-stack",
|
||||
})
|
||||
w, _ := st.GetWorkloadByRef(store.WorkloadKindStack, stack.ID)
|
||||
|
||||
// Pre-existing compose container — only carries compose's own labels,
|
||||
// no tinyforge.* labels at all.
|
||||
fake := &fakeDocker{items: []docker.ReconcileItem{{
|
||||
ID: "docker-xyz",
|
||||
Name: "tinyforge-compose-stack-worker-1",
|
||||
Image: "redis:7",
|
||||
State: "running",
|
||||
Labels: map[string]string{
|
||||
"com.docker.compose.project": "tinyforge-compose-stack",
|
||||
"com.docker.compose.service": "worker",
|
||||
},
|
||||
}}}
|
||||
|
||||
r := New(st, fake, 0)
|
||||
if err := r.ReconcileOnce(context.Background()); err != nil {
|
||||
t.Fatalf("ReconcileOnce: %v", err)
|
||||
}
|
||||
|
||||
rows, _ := st.ListContainersByWorkload(w.ID)
|
||||
if len(rows) != 1 {
|
||||
t.Fatalf("expected 1 row, got %d", len(rows))
|
||||
}
|
||||
if rows[0].Role != "worker" {
|
||||
t.Fatalf("role from compose label wrong: %q", rows[0].Role)
|
||||
}
|
||||
if rows[0].ContainerID != "docker-xyz" {
|
||||
t.Fatalf("container_id not bound: %q", rows[0].ContainerID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileMarksMissingRows(t *testing.T) {
|
||||
st := newTestStore(t)
|
||||
|
||||
stack, _ := st.CreateStack(store.Stack{
|
||||
Name: "missing-stack", ComposeProjectName: "tinyforge-missing-stack",
|
||||
})
|
||||
w, _ := st.GetWorkloadByRef(store.WorkloadKindStack, stack.ID)
|
||||
w := makeWorkload(t, st, "missing-stack", "stack")
|
||||
|
||||
// Pre-existing row with a real container_id that no longer exists.
|
||||
if err := st.UpsertContainer(store.Container{
|
||||
@@ -145,10 +110,7 @@ func TestReconcileMarksMissingRows(t *testing.T) {
|
||||
func TestReconcileSkipsRowsAwaitingDocker(t *testing.T) {
|
||||
st := newTestStore(t)
|
||||
|
||||
stack, _ := st.CreateStack(store.Stack{
|
||||
Name: "pending", ComposeProjectName: "tinyforge-pending",
|
||||
})
|
||||
w, _ := st.GetWorkloadByRef(store.WorkloadKindStack, stack.ID)
|
||||
w := makeWorkload(t, st, "pending", "stack")
|
||||
|
||||
// A row with empty container_id (deployer placeholder, awaiting docker
|
||||
// create). Reconciler must not mark this as missing.
|
||||
@@ -171,9 +133,8 @@ func TestReconcileSkipsRowsAwaitingDocker(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestReconcileIgnoresUnmanagedContainers(t *testing.T) {
|
||||
// A container without any tinyforge or compose labels would not even be
|
||||
// returned by ListAllForReconciler in production; but the dispatch must
|
||||
// be a no-op even if a stray item slips through.
|
||||
// A container without the canonical workload label is ignored even if
|
||||
// it carries other labels — only tinyforge.workload.id is honored.
|
||||
st := newTestStore(t)
|
||||
fake := &fakeDocker{items: []docker.ReconcileItem{{
|
||||
ID: "docker-foreign", Labels: map[string]string{"app": "other"},
|
||||
@@ -197,11 +158,7 @@ func TestReconcileDoesNotClobberDeployerFields(t *testing.T) {
|
||||
|
||||
// Project workload — exercises the path most affected by the regression
|
||||
// (proxies, blue-green slots, image-tag-based stale detection).
|
||||
project, err := st.CreateProject(store.Project{Name: "p", Image: "nginx"})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateProject: %v", err)
|
||||
}
|
||||
w, _ := st.GetWorkloadByRef(store.WorkloadKindProject, project.ID)
|
||||
w := makeWorkload(t, st, "p", "project")
|
||||
|
||||
// Deployer wrote the row with proxy / subdomain / image_tag / stage_id.
|
||||
deployerRow := store.Container{
|
||||
@@ -277,11 +234,7 @@ func TestReconcileRejectsForgedWorkloadLabel(t *testing.T) {
|
||||
// authoritative writer and inventing rows races with MaxInstances > 1 deploys.
|
||||
func TestReconcileSkipsProjectInsertWithoutDeployerRow(t *testing.T) {
|
||||
st := newTestStore(t)
|
||||
project, err := st.CreateProject(store.Project{Name: "p2", Image: "nginx"})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateProject: %v", err)
|
||||
}
|
||||
w, _ := st.GetWorkloadByRef(store.WorkloadKindProject, project.ID)
|
||||
w := makeWorkload(t, st, "p2", "project")
|
||||
|
||||
// Reconciler sees a real container with project labels but no deployer
|
||||
// row exists yet (race during deploy).
|
||||
@@ -306,10 +259,7 @@ func TestReconcileSkipsProjectInsertWithoutDeployerRow(t *testing.T) {
|
||||
|
||||
func TestReconcileNormalizesState(t *testing.T) {
|
||||
st := newTestStore(t)
|
||||
stack, _ := st.CreateStack(store.Stack{
|
||||
Name: "norm", ComposeProjectName: "tinyforge-norm",
|
||||
})
|
||||
w, _ := st.GetWorkloadByRef(store.WorkloadKindStack, stack.ID)
|
||||
w := makeWorkload(t, st, "norm", "stack")
|
||||
|
||||
fake := &fakeDocker{items: []docker.ReconcileItem{{
|
||||
ID: "docker-1",
|
||||
|
||||
Reference in New Issue
Block a user