feat(cutover): hard legacy cutover — drop projects/stacks/sites/deploys
Build / build (push) Successful in 10m39s

The clean-break delete that closes the workload-first refactor arc.
Net diff: ~30 backend files deleted, ~20 modified, ~12k LOC removed
on the Go side; entire /projects /stacks /sites /deploy frontend
trees gone; ~6.7k LOC removed on the Svelte/TypeScript side.

Backend
- API handlers gone: internal/api/{projects,stages,stage_env,stacks,
  static_sites,deploys,instances,volume_browser}.go
- Store CRUD + tests gone: internal/store/{projects,stages,stage_env,
  stacks,static_sites,static_site_secrets,deploys,poll_state,volumes,
  workload_sync}.go (+ _test.go siblings)
- Legacy deployer pipeline gone: internal/deployer/{bluegreen,promote,
  rollback,subdomain,resolver_test}.go; deployer.go trimmed to just the
  dispatch surface used by the plugin pipeline
- internal/staticsite/{manager,healthcheck}.go and
  internal/stack/manager.go gone (the rest of those packages stay as
  helpers imported by the static + compose plugins)
- internal/registry/poller.go gone (legacy registry poller)
- internal/volume.ResolvePath gone; ResolveWorkloadPath stays
- internal/webhook: handleWebhook (project) + handleSiteWebhook (site)
  gone; only POST /api/webhook/triggers/{secret} remains
- workload-side webhook URL handlers (getWorkloadWebhook +
  regenerateWorkloadWebhook + EnsureWorkloadWebhookSecret +
  SetWorkloadWebhookSecret + GetWorkloadByWebhookSecret) gone — they
  minted URLs that would 404 against the new trigger-only ingress
- cmd/server/main.go: dropped staticsite.Manager, stack.Manager,
  staticsite.HealthChecker, registry poller, SetSiteSyncTriggerer,
  SetStaticSiteManager, SetStackManager, wireStaticBackend
- store/store.go: idempotent DROP TABLE IF EXISTS for every legacy
  table (projects, stages, stage_env, volumes, deploys, deploy_logs,
  poll_states, stacks, stack_revisions, stack_deploys, static_sites,
  static_site_secrets); FK order children-then-parents
- store/models.go: dropped Project, Stage, Deploy, DeployLog, StageEnv,
  Volume, StaticSite, StaticSiteSecret, Stack, StackRevision,
  StackDeploy types; kept WorkloadKind constants as documented strings
- internal/store/helpers.go (new): BoolToInt, rowScanner,
  GenerateWebhookSecret extracted from deleted CRUD files
- internal/api/secrets.go (new): forwards to store.GenerateWebhookSecret
  so api + store paths share one secret-generation impl (no
  panic-vs-UUID-fallback divergence)
- internal/reconciler/reconciler.go: dropped legacy stack-by-compose
  + static-site label paths; only canonical tinyforge.workload.id
  dispatch remains
- providers (gitea_content/github_provider/gitlab_provider) gained
  path-traversal rejection on every tree entry
- internal/webhook ParsedImage / ParseImageRef demoted to package-
  private (no external callers)

Frontend
- /projects /stacks /sites /deploy routes deleted (entire trees)
- ProjectCard / InstanceCard / StaleContainerCard components deleted
- api.ts: dropped every project/stage/stack/site/deploy/instance
  helper + types (Project, Stage, Stack, StaticSite, Deploy,
  Instance, Volume, etc.); kept Workload, Container, App, Settings,
  Registry, EventTrigger, LogScanRule, webhook envelopes
- WorkloadWebhook type + getWorkloadWebhook/regenerateWorkloadWebhook
  api functions gone (mirror of the backend deletion above)
- web/src/routes/+layout.svelte: dropped /projects /sites /stacks
  /deploy nav entries, trimmed quick-nav keymap
- web/src/routes/+page.svelte: dashboard rewrite — reads
  listWorkloads + listContainers only; 4-card stat grid
  (workloads/running/failed/stale) + recent workloads strip
- navCounts.ts, SystemHealthCard.svelte, ContainerLogs.svelte,
  ContainerStats.svelte, StatusBadge.svelte, TagCombobox.svelte,
  proxies/+page.svelte, containers/+page.svelte all rewired to the
  workload-first surface
- AbortController plumbing on dashboard, nav-counts, stale page,
  SystemHealthCard so navigation doesn't leave dangling fetches
- i18n: dropped projects.*, projectDetail.*, envEditor.*,
  volumeEditor.*, volumeBrowser.*, quickDeploy.*, sites.*, stacks.*,
  instance.*, confirm.* namespaces; en/ru parity preserved (1042
  keys each)

Hardening from go-reviewer + security-reviewer + typescript-reviewer
subagent passes (0 CRITICAL across all three; 1 HIGH + ~12 MEDIUM
addressed inline before commit):

- Sec H1: dead-end workload webhook URL handlers (would mint URLs
  that 404 the new trigger-only ingress) deleted across backend +
  frontend
- Go M1: IsTerminalDeployStatus dropped (no production callers)
- Go M2: ParsedImage/ParseImageRef lowercased (in-package only)
- Go M6: generateWebhookSecret unified — api shim forwards to
  store.GenerateWebhookSecret
- Doc/comment freshness: stage_id (no longer FK), ProxyRoute legacy
  field names, workloadIDRow rationale, webhook_deliveries.target_type
  enum, WebhookDeliveryLog component header

Doc
- WORKLOAD_REFACTOR_TODO: cutover marked DONE; all three Priority 1
  items are now shipped. Next focus is Priority 3 polish (apps.* i18n
  + codemap entries) and Priority 4 tests.

Behavioral notes for operators upgrading from a pre-cutover build
- Existing rows in the dropped tables disappear on first boot.
- Legacy webhook URLs at /api/webhook/{secret} and
  /api/webhook/sites/{secret} return 404; CI configs must repoint to
  /api/webhook/triggers/{secret} (the trigger-split boot backfill
  lifted any embedded workload secret onto a Trigger row, so the
  secret value itself carries over).
- Frontend routes /projects /stacks /sites /deploy are gone; nav
  links replaced with /apps and /triggers.
This commit is contained in:
2026-05-16 06:00:21 +03:00
parent 234c3c711e
commit 739b67856a
101 changed files with 1116 additions and 20768 deletions
+59 -128
View File
@@ -3,7 +3,6 @@ package api
import (
"bufio"
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
@@ -14,17 +13,15 @@ import (
"sync"
"time"
"github.com/go-chi/chi/v5"
"github.com/alexei/tinyforge/internal/store"
)
// Limits and constants for the log endpoints.
const (
defaultLogTail = 200
maxLogTail = 5000
maxJSONLogBytes = 4 << 20 // 4 MiB cap for non-streaming log responses
maxLogLineBytes = 1 << 20 // 1 MiB max line length for the bufio.Scanner
defaultLogTail = 200
maxLogTail = 5000
maxJSONLogBytes = 4 << 20 // 4 MiB cap for non-streaming log responses
maxLogLineBytes = 1 << 20 // 1 MiB max line length for the bufio.Scanner
logHeartbeatPeriod = 20 * time.Second
)
@@ -37,82 +34,8 @@ var (
ctlBytePattern = regexp.MustCompile(`[\x00-\x08\x0b-\x1a\x1c-\x1f\x7f]`)
)
// listProjectImages handles GET /api/projects/{id}/images.
// Returns all local Docker images matching the project's image reference.
func (s *Server) listProjectImages(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
project, err := s.store.GetProjectByID(id)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
respondNotFound(w, "project")
return
}
slog.Error("failed to get project", "error", err)
respondError(w, http.StatusInternalServerError, "internal server error")
return
}
if s.docker == nil || project.Image == "" {
respondJSON(w, http.StatusOK, []any{})
return
}
images, err := s.docker.ListImagesByRef(r.Context(), project.Image)
if err != nil {
slog.Warn("list project images", "project", project.Name, "error", err)
respondJSON(w, http.StatusOK, []any{})
return
}
respondJSON(w, http.StatusOK, images)
}
// streamContainerLogs handles GET /api/projects/{id}/stages/{stage}/instances/{iid}/logs.
// Streams container logs via SSE. {iid} is the container row ID. Ownership is
// verified by joining through workload + stage so an attacker cannot stream
// logs for a foreign container by guessing IDs under the wrong project URL.
func (s *Server) streamContainerLogs(w http.ResponseWriter, r *http.Request) {
projectID := chi.URLParam(r, "id")
stageID := chi.URLParam(r, "stage")
containerRowID := chi.URLParam(r, "iid")
c, err := s.store.GetContainerByID(containerRowID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
respondNotFound(w, "container")
return
}
slog.Error("failed to get container", "error", err)
respondError(w, http.StatusInternalServerError, "internal server error")
return
}
wl, err := s.store.GetWorkloadByID(c.WorkloadID)
if err != nil {
respondNotFound(w, "container")
return
}
stage, err := s.store.GetStageByID(stageID)
if err != nil || stage.ProjectID != projectID {
respondNotFound(w, "container")
return
}
if wl.Kind != string(store.WorkloadKindProject) || wl.RefID != projectID || c.Role != stage.Name {
respondNotFound(w, "container")
return
}
if c.ContainerID == "" {
respondError(w, http.StatusBadRequest, "container row has no docker container bound")
return
}
s.streamLogsForContainer(w, r, c.ContainerID)
}
// streamLogsForContainer streams logs for an arbitrary container ID using the
// shared SSE/JSON dual-mode pattern. Owner-specific handlers (instance, site)
// shared SSE/JSON dual-mode pattern. Owner-specific handlers (workload-container)
// should validate ownership and then delegate here.
func (s *Server) streamLogsForContainer(w http.ResponseWriter, r *http.Request, containerID string) {
if s.docker == nil {
@@ -255,11 +178,7 @@ func sanitizeDockerLogLine(line string) string {
// by any container, computed in a single DB pass against the normalized
// containers index. Returning an error (rather than swallowing) prevents
// prune logic from treating a transient DB failure as "nothing is active".
func buildActiveImagesSet(st *store.Store, projects []store.Project) (map[string]bool, error) {
// `projects` is unused now — kept in the signature for back-compat with
// callers that already happen to have the slice. The image_ref column
// holds the full "image:tag" string written by the deployer.
_ = projects
func buildActiveImagesSet(st *store.Store) (map[string]bool, error) {
containers, err := st.ListContainers(store.ContainerFilter{})
if err != nil {
return nil, fmt.Errorf("list containers: %w", err)
@@ -274,8 +193,43 @@ func buildActiveImagesSet(st *store.Store, projects []store.Project) (map[string
return active, nil
}
// unusedImageStats handles GET /api/docker/unused-images.
// Returns the total size of unused project images and whether the threshold is exceeded.
// workloadImageBases returns the set of "image" strings (no tag) that
// some workload currently mounts to, derived from container.image_ref.
// This replaces the legacy "list all projects → projects[].Image" view
// after the workload-first cutover.
func workloadImageBases(st *store.Store) (map[string]bool, error) {
containers, err := st.ListContainers(store.ContainerFilter{})
if err != nil {
return nil, fmt.Errorf("list containers: %w", err)
}
bases := make(map[string]bool, len(containers))
for _, c := range containers {
if c.ImageRef == "" {
continue
}
ref, _ := splitImageTag(c.ImageRef)
if ref != "" {
bases[ref] = true
}
}
return bases, nil
}
// splitImageTag splits "image:tag" into image and tag parts. Returns the
// full string and empty tag if no colon separator is found. Inlined here
// because the legacy deploys.go that owned it was removed.
func splitImageTag(ref string) (string, string) {
if idx := strings.LastIndex(ref, ":"); idx != -1 {
afterColon := ref[idx+1:]
if !strings.Contains(afterColon, "/") {
return ref[:idx], afterColon
}
}
return ref, ""
}
// unusedImageStats handles GET /api/docker/unused-images. Returns the total
// size of unused workload images and whether the threshold is exceeded.
func (s *Server) unusedImageStats(w http.ResponseWriter, r *http.Request) {
if s.docker == nil {
respondJSON(w, http.StatusOK, map[string]any{
@@ -291,32 +245,25 @@ func (s *Server) unusedImageStats(w http.ResponseWriter, r *http.Request) {
return
}
projects, err := s.store.GetAllProjects()
imageBases, err := workloadImageBases(s.store)
if err != nil {
slog.Error("unused images: list projects", "error", err)
slog.Error("unused images: list workload images", "error", err)
respondError(w, http.StatusInternalServerError, "internal server error")
return
}
// Build set of active image refs in one DB pass instead of N×K queries.
// A flaky read here previously masqueraded as "no images are active",
// which on the prune endpoint would have deleted *running* images.
activeImages, err := buildActiveImagesSet(s.store, projects)
activeImages, err := buildActiveImagesSet(s.store)
if err != nil {
slog.Error("unused images: build active set", "error", err)
respondError(w, http.StatusInternalServerError, "internal server error")
return
}
// Sum unused image sizes.
ctx := r.Context()
var totalSize int64
var count int
for _, p := range projects {
if p.Image == "" {
continue
}
images, err := s.docker.ListImagesByRef(ctx, p.Image)
for base := range imageBases {
images, err := s.docker.ListImagesByRef(ctx, base)
if err != nil {
continue
}
@@ -339,69 +286,53 @@ func (s *Server) unusedImageStats(w http.ResponseWriter, r *http.Request) {
})
}
// pruneImages handles POST /api/docker/prune-images.
// Only removes images that belong to Tinyforge projects (not all system images).
// pruneImages handles POST /api/docker/prune-images. Only removes images that
// some workload references (via container.image_ref), never arbitrary host
// images.
func (s *Server) pruneImages(w http.ResponseWriter, r *http.Request) {
if s.docker == nil {
respondError(w, http.StatusServiceUnavailable, "Docker is not available")
return
}
// Collect all image references from our projects.
projects, err := s.store.GetAllProjects()
imageBases, err := workloadImageBases(s.store)
if err != nil {
slog.Error("prune: failed to list projects", "error", err)
slog.Error("prune: list workload images", "error", err)
respondError(w, http.StatusInternalServerError, "internal server error")
return
}
// Build a set of image refs used by active instances. Bail out on error
// — silently treating a DB blip as "no active images" would prune
// images currently in use by running containers.
activeImages, err := buildActiveImagesSet(s.store, projects)
activeImages, err := buildActiveImagesSet(s.store)
if err != nil {
slog.Error("prune: build active set", "error", err)
respondError(w, http.StatusInternalServerError, "internal server error")
return
}
// Collect all unique image bases from projects (without tags).
projectImages := make(map[string]bool)
for _, p := range projects {
if p.Image != "" {
projectImages[p.Image] = true
}
}
if len(projectImages) == 0 {
if len(imageBases) == 0 {
respondJSON(w, http.StatusOK, map[string]any{
"images_removed": 0,
"space_reclaimed_mb": 0,
"message": "No project images to clean up",
"message": "No workload images to clean up",
})
return
}
// List all local Docker images and find ones matching our projects but not actively used.
ctx := r.Context()
removed := 0
var reclaimedBytes int64
for imageBase := range projectImages {
// List all tags for this image.
images, err := s.docker.ListImagesByRef(ctx, imageBase)
for base := range imageBases {
images, err := s.docker.ListImagesByRef(ctx, base)
if err != nil {
slog.Warn("prune: list images", "image", imageBase, "error", err)
slog.Warn("prune: list images", "image", base, "error", err)
continue
}
for _, img := range images {
// Skip images that are actively used by running instances.
if activeImages[img.Ref] {
continue
}
// Remove unused image.
if err := s.docker.RemoveImage(ctx, img.ID); err != nil {
slog.Warn("prune: remove image", "image", img.Ref, "error", err)
continue