Files
tiny-forge/internal/store/store.go
T
alexei.dolgolyov 739b67856a
Build / build (push) Successful in 10m39s
feat(cutover): hard legacy cutover — drop projects/stacks/sites/deploys
The clean-break delete that closes the workload-first refactor arc.
Net diff: ~30 backend files deleted, ~20 modified, ~12k LOC removed
on the Go side; entire /projects /stacks /sites /deploy frontend
trees gone; ~6.7k LOC removed on the Svelte/TypeScript side.

Backend
- API handlers gone: internal/api/{projects,stages,stage_env,stacks,
  static_sites,deploys,instances,volume_browser}.go
- Store CRUD + tests gone: internal/store/{projects,stages,stage_env,
  stacks,static_sites,static_site_secrets,deploys,poll_state,volumes,
  workload_sync}.go (+ _test.go siblings)
- Legacy deployer pipeline gone: internal/deployer/{bluegreen,promote,
  rollback,subdomain,resolver_test}.go; deployer.go trimmed to just the
  dispatch surface used by the plugin pipeline
- internal/staticsite/{manager,healthcheck}.go and
  internal/stack/manager.go gone (the rest of those packages stay as
  helpers imported by the static + compose plugins)
- internal/registry/poller.go gone (legacy registry poller)
- internal/volume.ResolvePath gone; ResolveWorkloadPath stays
- internal/webhook: handleWebhook (project) + handleSiteWebhook (site)
  gone; only POST /api/webhook/triggers/{secret} remains
- workload-side webhook URL handlers (getWorkloadWebhook +
  regenerateWorkloadWebhook + EnsureWorkloadWebhookSecret +
  SetWorkloadWebhookSecret + GetWorkloadByWebhookSecret) gone — they
  minted URLs that would 404 against the new trigger-only ingress
- cmd/server/main.go: dropped staticsite.Manager, stack.Manager,
  staticsite.HealthChecker, registry poller, SetSiteSyncTriggerer,
  SetStaticSiteManager, SetStackManager, wireStaticBackend
- store/store.go: idempotent DROP TABLE IF EXISTS for every legacy
  table (projects, stages, stage_env, volumes, deploys, deploy_logs,
  poll_states, stacks, stack_revisions, stack_deploys, static_sites,
  static_site_secrets); FK order children-then-parents
- store/models.go: dropped Project, Stage, Deploy, DeployLog, StageEnv,
  Volume, StaticSite, StaticSiteSecret, Stack, StackRevision,
  StackDeploy types; kept WorkloadKind constants as documented strings
- internal/store/helpers.go (new): BoolToInt, rowScanner,
  GenerateWebhookSecret extracted from deleted CRUD files
- internal/api/secrets.go (new): forwards to store.GenerateWebhookSecret
  so api + store paths share one secret-generation impl (no
  panic-vs-UUID-fallback divergence)
- internal/reconciler/reconciler.go: dropped legacy stack-by-compose
  + static-site label paths; only canonical tinyforge.workload.id
  dispatch remains
- providers (gitea_content/github_provider/gitlab_provider) gained
  path-traversal rejection on every tree entry
- internal/webhook ParsedImage / ParseImageRef demoted to package-
  private (no external callers)

Frontend
- /projects /stacks /sites /deploy routes deleted (entire trees)
- ProjectCard / InstanceCard / StaleContainerCard components deleted
- api.ts: dropped every project/stage/stack/site/deploy/instance
  helper + types (Project, Stage, Stack, StaticSite, Deploy,
  Instance, Volume, etc.); kept Workload, Container, App, Settings,
  Registry, EventTrigger, LogScanRule, webhook envelopes
- WorkloadWebhook type + getWorkloadWebhook/regenerateWorkloadWebhook
  api functions gone (mirror of the backend deletion above)
- web/src/routes/+layout.svelte: dropped /projects /sites /stacks
  /deploy nav entries, trimmed quick-nav keymap
- web/src/routes/+page.svelte: dashboard rewrite — reads
  listWorkloads + listContainers only; 4-card stat grid
  (workloads/running/failed/stale) + recent workloads strip
- navCounts.ts, SystemHealthCard.svelte, ContainerLogs.svelte,
  ContainerStats.svelte, StatusBadge.svelte, TagCombobox.svelte,
  proxies/+page.svelte, containers/+page.svelte all rewired to the
  workload-first surface
- AbortController plumbing on dashboard, nav-counts, stale page,
  SystemHealthCard so navigation doesn't leave dangling fetches
- i18n: dropped projects.*, projectDetail.*, envEditor.*,
  volumeEditor.*, volumeBrowser.*, quickDeploy.*, sites.*, stacks.*,
  instance.*, confirm.* namespaces; en/ru parity preserved (1042
  keys each)

Hardening from go-reviewer + security-reviewer + typescript-reviewer
subagent passes (0 CRITICAL across all three; 1 HIGH + ~12 MEDIUM
addressed inline before commit):

- Sec H1: dead-end workload webhook URL handlers (would mint URLs
  that 404 the new trigger-only ingress) deleted across backend +
  frontend
- Go M1: IsTerminalDeployStatus dropped (no production callers)
- Go M2: ParsedImage/ParseImageRef lowercased (in-package only)
- Go M6: generateWebhookSecret unified — api shim forwards to
  store.GenerateWebhookSecret
- Doc/comment freshness: stage_id (no longer FK), ProxyRoute legacy
  field names, workloadIDRow rationale, webhook_deliveries.target_type
  enum, WebhookDeliveryLog component header

Doc
- WORKLOAD_REFACTOR_TODO: cutover marked DONE; all three Priority 1
  items are now shipped. Next focus is Priority 3 polish (apps.* i18n
  + codemap entries) and Priority 4 tests.

Behavioral notes for operators upgrading from a pre-cutover build
- Existing rows in the dropped tables disappear on first boot.
- Legacy webhook URLs at /api/webhook/{secret} and
  /api/webhook/sites/{secret} return 404; CI configs must repoint to
  /api/webhook/triggers/{secret} (the trigger-split boot backfill
  lifted any embedded workload secret onto a Trigger row, so the
  secret value itself carries over).
- Frontend routes /projects /stacks /sites /deploy are gone; nav
  links replaced with /apps and /triggers.
2026-05-16 06:00:21 +03:00

662 lines
28 KiB
Go

package store
import (
"database/sql"
"errors"
"fmt"
"log/slog"
"strings"
"time"
"github.com/google/uuid"
_ "modernc.org/sqlite"
)
// ErrNotFound is returned when a requested entity does not exist.
var ErrNotFound = errors.New("not found")
// ErrUnique is returned when a write violates a UNIQUE constraint.
// Translating the driver-specific message at the store boundary lets
// callers use errors.Is instead of fragile substring matching on
// err.Error(); the SQLite driver's wording is not part of any contract.
var ErrUnique = errors.New("unique constraint violation")
// translateSQLError maps a driver-level error onto one of the store's
// sentinel errors when possible. Returns the original error unchanged
// when no mapping applies. The returned error wraps the original via
// %w so callers that need the raw message still have it.
func translateSQLError(err error) error {
if err == nil {
return nil
}
msg := err.Error()
// modernc.org/sqlite returns text like
// "constraint failed: UNIQUE constraint failed: triggers.name (2067)"
// Match case-insensitively in case the driver wording shifts.
if strings.Contains(strings.ToUpper(msg), "UNIQUE") {
return fmt.Errorf("%w: %v", ErrUnique, err)
}
return err
}
// Store wraps the SQLite database connection and provides access to all query methods.
type Store struct {
db *sql.DB
}
// New opens a SQLite database at the given path and runs auto-migration.
func New(dbPath string) (*Store, error) {
db, err := sql.Open("sqlite", dbPath)
if err != nil {
return nil, fmt.Errorf("open database: %w", err)
}
// SQLite only allows one writer at a time. Limit connections to prevent SQLITE_BUSY.
db.SetMaxOpenConns(1)
db.SetConnMaxLifetime(0)
// Enable WAL mode and foreign keys for better concurrency and referential integrity.
pragmas := []string{
"PRAGMA journal_mode=WAL",
"PRAGMA foreign_keys=ON",
"PRAGMA busy_timeout=5000",
}
for _, p := range pragmas {
if _, err := db.Exec(p); err != nil {
db.Close()
return nil, fmt.Errorf("exec pragma %q: %w", p, err)
}
}
s := &Store{db: db}
if err := s.migrate(); err != nil {
db.Close()
return nil, fmt.Errorf("migrate: %w", err)
}
return s, nil
}
// Close closes the underlying database connection.
func (s *Store) Close() error {
return s.db.Close()
}
// DB returns the underlying *sql.DB for advanced operations like transactions.
func (s *Store) DB() *sql.DB {
return s.db
}
// migrate creates all tables if they do not already exist, then runs
// incremental migrations for schema changes added after initial release.
func (s *Store) migrate() error {
if _, err := s.db.Exec(schema); err != nil {
return err
}
return s.runMigrations()
}
// runMigrations applies additive schema changes that cannot be expressed
// with CREATE TABLE IF NOT EXISTS, plus the hard-cutover drops that
// remove every legacy project/stage/stack/static_site/deploy table.
func (s *Store) runMigrations() error {
migrations := []string{
// Set default network for existing databases with empty network.
`UPDATE settings SET network = 'tinyforge' WHERE network = ''`,
// Settings column adds that survive the cutover. SQLite is tolerant
// of "duplicate column" errors at the apply step, so re-running on
// a fully-migrated DB is a no-op.
`ALTER TABLE settings ADD COLUMN base_volume_path TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE settings ADD COLUMN ssl_certificate_id INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE settings ADD COLUMN stale_threshold_days INTEGER NOT NULL DEFAULT 7`,
`ALTER TABLE settings ADD COLUMN allowed_volume_paths TEXT NOT NULL DEFAULT '[]'`,
`ALTER TABLE settings ADD COLUMN wildcard_dns INTEGER NOT NULL DEFAULT 1`,
`ALTER TABLE settings ADD COLUMN dns_provider TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE settings ADD COLUMN cloudflare_api_token TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE settings ADD COLUMN cloudflare_zone_id TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE settings ADD COLUMN backup_enabled INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE settings ADD COLUMN backup_interval_hours INTEGER NOT NULL DEFAULT 24`,
`ALTER TABLE settings ADD COLUMN backup_retention_count INTEGER NOT NULL DEFAULT 10`,
`ALTER TABLE settings ADD COLUMN proxy_provider TEXT NOT NULL DEFAULT 'npm'`,
`ALTER TABLE settings ADD COLUMN traefik_entrypoint TEXT NOT NULL DEFAULT 'websecure'`,
`ALTER TABLE settings ADD COLUMN traefik_cert_resolver TEXT NOT NULL DEFAULT 'letsencrypt'`,
`ALTER TABLE settings ADD COLUMN traefik_network TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE settings ADD COLUMN traefik_api_url TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE settings ADD COLUMN npm_remote INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE settings ADD COLUMN npm_access_list_id INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE settings ADD COLUMN public_ip TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE settings ADD COLUMN image_prune_threshold_mb INTEGER NOT NULL DEFAULT 1024`,
`ALTER TABLE settings ADD COLUMN stats_interval_seconds INTEGER NOT NULL DEFAULT 15`,
`ALTER TABLE settings ADD COLUMN stats_retention_hours INTEGER NOT NULL DEFAULT 2`,
`ALTER TABLE settings ADD COLUMN notification_secret TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE settings ADD COLUMN auto_backup_before_deploy INTEGER NOT NULL DEFAULT 0`,
// Registries — owner column.
`ALTER TABLE registries ADD COLUMN owner TEXT NOT NULL DEFAULT ''`,
// Webhook delivery audit log persists every inbound webhook
// request so operators can debug "why didn't my deploy fire?"
// without grepping daemon logs.
`CREATE TABLE IF NOT EXISTS webhook_deliveries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
target_type TEXT NOT NULL,
target_id TEXT NOT NULL DEFAULT '',
target_name TEXT NOT NULL DEFAULT '',
received_at TEXT NOT NULL DEFAULT (datetime('now')),
source_ip TEXT NOT NULL DEFAULT '',
signature_state TEXT NOT NULL DEFAULT '',
status_code INTEGER NOT NULL DEFAULT 0,
outcome TEXT NOT NULL DEFAULT '',
detail TEXT NOT NULL DEFAULT '',
body_size INTEGER NOT NULL DEFAULT 0
)`,
`CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_target ON webhook_deliveries(target_type, target_id, received_at)`,
`CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_received_at ON webhook_deliveries(received_at)`,
// Containers — stage_id is now an opaque string set by the source
// plugin (image plugin uses it for the deploy-target tag). No FK
// semantics: the legacy `stages` table this column once joined to
// is gone; the column is just a free-form discriminator the
// proxies / dashboard views read to disambiguate sibling rows.
`ALTER TABLE containers ADD COLUMN stage_id TEXT NOT NULL DEFAULT ''`,
// Workload-first refactor columns. Land additively so old databases
// (which have a bare workloads table) pick them up on the next boot.
`ALTER TABLE workloads ADD COLUMN source_kind TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE workloads ADD COLUMN source_config TEXT NOT NULL DEFAULT '{}'`,
`ALTER TABLE workloads ADD COLUMN trigger_kind TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE workloads ADD COLUMN trigger_config TEXT NOT NULL DEFAULT '{}'`,
`ALTER TABLE workloads ADD COLUMN public_faces TEXT NOT NULL DEFAULT '[]'`,
`ALTER TABLE workloads ADD COLUMN parent_workload_id TEXT NOT NULL DEFAULT ''`,
// Hard cutover: drop every legacy table. Idempotent — DROP TABLE
// IF EXISTS is a no-op once the table is gone. Operators upgrading
// from a pre-cutover build will lose any project / stack / static
// site rows; the upgrade notes call this out explicitly.
`DROP TABLE IF EXISTS deploy_logs`,
`DROP TABLE IF EXISTS deploys`,
`DROP TABLE IF EXISTS stage_env`,
`DROP TABLE IF EXISTS stages`,
`DROP TABLE IF EXISTS poll_states`,
`DROP TABLE IF EXISTS volumes`,
`DROP TABLE IF EXISTS static_site_secrets`,
`DROP TABLE IF EXISTS static_sites`,
`DROP TABLE IF EXISTS stack_deploys`,
`DROP TABLE IF EXISTS stack_revisions`,
`DROP TABLE IF EXISTS stacks`,
`DROP TABLE IF EXISTS projects`,
}
// Workload refactor tables (2026-05-09). Workload is the unifying primitive
// over Project / Stack / StaticSite; Container is the normalized index of
// every Tinyforge-managed container; Apps is an optional grouping. These
// live alongside (not inside) the schema constant so existing databases
// pick them up on restart.
workloadTables := []string{
`CREATE TABLE IF NOT EXISTS workloads (
id TEXT PRIMARY KEY,
kind TEXT NOT NULL,
ref_id TEXT NOT NULL,
name TEXT NOT NULL,
app_id TEXT NOT NULL DEFAULT '',
source_kind TEXT NOT NULL DEFAULT '',
source_config TEXT NOT NULL DEFAULT '{}',
trigger_kind TEXT NOT NULL DEFAULT '',
trigger_config TEXT NOT NULL DEFAULT '{}',
public_faces TEXT NOT NULL DEFAULT '[]',
parent_workload_id TEXT NOT NULL DEFAULT '',
notification_url TEXT NOT NULL DEFAULT '',
notification_secret TEXT NOT NULL DEFAULT '',
webhook_secret TEXT NOT NULL DEFAULT '',
webhook_signing_secret TEXT NOT NULL DEFAULT '',
webhook_require_signature INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(kind, ref_id)
)`,
`CREATE TABLE IF NOT EXISTS containers (
id TEXT PRIMARY KEY,
workload_id TEXT NOT NULL,
workload_kind TEXT NOT NULL,
role TEXT NOT NULL DEFAULT '',
stage_id TEXT NOT NULL DEFAULT '',
container_id TEXT NOT NULL DEFAULT '',
image_ref TEXT NOT NULL DEFAULT '',
image_tag TEXT NOT NULL DEFAULT '',
host TEXT NOT NULL DEFAULT 'local',
state TEXT NOT NULL DEFAULT '',
port INTEGER NOT NULL DEFAULT 0,
subdomain TEXT NOT NULL DEFAULT '',
proxy_route_id TEXT NOT NULL DEFAULT '',
npm_proxy_id INTEGER NOT NULL DEFAULT 0,
last_seen_at TEXT NOT NULL DEFAULT '',
extra_json TEXT NOT NULL DEFAULT '{}',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)`,
`CREATE TABLE IF NOT EXISTS apps (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
description TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)`,
// workload_env: per-workload env overrides (encrypt-at-rest for
// secrets). Functional analog of stage_env. Workload deletion
// cascades through the FK so orphan rows are impossible.
`CREATE TABLE IF NOT EXISTS workload_env (
id TEXT PRIMARY KEY,
workload_id TEXT NOT NULL REFERENCES workloads(id) ON DELETE CASCADE,
key TEXT NOT NULL,
value TEXT NOT NULL DEFAULT '',
encrypted INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(workload_id, key)
)`,
// workload_volumes: per-workload mount declarations. Mirrors the
// legacy `volumes` table shape (source / target / scope / name)
// but keyed on workload_id. UNIQUE on (workload_id, target) so a
// re-add overwrites instead of duplicating.
`CREATE TABLE IF NOT EXISTS workload_volumes (
id TEXT PRIMARY KEY,
workload_id TEXT NOT NULL REFERENCES workloads(id) ON DELETE CASCADE,
source TEXT NOT NULL DEFAULT '',
target TEXT NOT NULL,
scope TEXT NOT NULL DEFAULT 'absolute',
name TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(workload_id, target)
)`,
// triggers: first-class redeploy signal sources. Webhook secrets
// move from workload onto the trigger so one webhook URL can fan
// out to multiple workloads via workload_trigger_bindings.
`CREATE TABLE IF NOT EXISTS triggers (
id TEXT PRIMARY KEY,
kind TEXT NOT NULL,
name TEXT NOT NULL UNIQUE,
config TEXT NOT NULL DEFAULT '{}',
webhook_secret TEXT NOT NULL DEFAULT '',
webhook_signing_secret TEXT NOT NULL DEFAULT '',
webhook_require_signature INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)`,
// workload_trigger_bindings: many-to-many between workloads and
// triggers. binding_config is the per-binding override applied on
// top of trigger.config (top-level JSON merge, binding wins).
`CREATE TABLE IF NOT EXISTS workload_trigger_bindings (
id TEXT PRIMARY KEY,
workload_id TEXT NOT NULL REFERENCES workloads(id) ON DELETE CASCADE,
trigger_id TEXT NOT NULL REFERENCES triggers(id) ON DELETE CASCADE,
binding_config TEXT NOT NULL DEFAULT '{}',
enabled INTEGER NOT NULL DEFAULT 1,
sort_order INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(workload_id, trigger_id)
)`,
}
for _, t := range workloadTables {
if _, err := s.db.Exec(t); err != nil {
return fmt.Errorf("create workload table: %w", err)
}
}
// Additive stack tables (2026-04-16). Created here rather than in the
// schema constant so older databases pick them up on restart.
statsTables := []string{
`CREATE TABLE IF NOT EXISTS container_stats_samples (
id INTEGER PRIMARY KEY AUTOINCREMENT,
container_id TEXT NOT NULL,
owner_type TEXT NOT NULL,
owner_id TEXT NOT NULL,
ts INTEGER NOT NULL,
cpu_percent REAL NOT NULL DEFAULT 0,
memory_usage INTEGER NOT NULL DEFAULT 0,
memory_limit INTEGER NOT NULL DEFAULT 0,
network_rx INTEGER NOT NULL DEFAULT 0,
network_tx INTEGER NOT NULL DEFAULT 0,
block_read INTEGER NOT NULL DEFAULT 0,
block_write INTEGER NOT NULL DEFAULT 0
)`,
`CREATE TABLE IF NOT EXISTS system_stats_samples (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts INTEGER NOT NULL,
ncpu INTEGER NOT NULL DEFAULT 0,
memory_total INTEGER NOT NULL DEFAULT 0,
workload_cpu_percent REAL NOT NULL DEFAULT 0,
workload_mem_usage INTEGER NOT NULL DEFAULT 0,
containers_running INTEGER NOT NULL DEFAULT 0,
disk_total_bytes INTEGER NOT NULL DEFAULT 0
)`,
}
for _, t := range statsTables {
if _, err := s.db.Exec(t); err != nil {
return fmt.Errorf("create stats table: %w", err)
}
}
// Observability: event_triggers — consume EventLog entries off the
// bus and dispatch webhook actions. Schema kept flat (comma-list
// filters, single optional regex) — see LOGSCAN_AND_TRIGGERS_TODO.md.
observabilityTables := []string{
`CREATE TABLE IF NOT EXISTS event_triggers (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
filter_severity TEXT NOT NULL DEFAULT '',
filter_source TEXT NOT NULL DEFAULT '',
filter_message_regex TEXT NOT NULL DEFAULT '',
action_type TEXT NOT NULL DEFAULT 'webhook',
action_target TEXT NOT NULL DEFAULT '',
action_secret TEXT NOT NULL DEFAULT '',
enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)`,
// log_scan_rules: regex patterns the log-scanner manager
// applies to container log lines. WorkloadID is nullable (via
// "" sentinel) so a global rule can have OverridesID = 0 and
// per-workload overrides reference the global's id.
`CREATE TABLE IF NOT EXISTS log_scan_rules (
id INTEGER PRIMARY KEY AUTOINCREMENT,
workload_id TEXT NOT NULL DEFAULT '',
overrides_id INTEGER NOT NULL DEFAULT 0,
name TEXT NOT NULL,
pattern TEXT NOT NULL,
severity TEXT NOT NULL DEFAULT 'warn',
streams TEXT NOT NULL DEFAULT 'all',
cooldown_seconds INTEGER NOT NULL DEFAULT 60,
enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)`,
`CREATE INDEX IF NOT EXISTS idx_log_scan_rules_workload ON log_scan_rules(workload_id)`,
`CREATE INDEX IF NOT EXISTS idx_log_scan_rules_overrides ON log_scan_rules(overrides_id)`,
}
for _, t := range observabilityTables {
if _, err := s.db.Exec(t); err != nil {
return fmt.Errorf("create observability table: %w", err)
}
}
for _, m := range migrations {
if _, err := s.db.Exec(m); err != nil {
// "duplicate column" / "already exists" are expected when a
// migration has already been applied. "no such table" is
// expected for obsolete ALTER TABLE migrations targeting tables
// the workload refactor dropped (e.g. instances). Anything
// else must surface — silently running against the wrong shape
// is worse than a startup failure.
msg := err.Error()
if !strings.Contains(msg, "duplicate column") &&
!strings.Contains(msg, "already exists") &&
!strings.Contains(msg, "no such table") {
return fmt.Errorf("apply migration %q: %w", m, err)
}
}
}
// Create indexes on foreign key columns for query performance. Only
// indexes targeting tables that still exist after the hard cutover.
indexes := []string{
`CREATE INDEX IF NOT EXISTS idx_event_log_severity ON event_log(severity)`,
`CREATE INDEX IF NOT EXISTS idx_event_log_source ON event_log(source)`,
`CREATE INDEX IF NOT EXISTS idx_event_log_created_at ON event_log(created_at)`,
`CREATE INDEX IF NOT EXISTS idx_dns_records_consumer ON dns_records(consumer_type, consumer_id)`,
`CREATE INDEX IF NOT EXISTS idx_container_stats_owner_ts ON container_stats_samples(owner_type, owner_id, ts)`,
`CREATE INDEX IF NOT EXISTS idx_container_stats_container_ts ON container_stats_samples(container_id, ts)`,
`CREATE INDEX IF NOT EXISTS idx_container_stats_ts ON container_stats_samples(ts)`,
`CREATE INDEX IF NOT EXISTS idx_system_stats_ts ON system_stats_samples(ts)`,
// Workload refactor indexes.
`CREATE INDEX IF NOT EXISTS idx_workloads_kind ON workloads(kind)`,
`CREATE INDEX IF NOT EXISTS idx_workloads_app_id ON workloads(app_id) WHERE app_id != ''`,
`CREATE INDEX IF NOT EXISTS idx_workloads_ref ON workloads(kind, ref_id)`,
`CREATE UNIQUE INDEX IF NOT EXISTS idx_workloads_webhook_secret ON workloads(webhook_secret) WHERE webhook_secret != ''`,
`CREATE INDEX IF NOT EXISTS idx_containers_workload ON containers(workload_id)`,
`CREATE INDEX IF NOT EXISTS idx_containers_state ON containers(state)`,
`CREATE INDEX IF NOT EXISTS idx_containers_container_id ON containers(container_id) WHERE container_id != ''`,
`CREATE INDEX IF NOT EXISTS idx_containers_kind ON containers(workload_kind)`,
`CREATE INDEX IF NOT EXISTS idx_containers_stage_id ON containers(stage_id) WHERE stage_id != ''`,
`CREATE INDEX IF NOT EXISTS idx_workload_env_workload ON workload_env(workload_id)`,
`CREATE INDEX IF NOT EXISTS idx_workload_volumes_workload ON workload_volumes(workload_id)`,
// Trigger-split indexes.
`CREATE INDEX IF NOT EXISTS idx_triggers_kind ON triggers(kind)`,
`CREATE UNIQUE INDEX IF NOT EXISTS idx_triggers_webhook_secret ON triggers(webhook_secret) WHERE webhook_secret != ''`,
`CREATE INDEX IF NOT EXISTS idx_bindings_workload ON workload_trigger_bindings(workload_id)`,
`CREATE INDEX IF NOT EXISTS idx_bindings_trigger ON workload_trigger_bindings(trigger_id)`,
}
for _, idx := range indexes {
if _, err := s.db.Exec(idx); err != nil {
return fmt.Errorf("create index: %w", err)
}
}
if err := s.backfillTriggersFromWorkloads(); err != nil {
slog.Warn("trigger backfill", "error", err)
}
return nil
}
// backfillTriggersFromWorkloads converts embedded trigger config on
// workload rows into standalone trigger + binding rows. Runs once per
// boot and is idempotent — only workloads with non-empty trigger_kind
// AND no existing binding produce a new trigger record.
//
// Each per-workload backfill runs inside a transaction so a partial
// failure (binding insert fails after trigger insert succeeds) rolls
// back cleanly; otherwise an orphan trigger row would survive forever
// because the next boot's bindings-count check sees zero bindings and
// tries to re-insert under the same UNIQUE name.
//
// Trigger names are unconditionally suffixed with the workload's id
// short-prefix to make collisions impossible across two workloads with
// identical (name, kind) — the "Foo [registry]" + "Foo [registry]" case
// would otherwise have one of them silently dropped.
//
// Why on every boot: the trigger-split refactor is a clean break (no
// formal migration). Existing dev databases have triggers embedded in
// workloads.trigger_kind / trigger_config; this lifts them into the new
// shape so URLs and behavior survive the upgrade.
func (s *Store) backfillTriggersFromWorkloads() error {
rows, err := s.db.Query(
`SELECT id, name, trigger_kind, trigger_config,
webhook_secret, webhook_signing_secret, webhook_require_signature
FROM workloads
WHERE trigger_kind != ''`,
)
if err != nil {
return fmt.Errorf("scan workloads for backfill: %w", err)
}
defer rows.Close()
type embedded struct {
id, name, kind, config string
webhookSecret, webhookSigningSecret string
requireSig int
}
var pending []embedded
for rows.Next() {
var e embedded
if err := rows.Scan(&e.id, &e.name, &e.kind, &e.config,
&e.webhookSecret, &e.webhookSigningSecret, &e.requireSig); err != nil {
return fmt.Errorf("scan workload row: %w", err)
}
pending = append(pending, e)
}
if err := rows.Err(); err != nil {
return err
}
for _, e := range pending {
if err := s.backfillOneTrigger(e.id, e.name, e.kind, e.config,
e.webhookSecret, e.webhookSigningSecret, e.requireSig); err != nil {
slog.Warn("trigger backfill: workload skipped",
"workload", e.id, "error", err)
}
}
return nil
}
// backfillOneTrigger lifts one embedded trigger into its own row + binding
// inside a single transaction. Idempotent: a workload that already has at
// least one binding is left alone.
func (s *Store) backfillOneTrigger(workloadID, workloadName, kind, config,
webhookSecret, webhookSigningSecret string, requireSig int) error {
tx, err := s.db.Begin()
if err != nil {
return fmt.Errorf("begin: %w", err)
}
defer func() { _ = tx.Rollback() }()
var existing int
if err := tx.QueryRow(
`SELECT COUNT(*) FROM workload_trigger_bindings WHERE workload_id = ?`,
workloadID,
).Scan(&existing); err != nil {
return fmt.Errorf("count bindings: %w", err)
}
if existing > 0 {
return nil
}
idShort := workloadID
if len(idShort) > 8 {
idShort = idShort[:8]
}
triggerName := fmt.Sprintf("%s [%s] %s", workloadName, kind, idShort)
triggerID := uuid.New().String()
now := Now()
if _, err := tx.Exec(
`INSERT INTO triggers (id, kind, name, config,
webhook_secret, webhook_signing_secret, webhook_require_signature,
created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
triggerID, kind, triggerName, config,
webhookSecret, webhookSigningSecret, requireSig,
now, now,
); err != nil {
return fmt.Errorf("insert trigger: %w", err)
}
bindingID := uuid.New().String()
if _, err := tx.Exec(
`INSERT INTO workload_trigger_bindings
(id, workload_id, trigger_id, binding_config, enabled, sort_order, created_at, updated_at)
VALUES (?, ?, ?, '{}', 1, 0, ?, ?)`,
bindingID, workloadID, triggerID, now, now,
); err != nil {
return fmt.Errorf("insert binding: %w", err)
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("commit: %w", err)
}
return nil
}
const schema = `
CREATE TABLE IF NOT EXISTS registries (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
url TEXT NOT NULL,
type TEXT NOT NULL DEFAULT 'generic',
token TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS settings (
id INTEGER PRIMARY KEY CHECK (id = 1),
domain TEXT NOT NULL DEFAULT '',
server_ip TEXT NOT NULL DEFAULT '',
public_ip TEXT NOT NULL DEFAULT '',
network TEXT NOT NULL DEFAULT 'tinyforge',
subdomain_pattern TEXT NOT NULL DEFAULT 'stage-{stage}-{project}',
notification_url TEXT NOT NULL DEFAULT '',
notification_secret TEXT NOT NULL DEFAULT '',
npm_url TEXT NOT NULL DEFAULT '',
npm_email TEXT NOT NULL DEFAULT '',
npm_password TEXT NOT NULL DEFAULT '',
webhook_secret TEXT NOT NULL DEFAULT '',
polling_interval TEXT NOT NULL DEFAULT '5m',
base_volume_path TEXT NOT NULL DEFAULT '',
ssl_certificate_id INTEGER NOT NULL DEFAULT 0,
npm_remote INTEGER NOT NULL DEFAULT 0,
image_prune_threshold_mb INTEGER NOT NULL DEFAULT 1024,
npm_access_list_id INTEGER NOT NULL DEFAULT 0,
traefik_entrypoint TEXT NOT NULL DEFAULT 'websecure',
traefik_cert_resolver TEXT NOT NULL DEFAULT 'letsencrypt',
traefik_network TEXT NOT NULL DEFAULT '',
traefik_api_url TEXT NOT NULL DEFAULT '',
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
username TEXT NOT NULL UNIQUE,
password_hash TEXT NOT NULL DEFAULT '',
email TEXT NOT NULL DEFAULT '',
role TEXT NOT NULL DEFAULT 'viewer',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS auth_settings (
id INTEGER PRIMARY KEY CHECK (id = 1),
auth_mode TEXT NOT NULL DEFAULT 'local',
oidc_client_id TEXT NOT NULL DEFAULT '',
oidc_client_secret TEXT NOT NULL DEFAULT '',
oidc_issuer_url TEXT NOT NULL DEFAULT '',
oidc_redirect_url TEXT NOT NULL DEFAULT ''
);
-- Seed the settings row if it does not exist.
INSERT OR IGNORE INTO settings (id) VALUES (1);
-- Seed the auth_settings row if it does not exist.
INSERT OR IGNORE INTO auth_settings (id) VALUES (1);
CREATE TABLE IF NOT EXISTS event_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source TEXT NOT NULL DEFAULT '',
severity TEXT NOT NULL DEFAULT 'info',
message TEXT NOT NULL DEFAULT '',
metadata TEXT NOT NULL DEFAULT '{}',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS standalone_proxies (
id TEXT PRIMARY KEY,
domain TEXT NOT NULL UNIQUE,
destination_url TEXT NOT NULL DEFAULT '',
destination_port INTEGER NOT NULL DEFAULT 0,
ssl_certificate_id INTEGER NOT NULL DEFAULT 0,
npm_proxy_id INTEGER NOT NULL DEFAULT 0,
health_status TEXT NOT NULL DEFAULT 'unknown',
health_checked_at TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS dns_records (
id TEXT PRIMARY KEY,
fqdn TEXT NOT NULL UNIQUE,
provider_record_id TEXT NOT NULL DEFAULT '',
consumer_type TEXT NOT NULL DEFAULT '',
consumer_id TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS backups (
id TEXT PRIMARY KEY,
filename TEXT NOT NULL UNIQUE,
size_bytes INTEGER NOT NULL DEFAULT 0,
backup_type TEXT NOT NULL DEFAULT 'manual',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
`
// Now returns the current time formatted for SQLite storage.
func Now() string {
return time.Now().UTC().Format("2006-01-02 15:04:05")
}