chore(workload): close the workload-first arc — apps i18n + codemap + tests
Build / build (push) Successful in 10m36s
Build / build (push) Successful in 10m36s
Closes the workload-first refactor by landing the Priority 3 polish items and the Priority 4 test gap. Net: ~2,400 lines added, ~350 lines modified across 13 files. Priority 3 — polish - apps.* i18n namespace: 276 new keys across apps.list.* (27), apps.new.* (91, sibling of existing apps.new.triggers.*), and apps.detail.* (158, sibling of existing apps.detail.bindings.*). EN+RU at 1314 keys each, perfectly in sync. /apps, /apps/new, /apps/[id] now render entirely from i18n. - New codemap docs/CODEMAPS/workload-plugin.md (238 lines): Source × Trigger contract, dispatch seam, webhook fan-out path, recipes for adding a new Source or Trigger kind. Plus docs/CODEMAPS/INDEX.md gateway. Priority 4 — tests - internal/api/workloads_test.go (new, ~30 subtests): /api/workloads CRUD + deploy + delete + env + volumes + chain + promote-from + triggers list/inline-bind + auth gating + standalone /api/triggers CRUD (create / dup-409 / kind filter / delete). Uses real POST handlers via httptest.NewServer + a fake plugin source registered under "testfakesource". - internal/deployer/dispatch_test.go (new, 11 tests): DispatchPlugin / DispatchTeardown / DispatchReconcile happy + unknown-kind + propagated-error each; PluginDeps wiring; a real 2s-bounded RWMutex deadlock probe on PluginDeps vs SetDNSProvider. - internal/workload/plugin/source/compose/compose_test.go (new, ~26 subtests): composeProjectName sanitization, writeYAML / writeYAMLIfChanged hash short-circuit, Validate happy + bad inputs, Kind / SchemaSample. Coverage delta on the workload-plugin path: - internal/api: 1.1% → 16.0% - internal/deployer: 0% → 54.1% - internal/workload/plugin/source/compose: 0% → 38.5% - Trigger plugins already at 87-95% from the trigger-split work. Production fix surfaced by the tests - store.CreateWorkload now self-references RefID = ID when caller leaves RefID empty (the typical plugin-native path). The api layer's broken backfill loop (called UpdateWorkload, which deliberately omits ref_id) is gone. Multiple sibling plugin workloads can now coexist under the UNIQUE(kind, ref_id) constraint. Review fixes addressed before commit - CRITICAL: deadlock-detect test gained a real 2s time.After (was selecting on context.Background().Done() which never fires). - HIGH: happy-path test now hard-asserts RefID = ID (was a t.Logf that would silently pass after a production fix). - HIGH: standalone /api/triggers CRUD coverage added (was bypassed by the workload-side bind flow). - HIGH: seedWorkload bypass deleted; tests now go through the real POST /api/workloads handler. - MEDIUM: withTempDir restore is a no-op (t.Setenv auto-restores); dead `old := os.Getenv(...)` capture removed. - MEDIUM: list-workloads test now asserts ID membership, not just count. Doc - WORKLOAD_REFACTOR_TODO: all three Priority 1 items, Priority 3 polish, and Priority 4 tests marked DONE. The workload-first arc is closed.
This commit is contained in:
@@ -0,0 +1,238 @@
|
||||
# Workload Plugin Architecture Codemap
|
||||
|
||||
**Last Updated:** 2026-05-16
|
||||
**Status:** Core contract for Source × Trigger plugin system (post-trigger-split refactor)
|
||||
|
||||
## Abstract
|
||||
|
||||
`internal/workload/plugin/` defines the **Source × Trigger plugin contracts** that decouple the deployer pipeline from specific deployable shapes (image, compose, static) and redeploy signals (registry push, git push, manual, cron). A Workload carries opaque config blobs; registry lookups route each to the matching plugin. New plugin kinds are added only via registration from init() — no changes needed to the API, deployer, or webhook handler.
|
||||
|
||||
## Key Files
|
||||
|
||||
| Path | Role |
|
||||
|------|------|
|
||||
| `internal/workload/plugin/plugin.go` | Package doc; `Deps` bundle (Store, Docker, Proxy, DNS, Health, Notifier, Events, EncKey) |
|
||||
| `internal/workload/plugin/types.go` | `Workload`, `DeploymentIntent`, `PublicFace`, `InboundEvent`, `ImagePushEvent`, `GitEvent`, `ManualEvent`; helpers `SourceConfigOf[T]`, `TriggerConfigOf[T]` |
|
||||
| `internal/workload/plugin/source.go` | `Source` interface (Kind / Validate / Deploy / Teardown / Reconcile); registries `RegisterSource` / `GetSource`; `Schemaer` optional interface; helpers `SourceKinds` / `SchemaSampleFor` |
|
||||
| `internal/workload/plugin/trigger.go` | `Trigger` interface (Kind / Validate / Match); registries `RegisterTrigger` / `GetTrigger`; helper `TriggerKinds` |
|
||||
| `internal/workload/plugin/binding.go` | `MergeJSONConfig` (top-level JSON merge for trigger + binding override); `WithEffectiveTrigger` (used by webhook fan-out to compose merged config) |
|
||||
| `internal/workload/plugin/registry.go` | `AllSources` / `AllTriggers` snapshot helpers (used by `/api/workloads/source-kinds` and `/api/workloads/trigger-kinds`) |
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Contract Surface: Source vs Trigger
|
||||
|
||||
```
|
||||
Workload (unifying user entity)
|
||||
├── SourceKind + SourceConfig (JSON blob)
|
||||
│ └── Source.Deploy() ← routes to image, compose, or static
|
||||
│ Source.Teardown()
|
||||
│ Source.Reconcile()
|
||||
│
|
||||
└── TriggerKind + TriggerConfig (JSON blob)
|
||||
└── Trigger.Match(InboundEvent) ← routes to registry, git, or manual
|
||||
returns DeploymentIntent
|
||||
```
|
||||
|
||||
- **Source** (stateless, 5 methods): owns full container lifecycle (deploy, tear down, reconcile state)
|
||||
- **Trigger** (stateless, 3 methods): given an inbound event + workload config, decide whether to fire a deploy intent
|
||||
|
||||
### Dispatch Seam: Deployer → Plugins
|
||||
|
||||
```
|
||||
deployer/dispatch.go
|
||||
├── DispatchPlugin(w, intent) → plugin.GetSource(w.SourceKind) → Source.Deploy()
|
||||
├── DispatchTeardown(w) → plugin.GetSource(w.SourceKind) → Source.Teardown()
|
||||
└── DispatchReconcile(w) → plugin.GetSource(w.SourceKind) → Source.Reconcile()
|
||||
|
||||
PluginDeps() assembles:
|
||||
├── Store (workload / container / webhook tables)
|
||||
├── Docker (container orchestration)
|
||||
├── Proxy (route manager)
|
||||
├── DNS (DNS provider, nil for wildcard)
|
||||
├── Health (status checker)
|
||||
├── Notifier (webhook client)
|
||||
├── Events (event bus for deploy lifecycle)
|
||||
└── EncKey (for crypto.Encrypt/Decrypt of config secrets)
|
||||
```
|
||||
|
||||
### Webhook Fan-Out Path: Trigger → Bindings
|
||||
|
||||
```
|
||||
webhook/trigger_handler.go: POST /api/webhook/triggers/{secret}
|
||||
├── Resolve secret → Trigger record
|
||||
├── Parse body → InboundEvent (auto-detects image-push, git-push, git-tag, manual, cron-tick)
|
||||
├── plugin.GetTrigger(trg.Kind) → Trigger plugin
|
||||
└── For each enabled workload_trigger_binding (bounded concurrency = 4):
|
||||
├── plugin.WithEffectiveTrigger()
|
||||
│ └── MergeJSONConfig(trigger.config, binding.binding_config)
|
||||
│ returns Workload copy with merged TriggerConfig
|
||||
├── Trigger.Match(evt, merged_workload)
|
||||
│ returns DeploymentIntent or nil
|
||||
└── If intent returned: DispatchPlugin(w, intent) → Source.Deploy()
|
||||
```
|
||||
|
||||
**Key design point**: MergeJSONConfig always returns freshly allocated slices (defensive copy) so binding fan-out never risks aliasing across goroutines.
|
||||
|
||||
## Concrete Implementations
|
||||
|
||||
### Sources
|
||||
|
||||
| Kind | Package | Files | Purpose |
|
||||
|------|---------|-------|---------|
|
||||
| `image` | `internal/workload/plugin/source/image/` | `image.go` + deps | Docker image deploys (blue-green multi-face proxy, registry auth) |
|
||||
| `compose` | `internal/workload/plugin/source/compose/` | `compose.go` + deps | docker-compose stacks via `internal/stack/` helpers |
|
||||
| `static` | `internal/workload/plugin/source/static/` | `deploy.go`, `teardown.go`, `reconcile.go`, `state.go`, `env.go`, `build.go`, `naming.go`, `static.go` | Git-folder-backed static site (nginx or Deno) via `internal/staticsite/` helpers |
|
||||
|
||||
**Static source inline port note:** The legacy `/api/sites/*` HTTP surface still exists for backwards compat; the static plugin operates directly on containers + workload_env tables without synthetic static_sites rows.
|
||||
|
||||
### Triggers
|
||||
|
||||
| Kind | Package | Files | Purpose |
|
||||
|------|---------|-------|---------|
|
||||
| `registry` | `internal/workload/plugin/trigger/registry/` | `registry.go` | Image push events (registry webhook or watcher) |
|
||||
| `git` | `internal/workload/plugin/trigger/git/` | `git.go` | Git push / tag-create (Gitea / GitHub / GitLab) |
|
||||
| `manual` | `internal/workload/plugin/trigger/manual/` | `manual.go` | Manual-only (no auto-fire) |
|
||||
|
||||
**Trigger lifecycle note:** As first-class records since the trigger-split refactor, triggers are bound to workloads via `workload_trigger_bindings` join table. Each binding carries optional `binding_config` (merged with trigger's `config` before Match is called).
|
||||
|
||||
## Data Flow: Example Webhook Dispatch
|
||||
|
||||
```
|
||||
1. Inbound POST /api/webhook/triggers/{secret}
|
||||
↓
|
||||
2. Lookup Trigger by secret
|
||||
↓
|
||||
3. Parse body → InboundEvent (detects kind: image-push, git-push, manual, ...)
|
||||
↓
|
||||
4. Load Trigger plugin (e.g. plugin.GetTrigger("git"))
|
||||
↓
|
||||
5. Load all bindings for this trigger
|
||||
↓
|
||||
6. For each binding (concurrent, max 4):
|
||||
a. Merge trigger.config + binding.binding_config
|
||||
b. Build Workload copy with merged TriggerConfig
|
||||
c. Call Trigger.Match(evt, merged_workload)
|
||||
d. If Match returns DeploymentIntent:
|
||||
- Call DispatchPlugin(w, intent)
|
||||
- Source.Deploy executes (e.g. pull image, build container)
|
||||
e. If Match returns (nil, nil): skip silently
|
||||
f. If Match returns error: log at warn level, continue to next binding
|
||||
↓
|
||||
7. Aggregate results (deployed count, skip reason counts)
|
||||
↓
|
||||
8. Return 200 with result summary
|
||||
```
|
||||
|
||||
## Integration Points
|
||||
|
||||
### API Layer (`internal/api/workloads.go`)
|
||||
|
||||
- `/api/workloads/{id}` — GET returns Workload with SourceKind + SourceConfig
|
||||
- `/api/workloads/{id}` — PUT/POST routes to Validate (source plugin checks config schema)
|
||||
- `/api/workloads/source-kinds` — GET calls `plugin.SourceKinds()` + `plugin.SchemaSampleFor()` per kind
|
||||
- `/api/workloads/trigger-kinds` — GET calls `plugin.TriggerKinds()` + `plugin.SchemaSampleFor()` per kind
|
||||
- `/api/workloads/{id}/deploy` — POST manual deploy: builds ManualEvent, calls webhook handler
|
||||
|
||||
### Webhook Ingress (`internal/webhook/`)
|
||||
|
||||
- `trigger_handler.go` — POST `/api/webhook/triggers/{secret}` implements the fan-out dispatcher (see Data Flow above)
|
||||
- `parse.go` — `buildInboundEvent()` normalizes vendor-specific payloads (Gitea / GitHub / GitLab / Docker Hub / generic registry) into `InboundEvent`
|
||||
|
||||
### Reconciler (`internal/reconciler/reconciler.go`)
|
||||
|
||||
- `reconcilePluginWorkloads()` — iterates every workload with `SourceKind != ""`, calls `DispatchReconcile(w)` on fixed schedule (e.g. every 5 minutes)
|
||||
- Keeps containers index in sync with deployed reality (garbage-collect orphaned containers, restart crashed services)
|
||||
|
||||
### Deployer (`internal/deployer/dispatch.go`)
|
||||
|
||||
- `DispatchPlugin()` / `DispatchTeardown()` / `DispatchReconcile()` — route calls to the matching Source plugin
|
||||
- `PluginDeps()` — assembles the stateless dependency bundle (called per-Deploy, per-Trigger.Match)
|
||||
|
||||
## External Dependencies
|
||||
|
||||
| Package | Version | Used For |
|
||||
|---------|---------|----------|
|
||||
| `encoding/json` | stdlib | Config marshaling / unmarshaling |
|
||||
| `sync` (RWMutex) | stdlib | Registry thread-safety (SourceKinds, TriggerKinds, Schemaer lookup) |
|
||||
| `context` | stdlib | Timeout control in Deploy / Teardown / Reconcile / Match |
|
||||
| `internal/store` | local | Workload / container / binding / trigger table access |
|
||||
| `internal/docker` | local | Container orchestration (Sources use this) |
|
||||
| `internal/proxy` | local | Route registration (Sources use this) |
|
||||
| `internal/dns` | local | DNS record creation (Sources use this, nil for wildcard DNS) |
|
||||
| `internal/health` | local | Status checks (available to plugin Deps) |
|
||||
| `internal/notify` | local | Webhook client (available to plugin Deps) |
|
||||
| `internal/events` | local | Event bus (Sources publish lifecycle events) |
|
||||
|
||||
## How to Add a New Source Kind
|
||||
|
||||
1. Create `internal/workload/plugin/source/{kind}/{kind}.go` with a struct implementing `Source`:
|
||||
|
||||
```go
|
||||
type source struct{}
|
||||
|
||||
func init() { plugin.RegisterSource(&source{}) }
|
||||
|
||||
func (s *source) Kind() string { return "k8s" }
|
||||
|
||||
func (s *source) Validate(cfg json.RawMessage) error { /* ... */ }
|
||||
|
||||
func (s *source) Deploy(ctx context.Context, deps plugin.Deps, w plugin.Workload, intent plugin.DeploymentIntent) error { /* ... */ }
|
||||
|
||||
func (s *source) Teardown(ctx context.Context, deps plugin.Deps, w plugin.Workload) error { /* ... */ }
|
||||
|
||||
func (s *source) Reconcile(ctx context.Context, deps plugin.Deps, w plugin.Workload) error { /* ... */ }
|
||||
|
||||
// Optional: implement Schemaer for JSON schema on /api/workloads/source-kinds
|
||||
func (s *source) SchemaSample() any { return Config{ /* sample fields */ } }
|
||||
```
|
||||
|
||||
2. Blank-import the sub-package from `cmd/server/main.go` so `init()` fires at boot:
|
||||
|
||||
```go
|
||||
import (
|
||||
_ "github.com/alexei/tinyforge/internal/workload/plugin/source/k8s"
|
||||
)
|
||||
```
|
||||
|
||||
3. Optionally ship a hand-rolled form in `web/src/routes/apps/{new,[id]}/+page.svelte` (per workload-first UX rule). The JSON editor remains a fallback for power users.
|
||||
|
||||
## How to Add a New Trigger Kind
|
||||
|
||||
1. Create `internal/workload/plugin/trigger/{kind}/{kind}.go` with a struct implementing `Trigger`:
|
||||
|
||||
```go
|
||||
type trigger struct{}
|
||||
|
||||
func init() { plugin.RegisterTrigger(&trigger{}) }
|
||||
|
||||
func (t *trigger) Kind() string { return "cron" }
|
||||
|
||||
func (t *trigger) Validate(cfg json.RawMessage) error { /* ... */ }
|
||||
|
||||
func (t *trigger) Match(ctx context.Context, deps plugin.Deps, w plugin.Workload, evt plugin.InboundEvent) (*plugin.DeploymentIntent, error) {
|
||||
// Decide whether this trigger fires for the given event + workload config
|
||||
// Return (nil, nil) to skip silently, (*intent, nil) to deploy, (nil, err) for config errors
|
||||
}
|
||||
|
||||
// Optional: implement Schemaer for JSON schema on /api/workloads/trigger-kinds
|
||||
func (t *trigger) SchemaSample() any { return Config{ /* sample fields */ } }
|
||||
```
|
||||
|
||||
2. Blank-import from `cmd/server/main.go`.
|
||||
|
||||
3. Ship a form variant in `web/src/lib/components/TriggerKindForm.svelte` so the `/triggers/new` page and workload bindings panel can author kind-specific config.
|
||||
|
||||
4. **Important**: Triggers that need to handle inbound webhooks should register a route in `internal/webhook/` for their vendor-specific payload format. The webhook ingress will auto-detect the kind and call `buildInboundEvent()` to normalize it into a standard `InboundEvent` before calling Match. Manual triggers do not need a webhook handler (they fire from the UI only).
|
||||
|
||||
## Related Areas
|
||||
|
||||
- **Workload Refactor** — Full context on the trigger-split, hard legacy cutover, and UI migration: [`docs/WORKLOAD_REFACTOR_TODO.md`](../WORKLOAD_REFACTOR_TODO.md)
|
||||
- **Webhook Signing** — HMAC-SHA256 verification, per-tier secret resolution, receiver code samples: [`docs/webhooks.md`](../webhooks.md)
|
||||
- **Log Scanning + Event Triggers** — Observability features that build on the trigger infrastructure: [`docs/LOGSCAN_AND_TRIGGERS_TODO.md`](../LOGSCAN_AND_TRIGGERS_TODO.md)
|
||||
|
||||
## Registry Details
|
||||
|
||||
Both Source and Trigger registries use sync.RWMutex for thread-safe lookup. Duplicate registration panics at init() (indicating a bug, not a runtime failure). Lookup errors surface missing kinds clearly so operators can diagnose when a workload references a kind whose package was not blank-imported.
|
||||
|
||||
**Lazy note**: Registries are not lazy — all kinds must be registered at boot before the HTTP server starts. This ensures consistent behavior across handlers and reconciler tasks.
|
||||
Reference in New Issue
Block a user