# Workload Plugin Architecture Codemap **Last Updated:** 2026-05-16 **Status:** Core contract for Source × Trigger plugin system (post-trigger-split refactor) ## Abstract `internal/workload/plugin/` defines the **Source × Trigger plugin contracts** that decouple the deployer pipeline from specific deployable shapes (image, compose, static) and redeploy signals (registry push, git push, manual, cron). A Workload carries opaque config blobs; registry lookups route each to the matching plugin. New plugin kinds are added only via registration from init() — no changes needed to the API, deployer, or webhook handler. ## Key Files | Path | Role | |------|------| | `internal/workload/plugin/plugin.go` | Package doc; `Deps` bundle (Store, Docker, Proxy, DNS, Health, Notifier, Events, EncKey) | | `internal/workload/plugin/types.go` | `Workload`, `DeploymentIntent`, `PublicFace`, `InboundEvent`, `ImagePushEvent`, `GitEvent`, `ManualEvent`; helpers `SourceConfigOf[T]`, `TriggerConfigOf[T]` | | `internal/workload/plugin/source.go` | `Source` interface (Kind / Validate / Deploy / Teardown / Reconcile); registries `RegisterSource` / `GetSource`; `Schemaer` optional interface; helpers `SourceKinds` / `SchemaSampleFor` | | `internal/workload/plugin/trigger.go` | `Trigger` interface (Kind / Validate / Match); registries `RegisterTrigger` / `GetTrigger`; helper `TriggerKinds` | | `internal/workload/plugin/binding.go` | `MergeJSONConfig` (top-level JSON merge for trigger + binding override); `WithEffectiveTrigger` (used by webhook fan-out to compose merged config) | | `internal/workload/plugin/registry.go` | `AllSources` / `AllTriggers` snapshot helpers (used by `/api/workloads/source-kinds` and `/api/workloads/trigger-kinds`) | ## Architecture Overview ### Contract Surface: Source vs Trigger ``` Workload (unifying user entity) ├── SourceKind + SourceConfig (JSON blob) │ └── Source.Deploy() ← routes to image, compose, or static │ Source.Teardown() │ Source.Reconcile() │ └── TriggerKind + TriggerConfig (JSON blob) └── Trigger.Match(InboundEvent) ← routes to registry, git, or manual returns DeploymentIntent ``` - **Source** (stateless, 5 methods): owns full container lifecycle (deploy, tear down, reconcile state) - **Trigger** (stateless, 3 methods): given an inbound event + workload config, decide whether to fire a deploy intent ### Dispatch Seam: Deployer → Plugins ``` deployer/dispatch.go ├── DispatchPlugin(w, intent) → plugin.GetSource(w.SourceKind) → Source.Deploy() ├── DispatchTeardown(w) → plugin.GetSource(w.SourceKind) → Source.Teardown() └── DispatchReconcile(w) → plugin.GetSource(w.SourceKind) → Source.Reconcile() PluginDeps() assembles: ├── Store (workload / container / webhook tables) ├── Docker (container orchestration) ├── Proxy (route manager) ├── DNS (DNS provider, nil for wildcard) ├── Health (status checker) ├── Notifier (webhook client) ├── Events (event bus for deploy lifecycle) └── EncKey (for crypto.Encrypt/Decrypt of config secrets) ``` ### Webhook Fan-Out Path: Trigger → Bindings ``` webhook/trigger_handler.go: POST /api/webhook/triggers/{secret} ├── Resolve secret → Trigger record ├── Parse body → InboundEvent (auto-detects image-push, git-push, git-tag, manual, cron-tick) ├── plugin.GetTrigger(trg.Kind) → Trigger plugin └── For each enabled workload_trigger_binding (bounded concurrency = 4): ├── plugin.WithEffectiveTrigger() │ └── MergeJSONConfig(trigger.config, binding.binding_config) │ returns Workload copy with merged TriggerConfig ├── Trigger.Match(evt, merged_workload) │ returns DeploymentIntent or nil └── If intent returned: DispatchPlugin(w, intent) → Source.Deploy() ``` **Key design point**: MergeJSONConfig always returns freshly allocated slices (defensive copy) so binding fan-out never risks aliasing across goroutines. ## Concrete Implementations ### Sources | Kind | Package | Files | Purpose | |------|---------|-------|---------| | `image` | `internal/workload/plugin/source/image/` | `image.go` + deps | Docker image deploys (blue-green multi-face proxy, registry auth) | | `compose` | `internal/workload/plugin/source/compose/` | `compose.go` + deps | docker-compose stacks via `internal/stack/` helpers | | `static` | `internal/workload/plugin/source/static/` | `deploy.go`, `teardown.go`, `reconcile.go`, `state.go`, `env.go`, `build.go`, `naming.go`, `static.go` | Git-folder-backed static site (nginx or Deno) via `internal/staticsite/` helpers | **Static source inline port note:** The legacy `/api/sites/*` HTTP surface still exists for backwards compat; the static plugin operates directly on containers + workload_env tables without synthetic static_sites rows. ### Triggers | Kind | Package | Files | Purpose | |------|---------|-------|---------| | `registry` | `internal/workload/plugin/trigger/registry/` | `registry.go` | Image push events (registry webhook or watcher) | | `git` | `internal/workload/plugin/trigger/git/` | `git.go` | Git push / tag-create (Gitea / GitHub / GitLab) | | `manual` | `internal/workload/plugin/trigger/manual/` | `manual.go` | Manual-only (no auto-fire) | **Trigger lifecycle note:** As first-class records since the trigger-split refactor, triggers are bound to workloads via `workload_trigger_bindings` join table. Each binding carries optional `binding_config` (merged with trigger's `config` before Match is called). ## Data Flow: Example Webhook Dispatch ``` 1. Inbound POST /api/webhook/triggers/{secret} ↓ 2. Lookup Trigger by secret ↓ 3. Parse body → InboundEvent (detects kind: image-push, git-push, manual, ...) ↓ 4. Load Trigger plugin (e.g. plugin.GetTrigger("git")) ↓ 5. Load all bindings for this trigger ↓ 6. For each binding (concurrent, max 4): a. Merge trigger.config + binding.binding_config b. Build Workload copy with merged TriggerConfig c. Call Trigger.Match(evt, merged_workload) d. If Match returns DeploymentIntent: - Call DispatchPlugin(w, intent) - Source.Deploy executes (e.g. pull image, build container) e. If Match returns (nil, nil): skip silently f. If Match returns error: log at warn level, continue to next binding ↓ 7. Aggregate results (deployed count, skip reason counts) ↓ 8. Return 200 with result summary ``` ## Integration Points ### API Layer (`internal/api/workloads.go`) - `/api/workloads/{id}` — GET returns Workload with SourceKind + SourceConfig - `/api/workloads/{id}` — PUT/POST routes to Validate (source plugin checks config schema) - `/api/workloads/source-kinds` — GET calls `plugin.SourceKinds()` + `plugin.SchemaSampleFor()` per kind - `/api/workloads/trigger-kinds` — GET calls `plugin.TriggerKinds()` + `plugin.SchemaSampleFor()` per kind - `/api/workloads/{id}/deploy` — POST manual deploy: builds ManualEvent, calls webhook handler ### Webhook Ingress (`internal/webhook/`) - `trigger_handler.go` — POST `/api/webhook/triggers/{secret}` implements the fan-out dispatcher (see Data Flow above) - `parse.go` — `buildInboundEvent()` normalizes vendor-specific payloads (Gitea / GitHub / GitLab / Docker Hub / generic registry) into `InboundEvent` ### Reconciler (`internal/reconciler/reconciler.go`) - `reconcilePluginWorkloads()` — iterates every workload with `SourceKind != ""`, calls `DispatchReconcile(w)` on fixed schedule (e.g. every 5 minutes) - Keeps containers index in sync with deployed reality (garbage-collect orphaned containers, restart crashed services) ### Deployer (`internal/deployer/dispatch.go`) - `DispatchPlugin()` / `DispatchTeardown()` / `DispatchReconcile()` — route calls to the matching Source plugin - `PluginDeps()` — assembles the stateless dependency bundle (called per-Deploy, per-Trigger.Match) ## External Dependencies | Package | Version | Used For | |---------|---------|----------| | `encoding/json` | stdlib | Config marshaling / unmarshaling | | `sync` (RWMutex) | stdlib | Registry thread-safety (SourceKinds, TriggerKinds, Schemaer lookup) | | `context` | stdlib | Timeout control in Deploy / Teardown / Reconcile / Match | | `internal/store` | local | Workload / container / binding / trigger table access | | `internal/docker` | local | Container orchestration (Sources use this) | | `internal/proxy` | local | Route registration (Sources use this) | | `internal/dns` | local | DNS record creation (Sources use this, nil for wildcard DNS) | | `internal/health` | local | Status checks (available to plugin Deps) | | `internal/notify` | local | Webhook client (available to plugin Deps) | | `internal/events` | local | Event bus (Sources publish lifecycle events) | ## How to Add a New Source Kind 1. Create `internal/workload/plugin/source/{kind}/{kind}.go` with a struct implementing `Source`: ```go type source struct{} func init() { plugin.RegisterSource(&source{}) } func (s *source) Kind() string { return "k8s" } func (s *source) Validate(cfg json.RawMessage) error { /* ... */ } func (s *source) Deploy(ctx context.Context, deps plugin.Deps, w plugin.Workload, intent plugin.DeploymentIntent) error { /* ... */ } func (s *source) Teardown(ctx context.Context, deps plugin.Deps, w plugin.Workload) error { /* ... */ } func (s *source) Reconcile(ctx context.Context, deps plugin.Deps, w plugin.Workload) error { /* ... */ } // Optional: implement Schemaer for JSON schema on /api/workloads/source-kinds func (s *source) SchemaSample() any { return Config{ /* sample fields */ } } ``` 2. Blank-import the sub-package from `cmd/server/main.go` so `init()` fires at boot: ```go import ( _ "github.com/alexei/tinyforge/internal/workload/plugin/source/k8s" ) ``` 3. Optionally ship a hand-rolled form in `web/src/routes/apps/{new,[id]}/+page.svelte` (per workload-first UX rule). The JSON editor remains a fallback for power users. ## How to Add a New Trigger Kind 1. Create `internal/workload/plugin/trigger/{kind}/{kind}.go` with a struct implementing `Trigger`: ```go type trigger struct{} func init() { plugin.RegisterTrigger(&trigger{}) } func (t *trigger) Kind() string { return "cron" } func (t *trigger) Validate(cfg json.RawMessage) error { /* ... */ } func (t *trigger) Match(ctx context.Context, deps plugin.Deps, w plugin.Workload, evt plugin.InboundEvent) (*plugin.DeploymentIntent, error) { // Decide whether this trigger fires for the given event + workload config // Return (nil, nil) to skip silently, (*intent, nil) to deploy, (nil, err) for config errors } // Optional: implement Schemaer for JSON schema on /api/workloads/trigger-kinds func (t *trigger) SchemaSample() any { return Config{ /* sample fields */ } } ``` 2. Blank-import from `cmd/server/main.go`. 3. Ship a form variant in `web/src/lib/components/TriggerKindForm.svelte` so the `/triggers/new` page and workload bindings panel can author kind-specific config. 4. **Important**: Triggers that need to handle inbound webhooks should register a route in `internal/webhook/` for their vendor-specific payload format. The webhook ingress will auto-detect the kind and call `buildInboundEvent()` to normalize it into a standard `InboundEvent` before calling Match. Manual triggers do not need a webhook handler (they fire from the UI only). ## Related Areas - **Workload Refactor** — Full context on the trigger-split, hard legacy cutover, and UI migration: [`docs/WORKLOAD_REFACTOR_TODO.md`](../WORKLOAD_REFACTOR_TODO.md) - **Webhook Signing** — HMAC-SHA256 verification, per-tier secret resolution, receiver code samples: [`docs/webhooks.md`](../webhooks.md) - **Log Scanning + Event Triggers** — Observability features that build on the trigger infrastructure: [`docs/LOGSCAN_AND_TRIGGERS_TODO.md`](../LOGSCAN_AND_TRIGGERS_TODO.md) ## Registry Details Both Source and Trigger registries use sync.RWMutex for thread-safe lookup. Duplicate registration panics at init() (indicating a bug, not a runtime failure). Lookup errors surface missing kinds clearly so operators can diagnose when a workload references a kind whose package was not blank-imported. **Lazy note**: Registries are not lazy — all kinds must be registered at boot before the HTTP server starts. This ensures consistent behavior across handlers and reconciler tasks.