tiny-forge/docs/CODEMAPS/workload-plugin.md

Workload Plugin Architecture Codemap

Last Updated: 2026-05-16
Status: Core contract for Source × Trigger plugin system (post-trigger-split refactor)

Abstract

internal/workload/plugin/ defines the Source × Trigger plugin contracts that decouple the deployer pipeline from specific deployable shapes (image, compose, static) and redeploy signals (registry push, git push, manual, cron). A Workload carries opaque config blobs; registry lookups route each to the matching plugin. New plugin kinds are added only via registration from init() — no changes needed to the API, deployer, or webhook handler.

Key Files

Path	Role
internal/workload/plugin/plugin.go	Package doc; Deps bundle (Store, Docker, Proxy, DNS, Health, Notifier, Events, EncKey)
internal/workload/plugin/types.go	Workload, DeploymentIntent, PublicFace, InboundEvent, ImagePushEvent, GitEvent, ManualEvent; helpers SourceConfigOf[T], TriggerConfigOf[T]
internal/workload/plugin/source.go	Source interface (Kind / Validate / Deploy / Teardown / Reconcile); registries RegisterSource / GetSource; Schemaer optional interface; helpers SourceKinds / SchemaSampleFor
internal/workload/plugin/trigger.go	Trigger interface (Kind / Validate / Match); registries RegisterTrigger / GetTrigger; helper TriggerKinds
internal/workload/plugin/binding.go	MergeJSONConfig (top-level JSON merge for trigger + binding override); WithEffectiveTrigger (used by webhook fan-out to compose merged config)
internal/workload/plugin/registry.go	AllSources / AllTriggers snapshot helpers (used by /api/workloads/source-kinds and /api/workloads/trigger-kinds)

Architecture Overview

Contract Surface: Source vs Trigger

Workload (unifying user entity)
├── SourceKind + SourceConfig (JSON blob)
│   └── Source.Deploy()      ← routes to image, compose, or static
│       Source.Teardown()
│       Source.Reconcile()
│
└── TriggerKind + TriggerConfig (JSON blob)
    └── Trigger.Match(InboundEvent)  ← routes to registry, git, or manual
        returns DeploymentIntent

• Source (stateless, 5 methods): owns full container lifecycle (deploy, tear down, reconcile state)
• Trigger (stateless, 3 methods): given an inbound event + workload config, decide whether to fire a deploy intent

Dispatch Seam: Deployer → Plugins

deployer/dispatch.go
├── DispatchPlugin(w, intent) → plugin.GetSource(w.SourceKind) → Source.Deploy()
├── DispatchTeardown(w)       → plugin.GetSource(w.SourceKind) → Source.Teardown()
└── DispatchReconcile(w)      → plugin.GetSource(w.SourceKind) → Source.Reconcile()

PluginDeps() assembles:
├── Store (workload / container / webhook tables)
├── Docker (container orchestration)
├── Proxy (route manager)
├── DNS (DNS provider, nil for wildcard)
├── Health (status checker)
├── Notifier (webhook client)
├── Events (event bus for deploy lifecycle)
└── EncKey (for crypto.Encrypt/Decrypt of config secrets)

Webhook Fan-Out Path: Trigger → Bindings

webhook/trigger_handler.go: POST /api/webhook/triggers/{secret}
├── Resolve secret → Trigger record
├── Parse body → InboundEvent (auto-detects image-push, git-push, git-tag, manual, cron-tick)
├── plugin.GetTrigger(trg.Kind) → Trigger plugin
└── For each enabled workload_trigger_binding (bounded concurrency = 4):
    ├── plugin.WithEffectiveTrigger()
    │   └── MergeJSONConfig(trigger.config, binding.binding_config)
    │       returns Workload copy with merged TriggerConfig
    ├── Trigger.Match(evt, merged_workload)
    │   returns DeploymentIntent or nil
    └── If intent returned: DispatchPlugin(w, intent) → Source.Deploy()

Key design point: MergeJSONConfig always returns freshly allocated slices (defensive copy) so binding fan-out never risks aliasing across goroutines.

Concrete Implementations

Sources

Kind	Package	Files	Purpose
image	internal/workload/plugin/source/image/	image.go + deps	Docker image deploys (blue-green multi-face proxy, registry auth)
compose	internal/workload/plugin/source/compose/	compose.go + deps	docker-compose stacks via internal/stack/ helpers
static	internal/workload/plugin/source/static/	deploy.go, teardown.go, reconcile.go, state.go, env.go, build.go, naming.go, static.go	Git-folder-backed static site (nginx or Deno) via internal/staticsite/ helpers

Static source inline port note: The legacy /api/sites/* HTTP surface still exists for backwards compat; the static plugin operates directly on containers + workload_env tables without synthetic static_sites rows.

Triggers

Kind	Package	Files	Purpose
registry	internal/workload/plugin/trigger/registry/	registry.go	Image push events (registry webhook or watcher)
git	internal/workload/plugin/trigger/git/	git.go	Git push / tag-create (Gitea / GitHub / GitLab)
manual	internal/workload/plugin/trigger/manual/	manual.go	Manual-only (no auto-fire)

Trigger lifecycle note: As first-class records since the trigger-split refactor, triggers are bound to workloads via workload_trigger_bindings join table. Each binding carries optional binding_config (merged with trigger's config before Match is called).

Data Flow: Example Webhook Dispatch

1. Inbound POST /api/webhook/triggers/{secret}
   ↓
2. Lookup Trigger by secret
   ↓
3. Parse body → InboundEvent (detects kind: image-push, git-push, manual, ...)
   ↓
4. Load Trigger plugin (e.g. plugin.GetTrigger("git"))
   ↓
5. Load all bindings for this trigger
   ↓
6. For each binding (concurrent, max 4):
   a. Merge trigger.config + binding.binding_config
   b. Build Workload copy with merged TriggerConfig
   c. Call Trigger.Match(evt, merged_workload)
   d. If Match returns DeploymentIntent:
      - Call DispatchPlugin(w, intent)
      - Source.Deploy executes (e.g. pull image, build container)
   e. If Match returns (nil, nil): skip silently
   f. If Match returns error: log at warn level, continue to next binding
   ↓
7. Aggregate results (deployed count, skip reason counts)
   ↓
8. Return 200 with result summary

Integration Points

API Layer (internal/api/workloads.go)

• /api/workloads/{id} — GET returns Workload with SourceKind + SourceConfig
• /api/workloads/{id} — PUT/POST routes to Validate (source plugin checks config schema)
• /api/workloads/source-kinds — GET calls plugin.SourceKinds() + plugin.SchemaSampleFor() per kind
• /api/workloads/trigger-kinds — GET calls plugin.TriggerKinds() + plugin.SchemaSampleFor() per kind
• /api/workloads/{id}/deploy — POST manual deploy: builds ManualEvent, calls webhook handler

Webhook Ingress (internal/webhook/)

• trigger_handler.go — POST /api/webhook/triggers/{secret} implements the fan-out dispatcher (see Data Flow above)
• parse.go — buildInboundEvent() normalizes vendor-specific payloads (Gitea / GitHub / GitLab / Docker Hub / generic registry) into InboundEvent

Reconciler (internal/reconciler/reconciler.go)

• reconcilePluginWorkloads() — iterates every workload with SourceKind != "", calls DispatchReconcile(w) on fixed schedule (e.g. every 5 minutes)
• Keeps containers index in sync with deployed reality (garbage-collect orphaned containers, restart crashed services)

Deployer (internal/deployer/dispatch.go)

• DispatchPlugin() / DispatchTeardown() / DispatchReconcile() — route calls to the matching Source plugin
• PluginDeps() — assembles the stateless dependency bundle (called per-Deploy, per-Trigger.Match)

External Dependencies

Package	Version	Used For
encoding/json	stdlib	Config marshaling / unmarshaling
sync (RWMutex)	stdlib	Registry thread-safety (SourceKinds, TriggerKinds, Schemaer lookup)
context	stdlib	Timeout control in Deploy / Teardown / Reconcile / Match
internal/store	local	Workload / container / binding / trigger table access
internal/docker	local	Container orchestration (Sources use this)
internal/proxy	local	Route registration (Sources use this)
internal/dns	local	DNS record creation (Sources use this, nil for wildcard DNS)
internal/health	local	Status checks (available to plugin Deps)
internal/notify	local	Webhook client (available to plugin Deps)
internal/events	local	Event bus (Sources publish lifecycle events)

How to Add a New Source Kind

1. Create internal/workload/plugin/source/{kind}/{kind}.go with a struct implementing Source:

type source struct{}

func init() { plugin.RegisterSource(&source{}) }

func (s *source) Kind() string { return "k8s" }

func (s *source) Validate(cfg json.RawMessage) error { /* ... */ }

func (s *source) Deploy(ctx context.Context, deps plugin.Deps, w plugin.Workload, intent plugin.DeploymentIntent) error { /* ... */ }

func (s *source) Teardown(ctx context.Context, deps plugin.Deps, w plugin.Workload) error { /* ... */ }

func (s *source) Reconcile(ctx context.Context, deps plugin.Deps, w plugin.Workload) error { /* ... */ }

// Optional: implement Schemaer for JSON schema on /api/workloads/source-kinds
func (s *source) SchemaSample() any { return Config{ /* sample fields */ } }

2. Blank-import the sub-package from cmd/server/main.go so init() fires at boot:

import (
    _ "github.com/alexei/tinyforge/internal/workload/plugin/source/k8s"
)

3. Optionally ship a hand-rolled form in web/src/routes/apps/{new,[id]}/+page.svelte (per workload-first UX rule). The JSON editor remains a fallback for power users.

How to Add a New Trigger Kind

1. Create internal/workload/plugin/trigger/{kind}/{kind}.go with a struct implementing Trigger:

type trigger struct{}

func init() { plugin.RegisterTrigger(&trigger{}) }

func (t *trigger) Kind() string { return "cron" }

func (t *trigger) Validate(cfg json.RawMessage) error { /* ... */ }

func (t *trigger) Match(ctx context.Context, deps plugin.Deps, w plugin.Workload, evt plugin.InboundEvent) (*plugin.DeploymentIntent, error) {
    // Decide whether this trigger fires for the given event + workload config
    // Return (nil, nil) to skip silently, (*intent, nil) to deploy, (nil, err) for config errors
}

// Optional: implement Schemaer for JSON schema on /api/workloads/trigger-kinds
func (t *trigger) SchemaSample() any { return Config{ /* sample fields */ } }

2. Blank-import from cmd/server/main.go.

3. Ship a form variant in web/src/lib/components/TriggerKindForm.svelte so the /triggers/new page and workload bindings panel can author kind-specific config.

4. Important: Triggers that need to handle inbound webhooks should register a route in internal/webhook/ for their vendor-specific payload format. The webhook ingress will auto-detect the kind and call buildInboundEvent() to normalize it into a standard InboundEvent before calling Match. Manual triggers do not need a webhook handler (they fire from the UI only).

Related Areas

• Workload Refactor — Full context on the trigger-split, hard legacy cutover, and UI migration: docs/WORKLOAD_REFACTOR_TODO.md
• Webhook Signing — HMAC-SHA256 verification, per-tier secret resolution, receiver code samples: docs/webhooks.md
• Log Scanning + Event Triggers — Observability features that build on the trigger infrastructure: docs/LOGSCAN_AND_TRIGGERS_TODO.md

Registry Details

Both Source and Trigger registries use sync.RWMutex for thread-safe lookup. Duplicate registration panics at init() (indicating a bug, not a runtime failure). Lookup errors surface missing kinds clearly so operators can diagnose when a workload references a kind whose package was not blank-imported.

Lazy note: Registries are not lazy — all kinds must be registered at boot before the HTTP server starts. This ensures consistent behavior across handlers and reconciler tasks.