From 0bb52f9ec62a75c8d76596735d3ec0b44210a0af Mon Sep 17 00:00:00 2001 From: "alexei.dolgolyov" Date: Fri, 27 Mar 2026 20:42:42 +0300 Subject: [PATCH] chore: add feature planner setup for docker-watcher-core Create structured plan files with 12 phases covering the full implementation: scaffold, store, crypto, Docker/NPM clients, registry poller, webhook, deployer, API layer, SvelteKit frontend, embedding, and hardening. --- plans/docker-watcher-core/CONTEXT.md | 53 +++++++++++++++ plans/docker-watcher-core/PLAN.md | 64 +++++++++++++++++++ .../phase-1-scaffold-store.md | 57 +++++++++++++++++ .../phase-10-settings-deploy.md | 56 ++++++++++++++++ .../docker-watcher-core/phase-11-embed-sse.md | 55 ++++++++++++++++ .../docker-watcher-core/phase-12-hardening.md | 59 +++++++++++++++++ .../phase-2-crypto-config.md | 50 +++++++++++++++ .../phase-3-docker-client.md | 52 +++++++++++++++ .../docker-watcher-core/phase-4-npm-client.md | 48 ++++++++++++++ .../phase-5-registry-poller.md | 49 ++++++++++++++ .../phase-6-webhook-handler.md | 48 ++++++++++++++ plans/docker-watcher-core/phase-7-deployer.md | 54 ++++++++++++++++ .../docker-watcher-core/phase-8-api-layer.md | 59 +++++++++++++++++ .../docker-watcher-core/phase-9-dashboard.md | 64 +++++++++++++++++++ 14 files changed, 768 insertions(+) create mode 100644 plans/docker-watcher-core/CONTEXT.md create mode 100644 plans/docker-watcher-core/PLAN.md create mode 100644 plans/docker-watcher-core/phase-1-scaffold-store.md create mode 100644 plans/docker-watcher-core/phase-10-settings-deploy.md create mode 100644 plans/docker-watcher-core/phase-11-embed-sse.md create mode 100644 plans/docker-watcher-core/phase-12-hardening.md create mode 100644 plans/docker-watcher-core/phase-2-crypto-config.md create mode 100644 plans/docker-watcher-core/phase-3-docker-client.md create mode 100644 plans/docker-watcher-core/phase-4-npm-client.md create mode 100644 plans/docker-watcher-core/phase-5-registry-poller.md create mode 100644 plans/docker-watcher-core/phase-6-webhook-handler.md create mode 100644 plans/docker-watcher-core/phase-7-deployer.md create mode 100644 plans/docker-watcher-core/phase-8-api-layer.md create mode 100644 plans/docker-watcher-core/phase-9-dashboard.md diff --git a/plans/docker-watcher-core/CONTEXT.md b/plans/docker-watcher-core/CONTEXT.md new file mode 100644 index 0000000..581c2fc --- /dev/null +++ b/plans/docker-watcher-core/CONTEXT.md @@ -0,0 +1,53 @@ +# Feature Context: Docker Watcher Core + +## Configuration +- **Development mode:** Automated +- **Execution mode:** Orchestrator +- **Strategy:** Big Bang (with per-phase code quality reviews) +- **Build (Go):** `go build ./cmd/server/` +- **Test (Go):** `go test ./...` +- **Lint (Go):** `golangci-lint run` +- **Build (Frontend):** `cd web && npm run build` +- **Test (Frontend):** `cd web && npm test` +- **Dev server:** `go run ./cmd/server/` + +## Current State +Greenfield project. Only PLAN.md exists with the architecture document. + +## Temporary Workarounds +None yet. + +## Cross-Phase Dependencies +- Phase 2 depends on Phase 1 (store CRUD for seed import) +- Phases 3 and 4 are independent of each other (can run in parallel) +- Phase 5 depends on Phase 1 (store for poll state) +- Phase 6 depends on Phase 3 (Docker inspect for auto-creation) and Phase 1 (store) +- Phase 7 depends on Phases 3, 4, 5 (Docker, NPM, registry clients) +- Phase 8 depends on Phases 1-7 (wires everything to HTTP) +- Phases 9 and 10 are independent of each other (can run in parallel) +- Phase 11 depends on Phases 8, 9, 10 (embeds frontend, SSE wires to API) +- Phase 12 depends on all prior phases + +## Deferred Work +None yet. + +## Failed Approaches +None yet. + +## Review Findings Log +None yet. + +## Phase Execution Log +| Phase | Agent Used | Test Writer | Parallel | Notes | +|-------|-----------|-------------|----------|-------| +| — | — | — | — | No phases executed yet | + +## Environment & Runtime Notes +- Platform: Windows 10 (development), Linux (deployment target) +- Docker socket: `/var/run/docker.sock` (Linux) — development may need Docker Desktop +- Go version: TBD (will be determined in Phase 1) + +## Implementation Notes +- Big Bang strategy: intermediate phases skip build/tests, code quality reviews after every phase +- Final phase (12) is the only phase where build + full test suite must pass +- Phases 3+4 and 9+10 identified for parallel execution diff --git a/plans/docker-watcher-core/PLAN.md b/plans/docker-watcher-core/PLAN.md new file mode 100644 index 0000000..5db0470 --- /dev/null +++ b/plans/docker-watcher-core/PLAN.md @@ -0,0 +1,64 @@ +# Feature: Docker Watcher Core + +**Branch:** `feature/docker-watcher-core` +**Base branch:** `main` +**Created:** 2026-03-27 +**Status:** 🟡 In Progress +**Strategy:** Big Bang (with per-phase code quality reviews) +**Mode:** Automated +**Execution:** Orchestrator + +## Summary + +A self-hosted tool that automates Docker container deployment with Nginx Proxy Manager integration. Detects new images from Gitea/GitHub registries, deploys containers, and configures reverse proxy routing — all from a web dashboard. Supports multiple simultaneous versions of the same project. + +## Build & Test Commands +- **Build (Go):** `go build ./cmd/server/` +- **Test (Go):** `go test ./...` +- **Lint (Go):** `golangci-lint run` +- **Build (Frontend):** `cd web && npm run build` +- **Test (Frontend):** `cd web && npm test` +- **Dev server:** `go run ./cmd/server/` + +## Phases + +- [ ] Phase 1: Project Scaffold & SQLite Store [domain: backend] → [subplan](./phase-1-scaffold-store.md) +- [ ] Phase 2: Crypto & Config Seed Loader [domain: backend] → [subplan](./phase-2-crypto-config.md) +- [ ] Phase 3: Docker Client [domain: backend] → [subplan](./phase-3-docker-client.md) +- [ ] Phase 4: NPM Client [domain: backend] → [subplan](./phase-4-npm-client.md) +- [ ] Phase 5: Registry Client & Poller [domain: backend] → [subplan](./phase-5-registry-poller.md) +- [ ] Phase 6: Webhook Handler [domain: backend] → [subplan](./phase-6-webhook-handler.md) +- [ ] Phase 7: Deployer & Health Checker [domain: backend] → [subplan](./phase-7-deployer.md) +- [ ] Phase 8: REST API Layer [domain: backend] → [subplan](./phase-8-api-layer.md) +- [ ] Phase 9: SvelteKit Dashboard & Project Views [domain: frontend] → [subplan](./phase-9-dashboard.md) +- [ ] Phase 10: Quick Deploy & Settings Pages [domain: frontend] → [subplan](./phase-10-settings-deploy.md) +- [ ] Phase 11: Frontend Embed & Real-Time Updates [domain: fullstack] → [subplan](./phase-11-embed-sse.md) +- [ ] Phase 12: Hardening [domain: backend] → [subplan](./phase-12-hardening.md) + +### Parallel Execution Notes +- Phases 3 and 4 are independent (Docker client vs NPM client) — can run in parallel +- Phases 9 and 10 are independent (dashboard vs settings pages) — can run in parallel + +## Phase Progress Log + +| Phase | Domain | Status | Review | Build | Committed | +|-------|--------|--------|--------|-------|-----------| +| Phase 1: Scaffold & Store | backend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 2: Crypto & Config | backend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 3: Docker Client | backend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 4: NPM Client | backend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 5: Registry & Poller | backend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 6: Webhook Handler | backend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 7: Deployer & Health | backend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 8: API Layer | backend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 9: Dashboard | frontend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 10: Settings & Deploy | frontend | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 11: Embed & SSE | fullstack | ⬜ Not Started | ⬜ | ⏭️ Skip (Big Bang) | ⬜ | +| Phase 12: Hardening | backend | ⬜ Not Started | ⬜ | ✅ Required (Final) | ⬜ | + +## Final Review +- [ ] Comprehensive code review +- [ ] Full build passes +- [ ] Full test suite passes +- [ ] Security review +- [ ] Merged to `main` diff --git a/plans/docker-watcher-core/phase-1-scaffold-store.md b/plans/docker-watcher-core/phase-1-scaffold-store.md new file mode 100644 index 0000000..362e1fc --- /dev/null +++ b/plans/docker-watcher-core/phase-1-scaffold-store.md @@ -0,0 +1,57 @@ +# Phase 1: Project Scaffold & SQLite Store + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Initialize the Go project, establish the directory structure, and implement the SQLite store with schema, migrations, and CRUD operations for all entities. + +## Tasks + +- [ ] Task 1: Initialize Go module (`go mod init`), create directory structure per PLAN.md +- [ ] Task 2: Add core dependencies to go.mod (sqlite, chi, yaml, uuid, cron) +- [ ] Task 3: Define SQLite schema — tables for projects, stages, registries, settings, instances, deploys, deploy_logs +- [ ] Task 4: Implement store initialization with auto-migration (create tables if not exist) +- [ ] Task 5: Implement projects CRUD (Create, GetByID, GetAll, Update, Delete) +- [ ] Task 6: Implement stages CRUD (Create, GetByProjectID, Update, Delete) +- [ ] Task 7: Implement registries CRUD (Create, GetByID, GetAll, Update, Delete) +- [ ] Task 8: Implement settings Get/Update (single-row config pattern) +- [ ] Task 9: Implement instances CRUD (Create, GetByStageID, GetByID, Update, Delete, UpdateStatus) +- [ ] Task 10: Implement deploys CRUD (Create, GetByProjectID, GetRecent, GetByID) + deploy_logs append +- [ ] Task 11: Create `cmd/server/main.go` entry point (minimal — just opens DB, defers close) + +## Files to Modify/Create +- `go.mod` — module definition and dependencies +- `go.sum` — dependency checksums +- `cmd/server/main.go` — entry point +- `internal/store/store.go` — DB connection, schema, migrations +- `internal/store/projects.go` — project queries +- `internal/store/stages.go` — stage queries +- `internal/store/registries.go` — registry queries +- `internal/store/settings.go` — settings queries +- `internal/store/instances.go` — instance queries +- `internal/store/deploys.go` — deploy history queries + +## Acceptance Criteria +- `go mod tidy` succeeds +- All store CRUD functions are implemented with proper error handling +- Schema covers all entities from the architecture plan +- Entry point compiles (may not fully run until later phases wire everything) + +## Notes +- Use `modernc.org/sqlite` for CGo-free SQLite +- Use `go-chi/chi/v5` for routing (will be wired in Phase 8) +- Settings table uses a single-row pattern (one row, upsert on update) +- Instance status should be an enum-like string: "running", "stopped", "failed", "removing" +- Deploy status: "pending", "pulling", "starting", "configuring_proxy", "health_checking", "success", "failed", "rolled_back" + +## Review Checklist +- [ ] All tasks completed +- [ ] Code follows Go conventions (gofmt, proper error returns) +- [ ] No unintended side effects +- [ ] Schema is normalized and covers all planned entities +- [ ] CRUD functions handle not-found cases properly + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-10-settings-deploy.md b/plans/docker-watcher-core/phase-10-settings-deploy.md new file mode 100644 index 0000000..26a7e82 --- /dev/null +++ b/plans/docker-watcher-core/phase-10-settings-deploy.md @@ -0,0 +1,56 @@ +# Phase 10: Quick Deploy & Settings Pages + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** frontend + +## Objective +Build the Quick Deploy page (paste image, auto-inspect, one-click deploy) and all Settings pages (registries, credentials, global settings, webhook URL). + +## Tasks + +- [ ] Task 1: Quick Deploy page (`routes/deploy/+page.svelte`) — image URL input, inspect button +- [ ] Task 2: Quick Deploy inspect flow — call /api/deploy/inspect, display auto-filled form (project name, port, stage, subdomain) +- [ ] Task 3: Quick Deploy submit — user reviews defaults, clicks Deploy, calls /api/deploy/quick +- [ ] Task 4: Settings layout (`routes/settings/+layout.svelte`) — sub-navigation for settings sections +- [ ] Task 5: Global settings page (`routes/settings/+page.svelte`) — domain, server IP, network, subdomain pattern, polling interval +- [ ] Task 6: Registries page (`routes/settings/registries/+page.svelte`) — list, add, edit, delete, test connection +- [ ] Task 7: Credentials page (`routes/settings/credentials/+page.svelte`) — NPM credentials, registry tokens (masked display) +- [ ] Task 8: Webhook URL display and regenerate button in settings +- [ ] Task 9: Projects config page (`routes/projects/config/+page.svelte` or integrated into project detail) — add/edit/delete projects, configure stages +- [ ] Task 10: Stage configuration form — tag patterns, auto_deploy toggle, max_instances, subdomain override +- [ ] Task 11: Form validation on all input pages — required fields, URL format, port range +- [ ] Task 12: Success/error toast notifications for all form submissions + +## Files to Modify/Create +- `web/src/routes/deploy/+page.svelte` — quick deploy +- `web/src/routes/settings/+layout.svelte` — settings layout +- `web/src/routes/settings/+page.svelte` — global settings +- `web/src/routes/settings/registries/+page.svelte` — registry management +- `web/src/routes/settings/credentials/+page.svelte` — credential management +- `web/src/lib/components/Toast.svelte` — toast notifications +- `web/src/lib/components/FormField.svelte` — reusable form field with validation + +## Acceptance Criteria +- Quick Deploy: paste image URL → inspect → review defaults → deploy works end-to-end +- All settings are editable and saved via API +- Registry test connection shows success/failure +- Credentials are masked in display (`••••••••`) +- Webhook URL is shown with copy button and regenerate option +- Form validation prevents bad submissions + +## Notes +- Quick Deploy is the zero-config entry point — should be dead simple UX +- Credential fields: show mask, edit replaces entirely (no partial edit) +- Registry test: calls POST /api/registries/:id/test, shows connection result +- Toast component: appears top-right, auto-dismiss after 5s, color-coded (green/red) + +## Review Checklist +- [ ] All tasks completed +- [ ] Quick deploy flow is intuitive (minimal clicks) +- [ ] Credentials never shown in plaintext in UI +- [ ] Form validation covers required fields and formats +- [ ] Error states are handled with user-friendly messages + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-11-embed-sse.md b/plans/docker-watcher-core/phase-11-embed-sse.md new file mode 100644 index 0000000..ebfb130 --- /dev/null +++ b/plans/docker-watcher-core/phase-11-embed-sse.md @@ -0,0 +1,55 @@ +# Phase 11: Frontend Embed & Real-Time Updates + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** fullstack + +## Objective +Build SvelteKit to static files, embed into the Go binary with `go:embed`, serve from Go, and implement SSE for real-time deploy progress and instance status updates. + +## Tasks + +- [ ] Task 1: Configure SvelteKit static adapter to output to `web/build/` +- [ ] Task 2: Add `//go:embed web/build` directive in Go — serve static files +- [ ] Task 3: Create Go handler for serving embedded SPA — serve index.html for all non-API routes (SPA fallback) +- [ ] Task 4: Implement SSE endpoint for deploy logs — `GET /api/deploys/:id/logs` streams deploy progress in real-time +- [ ] Task 5: Implement SSE endpoint for instance status — `GET /api/events` streams instance status changes +- [ ] Task 6: Create event bus/broadcaster in Go — publish events from deployer, subscribe from SSE handlers +- [ ] Task 7: Frontend: connect to SSE for deploy progress — update deploy log view in real-time +- [ ] Task 8: Frontend: connect to SSE for instance status — update dashboard/project views without refresh +- [ ] Task 9: Handle SSE reconnection in frontend — auto-reconnect with backoff on disconnect +- [ ] Task 10: Build script/Makefile — `make build` builds frontend then Go binary + +## Files to Modify/Create +- `web/svelte.config.js` — ensure static adapter outputs to `web/build/` +- `internal/api/static.go` — embedded static file server with SPA fallback +- `internal/api/sse.go` — SSE endpoints for deploy logs and instance events +- `internal/events/bus.go` — event bus for publishing/subscribing to events +- `web/src/lib/sse.ts` — SSE client helper with auto-reconnect +- `web/src/routes/+layout.svelte` — wire up global SSE connection for instance status +- `Makefile` — build frontend + backend +- `cmd/server/main.go` — wire embedded static serving and event bus + +## Acceptance Criteria +- `make build` produces a single Go binary with embedded frontend +- Go binary serves the SvelteKit SPA on all non-API routes +- Deploy progress streams in real-time via SSE +- Instance status updates appear without page refresh +- SSE reconnects automatically after network hiccups + +## Notes +- `go:embed` requires the embedded directory to be relative to the Go source file +- SPA fallback: any request that doesn't match `/api/*` gets `index.html` +- Event bus: simple pub/sub with channels — no external dependency needed +- SSE format: `data: {"type": "deploy_log", "payload": {...}}\n\n` +- Keep SSE connections lightweight — use context cancellation for cleanup + +## Review Checklist +- [ ] All tasks completed +- [ ] Single binary serves both API and frontend +- [ ] SSE handles multiple concurrent clients +- [ ] No goroutine leaks on SSE disconnect +- [ ] Build process is reproducible (Makefile) + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-12-hardening.md b/plans/docker-watcher-core/phase-12-hardening.md new file mode 100644 index 0000000..2576498 --- /dev/null +++ b/plans/docker-watcher-core/phase-12-hardening.md @@ -0,0 +1,59 @@ +# Phase 12: Hardening + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Production hardening — blue-green deploys, promote flow, dashboard auth, graceful shutdown, structured logging, and config export. + +## Tasks + +- [ ] Task 1: Blue-green deploys — start new container, health check, swap NPM proxy, then stop old container (zero downtime) +- [ ] Task 2: Promote flow — enforce `promote_from` for production deploys (only tags running in source stage are eligible) +- [ ] Task 3: Dashboard auth — basic auth or token-based authentication for the web UI +- [ ] Task 4: Auth middleware — protect all /api/* routes except webhook +- [ ] Task 5: Graceful shutdown — handle SIGTERM/SIGINT, drain in-progress deploys, close DB, stop poller +- [ ] Task 6: Structured logging — JSON logs with deploy context (project, stage, tag, instance ID) +- [ ] Task 7: Config export — download current SQLite state as YAML (reverse of seed import) +- [ ] Task 8: Dockerfile — multi-stage build (build frontend + Go, copy to minimal image) +- [ ] Task 9: docker-compose.yml — production-ready compose file with volumes, network, env +- [ ] Task 10: Final wiring review — ensure all services are properly initialized and shut down + +## Files to Modify/Create +- `internal/deployer/bluegreen.go` — blue-green deploy strategy +- `internal/deployer/promote.go` — promote flow logic +- `internal/api/auth.go` — authentication middleware +- `internal/config/export.go` — config export to YAML +- `internal/logging/logger.go` — structured JSON logger +- `cmd/server/main.go` — graceful shutdown, structured logging init +- `Dockerfile` — multi-stage build +- `docker-compose.yml` — production compose file + +## Acceptance Criteria +- Blue-green: zero downtime during deploy (old container serves until new one is healthy) +- Promote: production deploy only accepts tags from the specified source stage +- Auth: unauthenticated requests to /api/* (except webhook) return 401 +- Graceful shutdown: in-progress deploys complete before exit +- Logs are JSON-formatted with contextual fields +- Config export produces valid YAML that could be re-imported +- Docker image builds and runs correctly + +## Notes +- Blue-green: keep old container running until new one passes health check, then swap NPM proxy and stop old +- Auth: start simple (basic auth via env var), can be enhanced later (JWT, OIDC) +- SIGTERM handling: use Go's `os/signal` + `context.WithCancel` +- Structured logging: use `log/slog` (Go stdlib since 1.21) +- Dockerfile: build stage with Node.js + Go, runtime stage with scratch/alpine +- This is the FINAL phase — build and full test suite MUST pass here + +## Review Checklist +- [ ] All tasks completed +- [ ] Blue-green deploy handles rollback if new container fails +- [ ] Auth doesn't block webhook endpoint +- [ ] Graceful shutdown tested with concurrent deploys +- [ ] Dockerfile produces a minimal image +- [ ] docker-compose.yml matches the example in PLAN.md + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-2-crypto-config.md b/plans/docker-watcher-core/phase-2-crypto-config.md new file mode 100644 index 0000000..19fa0c2 --- /dev/null +++ b/plans/docker-watcher-core/phase-2-crypto-config.md @@ -0,0 +1,50 @@ +# Phase 2: Crypto & Config Seed Loader + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Implement AES-256 encryption for credential storage and the YAML seed config parser that imports into SQLite on first launch. + +## Tasks + +- [ ] Task 1: Implement AES-256-GCM encrypt/decrypt functions using Go stdlib `crypto/aes` + `crypto/cipher` +- [ ] Task 2: Key derivation from ENCRYPTION_KEY env var (SHA-256 hash to get 32 bytes) +- [ ] Task 3: Define YAML config structs matching the seed format from PLAN.md +- [ ] Task 4: Implement YAML parser — read and validate seed file +- [ ] Task 5: Implement seed importer — checks if DB is empty, if so imports YAML into SQLite via store CRUD +- [ ] Task 6: Encrypt credential fields (registry tokens, NPM password) during import +- [ ] Task 7: Create `docker-watcher.example.yaml` with documented example config +- [ ] Task 8: Wire seed import into `cmd/server/main.go` startup sequence + +## Files to Modify/Create +- `internal/crypto/crypto.go` — AES-256-GCM encrypt/decrypt +- `internal/config/config.go` — YAML structs and parser +- `internal/config/seed.go` — seed import logic (YAML → SQLite) +- `docker-watcher.example.yaml` — example seed config +- `cmd/server/main.go` — add seed import to startup + +## Acceptance Criteria +- Encrypt then decrypt round-trips correctly +- Different plaintexts produce different ciphertexts (random nonce) +- YAML parsing handles all fields from the seed format +- Seed import creates projects, stages, registries, and settings in SQLite +- Credentials are encrypted before storage +- Import is idempotent — skipped if DB already has data + +## Notes +- ENCRYPTION_KEY is the only secret env var — everything else is encrypted in SQLite +- Use GCM mode for authenticated encryption (integrity + confidentiality) +- Seed import should be transactional — all or nothing +- The example YAML should have placeholder values, not real credentials + +## Review Checklist +- [ ] All tasks completed +- [ ] Crypto uses secure practices (random nonce, GCM, no ECB) +- [ ] No hardcoded keys or secrets +- [ ] YAML parsing validates required fields +- [ ] Import is transactional + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-3-docker-client.md b/plans/docker-watcher-core/phase-3-docker-client.md new file mode 100644 index 0000000..c24868c --- /dev/null +++ b/plans/docker-watcher-core/phase-3-docker-client.md @@ -0,0 +1,52 @@ +# Phase 3: Docker Client + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Implement the Docker Engine API wrapper for container lifecycle management — pull images, inspect, create/start/stop/remove containers, and manage networks. + +## Tasks + +- [ ] Task 1: Create Docker client wrapper with socket connection (`/var/run/docker.sock`) +- [ ] Task 2: Implement `PullImage(ctx, image, tag, authConfig)` — pull with optional registry auth +- [ ] Task 3: Implement `InspectImage(ctx, image)` — extract EXPOSE ports, HEALTHCHECK, labels +- [ ] Task 4: Implement `CreateContainer(ctx, config)` — create with name, image, env, ports, network, labels +- [ ] Task 5: Implement `StartContainer(ctx, containerID)`, `StopContainer(ctx, containerID, timeout)`, `RemoveContainer(ctx, containerID, force)` +- [ ] Task 6: Implement `RestartContainer(ctx, containerID, timeout)` +- [ ] Task 7: Implement `ListContainers(ctx, filters)` — filter by labels to find managed containers +- [ ] Task 8: Implement `EnsureNetwork(ctx, networkName)` — create network if not exists +- [ ] Task 9: Implement `ConnectNetwork(ctx, networkID, containerID)` — attach container to network +- [ ] Task 10: Add docker-watcher labels to all managed containers (`docker-watcher.project`, `docker-watcher.stage`, `docker-watcher.instance-id`) + +## Files to Modify/Create +- `internal/docker/client.go` — Docker client wrapper, connection setup +- `internal/docker/container.go` — container lifecycle operations +- `internal/docker/image.go` — pull and inspect operations +- `internal/docker/network.go` — network management + +## Acceptance Criteria +- Client connects to Docker socket +- Pull handles both public and authenticated registries +- Image inspection extracts port, healthcheck, and label metadata +- Container creation applies all config (env, ports, network, labels) +- All operations return meaningful errors +- Managed containers are identifiable via labels + +## Notes +- Use `github.com/docker/docker/client` SDK +- Container names should be deterministic: `dw-{project}-{stage}-{tag-sanitized}` +- All containers should be on the shared network (e.g., `staging-net`) +- Port mapping: container's EXPOSE port → random host port (Docker auto-assigns) +- Auth config for private registries will come from the store (encrypted tokens) + +## Review Checklist +- [ ] All tasks completed +- [ ] Proper context propagation for cancellation +- [ ] Resource cleanup (close client, remove failed containers) +- [ ] No hardcoded values +- [ ] Error messages include container/image identifiers + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-4-npm-client.md b/plans/docker-watcher-core/phase-4-npm-client.md new file mode 100644 index 0000000..a67ab67 --- /dev/null +++ b/plans/docker-watcher-core/phase-4-npm-client.md @@ -0,0 +1,48 @@ +# Phase 4: NPM Client + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Implement the Nginx Proxy Manager API client — JWT authentication, CRUD for proxy hosts, and host lookup. + +## Tasks + +- [ ] Task 1: Create NPM client struct with base URL, cached JWT token, and auto-refresh +- [ ] Task 2: Implement `Authenticate(ctx, email, password)` — POST /api/tokens, store JWT +- [ ] Task 3: Implement `CreateProxyHost(ctx, config)` — POST /api/nginx/proxy-hosts +- [ ] Task 4: Implement `UpdateProxyHost(ctx, id, config)` — PUT /api/nginx/proxy-hosts/{id} +- [ ] Task 5: Implement `DeleteProxyHost(ctx, id)` — DELETE /api/nginx/proxy-hosts/{id} +- [ ] Task 6: Implement `ListProxyHosts(ctx)` — GET /api/nginx/proxy-hosts +- [ ] Task 7: Implement `FindProxyHostByDomain(ctx, domain)` — search existing hosts by domain name +- [ ] Task 8: Define proxy host config struct (domain, forward host/port, SSL settings, etc.) +- [ ] Task 9: Handle JWT token expiry — re-authenticate automatically on 401 + +## Files to Modify/Create +- `internal/npm/client.go` — NPM API client, auth, HTTP helpers +- `internal/npm/types.go` — request/response types for proxy hosts + +## Acceptance Criteria +- Client authenticates and caches JWT +- CRUD operations work for proxy hosts +- Token refresh happens transparently on expiry +- Proxy host config supports: domain, forward host, forward port, SSL (Let's Encrypt optional) +- FindByDomain enables checking if a proxy already exists before creating + +## Notes +- NPM API base: typically `http://npm:81/api` +- Forward host for containers: use container name on the shared Docker network +- Forward port: the container's internal port (from EXPOSE) +- SSL: for staging, can be disabled; production may want Let's Encrypt +- NPM credentials come from settings (encrypted in SQLite, decrypted at runtime) + +## Review Checklist +- [ ] All tasks completed +- [ ] JWT caching and refresh work correctly +- [ ] HTTP errors are properly handled (not just status code, but response body) +- [ ] No credentials logged or leaked in errors +- [ ] Struct types match NPM API contract + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-5-registry-poller.md b/plans/docker-watcher-core/phase-5-registry-poller.md new file mode 100644 index 0000000..5b849b2 --- /dev/null +++ b/plans/docker-watcher-core/phase-5-registry-poller.md @@ -0,0 +1,49 @@ +# Phase 5: Registry Client & Poller + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Implement the registry client interface with Gitea implementation, and the periodic tag polling scheduler. + +## Tasks + +- [ ] Task 1: Define `Registry` interface — `ListTags(ctx, image)`, `GetLatestTag(ctx, image, pattern)` +- [ ] Task 2: Implement Gitea registry client — uses Gitea API to list container image tags +- [ ] Task 3: Implement tag pattern matching — match tags against glob patterns (e.g., `dev-*`, `v*`) +- [ ] Task 4: Implement tag comparison — detect new tags since last poll (store last-seen tag per project/stage) +- [ ] Task 5: Create poller service — periodic scheduler using `robfig/cron` +- [ ] Task 6: Poller logic — for each project/stage with polling enabled, check for new tags, trigger deploy if auto_deploy +- [ ] Task 7: Add `last_polled_tag` field to instances or a new `poll_state` table in store +- [ ] Task 8: Implement registry factory — create client based on registry type (gitea, future: github, dockerhub) + +## Files to Modify/Create +- `internal/registry/registry.go` — interface definition + factory +- `internal/registry/gitea.go` — Gitea registry implementation +- `internal/registry/poller.go` — polling scheduler service +- `internal/store/poll_state.go` — poll state persistence (optional, or extend existing tables) + +## Acceptance Criteria +- Gitea client can list tags for a given image +- Tag pattern matching correctly filters tags (glob-style) +- Poller runs on configurable interval +- New tags are detected by comparing against stored state +- Registry factory returns correct client based on type + +## Notes +- Gitea API: `GET /api/v1/packages/{owner}/container/{image}/tags` (or similar, verify against Gitea docs) +- Auth: Bearer token from registry config +- Polling interval comes from global settings +- The poller is a fallback — webhooks are the primary detection mechanism (Phase 6) +- GitHub Container Registry support is future work — just define the interface now + +## Review Checklist +- [ ] All tasks completed +- [ ] Interface is clean and minimal +- [ ] Pattern matching handles edge cases (empty pattern, no tags) +- [ ] Poller doesn't leak goroutines +- [ ] Registry auth tokens handled securely + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-6-webhook-handler.md b/plans/docker-watcher-core/phase-6-webhook-handler.md new file mode 100644 index 0000000..0ec36e0 --- /dev/null +++ b/plans/docker-watcher-core/phase-6-webhook-handler.md @@ -0,0 +1,48 @@ +# Phase 6: Webhook Handler + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Implement the secret UUID-based webhook endpoint that receives image push notifications from CI systems, with auto-creation of unknown projects. + +## Tasks + +- [ ] Task 1: Implement webhook HTTP handler — `POST /api/webhook/:secret-uuid` +- [ ] Task 2: Validate incoming payload — extract image name and tag +- [ ] Task 3: Look up project by image name in store — match against configured project images +- [ ] Task 4: If known project: match tag to stage via tag patterns, determine if auto_deploy +- [ ] Task 5: If unknown project: auto-create project with defaults from image inspection (EXPOSE port, labels) +- [ ] Task 6: Generate and store webhook secret UUID in settings (on first launch) +- [ ] Task 7: Implement webhook URL regeneration (new UUID, invalidates old one) +- [ ] Task 8: Define webhook payload struct (`{"image": "registry/org/app:tag"}`) + +## Files to Modify/Create +- `internal/webhook/handler.go` — webhook HTTP handler + payload parsing +- `internal/webhook/matcher.go` — project/stage matching logic +- `internal/webhook/autocreate.go` — auto-create project from unknown image + +## Acceptance Criteria +- Valid webhook URL with correct UUID triggers processing +- Invalid/missing UUID returns 404 (no information leak) +- Known images are matched to projects and stages +- Unknown images trigger auto-creation with sensible defaults +- Webhook URL can be regenerated + +## Notes +- Webhook URL format: `POST /api/webhook/d8f2a1e9-...` +- No authentication needed beyond the secret UUID +- Auto-created projects use: image EXPOSE port, "dev" as default stage, auto_deploy: true +- The webhook handler calls into the deployer (Phase 7) — for now, define the interface/callback +- Keep the handler thin — it matches and delegates + +## Review Checklist +- [ ] All tasks completed +- [ ] No information leak on invalid UUIDs +- [ ] Payload validation rejects malformed input +- [ ] Auto-creation uses safe defaults +- [ ] Handler is stateless (delegates to store/deployer) + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-7-deployer.md b/plans/docker-watcher-core/phase-7-deployer.md new file mode 100644 index 0000000..5882f2f --- /dev/null +++ b/plans/docker-watcher-core/phase-7-deployer.md @@ -0,0 +1,54 @@ +# Phase 7: Deployer & Health Checker + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Implement the core deployment orchestrator: pull → start container → configure NPM proxy → health check → success/rollback. Plus multi-instance support and notifications. + +## Tasks + +- [ ] Task 1: Define deployer service struct — depends on Docker client, NPM client, store, notifier +- [ ] Task 2: Implement deploy flow: pull image → create container → start → connect to network → configure proxy → health check +- [ ] Task 3: Implement subdomain generation per convention: `stage-{stage}-{project}` for default, `stage-{stage}-{project}-{tag}` for specific +- [ ] Task 4: Sanitize tags for DNS (dots → dashes, lowercase, truncate) +- [ ] Task 5: Implement health checker — HTTP GET to `http://container:{port}{healthcheck_path}` with retries and timeout +- [ ] Task 6: Implement rollback on health check failure — remove new container, delete NPM proxy host, update instance status +- [ ] Task 7: Implement multi-instance support — multiple tags of same project/stage can run simultaneously +- [ ] Task 8: Implement max_instances enforcement — remove oldest instance when limit reached +- [ ] Task 9: Implement notification webhook — POST to configured URL on deploy success/failure +- [ ] Task 10: Create deploy history records in store (status, timestamps, logs) +- [ ] Task 11: Implement deploy log streaming — append log entries during deploy for real-time visibility + +## Files to Modify/Create +- `internal/deployer/deployer.go` — main deploy orchestrator +- `internal/deployer/subdomain.go` — subdomain generation and DNS sanitization +- `internal/deployer/rollback.go` — rollback logic +- `internal/health/checker.go` — HTTP health checker with retries +- `internal/notify/notifier.go` — webhook notification sender + +## Acceptance Criteria +- Full deploy flow works end-to-end (pull → proxy → health check) +- Failed health checks trigger automatic rollback +- Multi-instance: deploying a new tag doesn't stop existing instances +- max_instances removes oldest instance when exceeded +- Notifications fire on success and failure +- Deploy history is recorded with status and timestamps + +## Notes +- Health check: 3 retries, 5s between retries, 10s timeout per attempt (configurable later) +- Subdomain pattern comes from global settings +- Notifications are fire-and-forget (don't block deploy on notification failure) +- Deploy logs should be structured entries (timestamp + message) for SSE streaming later +- The deployer is the central orchestrator — webhook handler and poller both call into it + +## Review Checklist +- [ ] All tasks completed +- [ ] Rollback cleans up ALL resources (container, proxy, instance record) +- [ ] No goroutine leaks +- [ ] Error handling at every step of the deploy flow +- [ ] Subdomain generation produces valid DNS names + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-8-api-layer.md b/plans/docker-watcher-core/phase-8-api-layer.md new file mode 100644 index 0000000..a37a0d9 --- /dev/null +++ b/plans/docker-watcher-core/phase-8-api-layer.md @@ -0,0 +1,59 @@ +# Phase 8: REST API Layer + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** backend + +## Objective +Wire up all REST API endpoints using chi router, connecting the store, deployer, and other services to HTTP handlers. + +## Tasks + +- [ ] Task 1: Set up chi router with middleware (logging, recovery, CORS, JSON content-type) +- [ ] Task 2: Implement project endpoints — GET/POST /api/projects, GET/PUT/DELETE /api/projects/:id +- [ ] Task 3: Implement stage endpoints — POST /api/projects/:id/stages, PUT/DELETE /api/projects/:id/stages/:stage +- [ ] Task 4: Implement instance endpoints — GET /api/projects/:id/stages/:stage/instances, POST (deploy), DELETE (remove) +- [ ] Task 5: Implement instance control endpoints — POST .../instances/:iid/stop, start, restart +- [ ] Task 6: Implement quick deploy endpoints — POST /api/deploy/inspect, POST /api/deploy/quick +- [ ] Task 7: Implement registry endpoints — GET/POST /api/registries, PUT/DELETE /api/registries/:id, POST .../test +- [ ] Task 8: Implement settings endpoints — GET/PUT /api/settings, GET /api/settings/webhook-url, POST .../regenerate +- [ ] Task 9: Implement deploy history endpoints — GET /api/deploys, GET /api/deploys/:id/logs (SSE stub) +- [ ] Task 10: Implement registry tags endpoint — GET /api/registries/:id/tags/:image +- [ ] Task 11: Wire webhook handler into router — POST /api/webhook/:secret-uuid +- [ ] Task 12: Wire everything in main.go — initialize all services, start HTTP server + +## Files to Modify/Create +- `internal/api/router.go` — chi router setup, middleware +- `internal/api/projects.go` — project CRUD handlers +- `internal/api/stages.go` — stage CRUD handlers +- `internal/api/instances.go` — instance lifecycle handlers +- `internal/api/deploys.go` — deploy + quick deploy handlers +- `internal/api/registries.go` — registry CRUD + test + tags handlers +- `internal/api/settings.go` — settings handlers +- `internal/api/middleware.go` — middleware (logging, CORS, recovery) +- `internal/api/response.go` — consistent API response helpers (envelope format) +- `cmd/server/main.go` — full service wiring and HTTP server start + +## Acceptance Criteria +- All endpoints from the API spec in PLAN.md are implemented +- Consistent JSON envelope response format (success, data, error, metadata) +- CORS configured for frontend dev (localhost origins) +- Proper HTTP status codes (200, 201, 400, 404, 500) +- main.go starts a fully wired HTTP server + +## Notes +- Response envelope: `{"success": bool, "data": any, "error": string|null, "meta": {pagination}}` +- CORS: allow all origins in dev, restrict in production (configurable later) +- SSE for deploy logs is a stub in this phase — real implementation in Phase 11 +- Quick deploy: /inspect pulls and inspects image, returns defaults; /quick creates project + deploys +- All handlers should validate input and return 400 for bad requests + +## Review Checklist +- [ ] All tasks completed +- [ ] All API endpoints from PLAN.md are covered +- [ ] Consistent response format across all endpoints +- [ ] Input validation on all POST/PUT handlers +- [ ] No business logic in handlers (delegates to services) + +## Handoff to Next Phase + diff --git a/plans/docker-watcher-core/phase-9-dashboard.md b/plans/docker-watcher-core/phase-9-dashboard.md new file mode 100644 index 0000000..c4df7fa --- /dev/null +++ b/plans/docker-watcher-core/phase-9-dashboard.md @@ -0,0 +1,64 @@ +# Phase 9: SvelteKit Dashboard & Project Views + +**Status:** ⬜ Not Started +**Parent plan:** [PLAN.md](./PLAN.md) +**Domain:** frontend + +## Objective +Build the SvelteKit frontend with the dashboard overview and project detail views — project list, instance status, controls, and deploy history. + +## Tasks + +- [ ] Task 1: Initialize SvelteKit project in `web/` directory with TypeScript, static adapter +- [ ] Task 2: Set up Tailwind CSS (or project's preferred styling) +- [ ] Task 3: Create shared API client (`lib/api.ts`) — typed fetch wrapper for all backend endpoints +- [ ] Task 4: Define TypeScript types (`lib/types.ts`) — Project, Stage, Instance, Deploy, Registry, Settings +- [ ] Task 5: Create layout with navigation — sidebar or top nav with Dashboard, Projects, Deploy, Settings links +- [ ] Task 6: Dashboard page (`routes/+page.svelte`) — project overview cards with instance counts, status indicators, latest activity +- [ ] Task 7: Projects list page (`routes/projects/+page.svelte`) — all projects with quick stats, "Add Project" button +- [ ] Task 8: Project detail page (`routes/projects/[id]/+page.svelte`) — stages, instances per stage, controls +- [ ] Task 9: Instance controls — Stop, Start, Restart, Remove buttons with confirmation dialogs +- [ ] Task 10: Deploy history section in project detail — recent deploys with status, timestamp, tag +- [ ] Task 11: "Deploy new version" dropdown — list available tags from registry, trigger deploy +- [ ] Task 12: Create reusable components: StatusBadge, InstanceCard, ProjectCard, ConfirmDialog + +## Files to Modify/Create +- `web/package.json` — SvelteKit project config +- `web/svelte.config.js` — SvelteKit config with static adapter +- `web/vite.config.ts` — Vite config with API proxy for dev +- `web/src/app.html` — base HTML +- `web/src/lib/api.ts` — API client +- `web/src/lib/types.ts` — shared TypeScript types +- `web/src/routes/+layout.svelte` — app layout with navigation +- `web/src/routes/+page.svelte` — dashboard +- `web/src/routes/projects/+page.svelte` — project list +- `web/src/routes/projects/[id]/+page.svelte` — project detail +- `web/src/lib/components/StatusBadge.svelte` — status indicator +- `web/src/lib/components/InstanceCard.svelte` — instance display +- `web/src/lib/components/ProjectCard.svelte` — project summary card +- `web/src/lib/components/ConfirmDialog.svelte` — confirmation modal + +## Acceptance Criteria +- SvelteKit project builds to static output +- Dashboard shows all projects with live status +- Project detail shows stages, instances, and controls +- Instance controls trigger correct API calls +- Deploy dropdown fetches and displays available tags +- UI is responsive and clean + +## Notes +- SvelteKit static adapter for embedding in Go binary +- API proxy in vite.config.ts for dev: proxy `/api` to `http://localhost:8080` +- Use SvelteKit's `fetch` for SSR-compatible data loading +- Status colors: green=running, yellow=starting, red=failed, gray=stopped +- Keep components small and reusable + +## Review Checklist +- [ ] All tasks completed +- [ ] TypeScript types match backend API response format +- [ ] API client handles errors gracefully with user feedback +- [ ] No hardcoded API URLs (use relative paths) +- [ ] Components are reusable and well-structured + +## Handoff to Next Phase +