Commit Graph

25 Commits

Author SHA1 Message Date
alexei.dolgolyov 0405ecd9ce feat(notify): HMAC-signed outgoing webhooks with per-tier secrets and test sender
Build / build (push) Successful in 10m36s
Outgoing notifications were bare POSTs with no auth and no way to verify
they came from Tinyforge. They also went out from one global URL only,
even though stages had a notification_url field, and static-site sync
emitted no events at all.

Schema: add notification_url + notification_secret (lazy-generated) to
settings, projects, stages and static_sites. Migrations are additive.

Notifier: SendSigned computes HMAC-SHA256 over the exact body bytes and
sends X-Hub-Signature-256 (GitHub-compatible — receivers built for
GitHub/Gitea/Forgejo verify out of the box). Aux headers
X-Tinyforge-Event/Delivery/Timestamp/Tier are advisory and not signed.
Empty secret => unsigned send for back-compat.

Resolution: deploys fall through stage > project > settings, sites fall
through site > settings. The secret travels with the URL that sourced
it, so any tier can sign even when its parents are unsigned. Site sync
events now actually emit (site_sync_success / site_sync_failure).

API: 12 new endpoints — {GET secret, POST regenerate, POST disable,
POST test} for each of the 4 tiers. SendSyncForTest returns
status_code/latency_ms/signature_sent/delivery_id/response_snippet so
the UI surfaces receiver feedback inline.

UI: shared OutgoingWebhookPanel.svelte fits the existing card aesthetic.
Signing-state pill, secret reveal-on-demand, regenerate/disable behind
ConfirmDialog modals (not inline strips — too easy to misclick), send-
test result card with colour-coded status. Wired into Settings →
Integrations, project edit form, per-stage edit, and per-site detail.
EN + RU i18n.

Tests: round-trip (sender signs, receiver verifies), tampered-body and
wrong-secret rejection, unsigned-send omits header, send-test surfaces
4xx, concurrent fan-out via Drain. Resolver precedence locked for both
deploy and site paths.

Docs: docs/webhooks.md with header reference, verifier snippets in
Node/Python/Go, and a recipe for the service-to-notification-bridge
generic webhook provider.
2026-05-07 02:03:32 +03:00
alexei.dolgolyov 05440a5f92 feat(stats): resource metrics dashboard + sites logs/stats
Build / build (push) Successful in 10m50s
Background collector samples CPU/memory/network/block I/O for every
instance and site on a configurable interval (default 15s, range
5-300s), persists samples to SQLite with a configurable retention
window (default 2h, range 0-24h), and skips ticks gracefully when
the Docker daemon is unreachable. Settings are reloadable without
a restart — each tick re-reads them.

New API endpoints:
- GET /api/system/stats (host snapshot: info + df)
- GET /api/system/stats/history
- GET /api/system/stats/top?by=cpu|memory
- GET /api/projects/{id}/stages/{s}/instances/{iid}/stats/history
- GET /api/sites/{id}/stats[/history]
- GET /api/sites/{id}/logs (SSE + JSON, reuses instance log streamer)

Frontend:
- ECharts added with tree-shaken imports (~180KB gzip) for
  future-proof time-series/gantt/graph visualizations
- CollapsibleSection wraps all dashboard sections (system health,
  daemons, system resources, static sites, projects) with
  localStorage-persisted open state
- SystemResourcesCard shows capacity tiles, workload utilization
  chart with 30m/2h/6h/24h window picker, disk breakdown with
  reclaimable callouts, and top 5 consumers
- ContainerStats and ContainerLogs take a source discriminated union
  so sites reuse the same components as instances; sites detail page
  embeds both for Deno backend debugging
- Settings › Maintenance exposes collection interval + retention
- Docker-unavailable state returns 503 and renders an amber banner
  instead of a generic 500

Full i18n coverage (en + ru) for all new strings.
2026-04-24 15:02:43 +03:00
alexei.dolgolyov 0632f512e6 feat(webhook): per-project and per-site webhook URLs
Build / build (push) Successful in 10m25s
Replace the single global webhook secret with entity-scoped secrets stored
on each project and static site. Webhook-driven project autocreate is
removed — projects must exist before their URL can trigger deploys.

Also wires static-site webhooks (sync_trigger=push|tag), turning the
previously inert "push" trigger into a functional one: POST the site's
webhook URL from a Git provider and Tinyforge re-syncs on matching refs.

- Adds webhook_secret columns + unique indexes to projects and static_sites
- Per-entity GET/regenerate endpoints under /api/projects/{id}/webhook
  and /api/sites/{id}/webhook (admin-only)
- Removes /api/settings/webhook-url and the global webhook panel
- Reusable WebhookPanel Svelte component on both detail pages, i18n in en/ru
- Tests for matcher (siteRefMatches, ParseImageRef) and handler (project
  match/mismatch/404 and site push/manual/branch-skip)
2026-04-23 15:18:19 +03:00
alexei.dolgolyov 75424a5f25 feat: docker-compose stacks with Forge-themed UI
Build / build (push) Successful in 10m42s
Adds a new Stacks feature: upload/edit docker-compose YAML,
deploy as atomic units, browse revisions, roll back, and
stream logs. Backend in internal/stack + internal/api/stacks.go,
persistent storage in internal/store/stacks.go.

Stacks pages (list, new, detail) use a modern Forge aesthetic —
Instrument Serif display type, JetBrains Mono for meta/code,
indigo ember accents, dot-grid hero, registration marks on
hover, terminal panel for logs. Palette is sourced from the
app's existing design tokens so the feature remains consistent
with the rest of Tinyforge.

Fonts self-hosted via @fontsource/instrument-serif and
@fontsource/jetbrains-mono to satisfy the strict CSP.
2026-04-16 03:48:37 +03:00
alexei.dolgolyov b622384774 feat: persistent storage for Deno static sites
Build / build (push) Successful in 10m21s
- Add storage_enabled and storage_limit_mb columns to static_sites.
- Create/attach Docker volumes (tinyforge-site-{name}-data) for Deno
  sites with storage enabled, mounted at /app/data.
- Grant --allow-write=/app/data in Deno container CMD.
- Add storage usage API endpoint (GET /api/sites/{id}/storage).
- Show storage section in site detail page with usage bar.
- Add storage toggle and limit field to new site wizard.
- Use ConfirmDialog for secret deletion instead of inline delete.
2026-04-13 00:12:51 +03:00
alexei.dolgolyov 8d2c5a063b feat: static sites feature with Gitea/GitHub/GitLab support and Deno backend
Deploy static content from Git repository folders with optional server-side
API endpoints. Supports Gitea/Forgejo/Gogs, GitHub, and GitLab with provider
autodetection.

- New Sites entity with CRUD, encrypted secrets, and manual/push/tag sync triggers
- Pluggable GitProvider interface with three implementations
- Deno container mode: auto-generates router from API_{method}_{name} exports
- Static container mode: nginx serving files with optional markdown rendering
- Wizard UI with provider selector, repo picker, branch/folder tree pickers
- Deploy pipeline builds fresh image, starts container, configures NPM proxy
- Stop/Start buttons, force redeploy on manual trigger
- Periodic health checker detects crashed containers
- Proxy route existence check during auto-sync
2026-04-11 03:35:57 +03:00
alexei.dolgolyov b0816502bf feat: configurable unused images threshold with dashboard warning
- Add image_prune_threshold_mb setting (default 1024 MB)
- Add GET /api/docker/unused-images endpoint returning unused image count, size, and threshold status
- Dashboard shows amber warning banner when unused project images exceed threshold
- Banner links to settings page for pruning, shows count and human-readable size (MB/GB)
- Threshold configurable in Docker Image Cleanup section of settings
- DB migration + schema for image_prune_threshold_mb
2026-04-05 14:34:48 +03:00
alexei.dolgolyov 21ffef2ee2 feat: separate Public IP for DNS records from Server IP, improve settings help texts
- Add public_ip field to Settings for DNS A records (proxy/load balancer IP)
- DNS records now use public_ip, falling back to server_ip if empty
- Server IP renamed to "Server IP (Docker Host)" for clarity
- Public IP labeled "Public IP (DNS Target)"
- Updated help texts for domain, server IP, public IP, and Docker network
- DB migration + schema for public_ip column
2026-04-05 14:12:53 +03:00
alexei.dolgolyov 7550fe9e32 feat: CPU/RAM limits per stage, NPM access list (global + per-project)
Resource limits:
- Add cpu_limit (cores) and memory_limit (MB) fields to Stage model
- Pass limits to Docker container via NanoCPUs and Memory in HostConfig
- Add CPU/Memory fields to stage creation form in project detail
- 0 = unlimited (default)

NPM access list:
- Add npm_access_list_id to Settings (global default) and Project (per-project override)
- Per-project overrides global when > 0
- NPM provider passes access_list_id when configuring proxy hosts
- Add GET /api/settings/npm-access-lists endpoint to list NPM access lists
- Add access list picker on NPM settings page (global)
- Add access list ID field on project edit form (per-project)
- DB migrations for all new columns
2026-04-05 12:44:26 +03:00
alexei.dolgolyov 195ef3e7e5 feat: NPM remote mode for cross-machine deployments
- Add npm_remote setting: when enabled, proxy forwards to server_ip with
  published host ports instead of Docker container names
- Deployer looks up assigned host port via InspectContainerPort in remote mode
- Auto-remove stale containers with same name before creating new ones
- Add Remote NPM toggle with warning on NPM settings page
- DB migration + schema for npm_remote column
2026-04-05 02:18:06 +03:00
alexei.dolgolyov 308547a3d7 refactor: remove standalone proxies, add Traefik provider with Docker labels
Standalone proxy removal:
- Delete store, API handlers, proxy manager, health monitor, validator, hints
- Delete frontend pages (proxies list, create, edit) and components (ProxyCard, ProxyForm, ProxyFilter, ProxyGroup, ValidationChecklist)
- Remove proxy routes from router, nav items, dashboard references
- Clean up SystemHealthCard to remove proxy section

Traefik provider:
- Add TraefikProvider implementing proxy.Provider via Docker labels
- ContainerLabels() returns traefik.enable, router rule, entrypoints, service port, TLS cert resolver, docker network
- ConfigureRoute() returns router name (labels handle routing at container creation)
- DeleteRoute() is no-op (container removal auto-deregisters)
- Ping() checks Traefik API health (optional)
- Wire ContainerLabels into deployer (executeDeploy + blueGreenDeploy)
- Add Traefik settings: entrypoint, cert_resolver, network, api_url
- Add traefik option to proxy provider selector in settings UI
- Show conditional Traefik config fields
- Add i18n keys (EN + RU)
2026-04-04 22:54:31 +03:00
alexei.dolgolyov 7d6719da12 refactor: extract ProxyProvider interface with None and NPM implementations
Replace direct npm.Client usage throughout the codebase with the
proxy.Provider interface, enabling pluggable proxy backends. The
deployer, API layer, and proxy manager now use provider-agnostic
route management (ConfigureRoute/DeleteRoute) instead of NPM-specific
API calls. Adds ProxyRouteID (string) to Instance model and
ProxyProvider setting to Settings, with SQLite migrations for
backward compatibility.
2026-04-04 19:39:08 +03:00
alexei.dolgolyov 91b49cb5ed feat: expanded health checks, deploy filtering, per-project notifications, error sanitization, and audit trail
- Expand health endpoint to check DB, Docker, and NPM connectivity (FUNC-M4)
- Add project_id, stage_id, offset query params to deploys endpoint (FUNC-M5, FUNC-M6)
- Add notification_url field to Stage model for per-project overrides (FUNC-M2)
- Add NPM Ping method for health checking
- Sanitize all internal error messages in API handlers (SEC-M4)
- Add audit trail events for admin actions (FUNC-M3)
- Add EventLog event type to event bus
2026-04-04 13:10:10 +03:00
alexei.dolgolyov a9c7775bb7 feat: configuration backup management with manual and auto backup
Add backup/restore functionality for the SQLite database. Users can
trigger manual backups, configure automatic backups on an interval
with retention policies, list/download/delete backups, and restore
from any backup.

- Backup engine using VACUUM INTO (safe with WAL mode)
- Backup metadata tracked in DB, files stored in DATA_DIR/backups/
- Settings: backup_enabled, backup_interval_hours, backup_retention_count
- API: POST/GET/DELETE /api/backups, download, restore endpoints
- Autobackup via cron scheduler with configurable interval
- Retention: prune on startup, after each backup (manual and auto)
- Orphan cleanup: removes backup files without metadata on startup
- Restore: replaces DB and triggers graceful server shutdown
- Settings UI: /settings/backup with toggle, interval, retention config
- Backup list with download, delete, restore actions
- i18n: English and Russian translations
2026-04-02 15:32:15 +03:00
alexei.dolgolyov c730cfaa45 feat: Cloudflare DNS management with automatic record sync
Add flexible DNS management to Docker Watcher. By default, wildcard DNS
is assumed (current behavior). When disabled, users can configure a
Cloudflare DNS provider with API token and zone selection. DNS A records
are automatically created/updated/deleted in sync with proxy consumers
(deployed instances and standalone proxies).

- Settings: wildcard_dns toggle, dns_provider, cloudflare credentials
- Cloudflare client: Provider interface with EnsureRecord/DeleteRecord/ListRecords
- DNS lifecycle hooks in deployer and proxy manager (best-effort)
- Settings UI: DNS config section with provider picker, zone selector, test button
- DNS Records page at /dns with filtering, sync status, reconciliation
- Records visible in both wildcard and managed modes
- Cleanup on provider change: removes old records when switching modes
2026-04-02 14:49:21 +03:00
alexei.dolgolyov 582e7e39e3 feat(volume-browser): absolute scope with allowlist security
- Add 'absolute' volume scope for direct host paths (NFS, external mounts)
- Allowlist in settings: allowed_volume_paths (JSON array of prefixes)
- Validation: absolute source must be under an allowed prefix
- Empty allowlist = absolute scope disabled entirely
- Settings API exposes/validates allowed_volume_paths
- Frontend type updated with absolute scope
2026-04-01 23:31:27 +03:00
alexei.dolgolyov 8fb959f81f feat: volume scopes redesign — replace shared/isolated with 6 scopes
Replace confusing shared/isolated volume modes with explicit scopes:
- instance: per-deploy isolated directory
- stage: shared within a stage across deploys
- project: shared across all stages
- project_named: named group within a project
- named: global named volume across projects
- ephemeral: tmpfs in-memory mount

Includes schema migration (shared→project, isolated→instance),
backward-compatible deployer resolution, scope metadata API endpoint,
and redesigned volume editor UI with scope guide cards and hints.
2026-03-31 23:22:43 +03:00
alexei.dolgolyov aefecdffdf feat(observability): phase 2 - stale container detection
Add periodic scanner for stale containers:
- Cron-based scanner (hourly) detects non-running containers exceeding threshold
- last_alive_at tracking on instances, updated on deploy/start/restart
- API: GET /api/containers/stale, POST cleanup (single + bulk)
- Event log warnings emitted for newly stale containers
- Graceful handling of externally removed containers
2026-03-30 11:12:25 +03:00
alexei.dolgolyov c38b7d4c78 feat(observability): phase 1 - schema, models & event log backend
Add database foundation for observability features:
- event_log table with severity/source filtering and pagination
- standalone_proxies table for user-created reverse proxies
- stale_threshold_days setting (default 7 days)
- Auto-persist warn/error events from event bus to database
- SSE broadcast of persistent events for real-time UI updates
- Frontend types and API functions for downstream UI phases
2026-03-30 10:59:13 +03:00
alexei.dolgolyov 9f284932a1 feat: SSL wildcard certificate picker from NPM
- NPM client: ListCertificates endpoint
- API: GET /api/settings/npm-certificates (wildcard-only filter)
- Settings UI: EntityPicker for selecting wildcard certs
- Deployer: applies certificate_id + ssl_forced to proxy hosts
- Uses HTTPS subdomain URLs when SSL cert is configured
2026-03-29 13:07:58 +03:00
alexei.dolgolyov e94c4f9116 feat: optional NPM proxy per stage
Add enable_proxy boolean to stages (default true). When disabled,
the deployer skips NPM proxy host creation — useful for internal
services, workers, or externally-routed containers. UI shows
toggle in Add Stage form and "No Proxy" badge on stage header.
2026-03-29 12:58:13 +03:00
alexei.dolgolyov 1cfd23c431 feat: base volume path setting
Add global base_volume_path to settings. Relative volume source
paths are automatically prepended with the base path at deploy
time. Absolute paths are used as-is. Configurable in Settings >
General.
2026-03-28 15:21:37 +03:00
alexei.dolgolyov 77251c540b fix: align webhook regenerate route with frontend path 2026-03-28 13:58:53 +03:00
alexei.dolgolyov d4659146fc feat(docker-watcher): phase 13 - volumes & environment
Per-stage env var overrides with encryption for secrets.
Volume mounts with shared/isolated modes (isolated appends
/{stage}-{tag}/ to source path). Store CRUD, API endpoints,
and frontend editors for both. Env merge during deploy.
2026-03-27 23:28:59 +03:00
alexei.dolgolyov d63c831d15 feat(docker-watcher): phase 1 - project scaffold & SQLite store
Initialize Go module, directory structure, and full SQLite store layer:
- 7-table schema (projects, stages, registries, settings, instances,
  deploys, deploy_logs) with auto-migration
- CRUD operations for all entities with proper error handling
- ErrNotFound sentinel for distinguishing 404 from 500 in handlers
- WAL mode, foreign keys, busy timeout pragmas
2026-03-27 20:52:29 +03:00