ea55d31177
Build / build (push) Successful in 10m43s
Two-stage feature arc closing the gaps left by the hard legacy cutover.
The static-site creation wizard regains its auto-discovery + connection-test
flow; /apps/[id] grows the runtime/storage/lifecycle surface the legacy
/sites/[id] page used to expose.
Backend (Go)
- internal/api/discovery.go: six admin-gated endpoints wrapping
staticsite.GitProvider — POST /api/discovery/git/{detect-provider,
test-connection,repos,branches,tree} + GET /api/discovery/image/conflicts.
Identifier validation (validateGitIdent / validateGitBranch) at the
boundary so provider URL interpolation cannot be hijacked via `..`.
Upstream errors scrubbed: detailed slog on the server, generic 502 to
the client (mitigates token-reflection-in-error-page).
- internal/api/workload_runtime.go: four endpoints —
GET /api/workloads/{id}/runtime-state decodes containers.extra_json for
static workloads; GET /api/workloads/{id}/storage execs `du -sb /app/data`
with a 30s in-process cache (storageProbeCache) so polling can't turn
into per-request execs; POST /api/workloads/{id}/{stop,start} iterate
ListContainersByWorkload and call docker.StopContainer / StartContainer,
returning 200 / 409 (nothing to act on) / 502 (all failed).
- internal/staticsite/safehttp.go: NewSafeHTTPClient + ValidateBaseURL +
blockReason. DialContext re-resolves hostnames and refuses loopback /
link-local / multicast / unspecified addresses. RFC1918 + ULA explicitly
allowed (self-hosted Gitea on LAN is the dominant deployment).
Replaced four raw &http.Client{} constructions in the provider files.
- internal/staticsite/gitlab_provider.go: url.PathEscape each segment in
the raw-file URL builder for parity with projectPath().
- Test coverage: 26 cases in discovery_test.go (image-tag stripping,
source-config decoding, conflict scenarios, validator boundaries,
scheme rejection), 14 in workload_runtime_test.go (404 / 409 / nil-docker
/ probe-cache), 16 in safehttp_test.go (URL validation + block-reason
policy matrix + live dial against loopback + AWS metadata literals).
Frontend (Svelte 5 + runes)
- web/src/lib/api.ts: typed wrappers for every endpoint, AbortSignal
threaded through post(); ApiError exported so callers can narrow on
e.status; new DetectedGitProvider narrow union.
- web/src/routes/apps/new/+page.svelte: static-form discovery controls
(auto-detect provider, test connection, repo / branch / folder
EntityPickers, Deno auto-detect); image-form conflict panel with
debounced lookup + double-click submit guard ("Forge anyway") + Inspect
button that pre-fills port/healthcheck; English error fallbacks routed
through apps.new.errors.* (en + ru).
- web/src/routes/apps/[id]/+page.svelte: runtime-state panel + storage
panel + Stop / Start / Open-site toolbar; universal live-state badge
in the hero lede for image/compose/static (RUNNING / TRANSITIONING /
STOPPED / NOT DEPLOYED / MIXED · n/m RUNNING); ContainerStats panel
per row (auto-collapsing native <details> when N > 2); read-only
webhook bindings summary card; responsive toolbar overflow with native
<details> at <640px (z-index 100 above sticky nav).
- web/src/app.css: project-wide .forge-btn-ghost:focus-visible outline.
Hardening from go-reviewer + security-reviewer + typescript-reviewer +
frontend-design UI/UX subagents (0 CRITICAL, all HIGH/BLOCKER addressed
inline, IMPORTANT applied before commit):
- AbortController + per-call sequence tokens on every long-running
fetch (loadRuntimeState / loadStorage / loadTriggerMeta / inspectImage /
listImageConflicts) plus onDestroy cleanup so late resolves cannot
mutate dead component state.
- doStop / doStart snapshot and restore `error` across the finally-block
reload so a load()-cleared message doesn't hide a real failure.
- triggersById refreshed after inline trigger creation so the webhook
card doesn't silently exclude the just-created trigger.
- Live-state badge wraps in role=status / aria-live=polite (no redundant
aria-label).
- Webhook row has a single click target (was two pointing at the same URL).
- Empty webhook section hides entirely.
- Dropped role=menu / role=menuitem from the overflow menu (they would
promise arrow-key nav we don't wire; native Tab + ESC carry it).
Doc
- docs/CODEMAPS/INDEX.md + new docs/CODEMAPS/discovery-and-runtime.md
map the endpoint surface, security posture, frontend integration
patterns, and an "add a new probe" recipe.
Verification
- svelte-check: 0 errors, 3 pre-existing a11y warnings.
- go build + go vet + go test ./...: all green.
- i18n parity: en + ru at 1413 keys each.
- Live smoke against :8090: 404 / 409 / 502 envelopes correct, discovery
sanity passes, ProbeError surfaces on no-container path.
404 lines
13 KiB
Go
404 lines
13 KiB
Go
package api
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"log/slog"
|
|
"net/http"
|
|
"regexp"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/alexei/tinyforge/internal/staticsite"
|
|
)
|
|
|
|
// Discovery endpoints feed the /apps/new wizard's auto-discovery and
|
|
// connection-test flow. They wrap staticsite.GitProvider so the form
|
|
// can validate a repo + token before the workload is created, browse
|
|
// repos/branches/folders without leaving the page, and warn the operator
|
|
// when an image is already in use by another workload.
|
|
//
|
|
// The endpoints are workload-agnostic on purpose — they are scoped under
|
|
// /api/discovery rather than tied to the static_sites table the cutover
|
|
// dropped. Any future Git-driven source plugin can reuse them.
|
|
|
|
// Per-request budget for outbound calls. Short enough that a malicious
|
|
// or stuck upstream cannot pin a worker for long; long enough for slow
|
|
// self-hosted Gitea instances to respond.
|
|
const discoveryTimeout = 15 * time.Second
|
|
|
|
// gitProviderRequest is the shared request body for the four Git
|
|
// discovery endpoints. Token is plaintext over HTTPS — the wizard has
|
|
// not yet persisted it, so there is nothing to decrypt server-side.
|
|
// Empty Provider triggers DetectProviderWithProbe.
|
|
type gitProviderRequest struct {
|
|
Provider string `json:"provider"`
|
|
BaseURL string `json:"base_url"`
|
|
AccessToken string `json:"access_token"`
|
|
RepoOwner string `json:"repo_owner"`
|
|
RepoName string `json:"repo_name"`
|
|
Branch string `json:"branch"`
|
|
Query string `json:"query"`
|
|
}
|
|
|
|
// gitIdentRe accepts Git owner / repo identifiers as the major hosts
|
|
// (GitHub, GitLab, Gitea/Forgejo) accept them: alphanumeric plus dot,
|
|
// underscore, hyphen. Rejecting other characters at the API boundary
|
|
// prevents `..` traversal and URL injection in the provider code that
|
|
// interpolates these segments into request paths.
|
|
var gitIdentRe = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]*$`)
|
|
|
|
// gitBranchRe is more permissive than gitIdentRe: branches may contain
|
|
// `/` (e.g. `feature/foo`) but still cannot contain `..` or control
|
|
// characters. The check below pairs this regex with an explicit `..`
|
|
// reject so a `feature/../admin` value cannot slip through.
|
|
var gitBranchRe = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._/-]*$`)
|
|
|
|
// validateGitIdent guards owner / repo path segments at the boundary
|
|
// so the provider code can interpolate them with fmt.Sprintf without
|
|
// risking traversal. Empty input is reported with the supplied field
|
|
// name so the error message is actionable.
|
|
func validateGitIdent(field, value string) error {
|
|
v := strings.TrimSpace(value)
|
|
if v == "" {
|
|
return &apiError{msg: field + " is required"}
|
|
}
|
|
if !gitIdentRe.MatchString(v) {
|
|
return &apiError{msg: field + " contains invalid characters"}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// validateGitBranch is the branch-shaped variant of validateGitIdent.
|
|
// Branches legitimately contain `/`; the extra `..` reject covers the
|
|
// one traversal vector the regex still admits.
|
|
func validateGitBranch(value string) error {
|
|
v := strings.TrimSpace(value)
|
|
if v == "" {
|
|
return &apiError{msg: "branch is required"}
|
|
}
|
|
if strings.Contains(v, "..") {
|
|
return &apiError{msg: "branch contains invalid sequence '..'"}
|
|
}
|
|
if !gitBranchRe.MatchString(v) {
|
|
return &apiError{msg: "branch contains invalid characters"}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// apiError is a small typed error so handlers can distinguish a
|
|
// validation failure (→ 400) from any other error (→ 500/502). The
|
|
// type lives in this file because nothing outside discovery uses it
|
|
// yet — promote to response.go if other handlers need the same shape.
|
|
type apiError struct{ msg string }
|
|
|
|
func (e *apiError) Error() string { return e.msg }
|
|
|
|
// providerType normalizes the provider string into the typed enum used
|
|
// by staticsite.NewGitProvider. Empty input falls through to provider
|
|
// auto-detection inside NewGitProvider.
|
|
func (req gitProviderRequest) providerType() staticsite.ProviderType {
|
|
switch strings.ToLower(strings.TrimSpace(req.Provider)) {
|
|
case "github":
|
|
return staticsite.ProviderGitHub
|
|
case "gitlab":
|
|
return staticsite.ProviderGitLab
|
|
case "gitea":
|
|
return staticsite.ProviderGitea
|
|
default:
|
|
return ""
|
|
}
|
|
}
|
|
|
|
// newProvider constructs the GitProvider for the request, or writes a
|
|
// 400 to w and returns nil if the inputs are invalid. BaseURL is fully
|
|
// validated here (scheme + host shape); connect-time IP filtering is
|
|
// enforced inside the safe-HTTP transport the provider receives.
|
|
func (req gitProviderRequest) newProvider(w http.ResponseWriter) staticsite.GitProvider {
|
|
if err := staticsite.ValidateBaseURL(req.BaseURL); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return nil
|
|
}
|
|
provider, err := staticsite.NewGitProvider(req.providerType(), req.BaseURL, req.AccessToken)
|
|
if err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return nil
|
|
}
|
|
return provider
|
|
}
|
|
|
|
// upstreamError logs the detailed upstream failure server-side and
|
|
// writes a generic 502 to the client. Echoing the raw error string
|
|
// would leak any access token reflected by a misconfigured or
|
|
// attacker-controlled upstream into the response body.
|
|
func upstreamError(w http.ResponseWriter, op string, err error) {
|
|
slog.Warn("discovery upstream call failed", "op", op, "error", err)
|
|
respondError(w, http.StatusBadGateway, "upstream git provider returned an error")
|
|
}
|
|
|
|
// detectGitProviderRequest is the body for POST /api/discovery/git/detect-provider.
|
|
type detectGitProviderRequest struct {
|
|
BaseURL string `json:"base_url"`
|
|
}
|
|
|
|
// detectGitProvider probes the base URL for known Git provider API
|
|
// signatures so the wizard can auto-fill the provider dropdown.
|
|
// POST /api/discovery/git/detect-provider.
|
|
func (s *Server) detectGitProvider(w http.ResponseWriter, r *http.Request) {
|
|
var req detectGitProviderRequest
|
|
if !decodeJSON(w, r, &req) {
|
|
return
|
|
}
|
|
if err := staticsite.ValidateBaseURL(req.BaseURL); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
|
|
ctx, cancel := context.WithTimeout(r.Context(), discoveryTimeout)
|
|
defer cancel()
|
|
|
|
provider := staticsite.DetectProviderWithProbe(ctx, req.BaseURL)
|
|
respondJSON(w, http.StatusOK, map[string]string{"provider": string(provider)})
|
|
}
|
|
|
|
// testGitConnection verifies the configured base URL + token + repo
|
|
// reach the provider successfully so the wizard can fail fast.
|
|
// POST /api/discovery/git/test-connection.
|
|
func (s *Server) testGitConnection(w http.ResponseWriter, r *http.Request) {
|
|
var req gitProviderRequest
|
|
if !decodeJSON(w, r, &req) {
|
|
return
|
|
}
|
|
if err := validateGitIdent("repo_owner", req.RepoOwner); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
if err := validateGitIdent("repo_name", req.RepoName); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
provider := req.newProvider(w)
|
|
if provider == nil {
|
|
return
|
|
}
|
|
|
|
ctx, cancel := context.WithTimeout(r.Context(), discoveryTimeout)
|
|
defer cancel()
|
|
|
|
if err := provider.TestConnection(ctx, req.RepoOwner, req.RepoName); err != nil {
|
|
upstreamError(w, "test_connection", err)
|
|
return
|
|
}
|
|
respondJSON(w, http.StatusOK, map[string]string{"status": "ok"})
|
|
}
|
|
|
|
// listGitRepos returns repositories accessible with the supplied token,
|
|
// optionally filtered by a name query.
|
|
// POST /api/discovery/git/repos.
|
|
func (s *Server) listGitRepos(w http.ResponseWriter, r *http.Request) {
|
|
var req gitProviderRequest
|
|
if !decodeJSON(w, r, &req) {
|
|
return
|
|
}
|
|
provider := req.newProvider(w)
|
|
if provider == nil {
|
|
return
|
|
}
|
|
|
|
ctx, cancel := context.WithTimeout(r.Context(), discoveryTimeout)
|
|
defer cancel()
|
|
|
|
repos, err := provider.ListRepos(ctx, req.Query)
|
|
if err != nil {
|
|
upstreamError(w, "list_repos", err)
|
|
return
|
|
}
|
|
if repos == nil {
|
|
repos = []staticsite.RepoInfo{}
|
|
}
|
|
respondJSON(w, http.StatusOK, repos)
|
|
}
|
|
|
|
// listGitBranches returns the branch list for a repo.
|
|
// POST /api/discovery/git/branches.
|
|
func (s *Server) listGitBranches(w http.ResponseWriter, r *http.Request) {
|
|
var req gitProviderRequest
|
|
if !decodeJSON(w, r, &req) {
|
|
return
|
|
}
|
|
if err := validateGitIdent("repo_owner", req.RepoOwner); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
if err := validateGitIdent("repo_name", req.RepoName); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
provider := req.newProvider(w)
|
|
if provider == nil {
|
|
return
|
|
}
|
|
|
|
ctx, cancel := context.WithTimeout(r.Context(), discoveryTimeout)
|
|
defer cancel()
|
|
|
|
branches, err := provider.ListBranches(ctx, req.RepoOwner, req.RepoName)
|
|
if err != nil {
|
|
upstreamError(w, "list_branches", err)
|
|
return
|
|
}
|
|
if branches == nil {
|
|
branches = []string{}
|
|
}
|
|
respondJSON(w, http.StatusOK, branches)
|
|
}
|
|
|
|
// listGitTree returns the full directory tree for a branch so the
|
|
// wizard can render the folder picker.
|
|
// POST /api/discovery/git/tree.
|
|
func (s *Server) listGitTree(w http.ResponseWriter, r *http.Request) {
|
|
var req gitProviderRequest
|
|
if !decodeJSON(w, r, &req) {
|
|
return
|
|
}
|
|
if err := validateGitIdent("repo_owner", req.RepoOwner); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
if err := validateGitIdent("repo_name", req.RepoName); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
if err := validateGitBranch(req.Branch); err != nil {
|
|
respondError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
provider := req.newProvider(w)
|
|
if provider == nil {
|
|
return
|
|
}
|
|
|
|
ctx, cancel := context.WithTimeout(r.Context(), discoveryTimeout)
|
|
defer cancel()
|
|
|
|
tree, err := provider.ListTree(ctx, req.RepoOwner, req.RepoName, req.Branch)
|
|
if err != nil {
|
|
upstreamError(w, "list_tree", err)
|
|
return
|
|
}
|
|
if tree == nil {
|
|
tree = []staticsite.FolderEntry{}
|
|
}
|
|
respondJSON(w, http.StatusOK, tree)
|
|
}
|
|
|
|
// imageConflict is a slim projection of Workload, scoped to what the
|
|
// /apps/new conflict dialog needs to render.
|
|
type imageConflict struct {
|
|
ID string `json:"id"`
|
|
Name string `json:"name"`
|
|
Image string `json:"image"`
|
|
AppID string `json:"app_id,omitempty"`
|
|
}
|
|
|
|
// listImageConflicts finds existing image-source workloads whose
|
|
// configured image matches the supplied ref, with or without tag.
|
|
// GET /api/discovery/image/conflicts?image=<ref>.
|
|
//
|
|
// Matching mirrors the legacy quickDeploy behavior: collide on
|
|
// repository-without-tag so nginx:1.25 surfaces nginx, nginx:latest,
|
|
// and nginx:1.26 as conflicts. This is intentionally permissive — the
|
|
// wizard surfaces matches but lets the operator decide.
|
|
func (s *Server) listImageConflicts(w http.ResponseWriter, r *http.Request) {
|
|
image := strings.TrimSpace(r.URL.Query().Get("image"))
|
|
if image == "" {
|
|
respondError(w, http.StatusBadRequest, "image query parameter is required")
|
|
return
|
|
}
|
|
target := stripImageTag(image)
|
|
if target == "" {
|
|
respondError(w, http.StatusBadRequest, "image is empty after tag strip")
|
|
return
|
|
}
|
|
|
|
workloads, err := s.store.ListWorkloads("")
|
|
if err != nil {
|
|
slog.Error("list workloads for conflict check", "error", err)
|
|
respondError(w, http.StatusInternalServerError, "internal server error")
|
|
return
|
|
}
|
|
|
|
conflicts := []imageConflict{}
|
|
for _, wl := range workloads {
|
|
if wl.SourceKind != "image" {
|
|
continue
|
|
}
|
|
ref := imageRefFromSourceConfig(wl.SourceConfig)
|
|
if ref == "" {
|
|
continue
|
|
}
|
|
if stripImageTag(ref) != target {
|
|
continue
|
|
}
|
|
conflicts = append(conflicts, imageConflict{
|
|
ID: wl.ID,
|
|
Name: wl.Name,
|
|
Image: ref,
|
|
AppID: wl.AppID,
|
|
})
|
|
}
|
|
respondJSON(w, http.StatusOK, conflicts)
|
|
}
|
|
|
|
// stripImageTag returns the image reference with the trailing :tag
|
|
// removed, taking care to leave a registry port (e.g. registry:5000/foo)
|
|
// intact. Digest references (image@sha256:...) are returned unchanged.
|
|
func stripImageTag(ref string) string {
|
|
ref = strings.TrimSpace(ref)
|
|
if ref == "" {
|
|
return ""
|
|
}
|
|
// Digest reference: keep as-is so two pinned-by-digest workloads do
|
|
// not collide with each other or with tag-based refs unless the
|
|
// caller asks for exact-match (we currently don't).
|
|
if at := strings.Index(ref, "@"); at >= 0 {
|
|
return ref[:at]
|
|
}
|
|
// Strip a :tag suffix only when the colon is in the final path
|
|
// segment — earlier colons belong to a registry port.
|
|
lastSlash := strings.LastIndex(ref, "/")
|
|
tail := ref
|
|
if lastSlash >= 0 {
|
|
tail = ref[lastSlash+1:]
|
|
}
|
|
if colon := strings.LastIndex(tail, ":"); colon >= 0 {
|
|
// Only strip if the tag part looks like a tag (no slashes,
|
|
// non-empty). Otherwise leave alone. When lastSlash is -1 the
|
|
// arithmetic still yields the right cut point (-1 + 1 + colon
|
|
// == colon), so no special case is needed.
|
|
tag := tail[colon+1:]
|
|
if tag != "" && !strings.ContainsAny(tag, "/") {
|
|
return ref[:lastSlash+1+colon]
|
|
}
|
|
}
|
|
return ref
|
|
}
|
|
|
|
// imageRefFromSourceConfig extracts the "image" field from a workload's
|
|
// source_config JSON. Returns "" when the blob is missing, malformed,
|
|
// or has no image field — those workloads simply do not contribute to
|
|
// conflict detection.
|
|
func imageRefFromSourceConfig(raw string) string {
|
|
if raw == "" {
|
|
return ""
|
|
}
|
|
var cfg struct {
|
|
Image string `json:"image"`
|
|
}
|
|
if err := json.Unmarshal([]byte(raw), &cfg); err != nil {
|
|
return ""
|
|
}
|
|
return strings.TrimSpace(cfg.Image)
|
|
}
|
|
|