feat(discovery+runtime): restore static-site wizard discovery + close /sites/[id] feature parity
Build / build (push) Successful in 10m43s
Build / build (push) Successful in 10m43s
Two-stage feature arc closing the gaps left by the hard legacy cutover.
The static-site creation wizard regains its auto-discovery + connection-test
flow; /apps/[id] grows the runtime/storage/lifecycle surface the legacy
/sites/[id] page used to expose.
Backend (Go)
- internal/api/discovery.go: six admin-gated endpoints wrapping
staticsite.GitProvider — POST /api/discovery/git/{detect-provider,
test-connection,repos,branches,tree} + GET /api/discovery/image/conflicts.
Identifier validation (validateGitIdent / validateGitBranch) at the
boundary so provider URL interpolation cannot be hijacked via `..`.
Upstream errors scrubbed: detailed slog on the server, generic 502 to
the client (mitigates token-reflection-in-error-page).
- internal/api/workload_runtime.go: four endpoints —
GET /api/workloads/{id}/runtime-state decodes containers.extra_json for
static workloads; GET /api/workloads/{id}/storage execs `du -sb /app/data`
with a 30s in-process cache (storageProbeCache) so polling can't turn
into per-request execs; POST /api/workloads/{id}/{stop,start} iterate
ListContainersByWorkload and call docker.StopContainer / StartContainer,
returning 200 / 409 (nothing to act on) / 502 (all failed).
- internal/staticsite/safehttp.go: NewSafeHTTPClient + ValidateBaseURL +
blockReason. DialContext re-resolves hostnames and refuses loopback /
link-local / multicast / unspecified addresses. RFC1918 + ULA explicitly
allowed (self-hosted Gitea on LAN is the dominant deployment).
Replaced four raw &http.Client{} constructions in the provider files.
- internal/staticsite/gitlab_provider.go: url.PathEscape each segment in
the raw-file URL builder for parity with projectPath().
- Test coverage: 26 cases in discovery_test.go (image-tag stripping,
source-config decoding, conflict scenarios, validator boundaries,
scheme rejection), 14 in workload_runtime_test.go (404 / 409 / nil-docker
/ probe-cache), 16 in safehttp_test.go (URL validation + block-reason
policy matrix + live dial against loopback + AWS metadata literals).
Frontend (Svelte 5 + runes)
- web/src/lib/api.ts: typed wrappers for every endpoint, AbortSignal
threaded through post(); ApiError exported so callers can narrow on
e.status; new DetectedGitProvider narrow union.
- web/src/routes/apps/new/+page.svelte: static-form discovery controls
(auto-detect provider, test connection, repo / branch / folder
EntityPickers, Deno auto-detect); image-form conflict panel with
debounced lookup + double-click submit guard ("Forge anyway") + Inspect
button that pre-fills port/healthcheck; English error fallbacks routed
through apps.new.errors.* (en + ru).
- web/src/routes/apps/[id]/+page.svelte: runtime-state panel + storage
panel + Stop / Start / Open-site toolbar; universal live-state badge
in the hero lede for image/compose/static (RUNNING / TRANSITIONING /
STOPPED / NOT DEPLOYED / MIXED · n/m RUNNING); ContainerStats panel
per row (auto-collapsing native <details> when N > 2); read-only
webhook bindings summary card; responsive toolbar overflow with native
<details> at <640px (z-index 100 above sticky nav).
- web/src/app.css: project-wide .forge-btn-ghost:focus-visible outline.
Hardening from go-reviewer + security-reviewer + typescript-reviewer +
frontend-design UI/UX subagents (0 CRITICAL, all HIGH/BLOCKER addressed
inline, IMPORTANT applied before commit):
- AbortController + per-call sequence tokens on every long-running
fetch (loadRuntimeState / loadStorage / loadTriggerMeta / inspectImage /
listImageConflicts) plus onDestroy cleanup so late resolves cannot
mutate dead component state.
- doStop / doStart snapshot and restore `error` across the finally-block
reload so a load()-cleared message doesn't hide a real failure.
- triggersById refreshed after inline trigger creation so the webhook
card doesn't silently exclude the just-created trigger.
- Live-state badge wraps in role=status / aria-live=polite (no redundant
aria-label).
- Webhook row has a single click target (was two pointing at the same URL).
- Empty webhook section hides entirely.
- Dropped role=menu / role=menuitem from the overflow menu (they would
promise arrow-key nav we don't wire; native Tab + ESC carry it).
Doc
- docs/CODEMAPS/INDEX.md + new docs/CODEMAPS/discovery-and-runtime.md
map the endpoint surface, security posture, frontend integration
patterns, and an "add a new probe" recipe.
Verification
- svelte-check: 0 errors, 3 pre-existing a11y warnings.
- go build + go vet + go test ./...: all green.
- i18n parity: en + ru at 1413 keys each.
- Live smoke against :8090: 404 / 409 / 502 envelopes correct, discovery
sanity passes, ProbeError surfaces on no-container path.
This commit is contained in:
@@ -54,11 +54,9 @@ type GiteaContentFetcher struct {
|
||||
// token may be empty for public repositories.
|
||||
func NewGiteaContentFetcher(baseURL, token string) *GiteaContentFetcher {
|
||||
return &GiteaContentFetcher{
|
||||
baseURL: strings.TrimRight(baseURL, "/"),
|
||||
token: token,
|
||||
httpClient: &http.Client{
|
||||
Timeout: 60 * time.Second,
|
||||
},
|
||||
baseURL: strings.TrimRight(baseURL, "/"),
|
||||
token: token,
|
||||
httpClient: NewSafeHTTPClient(60 * time.Second),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -30,11 +30,9 @@ func NewGitHubProvider(baseURL, token string) *GitHubProvider {
|
||||
}
|
||||
|
||||
return &GitHubProvider{
|
||||
apiBase: apiBase,
|
||||
token: token,
|
||||
httpClient: &http.Client{
|
||||
Timeout: 60 * time.Second,
|
||||
},
|
||||
apiBase: apiBase,
|
||||
token: token,
|
||||
httpClient: NewSafeHTTPClient(60 * time.Second),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -26,12 +26,10 @@ type GitLabProvider struct {
|
||||
func NewGitLabProvider(baseURL, token string) *GitLabProvider {
|
||||
base := strings.TrimRight(baseURL, "/")
|
||||
return &GitLabProvider{
|
||||
apiBase: base + "/api/v4",
|
||||
rawBase: base,
|
||||
token: token,
|
||||
httpClient: &http.Client{
|
||||
Timeout: 60 * time.Second,
|
||||
},
|
||||
apiBase: base + "/api/v4",
|
||||
rawBase: base,
|
||||
token: token,
|
||||
httpClient: NewSafeHTTPClient(60 * time.Second),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -219,8 +217,14 @@ func (g *GitLabProvider) DownloadFolder(ctx context.Context, owner, repo, branch
|
||||
}
|
||||
|
||||
// GitLab raw file URL: {base}/{owner}/{repo}/-/raw/{branch}/{path}
|
||||
// Each segment is path-escaped to match projectPath()'s shape and
|
||||
// to refuse traversal sequences supplied via the request.
|
||||
fileURL := fmt.Sprintf("%s/%s/%s/-/raw/%s/%s",
|
||||
g.rawBase, owner, repo, branch, entry.Path)
|
||||
g.rawBase,
|
||||
url.PathEscape(owner),
|
||||
url.PathEscape(repo),
|
||||
url.PathEscape(branch),
|
||||
entry.Path)
|
||||
|
||||
if err := downloadFileHTTP(ctx, g.httpClient, fileURL, localPath, g.setAuth); err != nil {
|
||||
return fmt.Errorf("download %s: %w", relativePath, err)
|
||||
|
||||
@@ -101,8 +101,10 @@ func DetectProviderWithProbe(ctx context.Context, baseURL string) ProviderType {
|
||||
return urlBased
|
||||
}
|
||||
|
||||
// For unknown hosts, probe for Gitea/GitLab API signatures.
|
||||
client := &http.Client{Timeout: 5 * time.Second}
|
||||
// For unknown hosts, probe for Gitea/GitLab API signatures using the
|
||||
// SSRF-safe client so a probe URL cannot be used to reach loopback
|
||||
// or cloud-metadata addresses.
|
||||
client := NewSafeHTTPClient(5 * time.Second)
|
||||
base := strings.TrimRight(baseURL, "/")
|
||||
|
||||
// Try Gitea/Forgejo API.
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
package staticsite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ErrBlockedAddress is returned when the dialer refuses to connect
|
||||
// to a reserved IP (loopback / link-local / unspecified / multicast).
|
||||
// RFC1918 private ranges are intentionally allowed — self-hosted Gitea
|
||||
// on a LAN is the dominant deployment pattern.
|
||||
var ErrBlockedAddress = errors.New("connection to reserved address blocked")
|
||||
|
||||
// ValidateBaseURL enforces scheme + host shape on a user-supplied
|
||||
// provider base URL. Connect-time IP filtering happens later in the
|
||||
// safe-HTTP transport so DNS rebinding cannot bypass this check.
|
||||
func ValidateBaseURL(raw string) error {
|
||||
raw = strings.TrimSpace(raw)
|
||||
if raw == "" {
|
||||
return errors.New("base_url is required")
|
||||
}
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid base_url: %w", err)
|
||||
}
|
||||
if u.Scheme != "http" && u.Scheme != "https" {
|
||||
return fmt.Errorf("unsupported scheme %q (must be http or https)", u.Scheme)
|
||||
}
|
||||
if u.Host == "" {
|
||||
return errors.New("base_url is missing host")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// NewSafeHTTPClient returns an http.Client whose DialContext rejects
|
||||
// loopback, link-local, multicast, and unspecified addresses at connect
|
||||
// time. The dialer re-resolves and connects to the resolved IP so a
|
||||
// rebind between resolution and connect cannot slip through.
|
||||
//
|
||||
// RFC1918 / ULA private ranges are NOT blocked — operators routinely
|
||||
// point Tinyforge at self-hosted Gitea instances on private networks.
|
||||
// The threat model here is cloud-metadata exfiltration and loopback
|
||||
// service probing, not "any private IP is suspect".
|
||||
func NewSafeHTTPClient(timeout time.Duration) *http.Client {
|
||||
dialer := &net.Dialer{Timeout: 10 * time.Second, KeepAlive: 30 * time.Second}
|
||||
transport := &http.Transport{
|
||||
DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
host, port, err := net.SplitHostPort(addr)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// If the caller passed a literal IP, skip the DNS round-trip.
|
||||
if literal := net.ParseIP(host); literal != nil {
|
||||
if reason := blockReason(literal); reason != "" {
|
||||
return nil, fmt.Errorf("%w: %s (%s)", ErrBlockedAddress, literal, reason)
|
||||
}
|
||||
return dialer.DialContext(ctx, network, addr)
|
||||
}
|
||||
ips, err := net.DefaultResolver.LookupIPAddr(ctx, host)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(ips) == 0 {
|
||||
return nil, fmt.Errorf("no addresses for %s", host)
|
||||
}
|
||||
for _, ip := range ips {
|
||||
if reason := blockReason(ip.IP); reason != "" {
|
||||
return nil, fmt.Errorf("%w: %s (%s)", ErrBlockedAddress, ip.IP, reason)
|
||||
}
|
||||
}
|
||||
// Bind to the first resolved IP so a rebind between resolution
|
||||
// and connect cannot redirect the request to a blocked address.
|
||||
return dialer.DialContext(ctx, network, net.JoinHostPort(ips[0].IP.String(), port))
|
||||
},
|
||||
MaxIdleConns: 16,
|
||||
IdleConnTimeout: 30 * time.Second,
|
||||
TLSHandshakeTimeout: 10 * time.Second,
|
||||
}
|
||||
return &http.Client{Timeout: timeout, Transport: transport}
|
||||
}
|
||||
|
||||
// blockReason returns a human label for why an IP is rejected, or ""
|
||||
// if the IP is allowed. Centralized so all callers share the same
|
||||
// policy.
|
||||
func blockReason(ip net.IP) string {
|
||||
if ip == nil {
|
||||
return "nil address"
|
||||
}
|
||||
switch {
|
||||
case ip.IsLoopback():
|
||||
return "loopback"
|
||||
case ip.IsUnspecified():
|
||||
return "unspecified"
|
||||
case ip.IsLinkLocalUnicast():
|
||||
return "link-local"
|
||||
case ip.IsLinkLocalMulticast():
|
||||
return "link-local multicast"
|
||||
case ip.IsMulticast():
|
||||
return "multicast"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
package staticsite
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestValidateBaseURL(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
input string
|
||||
wantError bool
|
||||
}{
|
||||
{"https", "https://git.example.com", false},
|
||||
{"http", "http://git.example.com", false},
|
||||
{"trailing_slash", "https://git.example.com/", false},
|
||||
{"with_path", "https://git.example.com/sub", false},
|
||||
{"with_port", "https://git.example.com:8080", false},
|
||||
{"empty", "", true},
|
||||
{"whitespace_only", " ", true},
|
||||
{"ftp_scheme", "ftp://git.example.com", true},
|
||||
{"file_scheme", "file:///etc/passwd", true},
|
||||
{"no_scheme", "git.example.com", true},
|
||||
{"scheme_no_host", "https://", true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
err := ValidateBaseURL(tc.input)
|
||||
if tc.wantError && err == nil {
|
||||
t.Errorf("ValidateBaseURL(%q) = nil, want error", tc.input)
|
||||
}
|
||||
if !tc.wantError && err != nil {
|
||||
t.Errorf("ValidateBaseURL(%q) = %v, want nil", tc.input, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestBlockReason_PolicyMatrix(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
ip string
|
||||
wantBlocked bool
|
||||
}{
|
||||
// Allowed.
|
||||
{"public_v4", "8.8.8.8", false},
|
||||
{"rfc1918_10", "10.0.0.1", false},
|
||||
{"rfc1918_172_16", "172.16.0.1", false},
|
||||
{"rfc1918_192_168", "192.168.1.1", false},
|
||||
{"public_v6", "2606:4700:4700::1111", false},
|
||||
{"ula_v6", "fd00::1", false}, // ULA private — allowed, mirrors RFC1918
|
||||
|
||||
// Blocked.
|
||||
{"loopback_v4", "127.0.0.1", true},
|
||||
{"loopback_v6", "::1", true},
|
||||
{"unspecified_v4", "0.0.0.0", true},
|
||||
{"unspecified_v6", "::", true},
|
||||
{"link_local_v4_metadata", "169.254.169.254", true}, // AWS/GCP metadata
|
||||
{"link_local_v6", "fe80::1", true},
|
||||
{"multicast_v4", "224.0.0.1", true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
ip := net.ParseIP(tc.ip)
|
||||
if ip == nil {
|
||||
t.Fatalf("parse %q", tc.ip)
|
||||
}
|
||||
got := blockReason(ip)
|
||||
blocked := got != ""
|
||||
if blocked != tc.wantBlocked {
|
||||
t.Errorf("blockReason(%s) = %q (blocked=%v), want blocked=%v",
|
||||
tc.ip, got, blocked, tc.wantBlocked)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestSafeHTTPClient_RejectsLoopbackLiteral exercises the actual dial
|
||||
// path: a request to a loopback literal must fail before any TCP work
|
||||
// happens, with ErrBlockedAddress in the chain.
|
||||
func TestSafeHTTPClient_RejectsLoopbackLiteral(t *testing.T) {
|
||||
client := NewSafeHTTPClient(2 * time.Second)
|
||||
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, "http://127.0.0.1:1/", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("new request: %v", err)
|
||||
}
|
||||
_, err = client.Do(req)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !errors.Is(err, ErrBlockedAddress) && !strings.Contains(err.Error(), "blocked") {
|
||||
t.Errorf("err = %v, expected ErrBlockedAddress in chain or 'blocked' in message", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSafeHTTPClient_RejectsAWSMetadataLiteral mirrors the loopback
|
||||
// case but for the AWS/GCP cloud metadata IP (link-local).
|
||||
func TestSafeHTTPClient_RejectsAWSMetadataLiteral(t *testing.T) {
|
||||
client := NewSafeHTTPClient(2 * time.Second)
|
||||
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, "http://169.254.169.254/latest/meta-data/", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("new request: %v", err)
|
||||
}
|
||||
_, err = client.Do(req)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !errors.Is(err, ErrBlockedAddress) && !strings.Contains(err.Error(), "blocked") {
|
||||
t.Errorf("err = %v, expected ErrBlockedAddress in chain or 'blocked' in message", err)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user