Files
tiny-forge/internal/staticsite/gitea_content.go
T
alexei.dolgolyov ea55d31177
Build / build (push) Successful in 10m43s
feat(discovery+runtime): restore static-site wizard discovery + close /sites/[id] feature parity
Two-stage feature arc closing the gaps left by the hard legacy cutover.
The static-site creation wizard regains its auto-discovery + connection-test
flow; /apps/[id] grows the runtime/storage/lifecycle surface the legacy
/sites/[id] page used to expose.

Backend (Go)
- internal/api/discovery.go: six admin-gated endpoints wrapping
  staticsite.GitProvider — POST /api/discovery/git/{detect-provider,
  test-connection,repos,branches,tree} + GET /api/discovery/image/conflicts.
  Identifier validation (validateGitIdent / validateGitBranch) at the
  boundary so provider URL interpolation cannot be hijacked via `..`.
  Upstream errors scrubbed: detailed slog on the server, generic 502 to
  the client (mitigates token-reflection-in-error-page).
- internal/api/workload_runtime.go: four endpoints —
  GET /api/workloads/{id}/runtime-state decodes containers.extra_json for
  static workloads; GET /api/workloads/{id}/storage execs `du -sb /app/data`
  with a 30s in-process cache (storageProbeCache) so polling can't turn
  into per-request execs; POST /api/workloads/{id}/{stop,start} iterate
  ListContainersByWorkload and call docker.StopContainer / StartContainer,
  returning 200 / 409 (nothing to act on) / 502 (all failed).
- internal/staticsite/safehttp.go: NewSafeHTTPClient + ValidateBaseURL +
  blockReason. DialContext re-resolves hostnames and refuses loopback /
  link-local / multicast / unspecified addresses. RFC1918 + ULA explicitly
  allowed (self-hosted Gitea on LAN is the dominant deployment).
  Replaced four raw &http.Client{} constructions in the provider files.
- internal/staticsite/gitlab_provider.go: url.PathEscape each segment in
  the raw-file URL builder for parity with projectPath().
- Test coverage: 26 cases in discovery_test.go (image-tag stripping,
  source-config decoding, conflict scenarios, validator boundaries,
  scheme rejection), 14 in workload_runtime_test.go (404 / 409 / nil-docker
  / probe-cache), 16 in safehttp_test.go (URL validation + block-reason
  policy matrix + live dial against loopback + AWS metadata literals).

Frontend (Svelte 5 + runes)
- web/src/lib/api.ts: typed wrappers for every endpoint, AbortSignal
  threaded through post(); ApiError exported so callers can narrow on
  e.status; new DetectedGitProvider narrow union.
- web/src/routes/apps/new/+page.svelte: static-form discovery controls
  (auto-detect provider, test connection, repo / branch / folder
  EntityPickers, Deno auto-detect); image-form conflict panel with
  debounced lookup + double-click submit guard ("Forge anyway") + Inspect
  button that pre-fills port/healthcheck; English error fallbacks routed
  through apps.new.errors.* (en + ru).
- web/src/routes/apps/[id]/+page.svelte: runtime-state panel + storage
  panel + Stop / Start / Open-site toolbar; universal live-state badge
  in the hero lede for image/compose/static (RUNNING / TRANSITIONING /
  STOPPED / NOT DEPLOYED / MIXED · n/m RUNNING); ContainerStats panel
  per row (auto-collapsing native <details> when N > 2); read-only
  webhook bindings summary card; responsive toolbar overflow with native
  <details> at <640px (z-index 100 above sticky nav).
- web/src/app.css: project-wide .forge-btn-ghost:focus-visible outline.

Hardening from go-reviewer + security-reviewer + typescript-reviewer +
frontend-design UI/UX subagents (0 CRITICAL, all HIGH/BLOCKER addressed
inline, IMPORTANT applied before commit):
- AbortController + per-call sequence tokens on every long-running
  fetch (loadRuntimeState / loadStorage / loadTriggerMeta / inspectImage /
  listImageConflicts) plus onDestroy cleanup so late resolves cannot
  mutate dead component state.
- doStop / doStart snapshot and restore `error` across the finally-block
  reload so a load()-cleared message doesn't hide a real failure.
- triggersById refreshed after inline trigger creation so the webhook
  card doesn't silently exclude the just-created trigger.
- Live-state badge wraps in role=status / aria-live=polite (no redundant
  aria-label).
- Webhook row has a single click target (was two pointing at the same URL).
- Empty webhook section hides entirely.
- Dropped role=menu / role=menuitem from the overflow menu (they would
  promise arrow-key nav we don't wire; native Tab + ESC carry it).

Doc
- docs/CODEMAPS/INDEX.md + new docs/CODEMAPS/discovery-and-runtime.md
  map the endpoint surface, security posture, frontend integration
  patterns, and an "add a new probe" recipe.

Verification
- svelte-check: 0 errors, 3 pre-existing a11y warnings.
- go build + go vet + go test ./...: all green.
- i18n parity: en + ru at 1413 keys each.
- Live smoke against :8090: 404 / 409 / 502 envelopes correct, discovery
  sanity passes, ProbeError surfaces on no-container path.
2026-05-16 21:35:51 +03:00

370 lines
9.7 KiB
Go

package staticsite
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"time"
)
// giteaTreeEntry represents a single entry in a Gitea git tree response.
type giteaTreeEntry struct {
Path string `json:"path"`
Type string `json:"type"` // "blob" or "tree"
SHA string `json:"sha"`
Size int64 `json:"size"`
}
// giteaTreeResponse represents the Gitea git tree API response.
type giteaTreeResponse struct {
SHA string `json:"sha"`
Entries []giteaTreeEntry `json:"tree"`
Truncated bool `json:"truncated"`
}
// giteaBranch represents a branch from the Gitea API.
type giteaBranch struct {
Name string `json:"name"`
Commit struct {
ID string `json:"id"`
} `json:"commit"`
}
// giteaRef represents a git reference from the Gitea API.
type giteaRef struct {
Ref string `json:"ref"`
Object struct {
SHA string `json:"sha"`
} `json:"object"`
}
// GiteaContentFetcher downloads folder contents from a Gitea repository.
type GiteaContentFetcher struct {
baseURL string
token string
httpClient *http.Client
}
// NewGiteaContentFetcher creates a new content fetcher.
// token may be empty for public repositories.
func NewGiteaContentFetcher(baseURL, token string) *GiteaContentFetcher {
return &GiteaContentFetcher{
baseURL: strings.TrimRight(baseURL, "/"),
token: token,
httpClient: NewSafeHTTPClient(60 * time.Second),
}
}
// Name returns the provider identifier.
func (f *GiteaContentFetcher) Name() string { return "gitea" }
// ListRepos returns repositories accessible with the current token.
func (f *GiteaContentFetcher) ListRepos(ctx context.Context, query string) ([]RepoInfo, error) {
var allRepos []RepoInfo
page := 1
limit := 50
for {
url := fmt.Sprintf("%s/api/v1/repos/search?limit=%d&page=%d", f.baseURL, limit, page)
if query != "" {
url += "&q=" + query
}
if f.token != "" {
// When authenticated, include private repos.
url += "&private=true"
}
body, err := f.doGet(ctx, url)
if err != nil {
return nil, fmt.Errorf("list repos: %w", err)
}
var result struct {
Data []struct {
Owner struct {
Login string `json:"login"`
} `json:"owner"`
Name string `json:"name"`
FullName string `json:"full_name"`
Description string `json:"description"`
Private bool `json:"private"`
HTMLURL string `json:"html_url"`
} `json:"data"`
}
if err := json.Unmarshal(body, &result); err != nil {
// Gitea search wraps in {"data": [...]}, but some versions return a flat array.
var flat []struct {
Owner struct {
Login string `json:"login"`
} `json:"owner"`
Name string `json:"name"`
FullName string `json:"full_name"`
Description string `json:"description"`
Private bool `json:"private"`
HTMLURL string `json:"html_url"`
}
if err2 := json.Unmarshal(body, &flat); err2 != nil {
return nil, fmt.Errorf("decode repos: %w", err)
}
for _, r := range flat {
allRepos = append(allRepos, RepoInfo{
Owner: r.Owner.Login,
Name: r.Name,
FullName: r.FullName,
Description: r.Description,
Private: r.Private,
HTMLURL: r.HTMLURL,
})
}
if len(flat) < limit {
break
}
page++
continue
}
for _, r := range result.Data {
allRepos = append(allRepos, RepoInfo{
Owner: r.Owner.Login,
Name: r.Name,
FullName: r.FullName,
Description: r.Description,
Private: r.Private,
HTMLURL: r.HTMLURL,
})
}
if len(result.Data) < limit {
break
}
page++
}
return allRepos, nil
}
// ListBranches returns all branches for a repository.
func (f *GiteaContentFetcher) ListBranches(ctx context.Context, owner, repo string) ([]string, error) {
var allBranches []string
page := 1
limit := 50
for {
url := fmt.Sprintf("%s/api/v1/repos/%s/%s/branches?page=%d&limit=%d",
f.baseURL, owner, repo, page, limit)
body, err := f.doGet(ctx, url)
if err != nil {
return nil, fmt.Errorf("list branches: %w", err)
}
var branches []giteaBranch
if err := json.Unmarshal(body, &branches); err != nil {
return nil, fmt.Errorf("decode branches: %w", err)
}
for _, b := range branches {
allBranches = append(allBranches, b.Name)
}
if len(branches) < limit {
break
}
page++
}
return allBranches, nil
}
// GetLatestCommitSHA returns the latest commit SHA for a branch.
func (f *GiteaContentFetcher) GetLatestCommitSHA(ctx context.Context, owner, repo, branch string) (string, error) {
url := fmt.Sprintf("%s/api/v1/repos/%s/%s/branches/%s",
f.baseURL, owner, repo, branch)
body, err := f.doGet(ctx, url)
if err != nil {
return "", fmt.Errorf("get branch info: %w", err)
}
var b giteaBranch
if err := json.Unmarshal(body, &b); err != nil {
return "", fmt.Errorf("decode branch: %w", err)
}
return b.Commit.ID, nil
}
// FolderEntry represents a file or directory in the repo tree.
type FolderEntry struct {
Path string `json:"path"`
IsDir bool `json:"is_dir"`
}
// ListTree returns the full directory tree for a branch, useful for the folder picker.
func (f *GiteaContentFetcher) ListTree(ctx context.Context, owner, repo, branch string) ([]FolderEntry, error) {
url := fmt.Sprintf("%s/api/v1/repos/%s/%s/git/trees/%s?recursive=true",
f.baseURL, owner, repo, branch)
body, err := f.doGet(ctx, url)
if err != nil {
return nil, fmt.Errorf("list tree: %w", err)
}
var tree giteaTreeResponse
if err := json.Unmarshal(body, &tree); err != nil {
return nil, fmt.Errorf("decode tree: %w", err)
}
entries := make([]FolderEntry, 0, len(tree.Entries))
for _, e := range tree.Entries {
entries = append(entries, FolderEntry{
Path: e.Path,
IsDir: e.Type == "tree",
})
}
return entries, nil
}
// DownloadFolder downloads all files from a specific folder path in the repo
// to a local temporary directory. Returns the path to the temp directory.
func (f *GiteaContentFetcher) DownloadFolder(ctx context.Context, owner, repo, branch, folderPath, destDir string) error {
// Get the full tree.
url := fmt.Sprintf("%s/api/v1/repos/%s/%s/git/trees/%s?recursive=true",
f.baseURL, owner, repo, branch)
body, err := f.doGet(ctx, url)
if err != nil {
return fmt.Errorf("fetch tree: %w", err)
}
var tree giteaTreeResponse
if err := json.Unmarshal(body, &tree); err != nil {
return fmt.Errorf("decode tree: %w", err)
}
// Normalize folder path.
folderPath = strings.TrimPrefix(folderPath, "/")
folderPath = strings.TrimSuffix(folderPath, "/")
prefix := folderPath + "/"
// Download each file in the folder.
for _, entry := range tree.Entries {
if entry.Type != "blob" {
continue
}
if !strings.HasPrefix(entry.Path, prefix) {
continue
}
relativePath := strings.TrimPrefix(entry.Path, prefix)
localPath := filepath.Join(destDir, filepath.FromSlash(relativePath))
// Path-traversal defense: reject anything whose resolved
// destination escapes destDir. A hostile (or compromised)
// Gitea instance could return tree entries with `..` in
// the path; filepath.Join cleans them and would otherwise
// write outside the build context.
cleanDest := filepath.Clean(destDir)
if cleanRel := filepath.Clean(localPath); cleanRel != cleanDest &&
!strings.HasPrefix(cleanRel, cleanDest+string(os.PathSeparator)) {
return fmt.Errorf("rejecting tree entry outside dest: %s", relativePath)
}
// Create parent directories.
if err := os.MkdirAll(filepath.Dir(localPath), 0o755); err != nil {
return fmt.Errorf("create directory for %s: %w", relativePath, err)
}
// Download the file.
fileURL := fmt.Sprintf("%s/api/v1/repos/%s/%s/raw/%s?ref=%s",
f.baseURL, owner, repo, entry.Path, branch)
if err := f.downloadFile(ctx, fileURL, localPath); err != nil {
return fmt.Errorf("download %s: %w", relativePath, err)
}
}
return nil
}
// TestConnection verifies that the repository is accessible.
func (f *GiteaContentFetcher) TestConnection(ctx context.Context, owner, repo string) error {
url := fmt.Sprintf("%s/api/v1/repos/%s/%s", f.baseURL, owner, repo)
_, err := f.doGet(ctx, url)
if err != nil {
return fmt.Errorf("test connection: %w", err)
}
return nil
}
// doGet performs an authenticated GET request and returns the response body.
func (f *GiteaContentFetcher) doGet(ctx context.Context, url string) ([]byte, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
if f.token != "" {
req.Header.Set("Authorization", "token "+f.token)
}
req.Header.Set("Accept", "application/json")
resp, err := f.httpClient.Do(req)
if err != nil {
return nil, fmt.Errorf("execute request: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("read response: %w", err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("unexpected status %d: %s", resp.StatusCode, string(body))
}
return body, nil
}
// downloadFile downloads a URL to a local file path.
func (f *GiteaContentFetcher) downloadFile(ctx context.Context, url, localPath string) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return fmt.Errorf("create request: %w", err)
}
if f.token != "" {
req.Header.Set("Authorization", "token "+f.token)
}
resp, err := f.httpClient.Do(req)
if err != nil {
return fmt.Errorf("execute request: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected status %d for %s", resp.StatusCode, url)
}
file, err := os.Create(localPath)
if err != nil {
return fmt.Errorf("create file: %w", err)
}
defer file.Close()
if _, err := io.Copy(file, resp.Body); err != nil {
return fmt.Errorf("write file: %w", err)
}
return nil
}