Commit Graph

10 Commits

Author SHA1 Message Date
alexei.dolgolyov 2bec25353b ci: install editable packages inside a venv
Build and Test / test-frontend (push) Successful in 9m46s
Release / release (push) Has been cancelled
Release / test (push) Has been cancelled
Build and Test / build-image (push) Has been cancelled
Build and Test / test-backend (push) Has been cancelled
The hosted Gitea runner image pre-installs older versions of both packages
in its system Python site-packages and retains stale ~otify_bridge_core /
~otify_bridge_server dist-info directories from prior interrupted runs.
``pip install -e`` against the system interpreter tries to uninstall those,
the rollback fires mid-transaction, and the runner's
``/opt/hostedtoolcache/.../bin/notify-bridge`` console script disappears
before the new install can be placed:

  ERROR: Could not install packages due to an OSError:
  [Errno 2] No such file or directory:
  '/opt/hostedtoolcache/Python/3.12.12/x64/bin/notify-bridge'

Installing into a fresh venv sidesteps the pre-cached state entirely (and
is the recommendation pip itself prints on every run).
2026-04-23 20:23:42 +03:00
alexei.dolgolyov 920920bc67 feat: production-readiness hardening across security, async, DB, ops
Build and Test / test-frontend (push) Successful in 9m37s
Build and Test / test-backend (push) Successful in 10m53s
Build and Test / build-image (push) Failing after 14m52s
Security
- SSRF: async DNS resolver; allow_redirects=False on all outbound clients;
  matrix homeserver_url validated on create/update/test; update_provider
  and email_bot merge incoming config and reject ***-masked secrets.
- Auth: bcrypt offloaded to asyncio.to_thread; JWT now carries iss/aud +
  leeway and rejects missing claims; setup TOCTOU closed inside a
  transaction; rate limits extended (default 600/min, 10/min on password
  change, 30/min on needs-setup); constant-time login to prevent username
  enumeration.
- Config: rejects known dev secret keys; validates CORS origin schemes,
  port range, token lifetimes.
- Webhook handlers stream-read body with a 1 MiB cap; Discord 429 retries
  bounded (3 attempts, Retry-After capped at 60 s).
- CSP + HSTS added to SecurityHeadersMiddleware.

Async / runtime
- SQLite engine: WAL, synchronous=NORMAL, foreign_keys=ON, busy_timeout,
  pool_pre_ping, dispose on shutdown.
- Lifespan shutdown now stops scheduler before closing HTTP session and
  disposing the engine.
- Shared aiohttp session locked against concurrent first-caller races;
  core NotificationDispatcher accepts and reuses it.
- Storage and scheduled backup writes wrapped in asyncio.to_thread.
- NUT client writes bounded by asyncio.wait_for.
- Telegram poller switched from 3 s short-poll to 30 s interval + 25 s
  long-poll (~10x fewer API calls).

Database
- New performance-indexes migration covers every FK/owner column and
  hot-path composite (notification_tracker(provider_id, enabled);
  event_log(user_id, created_at DESC); webhook_payload_log(provider_id,
  created_at DESC); action_execution(action_id, started_at DESC)).
- New schema_version table for future upgrade gating.
- __system__ placeholder user (id=0) seeded so user_id=0 system defaults
  satisfy the newly enforced FK; filtered out of /auth/needs-setup,
  /api/users, and setup.
- list_notification_trackers rewritten to batched loads (was 1+N+N*M).
- Retention job extended to event_log, webhook_payload_log, and
  action_execution; retention days exposed as a setting.

Scheduler
- AsyncIOScheduler job_defaults: coalesce, misfire_grace_time=300,
  max_instances=1.

Ops
- uvicorn runs with proxy_headers, forwarded_allow_ips,
  timeout_graceful_shutdown; access log suppressed in non-debug.
- FastAPI version string now reads from importlib.metadata.
- New /api/ready endpoint separate from /api/health.
- docker-compose drops the ALLOW_PRIVATE_URLS=1 default, adds mem/cpu/pid
  limits, read_only + tmpfs, cap_drop:ALL, no-new-privileges; healthcheck
  targets /api/ready.
- CI now runs on push/PR with backend pytest, frontend svelte-check +
  build, and a non-push image build; release workflow gated on tests,
  publishes immutable sha-<commit> image tag, adds Trivy scan.

Tests
- New packages/server/tests/ with 29 passing tests: config validation,
  JWT round-trip + aud/alg=none rejection, SSRF scheme and private-range
  enforcement (sync + async), Discord bounded retry, and a lifespan-level
  /api/health + /api/ready smoke check.
- Renamed the misnamed services/test_dispatch.py to manual_dispatch.py so
  pytest never auto-collects production code.

Frontend
- /login now redirects already-authenticated users to /, shows a distinct
  'backend unreachable' banner (en/ru) when /auth/needs-setup fails.
2026-04-23 19:44:56 +03:00
alexei.dolgolyov f27fa42b87 fix(ci): build release payload via heredoc, drop broken env-var passing
Release / release (push) Successful in 24s
Previous attempt used `python3 -c "..." KEY=VALUE` which passes
KEY=VALUE as positional args, not environment variables — the python
block then crashed with KeyError: 'BODY' because nothing actually set
it in the environment.

Consolidate into a single heredoc-fed python3 block that reads
RELEASE_NOTES from the already-exported env var and reads TAG/VERSION/
IS_PRE after an explicit `export`. Uses <<'PY' so shell metachars in
the Python source (backticks, $, quotes) are not interpreted.

Also drops the redundant intermediate BODY variable — body is built
directly inside the single python invocation.
2026-04-21 20:16:27 +03:00
alexei.dolgolyov e12820f150 ci: robust Gitea release creation with HTTP status + diagnostics
Release / release (push) Failing after 21s
Previous implementation silently assumed any missing 'id' in POST
response meant "release already exists", then called an unguarded
python3 on the fallback response — which crashes (exit 1) if the
fallback also fails (e.g. release really doesn't exist).

New logic:
- Build JSON payload in Python (avoids shell escaping + CLI length limits)
- Capture HTTP status explicitly
- 201 → success
- 409 or "already exists" message → reuse existing (with HTTP check on fetch)
- Anything else → fail loudly with the response body printed

This also unblocks diagnosis of the current v0.1.0 failure by surfacing
the actual error the Gitea API is returning.
2026-04-21 20:09:55 +03:00
alexei.dolgolyov 866a8df310 ci: fix changelog step on shallow checkout and small repos
Release / release (push) Failing after 54s
- Set fetch-depth: 0 so previous tag lookups work across full history.
- Use `-n 20` instead of HEAD~20..HEAD, which fails when the repo has
  fewer than 20 commits (e.g. on the first release).
2026-04-21 19:59:40 +03:00
alexei.dolgolyov 56b345188e ci: consolidate release.yml into single checkout step
Release / release (push) Failing after 1m53s
The two-step pattern (sparse-checkout RELEASE_NOTES.md, then full
checkout) left sparse-checkout config active on the workspace, so the
second checkout still only restored RELEASE_NOTES.md. Docker build
then failed with "open Dockerfile: no such file or directory".

Since both RELEASE_NOTES.md and the full source are needed in the same
job, one full checkout is simpler and correct.
2026-04-21 19:50:49 +03:00
alexei.dolgolyov eecc9e295c ci: consolidate release tokens to single DEPLOY_TOKEN, rename redeploy step
- Use one DEPLOY_TOKEN for both registry login and Gitea release API,
  matching the claude-code-facts convention.
- Rename "Trigger Portainer redeploy" to "Trigger redeploy webhook" —
  the step calls a generic DOCKER_REDEPLOY_WEBHOOK_URL, not a
  Portainer-specific endpoint.
- Add .facts-sync.json to pin this project to the facts repo commit.
2026-04-21 19:35:50 +03:00
alexei.dolgolyov c41182ffd0 ci: sync release workflow with CI/CD docs, add manual build
- Fix github.* → gitea.* context consistency
- Add pre-release detection (skip :latest for alpha/beta/rc)
- Add release fallback (reuse existing if creation fails)
- Add prerelease field to release API call
- Use sparse-checkout for RELEASE_NOTES.md
- Skip Portainer redeploy for pre-releases
- Add version tag without v prefix
- Add manual build.yml for Docker image verification
2026-03-28 13:27:28 +03:00
alexei.dolgolyov b803d004e1 refactor: comprehensive codebase review — security, performance, quality, UX
Security:
- Fix NUT protocol command injection (validate names against safe regex)
- Enable Jinja2 autoescape=True to prevent HTML injection via external data
- Add WebhookProviderConfig validation model

Performance:
- Shared aiohttp.ClientSession singleton (replaces 40+ per-request sessions)
- Fix 4 N+1 queries with batch IN loads (poller, scheduler, memory, broadcast)
- asyncio.gather for Gitea commands and notification dispatcher
- Add DB indexes on NotificationTrackerState.tracker_id, CommandTrackerListener
- LRU cache for compiled Jinja2 templates
- Daily EventLog cleanup job (90-day retention)
- 30s HTTP timeout on all external calls
- GROUP BY for target type counts (replaces 7 sequential queries)

Code quality:
- Extract get_owned_entity() helper (replaces 11 duplicate functions)
- Extract slot_helpers.py (load_slots, save_slots, render_template_preview)
- Extract command_utils.py (tracker lookup, last event, collection IDs)
- Extract http_session.py (shared session lifecycle)
- Provider connection validation dedup (3x → 1 helper)
- Command dispatch tables replacing if/elif chains
- Album+links fetch helper (fetch_albums_with_links)
- Provider dispatch polymorphism (list_provider_collections)
- Immutable _enrich_assets (no longer mutates in-place)
- Fix _format_assets return type + handler unpacking

Frontend:
- Fix 18+ hardcoded English strings → t() with new i18n keys (en + ru)
- Mobile "More" nav panel with provider filter and search
- Shared Button.svelte component (4 variants, 2 sizes)
- Shared ErrorBanner.svelte component (8 pages updated)
- SvelteKit goto() replacing window.location.href
- Dashboard grid fixed for 4 cards, paginator opacity consistency

Functionality:
- max_instances=1 on scheduler jobs (prevents duplicate events)
- Webhook provider in watcher (prevents error spam)
- Fix stale SQLModel reference in poller
- Gitea get_repo() direct API call
2026-03-28 13:22:26 +03:00
alexei.dolgolyov 1ac6a17f6f feat: Docker deployment + Gitea CI/CD workflow
- Multi-stage Dockerfile: Node frontend build → Python wheel build → slim runtime
- Backend serves SvelteKit static output via FastAPI StaticFiles mount
- docker-compose.yml with named volume for /data persistence
- Gitea Actions workflow: build/push Docker image + create release on v* tags
- Add NOTIFY_BRIDGE_STATIC_DIR config for frontend path
- Fix run() to use configurable host/port
2026-03-23 02:14:14 +03:00