6877c4d272ef9f2b42cf167785be465d70c8a3da
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 alexei.dolgolyov
|
10d30fc956 |
feat: production readiness — security, perf, bug fixes, bridge self-monitoring
Comprehensive multi-area pass driven by a parallel 8-agent production
review. Frontend, backend, database, security, performance, operational,
plus a new self-monitoring feature.
## Critical fixes
- Planka webhook: reads bounded raw body (was NameError on every call)
- HA quiet hours: ha_state_changed/automation_triggered/service_called/
event_fired added to deferrable set (were silently dropped)
- DNS-rebinding SSRF: PinnedResolver wired into shared aiohttp session
- Telegram inbound webhook: secret now mandatory (401 without)
- Generic webhook: auth_mode="none" requires explicit
acknowledge_unauthenticated=true; per-IP rate limit 60/min
- svelte-check: 5 null-narrowing errors in EventDetailModal fixed
- Provider hardcoding: Immich-only block extracted to descriptor
featureDiscoveryHint
- command_sync: snapshot+expunge bot before exiting AsyncSession
## Bug fixes
- notifier asyncio.gather(return_exceptions=True) — one bad chat no longer
cancels peer sends
- NotificationDispatcher hoisted out of per-tracker loop
- Provider credential resolution unified across all 5 dispatch sites
- HA asyncio.shield now drains inner task on cancellation
- Provider construction switched from if/elif ladder to factory registry
- NUT first poll seeds silently (no spurious ups_on_battery)
- Quiet-hours gate: event-type-disabled now wins over deferral
- APScheduler drain job ID resolution upgraded to seconds
- HA on_status_change wired through to EventLog
- Webhook payload rollback failures now logged (not swallowed)
- Batched receivers/chats/bots in load_link_data (was per-target N+1)
- flag_modified on JSON column reassignments in deferred_dispatch
## Database
- UNIQUE indexes on service_provider.webhook_token,
telegram_bot.webhook_path_id, partial UNIQUE on telegram_bot.bot_id,
telegram_chat(bot_id, chat_id), notification_tracker_target unique link,
partial UNIQUE on bridge_self provider per user
- Composite ix_event_log_user_event_type_created index
- save_chat_from_webhook switched to ON CONFLICT DO UPDATE
- ondelete=CASCADE on user-id FKs (model annotation; app-side cascade
delete added for existing data)
- delete_notification_tracker converted from N+1 to bulk DELETE/UPDATE
- Module-level asyncio.Lock replaced with lazy _get_lock() pattern
- VACUUM INTO snapshot now PRAGMA integrity_check verified
## Performance
- Jinja2 template compilation LRU cached (lru_cache maxsize=512)
- Per-locale render cache in NotificationDispatcher (skips re-rendering
identical content for receivers sharing a locale)
- Tracker list cached per provider_id with 5s TTL + explicit invalidation
on tracker CRUD (relieves HA chat-bus rate query pressure)
- Nav-counts collapsed from 16 round-trips to single UNION ALL
- HA event_log: skip persisting empty assets_added/removed events
## Security hardening
- Mass-assignment guard on Action create/update; cron sub-minute reject
- Backup JSON depth/node-count cap (depth ≤ 10, nodes ≤ 100k)
- _sanitize_config extended to all JSON-typed fields on backup import
- Telegram _safe_get walks redirects manually with SSRF revalidation
- Bcrypt 72-byte password length cap with clear 422
- Webhook payload body redaction; sensitive substring set extended with
oauth/client_secret/webhook_secret/csrf in both header filter and
template extras filter
## Frontend
- 76 catch (err: any) sites converted to errMsg(err) helper
- globalProviderFilter: pure getter; reconciliation moved to one-time
$effect in +layout
- Provider-filter binding: removed paired $effects + _syncingFilter flag,
now one-way derived
- entity-cache: separate _refreshing flag for background re-fetches
- api.ts 401 handling: AuthRedirectError class + dedup _redirecting flag,
goto() instead of window.location.href
- a11y: aria-expanded on mobile More, role=switch + aria-checked on
Telegram bot toggles
## Tests & operations
- CI pytest gate added to .gitea/workflows/build.yml + release.yml
(wheel-built install to dodge editable-install slowness)
- /api/ready upgraded to deep healthcheck (db SELECT 1, scheduler.running,
HA supervisor presence) returning {ready, checks, errors, version}
- /api/metrics endpoint with prometheus_client (deferred_pending,
event_log_total, dispatch_duration, poll_failures, send_failures)
- New OPERATIONS.md covering deploy, healthchecks, metrics, backup/restore
procedures, log handling, common scenarios, upgrade flow
- New tests: test_bridge_self (11), test_gitea_parser (9),
test_planka_parser (6), test_immich_change_detector (6),
test_backup_roundtrip (1)
## New feature: bridge self-monitoring
- New bridge_self provider type — internal sink for bridge health events
- Three event types: bridge_self_poll_failures (consecutive tracker poll
failures), bridge_self_deferred_backlog (pending count crosses
threshold), bridge_self_target_failures (consecutive 5xx/network
failures per target)
- Per-user thresholds (defaults: 3 / 100 / 5) configurable via the
provider config form
- Auto-seeded on user create + /setup + boot backfill for existing users
- Anti-spam: counters reset after emission; backlog uses transition latch
- Self-loop guard: bridge_self failures don't count toward target-failure
thresholds (logged only) — wire to your own Telegram/Email/Matrix to
get notified when polls/dispatches/sends fail
- 6 default templates (3 events × 2 locales), tracking config columns
with backfill migration, frontend descriptor (excluded from "create
provider" wizard since auto-managed)
Operator-visible behavior changes (call out in release notes):
- NOTIFY_BRIDGE_TELEGRAM_WEBHOOK_SECRET now REQUIRED for webhook mode
- Existing webhook providers with auth_mode="none" need explicit opt-in
- Generic webhook endpoint rate-limited 60/min per source IP
- HA disconnect/reconnect writes ha_status_* EventLog rows
- Every user gets a bridge_self provider — wire it to a target to
receive failure alerts
Pre-existing test failures (test_ssrf, test_release_provider) on
Python 3.13 are unrelated; CI runs on 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
alexei.dolgolyov
|
ba199f24bd |
feat: deferred dispatch, release-check provider, settings polish
- Defer quiet-hours dispatches into new deferred_dispatch table; drain job + periodic catch-up scan re-fire at window end with coalescing on (link, event_type, collection_id). - Add ON DELETE SET NULL migration on event_log_id and partial unique index on (link_id, collection_id, event_type) WHERE status='pending'. - Add release-check provider abstraction (Gitea/GitHub) with SSRF-safe URL validation, settings UI cassette, and scheduled polling. - Replace importlib-only version lookup with version.py helper that prefers the higher of installed metadata vs source pyproject so stale editable dev installs stop misreporting. - Aurora frontend polish: MetaStrip component, ReleaseCassette, EventDetailModal expansion, and i18n additions. |
||
|
alexei.dolgolyov
|
b170c2b792 |
feat(frontend): smoother event refresh, localized crumbs, template config deep-link
- Auto-refresh ticker is now silent: skips ``eventsLoading`` so the
loading placeholder no longer flashes, uses ``(event.id)`` key on
the events ``{#each}`` so unchanged rows reuse their DOM nodes, and
short-circuits the array reassignment when the visible page is
identical to what we already rendered. No-op refreshes leave the
list completely untouched.
- ``PageHeader`` crumbs (Routing · Notification, Operators · Bots, …)
were hard-coded literals. Moved to a new ``crumbs`` i18n namespace
with 9 keys; updated all 15 call sites to ``t('crumbs.*')`` so they
switch with the language.
- Tracker form's Immich feature-discovery banner now exposes both
``Open Tracking Config`` and ``Open Template Config``. Added the
``?edit=<id>`` auto-open hook to ``/template-configs`` (mirrors the
existing one on ``/tracking-configs``) so the new link lands users
directly on the editor.
|
||
|
alexei.dolgolyov
|
5bd63a2191 |
feat(frontend): autogenerate entity names from type/provider
Mirror the providers form pattern (defaultName tied to type) across bots, targets, trackers, actions, and configs. Each form now derives form.name from the selected type or provider while the user hasn't manually edited it; switching to edit-mode flips the manualEdited flag so existing names are preserved. Defaults: bots → "<Type> Bot"; targets → type label; notification trackers → "<provider> Tracker"; command trackers → "<provider> Commands"; actions → "<provider> <Action Type>"; tracking/template/ command/command-template configs → "<descriptor.defaultName> <Suffix>". TargetForm and TrackerForm grew an optional onnameinput prop so parents can flag manual edits in subform inputs. |
||
|
alexei.dolgolyov
|
d662b50925 |
feat(redesign): roll subpage hero across all pages + Aurora Button + JinjaEditor + pulse fix
Big batch — every secondary page now wears the same glass-card hero that landed on Providers earlier: - notification-trackers, tracking-configs, template-configs - command-trackers, command-configs, command-template-configs - targets (with active-tab title), actions - bots (telegram / email / matrix tabs) - settings, settings/backup, users Each page picks an italic-em emphasis word, an editorial crumb (e.g. 'Routing · Notification', 'Operators · Bots', 'System · Maintenance'), a count meter, and entity-specific status pills derived from live data: 'X armed / Y paused' for trackers and actions, 'X types' for configs/templates, 'X channels' or '$N receivers' for targets. Other changes in this commit: - Button.svelte: redesigned. Primary variant becomes a real Aurora CTA — gradient lavender → orchid pill, 40px tall md / 34px sm, inset highlight, lift + glow on hover. Secondary, danger, ghost variants reworked to match. The 'Add <Type>' button on every page now reads as the page's primary action instead of a flat lavender rectangle. - JinjaEditor: overrode oneDark's hardcoded background with !important so the editor surface picks up var(--color-input-bg). Gutters / scroller / selection / autocomplete tooltip all match Aurora glass tokens now. Template editors stop visually clashing with the surrounding panel. - Aurora pulse dot: rewritten as a self-contained box-shadow glow pulse (no transform, no pseudo-element). The dot's bounding box is now stable so ancestors with overflow:hidden can never clip the visible dot — only the (decorative) outer glow halo. Fixes the 'half-moon clipping' on the dashboard 'On watch' deck. - topbar-action.svelte.ts left in tree but unused (topbar CTA was reverted per your call). Will clean up in a later commit. - Form input baseline styling moved into app.css (rounded 0.625rem, glass background, hover/focus rings) so untouched filter inputs on the per-type pages stop looking out of place. i18n: emphasis / countLabel / armed / paused / receiver / receivers / channelsCount keys added across en + ru. Build clean: 0 errors, 61 pre-existing a11y warnings unchanged. |
||
|
alexei.dolgolyov
|
a7a2b4efa4 |
feat: large polish pass — UX fixes, per-chat scope, restore/backup, action events
Backend
- Per-chat album scope for Immich commands (search/latest/memory/...): new
allowed_album_ids on CommandTrackerListener, threaded listener/page kwargs
through ProviderCommandHandler.handle; PATCH listener-scope endpoint.
- /search and /find accept a trailing page number; Immich client search_smart
/ search_metadata take a page param.
- Immich person-asset lookup switched from removed GET /api/people/{id}/assets
to POST /api/search/metadata with personIds (fixes /person command and
auto_organize rules silently returning zero candidates on Immich 1.106+).
- Auto_organize rule now sets the target album's thumbnail to the first added
image when missing (falls back to any asset type); failures do not fail the
rule. add_assets_to_album surfaces the Immich error body on non-2xx.
- EventLog.user_id / action_id / action_name columns with defensive migration
+ backfill. Status query filters by user_id directly; Immich/webhook paths
emit user_id explicitly. action_runner writes an action_success/partial/
failed event on each non-dry-run.
- Dashboard DELETE /api/status/events (scoped to user_id) + rendering live
tracker/provider/action names via FK join with snapshot fallback.
- PATCH /api/users/{id} for username/role change with last-admin guard.
- Deletion protection returns structured {message, entity, blocked_by}
(ApiError carries .blockedBy; frontend opens BlockedByModal).
- Backup prepare-restore → AppSetting markers + atomic write of
pending_restore.json; lifespan hook applies on next startup and archives
under data/applied_restores/. apply-restart sends SIGTERM so the lifespan
shutdown runs; NOTIFY_BRIDGE_SUPERVISED env override gates the button.
Manual POST /api/backup/files (same format as scheduled).
- New periodic-summary test path reuses shared collect_scheduled_assets
(limit=0) so test and future production code go through one primitive.
- Per-receiver locale for Telegram test messages (resolves
TelegramChat.language_override per chat instead of applying the first
receiver's locale to everyone).
- Bounded concurrency (semaphores) in NotificationDispatcher._preload_asset_data
and _refresh_telegram_chat_titles; chat title sweep extended to 24h since
save_chat_from_webhook covers active chats opportunistically.
- Telegram poller detects the \"webhook is active\" 409 and auto-calls
deleteWebhook for bots whose DB update_mode is polling (throttled per bot).
- TelegramClient.get_chat added (CLAUDE.md rule 6); set_album_thumbnail added.
- Seeds: rename \"Default Commands\" → \"Default Immich Commands\";
track_assets_removed default False.
Frontend
- Global provider selector visible when there is only one provider.
- Clear-events button + i18n + ConfirmModal on the dashboard; new icons/
labels/filters/colors for action_success / action_partial / action_failed.
- Auto-select first available tracking/template/command/config + bot on
create forms (trackers, command-trackers, targets, template/command
configs).
- Telegram target disable_url_preview defaults to true.
- BlockedByModal wired into 8 deletion flows; fetchAuth helper for
multipart/binary calls (reuses api()'s refresh + ApiError mapping).
- Immich tracker 'Checking links' parallelised (concurrency cap 6).
- Backup page: pending-restore banner + Apply-now / Apply-later modal,
restarting overlay polling /api/health, manual 'Create backup' button.
- Command-trackers listener row gets an 'Edit album scope' modal with
inherit/explicit multiselect.
- Users page: Edit user modal (username + role).
- parseDate helper for consistent UTC date rendering.
Migrations / schema
- event_log: + user_id, action_id, action_name (+ backfill user_id from
notification_tracker).
- command_tracker_listener: + allowed_album_ids.
|
||
|
alexei.dolgolyov
|
f0739ca949 |
feat: security hardening — SSRF guard, template sandbox timeout, webhook log prune, auth & backup polish
- Add outbound URL validation (SSRF) for webhook/Discord/Slack/ntfy/Matrix dispatch - Template renderer: input/output caps and thread-based render timeout - Webhook log filter: strip Authorization/signature/token-like headers; atomic prune - Auth/JWT/backup/config tightening; misc frontend UX fixes |
||
|
alexei.dolgolyov
|
734e5c9340 |
feat: UX improvements — secure webhooks, locale fixes, dynamic languages, UI polish
- Remove top paginator from dashboard events, keep only bottom - Fix test message locale: pass UI locale to email/matrix bot tests - Convert webhook auth mode from text input to icon grid selector - Generate secure UUID tokens for webhook URLs instead of sequential IDs - Move Recent Payloads into per-provider expandable container (lazy-loaded) - Make template config languages dynamic via app settings instead of hardcoded - Change default dev port to 5175 |
||
|
alexei.dolgolyov
|
6e51164f8e |
refactor: comprehensive consistency review — UI/UX, code quality, functional parity
Fix 19 issues across 3 priority tiers found during full-codebase review: CRITICAL: - Fix undefined --color-secondary CSS variable causing invisible UI elements - Fix Google Photos command templates using nonexistent asset.originalFileName - Fix scheduler template variable docs (tracker_name → schedule_name) - Add missing admin guard on notification template update endpoint HIGH: - Fix 5 hardcoded English strings missing i18n (MultiEntitySelect, actions, settings, TelegramBotTab, users) - Replace 17 raw <button> elements with shared <Button> component - Replace 5 raw error divs with shared <ErrorBanner> component - Refactor webhook handler duplication into shared _dispatch_webhook_event() - Add 30+ provider-specific fields to TrackingConfig TypeScript type - Add default TrackingConfig seeds for immich and google_photos - Add provider-specific command variable docs (Gitea, Planka, NUT, GP, Webhook) MEDIUM: - Replace hardcoded hex colors and Tailwind classes with CSS variable tokens - Remove dead code (unused imports, orphaned check_notification_tracker) - Fix Svelte 5 patterns ($state for _prevProviderId, remove unnecessary as any) - Fix inconsistent POST response shape (targets now returns full response) - Fix Google Photos template dead asset.year branches, clarify album_url docs |
||
|
alexei.dolgolyov
|
b803d004e1 |
refactor: comprehensive codebase review — security, performance, quality, UX
Security: - Fix NUT protocol command injection (validate names against safe regex) - Enable Jinja2 autoescape=True to prevent HTML injection via external data - Add WebhookProviderConfig validation model Performance: - Shared aiohttp.ClientSession singleton (replaces 40+ per-request sessions) - Fix 4 N+1 queries with batch IN loads (poller, scheduler, memory, broadcast) - asyncio.gather for Gitea commands and notification dispatcher - Add DB indexes on NotificationTrackerState.tracker_id, CommandTrackerListener - LRU cache for compiled Jinja2 templates - Daily EventLog cleanup job (90-day retention) - 30s HTTP timeout on all external calls - GROUP BY for target type counts (replaces 7 sequential queries) Code quality: - Extract get_owned_entity() helper (replaces 11 duplicate functions) - Extract slot_helpers.py (load_slots, save_slots, render_template_preview) - Extract command_utils.py (tracker lookup, last event, collection IDs) - Extract http_session.py (shared session lifecycle) - Provider connection validation dedup (3x → 1 helper) - Command dispatch tables replacing if/elif chains - Album+links fetch helper (fetch_albums_with_links) - Provider dispatch polymorphism (list_provider_collections) - Immutable _enrich_assets (no longer mutates in-place) - Fix _format_assets return type + handler unpacking Frontend: - Fix 18+ hardcoded English strings → t() with new i18n keys (en + ru) - Mobile "More" nav panel with provider filter and search - Shared Button.svelte component (4 variants, 2 sizes) - Shared ErrorBanner.svelte component (8 pages updated) - SvelteKit goto() replacing window.location.href - Dashboard grid fixed for 4 cards, paginator opacity consistency Functionality: - max_instances=1 on scheduler jobs (prevents duplicate events) - Webhook provider in watcher (prevents error spam) - Fix stale SQLModel reference in poller - Gitea get_repo() direct API call |
||
|
alexei.dolgolyov
|
6e35926772 |
feat: default tracker configs, email validation, expandable target links
- Tracker now has default_tracking_config_id and default_template_config_id that apply to all linked targets unless overridden per-target - Dispatch falls back to tracker defaults when per-link configs are null - Email bot creation validates SMTP connection before saving - Email notifications sent as HTML (links render properly) - Linked target items are expandable: collapsed shows config CrossLinks, expanded shows config selectors; action buttons always visible - Fix email bot test button icon (mdiEmailSend → mdiSend) - Fix target type icons in LinkedTargetsSection for all types - Provider filter moved above search in sidebar |
||
|
alexei.dolgolyov
|
e0bae394ee |
feat: comprehensive code review fixes — security, performance, quality
Backend security: - Reject Gitea webhooks when webhook_secret is empty (was silently skipping) - Add slowapi rate limiting on login (5/min) and setup (3/min) endpoints - Add CORS middleware with configurable origins - Mask telegram_webhook_secret in settings API response - Protect system-owned command template configs from regular user modification - Increase minimum password length to 8 characters Backend performance: - Batch queries in _resolve_command_context (3 queries instead of 3N) - Concurrent album fetching with asyncio.gather in immich commands - Singleton Jinja2 SandboxedEnvironment (reuse instead of per-render creation) - TTLCache for rate limits (bounded memory, auto-eviction) - Optional aiohttp session reuse in send_reply/send_media_group Backend code quality: - Extract dispatch_helpers.py (shared link_data loading + event filtering) - Extract database/seeds.py from main.py (490 lines → dedicated module) - Split immich_handler.py (415 lines) into commands/immich/ subpackage - Replace bare except blocks with logged warnings - Add per-provider config validation (Pydantic models) - Truncate command input to 512 chars - Expose usage_* and desc_* slots in capabilities and variables API Frontend security: - CSS.escape() for user-controlled querySelector in highlight.ts - Client-side password min 8 chars validation on setup and password change Frontend code quality: - Replace any types with proper interfaces across top files - Decompose targets/+page.svelte into TargetForm + ReceiverSection - Fix $derived.by usage, $state mutation patterns - Add console.warn to empty catch blocks Frontend UX: - Auth redirect via goto() with "Redirecting..." state - Platform-aware Ctrl/Cmd K keyboard hint - Remove stat-card hover transform Frontend accessibility: - Modal: role=dialog, aria-modal, focus trap, restore focus - EntitySelect/IconGridSelect: listbox/option roles, aria-selected/disabled |
||
|
alexei.dolgolyov
|
751097b347 |
feat: comprehensive code review fixes + receivers-only architecture
Security:
- Refuse startup with default secret_key in production (was just logging)
- Settings endpoint now requires admin role
- Password validation on initial setup
- DOM-based HTML sanitizer replaces regex in template previews
- Add *.log to .gitignore
Performance & reliability:
- Token refresh deduplication prevents race condition on concurrent 401s
- Theme media query listener registered once (no leak)
- IconPicker uses $derived instead of function call per render
- Snackbar uses single-batch state update instead of while loop
- Replace 11 inline hover handlers with CSS :hover in layout
Architecture - receivers-only:
- Delivery endpoints (chat_id, email, url, room_id, topic) now stored
exclusively in TargetReceiver rows, never in target.config
- Migration extracts existing delivery fields to receiver rows
- Notifier and dispatcher remove all config fallbacks
- Frontend targets page shows receivers list per target with
add/remove/toggle/test per receiver
- Single-receiver test endpoint: POST /targets/{id}/receivers/{id}/test
Code quality:
- Extract AuthLayout.svelte from login/setup (150 lines CSS dedup)
- Split telegram-bots page (754→51 lines + 3 tab components)
- Split notification-trackers page (547→432 lines + 4 components)
- Deduplicate _send_reply into shared handler.send_reply()
- Add locale column to template models, replace name-based detection
- Fix delete_notification_tracker dead protection check
- Fix check_telegram_bot query (filter by type, remove bogus OR)
- Add graceful scheduler shutdown in lifespan
- Consistent /bots?tab=telegram URLs across all nav links
i18n:
- Error page, chat actions, target types, provider types internationalized
- All new receiver UI strings in EN + RU
|