11593eaa7cd788b2372e8c72f33105013eafa63e
15 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 alexei.dolgolyov
|
10d30fc956 |
feat: production readiness — security, perf, bug fixes, bridge self-monitoring
Comprehensive multi-area pass driven by a parallel 8-agent production
review. Frontend, backend, database, security, performance, operational,
plus a new self-monitoring feature.
## Critical fixes
- Planka webhook: reads bounded raw body (was NameError on every call)
- HA quiet hours: ha_state_changed/automation_triggered/service_called/
event_fired added to deferrable set (were silently dropped)
- DNS-rebinding SSRF: PinnedResolver wired into shared aiohttp session
- Telegram inbound webhook: secret now mandatory (401 without)
- Generic webhook: auth_mode="none" requires explicit
acknowledge_unauthenticated=true; per-IP rate limit 60/min
- svelte-check: 5 null-narrowing errors in EventDetailModal fixed
- Provider hardcoding: Immich-only block extracted to descriptor
featureDiscoveryHint
- command_sync: snapshot+expunge bot before exiting AsyncSession
## Bug fixes
- notifier asyncio.gather(return_exceptions=True) — one bad chat no longer
cancels peer sends
- NotificationDispatcher hoisted out of per-tracker loop
- Provider credential resolution unified across all 5 dispatch sites
- HA asyncio.shield now drains inner task on cancellation
- Provider construction switched from if/elif ladder to factory registry
- NUT first poll seeds silently (no spurious ups_on_battery)
- Quiet-hours gate: event-type-disabled now wins over deferral
- APScheduler drain job ID resolution upgraded to seconds
- HA on_status_change wired through to EventLog
- Webhook payload rollback failures now logged (not swallowed)
- Batched receivers/chats/bots in load_link_data (was per-target N+1)
- flag_modified on JSON column reassignments in deferred_dispatch
## Database
- UNIQUE indexes on service_provider.webhook_token,
telegram_bot.webhook_path_id, partial UNIQUE on telegram_bot.bot_id,
telegram_chat(bot_id, chat_id), notification_tracker_target unique link,
partial UNIQUE on bridge_self provider per user
- Composite ix_event_log_user_event_type_created index
- save_chat_from_webhook switched to ON CONFLICT DO UPDATE
- ondelete=CASCADE on user-id FKs (model annotation; app-side cascade
delete added for existing data)
- delete_notification_tracker converted from N+1 to bulk DELETE/UPDATE
- Module-level asyncio.Lock replaced with lazy _get_lock() pattern
- VACUUM INTO snapshot now PRAGMA integrity_check verified
## Performance
- Jinja2 template compilation LRU cached (lru_cache maxsize=512)
- Per-locale render cache in NotificationDispatcher (skips re-rendering
identical content for receivers sharing a locale)
- Tracker list cached per provider_id with 5s TTL + explicit invalidation
on tracker CRUD (relieves HA chat-bus rate query pressure)
- Nav-counts collapsed from 16 round-trips to single UNION ALL
- HA event_log: skip persisting empty assets_added/removed events
## Security hardening
- Mass-assignment guard on Action create/update; cron sub-minute reject
- Backup JSON depth/node-count cap (depth ≤ 10, nodes ≤ 100k)
- _sanitize_config extended to all JSON-typed fields on backup import
- Telegram _safe_get walks redirects manually with SSRF revalidation
- Bcrypt 72-byte password length cap with clear 422
- Webhook payload body redaction; sensitive substring set extended with
oauth/client_secret/webhook_secret/csrf in both header filter and
template extras filter
## Frontend
- 76 catch (err: any) sites converted to errMsg(err) helper
- globalProviderFilter: pure getter; reconciliation moved to one-time
$effect in +layout
- Provider-filter binding: removed paired $effects + _syncingFilter flag,
now one-way derived
- entity-cache: separate _refreshing flag for background re-fetches
- api.ts 401 handling: AuthRedirectError class + dedup _redirecting flag,
goto() instead of window.location.href
- a11y: aria-expanded on mobile More, role=switch + aria-checked on
Telegram bot toggles
## Tests & operations
- CI pytest gate added to .gitea/workflows/build.yml + release.yml
(wheel-built install to dodge editable-install slowness)
- /api/ready upgraded to deep healthcheck (db SELECT 1, scheduler.running,
HA supervisor presence) returning {ready, checks, errors, version}
- /api/metrics endpoint with prometheus_client (deferred_pending,
event_log_total, dispatch_duration, poll_failures, send_failures)
- New OPERATIONS.md covering deploy, healthchecks, metrics, backup/restore
procedures, log handling, common scenarios, upgrade flow
- New tests: test_bridge_self (11), test_gitea_parser (9),
test_planka_parser (6), test_immich_change_detector (6),
test_backup_roundtrip (1)
## New feature: bridge self-monitoring
- New bridge_self provider type — internal sink for bridge health events
- Three event types: bridge_self_poll_failures (consecutive tracker poll
failures), bridge_self_deferred_backlog (pending count crosses
threshold), bridge_self_target_failures (consecutive 5xx/network
failures per target)
- Per-user thresholds (defaults: 3 / 100 / 5) configurable via the
provider config form
- Auto-seeded on user create + /setup + boot backfill for existing users
- Anti-spam: counters reset after emission; backlog uses transition latch
- Self-loop guard: bridge_self failures don't count toward target-failure
thresholds (logged only) — wire to your own Telegram/Email/Matrix to
get notified when polls/dispatches/sends fail
- 6 default templates (3 events × 2 locales), tracking config columns
with backfill migration, frontend descriptor (excluded from "create
provider" wizard since auto-managed)
Operator-visible behavior changes (call out in release notes):
- NOTIFY_BRIDGE_TELEGRAM_WEBHOOK_SECRET now REQUIRED for webhook mode
- Existing webhook providers with auth_mode="none" need explicit opt-in
- Generic webhook endpoint rate-limited 60/min per source IP
- HA disconnect/reconnect writes ha_status_* EventLog rows
- Every user gets a bridge_self provider — wire it to a target to
receive failure alerts
Pre-existing test failures (test_ssrf, test_release_provider) on
Python 3.13 are unrelated; CI runs on 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
alexei.dolgolyov
|
ef942b77cc |
feat(telegram): per-chat command localization + unified locale resolver
Two related Telegram changes:
1. Per-chat command localization. setMyCommands now accepts a scope
(BotCommandScopeChat) and deleteMyCommands clears scoped bindings.
Command registration runs three tiers: default → per-language
(Telegram client language) → per-chat (UI override). Saving a
chat's language_override or commands_enabled toggle pushes the
binding to Telegram inline rather than waiting on the 30s
debounced bot-wide sync.
2. Unified Telegram locale resolution. Three test paths (bot test_chat,
target receiver test, target-level fan-out) used to disagree on
locale priority — the target receiver test in particular only
consulted receiver.locale and ignored the chat's language_override.
Introduced pick_telegram_locale (pure) and
resolve_telegram_chat_locale (async DB lookup) in services/notifier
so all three paths share one priority order:
receiver.locale → chat.language_override → chat.language_code → fallback
Fan-out keeps batch-loading TelegramChat rows for efficiency, just
runs them through the same priority function now.
|
||
|
alexei.dolgolyov
|
56993d2ca3 |
fix(security,perf): harden restore, CSRF, token_version + perf pass
Security
- Sign pending_restore.json (SHA256 stored in AppSetting, verified on
startup apply) + refuse path outside data_dir, tighten to 0600.
- Require same-origin Origin/Referer on POST /api/backup/apply-restart —
Bearer-in-localStorage is CSRF-reachable from any XSS'd admin tab.
- Bump token_version on role/username change and admin password reset so
demoted admins lose admin in already-issued JWTs. Guard last-admin
TOCTOU via COUNT + post-commit re-check that rolls back a race.
- SSRF guard (validate_outbound_url) in ImmichClient.__init__ and the
external_domain setter — admin-mutable URLs were bypassing the check
that webhook/slack/discord paths already used. Dev restart script now
sets NOTIFY_BRIDGE_ALLOW_PRIVATE_URLS=1 so homelab Immich still works.
- Redact + cap Immich error bodies to ~120 chars before they flow into
ActionExecution.error / EventLog.details (both UI-visible).
- Deny-list sensitive keys (api_key / token / secret / password /
authorization / cookie / ...) in template-context merges so a rogue
template can't exfiltrate provider creds via {{ api_key }}.
- Cap user-controlled Immich search params (query ≤256, person_ids ≤50,
size ≤100) so a Telegram listener can't DoS upstream.
- Stream upload reads with running byte counter + content-length precheck
instead of buffering the full body and then rejecting.
- Log Telegram parse_mode fallbacks instead of swallowing silently;
template escape bugs now surface in server logs.
- Rollback partial imports on pending-restore failure (error recorded on
a fresh session).
Performance
- Fix N+1 in _refresh_telegram_chat_titles: single IN query instead of
session.get per chat.
- Parallelize album + shared-link fetches in test_dispatch (asyncio.gather)
and per-receiver Telegram test sends in notifier (semaphore 5).
- Early-exit collect_scheduled_assets(limit=0) so the periodic-summary
test path skips full per-album filter/sample (was O(album_assets)).
- Emit explicit CREATE INDEX IF NOT EXISTS for event_log user_id /
action_id / provider_id so the first boot after upgrade isn't left
unindexed for the dashboard query.
- Add AbortController timeout (120s) to fetchAuth so uploads/downloads
don't hang indefinitely.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
alexei.dolgolyov
|
a7a2b4efa4 |
feat: large polish pass — UX fixes, per-chat scope, restore/backup, action events
Backend
- Per-chat album scope for Immich commands (search/latest/memory/...): new
allowed_album_ids on CommandTrackerListener, threaded listener/page kwargs
through ProviderCommandHandler.handle; PATCH listener-scope endpoint.
- /search and /find accept a trailing page number; Immich client search_smart
/ search_metadata take a page param.
- Immich person-asset lookup switched from removed GET /api/people/{id}/assets
to POST /api/search/metadata with personIds (fixes /person command and
auto_organize rules silently returning zero candidates on Immich 1.106+).
- Auto_organize rule now sets the target album's thumbnail to the first added
image when missing (falls back to any asset type); failures do not fail the
rule. add_assets_to_album surfaces the Immich error body on non-2xx.
- EventLog.user_id / action_id / action_name columns with defensive migration
+ backfill. Status query filters by user_id directly; Immich/webhook paths
emit user_id explicitly. action_runner writes an action_success/partial/
failed event on each non-dry-run.
- Dashboard DELETE /api/status/events (scoped to user_id) + rendering live
tracker/provider/action names via FK join with snapshot fallback.
- PATCH /api/users/{id} for username/role change with last-admin guard.
- Deletion protection returns structured {message, entity, blocked_by}
(ApiError carries .blockedBy; frontend opens BlockedByModal).
- Backup prepare-restore → AppSetting markers + atomic write of
pending_restore.json; lifespan hook applies on next startup and archives
under data/applied_restores/. apply-restart sends SIGTERM so the lifespan
shutdown runs; NOTIFY_BRIDGE_SUPERVISED env override gates the button.
Manual POST /api/backup/files (same format as scheduled).
- New periodic-summary test path reuses shared collect_scheduled_assets
(limit=0) so test and future production code go through one primitive.
- Per-receiver locale for Telegram test messages (resolves
TelegramChat.language_override per chat instead of applying the first
receiver's locale to everyone).
- Bounded concurrency (semaphores) in NotificationDispatcher._preload_asset_data
and _refresh_telegram_chat_titles; chat title sweep extended to 24h since
save_chat_from_webhook covers active chats opportunistically.
- Telegram poller detects the \"webhook is active\" 409 and auto-calls
deleteWebhook for bots whose DB update_mode is polling (throttled per bot).
- TelegramClient.get_chat added (CLAUDE.md rule 6); set_album_thumbnail added.
- Seeds: rename \"Default Commands\" → \"Default Immich Commands\";
track_assets_removed default False.
Frontend
- Global provider selector visible when there is only one provider.
- Clear-events button + i18n + ConfirmModal on the dashboard; new icons/
labels/filters/colors for action_success / action_partial / action_failed.
- Auto-select first available tracking/template/command/config + bot on
create forms (trackers, command-trackers, targets, template/command
configs).
- Telegram target disable_url_preview defaults to true.
- BlockedByModal wired into 8 deletion flows; fetchAuth helper for
multipart/binary calls (reuses api()'s refresh + ApiError mapping).
- Immich tracker 'Checking links' parallelised (concurrency cap 6).
- Backup page: pending-restore banner + Apply-now / Apply-later modal,
restarting overlay polling /api/health, manual 'Create backup' button.
- Command-trackers listener row gets an 'Edit album scope' modal with
inherit/explicit multiselect.
- Users page: Edit user modal (username + role).
- parseDate helper for consistent UTC date rendering.
Migrations / schema
- event_log: + user_id, action_id, action_name (+ backfill user_id from
notification_tracker).
- command_tracker_listener: + allowed_album_ids.
|
||
|
alexei.dolgolyov
|
b803d004e1 |
refactor: comprehensive codebase review — security, performance, quality, UX
Security: - Fix NUT protocol command injection (validate names against safe regex) - Enable Jinja2 autoescape=True to prevent HTML injection via external data - Add WebhookProviderConfig validation model Performance: - Shared aiohttp.ClientSession singleton (replaces 40+ per-request sessions) - Fix 4 N+1 queries with batch IN loads (poller, scheduler, memory, broadcast) - asyncio.gather for Gitea commands and notification dispatcher - Add DB indexes on NotificationTrackerState.tracker_id, CommandTrackerListener - LRU cache for compiled Jinja2 templates - Daily EventLog cleanup job (90-day retention) - 30s HTTP timeout on all external calls - GROUP BY for target type counts (replaces 7 sequential queries) Code quality: - Extract get_owned_entity() helper (replaces 11 duplicate functions) - Extract slot_helpers.py (load_slots, save_slots, render_template_preview) - Extract command_utils.py (tracker lookup, last event, collection IDs) - Extract http_session.py (shared session lifecycle) - Provider connection validation dedup (3x → 1 helper) - Command dispatch tables replacing if/elif chains - Album+links fetch helper (fetch_albums_with_links) - Provider dispatch polymorphism (list_provider_collections) - Immutable _enrich_assets (no longer mutates in-place) - Fix _format_assets return type + handler unpacking Frontend: - Fix 18+ hardcoded English strings → t() with new i18n keys (en + ru) - Mobile "More" nav panel with provider filter and search - Shared Button.svelte component (4 variants, 2 sizes) - Shared ErrorBanner.svelte component (8 pages updated) - SvelteKit goto() replacing window.location.href - Dashboard grid fixed for 4 cards, paginator opacity consistency Functionality: - max_instances=1 on scheduler jobs (prevents duplicate events) - Webhook provider in watcher (prevents error spam) - Fix stale SQLModel reference in poller - Gitea get_repo() direct API call |
||
|
alexei.dolgolyov
|
6e35926772 |
feat: default tracker configs, email validation, expandable target links
- Tracker now has default_tracking_config_id and default_template_config_id that apply to all linked targets unless overridden per-target - Dispatch falls back to tracker defaults when per-link configs are null - Email bot creation validates SMTP connection before saving - Email notifications sent as HTML (links render properly) - Linked target items are expandable: collapsed shows config CrossLinks, expanded shows config selectors; action buttons always visible - Fix email bot test button icon (mdiEmailSend → mdiSend) - Fix target type icons in LinkedTargetsSection for all types - Provider filter moved above search in sidebar |
||
|
alexei.dolgolyov
|
d4cb388c74 |
refactor: unify test dispatch with real NotificationDispatcher
- Route scheduled/memory test sends through the same NotificationDispatcher the watcher uses — identical template rendering, media handling, caching - Add preview_url field to MediaAsset (transcoded mid-size), separate from thumbnail_url (small) and full_url (original). Dispatcher prefers preview_url - Fix sendMediaGroup cache: extract file_ids from Telegram response and store via async_set_many so repeat sends use cached file_ids - Parallelize asset downloads in _send_media_group with asyncio.gather - Filter unprocessed assets (archived/trashed/offline/no-thumbhash) at album parse time in ImmichAlbumData.from_api_response - Extract shared asset_to_media + collect_scheduled_assets into asset_utils.py (single source for test dispatch and future real scheduler) - Respect tracking config filters: limit, asset_type, favorite_only, min_rating - Random asset sampling for scheduled sends - Memory mode: "On This Day" date filter (same month+day, previous year) - Skip dispatch when no matching assets found - Remove ~250 lines of duplicated send logic from notifier.py - Fix restart-backend.sh: proper env var export, Python path resolution, error log |
||
|
alexei.dolgolyov
|
d8ecb60073 |
feat: broadcast notification target + UX improvements
Add broadcast target type that fans out notifications to multiple child targets. Dispatch expands broadcast into children in load_link_data() — dispatcher stays unaware. Children can be toggled on/off via disabled_child_ids in config. Also: dashboard provider card smaller font for names, scroll-to-form on target edit, broadcast nav tab with counter, flag_modified fix for JSON column updates, CLAUDE.md nav tree docs. |
||
|
alexei.dolgolyov
|
751097b347 |
feat: comprehensive code review fixes + receivers-only architecture
Security:
- Refuse startup with default secret_key in production (was just logging)
- Settings endpoint now requires admin role
- Password validation on initial setup
- DOM-based HTML sanitizer replaces regex in template previews
- Add *.log to .gitignore
Performance & reliability:
- Token refresh deduplication prevents race condition on concurrent 401s
- Theme media query listener registered once (no leak)
- IconPicker uses $derived instead of function call per render
- Snackbar uses single-batch state update instead of while loop
- Replace 11 inline hover handlers with CSS :hover in layout
Architecture - receivers-only:
- Delivery endpoints (chat_id, email, url, room_id, topic) now stored
exclusively in TargetReceiver rows, never in target.config
- Migration extracts existing delivery fields to receiver rows
- Notifier and dispatcher remove all config fallbacks
- Frontend targets page shows receivers list per target with
add/remove/toggle/test per receiver
- Single-receiver test endpoint: POST /targets/{id}/receivers/{id}/test
Code quality:
- Extract AuthLayout.svelte from login/setup (150 lines CSS dedup)
- Split telegram-bots page (754→51 lines + 3 tab components)
- Split notification-trackers page (547→432 lines + 4 components)
- Deduplicate _send_reply into shared handler.send_reply()
- Add locale column to template models, replace name-based detection
- Fix delete_notification_tracker dead protection check
- Fix check_telegram_bot query (filter by type, remove bogus OR)
- Add graceful scheduler shutdown in lifespan
- Consistent /bots?tab=telegram URLs across all nav links
i18n:
- Error page, chat actions, target types, provider types internationalized
- All new receiver UI strings in EN + RU
|
||
|
alexei.dolgolyov
|
3e3a6f0777 |
feat: Discord/Slack/ntfy/Matrix targets, command templates, delete protection, email/matrix bots
- Discord, Slack, ntfy, Matrix notification target types with clients and dispatch - MatrixBot model + API + frontend in Bots tab - Command template system fully wired into all handler commands - Default command templates seeded (EN/RU, 14 slots each) - Command template editor with variables reference including child fields - Delete protection on all 10 entity types (409 with consumer details) - Provider type selector on template config forms - Target type selector as dropdown with all 7 types - Response template selector on command config form - CLAUDE.md: mandatory server restart rule, child properties rule |
||
|
alexei.dolgolyov
|
846d480d38 |
feat: provider-strict configs, slot-based templates, broadcast targets, email bots, command templates
Major architectural improvements: - Provider-type enforcement: configs validated against provider type at assignment - TemplateConfig migrated to slot-based pattern (TemplateSlot child table) - Broadcast targets: TargetReceiver child table for multi-receiver dispatch - EmailBot: first-class email sender entity with SMTP config, test connection - CommandTemplateConfig: generic slot-based command response templates - Provider capability registry: dynamic slot/event/command definitions per provider - CommandTracker play/pause button matches NotificationTracker style |
||
|
alexei.dolgolyov
|
03ec9b3c86 |
feat: telegram commands, app settings, bot polling, webhook handling, UI improvements
Adds telegram bot command system with 13 commands (search, latest, random, etc.), webhook/polling handlers, rate limiting, app settings page, and various UI/UX improvements across all entity pages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
alexei.dolgolyov
|
5015e378fe |
feat: test menu dropdown, split text/media messages, target settings, provider URL links
- Replace 3 test buttons with unified dropdown menu (basic/periodic/scheduled/memory) - Send text message first, then assets as reply (not combined caption+media) - Pass all target config settings to Telegram client (disable_url_preview, max_media, chunk_delay, etc.) - Real data test notifications for periodic/scheduled/memory (fetch from Immich) - Provider card URL is now a clickable hyperlink - Localized test type labels (EN/RU) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
alexei.dolgolyov
|
03c5c66eed |
feat: UX & notification improvements — icons, events, chat names, link validation, templates
- Show entity icons on all cards with fallback defaults (providers, trackers, targets, bots)
- Enrich EventLog with provider_name, tracker_name, assets_count; add DB migration
- Dashboard events: filtering (type, provider, search), sorting, pagination, dynamic page size
- Friendly chat names on telegram target cards (resolve from TelegramChat table)
- Test message button on bot chat items with locale-aware messages
- Album public link validation on tracker save with auto-create dialog
- Support albums without public links: conditional <a href> in templates
- Fetch shared links during poll, enrich events with public_url/protected_url
- Per-asset public_url in template context ({share_url}/photos/{asset_id})
- Common date/location detection: common_date + common_location context vars
- Dual date formats: date_format (datetime) + date_only_format (date only)
- Template clone button, HTML link rendering in template preview
- Fix Telegram asset download 401: pass x-api-key headers through client
- Fix provider external_url matching for API key scoping
- Fix event timestamp timezone (append Z suffix for UTC)
- Localize event filter controls, test messages (EN/RU)
- Template variable UI helpers updated with all new fields
- CLAUDE.md: template system sync rules documentation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
alexei.dolgolyov
|
91e5cd58e9 |
fix: comprehensive API/UI review — 26 bug fixes and improvements
Backend: - Scheduler lifecycle sync: create/update/delete tracker now syncs APScheduler jobs - Test-periodic/test-memory endpoints render actual Jinja2 templates with sample data - Cascade cleanup on tracker delete (TrackerState removed, EventLog nullified) - Fix user_id=0 FK violation for system-owned TemplateConfig (removed FK constraint) - Fix API key leak: only attach x-api-key header for internal provider URLs - Validate config ownership in tracker_targets create/update - Fix _response() double-emit of created_at in template/tracking configs - Add per-target-link test endpoints (test, test-periodic, test-memory) Frontend: - Fix orphaned provider on test exception in providers/new - Add submitting guard + disabled state to targets save button - Move test buttons from tracker card to per-target-link rows - Fix Svelte 5 async $state reactivity (spread reassignment for all Record mutations) - i18n for dashboard timeAgo and event type badges (EN + RU) - Add required attribute to chat select dropdown in targets - Fix font CSS vars to prioritize imported DM Sans / JetBrains Mono - Standardize empty states with centered icon + text across all 6 list pages - Add stagger-children animation class to all list containers - Fix slide transition duration consistency (200ms everywhere) - Standardize border-radius to rounded-md across all form inputs - Fix providers/new page structure (h2 + mb-8 spacing) - Fix tracker card action row overflow (flex-wrap justify-end) - JinjaEditor dark mode reactivity (recreate editor on theme change) - Add aria-labels to mobile nav items - Make ConfirmModal confirm button label/icon configurable - Remove double error reporting on providers page - Add telegram bot edit functionality (name editing via PUT) - i18n for External Domain label on provider forms Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |