Commit Graph

13 Commits

Author SHA1 Message Date
alexei.dolgolyov 56993d2ca3 fix(security,perf): harden restore, CSRF, token_version + perf pass
Security
- Sign pending_restore.json (SHA256 stored in AppSetting, verified on
  startup apply) + refuse path outside data_dir, tighten to 0600.
- Require same-origin Origin/Referer on POST /api/backup/apply-restart —
  Bearer-in-localStorage is CSRF-reachable from any XSS'd admin tab.
- Bump token_version on role/username change and admin password reset so
  demoted admins lose admin in already-issued JWTs.  Guard last-admin
  TOCTOU via COUNT + post-commit re-check that rolls back a race.
- SSRF guard (validate_outbound_url) in ImmichClient.__init__ and the
  external_domain setter — admin-mutable URLs were bypassing the check
  that webhook/slack/discord paths already used.  Dev restart script now
  sets NOTIFY_BRIDGE_ALLOW_PRIVATE_URLS=1 so homelab Immich still works.
- Redact + cap Immich error bodies to ~120 chars before they flow into
  ActionExecution.error / EventLog.details (both UI-visible).
- Deny-list sensitive keys (api_key / token / secret / password /
  authorization / cookie / ...) in template-context merges so a rogue
  template can't exfiltrate provider creds via {{ api_key }}.
- Cap user-controlled Immich search params (query ≤256, person_ids ≤50,
  size ≤100) so a Telegram listener can't DoS upstream.
- Stream upload reads with running byte counter + content-length precheck
  instead of buffering the full body and then rejecting.
- Log Telegram parse_mode fallbacks instead of swallowing silently;
  template escape bugs now surface in server logs.
- Rollback partial imports on pending-restore failure (error recorded on
  a fresh session).

Performance
- Fix N+1 in _refresh_telegram_chat_titles: single IN query instead of
  session.get per chat.
- Parallelize album + shared-link fetches in test_dispatch (asyncio.gather)
  and per-receiver Telegram test sends in notifier (semaphore 5).
- Early-exit collect_scheduled_assets(limit=0) so the periodic-summary
  test path skips full per-album filter/sample (was O(album_assets)).
- Emit explicit CREATE INDEX IF NOT EXISTS for event_log user_id /
  action_id / provider_id so the first boot after upgrade isn't left
  unindexed for the dashboard query.
- Add AbortController timeout (120s) to fetchAuth so uploads/downloads
  don't hang indefinitely.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 02:28:55 +03:00
alexei.dolgolyov a7a2b4efa4 feat: large polish pass — UX fixes, per-chat scope, restore/backup, action events
Backend
- Per-chat album scope for Immich commands (search/latest/memory/...): new
  allowed_album_ids on CommandTrackerListener, threaded listener/page kwargs
  through ProviderCommandHandler.handle; PATCH listener-scope endpoint.
- /search and /find accept a trailing page number; Immich client search_smart
  / search_metadata take a page param.
- Immich person-asset lookup switched from removed GET /api/people/{id}/assets
  to POST /api/search/metadata with personIds (fixes /person command and
  auto_organize rules silently returning zero candidates on Immich 1.106+).
- Auto_organize rule now sets the target album's thumbnail to the first added
  image when missing (falls back to any asset type); failures do not fail the
  rule. add_assets_to_album surfaces the Immich error body on non-2xx.
- EventLog.user_id / action_id / action_name columns with defensive migration
  + backfill. Status query filters by user_id directly; Immich/webhook paths
  emit user_id explicitly. action_runner writes an action_success/partial/
  failed event on each non-dry-run.
- Dashboard DELETE /api/status/events (scoped to user_id) + rendering live
  tracker/provider/action names via FK join with snapshot fallback.
- PATCH /api/users/{id} for username/role change with last-admin guard.
- Deletion protection returns structured {message, entity, blocked_by}
  (ApiError carries .blockedBy; frontend opens BlockedByModal).
- Backup prepare-restore → AppSetting markers + atomic write of
  pending_restore.json; lifespan hook applies on next startup and archives
  under data/applied_restores/. apply-restart sends SIGTERM so the lifespan
  shutdown runs; NOTIFY_BRIDGE_SUPERVISED env override gates the button.
  Manual POST /api/backup/files (same format as scheduled).
- New periodic-summary test path reuses shared collect_scheduled_assets
  (limit=0) so test and future production code go through one primitive.
- Per-receiver locale for Telegram test messages (resolves
  TelegramChat.language_override per chat instead of applying the first
  receiver's locale to everyone).
- Bounded concurrency (semaphores) in NotificationDispatcher._preload_asset_data
  and _refresh_telegram_chat_titles; chat title sweep extended to 24h since
  save_chat_from_webhook covers active chats opportunistically.
- Telegram poller detects the \"webhook is active\" 409 and auto-calls
  deleteWebhook for bots whose DB update_mode is polling (throttled per bot).
- TelegramClient.get_chat added (CLAUDE.md rule 6); set_album_thumbnail added.
- Seeds: rename \"Default Commands\" → \"Default Immich Commands\";
  track_assets_removed default False.

Frontend
- Global provider selector visible when there is only one provider.
- Clear-events button + i18n + ConfirmModal on the dashboard; new icons/
  labels/filters/colors for action_success / action_partial / action_failed.
- Auto-select first available tracking/template/command/config + bot on
  create forms (trackers, command-trackers, targets, template/command
  configs).
- Telegram target disable_url_preview defaults to true.
- BlockedByModal wired into 8 deletion flows; fetchAuth helper for
  multipart/binary calls (reuses api()'s refresh + ApiError mapping).
- Immich tracker 'Checking links' parallelised (concurrency cap 6).
- Backup page: pending-restore banner + Apply-now / Apply-later modal,
  restarting overlay polling /api/health, manual 'Create backup' button.
- Command-trackers listener row gets an 'Edit album scope' modal with
  inherit/explicit multiselect.
- Users page: Edit user modal (username + role).
- parseDate helper for consistent UTC date rendering.

Migrations / schema
- event_log: + user_id, action_id, action_name (+ backfill user_id from
  notification_tracker).
- command_tracker_listener: + allowed_album_ids.
2026-04-22 01:13:11 +03:00
alexei.dolgolyov b803d004e1 refactor: comprehensive codebase review — security, performance, quality, UX
Security:
- Fix NUT protocol command injection (validate names against safe regex)
- Enable Jinja2 autoescape=True to prevent HTML injection via external data
- Add WebhookProviderConfig validation model

Performance:
- Shared aiohttp.ClientSession singleton (replaces 40+ per-request sessions)
- Fix 4 N+1 queries with batch IN loads (poller, scheduler, memory, broadcast)
- asyncio.gather for Gitea commands and notification dispatcher
- Add DB indexes on NotificationTrackerState.tracker_id, CommandTrackerListener
- LRU cache for compiled Jinja2 templates
- Daily EventLog cleanup job (90-day retention)
- 30s HTTP timeout on all external calls
- GROUP BY for target type counts (replaces 7 sequential queries)

Code quality:
- Extract get_owned_entity() helper (replaces 11 duplicate functions)
- Extract slot_helpers.py (load_slots, save_slots, render_template_preview)
- Extract command_utils.py (tracker lookup, last event, collection IDs)
- Extract http_session.py (shared session lifecycle)
- Provider connection validation dedup (3x → 1 helper)
- Command dispatch tables replacing if/elif chains
- Album+links fetch helper (fetch_albums_with_links)
- Provider dispatch polymorphism (list_provider_collections)
- Immutable _enrich_assets (no longer mutates in-place)
- Fix _format_assets return type + handler unpacking

Frontend:
- Fix 18+ hardcoded English strings → t() with new i18n keys (en + ru)
- Mobile "More" nav panel with provider filter and search
- Shared Button.svelte component (4 variants, 2 sizes)
- Shared ErrorBanner.svelte component (8 pages updated)
- SvelteKit goto() replacing window.location.href
- Dashboard grid fixed for 4 cards, paginator opacity consistency

Functionality:
- max_instances=1 on scheduler jobs (prevents duplicate events)
- Webhook provider in watcher (prevents error spam)
- Fix stale SQLModel reference in poller
- Gitea get_repo() direct API call
2026-03-28 13:22:26 +03:00
alexei.dolgolyov 6e35926772 feat: default tracker configs, email validation, expandable target links
- Tracker now has default_tracking_config_id and default_template_config_id
  that apply to all linked targets unless overridden per-target
- Dispatch falls back to tracker defaults when per-link configs are null
- Email bot creation validates SMTP connection before saving
- Email notifications sent as HTML (links render properly)
- Linked target items are expandable: collapsed shows config CrossLinks,
  expanded shows config selectors; action buttons always visible
- Fix email bot test button icon (mdiEmailSend → mdiSend)
- Fix target type icons in LinkedTargetsSection for all types
- Provider filter moved above search in sidebar
2026-03-24 22:32:37 +03:00
alexei.dolgolyov d4cb388c74 refactor: unify test dispatch with real NotificationDispatcher
- Route scheduled/memory test sends through the same NotificationDispatcher
  the watcher uses — identical template rendering, media handling, caching
- Add preview_url field to MediaAsset (transcoded mid-size), separate from
  thumbnail_url (small) and full_url (original). Dispatcher prefers preview_url
- Fix sendMediaGroup cache: extract file_ids from Telegram response and store
  via async_set_many so repeat sends use cached file_ids
- Parallelize asset downloads in _send_media_group with asyncio.gather
- Filter unprocessed assets (archived/trashed/offline/no-thumbhash) at album
  parse time in ImmichAlbumData.from_api_response
- Extract shared asset_to_media + collect_scheduled_assets into asset_utils.py
  (single source for test dispatch and future real scheduler)
- Respect tracking config filters: limit, asset_type, favorite_only, min_rating
- Random asset sampling for scheduled sends
- Memory mode: "On This Day" date filter (same month+day, previous year)
- Skip dispatch when no matching assets found
- Remove ~250 lines of duplicated send logic from notifier.py
- Fix restart-backend.sh: proper env var export, Python path resolution, error log
2026-03-24 19:32:40 +03:00
alexei.dolgolyov d8ecb60073 feat: broadcast notification target + UX improvements
Add broadcast target type that fans out notifications to multiple
child targets. Dispatch expands broadcast into children in
load_link_data() — dispatcher stays unaware. Children can be
toggled on/off via disabled_child_ids in config.

Also: dashboard provider card smaller font for names, scroll-to-form
on target edit, broadcast nav tab with counter, flag_modified fix
for JSON column updates, CLAUDE.md nav tree docs.
2026-03-24 15:15:41 +03:00
alexei.dolgolyov 751097b347 feat: comprehensive code review fixes + receivers-only architecture
Security:
- Refuse startup with default secret_key in production (was just logging)
- Settings endpoint now requires admin role
- Password validation on initial setup
- DOM-based HTML sanitizer replaces regex in template previews
- Add *.log to .gitignore

Performance & reliability:
- Token refresh deduplication prevents race condition on concurrent 401s
- Theme media query listener registered once (no leak)
- IconPicker uses $derived instead of function call per render
- Snackbar uses single-batch state update instead of while loop
- Replace 11 inline hover handlers with CSS :hover in layout

Architecture - receivers-only:
- Delivery endpoints (chat_id, email, url, room_id, topic) now stored
  exclusively in TargetReceiver rows, never in target.config
- Migration extracts existing delivery fields to receiver rows
- Notifier and dispatcher remove all config fallbacks
- Frontend targets page shows receivers list per target with
  add/remove/toggle/test per receiver
- Single-receiver test endpoint: POST /targets/{id}/receivers/{id}/test

Code quality:
- Extract AuthLayout.svelte from login/setup (150 lines CSS dedup)
- Split telegram-bots page (754→51 lines + 3 tab components)
- Split notification-trackers page (547→432 lines + 4 components)
- Deduplicate _send_reply into shared handler.send_reply()
- Add locale column to template models, replace name-based detection
- Fix delete_notification_tracker dead protection check
- Fix check_telegram_bot query (filter by type, remove bogus OR)
- Add graceful scheduler shutdown in lifespan
- Consistent /bots?tab=telegram URLs across all nav links

i18n:
- Error page, chat actions, target types, provider types internationalized
- All new receiver UI strings in EN + RU
2026-03-22 02:19:31 +03:00
alexei.dolgolyov 3e3a6f0777 feat: Discord/Slack/ntfy/Matrix targets, command templates, delete protection, email/matrix bots
- Discord, Slack, ntfy, Matrix notification target types with clients and dispatch
- MatrixBot model + API + frontend in Bots tab
- Command template system fully wired into all handler commands
- Default command templates seeded (EN/RU, 14 slots each)
- Command template editor with variables reference including child fields
- Delete protection on all 10 entity types (409 with consumer details)
- Provider type selector on template config forms
- Target type selector as dropdown with all 7 types
- Response template selector on command config form
- CLAUDE.md: mandatory server restart rule, child properties rule
2026-03-21 20:36:12 +03:00
alexei.dolgolyov 846d480d38 feat: provider-strict configs, slot-based templates, broadcast targets, email bots, command templates
Major architectural improvements:
- Provider-type enforcement: configs validated against provider type at assignment
- TemplateConfig migrated to slot-based pattern (TemplateSlot child table)
- Broadcast targets: TargetReceiver child table for multi-receiver dispatch
- EmailBot: first-class email sender entity with SMTP config, test connection
- CommandTemplateConfig: generic slot-based command response templates
- Provider capability registry: dynamic slot/event/command definitions per provider
- CommandTracker play/pause button matches NotificationTracker style
2026-03-21 16:33:24 +03:00
alexei.dolgolyov 03ec9b3c86 feat: telegram commands, app settings, bot polling, webhook handling, UI improvements
Adds telegram bot command system with 13 commands (search, latest, random, etc.),
webhook/polling handlers, rate limiting, app settings page, and various UI/UX
improvements across all entity pages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 23:11:42 +03:00
alexei.dolgolyov 5015e378fe feat: test menu dropdown, split text/media messages, target settings, provider URL links
- Replace 3 test buttons with unified dropdown menu (basic/periodic/scheduled/memory)
- Send text message first, then assets as reply (not combined caption+media)
- Pass all target config settings to Telegram client (disable_url_preview, max_media, chunk_delay, etc.)
- Real data test notifications for periodic/scheduled/memory (fetch from Immich)
- Provider card URL is now a clickable hyperlink
- Localized test type labels (EN/RU)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 16:34:25 +03:00
alexei.dolgolyov 03c5c66eed feat: UX & notification improvements — icons, events, chat names, link validation, templates
- Show entity icons on all cards with fallback defaults (providers, trackers, targets, bots)
- Enrich EventLog with provider_name, tracker_name, assets_count; add DB migration
- Dashboard events: filtering (type, provider, search), sorting, pagination, dynamic page size
- Friendly chat names on telegram target cards (resolve from TelegramChat table)
- Test message button on bot chat items with locale-aware messages
- Album public link validation on tracker save with auto-create dialog
- Support albums without public links: conditional <a href> in templates
- Fetch shared links during poll, enrich events with public_url/protected_url
- Per-asset public_url in template context ({share_url}/photos/{asset_id})
- Common date/location detection: common_date + common_location context vars
- Dual date formats: date_format (datetime) + date_only_format (date only)
- Template clone button, HTML link rendering in template preview
- Fix Telegram asset download 401: pass x-api-key headers through client
- Fix provider external_url matching for API key scoping
- Fix event timestamp timezone (append Z suffix for UTC)
- Localize event filter controls, test messages (EN/RU)
- Template variable UI helpers updated with all new fields
- CLAUDE.md: template system sync rules documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 16:18:03 +03:00
alexei.dolgolyov 91e5cd58e9 fix: comprehensive API/UI review — 26 bug fixes and improvements
Backend:
- Scheduler lifecycle sync: create/update/delete tracker now syncs APScheduler jobs
- Test-periodic/test-memory endpoints render actual Jinja2 templates with sample data
- Cascade cleanup on tracker delete (TrackerState removed, EventLog nullified)
- Fix user_id=0 FK violation for system-owned TemplateConfig (removed FK constraint)
- Fix API key leak: only attach x-api-key header for internal provider URLs
- Validate config ownership in tracker_targets create/update
- Fix _response() double-emit of created_at in template/tracking configs
- Add per-target-link test endpoints (test, test-periodic, test-memory)

Frontend:
- Fix orphaned provider on test exception in providers/new
- Add submitting guard + disabled state to targets save button
- Move test buttons from tracker card to per-target-link rows
- Fix Svelte 5 async $state reactivity (spread reassignment for all Record mutations)
- i18n for dashboard timeAgo and event type badges (EN + RU)
- Add required attribute to chat select dropdown in targets
- Fix font CSS vars to prioritize imported DM Sans / JetBrains Mono
- Standardize empty states with centered icon + text across all 6 list pages
- Add stagger-children animation class to all list containers
- Fix slide transition duration consistency (200ms everywhere)
- Standardize border-radius to rounded-md across all form inputs
- Fix providers/new page structure (h2 + mb-8 spacing)
- Fix tracker card action row overflow (flex-wrap justify-end)
- JinjaEditor dark mode reactivity (recreate editor on theme change)
- Add aria-labels to mobile nav items
- Make ConfirmModal confirm button label/icon configurable
- Remove double error reporting on providers page
- Add telegram bot edit functionality (name editing via PUT)
- i18n for External Domain label on provider forms

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 14:26:20 +03:00