feat: large polish pass — UX fixes, per-chat scope, restore/backup, action events

Backend
- Per-chat album scope for Immich commands (search/latest/memory/...): new
  allowed_album_ids on CommandTrackerListener, threaded listener/page kwargs
  through ProviderCommandHandler.handle; PATCH listener-scope endpoint.
- /search and /find accept a trailing page number; Immich client search_smart
  / search_metadata take a page param.
- Immich person-asset lookup switched from removed GET /api/people/{id}/assets
  to POST /api/search/metadata with personIds (fixes /person command and
  auto_organize rules silently returning zero candidates on Immich 1.106+).
- Auto_organize rule now sets the target album's thumbnail to the first added
  image when missing (falls back to any asset type); failures do not fail the
  rule. add_assets_to_album surfaces the Immich error body on non-2xx.
- EventLog.user_id / action_id / action_name columns with defensive migration
  + backfill. Status query filters by user_id directly; Immich/webhook paths
  emit user_id explicitly. action_runner writes an action_success/partial/
  failed event on each non-dry-run.
- Dashboard DELETE /api/status/events (scoped to user_id) + rendering live
  tracker/provider/action names via FK join with snapshot fallback.
- PATCH /api/users/{id} for username/role change with last-admin guard.
- Deletion protection returns structured {message, entity, blocked_by}
  (ApiError carries .blockedBy; frontend opens BlockedByModal).
- Backup prepare-restore → AppSetting markers + atomic write of
  pending_restore.json; lifespan hook applies on next startup and archives
  under data/applied_restores/. apply-restart sends SIGTERM so the lifespan
  shutdown runs; NOTIFY_BRIDGE_SUPERVISED env override gates the button.
  Manual POST /api/backup/files (same format as scheduled).
- New periodic-summary test path reuses shared collect_scheduled_assets
  (limit=0) so test and future production code go through one primitive.
- Per-receiver locale for Telegram test messages (resolves
  TelegramChat.language_override per chat instead of applying the first
  receiver's locale to everyone).
- Bounded concurrency (semaphores) in NotificationDispatcher._preload_asset_data
  and _refresh_telegram_chat_titles; chat title sweep extended to 24h since
  save_chat_from_webhook covers active chats opportunistically.
- Telegram poller detects the \"webhook is active\" 409 and auto-calls
  deleteWebhook for bots whose DB update_mode is polling (throttled per bot).
- TelegramClient.get_chat added (CLAUDE.md rule 6); set_album_thumbnail added.
- Seeds: rename \"Default Commands\" → \"Default Immich Commands\";
  track_assets_removed default False.

Frontend
- Global provider selector visible when there is only one provider.
- Clear-events button + i18n + ConfirmModal on the dashboard; new icons/
  labels/filters/colors for action_success / action_partial / action_failed.
- Auto-select first available tracking/template/command/config + bot on
  create forms (trackers, command-trackers, targets, template/command
  configs).
- Telegram target disable_url_preview defaults to true.
- BlockedByModal wired into 8 deletion flows; fetchAuth helper for
  multipart/binary calls (reuses api()'s refresh + ApiError mapping).
- Immich tracker 'Checking links' parallelised (concurrency cap 6).
- Backup page: pending-restore banner + Apply-now / Apply-later modal,
  restarting overlay polling /api/health, manual 'Create backup' button.
- Command-trackers listener row gets an 'Edit album scope' modal with
  inherit/explicit multiselect.
- Users page: Edit user modal (username + role).
- parseDate helper for consistent UTC date rendering.

Migrations / schema
- event_log: + user_id, action_id, action_name (+ backfill user_id from
  notification_tracker).
- command_tracker_listener: + allowed_album_ids.
This commit is contained in:
2026-04-22 01:13:11 +03:00
parent b5ffab7ece
commit a7a2b4efa4
57 changed files with 2452 additions and 335 deletions
@@ -1,10 +1,13 @@
"""Configuration backup/restore API (admin only)."""
import asyncio
import json
import logging
import os
import signal
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, HTTPException, UploadFile, File, Query
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, UploadFile, File, Query
from fastapi.responses import JSONResponse
from sqlmodel.ext.asyncio.session import AsyncSession
@@ -16,10 +19,24 @@ from ..services.backup_schema import (
ALL_CATEGORIES, BackupCategory, BackupFile, ConflictMode, SecretsMode,
)
from ..services.backup_service import (
cleanup_old_backups, export_backup, import_backup, list_backup_files,
validate_backup,
cleanup_old_backups, export_backup, export_backup_to_file, import_backup,
list_backup_files, validate_backup,
)
# Pending-restore marker keys (single source of truth consumed at startup)
PENDING_RESTORE_PATH_KEY = "pending_restore_path"
PENDING_RESTORE_CONFLICT_KEY = "pending_restore_conflict_mode"
PENDING_RESTORE_UPLOADED_AT_KEY = "pending_restore_uploaded_at"
PENDING_RESTORE_UPLOADED_BY_KEY = "pending_restore_uploaded_by"
def _pending_restore_path():
return app_config.data_dir / "pending_restore.json"
def _applied_restores_dir():
return app_config.data_dir / "applied_restores"
_LOGGER = logging.getLogger(__name__)
router = APIRouter(prefix="/api/backup", tags=["backup"])
@@ -131,6 +148,188 @@ async def import_config(
return result.model_dump()
# ---------------------------------------------------------------------------
# Pending restore (prepare → apply on next restart)
# ---------------------------------------------------------------------------
async def _set_app_setting(session: AsyncSession, key: str, value: str) -> None:
row = await session.get(AppSetting, key)
if row:
row.value = value
else:
row = AppSetting(key=key, value=value)
session.add(row)
async def _clear_pending_restore_markers(session: AsyncSession) -> None:
for key in (
PENDING_RESTORE_PATH_KEY,
PENDING_RESTORE_CONFLICT_KEY,
PENDING_RESTORE_UPLOADED_AT_KEY,
PENDING_RESTORE_UPLOADED_BY_KEY,
):
row = await session.get(AppSetting, key)
if row:
await session.delete(row)
@router.post("/prepare-restore")
async def prepare_restore(
file: UploadFile = File(...),
conflict_mode: ConflictMode = Query(default=ConflictMode.SKIP),
user: User = Depends(require_admin),
session: AsyncSession = Depends(get_session),
):
"""Stage a backup for restore on next backend restart.
Validates the uploaded file, writes it to ``data/pending_restore.json``,
and persists marker settings so startup will apply it atomically.
"""
content = await file.read()
if len(content) > MAX_UPLOAD_SIZE:
raise HTTPException(status_code=400, detail="File too large (max 10 MB)")
try:
raw = json.loads(content)
except json.JSONDecodeError as e:
raise HTTPException(status_code=400, detail=f"Invalid JSON: {e}")
validation = validate_backup(raw)
if not validation.valid:
raise HTTPException(
status_code=400,
detail=f"Invalid backup: {'; '.join(validation.errors)}",
)
pending_path = _pending_restore_path()
pending_path.parent.mkdir(parents=True, exist_ok=True)
# Atomic write: write to tmp then rename, so a crash mid-write never
# leaves a truncated pending_restore.json that would break startup apply.
tmp_path = pending_path.with_suffix(pending_path.suffix + ".tmp")
tmp_path.write_text(json.dumps(raw), encoding="utf-8")
os.replace(tmp_path, pending_path)
now_iso = datetime.now(timezone.utc).isoformat()
await _set_app_setting(session, PENDING_RESTORE_PATH_KEY, str(pending_path))
await _set_app_setting(session, PENDING_RESTORE_CONFLICT_KEY, conflict_mode.value)
await _set_app_setting(session, PENDING_RESTORE_UPLOADED_AT_KEY, now_iso)
await _set_app_setting(session, PENDING_RESTORE_UPLOADED_BY_KEY, user.username)
await session.commit()
return {
"pending": True,
"uploaded_at": now_iso,
"uploaded_by": user.username,
"conflict_mode": conflict_mode.value,
"validation": validation.model_dump(),
"supervised": _is_supervised(),
}
@router.get("/pending-restore")
async def get_pending_restore(
user: User = Depends(require_admin),
session: AsyncSession = Depends(get_session),
):
"""Return current pending-restore state, or null if none."""
path_row = await session.get(AppSetting, PENDING_RESTORE_PATH_KEY)
if not path_row or not path_row.value:
return {"pending": False, "supervised": _is_supervised()}
conflict_row = await session.get(AppSetting, PENDING_RESTORE_CONFLICT_KEY)
uploaded_at_row = await session.get(AppSetting, PENDING_RESTORE_UPLOADED_AT_KEY)
uploaded_by_row = await session.get(AppSetting, PENDING_RESTORE_UPLOADED_BY_KEY)
return {
"pending": True,
"uploaded_at": uploaded_at_row.value if uploaded_at_row else None,
"uploaded_by": uploaded_by_row.value if uploaded_by_row else None,
"conflict_mode": (conflict_row.value if conflict_row else ConflictMode.SKIP.value),
"supervised": _is_supervised(),
}
@router.delete("/pending-restore")
async def cancel_pending_restore(
user: User = Depends(require_admin),
session: AsyncSession = Depends(get_session),
):
"""Cancel a pending restore."""
pending_path = _pending_restore_path()
if pending_path.exists():
pending_path.unlink()
await _clear_pending_restore_markers(session)
await session.commit()
return {"cancelled": True}
def _is_supervised() -> bool:
"""Heuristic: is this process managed by something that will respawn it?
Priority order:
1. Explicit operator override: ``NOTIFY_BRIDGE_SUPERVISED`` env var or
the ``supervised`` AppSetting (values: ``true``/``false``/``auto``).
``auto`` (or unset) falls through to the detection heuristic.
2. Heuristic: look at common container/service-manager env vars.
Used by the frontend to decide whether to offer "Restart now" — a bad
guess here is a foot-gun (process exits, stays dead), so err on the side
of false when unsure.
"""
override = os.environ.get("NOTIFY_BRIDGE_SUPERVISED", "").strip().lower()
if override in ("true", "1", "yes", "on"):
return True
if override in ("false", "0", "no", "off"):
return False
for var in ("CONTAINER", "DOCKER_CONTAINER", "KUBERNETES_SERVICE_HOST",
"INVOCATION_ID", "PM2_HOME"):
if os.environ.get(var):
return True
if os.path.exists("/.dockerenv"):
return True
return False
@router.post("/apply-restart")
async def apply_and_restart(
background_tasks: BackgroundTasks,
user: User = Depends(require_admin),
session: AsyncSession = Depends(get_session),
):
"""Trigger a graceful exit so the supervisor respawns and applies the pending restore.
Only allowed when a pending restore is staged AND the process is supervised.
"""
path_row = await session.get(AppSetting, PENDING_RESTORE_PATH_KEY)
if not path_row or not path_row.value:
raise HTTPException(status_code=409, detail="No pending restore to apply")
if not _is_supervised():
raise HTTPException(
status_code=409,
detail=(
"This process is not supervised. Restart the backend manually to apply "
"the pending restore, or use the Cancel button."
),
)
async def _shutdown_soon() -> None:
# Small delay so the HTTP response flushes before the signal fires.
await asyncio.sleep(0.5)
_LOGGER.warning("Admin triggered restart to apply pending restore")
# SIGTERM lets uvicorn run its normal graceful shutdown:
# drain in-flight requests, fire the lifespan shutdown hooks
# (close_http_session, scheduler.shutdown), then exit. The
# supervisor respawns, and startup applies the pending restore.
try:
os.kill(os.getpid(), signal.SIGTERM)
except Exception: # noqa: BLE001 — last-resort fallback on platforms that reject SIGTERM
_LOGGER.exception("SIGTERM delivery failed; falling back to os._exit")
os._exit(0)
background_tasks.add_task(_shutdown_soon)
return {"restart_requested": True}
# ---------------------------------------------------------------------------
# Scheduled backup settings
# ---------------------------------------------------------------------------
@@ -205,6 +404,37 @@ async def get_backup_files(
return list_backup_files(_backup_dir())
@router.post("/files")
async def create_manual_backup(
secrets_mode: SecretsMode = Query(default=SecretsMode.EXCLUDE),
user: User = Depends(require_admin),
session: AsyncSession = Depends(get_session),
):
"""Create a backup file in the backups directory (manual checkpoint).
Produces the same JSON format as scheduled backups, saved under
``data/backups/backup-<timestamp>.json``. Retention is managed by the
existing scheduled-backup settings (``backup_retention_count``).
"""
backup_dir = _backup_dir()
filepath = await export_backup_to_file(session, user.id, backup_dir, secrets_mode)
# Apply the same retention as scheduled backups if configured.
retention_row = await session.get(AppSetting, "backup_retention_count")
if retention_row and retention_row.value:
try:
retention = int(retention_row.value)
if retention > 0:
cleanup_old_backups(backup_dir, keep=retention)
except ValueError:
pass
stat = filepath.stat()
return {
"filename": filepath.name,
"size": stat.st_size,
"created_at": datetime.fromtimestamp(stat.st_mtime, tz=timezone.utc).isoformat(),
}
@router.get("/files/{filename}")
async def download_backup_file(
filename: str,
@@ -42,6 +42,11 @@ class CommandTrackerUpdate(BaseModel):
class ListenerCreate(BaseModel):
listener_type: str
listener_id: int
allowed_album_ids: list[str] | None = None
class ListenerUpdate(BaseModel):
allowed_album_ids: list[str] | None = None
# --- Command Tracker CRUD ---
@@ -299,6 +304,7 @@ async def add_listener(
command_tracker_id=tracker_id,
listener_type=body.listener_type,
listener_id=body.listener_id,
allowed_album_ids=body.allowed_album_ids,
)
session.add(listener)
await session.commit()
@@ -316,6 +322,30 @@ async def add_listener(
return await _listener_response(session, listener)
@router.patch("/{tracker_id}/listeners/{listener_id}")
async def update_listener(
tracker_id: int,
listener_id: int,
body: ListenerUpdate,
user: User = Depends(get_current_user),
session: AsyncSession = Depends(get_session),
):
"""Update a listener's per-chat settings (currently just allowed_album_ids)."""
await _get_user_tracker(session, tracker_id, user.id)
listener = await session.get(CommandTrackerListener, listener_id)
if not listener or listener.command_tracker_id != tracker_id:
raise HTTPException(status_code=404, detail="Listener not found")
# Empty list means "no albums" which is rarely useful; treat as null (inherit).
if body.allowed_album_ids is not None and len(body.allowed_album_ids) == 0:
listener.allowed_album_ids = None
else:
listener.allowed_album_ids = body.allowed_album_ids
session.add(listener)
await session.commit()
await session.refresh(listener)
return await _listener_response(session, listener)
@router.delete("/{tracker_id}/listeners/{listener_id}", status_code=status.HTTP_204_NO_CONTENT)
async def remove_listener(
tracker_id: int,
@@ -394,6 +424,7 @@ async def _listener_response(session: AsyncSession, l: CommandTrackerListener) -
"command_tracker_id": l.command_tracker_id,
"listener_type": l.listener_type,
"listener_id": l.listener_id,
"allowed_album_ids": l.allowed_album_ids,
"name": name,
"created_at": l.created_at.isoformat(),
}
@@ -19,10 +19,22 @@ from ..database.models import (
def raise_if_used(consumers: list[str], entity_name: str) -> None:
"""Raise 409 Conflict if the entity has consumers."""
"""Raise 409 Conflict if the entity has consumers.
Produces a human-readable summary string (kept as the primary ``detail``)
plus a structured ``blocked_by`` list so the frontend can render a
clickable warning modal.
"""
if consumers:
detail = f"Cannot delete {entity_name}: used by {len(consumers)} consumer(s). " + "; ".join(consumers)
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=detail)
summary = f"Cannot delete {entity_name}: used by {len(consumers)} consumer(s)."
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail={
"message": summary,
"entity": entity_name,
"blocked_by": consumers,
},
)
async def check_service_provider(session: AsyncSession, provider_id: int) -> list[str]:
@@ -3,12 +3,14 @@
from datetime import datetime, timedelta, timezone
from fastapi import APIRouter, Depends, Query
from sqlalchemy import delete as sa_delete
from sqlmodel import func, select
from sqlmodel.ext.asyncio.session import AsyncSession
from ..auth.dependencies import get_current_user
from ..database.engine import get_session
from ..database.models import (
Action,
CommandConfig,
CommandTemplateConfig,
CommandTracker,
@@ -54,12 +56,10 @@ async def get_status(
select(func.count()).select_from(NotificationTarget).where(NotificationTarget.user_id == user.id)
)).one()
# Build events query with filters
events_query = (
select(EventLog)
.join(NotificationTracker, EventLog.tracker_id == NotificationTracker.id)
.where(NotificationTracker.user_id == user.id)
)
# Build events query with filters. EventLog.user_id is the owner column;
# action events (event_type starts with "action_") have tracker_id NULL but
# user_id set, so we filter by user_id directly.
events_query = select(EventLog).where(EventLog.user_id == user.id)
if event_type:
events_query = events_query.where(EventLog.event_type == event_type)
@@ -69,6 +69,7 @@ async def get_status(
events_query = events_query.where(
EventLog.collection_name.contains(search)
| EventLog.tracker_name.contains(search)
| EventLog.action_name.contains(search)
| EventLog.provider_name.contains(search)
)
@@ -84,6 +85,65 @@ async def get_status(
events_query = events_query.offset(offset).limit(limit)
recent_events = await session.exec(events_query)
event_rows = recent_events.all()
# Resolve live tracker names from FK (fall back to stored snapshot when deleted)
tracker_ids = {e.tracker_id for e in event_rows if e.tracker_id is not None}
tracker_name_map: dict[int, str] = {}
if tracker_ids:
tracker_rows = (await session.exec(
select(NotificationTracker.id, NotificationTracker.name).where(
NotificationTracker.id.in_(tracker_ids)
)
)).all()
tracker_name_map = {tid: tname for tid, tname in tracker_rows}
# Resolve live provider names similarly
provider_ids = {e.provider_id for e in event_rows if e.provider_id is not None}
provider_name_map: dict[int, str] = {}
if provider_ids:
provider_rows = (await session.exec(
select(ServiceProvider.id, ServiceProvider.name).where(
ServiceProvider.id.in_(provider_ids)
)
)).all()
provider_name_map = {pid: pname for pid, pname in provider_rows}
# Resolve live action names so renames are reflected; fall back to snapshot.
action_ids = {e.action_id for e in event_rows if e.action_id is not None}
action_name_map: dict[int, str] = {}
if action_ids:
action_rows = (await session.exec(
select(Action.id, Action.name).where(Action.id.in_(action_ids))
)).all()
action_name_map = {aid: aname for aid, aname in action_rows}
def _display_tracker_name(e: EventLog) -> str:
if e.tracker_id is not None and e.tracker_id in tracker_name_map:
return tracker_name_map[e.tracker_id]
return f"(deleted) {e.tracker_name}" if e.tracker_name else "(deleted)"
def _display_provider_name(e: EventLog) -> str:
if e.provider_id is not None and e.provider_id in provider_name_map:
return provider_name_map[e.provider_id]
return e.provider_name or ""
def _display_action_name(e: EventLog) -> str:
if e.action_id is not None and e.action_id in action_name_map:
return action_name_map[e.action_id]
if e.action_name:
return f"(deleted) {e.action_name}"
return ""
def _display_subject(e: EventLog) -> str:
"""The primary label shown on the event row.
For action events the ``collection_name`` stores the action name;
use the live-resolved action name when available so renames show.
"""
if e.action_id is not None or (e.event_type or "").startswith("action_"):
return _display_action_name(e) or e.collection_name
return e.collection_name
return {
"providers": providers_count,
@@ -94,19 +154,43 @@ async def get_status(
{
"id": e.id,
"event_type": e.event_type,
"collection_name": e.collection_name,
"tracker_name": e.tracker_name or "",
"provider_name": e.provider_name or "",
"collection_name": _display_subject(e),
"tracker_name": _display_tracker_name(e),
"action_id": e.action_id,
"action_name": _display_action_name(e),
"provider_name": _display_provider_name(e),
"provider_id": e.provider_id,
"assets_count": e.assets_count or 0,
"created_at": e.created_at.isoformat() + ("Z" if not e.created_at.tzinfo else ""),
"details": e.details or {},
}
for e in recent_events.all()
for e in event_rows
],
}
@router.delete("/events")
async def clear_events(
user: User = Depends(get_current_user),
session: AsyncSession = Depends(get_session),
older_than_days: int | None = Query(None, ge=0),
):
"""Delete all event log entries for the current user.
Optionally keep events newer than `older_than_days` days.
"""
stmt = sa_delete(EventLog).where(EventLog.user_id == user.id)
if older_than_days is not None:
cutoff = datetime.now(timezone.utc) - timedelta(days=older_than_days)
stmt = stmt.where(EventLog.created_at < cutoff)
# Use session.execute() for DELETE (consistent with other endpoints and
# avoids sqlmodel wrapping a CursorResult that may drop rowcount).
result = await session.execute(stmt)
await session.commit()
return {"deleted": result.rowcount or 0}
@router.get("/counts")
async def get_nav_counts(
user: User = Depends(get_current_user),
@@ -192,8 +276,7 @@ async def get_event_chart(
EventLog.event_type,
func.count().label("total"),
)
.join(NotificationTracker, EventLog.tracker_id == NotificationTracker.id)
.where(NotificationTracker.user_id == user.id, EventLog.created_at >= cutoff)
.where(EventLog.user_id == user.id, EventLog.created_at >= cutoff)
)
if event_type:
@@ -204,6 +287,7 @@ async def get_event_chart(
query = query.where(
EventLog.collection_name.contains(search)
| EventLog.tracker_name.contains(search)
| EventLog.action_name.contains(search)
| EventLog.provider_name.contains(search)
)
@@ -162,8 +162,9 @@ async def get_template_variables(
"city": "City name",
"state": "State/region name",
"country": "Country name",
"file_size": "File size in bytes (null if unknown)",
"oversized": "Whether video exceeds the target's size limit (boolean, videos only)",
"file_size": "Original asset size in bytes (null if unknown)",
"playback_size": "Size in bytes of the media we actually upload — for Immich videos this is the transcoded /video/playback (null for photos or when unknown)",
"oversized": "Whether the asset's playback_size exceeds the target's size limit (boolean, videos only)",
"public_url": "Per-asset public share URL (empty if no album link)",
"url": "Public viewer URL (if shared)",
"download_url": "Direct download URL (if shared)",
@@ -69,6 +69,45 @@ async def create_user(
return {"id": user.id, "username": user.username, "role": user.role}
@router.patch("/{user_id}")
async def update_user(
user_id: int,
body: UserUpdate,
admin: User = Depends(require_admin),
session: AsyncSession = Depends(get_session),
):
"""Update username and/or role for a user (admin only)."""
user = await session.get(User, user_id)
if not user:
raise HTTPException(status_code=404, detail="User not found")
if body.username is not None and body.username != user.username:
new_username = body.username.strip()
if not new_username:
raise HTTPException(status_code=400, detail="Username cannot be empty")
dup = await session.exec(select(User).where(User.username == new_username))
if dup.first():
raise HTTPException(status_code=409, detail="Username already exists")
user.username = new_username
if body.role is not None and body.role != user.role:
if body.role not in ("admin", "user"):
raise HTTPException(status_code=400, detail="Invalid role")
# Prevent demoting the last admin
if user.role == "admin" and body.role != "admin":
admins = (await session.exec(
select(User).where(User.role == "admin")
)).all()
if len(admins) <= 1:
raise HTTPException(status_code=400, detail="Cannot demote the last admin")
user.role = body.role
session.add(user)
await session.commit()
await session.refresh(user)
return {"id": user.id, "username": user.username, "role": user.role}
class ResetPasswordRequest(BaseModel):
new_password: str
@@ -147,6 +147,7 @@ async def _dispatch_webhook_event(
# Log event
extra_details = {k: v for k, v in event.extra.items() if k in detail_keys}
session.add(EventLog(
user_id=tracker.user_id,
tracker_id=tracker.id,
tracker_name=tracker.name,
provider_id=provider_id,