perf(immich): skip full album fetch on idle ticks; delta-fetch for active ones

Optimizes polling for large Immich albums (tested path targets ~200k
assets). Combined impact on idle albums drops per-tick cost from ~150 MB
fetch to ~few hundred bytes; active albums fetch O(changes) instead of
O(library).

Core changes
- ImmichAlbumMeta + get_album_meta() using ?withoutAssets=true as a
  cheap change-detection probe.
- poll() fast-path: skip full fetch when meta fingerprint matches and
  no pending assets are outstanding.
- poll() delta-path: search/metadata with updatedAfter when fingerprint
  changed, falling back to full fetch on count decrease or mixed
  add+remove that delta can't reconcile.
- asyncio.gather over meta probes so a 20-album tracker pays one
  round-trip of latency instead of 20.
- Event payload cap (50 added / 200 removed) so a bulk import can't
  explode a Jinja template or exceed Telegram's message limits.
- Module-level users cache (1h TTL, sha256-keyed) shared across
  providers on the same Immich server.
- Tick-scoped shared-links cache via new
  get_all_shared_links_by_album() — one /api/shared-links request per
  tick instead of one per changed album.

Server changes
- meta_fingerprint JSON column on NotificationTrackerState + migration.
- watcher skips the asset_ids DB rewrite when the fingerprint didn't
  change, avoiding ~8 MB JSON writes on idle ticks for huge albums.
- Adaptive polling: after 10 empty ticks skip 1-in-2, after 30 skip
  1-in-4, reset on first detected change; resets on schedule changes.
- APScheduler jitter (interval/4, capped at 30s) to smooth thundering-
  herd bursts when many trackers share the same scan_interval.
This commit is contained in:
2026-04-22 18:55:26 +03:00
parent d02616069d
commit fe38d20b96
8 changed files with 796 additions and 40 deletions
@@ -187,8 +187,17 @@ async def check_tracker(tracker_id: int) -> dict[str, Any]:
"asset_ids": s.asset_ids,
"pending_asset_ids": s.pending_asset_ids,
"shared": bool(s.shared),
"meta_fingerprint": s.meta_fingerprint or {},
}
# Snapshot the original fingerprint per collection so we can skip the
# (expensive) asset_ids rewrite when nothing changed. For a 200k-asset
# album this avoids a ~7 MB JSON write to the state row every tick.
original_fingerprints: dict[str, dict[str, Any]] = {
cid: dict(cstate.get("meta_fingerprint") or {})
for cid, cstate in state_dict.items()
}
# Load tracker-target links
link_data = await load_link_data(session, tracker_id)
@@ -279,11 +288,20 @@ async def check_tracker(tracker_id: int) -> dict[str, Any]:
existing = s
break
current_fingerprint = dict(cstate.get("meta_fingerprint") or {})
prior_fingerprint = original_fingerprints.get(cid, {})
# Skip the DB update when the provider reported no meaningful
# change. ``existing`` is None on first-ever fetch for a
# collection — that path always writes so the row gets created.
if existing is not None and current_fingerprint == prior_fingerprint:
continue
if existing:
existing.asset_ids = cstate.get("asset_ids", [])
existing.pending_asset_ids = cstate.get("pending_asset_ids", [])
existing.collection_name = cstate.get("name", "")
existing.shared = cstate.get("shared", False)
existing.meta_fingerprint = current_fingerprint
session.add(existing)
else:
new_ts = NotificationTrackerState(
@@ -293,6 +311,7 @@ async def check_tracker(tracker_id: int) -> dict[str, Any]:
shared=cstate.get("shared", False),
asset_ids=cstate.get("asset_ids", []),
pending_asset_ids=cstate.get("pending_asset_ids", []),
meta_fingerprint=current_fingerprint,
)
session.add(new_ts)