Files
notify-bridge/packages/server/src/notify_bridge_server/database/models.py
T
alexei.dolgolyov 10d30fc956 feat: production readiness — security, perf, bug fixes, bridge self-monitoring
Comprehensive multi-area pass driven by a parallel 8-agent production
review. Frontend, backend, database, security, performance, operational,
plus a new self-monitoring feature.

## Critical fixes
- Planka webhook: reads bounded raw body (was NameError on every call)
- HA quiet hours: ha_state_changed/automation_triggered/service_called/
  event_fired added to deferrable set (were silently dropped)
- DNS-rebinding SSRF: PinnedResolver wired into shared aiohttp session
- Telegram inbound webhook: secret now mandatory (401 without)
- Generic webhook: auth_mode="none" requires explicit
  acknowledge_unauthenticated=true; per-IP rate limit 60/min
- svelte-check: 5 null-narrowing errors in EventDetailModal fixed
- Provider hardcoding: Immich-only block extracted to descriptor
  featureDiscoveryHint
- command_sync: snapshot+expunge bot before exiting AsyncSession

## Bug fixes
- notifier asyncio.gather(return_exceptions=True) — one bad chat no longer
  cancels peer sends
- NotificationDispatcher hoisted out of per-tracker loop
- Provider credential resolution unified across all 5 dispatch sites
- HA asyncio.shield now drains inner task on cancellation
- Provider construction switched from if/elif ladder to factory registry
- NUT first poll seeds silently (no spurious ups_on_battery)
- Quiet-hours gate: event-type-disabled now wins over deferral
- APScheduler drain job ID resolution upgraded to seconds
- HA on_status_change wired through to EventLog
- Webhook payload rollback failures now logged (not swallowed)
- Batched receivers/chats/bots in load_link_data (was per-target N+1)
- flag_modified on JSON column reassignments in deferred_dispatch

## Database
- UNIQUE indexes on service_provider.webhook_token,
  telegram_bot.webhook_path_id, partial UNIQUE on telegram_bot.bot_id,
  telegram_chat(bot_id, chat_id), notification_tracker_target unique link,
  partial UNIQUE on bridge_self provider per user
- Composite ix_event_log_user_event_type_created index
- save_chat_from_webhook switched to ON CONFLICT DO UPDATE
- ondelete=CASCADE on user-id FKs (model annotation; app-side cascade
  delete added for existing data)
- delete_notification_tracker converted from N+1 to bulk DELETE/UPDATE
- Module-level asyncio.Lock replaced with lazy _get_lock() pattern
- VACUUM INTO snapshot now PRAGMA integrity_check verified

## Performance
- Jinja2 template compilation LRU cached (lru_cache maxsize=512)
- Per-locale render cache in NotificationDispatcher (skips re-rendering
  identical content for receivers sharing a locale)
- Tracker list cached per provider_id with 5s TTL + explicit invalidation
  on tracker CRUD (relieves HA chat-bus rate query pressure)
- Nav-counts collapsed from 16 round-trips to single UNION ALL
- HA event_log: skip persisting empty assets_added/removed events

## Security hardening
- Mass-assignment guard on Action create/update; cron sub-minute reject
- Backup JSON depth/node-count cap (depth ≤ 10, nodes ≤ 100k)
- _sanitize_config extended to all JSON-typed fields on backup import
- Telegram _safe_get walks redirects manually with SSRF revalidation
- Bcrypt 72-byte password length cap with clear 422
- Webhook payload body redaction; sensitive substring set extended with
  oauth/client_secret/webhook_secret/csrf in both header filter and
  template extras filter

## Frontend
- 76 catch (err: any) sites converted to errMsg(err) helper
- globalProviderFilter: pure getter; reconciliation moved to one-time
  $effect in +layout
- Provider-filter binding: removed paired $effects + _syncingFilter flag,
  now one-way derived
- entity-cache: separate _refreshing flag for background re-fetches
- api.ts 401 handling: AuthRedirectError class + dedup _redirecting flag,
  goto() instead of window.location.href
- a11y: aria-expanded on mobile More, role=switch + aria-checked on
  Telegram bot toggles

## Tests & operations
- CI pytest gate added to .gitea/workflows/build.yml + release.yml
  (wheel-built install to dodge editable-install slowness)
- /api/ready upgraded to deep healthcheck (db SELECT 1, scheduler.running,
  HA supervisor presence) returning {ready, checks, errors, version}
- /api/metrics endpoint with prometheus_client (deferred_pending,
  event_log_total, dispatch_duration, poll_failures, send_failures)
- New OPERATIONS.md covering deploy, healthchecks, metrics, backup/restore
  procedures, log handling, common scenarios, upgrade flow
- New tests: test_bridge_self (11), test_gitea_parser (9),
  test_planka_parser (6), test_immich_change_detector (6),
  test_backup_roundtrip (1)

## New feature: bridge self-monitoring
- New bridge_self provider type — internal sink for bridge health events
- Three event types: bridge_self_poll_failures (consecutive tracker poll
  failures), bridge_self_deferred_backlog (pending count crosses
  threshold), bridge_self_target_failures (consecutive 5xx/network
  failures per target)
- Per-user thresholds (defaults: 3 / 100 / 5) configurable via the
  provider config form
- Auto-seeded on user create + /setup + boot backfill for existing users
- Anti-spam: counters reset after emission; backlog uses transition latch
- Self-loop guard: bridge_self failures don't count toward target-failure
  thresholds (logged only) — wire to your own Telegram/Email/Matrix to
  get notified when polls/dispatches/sends fail
- 6 default templates (3 events × 2 locales), tracking config columns
  with backfill migration, frontend descriptor (excluded from "create
  provider" wizard since auto-managed)

Operator-visible behavior changes (call out in release notes):
- NOTIFY_BRIDGE_TELEGRAM_WEBHOOK_SECRET now REQUIRED for webhook mode
- Existing webhook providers with auth_mode="none" need explicit opt-in
- Generic webhook endpoint rate-limited 60/min per source IP
- HA disconnect/reconnect writes ha_status_* EventLog rows
- Every user gets a bridge_self provider — wire it to a target to
  receive failure alerts

Pre-existing test failures (test_ssrf, test_release_provider) on
Python 3.13 are unrelated; CI runs on 3.12.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 02:16:49 +03:00

814 lines
31 KiB
Python

"""SQLModel database table definitions for Notify Bridge."""
from __future__ import annotations
from datetime import datetime, timezone
from typing import Any
from uuid import uuid4
from sqlalchemy import ForeignKey, Index, UniqueConstraint, Text
from sqlmodel import JSON, Column, Field, SQLModel
def _utcnow() -> datetime:
return datetime.now(timezone.utc)
class User(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
username: str = Field(index=True, unique=True)
hashed_password: str
role: str = Field(default="user")
token_version: int = Field(default=1)
created_at: datetime = Field(default_factory=_utcnow)
class ServiceProvider(SQLModel, table=True):
"""A service provider instance (e.g., an Immich server)."""
__tablename__ = "service_provider"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
type: str # ServiceProviderType value ("immich")
name: str
icon: str = Field(default="")
config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
# Webhook token is the shared secret embedded in inbound webhook URLs.
# Must be unique so a token uniquely identifies a provider; indexed so
# the webhook router does an O(log n) lookup on every inbound request.
webhook_token: str = Field(
default_factory=lambda: uuid4().hex,
unique=True,
index=True,
)
created_at: datetime = Field(default_factory=_utcnow)
class TelegramBot(SQLModel, table=True):
__tablename__ = "telegram_bot"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
name: str
token: str
icon: str = Field(default="")
bot_username: str = Field(default="")
# bot_id=0 is a sentinel meaning "Telegram has not yet returned a numeric
# ID for this bot" (i.e. token never validated). Multiple unverified bots
# may legitimately carry 0, so we only enforce uniqueness for non-sentinel
# values via a partial index added in migrate_uniqueness_constraints.
bot_id: int = Field(default=0, index=True)
# URL-path embedded in Telegram's setWebhook callback URL. Must be unique
# so the inbound dispatcher resolves a single bot per incoming request.
webhook_path_id: str = Field(
default_factory=lambda: uuid4().hex,
unique=True,
index=True,
)
update_mode: str = Field(default="none") # "none", "polling", or "webhook"
# NOTE: commands_config column remains in the DB for backward compat,
# but is no longer part of the SQLModel class. Data migrated to CommandConfig.
created_at: datetime = Field(default_factory=_utcnow)
class MatrixBot(SQLModel, table=True):
"""Matrix bot — homeserver connection for sending messages to rooms."""
__tablename__ = "matrix_bot"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
name: str
icon: str = Field(default="")
homeserver_url: str # e.g. https://matrix.org
access_token: str
display_name: str = Field(default="")
created_at: datetime = Field(default_factory=_utcnow)
class EmailBot(SQLModel, table=True):
"""Email sender — SMTP connection for sending email notifications."""
__tablename__ = "email_bot"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
name: str
icon: str = Field(default="")
email: str # From address
smtp_host: str
smtp_port: int = Field(default=587)
smtp_username: str = Field(default="")
smtp_password: str = Field(default="")
smtp_use_tls: bool = Field(default=True)
created_at: datetime = Field(default_factory=_utcnow)
class TelegramChat(SQLModel, table=True):
__tablename__ = "telegram_chat"
# (bot_id, chat_id) uniquely identifies a chat. The composite index is
# the access pattern for save_chat_from_webhook ON CONFLICT updates and
# for any "lookup by (bot, chat)" callers.
__table_args__ = (
UniqueConstraint("bot_id", "chat_id", name="uq_telegram_chat_bot_chat"),
Index("ix_telegram_chat_bot_chat", "bot_id", "chat_id"),
)
id: int | None = Field(default=None, primary_key=True)
bot_id: int = Field(foreign_key="telegram_bot.id")
chat_id: str
title: str = Field(default="")
chat_type: str = Field(default="private")
username: str = Field(default="")
language_code: str = Field(default="") # auto-detected from Telegram
language_override: str = Field(default="") # manual override set by user
commands_enabled: bool = Field(default=False)
discovered_at: datetime = Field(default_factory=_utcnow)
class TrackingConfig(SQLModel, table=True):
"""What events to track + scheduling rules. Tied to a provider type."""
__tablename__ = "tracking_config"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
provider_type: str # Must match provider's type
name: str
icon: str = Field(default="")
# Event-driven tracking
track_assets_added: bool = Field(default=True)
track_assets_removed: bool = Field(default=False)
track_collection_renamed: bool = Field(default=True)
track_collection_deleted: bool = Field(default=True)
track_sharing_changed: bool = Field(default=False)
# Gitea event tracking
track_push: bool = Field(default=True)
track_issue_opened: bool = Field(default=True)
track_issue_closed: bool = Field(default=True)
track_issue_commented: bool = Field(default=False)
track_pr_opened: bool = Field(default=True)
track_pr_closed: bool = Field(default=True)
track_pr_merged: bool = Field(default=True)
track_pr_commented: bool = Field(default=False)
track_release_published: bool = Field(default=True)
# Planka event tracking
track_card_created: bool = Field(default=True)
track_card_updated: bool = Field(default=False)
track_card_moved: bool = Field(default=True)
track_card_deleted: bool = Field(default=False)
track_card_commented: bool = Field(default=True)
track_comment_updated: bool = Field(default=False)
track_board_created: bool = Field(default=True)
track_board_updated: bool = Field(default=False)
track_board_deleted: bool = Field(default=True)
track_list_created: bool = Field(default=False)
track_list_updated: bool = Field(default=False)
track_list_deleted: bool = Field(default=False)
track_attachment_created: bool = Field(default=True)
track_card_label_added: bool = Field(default=False)
track_task_completed: bool = Field(default=True)
# Scheduler event tracking
track_scheduled_message: bool = Field(default=True)
# NUT (UPS) event tracking
track_ups_online: bool = Field(default=True)
track_ups_on_battery: bool = Field(default=True)
track_ups_low_battery: bool = Field(default=True)
track_ups_battery_restored: bool = Field(default=True)
track_ups_comms_lost: bool = Field(default=True)
track_ups_comms_restored: bool = Field(default=True)
track_ups_replace_battery: bool = Field(default=True)
track_ups_overload: bool = Field(default=True)
# Generic Webhook event tracking
track_webhook_received: bool = Field(default=True)
# Home Assistant event tracking
track_ha_state_changed: bool = Field(default=True)
track_ha_automation_triggered: bool = Field(default=False)
track_ha_service_called: bool = Field(default=False)
track_ha_event_fired: bool = Field(default=False)
# Bridge self-monitoring event tracking — defaults ON because the whole
# point of the provider is to alert on these conditions.
track_bridge_self_poll_failures: bool = Field(default=True)
track_bridge_self_deferred_backlog: bool = Field(default=True)
track_bridge_self_target_failures: bool = Field(default=True)
# Immich asset display
track_images: bool = Field(default=True)
track_videos: bool = Field(default=True)
notify_favorites_only: bool = Field(default=False)
# Asset display
include_tags: bool = Field(default=True)
include_asset_details: bool = Field(default=False)
max_assets_to_show: int = Field(default=10)
assets_order_by: str = Field(default="none")
assets_order: str = Field(default="descending")
# Periodic summary
periodic_enabled: bool = Field(default=False)
periodic_interval_days: int = Field(default=1)
periodic_start_date: str = Field(default="2025-01-01")
periodic_times: str = Field(default="12:00")
# Scheduled assets
scheduled_enabled: bool = Field(default=False)
scheduled_times: str = Field(default="09:00")
scheduled_collection_mode: str = Field(default="per_collection")
scheduled_limit: int = Field(default=10)
scheduled_favorite_only: bool = Field(default=False)
scheduled_asset_type: str = Field(default="all")
scheduled_min_rating: int = Field(default=0)
scheduled_order_by: str = Field(default="random")
scheduled_order: str = Field(default="descending")
# Memory mode
memory_enabled: bool = Field(default=False)
memory_source: str = Field(default="albums") # "albums" or "native"
memory_times: str = Field(default="09:00")
memory_collection_mode: str = Field(default="combined")
memory_limit: int = Field(default=10)
memory_favorite_only: bool = Field(default=False)
memory_asset_type: str = Field(default="all")
memory_min_rating: int = Field(default=0)
# Quiet hours — HH:MM strings interpreted in the app-level timezone
# (AppSetting "timezone"). Gated by quiet_hours_enabled so an empty window
# still represents "explicitly disabled" vs "not yet configured".
quiet_hours_enabled: bool = Field(default=False)
quiet_hours_start: str | None = Field(default=None)
quiet_hours_end: str | None = Field(default=None)
created_at: datetime = Field(default_factory=_utcnow)
class TemplateConfig(SQLModel, table=True):
"""Jinja2 message templates. Tied to a provider type.
Template content is stored in TemplateSlot child rows (one per slot).
"""
__tablename__ = "template_config"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(default=0) # 0 = system-owned (no FK to allow sentinel)
provider_type: str # Must match provider's type
name: str
description: str = Field(default="")
icon: str = Field(default="")
locale: str = Field(default="") # e.g. "en", "ru"; empty = unspecified
date_format: str = Field(default="%d.%m.%Y, %H:%M UTC")
date_only_format: str = Field(default="%d.%m.%Y")
created_at: datetime = Field(default_factory=_utcnow)
class TemplateSlot(SQLModel, table=True):
"""One Jinja2 template for a specific slot and locale within a TemplateConfig.
Slot names are provider-specific (e.g. 'message_assets_added' for Immich).
Each (config, slot, locale) triple holds a separate template.
"""
__tablename__ = "template_slot"
__table_args__ = (
UniqueConstraint("config_id", "slot_name", "locale", name="uq_template_slot_locale"),
)
id: int | None = Field(default=None, primary_key=True)
config_id: int = Field(
foreign_key="template_config.id",
index=True,
)
slot_name: str
locale: str = Field(default="en")
template: str = Field(default="", sa_column=Column(Text, default=""))
class NotificationTarget(SQLModel, table=True):
"""Where to send notifications. Pure delivery endpoint.
Target-level config holds connection/display settings (e.g. bot_token,
disable_url_preview). Actual delivery endpoints live in TargetReceiver rows.
"""
__tablename__ = "notification_target"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
type: str # "telegram", "webhook", "email", "discord", "slack", "ntfy", "matrix"
name: str
icon: str = Field(default="")
config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
chat_action: str | None = Field(default="typing") # e.g. "typing", "upload_photo"
created_at: datetime = Field(default_factory=_utcnow)
class TargetReceiver(SQLModel, table=True):
"""One delivery endpoint within a NotificationTarget (broadcast support).
For Telegram: config = {"chat_id": "12345"}
For Webhook: config = {"url": "https://...", "headers": {...}}
For Email: config = {"email": "user@example.com", "name": "..."}
"""
__tablename__ = "target_receiver"
__table_args__ = (
UniqueConstraint("target_id", "receiver_key", name="uq_target_receiver"),
)
id: int | None = Field(default=None, primary_key=True)
target_id: int = Field(
foreign_key="notification_target.id",
index=True,
)
name: str = Field(default="")
config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
receiver_key: str = Field(default="") # dedup key (e.g. chat_id, url, email)
locale: str = Field(default="") # e.g. "en", "ru"; empty = use target default
enabled: bool = Field(default=True)
created_at: datetime = Field(default_factory=_utcnow)
class NotificationTracker(SQLModel, table=True):
"""Watches a provider's collections for changes."""
__tablename__ = "notification_tracker"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
provider_id: int = Field(foreign_key="service_provider.id")
name: str
icon: str = Field(default="")
collection_ids: list[str] = Field(default_factory=list, sa_column=Column(JSON))
filters: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
scan_interval: int = Field(default=60)
# Cap on the adaptive-polling skip factor (see services/scheduler.py).
# None or 0 disables adaptive back-off entirely — every scheduled tick
# runs. Positive values (2..N) enable skipping up to (N-1) out of N ticks
# once the tracker has been idle long enough. Per-tracker so an operator
# can opt a latency-sensitive tracker out of the global heuristic.
adaptive_max_skip: int | None = Field(default=None)
default_tracking_config_id: int | None = Field(default=None, foreign_key="tracking_config.id")
default_template_config_id: int | None = Field(default=None, foreign_key="template_config.id")
enabled: bool = Field(default=True)
created_at: datetime = Field(default_factory=_utcnow)
class NotificationTrackerTarget(SQLModel, table=True):
"""Junction between NotificationTracker and NotificationTarget with per-link config."""
__tablename__ = "notification_tracker_target"
# A tracker should never link to the same target twice — duplicate links
# would deliver the same notification multiple times. Enforced at the DB
# level so concurrent inserts can't bypass an application-side check.
__table_args__ = (
UniqueConstraint(
"notification_tracker_id", "target_id",
name="uq_ntt_tracker_target",
),
)
id: int | None = Field(default=None, primary_key=True)
# Python attr stays as tracker_id for backward compat; DB column is notification_tracker_id
tracker_id: int = Field(
foreign_key="notification_tracker.id",
index=True,
sa_column_kwargs={"name": "notification_tracker_id"},
)
target_id: int = Field(
foreign_key="notification_target.id",
index=True,
)
tracking_config_id: int | None = Field(
default=None, foreign_key="tracking_config.id"
)
template_config_id: int | None = Field(
default=None, foreign_key="template_config.id"
)
enabled: bool = Field(default=True)
quiet_hours_start: str | None = None
quiet_hours_end: str | None = None
# NOTE: commands_config column remains in the DB for backward compat,
# but is no longer part of the SQLModel class. Data migrated to CommandConfig.
created_at: datetime = Field(default_factory=_utcnow)
class NotificationTrackerState(SQLModel, table=True):
"""Persisted state for change detection."""
__tablename__ = "notification_tracker_state"
id: int | None = Field(default=None, primary_key=True)
# Python attr stays as tracker_id for backward compat; DB column is notification_tracker_id
tracker_id: int = Field(
foreign_key="notification_tracker.id",
index=True,
sa_column_kwargs={"name": "notification_tracker_id"},
)
collection_id: str
collection_name: str = Field(default="")
shared: bool = Field(default=False)
asset_ids: list[str] = Field(default_factory=list, sa_column=Column(JSON))
pending_asset_ids: list[str] = Field(default_factory=list, sa_column=Column(JSON))
# Lightweight fingerprint ({updated_at, asset_count, shared, name, ...})
# captured from the provider's cheap meta probe. Letting this differ from
# the current provider response is what tells the watcher a full fetch is
# actually required — letting it match lets the watcher skip the big read.
meta_fingerprint: dict[str, Any] = Field(
default_factory=dict, sa_column=Column(JSON)
)
last_updated: datetime = Field(default_factory=_utcnow)
class CommandConfig(SQLModel, table=True):
"""Configuration for bot commands (e.g., which commands are enabled, rate limits)."""
__tablename__ = "command_config"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
provider_type: str
name: str
icon: str = Field(default="")
enabled_commands: list[str] = Field(default_factory=list, sa_column=Column(JSON))
locale: str = Field(default="en")
response_mode: str = Field(default="media") # "media" or "text"
default_count: int = Field(default=5)
rate_limits: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
command_template_config_id: int | None = Field(
default=None, foreign_key="command_template_config.id"
)
created_at: datetime = Field(default_factory=_utcnow)
class CommandTemplateConfig(SQLModel, table=True):
"""Jinja2 templates for command responses. Provider-specific via slots."""
__tablename__ = "command_template_config"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(default=0) # 0 = system-owned
provider_type: str
name: str
description: str = Field(default="")
icon: str = Field(default="")
locale: str = Field(default="") # e.g. "en", "ru"; empty = unspecified
created_at: datetime = Field(default_factory=_utcnow)
class CommandTemplateSlot(SQLModel, table=True):
"""One Jinja2 template for a specific command response slot and locale.
Slot names match command names (e.g. 'status', 'help', 'albums').
Description slots use 'desc_' prefix (e.g. 'desc_status', 'desc_help').
Each (config, slot, locale) triple holds a separate template.
"""
__tablename__ = "command_template_slot"
__table_args__ = (
UniqueConstraint("config_id", "slot_name", "locale", name="uq_cmd_slot_locale"),
)
id: int | None = Field(default=None, primary_key=True)
config_id: int = Field(
foreign_key="command_template_config.id",
index=True,
)
slot_name: str
locale: str = Field(default="en")
template: str = Field(default="", sa_column=Column(Text, default=""))
class CommandTracker(SQLModel, table=True):
"""Links a provider to a command config for interactive bot commands."""
__tablename__ = "command_tracker"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
provider_id: int = Field(foreign_key="service_provider.id")
command_config_id: int = Field(foreign_key="command_config.id")
name: str
icon: str = Field(default="")
enabled: bool = Field(default=True)
created_at: datetime = Field(default_factory=_utcnow)
class CommandTrackerListener(SQLModel, table=True):
"""Links a CommandTracker to a listener (e.g., a telegram bot chat)."""
__tablename__ = "command_tracker_listener"
__table_args__ = (
UniqueConstraint(
"command_tracker_id", "listener_type", "listener_id",
name="uq_command_tracker_listener",
),
)
id: int | None = Field(default=None, primary_key=True)
command_tracker_id: int = Field(
foreign_key="command_tracker.id",
index=True,
)
listener_type: str # e.g. "telegram_bot"
listener_id: int
# Optional per-chat album scope. None = inherit from tracker (use all).
# When set, only these album/collection ids are queryable from this chat.
allowed_album_ids: list[str] | None = Field(
default=None, sa_column=Column(JSON, nullable=True),
)
created_at: datetime = Field(default_factory=_utcnow)
class DeferredDispatch(SQLModel, table=True):
"""A dispatch held back by quiet hours, waiting for the window to end.
One row per ``(link, event_type, collection_id)`` for asset events — newly
arriving events for the same key coalesce into the existing row's
``event_payload`` (union of added/removed asset sets) instead of inserting
a duplicate row. Non-asset events (push, pr_opened, ups_*, …) get a fresh
row each time because they aren't logically cancellable.
At drain time the scheduler picks up rows where ``status='pending'`` and
``fire_at <= now``, re-resolves the link/target/config against current
state (so subsequent config edits apply), and dispatches.
"""
__tablename__ = "deferred_dispatch"
id: int | None = Field(default=None, primary_key=True)
user_id: int | None = Field(
default=None,
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=True,
index=True,
),
)
tracker_id: int = Field(foreign_key="notification_tracker.id", index=True)
# The specific link this deferral targets. On drain we re-fetch by ID; if
# the link was disabled or removed in the meantime we drop with a
# ``deferred_then_dropped`` log row instead of dispatching to nothing.
link_id: int = Field(
foreign_key="notification_tracker_target.id", index=True,
)
# The event_log row written when the event was first detected. The drain
# writes a follow-up event_log row referencing this id so the dashboard
# can show "delivered at HH:MM, originally detected at HH:MM".
#
# ``ondelete="SET NULL"`` matters because the daily ``_cleanup_old_events``
# job hard-deletes event_log rows past the retention horizon. Without
# SET NULL, an old pending DeferredDispatch row referencing an aging-out
# event_log row would either (a) prevent the delete with an FK violation
# under SQLite's enforced foreign_keys PRAGMA, or (b) leave a dangling
# reference on engines that don't enforce.
event_log_id: int | None = Field(
default=None,
sa_column=Column(
"event_log_id",
ForeignKey("event_log.id", ondelete="SET NULL"),
nullable=True,
index=True,
),
)
event_type: str = Field(index=True)
collection_id: str = Field(default="", index=True)
# ``dataclasses.asdict(ServiceEvent)`` with datetime/enum normalisation —
# round-tripped via the helpers in ``services.deferred_dispatch``.
event_payload: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
fire_at: datetime = Field(index=True)
# ``pending`` until the drain runs; then ``fired``, ``dropped`` (link
# gone / event-type disabled after defer), or ``cancelled`` (coalesced
# away by a counter-event).
status: str = Field(default="pending", index=True)
fired_at: datetime | None = Field(default=None)
created_at: datetime = Field(default_factory=_utcnow)
class EventLog(SQLModel, table=True):
"""Log of detected events."""
__tablename__ = "event_log"
id: int | None = Field(default=None, primary_key=True)
# Owner. Indexed for the dashboard events query. Nullable only because
# historical rows (pre-user_id column) may have no owner; new rows always
# set this directly. SET NULL on user delete preserves the audit trail
# while letting the user record itself be removed.
user_id: int | None = Field(
default=None,
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="SET NULL"),
nullable=True,
index=True,
),
)
# Python attr stays as tracker_id for backward compat; DB column is notification_tracker_id
tracker_id: int | None = Field(
default=None,
foreign_key="notification_tracker.id",
index=True,
sa_column_kwargs={"name": "notification_tracker_id"},
)
tracker_name: str = Field(default="")
# Links an event back to an Action when the event was emitted by the
# action runner (``event_type`` starts with ``action_``). Null for
# notification-tracker events.
action_id: int | None = Field(
default=None, foreign_key="action.id", index=True,
)
action_name: str = Field(default="")
# Bot command provenance. Populated when ``event_type`` starts with
# ``command_`` so the dashboard can render command activity alongside
# tracker and action events. NULL for non-command rows.
command_tracker_id: int | None = Field(
default=None, foreign_key="command_tracker.id", index=True,
)
command_tracker_name: str = Field(default="")
telegram_bot_id: int | None = Field(
default=None, foreign_key="telegram_bot.id", index=True,
)
bot_name: str = Field(default="")
# FK to service_provider with SET NULL so deleting a provider leaves
# historical event_log rows intact (provider_name preserves the label
# for display). The FK only takes effect on freshly created tables —
# SQLite cannot ALTER a constraint into an existing table without a
# rebuild, so application code in api/providers.delete_provider also
# nulls these explicitly. See migrate_eventlog_provider_fk.
provider_id: int | None = Field(
default=None,
sa_column=Column(
"provider_id",
ForeignKey("service_provider.id", ondelete="SET NULL"),
nullable=True,
index=True,
),
)
provider_name: str = Field(default="")
event_type: str = Field(index=True)
collection_id: str
collection_name: str
assets_count: int = Field(default=0)
details: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
created_at: datetime = Field(default_factory=_utcnow)
class Action(SQLModel, table=True):
"""A scheduled action that mutates an external service."""
__tablename__ = "action"
id: int | None = Field(default=None, primary_key=True)
user_id: int = Field(
sa_column=Column(
"user_id",
ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
)
provider_id: int = Field(foreign_key="service_provider.id")
name: str
icon: str = Field(default="")
action_type: str # e.g. "auto_organize"
config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
schedule_type: str = Field(default="interval") # "interval" or "cron"
schedule_interval: int = Field(default=3600) # seconds
schedule_cron: str = Field(default="")
enabled: bool = Field(default=False) # default disabled for safety
last_run_at: datetime | None = Field(default=None)
last_run_status: str = Field(default="") # "success", "partial", "failed", ""
created_at: datetime = Field(default_factory=_utcnow)
class ActionRule(SQLModel, table=True):
"""One rule within an Action. Executed in order."""
__tablename__ = "action_rule"
id: int | None = Field(default=None, primary_key=True)
action_id: int = Field(foreign_key="action.id", index=True)
name: str = Field(default="")
rule_config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
enabled: bool = Field(default=True)
order: int = Field(default=0)
created_at: datetime = Field(default_factory=_utcnow)
class ActionExecution(SQLModel, table=True):
"""Log of an action execution (scheduled, manual, or dry-run)."""
__tablename__ = "action_execution"
id: int | None = Field(default=None, primary_key=True)
action_id: int = Field(foreign_key="action.id", index=True)
started_at: datetime = Field(default_factory=_utcnow)
finished_at: datetime | None = Field(default=None)
status: str = Field(default="running") # "running", "success", "partial", "failed"
rules_processed: int = Field(default=0)
rules_succeeded: int = Field(default=0)
rules_failed: int = Field(default=0)
total_items_affected: int = Field(default=0)
summary: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
error: str = Field(default="")
trigger: str = Field(default="scheduled") # "scheduled", "manual", "dry_run"
class WebhookPayloadLog(SQLModel, table=True):
"""Log of incoming webhook payloads for debugging and replay."""
__tablename__ = "webhook_payload_log"
id: int | None = Field(default=None, primary_key=True)
provider_id: int = Field(foreign_key="service_provider.id", index=True)
method: str = Field(default="POST")
headers: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
body: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
status: str = Field(default="matched") # "matched" | "unmatched" | "error"
extracted_fields: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
error_message: str = Field(default="")
created_at: datetime = Field(default_factory=_utcnow)
class AppSetting(SQLModel, table=True):
"""Key-value app-level settings (admin-configurable)."""
__tablename__ = "app_setting"
key: str = Field(primary_key=True)
value: str = Field(default="")