10d30fc956
Comprehensive multi-area pass driven by a parallel 8-agent production
review. Frontend, backend, database, security, performance, operational,
plus a new self-monitoring feature.
## Critical fixes
- Planka webhook: reads bounded raw body (was NameError on every call)
- HA quiet hours: ha_state_changed/automation_triggered/service_called/
event_fired added to deferrable set (were silently dropped)
- DNS-rebinding SSRF: PinnedResolver wired into shared aiohttp session
- Telegram inbound webhook: secret now mandatory (401 without)
- Generic webhook: auth_mode="none" requires explicit
acknowledge_unauthenticated=true; per-IP rate limit 60/min
- svelte-check: 5 null-narrowing errors in EventDetailModal fixed
- Provider hardcoding: Immich-only block extracted to descriptor
featureDiscoveryHint
- command_sync: snapshot+expunge bot before exiting AsyncSession
## Bug fixes
- notifier asyncio.gather(return_exceptions=True) — one bad chat no longer
cancels peer sends
- NotificationDispatcher hoisted out of per-tracker loop
- Provider credential resolution unified across all 5 dispatch sites
- HA asyncio.shield now drains inner task on cancellation
- Provider construction switched from if/elif ladder to factory registry
- NUT first poll seeds silently (no spurious ups_on_battery)
- Quiet-hours gate: event-type-disabled now wins over deferral
- APScheduler drain job ID resolution upgraded to seconds
- HA on_status_change wired through to EventLog
- Webhook payload rollback failures now logged (not swallowed)
- Batched receivers/chats/bots in load_link_data (was per-target N+1)
- flag_modified on JSON column reassignments in deferred_dispatch
## Database
- UNIQUE indexes on service_provider.webhook_token,
telegram_bot.webhook_path_id, partial UNIQUE on telegram_bot.bot_id,
telegram_chat(bot_id, chat_id), notification_tracker_target unique link,
partial UNIQUE on bridge_self provider per user
- Composite ix_event_log_user_event_type_created index
- save_chat_from_webhook switched to ON CONFLICT DO UPDATE
- ondelete=CASCADE on user-id FKs (model annotation; app-side cascade
delete added for existing data)
- delete_notification_tracker converted from N+1 to bulk DELETE/UPDATE
- Module-level asyncio.Lock replaced with lazy _get_lock() pattern
- VACUUM INTO snapshot now PRAGMA integrity_check verified
## Performance
- Jinja2 template compilation LRU cached (lru_cache maxsize=512)
- Per-locale render cache in NotificationDispatcher (skips re-rendering
identical content for receivers sharing a locale)
- Tracker list cached per provider_id with 5s TTL + explicit invalidation
on tracker CRUD (relieves HA chat-bus rate query pressure)
- Nav-counts collapsed from 16 round-trips to single UNION ALL
- HA event_log: skip persisting empty assets_added/removed events
## Security hardening
- Mass-assignment guard on Action create/update; cron sub-minute reject
- Backup JSON depth/node-count cap (depth ≤ 10, nodes ≤ 100k)
- _sanitize_config extended to all JSON-typed fields on backup import
- Telegram _safe_get walks redirects manually with SSRF revalidation
- Bcrypt 72-byte password length cap with clear 422
- Webhook payload body redaction; sensitive substring set extended with
oauth/client_secret/webhook_secret/csrf in both header filter and
template extras filter
## Frontend
- 76 catch (err: any) sites converted to errMsg(err) helper
- globalProviderFilter: pure getter; reconciliation moved to one-time
$effect in +layout
- Provider-filter binding: removed paired $effects + _syncingFilter flag,
now one-way derived
- entity-cache: separate _refreshing flag for background re-fetches
- api.ts 401 handling: AuthRedirectError class + dedup _redirecting flag,
goto() instead of window.location.href
- a11y: aria-expanded on mobile More, role=switch + aria-checked on
Telegram bot toggles
## Tests & operations
- CI pytest gate added to .gitea/workflows/build.yml + release.yml
(wheel-built install to dodge editable-install slowness)
- /api/ready upgraded to deep healthcheck (db SELECT 1, scheduler.running,
HA supervisor presence) returning {ready, checks, errors, version}
- /api/metrics endpoint with prometheus_client (deferred_pending,
event_log_total, dispatch_duration, poll_failures, send_failures)
- New OPERATIONS.md covering deploy, healthchecks, metrics, backup/restore
procedures, log handling, common scenarios, upgrade flow
- New tests: test_bridge_self (11), test_gitea_parser (9),
test_planka_parser (6), test_immich_change_detector (6),
test_backup_roundtrip (1)
## New feature: bridge self-monitoring
- New bridge_self provider type — internal sink for bridge health events
- Three event types: bridge_self_poll_failures (consecutive tracker poll
failures), bridge_self_deferred_backlog (pending count crosses
threshold), bridge_self_target_failures (consecutive 5xx/network
failures per target)
- Per-user thresholds (defaults: 3 / 100 / 5) configurable via the
provider config form
- Auto-seeded on user create + /setup + boot backfill for existing users
- Anti-spam: counters reset after emission; backlog uses transition latch
- Self-loop guard: bridge_self failures don't count toward target-failure
thresholds (logged only) — wire to your own Telegram/Email/Matrix to
get notified when polls/dispatches/sends fail
- 6 default templates (3 events × 2 locales), tracking config columns
with backfill migration, frontend descriptor (excluded from "create
provider" wizard since auto-managed)
Operator-visible behavior changes (call out in release notes):
- NOTIFY_BRIDGE_TELEGRAM_WEBHOOK_SECRET now REQUIRED for webhook mode
- Existing webhook providers with auth_mode="none" need explicit opt-in
- Generic webhook endpoint rate-limited 60/min per source IP
- HA disconnect/reconnect writes ha_status_* EventLog rows
- Every user gets a bridge_self provider — wire it to a target to
receive failure alerts
Pre-existing test failures (test_ssrf, test_release_provider) on
Python 3.13 are unrelated; CI runs on 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
814 lines
31 KiB
Python
814 lines
31 KiB
Python
"""SQLModel database table definitions for Notify Bridge."""
|
|
|
|
from __future__ import annotations
|
|
|
|
from datetime import datetime, timezone
|
|
from typing import Any
|
|
from uuid import uuid4
|
|
|
|
from sqlalchemy import ForeignKey, Index, UniqueConstraint, Text
|
|
from sqlmodel import JSON, Column, Field, SQLModel
|
|
|
|
|
|
def _utcnow() -> datetime:
|
|
return datetime.now(timezone.utc)
|
|
|
|
|
|
class User(SQLModel, table=True):
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
username: str = Field(index=True, unique=True)
|
|
hashed_password: str
|
|
role: str = Field(default="user")
|
|
token_version: int = Field(default=1)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class ServiceProvider(SQLModel, table=True):
|
|
"""A service provider instance (e.g., an Immich server)."""
|
|
|
|
__tablename__ = "service_provider"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
type: str # ServiceProviderType value ("immich")
|
|
name: str
|
|
icon: str = Field(default="")
|
|
config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
# Webhook token is the shared secret embedded in inbound webhook URLs.
|
|
# Must be unique so a token uniquely identifies a provider; indexed so
|
|
# the webhook router does an O(log n) lookup on every inbound request.
|
|
webhook_token: str = Field(
|
|
default_factory=lambda: uuid4().hex,
|
|
unique=True,
|
|
index=True,
|
|
)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class TelegramBot(SQLModel, table=True):
|
|
__tablename__ = "telegram_bot"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
name: str
|
|
token: str
|
|
icon: str = Field(default="")
|
|
bot_username: str = Field(default="")
|
|
# bot_id=0 is a sentinel meaning "Telegram has not yet returned a numeric
|
|
# ID for this bot" (i.e. token never validated). Multiple unverified bots
|
|
# may legitimately carry 0, so we only enforce uniqueness for non-sentinel
|
|
# values via a partial index added in migrate_uniqueness_constraints.
|
|
bot_id: int = Field(default=0, index=True)
|
|
# URL-path embedded in Telegram's setWebhook callback URL. Must be unique
|
|
# so the inbound dispatcher resolves a single bot per incoming request.
|
|
webhook_path_id: str = Field(
|
|
default_factory=lambda: uuid4().hex,
|
|
unique=True,
|
|
index=True,
|
|
)
|
|
update_mode: str = Field(default="none") # "none", "polling", or "webhook"
|
|
# NOTE: commands_config column remains in the DB for backward compat,
|
|
# but is no longer part of the SQLModel class. Data migrated to CommandConfig.
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class MatrixBot(SQLModel, table=True):
|
|
"""Matrix bot — homeserver connection for sending messages to rooms."""
|
|
|
|
__tablename__ = "matrix_bot"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
name: str
|
|
icon: str = Field(default="")
|
|
homeserver_url: str # e.g. https://matrix.org
|
|
access_token: str
|
|
display_name: str = Field(default="")
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class EmailBot(SQLModel, table=True):
|
|
"""Email sender — SMTP connection for sending email notifications."""
|
|
|
|
__tablename__ = "email_bot"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
name: str
|
|
icon: str = Field(default="")
|
|
email: str # From address
|
|
smtp_host: str
|
|
smtp_port: int = Field(default=587)
|
|
smtp_username: str = Field(default="")
|
|
smtp_password: str = Field(default="")
|
|
smtp_use_tls: bool = Field(default=True)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class TelegramChat(SQLModel, table=True):
|
|
__tablename__ = "telegram_chat"
|
|
# (bot_id, chat_id) uniquely identifies a chat. The composite index is
|
|
# the access pattern for save_chat_from_webhook ON CONFLICT updates and
|
|
# for any "lookup by (bot, chat)" callers.
|
|
__table_args__ = (
|
|
UniqueConstraint("bot_id", "chat_id", name="uq_telegram_chat_bot_chat"),
|
|
Index("ix_telegram_chat_bot_chat", "bot_id", "chat_id"),
|
|
)
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
bot_id: int = Field(foreign_key="telegram_bot.id")
|
|
chat_id: str
|
|
title: str = Field(default="")
|
|
chat_type: str = Field(default="private")
|
|
username: str = Field(default="")
|
|
language_code: str = Field(default="") # auto-detected from Telegram
|
|
language_override: str = Field(default="") # manual override set by user
|
|
commands_enabled: bool = Field(default=False)
|
|
discovered_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class TrackingConfig(SQLModel, table=True):
|
|
"""What events to track + scheduling rules. Tied to a provider type."""
|
|
|
|
__tablename__ = "tracking_config"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
provider_type: str # Must match provider's type
|
|
name: str
|
|
icon: str = Field(default="")
|
|
|
|
# Event-driven tracking
|
|
track_assets_added: bool = Field(default=True)
|
|
track_assets_removed: bool = Field(default=False)
|
|
track_collection_renamed: bool = Field(default=True)
|
|
track_collection_deleted: bool = Field(default=True)
|
|
track_sharing_changed: bool = Field(default=False)
|
|
|
|
# Gitea event tracking
|
|
track_push: bool = Field(default=True)
|
|
track_issue_opened: bool = Field(default=True)
|
|
track_issue_closed: bool = Field(default=True)
|
|
track_issue_commented: bool = Field(default=False)
|
|
track_pr_opened: bool = Field(default=True)
|
|
track_pr_closed: bool = Field(default=True)
|
|
track_pr_merged: bool = Field(default=True)
|
|
track_pr_commented: bool = Field(default=False)
|
|
track_release_published: bool = Field(default=True)
|
|
|
|
# Planka event tracking
|
|
track_card_created: bool = Field(default=True)
|
|
track_card_updated: bool = Field(default=False)
|
|
track_card_moved: bool = Field(default=True)
|
|
track_card_deleted: bool = Field(default=False)
|
|
track_card_commented: bool = Field(default=True)
|
|
track_comment_updated: bool = Field(default=False)
|
|
track_board_created: bool = Field(default=True)
|
|
track_board_updated: bool = Field(default=False)
|
|
track_board_deleted: bool = Field(default=True)
|
|
track_list_created: bool = Field(default=False)
|
|
track_list_updated: bool = Field(default=False)
|
|
track_list_deleted: bool = Field(default=False)
|
|
track_attachment_created: bool = Field(default=True)
|
|
track_card_label_added: bool = Field(default=False)
|
|
track_task_completed: bool = Field(default=True)
|
|
|
|
# Scheduler event tracking
|
|
track_scheduled_message: bool = Field(default=True)
|
|
|
|
# NUT (UPS) event tracking
|
|
track_ups_online: bool = Field(default=True)
|
|
track_ups_on_battery: bool = Field(default=True)
|
|
track_ups_low_battery: bool = Field(default=True)
|
|
track_ups_battery_restored: bool = Field(default=True)
|
|
track_ups_comms_lost: bool = Field(default=True)
|
|
track_ups_comms_restored: bool = Field(default=True)
|
|
track_ups_replace_battery: bool = Field(default=True)
|
|
track_ups_overload: bool = Field(default=True)
|
|
|
|
# Generic Webhook event tracking
|
|
track_webhook_received: bool = Field(default=True)
|
|
|
|
# Home Assistant event tracking
|
|
track_ha_state_changed: bool = Field(default=True)
|
|
track_ha_automation_triggered: bool = Field(default=False)
|
|
track_ha_service_called: bool = Field(default=False)
|
|
track_ha_event_fired: bool = Field(default=False)
|
|
|
|
# Bridge self-monitoring event tracking — defaults ON because the whole
|
|
# point of the provider is to alert on these conditions.
|
|
track_bridge_self_poll_failures: bool = Field(default=True)
|
|
track_bridge_self_deferred_backlog: bool = Field(default=True)
|
|
track_bridge_self_target_failures: bool = Field(default=True)
|
|
|
|
# Immich asset display
|
|
track_images: bool = Field(default=True)
|
|
track_videos: bool = Field(default=True)
|
|
notify_favorites_only: bool = Field(default=False)
|
|
|
|
# Asset display
|
|
include_tags: bool = Field(default=True)
|
|
include_asset_details: bool = Field(default=False)
|
|
max_assets_to_show: int = Field(default=10)
|
|
assets_order_by: str = Field(default="none")
|
|
assets_order: str = Field(default="descending")
|
|
|
|
# Periodic summary
|
|
periodic_enabled: bool = Field(default=False)
|
|
periodic_interval_days: int = Field(default=1)
|
|
periodic_start_date: str = Field(default="2025-01-01")
|
|
periodic_times: str = Field(default="12:00")
|
|
|
|
# Scheduled assets
|
|
scheduled_enabled: bool = Field(default=False)
|
|
scheduled_times: str = Field(default="09:00")
|
|
scheduled_collection_mode: str = Field(default="per_collection")
|
|
scheduled_limit: int = Field(default=10)
|
|
scheduled_favorite_only: bool = Field(default=False)
|
|
scheduled_asset_type: str = Field(default="all")
|
|
scheduled_min_rating: int = Field(default=0)
|
|
scheduled_order_by: str = Field(default="random")
|
|
scheduled_order: str = Field(default="descending")
|
|
|
|
# Memory mode
|
|
memory_enabled: bool = Field(default=False)
|
|
memory_source: str = Field(default="albums") # "albums" or "native"
|
|
memory_times: str = Field(default="09:00")
|
|
memory_collection_mode: str = Field(default="combined")
|
|
memory_limit: int = Field(default=10)
|
|
memory_favorite_only: bool = Field(default=False)
|
|
memory_asset_type: str = Field(default="all")
|
|
memory_min_rating: int = Field(default=0)
|
|
|
|
# Quiet hours — HH:MM strings interpreted in the app-level timezone
|
|
# (AppSetting "timezone"). Gated by quiet_hours_enabled so an empty window
|
|
# still represents "explicitly disabled" vs "not yet configured".
|
|
quiet_hours_enabled: bool = Field(default=False)
|
|
quiet_hours_start: str | None = Field(default=None)
|
|
quiet_hours_end: str | None = Field(default=None)
|
|
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class TemplateConfig(SQLModel, table=True):
|
|
"""Jinja2 message templates. Tied to a provider type.
|
|
|
|
Template content is stored in TemplateSlot child rows (one per slot).
|
|
"""
|
|
|
|
__tablename__ = "template_config"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(default=0) # 0 = system-owned (no FK to allow sentinel)
|
|
provider_type: str # Must match provider's type
|
|
name: str
|
|
description: str = Field(default="")
|
|
icon: str = Field(default="")
|
|
locale: str = Field(default="") # e.g. "en", "ru"; empty = unspecified
|
|
|
|
date_format: str = Field(default="%d.%m.%Y, %H:%M UTC")
|
|
date_only_format: str = Field(default="%d.%m.%Y")
|
|
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class TemplateSlot(SQLModel, table=True):
|
|
"""One Jinja2 template for a specific slot and locale within a TemplateConfig.
|
|
|
|
Slot names are provider-specific (e.g. 'message_assets_added' for Immich).
|
|
Each (config, slot, locale) triple holds a separate template.
|
|
"""
|
|
|
|
__tablename__ = "template_slot"
|
|
__table_args__ = (
|
|
UniqueConstraint("config_id", "slot_name", "locale", name="uq_template_slot_locale"),
|
|
)
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
config_id: int = Field(
|
|
foreign_key="template_config.id",
|
|
index=True,
|
|
|
|
|
|
)
|
|
slot_name: str
|
|
locale: str = Field(default="en")
|
|
template: str = Field(default="", sa_column=Column(Text, default=""))
|
|
|
|
|
|
class NotificationTarget(SQLModel, table=True):
|
|
"""Where to send notifications. Pure delivery endpoint.
|
|
|
|
Target-level config holds connection/display settings (e.g. bot_token,
|
|
disable_url_preview). Actual delivery endpoints live in TargetReceiver rows.
|
|
"""
|
|
|
|
__tablename__ = "notification_target"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
type: str # "telegram", "webhook", "email", "discord", "slack", "ntfy", "matrix"
|
|
name: str
|
|
icon: str = Field(default="")
|
|
config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
chat_action: str | None = Field(default="typing") # e.g. "typing", "upload_photo"
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class TargetReceiver(SQLModel, table=True):
|
|
"""One delivery endpoint within a NotificationTarget (broadcast support).
|
|
|
|
For Telegram: config = {"chat_id": "12345"}
|
|
For Webhook: config = {"url": "https://...", "headers": {...}}
|
|
For Email: config = {"email": "user@example.com", "name": "..."}
|
|
"""
|
|
|
|
__tablename__ = "target_receiver"
|
|
__table_args__ = (
|
|
UniqueConstraint("target_id", "receiver_key", name="uq_target_receiver"),
|
|
)
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
target_id: int = Field(
|
|
foreign_key="notification_target.id",
|
|
index=True,
|
|
|
|
|
|
)
|
|
name: str = Field(default="")
|
|
config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
receiver_key: str = Field(default="") # dedup key (e.g. chat_id, url, email)
|
|
locale: str = Field(default="") # e.g. "en", "ru"; empty = use target default
|
|
enabled: bool = Field(default=True)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class NotificationTracker(SQLModel, table=True):
|
|
"""Watches a provider's collections for changes."""
|
|
|
|
__tablename__ = "notification_tracker"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
provider_id: int = Field(foreign_key="service_provider.id")
|
|
name: str
|
|
icon: str = Field(default="")
|
|
collection_ids: list[str] = Field(default_factory=list, sa_column=Column(JSON))
|
|
filters: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
scan_interval: int = Field(default=60)
|
|
# Cap on the adaptive-polling skip factor (see services/scheduler.py).
|
|
# None or 0 disables adaptive back-off entirely — every scheduled tick
|
|
# runs. Positive values (2..N) enable skipping up to (N-1) out of N ticks
|
|
# once the tracker has been idle long enough. Per-tracker so an operator
|
|
# can opt a latency-sensitive tracker out of the global heuristic.
|
|
adaptive_max_skip: int | None = Field(default=None)
|
|
default_tracking_config_id: int | None = Field(default=None, foreign_key="tracking_config.id")
|
|
default_template_config_id: int | None = Field(default=None, foreign_key="template_config.id")
|
|
enabled: bool = Field(default=True)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class NotificationTrackerTarget(SQLModel, table=True):
|
|
"""Junction between NotificationTracker and NotificationTarget with per-link config."""
|
|
|
|
__tablename__ = "notification_tracker_target"
|
|
# A tracker should never link to the same target twice — duplicate links
|
|
# would deliver the same notification multiple times. Enforced at the DB
|
|
# level so concurrent inserts can't bypass an application-side check.
|
|
__table_args__ = (
|
|
UniqueConstraint(
|
|
"notification_tracker_id", "target_id",
|
|
name="uq_ntt_tracker_target",
|
|
),
|
|
)
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
# Python attr stays as tracker_id for backward compat; DB column is notification_tracker_id
|
|
tracker_id: int = Field(
|
|
foreign_key="notification_tracker.id",
|
|
index=True,
|
|
sa_column_kwargs={"name": "notification_tracker_id"},
|
|
)
|
|
target_id: int = Field(
|
|
foreign_key="notification_target.id",
|
|
index=True,
|
|
|
|
|
|
)
|
|
tracking_config_id: int | None = Field(
|
|
default=None, foreign_key="tracking_config.id"
|
|
)
|
|
template_config_id: int | None = Field(
|
|
default=None, foreign_key="template_config.id"
|
|
)
|
|
enabled: bool = Field(default=True)
|
|
quiet_hours_start: str | None = None
|
|
quiet_hours_end: str | None = None
|
|
# NOTE: commands_config column remains in the DB for backward compat,
|
|
# but is no longer part of the SQLModel class. Data migrated to CommandConfig.
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class NotificationTrackerState(SQLModel, table=True):
|
|
"""Persisted state for change detection."""
|
|
|
|
__tablename__ = "notification_tracker_state"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
# Python attr stays as tracker_id for backward compat; DB column is notification_tracker_id
|
|
tracker_id: int = Field(
|
|
foreign_key="notification_tracker.id",
|
|
index=True,
|
|
sa_column_kwargs={"name": "notification_tracker_id"},
|
|
)
|
|
collection_id: str
|
|
collection_name: str = Field(default="")
|
|
shared: bool = Field(default=False)
|
|
asset_ids: list[str] = Field(default_factory=list, sa_column=Column(JSON))
|
|
pending_asset_ids: list[str] = Field(default_factory=list, sa_column=Column(JSON))
|
|
# Lightweight fingerprint ({updated_at, asset_count, shared, name, ...})
|
|
# captured from the provider's cheap meta probe. Letting this differ from
|
|
# the current provider response is what tells the watcher a full fetch is
|
|
# actually required — letting it match lets the watcher skip the big read.
|
|
meta_fingerprint: dict[str, Any] = Field(
|
|
default_factory=dict, sa_column=Column(JSON)
|
|
)
|
|
last_updated: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class CommandConfig(SQLModel, table=True):
|
|
"""Configuration for bot commands (e.g., which commands are enabled, rate limits)."""
|
|
|
|
__tablename__ = "command_config"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
provider_type: str
|
|
name: str
|
|
icon: str = Field(default="")
|
|
enabled_commands: list[str] = Field(default_factory=list, sa_column=Column(JSON))
|
|
locale: str = Field(default="en")
|
|
response_mode: str = Field(default="media") # "media" or "text"
|
|
default_count: int = Field(default=5)
|
|
rate_limits: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
command_template_config_id: int | None = Field(
|
|
default=None, foreign_key="command_template_config.id"
|
|
)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class CommandTemplateConfig(SQLModel, table=True):
|
|
"""Jinja2 templates for command responses. Provider-specific via slots."""
|
|
|
|
__tablename__ = "command_template_config"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(default=0) # 0 = system-owned
|
|
provider_type: str
|
|
name: str
|
|
description: str = Field(default="")
|
|
icon: str = Field(default="")
|
|
locale: str = Field(default="") # e.g. "en", "ru"; empty = unspecified
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class CommandTemplateSlot(SQLModel, table=True):
|
|
"""One Jinja2 template for a specific command response slot and locale.
|
|
|
|
Slot names match command names (e.g. 'status', 'help', 'albums').
|
|
Description slots use 'desc_' prefix (e.g. 'desc_status', 'desc_help').
|
|
Each (config, slot, locale) triple holds a separate template.
|
|
"""
|
|
|
|
__tablename__ = "command_template_slot"
|
|
__table_args__ = (
|
|
UniqueConstraint("config_id", "slot_name", "locale", name="uq_cmd_slot_locale"),
|
|
)
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
config_id: int = Field(
|
|
foreign_key="command_template_config.id",
|
|
index=True,
|
|
|
|
|
|
)
|
|
slot_name: str
|
|
locale: str = Field(default="en")
|
|
template: str = Field(default="", sa_column=Column(Text, default=""))
|
|
|
|
|
|
class CommandTracker(SQLModel, table=True):
|
|
"""Links a provider to a command config for interactive bot commands."""
|
|
|
|
__tablename__ = "command_tracker"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
provider_id: int = Field(foreign_key="service_provider.id")
|
|
command_config_id: int = Field(foreign_key="command_config.id")
|
|
name: str
|
|
icon: str = Field(default="")
|
|
enabled: bool = Field(default=True)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class CommandTrackerListener(SQLModel, table=True):
|
|
"""Links a CommandTracker to a listener (e.g., a telegram bot chat)."""
|
|
|
|
__tablename__ = "command_tracker_listener"
|
|
__table_args__ = (
|
|
UniqueConstraint(
|
|
"command_tracker_id", "listener_type", "listener_id",
|
|
name="uq_command_tracker_listener",
|
|
),
|
|
)
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
command_tracker_id: int = Field(
|
|
foreign_key="command_tracker.id",
|
|
index=True,
|
|
|
|
)
|
|
listener_type: str # e.g. "telegram_bot"
|
|
listener_id: int
|
|
# Optional per-chat album scope. None = inherit from tracker (use all).
|
|
# When set, only these album/collection ids are queryable from this chat.
|
|
allowed_album_ids: list[str] | None = Field(
|
|
default=None, sa_column=Column(JSON, nullable=True),
|
|
)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class DeferredDispatch(SQLModel, table=True):
|
|
"""A dispatch held back by quiet hours, waiting for the window to end.
|
|
|
|
One row per ``(link, event_type, collection_id)`` for asset events — newly
|
|
arriving events for the same key coalesce into the existing row's
|
|
``event_payload`` (union of added/removed asset sets) instead of inserting
|
|
a duplicate row. Non-asset events (push, pr_opened, ups_*, …) get a fresh
|
|
row each time because they aren't logically cancellable.
|
|
|
|
At drain time the scheduler picks up rows where ``status='pending'`` and
|
|
``fire_at <= now``, re-resolves the link/target/config against current
|
|
state (so subsequent config edits apply), and dispatches.
|
|
"""
|
|
|
|
__tablename__ = "deferred_dispatch"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int | None = Field(
|
|
default=None,
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=True,
|
|
index=True,
|
|
),
|
|
)
|
|
tracker_id: int = Field(foreign_key="notification_tracker.id", index=True)
|
|
# The specific link this deferral targets. On drain we re-fetch by ID; if
|
|
# the link was disabled or removed in the meantime we drop with a
|
|
# ``deferred_then_dropped`` log row instead of dispatching to nothing.
|
|
link_id: int = Field(
|
|
foreign_key="notification_tracker_target.id", index=True,
|
|
)
|
|
# The event_log row written when the event was first detected. The drain
|
|
# writes a follow-up event_log row referencing this id so the dashboard
|
|
# can show "delivered at HH:MM, originally detected at HH:MM".
|
|
#
|
|
# ``ondelete="SET NULL"`` matters because the daily ``_cleanup_old_events``
|
|
# job hard-deletes event_log rows past the retention horizon. Without
|
|
# SET NULL, an old pending DeferredDispatch row referencing an aging-out
|
|
# event_log row would either (a) prevent the delete with an FK violation
|
|
# under SQLite's enforced foreign_keys PRAGMA, or (b) leave a dangling
|
|
# reference on engines that don't enforce.
|
|
event_log_id: int | None = Field(
|
|
default=None,
|
|
sa_column=Column(
|
|
"event_log_id",
|
|
ForeignKey("event_log.id", ondelete="SET NULL"),
|
|
nullable=True,
|
|
index=True,
|
|
),
|
|
)
|
|
event_type: str = Field(index=True)
|
|
collection_id: str = Field(default="", index=True)
|
|
# ``dataclasses.asdict(ServiceEvent)`` with datetime/enum normalisation —
|
|
# round-tripped via the helpers in ``services.deferred_dispatch``.
|
|
event_payload: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
fire_at: datetime = Field(index=True)
|
|
# ``pending`` until the drain runs; then ``fired``, ``dropped`` (link
|
|
# gone / event-type disabled after defer), or ``cancelled`` (coalesced
|
|
# away by a counter-event).
|
|
status: str = Field(default="pending", index=True)
|
|
fired_at: datetime | None = Field(default=None)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class EventLog(SQLModel, table=True):
|
|
"""Log of detected events."""
|
|
|
|
__tablename__ = "event_log"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
# Owner. Indexed for the dashboard events query. Nullable only because
|
|
# historical rows (pre-user_id column) may have no owner; new rows always
|
|
# set this directly. SET NULL on user delete preserves the audit trail
|
|
# while letting the user record itself be removed.
|
|
user_id: int | None = Field(
|
|
default=None,
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="SET NULL"),
|
|
nullable=True,
|
|
index=True,
|
|
),
|
|
)
|
|
# Python attr stays as tracker_id for backward compat; DB column is notification_tracker_id
|
|
tracker_id: int | None = Field(
|
|
default=None,
|
|
foreign_key="notification_tracker.id",
|
|
index=True,
|
|
sa_column_kwargs={"name": "notification_tracker_id"},
|
|
)
|
|
tracker_name: str = Field(default="")
|
|
# Links an event back to an Action when the event was emitted by the
|
|
# action runner (``event_type`` starts with ``action_``). Null for
|
|
# notification-tracker events.
|
|
action_id: int | None = Field(
|
|
default=None, foreign_key="action.id", index=True,
|
|
)
|
|
action_name: str = Field(default="")
|
|
# Bot command provenance. Populated when ``event_type`` starts with
|
|
# ``command_`` so the dashboard can render command activity alongside
|
|
# tracker and action events. NULL for non-command rows.
|
|
command_tracker_id: int | None = Field(
|
|
default=None, foreign_key="command_tracker.id", index=True,
|
|
)
|
|
command_tracker_name: str = Field(default="")
|
|
telegram_bot_id: int | None = Field(
|
|
default=None, foreign_key="telegram_bot.id", index=True,
|
|
)
|
|
bot_name: str = Field(default="")
|
|
# FK to service_provider with SET NULL so deleting a provider leaves
|
|
# historical event_log rows intact (provider_name preserves the label
|
|
# for display). The FK only takes effect on freshly created tables —
|
|
# SQLite cannot ALTER a constraint into an existing table without a
|
|
# rebuild, so application code in api/providers.delete_provider also
|
|
# nulls these explicitly. See migrate_eventlog_provider_fk.
|
|
provider_id: int | None = Field(
|
|
default=None,
|
|
sa_column=Column(
|
|
"provider_id",
|
|
ForeignKey("service_provider.id", ondelete="SET NULL"),
|
|
nullable=True,
|
|
index=True,
|
|
),
|
|
)
|
|
provider_name: str = Field(default="")
|
|
event_type: str = Field(index=True)
|
|
collection_id: str
|
|
collection_name: str
|
|
assets_count: int = Field(default=0)
|
|
details: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class Action(SQLModel, table=True):
|
|
"""A scheduled action that mutates an external service."""
|
|
|
|
__tablename__ = "action"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
user_id: int = Field(
|
|
sa_column=Column(
|
|
"user_id",
|
|
ForeignKey("user.id", ondelete="CASCADE"),
|
|
nullable=False,
|
|
),
|
|
)
|
|
provider_id: int = Field(foreign_key="service_provider.id")
|
|
name: str
|
|
icon: str = Field(default="")
|
|
action_type: str # e.g. "auto_organize"
|
|
config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
schedule_type: str = Field(default="interval") # "interval" or "cron"
|
|
schedule_interval: int = Field(default=3600) # seconds
|
|
schedule_cron: str = Field(default="")
|
|
enabled: bool = Field(default=False) # default disabled for safety
|
|
last_run_at: datetime | None = Field(default=None)
|
|
last_run_status: str = Field(default="") # "success", "partial", "failed", ""
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class ActionRule(SQLModel, table=True):
|
|
"""One rule within an Action. Executed in order."""
|
|
|
|
__tablename__ = "action_rule"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
action_id: int = Field(foreign_key="action.id", index=True)
|
|
name: str = Field(default="")
|
|
rule_config: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
enabled: bool = Field(default=True)
|
|
order: int = Field(default=0)
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class ActionExecution(SQLModel, table=True):
|
|
"""Log of an action execution (scheduled, manual, or dry-run)."""
|
|
|
|
__tablename__ = "action_execution"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
action_id: int = Field(foreign_key="action.id", index=True)
|
|
started_at: datetime = Field(default_factory=_utcnow)
|
|
finished_at: datetime | None = Field(default=None)
|
|
status: str = Field(default="running") # "running", "success", "partial", "failed"
|
|
rules_processed: int = Field(default=0)
|
|
rules_succeeded: int = Field(default=0)
|
|
rules_failed: int = Field(default=0)
|
|
total_items_affected: int = Field(default=0)
|
|
summary: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
error: str = Field(default="")
|
|
trigger: str = Field(default="scheduled") # "scheduled", "manual", "dry_run"
|
|
|
|
|
|
class WebhookPayloadLog(SQLModel, table=True):
|
|
"""Log of incoming webhook payloads for debugging and replay."""
|
|
|
|
__tablename__ = "webhook_payload_log"
|
|
|
|
id: int | None = Field(default=None, primary_key=True)
|
|
provider_id: int = Field(foreign_key="service_provider.id", index=True)
|
|
method: str = Field(default="POST")
|
|
headers: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
body: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
status: str = Field(default="matched") # "matched" | "unmatched" | "error"
|
|
extracted_fields: dict[str, Any] = Field(default_factory=dict, sa_column=Column(JSON))
|
|
error_message: str = Field(default="")
|
|
created_at: datetime = Field(default_factory=_utcnow)
|
|
|
|
|
|
class AppSetting(SQLModel, table=True):
|
|
"""Key-value app-level settings (admin-configurable)."""
|
|
|
|
__tablename__ = "app_setting"
|
|
|
|
key: str = Field(primary_key=True)
|
|
value: str = Field(default="")
|