feat: core ORM models + initial migration
Implements the 14 core ORM models for the yunke data-mirror system using SQLAlchemy 2.0 Mapped/mapped_column style, driven by the field lists in docs/superpowers/specs/2026-08-01-yunke-data-dictionary.md: Department, Staff, WechatAccount, WhatsappAccount, Customer, ChatGroup, GroupMember, Message, Media, PhoneCall, Sms, SyncState, YkAccount, SyncRun. Conventions: - Natural source PKs where they exist (friend_id, wx_id, msg_id, room_id, media_hash, call_id, sms_id, dept_id, user_id); surrogate Integer PKs otherwise (SyncState, YkAccount, SyncRun). GroupMember uses a composite (channel, room_id, member_id) PK. - Common audit columns version/raw_json/fetched_at/updated_at on every source-mirrored table. - channel only where meaningful: Customer/Message/Media/ChatGroup/ GroupMember/Sms use messaging channels; PhoneCall.channel is call-type; SyncState.channel scopes the cursor. Omitted on Staff/Department/ WechatAccount/WhatsappAccount/YkAccount/SyncRun. - Message composite index on (channel, owner_account_id, talker_id, timestamp) for the hot per-talker time-order query path. - message_table_name(channel, timestamp) helper documents the future message_<channel>_<YYYYMM> sharding convention; the Message model still maps to a single physical table for now (sharding is a later task). All types are portable generic SQLAlchemy types so tests run on in-memory SQLite while prod targets MySQL. Tests verify Customer round-trip, raw_json JSON round-trip, Message natural PK, GroupMember composite PK, the Message composite index, and full table creation on SQLite. Alembic wired via migrations/env.py to Base.metadata + get_settings().db_url with an async->sync driver swap (mysql+asyncmy -> mysql+pymysql, sqlite+aiosqlite -> sqlite). Initial migration autogenerated against an empty SQLite DB; server defaults normalized to sa.func.now() so the file is MySQL-portable. upgrade head is intentionally not run (no MySQL yet). Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
浅念
@@ -0,0 +1,613 @@
|
||||
"""Core ORM models for the yunke data-mirror system.
|
||||
|
||||
Every model maps an entity observed in the yunke CRM API (see
|
||||
``docs/superpowers/specs/2026-08-01-yunke-data-dictionary.md``) to a SQL table.
|
||||
Models target **MySQL in production** but keep column types portable so the
|
||||
suite can exercise them on **in-memory SQLite** (see ``tests/test_models.py``).
|
||||
|
||||
Conventions
|
||||
-----------
|
||||
* SQLAlchemy 2.0 ``Mapped``/``mapped_column`` style; all classes extend
|
||||
``app.db.base.Base``.
|
||||
* API field names are mapped to ``snake_case`` columns; the original payload is
|
||||
preserved verbatim in ``raw_json`` (``sa.JSON``).
|
||||
* Common audit columns on every source-mirrored table:
|
||||
- ``version`` — source update stamp (string when the API gives a version
|
||||
token, BigInteger when it is a ms timestamp).
|
||||
- ``raw_json`` — the original API record (``sa.JSON``).
|
||||
- ``fetched_at`` — when we pulled the row (server default ``now()``).
|
||||
- ``updated_at`` — when the row was last written (server default + onupdate).
|
||||
* ``channel`` is only added where it is *meaningful*:
|
||||
- Communication tables (``Customer``, ``Message``, ``Media``, ``ChatGroup``,
|
||||
``GroupMember``, ``Sms``) use ``channel`` in {``wechat``, ``whatsapp``,
|
||||
``sms``}.
|
||||
- ``PhoneCall`` uses ``channel`` to mean call-type in {``phone_call``,
|
||||
``voice_call``} (calls are not tied to a messaging channel).
|
||||
- ``SyncState`` uses ``channel`` to scope a sync cursor (together with
|
||||
``entity_type``).
|
||||
- ``Staff``, ``Department``, ``WechatAccount``, ``WhatsappAccount``,
|
||||
``YkAccount``, ``SyncRun`` carry no ``channel`` column.
|
||||
* Primary keys use the natural source id where one exists; otherwise an
|
||||
integer surrogate PK is used (``SyncState``, ``YkAccount``, ``SyncRun``).
|
||||
* ``GroupMember`` uses a composite PK ``(channel, room_id, member_id)``.
|
||||
|
||||
The ``Message`` model currently maps to a single physical ``message`` table.
|
||||
Real-world volume is expected at millions of rows/month, so the long-term plan
|
||||
is time-sharded physical tables named via :func:`message_table_name`. That
|
||||
sharding is a LATER task — for now every message row lives in ``message`` and
|
||||
the helper below is provided so call-sites can already agree on the naming
|
||||
convention.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
from typing import Any
|
||||
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy import BigInteger, Boolean, DateTime, Index, Integer, String, Text
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
from sqlalchemy.types import JSON
|
||||
|
||||
from app.db.base import Base
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def message_table_name(channel: str, timestamp: int | dt.datetime) -> str:
|
||||
"""Return the conventional sharded table name for a message.
|
||||
|
||||
Format: ``message_<channel>_<YYYYMM>`` where ``YYYYMM`` is derived from the
|
||||
message timestamp. ``timestamp`` may be either:
|
||||
|
||||
* a Unix epoch in **milliseconds** (``int``) — the form used by the
|
||||
``Message.timestamp`` column and the yunke API, or
|
||||
* a ``datetime.datetime`` instance (any tz).
|
||||
|
||||
Examples
|
||||
--------
|
||||
>>> import datetime as dt
|
||||
>>> message_table_name("wechat", 1_700_000_000_000)
|
||||
'message_wechat_202311'
|
||||
>>> message_table_name("whatsapp", dt.datetime(2024, 1, 5))
|
||||
'message_whatsapp_202401'
|
||||
"""
|
||||
if isinstance(timestamp, dt.datetime):
|
||||
ts = timestamp
|
||||
else:
|
||||
# Treat int as ms-since-epoch (UTC). ``float`` also tolerated.
|
||||
ts = dt.datetime.fromtimestamp(int(timestamp) / 1000, tz=dt.UTC)
|
||||
return f"message_{channel}_{ts.strftime('%Y%m')}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shared column defaults
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _fetched_at() -> Mapped[dt.datetime]:
|
||||
return mapped_column(
|
||||
DateTime, server_default=sa.func.now(), nullable=False
|
||||
)
|
||||
|
||||
|
||||
def _updated_at() -> Mapped[dt.datetime]:
|
||||
return mapped_column(
|
||||
DateTime,
|
||||
server_default=sa.func.now(),
|
||||
onupdate=sa.func.now(),
|
||||
nullable=False,
|
||||
)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# 1. Organization & people
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class Department(Base):
|
||||
"""云客 部门/分组 — source: ``POST /usercenter/dept/tree``."""
|
||||
|
||||
__tablename__ = "department"
|
||||
|
||||
dept_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
parent_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
path: Mapped[str | None] = mapped_column(String(512), nullable=True)
|
||||
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
|
||||
class Staff(Base):
|
||||
"""员工 — embedded in account lists (``userId`` + ``userName``)."""
|
||||
|
||||
__tablename__ = "staff"
|
||||
|
||||
user_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
sale_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
dept_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
is_on_job: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
||||
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# 2. Staff messaging accounts (channel identity, so no `channel` column)
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class WechatAccount(Base):
|
||||
"""员工微信号 — source: ``POST /wechatweb/wxUser/list`` (wechatId PK)."""
|
||||
|
||||
__tablename__ = "wechat_account"
|
||||
|
||||
wx_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
||||
nick_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
alias: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
user_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
contacts_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
group_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
send_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
receive_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
is_on_job: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
||||
db_size: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
online_status: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
||||
last_update: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
|
||||
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
|
||||
class WhatsappAccount(Base):
|
||||
"""员工 WhatsApp 账号 — source: ``POST /qqweb/WhatsAppUser/businessSalesList``.
|
||||
|
||||
PK is ``uid`` of the form ``<phone>@s.whatsapp.net``.
|
||||
"""
|
||||
|
||||
__tablename__ = "whatsapp_account"
|
||||
|
||||
uid: Mapped[str] = mapped_column(String(128), primary_key=True)
|
||||
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
user_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
nick_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
sale_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
sales_bind_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
contacts_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
group_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
talker_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
friend_add_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
is_on_job: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
||||
last_update: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
|
||||
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# 3. Communication entities (carry a meaningful `channel` column)
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class Customer(Base):
|
||||
"""客户/好友 (WeChat friend OR WhatsApp contact OR SMS peer).
|
||||
|
||||
Source: ``POST /wechatweb/wxFriends/friend_list/es`` (WeChat) or
|
||||
``POST /qqweb/WhatsAppUser/businessFriendList`` (WhatsApp). PK is the
|
||||
source friend id: ``friendWxId`` for WeChat, ``<phone>@s.whatsapp.net``
|
||||
for WhatsApp.
|
||||
"""
|
||||
|
||||
__tablename__ = "customer"
|
||||
|
||||
friend_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
||||
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
owner_account_id: Mapped[str | None] = mapped_column(
|
||||
String(128), nullable=True
|
||||
) # wxId (WeChat) / uid (WhatsApp) — owning staff account
|
||||
friend_nickname: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
friend_remark: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
real_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
wechat_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
gender: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
region: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
description: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
sale_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
dept_route: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
dept_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
head_image_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
last_chat_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
send_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
receive_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
message_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
delete_flag: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
is_block: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
||||
friend_state: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
|
||||
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (
|
||||
Index(
|
||||
"ix_customer_owner_friend",
|
||||
"channel",
|
||||
"owner_account_id",
|
||||
"friend_id",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class ChatGroup(Base):
|
||||
"""聊天群 — source: ``POST /wechatweb/chatroom/getChatroomList``.
|
||||
|
||||
PK ``room_id`` is the source ``chatRoomId`` (ends with ``@chatroom``).
|
||||
The raw ``memberIds`` (semicolon-joined wxids) is kept verbatim in
|
||||
``member_ids``; the parsed members are mirrored into :class:`GroupMember`.
|
||||
"""
|
||||
|
||||
__tablename__ = "chat_group"
|
||||
|
||||
room_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
||||
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
room_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
owner_account_id: Mapped[str | None] = mapped_column(
|
||||
String(128), nullable=True
|
||||
) # wechatId / uid of owning staff account
|
||||
owner_account_nickname: Mapped[str | None] = mapped_column(
|
||||
String(255), nullable=True
|
||||
)
|
||||
member_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
member_ids: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
head_image_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
owner_user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
owner_user_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
message_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
own_message_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
loss_members: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
add_members: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
last_chat_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
update_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
|
||||
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (
|
||||
Index(
|
||||
"ix_chat_group_owner",
|
||||
"channel",
|
||||
"owner_account_id",
|
||||
"room_id",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class GroupMember(Base):
|
||||
"""群成员 — composite PK (channel, room_id, member_id).
|
||||
|
||||
Source: the ``memberIds`` field of a chatroom payload, parsed into one row
|
||||
per member. Per-member nickname/avatar may require a separate API call
|
||||
(see data-dictionary §9.6) and are therefore nullable.
|
||||
"""
|
||||
|
||||
__tablename__ = "group_member"
|
||||
|
||||
channel: Mapped[str] = mapped_column(String(16), primary_key=True)
|
||||
room_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
||||
member_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
||||
nickname: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
avatar: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
display_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
|
||||
class Message(Base):
|
||||
"""聊天消息 — WeChat ``msgSvrId`` / WhatsApp ``keyId`` as natural PK.
|
||||
|
||||
Sources:
|
||||
* WeChat: ``POST /wechatweb/msg/chats``
|
||||
* WhatsApp: ``POST /qqweb/WhatsAppUser/chatRecords``
|
||||
|
||||
The model currently maps to a **single** ``message`` table. The real
|
||||
sharding across ``message_<channel>_<YYYYMM>`` physical tables (see
|
||||
:func:`message_table_name`) is a LATER task; this single-table mapping is
|
||||
sufficient for ingestion v1.
|
||||
"""
|
||||
|
||||
__tablename__ = "message"
|
||||
|
||||
msg_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
owner_account_id: Mapped[str | None] = mapped_column(
|
||||
String(128), nullable=True
|
||||
) # wechatId / uid of sender staff account
|
||||
talker_id: Mapped[str | None] = mapped_column(
|
||||
String(128), nullable=True
|
||||
) # talker / friendUid — the other party
|
||||
ori_talker: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
room_id: Mapped[str | None] = mapped_column(
|
||||
String(128), nullable=True
|
||||
) # @chatroom id; NULL/empty = direct message
|
||||
is_send: Mapped[bool | None] = mapped_column(Boolean, nullable=True) # mine/isSend
|
||||
type: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
text: Mapped[str | None] = mapped_column(Text, nullable=True) # content/text
|
||||
file_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
file_th_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
file_hd_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
md5: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
md5_th: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
md5_hd: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
duration: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
transcription_text: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
img_path: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
original_image: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
pic_type: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
pay_msg_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
refer_msg: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
sender_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
sender_head_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
timestamp: Mapped[int | None] = mapped_column(BigInteger, nullable=True) # ms epoch
|
||||
is_del: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
||||
|
||||
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (
|
||||
# Hot query path: per-account, per-talker, time-ordered pagination.
|
||||
Index(
|
||||
"ix_message_channel_owner_talker_time",
|
||||
"channel",
|
||||
"owner_account_id",
|
||||
"talker_id",
|
||||
"timestamp",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class Media(Base):
|
||||
"""Locally cached media (image / voice / video / file) — keyed by hash.
|
||||
|
||||
Deduplicates downloads across messages that reference the same file (via
|
||||
``md5``). ``local_path`` is set once the blob has been fetched into the
|
||||
local media store.
|
||||
"""
|
||||
|
||||
__tablename__ = "media"
|
||||
|
||||
media_hash: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
media_type: Mapped[str | None] = mapped_column(
|
||||
String(32), nullable=True
|
||||
) # image|voice|video|file|...
|
||||
original_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
original_th_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
original_hd_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
local_path: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
size_bytes: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
mime_type: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
duration: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (
|
||||
Index("ix_media_type_channel", "media_type", "channel"),
|
||||
)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# 4. Phone calls & SMS (prefix /pc/)
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class PhoneCall(Base):
|
||||
"""电话/语音通话记录 — source: ``POST /pc/callLog/list`` (phone calls) and
|
||||
``POST /wechatweb/callRecord/queryWeChatVoiceList`` (voice calls).
|
||||
|
||||
``channel`` here means **call-type** in {``phone_call``, ``voice_call``}
|
||||
(calls are not tied to a messaging channel). PK is the source ``id``.
|
||||
"""
|
||||
|
||||
__tablename__ = "phone_call"
|
||||
|
||||
call_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
start_call_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
call_duration: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
ring_time: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
call_status: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
incoming_call: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
user_id_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
department_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
call_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
sim_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
sim_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
imei: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
record_file: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
is_upload: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
||||
download_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
ai_asr: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
qc_status: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
plan_customer_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
|
||||
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (
|
||||
Index("ix_phone_call_user_time", "user_id", "start_call_time"),
|
||||
)
|
||||
|
||||
|
||||
class Sms(Base):
|
||||
"""短信记录 — source: ``POST /pc/customer/smsLogsList``."""
|
||||
|
||||
__tablename__ = "sms"
|
||||
|
||||
sms_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
channel: Mapped[str] = mapped_column(
|
||||
String(16), nullable=False, default="sms"
|
||||
) # always 'sms' today; kept for symmetry
|
||||
customer_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
content: Mapped[str | None] = mapped_column(Text, nullable=True) # full sms text
|
||||
content_preview: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
sender: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
direction: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
sender_department: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
sender_department_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
receive_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
create_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
sim_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
sim_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
source: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
|
||||
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (Index("ix_sms_user_time", "user_id", "create_time"),)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# 5. Operational / bookkeeping tables (local; no raw_json/version)
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class SyncState(Base):
|
||||
"""Resumable sync cursor — scoped by (entity_type, channel, scope_id).
|
||||
|
||||
``scope_id`` qualifies the cursor for a sub-stream (e.g. the owning staff
|
||||
account id when syncing per-account message streams). ``last_version``
|
||||
holds the opaque source update stamp; ``last_timestamp`` the ms epoch.
|
||||
"""
|
||||
|
||||
__tablename__ = "sync_state"
|
||||
|
||||
state_id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
entity_type: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
channel: Mapped[str | None] = mapped_column(String(16), nullable=True)
|
||||
scope_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
last_version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
last_timestamp: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
last_synced_at: Mapped[dt.datetime | None] = mapped_column(
|
||||
DateTime, nullable=True
|
||||
)
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (
|
||||
Index(
|
||||
"ix_sync_state_scope",
|
||||
"entity_type",
|
||||
"channel",
|
||||
"scope_id",
|
||||
unique=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class YkAccount(Base):
|
||||
"""云客 CRM login account — credentials / session for the upstream API.
|
||||
|
||||
The session cookie / password is stored encrypted with
|
||||
``Settings.encryption_key`` (Fernet). ``user_id`` + ``company`` together
|
||||
identify a yunke session (the two ``userid`` / ``company`` request headers).
|
||||
"""
|
||||
|
||||
__tablename__ = "yk_account"
|
||||
|
||||
account_id: Mapped[int] = mapped_column(
|
||||
Integer, primary_key=True, autoincrement=True
|
||||
)
|
||||
user_id: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
company: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
username: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
session_blob_enc: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
last_login_at: Mapped[dt.datetime | None] = mapped_column(
|
||||
DateTime, nullable=True
|
||||
)
|
||||
is_active: Mapped[bool] = mapped_column(
|
||||
Boolean, nullable=False, default=True, server_default=sa.text("1")
|
||||
)
|
||||
|
||||
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (
|
||||
Index("ix_yk_account_user_company", "user_id", "company", unique=True),
|
||||
)
|
||||
|
||||
|
||||
class SyncRun(Base):
|
||||
"""Audit log for one sync execution (start / end / status / counts)."""
|
||||
|
||||
__tablename__ = "sync_run"
|
||||
|
||||
run_id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
entity_type: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
channel: Mapped[str | None] = mapped_column(String(16), nullable=True)
|
||||
scope_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
status: Mapped[str] = mapped_column(String(16), nullable=False) # running|success|failed
|
||||
started_at: Mapped[dt.datetime] = mapped_column(
|
||||
DateTime, server_default=sa.func.now(), nullable=False
|
||||
)
|
||||
finished_at: Mapped[dt.datetime | None] = mapped_column(DateTime, nullable=True)
|
||||
records_fetched: Mapped[int] = mapped_column(
|
||||
Integer, nullable=False, default=0, server_default=sa.text("0")
|
||||
)
|
||||
records_upserted: Mapped[int] = mapped_column(
|
||||
Integer, nullable=False, default=0, server_default=sa.text("0")
|
||||
)
|
||||
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
|
||||
updated_at: Mapped[dt.datetime] = _updated_at()
|
||||
|
||||
__table_args__ = (Index("ix_sync_run_entity_status", "entity_type", "status"),)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ChatGroup",
|
||||
"Customer",
|
||||
"Department",
|
||||
"GroupMember",
|
||||
"Media",
|
||||
"Message",
|
||||
"PhoneCall",
|
||||
"Sms",
|
||||
"Staff",
|
||||
"SyncRun",
|
||||
"SyncState",
|
||||
"WechatAccount",
|
||||
"WhatsappAccount",
|
||||
"YkAccount",
|
||||
"message_table_name",
|
||||
]
|
||||
Reference in New Issue
Block a user