浅念
9cf1c9ab11
Implements the 14 core ORM models for the yunke data-mirror system using SQLAlchemy 2.0 Mapped/mapped_column style, driven by the field lists in docs/superpowers/specs/2026-08-01-yunke-data-dictionary.md: Department, Staff, WechatAccount, WhatsappAccount, Customer, ChatGroup, GroupMember, Message, Media, PhoneCall, Sms, SyncState, YkAccount, SyncRun. Conventions: - Natural source PKs where they exist (friend_id, wx_id, msg_id, room_id, media_hash, call_id, sms_id, dept_id, user_id); surrogate Integer PKs otherwise (SyncState, YkAccount, SyncRun). GroupMember uses a composite (channel, room_id, member_id) PK. - Common audit columns version/raw_json/fetched_at/updated_at on every source-mirrored table. - channel only where meaningful: Customer/Message/Media/ChatGroup/ GroupMember/Sms use messaging channels; PhoneCall.channel is call-type; SyncState.channel scopes the cursor. Omitted on Staff/Department/ WechatAccount/WhatsappAccount/YkAccount/SyncRun. - Message composite index on (channel, owner_account_id, talker_id, timestamp) for the hot per-talker time-order query path. - message_table_name(channel, timestamp) helper documents the future message_<channel>_<YYYYMM> sharding convention; the Message model still maps to a single physical table for now (sharding is a later task). All types are portable generic SQLAlchemy types so tests run on in-memory SQLite while prod targets MySQL. Tests verify Customer round-trip, raw_json JSON round-trip, Message natural PK, GroupMember composite PK, the Message composite index, and full table creation on SQLite. Alembic wired via migrations/env.py to Base.metadata + get_settings().db_url with an async->sync driver swap (mysql+asyncmy -> mysql+pymysql, sqlite+aiosqlite -> sqlite). Initial migration autogenerated against an empty SQLite DB; server defaults normalized to sa.func.now() so the file is MySQL-portable. upgrade head is intentionally not run (no MySQL yet). Co-Authored-By: Claude <noreply@anthropic.com>
614 lines
27 KiB
Python
614 lines
27 KiB
Python
"""Core ORM models for the yunke data-mirror system.
|
|
|
|
Every model maps an entity observed in the yunke CRM API (see
|
|
``docs/superpowers/specs/2026-08-01-yunke-data-dictionary.md``) to a SQL table.
|
|
Models target **MySQL in production** but keep column types portable so the
|
|
suite can exercise them on **in-memory SQLite** (see ``tests/test_models.py``).
|
|
|
|
Conventions
|
|
-----------
|
|
* SQLAlchemy 2.0 ``Mapped``/``mapped_column`` style; all classes extend
|
|
``app.db.base.Base``.
|
|
* API field names are mapped to ``snake_case`` columns; the original payload is
|
|
preserved verbatim in ``raw_json`` (``sa.JSON``).
|
|
* Common audit columns on every source-mirrored table:
|
|
- ``version`` — source update stamp (string when the API gives a version
|
|
token, BigInteger when it is a ms timestamp).
|
|
- ``raw_json`` — the original API record (``sa.JSON``).
|
|
- ``fetched_at`` — when we pulled the row (server default ``now()``).
|
|
- ``updated_at`` — when the row was last written (server default + onupdate).
|
|
* ``channel`` is only added where it is *meaningful*:
|
|
- Communication tables (``Customer``, ``Message``, ``Media``, ``ChatGroup``,
|
|
``GroupMember``, ``Sms``) use ``channel`` in {``wechat``, ``whatsapp``,
|
|
``sms``}.
|
|
- ``PhoneCall`` uses ``channel`` to mean call-type in {``phone_call``,
|
|
``voice_call``} (calls are not tied to a messaging channel).
|
|
- ``SyncState`` uses ``channel`` to scope a sync cursor (together with
|
|
``entity_type``).
|
|
- ``Staff``, ``Department``, ``WechatAccount``, ``WhatsappAccount``,
|
|
``YkAccount``, ``SyncRun`` carry no ``channel`` column.
|
|
* Primary keys use the natural source id where one exists; otherwise an
|
|
integer surrogate PK is used (``SyncState``, ``YkAccount``, ``SyncRun``).
|
|
* ``GroupMember`` uses a composite PK ``(channel, room_id, member_id)``.
|
|
|
|
The ``Message`` model currently maps to a single physical ``message`` table.
|
|
Real-world volume is expected at millions of rows/month, so the long-term plan
|
|
is time-sharded physical tables named via :func:`message_table_name`. That
|
|
sharding is a LATER task — for now every message row lives in ``message`` and
|
|
the helper below is provided so call-sites can already agree on the naming
|
|
convention.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import datetime as dt
|
|
from typing import Any
|
|
|
|
import sqlalchemy as sa
|
|
from sqlalchemy import BigInteger, Boolean, DateTime, Index, Integer, String, Text
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
from sqlalchemy.types import JSON
|
|
|
|
from app.db.base import Base
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def message_table_name(channel: str, timestamp: int | dt.datetime) -> str:
|
|
"""Return the conventional sharded table name for a message.
|
|
|
|
Format: ``message_<channel>_<YYYYMM>`` where ``YYYYMM`` is derived from the
|
|
message timestamp. ``timestamp`` may be either:
|
|
|
|
* a Unix epoch in **milliseconds** (``int``) — the form used by the
|
|
``Message.timestamp`` column and the yunke API, or
|
|
* a ``datetime.datetime`` instance (any tz).
|
|
|
|
Examples
|
|
--------
|
|
>>> import datetime as dt
|
|
>>> message_table_name("wechat", 1_700_000_000_000)
|
|
'message_wechat_202311'
|
|
>>> message_table_name("whatsapp", dt.datetime(2024, 1, 5))
|
|
'message_whatsapp_202401'
|
|
"""
|
|
if isinstance(timestamp, dt.datetime):
|
|
ts = timestamp
|
|
else:
|
|
# Treat int as ms-since-epoch (UTC). ``float`` also tolerated.
|
|
ts = dt.datetime.fromtimestamp(int(timestamp) / 1000, tz=dt.UTC)
|
|
return f"message_{channel}_{ts.strftime('%Y%m')}"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Shared column defaults
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _fetched_at() -> Mapped[dt.datetime]:
|
|
return mapped_column(
|
|
DateTime, server_default=sa.func.now(), nullable=False
|
|
)
|
|
|
|
|
|
def _updated_at() -> Mapped[dt.datetime]:
|
|
return mapped_column(
|
|
DateTime,
|
|
server_default=sa.func.now(),
|
|
onupdate=sa.func.now(),
|
|
nullable=False,
|
|
)
|
|
|
|
|
|
# ===========================================================================
|
|
# 1. Organization & people
|
|
# ===========================================================================
|
|
|
|
|
|
class Department(Base):
|
|
"""云客 部门/分组 — source: ``POST /usercenter/dept/tree``."""
|
|
|
|
__tablename__ = "department"
|
|
|
|
dept_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
|
parent_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
path: Mapped[str | None] = mapped_column(String(512), nullable=True)
|
|
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
|
|
class Staff(Base):
|
|
"""员工 — embedded in account lists (``userId`` + ``userName``)."""
|
|
|
|
__tablename__ = "staff"
|
|
|
|
user_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
|
name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
sale_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
dept_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
is_on_job: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
|
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
|
|
# ===========================================================================
|
|
# 2. Staff messaging accounts (channel identity, so no `channel` column)
|
|
# ===========================================================================
|
|
|
|
|
|
class WechatAccount(Base):
|
|
"""员工微信号 — source: ``POST /wechatweb/wxUser/list`` (wechatId PK)."""
|
|
|
|
__tablename__ = "wechat_account"
|
|
|
|
wx_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
|
nick_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
alias: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
user_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
contacts_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
group_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
send_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
receive_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
is_on_job: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
|
db_size: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
online_status: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
|
last_update: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
|
|
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
|
|
class WhatsappAccount(Base):
|
|
"""员工 WhatsApp 账号 — source: ``POST /qqweb/WhatsAppUser/businessSalesList``.
|
|
|
|
PK is ``uid`` of the form ``<phone>@s.whatsapp.net``.
|
|
"""
|
|
|
|
__tablename__ = "whatsapp_account"
|
|
|
|
uid: Mapped[str] = mapped_column(String(128), primary_key=True)
|
|
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
user_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
nick_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
sale_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
sales_bind_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
contacts_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
group_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
talker_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
friend_add_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
is_on_job: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
|
last_update: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
|
|
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
|
|
# ===========================================================================
|
|
# 3. Communication entities (carry a meaningful `channel` column)
|
|
# ===========================================================================
|
|
|
|
|
|
class Customer(Base):
|
|
"""客户/好友 (WeChat friend OR WhatsApp contact OR SMS peer).
|
|
|
|
Source: ``POST /wechatweb/wxFriends/friend_list/es`` (WeChat) or
|
|
``POST /qqweb/WhatsAppUser/businessFriendList`` (WhatsApp). PK is the
|
|
source friend id: ``friendWxId`` for WeChat, ``<phone>@s.whatsapp.net``
|
|
for WhatsApp.
|
|
"""
|
|
|
|
__tablename__ = "customer"
|
|
|
|
friend_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
|
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
owner_account_id: Mapped[str | None] = mapped_column(
|
|
String(128), nullable=True
|
|
) # wxId (WeChat) / uid (WhatsApp) — owning staff account
|
|
friend_nickname: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
friend_remark: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
real_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
wechat_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
gender: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
region: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
description: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
sale_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
dept_route: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
dept_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
head_image_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
last_chat_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
send_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
receive_num: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
message_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
delete_flag: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
is_block: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
|
friend_state: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
|
|
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (
|
|
Index(
|
|
"ix_customer_owner_friend",
|
|
"channel",
|
|
"owner_account_id",
|
|
"friend_id",
|
|
),
|
|
)
|
|
|
|
|
|
class ChatGroup(Base):
|
|
"""聊天群 — source: ``POST /wechatweb/chatroom/getChatroomList``.
|
|
|
|
PK ``room_id`` is the source ``chatRoomId`` (ends with ``@chatroom``).
|
|
The raw ``memberIds`` (semicolon-joined wxids) is kept verbatim in
|
|
``member_ids``; the parsed members are mirrored into :class:`GroupMember`.
|
|
"""
|
|
|
|
__tablename__ = "chat_group"
|
|
|
|
room_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
|
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
room_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
owner_account_id: Mapped[str | None] = mapped_column(
|
|
String(128), nullable=True
|
|
) # wechatId / uid of owning staff account
|
|
owner_account_nickname: Mapped[str | None] = mapped_column(
|
|
String(255), nullable=True
|
|
)
|
|
member_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
member_ids: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
head_image_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
owner_user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
owner_user_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
message_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
own_message_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
loss_members: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
add_members: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
last_chat_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
update_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
|
|
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (
|
|
Index(
|
|
"ix_chat_group_owner",
|
|
"channel",
|
|
"owner_account_id",
|
|
"room_id",
|
|
),
|
|
)
|
|
|
|
|
|
class GroupMember(Base):
|
|
"""群成员 — composite PK (channel, room_id, member_id).
|
|
|
|
Source: the ``memberIds`` field of a chatroom payload, parsed into one row
|
|
per member. Per-member nickname/avatar may require a separate API call
|
|
(see data-dictionary §9.6) and are therefore nullable.
|
|
"""
|
|
|
|
__tablename__ = "group_member"
|
|
|
|
channel: Mapped[str] = mapped_column(String(16), primary_key=True)
|
|
room_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
|
member_id: Mapped[str] = mapped_column(String(128), primary_key=True)
|
|
nickname: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
avatar: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
display_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
|
|
class Message(Base):
|
|
"""聊天消息 — WeChat ``msgSvrId`` / WhatsApp ``keyId`` as natural PK.
|
|
|
|
Sources:
|
|
* WeChat: ``POST /wechatweb/msg/chats``
|
|
* WhatsApp: ``POST /qqweb/WhatsAppUser/chatRecords``
|
|
|
|
The model currently maps to a **single** ``message`` table. The real
|
|
sharding across ``message_<channel>_<YYYYMM>`` physical tables (see
|
|
:func:`message_table_name`) is a LATER task; this single-table mapping is
|
|
sufficient for ingestion v1.
|
|
"""
|
|
|
|
__tablename__ = "message"
|
|
|
|
msg_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
|
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
owner_account_id: Mapped[str | None] = mapped_column(
|
|
String(128), nullable=True
|
|
) # wechatId / uid of sender staff account
|
|
talker_id: Mapped[str | None] = mapped_column(
|
|
String(128), nullable=True
|
|
) # talker / friendUid — the other party
|
|
ori_talker: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
|
room_id: Mapped[str | None] = mapped_column(
|
|
String(128), nullable=True
|
|
) # @chatroom id; NULL/empty = direct message
|
|
is_send: Mapped[bool | None] = mapped_column(Boolean, nullable=True) # mine/isSend
|
|
type: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
text: Mapped[str | None] = mapped_column(Text, nullable=True) # content/text
|
|
file_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
file_th_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
file_hd_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
md5: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
md5_th: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
md5_hd: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
duration: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
transcription_text: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
img_path: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
original_image: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
pic_type: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
pay_msg_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
refer_msg: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
sender_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
sender_head_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
timestamp: Mapped[int | None] = mapped_column(BigInteger, nullable=True) # ms epoch
|
|
is_del: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
|
|
|
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (
|
|
# Hot query path: per-account, per-talker, time-ordered pagination.
|
|
Index(
|
|
"ix_message_channel_owner_talker_time",
|
|
"channel",
|
|
"owner_account_id",
|
|
"talker_id",
|
|
"timestamp",
|
|
),
|
|
)
|
|
|
|
|
|
class Media(Base):
|
|
"""Locally cached media (image / voice / video / file) — keyed by hash.
|
|
|
|
Deduplicates downloads across messages that reference the same file (via
|
|
``md5``). ``local_path`` is set once the blob has been fetched into the
|
|
local media store.
|
|
"""
|
|
|
|
__tablename__ = "media"
|
|
|
|
media_hash: Mapped[str] = mapped_column(String(64), primary_key=True)
|
|
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
media_type: Mapped[str | None] = mapped_column(
|
|
String(32), nullable=True
|
|
) # image|voice|video|file|...
|
|
original_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
original_th_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
original_hd_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
local_path: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
size_bytes: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
mime_type: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
|
duration: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (
|
|
Index("ix_media_type_channel", "media_type", "channel"),
|
|
)
|
|
|
|
|
|
# ===========================================================================
|
|
# 4. Phone calls & SMS (prefix /pc/)
|
|
# ===========================================================================
|
|
|
|
|
|
class PhoneCall(Base):
|
|
"""电话/语音通话记录 — source: ``POST /pc/callLog/list`` (phone calls) and
|
|
``POST /wechatweb/callRecord/queryWeChatVoiceList`` (voice calls).
|
|
|
|
``channel`` here means **call-type** in {``phone_call``, ``voice_call``}
|
|
(calls are not tied to a messaging channel). PK is the source ``id``.
|
|
"""
|
|
|
|
__tablename__ = "phone_call"
|
|
|
|
call_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
|
channel: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
start_call_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
call_duration: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
ring_time: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
call_status: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
incoming_call: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
user_id_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
department_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
call_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
sim_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
sim_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
imei: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
|
record_file: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
|
is_upload: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
|
download_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
ai_asr: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
qc_status: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
plan_customer_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
|
|
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (
|
|
Index("ix_phone_call_user_time", "user_id", "start_call_time"),
|
|
)
|
|
|
|
|
|
class Sms(Base):
|
|
"""短信记录 — source: ``POST /pc/customer/smsLogsList``."""
|
|
|
|
__tablename__ = "sms"
|
|
|
|
sms_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
|
channel: Mapped[str] = mapped_column(
|
|
String(16), nullable=False, default="sms"
|
|
) # always 'sms' today; kept for symmetry
|
|
customer_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
content: Mapped[str | None] = mapped_column(Text, nullable=True) # full sms text
|
|
content_preview: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
sender: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
|
direction: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
sender_department: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
sender_department_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
receive_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
create_time: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
sim_phone: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
sim_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
source: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
|
|
version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (Index("ix_sms_user_time", "user_id", "create_time"),)
|
|
|
|
|
|
# ===========================================================================
|
|
# 5. Operational / bookkeeping tables (local; no raw_json/version)
|
|
# ===========================================================================
|
|
|
|
|
|
class SyncState(Base):
|
|
"""Resumable sync cursor — scoped by (entity_type, channel, scope_id).
|
|
|
|
``scope_id`` qualifies the cursor for a sub-stream (e.g. the owning staff
|
|
account id when syncing per-account message streams). ``last_version``
|
|
holds the opaque source update stamp; ``last_timestamp`` the ms epoch.
|
|
"""
|
|
|
|
__tablename__ = "sync_state"
|
|
|
|
state_id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
|
entity_type: Mapped[str] = mapped_column(String(32), nullable=False)
|
|
channel: Mapped[str | None] = mapped_column(String(16), nullable=True)
|
|
scope_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
|
last_version: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
last_timestamp: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
last_synced_at: Mapped[dt.datetime | None] = mapped_column(
|
|
DateTime, nullable=True
|
|
)
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (
|
|
Index(
|
|
"ix_sync_state_scope",
|
|
"entity_type",
|
|
"channel",
|
|
"scope_id",
|
|
unique=True,
|
|
),
|
|
)
|
|
|
|
|
|
class YkAccount(Base):
|
|
"""云客 CRM login account — credentials / session for the upstream API.
|
|
|
|
The session cookie / password is stored encrypted with
|
|
``Settings.encryption_key`` (Fernet). ``user_id`` + ``company`` together
|
|
identify a yunke session (the two ``userid`` / ``company`` request headers).
|
|
"""
|
|
|
|
__tablename__ = "yk_account"
|
|
|
|
account_id: Mapped[int] = mapped_column(
|
|
Integer, primary_key=True, autoincrement=True
|
|
)
|
|
user_id: Mapped[str] = mapped_column(String(64), nullable=False)
|
|
company: Mapped[str] = mapped_column(String(64), nullable=False)
|
|
username: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
|
session_blob_enc: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
last_login_at: Mapped[dt.datetime | None] = mapped_column(
|
|
DateTime, nullable=True
|
|
)
|
|
is_active: Mapped[bool] = mapped_column(
|
|
Boolean, nullable=False, default=True, server_default=sa.text("1")
|
|
)
|
|
|
|
raw_json: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
|
fetched_at: Mapped[dt.datetime] = _fetched_at()
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (
|
|
Index("ix_yk_account_user_company", "user_id", "company", unique=True),
|
|
)
|
|
|
|
|
|
class SyncRun(Base):
|
|
"""Audit log for one sync execution (start / end / status / counts)."""
|
|
|
|
__tablename__ = "sync_run"
|
|
|
|
run_id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
|
entity_type: Mapped[str] = mapped_column(String(32), nullable=False)
|
|
channel: Mapped[str | None] = mapped_column(String(16), nullable=True)
|
|
scope_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
|
status: Mapped[str] = mapped_column(String(16), nullable=False) # running|success|failed
|
|
started_at: Mapped[dt.datetime] = mapped_column(
|
|
DateTime, server_default=sa.func.now(), nullable=False
|
|
)
|
|
finished_at: Mapped[dt.datetime | None] = mapped_column(DateTime, nullable=True)
|
|
records_fetched: Mapped[int] = mapped_column(
|
|
Integer, nullable=False, default=0, server_default=sa.text("0")
|
|
)
|
|
records_upserted: Mapped[int] = mapped_column(
|
|
Integer, nullable=False, default=0, server_default=sa.text("0")
|
|
)
|
|
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
|
|
updated_at: Mapped[dt.datetime] = _updated_at()
|
|
|
|
__table_args__ = (Index("ix_sync_run_entity_status", "entity_type", "status"),)
|
|
|
|
|
|
__all__ = [
|
|
"ChatGroup",
|
|
"Customer",
|
|
"Department",
|
|
"GroupMember",
|
|
"Media",
|
|
"Message",
|
|
"PhoneCall",
|
|
"Sms",
|
|
"Staff",
|
|
"SyncRun",
|
|
"SyncState",
|
|
"WechatAccount",
|
|
"WhatsappAccount",
|
|
"YkAccount",
|
|
"message_table_name",
|
|
]
|