feat: wechat voice-call scraper + group message coverage

- scrape_wechat_voice_calls: wraps POST /wechatweb/callRecord/queryWeChatVoiceList
  with page-number pagination (reuses paginate_by_page); normalizes records to
  the PhoneCall column shape with channel='voice_call'. Field mapping per
  phase0-findings §4: id->call_id, duration(str s)->call_duration(int),
  startTime(YYYY-MM-DD HH:MM:SS)->start_call_time(ms epoch), callStatus->
  call_status, talker->call_number, talkerNickName->plan_customer_name,
  ossFileName->record_file (may be null). isSend (0=in/1=out) is inverted to
  incoming_call (1=in/0=out) so the column's meaning matches the phone_call
  channel.
- upsert_voice_call: idempotent Core upsert into PhoneCall by call_id
  (sqlite/mysql dialect-aware). Adds call_duration/start_call_time to the
  _INT_FIELDS coercion net (scraper already emits ints; defensive for
  hand-built records).
- Group-message test (Task 2.3): locks in that scrape_wechat_messages already
  handles chatroom conversations (friendId=@chatroom, roomid set, talker=member
  wxid) — yields room_id populated and talker_id = the sending member. No
  scraper change required.

Suite: 123 -> 130 passing. ruff clean.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
浅念
2026-08-01 14:03:47 +08:00
parent 2a74d3888d
commit 41b9ec2571
6 changed files with 923 additions and 5 deletions
+34 -3
View File
@@ -19,13 +19,18 @@ Upstream CRM values are not always typed the way the columns want them. The
following coercions are applied when building column values from the input
dict (see :func:`_coerce_bool`, :func:`_coerce_int`):
* Booleans (``is_block`` / ``is_send`` / ``is_del`` / ``is_on_job``):
``1`` / ``"1"`` / ``True`` -> ``True``; ``0`` / ``"0"`` / ``False`` ->
``False``; ``None`` preserved.
* Booleans (``is_block`` / ``is_send`` / ``is_del`` / ``is_on_job`` /
``is_upload``): ``1`` / ``"1"`` / ``True`` -> ``True``; ``0`` / ``"0"`` /
``False`` -> ``False``; ``None`` preserved.
* ``gender``: ``"0"`` / ``"1"`` / ``"2"`` (str) -> ``int`` when not ``None``.
* ``type_raw`` (Message): the raw upstream type code, coerced to
``int | None``. ``Message.type`` is a normalized label string and is
stored as-is (no int coercion).
* PhoneCall int fields (``call_duration`` / ``start_call_time`` /
``call_status`` / ``incoming_call`` / ``qc_status``): coerced to
``int | None``. The voice-call scraper already parses these to ints
(incl. ``startTime`` datetime -> ms epoch); the repo coercion is a
defensive net for hand-built records that arrive as numeric strings.
* Timestamps (``last_chat_time`` / ``timestamp`` / ``last_update`` / ...):
passed through as-is (ms-epoch ``int`` or ``None``).
* ``raw_json``: the original record dict, preserved verbatim. When the input
@@ -54,6 +59,7 @@ from app.db.models import (
Department,
GroupMember,
Message,
PhoneCall,
Staff,
WechatAccount,
)
@@ -66,6 +72,7 @@ __all__ = [
"upsert_messages",
"upsert_staff",
"upsert_staff_from_account",
"upsert_voice_call",
"upsert_wechat_account",
]
@@ -90,6 +97,8 @@ _INT_FIELDS: frozenset[str] = frozenset(
"direction",
"incoming_call",
"call_status",
"call_duration",
"start_call_time",
"qc_status",
}
)
@@ -443,3 +452,25 @@ async def upsert_chat_group(session: AsyncSession, rec: dict[str, Any]) -> None:
"member_id": member_id,
}
await _upsert_one(session, GroupMember, member_rec)
# ---------------------------------------------------------------------------
# PhoneCall (voice_call) upsert (M2, Task 2.4)
# ---------------------------------------------------------------------------
async def upsert_voice_call(session: AsyncSession, rec: dict[str, Any]) -> None:
"""Idempotently upsert one :class:`PhoneCall` (PK = ``call_id``).
Same ``call_id`` twice -> one row, with non-PK fields refreshed from the
later record. Applies the documented int coercions (``call_duration`` /
``start_call_time`` / ``call_status`` / ``incoming_call`` -> int when they
arrive as numeric strings; the voice-call scraper already emits ints, but
a hand-built record may not). ``is_upload`` is a bool field and goes
through :func:`_coerce_bool` like the other bool columns.
The voice-call channel (``channel="voice_call"``) and the phone-call
channel (``channel="phone_call"``) share the ``PhoneCall`` table; the
per-row ``channel`` value disambiguates them.
"""
await _upsert_one(session, PhoneCall, rec)