feat: wechat voice-call scraper + group message coverage
- scrape_wechat_voice_calls: wraps POST /wechatweb/callRecord/queryWeChatVoiceList with page-number pagination (reuses paginate_by_page); normalizes records to the PhoneCall column shape with channel='voice_call'. Field mapping per phase0-findings §4: id->call_id, duration(str s)->call_duration(int), startTime(YYYY-MM-DD HH:MM:SS)->start_call_time(ms epoch), callStatus-> call_status, talker->call_number, talkerNickName->plan_customer_name, ossFileName->record_file (may be null). isSend (0=in/1=out) is inverted to incoming_call (1=in/0=out) so the column's meaning matches the phone_call channel. - upsert_voice_call: idempotent Core upsert into PhoneCall by call_id (sqlite/mysql dialect-aware). Adds call_duration/start_call_time to the _INT_FIELDS coercion net (scraper already emits ints; defensive for hand-built records). - Group-message test (Task 2.3): locks in that scrape_wechat_messages already handles chatroom conversations (friendId=@chatroom, roomid set, talker=member wxid) — yields room_id populated and talker_id = the sending member. No scraper change required. Suite: 123 -> 130 passing. ruff clean. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -19,13 +19,18 @@ Upstream CRM values are not always typed the way the columns want them. The
|
||||
following coercions are applied when building column values from the input
|
||||
dict (see :func:`_coerce_bool`, :func:`_coerce_int`):
|
||||
|
||||
* Booleans (``is_block`` / ``is_send`` / ``is_del`` / ``is_on_job``):
|
||||
``1`` / ``"1"`` / ``True`` -> ``True``; ``0`` / ``"0"`` / ``False`` ->
|
||||
``False``; ``None`` preserved.
|
||||
* Booleans (``is_block`` / ``is_send`` / ``is_del`` / ``is_on_job`` /
|
||||
``is_upload``): ``1`` / ``"1"`` / ``True`` -> ``True``; ``0`` / ``"0"`` /
|
||||
``False`` -> ``False``; ``None`` preserved.
|
||||
* ``gender``: ``"0"`` / ``"1"`` / ``"2"`` (str) -> ``int`` when not ``None``.
|
||||
* ``type_raw`` (Message): the raw upstream type code, coerced to
|
||||
``int | None``. ``Message.type`` is a normalized label string and is
|
||||
stored as-is (no int coercion).
|
||||
* PhoneCall int fields (``call_duration`` / ``start_call_time`` /
|
||||
``call_status`` / ``incoming_call`` / ``qc_status``): coerced to
|
||||
``int | None``. The voice-call scraper already parses these to ints
|
||||
(incl. ``startTime`` datetime -> ms epoch); the repo coercion is a
|
||||
defensive net for hand-built records that arrive as numeric strings.
|
||||
* Timestamps (``last_chat_time`` / ``timestamp`` / ``last_update`` / ...):
|
||||
passed through as-is (ms-epoch ``int`` or ``None``).
|
||||
* ``raw_json``: the original record dict, preserved verbatim. When the input
|
||||
@@ -54,6 +59,7 @@ from app.db.models import (
|
||||
Department,
|
||||
GroupMember,
|
||||
Message,
|
||||
PhoneCall,
|
||||
Staff,
|
||||
WechatAccount,
|
||||
)
|
||||
@@ -66,6 +72,7 @@ __all__ = [
|
||||
"upsert_messages",
|
||||
"upsert_staff",
|
||||
"upsert_staff_from_account",
|
||||
"upsert_voice_call",
|
||||
"upsert_wechat_account",
|
||||
]
|
||||
|
||||
@@ -90,6 +97,8 @@ _INT_FIELDS: frozenset[str] = frozenset(
|
||||
"direction",
|
||||
"incoming_call",
|
||||
"call_status",
|
||||
"call_duration",
|
||||
"start_call_time",
|
||||
"qc_status",
|
||||
}
|
||||
)
|
||||
@@ -443,3 +452,25 @@ async def upsert_chat_group(session: AsyncSession, rec: dict[str, Any]) -> None:
|
||||
"member_id": member_id,
|
||||
}
|
||||
await _upsert_one(session, GroupMember, member_rec)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PhoneCall (voice_call) upsert (M2, Task 2.4)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def upsert_voice_call(session: AsyncSession, rec: dict[str, Any]) -> None:
|
||||
"""Idempotently upsert one :class:`PhoneCall` (PK = ``call_id``).
|
||||
|
||||
Same ``call_id`` twice -> one row, with non-PK fields refreshed from the
|
||||
later record. Applies the documented int coercions (``call_duration`` /
|
||||
``start_call_time`` / ``call_status`` / ``incoming_call`` -> int when they
|
||||
arrive as numeric strings; the voice-call scraper already emits ints, but
|
||||
a hand-built record may not). ``is_upload`` is a bool field and goes
|
||||
through :func:`_coerce_bool` like the other bool columns.
|
||||
|
||||
The voice-call channel (``channel="voice_call"``) and the phone-call
|
||||
channel (``channel="phone_call"``) share the ``PhoneCall`` table; the
|
||||
per-row ``channel`` value disambiguates them.
|
||||
"""
|
||||
await _upsert_one(session, PhoneCall, rec)
|
||||
|
||||
Reference in New Issue
Block a user