feat: department + wechat account/staff scrapers
M2 organization scrapers for yunke-server: - scrape_departments: POST /usercenter/dept/tree. Single request (no pagination). Defensively unwraps the envelope (data is node list OR an object wrapping nodes under children), flattens BOTH nested and flat shapes, and reconstructs the materialized path by walking the pId chain (ancestor names joined with /). Maps id/pId/name -> dept_id/parent_id/name. - scrape_wechat_accounts: POST /wechatweb/wxUser/list. Page-number pagination via the existing paginate_by_page helper. Envelope differs from friend_list (no basicResult wrapper): data.data/data.total. Maps wechatId/nickName/alias/phone/userId/userName/contactsNum/groupNum/ sendNum/receiveNum/isOnJob/dbSize/onlineStatus/lastUpdate/version to WechatAccount columns; carries embedded staff pair for the repo layer. Repos: - upsert_department: idempotent upsert into Department (PK dept_id). - upsert_staff: primitive idempotent staff upsert (PK user_id, is_on_job coerced to bool). - upsert_wechat_account: now ALSO upserts the embedded Staff row via upsert_staff_from_account (best-effort; skips when account has no user_id). - upsert_staff_from_account: refactored to delegate to upsert_staff and forward phone/sale_phone/is_on_job when present. Tests (TDD red->green): test_scrape_departments (nested + flat + wrapped-data + empty), test_scrape_wechat_accounts (2-page pagination + body advance + empty first page), test_repos extended (dept idempotency/root parent, direct staff upsert + bool coercion, account-also-upserts-staff, re-upsert updates both, no-user_id skips staff). Cassettes: dept_tree_fixture (same 4 nodes in both nested + flat shapes), wxUser_fixture (4 capture-shaped accounts incl. departed + orphan). Suite: 97 -> 111 passing. ruff check clean. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
浅念
@@ -0,0 +1,107 @@
|
||||
"""Fixture payloads for ``POST /usercenter/dept/tree`` (Task 2.0).
|
||||
|
||||
yunke's dept/tree endpoint follows the same convention as the permissions API:
|
||||
nodes carry ``id`` / ``pId`` / ``name``, and the tree MAY be delivered either
|
||||
as a flat list of nodes or as a nested tree (each node optionally carrying a
|
||||
``children`` array). The scraper must flatten both shapes into one flat list
|
||||
of nodes and reconstruct ``parent_id`` from ``pId``.
|
||||
|
||||
This cassette provides TWO realistic capture-shaped payloads so the scraper
|
||||
tests can prove both shapes flatten to the SAME set of nodes:
|
||||
|
||||
* ``NESTED_TREE_PAYLOAD`` — a root with two children, one of which has its own
|
||||
sub-child (3 levels deep). Nested via ``children``.
|
||||
* ``FLAT_TREE_PAYLOAD`` — the exact same 4 nodes delivered as a flat list
|
||||
(parent links carried by ``pId``).
|
||||
|
||||
Both decode to the same 4 departments with the same ids / parents / names /
|
||||
paths, so the scraper test can swap the envelope and re-assert.
|
||||
|
||||
The 4 nodes:
|
||||
root "Sales" (id="1", pId="0")
|
||||
├── "East" (id="1.1", pId="1")
|
||||
│ └── "TeamA" (id="1.1.1", pId="1.1")
|
||||
└── "North" (id="1.2", pId="1")
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The canonical 4 nodes (single source of truth for both shapes below)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Root has pId="0" — the common yunke convention for "no parent". The scraper
|
||||
# treats any falsy pId (None / "" / "0") OR an unknown pId as "root".
|
||||
ROOT_NODE: dict[str, Any] = {"id": "1", "pId": "0", "name": "Sales"}
|
||||
EAST_NODE: dict[str, Any] = {"id": "1.1", "pId": "1", "name": "East"}
|
||||
TEAMA_NODE: dict[str, Any] = {"id": "1.1.1", "pId": "1.1", "name": "TeamA"}
|
||||
NORTH_NODE: dict[str, Any] = {"id": "1.2", "pId": "1", "name": "North"}
|
||||
|
||||
# All four nodes, in a stable order matching a depth-first traversal of the
|
||||
# nested tree. Tests assert the scraper yields exactly these (order is not
|
||||
# load-bearing — tests compare as sets where appropriate).
|
||||
ALL_NODES: list[dict[str, Any]] = [ROOT_NODE, EAST_NODE, TEAMA_NODE, NORTH_NODE]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Nested envelope — root carries children, recursively
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
NESTED_TREE_NODES: list[dict[str, Any]] = [
|
||||
{
|
||||
**ROOT_NODE,
|
||||
"children": [
|
||||
{
|
||||
**EAST_NODE,
|
||||
"children": [{**TEAMA_NODE}],
|
||||
},
|
||||
{**NORTH_NODE},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Flat envelope — every node at the top level, parents via pId
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
FLAT_TREE_NODES: list[dict[str, Any]] = [
|
||||
{**ROOT_NODE},
|
||||
{**EAST_NODE},
|
||||
{**TEAMA_NODE},
|
||||
{**NORTH_NODE},
|
||||
]
|
||||
|
||||
|
||||
def make_dept_tree_response(
|
||||
nodes: list[dict[str, Any]],
|
||||
*,
|
||||
nested_in_data_object: bool = False,
|
||||
trace_id: str = "trace-dept",
|
||||
) -> dict[str, Any]:
|
||||
"""Build a capture-shaped ``dept/tree`` response envelope.
|
||||
|
||||
The yunke dept/tree response top-level is the standard envelope
|
||||
``{message, code:"10000", success, traceId, data}``. The shape of ``data``
|
||||
itself varies between captures — be defensive:
|
||||
|
||||
* ``data`` is the node list directly (the common case), OR
|
||||
* ``data`` is an object wrapping the node list under ``children`` /
|
||||
``list`` / etc.
|
||||
|
||||
The ``nested_in_data_object`` flag selects the second variant for the
|
||||
"wrapped data object" path so a test can prove the scraper unwraps it.
|
||||
The default (``False``) emits the most common flat-data array shape.
|
||||
"""
|
||||
data: Any = list(nodes)
|
||||
if nested_in_data_object:
|
||||
data = {"children": list(nodes)}
|
||||
return {
|
||||
"message": "调用成功",
|
||||
"code": "10000",
|
||||
"success": True,
|
||||
"traceId": trace_id,
|
||||
"data": data,
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
"""Fixture payloads for ``POST /wechatweb/wxUser/list`` (Task 2.1).
|
||||
|
||||
These records mirror the real field names captured from the live CRM (see the
|
||||
data dictionary §2: ``wechatId, nickName, alias, phone, userId,
|
||||
userName(员工真名), contactsNum, groupNum, sendNum/receiveNum, isOnJob,
|
||||
dbSize, 在线状态, lastUpdate``; total=55).
|
||||
|
||||
The envelope differs from ``wxFriends/friend_list/es``: there is **no**
|
||||
``basicResult`` wrapper. The pagination fields sit directly under ``data``::
|
||||
|
||||
{"code":"10000","data":{"data":[...], "total":.., "pageNum":.., "pageSize":..}}
|
||||
|
||||
The cassette ships two pages so a test can drive the page-number paginator
|
||||
across the empty-page stop signal:
|
||||
|
||||
* page 1 -> 2 of the 4 fixture records, ``total=4``
|
||||
* page 2 -> the other 2 records, ``total=4``
|
||||
* (page 3 -> empty ``data`` array, served inline by the test)
|
||||
|
||||
Records carry every field with a target ``WechatAccount`` column so the test
|
||||
can assert a full mapping end-to-end, plus the embedded ``userId``+``userName``
|
||||
staff pair (the owning employee) so the embedded-staff repo upsert can be
|
||||
exercised against capture-shaped input.
|
||||
"""
|
||||
|
||||
# The module name uses camelCase deliberately to mirror the upstream endpoint
|
||||
# name (``wxUser``); the Task 2.1 spec mandates this exact filename.
|
||||
# ruff: noqa: N999
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Capture-shaped WeChat staff-account records (4 total across 2 pages).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
WXUSER_RECORDS: list[dict[str, Any]] = [
|
||||
{
|
||||
"wechatId": "wxid_staff_alice01",
|
||||
"nickName": "Alice WeChat",
|
||||
"alias": "alice_wx",
|
||||
"phone": "13800000001",
|
||||
"userId": "staff_user_001",
|
||||
"userName": "Alice Wang",
|
||||
"contactsNum": 312,
|
||||
"groupNum": 18,
|
||||
"sendNum": 1204,
|
||||
"receiveNum": 2387,
|
||||
"isOnJob": 1,
|
||||
"dbSize": 482103,
|
||||
"onlineStatus": "online",
|
||||
"headImageUrl": "https://wx.qlogo.cn/avatar_alice.png",
|
||||
"lastUpdate": 1700000000123,
|
||||
"version": "v1.20260731.alice",
|
||||
},
|
||||
{
|
||||
"wechatId": "wxid_staff_bob02",
|
||||
"nickName": "Bob WeChat",
|
||||
"alias": "bob_wx",
|
||||
"phone": "13800000002",
|
||||
"userId": "staff_user_002",
|
||||
"userName": "Bob Chen",
|
||||
"contactsNum": 87,
|
||||
"groupNum": 4,
|
||||
"sendNum": 211,
|
||||
"receiveNum": 402,
|
||||
"isOnJob": 1,
|
||||
"dbSize": 91205,
|
||||
"onlineStatus": "offline",
|
||||
"headImageUrl": "https://wx.qlogo.cn/avatar_bob.png",
|
||||
"lastUpdate": 1700000000456,
|
||||
"version": "v1.20260731.bob",
|
||||
},
|
||||
{
|
||||
"wechatId": "wxid_staff_carol03",
|
||||
"nickName": "Carol We Chat",
|
||||
"alias": "carol_wx",
|
||||
"phone": "13800000003",
|
||||
"userId": "staff_user_003",
|
||||
"userName": "Carol Liu",
|
||||
"contactsNum": 1502,
|
||||
"groupNum": 72,
|
||||
"sendNum": 9034,
|
||||
"receiveNum": 11245,
|
||||
"isOnJob": 0, # departed staff — exercises the falsy isOnJob path
|
||||
"dbSize": 2_100_445,
|
||||
"onlineStatus": "offline",
|
||||
"headImageUrl": "https://wx.qlogo.cn/avatar_carol.png",
|
||||
"lastUpdate": 1700000000789,
|
||||
"version": "v1.20260731.carol",
|
||||
},
|
||||
{
|
||||
# Account with NO embedded staff (userId missing) — the scraper must
|
||||
# still yield it; the embedded-staff repo upsert must skip it cleanly.
|
||||
"wechatId": "wxid_staff_orphan04",
|
||||
"nickName": "Orphan WeChat",
|
||||
"alias": "orphan_wx",
|
||||
"phone": "13800000004",
|
||||
"userId": None,
|
||||
"userName": None,
|
||||
"contactsNum": 5,
|
||||
"groupNum": 0,
|
||||
"sendNum": 1,
|
||||
"receiveNum": 2,
|
||||
"isOnJob": 0,
|
||||
"dbSize": 1024,
|
||||
"onlineStatus": "offline",
|
||||
"headImageUrl": "https://wx.qlogo.cn/avatar_orphan.png",
|
||||
"lastUpdate": 1700000001000,
|
||||
"version": "v1.20260731.orphan",
|
||||
},
|
||||
]
|
||||
|
||||
# Default page size for the fixture (and the scraper). total=4 means the
|
||||
# fixture fills exactly 2 pages of size 2 plus a terminating empty page.
|
||||
TOTAL = len(WXUSER_RECORDS)
|
||||
DEFAULT_PAGE_SIZE = 2
|
||||
|
||||
|
||||
def make_wxuser_list_response(
|
||||
records: list[dict[str, Any]],
|
||||
*,
|
||||
total: int,
|
||||
page_num: int = 1,
|
||||
page_size: int = DEFAULT_PAGE_SIZE,
|
||||
trace_id: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Build a capture-shaped ``wxUser/list`` response envelope.
|
||||
|
||||
Shape (no ``basicResult`` wrapper)::
|
||||
|
||||
{"message":"调用成功","code":"10000","success":true,"traceId":"...",
|
||||
"data":{"data":[...], "total":.., "pageNum":.., "pageSize":..}}
|
||||
"""
|
||||
return {
|
||||
"message": "调用成功",
|
||||
"code": "10000",
|
||||
"success": True,
|
||||
"traceId": trace_id or f"trace-{page_num}",
|
||||
"data": {
|
||||
"data": list(records),
|
||||
"total": total,
|
||||
"pageNum": page_num,
|
||||
"pageSize": page_size,
|
||||
},
|
||||
}
|
||||
Reference in New Issue
Block a user