Backend (scrape_wechat_accounts):
- Send wxUser/list as form-urlencoded (not JSON) with the REQUIRED filters
isOnJob / departmentId / startdate / enddate + empty userId/salesname/
weChatNickName, matching the live request (empty fails with code -90000).
- scrape_wechat_accounts now takes department_id (the company-root dept id)
plus optional start/end defaulting to a recent 7-day UTC window.
- paginate_by_page gains a form=True mode (data= + form content-type).
- Orchestrator sync_wechat_accounts resolves the root dept from the
Department table (or accepts one) and skips with a warning when unknown.
- seed_accounts.py resolves the root via scrape_departments.
Frontend (yunkes):
- Add StaffList.vue (/staff) with is_on_job filter + pagination.
- Add /staff route and a 员工 nav item between 客户 and 群.
Co-Authored-By: Claude <noreply@anthropic.com>
Task 2.7 — the coordination layer tying scrapers + repos together with
resumable, incremental sync.
app/sync/watermarks.py — SyncState CRUD keyed on (entity_type, channel,
scope_id): get_watermark / get_last_timestamp / get_last_version /
set_watermark (upsert). A partial advance (timestamp-only) does not clobber
the other cursor with NULL.
app/sync/orchestrator.py — the sync brain:
* sync_wechat_messages: time-window incremental. Reads last_timestamp for
(message, wechat, "<wechat_id>:<friend_id>"), queries [last_ts, now] via
scrape_wechat_messages, upserts each, best-effort downloads media, advances
the watermark to max(timestamp). A re-run with no new messages queries an
empty window and fetches nothing — the watermark advanced past.
* sync_{departments,wechat_accounts,wechat_friends,wechat_groups,
wechat_voice_calls}: full re-scrape + idempotent upsert (these list APIs
have no since-param; 'incremental' = re-scrape all, rely on PK idempotency).
Watermark records last_synced_at only.
* sync_all_wechat: coordinator running departments -> accounts -> friends ->
groups -> voice_calls, then per-conversation messages for an explicit
(wechat_id, friend_id) list. Auto-discovery deferred.
Each run logs a SyncRun row (started -> success/failed, counts, error).
Failure policy: a scraper that raises is caught, recorded as failed, and a
failed SyncResult is returned — the orchestrator does not re-raise, so one
broken entity does not abort the batch. Media-download failures are
best-effort. The orchestrator never commits; callers own the transaction.
Tests (TDD red -> green; in-memory SQLite; scrapers/repos monkeypatched):
- watermarks: set/get, upsert (same key twice -> 1 row), partial-advance
preserves the other cursor, defaults when absent.
- orchestrator messages: first run upserts 2 + advances watermark to T2;
second run scrape called with start==T2 and upserts 0 (incremental proof);
media failure does not abort; SyncRun success recorded.
- orchestrator friends: 3 records upserted, SyncRun success, watermark
last_synced_at set.
- failure policy: scraper raises -> SyncRun failed + failed result, no raise
(both friends and message paths).
- sync_all_wechat: 5 entities + 2 message runs in order, all success.
Suite: 149 -> 163 passing. ruff clean.
Co-Authored-By: Claude <noreply@anthropic.com>
浅念