Add the first list scraper for the yunke data-mirror system, establishing
the reusable pagination pattern all later list scrapers will build on.
- app/scraper/base.py: generic `paginate_by_page` async generator that walks
page-numbered endpoints until a page comes back empty (the yunke CRM's
primary "no more rows" signal), with a defensive over-fetch ceiling keyed
off the declared total (strict `>` so an honest source yields exactly
`total`). Plus a tiny `RateLimiter` enforcing a min interval between calls.
- app/scraper/wechat.py: `scrape_wechat_friends` wraps
`POST /wechatweb/wxFriends/friend_list/es`, maps each record to the
`Customer` column shape (snake_case) with `channel="wechat"` and a verbatim
`raw_json` copy. Optional `department_id`/`user_id` filters pass through to
the request body.
- tests/cassettes/wxFriends_fixture.py: two realistic, capture-shaped friend
records + a response-envelope builder, reused by the scraper tests.
- tests/test_scrape_wechat_friends.py: drives the real pagination +
normalization logic through respx (page1 returns both records, page2 empty
-> stop). Asserts exact field mappings, `raw_json` fidelity, and exactly 2
upstream calls.
- tests/test_scrape_base.py: smoke tests for the RateLimiter spacing.
Persistence (upsert into `Customer`) is deliberately deferred to Task 1.7;
scrapers remain pure producers of normalized dicts.
Co-Authored-By: Claude <noreply@anthropic.com>
浅念