Upstream API records may contain lone surrogate chars (emoji fragments)
that break SQLite/MySQL utf-8 encoding. Now sanitized to U+FFFD before
storage. Includes backfill_customers.py for full customer backfill.
Co-Authored-By: Claude <noreply@anthropic.com>
Live validation: businessFriendList uses 'page' (not pageNum) and REQUIRES
departmentId (root) + type or returns empty. Same class of bug as wxUser/list.
Co-Authored-By: Claude <noreply@anthropic.com>
Backend (scrape_wechat_accounts):
- Send wxUser/list as form-urlencoded (not JSON) with the REQUIRED filters
isOnJob / departmentId / startdate / enddate + empty userId/salesname/
weChatNickName, matching the live request (empty fails with code -90000).
- scrape_wechat_accounts now takes department_id (the company-root dept id)
plus optional start/end defaulting to a recent 7-day UTC window.
- paginate_by_page gains a form=True mode (data= + form content-type).
- Orchestrator sync_wechat_accounts resolves the root dept from the
Department table (or accepts one) and skips with a warning when unknown.
- seed_accounts.py resolves the root via scrape_departments.
Frontend (yunkes):
- Add StaffList.vue (/staff) with is_on_job filter + pagination.
- Add /staff route and a 员工 nav item between 客户 and 群.
Co-Authored-By: Claude <noreply@anthropic.com>
Cleanup after successful live validation (10 real customers scraped+stored):
- app/config.py: add yk_userid / yk_company / yk_cookie Settings fields so
existing-session creds can be supplied via .env (bypasses programmatic
login for validation with a live browser session).
- app/storage/repos.py: coerce list values to ';'-joined strings in
_build_column_values. Fixes a real bug: the live friends-list API returns
Customer.deptRoute as a LIST, but dept_route is a scalar String(255)
column. raw_json still preserves the original list (no data loss).
- scripts/smoke_wechat.py:
* Revert build_client_from_env to read os.environ ONLY (no get_settings
fallbacks), so the monkeypatch.setenv unit tests stay hermetic and the
real .env stops leaking in. 3 previously-red tests green again.
* Add _push_settings_into_environ() (Settings -> os.environ, real env
wins, only fills gaps) and call it as the first line of main(); so the
smoke run still honors .env without the pure builder depending on
Settings. .env -> Settings -> os.environ (once) -> build_client_from_env.
* Keep the await _print_summary fix from validation.
- tests/test_repos.py: regression test for list dept_route coercion
(stored joined 'A;B', raw_json keeps the list).
Co-Authored-By: Claude <noreply@anthropic.com>
Replace the Phase-0 placeholder password encryption with real RSA
(PKCS#1 v1.5). encrypt_password now builds an RSA public key from the
getSecureKey modulus/exponent (RSAPublicNumbers, e=int(hex,16),
n=int(hex,16)), encrypts the UTF-8 password, and returns hex (documented
assumption -- base64 is the one-line fallback noted in the docstring).
get_secure_key now returns the inner data dict to match the new contract;
pc_login keeps {account, password} with a comment on the username/password
and account/pwd fallbacks if pcLogin rejects.
Extend normalize_wechat_type with the Phase-0 confirmed codes:
2 -> image, 3 -> image_legacy, 4 -> video, 10 -> system (1 -> text and
-2 -> divider unchanged; 6/9/13/15/21/22 stay unknown until confirmed).
Tests:
- encrypt_password round-trip: generate an RSA keypair, build secure_key
from its public numbers, encrypt, then decrypt with the private key and
assert the original password comes back. Pins 512-hex output for a
2048-bit key. No network.
- Parser parametrize table extended for the new codes (and a downstream
message-fixture assertion updated from unknown -> image_legacy).
Suite: 87 -> 97 passing. ruff check clean.
Co-Authored-By: Claude <noreply@anthropic.com>
Wires the existing pieces together into a runnable integration check:
scrape ~10 WeChat friends + ~50 of their messages (last 7 days) from the
real yunke CRM (read-only), upsert into a local SQLite smoke DB, then
print a summary that verifies rows landed.
Two auth modes:
* Existing-session (preferred): YK_USERID/YK_COMPANY[/YK_COOKIE] -> YunkeClient
constructed from a captured session, login bypassed.
* Login mode (Phase-0 blocked): YUNKE_USERNAME/YUNKE_PASSWORD -> attempts
client.login(), which the real CRM will reject until the placeholder
password encryption is replaced. Prints a clear message and exits 3.
build_client_from_env() is split out as a pure, network-free helper so
auth-mode selection is unit-testable. main() is async (asyncio.run in
__main__). HTTPStatusError from a 401/expired session is wrapped into a
helpful message + exit 4 (no traceback). client.aclose() + engine.dispose()
always run in finally.
Defaults to SMOKE_DB_URL=sqlite+aiosqlite:///./smoke.db so no MySQL is
required. Run from yunke-server/ with the venv active:
python -m scripts.smoke_wechat
Tests cover the entrypoint shape and auth-mode selection (no network).
Suite: 77 -> 87 passing.
Co-Authored-By: Claude <noreply@anthropic.com>
浅念