address review Minors: drop redundant filename replace, assert 暂无消息,
tighten downloadBlob return type + try/finally, fix plan group time-picker wording.
Co-Authored-By: Claude <noreply@anthropic.com>
1. .yk-staffinfo__nick display:block (ellipsis now applies to WA inline span). 2. openAdded/openTalked modal use owner_account_id (always present) instead of user_id! (null would leak whole customer table). 3. viewChat guards null talker+room + awaits router.push. 4. WA 添加人数 title → 客户列表 (honest: WA has no add_time, shows all).
Co-Authored-By: Claude <noreply@anthropic.com>
YkStaffDetailModal Col gains optional formatter; baseCustomerCols time cols render via formatTime instead of raw epoch.
Co-Authored-By: Claude <noreply@anthropic.com>
Replace the sparse staff-record table on /staff with a channel-tabbed
employee account view mirroring the 云客 backend's 员工微信列表 +
WhatsApp 员工列表. Each tab loads /api/accounts?channel=<channel> with
its own pagination state; rows open the existing account all-fields
drawer (reused from AccountList via accountFields.ts).
WeChat tab columns: 头像/微信昵称/微信号/手机号/员工姓名/好友数/群数/
发送/接收/数据库大小/在职/在线状态/最后更新. WhatsApp tab columns:
头像(el-avatar initials)/昵称/员工姓名/手机/销售手机/好友数/群数/
联系人数/新增好友/在职/最后更新. Reads raw_json via the structured-first
readField helper; tolerates boolean and 1/0 forms of isOnJob/onlineStatus.
Also fix the avatar field key in accountFields.ts (headImageUrl, with
headImgUrl as a legacy alias) so the WeChat drawer avatar renders — the
upstream wxUser record uses headImageUrl.
Co-Authored-By: Claude <noreply@anthropic.com>
M6 — the scheduling/deployment layer. Makes the system runnable as a
scheduled service + deployable via Docker.
Part A — SyncScheduler (app/sync/scheduler.py):
- APScheduler (AsyncIOScheduler) wraps the orchestrator's sync_* functions.
- Daily full sync at FULL_SYNC_HOUR (default 02:00 local) + incremental
message sync every MESSAGE_INTERVAL_HOURS (default 4).
- ENTITY_FUNCS registry dispatches run_once("<entity>") to the right runner
(departments/accounts/friends/groups/voice_calls/phone_calls/sms/messages/
all_wechat/all).
- Conversations for the message run are discovered from the local customer
mirror (no extra upstream call).
- Failure policy: every run is wrapped — a blown entity is logged + swallowed
so the scheduler keeps its next appointment.
- Per-run JSON-line log (level=info/warning/error, totals, failed entities).
Part B — CLI entry point (app/sync/cli.py):
- `python -m app.sync.cli sync [--entity <name>] [--quiet]` — one-shot sync.
- `python -m app.sync.cli scheduler [--full-sync-hour H] [--message-interval-hours N]`
— long-running scheduler (blocks until SIGINT/SIGTERM).
- Uses _push_settings_into_environ + build_client_from_env (session-cookie
mode preferred; pcLogin deferred).
- Base.metadata.create_all on every command (fresh-checkout friendly).
- Exit codes: 0 success, 1 generic failure, 2 bad CLI / auth misconfig.
Part C — Docker stack:
- yunke-server/Dockerfile: python:3.12-slim, installs deps + app, single
image serves both API (uvicorn) + scheduler (CLI).
- yunkes/Dockerfile: multi-stage node:20 build → nginx serves dist/ +
reverse-proxies /api → backend:8000.
- yunkes/nginx.conf: /api proxy with proxy_buffering off (streaming media),
SPA fallback, asset caching.
- docker-compose.yml: mysql (8.0, healthcheck, volume) + redis (7-alpine) +
backend + scheduler + frontend (port 5173 → 80). Env-driven creds + cadence.
Part D — README.md at repo root: dev mode + docker mode quick-start, config
table, architecture overview, current limitations (pcLogin deferred, WA voice/
media deferred, conservative pacing, conversation discovery is local-only).
Verification:
- pytest: 323 passed (was 294; +29 new for scheduler + CLI).
- npm run build: clean.
- python -m app.sync.cli --help: prints usage.
- docker compose config: VALID.
Co-Authored-By: Claude <noreply@anthropic.com>
Upstream API records may contain lone surrogate chars (emoji fragments)
that break SQLite/MySQL utf-8 encoding. Now sanitized to U+FFFD before
storage. Includes backfill_customers.py for full customer backfill.
Co-Authored-By: Claude <noreply@anthropic.com>
MessageBubble.vue was keys off the (incomplete) type code, so many messages
with real OSS media URLs (https://wx.oss.yunkecn.com/.../*.webp|*.mp4) showed
as '[未知消息]'. Rendering is now URL-driven: pick the best http URL from
file_url > file_hd_url > file_th_url, then render by extension
(image/video/audio/file). Type code is only used as a last-resort label.
- MessageBubble.vue: URL-driven cascade (URL by ext → duration →
transcription → text → structured-type placeholder → localized tag).
Image/video/audio/file children @click.stop so the all-fields drawer
still opens on bubble click but not on media interactions.
- format.ts: wechatTypeLabel(type_raw) maps known codes (3→图片, 34→语音,
43→图片/表情, 49→文件/名片, 42→名片, 9→表情, 20→红包/转账).
- ChatTimeline.vue drawer type tag uses wechatTypeLabel for WeChat msgs.
- style.css: compact .yk-msg__audio player for voice-with-URL messages.
Verified against smoke.db: 25 type-3 images + 3 type-43 videos that
previously showed '[未知消息]' now render as images/video. Build clean.
Co-Authored-By: Claude <noreply@anthropic.com>
WA list endpoints return wall-clock strings ('2026-08-01 16:39') for
lastUpdate; storing raw broke CustomerOut.last_chat_time (int) validation
-> 500 on /api/customers?channel=whatsapp. Now parsed via parse_datetime_to_ms.
Co-Authored-By: Claude <noreply@anthropic.com>
Live validation: businessFriendList uses 'page' (not pageNum) and REQUIRES
departmentId (root) + type or returns empty. Same class of bug as wxUser/list.
Co-Authored-By: Claude <noreply@anthropic.com>
Replicates the Customer all-fields pattern across the remaining three
entities. Backend now exposes `raw_json` on MessageOut / StaffOut /
AccountOut so the frontend can render every 云客 source field without a
second endpoint. Each entity gains a Chinese-label field utility + an
el-drawer surfacing all fields (structured-first, raw_json fallback,
unknown keys under 其他).
Backend:
- schemas.py: add `raw_json: dict | None` to MessageOut / StaffOut /
AccountOut. MessageOut/StaffOut pick the column up via from_attributes;
AccountOut is built explicitly in the router, which now threads
`raw_json` through from both WechatAccount and WhatsappAccount rows.
- tests: new raw_json serialization tests for messages, staff, accounts
(assert populated dict round-trips verbatim + NULL serializes as null).
Frontend (yunkes/):
- New utils: messageFields.ts, staffFields.ts, accountFields.ts — each
with a 中文 label map, ordered groups, structured-column preference,
and an 其他 fallback for unknown raw_json keys.
- Message: clicking a chat bubble opens an all-fields drawer (referMsg
renders as pretty-printed JSON). Image preview + file download links
stopPropagation so their own interactions still work.
- Staff: row click → drawer; columns relabeled (员工ID, 销售手机).
- Accounts: new AccountList.vue at /accounts with a 账号 nav item,
unified WeChat/WhatsApp table (渠道/账号ID/昵称/微信号/归属员工/好友数/
群数/收发/在职/最后更新), channel filter, pagination, row-click drawer.
- api: add Account type + getAccounts(); add raw_json to Message/Staff.
Co-Authored-By: Claude <noreply@anthropic.com>
Backend:
- CustomerOut now exposes raw_json (verbatim upstream friend record) so the
frontend can render every 云客 field without per-field structured columns.
- The verbatim blob round-trips via from_attributes; no model change needed.
Frontend:
- CustomerList table mirrors the 云客 客户列表 column set (头像/微信昵称/微信号/
备注/真实姓名/手机/性别/地区/描述/销售员工/部门/来源/添加方式/添加时间/最近
聊天/消息数/渠道). A readField helper reads the structured column first and
falls back to raw_json.<camelCase>.
- Row click opens an all-fields drawer (right side) rendering EVERY field of
the record with Chinese labels, grouped (基本信息/联系方式/归属员工与部门/时间
与来源/消息统计/好友状态/其他). Timestamps format to local datetime, flags
render 是/否, image URLs render as el-image thumbnails, empty values show —.
- New src/utils/customerFields.ts holds the camelCase→中文 label map, the
structured-column alias map, group definitions, and value-type formatters;
unknown raw_json keys auto-flow into an 其他 section so new upstream fields
surface without code changes.
- Drawer 查看聊天记录 action navigates to /chat/:channel/:ownerId/:talkerId
(owner_account_id + friend_id + channel), preserving the existing path.
Co-Authored-By: Claude <noreply@anthropic.com>
Backend (scrape_wechat_accounts):
- Send wxUser/list as form-urlencoded (not JSON) with the REQUIRED filters
isOnJob / departmentId / startdate / enddate + empty userId/salesname/
weChatNickName, matching the live request (empty fails with code -90000).
- scrape_wechat_accounts now takes department_id (the company-root dept id)
plus optional start/end defaulting to a recent 7-day UTC window.
- paginate_by_page gains a form=True mode (data= + form content-type).
- Orchestrator sync_wechat_accounts resolves the root dept from the
Department table (or accepts one) and skips with a warning when unknown.
- seed_accounts.py resolves the root via scrape_departments.
Frontend (yunkes):
- Add StaffList.vue (/staff) with is_on_job filter + pagination.
- Add /staff route and a 员工 nav item between 客户 and 群.
Co-Authored-By: Claude <noreply@anthropic.com>
Live validation found two real bugs:
- YunkeClient sent 'source: yunkepc' on every request; msg/chats binds
source as Integer -> 500. Removed (cookie auth suffices, per Phase 0).
- WeChat message talker_id used the masked 'talker' field; the timeline
joins on the full peer id. Now: 1:1 -> oriTalker (full), group -> talker
(member). ori_talker keeps the raw oriTalker (peer/chatroom).
Co-Authored-By: Claude <noreply@anthropic.com>
Backend (Part A):
- MessageOut: add file_th_url, file_hd_url, duration, transcription_text, md5
(room_id + sender_head_url already existed) so the chat timeline can render
image thumbnails, voice duration/transcription, and avatars from a single
response.
- PhoneCallOut: add record_file so the Calls view can offer recording playback.
- SmsOut: add content (full body) so the SMS detail dialog shows the whole text.
- Tests: red->green for each new field (media fields default None + populated,
record_file round-trip, content round-trip). Suite 267 -> 271.
Frontend (Part B):
- api/index.ts: wrappers getGroups/getGroupMembers/getCalls/getSms/getSyncStatus
+ Group/GroupMember/PhoneCall/Sms/SyncState/SyncRun/SyncStatus types in
types.ts (mirror the *Out pydantic schemas).
- views/GroupList.vue (/groups): table + channel filter + pagination; row click
opens the group timeline.
- views/CallList.vue (/calls): table + call-type filter (phone_call/voice_call/
all); "播放录音" action opens an el-dialog with <audio controls>.
- views/SmsList.vue (/sms): table; row click opens a detail dialog with the full
content.
- views/SyncStatus.vue (/sync): two tables (states + recent_runs) with a manual
refresh; run status color-coded.
- ChatTimeline.vue: reused for both 1:1 DM (talkerId) and group (roomId) by
branching the query; group title + per-member sender already handled by
MessageBubble.
- router: add /groups /calls /sms /sync + /group/:channel/:ownerId/:roomId.
- layouts/MainLayout.vue: replace the single 客户列表 button with an el-menu
(客户 / 群 / 通话 / 短信 / 同步); org tree kept on the left.
Co-Authored-By: Claude <noreply@anthropic.com>
M5b viewing frontend for the yunke data-mirror, under yunkes/.
- Scaffold: Vite + Vue 3 + TS, with element-plus, @element-plus/icons-vue,
pinia, vue-router@4, axios. Dev server on :5173 with /api and /api/media
proxied to the FastAPI backend on :8000.
- API client (src/api/): axios instance (baseURL '/api') whose response
interceptor unwraps the axios envelope; typed wrappers (getCustomers,
getMessages, getDepartments, getStaff) returning Page<T> / arrays; types
mirroring the backend *Out schemas (Message typed as a permissive superset
so file_th_url/duration/transcription_text degrade gracefully today and
light up once MessageOut exposes them).
- Layout: el-header top bar (title + channel switch) + el-aside department
tree (built from the flat parent_id list via useOrgStore, with search
filter) + el-main router-view.
- CustomerList (/customers): el-table with channel filter, department filter
driven by the tree selection (route query), pagination, and row-click into
the chat timeline (channel + owner_account_id + talker_id).
- ChatTimeline (/chat/:channel/:ownerId/:talkerId): asc-ordered message
bubbles keyed by is_send (direction) and type (text/image/video/voice/file/
divider/system/unknown); lands on the most-recent page and prepends older
pages via load-more with scroll anchoring.
- Home '/' redirects to '/customers'.
npm run build (vue-tsc -b + vite build) is clean.
Co-Authored-By: Claude <noreply@anthropic.com>
Mirrors the WeChat scraper patterns (M3) using the shared paginator +
parser + repo building blocks. Three new async generators yield
snake_case dicts + verbatim raw_json, ready for the channel-generic
upsert_customer / upsert_messages repos.
- app/scraper/whatsapp.py (new): scrape_whatsapp_accounts /
scrape_whatsapp_friends (page-number pagination over the
data.overviews / data.totalCount envelope) and scrape_whatsapp_messages
(backward-cursor pagination, JSON body {uid,friendUid,start,end} —
NOT form-encoded like WeChat msg/chats). Drops type==-2 dividers;
maps mediaName -> file_url as a deferred-download key.
- app/scraper/parsers.py: add normalize_whatsapp_type (0->text,
1->image, -2->divider, else unknown; never raises) and a shared
coerce_int_or_none scraper helper (mirrors repos._coerce_int so the
scraper emits type_raw ints without a DB dependency).
- app/storage/repos.py: add upsert_whatsapp_account (PK uid) which
also upserts the embedded Staff(user_id, name) via the existing
upsert_staff_from_account helper; WhatsappAccount imported.
Tests (respx mocks, no network): 3 cassette fixtures + 3 scraper test
modules (accounts/friends/messages), parsers + repos extended with WA
cases. Divider drop, backward-cursor advance (end == E1-1), JSON body
asserts, embedded-staff upsert, and channel-generic Customer/Message
upserts all covered.
Suite: 182 -> 211 passing. ruff clean.
Co-Authored-By: Claude <noreply@anthropic.com>
Cleanup after successful live validation (10 real customers scraped+stored):
- app/config.py: add yk_userid / yk_company / yk_cookie Settings fields so
existing-session creds can be supplied via .env (bypasses programmatic
login for validation with a live browser session).
- app/storage/repos.py: coerce list values to ';'-joined strings in
_build_column_values. Fixes a real bug: the live friends-list API returns
Customer.deptRoute as a LIST, but dept_route is a scalar String(255)
column. raw_json still preserves the original list (no data loss).
- scripts/smoke_wechat.py:
* Revert build_client_from_env to read os.environ ONLY (no get_settings
fallbacks), so the monkeypatch.setenv unit tests stay hermetic and the
real .env stops leaking in. 3 previously-red tests green again.
* Add _push_settings_into_environ() (Settings -> os.environ, real env
wins, only fills gaps) and call it as the first line of main(); so the
smoke run still honors .env without the pure builder depending on
Settings. .env -> Settings -> os.environ (once) -> build_client_from_env.
* Keep the await _print_summary fix from validation.
- tests/test_repos.py: regression test for list dept_route coercion
(stored joined 'A;B', raw_json keeps the list).
Co-Authored-By: Claude <noreply@anthropic.com>
Centralize scraping stealth in YunkeClient.request() so automated traffic
resembles the normal CRM web app and is paced conservatively to avoid
triggering 云客 risk control (pulls M6 rate-limit work forward):
- app/config.py: add 6 tunable settings with conservative defaults
(request_min_interval=1.0, request_jitter=0.8, max_requests_per_run=0,
risk_backoff_{base=2.0, max=60.0, attempts=4}).
- app/scraper/base.py: RateLimiter gains optional jitter + a new wait()
method (fixed min+jitter per-call sleep); gap-based acquire() is unchanged
for backward compat (default jitter=0.0).
- app/session/yunke_client.py: every request now carries browser-identical
headers (UA/sec-ch-ua/accept/accept-language/referer) merged with the auth
headers (per-request overrides win); awaits the shared jittered limiter
before sending; enforces a per-run request cap (RequestCapReached); and
backs off exponentially on risk signals (HTTP 429 or 200 body carrying
频繁/风控/限制/稍后), distinct from the existing 401 relogin-once path.
- tests/conftest.py: zero the throttle env in the test suite so the
conservative prod defaults don't slow unit tests.
- tests/test_yunke_client_stealth.py: 9 new tests (browser headers,
override, limiter spy, 429 backoff, 200-marker backoff, persistent-429
raise, volume cap, unlimited, 401-relogin preserved). TDD red->green.
Suite: 171 -> 181 passing. ruff clean.
Co-Authored-By: Claude <noreply@anthropic.com>
浅念
