xinyouhe/yunke
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
403 Commits 1 Branch 0 Tags
602bee8fdfd7a8a8bde9e0e56cd71cb226e5042a
Commit Graph

32 Commits

Author SHA1 Message Date
浅念 b0ebe8a15a feat(group-stats): 微信群列表对齐云客「微信群统计」——getChatroomList 数据源+窗内统计 
用户报微信群列表数据/内容不对,应对齐云客「沟通→微信号管理→微信群统计」
(/cms/wechat/wechatgroup)。真机探明(表单登录+页面导航抓 network):

- 数据源=POST /wechatweb/chatroom/getChatroomList(form:departmentId+
  startTime/endTime 窗+分页;信封 data.overviews[]/totalCount 1,854 行)
- 云客窗是任意时间窗实时算:沟通消息数实测同群全史 417/7天 36/单日 6
- **修正存量映射错误**:member_count 原存 friendNum(员工好友数),真群成员
  数是 friendCount/getChatroomList.memberCount
- 新增/退出成员数云客按窗实时算,本地无事件流 → 按天积累

实现:
- ChatGroup 补 group_remark 列(备注仅 friend_list/es 携带,两归一化均
  「有值才写」防互清);迁移含存量 raw_json 回填(须先于基线抓取执行)
- 新表 chat_group_member_stats(渠道×账号×群×天一行 added/lost)
- 新实体 chatroom-stats:全窗刷档案列(真成员数/群备注/员工信息,owner
  经 wechat_account alias 解析)+ --day 单日窗落进出日行;full_sync 5c 步
- /api/groups 扩展:msg_start/msg_end 窗内聚合(window_msg_count=
  msg_daily_stats 按天和、window_added/lost=日表和)、last_start/end
  最新消息窗、group_remark 子串;行补员工微信/员工信息关联列
- GroupList 微信 tab:统计时段(默认近7天)/最新消息时间段/群名称/群备注/
  员工下拉/查询 + 9 列(群信息|群备注|员工微信|员工信息|群成员数量|
  新增成员数|退出成员数|沟通消息数|操作);WA tab 不动

700 测试绿(新增窗聚合/筛选/员工列 3 例)。本地实测:基线 1,854 群=云客
total 一字不差,今日日窗 added 407/lost 166 落表,页面 9 列/员工列渲染正常。

Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-09 18:09:20 +08:00
浅念 7758a3351f docs(management): 批次A终评修正——部署顺序理由/data_complete语义/通话类型注释 
终评 Approve 后三处文档级修正:
- CLI help与DEPLOY的顺序理由改为事实(宽表不读fact,先跑不会出0;
  顺序约定保留,fact是批次B评分/下钻数据源)
- data_complete 注释改为预留语义待批次B定义(当前恒True)
- EmployeeCallDaily 注释通话类型实测值(voice_call/wa_voice_call)
 2026-09-09 16:30:05 +08:00
浅念 58d274096d fix(management): first_reply_fact 同毫秒并列主键冲突+定序抖动——reply_msg_id 进 PK/按(ts,msg_id)排序 
评审修正(代码质量评审 Critical#1 + Important#2/#3):
- reply_msg_id 进复合 PK(模型+migration b7c8d9e0f1a2 原地改,本地
  down/up 重建,表空未部署):同毫秒 F,T,F,T 交换的两个样本
  (thread_end_ts, reply_ts) 完全并列不再撞主键(原会中断整个
  recompute_all 事务);兼作下钻证据定位回复消息。
- load_conversation 增查 msg_id 并按 (ts, msg_id) 排序,
  compute_conversation 协议改 3 元组——SQL 等键顺序未定义,原实现
  同毫秒 [F,T] 归属随查询抖动(0/1 answered 摇摆)。
- recompute_pairs 每 500 会话显式 flush+进度日志(内存上界对齐
  summary.recompute_all 惯例,不再依赖隐式 autoflush)。
- 测试:旧 9 个更新协议;新增同毫秒 F,T,F,T 回归/并列确定性/
  空会话幂等清空,fr_sum 总量断言并入双串测试;
  test_management_models 构造补新主键列。

Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-09 13:18:08 +08:00
浅念 2b1f91762b fix(management): 评审修正——config_json注解对齐NOT NULL+补3个约束测试 
- EmployeeScoreConfig.config_json 去掉 | None,注解与 DDL(nullable=False)一致
- 删测试死 import datetime as dt
- 补约束测试: metric_daily 复合PK防重/ call_daily PK防重/ config_json NOT NULL
  (JSON 列 Python None 序列化为 'null' 字符串,测 NOT NULL 须 sa.null())
- EmployeeScoreConfig docstring 补 JSON 列整列重赋值注记

Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-09 12:24:02 +08:00
浅念 e0c764997f feat(management): 批次A六表模型——fact/宽表/通话小表+评分配置/快照/辅导 
Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-09 12:04:54 +08:00
浅念 d42895337d feat(link): customer_account_link 账号↔客户多对多——共享好友员工维度对齐云客 
用户报「Lily 好友数 36,点进员工客户页只有 12」。真机探明根因:那 24 个
是共享客户(同一人加多个员工),云客 per-employee 各算一个;而 customer
表 PK=friend_id 全局单行、归属 last-wins,其他员工名下必然「消失」。
抓取器本身每员工都拿全了(日志 declared=36 rows=36),丢在入库模型层。

- 新表 customer_account_link(渠道×账号×客户一行,per-relation 快照:
  昵称/备注/头像/手机号/收发计数/最近沟通/add_time/fromType/isDel)
- 写:repos.upsert_customer 随档案同事务维护(所有好友抓取路径一处挂全);
  新增 prune_customer_links 对账
- 读:/api/customers?owner_account_id= 改读 link(与云客 contacts_num 口径
  一致);该账号 link 为空时回退档案路径(过渡安全);与 dept/user 等
  过滤组合时维持旧行为;CustomerOut 透传 from_type
- 抓取器:friends/wa-friends 逐员工整批 upsert 后按名单对账(取满声明
  total 才删,防半份名单误删)
- 前端:StaffFriends 数据来源列优先 link 的 from_type

611 测试绿(link 新 6 例);本地验证空表回退/造数接管/from_type 透传。
上线后需重跑 friends+wa-friends 回填 link(存量关系无法从档案反推)。

Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-08 17:17:06 +08:00
浅念 9d3cc8e76e fix(messages): 聊天记录大对话超时——DESC首屏+keyset游标+群聊直达索引 
用户报:聊天记录多时「聊天记录加载失败：请求失败 (ECONNABORTED)」(2026-09-07)。
服务器 EXPLAIN ANALYZE 实测两级根因:
- 单聊 1 万条:ASC 末页 OFFSET 10486 逐条回表 2515ms(DESC LIMIT 15ms,167 倍);
- 群聊 16,290 条:查询刻意不带 owner(chat_group 去重后 owner 错位),现有
  (channel,owner,room,timestamp) 索引被 owner 挡住,退化到 room 单列索引+
  filter+filesort——count 4.7s/首页 5.1s/末页 10.7s;真凶群 53132846289@chatroom
  40,863 条,三段串行 ~40s 必撞前端 axios 15s 超时。

修法(不动共享 MySQL 配置,沿用既往修复模式):
1. 迁移 e6f7a8b9c0d1:message 补 (channel, room_id, timestamp) 索引,群查询
   seek+排序直达+覆盖 count;模型侧双声明防 autogenerate 反 DROP。
2. /api/messages 加 order=asc|desc(默认 asc 不变)与 before_ts/before_id
   keyset 游标(成对,单传 422)——排序列 (timestamp,msg_id) 全序,平手按
   msg_id 决胜,游标页恒 O(page_size)。
3. 前端 ChatMain/ChatTimeline 首屏改「order=desc 取最新 50 反转」单请求
   (原 page1 拿 total+深 OFFSET 末页两请求),「加载更多」改游标上翻;
   hasMoreOlder=已加载<总数。SessionPane 搜索等 order=asc 默认路径不受影响。

本地实测(37,532 条群):desc 首页 0.39s、游标上翻 0.24s,浏览器真机
渲染+上翻 150 条无错。测试 8 增(desc/游标/平手边界/422/窗口叠加/
前端流程不变式),全量 601 绿。

Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-07 16:59:04 +08:00
浅念 feb3b98665 feat(analytics): 首响时长+回复率数据管道——消息串算法/日汇总扩展列/员工聚合 
批1(A1+A2):管理层要看「每个人的响应质量」。

- 迁移 d9e0f1a2b3c4:msg_daily_stats/stat_daily_staff 各加
  cust_threads/threads_answered/fr_sum_ms(NOT NULL server_default 0,
  幂等守卫照 9b0d0c133035 惯例;models.py 双声明同步)
- summary.py _thread_stats:串=客户连续消息不被员工消息打断的段,
  is_send=None 不参与串判定;fr=员工首条回复−串末条客户消息;
  日切近似 v1(日末未闭合串计 threads 不计 answered,口径写进 docstring)
- recompute_days/recompute_all 都先按 ts 排序再算串(乱序防护)
- staff_summary 聚合三列按账号→员工累加(fr_cnt≡threads_answered 不单列)
- 测试 +10:串算法矩阵 9 例+staff 聚合传递;全量 589 绿
- 本地 86 万行回填验证:daily-stats 60s/staff 10s,rollup 总和与
  明细完全一致,随机 5 行 SQL 独立复算全对

Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-07 14:13:09 +08:00
浅念 af05e58976 perf(analytics): 5表 updated_at 窄索引——health MAX 消除全表扫(128MB buffer pool) 
Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-04 16:40:01 +08:00
浅念 8b29326b2e fix(analytics): 命中表索引幂等守卫独立+created_at 去掉 onupdate 语义 
Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-04 15:30:14 +08:00
浅念 565006239b feat(analytics): 数据分析板块四新表+迁移——stat_daily_staff/敏感词三表/fetched_at 索引 
Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-04 15:04:33 +08:00
浅念 421998be54 perf(api): 会话/5数字弹窗超时修复——日粒度汇总层 msg_daily_stats 
根因(服务器实测):sessions 对 owner 全量 message GROUP BY,重账号 5.2万条
3~5s;staff-chat-stats 全时段窗 6.9s——128MB buffer pool 下被推过前端
15s 超时(ECONNABORTED)。EXPLAIN ANALYZE 证实最优 covering 索引也要
18.9s,瓶颈是磁盘不是查询形状。

修复(预聚合,云客同款思路):
- msg_daily_stats 表:每「渠道×账号×会话×上海自然日」一行(收发计数/
  当日最后时间与方向);迁移 a7d3e9f1c2b4(幂等建表)
- 写路径:repos.upsert_messages 落库自动重算本批触及的会话×天(所有
  抓取路径汇到该函数,一处挂全;失败不阻塞写入)
- 读路径:sessions/staff-chat-stats 表有行时读汇总(namedtuple 同形,
  端点主体零改动),空表守卫回退旧聚合(部署回填前不空白)
- 自愈:CLI daily-stats 全量重建(幂等);full_sync_all.sh 夜间追加第 11
  步——顺带修掉尾部 sqlite3 直连 smoke.db 的硬编码(服务器 MySQL 上会炸)
- 口径统一:day 一律 Asia/Shanghai Python 计算(旧 from_unixtime 跟随
  DB 会话时区与今日数割裂);统计窗口对齐自然日(文档化语义变化)

测试:12 新增(汇总层 8 + 端点判别/对拍 4,判别手法=回填后删光 message
仍出完整数据),全量 512 绿。

Co-Authored-By: Claude Code <noreply@anthropic.com>
 2026-09-03 16:37:36 +08:00
浅念 0b03c9fb0d perf(customers): message 补 (channel,owner,timestamp) 索引,列表 7.6s→0.03s 
客户列表行内「今日收发」聚合按 channel=?+owner IN+timestamp>= GROUP BY,
两个既有索引把 timestamp 排在 talker/room 后无法 seek——asyncmy 服务端
预编译时优化器弃索引全表扫 86 万含大文本宽行,实测 7.7s/次,前端每次翻页
都撞(用户报微信客户列表加载慢)。补三列窄索引后 owner 前缀直接落入当天
几百行:GROUP BY 0.14s、API 全链 0.032s(78 倍,翻页/WA/员工过滤同测)。
模型+迁移同步声明(防 autogenerate 反向 DROP),inspect 幂等。

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-27 16:23:04 +08:00
浅念 35bfebd3e3 chore(backend): list_order迁移补齐+Task14类型修正落链+moments每页日志+MsgTypeParam派生+3处文档修正+Dockerfile打包migrations+DEPLOY补迁移步骤 
- 9b0d0c133035: wechat_account.list_order 进迁移链(此前仅模型声明,
  干净库 /api/accounts 即 500);sa.inspect 幂等守卫兼容已手动加列的
  smoke.db
- fe63c6789e94(验证中发现的预存缺口,超出原清单): Task 14 的 5 列模型
  类型修正(4×URL String(1024)→Text, db_size Integer→BigInteger)从未落
  链——纯 alembic 新库会复活 >1KB 链接截断;对齐后干净库 alembic check
  全绿。SQLite message 表 batch 重建会掉 FTS 触发器+rowid 脱节(docstring
  记明 ensure_fts+rebuild_fts 修复,smoke.db 已修复验证)
- scrape_via_page: _iter_moments_pages 每页一行进度日志(镜像 _fetch_pages)
- messages: MsgTypeParam=Literal[tuple(_MSG_TYPE_FILTER)] 派生,杜绝
  双处罗列漂移(openapi enum 验证一致,492 绿含 422 行为)
- 文档: ChatGroup 复合PK docstring/groups room_id 多行表述/测试名去
  单行假设;Dockerfile 补 alembic.ini+migrations;DEPLOY 增「数据库
  迁移」节(新库/旧库stamp路径/回填脚本显式DB_URL)+运维表升级行

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-25 20:32:25 +08:00
浅念 6ff288f1d3 feat(groups): chat_group 补 6 群主/建群列 + 复合PK迁移修复 + 存量回填 
对齐云客员工聊天窗口·微信群 tab(2026-08-25 真机验证 raw_json 键):
- ChatGroup 加列 owner_name/owner_wxid/owner_alias/owner_avatar/
  friend_count/create_time(全 nullable)
- scraper _normalize_chat_group 同名提取;createTime 兼容 ms-int 与
  'YYYY-MM-DD HH:mm:ss'(smoke.db 实测 1105 行全为后者),复用
  _parse_add_time 归一 ms-epoch
- 迁移 099ac2097fa2:加 6 列 + 修复 Task1 评审遗留 drift——PK 由单列
  room_id 变更为复合 (room_id, channel, owner_account_id) 且
  owner_account_id NOT NULL(对齐模型 8a99940;smoke.db 1105 行无 NULL,
  未兜底编造数据);SQLite 走 batch 重建(copy_from 显式旧结构),
  MySQL 渲染为直接 ALTER(离线 --sql 已验证)
- 回填脚本 scripts/backfill_chat_group_fields.py(幂等,逐列只补空);
  smoke.db 实跑 backfilled 1105/1105,重跑 0/1105
- GroupOut 加 6 字段(model_validate 自动输出,路由无改)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-25 14:32:12 +08:00
浅念 b7669199f8 feat(db): moments_post 表+upsert(朋友圈实体,PK=云客data.id,create_time统一ms) 
- MomentsPost 模型:moments_post 表,PK=id(云客 getFriendsterInfo data.id),
  wechat_id/create_time 建索引,praise/comment/image 存 JSON 列
- 迁移 524da8d2d597(手写 create_table,已应用到 smoke.db 并在临时 SQLite
  全链路 upgrade/downgrade 验证)
- repos.upsert_moments_posts:批量幂等 upsert;附带修 _build_column_values
  对 JSON 列表的误 join(标量列 join 保持不变,dept_route 旧测试仍过)
- 修复 3 个 chat_group 旧测试缺 owner_account_id(2026-08-24 复合主键遗留)
- 全套 430 测试通过

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-25 13:44:54 +08:00
浅念 8a99940554 feat(groups): chat_group改复合主键(channel,owner,room)——多员工同群不再去重(云客张士蓓64群vs原42) 
原 room_id 单列 PK 把多员工同群合并成一行(末位覆盖owner),其他员工
群列表缺数据。改复合主键后同群每个员工各一行,对齐云客按员工查看。
重扫后 1590 行/865 去重群/张士蓓 64 群(与云客一致)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-24 12:01:25 +08:00
浅念 804addf725 feat(staff): 员工列表排序对齐云客API返回顺序(list_order列+抓取记序+API按序排) 
- wechat_account 加 list_order 列(云客 wxUser/list 返回顺序)
- 抓取器记录每行在API响应中的序号
- /api/accounts ORDER BY list_order(对齐云客员工列表排列)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-24 11:25:33 +08:00
浅念 00ae13b5c2 feat(deploy): 完成数据迁移与MySQL全链路(14表698k消息100%一致+5方言补丁+bind mount) 
- migrate_sqlite_to_mysql.py: 逐表清空重导(幂等),复用repos强转,JSON列
  解析,67万消息按msg_id键集分批,467s跑完14表全一致
- 修4个MySQL实测bug: NULLS LAST语法(compat.py编译补丁)、message.text
  TEXT→MEDIUMTEXT(64KB字节上限,实测20.8万字符车源清单)、URL列
  String(1024)→Text(分享链接超1KB)、垃圾超长member_id源头清理
- compose: media改bind mount(不占C盘Docker虚拟磁盘,实测2.5GB卷把
  C盘撑爆引擎500);mysql不映射宿主机3306
- 前端Dockerfile: npm ci→install(本地/容器npm版本锁漂移)
- 五服务实测:nginx→backend→MySQL全链路返回真实数据(客户6859/A**杰25条
  /搜索56条),媒体49536文件bind mount直读

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-22 12:40:44 +08:00
浅念 2175cb0169 feat(deploy): MySQL容器实测打通(3个方言bug修复)+部署文档 
- repos: on_duplicate只更新实际插入列(fetched_at等Python-default列,
  SQLite excluded容忍/MySQL new.*直接报Unknown column)
- repos: _coerce_int增加日期字符串→ms epoch分支 + _INT_FIELDS补
  last_update/last_chat_time/timestamp/update_time(字符串直通BigInteger)
- models: wechat_account.db_size Integer→BigInteger(实测2.17GB超32位上限)
- compose: mysql不映射宿主机3306(本机已占;容器内服务名互联)
- docs/DEPLOY.md: 服务器部署全指南(架构/步骤/数据迁移/运维命令/注意事项)
- 容器实测:Playwright抓取55账号→MySQL upsert全通

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-22 10:40:04 +08:00
浅念 e8c77412fa feat(search): FTS5 trigram全文索引(中文搜索3.95s→0.003s,1300倍) 
- message_fts 外部内容虚拟表+trigram分词(中文子串可MATCH,无需分词库)
- 3个触发器随ORM upsert自动同步索引(抓取器零改动)
- 搜索路由:≥3字走FTS MATCH,<3字/MySQL回退LIKE
- 实测:搜「大众途安」56条,LIKE全扫3.95s→FTS 0.003s;索引+300MB
- 7个新测试(建表幂等/触发器同步/中文匹配/短查询回退/未建索引回退),426绿

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-21 15:08:26 +08:00
浅念 f92aac1a3d feat(scraper): 微信群消息抓取(368778条/750群)+浏览器守护循环+SQLite忙等 
- 新 group-messages 实体:真机探明群消息=同一 msg/chats 端点 friendId 传 roomId;
  talker=发送者wxid、msgId=null→owner:room:createTime:talker 合成稳定主键(重扫幂等)
- 浏览器守护循环:超大群撑崩渲染进程时重启浏览器跳过毒群继续(实战2次自愈);
  200页上限;恢复路径补 await
- base.py SQLite connect_args timeout=60:并发写忙等而非撞锁报错
  (双抓取器并发写实测 database is locked 崩任务)
- 5个新单测,全量410绿

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-20 09:09:32 +08:00
浅念 9a953553ec fix(xss+perf): SearchView XSS消除(无v-html)+room_id索引+pool_recycle 
- SearchView.vue: 用 highlightSegments() 返回 {text,hit} 数组替代 v-html+escapeHtml+regex,模板用 <template v-for> + <mark>/<span> 渲染,从结构上消除 XSS 风险(4 处 v-html 全部移除: 消息片段/客户昵称/realName/手机号)
- models.py: Message 新增 ix_message_channel_owner_room_time(channel,owner_account_id,room_id,timestamp),群聊查询过滤 room_id 而非 talker_id,原 talker 索引无法覆盖
- migrations/d8eee96cd017: 新增 room_id 索引的 upgrade/downgrade (offline SQL 已验证)
- base.py + scheduler.py + cli.py: 所有生产 create_async_engine 增加 pool_recycle=3600,在 MySQL wait_timeout(默认 8h)断连前主动回收

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-12 11:50:45 +08:00
浅念 5ce33232c1 feat(sync): 聊天图片本地化(message media_hash+download_message_images+local-first rendering) 
- Message 模型新增 media_hash 列 (sha256 FK -> Media.media_hash)
- Alembic 迁移 e5f6a7b8c9d0_message_media_hash (batch_alter_table, SQLite/MySQL 通用)
- app/scraper/media.py:download_message_images 批量下载器，镜像 download_call_recordings
  - 候选: media_hash IS NULL AND channel IN ('wechat','whatsapp') AND type IN ('image','video','file') AND 任一 file_*_url LIKE 'http%'
  - image 类型优先 file_th_url (缩略图，小快省)，video/file 优先 file_url
  - per-row try/except + per-run tried set，失败不挂循环
- sync_message_images orchestrator 入口 + ENTITY_MESSAGE_IMAGES 常量 (提交自身 SyncRun，镜像 call_recordings)
- scheduler: _run_message_images + ENTITY_FUNCS['message_images'] + 加入 daily all job (尾部)
- MessageOut schema 暴露 media_hash 字段
- MessageBubble.vue local-first: media_hash 优先 /api/media/{hash}，否则回退 OSS file_url/file_th_url/file_hd_url
- types.ts: Message 接口添加 media_hash 字段
- 新增 test_message_images_downloader.py (6 tests, 镜像 test_recording_downloader)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 18:24:11 +08:00
浅念 538a0dc635 feat(sync): 长语音音频本地化(voice_clip media_hash+download+local-first playback) 
Mirror download_call_recordings pattern for voice_clips:
- VoiceClip.media_hash column + Alembic migration (c4d5e6f7a8b9)
- VoiceClipOut.media_hash + frontend VoiceClip.media_hash type
- download_voice_clip_audio in app/scraper/media.py (resilient, sha256-dedup,
  same download_url/upsert_media helpers as call recordings)
- sync_voice_clip_audio orchestrator wrapper (commits own SyncRun, mirrors
  sync_call_recordings semantics)
- scheduler: _run_voice_clip_audio + ENTITY_FUNCS['voice_clip_audio'] +
  appended to daily _run_all job (after voice_clips)
- VoiceClipModal playClip local-first: /api/media/{media_hash} with OSS
  oss_file_name fallback (mirrors CallList.playRecord)

Tested: 376 pytest pass; vue-tsc clean; migration applied; 1523/1533 clips
localized (10 pending are genuine upstream HTTP 404s — resilient skip).

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 12:06:26 +08:00
浅念 9da272b992 feat(calls): voice_clip 手动匹配(voice_clip↔customer 本地关联写操作) 
- VoiceClip: 新增 matched_friend_id 列 (String(128), nullable) + alembic 迁移 b3c4d5e6f7a8
- PATCH /api/voice-clips/{id}/match: friend_id 关联 / null 取消 (本地写, 不回写云客)
- GET /api/voice-clips: 批量解析 matched_friend_name (remark>nickname>real>id)
- VoiceClipModal: 已匹配客户列 + 匹配/取消匹配按钮 + 嵌套客户选择 picker (默认按员工列客户, 名搜索走 /search)
- 测试: 17 passed (新增 7 个 match 用例覆盖 set/clear/name 解析/404)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 10:43:34 +08:00
浅念 d9060b6920 feat(api): 长语音 voice_clip 后端(model+migration+scraper+sync+endpoint) 
- VoiceClip model (BigInteger PK voice_clip_id, structured cols + raw_json)
- Alembic migration a1b2c3d4e5f6 (chains from 8c4f1d2a9e6b)
- scrape_voice_clips(client, user_id, *, channel, page_size) per-employee scraper
  over POST /wechatweb/call/list {pageNum, pageSize, userId}; channel stamped
  locally (wechat/whatsapp), startTime/endTime ms-epoch string -> int
- upsert_voice_clip repo + start_time/end_time/mp3_duration int coercion
- sync_voice_clips orchestrator: iterates distinct user_id across
  WechatAccount + WhatsappAccount, stamps channel per account table
- scheduler ENTITY_FUNCS['voice_clips'] + added to daily 'all' job
- GET /api/voice-clips (user_id exact beats user_real_name fuzzy; channel;
  company_code; start/end window on end_time; ordered end_time desc)
- VoiceClipOut schema + router registered in main.py
- tests: scraper (5) + API (11) = 16 new; full suite 370 green

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 18:12:28 +08:00
浅念 8f70ac5220 feat(db): phone_call.record_media_hash 列 + 迁移（录音本地化 FK→Media） 
Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-07 16:20:18 +08:00
浅念 b5733f15d2 feat(db): add Customer.add_time column + migration 
Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-05 18:39:17 +08:00
浅念 5a2ff58af3 fix: split message type into normalized label + raw code 
The scraper was writing a normalized type label ("text"/"unknown"/...) into
Message.type, but the column was Integer, so the storage layer coerced it to
NULL and the raw code was only recoverable from raw_json. Align with design
§4.3, which specifies two fields: `type` (normalized enum) and `type_raw`
(original code).

- Model (Message): type -> String(32) normalized label; add type_raw Integer
  for the raw upstream type code.
- Scraper: _normalize_message sets both `type` (label) and `type_raw`
  (coerced int from messageType/type); _is_divider now checks raw -2.
- Repos: type_raw joins _INT_FIELDS; Message.type stored as-is (string).
- Migration 96975050fa79: add type_raw + alter type Integer->String(32),
  wrapped in batch_alter_table so it runs on SQLite and MySQL.
- env.py: enable render_as_batch on SQLite (dev/test) for portable batch
  migrations; MySQL still uses direct ALTER.
- Tests updated end-to-end (scraper, repos, models).

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-01 12:04:08 +08:00
浅念 9cf1c9ab11 feat: core ORM models + initial migration 
Implements the 14 core ORM models for the yunke data-mirror system using
SQLAlchemy 2.0 Mapped/mapped_column style, driven by the field lists in
docs/superpowers/specs/2026-08-01-yunke-data-dictionary.md:

  Department, Staff, WechatAccount, WhatsappAccount, Customer, ChatGroup,
  GroupMember, Message, Media, PhoneCall, Sms, SyncState, YkAccount, SyncRun.

Conventions:
- Natural source PKs where they exist (friend_id, wx_id, msg_id, room_id,
  media_hash, call_id, sms_id, dept_id, user_id); surrogate Integer PKs
  otherwise (SyncState, YkAccount, SyncRun). GroupMember uses a composite
  (channel, room_id, member_id) PK.
- Common audit columns version/raw_json/fetched_at/updated_at on every
  source-mirrored table.
- channel only where meaningful: Customer/Message/Media/ChatGroup/
  GroupMember/Sms use messaging channels; PhoneCall.channel is call-type;
  SyncState.channel scopes the cursor. Omitted on Staff/Department/
  WechatAccount/WhatsappAccount/YkAccount/SyncRun.
- Message composite index on (channel, owner_account_id, talker_id,
  timestamp) for the hot per-talker time-order query path.
- message_table_name(channel, timestamp) helper documents the future
  message_<channel>_<YYYYMM> sharding convention; the Message model still
  maps to a single physical table for now (sharding is a later task).

All types are portable generic SQLAlchemy types so tests run on in-memory
SQLite while prod targets MySQL. Tests verify Customer round-trip,
raw_json JSON round-trip, Message natural PK, GroupMember composite PK,
the Message composite index, and full table creation on SQLite.

Alembic wired via migrations/env.py to Base.metadata + get_settings().db_url
with an async->sync driver swap (mysql+asyncmy -> mysql+pymysql,
sqlite+aiosqlite -> sqlite). Initial migration autogenerated against an
empty SQLite DB; server defaults normalized to sa.func.now() so the file
is MySQL-portable. upgrade head is intentionally not run (no MySQL yet).

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-01 11:15:34 +08:00
浅念 b9fcf17ae4 feat: db base + config tests 
Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-01 11:04:44 +08:00