xinyouhe/yunke
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(analytics): 首响时长+回复率数据管道——消息串算法/日汇总扩展列/员工聚合

Browse Source 
批1(A1+A2):管理层要看「每个人的响应质量」。

- 迁移 d9e0f1a2b3c4:msg_daily_stats/stat_daily_staff 各加
  cust_threads/threads_answered/fr_sum_ms(NOT NULL server_default 0,
  幂等守卫照 9b0d0c133035 惯例;models.py 双声明同步)
- summary.py _thread_stats:串=客户连续消息不被员工消息打断的段,
  is_send=None 不参与串判定;fr=员工首条回复−串末条客户消息;
  日切近似 v1(日末未闭合串计 threads 不计 answered,口径写进 docstring)
- recompute_days/recompute_all 都先按 ts 排序再算串(乱序防护)
- staff_summary 聚合三列按账号→员工累加(fr_cnt≡threads_answered 不单列)
- 测试 +10:串算法矩阵 9 例+staff 聚合传递;全量 589 绿
- 本地 86 万行回填验证:daily-stats 60s/staff 10s,rollup 总和与
  明细完全一致,随机 5 行 SQL 独立复算全对

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
浅念
2026-09-07 14:13:09 +08:00
parent 39791b2a49
commit feb3b98665
6 changed files with 359 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
yunke-server/app/db/models.py
+27
View File
@@ -776,6 +776,10 @@ class MsgDailyStats(Base):
- ``send_cnt``/``recv_cnt`` 仅计 is_send 真/假;NULL 两边都不计(对齐
旧聚合的 case 语义)。
- ``last_ts``/``last_is_send`` 为该日最后一条消息的时间与方向。
- 首响/回复率三列(2026-09-07 批1)按「消息串」口径,见
:func:`app.storage.summary._thread_stats`:串=客户连续消息不被员工
消息打断的段(is_send=NULL 不参与串判定);日切近似——日末未闭合
串计 threads 不计 answered,跨夜的串在次日段重新起算。
"""
__tablename__ = "msg_daily_stats"
@@ -796,6 +800,18 @@ class MsgDailyStats(Base):
# 当日最后一条消息(ms);当天无有效方向计数的消息也存在(is_send=NULL)
last_ts: Mapped[int | None] = mapped_column(sa.BigInteger, nullable=True)
last_is_send: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
# 当日客户串数(串=客户连续消息不被员工消息打断的段;NULL 方向不参与)
cust_threads: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default=sa.text("0")
)
# 被员工回复的串数 ≡ 首响样本数 fr_cnt(fr_cnt=threads_answered 不单列)
threads_answered: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default=sa.text("0")
)
# 首响样本时长和(ms):展示用首响时长 = fr_sum_ms / threads_answered
fr_sum_ms: Mapped[int] = mapped_column(
sa.BigInteger, nullable=False, default=0, server_default=sa.text("0")
)
updated_at: Mapped[dt.datetime] = _updated_at()
@@ -829,6 +845,17 @@ class StatDailyStaff(Base):
active_rooms: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default=sa.text("0")
)
# 首响/回复率三列(2026-09-07 批1):msg_daily_stats 同义列按账号→员工
# 累加。回复率=Σanswered/Σthreads;首响时长=Σfr_sum_ms/Σanswered。
cust_threads: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default=sa.text("0")
)
threads_answered: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default=sa.text("0")
)
fr_sum_ms: Mapped[int] = mapped_column(
sa.BigInteger, nullable=False, default=0, server_default=sa.text("0")
)
updated_at: Mapped[dt.datetime] = _updated_at()
yunke-server/app/storage/staff_summary.py
+21 -2
View File
@@ -89,6 +89,10 @@ async def recompute_days(
MsgDailyStats.peer_id,
MsgDailyStats.send_cnt,
MsgDailyStats.recv_cnt,
# 首响/回复率三列(2026-09-07 批1):同义聚合累加
MsgDailyStats.cust_threads,
MsgDailyStats.threads_answered,
MsgDailyStats.fr_sum_ms,
).where(
MsgDailyStats.day == day, MsgDailyStats.channel == channel
)
@@ -102,13 +106,25 @@ async def recompute_days(
if not rows:
continue
agg: dict[str, dict] = {}
for owner, kind, peer, s, r in rows:
for owner, kind, peer, s, r, t, a_ns, fr in rows:
uid = amap.get(owner, owner) # 映射不上按 owner 原值(读侧标注)
a = agg.setdefault(
uid, {"sent": 0, "recv": 0, "friends": set(), "rooms": set()}
uid,
{
"sent": 0,
"recv": 0,
"friends": set(),
"rooms": set(),
"threads": 0,
"answered": 0,
"fr": 0,
},
)
a["sent"] += int(s or 0)
a["recv"] += int(r or 0)
a["threads"] += int(t or 0)
a["answered"] += int(a_ns or 0)
a["fr"] += int(fr or 0)
if kind == "friend": # 显式分支:未来新 kind 跳过,不误入 rooms
a["friends"].add(peer)
elif kind == "room":
@@ -123,6 +139,9 @@ async def recompute_days(
msg_recv=a["recv"],
active_friends=len(a["friends"]),
active_rooms=len(a["rooms"]),
cust_threads=a["threads"],
threads_answered=a["answered"],
fr_sum_ms=a["fr"],
)
)
n += 1
yunke-server/app/storage/summary.py
+55 -2
View File
@@ -14,6 +14,17 @@
计数语义与旧实时聚合完全一致:is_send=True 计 send、False 计 recv、
NULL 两边不计;timestamp 为 NULL 的消息不参与统计。
首响/回复率三列(2026-09-07 批1,A1+A2)——「消息串」口径(与 8-24
spec 对齐,定案):
- **串**:同会话内客户连续消息(is_send=False)不被员工消息(is_send=True)
打断的段;is_send=None(unknown)不参与串判定(跳过——不打断也不起串)。
- **首响样本**:每个被回串一个样本 fr_ms = 员工首条回复 ts − 串末条客户
消息 ts。fr_cnt ≡ threads_answered(被回串数=样本数),不单列。
- **日切近似**(v1):单日内计算,日末未闭合的串计 threads 不计 answered,
跨夜的串在次日段重新起算——8-24 spec 的 24h 窗口径的简化,读端拼均值
(fr_sum_ms / threads_answered)不受影响,仅跨夜串的归属日有偏差。
"""
from __future__ import annotations
@@ -113,6 +124,36 @@ def _peer_filter(kind: str, peer: str) -> list:
]
def _thread_stats(rows: list) -> tuple[int, int, int]:
"""消息串口径三值:(cust_threads, threads_answered, fr_sum_ms)。
- 串 = 客户连续消息(is_send=False)不被员工消息(is_send=True)打断
的段;is_send=None(unknown)不参与串判定——跳过,不打断也不起串。
- 每个被回串一个首响样本 fr_ms = 员工首条回复 ts − 串**末条**客户消息
ts;fr_sum_ms 为样本时长和(展示用首响时长 = fr_sum_ms/answered)。
- 日切近似 v1:日末未闭合的串计 threads 不计 answered(跨夜的串在次日
段重新起算——8-24 spec 的 24h 窗口径的简化)。
- rows 须先按 ts 升序(调用方 sorted;SQL 无 ORDER BY,乱序会错算串)。
"""
threads = answered = 0
fr_sum = 0
in_thread = False
thread_last_ts: int | None = None
for ts, is_send in rows:
if is_send is False:
if not in_thread:
in_thread = True
threads += 1
thread_last_ts = int(ts)
elif is_send is True:
if in_thread and thread_last_ts is not None:
answered += 1
fr_sum += int(ts) - thread_last_ts
in_thread = False
# is_send=None:跳过(不参与串判定)
return threads, answered, fr_sum
async def recompute_days(
session: AsyncSession,
keys: set[tuple[str, str, str, str, int]] | list[tuple[str, str, str, str, int]],
@@ -140,7 +181,10 @@ async def recompute_days(
send = recv = 0
last_ts: int | None = None
last_is_send: bool | None = None
for ts, is_send in rows:
# 串算法需要严格时序:先排序(SQL 无 ORDER BY,旧代码遍历中 last_ts
# 用 >= 兜乱序;排序后该兜底语义不变——max ts 与并列末条方向一致)。
ordered = sorted(rows, key=lambda r: r[0])
for ts, is_send in ordered:
if is_send is True:
send += 1
elif is_send is False:
@@ -148,6 +192,7 @@ async def recompute_days(
if last_ts is None or ts >= last_ts:
last_ts = int(ts)
last_is_send = is_send
threads, answered, fr_sum = _thread_stats(ordered)
await session.execute(
sa.delete(MsgDailyStats).where(
MsgDailyStats.channel == channel,
@@ -169,6 +214,9 @@ async def recompute_days(
recv_cnt=recv,
last_ts=last_ts,
last_is_send=last_is_send,
cust_threads=threads,
threads_answered=answered,
fr_sum_ms=fr_sum,
)
)
n += 1
@@ -224,7 +272,8 @@ async def recompute_all(session: AsyncSession) -> int:
send = recv = 0
last_ts = None
last_is_send = None
for ts, is_send in day_rows:
ordered = sorted(day_rows, key=lambda r: r[0]) # 串算法需严格时序
for ts, is_send in ordered:
if is_send is True:
send += 1
elif is_send is False:
@@ -232,6 +281,7 @@ async def recompute_all(session: AsyncSession) -> int:
if last_ts is None or ts >= last_ts:
last_ts = int(ts)
last_is_send = is_send
threads, answered, fr_sum = _thread_stats(ordered)
session.add(
MsgDailyStats(
channel=channel,
@@ -243,6 +293,9 @@ async def recompute_all(session: AsyncSession) -> int:
recv_cnt=recv,
last_ts=last_ts,
last_is_send=last_is_send,
cust_threads=threads,
threads_answered=answered,
fr_sum_ms=fr_sum,
)
)
total_pairs += 1