浅念
322 lines
19 KiB
Markdown
322 lines
19 KiB
Markdown
# 采集阶段B：业务事件体系 — 实现计划
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** 零网站改动的业务事件体系——SDK 自动监听咨询点击/CF7 表单全过程/产品页，服务端契约+6 事件，D06 归因 paid_social，admin 时间线+询盘漏斗卡。
|
||
|
||
**Architecture:** SDK 新模块 business.ts（与 page.ts/stay.ts 同模式，事件携带阶段A 归属）；server 白名单+校验扩展+归因规则 v2（fb+paid 优先）；admin 时间线徽章+漏斗 API/卡片。CF7 事实已按线上 6.1.6 源码核验写入 spec §1.2。
|
||
|
||
**Tech Stack:** sdk/（TS+vitest+esbuild，基线 100 测试）；server/（Django+DRF，基线 161 测试）；admin/（Vue3+Element Plus）。
|
||
|
||
**Spec:** docs/superpowers/specs/2026-09-18-collection-phase-b-design.md（§1.2 已含 CF7 实测事实——wpcf7statuschanged 状态机/原生 submit capture/FormData hidden 采集）
|
||
|
||
**执行顺序：** T1(server 契约) → T2(归因 D06) → T3(漏斗 API) → T4(SDK business.ts) → T5(track 修复+接线) → T6(admin) → T7(部署，排 9/20 观察窗后) → T8(线上验证+测试询盘)
|
||
|
||
**测试命令：** server `cd server && python -m pytest tracker/tests/ -q`；sdk `cd sdk && npx vitest run`。
|
||
|
||
**CF7 已核验事实（实现者必读，spec §1.2）：**
|
||
- CF7 拦截原生 submit 于 form 元素冒泡阶段（`addEventListener("submit", e => { wpcf7.submit(...); e.preventDefault() })`）→ SDK 的 **document capture 委托必然先执行**
|
||
- 状态事件 `wpcf7statuschanged`（bubbles:true，`detail.status` ∈ init/validating/invalid/unaccepted/spam/mail_sent/mail_failed/resetting）
|
||
- FormData：`:where(input,textarea,select):enabled` 逐字段 append，hidden 走 default 分支（`_ctk_sid` 会被采集）
|
||
- `wpcf7submit` 在 AJAX 响应后才触发——**弃用**
|
||
|
||
---
|
||
|
||
### Task 1: Server 契约扩展（+6 事件与校验）
|
||
|
||
**Files:**
|
||
- Modify: `server/tracker/constants.py`（EVENT_NAMES/EVENT_PROP_WHITELIST）
|
||
- Modify: `server/tracker/services/validation.py`（validate_event 新键校验）
|
||
- Test: `server/tracker/tests/test_validation.py`
|
||
|
||
- [ ] **Step 1: 失败测试**（追加）：
|
||
|
||
```python
|
||
import pytest
|
||
|
||
def _biz(name, **props):
|
||
return {"event_id": f"biz-{name}", "event_name": name, "occurred_at": timezone.now(),
|
||
"path": "/p", "title": "t", "site_id": SITE_ID, "visitor_id": "v1",
|
||
"session_id": "s1", "page_id": "p1", "seq": 1, "properties": props}
|
||
|
||
@pytest.mark.parametrize("ev,expect_reason", [
|
||
(_biz("contact_click", channel="whatsapp"), None),
|
||
(_biz("contact_click", channel="im"), "property_not_allowed"), # 枚举外
|
||
(_biz("form_start", form_id="cf7-156"), None),
|
||
(_biz("form_start", form_id="wpcf7-f156-p20"), "property_not_allowed"), # 格式不符
|
||
(_biz("form_submit", form_id="cf7-156", submission_id="9c8b7a6d-1111-4222-8333-444455556666"), None),
|
||
(_biz("form_submit", form_id="cf7-156", submission_id=""), None), # 空串合法（极端时序）
|
||
(_biz("form_submit", form_id="cf7-156", submission_id="not-uuid"), "property_not_allowed"),
|
||
(_biz("form_error", form_id="cf7-156", submission_id="", error_code="invalid", field_count=2), None),
|
||
(_biz("form_error", form_id="cf7-156", submission_id="", error_code="bogus"), "property_not_allowed"),
|
||
(_biz("form_error", form_id="cf7-156", submission_id="", error_code="invalid", field_count=-1), "property_not_allowed"),
|
||
(_biz("form_submit_result", form_id="cf7-156", submission_id="", result_code="mail_sent"), None),
|
||
(_biz("form_submit_result", form_id="cf7-156", submission_id="", result_code="ok"), "property_not_allowed"),
|
||
(_biz("product_view", product_id="byd-equation-leopard-5", inferred=True, rule_version="v1"), None),
|
||
(_biz("product_view", product_id="x"*101, inferred=True, rule_version="v1"), "property_not_allowed"),
|
||
])
|
||
def test_biz_event_validation(ev, expect_reason):
|
||
assert validate_event(ev, v2=True, batch_site_id=SITE_ID, batch_visitor_id="v1") == expect_reason
|
||
```
|
||
|
||
（SITE_ID/imports 参照该文件既有测试；timezone 从 django.utils 导入。）
|
||
|
||
- [ ] **Step 2: 跑失败**（property_not_allowed——白名单无新事件）
|
||
|
||
- [ ] **Step 3: 实现——constants.py**：
|
||
|
||
```python
|
||
EVENT_NAMES = {..., "contact_click", "form_start", "form_error", "form_submit", "form_submit_result", "product_view"}
|
||
EVENT_PROP_WHITELIST = {...,
|
||
"contact_click": {"channel"},
|
||
"form_start": {"form_id"},
|
||
"form_submit": {"form_id", "submission_id"},
|
||
"form_error": {"form_id", "submission_id", "error_code", "field_count"},
|
||
"form_submit_result": {"form_id", "submission_id", "result_code"},
|
||
"product_view": {"product_id", "inferred", "rule_version"},
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: 实现——validation.py**（validate_event 尾部新增，白名单通过后）：
|
||
|
||
```python
|
||
_CHANNEL = {"whatsapp", "phone", "email"}
|
||
_RESULT = {"mail_sent", "mail_failed"}
|
||
_ERROR = {"invalid", "spam", "other"}
|
||
_SUBMIT_RE = re.compile(r"^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$|^$")
|
||
_FORM_RE = re.compile(r"^cf7-\d{1,10}$")
|
||
_PROD_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,99}$")
|
||
|
||
def _biz_props_ok(name, props) -> bool:
|
||
ch = lambda k: isinstance(props.get(k), str)
|
||
if name == "contact_click": return ch("channel") and props["channel"] in _CHANNEL
|
||
if name == "form_start": return ch("form_id") and _FORM_RE.match(props["form_id"])
|
||
if name in ("form_submit", "form_submit_result"):
|
||
if not (ch("form_id") and _FORM_RE.match(props["form_id"])): return False
|
||
sid = props.get("submission_id", "")
|
||
return isinstance(sid, str) and bool(_SUBMIT_RE.match(sid))
|
||
if name == "form_error":
|
||
if not (ch("form_id") and _FORM_RE.match(props["form_id"])): return False
|
||
sid = props.get("submission_id", "")
|
||
fc = props.get("field_count")
|
||
return (isinstance(sid, str) and bool(_SUBMIT_RE.match(sid))
|
||
and ch("error_code") and props["error_code"] in _ERROR
|
||
and isinstance(fc, int) and 0 <= fc <= 99 and not isinstance(fc, bool))
|
||
if name == "product_view":
|
||
return (ch("product_id") and bool(_PROD_RE.match(props["product_id"]))
|
||
and isinstance(props.get("inferred"), bool)
|
||
and ch("rule_version") and len(props["rule_version"]) <= 16)
|
||
return True
|
||
```
|
||
|
||
（调用点：validate_event 白名单检查后 `if ev["event_name"] in C.BIZ_EVENT_NAMES and not _biz_props_ok(...): return "property_not_allowed"`——枚举常量放 constants：`BIZ_EVENT_NAMES = {...6 个}`，EVENT_NAMES = 旧4 ∪ BIZ。）
|
||
|
||
- [ ] **Step 5: 全量回归 161+新增全过 → Commit** `feat(server): 业务事件契约——白名单+6事件与枚举校验`
|
||
|
||
---
|
||
|
||
### Task 2: D06 归因扩展（paid_social + fb 优先规则）
|
||
|
||
**Files:**
|
||
- Modify: `server/tracker/constants.py`（CHANNELS/ATTRIBUTION_RULE_VERSION）
|
||
- Modify: `server/tracker/services/attribution.py`（classify 分支）
|
||
- Test: `server/tracker/tests/test_attribution.py`
|
||
|
||
- [ ] **Step 1: 失败测试**（追加；helper 沿文件既有 classify 调用形态）：
|
||
|
||
```python
|
||
def test_fb_paid_medium_priority():
|
||
# 实测主导模式：utm_medium=paid&utm_source=fb → paid_social（优先于强化集）
|
||
r = classify(referrer_url="", utm={"utm_source": "fb", "utm_medium": "paid"},
|
||
landing_path="/", ua="", internal_domains=[])
|
||
assert r["channel"] == "paid_social"
|
||
|
||
def test_fb_family_medium_union():
|
||
for m in ("cpc", "social", "paid_social", "social_ads"):
|
||
r = classify(referrer_url="", utm={"utm_source": "facebook.com", "utm_medium": m},
|
||
landing_path="/", ua="", internal_domains=[])
|
||
assert r["channel"] == "paid_social", m
|
||
|
||
def test_nonfb_paid_medium_stays_paid_search_label_only():
|
||
r = classify(referrer_url="", utm={"utm_source": "google", "utm_medium": "cpc"},
|
||
landing_path="/", ua="", internal_domains=[])
|
||
assert r["channel"] == "paid_search" and r["resolution"] == "label_only"
|
||
|
||
def test_fbclid_now_paid_social():
|
||
r = classify(referrer_url="", utm={}, landing_path="/?fbclid=IwARxyz", ua="", internal_domains=[])
|
||
assert r["channel"] == "paid_social"
|
||
|
||
def test_gclid_still_paid_search():
|
||
r = classify(referrer_url="", utm={}, landing_path="/?gclid=EAIa", ua="", internal_domains=[])
|
||
assert r["channel"] == "paid_search"
|
||
```
|
||
|
||
- [ ] **Step 2: 跑失败**（paid_social 不存在）
|
||
|
||
- [ ] **Step 3: 实现**：
|
||
- constants：`CHANNELS = (..., "paid_social", ...)`（paid_search 之后）；`ATTRIBUTION_RULE_VERSION = "v2"`；付费 medium 全集常量 `PAID_MEDIUMS = {"cpc","ppc","paidsearch","paid","social","paid_social","social_ads","paidsocial"}`、`SOCIAL_MEDIUMS = {"social","paid_social","social_ads","paidsocial"}`
|
||
- attribution.py classify：现有 `_paid_param` 分支改：fbclid → paid_social / gclid/msclkid/yclid → paid_search；**新增分支（在 _paid_param 之后、referrer 判定之前）**：无点击参数时若 utm_source 命中 fb 家族（复用 `_AD_FAMILIES`/家族表）且 utm_medium∈PAID_MEDIUMS → paid_social（evidence_kind="utm_label", resolution="label_only"）；utm_source 非 fb 家族但 medium∈PAID_MEDIUMS-\SOCIAL_MEDIUMS → paid_search label_only（既有强化行为保持）
|
||
- [ ] **Step 4: 全量回归 + 既有归因测试按新口径更新**（fbclid 断言从 paid_search 改 paid_social 等预期内变更）
|
||
- [ ] **Step 5: Commit** `feat(server): 归因v2——paid_social渠道与fb+付费medium优先规则`
|
||
|
||
---
|
||
|
||
### Task 3: 询盘漏斗 API
|
||
|
||
**Files:**
|
||
- Create/Modify: `server/tracker/api/dashboard.py`（加 funnel_summary 视图或并入既有 dashboard 视图集）+ `server/config/urls.py`（路由）
|
||
- Test: `server/tracker/tests/test_dashboard.py`
|
||
|
||
- [ ] **Step 1: 失败测试**：
|
||
|
||
```python
|
||
def test_funnel_summary_independent_session_counting(db, ...):
|
||
# 会话A：2次page_view+1次contact_click；会话B：1次page_view+1次form_submit；bot事件若干
|
||
# 断言五级 = [2, 1, 0, 1, 0]（distinct session，非嵌套非事件数）
|
||
def test_funnel_summary_excludes_bots(db, ...):
|
||
# is_bot=True 的事件不计入任何级
|
||
```
|
||
|
||
- [ ] **Step 2: 实现**：`GET /api/admin/funnel-summary?days=N`（默认 7；权限/site 过滤沿 dashboard 既有模式）。五级 event_name 序列 `["page_view","contact_click","form_start","form_submit","form_submit_result"]`，每级 `Event.objects.filter(site=, occurred_at__gte=窗口, is_bot=False, event_name=级).values("session").distinct().count()`。
|
||
|
||
- [ ] **Step 3: 回归 + Commit** `feat(server): 询盘漏斗API——五级独立distinct会话计数/排除机器人`
|
||
|
||
---
|
||
|
||
### Task 4: SDK business.ts（核心模块）
|
||
|
||
**Files:**
|
||
- Create: `sdk/src/business.ts`
|
||
- Modify: `sdk/src/page.ts`（onSegmentChange 钩子）
|
||
- Test: `sdk/test/business.test.ts`（新建）
|
||
|
||
- [ ] **Step 1: page.ts 加 onSegmentChange**（先失败测试再实现——与 onClose 同模式）：
|
||
|
||
```ts
|
||
// 模块级 segmentListeners: Set<(ps: PageSession) => void>
|
||
export function onSegmentChange(fn: (ps: PageSession) => void): void
|
||
// onRouteChange 内新 PageSession 创建并 emitPageView 之后：for fn of segmentListeners → try fn(currentPs) catch 静默
|
||
```
|
||
|
||
（**不在 close() 内触发**——close 覆盖 pagehide/重复 init，会导致卸载重复 product_view，spec §1.3。）
|
||
|
||
- [ ] **Step 2: business.ts 失败测试清单**（fake timers + happy-dom，沿 page.test.ts 模式）：
|
||
|
||
```
|
||
contact_click：wa.me/tel:/mailto: 链接点击 → 事件 channel 正确 + flush(true) 被调
|
||
（非链接点击/其他 http 链接不触发；wa.me 与 api.whatsapp.com 与 whatsapp:// 三形态）
|
||
form_start：.wpcf7-form 内 isTrusted focusin → 事件 + _ctk_sid 空占位注入
|
||
（同表单第二次 focusin 不重发；isTrusted=false 的合成 focus 不触发）
|
||
form_submit：表单原生 submit（capture 委托）→ submission_id 生成 + _ctk_sid 值更新 + 事件引用同 ID
|
||
（两次 submit → 两个不同 submission_id）
|
||
form_error：dispatch wpcf7statuschanged(status="invalid") → error_code=invalid；
|
||
status="spam" → spam；field_count 从 DOM .wpcf7-not-valid-tip 计数
|
||
form_submit_result：status="mail_sent" → result_code=mail_sent；"mail_failed" → mail_failed
|
||
form_id：action="#wpcf7-f156-p20-o1" → "cf7-156"
|
||
product_view：/product/slug/ 与 /de/product/slug/（非锚定段匹配）→ 事件 inferred=true；
|
||
/product-category/suv/ 不触发；onSegmentChange 切段到产品页 → 事件带新 page_id
|
||
红线：监听器内 throw 不冒泡
|
||
```
|
||
|
||
- [ ] **Step 3: 实现 business.ts**（结构）：
|
||
|
||
```ts
|
||
export function startBusinessTracking(t: Transport): void {
|
||
// 1) contact_click：document click capture 委托——closest("a[href]") 命中协议前缀
|
||
// （wa.me|api.whatsapp.com|whatsapp:// → whatsapp；tel: → phone；mailto: → email）
|
||
// emit 后 t.flush(true)
|
||
// 2) form_*：document focusin+input capture（.wpcf7-form 委托，isTrusted，每 form WeakSet 一次）
|
||
// → form_start + 注入 <input type=hidden name=_ctk_sid value="">
|
||
// document submit capture：target.closest(".wpcf7-form") → submission_id=uuid()
|
||
// → 更新 _ctk_sid 值 → form_submit 事件引用
|
||
// document wpcf7statuschanged：detail.status 映射 form_error/form_submit_result
|
||
// （引用最近 submission_id——WeakMap<FormElement, string>；field_count =
|
||
// form.querySelectorAll(".wpcf7-not-valid-tip").length）
|
||
// 3) form_id：从 form action/id 正则 /wpcf7-f(\d+)/ → `cf7-${ID}`；解析不出 → "cf7-0"
|
||
// 4) product_view：matchProduct(path)（段匹配：路径段数组含 "product" 且其后一段为 slug）
|
||
// init 时对当前 path 判定 + onSegmentChange(ps => matchProduct(location.pathname))
|
||
// 所有 emit：事件结构 = events 形态 + ps.attribution() 顶层展开（同 stay.ts 模式）
|
||
// 所有回调 try/catch 静默（红线）
|
||
}
|
||
function matchProduct(path: string): string | null {
|
||
const segs = path.split("/").filter(Boolean);
|
||
const i = segs.indexOf("product");
|
||
if (i >= 0 && i + 1 < segs.length && segs[i + 1]) return segs[i + 1].slice(0, 100);
|
||
return null;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: 全量 `npx vitest run`（100+新增）+ tsc 干净 → Commit** `feat(sdk): business.ts——咨询点击/CF7状态机/产品页自动监听`
|
||
|
||
---
|
||
|
||
### Task 5: track 通道修复 + 0.2.1 接线
|
||
|
||
**Files:**
|
||
- Modify: `sdk/src/index.ts`（track 补归属 + business 接线 + 版本 0.2.1）
|
||
- Modify: `sdk/package.json`（version 0.2.1）
|
||
- Modify: `sdk/src/types.ts`（SDK_VERSION）
|
||
- Test: `sdk/test/index.test.ts`
|
||
|
||
- [ ] **Step 1: 失败测试**：
|
||
- track 事件带归属顶层字段（site/visitor/session/page/seq 来自当前 PageSession）
|
||
- init 后队列含 page_view/sdk_loaded/product_view（产品页路径下）/business 监听已装（可从 contact_click 事件冒烟验证）
|
||
- SDK_VERSION 断言 "0.2.1"
|
||
- [ ] **Step 2: 实现**：index.ts init 分支加 `startBusinessTracking(transport)`（startStayTracking 之后）；track 分支 `transport.push({ ...arg, ...getPageSession()?.attribution() })`（PageSession 未启动时保持原样——兜底组路径）
|
||
- [ ] **Step 3: 全量 + build（记录 gzip）→ Commit** `feat(sdk): 0.2.1——track补归属+business接线`
|
||
|
||
---
|
||
|
||
### Task 6: Admin（时间线新事件 + 漏斗卡）
|
||
|
||
**Files:**
|
||
- Modify: `admin/src/views/tracker/visitor/components/EventTimeline.vue`（EVENT_META 扩展）
|
||
- Modify: `admin/src/views/tracker/dashboard/index.vue`（漏斗卡）
|
||
- Modify: `admin/src/api/`（漏斗 API 函数，沿 dashboard 既有 API 模块）
|
||
- Test: Playwright 视觉验证（T6 内含）
|
||
|
||
- [ ] **Step 1: EventTimeline EVENT_META 扩展**（先读现有 EVENT_META 结构）：
|
||
|
||
```
|
||
contact_click → 图标 ChatDotRound，label 咨询点击，副行显示 channel 中文（whatsapp→WhatsApp 等）
|
||
form_start → EditPen 表单开始
|
||
form_submit → Promotion 表单提交（submission_id 前 8 位）
|
||
form_error → WarningFilled 表单校验未过（error_code+field_count）
|
||
form_submit_result → 成功/失败按 result_code 区分（mail_sent→SuccessFilled 邮件已发（未验证）/mail_failed→CircleCloseFilled 发送失败）
|
||
product_view → Van 产品查看（product_id，inferred 标记）
|
||
```
|
||
|
||
- [ ] **Step 2: 漏斗卡**（dashboard）：
|
||
- API：`getFunnelSummary(days)` → `{levels: [{key, label, count}]}`；当日/近7天切换（days=1/7）
|
||
- 卡片 UI：五级横向计数条（各级 label+count，**无百分比**）；末级 label"邮件已发（未验证）"；样式沿客迹 V1 卡片（参考现有渠道分布卡的写法）；位置：渠道分布卡下方或旁（看排版）
|
||
- 渠道徽章：grep admin 中 paid_search/付费搜索 的标签映射处，加 paid_social→付费社交
|
||
|
||
- [ ] **Step 3: pnpm build 通过 → Commit** `feat(admin): 时间线业务事件徽章+询盘漏斗卡`
|
||
|
||
---
|
||
|
||
### Task 7: 部署（排 9/20 观察窗 D02 确认后执行）
|
||
|
||
- [ ] **Step 1: 前置确认**：观察窗指标（孤立 page_stay 归零）已核验；server/sdk/admin 全量测试绿
|
||
- [ ] **Step 2: server 先行**：tar → scp 经跳板 → /opt/cms-cj 原地覆盖 → docker compose up -d --build web → 容器/探活 → v1/v2 回归批
|
||
- [ ] **Step 3: 归因重算**：`docker exec tracker-web python manage.py recompute_attribution --site site_T3Fkse332jjbejPF` → 汇报渠道口径变化（付费搜索部分移至付费社交——预期内）
|
||
- [ ] **Step 4: admin**：build → tar → scp → 解压 → **cp -rf dist/* admin/dist/**（真实挂载点）→ curl 哈希验证
|
||
- [ ] **Step 5: SDK 0.2.1**：build → gzip 记录 → tar → scp → **cp 到 sdk/dist**（真实挂载点）→ curl 验证 build_id
|
||
|
||
### Task 8: 线上验证 + 测试询盘闭环
|
||
|
||
- [ ] **Step 1: contact_click/product_view 验证**：Playwright 访问 hesucar 产品页（等 15s flush 或触发交互）→ 生产库查 product_view 落库（inferred=true）；点 WhatsApp 链接 → contact_click 落库
|
||
- [ ] **Step 2: form_ 全链路（测试询盘，用户已接受）**：hesucar 联系页表单填 Name="TEST-KEJI"、Tel/Message 合法值 → 提交 → 生产库断言 form_start/form_submit（submission_id 非空）/form_submit_result(mail_sent) 落库；漏斗 API 五级数字 +1 验证；_ctk_sid 已随 POST（可通过 RawBatch 批次体佐证——form_submit 事件的 submission_id 非空即间接证明 capture 时序成立）
|
||
- [ ] **Step 3: admin 视觉**：登录后台 → 访客详情时间线显示新事件徽章 → 数据总览漏斗卡渲染正确
|
||
- [ ] **Step 4: 清理与汇报**：测试询盘数据保留（TEST-KEJI 标记可辨识，商家可自行删除邮件）；汇报部署版本/验证结果/归因口径变化
|
||
|
||
---
|
||
|
||
## 风险提示（执行者必读）
|
||
|
||
1. **CF7 事实已核验**（spec §1.2）——不要按旧文档/记忆使用 wpcf7invalid/wpcf7mailsent；一切以 wpcf7statuschanged 状态机为准。
|
||
2. SDK 红线：所有回调 try/catch 静默（§4.5）；business.ts 的 DOM 监听全部 capture 委托（时序确定性依赖）。
|
||
3. Task 2 既有归因测试的口径变更是预期内（fbclid→paid_social 等），逐个核对更新而非批量跳过。
|
||
4. 部署顺序硬约束：server → admin → SDK；T7 整体排在观察窗确认后。
|
||
5. `_ctk_sid` 注入使用 DOM API（createElement+appendChild），勿用 innerHTML（宿主页安全）。
|