18 KiB
插件推送改造 实施计划 (Plan 2)
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: 在现有 d:\alibb\ Chrome 插件(已实现抓取 + API 直拉聊天)上新增推送模块:抓完一个询盘自动入队,由 service worker 经 chrome.alarms 消费、POST 到 Plan 1 后端 /api/ingest。带 IndexedDB 队列(防丢)+ 指数退避重试 + 配置页 + 运行时域名授权。保留现有本地导出能力(双写)。
Architecture:
background.js(SW)改 ES module(manifesttype:module),import推送逻辑push/*.js- 推送逻辑分三个 ES module(可被 node + vitest 测试):
push/queue.js(IndexedDB 队列)、push/payload.js(组装后端 payload)、push/worker.js(消费 + POST + 重试) content.js保持经典脚本,抓完一条 →chrome.runtime.sendMessage({type:'enqueue', record, messages})发原始数据给 SW;SW 组装 payload + 入队- IndexedDB 由 SW 落盘(content script 的 IDB 属页面源,v3 修订),
chrome.storage.local只存queue_index+ 元数据 popup加「后端设置」(URL/token/启用)+ 推送状态显示;chrome.permissions.request运行时授权后端域名
Tech Stack: 现有 JS 插件 + vitest 2 + fake-indexeddb(node 测队列)+ Chrome MV3 API(alarms/indexedDB/permissions/storage)。
关联: spec docs/superpowers/specs/2026-07-13-alibaba-inquiry-backend-design.md §8;后端 Plan 1 已就绪(d:\alibb-server\backend,/api/ingest)。
前提: d:\alibb 当前非 git 仓库,Task 1 会 git init。Plan 1 后端能 docker compose up 跑起来供端到端验证。
文件结构
d:\alibb\ # 插件根(本 plan 改这里)
manifest.json # 改:type:module + permissions(alarms/unlimitedStorage) + optional_host_permissions
background.js # 改:ES module,import push/*,alarms worker,接收 enqueue;保留 storeDetail/getBgData(本地导出)
content.js # 改:apiMsgToChat 扩展(messageId/sendTime ISO/translation);fetchChatByApi 扩展(meta);processCurrentDetailPage 加 enqueue
popup.html / popup.js # 改:加后端设置区 + 推送状态显示
push\
queue.js # 新:IndexedDB 队列操作(addItem/getPending/markStatus/remove/getStats)
payload.js # 新:buildIngestPayload(record, messages) → 适配后端 zod
worker.js # 新:processQueue(消费 + POST + 指数退避)
tests\
queue.test.js # vitest + fake-indexeddb
payload.test.js # 纯函数测试
worker.test.js # mock fetch + fake-indexeddb
package.json # 新:vitest + fake-indexeddb
vitest.config.ts
Task 1: node 测试基础 + git init
Files: package.json, vitest.config.ts, .gitignore
- Step 1:
cd d:/alibb && git init(若已是 git 仓库则跳过) - Step 2:
package.json:{ "name":"alibaba-inquiry-plugin", "type":"module", "scripts":{"test":"vitest run"} } - Step 3:
npm install -D vitest fake-indexeddb - Step 4:
vitest.config.ts:{ test:{ globals:true, testTimeout:30000, poolOptions:{ forks:{ singleFork:true } }, setupFiles:['fake-indexeddb/auto'] } } - Step 5:
.gitignore:node_modules/ .env(注意:不忽略 push/*.js、content.js 等插件文件) - Step 6: commit
chore: add node test harness (vitest + fake-indexeddb)
Task 2: manifest 改造
Files: manifest.json
- Step 1: permissions 加
alarms、unlimitedStorage→["storage","activeTab","alarms","unlimitedStorage"] - Step 2: 加
optional_host_permissions: ["http://*/*","https://*/*"](后端域名运行时授权) - Step 3: background 加
"type":"module"→{ "service_worker":"background.js", "type":"module" } - Step 4: commit
feat(manifest): alarms/unlimitedStorage + optional_host_permissions + sw module
Task 3: push/queue.js(IndexedDB 队列)
Files: push/queue.js, tests/queue.test.js
- Step 1: 写
tests/queue.test.ts(TDD):addItem → getPending 返回含它;markStatus('sent') → 不在 pending;getStats → {pending,failed,sent} 计数 - Step 2: 跑测试确认失败
- Step 3: 写
push/queue.js:
const DB_NAME = 'inquiry_push', STORE = 'queue', VERSION = 1;
function openDb() {
return new Promise((resolve, reject) => {
const req = indexedDB.open(DB_NAME, VERSION);
req.onupgradeneeded = () => req.result.createObjectStore(STORE, { keyPath: 'id' });
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
}
function tx(db, mode) { return db.transaction(STORE, mode).objectStore(STORE); }
function req2promise(r) { return new Promise((res, rej) => { r.onsuccess = () => res(r.result); r.onerror = () => rej(r.error); }); }
export async function addItem(item) {
const db = await openDb();
const rec = { id: crypto.randomUUID(), payload: item.payload, inquiryId: item.inquiryId,
status: 'pending', retries: 0, nextRetry: Date.now(), createdAt: Date.now() };
await req2promise(tx(db, 'readwrite').add(rec));
db.close();
return rec.id;
}
export async function getPending(limit = 5) {
const db = await openDb();
const all = await req2promise(tx(db, 'readonly').getAll());
db.close();
const now = Date.now();
return all.filter(r => r.status === 'pending' && r.nextRetry <= now)
.sort((a, b) => a.nextRetry - b.nextRetry)
.slice(0, limit);
}
export async function markStatus(id, status, opts = {}) {
const db = await openDb();
const store = tx(db, 'readwrite');
const r = await req2promise(store.get(id));
if (!r) { db.close(); return; }
Object.assign(r, { status, ...opts });
await req2promise(store.put(r));
db.close();
}
export async function remove(id) {
const db = await openDb();
await req2promise(tx(db, 'readwrite').delete(id));
db.close();
}
export async function getStats() {
const db = await openDb();
const all = await req2promise(tx(db, 'readonly').getAll());
db.close();
return all.reduce((a, r) => { a[r.status] = (a[r.status] || 0) + 1; return a; }, {});
}
crypto.randomUUID()在 SW 和 node 18+ 都可用。fake-indexeddb 提供 indexedDB 全局。
- Step 4: 测试通过 + commit
feat(push): IndexedDB queue (add/getPending/markStatus/remove/stats)
Task 4: push/payload.js(组装后端 payload)
Files: push/payload.js, tests/payload.test.js
- Step 1: 测试:
buildIngestPayload(record, messages)输出匹配后端 zod(inquiryId/createTime ISO/updateTime ISO/customer{customerId,country,...}/messages[{messageId,sendTime ISO,direction,content,...}]);空字符串 behavior 归 null;messages 为空时抛错(后端 zod min(1)) - Step 2: 写
push/payload.js:
const s = v => (v == null || v === '') ? null : v;
const iso = ms => new Date(ms).toISOString();
export function buildIngestPayload(record, messages) {
if (!messages || messages.length === 0) throw new Error('messages required');
const iq = record || {};
return {
inquiry: {
inquiryId: String(iq.inquiryId),
manager: s(iq.manager) || undefined,
// 优先用 meta 的 gmtCreate/gmtModified(record.createTimeFromApi),回退列表 createTime
createTime: iq.createTimeFromApi || iq.createTime || iso(Date.now()),
updateTime: iq.updateTimeFromApi || iq.updateTime || iq.createTimeFromApi || iso(Date.now()),
status: s(iq.status) || undefined,
msgPreview: s(iq.msgPreview) || undefined,
isFlagged: !!iq.isFlagged,
},
customer: iq.customerId ? {
customerId: String(iq.customerId),
name: s(iq.detailBuyerName) || s(iq.buyerName),
country: s(iq.detailCountry) || s(iq.countryCode) ? (s(iq.detailCountry) || s(iq.countryCode)) : null,
email: s(iq.email), company: s(iq.company),
registerTime: s(iq.registerTime) || null,
avatarUrl: s(iq.detailAvatarUrl) || s(iq.avatarUrl) || undefined,
buyerTags: s(iq.buyerTags) || undefined, buyerFeatures: s(iq.buyerFeatures) || undefined,
behavior: iq.behavior ? {
productViews: n(iq.behavior.productViews), validInquiries: n(iq.behavior.validInquiries),
validRfq: n(iq.behavior.validRfq), loginDays: n(iq.behavior.loginDays),
spamInquiries: n(iq.behavior.spamInquiries), topIndustry: s(iq.behavior.topIndustry),
} : undefined,
} : null,
messages: messages.map(m => ({
messageId: String(m.messageId), sendTime: m.sendTime,
sender: s(m.sender) || undefined, direction: m.direction,
content: m.original ?? m.content ?? '', translation: s(m.translation) || '',
})),
};
}
function n(v) { const x = Number(v); return Number.isFinite(x) ? x : null; }
注:
messages已是 apiMsgToChat 转换后的对象(messageId/sendTime ISO/sender/original/direction/translation)。content 映射用 original(apiMsgToChat 产出 original)。
- Step 3: 测试通过(对照 Plan 1 后端 PayloadSchema 语义)+ commit
feat(push): buildIngestPayload adapter
Task 5: push/worker.js(消费 + POST + 重试)
Files: push/worker.js, tests/worker.test.js
- Step 1: 测试:mock global.fetch(成功 → markStatus sent 风格的 remove);失败 → retries++ nextRetry 退避;超过 8 次 → markStatus failed。注入
{addItem,getPending,markStatus,remove}mock 或用真实 fake-indexeddb + fetch mock - Step 2: 写
push/worker.js:
import { getPending, markStatus, remove } from './queue.js';
import { buildIngestPayload } from './payload.js';
const MAX_RETRIES = 8;
let _config = null; // {serverUrl, token, enabled}
export function configure(cfg) { _config = cfg; }
export async function processQueue() {
if (!_config || !_config.enabled || !_config.serverUrl || !_config.token) return { processed: 0 };
const items = await getPending(3);
let processed = 0;
for (const it of items) {
try {
const payload = buildIngestPayload(it.payload.record, it.payload.messages);
const res = await fetch(`${_config.serverUrl}/api/ingest`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${_config.token}`, 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
});
if (res.ok) { await remove(it.id); processed++; }
else { await bumpRetry(it, `HTTP ${res.status}`); }
} catch (e) {
await bumpRetry(it, String(e && e.message || e));
}
}
return { processed };
}
async function bumpRetry(it, err) {
const retries = it.retries + 1;
if (retries >= MAX_RETRIES) {
await markStatus(it.id, 'failed', { retries, lastError: err });
} else {
const delay = Math.min(Math.pow(2, retries) * 1000, 3600 * 1000);
await markStatus(it.id, 'pending', { retries, nextRetry: Date.now() + delay, lastError: err });
}
}
- Step 3: 测试通过 + commit
feat(push): worker (consume + POST + backoff)
Task 6: background.js(SW,alarms worker + 接收 enqueue)
Files: background.js
- Step 1: 顶部改 ES module import:
import { addItem, getStats } from './push/queue.js';
import { processQueue, configure } from './push/worker.js';
- Step 2: 保留现有 storeDetail/getBgData/clearBgData(本地导出)不动
- Step 3: 加配置加载 + alarms 注册 + enqueue 处理:
const CONFIG_KEY = 'push_config';
const ALARM = 'push-tick';
async function loadConfig() {
const d = await chrome.storage.local.get(CONFIG_KEY);
return d[CONFIG_KEY] || { enabled: false, serverUrl: '', token: '' };
}
// 启动 + 配置变更时刷新 worker 配置 + alarm
async function refresh() {
const cfg = await loadConfig();
configure(cfg);
if (cfg.enabled) {
await chrome.alarms.create(ALARM, { periodInMinutes: 1 });
processQueue(); // 立即跑一次
} else {
await chrome.alarms.clear(ALARM);
}
}
chrome.runtime.onInstalled.addListener(refresh);
chrome.runtime.onStartup.addListener(refresh);
chrome.alarms.onAlarm.addListener(a => { if (a.name === ALARM) processQueue(); });
chrome.runtime.onMessage.addListener((msg, sender, sendResponse) => {
if (msg.type === 'enqueue') {
(async () => {
const cfg = await loadConfig();
if (!cfg.enabled) { sendResponse({ ok: false, reason: 'disabled' }); return; }
const id = await addItem({ payload: { record: msg.record, messages: msg.messages }, inquiryId: msg.record?.inquiryId });
sendResponse({ ok: true, id });
processQueue(); // 立即尝试
})();
return true;
}
if (msg.type === 'getPushStats') {
(async () => sendResponse(await getStats()))();
return true;
}
if (msg.type === 'setPushConfig') {
(async () => {
await chrome.storage.local.set({ [CONFIG_KEY]: msg.config });
await refresh();
sendResponse({ ok: true });
})();
return true;
}
});
- Step 4: commit
feat(bg): push worker + alarms + enqueue handler (keep local export)
Task 7: content.js — apiMsgToChat 扩展
Files: content.js(apiMsgToChat ~434)
- Step 1: apiMsgToChat 改产出
{ messageId, sendTime(ISO), sender, original, translation, direction }:messageId: String(m.messageId)sendTime: new Date(m.sendTime).toISOString()(替代原来的本地时区 "YYYY-MM-DD HH:MM:SS")translation:从m.extInfo?.icbuData?.extend解析(extend 是 JSON 字符串,内含 extParams.inputTranslateInfo.translate);解析失败留''- sender/original/direction 保持
- Step 2: translation 解析加双重 JSON.parse 兜底(extend 可能是字符串里的字符串),失败不抛(留 '')
- Step 3: 手动/Playwright 验证(无 node 测试,因 content.js 非 module):用 Playwright 注入新版 apiMsgToChat + 真实 m 样本,断言 messageId/sendTime/translation 正确
- Step 4: commit
feat(content): apiMsgToChat emits messageId/sendTime(ISO)/translation
Task 8: content.js — fetchChatByApi 扩展(提取 gmtCreate/country)
Files: content.js(fetchChatByApi ~451)
- Step 1: fetchChatByApi 在调 querySummary 后,额外提取
d.gmtCreate/d.gmtModified(毫秒→ISO)存入返回 meta;d.contact?.countryCode或d.cid派生也行(spec 用 contact.countryCode)。返回{ messages, meta: { gmtCreate, gmtModified } } - Step 2: 调用方(processCurrentDetailPage / scrapeDetailData)把 meta 挂到 detail:
detail.createTimeFromApi = meta.gmtCreate; detail.updateTimeFromApi = meta.gmtModified - Step 3: commit
feat(content): fetchChatByApi extracts gmtCreate/gmtModified for payload
Task 9: content.js — processCurrentDetailPage 加 enqueue
Files: content.js(processCurrentDetailPage ~347,storeDetail 之后)
- Step 1: 在 storeDetail(本地导出)之后加 enqueue(推送):
// 推送到后端(Plan 2):发 record + messages 给 SW 入队
try {
chrome.runtime.sendMessage({
type: 'enqueue',
record: merged, // {item, detail} 合并
messages: detail.chatMessages || [], // apiMsgToChat 产出数组
}, () => {}); // fire-and-forget,失败不影响本地导出
} catch (e) {}
detail.chatMessages需在 scrapeDetailData 产出(现有 chatText 是序列化串,加一个chatMessages数组字段)。确保 merged 含createTimeFromApi/updateTimeFromApi(Task 8)。
- Step 2: commit
feat(content): enqueue record+messages to SW after detail scrape
Task 10: popup 配置页 + permissions.request
Files: popup.html, popup.js
- Step 1: popup.html 加「后端设置」区:服务器 URL 输入、Token 输入、启用推送 checkbox、「授权域名」按钮、「保存」按钮
- Step 2: popup.js:加载时
chrome.runtime.sendMessage({type:'getPushConfig'})回填(或直接chrome.storage.local.get);保存时 setPushConfig - Step 3: 「授权域名」按钮 →
chrome.permissions.request({origins:[serverUrl + '/*']})(用户手势触发);成功后提示 - Step 4: commit
feat(popup): backend config UI + runtime origin permission
Task 11: popup 推送状态显示
Files: popup.html, popup.js
- Step 1: popup 加状态条:「已推 N / 待推 M / 失败 K」
- Step 2: popup 打开时 + 每 2s
chrome.runtime.sendMessage({type:'getPushStats'})刷新 - Step 3: 失败 K > 0 时红字 + 可点击查看 lastError(简化:弹 alert 或展开列表)
- Step 4: commit
feat(popup): push stats display (pending/failed/sent)
Task 12: 端到端验证
Files: scripts/e2e-plugin.md(验证步骤文档)
- Step 1: 起 Plan 1 后端:
cd d:/alibb-server/backend && docker compose up -d --build,等 health ok - Step 2: Chrome
chrome://extensions开发者模式 → 加载d:\alibb(本 plan 改后的插件) - Step 3: 点插件图标 → 后端设置填
http://localhost:8080+ tokendev-ingest-token-1234+ 启用 → 授权域名 → 保存 - Step 4: 打开
message.alibaba.com→ 抓一个询盘(用插件现有抓取流程,或 Playwright 注入触发) - Step 5: 验证:
curl -H "Authorization: Bearer <jwt>" http://localhost:8080/api/inquiries能看到刚抓的询盘;docker compose exec mysql mysql -uroot -proot inquiry -e "SELECT inquiry_id, messages_count FROM inquiries"数据在 - Step 6: popup 状态条显示「已推 1」
- Step 7: 验证幂等:再抓同一询盘 → 后端 messages_count 不变(早返回)
- Step 8: commit
test: e2e plugin→backend validation doc
完成标准(DoD)
- manifest:
type:module+ alarms/unlimitedStorage + optional_host_permissions push/queue.js/payload.js/worker.js三个 module + vitest 测试全绿background.js:alarms(1 分钟)消费队列,POST/api/ingest,失败指数退避(≤8 次);保留本地 storeDetail/getBgDatacontent.js:apiMsgToChat 产 messageId/sendTime(ISO)/translation;fetchChatByApi 提取 gmtCreate/gmtModified;processCurrentDetailPage enqueue- popup:后端设置(URL/token/启用)+ permissions.request 授权 + 状态显示(已推/待推/失败)
- 端到端:抓一条询盘 → Plan 1 后端能看到数据;再抓同一条幂等(messages_count 不变)
- 断网/后端停 → 数据进队列不丢;后端恢复 → 自动推完
后续(本 plan 不含)
- Plan 3:管理后台前端(查看/筛选/导出/统计)
- Plan 4:正式部署(域名 + HTTPS + 备份定时 + 监控)
- 增量推送(插件记录 last sendTime,只推新增消息)
- 失败项重试 UI(popup 查看失败 + 手动重推)