Files
alibaba/docs/superpowers/plans/2026-07-14-plugin-push.md
T
2026-08-27 16:12:37 +08:00

18 KiB

插件推送改造 实施计划 (Plan 2)

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: 在现有 d:\alibb\ Chrome 插件(已实现抓取 + API 直拉聊天)上新增推送模块:抓完一个询盘自动入队,由 service worker 经 chrome.alarms 消费、POST 到 Plan 1 后端 /api/ingest。带 IndexedDB 队列(防丢)+ 指数退避重试 + 配置页 + 运行时域名授权。保留现有本地导出能力(双写)。

Architecture:

  • background.js(SW)改 ES module(manifest type:module),import 推送逻辑 push/*.js
  • 推送逻辑分三个 ES module(可被 node + vitest 测试):push/queue.js(IndexedDB 队列)、push/payload.js(组装后端 payload)、push/worker.js(消费 + POST + 重试)
  • content.js 保持经典脚本,抓完一条 → chrome.runtime.sendMessage({type:'enqueue', record, messages}) 发原始数据给 SW;SW 组装 payload + 入队
  • IndexedDB 由 SW 落盘(content script 的 IDB 属页面源,v3 修订),chrome.storage.local 只存 queue_index + 元数据
  • popup 加「后端设置」(URL/token/启用)+ 推送状态显示;chrome.permissions.request 运行时授权后端域名

Tech Stack: 现有 JS 插件 + vitest 2 + fake-indexeddb(node 测队列)+ Chrome MV3 API(alarms/indexedDB/permissions/storage)。

关联: spec docs/superpowers/specs/2026-07-13-alibaba-inquiry-backend-design.md §8;后端 Plan 1 已就绪(d:\alibb-server\backend,/api/ingest)。

前提: d:\alibb 当前非 git 仓库,Task 1 会 git init。Plan 1 后端能 docker compose up 跑起来供端到端验证。


文件结构

d:\alibb\                      # 插件根(本 plan 改这里)
  manifest.json                # 改:type:module + permissions(alarms/unlimitedStorage) + optional_host_permissions
  background.js                # 改:ES module,import push/*,alarms worker,接收 enqueue;保留 storeDetail/getBgData(本地导出)
  content.js                   # 改:apiMsgToChat 扩展(messageId/sendTime ISO/translation);fetchChatByApi 扩展(meta);processCurrentDetailPage 加 enqueue
  popup.html / popup.js        # 改:加后端设置区 + 推送状态显示
  push\
    queue.js                   # 新:IndexedDB 队列操作(addItem/getPending/markStatus/remove/getStats)
    payload.js                 # 新:buildIngestPayload(record, messages) → 适配后端 zod
    worker.js                  # 新:processQueue(消费 + POST + 指数退避)
  tests\
    queue.test.js              # vitest + fake-indexeddb
    payload.test.js            # 纯函数测试
    worker.test.js             # mock fetch + fake-indexeddb
  package.json                 # 新:vitest + fake-indexeddb
  vitest.config.ts

Task 1: node 测试基础 + git init

Files: package.json, vitest.config.ts, .gitignore

  • Step 1: cd d:/alibb && git init(若已是 git 仓库则跳过)
  • Step 2: package.json:{ "name":"alibaba-inquiry-plugin", "type":"module", "scripts":{"test":"vitest run"} }
  • Step 3: npm install -D vitest fake-indexeddb
  • Step 4: vitest.config.ts:{ test:{ globals:true, testTimeout:30000, poolOptions:{ forks:{ singleFork:true } }, setupFiles:['fake-indexeddb/auto'] } }
  • Step 5: .gitignore:node_modules/ .env(注意:不忽略 push/*.js、content.js 等插件文件)
  • Step 6: commit chore: add node test harness (vitest + fake-indexeddb)

Task 2: manifest 改造

Files: manifest.json

  • Step 1: permissions 加 alarms、unlimitedStorage → ["storage","activeTab","alarms","unlimitedStorage"]
  • Step 2: 加 optional_host_permissions: ["http://*/*","https://*/*"](后端域名运行时授权)
  • Step 3: background 加 "type":"module" → { "service_worker":"background.js", "type":"module" }
  • Step 4: commit feat(manifest): alarms/unlimitedStorage + optional_host_permissions + sw module

Task 3: push/queue.js(IndexedDB 队列)

Files: push/queue.js, tests/queue.test.js

  • Step 1: 写 tests/queue.test.ts(TDD):addItem → getPending 返回含它;markStatus('sent') → 不在 pending;getStats → {pending,failed,sent} 计数
  • Step 2: 跑测试确认失败
  • Step 3: 写 push/queue.js:
const DB_NAME = 'inquiry_push', STORE = 'queue', VERSION = 1;
function openDb() {
  return new Promise((resolve, reject) => {
    const req = indexedDB.open(DB_NAME, VERSION);
    req.onupgradeneeded = () => req.result.createObjectStore(STORE, { keyPath: 'id' });
    req.onsuccess = () => resolve(req.result);
    req.onerror = () => reject(req.error);
  });
}
function tx(db, mode) { return db.transaction(STORE, mode).objectStore(STORE); }
function req2promise(r) { return new Promise((res, rej) => { r.onsuccess = () => res(r.result); r.onerror = () => rej(r.error); }); }

export async function addItem(item) {
  const db = await openDb();
  const rec = { id: crypto.randomUUID(), payload: item.payload, inquiryId: item.inquiryId,
    status: 'pending', retries: 0, nextRetry: Date.now(), createdAt: Date.now() };
  await req2promise(tx(db, 'readwrite').add(rec));
  db.close();
  return rec.id;
}
export async function getPending(limit = 5) {
  const db = await openDb();
  const all = await req2promise(tx(db, 'readonly').getAll());
  db.close();
  const now = Date.now();
  return all.filter(r => r.status === 'pending' && r.nextRetry <= now)
            .sort((a, b) => a.nextRetry - b.nextRetry)
            .slice(0, limit);
}
export async function markStatus(id, status, opts = {}) {
  const db = await openDb();
  const store = tx(db, 'readwrite');
  const r = await req2promise(store.get(id));
  if (!r) { db.close(); return; }
  Object.assign(r, { status, ...opts });
  await req2promise(store.put(r));
  db.close();
}
export async function remove(id) {
  const db = await openDb();
  await req2promise(tx(db, 'readwrite').delete(id));
  db.close();
}
export async function getStats() {
  const db = await openDb();
  const all = await req2promise(tx(db, 'readonly').getAll());
  db.close();
  return all.reduce((a, r) => { a[r.status] = (a[r.status] || 0) + 1; return a; }, {});
}

crypto.randomUUID() 在 SW 和 node 18+ 都可用。fake-indexeddb 提供 indexedDB 全局。

  • Step 4: 测试通过 + commit feat(push): IndexedDB queue (add/getPending/markStatus/remove/stats)

Task 4: push/payload.js(组装后端 payload)

Files: push/payload.js, tests/payload.test.js

  • Step 1: 测试:buildIngestPayload(record, messages) 输出匹配后端 zod(inquiryId/createTime ISO/updateTime ISO/customer{customerId,country,...}/messages[{messageId,sendTime ISO,direction,content,...}]);空字符串 behavior 归 null;messages 为空时抛错(后端 zod min(1))
  • Step 2: 写 push/payload.js:
const s = v => (v == null || v === '') ? null : v;
const iso = ms => new Date(ms).toISOString();

export function buildIngestPayload(record, messages) {
  if (!messages || messages.length === 0) throw new Error('messages required');
  const iq = record || {};
  return {
    inquiry: {
      inquiryId: String(iq.inquiryId),
      manager: s(iq.manager) || undefined,
      // 优先用 meta 的 gmtCreate/gmtModified(record.createTimeFromApi),回退列表 createTime
      createTime: iq.createTimeFromApi || iq.createTime || iso(Date.now()),
      updateTime: iq.updateTimeFromApi || iq.updateTime || iq.createTimeFromApi || iso(Date.now()),
      status: s(iq.status) || undefined,
      msgPreview: s(iq.msgPreview) || undefined,
      isFlagged: !!iq.isFlagged,
    },
    customer: iq.customerId ? {
      customerId: String(iq.customerId),
      name: s(iq.detailBuyerName) || s(iq.buyerName),
      country: s(iq.detailCountry) || s(iq.countryCode) ? (s(iq.detailCountry) || s(iq.countryCode)) : null,
      email: s(iq.email), company: s(iq.company),
      registerTime: s(iq.registerTime) || null,
      avatarUrl: s(iq.detailAvatarUrl) || s(iq.avatarUrl) || undefined,
      buyerTags: s(iq.buyerTags) || undefined, buyerFeatures: s(iq.buyerFeatures) || undefined,
      behavior: iq.behavior ? {
        productViews: n(iq.behavior.productViews), validInquiries: n(iq.behavior.validInquiries),
        validRfq: n(iq.behavior.validRfq), loginDays: n(iq.behavior.loginDays),
        spamInquiries: n(iq.behavior.spamInquiries), topIndustry: s(iq.behavior.topIndustry),
      } : undefined,
    } : null,
    messages: messages.map(m => ({
      messageId: String(m.messageId), sendTime: m.sendTime,
      sender: s(m.sender) || undefined, direction: m.direction,
      content: m.original ?? m.content ?? '', translation: s(m.translation) || '',
    })),
  };
}
function n(v) { const x = Number(v); return Number.isFinite(x) ? x : null; }

注:messages 已是 apiMsgToChat 转换后的对象(messageId/sendTime ISO/sender/original/direction/translation)。content 映射用 original(apiMsgToChat 产出 original)。

  • Step 3: 测试通过(对照 Plan 1 后端 PayloadSchema 语义)+ commit feat(push): buildIngestPayload adapter

Task 5: push/worker.js(消费 + POST + 重试)

Files: push/worker.js, tests/worker.test.js

  • Step 1: 测试:mock global.fetch(成功 → markStatus sent 风格的 remove);失败 → retries++ nextRetry 退避;超过 8 次 → markStatus failed。注入 {addItem,getPending,markStatus,remove} mock 或用真实 fake-indexeddb + fetch mock
  • Step 2: 写 push/worker.js:
import { getPending, markStatus, remove } from './queue.js';
import { buildIngestPayload } from './payload.js';

const MAX_RETRIES = 8;
let _config = null;   // {serverUrl, token, enabled}

export function configure(cfg) { _config = cfg; }

export async function processQueue() {
  if (!_config || !_config.enabled || !_config.serverUrl || !_config.token) return { processed: 0 };
  const items = await getPending(3);
  let processed = 0;
  for (const it of items) {
    try {
      const payload = buildIngestPayload(it.payload.record, it.payload.messages);
      const res = await fetch(`${_config.serverUrl}/api/ingest`, {
        method: 'POST',
        headers: { 'Authorization': `Bearer ${_config.token}`, 'Content-Type': 'application/json' },
        body: JSON.stringify(payload),
      });
      if (res.ok) { await remove(it.id); processed++; }
      else { await bumpRetry(it, `HTTP ${res.status}`); }
    } catch (e) {
      await bumpRetry(it, String(e && e.message || e));
    }
  }
  return { processed };
}

async function bumpRetry(it, err) {
  const retries = it.retries + 1;
  if (retries >= MAX_RETRIES) {
    await markStatus(it.id, 'failed', { retries, lastError: err });
  } else {
    const delay = Math.min(Math.pow(2, retries) * 1000, 3600 * 1000);
    await markStatus(it.id, 'pending', { retries, nextRetry: Date.now() + delay, lastError: err });
  }
}
  • Step 3: 测试通过 + commit feat(push): worker (consume + POST + backoff)

Task 6: background.js(SW,alarms worker + 接收 enqueue)

Files: background.js

  • Step 1: 顶部改 ES module import:
import { addItem, getStats } from './push/queue.js';
import { processQueue, configure } from './push/worker.js';
  • Step 2: 保留现有 storeDetail/getBgData/clearBgData(本地导出)不动
  • Step 3: 加配置加载 + alarms 注册 + enqueue 处理:
const CONFIG_KEY = 'push_config';
const ALARM = 'push-tick';

async function loadConfig() {
  const d = await chrome.storage.local.get(CONFIG_KEY);
  return d[CONFIG_KEY] || { enabled: false, serverUrl: '', token: '' };
}

// 启动 + 配置变更时刷新 worker 配置 + alarm
async function refresh() {
  const cfg = await loadConfig();
  configure(cfg);
  if (cfg.enabled) {
    await chrome.alarms.create(ALARM, { periodInMinutes: 1 });
    processQueue();   // 立即跑一次
  } else {
    await chrome.alarms.clear(ALARM);
  }
}

chrome.runtime.onInstalled.addListener(refresh);
chrome.runtime.onStartup.addListener(refresh);
chrome.alarms.onAlarm.addListener(a => { if (a.name === ALARM) processQueue(); });

chrome.runtime.onMessage.addListener((msg, sender, sendResponse) => {
  if (msg.type === 'enqueue') {
    (async () => {
      const cfg = await loadConfig();
      if (!cfg.enabled) { sendResponse({ ok: false, reason: 'disabled' }); return; }
      const id = await addItem({ payload: { record: msg.record, messages: msg.messages }, inquiryId: msg.record?.inquiryId });
      sendResponse({ ok: true, id });
      processQueue();   // 立即尝试
    })();
    return true;
  }
  if (msg.type === 'getPushStats') {
    (async () => sendResponse(await getStats()))();
    return true;
  }
  if (msg.type === 'setPushConfig') {
    (async () => {
      await chrome.storage.local.set({ [CONFIG_KEY]: msg.config });
      await refresh();
      sendResponse({ ok: true });
    })();
    return true;
  }
});
  • Step 4: commit feat(bg): push worker + alarms + enqueue handler (keep local export)

Task 7: content.js — apiMsgToChat 扩展

Files: content.js(apiMsgToChat ~434)

  • Step 1: apiMsgToChat 改产出 { messageId, sendTime(ISO), sender, original, translation, direction }:
    • messageId: String(m.messageId)
    • sendTime: new Date(m.sendTime).toISOString()(替代原来的本地时区 "YYYY-MM-DD HH:MM:SS")
    • translation:从 m.extInfo?.icbuData?.extend 解析(extend 是 JSON 字符串,内含 extParams.inputTranslateInfo.translate);解析失败留 ''
    • sender/original/direction 保持
  • Step 2: translation 解析加双重 JSON.parse 兜底(extend 可能是字符串里的字符串),失败不抛(留 '')
  • Step 3: 手动/Playwright 验证(无 node 测试,因 content.js 非 module):用 Playwright 注入新版 apiMsgToChat + 真实 m 样本,断言 messageId/sendTime/translation 正确
  • Step 4: commit feat(content): apiMsgToChat emits messageId/sendTime(ISO)/translation

Task 8: content.js — fetchChatByApi 扩展(提取 gmtCreate/country)

Files: content.js(fetchChatByApi ~451)

  • Step 1: fetchChatByApi 在调 querySummary 后,额外提取 d.gmtCreate/d.gmtModified(毫秒→ISO)存入返回 meta;d.contact?.countryCode 或 d.cid 派生也行(spec 用 contact.countryCode)。返回 { messages, meta: { gmtCreate, gmtModified } }
  • Step 2: 调用方(processCurrentDetailPage / scrapeDetailData)把 meta 挂到 detail:detail.createTimeFromApi = meta.gmtCreate; detail.updateTimeFromApi = meta.gmtModified
  • Step 3: commit feat(content): fetchChatByApi extracts gmtCreate/gmtModified for payload

Task 9: content.js — processCurrentDetailPage 加 enqueue

Files: content.js(processCurrentDetailPage ~347,storeDetail 之后)

  • Step 1: 在 storeDetail(本地导出)之后加 enqueue(推送):
// 推送到后端(Plan 2):发 record + messages 给 SW 入队
try {
  chrome.runtime.sendMessage({
    type: 'enqueue',
    record: merged,            // {item, detail} 合并
    messages: detail.chatMessages || [],   // apiMsgToChat 产出数组
  }, () => {});  // fire-and-forget,失败不影响本地导出
} catch (e) {}

detail.chatMessages 需在 scrapeDetailData 产出(现有 chatText 是序列化串,加一个 chatMessages 数组字段)。确保 merged 含 createTimeFromApi/updateTimeFromApi(Task 8)。

  • Step 2: commit feat(content): enqueue record+messages to SW after detail scrape

Task 10: popup 配置页 + permissions.request

Files: popup.html, popup.js

  • Step 1: popup.html 加「后端设置」区:服务器 URL 输入、Token 输入、启用推送 checkbox、「授权域名」按钮、「保存」按钮
  • Step 2: popup.js:加载时 chrome.runtime.sendMessage({type:'getPushConfig'}) 回填(或直接 chrome.storage.local.get);保存时 setPushConfig
  • Step 3: 「授权域名」按钮 → chrome.permissions.request({origins:[serverUrl + '/*']})(用户手势触发);成功后提示
  • Step 4: commit feat(popup): backend config UI + runtime origin permission

Task 11: popup 推送状态显示

Files: popup.html, popup.js

  • Step 1: popup 加状态条:「已推 N / 待推 M / 失败 K」
  • Step 2: popup 打开时 + 每 2s chrome.runtime.sendMessage({type:'getPushStats'}) 刷新
  • Step 3: 失败 K > 0 时红字 + 可点击查看 lastError(简化:弹 alert 或展开列表)
  • Step 4: commit feat(popup): push stats display (pending/failed/sent)

Task 12: 端到端验证

Files: scripts/e2e-plugin.md(验证步骤文档)

  • Step 1: 起 Plan 1 后端:cd d:/alibb-server/backend && docker compose up -d --build,等 health ok
  • Step 2: Chrome chrome://extensions 开发者模式 → 加载 d:\alibb(本 plan 改后的插件)
  • Step 3: 点插件图标 → 后端设置填 http://localhost:8080 + token dev-ingest-token-1234 + 启用 → 授权域名 → 保存
  • Step 4: 打开 message.alibaba.com → 抓一个询盘(用插件现有抓取流程,或 Playwright 注入触发)
  • Step 5: 验证:curl -H "Authorization: Bearer <jwt>" http://localhost:8080/api/inquiries 能看到刚抓的询盘;docker compose exec mysql mysql -uroot -proot inquiry -e "SELECT inquiry_id, messages_count FROM inquiries" 数据在
  • Step 6: popup 状态条显示「已推 1」
  • Step 7: 验证幂等:再抓同一询盘 → 后端 messages_count 不变(早返回)
  • Step 8: commit test: e2e plugin→backend validation doc

完成标准(DoD)

  • manifest:type:module + alarms/unlimitedStorage + optional_host_permissions
  • push/queue.js / payload.js / worker.js 三个 module + vitest 测试全绿
  • background.js:alarms(1 分钟)消费队列,POST /api/ingest,失败指数退避(≤8 次);保留本地 storeDetail/getBgData
  • content.js:apiMsgToChat 产 messageId/sendTime(ISO)/translation;fetchChatByApi 提取 gmtCreate/gmtModified;processCurrentDetailPage enqueue
  • popup:后端设置(URL/token/启用)+ permissions.request 授权 + 状态显示(已推/待推/失败)
  • 端到端:抓一条询盘 → Plan 1 后端能看到数据;再抓同一条幂等(messages_count 不变)
  • 断网/后端停 → 数据进队列不丢;后端恢复 → 自动推完

后续(本 plan 不含)

  • Plan 3:管理后台前端(查看/筛选/导出/统计)
  • Plan 4:正式部署(域名 + HTTPS + 备份定时 + 监控)
  • 增量推送(插件记录 last sendTime,只推新增消息)
  • 失败项重试 UI(popup 查看失败 + 手动重推)