test(evidence): 阶段2真实WXR冒烟通过
真实 hesupipe WXR(5,490,399B,SHA-256 与 Plan 1 基线一致)跑 wxr:probe 全链: 计数 18 项对照全过(rows 1047 / terms 451 / unsupported 75 细分一致 / failed 0 / item 总数 1122);heapUsed max 62.2MiB < 100MiB(setInterval(100ms) 双 max 口径, 全程 167ms 仅 2 样本如实注明);parseMs 含流 IO 等待口径注明。 nicename 锚点核对:在表 taxonomy 706/706 全命中(nicename≡slug,违例 0); category/post_tag 共 807 引用因 wp:term 表缺该 taxonomy 不可解析——阶段 3 需按 (domain, nicename) 自建缺失 term,不能假设 term 表完备。 eslint.config.js:scripts/*.mjs 显式 node 全局(no-undef 误报)。 Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,143 @@
|
|||||||
|
# 阶段 2 真实 WXR 冒烟证据(Plan 2 · Task 8)
|
||||||
|
|
||||||
|
**verdict:PASS**(计数对照 18 项全过 + 内存断言通过 + nicename 锚点核对完成)
|
||||||
|
|
||||||
|
| 项 | 值 |
|
||||||
|
|---|---|
|
||||||
|
| 日期 | 2026-10-21 |
|
||||||
|
| 环境 | Windows 11 · Node v24.13.0 · pnpm 10.7.1 · tsx 直吃 TS 源码(与 sf CLI 同机制) |
|
||||||
|
| 输入 | `D:/project/cms/storage/wxr/fd04d6ba8a1316b6.xml`(5,490,399 B ≈ 5.5MB,绝对只读) |
|
||||||
|
| 输入完整性 | SHA-256 `f9bc5d4497a1792af37abff78e0096268bda975cb367a956da42bcf9552b9f62` 与 Plan 1 基线一致——文件自 Task 0.4 起未变 |
|
||||||
|
| 冒烟脚本 | `scripts/smoke-real-wxr.mjs`(probe 全链 + setInterval(100ms) 内存采样 + 基线断言 + nicename 核对) |
|
||||||
|
| 证据 run | `artifacts/probe/20261021T132153-1yft49/`(smoke.json / report.json / report.html) |
|
||||||
|
| CLI run | `artifacts/probe/20261021T132302-26azne/`(report.json / report.html) |
|
||||||
|
|
||||||
|
## 实际执行的命令原文
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 冒烟主路径(仓库根 D:\project\cms-one):probe 同链路 + 内存采样 + 基线断言 + nicename 核对,stdout 出 JSON
|
||||||
|
pnpm exec tsx scripts/smoke-real-wxr.mjs # 缺省 --wxr 即 hesupipe 文件;exit 0 = PASS
|
||||||
|
|
||||||
|
# 计划钉死的 CLI 命令(Step 1 原文,只读诊断通道直读 cms)——计数与冒烟脚本一致
|
||||||
|
pnpm sf wxr:probe --wxr "D:/project/cms/storage/wxr/fd04d6ba8a1316b6.xml"
|
||||||
|
```
|
||||||
|
|
||||||
|
注:smoke 脚本为 .mjs,但 `@sf/adapter-wp` 经根 tsconfig paths 直连 packages 的 TS 源码,故须 tsx 运行(纯 node 无法解析 workspace TS 包);与 `scripts/sf.ts` 的 CLI 执行机制(Plan 2 评审钉死)相同。
|
||||||
|
|
||||||
|
## Step 1 计数对照(实际 vs 基线,逐项)
|
||||||
|
|
||||||
|
基线:`docs/evidence/real-wxr-baseline.md`。断言规则:**任何一项不一致都算 FAIL**。以下 18 项全部 `ok`(smoke.json `checks` 数组原样可查):
|
||||||
|
|
||||||
|
| 对照项 | 基线 | 实际 | 结果 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 输入字节数 | 5,490,399 | 5,490,399 | ok |
|
||||||
|
| 输入 SHA-256 | f9bc5d44…b9f62 | f9bc5d44…b9f62 | ok |
|
||||||
|
| totals.rows | 1,047(855+129+48+15) | 1,047 | ok |
|
||||||
|
| totals.terms | 451 | 451 | ok |
|
||||||
|
| totals.unsupported | 75 | 75 | ok |
|
||||||
|
| totals.failed | 0 | 0 | ok |
|
||||||
|
| item 总数(rows+unsupported+failed) | 1,122 | 1,122 | ok |
|
||||||
|
| rows `wp:attachment` | 855 | 855 | ok |
|
||||||
|
| rows `wp:post` | 129 | 129 | ok |
|
||||||
|
| rows `wp:product` | 48 | 48 | ok |
|
||||||
|
| rows `wp:page` | 15 | 15 | ok |
|
||||||
|
| unsupported `wp:nav_menu_item` | 68 | 68 | ok |
|
||||||
|
| unsupported `wp:wpcf7_contact_form` | 3 | 3 | ok |
|
||||||
|
| unsupported `wp:elementor_library` | 2 | 2 | ok |
|
||||||
|
| unsupported `wp:wp_navigation` | 2 | 2 | ok |
|
||||||
|
| counts 桶键集合(无多余/缺失桶) | 上述 8 键 | 完全一致 | ok |
|
||||||
|
| probe.fatalError | null | null | ok |
|
||||||
|
| 内存 heapUsed max | < 100MB | 65,268,760 B(62.2MiB) | ok |
|
||||||
|
|
||||||
|
计数细分(created=清洗有变化将产新内容 / skipped=无变化,导入前投影语义):
|
||||||
|
|
||||||
|
| sourceType | created | skipped | unsupported | 基线口径 |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| wp:attachment | 0 | 855 | 0 | 855 rows |
|
||||||
|
| wp:post | 87 | 42 | 0 | 129 rows |
|
||||||
|
| wp:product | 47 | 1 | 0 | 48 rows |
|
||||||
|
| wp:page | 4 | 11 | 0 | 15 rows |
|
||||||
|
| wp:nav_menu_item | – | – | 68 | 68 unsupported |
|
||||||
|
| wp:wpcf7_contact_form | – | – | 3 | 3 unsupported |
|
||||||
|
| wp:elementor_library | – | – | 2 | 2 unsupported |
|
||||||
|
| wp:wp_navigation | – | – | 2 | 2 unsupported |
|
||||||
|
|
||||||
|
清洗统计(背景数据,非对照项):changedRows=138(post 87 + product 47 + page 4;attachment 无正文零清洗全 skipped);removed-tag=3、removed-attr=4,667;rowErrors 固化 4,793 条(上限 10,000,未触顶 truncated=0)。
|
||||||
|
|
||||||
|
## Step 2 内存与耗时
|
||||||
|
|
||||||
|
**内存(计划钉死口径:smoke 脚本内 `setInterval(100ms)` 采 `process.memoryUsage()` 的 heapUsed/rss 双 max)**
|
||||||
|
|
||||||
|
| 指标 | 值 |
|
||||||
|
|---|---|
|
||||||
|
| heapUsed max | **65,268,760 B = 62.2MiB < 100MiB(断言通过)** |
|
||||||
|
| rss max | 175,218,688 B = 167.1MiB(进程整体常驻,含 node+tsx 运行时基线,仅记录不断言) |
|
||||||
|
| 采样样本数 | 2(见下方口径说明) |
|
||||||
|
|
||||||
|
口径说明(如实记录):
|
||||||
|
|
||||||
|
- 采样的是**整个进程**的 heapUsed/rss——含 tsx 加载器与模块基线,非解析增量;断言语义是"流式解析 5.5MB 文件时进程堆峰值仍 < 100MB",通过即证明无整文件缓冲(整读 5.5MB 原文 + 解析树必然显著更高,且会随文件增长线性突破)。
|
||||||
|
- probe 全程仅 ~167ms(见下),100ms 间隔只落到 2 个样本(1 个 interval tick + 终末补采 1 次)。max 以**实测下界**口径记录;考虑解析器 per-item yield、不 buffer item 列表的结构保证(Task 3 Step 2 钉死),两样本已覆盖解析爬坡段与收尾段,实测 62.2MiB 与 100MiB 线余量充足。
|
||||||
|
|
||||||
|
**耗时(smoke run / CLI run 各一次,量级一致)**
|
||||||
|
|
||||||
|
| 指标 | smoke run(1yft49) | CLI run(26azne) |
|
||||||
|
|---|---|---|
|
||||||
|
| parseMs | 73.6ms | 79.2ms |
|
||||||
|
| sanitizeMs | 81.7ms | 79.2ms |
|
||||||
|
| totalMs | 166.9ms | 169.5ms |
|
||||||
|
|
||||||
|
**parseMs 口径(Task 6 移交,务必连同数字一起引用)**:parseMs 是 `parser.next()` await 窗口累计——**含流式 IO 等待**(磁盘读、backpressure),非纯 CPU 解析耗时;totalMs 覆盖整个消费循环(含映射与报告聚合);totalMs ≥ parseMs+sanitizeMs 的差值即映射/聚合时间。
|
||||||
|
|
||||||
|
## nicename 锚点核对(Task 4 冻结点)
|
||||||
|
|
||||||
|
**核对方法**:第二遍独立流式解析(同一 parseWxrStream),全量收集 wp:term 表(taxonomy → slug 集)与 item 引用(domain → nicename 计数),流结束后交叉验证;抽检规则=文档序前 10 个含 `domain="category"` 引用的 item(该类 item 共 129 个)。
|
||||||
|
|
||||||
|
**抽检 10 条明细**(全部未解析到 term slug,原因见下):
|
||||||
|
|
||||||
|
| rawLocation | postId | post_type | category nicename(value) | ∈ wp:term slug 集 |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| wxr://item/941 | 1646 | post | company-news(Company News) | 否 |
|
||||||
|
| wxr://item/942 | 2281 | post | trade-news(Trade News) | 否 |
|
||||||
|
| wxr://item/943 | 2549 | post | progect(Progects) | 否 |
|
||||||
|
| wxr://item/944 | 2833 | post | progect(Progects) | 否 |
|
||||||
|
| wxr://item/945 | 3121 | post | progect(Progects) | 否 |
|
||||||
|
| wxr://item/946 | 3124 | post | progect(Progects) | 否 |
|
||||||
|
| wxr://item/947 | 3127 | post | progect(Progects) | 否 |
|
||||||
|
| wxr://item/948 | 3130 | post | progect(Progects) | 否 |
|
||||||
|
| wxr://item/949 | 3138 | post | progect(Progects) | 否 |
|
||||||
|
| wxr://item/950 | 3143 | post | progect(Progects) | 否 |
|
||||||
|
|
||||||
|
**抽检一致率(category nicename 可解析到 wp:term slug):0/10。根因不是 nicename≠slug,而是该 WXR 的 wp:term 表根本没有 category/post_tag 条目**。全量分层核对(grep 级独立复核结论一致):
|
||||||
|
|
||||||
|
| domain(item 引用数) | 可解析 | wp:term 表中有该 taxonomy? | 结论 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| product_tag(588) | 588/588 = 100% | 有(439 term) | nicename ≡ term_slug |
|
||||||
|
| product_cat(49) | 49/49 = 100% | 有(10 term) | nicename ≡ term_slug |
|
||||||
|
| nav_menu(68) | 68/68 = 100% | 有(1 term:mainmenu) | nicename ≡ term_slug |
|
||||||
|
| elementor_library_type(1) | 1/1 = 100% | 有(1 term) | nicename ≡ term_slug |
|
||||||
|
| category(148 引用,5 个不同 nicename:trade-news 59 / progect 29 / news 28 / company-news 20 / notification 12) | 0/148 | **无(0 term)** | taxonomy 未导出,锚点无从对拍 |
|
||||||
|
| post_tag(659 引用,471 个不同 nicename) | 0/659 | **无(0 term)** | taxonomy 未导出,锚点无从对拍 |
|
||||||
|
|
||||||
|
汇总:**wp:term 表内 taxonomy 的全部 706 条引用 100% 命中 slug,nicename≠slug 违例 0 条**;category/post_tag 共 807 条引用因 term 表缺该 taxonomy 不可解析。(wp:term 总数 451 = product_tag 439 + product_cat 10 + nav_menu 1 + elementor_library_type 1,与 totals.terms 一致。)
|
||||||
|
|
||||||
|
**对阶段 3 taxonomyRefs 映射策略的结论**:
|
||||||
|
|
||||||
|
1. Task 4 的锚点设计(`sourceTermId` = item 内 @nicename)被实数据**正向验证**:凡 term 表覆盖的 taxonomy,nicename 与 term_slug 完全等同(706/706),"真实 WP 中 category @nicename 即 term slug"的假设在本文件所有可对拍域上成立;
|
||||||
|
2. 但**不能假设 wp:term 表完备**:本 WXR 未导出 category/post_tag 的 term 条目(WordPress 7.1.3 导出行为,含 nav_menu/elementor/product 域却缺这两个常规域)。阶段 3 做 slug → term_id 解析时,category(148 引用/5 nicename)与 post_tag(659 引用/471 nicename)将全部 miss——需按 (domain, nicename) 自建缺失 term(value 可作 name),不能只依赖 wp:term 事件;
|
||||||
|
3. 数据怪癖原样保留:`progect`(源站拼写如此,29 处引用)按原文迁移不修正。
|
||||||
|
|
||||||
|
## 报告产物
|
||||||
|
|
||||||
|
| 产物 | 路径 |
|
||||||
|
|---|---|
|
||||||
|
| 冒烟结果(断言/内存/nicename 全量数据,JSON) | `artifacts/probe/20261021T132153-1yft49/smoke.json` |
|
||||||
|
| probe 报告 JSON(smoke run) | `artifacts/probe/20261021T132153-1yft49/report.json` |
|
||||||
|
| probe 报告 HTML(smoke run) | `artifacts/probe/20261021T132153-1yft49/report.html` |
|
||||||
|
| probe 报告 JSON/HTML(CLI run) | `artifacts/probe/20261021T132302-26azne/report.{json,html}` |
|
||||||
|
|
||||||
|
(artifacts/ 不入库(.gitignore),产物仅存本地;smoke.json 的 `checks` 数组即上表 18 项断言的机器可读原貌。)
|
||||||
|
|
||||||
|
## 只读声明
|
||||||
|
|
||||||
|
本次冒烟对 `D:/project/cms/storage/wxr/fd04d6ba8a1316b6.xml` 仅执行读取操作(stat、sha256 流式读、两遍 createReadStream 解析读),未修改该文件及 `D:\project\cms` 下任何内容;全部写入仅发生在本仓库 `artifacts/probe/` 与上述两个新建文件。
|
||||||
@@ -26,4 +26,22 @@ export default tseslint.config(
|
|||||||
'@typescript-eslint/no-explicit-any': 'off',
|
'@typescript-eslint/no-explicit-any': 'off',
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
// Node 直跑的 .mjs 工具脚本(如 scripts/smoke-real-wxr.mjs):no-undef 对 TS 文件
|
||||||
|
// 由 TS 接管而关闭,但 .mjs 仍生效且无 node 全局——此处显式声明 node 全局,
|
||||||
|
// 否则 process/setInterval 等误报 no-undef。
|
||||||
|
files: ['scripts/**/*.mjs'],
|
||||||
|
languageOptions: {
|
||||||
|
globals: {
|
||||||
|
process: 'readonly',
|
||||||
|
setInterval: 'readonly',
|
||||||
|
clearInterval: 'readonly',
|
||||||
|
setTimeout: 'readonly',
|
||||||
|
clearTimeout: 'readonly',
|
||||||
|
console: 'readonly',
|
||||||
|
performance: 'readonly',
|
||||||
|
Buffer: 'readonly',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -0,0 +1,357 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
// 真实 WXR 冒烟(Plan 2 · Task 8,阶段 2 验收):
|
||||||
|
// 对 hesupipe 真实导出(只读)跑 wxr:probe 同链路(runWxrAdapterReport),计数对照
|
||||||
|
// Plan 1 基线 docs/evidence/real-wxr-baseline.md,内存按计划钉死口径采样——
|
||||||
|
// setInterval(100ms) 采 process.memoryUsage() 的 heapUsed/rss 双 max。
|
||||||
|
//
|
||||||
|
// 运行方式(仓库根):
|
||||||
|
// pnpm exec tsx scripts/smoke-real-wxr.mjs [--wxr <path>] [--out <dir>]
|
||||||
|
// 与 scripts/sf.ts 同机制:@sf/adapter-wp 经根 tsconfig paths 直连 TS 源码(需 tsx,
|
||||||
|
// 纯 node 无法解析 workspace TS 包——证据文档须记实际执行命令原文)。
|
||||||
|
//
|
||||||
|
// 断言语义(任一不一致 → verdict FAIL + exitCode 1,如实报告不凑解释):
|
||||||
|
// - totals 与基线逐项一致(rows/terms/unsupported/failed 与 item 总数);
|
||||||
|
// - unsupported 按 sourceType(`wp:<post_type>`)细分计数一致;
|
||||||
|
// - heapUsed max < 100MB(流式验收线);fatalError 必须为 null。
|
||||||
|
// nicename 锚点核对(Task 4 冻结点)为证据采集而非 pass/fail 断言:不一致逐条列出,
|
||||||
|
// 供阶段 3 taxonomyRefs 映射策略决策。
|
||||||
|
//
|
||||||
|
// 计时口径(Task 6 移交,报告须注明):parseMs 是 parser.next() await 窗口累计,
|
||||||
|
// 含流式 IO 等待(磁盘读/backpressure),非纯 CPU 解析耗时;totalMs 覆盖整个消费循环。
|
||||||
|
//
|
||||||
|
// 输出:stdout 纯 JSON(机器可消费);人类可读进度走 stderr。产物落
|
||||||
|
// artifacts/probe/<runId>/{report.json,report.html,smoke.json}(与 wxr:probe CLI 同布局)。
|
||||||
|
|
||||||
|
import { createHash } from 'node:crypto'
|
||||||
|
import { createReadStream } from 'node:fs'
|
||||||
|
import { mkdir, stat, writeFile } from 'node:fs/promises'
|
||||||
|
import { dirname, join, resolve } from 'node:path'
|
||||||
|
import { fileURLToPath } from 'node:url'
|
||||||
|
import { parseWxrStream, renderReportHtml, runWxrAdapterReport } from '@sf/adapter-wp'
|
||||||
|
import { makeRunId } from '../apps/api/src/modules/source/register.service.js'
|
||||||
|
|
||||||
|
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..')
|
||||||
|
|
||||||
|
/** 冒烟对象(缺省):hesupipe 真实导出,绝对只读 */
|
||||||
|
const DEFAULT_WXR = 'D:/project/cms/storage/wxr/fd04d6ba8a1316b6.xml'
|
||||||
|
|
||||||
|
/** 流式验收线:heapUsed max < 100MB(计划钉死) */
|
||||||
|
const HEAP_LIMIT_BYTES = 100 * 1024 * 1024
|
||||||
|
|
||||||
|
/** 内存采样周期(计划钉死口径:setInterval(100ms) 双 max) */
|
||||||
|
const SAMPLE_INTERVAL_MS = 100
|
||||||
|
|
||||||
|
/** nicename 锚点抽检条数:文档序前 N 个含 domain="category" 引用的 item */
|
||||||
|
const NICENAME_SAMPLE_ITEMS = 10
|
||||||
|
|
||||||
|
/** Plan 1 基线(docs/evidence/real-wxr-baseline.md,对照用冻结值) */
|
||||||
|
const BASELINE = {
|
||||||
|
fileBytes: 5_490_399,
|
||||||
|
sha256: 'f9bc5d4497a1792af37abff78e0096268bda975cb367a956da42bcf9552b9f62',
|
||||||
|
itemsTotal: 1122,
|
||||||
|
rowsTotal: 1047, // 855 attachment + 129 post + 48 product + 15 page
|
||||||
|
terms: 451,
|
||||||
|
unsupportedTotal: 75,
|
||||||
|
failedTotal: 0,
|
||||||
|
/** 产 row 的四类(created+skipped+failed 之和对照) */
|
||||||
|
rowsByType: { 'wp:attachment': 855, 'wp:post': 129, 'wp:product': 48, 'wp:page': 15 },
|
||||||
|
/** unsupported 细分(`wp:<post_type>` 归桶) */
|
||||||
|
unsupportedByType: {
|
||||||
|
'wp:nav_menu_item': 68,
|
||||||
|
'wp:wpcf7_contact_form': 3,
|
||||||
|
'wp:elementor_library': 2,
|
||||||
|
'wp:wp_navigation': 2,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 期望出现的 counts 键全集 = rowsByType ∪ unsupportedByType(多一个少一个都算不一致) */
|
||||||
|
const EXPECTED_COUNTS_KEYS = [...Object.keys(BASELINE.rowsByType), ...Object.keys(BASELINE.unsupportedByType)].sort()
|
||||||
|
|
||||||
|
/** 断言登记(顺序即证据表行序) */
|
||||||
|
function makeChecks() {
|
||||||
|
const checks = []
|
||||||
|
const check = (name, expected, actual, pass) => checks.push({ name, expected, actual, pass })
|
||||||
|
const checkEq = (name, expected, actual) => check(name, expected, actual, Object.is(expected, actual))
|
||||||
|
return { checks, check, checkEq }
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 流式 sha256(64KB 块,与 register 服务的哈希口径同型) */
|
||||||
|
async function sha256File(path) {
|
||||||
|
const hash = createHash('sha256')
|
||||||
|
for await (const chunk of createReadStream(path, { highWaterMark: 64 * 1024 })) {
|
||||||
|
hash.update(chunk)
|
||||||
|
}
|
||||||
|
return hash.digest('hex')
|
||||||
|
}
|
||||||
|
|
||||||
|
/** --wxr <path> / --wxr=<path> / --out <dir> / --out=<dir>(缺省 DEFAULT_WXR 与 artifacts/probe) */
|
||||||
|
function parseArgs(argv) {
|
||||||
|
const out = { wxr: DEFAULT_WXR, outDir: join(REPO_ROOT, 'artifacts', 'probe') }
|
||||||
|
for (let i = 0; i < argv.length; i++) {
|
||||||
|
const arg = argv[i]
|
||||||
|
if (arg === '--wxr' || arg === '--out') {
|
||||||
|
const value = argv[++i]
|
||||||
|
if (value === undefined) throw new Error(`${arg} 缺少值参数`)
|
||||||
|
if (arg === '--wxr') out.wxr = value
|
||||||
|
else out.outDir = value
|
||||||
|
} else if (arg !== undefined && arg.startsWith('--wxr=')) out.wxr = arg.slice(6)
|
||||||
|
else if (arg !== undefined && arg.startsWith('--out=')) out.outDir = arg.slice(6)
|
||||||
|
else throw new Error(`未知参数:${arg}(支持 --wxr <path> 与 --out <dir>)`)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 单域 unresolved nicename 明细固化上限(防 post_tag 这类大域使 smoke.json 失去上界) */
|
||||||
|
const UNRESOLVED_LIST_CAP = 50
|
||||||
|
|
||||||
|
/**
|
||||||
|
* nicename 锚点核对(Task 4 冻结点:sourceTermId 以 item 内 @nicename 为锚)。
|
||||||
|
* 单遍流式(与 probe 独立的第二遍读,采样器已停):全量收集 wp:term 的
|
||||||
|
* (taxonomy → Set(slug)) 与 item 引用 (domain → Map(nicename → 计数))、
|
||||||
|
* 文档序前 10 个含 category 域引用的 item 明细——流结束后统一比对
|
||||||
|
* (不依赖 wp:term 与 item 的文档先后顺序)。
|
||||||
|
*
|
||||||
|
* 证据分两层(阶段 3 taxonomyRefs 映射策略的决策依据):
|
||||||
|
* - 任务要求的 10 条抽检:item 的 category 域 nicename ∈ wp:term(category) 的 slug 集;
|
||||||
|
* - 全量分层核对(每个 domain):区分「taxonomy 未出现在 wp:term 表」(导出缺该
|
||||||
|
* taxonomy,锚点无从对拍)与「taxonomy 在表但 nicename 无对应 slug」(真正的
|
||||||
|
* nicename≠slug 不一致)。前者是本 WXR 的实际形态(category/post_tag 未导出
|
||||||
|
* term 表),如实记录而非记为锚点失败。
|
||||||
|
*/
|
||||||
|
async function nicenameAudit(wxrPath) {
|
||||||
|
const termSlugsByTaxonomy = new Map()
|
||||||
|
const domainCounts = new Map()
|
||||||
|
const nicenameCountsByDomain = new Map()
|
||||||
|
const sampledItems = []
|
||||||
|
let itemsSeen = 0
|
||||||
|
let itemsWithCategoryRef = 0
|
||||||
|
|
||||||
|
for await (const ev of parseWxrStream(createReadStream(wxrPath))) {
|
||||||
|
if (ev.kind === 'term') {
|
||||||
|
const { taxonomy, slug } = ev.term
|
||||||
|
let set = termSlugsByTaxonomy.get(taxonomy)
|
||||||
|
if (set === undefined) {
|
||||||
|
set = new Set()
|
||||||
|
termSlugsByTaxonomy.set(taxonomy, set)
|
||||||
|
}
|
||||||
|
set.add(slug)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
itemsSeen++
|
||||||
|
const categoryRefs = ev.item.categories.filter((c) => c.domain === 'category')
|
||||||
|
for (const ref of ev.item.categories) {
|
||||||
|
domainCounts.set(ref.domain, (domainCounts.get(ref.domain) ?? 0) + 1)
|
||||||
|
let byNicename = nicenameCountsByDomain.get(ref.domain)
|
||||||
|
if (byNicename === undefined) {
|
||||||
|
byNicename = new Map()
|
||||||
|
nicenameCountsByDomain.set(ref.domain, byNicename)
|
||||||
|
}
|
||||||
|
byNicename.set(ref.nicename, (byNicename.get(ref.nicename) ?? 0) + 1)
|
||||||
|
}
|
||||||
|
if (categoryRefs.length === 0) continue
|
||||||
|
itemsWithCategoryRef++
|
||||||
|
if (sampledItems.length < NICENAME_SAMPLE_ITEMS) {
|
||||||
|
sampledItems.push({
|
||||||
|
rawLocation: ev.item.rawLocation,
|
||||||
|
postId: ev.item.postId,
|
||||||
|
postType: ev.item.postType,
|
||||||
|
refs: categoryRefs.map((ref) => ({ nicename: ref.nicename, value: ref.value })),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const categorySlugs = termSlugsByTaxonomy.get('category') ?? new Set()
|
||||||
|
|
||||||
|
// 抽检 10 条:明细级比对(nicename ∈ category 域 wp:term 的 term_slug 集)
|
||||||
|
let sampleMatched = 0
|
||||||
|
let sampleTotal = 0
|
||||||
|
const sampleMismatches = []
|
||||||
|
for (const item of sampledItems) {
|
||||||
|
for (const ref of item.refs) {
|
||||||
|
sampleTotal++
|
||||||
|
const resolved = categorySlugs.has(ref.nicename)
|
||||||
|
if (resolved) sampleMatched++
|
||||||
|
else sampleMismatches.push({ rawLocation: item.rawLocation, postId: item.postId, ...ref })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 全量分层核对(每个 domain 一行):taxonomy 在表与否 × nicename 可解析与否
|
||||||
|
const perDomain = [...nicenameCountsByDomain.entries()]
|
||||||
|
.sort(([a], [b]) => a.localeCompare(b))
|
||||||
|
.map(([domain, byNicename]) => {
|
||||||
|
const slugs = termSlugsByTaxonomy.get(domain)
|
||||||
|
let refs = 0
|
||||||
|
let matched = 0
|
||||||
|
let distinctUnresolved = 0
|
||||||
|
const unresolved = []
|
||||||
|
for (const [nicename, count] of byNicename) {
|
||||||
|
refs += count
|
||||||
|
if (slugs !== undefined && slugs.has(nicename)) {
|
||||||
|
matched += count
|
||||||
|
} else {
|
||||||
|
distinctUnresolved++
|
||||||
|
if (unresolved.length < UNRESOLVED_LIST_CAP) unresolved.push({ nicename, refCount: count })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
domain,
|
||||||
|
refs,
|
||||||
|
matched,
|
||||||
|
rate: refs === 0 ? null : matched / refs,
|
||||||
|
taxonomyInTermTable: slugs !== undefined,
|
||||||
|
distinctUnresolved,
|
||||||
|
unresolvedCapped: unresolved.length < distinctUnresolved,
|
||||||
|
unresolved,
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
const conclusion = {
|
||||||
|
/** taxonomy 在 wp:term 表中的域的引用数(这些引用可严格对拍 nicename vs slug) */
|
||||||
|
refsAgainstExistingTaxonomy: 0,
|
||||||
|
matchedAgainstExistingTaxonomy: 0,
|
||||||
|
/** taxonomy 不在 wp:term 表的域的引用数(锚点无从对拍——导出缺表,非锚点不一致) */
|
||||||
|
refsOfAbsentTaxonomy: 0,
|
||||||
|
/** taxonomy 在表但 nicename 无对应 slug 的引用数(真正的 nicename≠slug 违例) */
|
||||||
|
nicenameEqualsSlugViolations: 0,
|
||||||
|
}
|
||||||
|
for (const row of perDomain) {
|
||||||
|
if (row.taxonomyInTermTable) {
|
||||||
|
conclusion.refsAgainstExistingTaxonomy += row.refs
|
||||||
|
conclusion.matchedAgainstExistingTaxonomy += row.matched
|
||||||
|
conclusion.nicenameEqualsSlugViolations += row.refs - row.matched
|
||||||
|
} else {
|
||||||
|
conclusion.refsOfAbsentTaxonomy += row.refs
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
sampleRule: `文档序前 ${NICENAME_SAMPLE_ITEMS} 个含 domain="category" 引用的 item(共 ${itemsWithCategoryRef} 个 item 含 category 域引用,items 总数 ${itemsSeen})`,
|
||||||
|
sampledItems: sampledItems.map((item) => ({
|
||||||
|
...item,
|
||||||
|
refs: item.refs.map((ref) => ({ ...ref, resolvedToTermSlug: categorySlugs.has(ref.nicename) })),
|
||||||
|
})),
|
||||||
|
sampleMatched,
|
||||||
|
sampleTotal,
|
||||||
|
sampleRate: sampleTotal === 0 ? null : sampleMatched / sampleTotal,
|
||||||
|
sampleMismatches,
|
||||||
|
perDomain,
|
||||||
|
conclusion,
|
||||||
|
domainCounts: Object.fromEntries([...domainCounts.entries()].sort(([a], [b]) => a.localeCompare(b))),
|
||||||
|
termTaxonomies: Object.fromEntries(
|
||||||
|
[...termSlugsByTaxonomy.entries()]
|
||||||
|
.sort(([a], [b]) => a.localeCompare(b))
|
||||||
|
.map(([taxonomy, set]) => [taxonomy, set.size]),
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const args = parseArgs(process.argv.slice(2))
|
||||||
|
const wxrPath = resolve(args.wxr)
|
||||||
|
const { checks, check, checkEq } = makeChecks()
|
||||||
|
|
||||||
|
process.stderr.write(`[smoke] 输入:${wxrPath}\n`)
|
||||||
|
const fileStat = await stat(wxrPath)
|
||||||
|
const sha256 = await sha256File(wxrPath)
|
||||||
|
checkEq('input.bytes', BASELINE.fileBytes, fileStat.size)
|
||||||
|
checkEq('input.sha256', BASELINE.sha256, sha256)
|
||||||
|
|
||||||
|
// —— probe + 内存采样(计划钉死口径)——
|
||||||
|
process.stderr.write('[smoke] 运行 probe(parse→map→sanitize 全链)+ setInterval(100ms) 内存采样…\n')
|
||||||
|
let maxHeapUsed = 0
|
||||||
|
let maxRss = 0
|
||||||
|
let samples = 0
|
||||||
|
const sample = () => {
|
||||||
|
const mu = process.memoryUsage()
|
||||||
|
if (mu.heapUsed > maxHeapUsed) maxHeapUsed = mu.heapUsed
|
||||||
|
if (mu.rss > maxRss) maxRss = mu.rss
|
||||||
|
samples++
|
||||||
|
}
|
||||||
|
const sampler = setInterval(sample, SAMPLE_INTERVAL_MS)
|
||||||
|
const report = await runWxrAdapterReport(createReadStream(wxrPath))
|
||||||
|
clearInterval(sampler)
|
||||||
|
sample() // 终末补采一次,覆盖收尾段(报告对象组装等)
|
||||||
|
|
||||||
|
// —— 计数对照(基线逐项)——
|
||||||
|
const t = report.totals
|
||||||
|
checkEq('totals.rows', BASELINE.rowsTotal, t.rows)
|
||||||
|
checkEq('totals.terms', BASELINE.terms, t.terms)
|
||||||
|
checkEq('totals.unsupported', BASELINE.unsupportedTotal, t.unsupported)
|
||||||
|
checkEq('totals.failed', BASELINE.failedTotal, t.failed)
|
||||||
|
checkEq('items.total(rows+unsupported+failed)', BASELINE.itemsTotal, t.rows + t.unsupported + t.failed)
|
||||||
|
|
||||||
|
for (const [type, expected] of Object.entries(BASELINE.rowsByType)) {
|
||||||
|
const c = report.counts[type]
|
||||||
|
checkEq(`rows.${type}(created+skipped+failed)`, expected, c === undefined ? 0 : c.created + c.skipped + c.failed)
|
||||||
|
}
|
||||||
|
for (const [type, expected] of Object.entries(BASELINE.unsupportedByType)) {
|
||||||
|
const c = report.counts[type]
|
||||||
|
checkEq(`unsupported.${type}`, expected, c === undefined ? 0 : c.unsupported)
|
||||||
|
}
|
||||||
|
checkEq(
|
||||||
|
'counts.keys(无多余/缺失桶)',
|
||||||
|
EXPECTED_COUNTS_KEYS.join(','),
|
||||||
|
Object.keys(report.counts).sort().join(','),
|
||||||
|
)
|
||||||
|
checkEq('probe.fatalError', null, report.fatalError)
|
||||||
|
check('memory.heapUsedMax<100MB', `< ${HEAP_LIMIT_BYTES}`, maxHeapUsed, maxHeapUsed < HEAP_LIMIT_BYTES)
|
||||||
|
|
||||||
|
// —— nicename 锚点核对(证据采集,非 pass/fail 断言)——
|
||||||
|
process.stderr.write('[smoke] nicename 锚点核对(第二遍流式)…\n')
|
||||||
|
const nicename = await nicenameAudit(wxrPath)
|
||||||
|
|
||||||
|
const verdict = checks.every((c) => c.pass) ? 'PASS' : 'FAIL'
|
||||||
|
|
||||||
|
// —— 产物(与 wxr:probe CLI 同布局 + smoke.json)——
|
||||||
|
const runId = makeRunId()
|
||||||
|
const dir = join(resolve(args.outDir), runId)
|
||||||
|
await mkdir(dir, { recursive: true })
|
||||||
|
await writeFile(join(dir, 'report.json'), `${JSON.stringify(report, null, 2)}\n`, 'utf8')
|
||||||
|
await writeFile(join(dir, 'report.html'), `${renderReportHtml(report)}\n`, 'utf8')
|
||||||
|
const smoke = {
|
||||||
|
runId,
|
||||||
|
generatedAt: new Date().toISOString(),
|
||||||
|
nodeVersion: process.version,
|
||||||
|
input: { path: wxrPath, bytes: fileStat.size, sha256 },
|
||||||
|
probe: {
|
||||||
|
totals: report.totals,
|
||||||
|
counts: report.counts,
|
||||||
|
sanitize: report.sanitize,
|
||||||
|
timing: report.timing,
|
||||||
|
rowErrors: { recorded: report.rowErrors.length, truncated: report.rowErrorsTruncated },
|
||||||
|
fatalError: report.fatalError,
|
||||||
|
},
|
||||||
|
memory: {
|
||||||
|
intervalMs: SAMPLE_INTERVAL_MS,
|
||||||
|
samples,
|
||||||
|
maxHeapUsedBytes: maxHeapUsed,
|
||||||
|
maxRssBytes: maxRss,
|
||||||
|
heapLimitBytes: HEAP_LIMIT_BYTES,
|
||||||
|
heapUnderLimit: maxHeapUsed < HEAP_LIMIT_BYTES,
|
||||||
|
},
|
||||||
|
nicenameAudit: nicename,
|
||||||
|
checks,
|
||||||
|
verdict,
|
||||||
|
artifacts: { dir, report: join(dir, 'report.json'), smoke: join(dir, 'smoke.json') },
|
||||||
|
}
|
||||||
|
await writeFile(join(dir, 'smoke.json'), `${JSON.stringify(smoke, null, 2)}\n`, 'utf8')
|
||||||
|
|
||||||
|
for (const c of checks) {
|
||||||
|
process.stderr.write(` [${c.pass ? 'ok' : 'FAIL'}] ${c.name}: expected=${JSON.stringify(c.expected)} actual=${JSON.stringify(c.actual)}\n`)
|
||||||
|
}
|
||||||
|
const nc = nicename.conclusion
|
||||||
|
process.stderr.write(
|
||||||
|
`[smoke] verdict=${verdict}(nicename 抽检 ${nicename.sampleMatched}/${nicename.sampleTotal};` +
|
||||||
|
`在表 taxonomy 对拍 ${nc.matchedAgainstExistingTaxonomy}/${nc.refsAgainstExistingTaxonomy},` +
|
||||||
|
`nicename≠slug 违例 ${nc.nicenameEqualsSlugViolations},缺表 taxonomy 引用 ${nc.refsOfAbsentTaxonomy})\n`,
|
||||||
|
)
|
||||||
|
process.stderr.write(`[smoke] 产物:${join(dir, 'smoke.json')}\n`)
|
||||||
|
|
||||||
|
process.stdout.write(`${JSON.stringify(smoke, null, 2)}\n`)
|
||||||
|
process.exitCode = verdict === 'PASS' ? 0 : 1
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
process.stderr.write(`[smoke] 未预期异常:${err instanceof Error ? err.stack : String(err)}\n`)
|
||||||
|
process.exitCode = 1
|
||||||
|
})
|
||||||
Reference in New Issue
Block a user