feat(deploy): MySQL容器实测打通(3个方言bug修复)+部署文档

- repos: on_duplicate只更新实际插入列(fetched_at等Python-default列,
  SQLite excluded容忍/MySQL new.*直接报Unknown column)
- repos: _coerce_int增加日期字符串→ms epoch分支 + _INT_FIELDS补
  last_update/last_chat_time/timestamp/update_time(字符串直通BigInteger)
- models: wechat_account.db_size Integer→BigInteger(实测2.17GB超32位上限)
- compose: mysql不映射宿主机3306(本机已占;容器内服务名互联)
- docs/DEPLOY.md: 服务器部署全指南(架构/步骤/数据迁移/运维命令/注意事项)
- 容器实测:Playwright抓取55账号→MySQL upsert全通

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
浅念
2026-08-22 10:40:04 +08:00
parent 3dc80805fa
commit 2175cb0169
4 changed files with 168 additions and 7 deletions
+31 -2
View File
@@ -116,6 +116,12 @@ _INT_FIELDS: frozenset[str] = frozenset(
"start_time",
"end_time",
"mp3_duration",
# 时间戳(ms-epoch)列:上游可能给日期字符串而非数字,走 _coerce_int
# 的日期解析分支(MySQL 严格类型实测修复,2026-08-21)
"last_update",
"last_chat_time",
"timestamp",
"update_time",
}
)
@@ -171,7 +177,26 @@ def _coerce_int(value: Any) -> int | None:
try:
return int(s)
except ValueError:
return None
pass
# 日期时间字符串 → ms epoch(如 "2026-08-21 23:29:46");MySQL 的
# BigInteger 列拒绝字符串,SQLite 容忍(2026-08-21 部署实测修复)
ts = _parse_datetime_str_to_ms(s)
if ts is not None:
return ts
return None
def _parse_datetime_str_to_ms(s: str) -> int | None:
"""Best-effort datetime-string → ms epoch for timestamp int columns."""
import datetime as _dt
for fmt in ("%Y-%m-%d %H:%M:%S", "%Y-%m-%d %H:%M", "%Y-%m-%d"):
try:
d = _dt.datetime.strptime(s, fmt)
# 与 scraper 层一致的 naive-UTC 约定
return int(d.replace(tzinfo=_dt.UTC).timestamp() * 1000)
except ValueError:
continue
return None
@@ -263,7 +288,11 @@ def _build_upsert_stmt(
"""
pk_cols = _pk_col_names(model)
# Non-PK columns to refresh on conflict (volatile fields).
update_col_names = [c.name for c in model.__table__.columns if c.name not in pk_cols]
# 只刷新实际插入的列(2026-08-21 MySQL 实测修复):全列取会把
# fetched_at/updated_at 这类 Python-default 列拉进 UPDATE 子句 ——
# SQLite 的 excluded.* 容忍(取 DEFAULT),MySQL 的 new.* 直接报
# "Unknown column"。
update_col_names = [name for name in values if name not in pk_cols]
# NB: access ``excluded`` / ``inserted`` on the BARE insert (before
# ``.values(...)``) and reuse that accessor when building the SET dict.