fix: split message type into normalized label + raw code

The scraper was writing a normalized type label ("text"/"unknown"/...) into
Message.type, but the column was Integer, so the storage layer coerced it to
NULL and the raw code was only recoverable from raw_json. Align with design
§4.3, which specifies two fields: `type` (normalized enum) and `type_raw`
(original code).

- Model (Message): type -> String(32) normalized label; add type_raw Integer
  for the raw upstream type code.
- Scraper: _normalize_message sets both `type` (label) and `type_raw`
  (coerced int from messageType/type); _is_divider now checks raw -2.
- Repos: type_raw joins _INT_FIELDS; Message.type stored as-is (string).
- Migration 96975050fa79: add type_raw + alter type Integer->String(32),
  wrapped in batch_alter_table so it runs on SQLite and MySQL.
- env.py: enable render_as_batch on SQLite (dev/test) for portable batch
  migrations; MySQL still uses direct ALTER.
- Tests updated end-to-end (scraper, repos, models).

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
浅念
2026-08-01 12:04:08 +08:00
parent 854efd3119
commit 5a2ff58af3
8 changed files with 176 additions and 29 deletions
+9
View File
@@ -50,6 +50,11 @@ def _sync_url(url: str) -> str:
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode (emit SQL without a live connection)."""
url = _sync_url(get_settings().db_url)
# SQLite cannot ALTER most column properties directly; Alembic's batch mode
# (table-recreate) handles it. Enable it for SQLite so the SAME migration
# scripts run on both SQLite (dev/test) and MySQL (prod). MySQL needs no
# batch mode and gets direct ALTERs.
render_as_batch = url.startswith("sqlite")
context.configure(
url=url,
target_metadata=target_metadata,
@@ -57,6 +62,7 @@ def run_migrations_offline() -> None:
dialect_opts={"paramstyle": "named"},
compare_type=True,
compare_server_default=True,
render_as_batch=render_as_batch,
)
with context.begin_transaction():
@@ -74,11 +80,14 @@ def run_migrations_online() -> None:
)
with connectable.connect() as connection:
# See offline mode for the rationale on batch mode per dialect.
render_as_batch = connection.dialect.name == "sqlite"
context.configure(
connection=connection,
target_metadata=target_metadata,
compare_type=True,
compare_server_default=True,
render_as_batch=render_as_batch,
)
with context.begin_transaction():