fix: split message type into normalized label + raw code
The scraper was writing a normalized type label ("text"/"unknown"/...) into
Message.type, but the column was Integer, so the storage layer coerced it to
NULL and the raw code was only recoverable from raw_json. Align with design
§4.3, which specifies two fields: `type` (normalized enum) and `type_raw`
(original code).
- Model (Message): type -> String(32) normalized label; add type_raw Integer
for the raw upstream type code.
- Scraper: _normalize_message sets both `type` (label) and `type_raw`
(coerced int from messageType/type); _is_divider now checks raw -2.
- Repos: type_raw joins _INT_FIELDS; Message.type stored as-is (string).
- Migration 96975050fa79: add type_raw + alter type Integer->String(32),
wrapped in batch_alter_table so it runs on SQLite and MySQL.
- env.py: enable render_as_batch on SQLite (dev/test) for portable batch
migrations; MySQL still uses direct ALTER.
- Tests updated end-to-end (scraper, repos, models).
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -351,7 +351,12 @@ class Message(Base):
|
||||
String(128), nullable=True
|
||||
) # @chatroom id; NULL/empty = direct message
|
||||
is_send: Mapped[bool | None] = mapped_column(Boolean, nullable=True) # mine/isSend
|
||||
type: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
type: Mapped[str | None] = mapped_column(
|
||||
String(32), nullable=True
|
||||
) # normalized enum label (text/image/voice/.../unknown); see design §4.3
|
||||
type_raw: Mapped[int | None] = mapped_column(
|
||||
Integer, nullable=True
|
||||
) # original upstream type code (e.g. WeChat 1=text, -2=divider)
|
||||
text: Mapped[str | None] = mapped_column(Text, nullable=True) # content/text
|
||||
file_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
file_th_url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
|
||||
Reference in New Issue
Block a user