fix(migrate): message.text 建 MySQL ngram FULLTEXT 索引(搜索16s回归)
_search_messages_mysql 一直在用 MATCH..AGAINST,但 migrate 按模型建表不含 FULLTEXT、迁移链也从没建过——索引缺失时 MATCH 报错回退 LIKE 全扫:860k 行 LIKE '%q%' 实测 16.3s,axios 15s 超时 ECONNABORTED,搜索页全空(切 Docker MySQL 单库后暴露;8-22 部署验证时已知 LIKE 慢但索引创建环节漏了)。补迁移: 仅 MySQL 方言,原生 DDL(本版 SQLAlchemy Index 不认 WITH PARSER 方言参数), inspect 幂等。实测建成后 документ 0.73s / 报关 0.40s(22~40 倍提速), ngram 对中文俄文按 bigram 切词,子串语义贴近原 LIKE 路径。 Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,68 @@
|
||||
"""MySQL message.text 建 ngram FULLTEXT 索引(搜索 LIKE 全扫 16s 回归修复)
|
||||
|
||||
Revision ID: b41e8fa0c7d2
|
||||
Revises: fe63c6789e94
|
||||
Create Date: 2026-08-27 15:55:00.000000
|
||||
|
||||
内容
|
||||
----
|
||||
``_search_messages_mysql``(search router)早就在用 ``MATCH(m.text) AGAINST``
|
||||
查询,但**没有任何环节在 MySQL 上建过 FULLTEXT 索引**——migrate_sqlite_to_mysql
|
||||
按模型 create_all 建表(不含 FULLTEXT),迁移链也没有——索引缺失时 MATCH 报
|
||||
错、代码回退 LIKE 全扫:2026-08-27 实测 860k 行 ``LIKE '%q%'`` 要 16.3s,前端
|
||||
axios 15s 超时直接 ECONNABORTED,搜索页全空(切 Docker MySQL 单库后暴露)。
|
||||
|
||||
本迁移补上索引:
|
||||
|
||||
* 仅 MySQL 方言执行;SQLite 跳过(那边 FTS5 虚表走 app/db/fts.py 的
|
||||
ensure_fts 体系,与本索引无关)。
|
||||
* ``WITH PARSER ngram``:中文/俄文等无空格分词文本按 bigram 切词,子串式
|
||||
匹配语义最接近原 LIKE 路径。
|
||||
* 幂等:``sa.inspect`` 先查同名索引,存在即跳过(重放安全)。
|
||||
|
||||
注意:86 万行建 ngram 索引约 1-3 分钟(表重建),避开抓取窗口执行。
|
||||
"""
|
||||
from collections.abc import Sequence
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = 'b41e8fa0c7d2'
|
||||
down_revision: str | Sequence[str] | None = 'fe63c6789e94'
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
_INDEX_NAME = 'ft_message_text'
|
||||
|
||||
|
||||
def _has_index(table: str, index: str) -> bool:
|
||||
insp = sa.inspect(op.get_bind())
|
||||
return index in {i['name'] for i in insp.get_indexes(table)}
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Upgrade schema."""
|
||||
bind = op.get_bind()
|
||||
if bind.dialect.name != 'mysql':
|
||||
return # SQLite 的 FTS5 虚表由 app/db/fts.py ensure_fts 管,与此无关
|
||||
if _has_index('message', _INDEX_NAME):
|
||||
return # 已建过,幂等跳过
|
||||
# 原生 DDL:本版 SQLAlchemy 的 Index 不认 WITH PARSER 方言参数
|
||||
bind.execute(
|
||||
sa.text(
|
||||
f"CREATE FULLTEXT INDEX {_INDEX_NAME} ON message (text) "
|
||||
f"WITH PARSER ngram"
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Downgrade schema."""
|
||||
bind = op.get_bind()
|
||||
if bind.dialect.name != 'mysql':
|
||||
return
|
||||
if not _has_index('message', _INDEX_NAME):
|
||||
return
|
||||
bind.execute(sa.text(f"DROP INDEX {_INDEX_NAME} ON message"))
|
||||
Reference in New Issue
Block a user