feat(sync): 聊天图片本地化(message media_hash+download_message_images+local-first rendering)
- Message 模型新增 media_hash 列 (sha256 FK -> Media.media_hash)
- Alembic 迁移 e5f6a7b8c9d0_message_media_hash (batch_alter_table, SQLite/MySQL 通用)
- app/scraper/media.py:download_message_images 批量下载器,镜像 download_call_recordings
- 候选: media_hash IS NULL AND channel IN ('wechat','whatsapp') AND type IN ('image','video','file') AND 任一 file_*_url LIKE 'http%'
- image 类型优先 file_th_url (缩略图,小快省),video/file 优先 file_url
- per-row try/except + per-run tried set,失败不挂循环
- sync_message_images orchestrator 入口 + ENTITY_MESSAGE_IMAGES 常量 (提交自身 SyncRun,镜像 call_recordings)
- scheduler: _run_message_images + ENTITY_FUNCS['message_images'] + 加入 daily all job (尾部)
- MessageOut schema 暴露 media_hash 字段
- MessageBubble.vue local-first: media_hash 优先 /api/media/{hash},否则回退 OSS file_url/file_th_url/file_hd_url
- types.ts: Message 接口添加 media_hash 字段
- 新增 test_message_images_downloader.py (6 tests, 镜像 test_recording_downloader)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -57,6 +57,7 @@ from app.sync.orchestrator import (
|
||||
sync_all_wechat,
|
||||
sync_call_recordings,
|
||||
sync_departments,
|
||||
sync_message_images,
|
||||
sync_phone_calls,
|
||||
sync_sms,
|
||||
sync_voice_clip_audio,
|
||||
@@ -218,6 +219,13 @@ async def _run_voice_clip_audio(
|
||||
return [await sync_voice_clip_audio(session, client, store)]
|
||||
|
||||
|
||||
async def _run_message_images(
|
||||
session: AsyncSession, client: YunkeClient, store: ObjectStore
|
||||
) -> list[SyncResult]:
|
||||
"""Image / media localization pass for already-mirrored ``message`` rows."""
|
||||
return [await sync_message_images(session, client, store)]
|
||||
|
||||
|
||||
async def _run_messages(
|
||||
session: AsyncSession, client: YunkeClient, store: ObjectStore
|
||||
) -> list[SyncResult]:
|
||||
@@ -301,6 +309,13 @@ async def _run_all(
|
||||
# in the daily job means no later in-batch entity can be affected by the
|
||||
# mid-batch commit boundary.
|
||||
results.append(await sync_voice_clip_audio(session, client, store))
|
||||
# Message image / media localization: download OSS images / videos / files
|
||||
# for already-mirrored ``message`` rows. Runs AFTER messages (synced
|
||||
# earlier in this same daily job via _run_all_wechat + _run_wa_messages)
|
||||
# so freshly-scraped rows are eligible. Commits internally (mirrors
|
||||
# call_recordings / voice_clip_audio) — also placed at the tail of the
|
||||
# daily job for the same mid-batch commit-boundary reason.
|
||||
results.append(await sync_message_images(session, client, store))
|
||||
return results
|
||||
|
||||
|
||||
@@ -324,6 +339,7 @@ ENTITY_FUNCS: dict[
|
||||
"voice_clips": _run_voice_clips,
|
||||
"call_recordings": _run_call_recordings,
|
||||
"voice_clip_audio": _run_voice_clip_audio,
|
||||
"message_images": _run_message_images,
|
||||
"messages": _run_messages,
|
||||
"wa_messages": _run_wa_messages,
|
||||
"all_wechat": _run_all_wechat,
|
||||
|
||||
Reference in New Issue
Block a user