xinyouhe/yunke
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(scraper): WA群抓取(type=1真机验证,223群)+群成员派生(12840成员/750群)

Browse Source 
- wa-groups 实体:businessFriendList type=1 真机验证返回群,行结构是好友字段名
  (friendUid=@g.us 群ID/nickName=群名),专用 _normalize_wa_group_row
- backfill_group_members.py:无独立成员API,从群消息派生(talker=成员,sender_name=昵称)
- full_sync_all.sh:+4步(5b WA群/语音后录音重链接/8b 群消息+成员/消息上限99999)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
浅念
2026-08-20 09:21:20 +08:00
parent f92aac1a3d
commit 02dd77a62f
3 changed files with 153 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
yunke-server/scripts/full_sync_all.sh
+9 -2
View File
@@ -18,14 +18,21 @@ echo "$(date) --- 4/10 WA customers ---"
$PY scripts/scrape_via_page.py --entity wa-friends
echo "$(date) --- 5/10 WeChat groups ---"
$PY scripts/scrape_via_page.py --entity groups --max-employees 100
echo "$(date) --- 5b/10 WA groups ---"
$PY scripts/scrape_via_page.py --entity wa-groups
echo "$(date) --- 6/10 Voice calls (WeChat + WA) ---"
$PY scripts/scrape_via_page.py --entity voice-calls
# 语音重扫会把 record_media_hash 覆盖回 None —— 必须紧跟录音重链接
$PY -m app.sync.cli sync --entity call_recordings
echo "$(date) --- 7/10 Voice clips ---"
$PY scripts/scrape_via_page.py --entity voice-clips
echo "$(date) --- 8/10 WeChat messages ---"
$PY scripts/scrape_via_page.py --entity messages --max-conversations 1000
$PY scripts/scrape_via_page.py --entity messages --max-conversations 99999
echo "$(date) --- 8b/10 WeChat group messages + members ---"
$PY scripts/scrape_via_page.py --entity group-messages --max-conversations 99999
$PY -m scripts.backfill_group_members
echo "$(date) --- 9/10 WA messages ---"
$PY scripts/scrape_via_page.py --entity wa-messages --max-conversations 500
$PY scripts/scrape_via_page.py --entity wa-messages --max-conversations 99999
echo "$(date) --- 10/10 Media download (audio+images) ---"
$PY scripts/scrape_via_page.py --entity media