Files
xhs_crawler_claude/app/__pycache__/web.cpython-311.pyc
T

100 lines
23 KiB
Plaintext
Raw Normal View History

2026-01-12 18:58:23 +08:00
§
2026-01-12 19:07:00 +08:00
BÕdiU5ã ó—dZddlmZddlZddlZddlZddlmZddlm Z m
2026-01-12 18:58:23 +08:00
Z
m Z m Z ddl
mZmZmZddlmZmZmZddlmZdd lmZdd
lmZdd lmZdd lmZdd
lm Z m!Z!ddl"m#Z#ddl$m%Z%ddl&m'Z'ddl"m(Z(ddl"m)Z)eGd„d¦«¦«Z*e*ej+¦«¬¦«Z,e%¦«Z-edd¬¦«Z"e" .edgddgdgdg¬¦«e" /deej0 1e#j2d¦«¬¦«d¬¦«eej0 1e#j2d ¦«¬¦«Z3e" 4d!¦«d"„¦«Z5e" 4d#¦«d$„¦«Z6Gd%„d&e¦«Z7Gd'„d(e¦«Z8Gd)„d*e¦«Z9Gd+„d,e¦«Z:Gd-„d.e¦«Z;e" <d/e¬0¦«ded3„¦«Z=dfd6„Z>dgd8„Z?dhd;„Z@did=„ZAe" <d>e8¬?¦«d@„¦«ZBe" CdA¦«dB„¦«ZDe" <dC¦«djdkdG„¦«ZEe" <dHe9¬?¦«dI„¦«ZFe" CdH¦«dldK„¦«ZGe" HdL¦«dldM„¦«ZIe" HdH¦«dN„¦«ZJe" <dO¦«dP„¦«ZKe" <dQe:¬?¦«dR„¦«ZLe" <dSe;¬?¦«dT„¦«ZMe" CdU¦«dV„¦«ZNe" CdW¦«dX„¦«ZOe" CdY¦«dZ„¦«ZPe" Cd[¦«dmd^„¦«ZQe" Cd_¦«d`„¦«ZRe" <da¦«dldb„¦«ZSe" <dc¦«dd„¦«ZTdS)nu€web.py - FastAPI å�Žç«¯
目标:
- �供基本的 Web UI + API(�动/�止采集�导入用户�登录/验��日志�SSE)
- �账�登录�长期�用(storage_state �存)
- txt 导入:支�“数字�红书��自动解�为 profile_id(并缓存,低频,�少风控)
- 移除无用/冗余的调试/自更新接�,代���清晰�维护
é)Ú annotationsN)Ú dataclass)ÚListÚOptionalÚDictÚAny)ÚFastAPIÚ
HTTPExceptionÚBackgroundTasks)Ú HTMLResponseÚStreamingResponseÚPlainTextResponse)Ú StaticFiles)ÚJinja2Templates)ÚRequest)ÚCORSMiddleware)Ú BaseModel)Úasync_playwrightÚBrowser)Úconfig)Ú
LogManager)ÚXiaoHongShuScraper)Ú auth_manager)Ú
2026-01-12 19:07:00 +08:00
user_resolvercóV—eZdZUdZded<dZded<dZded<dZd ed
<dZd ed <dS)
ÚAppStateFÚboolÚ
is_runningNzOptional[XiaoHongShuScraper]ÚscraperzOptional[asyncio.Queue]Ú event_queuerÚ
playwrightzOptional[Browser]Úbrowser) Ú__name__Ú
__module__Ú __qualname__rÚ__annotations__rr r!r"©óú/tmp/tmpdw5zp1n_/app/web.pyrr(sg€€€€€€à€JÐÐÐÑØ,0€GÐ0Ð0Ð0Ñ0Ø+/€KÐ/Ð/Ð/Ñ/Ø€JÐÐÐÑØ!%€GÐ%Ð%Ð%Ñ%Ð%Ð%r(r)r uå°�红书数æ�®é‡‡é›†å™¨z1.1.0)ÚtitleÚversionÚ*T)Ú
allow_originsÚallow_credentialsÚ
allow_methodsÚ
allow_headersÚexpose_headersz/staticÚstatic)Ú directory)ÚnameÚ templatesÚstartupcƒópK—dt_dt_t dd¦«dS)u|
��轻��始化,�在�动时强制拉起�览器。
原因:
- 很多环境首次�行未执行 `playwright install chromium`,如果�动� launch,会导致 Web 直接起��(你看到的“打�开�)。
2026-01-12 18:58:23 +08:00
- �览器改为按需�动:第一次需�登录/解�/采集时��动,失败也能在 UI 里看到明确错误。
NÚINFOz7App started (browser will be launched lazily on demand))Ú app_stater!r"Ú log_managerÚappendr'r(r)Ú
on_startupr<Is5èè€ð …IÔØ…IÔÝ×Ò�vÐXÑYÔYÐYÐYÐYr(ÚshutdowncƒóDK— tjr$tj ¦«ƒd{V—†dt_n#dt_wxYw tjr$tj ¦«ƒd{V—†dt_dS#dt_wxYw)N)r9r"Úcloser!Ústopr'r(r)Ú on_shutdownrAVsµèè€ð!Ý Ô ð ,ÝÔ#×)Ò)Ñ+Ô+Ð +Ð +Ð +Ð +Ð +Ð +Ð +à � ÔÐø˜D� ÔÐ Ð Ð Ð ð$Ý Ô ð .ÝÔ&×+Ò+Ñ-Ô-Ð -Ð -Ð -Ð -Ð -Ð -Ð -à#� ÔÐÐø˜t� ÔÐ#Ð#Ð#Ð#s„0AÁAÁ0BÂBcó,—eZdZUdZded<dZded<dS)Ú StartRequestÚsingleÚstrÚmodeéÚintÚ max_threadsN)r#r$r%rFr&rIr'r(r)rCrCjs4€€€€€€Ø€DÐÐÐÑØ€KÐÐÐÑÐÐr(rCcó6—eZdZUded<dZded<dZded<dS) ÚStatusResponserÚrunningNú
2026-01-12 19:07:00 +08:00
Optional[str]rFrrHÚ
user_count)r#r$r%r&rFrNr'r(r)rKrKos=€€€€€€Ø€M€M�MØ€DÐÐÐÑØ€JÐÐÐÑÐÐr(rKcó$—eZdZUded<ded<dS)Ú
UsersResponseú List[str]ÚusersrHÚcountN)r#r$r%r&r'r(r)rPrPus%€€€€€€ØÐÐÑØ€J€J�J€J€Jr(rPcó‚—eZdZUded<ded<ded<ded<ded<ded<ded <ded
<ded <d ed
<gZded<dS)ÚImportResponserQrRrHrSÚ total_linesÚimportedÚskippedÚresolvedÚfailed_resolveÚ
duplicatesÚinvalidrEÚmessageÚfailed_examplesN)r#r$r%r&r^r'r(r)rUrUzs†€€€€€€ØÐÐÑØ€J€J�JØÐÐÑØ€M€M�MØ€L€L�LØ€M€M�MØÐÐÑØ€O€O�OØ€L€L�LØ€L€L�Là!#€OÐ#Ð#Ð#Ñ#Ð#Ð#r(rUcóD—eZdZUded<dZded<dZded<dZded<dS)ÚLoginStatusResponserÚ logged_inNrMÚ
login_timeÚlast_verified_timer])r#r$r%r&rbrcr]r'r(r)r`r`‰sO€€€€€€Ø€O€O�OØ $€JÐ$Ð$Ð$Ñ$Ø(,ÐÐ,Ð,Ð,Ñ,Ø!€GÐ!Ð!Ð!Ñ!Ð!Ð!r(r`ú/)Úresponse_classÚrequestrcƒó@K—t dd|i¦«S)Nz
index.htmlrf)r5ÚTemplateResponse)rfs r)Úindexri”s!èè€å × %Ò % l°YÀÐ4HÑ IÔ IÐIr(ÚreturnrEcóV—tj tjd¦«S)Nz users.txt)ÚosÚpathÚjoinrÚDATA_DIRr'r(r)Ú _users_filerp�s€Ý
Œ7�<Š<�œ¨Ñ 5Ô 5Ð5r(rQcó¨—t¦«}tj |¦«sgSg}t |dd¬¦«5}|D].}| ¦«}|sŒ| |¦«Œ/ ddd¦«n #1swxYwYt¦«}g}|D]1}||vrŒ| |¦«| |¦«Œ2|S)NÚrúutf-8©Úencoding) rprlrmÚexistsÚopenÚstripr;ÚsetÚadd)rmrRÚfÚlineÚseenÚoutÚus r)Úread_users_txtr€¡s#€Ý ‰=Œ=€DÝ
Œ7�>Š>˜$Ñ Ô ðØˆ Ø€EÝ
ˆd�C 'Ð *Ñ *Ô *ð¨aØð ð ˆDØ—:’:‘<”<ˆDØð
2026-01-12 18:58:23 +08:00
ØØ �LŠL˜Ñ Ô Ð Ð ð  ðððñôððððððøøøððððõ ‰5Œ5€DØ
€CØ
ððˆØ �ˆ9ˆ9Ø Ø �Š�‰ Œ ˆ Ø �
Š
�1‰
Œ
ˆ
ˆ
Ø €JsÁ2BÂBÂ
BrRÚNonecóæ—t¦«}|dz}t|dd¬¦«5}|D]}| |dz¦«Œ ddd¦«n #1swxYwYtj||¦«dS)Nz.tmpÚwrsrtú
)rprwÚwriterlÚreplace)rRrmÚtmpr{rs r)Úwrite_users_txtrˆ·sÂ€Ý ‰=Œ=€DØ
�‰-€CÝ
ˆc�3 Ð )Ñ )Ô )ð¨QØð ð ˆAØ
�GŠG�A˜‘HÑ Ô Ð Ð ð ðððñôððððððøøøððððõ„Jˆs�DÑÔÐÐÐs¦AÁAÁArcóT—tj€tdd¬¦«‚tjS)NéôzBrowser not ready©Ú status_codeÚdetail)r9r"r
2026-01-12 19:07:00 +08:00
r'r(r)Úrequire_browserrŽÀs(€ÝÔРݨÐ4GÐHÑHÔHÐHÝ Ô Ðr(z /api/status)Úresponse_modelcƒó¾K—ttjtjrt tjdd¦«ndt t
¦«¦«¬¦«S)NrF)rLrFrN)rKr9rrÚgetattrÚlenr€r'r(r)Ú
api_statusr“ÊsTèè€å ÝÔ$Ý9BÔ9JÐ
T�W•YÔ&¨°Ñ
2026-01-12 18:58:23 +08:00
5Ô
2026-01-12 19:07:00 +08:00
5Ð
2026-01-12 18:58:23 +08:00
5ÐPTÝ•~Ñ'Ô'Ñ(Ô(ð ñ ô ðr(z/api/logs/clearcƒó@K—t ¦«ddiS©NÚsuccessT)r:Úclearr'r(r)Ú
clear_logsr˜Ós#èè€å×ÒÑÔÐØ �tÐ Ðr(z /api/logséÈÚlimitrHcƒó@K—dt |¬¦«iS)NÚlogs©rš)r:Ú
get_latestr�s r)Úget_logsrŸÙs"èè€à •K×*Ò*°Ð*Ñ7Ô7Ð 8Ð8r(z
2026-01-12 19:07:00 +08:00
/api/userscƒó^K—t¦«}t|t|¦«¬¦«S)N)rRrS)r€rPr’)rRs r)Ú get_usersr¡âs*èè€å Ñ Ô €EÝ ˜u­C°©J¬JÐ 7Ñ 7Ô 7Ð7r(Úuser_idcƒóÖK—|pd ¦«}|stdd¬¦«‚t¦«}||vrdddœS| |¦«t |¦«ddiS) NÚé�uuser_id ä¸�能为空r‹Tu 已存在©r–r]r–)rxr
r€r;rˆ©r¢rRs r)Úadd_userr¨ès‡èè€àˆ}˜"×#Ò#Ñ%Ô%€GØ ðLݨÐ4JÐKÑKÔKÐKå Ñ Ô €EØ�%ÐÐØ¨KÐ8Ð8Ð8à ‡L‚L�ÑÔÐÝ�EÑÔÐØ �tÐ Ðr(z/api/users/{user_id}cƒóf‡K—t¦«}ˆfd„|D¦«}t|¦«ddiS)Ncó •—g|]
2026-01-12 18:58:23 +08:00
}|‰k¯|‘Œ Sr'r')Ú.0rr¢s €r)ú
<listcomp>zdelete_user.<locals>.<listcomp>úsø€Ð .Ð .Ð .�1  g¢ ˆQ   r(r–T)r€rˆr§s` r)Ú delete_userr­÷sDøèè€å Ñ Ô €EØ .Ð .Ð .Ð .˜Ð .Ñ .Ô .€EÝ�EÑÔÐØ �tÐ Ðr(cƒó,K—tg¦«ddiSr•)rˆr'r(r)Údelete_all_usersr¯ÿsèè€å�BÑÔÐØ �tÐ Ðr(z/api/users/export/txtcƒóbK—td t¦«¦«¦«S)Nz\n)rrnr€r'r(r)Úexport_users_txtr±s&èè€å ˜UŸZšZ­Ñ(8Ô(8Ñ9Ô9Ñ :Ô :Ð:r(z/api/users/import/txtc
ƒóæK—t¦«}tj |¦«st dd¬¦«‚t |dd¬¦«5}d„|D¦«}ddd¦«n #1swxYwYt
2026-01-12 19:07:00 +08:00
|¦«}t¦«}g}d }d }d }d } d }
2026-01-12 18:58:23 +08:00
g} t¦«} g}
2026-01-12 19:07:00 +08:00
|D�]}tj
|¦«\}}}|rI|  ¦«}|| vr|d
2026-01-12 18:58:23 +08:00
z
}Œ;|   |¦«| 
|¦«Œf|r|
 
||f¦«Œ€| ¦«rN| ¦« d ¦«s'| ¦« d ¦«r|d
2026-01-12 19:07:00 +08:00
z
}n0|d
2026-01-12 18:58:23 +08:00
z
}t
2026-01-12 19:07:00 +08:00
| ¦«d
kr|  
d|›�¦«�Œ|
D]Æ\}}tj||d¬¦«ƒd{V—†\}}|s8|
d
z
2026-01-12 18:58:23 +08:00
}
t
| ¦«d
kr|  
d|›d|›d|›d�¦«Œ_|  ¦«}|| vr|d
z
}Œ}|   |¦«| 
|¦«| d
z
} t#jd¦«ƒd{V—†ŒÇt'|¦«t
|¦«}d|›d| ›d�}t(