Files
yunke/yunke-server/app/db/base.py
T
浅念 f92aac1a3d feat(scraper): 微信群消息抓取(368778条/750群)+浏览器守护循环+SQLite忙等
- 新 group-messages 实体:真机探明群消息=同一 msg/chats 端点 friendId 传 roomId;
  talker=发送者wxid、msgId=null→owner:room:createTime:talker 合成稳定主键(重扫幂等)
- 浏览器守护循环:超大群撑崩渲染进程时重启浏览器跳过毒群继续(实战2次自愈);
  200页上限;恢复路径补 await
- base.py SQLite connect_args timeout=60:并发写忙等而非撞锁报错
  (双抓取器并发写实测 database is locked 崩任务)
- 5个新单测,全量410绿

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-20 09:09:32 +08:00

55 lines
1.6 KiB
Python

"""Database core: declarative base, async engine, session factory.
Production targets MySQL via `asyncmy` (configured by `Settings.db_url`).
Unit tests build their own in-memory SQLite engines (see `tests/test_db_base.py`)
and only import `Base` / `async_session_maker` from here.
"""
from collections.abc import AsyncIterator
from sqlalchemy.ext.asyncio import (
AsyncSession,
async_sessionmaker,
create_async_engine,
)
from sqlalchemy.orm import DeclarativeBase
from app.config import get_settings
class Base(DeclarativeBase):
"""Common declarative base for all ORM models in the project."""
settings = get_settings()
# SQLite: 写锁竞争时忙等 60s 而不是立刻报 "database is locked"
# (多进程并发写的场景:页面抓取器 × 2 + 后端 API 同时跑,2026-08-19 实测撞锁崩任务)。
_connect_args = {"timeout": 60} if settings.db_url.startswith("sqlite") else {}
async_engine = create_async_engine(
settings.db_url,
pool_pre_ping=True,
pool_recycle=3600, # recycle connections before MySQL wait_timeout (default 8h) closes them
connect_args=_connect_args,
future=True,
)
async_session_maker: async_sessionmaker[AsyncSession] = async_sessionmaker(
async_engine,
expire_on_commit=False,
)
async def get_session() -> AsyncIterator[AsyncSession]:
"""FastAPI dependency that yields an `AsyncSession`.
Usage::
@app.get("/")
async def index(session: AsyncSession = Depends(get_session)):
...
"""
async with async_session_maker() as session:
yield session