feat(site-import): POST/GET API 与挂载

Task 7(spec §4):/api/platform/tenants/{slug}/site-import 两端点,
_tenant_guard 守租户(两角色)。POST 校验链:鉴权 → 租户存在/未停用
(404/403) → 形态预检(retry 只认 generate+task_id 非空;新建 URL 复用
safe_fetch._validate_url 单一事实源——scheme/内网字面量/端口,POST 即拒
400「仅支持公网网站地址」) → task.start;异常映射 ImportRateLimited→429 /
ConcurrentLimit→409 / LookupError→404 / ValueError→409(str(exc) 作
detail,任务层白话文案透出)。GET 五键 status/phase/pages/draft/error,
(slug,task_id) 联合索引即越权防线,过期 404「任务不存在或已过期」。
main.py 照 config_routes 位置挂载。测试 15 条(管线全打桩)。

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
浅念
2026-10-20 21:33:00 +08:00
parent e1e97dac2c
commit 57adf8e2a6
3 changed files with 397 additions and 0 deletions
+95
View File
@@ -0,0 +1,95 @@
"""site_import API(spec §4)——2 端点,两角色可用,_tenant_guard 守租户。
- POST /{slug}/site-import:新建(body.url)或复活(body.retry="generate"
+ body.task_id)。校验次序:鉴权(_tenant_guard 依赖链,401/403/409 门
连锁)→ 租户存在且未停用(404/403)→ 形态预检(retry 只认 generate 且
task_id 非空;新建 URL 过 safe_fetch 的 URL 层预检——scheme/host 字面量
禁段/端口白名单,spec §5「内网/非法地址 POST 即拒」,连接层二次兜底)
→ task.start。异常映射(Task 7 备忘):ImportRateLimited→429 /
ConcurrentLimit→409 / LookupError→404(retry 目标不存在)/ ValueError→
409(非 generate-failed 态),str(exc) 作 detail——任务层的白话文案
原样透出,API 层不二次翻译(FetchError 已在任务层白话化,同理由)。
- GET /{slug}/site-import/{task_id}:轮询。task.get 的 (slug, task_id)
联合索引即越权防线(跨租户持他人 id → None → 404);30 分钟过期同理
404「任务不存在或已过期」(前端按终态「任务已过期,请重新导入」处理)。
响应五键(spec §4):status(=任务 state)/ phase / pages(=任务
pages_fetched)/ draft(仅 done,spec §4 草稿包整包)/ error。
启动只做校验+建任务——pipeline 由任务模块的后台 aio task 跑,本层不
感知 crawler/generator(测试也因此能整体打桩 task.start/task.get)。
"""
from fastapi import APIRouter, Depends, HTTPException, Request
from pydantic import BaseModel
from sqlalchemy.ext.asyncio import AsyncSession
from app.platform.routes.tenant_routes import _get_tenant_or_404, _tenant_guard
from app.platform.site_import import safe_fetch, task
router = APIRouter(prefix="/api/platform/tenants", tags=["platform-site-import"])
class StartRequest(BaseModel):
url: str = ""
retry: str = "" # "generate" 复活模式(spec §3.3)
task_id: str = "" # retry 时指定任务
@router.post("/{slug}/site-import")
async def start_import(slug: str, body: StartRequest, request: Request,
user=Depends(_tenant_guard)):
"""启动/复活导入任务 → {task_id}。校验次序与异常映射见模块 docstring。"""
engine = request.app.state.platform_engine
async with AsyncSession(engine, expire_on_commit=False) as session:
tenant = await _get_tenant_or_404(session, slug)
if tenant.status != "active":
raise HTTPException(
status_code=403, detail=f"租户 {slug!r} 已停用,无法发起导入")
url: str | None = None
retry_task_id: str | None = None
if body.retry:
if body.retry != "generate":
raise HTTPException(
status_code=400,
detail=f"retry 只支持 generate(实际 {body.retry!r})")
if not body.task_id:
raise HTTPException(
status_code=400, detail="retry 形态必须带 task_id")
retry_task_id = body.task_id
else:
# URL 层预检单一事实源在 safe_fetch(scheme/禁段字面量/端口)——
# 私有口内部复用,不另抄一份漂移;不公网形态统一白话 400
try:
safe_fetch._validate_url(body.url)
except safe_fetch.FetchError as exc:
raise HTTPException(
status_code=400, detail="仅支持公网网站地址") from exc
url = body.url
try:
task_id = await task.start(slug, url, retry_task_id=retry_task_id)
except task.ImportRateLimited as exc:
raise HTTPException(status_code=429, detail=str(exc)) from exc
except task.ConcurrentLimit as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
except LookupError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {"task_id": task_id}
@router.get("/{slug}/site-import/{task_id}")
async def get_import_task(slug: str, task_id: str,
user=Depends(_tenant_guard)):
"""轮询任务状态(spec §4 五键;404=不存在/已过期/跨租户)。"""
t = task.get(slug, task_id)
if t is None:
raise HTTPException(status_code=404, detail="任务不存在或已过期")
return {
"status": t.state,
"phase": t.phase,
"pages": t.pages_fetched,
"draft": t.draft if t.state == task.TaskState.DONE else None,
"error": t.error,
}