13 lines
604 B
Python
13 lines
604 B
Python
|
"""Scraper layer for the yunke data-mirror system.
|
||
|
|
|
||
|
|
Scrapers sit between the session layer (:mod:`app.session.yunke_client`) and
|
||
|
|
the persistence layer (a LATER task). Each scraper is an async generator that
|
||
|
|
walks one upstream list endpoint and yields **normalized dicts** —
|
||
|
|
snake_case keys matching the target ORM model columns, plus a ``raw_json`` key
|
||
|
|
holding the original upstream record verbatim.
|
||
|
|
|
||
|
|
Persistence (upsert into ORM tables) is intentionally out of scope here: it is
|
||
|
|
Task 1.7. Keeping scrapers pure producers lets us test ingestion logic against
|
||
|
|
``respx`` fixtures without a database.
|
||
|
|
"""
|