docs(sp42): skills cyclone-store/tail — refresh module list, event kinds, streaming endpoint count
This commit is contained in:
Nora
@@ -6,17 +6,9 @@ description: "Cyclone store write-paths, the pubsub event contract (claim_writte
|
||||
# cyclone-store
|
||||
|
||||
Cyclone persists every parsed X12 batch through one facade,
|
||||
`CycloneStore` (`backend/src/cyclone/store.py:882`). Every write
|
||||
inserts the row AND publishes a pubsub event on the in-process
|
||||
`EventBus` (`backend/src/cyclone/pubsub.py:20`) so live-tail pages
|
||||
see new rows the moment they land. The event contract is the seam
|
||||
between persistence and streaming — a wrong event name silently
|
||||
goes stale.
|
||||
`CycloneStore` (now exposed via `backend/src/cyclone/store/__init__.py` after the SP21 split landed; the public import surface `from cyclone.store import …` is unchanged). Every write inserts the row AND publishes a pubsub event on the in-process `EventBus` (`backend/src/cyclone/pubsub.py:20`) so live-tail pages see new rows the moment they land. The event contract is the seam between persistence and streaming — a wrong event name silently goes stale.
|
||||
|
||||
As of this writing: the store is a **single 2412-line module** and
|
||||
the SP21 split into `backend/src/cyclone/store/` is in progress.
|
||||
Three event kinds: `claim_written`, `remittance_written`,
|
||||
`activity_recorded`. Next: **SP22**.
|
||||
As of this writing: the store is a **16-module subpackage** under `backend/src/cyclone/store/` (SP21 split landed). Event kinds in use today: `claim_written`, `remittance_written`, `activity_recorded`, plus the SP-acks-tail additions `ack_written` and `ta1_ack_written` for the `/api/acks/stream` and `/api/ta1-acks/stream` endpoints. Next: **SP43** (after the SP42 doc-pass merges).
|
||||
|
||||
## When to use
|
||||
|
||||
@@ -45,12 +37,11 @@ Three event kinds: `claim_written`, `remittance_written`,
|
||||
2. **Every write publishes an event.** The event name matches the
|
||||
entity: `claim_written`, `remittance_written`,
|
||||
`activity_recorded` (the trailing `_recorded` signals a
|
||||
non-canonical row — activity events are derived, not first-class;
|
||||
current names at `store.py:1072,1081,1096`). New entities get
|
||||
`<entity>_written`; activity-style side rows get
|
||||
`<entity>_recorded`. Publish is **best-effort** — failures are
|
||||
logged but never roll back the persisted batch
|
||||
(`store.py:1097-1098`).
|
||||
non-canonical row — activity events are derived, not first-class),
|
||||
plus `ack_written` and `ta1_ack_written` for the SP-acks-tail
|
||||
streams. New entities get `<entity>_written`; activity-style side
|
||||
rows get `<entity>_recorded`. Publish is **best-effort** —
|
||||
failures are logged but never roll back the persisted batch.
|
||||
3. **Snapshot shape.** Each entity has a `to_ui_<entity>` serializer
|
||||
(plain Python function returning a dict) — currently
|
||||
`to_ui_claim`, `to_ui_remittance`, `to_ui_claim_from_orm`,
|
||||
@@ -61,13 +52,15 @@ Three event kinds: `claim_written`, `remittance_written`,
|
||||
that row (`store.py:1052-1054`).
|
||||
4. **SP21 boundaries.** Post-split, each domain lives in its own
|
||||
module under `backend/src/cyclone/store/`: `__init__.py` (facade
|
||||
+ `CycloneStore` class), `exceptions.py`, `records.py`,
|
||||
`orm_builders.py`, `ui.py`, `write.py`, `batches.py`,
|
||||
`claim_detail.py`, `acks.py`, `backups.py`, `inbox.py`,
|
||||
`providers.py`. Cross-module writes go through `CycloneStore`
|
||||
facade methods, not direct module access. The facade re-exports
|
||||
every name callers currently import from `cyclone.store`. Full
|
||||
list: `docs/superpowers/plans/2026-06-21-cyclone-store-split.md:25-37`.
|
||||
+ `CycloneStore` class), `acks.py`, `backfill.py`, `backups.py`,
|
||||
`batches.py`, `claim_acks.py`, `claim_detail.py`, `exceptions.py`,
|
||||
`inbox.py`, `kpis.py`, `orm_builders.py`, `providers.py`,
|
||||
`records.py`, `resubmissions.py`, `submission_dedup.py`, `ui.py`,
|
||||
`write.py` — 16 modules total. Cross-module writes go through
|
||||
`CycloneStore` facade methods, not direct module access. The
|
||||
facade re-exports every name callers currently import from
|
||||
`cyclone.store`. Full split plan:
|
||||
`docs/superpowers/plans/2026-06-21-cyclone-store-split.md`.
|
||||
5. **No business logic in route handlers.** A route handler validates
|
||||
input, calls `store.<method>(...)`, passes the `event_bus`,
|
||||
returns the serialized result. Reconciliation, idempotency
|
||||
@@ -77,7 +70,7 @@ Three event kinds: `claim_written`, `remittance_written`,
|
||||
|
||||
### A `CycloneStore.add` write — publishes events from inserted rows
|
||||
|
||||
Taken from `backend/src/cyclone/store.py:898-1107`. The method opens
|
||||
Taken from `backend/src/cyclone/store/write.py` (post-SP21). The method opens
|
||||
a session, inserts rows, then runs a sync `_publish_events_sync`
|
||||
after commit so subscribers can immediately re-fetch consistent data.
|
||||
|
||||
@@ -118,12 +111,13 @@ FastAPI handlers to await.
|
||||
|
||||
### Backend `/api/<resource>/stream` endpoint — subscribes to the event
|
||||
|
||||
Taken from `backend/src/cyclone/api.py:1380-1401`. Two phases — eager
|
||||
Streaming endpoints live in `backend/src/cyclone/api_routers/<topic>.py`
|
||||
post-SP36 (e.g. `api_routers/claims.py`). Two phases — eager
|
||||
snapshot, then live subscription — wrapped in `StreamingResponse`
|
||||
with `media_type="application/x-ndjson"`.
|
||||
|
||||
```python
|
||||
@app.get("/api/claims/stream")
|
||||
@router.get("/api/claims/stream")
|
||||
async def claims_stream(request: Request, ...) -> StreamingResponse:
|
||||
bus: EventBus = request.app.state.event_bus
|
||||
|
||||
@@ -139,6 +133,10 @@ async def claims_stream(request: Request, ...) -> StreamingResponse:
|
||||
return StreamingResponse(gen(), media_type="application/x-ndjson")
|
||||
```
|
||||
|
||||
The same shape is repeated in `api_routers/remittances.py`,
|
||||
`api_routers/activity.py`, `api_routers/acks.py`, and
|
||||
`api_routers/ta1_acks.py` (five streaming endpoints total).
|
||||
|
||||
`_ndjson_line` and `_tail_events` live in
|
||||
`backend/src/cyclone/api_helpers.py`. The `["remittance_written"]`
|
||||
and `["activity_recorded"]` subscriptions are at `api.py:1891,2002`.
|
||||
@@ -172,11 +170,11 @@ def test_publishes_claim_written_event(client: TestClient) -> None:
|
||||
equivalent `get_*` facade method).
|
||||
- **Don't introduce a new event name without updating the subscriber
|
||||
list.** Today every `<entity>_written` event has exactly one
|
||||
consumer — the matching `/api/<resource>/stream` endpoint,
|
||||
currently in `backend/src/cyclone/api.py` (claims at `:1398`,
|
||||
remittances at `:1891`, activity at `:2002`). Any future split
|
||||
into `backend/src/cyclone/api_routers/` must wire the same
|
||||
subscription.
|
||||
consumer — the matching `/api/<resource>/stream` endpoint, now
|
||||
split across `api_routers/{claims,remittances,activity,acks,ta1_acks}.py`
|
||||
(post-SP36 split). Any new event must wire its subscription in
|
||||
the matching router and add the resource name to the hook
|
||||
triplet in the corresponding page.
|
||||
- **Don't merge a write method with its event publication into
|
||||
separate places.** `_publish_events_sync` lives next to `add` in
|
||||
`store.py:1042-1107` so reviewers see both halves of the contract
|
||||
|
||||
Reference in New Issue
Block a user