diff --git a/.superpowers/skills/cyclone-store/SKILL.md b/.superpowers/skills/cyclone-store/SKILL.md index 8d7d2a2..18ffa49 100644 --- a/.superpowers/skills/cyclone-store/SKILL.md +++ b/.superpowers/skills/cyclone-store/SKILL.md @@ -6,17 +6,9 @@ description: "Cyclone store write-paths, the pubsub event contract (claim_writte # cyclone-store Cyclone persists every parsed X12 batch through one facade, -`CycloneStore` (`backend/src/cyclone/store.py:882`). Every write -inserts the row AND publishes a pubsub event on the in-process -`EventBus` (`backend/src/cyclone/pubsub.py:20`) so live-tail pages -see new rows the moment they land. The event contract is the seam -between persistence and streaming — a wrong event name silently -goes stale. +`CycloneStore` (now exposed via `backend/src/cyclone/store/__init__.py` after the SP21 split landed; the public import surface `from cyclone.store import …` is unchanged). Every write inserts the row AND publishes a pubsub event on the in-process `EventBus` (`backend/src/cyclone/pubsub.py:20`) so live-tail pages see new rows the moment they land. The event contract is the seam between persistence and streaming — a wrong event name silently goes stale. -As of this writing: the store is a **single 2412-line module** and -the SP21 split into `backend/src/cyclone/store/` is in progress. -Three event kinds: `claim_written`, `remittance_written`, -`activity_recorded`. Next: **SP22**. +As of this writing: the store is a **16-module subpackage** under `backend/src/cyclone/store/` (SP21 split landed). Event kinds in use today: `claim_written`, `remittance_written`, `activity_recorded`, plus the SP-acks-tail additions `ack_written` and `ta1_ack_written` for the `/api/acks/stream` and `/api/ta1-acks/stream` endpoints. Next: **SP43** (after the SP42 doc-pass merges). ## When to use @@ -45,12 +37,11 @@ Three event kinds: `claim_written`, `remittance_written`, 2. **Every write publishes an event.** The event name matches the entity: `claim_written`, `remittance_written`, `activity_recorded` (the trailing `_recorded` signals a - non-canonical row — activity events are derived, not first-class; - current names at `store.py:1072,1081,1096`). New entities get - `_written`; activity-style side rows get - `_recorded`. Publish is **best-effort** — failures are - logged but never roll back the persisted batch - (`store.py:1097-1098`). + non-canonical row — activity events are derived, not first-class), + plus `ack_written` and `ta1_ack_written` for the SP-acks-tail + streams. New entities get `_written`; activity-style side + rows get `_recorded`. Publish is **best-effort** — + failures are logged but never roll back the persisted batch. 3. **Snapshot shape.** Each entity has a `to_ui_` serializer (plain Python function returning a dict) — currently `to_ui_claim`, `to_ui_remittance`, `to_ui_claim_from_orm`, @@ -61,13 +52,15 @@ Three event kinds: `claim_written`, `remittance_written`, that row (`store.py:1052-1054`). 4. **SP21 boundaries.** Post-split, each domain lives in its own module under `backend/src/cyclone/store/`: `__init__.py` (facade - + `CycloneStore` class), `exceptions.py`, `records.py`, - `orm_builders.py`, `ui.py`, `write.py`, `batches.py`, - `claim_detail.py`, `acks.py`, `backups.py`, `inbox.py`, - `providers.py`. Cross-module writes go through `CycloneStore` - facade methods, not direct module access. The facade re-exports - every name callers currently import from `cyclone.store`. Full - list: `docs/superpowers/plans/2026-06-21-cyclone-store-split.md:25-37`. + + `CycloneStore` class), `acks.py`, `backfill.py`, `backups.py`, + `batches.py`, `claim_acks.py`, `claim_detail.py`, `exceptions.py`, + `inbox.py`, `kpis.py`, `orm_builders.py`, `providers.py`, + `records.py`, `resubmissions.py`, `submission_dedup.py`, `ui.py`, + `write.py` — 16 modules total. Cross-module writes go through + `CycloneStore` facade methods, not direct module access. The + facade re-exports every name callers currently import from + `cyclone.store`. Full split plan: + `docs/superpowers/plans/2026-06-21-cyclone-store-split.md`. 5. **No business logic in route handlers.** A route handler validates input, calls `store.(...)`, passes the `event_bus`, returns the serialized result. Reconciliation, idempotency @@ -77,7 +70,7 @@ Three event kinds: `claim_written`, `remittance_written`, ### A `CycloneStore.add` write — publishes events from inserted rows -Taken from `backend/src/cyclone/store.py:898-1107`. The method opens +Taken from `backend/src/cyclone/store/write.py` (post-SP21). The method opens a session, inserts rows, then runs a sync `_publish_events_sync` after commit so subscribers can immediately re-fetch consistent data. @@ -118,12 +111,13 @@ FastAPI handlers to await. ### Backend `/api//stream` endpoint — subscribes to the event -Taken from `backend/src/cyclone/api.py:1380-1401`. Two phases — eager +Streaming endpoints live in `backend/src/cyclone/api_routers/.py` +post-SP36 (e.g. `api_routers/claims.py`). Two phases — eager snapshot, then live subscription — wrapped in `StreamingResponse` with `media_type="application/x-ndjson"`. ```python -@app.get("/api/claims/stream") +@router.get("/api/claims/stream") async def claims_stream(request: Request, ...) -> StreamingResponse: bus: EventBus = request.app.state.event_bus @@ -139,6 +133,10 @@ async def claims_stream(request: Request, ...) -> StreamingResponse: return StreamingResponse(gen(), media_type="application/x-ndjson") ``` +The same shape is repeated in `api_routers/remittances.py`, +`api_routers/activity.py`, `api_routers/acks.py`, and +`api_routers/ta1_acks.py` (five streaming endpoints total). + `_ndjson_line` and `_tail_events` live in `backend/src/cyclone/api_helpers.py`. The `["remittance_written"]` and `["activity_recorded"]` subscriptions are at `api.py:1891,2002`. @@ -172,11 +170,11 @@ def test_publishes_claim_written_event(client: TestClient) -> None: equivalent `get_*` facade method). - **Don't introduce a new event name without updating the subscriber list.** Today every `_written` event has exactly one - consumer — the matching `/api//stream` endpoint, - currently in `backend/src/cyclone/api.py` (claims at `:1398`, - remittances at `:1891`, activity at `:2002`). Any future split - into `backend/src/cyclone/api_routers/` must wire the same - subscription. + consumer — the matching `/api//stream` endpoint, now + split across `api_routers/{claims,remittances,activity,acks,ta1_acks}.py` + (post-SP36 split). Any new event must wire its subscription in + the matching router and add the resource name to the hook + triplet in the corresponding page. - **Don't merge a write method with its event publication into separate places.** `_publish_events_sync` lives next to `add` in `store.py:1042-1107` so reviewers see both halves of the contract diff --git a/.superpowers/skills/cyclone-tail/SKILL.md b/.superpowers/skills/cyclone-tail/SKILL.md index f57554e..f941f6b 100644 --- a/.superpowers/skills/cyclone-tail/SKILL.md +++ b/.superpowers/skills/cyclone-tail/SKILL.md @@ -5,14 +5,14 @@ description: "Cyclone live-tail streaming wire format and the useTailStream / us # cyclone-tail -Cyclone keeps the Claims, Remittances, and Activity pages live without +Cyclone keeps the Claims, Remittances, Activity, Acks, and Ta1Acks pages live without polling: every store write publishes an internal EventBus event, the page opens a `GET /api//stream` HTTP/1.1 chunked-NDJSON connection, and new rows land in the table the moment they hit the database. This skill codifies the wire format, the hook triplet, and the backoff/stall machinery so additions stay consistent with the -three streaming pages already shipped (`Claims`, `Remittances`, -`ActivityLog`). +five streaming pages already shipped (`Claims`, `Remittances`, +`ActivityLog`, `Acks`, `Ta1Acks`). ## When to use @@ -42,8 +42,8 @@ three streaming pages already shipped (`Claims`, `Remittances`, `item_dropped` (`{"id": "..."}` queue-overflow notice), `error` (`{"message": "..."}` promoted to a thrown error by the hook). Defined at `src/lib/tail-stream.ts:22-44`; emitted by - `backend/src/cyclone/api.py:1357-2006`. See - `references/wire-format.md`. + the streaming routers in `backend/src/cyclone/api_routers/{claims,remittances,activity,acks,ta1_acks}.py` + (post-SP36 split). See `references/wire-format.md`. 2. **Hook triplet.** Streaming pages compose three pieces: `useTailStream(resource)` (`src/hooks/useTailStream.ts:80` — opens the stream, drives the backoff/stall state machine, dispatches @@ -52,8 +52,9 @@ three streaming pages already shipped (`Claims`, `Remittances`, (`src/hooks/useMergedTail.ts:25` — returns `baseItems + tailSlice` dedup'd against `baseItems`), and a per-resource initial-fetch hook (`useClaims`, `useRemittances`, - `useActivity`). The page wires all three — see - `src/pages/Claims.tsx:87-89`. + `useActivity`, `useAcks`, `useTa1Acks`). The page wires all three — + see `src/pages/Claims.tsx:87-89`. The five resources share the + same triplet; only the resource string differs. 3. **Stall threshold.** 30 seconds of total silence — heartbeats included — flips status to `stalled` and surfaces the `↻ Reconnect` button on ``. Constant: @@ -67,8 +68,8 @@ three streaming pages already shipped (`Claims`, `Remittances`, `` from `connecting` to `live` and to reset the reconnect backoff counter (`useTailStream.ts:174-180`). 5. **Content-Type.** Stream endpoints respond with - `media_type="application/x-ndjson"` — see - `backend/src/cyclone/api.py:1401,1894,2005`. Frontend sets + `media_type="application/x-ndjson"` — set in each + `api_routers/.py` stream handler. Frontend sets `Accept: application/x-ndjson` at `src/lib/tail-stream.ts:67-70`. Never `application/json` for a stream endpoint. 6. **Backoff.** Transient errors retry with `1s → 2s → 4s → 8s → 16s @@ -108,7 +109,7 @@ export function ClaimsPage() { ### Backend `/api/foo/stream` endpoint -Pattern from `backend/src/cyclone/api.py:1357-1401`. Register BEFORE +Pattern from `backend/src/cyclone/api_routers/claims.py`. Register BEFORE `/api/foo/{foo_id}` so the literal `stream` segment doesn't match as an id. Two phases — eager snapshot, then live subscription — wrapped in `StreamingResponse` with `media_type="application/x-ndjson"`. @@ -160,7 +161,7 @@ takes `status` + `lastEventAt` from `useTailStream` plus - **Don't change the wire format on one endpoint without updating the others.** The parser (`src/lib/tail-stream.ts`) and the hook dispatch switch (`useTailStream.ts:173-195`) are shared across all - three live streams. Adding a new event type means updating + five live streams. Adding a new event type means updating `TailEvent`, `KNOWN_TYPES`, the dispatch `case`, the `armStall` re-arm list, and the backend emitter — in that order. - **Don't emit `item` events before `snapshot_end`.** The hook uses @@ -168,7 +169,7 @@ takes `status` + `lastEventAt` from `useTailStream` plus `connecting` to `live` and to reset the reconnect backoff counter. Emitting `item`s first lands rows in `useTailStore` but the UI still reads "Connecting" — operators see a flash of stale state. - Emit snapshot, then `snapshot_end`, then live (`api.py:1386-1401`). + Emit snapshot, then `snapshot_end`, then live. - **Don't change the 30s stall threshold without updating the README "Status pill" table.** The constant (`STALL_TIMEOUT_MS = 30_000` at `useTailStream.ts:53`) is the @@ -176,7 +177,7 @@ takes `status` + `lastEventAt` from `useTailStream` plus edit at `README.md:118`. - **Don't add `useTailStream` calls from `useFoo.ts` hooks.** Streaming connections belong on the page (`src/pages/Claims.tsx`, - `Remittances.tsx`, `ActivityLog.tsx`). Hoisting into `useFoo` + `Remittances.tsx`, `ActivityLog.tsx`, `Acks.tsx`, `Ta1Acks.tsx`). Hoisting into `useFoo` couples the lifecycle to whoever mounts it and breaks one-resource-one-page ownership. @@ -185,16 +186,16 @@ takes `status` + `lastEventAt` from `useTailStream` plus - **`cyclone-frontend-page`** — page components live in `src/pages/`; load when adding or refactoring a streaming page to confirm the route + `PageHeader` + table conventions. -- **`cyclone-api-router`** — endpoint conventions (`api_routers/` for - resource-group routers, `api.py` for the live-tail endpoints); load - when adding or changing an HTTP endpoint that surfaces streamed or - paginated data. +- **`cyclone-api-router`** — endpoint conventions (`api_routers/` + hosts the streaming endpoints post-SP36, alongside the other + resource-group routers); load when adding or changing an HTTP + endpoint that surfaces streamed or paginated data. - **`cyclone-store`** — write-path conventions in `store.py` and the pubsub event contract (`claim_written`, `remittance_written`, `activity_recorded`); load when adding a new entity whose writes should fan out to a stream endpoint. - **`cyclone-tests`** — frontend `*.test.tsx` siblings cover `useTailStream`, `useMergedTail`, `TailStatusPill`; backend - `test_api_stream_live.py` covers the three live-tail endpoints; - load when the increment changes wire-format behavior or adds a - streaming hook. \ No newline at end of file + `test_api_stream_live.py` covers the five live-tail endpoints + (claims, remittances, activity, acks, ta1-acks); load when the + increment changes wire-format behavior or adds a streaming hook. \ No newline at end of file