From beff7b2128b7cedd57c7464c77f21b85c240b6ae Mon Sep 17 00:00:00 2001 From: Tyler Date: Sat, 20 Jun 2026 17:18:36 -0600 Subject: [PATCH] docs(readme): document SP5 live-tail behavior, endpoints, status pill --- README.md | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 102 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 89a11af..0f04602 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,69 @@ npm run build npm test ``` +## Live updates + +The Claims, Remittances, and Activity pages stay current without +manual refresh. The backend publishes an internal event on every +store write, the page opens a streaming HTTP connection to the +matching `/api//stream` endpoint, and new rows are +appended to the table the moment they hit the database. + +### Wire format + +Each stream endpoint emits newline-delimited JSON. The first batch +is the **snapshot** of currently-known rows; after that comes +**`snapshot_end`** with the count, then the **live** events. + +```json +{"type":"item","data":{"id":"CLM-1", "...":"..."}} +{"type":"item","data":{"id":"CLM-2", "...":"..."}} +{"type":"snapshot_end","data":{"count":2}} +{"type":"item","data":{"id":"CLM-3", "...":"..."}} ← live +{"type":"heartbeat","data":{"ts":"2026-06-20T23:17:09Z"}} ← idle keep-alive +``` + +Lines are `{"type": ..., "data": ...}`; known types are `item`, +`snapshot_end`, `heartbeat`, and (rare) `item_dropped` / +`error`. Heartbeats keep the connection alive when nothing is +happening — clients flip to `stalled` after 30s of total silence +(heartbeat or otherwise) and surface a **↻ Reconnect** button. + +### Endpoints + +| Method | Path | Subscribes to | Default sort | +| ------ | -------------------------- | ------------------- | ------------------- | +| GET | `/api/claims/stream` | `claim_written` | `-submission_date` | +| GET | `/api/remittances/stream` | `remittance_written`| `-received_date` | +| GET | `/api/activity/stream` | `activity_recorded` | `-timestamp` (limit 50) | + +All three accept the same query params as their non-streaming +counterparts (`status`, `payer`, `date_from`, …) so a frontend can +swap a one-shot fetch for a tail with no URL surgery. Responses +are `Content-Type: application/x-ndjson`. + +### Status pill + +The pill in each toolbar reflects the current connection state: + +| Status | Badge variant | What it means | +| ------------- | ------------- | ------------------------------------------------------------------ | +| `live` | success | snapshot received, listening for new events | +| `connecting` | warning | opening the stream (initial mount) | +| `reconnecting`| warning | previous attempt failed, backing off before retry | +| `stalled` | destructive | no event (including heartbeat) for 30s — click **↻ Reconnect** | +| `error` | destructive | stream errored — click **↻ Reconnect** | +| `closed` | destructive | page unmounted | + +The reconnect button is only shown on `stalled` and `error`. The +backoff schedule on error is `1s → 2s → 4s → 8s → 16s → 30s` capped. + +### Knobs + +| Env var | Default | Effect | +| ---------------------------- | ------- | ------------------------------------------------------- | +| `CYCLONE_TAIL_HEARTBEAT_S` | `15` | Idle heartbeat interval (seconds). Lower it for tests. | + ## Persistence Parsed batches, claims, remittances, matches, and activity events are @@ -94,8 +157,9 @@ backup API). . ├── backend/ │ ├── src/cyclone/ -│ │ ├── api.py # FastAPI app, GET + parse routes -│ │ ├── store.py # InMemoryStore, mappers +│ │ ├── api.py # FastAPI app, GET + parse routes, /api/{resource}/stream +│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out) +│ │ ├── store.py # InMemoryStore, mappers, publish-on-write │ │ ├── __main__.py # `python -m cyclone serve` │ │ ├── cli.py # click CLI │ │ └── parsers/ # X12 tokenizer, models, validator, writers @@ -105,14 +169,20 @@ backup API). │ ├── test_api_gets.py # 6 GET endpoints │ ├── test_store.py # store + mappers + iterators │ ├── test_api_streaming.py +│ ├── test_api_stream_live.py # 3 live-tail endpoints + disconnect cleanup +│ ├── test_pubsub.py # EventBus + subscribe/unsubscribe │ └── test_api_parse_persists.py ├── src/ # React + Vite + TypeScript UI │ ├── components/ -│ │ └── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, … +│ │ ├── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, … +│ │ └── TailStatusPill.tsx # live-tail status badge + reconnect button │ ├── pages/ # Claims, Remittances, Providers, Activity, Upload │ ├── hooks/ # useBatches, useClaims, useRemittances, useProviders, useActivity, useParse +│ │ # + useTailStream, useMergedTail (live tail) │ ├── lib/ # api.ts (6 GET + parse837/parse835/health), format.ts, utils.ts +│ │ # + tail-stream.ts (NDJSON parser) │ ├── store/ # zustand sample-data + parsed-batches store +│ │ # + tail-store.ts (FIFO-capped live tail slices) │ └── types/ # shared TS types ├── docs/ │ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes @@ -123,7 +193,7 @@ backup API). ## Roadmap -Sub-projects 2, 3, and the first half of 4 are **shipped**. Next up: +Sub-projects 2, 3, 4, and 5 are **shipped**. Next up: - **Sub-project 3 (shipped) — More 837P/835 features.** - **837P validation rules:** R034 enforces `REF*G1` on frequency-code @@ -154,9 +224,23 @@ Sub-projects 2, 3, and the first half of 4 are **shipped**. Next up: (404) states are all distinct. Powered by a new backend endpoint `GET /api/claims/{claim_id}` that returns the full drawer payload in a single round-trip. - - **Remaining SP4:** batch diff view, real-time streaming of NDJSON - GET responses, advanced filters (date range, multi-status, saved - filter sets). + - **Remaining SP4:** batch diff view, advanced filters (date range, + multi-status, saved filter sets). +- **Sub-project 5 (shipped) — Live updates.** + - The Claims, Remittances, and Activity pages stream new rows in + real time over Server-Sent Events encoded as newline-delimited + JSON. The backend publishes a `claim_written` / + `remittance_written` / `activity_recorded` event on every store + write; the frontend opens an HTTP stream and dispatches each + event into a per-resource Zustand store that the page reads on + top of its base query results. + - A small status pill in each toolbar surfaces the connection + state — `live` / `connecting` / `reconnecting` / `stalled` / + `error` / `closed` — and offers a manual **↻ Reconnect** button + on `stalled` or `error`. Stale connections (no event for 30s, + heartbeat included) flip to `stalled` automatically; the + back-off ladder on errors is 1s → 2s → 4s → 8s → 16s → 30s + capped. See the "Live updates" section below for details. ### SP3 endpoints @@ -184,6 +268,17 @@ ACKs and lets you download the regenerated 999 text. when the claim doesn't exist — distinct from a transient fetch failure so the UI can render a dedicated not-found state. +### SP5 endpoints (live updates) + +- `GET /api/claims/stream` — NDJSON stream of claim events. Emits a + snapshot (filtered by the same query params as `GET /api/claims`), + then `snapshot_end`, then live `claim_written` events as they + arrive, with a 15s idle heartbeat. +- `GET /api/remittances/stream` — same shape for remittances + (subscribes to `remittance_written`, default sort `-received_date`). +- `GET /api/activity/stream` — same shape for activity events + (subscribes to `activity_recorded`, default `limit=50`). + ## License No license file yet; this is internal-use software. Add a `LICENSE` file