Tyler
326 lines
14 KiB
Markdown
326 lines
14 KiB
Markdown
# Cyclone
|
||
|
||
A self-hosted EDI claims management suite for a single billing office.
|
||
Parses 837P professional claims and 835 ERA remittances (X12 005010X222A1
|
||
and 005010X221A1), with a local-only FastAPI backend and a React UI for
|
||
browsing, filtering, and inspecting the parsed data.
|
||
|
||
Local-only on purpose: binds to `127.0.0.1`, no auth, no internet exposure.
|
||
Built for one operator, one machine, one trading partner (Colorado
|
||
Medicaid, currently).
|
||
|
||
## Install
|
||
|
||
```bash
|
||
# Backend (Python 3.11+)
|
||
cd backend
|
||
python -m venv .venv
|
||
.venv/bin/pip install -e '.[dev]'
|
||
|
||
# Frontend (Node 20+)
|
||
cd ..
|
||
npm install
|
||
```
|
||
|
||
## Dev
|
||
|
||
Two terminals:
|
||
|
||
```bash
|
||
# Terminal 1 — backend
|
||
cd backend
|
||
.venv/bin/python -m cyclone serve
|
||
# (defaults to 127.0.0.1:8000; override with CYCLONE_PORT=...; reload with CYCLONE_RELOAD=1)
|
||
|
||
# Terminal 2 — frontend
|
||
npm run dev
|
||
# (Vite on http://localhost:5173)
|
||
```
|
||
|
||
Then open `http://localhost:5173`. Drop an `.txt` 837P or 835 file on the
|
||
Upload page; navigate to Claims, Remittances, Providers, or Activity to
|
||
see the parsed data.
|
||
|
||
The frontend reads its backend URL from `VITE_API_BASE_URL` (default
|
||
empty). Create a `.env.local` at the repo root with:
|
||
|
||
```
|
||
VITE_API_BASE_URL=http://127.0.0.1:8000
|
||
```
|
||
|
||
Without that, the UI falls back to its in-memory sample store via the
|
||
existing `data` adapter (parses are disabled).
|
||
|
||
## Test
|
||
|
||
```bash
|
||
# Backend
|
||
cd backend && .venv/bin/pytest
|
||
|
||
# Frontend type-check + build
|
||
npm run typecheck
|
||
npm run build
|
||
|
||
# Frontend unit tests
|
||
npm test
|
||
```
|
||
|
||
## Live updates
|
||
|
||
The Claims, Remittances, and Activity pages stay current without
|
||
manual refresh. The backend publishes an internal event on every
|
||
store write, the page opens a streaming HTTP connection to the
|
||
matching `/api/<resource>/stream` endpoint, and new rows are
|
||
appended to the table the moment they hit the database.
|
||
|
||
### Wire format
|
||
|
||
Each stream endpoint emits newline-delimited JSON. The first batch
|
||
is the **snapshot** of currently-known rows; after that comes
|
||
**`snapshot_end`** with the count, then the **live** events.
|
||
|
||
```json
|
||
{"type":"item","data":{"id":"CLM-1", "...":"..."}}
|
||
{"type":"item","data":{"id":"CLM-2", "...":"..."}}
|
||
{"type":"snapshot_end","data":{"count":2}}
|
||
{"type":"item","data":{"id":"CLM-3", "...":"..."}} ← live
|
||
{"type":"heartbeat","data":{"ts":"2026-06-20T23:17:09Z"}} ← idle keep-alive
|
||
```
|
||
|
||
Lines are `{"type": ..., "data": ...}`; known types are `item`,
|
||
`snapshot_end`, `heartbeat`, and (rare) `item_dropped` /
|
||
`error`. Heartbeats keep the connection alive when nothing is
|
||
happening — clients flip to `stalled` after 30s of total silence
|
||
(heartbeat or otherwise) and surface a **↻ Reconnect** button.
|
||
|
||
### Endpoints
|
||
|
||
| Method | Path | Subscribes to | Default sort |
|
||
| ------ | -------------------------- | ------------------- | ------------------- |
|
||
| GET | `/api/claims/stream` | `claim_written` | `-submission_date` |
|
||
| GET | `/api/remittances/stream` | `remittance_written`| `-received_date` |
|
||
| GET | `/api/activity/stream` | `activity_recorded` | `-timestamp` (limit 50) |
|
||
|
||
All three accept the same query params as their non-streaming
|
||
counterparts (`status`, `payer`, `date_from`, …) so a frontend can
|
||
swap a one-shot fetch for a tail with no URL surgery. Responses
|
||
are `Content-Type: application/x-ndjson`.
|
||
|
||
### Status pill
|
||
|
||
The pill in each toolbar reflects the current connection state:
|
||
|
||
| Status | Badge variant | What it means |
|
||
| ------------- | ------------- | ------------------------------------------------------------------ |
|
||
| `live` | success | snapshot received, listening for new events |
|
||
| `connecting` | warning | opening the stream (initial mount) |
|
||
| `reconnecting`| warning | previous attempt failed, backing off before retry |
|
||
| `stalled` | destructive | no event (including heartbeat) for 30s — click **↻ Reconnect** |
|
||
| `error` | destructive | stream errored — click **↻ Reconnect** |
|
||
| `closed` | destructive | page unmounted |
|
||
|
||
The reconnect button is only shown on `stalled` and `error`. The
|
||
backoff schedule on error is `1s → 2s → 4s → 8s → 16s → 30s` capped.
|
||
|
||
### Knobs
|
||
|
||
| Env var | Default | Effect |
|
||
| ---------------------------- | ------- | ------------------------------------------------------- |
|
||
| `CYCLONE_TAIL_HEARTBEAT_S` | `15` | Idle heartbeat interval (seconds). Lower it for tests. |
|
||
|
||
## Inbox
|
||
|
||
`/inbox` is the working surface. Four lanes, dark by default (Ticker Tape
|
||
aesthetic):
|
||
|
||
- **Rejected** — claims whose 999 set-level response was R or E. Re-submit in bulk.
|
||
- **Candidates** — remits whose CLP-claim-id didn't match exactly; each one shows its top scored claim. One-click manual match or dismiss.
|
||
- **Unmatched** — claims still waiting for a remit, and remits with no candidates above the threshold.
|
||
- **Done today** — terminal state transitions in the last 24 hours.
|
||
|
||
The page subscribes to the SP5 claim and remittance tail streams
|
||
(`/api/claims/stream` and `/api/remittances/stream`); a new `item`
|
||
event refetches the lane payload (debounced ~250ms). When a 999
|
||
parses and rejects claims, the inbox reflects the new
|
||
**Rejected** rows within a fraction of a second.
|
||
|
||
### Inbox endpoints
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
||
| GET | `/api/inbox/lanes` | All four lanes in one call. |
|
||
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
|
||
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
|
||
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
|
||
| GET | `/api/inbox/export.csv?lane=<lane>` | Streams CSV of the lane's rows. |
|
||
|
||
### Scoring
|
||
|
||
4-field weighted (sum = 100):
|
||
|
||
| Field | Weight | Rule |
|
||
| ----------------------- | ------ | ------------------------------------------------------------- |
|
||
| Patient control number | 40 | Exact match, normalized (case + leading zeros). |
|
||
| Service date | 25 | Linear decay over ±3 days. |
|
||
| Charge amount | 20 | Linear decay over ±10%. |
|
||
| Provider NPI | 15 | Exact match. Missing → 0. |
|
||
|
||
Tiers: **strong** (≥75, full opacity, Match enabled), **weak**
|
||
(50–74, dimmed), **hidden** (<50, not surfaced).
|
||
|
||
## Persistence
|
||
|
||
Parsed batches, claims, remittances, matches, and activity events are
|
||
stored in a SQLite file at `~/.local/share/cyclone/cyclone.db` by
|
||
default. The directory is auto-created on first run.
|
||
|
||
To use a different location, set `CYCLONE_DB_URL`:
|
||
|
||
```bash
|
||
export CYCLONE_DB_URL=sqlite:///path/to/cyclone.db
|
||
# or
|
||
export CYCLONE_DB_URL=postgresql://user:pass@host:5432/cyclone
|
||
```
|
||
|
||
### Backup
|
||
|
||
```bash
|
||
sqlite3 ~/.local/share/cyclone/cyclone.db ".backup /path/to/backup.db"
|
||
```
|
||
|
||
This is safe to run while the backend is running (uses SQLite's online
|
||
backup API).
|
||
|
||
## Project layout
|
||
|
||
```
|
||
.
|
||
├── backend/
|
||
│ ├── src/cyclone/
|
||
│ │ ├── api.py # FastAPI app, GET + parse routes, /api/{resource}/stream
|
||
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
|
||
│ │ ├── store.py # InMemoryStore, mappers, publish-on-write
|
||
│ │ ├── __main__.py # `python -m cyclone serve`
|
||
│ │ ├── cli.py # click CLI
|
||
│ │ └── parsers/ # X12 tokenizer, models, validator, writers
|
||
│ └── tests/
|
||
│ ├── fixtures/ # co_medicaid_*.txt, minimal_*.txt
|
||
│ ├── test_api.py # parse-837/835 round-trip
|
||
│ ├── test_api_gets.py # 6 GET endpoints
|
||
│ ├── test_store.py # store + mappers + iterators
|
||
│ ├── test_api_streaming.py
|
||
│ ├── test_api_stream_live.py # 3 live-tail endpoints + disconnect cleanup
|
||
│ ├── test_pubsub.py # EventBus + subscribe/unsubscribe
|
||
│ └── test_api_parse_persists.py
|
||
├── src/ # React + Vite + TypeScript UI
|
||
│ ├── components/
|
||
│ │ ├── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, …
|
||
│ │ └── TailStatusPill.tsx # live-tail status badge + reconnect button
|
||
│ ├── pages/ # Claims, Remittances, Providers, Activity, Upload
|
||
│ ├── hooks/ # useBatches, useClaims, useRemittances, useProviders, useActivity, useParse
|
||
│ │ # + useTailStream, useMergedTail (live tail)
|
||
│ ├── lib/ # api.ts (6 GET + parse837/parse835/health), format.ts, utils.ts
|
||
│ │ # + tail-stream.ts (NDJSON parser)
|
||
│ ├── store/ # zustand sample-data + parsed-batches store
|
||
│ │ # + tail-store.ts (FIFO-capped live tail slices)
|
||
│ └── types/ # shared TS types
|
||
├── docs/
|
||
│ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes
|
||
│ └── superpowers/plans/ # implementation plan
|
||
├── tailwind.config.js # shimmer, scan, row-flash keyframes
|
||
└── package.json
|
||
```
|
||
|
||
## Roadmap
|
||
|
||
Sub-projects 2, 3, 4, and 5 are **shipped**. Next up:
|
||
|
||
- **Sub-project 3 (shipped) — More 837P/835 features.**
|
||
- **837P validation rules:** R034 enforces `REF*G1` on frequency-code
|
||
7/8 claims; R035 enforces `BHT06` transaction-type-code is in the
|
||
allowed set per payer config.
|
||
- **835 CAS deep-parsing:** every CAS adjustment now surfaces with its
|
||
CARC reason code + label (e.g. `CO-29: The time limit for filing has
|
||
expired`), surfaced via `GET /api/remittances/{id}` and rendered as
|
||
an expansion row on the Remittances page.
|
||
- **999 ACK transaction set:** full inbound parser + outbound
|
||
serializer. Auto-generated on 837 ingest when `?ack=true` is passed
|
||
to `POST /api/parse-837`. Persisted to the `acks` table, browsable
|
||
on a new `/acks` page, and downloadable as the regenerated raw
|
||
999 text.
|
||
- **270/271 eligibility (API-only):** `POST /api/eligibility/request`
|
||
builds a 270 from a JSON payload (subscriber, provider, payer,
|
||
service type); `POST /api/eligibility/parse-271` ingests the
|
||
response and returns structured `coverage_benefits`.
|
||
- **Sub-project 4 (partially shipped) — Frontend features.**
|
||
- **Per-claim detail drawer (shipped):** click any row on the Claims
|
||
page to open a side-panel drawer with the full claim context —
|
||
state + amount header, validation panel, service lines, diagnoses,
|
||
parties (billing provider, subscriber, payer), raw X12 segments,
|
||
matched remittance summary, and a vertical state-history timeline.
|
||
URL is synced (`?claim=...`) so links and back-button restore the
|
||
drawer. Keyboard nav: `j`/`k` move between claims, `esc` closes,
|
||
`?` opens the cheatsheet overlay. Skeleton + error + not-found
|
||
(404) states are all distinct. Powered by a new backend endpoint
|
||
`GET /api/claims/{claim_id}` that returns the full drawer payload
|
||
in a single round-trip.
|
||
- **Remaining SP4:** batch diff view, advanced filters (date range,
|
||
multi-status, saved filter sets).
|
||
- **Sub-project 5 (shipped) — Live updates.**
|
||
- The Claims, Remittances, and Activity pages stream new rows in
|
||
real time over Server-Sent Events encoded as newline-delimited
|
||
JSON. The backend publishes a `claim_written` /
|
||
`remittance_written` / `activity_recorded` event on every store
|
||
write; the frontend opens an HTTP stream and dispatches each
|
||
event into a per-resource Zustand store that the page reads on
|
||
top of its base query results.
|
||
- A small status pill in each toolbar surfaces the connection
|
||
state — `live` / `connecting` / `reconnecting` / `stalled` /
|
||
`error` / `closed` — and offers a manual **↻ Reconnect** button
|
||
on `stalled` or `error`. Stale connections (no event for 30s,
|
||
heartbeat included) flip to `stalled` automatically; the
|
||
back-off ladder on errors is 1s → 2s → 4s → 8s → 16s → 30s
|
||
capped. See the "Live updates" section below for details.
|
||
|
||
### SP3 endpoints
|
||
|
||
- `POST /api/parse-837?ack=true` — existing 837 parse, plus optional
|
||
auto-generated 999 ACK persisted alongside the batch.
|
||
- `POST /api/parse-999` — parse an inbound 999 ACK and persist it.
|
||
- `GET /api/acks` — list ACKs.
|
||
- `GET /api/acks/{id}` — ACK detail, including the regenerated
|
||
`raw_999_text`.
|
||
- `POST /api/eligibility/request` — build a 270 from JSON.
|
||
- `POST /api/eligibility/parse-271` — ingest a 271 and return parsed
|
||
coverage benefits.
|
||
|
||
The UI gains a new **Acks** page (sidebar entry) that lists persisted
|
||
ACKs and lets you download the regenerated 999 text.
|
||
|
||
### SP4 endpoints (claim drawer)
|
||
|
||
- `GET /api/claims/{claim_id}` — full claim context for the drawer
|
||
payload: header fields, diagnoses, service lines, parties
|
||
(billing provider / subscriber / payer), validation result, raw
|
||
X12 segments, matched remittance summary (or `null`), and the
|
||
claim's state history (ordered newest-first). 404 with a structured
|
||
`{ "error": "Not found", "detail": "Claim {id} not found" }` body
|
||
when the claim doesn't exist — distinct from a transient fetch
|
||
failure so the UI can render a dedicated not-found state.
|
||
|
||
### SP5 endpoints (live updates)
|
||
|
||
- `GET /api/claims/stream` — NDJSON stream of claim events. Emits a
|
||
snapshot (filtered by the same query params as `GET /api/claims`),
|
||
then `snapshot_end`, then live `claim_written` events as they
|
||
arrive, with a 15s idle heartbeat.
|
||
- `GET /api/remittances/stream` — same shape for remittances
|
||
(subscribes to `remittance_written`, default sort `-received_date`).
|
||
- `GET /api/activity/stream` — same shape for activity events
|
||
(subscribes to `activity_recorded`, default `limit=50`).
|
||
|
||
## License
|
||
|
||
No license file yet; this is internal-use software. Add a `LICENSE` file
|
||
when one is decided.
|