tyler/cyclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ea64e6e0f070a0d84e8fce30495625c3a7abce89
cyclone/README.md
T
Tyler ea64e6e0f0 docs: list all 8 api_routers/ subpackages in Project layout 
Four additional routers landed since the last doc pass:
- activity.py (GET /api/activity, /api/activity/stream)
- batches.py (GET /api/batches, /api/batches/{id}, /api/batch-diff)
- providers.py (GET /api/providers)
- clearhouse.py (GET /api/clearhouse, POST /api/clearhouse/submit)
- config.py (/api/config/providers[/...], /api/config/payers[/...],
  POST /api/admin/reload-config)

Update the Project layout block to list all 8 routers with the routes
they own. No new functionality is introduced — the refactor is purely
a code-organization move from api.py into the api_routers/ subpackage.
No code changes; no tests touched.
2026-06-21 03:05:52 -06:00

868 lines
46 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Cyclone
A self-hosted EDI claims management suite for a single billing office.
Parses 837P professional claims and 835 ERA remittances (X12 005010X222A1
and 005010X221A1), with a local-only FastAPI backend and a React UI for
browsing, filtering, and inspecting the parsed data.
Local-only on purpose: binds to `127.0.0.1`, no auth, no internet exposure.
Built for one operator, one machine, one trading partner (Colorado
Medicaid, currently).
## Install
```bash
# Backend (Python 3.11+)
cd backend
python -m venv .venv
.venv/bin/pip install -e '.[dev]'
# Frontend (Node 20+)
cd ..
npm install
```
## Dev
Two terminals:
```bash
# Terminal 1 — backend
cd backend
.venv/bin/python -m cyclone serve
# (defaults to 127.0.0.1:8000; override with CYCLONE_PORT=...; reload with CYCLONE_RELOAD=1)
# Terminal 2 — frontend
npm run dev
# (Vite on http://localhost:5173)
```
Then open `http://localhost:5173`. Drop an `.txt` 837P or 835 file on the
Upload page; navigate to Claims, Remittances, Providers, or Activity to
see the parsed data.
The frontend reads its backend URL from `VITE_API_BASE_URL` (default
empty). Create a `.env.local` at the repo root with:
```
VITE_API_BASE_URL=http://127.0.0.1:8000
```
Without that, the UI falls back to its in-memory sample store via the
existing `data` adapter (parses are disabled).
## Test
```bash
# Backend
cd backend && .venv/bin/pytest
# Frontend type-check + build
npm run typecheck
npm run build
# Frontend unit tests
npm test
```
## Live updates
The Claims, Remittances, and Activity pages stay current without
manual refresh. The backend publishes an internal event on every
store write, the page opens a streaming HTTP connection to the
matching `/api/<resource>/stream` endpoint, and new rows are
appended to the table the moment they hit the database.
### Wire format
Each stream endpoint emits newline-delimited JSON. The first batch
is the **snapshot** of currently-known rows; after that comes
**`snapshot_end`** with the count, then the **live** events.
```json
{"type":"item","data":{"id":"CLM-1", "...":"..."}}
{"type":"item","data":{"id":"CLM-2", "...":"..."}}
{"type":"snapshot_end","data":{"count":2}}
{"type":"item","data":{"id":"CLM-3", "...":"..."}} ← live
{"type":"heartbeat","data":{"ts":"2026-06-20T23:17:09Z"}} ← idle keep-alive
```
Lines are `{"type": ..., "data": ...}`; known types are `item`,
`snapshot_end`, `heartbeat`, and (rare) `item_dropped` /
`error`. Heartbeats keep the connection alive when nothing is
happening — clients flip to `stalled` after 30s of total silence
(heartbeat or otherwise) and surface a **↻ Reconnect** button.
### Endpoints
| Method | Path | Subscribes to | Default sort |
| ------ | -------------------------- | ------------------- | ------------------- |
| GET | `/api/claims/stream` | `claim_written` | `-submission_date` |
| GET | `/api/remittances/stream` | `remittance_written`| `-received_date` |
| GET | `/api/activity/stream` | `activity_recorded` | `-timestamp` (limit 50) |
All three accept the same query params as their non-streaming
counterparts (`status`, `payer`, `date_from`, …) so a frontend can
swap a one-shot fetch for a tail with no URL surgery. Responses
are `Content-Type: application/x-ndjson`.
### Status pill
The pill in each toolbar reflects the current connection state:
| Status | Badge variant | What it means |
| ------------- | ------------- | ------------------------------------------------------------------ |
| `live` | success | snapshot received, listening for new events |
| `connecting` | warning | opening the stream (initial mount) |
| `reconnecting`| warning | previous attempt failed, backing off before retry |
| `stalled` | destructive | no event (including heartbeat) for 30s — click **↻ Reconnect** |
| `error` | destructive | stream errored — click **↻ Reconnect** |
| `closed` | destructive | page unmounted |
The reconnect button is only shown on `stalled` and `error`. The
backoff schedule on error is `1s → 2s → 4s → 8s → 16s → 30s` capped.
### Knobs
| Env var | Default | Effect |
| ---------------------------- | ------- | ------------------------------------------------------- |
| `CYCLONE_TAIL_HEARTBEAT_S` | `15` | Idle heartbeat interval (seconds). Lower it for tests. |
## Inbox
`/inbox` is the working surface. Five lanes, dark by default (Ticker Tape
aesthetic):
- **Rejected** — claims whose 999 set-level response was R or E. Re-submit in bulk.
- **Payer-rejected** — claims whose 277CA STC category is A4, A6, or A7 (the payer
accepted the file but denied the claim). Stamped at 277CA ingest time and never
overwritten by a looser later 277CA.
- **Candidates** — remits whose CLP-claim-id didn't match exactly; each one shows its top scored claim. One-click manual match or dismiss.
- **Unmatched** — claims still waiting for a remit, and remits with no candidates above the threshold.
- **Done today** — terminal state transitions in the last 24 hours.
The page subscribes to the SP5 claim and remittance tail streams
(`/api/claims/stream` and `/api/remittances/stream`); a new `item`
event refetches the lane payload (debounced ~250ms). When a 999
parses and rejects claims, the inbox reflects the new
**Rejected** rows within a fraction of a second.
### Inbox endpoints
| Method | Path | Notes |
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
| GET | `/api/inbox/lanes` | All five lanes in one call. |
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
| POST | `/api/inbox/payer-rejected/acknowledge` | `{claim_ids: [...], actor: "..."}`. Bulk-acknowledge payer rejections. Idempotent. `200` with `transitioned` / `already_acked` / `not_found` / `not_rejected` counts. Writes an audit event per transition; never overwrites the underlying 277CA rejection. |
| GET | `/api/inbox/export.csv?lane=<lane>` | Streams CSV of the lane's rows. |
## Outbound 837 Serializer
The same canonical `ClaimOutput` model that powers 837 ingestion also
drives **outbound** 837P regeneration. Operators can take a claim
that's been parsed, edited, or rejected and emit a byte-faithful X12
837P file — closing the resubmit loop without leaving the app.
Two surfaces:
- **Single claim → .x12 download** from the ClaimDrawer header. Click
the **Download 837** icon, get a `claim-{id}.x12` attachment. The
file is a complete ISA/GS/ST envelope + hierarchy + CLM/SV1 lines,
regenerated from the canonical `ClaimOutput` fields (not by patching
`raw_segments`).
- **Multi-claim → .zip bundle** from the Inbox rejected lane. Select
N>1 rejected claims, hit **Resubmit**, choose **Resubmit + Download**
in the confirm modal, and the backend hands you
`resubmit-{N}-claims.zip` with one `.x12` per successfully
resubmitted claim. Each file gets a unique ISA13/GS06 so back-to-back
claims don't collide on control numbers.
Design spec at `docs/superpowers/specs/2026-06-20-cyclone-serialize-837-design.md`.
Implementation note: the spec proposed a hybrid "patch `raw_segments`"
approach, but `raw_segments` only captures post-CLM segments (CLM,
REF*G1, HI, LX, SV1 pairs) — the envelope and hierarchy segments
(NM1, N3, N4, HL, PRV, SBR, DMG) are not captured, so a pass-through
serializer cannot regenerate them byte-for-byte. The shipped
implementation rebuilds the entire file from canonical fields
("Approach A" in the plan). The round-trip guarantee still holds:
all 113 prodfiles in `docs/prodfiles/claims/*.x12` parse → serialize →
parse back to the same claim id.
### Where to find it
- Backend serializer: `backend/src/cyclone/parsers/serialize_837.py`
- Single-claim download API: `GET /api/claims/{id}/serialize-837`
- Bundle API: `POST /api/inbox/rejected/resubmit?download=true`
- Frontend helper: `src/lib/api.ts:serializeClaim837`,
`src/lib/inbox-api.ts:resubmitRejectedWithDownload`
- Browser download utility: `src/lib/download.ts` (`downloadTextFile`,
`downloadBlob`)
- UI:
- `src/components/ClaimDrawer/ClaimDrawerHeader.tsx` — Download icon
- `src/pages/Inbox.tsx` — Resubmit bundle modal + progress overlay
## Per-Line Adjustment Audit
Every 835 CAS segment is tied back to the specific 837 service line it
adjudicates, instead of being aggregated to the claim level. The match
runs **eagerly** at 835 ingest time so the audit trail is stable across
re-reads (a re-migration is required if the algorithm changes).
### Where to find it
- **ClaimDrawer → "Line Reconciliation" tab** — side-by-side view of the
837 SV1 lines and 835 SVC composites with per-line CAS reasons.
- **ClaimDrawer → ServiceLinesTable** — every billed line now shows its
paid amount and adjustment sum on the right.
- **RemitDrawer → CasAdjustmentsPanel** — each CAS row carries a
`proc · #N` reference; claim-level CAS clusters separately.
- **ClaimDrawer → MatchedRemitCard** — a `lines: N/M matched` badge
lights up amber when at least one line is unmatched.
### Match criteria (strict)
A 835 SVC composite matches a 837 SV1 line iff all four criteria align:
| Field | Rule |
| ------------- | ----------------------------------------------- |
| Procedure | Exact match (case-insensitive, uppercased). |
| Modifiers | Set-equal (order-independent). |
| Service date | Exact match (both null counts as a match). |
| Units | Exact match (both null counts as a match). |
Unmatched lines surface as a **soft warning** — the claim still posts to
PAID/PARTIAL/RECEIVED, but the Inbox shows the unmatched count and the
drawer surfaces a per-line "no 837 line matched" note.
### Line-level endpoints
| Method | Path | Notes |
| ------ | --------------------------------------------------- | -------------------------------------------------------------------------------------- |
| GET | `/api/claims/{claim_id}/line-reconciliation` | Dedicated view for the drawer tab. |
| GET | `/api/claims/{claim_id}` | Now includes a slim `lineReconciliation[]`. |
| GET | `/api/remittances/{remittance_id}` | Now includes `serviceLinePayments[]` + `claimLevelAdjustments[]`. |
| GET | `/api/inbox/lanes` | `matched_remittance` payload gains `matched_lines` + `total_lines`. |
### Scoring
4-field weighted (sum = 100):
| Field | Weight | Rule |
| ----------------------- | ------ | ------------------------------------------------------------- |
| Patient control number | 40 | Exact match, normalized (case + leading zeros). |
| Service date | 25 | Linear decay over ±3 days. |
| Charge amount | 20 | Linear decay over ±10%. |
| Provider NPI | 15 | Exact match. Missing → 0. |
Tiers: **strong** (≥75, full opacity, Match enabled), **weak**
(50–74, dimmed), **hidden** (<50, not surfaced).
## Multi-Payer, Multi-NPI & Clearhouse
The payer and provider identity that used to live as a single hard-coded
`PayerConfig` dict in the backend is now data, not code. Three new tables
plus a YAML file drive the entire configuration:
- **`providers` table** — one row per billing-provider NPI (Montrose
`1881068062`, Delta `1851446637`, Salida `1467507269`). All three share
the same `TOC, Inc.` legal name, tax ID `721587149`, and taxonomy
`251E00000X`. Outbound 837 files pick the right `BillingProvider` by
NPI; `claim.party.npi` is now a foreign key into `providers`.
- **`payers` table** — one row per payer (`CO_TXIX`, …) with its
receiver identity (NM1*40 / ISA08 / GS03).
- **`payer_configs` join table** — one row per `(payer_id, transaction_type)`
pair. 837P and 835 can carry different `BHT06`, SBR defaults, and
allowed status codes per payer.
- **`clearhouse` single-row config** — dzinesco's identity: TPID
`11525703`, submitter name, MT-clock file-naming block, SFTP block.
- **`config/payers.yaml`** — the on-disk source for everything above,
schema-validated at boot against a Pydantic model. A typo or missing
field fails the boot with a precise error. The original in-code
`PAYER_FACTORIES` dict is kept as a fallback for ad-hoc testing.
### Config + clearhouse endpoints
| Method | Path | Notes |
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
| GET | `/api/clearhouse` | The `clearhouse` singleton (name, TPID, file/SFTP blocks). |
| POST | `/api/clearhouse/submit` | Push a batch of generated 837 files via SFTP (see SFTP section). |
| GET | `/api/config/providers` | All providers. |
| GET | `/api/config/providers/{npi}` | One provider. |
| GET | `/api/config/payers` | All payers. |
| GET | `/api/config/payers/{payer_id}/configs` | All `(payer_id, transaction_type)` configs for one payer. |
| POST | `/api/admin/reload-config` | Re-read `config/payers.yaml` and refresh the in-process cache. |
## 277CA Claim Acknowledgment
A 277CA (`005010X214`) is the per-claim acknowledgment CMS and
Colorado Medicaid rely on: the file was syntactically valid *and* each
named claim was accepted, pended, or rejected by the payer at the claim
level. It is distinct from a 999 (file-level) and a TA1 (envelope-level).
Cyclone ingests 277CA files the same way it ingests 999 / 835 — drop the
file on the Upload page or `POST /api/parse-277ca` — and stamps every
claim whose `STC` category is `A4`, `A6`, or `A7` with a non-null
`payer_rejected_at` + `payer_rejected_reason` + originating 277CA row id.
| Method | Path | Notes |
| ------ | -------------------------- | ---------------------------------------------------------------- |
| POST | `/api/parse-277ca` | Upload a 277CA, persist the parsed status rows. |
| GET | `/api/277ca-acks` | List 277CA acks (filterable by date / payer). |
| GET | `/api/277ca-acks/{id}` | One 277CA ack with its per-claim status rows + regenerated text. |
The `payer_rejected` stamp is **monotonic**: a later 277CA with a looser
status set cannot clear a previous rejection. The Payer-Rejected inbox
lane surfaces every claim with a non-null `payer_rejected_at` — it is
distinct from the 999 `Rejected` lane (envelope reject) and they can
both be true for the same claim.
## Tamper-Evident Audit Log
The `audit_log` table is the canonical record of every state transition
the system has ever observed — claim lifecycle, reconciliation
decisions, config reloads, SFTP submissions, 277CA rejects, Payer-Rejected
acknowledgements, SQLCipher key rotations. SP11 made it tamper-evident:
every row carries a SHA-256 hash of
`(prev_hash || row_payload)`, forming a chain back to a genesis row.
Any `INSERT`, `UPDATE`, or `DELETE` that breaks the chain is detectable
in a single walk.
| Method | Path | Notes |
| ------ | --------------------------------- | -------------------------------------------------------------- |
| GET | `/api/admin/audit-log` | Paginated audit log (filterable by event type / actor / date). |
| GET | `/api/admin/audit-log/verify` | Walk the chain; return the first broken link, or `{ok: true}`. |
`verify_chain` is the integrity check that backs the audit promise —
it is intentionally cheap (one indexed walk) and intentionally
side-effect-free so a scheduler can run it on a cron and alert on any
non-`{ok: true}` result. Chain verification is **not** access-gated
beyond the same `127.0.0.1` bind the rest of the API uses; for a
hostile multi-operator deployment, wrap the route in your reverse proxy.
## Encryption at Rest
When the macOS Keychain carries an entry at service `cyclone`, account
`cyclone.db.key`, and the optional `sqlcipher3` Python package is
installed, the SQLite file at `~/.local/share/cyclone/cyclone.db` is
opened with SQLCipher (AES-256). The key is read from the Keychain
once at process start, applied via a SQLAlchemy `connect` event so
every connection — including migrations and tests — gets the same
`PRAGMA key`. The key is never written to disk or to a Python global.
When the Keychain entry is missing **or** `sqlcipher3` is not
installed, the DB falls back to plain SQLite. The intent is a graceful
default for developers and CI; the production posture is that every
operator has created the Keychain entry on first run. See
[docs/reference/co-medicaid.md §Keychain setup](docs/reference/co-medicaid.md)
for the one-time setup recipe and the HIPAA Security Rule §164.312(a)(2)(iv)
mapping.
### Key rotation
The DB encryption key is rotated in place via `POST /api/admin/db/rotate-key`.
The handler:
1. Generates a fresh 256-bit CSPRNG key (`db_crypto.generate_db_key()`).
2. Persists the new key to the Keychain under the same
`cyclone.db.key` account (overwriting the old one).
3. Disposes the SQLAlchemy engine so pooled connections release the
file (SQLCipher refuses to `PRAGMA rekey` while another connection
holds the DB).
4. Opens with the old key, issues `PRAGMA rekey`, and verifies the
schema survived (table-count sanity check).
5. Rebuilds the engine with the new key.
6. Writes a `db.key_rotated` audit event carrying the SHA-256
fingerprints of the old and new keys (first 8 hex chars) plus the
post-rotation `table_count`.
When SQLCipher is enabled the engine uses SQLAlchemy's `NullPool`
instead of the default `QueuePool`. `QueuePool` returns connections to
a shared queue that any thread can pull from, which breaks SQLCipher's
thread affinity. `NullPool` trades connection reuse for thread safety
— the only correct behavior under FastAPI's per-request threadpool.
The rotation endpoint is serialized with a module-level
`threading.Lock` (one rotation in flight at a time), returns `409` if
a rotation is already running, `400` if encryption is not enabled,
and `503` with a `reason` on `PRAGMA rekey` or Keychain failure so the
operator can take the next step without parsing the traceback.
| Method | Path | Notes |
| ------ | --------------------------------- | -------------------------------------------------------------- |
| POST | `/api/admin/db/rotate-key` | Rotate the SQLCipher key in place. Audit-logged. |
## SFTP Wire-Up (paramiko)
The `clearhouse.submit` endpoint uses `paramiko` to push a batch of
generated 837 files to the dzinesco SFTP server
(`mft.gainwelltechnologies.com:22`, path
`/CO XIX/PROD/coxix_prod_11525703/FromHPE`). The SFTP credential is
fetched from the macOS Keychain at call time — never read from YAML,
never logged, never written to disk. The wire-up honors the file-naming
template stored in the `clearhouse` config:
```
outbound: {tpid}-{tx}-{ts_mt}-1of1.{ext} e.g. 11525703-837P-20260620181814559-1of1.txt
inbound: TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
```
where `{ts_mt}` is a 17-digit `yyyymmddhhmmssSSS` Mountain Time stamp.
Inbound filenames are routed by `<FileType>` and `<OrigTx>` to the
matching parser (`999`, `TA1`, `271`, `277`, `277CA`, `835`).
The `SftpClient` interface is the same one the SP9 stub used — the swap
was a one-file change (`sftp_paramiko.py` replacing `sftp_stub.py`).
`paramiko` is an optional dependency; the stub remains the default when
the `paramiko` extras aren't installed so the test suite stays green on
Linux dev boxes.
## Batches, Reconciliation, and Activity
These read/write endpoints are the core operator surface for browsing
the parsed record and acting on matches. They predate the per-SP
endpoint reference sections in the **Roadmap** and are listed here in
one place so the route inventory stays discoverable.
| Method | Path | Notes |
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
| GET | `/api/batches` | All parsed batches, newest first. `?limit=` (1–1000, default 100). |
| GET | `/api/batches/{batch_id}` | One batch detail (envelope, claims / remits, validation summary). |
| GET | `/api/batch-diff?a=<id>&b=<id>` | Side-by-side diff of two batches: `added` / `removed` / `changed` claims + envelope metadata for each. Both query params required. |
| GET | `/api/reconciliation/unmatched` | `{"claims": [...], "remittances": [...]}` — every Claim with no paired Remittance and vice versa. The two lists are always present (empty list, never absent) so the UI can index unconditionally. |
| POST | `/api/reconciliation/match` | Body `{claim_id, remit_id}`. Manually pair a claim with a remit. `400` on missing ids, `404` on unknown id, `409` on already-matched or terminal-state claim. |
| POST | `/api/reconciliation/unmatch` | Body `{claim_id}`. Remove the current match and reset the claim to `submitted`. `404` on unknown id, `409` on no current match. |
| GET | `/api/providers` | Distinct providers across parsed claims. `?npi=`, `?state=`, `?limit=`, `?offset=`. Distinct from `/api/config/providers/{npi}` (SP9 config table) — this endpoint surfaces providers derived from the parsed claim stream. |
| GET | `/api/activity` | Recent activity events. `?kind=`, `?since=`, `?limit=` (1–500, default 200). Powers the Activity page; the streaming counterpart `/api/activity/stream` is documented under **Live updates**. |
The per-SP endpoint blocks at the bottom of the **Roadmap** cover the
SP-specific routes (parse, 999/TA1/277CA, Inbox, claim drawer, line
reconciliation, outbound 837, multi-payer config, audit log, 277CA,
key rotation, payer-rejected acknowledge).
## Persistence
Parsed batches, claims, remittances, matches, 277CA rejections,
hash-chained audit log entries, and SFTP submission history are stored
in a SQLite file at `~/.local/share/cyclone/cyclone.db` by default.
The directory is auto-created on first run. The DB is optionally
encrypted with SQLCipher — see
[Encryption at Rest](#encryption-at-rest) for the Keychain-driven
setup.
To use a different location, set `CYCLONE_DB_URL`:
```bash
export CYCLONE_DB_URL=sqlite:///path/to/cyclone.db
# or
export CYCLONE_DB_URL=postgresql://user:pass@host:5432/cyclone
```
### Backup
```bash
sqlite3 ~/.local/share/cyclone/cyclone.db ".backup /path/to/backup.db"
```
This is safe to run while the backend is running (uses SQLite's online
backup API).
## Project layout
```
.
├── backend/
│ ├── src/cyclone/
│ │ ├── api.py # FastAPI app + parse routes; mounts api_routers/* sub-apps
│ │ ├── api_helpers.py # NDJSON / content-negotiation / live-tail helpers
│ │ ├── api_routers/ # FastAPI APIRouters extracted from api.py
│ │ │ ├── health.py # GET /api/health
│ │ │ ├── acks.py # GET /api/acks, /api/acks/{id}
│ │ │ ├── ta1_acks.py # GET /api/ta1-acks, /api/ta1-acks/{id}
│ │ │ ├── activity.py # GET /api/activity, /api/activity/stream
│ │ │ ├── batches.py # GET /api/batches, /api/batches/{id}, /api/batch-diff
│ │ │ ├── providers.py # GET /api/providers
│ │ │ ├── clearhouse.py # GET /api/clearhouse, POST /api/clearhouse/submit
│ │ │ └── config.py # /api/config/providers[/...], /api/config/payers[/...], /api/admin/reload-config
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
│ │ ├── store.py # CycloneStore, mappers, publish-on-write
│ │ ├── db.py # SQLAlchemy engine, session factory, ORM models
│ │ ├── db_migrate.py # PRAGMA user_version migration runner
│ │ ├── db_crypto.py # optional SQLCipher encryption at rest (SP12)
│ │ ├── audit_log.py # tamper-evident hash-chained audit_log (SP11)
│ │ ├── inbox_lanes.py # rejected / payer_rejected / candidates / unmatched / done_today
│ │ ├── inbox_state.py # 999 envelope reject → claim state transitions
│ │ ├── inbox_state_277ca.py # 277CA STC A4/A6/A7 → payer_rejected stamp (SP10)
│ │ ├── providers.py # multi-NPI provider lookups (SP9)
│ │ ├── payers.py # payer / payer_config lookups (SP9)
│ │ ├── secrets.py # macOS Keychain-backed secret fetcher
│ │ ├── reconcile.py # pure-function 835→claim match + line-level match
│ │ ├── __main__.py # `python -m cyclone serve`
│ │ ├── cli.py # click CLI
│ │ └── parsers/ # X12 tokenizer, models, validator, writers, 277CA, 999, TA1, 270, 271
│ └── tests/
│ ├── fixtures/ # co_medicaid_*.txt, minimal_*.txt
│ ├── test_api.py # parse-837/835 round-trip
│ ├── test_api_gets.py # 6 GET endpoints
│ ├── test_store.py # store + mappers + iterators
│ ├── test_api_streaming.py
│ ├── test_api_stream_live.py # 3 live-tail endpoints + disconnect cleanup
│ ├── test_pubsub.py # EventBus + subscribe/unsubscribe
│ ├── test_api_parse_persists.py
│ ├── test_db.py / test_db_crypto.py / test_db_migrate.py
│ ├── test_audit_log.py
│ ├── test_inbox_lanes.py / test_inbox_state.py
│ ├── test_apply_277ca_rejections.py
│ ├── test_sftp_stub.py / test_sftp_paramiko.py
│ └── test_providers_seed.py / test_payer_config_loading.py
├── src/ # React + Vite + TypeScript UI
│ ├── components/
│ │ ├── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, …
│ │ └── TailStatusPill.tsx # live-tail status badge + reconnect button
│ ├── pages/ # Claims, Remittances, Providers, Acks, Activity, Upload, Inbox, …
│ ├── hooks/ # useBatches, useClaims, useRemittances, useProviders, useActivity, useParse
│ │ # + useTailStream, useMergedTail (live tail)
│ ├── lib/ # api.ts, format.ts, utils.ts
│ │ # + tail-stream.ts (NDJSON parser)
│ ├── store/ # zustand sample-data + parsed-batches store
│ │ # + tail-store.ts (FIFO-capped live tail slices)
│ └── types/ # shared TS types
├── config/
│ └── payers.yaml # YAML-driven payer + clearhouse config (SP9)
├── docs/
│ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes (incl. Keychain setup)
│ ├── reviews/ # post-SP completeness reviews
│ ├── superpowers/plans/ # implementation plans
│ └── superpowers/specs/ # design specs (incl. SP9-SP13)
├── tailwind.config.js # shimmer, scan, row-flash keyframes
└── package.json
```
## Roadmap
Sub-projects 2 through 15 are **shipped**. See the [completeness
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
the honest gap analysis against the industry definition of a HIPAA
clearinghouse — the short version is that the local-only,
single-operator, single-payer design contract is honored, and the
items that would be needed to expand that contract (AS2/AS4, SNIP 1–7,
HITRUST, 276/277 status, 278 referrals, COB) are intentionally out of
scope.
Shipped sub-projects (most recent first):
- **Sub-project 15 (shipped) — SQLCipher key rotation.** In-place
rotation via `PRAGMA rekey`, serialized through a module-level
`threading.Lock` and a SQLAlchemy `NullPool` to keep SQLCipher
thread-affine under FastAPI's per-request threadpool. Writes a
`db.key_rotated` audit event with old + new key fingerprints and
post-rotation `table_count`. See
[Encryption at Rest — Key rotation](#key-rotation).
- **Sub-project 14 (shipped) — 5-lane Inbox UI.** The Payer-Rejected
lane is now rendered in the Inbox alongside Rejected / Candidates /
Unmatched / Done today. New bulk action
`POST /api/inbox/payer-rejected/acknowledge` drops claims from the
working surface without erasing the original 277CA rejection event
(audit log stays intact, SP11).
- **Sub-project 13 (shipped) — SFTP wire-up.** `paramiko`-backed
`SftpClient` replaces the SP9 stub. The clearhouse.submit endpoint
actually pushes to
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
SFTP credentials are read from the macOS Keychain at call time.
- **Sub-project 12 (shipped) — Encryption at rest.** Optional
SQLCipher AES-256 encryption of the SQLite file, with the key
fetched from the macOS Keychain. Falls back to plain SQLite when
the Keychain entry is missing or `sqlcipher3` isn't installed.
- **Sub-project 11 (shipped) — Tamper-evident audit log.** Every
`audit_log` row carries a SHA-256 hash chained to the previous row;
a single walk via `GET /api/admin/audit-log/verify` detects any
break.
- **Sub-project 10 (shipped) — 277CA + Payer-Rejected lane.** Inbound
277CA parser + a new Payer-Rejected inbox lane distinct from the
999-envelope Rejected lane. The rejection stamp is monotonic.
- **Sub-project 9 (shipped) — Multi-payer, multi-NPI, SFTP stub.** The
in-code `PAYER_FACTORIES` dict is replaced by a `config/payers.yaml`
+ 3 new DB tables (`providers`, `payers`, `payer_configs`) +
`clearhouse` singleton. Added a `POST /api/clearhouse/submit` stub
that writes to a local `staging_dir` — swapped for real `paramiko`
in SP13.
- **Sub-project 8 (shipped) — Outbound 837P serializer.** Closes the
resubmit loop: rejected claims can be regenerated back to an X12 837P
file (single download from the claim drawer, or ZIP bundle from the
inbox rejected lane). See the "Outbound 837 Serializer" section
below for details.
- **Sub-project 3 (shipped) — More 837P/835 features.**
- **837P validation rules:** R034 enforces `REF*G1` on frequency-code
7/8 claims; R035 enforces `BHT06` transaction-type-code is in the
allowed set per payer config.
- **835 CAS deep-parsing:** every CAS adjustment now surfaces with its
CARC reason code + label (e.g. `CO-29: The time limit for filing has
expired`), surfaced via `GET /api/remittances/{id}` and rendered as
an expansion row on the Remittances page.
- **999 ACK transaction set:** full inbound parser + outbound
serializer. Auto-generated on 837 ingest when `?ack=true` is passed
to `POST /api/parse-837`. Persisted to the `acks` table, browsable
on a new `/acks` page, and downloadable as the regenerated raw
999 text.
- **270/271 eligibility (API-only):** `POST /api/eligibility/request`
builds a 270 from a JSON payload (subscriber, provider, payer,
service type); `POST /api/eligibility/parse-271` ingests the
response and returns structured `coverage_benefits`.
- **TA1 interchange ACK:** parser + persistence + API for the
envelope-level TA1 acknowledgment (separate from the 999
transaction-set ACK).
- **Sub-project 4 (shipped) — Frontend features.**
- **Per-claim detail drawer:** click any row on the Claims page to
open a side-panel drawer with the full claim context — state +
amount header, validation panel, service lines, diagnoses, parties
(billing provider, subscriber, payer), raw X12 segments, matched
remittance summary, and a vertical state-history timeline. URL is
synced (`?claim=...`) so links and back-button restore the drawer.
Keyboard nav: `j`/`k` move between claims, `esc` closes, `?` opens
the cheatsheet overlay. Skeleton + error + not-found (404) states
are all distinct. Powered by a new backend endpoint
`GET /api/claims/{claim_id}` that returns the full drawer payload
in a single round-trip.
- **Remit drawer:** mirror of the claim drawer for remittances —
payer + provider header, financial summary, claim-payments table,
CAS adjustments panel, parties grid, raw X12 segments. URL is
synced (`?remit=...`).
- **Batch diff view:** side-by-side comparison of two batches
(added / removed / changed claims) reachable from the Batches
page. Powers the `BatchDetail` and `BatchDiff` views.
- **Global Cmd-K search:** cross-resource search across claims,
remittances, and activity; opens from anywhere via the keyboard
shortcut.
- **CSV export:** per-page export buttons (claims, remittances, lanes)
that stream CSV directly from the backend.
- **Accessibility + print styles:** keyboard navigation focus rings,
`aria` roles, and `@media print` stylesheets for paper-friendly
output.
- **Sub-project 5 (shipped) — Live updates.**
- The Claims, Remittances, and Activity pages stream new rows in
real time over Server-Sent Events encoded as newline-delimited
JSON. The backend publishes a `claim_written` /
`remittance_written` / `activity_recorded` event on every store
write; the frontend opens an HTTP stream and dispatches each
event into a per-resource Zustand store that the page reads on
top of its base query results.
- A small status pill in each toolbar surfaces the connection
state — `live` / `connecting` / `reconnecting` / `stalled` /
`error` / `closed` — and offers a manual **↻ Reconnect** button
on `stalled` or `error`. Stale connections (no event for 30s,
heartbeat included) flip to `stalled` automatically; the
back-off ladder on errors is `1s → 2s → 4s → 8s → 16s → 30s`
capped. See the "Live updates" section below for details.
- **Sub-project 6 (shipped) — Inbox workflow automation.**
- **Ticker-Tape inbox (`/inbox`):** five lanes ordered by urgency —
**Rejected** (claims whose 999 rejected them), **Payer-rejected**
(claims whose 277CA denied them — added in SP10), **Candidates**
(remits that didn't auto-match a claim), **Unmatched** (claims
still waiting for a remit), and **Done today** (terminal
transitions in the last 24 hours). The page subscribes to the
claim and remittance tail streams so new rejected claims land in
the Rejected lane within a fraction of a second.
- **4-field weighted scoring:** every candidate remit is scored on
patient control number (40), service date (25), charge amount
(20), and provider NPI (15). Tiers: strong (≥75, Match enabled),
weak (50–74, dimmed), hidden (<50).
- **Bulk actions:** multi-select rows in any lane to dismiss
candidates, resubmit rejected claims, or export the lane as CSV
in a single round-trip.
- **Sub-project 7 (shipped) — Per-line adjustment audit.**
- Every 835 CAS segment is tied back to the specific 837 service
line it adjudicates, instead of being aggregated to the claim
level. The match runs **eagerly** at 835 ingest time so the audit
trail is stable across re-reads.
- New endpoints surface the per-line projection: `GET
/api/claims/{id}/line-reconciliation` for the dedicated tab in
the ClaimDrawer; the existing claim + remit detail endpoints
gained slim `lineReconciliation[]` / `serviceLinePayments[]` /
`claimLevelAdjustments[]` fields; the inbox lanes include
`matched_lines` / `total_lines` badges on matched remittances.
- The ClaimDrawer gains a **Line Reconciliation** tab (alongside
Details) that pairs every 837 SV1 line with its 835 SVC composite
and surfaces per-line CAS reasons + unmatched-line warnings.
- See the "Per-Line Adjustment Audit" section below for details.
### SP3 endpoints
- `POST /api/parse-837?ack=true` — existing 837 parse, plus optional
auto-generated 999 ACK persisted alongside the batch.
- `POST /api/parse-999` — parse an inbound 999 ACK and persist it.
- `GET /api/acks` — list ACKs.
- `GET /api/acks/{id}` — ACK detail, including the regenerated
`raw_999_text`.
- `POST /api/parse-ta1` — parse an inbound TA1 envelope ACK and persist it.
- `GET /api/ta1-acks` — list TA1 acks.
- `GET /api/ta1-acks/{id}` — TA1 ack detail (envelope control segments
+ the parser's accept/reject verdict).
- `POST /api/eligibility/request` — build a 270 from JSON.
- `POST /api/eligibility/parse-271` — ingest a 271 and return parsed
coverage benefits.
The UI gains a new **Acks** page (sidebar entry) that lists persisted
ACKs and lets you download the regenerated 999 text.
### SP4 endpoints (claim drawer)
- `GET /api/claims/{claim_id}` — full claim context for the drawer
payload: header fields, diagnoses, service lines, parties
(billing provider / subscriber / payer), validation result, raw
X12 segments, matched remittance summary (or `null`), and the
claim's state history (ordered newest-first). 404 with a structured
`{ "error": "Not found", "detail": "Claim {id} not found" }` body
when the claim doesn't exist — distinct from a transient fetch
failure so the UI can render a dedicated not-found state.
### SP5 endpoints (live updates)
- `GET /api/claims/stream` — NDJSON stream of claim events. Emits a
snapshot (filtered by the same query params as `GET /api/claims`),
then `snapshot_end`, then live `claim_written` events as they
arrive, with a 15s idle heartbeat.
- `GET /api/remittances/stream` — same shape for remittances
(subscribes to `remittance_written`, default sort `-received_date`).
- `GET /api/activity/stream` — same shape for activity events
(subscribes to `activity_recorded`, default `limit=50`).
### SP6 endpoints (inbox)
- `GET /api/inbox/lanes` — all five lanes (rejected, payer_rejected,
candidates, unmatched, done_today) in a single round-trip, with
row-level scoring and matched-remit context.
- `POST /api/inbox/candidates/{remit_id}/match` — manual match of a
candidate remit to one of its scored claims; `409` if the claim
state moved out from under us.
- `POST /api/inbox/candidates/dismiss` — `{pairs: [{claim_id,
remit_id}]}`. Session-scoped, no DB persistence.
- `POST /api/inbox/rejected/resubmit` — `{claim_ids: [...]}`. `200`
with `conflicts` for non-rejected claims in the batch.
- `GET /api/inbox/export.csv?lane=<lane>` — streams CSV of the
lane's rows.
### SP7 endpoints (per-line reconciliation)
- `GET /api/claims/{id}/line-reconciliation` — dedicated view for
the ClaimDrawer's Line Reconciliation tab; returns the slim
per-line projection (`summary` + `lines[]`).
- `GET /api/claims/{id}` — now includes a slim `lineReconciliation[]`
so the ServiceLinesTable can show paid + adjustments columns
without a second fetch.
- `GET /api/remittances/{id}` — now includes `serviceLinePayments[]`
+ `claimLevelAdjustments[]`; the CAS panel renders per-line + claim
level separately.
- `GET /api/inbox/lanes` — `matched_remittance` payload gained
`matched_lines` + `total_lines` so the MatchedRemitCard can badge
partial matches.
### SP8 endpoints (outbound 837P)
- `GET /api/claims/{id}/serialize-837` — regenerate the persisted
claim as a byte-faithful X12 837P file (`text/x12` body,
`Content-Disposition: attachment; filename="claim-{id}.x12"`).
404 if the claim doesn't exist, 422 if the stored `raw_json`
can't be revalidated as a `ClaimOutput`. Drives the **Download 837**
icon in the ClaimDrawer header.
- `POST /api/inbox/rejected/resubmit?download=true` — same mass-resubmit
endpoint as SP6, but the response is a ZIP archive (`application/zip`)
containing one `claim-{id}.x12` per successfully resubmitted claim.
Each file uses `serialize_837_for_resubmit` so back-to-back files in
the bundle get unique ISA13/GS06 control numbers (back-to-back
resubmits would otherwise all share `000000001`). Conflicts and
missing ids are deliberately omitted from the ZIP — the user already
saw them in the JSON response on prior calls. Per-claim regenerate
failures (rare — usually means `raw_json` is corrupted for one claim)
are surfaced via the `X-Cyclone-Serialize-Errors` response header
(JSON-encoded array) so the UI can show "10 resubmitted, 2 couldn't
be regenerated" without parsing the binary. Drives the **Resubmit +
Download** button in the Inbox rejected-lane BulkBar (N>1 modal
prompt).
### SP9 endpoints (multi-payer, multi-NPI, SFTP stub)
- `GET /api/clearhouse` — the `clearhouse` singleton (name, TPID,
file-naming block, SFTP block).
- `POST /api/clearhouse/submit` — push a batch of generated 837 files.
The SP9 implementation writes to a local `staging_dir`; the SP13
swap replaces the write with a real `paramiko` SFTP push without
changing the route shape.
- `GET /api/config/providers` and `GET /api/config/providers/{npi}` —
list / fetch providers from the new `providers` table.
- `GET /api/config/payers` and
`GET /api/config/payers/{payer_id}/configs` — list payers; for a
given payer, return the per-transaction-type `payer_configs` rows.
- `POST /api/admin/reload-config` — re-read `config/payers.yaml` and
refresh the in-process cache without a server restart.
### SP10 endpoints (277CA + Payer-Rejected lane)
- `POST /api/parse-277ca` — upload a 277CA file; persist the parsed
`ClaimStatus` rows and stamp the matching claims with
`payer_rejected_at` (monotonic, never overwritten by `NULL`).
- `GET /api/277ca-acks` — list 277CA acks.
- `GET /api/277ca-acks/{id}` — one 277CA ack with its per-claim
`ClaimStatus` rows + regenerated text.
- `GET /api/inbox/lanes` — the response now also carries a
`payer_rejected` lane populated from
`Claim.payer_rejected_at IS NOT NULL`.
### SP11 endpoints (tamper-evident audit log)
- `GET /api/admin/audit-log` — paginated audit log. Each row carries
`(id, prev_hash, row_hash, event_type, actor, payload_json,
created_at)` where `row_hash = sha256(prev_hash || canonical_json(payload))`.
- `GET /api/admin/audit-log/verify` — walk the chain in insertion
order; return `{ok: true}` or the first `{id, expected, got}`
mismatch. The walk is O(n) with one indexed lookup per row.
### SP12 (encryption at rest — no new routes)
SP12 introduces no API routes. The `cyclone.db.key` Keychain entry +
optional `sqlcipher3` dependency enable AES-256 encryption transparently
on the next connection. See [Encryption at Rest](#encryption-at-rest)
and [docs/reference/co-medicaid.md](docs/reference/co-medicaid.md) for
the one-time setup recipe.
### SP13 endpoints (paramiko SFTP)
- `POST /api/clearhouse/submit` — same endpoint as SP9; the
implementation is now a real `paramiko` `SftpClient.write` to
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
SFTP credentials are fetched from the macOS Keychain at call time.
### SP14 endpoints (5-lane Inbox UI + acknowledge)
- `GET /api/inbox/lanes` — same shape as SP6; the `payer_rejected`
lane payload now also includes
`payer_rejected_acknowledged_at` + `payer_rejected_acknowledged_actor`
per row so the UI can badge acknowledged claims (forward-compat for
a future "Recently acknowledged" view).
- `POST /api/inbox/payer-rejected/acknowledge` — bulk-acknowledge
Payer-Rejected claims. Body: `{claim_ids: [...], actor: "..."}`.
Idempotent. Response: `200` with
`{transitioned, already_acked, not_found, not_rejected}`. Writes an
`inbox.payer_rejected_acknowledged` audit event per transition.
Acknowledgement hides a claim from the lane but never overwrites
`payer_rejected_at` / `payer_rejected_reason` /
`payer_rejected_by_277ca_id` — the original 277CA evidence stays
intact in the audit log.
### SP15 endpoints (SQLCipher key rotation)
- `POST /api/admin/db/rotate-key` — rotate the SQLCipher DB key in
place. Generates a fresh 256-bit key, writes it to the Keychain
(overwriting `cyclone.db.key`), disposes the engine, issues
`PRAGMA rekey`, verifies the schema, rebuilds the engine. Writes
a `db.key_rotated` audit event with old + new key fingerprints
and `table_count`. Returns `409` when a rotation is already in
flight, `400` when encryption is not enabled, `503` with a
`reason` on `PRAGMA rekey` or Keychain failure.
## License
No license file yet; this is internal-use software. Add a `LICENSE` file
when one is decided.
Reference in New Issue View Git Blame Copy Permalink