tyler/cyclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: sync READMEs with SP14 (Payer-Rejected lane) + endpoint inventory

Browse Source
This commit is contained in:
Tyler
2026-06-21 00:36:33 -06:00
parent ab00909715
commit d4f6fdd49c
2 changed files with 279 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+260 -18
View File
@@ -130,10 +130,13 @@ backoff schedule on error is `1s → 2s → 4s → 8s → 16s → 30s` capped.
## Inbox
`/inbox` is the working surface. Four lanes, dark by default (Ticker Tape
`/inbox` is the working surface. Five lanes, dark by default (Ticker Tape
aesthetic):
- **Rejected** — claims whose 999 set-level response was R or E. Re-submit in bulk.
- **Payer-rejected** — claims whose 277CA STC category is A4, A6, or A7 (the payer
accepted the file but denied the claim). Stamped at 277CA ingest time and never
overwritten by a looser later 277CA.
- **Candidates** — remits whose CLP-claim-id didn't match exactly; each one shows its top scored claim. One-click manual match or dismiss.
- **Unmatched** — claims still waiting for a remit, and remits with no candidates above the threshold.
- **Done today** — terminal state transitions in the last 24 hours.
@@ -148,7 +151,7 @@ parses and rejects claims, the inbox reflects the new
| Method | Path | Notes |
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
| GET | `/api/inbox/lanes` | All four lanes in one call. |
| GET | `/api/inbox/lanes` | All five lanes in one call. |
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
@@ -255,11 +258,139 @@ drawer surfaces a per-line "no 837 line matched" note.
Tiers: **strong** (≥75, full opacity, Match enabled), **weak**
(50–74, dimmed), **hidden** (<50, not surfaced).
## Multi-Payer, Multi-NPI & Clearhouse
The payer and provider identity that used to live as a single hard-coded
`PayerConfig` dict in the backend is now data, not code. Three new tables
plus a YAML file drive the entire configuration:
- **`providers` table** — one row per billing-provider NPI (Montrose
`1881068062`, Delta `1851446637`, Salida `1467507269`). All three share
the same `TOC, Inc.` legal name, tax ID `721587149`, and taxonomy
`251E00000X`. Outbound 837 files pick the right `BillingProvider` by
NPI; `claim.party.npi` is now a foreign key into `providers`.
- **`payers` table** — one row per payer (`CO_TXIX`, …) with its
receiver identity (NM1*40 / ISA08 / GS03).
- **`payer_configs` join table** — one row per `(payer_id, transaction_type)`
pair. 837P and 835 can carry different `BHT06`, SBR defaults, and
allowed status codes per payer.
- **`clearhouse` single-row config** — dzinesco's identity: TPID
`11525703`, submitter name, MT-clock file-naming block, SFTP block.
- **`config/payers.yaml`** — the on-disk source for everything above,
schema-validated at boot against a Pydantic model. A typo or missing
field fails the boot with a precise error. The original in-code
`PAYER_FACTORIES` dict is kept as a fallback for ad-hoc testing.
### Config + clearhouse endpoints
| Method | Path | Notes |
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
| GET | `/api/clearhouse` | The `clearhouse` singleton (name, TPID, file/SFTP blocks). |
| POST | `/api/clearhouse/submit` | Push a batch of generated 837 files via SFTP (see SFTP section). |
| GET | `/api/config/providers` | All providers. |
| GET | `/api/config/providers/{npi}` | One provider. |
| GET | `/api/config/payers` | All payers. |
| GET | `/api/config/payers/{payer_id}/configs` | All `(payer_id, transaction_type)` configs for one payer. |
| POST | `/api/admin/reload-config` | Re-read `config/payers.yaml` and refresh the in-process cache. |
## 277CA Claim Acknowledgment
A 277CA (`005010X214`) is the per-claim acknowledgment CMS and
Colorado Medicaid rely on: the file was syntactically valid *and* each
named claim was accepted, pended, or rejected by the payer at the claim
level. It is distinct from a 999 (file-level) and a TA1 (envelope-level).
Cyclone ingests 277CA files the same way it ingests 999 / 835 — drop the
file on the Upload page or `POST /api/parse-277ca` — and stamps every
claim whose `STC` category is `A4`, `A6`, or `A7` with a non-null
`payer_rejected_at` + `payer_rejected_reason` + originating 277CA row id.
| Method | Path | Notes |
| ------ | -------------------------- | ---------------------------------------------------------------- |
| POST | `/api/parse-277ca` | Upload a 277CA, persist the parsed status rows. |
| GET | `/api/277ca-acks` | List 277CA acks (filterable by date / payer). |
| GET | `/api/277ca-acks/{id}` | One 277CA ack with its per-claim status rows + regenerated text. |
The `payer_rejected` stamp is **monotonic**: a later 277CA with a looser
status set cannot clear a previous rejection. The Payer-Rejected inbox
lane surfaces every claim with a non-null `payer_rejected_at` — it is
distinct from the 999 `Rejected` lane (envelope reject) and they can
both be true for the same claim.
## Tamper-Evident Audit Log
The `audit_log` table is the canonical record of every state transition
the system has ever observed — claim lifecycle, reconciliation
decisions, config reloads, SFTP submissions, 277CA rejects. SP11 made
it tamper-evident: every row carries a SHA-256 hash of
`(prev_hash || row_payload)`, forming a chain back to a genesis row.
Any `INSERT`, `UPDATE`, or `DELETE` that breaks the chain is detectable
in a single walk.
| Method | Path | Notes |
| ------ | --------------------------------- | -------------------------------------------------------------- |
| GET | `/api/admin/audit-log` | Paginated audit log (filterable by event type / actor / date). |
| GET | `/api/admin/audit-log/verify` | Walk the chain; return the first broken link, or `{ok: true}`. |
`verify_chain` is the integrity check that backs the audit promise —
it is intentionally cheap (one indexed walk) and intentionally
side-effect-free so a scheduler can run it on a cron and alert on any
non-`{ok: true}` result. Chain verification is **not** access-gated
beyond the same `127.0.0.1` bind the rest of the API uses; for a
hostile multi-operator deployment, wrap the route in your reverse proxy.
## Encryption at Rest
When the macOS Keychain carries an entry at service `cyclone`, account
`cyclone.db.key`, and the optional `sqlcipher3` Python package is
installed, the SQLite file at `~/.local/share/cyclone/cyclone.db` is
opened with SQLCipher (AES-256). The key is read from the Keychain
once at process start, applied via a SQLAlchemy `connect` event so
every connection — including migrations and tests — gets the same
`PRAGMA key`. The key is never written to disk or to a Python global.
When the Keychain entry is missing **or** `sqlcipher3` is not
installed, the DB falls back to plain SQLite. The intent is a graceful
default for developers and CI; the production posture is that every
operator has created the Keychain entry on first run. See
[docs/reference/co-medicaid.md §Keychain setup](docs/reference/co-medicaid.md)
for the one-time setup recipe and the HIPAA Security Rule §164.312(a)(2)(iv)
mapping.
## SFTP Wire-Up (paramiko)
The `clearhouse.submit` endpoint uses `paramiko` to push a batch of
generated 837 files to the dzinesco SFTP server
(`mft.gainwelltechnologies.com:22`, path
`/CO XIX/PROD/coxix_prod_11525703/FromHPE`). The SFTP credential is
fetched from the macOS Keychain at call time — never read from YAML,
never logged, never written to disk. The wire-up honors the file-naming
template stored in the `clearhouse` config:
```
outbound: {tpid}-{tx}-{ts_mt}-1of1.{ext} e.g. 11525703-837P-20260620181814559-1of1.txt
inbound: TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
```
where `{ts_mt}` is a 17-digit `yyyymmddhhmmssSSS` Mountain Time stamp.
Inbound filenames are routed by `<FileType>` and `<OrigTx>` to the
matching parser (`999`, `TA1`, `271`, `277`, `277CA`, `835`).
The `SftpClient` interface is the same one the SP9 stub used — the swap
was a one-file change (`sftp_paramiko.py` replacing `sftp_stub.py`).
`paramiko` is an optional dependency; the stub remains the default when
the `paramiko` extras aren't installed so the test suite stays green on
Linux dev boxes.
## Persistence
Parsed batches, claims, remittances, matches, and activity events are
stored in a SQLite file at `~/.local/share/cyclone/cyclone.db` by
default. The directory is auto-created on first run.
Parsed batches, claims, remittances, matches, 277CA rejections,
hash-chained audit log entries, and SFTP submission history are stored
in a SQLite file at `~/.local/share/cyclone/cyclone.db` by default.
The directory is auto-created on first run. The DB is optionally
encrypted with SQLCipher — see
[Encryption at Rest](#encryption-at-rest) for the Keychain-driven
setup.
To use a different location, set `CYCLONE_DB_URL`:
@@ -285,11 +416,23 @@ backup API).
├── backend/
│ ├── src/cyclone/
│ │ ├── api.py # FastAPI app, GET + parse routes, /api/{resource}/stream
│ │ ├── api_helpers.py # NDJSON / content-negotiation / live-tail helpers
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
│ │ ├── store.py # InMemoryStore, mappers, publish-on-write
│ │ ├── store.py # CycloneStore, mappers, publish-on-write
│ │ ├── db.py # SQLAlchemy engine, session factory, ORM models
│ │ ├── db_migrate.py # PRAGMA user_version migration runner
│ │ ├── db_crypto.py # optional SQLCipher encryption at rest (SP12)
│ │ ├── audit_log.py # tamper-evident hash-chained audit_log (SP11)
│ │ ├── inbox_lanes.py # rejected / payer_rejected / candidates / unmatched / done_today
│ │ ├── inbox_state.py # 999 envelope reject → claim state transitions
│ │ ├── inbox_state_277ca.py # 277CA STC A4/A6/A7 → payer_rejected stamp (SP10)
│ │ ├── providers.py # multi-NPI provider lookups (SP9)
│ │ ├── payers.py # payer / payer_config lookups (SP9)
│ │ ├── secrets.py # macOS Keychain-backed secret fetcher
│ │ ├── reconcile.py # pure-function 835→claim match + line-level match
│ │ ├── __main__.py # `python -m cyclone serve`
│ │ ├── cli.py # click CLI
│ │ └── parsers/ # X12 tokenizer, models, validator, writers
│ │ └── parsers/ # X12 tokenizer, models, validator, writers, 277CA, 999, TA1, 270, 271
│ └── tests/
│ ├── fixtures/ # co_medicaid_*.txt, minimal_*.txt
│ ├── test_api.py # parse-837/835 round-trip
@@ -298,29 +441,71 @@ backup API).
│ ├── test_api_streaming.py
│ ├── test_api_stream_live.py # 3 live-tail endpoints + disconnect cleanup
│ ├── test_pubsub.py # EventBus + subscribe/unsubscribe
│ └── test_api_parse_persists.py
│ ├── test_api_parse_persists.py
│ ├── test_db.py / test_db_crypto.py / test_db_migrate.py
│ ├── test_audit_log.py
│ ├── test_inbox_lanes.py / test_inbox_state.py
│ ├── test_apply_277ca_rejections.py
│ ├── test_sftp_stub.py / test_sftp_paramiko.py
│ └── test_providers_seed.py / test_payer_config_loading.py
├── src/ # React + Vite + TypeScript UI
│ ├── components/
│ │ ├── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, …
│ │ └── TailStatusPill.tsx # live-tail status badge + reconnect button
│ ├── pages/ # Claims, Remittances, Providers, Activity, Upload
│ ├── pages/ # Claims, Remittances, Providers, Acks, Activity, Upload, Inbox, …
│ ├── hooks/ # useBatches, useClaims, useRemittances, useProviders, useActivity, useParse
│ │ # + useTailStream, useMergedTail (live tail)
│ ├── lib/ # api.ts (6 GET + parse837/parse835/health), format.ts, utils.ts
│ ├── lib/ # api.ts, format.ts, utils.ts
│ │ # + tail-stream.ts (NDJSON parser)
│ ├── store/ # zustand sample-data + parsed-batches store
│ │ # + tail-store.ts (FIFO-capped live tail slices)
│ └── types/ # shared TS types
├── config/
│ └── payers.yaml # YAML-driven payer + clearhouse config (SP9)
├── docs/
│ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes
│ └── superpowers/plans/ # implementation plan
│ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes (incl. Keychain setup)
│ ├── reviews/ # post-SP completeness reviews
│ ├── superpowers/plans/ # implementation plans
│ └── superpowers/specs/ # design specs (incl. SP9-SP13)
├── tailwind.config.js # shimmer, scan, row-flash keyframes
└── package.json
```
## Roadmap
Sub-projects 2 through 8 are **shipped**. Next up:
Sub-projects 2 through 13 are **shipped**. See the [completeness
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
the honest gap analysis against the industry definition of a HIPAA
clearinghouse — the short version is that the local-only,
single-operator, single-payer design contract is honored, and the
items that would be needed to expand that contract (AS2/AS4, SNIP 1–7,
HITRUST, 276/277 status, 278 referrals, COB) are intentionally out of
scope.
Shipped sub-projects (most recent first):
- **Sub-project 13 (shipped) — SFTP wire-up.** `paramiko`-backed
`SftpClient` replaces the SP9 stub. The clearhouse.submit endpoint
actually pushes to
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
SFTP credentials are read from the macOS Keychain at call time.
- **Sub-project 12 (shipped) — Encryption at rest.** Optional
SQLCipher AES-256 encryption of the SQLite file, with the key
fetched from the macOS Keychain. Falls back to plain SQLite when
the Keychain entry is missing or `sqlcipher3` isn't installed.
- **Sub-project 11 (shipped) — Tamper-evident audit log.** Every
`audit_log` row carries a SHA-256 hash chained to the previous row;
a single walk via `GET /api/admin/audit-log/verify` detects any
break.
- **Sub-project 10 (shipped) — 277CA + Payer-Rejected lane.** Inbound
277CA parser + a new Payer-Rejected inbox lane distinct from the
999-envelope Rejected lane. The rejection stamp is monotonic.
- **Sub-project 9 (shipped) — Multi-payer, multi-NPI, SFTP stub.** The
in-code `PAYER_FACTORIES` dict is replaced by a `config/payers.yaml`
+ 3 new DB tables (`providers`, `payers`, `payer_configs`) +
`clearhouse` singleton. Added a `POST /api/clearhouse/submit` stub
that writes to a local `staging_dir` — swapped for real `paramiko`
in SP13.
- **Sub-project 8 (shipped) — Outbound 837P serializer.** Closes the
resubmit loop: rejected claims can be regenerated back to an X12 837P
@@ -390,8 +575,9 @@ Sub-projects 2 through 8 are **shipped**. Next up:
back-off ladder on errors is `1s → 2s → 4s → 8s → 16s → 30s`
capped. See the "Live updates" section below for details.
- **Sub-project 6 (shipped) — Inbox workflow automation.**
- **Ticker-Tape inbox (`/inbox`):** four lanes ordered by urgency —
**Rejected** (claims whose 999 rejected them), **Candidates**
- **Ticker-Tape inbox (`/inbox`):** five lanes ordered by urgency —
**Rejected** (claims whose 999 rejected them), **Payer-rejected**
(claims whose 277CA denied them — added in SP10), **Candidates**
(remits that didn't auto-match a claim), **Unmatched** (claims
still waiting for a remit), and **Done today** (terminal
transitions in the last 24 hours). The page subscribes to the
@@ -428,6 +614,10 @@ Sub-projects 2 through 8 are **shipped**. Next up:
- `GET /api/acks` — list ACKs.
- `GET /api/acks/{id}` — ACK detail, including the regenerated
`raw_999_text`.
- `POST /api/parse-ta1` — parse an inbound TA1 envelope ACK and persist it.
- `GET /api/ta1-acks` — list TA1 acks.
- `GET /api/ta1-acks/{id}` — TA1 ack detail (envelope control segments
+ the parser's accept/reject verdict).
- `POST /api/eligibility/request` — build a 270 from JSON.
- `POST /api/eligibility/parse-271` — ingest a 271 and return parsed
coverage benefits.
@@ -459,9 +649,9 @@ ACKs and lets you download the regenerated 999 text.
### SP6 endpoints (inbox)
- `GET /api/inbox/lanes` — all four lanes (rejected, candidates,
unmatched, done_today) in a single round-trip, with row-level
scoring and matched-remit context.
- `GET /api/inbox/lanes` — all five lanes (rejected, payer_rejected,
candidates, unmatched, done_today) in a single round-trip, with
row-level scoring and matched-remit context.
- `POST /api/inbox/candidates/{remit_id}/match` — manual match of a
candidate remit to one of its scored claims; `409` if the claim
state moved out from under us.
@@ -510,6 +700,58 @@ ACKs and lets you download the regenerated 999 text.
Download** button in the Inbox rejected-lane BulkBar (N>1 modal
prompt).
### SP9 endpoints (multi-payer, multi-NPI, SFTP stub)
- `GET /api/clearhouse` — the `clearhouse` singleton (name, TPID,
file-naming block, SFTP block).
- `POST /api/clearhouse/submit` — push a batch of generated 837 files.
The SP9 implementation writes to a local `staging_dir`; the SP13
swap replaces the write with a real `paramiko` SFTP push without
changing the route shape.
- `GET /api/config/providers` and `GET /api/config/providers/{npi}` —
list / fetch providers from the new `providers` table.
- `GET /api/config/payers` and
`GET /api/config/payers/{payer_id}/configs` — list payers; for a
given payer, return the per-transaction-type `payer_configs` rows.
- `POST /api/admin/reload-config` — re-read `config/payers.yaml` and
refresh the in-process cache without a server restart.
### SP10 endpoints (277CA + Payer-Rejected lane)
- `POST /api/parse-277ca` — upload a 277CA file; persist the parsed
`ClaimStatus` rows and stamp the matching claims with
`payer_rejected_at` (monotonic, never overwritten by `NULL`).
- `GET /api/277ca-acks` — list 277CA acks.
- `GET /api/277ca-acks/{id}` — one 277CA ack with its per-claim
`ClaimStatus` rows + regenerated text.
- `GET /api/inbox/lanes` — the response now also carries a
`payer_rejected` lane populated from
`Claim.payer_rejected_at IS NOT NULL`.
### SP11 endpoints (tamper-evident audit log)
- `GET /api/admin/audit-log` — paginated audit log. Each row carries
`(id, prev_hash, row_hash, event_type, actor, payload_json,
created_at)` where `row_hash = sha256(prev_hash || canonical_json(payload))`.
- `GET /api/admin/audit-log/verify` — walk the chain in insertion
order; return `{ok: true}` or the first `{id, expected, got}`
mismatch. The walk is O(n) with one indexed lookup per row.
### SP12 (encryption at rest — no new routes)
SP12 introduces no API routes. The `cyclone.db.key` Keychain entry +
optional `sqlcipher3` dependency enable AES-256 encryption transparently
on the next connection. See [Encryption at Rest](#encryption-at-rest)
and [docs/reference/co-medicaid.md](docs/reference/co-medicaid.md) for
the one-time setup recipe.
### SP13 endpoints (paramiko SFTP)
- `POST /api/clearhouse/submit` — same endpoint as SP9; the
implementation is now a real `paramiko` `SftpClient.write` to
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
SFTP credentials are fetched from the macOS Keychain at call time.
## License
No license file yet; this is internal-use software. Add a `LICENSE` file
backend/README.md
+15 -1
View File
@@ -79,10 +79,24 @@ python -m cyclone serve
### Endpoints
| Method | Path | Purpose |
| ------ | ---------------- | -------------------------------------------------- |
| ------ | --------------------- | ------------------------------------------------------------------ |
| GET | `/api/health` | Liveness probe: `{"status": "ok", "version": ...}` |
| POST | `/api/parse-837` | Upload an X12 837P file, get parsed claims back |
| POST | `/api/parse-835` | Upload an X12 835 ERA file, get parsed payouts back |
| POST | `/api/parse-999` | Upload an inbound 999 ACK and persist it |
| POST | `/api/parse-ta1` | Upload an inbound TA1 envelope ACK and persist it |
| POST | `/api/parse-277ca` | Upload a 277CA claim acknowledgment and stamp payer_rejected claims |
| POST | `/api/eligibility/request` | Build a 270 from JSON (subscriber / provider / payer) |
| POST | `/api/eligibility/parse-271` | Ingest a 271 and return structured `coverage_benefits` |
| GET | `/api/clearhouse` | The `clearhouse` singleton (SP9) |
| POST | `/api/clearhouse/submit` | Push a batch of generated 837 files via SFTP (SP9 stub, SP13 real) |
| POST | `/api/admin/reload-config` | Re-read `config/payers.yaml` and refresh the in-process cache |
| GET | `/api/admin/audit-log` | Paginated tamper-evident audit log (SP11) |
| GET | `/api/admin/audit-log/verify` | Walk the audit_log hash chain (SP11) |
The full surface — claim / remittance / batch / inbox / stream
endpoints, config lookups, and the 270/271 builder — is enumerated in
the root [README](../README.md#multi-payer-multi-npi--clearhouse).
`POST /api/parse-837` accepts `multipart/form-data` with a single `file`
field. Optional query parameters: