Tyler
c835996bd6
Three pure-ASGI middlewares close completeness-review gaps §3.1.4 (no body/rate limits) and §3.1.25 (no security headers): - BodySizeLimitMiddleware — rejects oversized uploads (50 MB default, CYCLONE_MAX_BODY_BYTES override). 413 on over-cap Content-Length and on chunked reads that cross the cap. - RateLimitMiddleware — sliding-window per-IP limiter (300/min default, CYCLONE_RATE_LIMIT_PER_MIN override). 429 over the window. /api/health is exempt. - SecurityHeadersMiddleware — stamps X-Content-Type-Options, X-Frame-Options, Referrer-Policy, Permissions-Policy, and a strict Content-Security-Policy on every response. Every 413/429 also writes a tamper-evident api.request_rejected event into the SP11 audit chain so an operator can correlate rejections with the SP18 JSON logs. GET /api/health is rewritten to return a subsystem snapshot: DB connectivity (SELECT 1), MFT scheduler state, backup scheduler state, live pubsub subscriber counts, last batch id + timestamp. Returns status='degraded' if any subsystem is unhappy; per-subsystem errors surfaced in the respective dict. Cyclone.pubsub.EventBus.stats() — new method for live subscriber counts. 13 new tests (test_security.py) + 1 updated (test_api.py health endpoint). All 883 backend tests pass.
994 lines
50 KiB
Markdown
994 lines
50 KiB
Markdown
# Cyclone
|
||
|
||
A self-hosted EDI claims management suite for a single billing office.
|
||
Parses 837P professional claims and 835 ERA remittances (X12 005010X222A1
|
||
and 005010X221A1), with a local-only FastAPI backend and a React UI for
|
||
browsing, filtering, and inspecting the parsed data.
|
||
|
||
Local-only on purpose: binds to `127.0.0.1`, no auth, no internet exposure.
|
||
Built for one operator, one machine, one trading partner (Colorado
|
||
Medicaid, currently).
|
||
|
||
## Install
|
||
|
||
```bash
|
||
# Backend (Python 3.11+)
|
||
cd backend
|
||
python -m venv .venv
|
||
.venv/bin/pip install -e '.[dev]'
|
||
|
||
# Frontend (Node 20+)
|
||
cd ..
|
||
npm install
|
||
```
|
||
|
||
## Dev
|
||
|
||
Two terminals:
|
||
|
||
```bash
|
||
# Terminal 1 — backend
|
||
cd backend
|
||
.venv/bin/python -m cyclone serve
|
||
# (defaults to 127.0.0.1:8000; override with CYCLONE_PORT=...; reload with CYCLONE_RELOAD=1)
|
||
|
||
# Terminal 2 — frontend
|
||
npm run dev
|
||
# (Vite on http://localhost:5173)
|
||
```
|
||
|
||
Then open `http://localhost:5173`. Drop an `.txt` 837P or 835 file on the
|
||
Upload page; navigate to Claims, Remittances, Providers, or Activity to
|
||
see the parsed data.
|
||
|
||
The frontend reads its backend URL from `VITE_API_BASE_URL` (default
|
||
empty). Create a `.env.local` at the repo root with:
|
||
|
||
```
|
||
VITE_API_BASE_URL=http://127.0.0.1:8000
|
||
```
|
||
|
||
Without that, the UI falls back to its in-memory sample store via the
|
||
existing `data` adapter (parses are disabled).
|
||
|
||
## Test
|
||
|
||
```bash
|
||
# Backend
|
||
cd backend && .venv/bin/pytest
|
||
|
||
# Frontend type-check + build
|
||
npm run typecheck
|
||
npm run build
|
||
|
||
# Frontend unit tests
|
||
npm test
|
||
```
|
||
|
||
## Live updates
|
||
|
||
The Claims, Remittances, and Activity pages stay current without
|
||
manual refresh. The backend publishes an internal event on every
|
||
store write, the page opens a streaming HTTP connection to the
|
||
matching `/api/<resource>/stream` endpoint, and new rows are
|
||
appended to the table the moment they hit the database.
|
||
|
||
### Wire format
|
||
|
||
Each stream endpoint emits newline-delimited JSON. The first batch
|
||
is the **snapshot** of currently-known rows; after that comes
|
||
**`snapshot_end`** with the count, then the **live** events.
|
||
|
||
```json
|
||
{"type":"item","data":{"id":"CLM-1", "...":"..."}}
|
||
{"type":"item","data":{"id":"CLM-2", "...":"..."}}
|
||
{"type":"snapshot_end","data":{"count":2}}
|
||
{"type":"item","data":{"id":"CLM-3", "...":"..."}} ← live
|
||
{"type":"heartbeat","data":{"ts":"2026-06-20T23:17:09Z"}} ← idle keep-alive
|
||
```
|
||
|
||
Lines are `{"type": ..., "data": ...}`; known types are `item`,
|
||
`snapshot_end`, `heartbeat`, and (rare) `item_dropped` /
|
||
`error`. Heartbeats keep the connection alive when nothing is
|
||
happening — clients flip to `stalled` after 30s of total silence
|
||
(heartbeat or otherwise) and surface a **↻ Reconnect** button.
|
||
|
||
### Endpoints
|
||
|
||
| Method | Path | Subscribes to | Default sort |
|
||
| ------ | -------------------------- | ------------------- | ------------------- |
|
||
| GET | `/api/claims/stream` | `claim_written` | `-submission_date` |
|
||
| GET | `/api/remittances/stream` | `remittance_written`| `-received_date` |
|
||
| GET | `/api/activity/stream` | `activity_recorded` | `-timestamp` (limit 50) |
|
||
|
||
All three accept the same query params as their non-streaming
|
||
counterparts (`status`, `payer`, `date_from`, …) so a frontend can
|
||
swap a one-shot fetch for a tail with no URL surgery. Responses
|
||
are `Content-Type: application/x-ndjson`.
|
||
|
||
### Status pill
|
||
|
||
The pill in each toolbar reflects the current connection state:
|
||
|
||
| Status | Badge variant | What it means |
|
||
| ------------- | ------------- | ------------------------------------------------------------------ |
|
||
| `live` | success | snapshot received, listening for new events |
|
||
| `connecting` | warning | opening the stream (initial mount) |
|
||
| `reconnecting`| warning | previous attempt failed, backing off before retry |
|
||
| `stalled` | destructive | no event (including heartbeat) for 30s — click **↻ Reconnect** |
|
||
| `error` | destructive | stream errored — click **↻ Reconnect** |
|
||
| `closed` | destructive | page unmounted |
|
||
|
||
The reconnect button is only shown on `stalled` and `error`. The
|
||
backoff schedule on error is `1s → 2s → 4s → 8s → 16s → 30s` capped.
|
||
|
||
### Knobs
|
||
|
||
| Env var | Default | Effect |
|
||
| ---------------------------- | ------- | ------------------------------------------------------- |
|
||
| `CYCLONE_TAIL_HEARTBEAT_S` | `15` | Idle heartbeat interval (seconds). Lower it for tests. |
|
||
|
||
## Inbox
|
||
|
||
`/inbox` is the working surface. Five lanes, dark by default (Ticker Tape
|
||
aesthetic):
|
||
|
||
- **Rejected** — claims whose 999 set-level response was R or E. Re-submit in bulk.
|
||
- **Payer-rejected** — claims whose 277CA STC category is A4, A6, or A7 (the payer
|
||
accepted the file but denied the claim). Stamped at 277CA ingest time and never
|
||
overwritten by a looser later 277CA.
|
||
- **Candidates** — remits whose CLP-claim-id didn't match exactly; each one shows its top scored claim. One-click manual match or dismiss.
|
||
- **Unmatched** — claims still waiting for a remit, and remits with no candidates above the threshold.
|
||
- **Done today** — terminal state transitions in the last 24 hours.
|
||
|
||
The page subscribes to the SP5 claim and remittance tail streams
|
||
(`/api/claims/stream` and `/api/remittances/stream`); a new `item`
|
||
event refetches the lane payload (debounced ~250ms). When a 999
|
||
parses and rejects claims, the inbox reflects the new
|
||
**Rejected** rows within a fraction of a second.
|
||
|
||
### Inbox endpoints
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
||
| GET | `/api/inbox/lanes` | All five lanes in one call. |
|
||
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
|
||
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
|
||
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
|
||
| GET | `/api/inbox/export.csv?lane=<lane>` | Streams CSV of the lane's rows. |
|
||
|
||
## Outbound 837 Serializer
|
||
|
||
The same canonical `ClaimOutput` model that powers 837 ingestion also
|
||
drives **outbound** 837P regeneration. Operators can take a claim
|
||
that's been parsed, edited, or rejected and emit a byte-faithful X12
|
||
837P file — closing the resubmit loop without leaving the app.
|
||
|
||
Two surfaces:
|
||
|
||
- **Single claim → .x12 download** from the ClaimDrawer header. Click
|
||
the **Download 837** icon, get a `claim-{id}.x12` attachment. The
|
||
file is a complete ISA/GS/ST envelope + hierarchy + CLM/SV1 lines,
|
||
regenerated from the canonical `ClaimOutput` fields (not by patching
|
||
`raw_segments`).
|
||
- **Multi-claim → .zip bundle** from the Inbox rejected lane. Select
|
||
N>1 rejected claims, hit **Resubmit**, choose **Resubmit + Download**
|
||
in the confirm modal, and the backend hands you
|
||
`resubmit-{N}-claims.zip` with one `.x12` per successfully
|
||
resubmitted claim. Each file gets a unique ISA13/GS06 so back-to-back
|
||
claims don't collide on control numbers.
|
||
|
||
Design spec at `docs/superpowers/specs/2026-06-20-cyclone-serialize-837-design.md`.
|
||
Implementation note: the spec proposed a hybrid "patch `raw_segments`"
|
||
approach, but `raw_segments` only captures post-CLM segments (CLM,
|
||
REF*G1, HI, LX, SV1 pairs) — the envelope and hierarchy segments
|
||
(NM1, N3, N4, HL, PRV, SBR, DMG) are not captured, so a pass-through
|
||
serializer cannot regenerate them byte-for-byte. The shipped
|
||
implementation rebuilds the entire file from canonical fields
|
||
("Approach A" in the plan). The round-trip guarantee still holds:
|
||
all 113 prodfiles in `docs/prodfiles/claims/*.x12` parse → serialize →
|
||
parse back to the same claim id.
|
||
|
||
### Where to find it
|
||
|
||
- Backend serializer: `backend/src/cyclone/parsers/serialize_837.py`
|
||
- Single-claim download API: `GET /api/claims/{id}/serialize-837`
|
||
- Bundle API: `POST /api/inbox/rejected/resubmit?download=true`
|
||
- Frontend helper: `src/lib/api.ts:serializeClaim837`,
|
||
`src/lib/inbox-api.ts:resubmitRejectedWithDownload`
|
||
- Browser download utility: `src/lib/download.ts` (`downloadTextFile`,
|
||
`downloadBlob`)
|
||
- UI:
|
||
- `src/components/ClaimDrawer/ClaimDrawerHeader.tsx` — Download icon
|
||
- `src/pages/Inbox.tsx` — Resubmit bundle modal + progress overlay
|
||
|
||
## Per-Line Adjustment Audit
|
||
|
||
Every 835 CAS segment is tied back to the specific 837 service line it
|
||
adjudicates, instead of being aggregated to the claim level. The match
|
||
runs **eagerly** at 835 ingest time so the audit trail is stable across
|
||
re-reads (a re-migration is required if the algorithm changes).
|
||
|
||
### Where to find it
|
||
|
||
- **ClaimDrawer → "Line Reconciliation" tab** — side-by-side view of the
|
||
837 SV1 lines and 835 SVC composites with per-line CAS reasons.
|
||
- **ClaimDrawer → ServiceLinesTable** — every billed line now shows its
|
||
paid amount and adjustment sum on the right.
|
||
- **RemitDrawer → CasAdjustmentsPanel** — each CAS row carries a
|
||
`proc · #N` reference; claim-level CAS clusters separately.
|
||
- **ClaimDrawer → MatchedRemitCard** — a `lines: N/M matched` badge
|
||
lights up amber when at least one line is unmatched.
|
||
|
||
### Match criteria (strict)
|
||
|
||
A 835 SVC composite matches a 837 SV1 line iff all four criteria align:
|
||
|
||
| Field | Rule |
|
||
| ------------- | ----------------------------------------------- |
|
||
| Procedure | Exact match (case-insensitive, uppercased). |
|
||
| Modifiers | Set-equal (order-independent). |
|
||
| Service date | Exact match (both null counts as a match). |
|
||
| Units | Exact match (both null counts as a match). |
|
||
|
||
Unmatched lines surface as a **soft warning** — the claim still posts to
|
||
PAID/PARTIAL/RECEIVED, but the Inbox shows the unmatched count and the
|
||
drawer surfaces a per-line "no 837 line matched" note.
|
||
|
||
### Line-level endpoints
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
||
| GET | `/api/claims/{claim_id}/line-reconciliation` | Dedicated view for the drawer tab. |
|
||
| GET | `/api/claims/{claim_id}` | Now includes a slim `lineReconciliation[]`. |
|
||
| GET | `/api/remittances/{remittance_id}` | Now includes `serviceLinePayments[]` + `claimLevelAdjustments[]`. |
|
||
| GET | `/api/inbox/lanes` | `matched_remittance` payload gains `matched_lines` + `total_lines`. |
|
||
|
||
### Scoring
|
||
|
||
4-field weighted (sum = 100):
|
||
|
||
| Field | Weight | Rule |
|
||
| ----------------------- | ------ | ------------------------------------------------------------- |
|
||
| Patient control number | 40 | Exact match, normalized (case + leading zeros). |
|
||
| Service date | 25 | Linear decay over ±3 days. |
|
||
| Charge amount | 20 | Linear decay over ±10%. |
|
||
| Provider NPI | 15 | Exact match. Missing → 0. |
|
||
|
||
Tiers: **strong** (≥75, full opacity, Match enabled), **weak**
|
||
(50–74, dimmed), **hidden** (<50, not surfaced).
|
||
|
||
## Multi-Payer, Multi-NPI & Clearhouse
|
||
|
||
The payer and provider identity that used to live as a single hard-coded
|
||
`PayerConfig` dict in the backend is now data, not code. Three new tables
|
||
plus a YAML file drive the entire configuration:
|
||
|
||
- **`providers` table** — one row per billing-provider NPI (Montrose
|
||
`1881068062`, Delta `1851446637`, Salida `1467507269`). All three share
|
||
the same `TOC, Inc.` legal name, tax ID `721587149`, and taxonomy
|
||
`251E00000X`. Outbound 837 files pick the right `BillingProvider` by
|
||
NPI; `claim.party.npi` is now a foreign key into `providers`.
|
||
- **`payers` table** — one row per payer (`CO_TXIX`, …) with its
|
||
receiver identity (NM1*40 / ISA08 / GS03).
|
||
- **`payer_configs` join table** — one row per `(payer_id, transaction_type)`
|
||
pair. 837P and 835 can carry different `BHT06`, SBR defaults, and
|
||
allowed status codes per payer.
|
||
- **`clearhouse` single-row config** — dzinesco's identity: TPID
|
||
`11525703`, submitter name, MT-clock file-naming block, SFTP block.
|
||
- **`config/payers.yaml`** — the on-disk source for everything above,
|
||
schema-validated at boot against a Pydantic model. A typo or missing
|
||
field fails the boot with a precise error. The original in-code
|
||
`PAYER_FACTORIES` dict is kept as a fallback for ad-hoc testing.
|
||
|
||
### Config + clearhouse endpoints
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
||
| GET | `/api/clearhouse` | The `clearhouse` singleton (name, TPID, file/SFTP blocks). |
|
||
| POST | `/api/clearhouse/submit` | Push a batch of generated 837 files via SFTP (see SFTP section). |
|
||
| GET | `/api/config/providers` | All providers. |
|
||
| GET | `/api/config/providers/{npi}` | One provider. |
|
||
| GET | `/api/config/payers` | All payers. |
|
||
| GET | `/api/config/payers/{payer_id}/configs` | All `(payer_id, transaction_type)` configs for one payer. |
|
||
| POST | `/api/admin/reload-config` | Re-read `config/payers.yaml` and refresh the in-process cache. |
|
||
|
||
## 277CA Claim Acknowledgment
|
||
|
||
A 277CA (`005010X214`) is the per-claim acknowledgment CMS and
|
||
Colorado Medicaid rely on: the file was syntactically valid *and* each
|
||
named claim was accepted, pended, or rejected by the payer at the claim
|
||
level. It is distinct from a 999 (file-level) and a TA1 (envelope-level).
|
||
|
||
Cyclone ingests 277CA files the same way it ingests 999 / 835 — drop the
|
||
file on the Upload page or `POST /api/parse-277ca` — and stamps every
|
||
claim whose `STC` category is `A4`, `A6`, or `A7` with a non-null
|
||
`payer_rejected_at` + `payer_rejected_reason` + originating 277CA row id.
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | -------------------------- | ---------------------------------------------------------------- |
|
||
| POST | `/api/parse-277ca` | Upload a 277CA, persist the parsed status rows. |
|
||
| GET | `/api/277ca-acks` | List 277CA acks (filterable by date / payer). |
|
||
| GET | `/api/277ca-acks/{id}` | One 277CA ack with its per-claim status rows + regenerated text. |
|
||
|
||
The `payer_rejected` stamp is **monotonic**: a later 277CA with a looser
|
||
status set cannot clear a previous rejection. The Payer-Rejected inbox
|
||
lane surfaces every claim with a non-null `payer_rejected_at` — it is
|
||
distinct from the 999 `Rejected` lane (envelope reject) and they can
|
||
both be true for the same claim.
|
||
|
||
## Tamper-Evident Audit Log
|
||
|
||
The `audit_log` table is the canonical record of every state transition
|
||
the system has ever observed — claim lifecycle, reconciliation
|
||
decisions, config reloads, SFTP submissions, 277CA rejects. SP11 made
|
||
it tamper-evident: every row carries a SHA-256 hash of
|
||
`(prev_hash || row_payload)`, forming a chain back to a genesis row.
|
||
Any `INSERT`, `UPDATE`, or `DELETE` that breaks the chain is detectable
|
||
in a single walk.
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------- | -------------------------------------------------------------- |
|
||
| GET | `/api/admin/audit-log` | Paginated audit log (filterable by event type / actor / date). |
|
||
| GET | `/api/admin/audit-log/verify` | Walk the chain; return the first broken link, or `{ok: true}`. |
|
||
|
||
`verify_chain` is the integrity check that backs the audit promise —
|
||
it is intentionally cheap (one indexed walk) and intentionally
|
||
side-effect-free so a scheduler can run it on a cron and alert on any
|
||
non-`{ok: true}` result. Chain verification is **not** access-gated
|
||
beyond the same `127.0.0.1` bind the rest of the API uses; for a
|
||
hostile multi-operator deployment, wrap the route in your reverse proxy.
|
||
|
||
## Encryption at Rest
|
||
|
||
When the macOS Keychain carries an entry at service `cyclone`, account
|
||
`cyclone.db.key`, and the optional `sqlcipher3` Python package is
|
||
installed, the SQLite file at `~/.local/share/cyclone/cyclone.db` is
|
||
opened with SQLCipher (AES-256). The key is read from the Keychain
|
||
once at process start, applied via a SQLAlchemy `connect` event so
|
||
every connection — including migrations and tests — gets the same
|
||
`PRAGMA key`. The key is never written to disk or to a Python global.
|
||
|
||
When the Keychain entry is missing **or** `sqlcipher3` is not
|
||
installed, the DB falls back to plain SQLite. The intent is a graceful
|
||
default for developers and CI; the production posture is that every
|
||
operator has created the Keychain entry on first run. See
|
||
[docs/reference/co-medicaid.md §Keychain setup](docs/reference/co-medicaid.md)
|
||
for the one-time setup recipe and the HIPAA Security Rule §164.312(a)(2)(iv)
|
||
mapping.
|
||
|
||
### Key rotation (SP15)
|
||
|
||
`POST /api/admin/db/rotate-key` re-encrypts the SQLite file in place
|
||
with a fresh SQLCipher key via `PRAGMA rekey`, then updates the
|
||
Keychain so subsequent connections open with the new key. The
|
||
rotation holds a module-level `threading.Lock` (so two concurrent
|
||
requests can't race), disposes + rebuilds the SQLAlchemy engine with
|
||
`NullPool` (so SQLCipher's thread affinity is honored), and writes a
|
||
tamper-evident `db.key_rotated` audit event with old + new
|
||
fingerprints and the post-rotation table count. The old key is
|
||
retained in the `cyclone.db.key.previous` Keychain account for a
|
||
grace period so a botched rotation can be rolled back by hand.
|
||
|
||
## Security hardening (SP19)
|
||
|
||
Three pure-ASGI middlewares sit in front of every FastAPI request.
|
||
They're sized for Cyclone's local-only posture — a misconfigured
|
||
Tailscale / ngrok bind, a buggy cron job uploading a 4 GB file, or a
|
||
port-scraper — not for hostile internet exposure.
|
||
|
||
| Middleware | Default | Override | Reject |
|
||
|------------|---------|----------|--------|
|
||
| `BodySizeLimitMiddleware` | 50 MB | `CYCLONE_MAX_BODY_BYTES` | `413 body_too_large` over Content-Length cap; chunked reads capped too |
|
||
| `RateLimitMiddleware` | 300 req/min/IP | `CYCLONE_RATE_LIMIT_PER_MIN` | `429 rate_limited` over the sliding window; `/api/health` exempt |
|
||
| `SecurityHeadersMiddleware` | always on | n/a | stamps `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: same-origin`, `Permissions-Policy`, `Content-Security-Policy: default-src 'none'; frame-ancestors 'none'` |
|
||
|
||
Every rejection (413 / 429) also writes a tamper-evident
|
||
`api.request_rejected` event into the SP11 audit chain so an
|
||
operator can correlate a misbehaving client with the SP18 JSON logs:
|
||
|
||
```json
|
||
{"event_type":"api.request_rejected","entity_id":"POST /api/parse-837","payload":{"status":413,"reason":"body_too_large","path":"/api/parse-837","method":"POST","ip":"127.0.0.1"}}
|
||
```
|
||
|
||
### Health probe
|
||
|
||
`GET /api/health` now returns a subsystem snapshot:
|
||
|
||
```json
|
||
{
|
||
"status": "ok",
|
||
"version": "0.1.0",
|
||
"db": {"ok": true},
|
||
"scheduler": {"running": true, "interval_s": 60, "sftp_block": "co_medicaid",
|
||
"backup_scheduler_running": false, "backup_interval_hours": 24.0},
|
||
"pubsub": {"parse_completed": 1, "batch_added": 1},
|
||
"batch": {"last_batch_id": 42, "last_batch_kind": "837P",
|
||
"last_batch_at": "2026-06-21T15:30:00.123Z",
|
||
"last_batch_filename": "TP11525703-837P-..."}
|
||
}
|
||
```
|
||
|
||
Returns `"status": "degraded"` if any subsystem reports an error —
|
||
the per-subsystem dict still surfaces so an operator can see which
|
||
one is unhappy. `/api/health` is rate-limit exempt so a load balancer
|
||
hammering the endpoint doesn't trip the limiter.
|
||
|
||
### Files
|
||
|
||
* `cyclone.security` — `BodySizeLimitMiddleware`,
|
||
`RateLimitMiddleware`, `SecurityHeadersMiddleware`, and
|
||
`get_health_snapshot()` (~330 LOC).
|
||
* `cyclone.api_routers.health` rewritten to use `get_health_snapshot()`.
|
||
* `cyclone.pubsub.EventBus.stats()` — new method that returns
|
||
per-kind subscriber counts.
|
||
* `tests/test_security.py` — 13 new tests.
|
||
|
||
## Structured logging (SP18)
|
||
|
||
Cyclone emits newline-delimited JSON to stderr by default — readable
|
||
by `jq`, Loki, Vector, ELK, or any log shipper. Every record carries
|
||
`ts` (ISO 8601 ms UTC), `level`, `logger`, `msg`, and an optional
|
||
`extra` dict for structured fields. Exceptions render as a
|
||
`traceback` string.
|
||
|
||
```
|
||
{"ts":"2026-06-21T15:30:00.123Z","level":"INFO","logger":"cyclone.scheduler","msg":"Processed inbound file","extra":{"input_filename":"ACK_999.x12","parser":"parse_999","claims":3}}
|
||
{"ts":"2026-06-21T15:30:01.456Z","level":"ERROR","logger":"cyclone.api","msg":"Backup create failed","extra":{"reason":"BackupError: passphrase mismatch"},"traceback":"Traceback ..."}
|
||
```
|
||
|
||
### PII scrubbing
|
||
|
||
A `PiiScrubber` filter is attached to the root logger and rewrites
|
||
obvious PHI patterns to `<redacted:npi>` / `<redacted:ssn>` /
|
||
`<redacted:dob>` / `<redacted:patient_name>` before any handler sees
|
||
the record:
|
||
|
||
| Pattern | Replacement |
|
||
|---------|-------------|
|
||
| `\b\d{10}\b` | `<redacted:npi>` |
|
||
| `\b\d{3}-\d{2}-\d{4}\b` or `\b\d{9}\b` at phrase boundary | `<redacted:ssn>` |
|
||
| `(dob\|date_of_birth)[:=]\s*\d{4}-\d{2}-\d{2}` | preserves the key, redacts the date |
|
||
| `patient_name=...` | full chunk redacted |
|
||
| Extras with key `dob`/`ssn`/`npi`/`patient_name`/… | value redacted regardless of shape |
|
||
|
||
The scrubber is conservative — bare ISO dates without a `dob=` prefix
|
||
are **not** scrubbed (they're too often timestamps or batch IDs), and
|
||
11+ digit numbers are left alone (they can't be NPIs). Disable for
|
||
forensic mode with `CYCLONE_LOG_NO_PII_SCRUB=1`.
|
||
|
||
### Knobs
|
||
|
||
| Env var | Default | Meaning |
|
||
|---------|---------|---------|
|
||
| `CYCLONE_LOG_LEVEL` | `INFO` | Root logger level. `DEBUG` for troubleshooting. |
|
||
| `CYCLONE_LOG_FILE` | (none) | Write to this path via `RotatingFileHandler` (10 MB × 5 backups). |
|
||
| `CYCLONE_LOG_JSON` | `true` | `false` uses the dev tabular formatter. |
|
||
| `CYCLONE_LOG_NO_PII_SCRUB` | (none) | `1` disables scrubbing. |
|
||
|
||
CLI:
|
||
|
||
```
|
||
cyclone --log-format=dev parse-837 sample.x12 --output-dir out/ # tabular for tail -f
|
||
cyclone --log-file=/var/log/cyclone.log backup create # JSON to rotating file
|
||
```
|
||
|
||
The `parse-837` / `parse-835` subcommands also accept `--log-level`
|
||
which re-runs `setup_logging()` so the per-invocation level overrides
|
||
the group default.
|
||
|
||
### Files
|
||
|
||
* `cyclone.logging_config` — `JsonFormatter`, `CycloneDevFormatter`,
|
||
`PiiScrubber`, `setup_logging()`.
|
||
* `tests/test_logging_formatter.py` (11), `test_logging_scrubber.py`
|
||
(13), `test_logging_setup.py` (10) — 34 new tests.
|
||
* `cyclone.api` lifespan calls `setup_logging()` first; the CLI
|
||
`main` group does the same.
|
||
|
||
## Encrypted Backups (SP17)
|
||
|
||
The BackupService takes an online consistent snapshot of the live
|
||
SQLite file via SQLite's `.backup()` API, encrypts the bytes with
|
||
AES-256-GCM, and writes a `.bin` + `.meta.json` pair into the backup
|
||
directory (default `~/.local/share/cyclone/backups/`). The encryption
|
||
key is derived from a separate passphrase in the macOS Keychain
|
||
(PBKDF2-HMAC-SHA256, 200,000 iterations, 16-byte salt persisted to
|
||
Keychain) — so a SQLCipher DB-key compromise does not unlock the
|
||
backups, and a backup-passphrase compromise does not unlock the live
|
||
DB. If neither is set, the service refuses (`BackupError`) rather than
|
||
silently writing plaintext.
|
||
|
||
| Method | Path | Purpose |
|
||
| ------ | ---- | ------- |
|
||
| POST | `/api/admin/backup/create` | Take an encrypted backup now. |
|
||
| GET | `/api/admin/backup/list` | List `db_backups` rows (newest first, filterable). |
|
||
| GET | `/api/admin/backup/status` | Counts, disk usage, last-run timestamp, scheduler snapshot. |
|
||
| POST | `/api/admin/backup/{id}/verify` | Decrypt + SHA-256 verify against the sidecar. |
|
||
| POST | `/api/admin/backup/{id}/restore/initiate` | First step: get `restore_token` + preview (fingerprints of backup vs live). |
|
||
| POST | `/api/admin/backup/{id}/restore/confirm` | Second step: dispose engine, copy decrypted DB, rebuild engine. |
|
||
| POST | `/api/admin/backup/prune` | Apply retention policy now. |
|
||
| POST | `/api/admin/backup/scheduler/{start,stop,tick}` | Operate the backup scheduler. |
|
||
|
||
Restore is two-step by design: an idle browser tab can't nuke the
|
||
live DB. The first call returns a one-shot 64-char hex
|
||
`restore_token` plus a side-by-side preview (`backup_db_fingerprint`,
|
||
`backup_table_count`, `current_db_fingerprint`, `current_table_count`).
|
||
The second call swaps the live engine only if the token matches
|
||
within a 5-minute TTL.
|
||
|
||
The scheduler (auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`)
|
||
ticks every `CYCLONE_BACKUP_INTERVAL_HOURS` (default 24), runs
|
||
`create_now` + `prune`, and writes audit events for each outcome
|
||
(`db.backup_created`, `db.backup_failed`, `db.backup_pruned`,
|
||
`db.backup_restored`). The CLI mirrors the API surface:
|
||
|
||
```bash
|
||
cyclone backup init-passphrase # one-time; interactive
|
||
cyclone backup create
|
||
cyclone backup list
|
||
cyclone backup verify <id>
|
||
cyclone backup restore <id> --yes
|
||
cyclone backup prune --yes
|
||
cyclone backup status
|
||
```
|
||
|
||
Retention defaults to 30 days (`CYCLONE_BACKUP_RETENTION_DAYS`). The
|
||
retention policy is best-effort: an operator who runs `cyclone
|
||
backup create` manually retains full control.
|
||
|
||
## SFTP Wire-Up (paramiko)
|
||
|
||
The `clearhouse.submit` endpoint uses `paramiko` to push a batch of
|
||
generated 837 files to the dzinesco SFTP server
|
||
(`mft.gainwelltechnologies.com:22`, path
|
||
`/CO XIX/PROD/coxix_prod_11525703/FromHPE`). The SFTP credential is
|
||
fetched from the macOS Keychain at call time — never read from YAML,
|
||
never logged, never written to disk. The wire-up honors the file-naming
|
||
template stored in the `clearhouse` config:
|
||
|
||
```
|
||
outbound: {tpid}-{tx}-{ts_mt}-1of1.{ext} e.g. 11525703-837P-20260620181814559-1of1.txt
|
||
inbound: TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
|
||
```
|
||
|
||
where `{ts_mt}` is a 17-digit `yyyymmddhhmmssSSS` Mountain Time stamp.
|
||
Inbound filenames are routed by `<FileType>` and `<OrigTx>` to the
|
||
matching parser (`999`, `TA1`, `271`, `277`, `277CA`, `835`).
|
||
|
||
The `SftpClient` interface is the same one the SP9 stub used — the swap
|
||
was a one-file change (`sftp_paramiko.py` replacing `sftp_stub.py`).
|
||
`paramiko` is an optional dependency; the stub remains the default when
|
||
the `paramiko` extras aren't installed so the test suite stays green on
|
||
Linux dev boxes.
|
||
|
||
## Persistence
|
||
|
||
Parsed batches, claims, remittances, matches, 277CA rejections,
|
||
hash-chained audit log entries, and SFTP submission history are stored
|
||
in a SQLite file at `~/.local/share/cyclone/cyclone.db` by default.
|
||
The directory is auto-created on first run. The DB is optionally
|
||
encrypted with SQLCipher — see
|
||
[Encryption at Rest](#encryption-at-rest) for the Keychain-driven
|
||
setup.
|
||
|
||
To use a different location, set `CYCLONE_DB_URL`:
|
||
|
||
```bash
|
||
export CYCLONE_DB_URL=sqlite:///path/to/cyclone.db
|
||
# or
|
||
export CYCLONE_DB_URL=postgresql://user:pass@host:5432/cyclone
|
||
```
|
||
|
||
### Backup
|
||
|
||
```bash
|
||
sqlite3 ~/.local/share/cyclone/cyclone.db ".backup /path/to/backup.db"
|
||
```
|
||
|
||
This is safe to run while the backend is running (uses SQLite's online
|
||
backup API).
|
||
|
||
## Project layout
|
||
|
||
```
|
||
.
|
||
├── backend/
|
||
│ ├── src/cyclone/
|
||
│ │ ├── api.py # FastAPI app, GET + parse routes, /api/{resource}/stream
|
||
│ │ ├── api_helpers.py # NDJSON / content-negotiation / live-tail helpers
|
||
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
|
||
│ │ ├── store.py # CycloneStore, mappers, publish-on-write
|
||
│ │ ├── db.py # SQLAlchemy engine, session factory, ORM models
|
||
│ │ ├── db_migrate.py # PRAGMA user_version migration runner
|
||
│ │ ├── db_crypto.py # optional SQLCipher encryption at rest (SP12)
|
||
│ │ ├── audit_log.py # tamper-evident hash-chained audit_log (SP11)
|
||
│ │ ├── inbox_lanes.py # rejected / payer_rejected / candidates / unmatched / done_today
|
||
│ │ ├── inbox_state.py # 999 envelope reject → claim state transitions
|
||
│ │ ├── inbox_state_277ca.py # 277CA STC A4/A6/A7 → payer_rejected stamp (SP10)
|
||
│ │ ├── providers.py # multi-NPI provider lookups (SP9)
|
||
│ │ ├── payers.py # payer / payer_config lookups (SP9)
|
||
│ │ ├── secrets.py # macOS Keychain-backed secret fetcher
|
||
│ │ ├── reconcile.py # pure-function 835→claim match + line-level match
|
||
│ │ ├── __main__.py # `python -m cyclone serve`
|
||
│ │ ├── cli.py # click CLI
|
||
│ │ └── parsers/ # X12 tokenizer, models, validator, writers, 277CA, 999, TA1, 270, 271
|
||
│ └── tests/
|
||
│ ├── fixtures/ # co_medicaid_*.txt, minimal_*.txt
|
||
│ ├── test_api.py # parse-837/835 round-trip
|
||
│ ├── test_api_gets.py # 6 GET endpoints
|
||
│ ├── test_store.py # store + mappers + iterators
|
||
│ ├── test_api_streaming.py
|
||
│ ├── test_api_stream_live.py # 3 live-tail endpoints + disconnect cleanup
|
||
│ ├── test_pubsub.py # EventBus + subscribe/unsubscribe
|
||
│ ├── test_api_parse_persists.py
|
||
│ ├── test_db.py / test_db_crypto.py / test_db_migrate.py
|
||
│ ├── test_audit_log.py
|
||
│ ├── test_inbox_lanes.py / test_inbox_state.py
|
||
│ ├── test_apply_277ca_rejections.py
|
||
│ ├── test_sftp_stub.py / test_sftp_paramiko.py
|
||
│ └── test_providers_seed.py / test_payer_config_loading.py
|
||
├── src/ # React + Vite + TypeScript UI
|
||
│ ├── components/
|
||
│ │ ├── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, …
|
||
│ │ └── TailStatusPill.tsx # live-tail status badge + reconnect button
|
||
│ ├── pages/ # Claims, Remittances, Providers, Acks, Activity, Upload, Inbox, …
|
||
│ ├── hooks/ # useBatches, useClaims, useRemittances, useProviders, useActivity, useParse
|
||
│ │ # + useTailStream, useMergedTail (live tail)
|
||
│ ├── lib/ # api.ts, format.ts, utils.ts
|
||
│ │ # + tail-stream.ts (NDJSON parser)
|
||
│ ├── store/ # zustand sample-data + parsed-batches store
|
||
│ │ # + tail-store.ts (FIFO-capped live tail slices)
|
||
│ └── types/ # shared TS types
|
||
├── config/
|
||
│ └── payers.yaml # YAML-driven payer + clearhouse config (SP9)
|
||
├── docs/
|
||
│ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes (incl. Keychain setup)
|
||
│ ├── reviews/ # post-SP completeness reviews
|
||
│ ├── superpowers/plans/ # implementation plans
|
||
│ └── superpowers/specs/ # design specs (incl. SP9-SP13)
|
||
├── tailwind.config.js # shimmer, scan, row-flash keyframes
|
||
└── package.json
|
||
```
|
||
|
||
## Roadmap
|
||
|
||
Sub-projects 2 through 19 are **shipped**. See the [completeness
|
||
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
|
||
the honest gap analysis against the industry definition of a HIPAA
|
||
clearinghouse — the short version is that the local-only,
|
||
single-operator, single-payer design contract is honored, and the
|
||
items that would be needed to expand that contract (AS2/AS4, SNIP 1–7,
|
||
HITRUST, 276/277 status, 278 referrals, COB) are intentionally out of
|
||
scope.
|
||
|
||
Shipped sub-projects (most recent first):
|
||
|
||
- **Sub-project 19 (shipped) — Security hardening + health probe.**
|
||
Three pure-ASGI middlewares (`BodySizeLimitMiddleware`,
|
||
`RateLimitMiddleware`, `SecurityHeadersMiddleware`) close the
|
||
completeness-review gaps §3.1.4 (no body/rate limits) and §3.1.25
|
||
(no CSP / security headers). 413/429 rejections emit a
|
||
tamper-evident `api.request_rejected` audit event (SP11 chain).
|
||
`/api/health` is now a rich subsystem snapshot — DB connectivity,
|
||
MFT scheduler state, backup scheduler state, live pubsub
|
||
subscriber counts, last batch id + timestamp. See
|
||
[Security hardening (SP19)](#security-hardening-sp19) below.
|
||
- **Sub-project 18 (shipped) — Structured JSON logging.** All logs
|
||
emitted by the API, CLI, scheduler tick loop, and backup service
|
||
flow through a `JsonFormatter` (newline-delimited JSON, ISO-8601 ms
|
||
timestamps) by default. A `PiiScrubber` filter redacts obvious PHI
|
||
(NPIs, SSNs, DOBs, patient names) from message + extras — both via
|
||
inline patterns (`npi 1881068062`) and via PHI-keyed extras
|
||
(`extra={"dob": "1980-04-12"}`). Configurable via env vars
|
||
(`CYCLONE_LOG_LEVEL`, `CYCLONE_LOG_FILE`, `CYCLONE_LOG_JSON`,
|
||
`CYCLONE_LOG_NO_PII_SCRUB`) and CLI flags
|
||
(`--log-format=json|dev`, `--log-file=…`); a tabular `CycloneDevFormatter`
|
||
is the opt-out for `tail -f` in dev. See
|
||
[Structured logging](#structured-logging-sp18) below.
|
||
- **Sub-project 17 (shipped) — Encrypted DB backups.** Automated
|
||
encrypted backups via AES-256-GCM (PBKDF2-HMAC-SHA256, 200k iters).
|
||
The operator sets a separate passphrase in the macOS Keychain
|
||
(`cyclone backup init-passphrase`); if missing, the service falls
|
||
back to deriving from the SQLCipher DB key with a WARNING. Online
|
||
backups via SQLite `.backup()`, two-step restore (`initiate` →
|
||
`confirm` with one-shot 64-char hex token), retention pruning with
|
||
a 30-day default, and a tamper-evident audit chain (`db.backup_created`,
|
||
`db.backup_failed`, `db.backup_pruned`, `db.backup_restored`,
|
||
`db.backup_passphrase_set`). Backup scheduler ticks every 24h
|
||
(configurable); auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`.
|
||
Seven admin endpoints + six CLI subcommands. See
|
||
[Encrypted Backups](#encrypted-backups) below.
|
||
- **Sub-project 16 (shipped) — Live MFT polling scheduler.** asyncio
|
||
background loop polls the Gainwell MFT inbound path, downloads
|
||
new files, and routes them through the right parser (999 / 835 /
|
||
277CA / TA1). Idempotent (re-ticks skip already-processed files
|
||
via the new `processed_inbound_files` table). Crash-safe (per-file
|
||
try/except so a bad file doesn't stop the loop). Five admin
|
||
endpoints (`/api/admin/scheduler/{status,start,stop,tick,processed-files}`).
|
||
- **Sub-project 15 (shipped) — SQLCipher key rotation.** In-place
|
||
rotation via `PRAGMA rekey`, serialized through a module-level
|
||
`threading.Lock` and a SQLAlchemy `NullPool` to keep SQLCipher
|
||
thread-affine under FastAPI's per-request threadpool. Writes a
|
||
`db.key_rotated` audit event with old + new key fingerprints and
|
||
post-rotation `table_count`. See
|
||
[Encryption at Rest — Key rotation](#key-rotation).
|
||
- **Sub-project 14 (shipped) — 5-lane Inbox UI.** The Payer-Rejected
|
||
lane is now rendered in the Inbox alongside Rejected / Candidates /
|
||
Unmatched / Done today. New bulk action
|
||
`POST /api/inbox/payer-rejected/acknowledge` drops claims from the
|
||
working surface without erasing the original 277CA rejection event
|
||
(audit log stays intact, SP11).
|
||
- **Sub-project 13 (shipped) — SFTP wire-up.** `paramiko`-backed
|
||
`SftpClient` replaces the SP9 stub. The clearhouse.submit endpoint
|
||
actually pushes to
|
||
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
|
||
SFTP credentials are read from the macOS Keychain at call time.
|
||
- **Sub-project 12 (shipped) — Encryption at rest.** Optional
|
||
SQLCipher AES-256 encryption of the SQLite file, with the key
|
||
fetched from the macOS Keychain. Falls back to plain SQLite when
|
||
the Keychain entry is missing or `sqlcipher3` isn't installed.
|
||
- **Sub-project 11 (shipped) — Tamper-evident audit log.** Every
|
||
`audit_log` row carries a SHA-256 hash chained to the previous row;
|
||
a single walk via `GET /api/admin/audit-log/verify` detects any
|
||
break.
|
||
- **Sub-project 10 (shipped) — 277CA + Payer-Rejected lane.** Inbound
|
||
277CA parser + a new Payer-Rejected inbox lane distinct from the
|
||
999-envelope Rejected lane. The rejection stamp is monotonic.
|
||
- **Sub-project 9 (shipped) — Multi-payer, multi-NPI, SFTP stub.** The
|
||
in-code `PAYER_FACTORIES` dict is replaced by a `config/payers.yaml`
|
||
+ 3 new DB tables (`providers`, `payers`, `payer_configs`) +
|
||
`clearhouse` singleton. Added a `POST /api/clearhouse/submit` stub
|
||
that writes to a local `staging_dir` — swapped for real `paramiko`
|
||
in SP13.
|
||
|
||
- **Sub-project 8 (shipped) — Outbound 837P serializer.** Closes the
|
||
resubmit loop: rejected claims can be regenerated back to an X12 837P
|
||
file (single download from the claim drawer, or ZIP bundle from the
|
||
inbox rejected lane). See the "Outbound 837 Serializer" section
|
||
below for details.
|
||
- **Sub-project 3 (shipped) — More 837P/835 features.**
|
||
- **837P validation rules:** R034 enforces `REF*G1` on frequency-code
|
||
7/8 claims; R035 enforces `BHT06` transaction-type-code is in the
|
||
allowed set per payer config.
|
||
- **835 CAS deep-parsing:** every CAS adjustment now surfaces with its
|
||
CARC reason code + label (e.g. `CO-29: The time limit for filing has
|
||
expired`), surfaced via `GET /api/remittances/{id}` and rendered as
|
||
an expansion row on the Remittances page.
|
||
- **999 ACK transaction set:** full inbound parser + outbound
|
||
serializer. Auto-generated on 837 ingest when `?ack=true` is passed
|
||
to `POST /api/parse-837`. Persisted to the `acks` table, browsable
|
||
on a new `/acks` page, and downloadable as the regenerated raw
|
||
999 text.
|
||
- **270/271 eligibility (API-only):** `POST /api/eligibility/request`
|
||
builds a 270 from a JSON payload (subscriber, provider, payer,
|
||
service type); `POST /api/eligibility/parse-271` ingests the
|
||
response and returns structured `coverage_benefits`.
|
||
- **TA1 interchange ACK:** parser + persistence + API for the
|
||
envelope-level TA1 acknowledgment (separate from the 999
|
||
transaction-set ACK).
|
||
- **Sub-project 4 (shipped) — Frontend features.**
|
||
- **Per-claim detail drawer:** click any row on the Claims page to
|
||
open a side-panel drawer with the full claim context — state +
|
||
amount header, validation panel, service lines, diagnoses, parties
|
||
(billing provider, subscriber, payer), raw X12 segments, matched
|
||
remittance summary, and a vertical state-history timeline. URL is
|
||
synced (`?claim=...`) so links and back-button restore the drawer.
|
||
Keyboard nav: `j`/`k` move between claims, `esc` closes, `?` opens
|
||
the cheatsheet overlay. Skeleton + error + not-found (404) states
|
||
are all distinct. Powered by a new backend endpoint
|
||
`GET /api/claims/{claim_id}` that returns the full drawer payload
|
||
in a single round-trip.
|
||
- **Remit drawer:** mirror of the claim drawer for remittances —
|
||
payer + provider header, financial summary, claim-payments table,
|
||
CAS adjustments panel, parties grid, raw X12 segments. URL is
|
||
synced (`?remit=...`).
|
||
- **Batch diff view:** side-by-side comparison of two batches
|
||
(added / removed / changed claims) reachable from the Batches
|
||
page. Powers the `BatchDetail` and `BatchDiff` views.
|
||
- **Global Cmd-K search:** cross-resource search across claims,
|
||
remittances, and activity; opens from anywhere via the keyboard
|
||
shortcut.
|
||
- **CSV export:** per-page export buttons (claims, remittances, lanes)
|
||
that stream CSV directly from the backend.
|
||
- **Accessibility + print styles:** keyboard navigation focus rings,
|
||
`aria` roles, and `@media print` stylesheets for paper-friendly
|
||
output.
|
||
- **Sub-project 5 (shipped) — Live updates.**
|
||
- The Claims, Remittances, and Activity pages stream new rows in
|
||
real time over Server-Sent Events encoded as newline-delimited
|
||
JSON. The backend publishes a `claim_written` /
|
||
`remittance_written` / `activity_recorded` event on every store
|
||
write; the frontend opens an HTTP stream and dispatches each
|
||
event into a per-resource Zustand store that the page reads on
|
||
top of its base query results.
|
||
- A small status pill in each toolbar surfaces the connection
|
||
state — `live` / `connecting` / `reconnecting` / `stalled` /
|
||
`error` / `closed` — and offers a manual **↻ Reconnect** button
|
||
on `stalled` or `error`. Stale connections (no event for 30s,
|
||
heartbeat included) flip to `stalled` automatically; the
|
||
back-off ladder on errors is `1s → 2s → 4s → 8s → 16s → 30s`
|
||
capped. See the "Live updates" section below for details.
|
||
- **Sub-project 6 (shipped) — Inbox workflow automation.**
|
||
- **Ticker-Tape inbox (`/inbox`):** five lanes ordered by urgency —
|
||
**Rejected** (claims whose 999 rejected them), **Payer-rejected**
|
||
(claims whose 277CA denied them — added in SP10), **Candidates**
|
||
(remits that didn't auto-match a claim), **Unmatched** (claims
|
||
still waiting for a remit), and **Done today** (terminal
|
||
transitions in the last 24 hours). The page subscribes to the
|
||
claim and remittance tail streams so new rejected claims land in
|
||
the Rejected lane within a fraction of a second.
|
||
- **4-field weighted scoring:** every candidate remit is scored on
|
||
patient control number (40), service date (25), charge amount
|
||
(20), and provider NPI (15). Tiers: strong (≥75, Match enabled),
|
||
weak (50–74, dimmed), hidden (<50).
|
||
- **Bulk actions:** multi-select rows in any lane to dismiss
|
||
candidates, resubmit rejected claims, or export the lane as CSV
|
||
in a single round-trip.
|
||
- **Sub-project 7 (shipped) — Per-line adjustment audit.**
|
||
- Every 835 CAS segment is tied back to the specific 837 service
|
||
line it adjudicates, instead of being aggregated to the claim
|
||
level. The match runs **eagerly** at 835 ingest time so the audit
|
||
trail is stable across re-reads.
|
||
- New endpoints surface the per-line projection: `GET
|
||
/api/claims/{id}/line-reconciliation` for the dedicated tab in
|
||
the ClaimDrawer; the existing claim + remit detail endpoints
|
||
gained slim `lineReconciliation[]` / `serviceLinePayments[]` /
|
||
`claimLevelAdjustments[]` fields; the inbox lanes include
|
||
`matched_lines` / `total_lines` badges on matched remittances.
|
||
- The ClaimDrawer gains a **Line Reconciliation** tab (alongside
|
||
Details) that pairs every 837 SV1 line with its 835 SVC composite
|
||
and surfaces per-line CAS reasons + unmatched-line warnings.
|
||
- See the "Per-Line Adjustment Audit" section below for details.
|
||
|
||
### SP3 endpoints
|
||
|
||
- `POST /api/parse-837?ack=true` — existing 837 parse, plus optional
|
||
auto-generated 999 ACK persisted alongside the batch.
|
||
- `POST /api/parse-999` — parse an inbound 999 ACK and persist it.
|
||
- `GET /api/acks` — list ACKs.
|
||
- `GET /api/acks/{id}` — ACK detail, including the regenerated
|
||
`raw_999_text`.
|
||
- `POST /api/parse-ta1` — parse an inbound TA1 envelope ACK and persist it.
|
||
- `GET /api/ta1-acks` — list TA1 acks.
|
||
- `GET /api/ta1-acks/{id}` — TA1 ack detail (envelope control segments
|
||
+ the parser's accept/reject verdict).
|
||
- `POST /api/eligibility/request` — build a 270 from JSON.
|
||
- `POST /api/eligibility/parse-271` — ingest a 271 and return parsed
|
||
coverage benefits.
|
||
|
||
The UI gains a new **Acks** page (sidebar entry) that lists persisted
|
||
ACKs and lets you download the regenerated 999 text.
|
||
|
||
### SP4 endpoints (claim drawer)
|
||
|
||
- `GET /api/claims/{claim_id}` — full claim context for the drawer
|
||
payload: header fields, diagnoses, service lines, parties
|
||
(billing provider / subscriber / payer), validation result, raw
|
||
X12 segments, matched remittance summary (or `null`), and the
|
||
claim's state history (ordered newest-first). 404 with a structured
|
||
`{ "error": "Not found", "detail": "Claim {id} not found" }` body
|
||
when the claim doesn't exist — distinct from a transient fetch
|
||
failure so the UI can render a dedicated not-found state.
|
||
|
||
### SP5 endpoints (live updates)
|
||
|
||
- `GET /api/claims/stream` — NDJSON stream of claim events. Emits a
|
||
snapshot (filtered by the same query params as `GET /api/claims`),
|
||
then `snapshot_end`, then live `claim_written` events as they
|
||
arrive, with a 15s idle heartbeat.
|
||
- `GET /api/remittances/stream` — same shape for remittances
|
||
(subscribes to `remittance_written`, default sort `-received_date`).
|
||
- `GET /api/activity/stream` — same shape for activity events
|
||
(subscribes to `activity_recorded`, default `limit=50`).
|
||
|
||
### SP6 endpoints (inbox)
|
||
|
||
- `GET /api/inbox/lanes` — all five lanes (rejected, payer_rejected,
|
||
candidates, unmatched, done_today) in a single round-trip, with
|
||
row-level scoring and matched-remit context.
|
||
- `POST /api/inbox/candidates/{remit_id}/match` — manual match of a
|
||
candidate remit to one of its scored claims; `409` if the claim
|
||
state moved out from under us.
|
||
- `POST /api/inbox/candidates/dismiss` — `{pairs: [{claim_id,
|
||
remit_id}]}`. Session-scoped, no DB persistence.
|
||
- `POST /api/inbox/rejected/resubmit` — `{claim_ids: [...]}`. `200`
|
||
with `conflicts` for non-rejected claims in the batch.
|
||
- `GET /api/inbox/export.csv?lane=<lane>` — streams CSV of the
|
||
lane's rows.
|
||
|
||
### SP7 endpoints (per-line reconciliation)
|
||
|
||
- `GET /api/claims/{id}/line-reconciliation` — dedicated view for
|
||
the ClaimDrawer's Line Reconciliation tab; returns the slim
|
||
per-line projection (`summary` + `lines[]`).
|
||
- `GET /api/claims/{id}` — now includes a slim `lineReconciliation[]`
|
||
so the ServiceLinesTable can show paid + adjustments columns
|
||
without a second fetch.
|
||
- `GET /api/remittances/{id}` — now includes `serviceLinePayments[]`
|
||
+ `claimLevelAdjustments[]`; the CAS panel renders per-line + claim
|
||
level separately.
|
||
- `GET /api/inbox/lanes` — `matched_remittance` payload gained
|
||
`matched_lines` + `total_lines` so the MatchedRemitCard can badge
|
||
partial matches.
|
||
|
||
### SP8 endpoints (outbound 837P)
|
||
|
||
- `GET /api/claims/{id}/serialize-837` — regenerate the persisted
|
||
claim as a byte-faithful X12 837P file (`text/x12` body,
|
||
`Content-Disposition: attachment; filename="claim-{id}.x12"`).
|
||
404 if the claim doesn't exist, 422 if the stored `raw_json`
|
||
can't be revalidated as a `ClaimOutput`. Drives the **Download 837**
|
||
icon in the ClaimDrawer header.
|
||
- `POST /api/inbox/rejected/resubmit?download=true` — same mass-resubmit
|
||
endpoint as SP6, but the response is a ZIP archive (`application/zip`)
|
||
containing one `claim-{id}.x12` per successfully resubmitted claim.
|
||
Each file uses `serialize_837_for_resubmit` so back-to-back files in
|
||
the bundle get unique ISA13/GS06 control numbers (back-to-back
|
||
resubmits would otherwise all share `000000001`). Conflicts and
|
||
missing ids are deliberately omitted from the ZIP — the user already
|
||
saw them in the JSON response on prior calls. Per-claim regenerate
|
||
failures (rare — usually means `raw_json` is corrupted for one claim)
|
||
are surfaced via the `X-Cyclone-Serialize-Errors` response header
|
||
(JSON-encoded array) so the UI can show "10 resubmitted, 2 couldn't
|
||
be regenerated" without parsing the binary. Drives the **Resubmit +
|
||
Download** button in the Inbox rejected-lane BulkBar (N>1 modal
|
||
prompt).
|
||
|
||
### SP9 endpoints (multi-payer, multi-NPI, SFTP stub)
|
||
|
||
- `GET /api/clearhouse` — the `clearhouse` singleton (name, TPID,
|
||
file-naming block, SFTP block).
|
||
- `POST /api/clearhouse/submit` — push a batch of generated 837 files.
|
||
The SP9 implementation writes to a local `staging_dir`; the SP13
|
||
swap replaces the write with a real `paramiko` SFTP push without
|
||
changing the route shape.
|
||
- `GET /api/config/providers` and `GET /api/config/providers/{npi}` —
|
||
list / fetch providers from the new `providers` table.
|
||
- `GET /api/config/payers` and
|
||
`GET /api/config/payers/{payer_id}/configs` — list payers; for a
|
||
given payer, return the per-transaction-type `payer_configs` rows.
|
||
- `POST /api/admin/reload-config` — re-read `config/payers.yaml` and
|
||
refresh the in-process cache without a server restart.
|
||
|
||
### SP10 endpoints (277CA + Payer-Rejected lane)
|
||
|
||
- `POST /api/parse-277ca` — upload a 277CA file; persist the parsed
|
||
`ClaimStatus` rows and stamp the matching claims with
|
||
`payer_rejected_at` (monotonic, never overwritten by `NULL`).
|
||
- `GET /api/277ca-acks` — list 277CA acks.
|
||
- `GET /api/277ca-acks/{id}` — one 277CA ack with its per-claim
|
||
`ClaimStatus` rows + regenerated text.
|
||
- `GET /api/inbox/lanes` — the response now also carries a
|
||
`payer_rejected` lane populated from
|
||
`Claim.payer_rejected_at IS NOT NULL`.
|
||
|
||
### SP11 endpoints (tamper-evident audit log)
|
||
|
||
- `GET /api/admin/audit-log` — paginated audit log. Each row carries
|
||
`(id, prev_hash, row_hash, event_type, actor, payload_json,
|
||
created_at)` where `row_hash = sha256(prev_hash || canonical_json(payload))`.
|
||
- `GET /api/admin/audit-log/verify` — walk the chain in insertion
|
||
order; return `{ok: true}` or the first `{id, expected, got}`
|
||
mismatch. The walk is O(n) with one indexed lookup per row.
|
||
|
||
### SP12 (encryption at rest — no new routes)
|
||
|
||
SP12 introduces no API routes. The `cyclone.db.key` Keychain entry +
|
||
optional `sqlcipher3` dependency enable AES-256 encryption transparently
|
||
on the next connection. See [Encryption at Rest](#encryption-at-rest)
|
||
and [docs/reference/co-medicaid.md](docs/reference/co-medicaid.md) for
|
||
the one-time setup recipe.
|
||
|
||
### SP13 endpoints (paramiko SFTP)
|
||
|
||
- `POST /api/clearhouse/submit` — same endpoint as SP9; the
|
||
implementation is now a real `paramiko` `SftpClient.write` to
|
||
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
|
||
SFTP credentials are fetched from the macOS Keychain at call time.
|
||
|
||
## License
|
||
|
||
No license file yet; this is internal-use software. Add a `LICENSE` file
|
||
when one is decided.
|