tyler
edca04868d
Reachability is controlled by the host firewall / compose port publishing, not the bind address. Backend, compose, and docs all now default to 0.0.0.0 so the API is reachable from the frontend container, the host, and the LAN without per-env overrides. Files: - backend/src/cyclone/__main__.py: default host = 0.0.0.0 - docker-compose.yml: refresh the comment to match the new posture - CLAUDE.md / README.md / docs/ARCHITECTURE.md / docs/REQUIREMENTS.md: reframe the bind note accordingly
1170 lines
59 KiB
Markdown
1170 lines
59 KiB
Markdown
# Cyclone
|
||
|
||
A self-hosted EDI claims management suite for a single billing office.
|
||
Parses 837P professional claims and 835 ERA remittances (X12 005010X222A1
|
||
and 005010X221A1), with a local-only FastAPI backend and a React UI for
|
||
browsing, filtering, and inspecting the parsed data.
|
||
|
||
Local-only on purpose: binds to `127.0.0.1`, no auth, no internet exposure.
|
||
Built for one operator, one machine, one trading partner (Colorado
|
||
Medicaid, currently).
|
||
|
||
## Install
|
||
|
||
```bash
|
||
# Backend (Python 3.11+)
|
||
cd backend
|
||
python -m venv .venv
|
||
.venv/bin/pip install -e '.[dev]'
|
||
|
||
# Frontend (Node 20+)
|
||
cd ..
|
||
npm install
|
||
```
|
||
|
||
## Dev
|
||
|
||
Two terminals:
|
||
|
||
```bash
|
||
# Terminal 1 — backend
|
||
cd backend
|
||
.venv/bin/python -m cyclone serve
|
||
# (defaults to 0.0.0.0:8000; override with CYCLONE_HOST / CYCLONE_PORT / CYCLONE_RELOAD=1)
|
||
|
||
# Terminal 2 — frontend
|
||
npm run dev
|
||
# (Vite on http://localhost:5173)
|
||
```
|
||
|
||
Then open `http://localhost:5173`. Drop an `.txt` 837P or 835 file on the
|
||
Upload page; navigate to Claims, Remittances, Providers, or Activity to
|
||
see the parsed data.
|
||
|
||
The frontend reads its backend URL from `VITE_API_BASE_URL` (default
|
||
empty). Create a `.env.local` at the repo root with:
|
||
|
||
```
|
||
VITE_API_BASE_URL=http://127.0.0.1:8000
|
||
```
|
||
|
||
Without that, the UI falls back to its in-memory sample store via the
|
||
existing `data` adapter (parses are disabled).
|
||
|
||
## Pipeline automation agent
|
||
|
||
For unattended round-trips (an agent / scheduler drops an 837P file
|
||
into the pipeline and waits for the 999 back from Gainwell), see the
|
||
`cyclone-pipeline` sibling project at `/Users/openclaw/dev/cyclone-pipeline/`.
|
||
It drives the full 7-phase state machine — preflight → browser upload →
|
||
parse verification → SFTP submit → TA1 wait → 999 wait → scan +
|
||
report — with structured JSON logs, crash-safe resume, and a
|
||
self-contained per-run folder under `./runs/`.
|
||
|
||
```bash
|
||
# Single file
|
||
cyclone-pipeline run /path/to/axiscare-837p.txt
|
||
|
||
# Resume a crashed run
|
||
cyclone-pipeline resume 2026-06-21-1430-001
|
||
|
||
# On/after the following Monday, verify the 835 arrived
|
||
cyclone-pipeline check-835 2026-06-21-1430-001
|
||
```
|
||
|
||
The 835 is **not** waited for inline (it lands the following Monday on
|
||
the CO Medicaid payment cycle). See
|
||
[`cyclone-pipeline/README.md`](../cyclone-pipeline/README.md) for
|
||
install, embed-in-agent example, exit codes, and the report format.
|
||
|
||
## Skills
|
||
|
||
Cyclone ships 8 project-scoped AI-assistant skills under
|
||
[`.superpowers/skills/`](.superpowers/skills/). Each one codifies the
|
||
conventions for a major subsystem so the next contributor (human or
|
||
AI) gets the lay of the land automatically.
|
||
|
||
| Skill | Owns |
|
||
|-------|------|
|
||
| [`cyclone-spec`](.superpowers/skills/cyclone-spec/SKILL.md) | The SP-N spec → plan → implement → merge flow. |
|
||
| [`cyclone-tests`](.superpowers/skills/cyclone-tests/SKILL.md) | pytest + vitest fixture patterns, prodfiles drop-in. |
|
||
| [`cyclone-edi`](.superpowers/skills/cyclone-edi/SKILL.md) | EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1). |
|
||
| [`cyclone-tail`](.superpowers/skills/cyclone-tail/SKILL.md) | Live-tail streaming wire format and the hook triplet. |
|
||
| [`cyclone-store`](.superpowers/skills/cyclone-store/SKILL.md) | Store write-paths, pubsub event contract, SP21 split map. |
|
||
| [`cyclone-api-router`](.superpowers/skills/cyclone-api-router/SKILL.md) | FastAPI router conventions (`api_routers/`, `api_helpers.py`). |
|
||
| [`cyclone-frontend-page`](.superpowers/skills/cyclone-frontend-page/SKILL.md) | React page conventions (TanStack Query, drawer, URL state). |
|
||
| [`cyclone-cli`](.superpowers/skills/cyclone-cli/SKILL.md) | CLI subcommand conventions (`cli.py`, exit codes, smoke tests). |
|
||
|
||
Skills auto-load by description match — no slash command needed.
|
||
|
||
## Test
|
||
|
||
```bash
|
||
# Backend
|
||
cd backend && .venv/bin/pytest
|
||
|
||
# Frontend type-check + build
|
||
npm run typecheck
|
||
npm run build
|
||
|
||
# Frontend unit tests
|
||
npm test
|
||
```
|
||
|
||
## Authentication
|
||
|
||
Cyclone ships with username/password authentication and three predefined roles.
|
||
|
||
**Roles:**
|
||
|
||
| Role | Can read | Can write (upload, parse, reconcile) | Can manage users |
|
||
| -------- | -------- | ------------------------------------ | ---------------- |
|
||
| `viewer` | ✅ | ❌ | ❌ |
|
||
| `user` | ✅ | ✅ | ❌ |
|
||
| `admin` | ✅ | ✅ | ✅ |
|
||
|
||
**Bootstrap.** On first start, set `CYCLONE_ADMIN_USERNAME` and
|
||
`CYCLONE_ADMIN_PASSWORD` (min 12 chars) in your environment. Cyclone creates the
|
||
first admin automatically. On subsequent starts these env vars are ignored, so
|
||
rotating the bootstrap password doesn't affect an already-seeded admin — use the
|
||
CLI below to reset it. When running via `docker compose`, both vars are
|
||
required: compose refuses to start with a clear error if either is missing.
|
||
|
||
**CLI.** Manage users from the command line:
|
||
|
||
```
|
||
python -m cyclone users create alice --role user --password 'hunter2hunter2'
|
||
python -m cyclone users list
|
||
python -m cyclone users disable alice
|
||
python -m cyclone users reset-password alice
|
||
python -m cyclone users set-role alice --role admin
|
||
```
|
||
|
||
**Login.** Browse to `http://localhost:5173` (dev) or `http://localhost:8081`
|
||
(Docker), sign in on the `/login` page, and you'll be redirected to the
|
||
dashboard. Sessions are stored server-side in SQLite with a 24-hour sliding
|
||
expiry — every authenticated request refreshes the TTL, so an active user
|
||
never gets logged out.
|
||
|
||
**Dev escape hatch.** Set `CYCLONE_AUTH_DISABLED=1` to bypass auth entirely
|
||
(the backend auto-grants admin on every request). **NEVER set this in
|
||
production** — it's a single env-var trip from wide-open to the public
|
||
internet. The Docker compose file does not honor this flag.
|
||
|
||
See `docs/superpowers/specs/2026-06-22-cyclone-auth-design.md` for the full
|
||
design.
|
||
|
||
## Live updates
|
||
|
||
The Claims, Remittances, and Activity pages stay current without
|
||
manual refresh. The backend publishes an internal event on every
|
||
store write, the page opens a streaming HTTP connection to the
|
||
matching `/api/<resource>/stream` endpoint, and new rows are
|
||
appended to the table the moment they hit the database.
|
||
|
||
### Wire format
|
||
|
||
Each stream endpoint emits newline-delimited JSON. The first batch
|
||
is the **snapshot** of currently-known rows; after that comes
|
||
**`snapshot_end`** with the count, then the **live** events.
|
||
|
||
```json
|
||
{"type":"item","data":{"id":"CLM-1", "...":"..."}}
|
||
{"type":"item","data":{"id":"CLM-2", "...":"..."}}
|
||
{"type":"snapshot_end","data":{"count":2}}
|
||
{"type":"item","data":{"id":"CLM-3", "...":"..."}} ← live
|
||
{"type":"heartbeat","data":{"ts":"2026-06-20T23:17:09Z"}} ← idle keep-alive
|
||
```
|
||
|
||
Lines are `{"type": ..., "data": ...}`; known types are `item`,
|
||
`snapshot_end`, `heartbeat`, and (rare) `item_dropped` /
|
||
`error`. Heartbeats keep the connection alive when nothing is
|
||
happening — clients flip to `stalled` after 30s of total silence
|
||
(heartbeat or otherwise) and surface a **↻ Reconnect** button.
|
||
|
||
### Endpoints
|
||
|
||
| Method | Path | Subscribes to | Default sort |
|
||
| ------ | -------------------------- | ------------------- | ------------------- |
|
||
| GET | `/api/claims/stream` | `claim_written` | `-submission_date` |
|
||
| GET | `/api/remittances/stream` | `remittance_written`| `-received_date` |
|
||
| GET | `/api/activity/stream` | `activity_recorded` | `-timestamp` (limit 50) |
|
||
|
||
All three accept the same query params as their non-streaming
|
||
counterparts (`status`, `payer`, `date_from`, …) so a frontend can
|
||
swap a one-shot fetch for a tail with no URL surgery. Responses
|
||
are `Content-Type: application/x-ndjson`.
|
||
|
||
### Status pill
|
||
|
||
The pill in each toolbar reflects the current connection state:
|
||
|
||
| Status | Badge variant | What it means |
|
||
| ------------- | ------------- | ------------------------------------------------------------------ |
|
||
| `live` | success | snapshot received, listening for new events |
|
||
| `connecting` | warning | opening the stream (initial mount) |
|
||
| `reconnecting`| warning | previous attempt failed, backing off before retry |
|
||
| `stalled` | destructive | no event (including heartbeat) for 30s — click **↻ Reconnect** |
|
||
| `error` | destructive | stream errored — click **↻ Reconnect** |
|
||
| `closed` | destructive | page unmounted |
|
||
|
||
The reconnect button is only shown on `stalled` and `error`. The
|
||
backoff schedule on error is `1s → 2s → 4s → 8s → 16s → 30s` capped.
|
||
|
||
### Knobs
|
||
|
||
| Env var | Default | Effect |
|
||
| ---------------------------- | ------- | ------------------------------------------------------- |
|
||
| `CYCLONE_TAIL_HEARTBEAT_S` | `15` | Idle heartbeat interval (seconds). Lower it for tests. |
|
||
|
||
## Inbox
|
||
|
||
`/inbox` is the working surface. Five lanes, dark by default (Ticker Tape
|
||
aesthetic):
|
||
|
||
- **Rejected** — claims whose 999 set-level response was R or E. Re-submit in bulk.
|
||
- **Payer-rejected** — claims whose 277CA STC category is A4, A6, or A7 (the payer
|
||
accepted the file but denied the claim). Stamped at 277CA ingest time and never
|
||
overwritten by a looser later 277CA.
|
||
- **Candidates** — remits whose CLP-claim-id didn't match exactly; each one shows its top scored claim. One-click manual match or dismiss.
|
||
- **Unmatched** — claims still waiting for a remit, and remits with no candidates above the threshold.
|
||
- **Done today** — terminal state transitions in the last 24 hours.
|
||
|
||
The page subscribes to the SP5 claim and remittance tail streams
|
||
(`/api/claims/stream` and `/api/remittances/stream`); a new `item`
|
||
event refetches the lane payload (debounced ~250ms). When a 999
|
||
parses and rejects claims, the inbox reflects the new
|
||
**Rejected** rows within a fraction of a second.
|
||
|
||
### Inbox endpoints
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
||
| GET | `/api/inbox/lanes` | All five lanes in one call. |
|
||
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
|
||
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
|
||
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
|
||
| GET | `/api/inbox/export.csv?lane=<lane>` | Streams CSV of the lane's rows. |
|
||
|
||
## Outbound 837 Serializer
|
||
|
||
The same canonical `ClaimOutput` model that powers 837 ingestion also
|
||
drives **outbound** 837P regeneration. Operators can take a claim
|
||
that's been parsed, edited, or rejected and emit a byte-faithful X12
|
||
837P file — closing the resubmit loop without leaving the app.
|
||
|
||
Two surfaces:
|
||
|
||
- **Single claim → .x12 download** from the ClaimDrawer header. Click
|
||
the **Download 837** icon, get a `claim-{id}.x12` attachment. The
|
||
file is a complete ISA/GS/ST envelope + hierarchy + CLM/SV1 lines,
|
||
regenerated from the canonical `ClaimOutput` fields (not by patching
|
||
`raw_segments`).
|
||
- **Multi-claim → .zip bundle** from the Inbox rejected lane. Select
|
||
N>1 rejected claims, hit **Resubmit**, choose **Resubmit + Download**
|
||
in the confirm modal, and the backend hands you
|
||
`resubmit-{N}-claims.zip` with one `.x12` per successfully
|
||
resubmitted claim. Each file gets a unique ISA13/GS06 so back-to-back
|
||
claims don't collide on control numbers.
|
||
|
||
Design spec at `docs/superpowers/specs/2026-06-20-cyclone-serialize-837-design.md`.
|
||
Implementation note: the spec proposed a hybrid "patch `raw_segments`"
|
||
approach, but `raw_segments` only captures post-CLM segments (CLM,
|
||
REF*G1, HI, LX, SV1 pairs) — the envelope and hierarchy segments
|
||
(NM1, N3, N4, HL, PRV, SBR, DMG) are not captured, so a pass-through
|
||
serializer cannot regenerate them byte-for-byte. The shipped
|
||
implementation rebuilds the entire file from canonical fields
|
||
("Approach A" in the plan). The round-trip guarantee still holds:
|
||
all 113 prodfiles in `docs/prodfiles/claims/*.x12` parse → serialize →
|
||
parse back to the same claim id.
|
||
|
||
### Where to find it
|
||
|
||
- Backend serializer: `backend/src/cyclone/parsers/serialize_837.py`
|
||
- Single-claim download API: `GET /api/claims/{id}/serialize-837`
|
||
- Bundle API: `POST /api/inbox/rejected/resubmit?download=true`
|
||
- Frontend helper: `src/lib/api.ts:serializeClaim837`,
|
||
`src/lib/inbox-api.ts:resubmitRejectedWithDownload`
|
||
- Browser download utility: `src/lib/download.ts` (`downloadTextFile`,
|
||
`downloadBlob`)
|
||
- UI:
|
||
- `src/components/ClaimDrawer/ClaimDrawerHeader.tsx` — Download icon
|
||
- `src/pages/Inbox.tsx` — Resubmit bundle modal + progress overlay
|
||
|
||
## Per-Line Adjustment Audit
|
||
|
||
Every 835 CAS segment is tied back to the specific 837 service line it
|
||
adjudicates, instead of being aggregated to the claim level. The match
|
||
runs **eagerly** at 835 ingest time so the audit trail is stable across
|
||
re-reads (a re-migration is required if the algorithm changes).
|
||
|
||
### Where to find it
|
||
|
||
- **ClaimDrawer → "Line Reconciliation" tab** — side-by-side view of the
|
||
837 SV1 lines and 835 SVC composites with per-line CAS reasons.
|
||
- **ClaimDrawer → ServiceLinesTable** — every billed line now shows its
|
||
paid amount and adjustment sum on the right.
|
||
- **RemitDrawer → CasAdjustmentsPanel** — each CAS row carries a
|
||
`proc · #N` reference; claim-level CAS clusters separately.
|
||
- **ClaimDrawer → MatchedRemitCard** — a `lines: N/M matched` badge
|
||
lights up amber when at least one line is unmatched.
|
||
|
||
### Match criteria (strict)
|
||
|
||
A 835 SVC composite matches a 837 SV1 line iff all four criteria align:
|
||
|
||
| Field | Rule |
|
||
| ------------- | ----------------------------------------------- |
|
||
| Procedure | Exact match (case-insensitive, uppercased). |
|
||
| Modifiers | Set-equal (order-independent). |
|
||
| Service date | Exact match (both null counts as a match). |
|
||
| Units | Exact match (both null counts as a match). |
|
||
|
||
Unmatched lines surface as a **soft warning** — the claim still posts to
|
||
PAID/PARTIAL/RECEIVED, but the Inbox shows the unmatched count and the
|
||
drawer surfaces a per-line "no 837 line matched" note.
|
||
|
||
### Line-level endpoints
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
||
| GET | `/api/claims/{claim_id}/line-reconciliation` | Dedicated view for the drawer tab. |
|
||
| GET | `/api/claims/{claim_id}` | Now includes a slim `lineReconciliation[]`. |
|
||
| GET | `/api/remittances/{remittance_id}` | Now includes `serviceLinePayments[]` + `claimLevelAdjustments[]`. |
|
||
| GET | `/api/inbox/lanes` | `matched_remittance` payload gains `matched_lines` + `total_lines`. |
|
||
|
||
### Scoring
|
||
|
||
4-field weighted (sum = 100):
|
||
|
||
| Field | Weight | Rule |
|
||
| ----------------------- | ------ | ------------------------------------------------------------- |
|
||
| Patient control number | 40 | Exact match, normalized (case + leading zeros). |
|
||
| Service date | 25 | Linear decay over ±3 days. |
|
||
| Charge amount | 20 | Linear decay over ±10%. |
|
||
| Provider NPI | 15 | Exact match. Missing → 0. |
|
||
|
||
Tiers: **strong** (≥75, full opacity, Match enabled), **weak**
|
||
(50–74, dimmed), **hidden** (<50, not surfaced).
|
||
|
||
## Multi-Payer, Multi-NPI & Clearhouse
|
||
|
||
The payer and provider identity that used to live as a single hard-coded
|
||
`PayerConfig` dict in the backend is now data, not code. Three new tables
|
||
plus a YAML file drive the entire configuration:
|
||
|
||
- **`providers` table** — one row per billing-provider NPI (Montrose
|
||
`1881068062`, Delta `1851446637`, Salida `1467507269`). All three share
|
||
the same `TOC, Inc.` legal name, tax ID `721587149`, and taxonomy
|
||
`251E00000X`. Outbound 837 files pick the right `BillingProvider` by
|
||
NPI; `claim.party.npi` is now a foreign key into `providers`.
|
||
- **`payers` table** — one row per payer (`CO_TXIX`, …) with its
|
||
receiver identity (NM1*40 / ISA08 / GS03).
|
||
- **`payer_configs` join table** — one row per `(payer_id, transaction_type)`
|
||
pair. 837P and 835 can carry different `BHT06`, SBR defaults, and
|
||
allowed status codes per payer.
|
||
- **`clearhouse` single-row config** — dzinesco's identity: TPID
|
||
`11525703`, submitter name, MT-clock file-naming block, SFTP block.
|
||
- **`config/payers.yaml`** — the on-disk source for everything above,
|
||
schema-validated at boot against a Pydantic model. A typo or missing
|
||
field fails the boot with a precise error. The original in-code
|
||
`PAYER_FACTORIES` dict is kept as a fallback for ad-hoc testing.
|
||
|
||
### Config + clearhouse endpoints
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
||
| GET | `/api/clearhouse` | The `clearhouse` singleton (name, TPID, file/SFTP blocks). |
|
||
| POST | `/api/clearhouse/submit` | Push a batch of generated 837 files via SFTP (see SFTP section). |
|
||
| GET | `/api/config/providers` | All providers. |
|
||
| GET | `/api/config/providers/{npi}` | One provider. |
|
||
| GET | `/api/config/payers` | All payers. |
|
||
| GET | `/api/config/payers/{payer_id}/configs` | All `(payer_id, transaction_type)` configs for one payer. |
|
||
| POST | `/api/admin/reload-config` | Re-read `config/payers.yaml` and refresh the in-process cache. |
|
||
|
||
## 277CA Claim Acknowledgment
|
||
|
||
A 277CA (`005010X214`) is the per-claim acknowledgment CMS and
|
||
Colorado Medicaid rely on: the file was syntactically valid *and* each
|
||
named claim was accepted, pended, or rejected by the payer at the claim
|
||
level. It is distinct from a 999 (file-level) and a TA1 (envelope-level).
|
||
|
||
Cyclone ingests 277CA files the same way it ingests 999 / 835 — drop the
|
||
file on the Upload page or `POST /api/parse-277ca` — and stamps every
|
||
claim whose `STC` category is `A4`, `A6`, or `A7` with a non-null
|
||
`payer_rejected_at` + `payer_rejected_reason` + originating 277CA row id.
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | -------------------------- | ---------------------------------------------------------------- |
|
||
| POST | `/api/parse-277ca` | Upload a 277CA, persist the parsed status rows. |
|
||
| GET | `/api/277ca-acks` | List 277CA acks (filterable by date / payer). |
|
||
| GET | `/api/277ca-acks/{id}` | One 277CA ack with its per-claim status rows + regenerated text. |
|
||
|
||
The `payer_rejected` stamp is **monotonic**: a later 277CA with a looser
|
||
status set cannot clear a previous rejection. The Payer-Rejected inbox
|
||
lane surfaces every claim with a non-null `payer_rejected_at` — it is
|
||
distinct from the 999 `Rejected` lane (envelope reject) and they can
|
||
both be true for the same claim.
|
||
|
||
## Tamper-Evident Audit Log
|
||
|
||
The `audit_log` table is the canonical record of every state transition
|
||
the system has ever observed — claim lifecycle, reconciliation
|
||
decisions, config reloads, SFTP submissions, 277CA rejects. SP11 made
|
||
it tamper-evident: every row carries a SHA-256 hash of
|
||
`(prev_hash || row_payload)`, forming a chain back to a genesis row.
|
||
Any `INSERT`, `UPDATE`, or `DELETE` that breaks the chain is detectable
|
||
in a single walk.
|
||
|
||
| Method | Path | Notes |
|
||
| ------ | --------------------------------- | -------------------------------------------------------------- |
|
||
| GET | `/api/admin/audit-log` | Paginated audit log (filterable by event type / actor / date). |
|
||
| GET | `/api/admin/audit-log/verify` | Walk the chain; return the first broken link, or `{ok: true}`. |
|
||
|
||
`verify_chain` is the integrity check that backs the audit promise —
|
||
it is intentionally cheap (one indexed walk) and intentionally
|
||
side-effect-free so a scheduler can run it on a cron and alert on any
|
||
non-`{ok: true}` result. Chain verification is **not** access-gated
|
||
beyond the same `127.0.0.1` bind the rest of the API uses; for a
|
||
hostile multi-operator deployment, wrap the route in your reverse proxy.
|
||
|
||
## Encryption at Rest
|
||
|
||
When the macOS Keychain carries an entry at service `cyclone`, account
|
||
`cyclone.db.key`, and the optional `sqlcipher3` Python package is
|
||
installed, the SQLite file at `~/.local/share/cyclone/cyclone.db` is
|
||
opened with SQLCipher (AES-256). The key is read from the Keychain
|
||
once at process start, applied via a SQLAlchemy `connect` event so
|
||
every connection — including migrations and tests — gets the same
|
||
`PRAGMA key`. The key is never written to disk or to a Python global.
|
||
|
||
When the Keychain entry is missing **or** `sqlcipher3` is not
|
||
installed, the DB falls back to plain SQLite. The intent is a graceful
|
||
default for developers and CI; the production posture is that every
|
||
operator has created the Keychain entry on first run. See
|
||
[docs/reference/co-medicaid.md §Keychain setup](docs/reference/co-medicaid.md)
|
||
for the one-time setup recipe and the HIPAA Security Rule §164.312(a)(2)(iv)
|
||
mapping.
|
||
|
||
### Key rotation (SP15)
|
||
|
||
`POST /api/admin/db/rotate-key` re-encrypts the SQLite file in place
|
||
with a fresh SQLCipher key via `PRAGMA rekey`, then updates the
|
||
Keychain so subsequent connections open with the new key. The
|
||
rotation holds a module-level `threading.Lock` (so two concurrent
|
||
requests can't race), disposes + rebuilds the SQLAlchemy engine with
|
||
`NullPool` (so SQLCipher's thread affinity is honored), and writes a
|
||
tamper-evident `db.key_rotated` audit event with old + new
|
||
fingerprints and the post-rotation table count. The old key is
|
||
retained in the `cyclone.db.key.previous` Keychain account for a
|
||
grace period so a botched rotation can be rolled back by hand.
|
||
|
||
## NPI checksum + Tax ID format validation (SP20)
|
||
|
||
Two pure local validators — no NPPES round-trip, no IRS e-file
|
||
lookup. Catches the 99% case (a typo at the end of an NPI, a letter in
|
||
an EIN, an extra digit, the reserved `00`/`07`/`8X` EIN prefix).
|
||
|
||
| Check | Algorithm | Surface |
|
||
|-------|-----------|---------|
|
||
| NPI | 10 digits where the last is a Luhn checksum over `80840 + body`. CMS-published example: body `123456789` → check `3` → valid NPI `1234567893`. | `cyclone.npi.is_valid_npi`, CLI `cyclone validate-npi <npi>`, API `GET /api/admin/validate-provider?npi=...`, validator rule `R021_npi_checksum` (warning) |
|
||
| Tax ID (EIN) | 9 digits, optional `XX-XXXXXXX` formatting. Rejects reserved prefixes `00`, `07`, `80`–`89` (IRS Pension Plan Branch). | `cyclone.npi.is_valid_tax_id`, CLI `cyclone validate-tax-id <ein>`, API `GET /api/admin/validate-provider?tax_id=...` |
|
||
|
||
### CLI
|
||
|
||
```bash
|
||
$ cyclone validate-npi 1234567893
|
||
OK: 10-digit NPI passes Luhn checksum
|
||
$ cyclone validate-npi 1234567890
|
||
INVALID: '1234567890' fails NPI Luhn checksum # exit 1
|
||
|
||
$ cyclone validate-tax-id 72-1587149
|
||
OK: 9-digit EIN (normalized=721587149)
|
||
$ cyclone validate-tax-id 00-1234567
|
||
INVALID: 9-digit EIN has reserved prefix (00); EIN is not assignable by IRS # exit 1
|
||
```
|
||
|
||
### API
|
||
|
||
```bash
|
||
curl 'http://localhost:8000/api/admin/validate-provider?npi=1234567893&tax_id=72-1587149'
|
||
# {
|
||
# "npi": {"valid": true, "skipped": false},
|
||
# "tax_id": {"valid": true, "skipped": false, "normalized": "721587149"}
|
||
# }
|
||
```
|
||
|
||
Both query params are optional; omitted fields return
|
||
`{"valid": null, "skipped": true}` so the caller can render "no check
|
||
performed" rather than treating absent input as a hard fail.
|
||
|
||
### Parser integration
|
||
|
||
The `R021_npi_checksum` rule runs alongside the existing `R020_npi_format`
|
||
in `cyclone.parsers.validator`. A billing-provider NPI that passes
|
||
R020 (right shape) but fails R021 (bad Luhn) is yielded as a
|
||
**warning**, not an error — operators sometimes ingest test fixtures
|
||
with placeholder NPIs (e.g. all-same-digit) and we don't want to block
|
||
that path. In strict mode (`--strict` / `?strict=true`) warnings are
|
||
promoted to errors.
|
||
|
||
### Files
|
||
|
||
* `cyclone/npi.py` — new module (~155 LOC).
|
||
* `cyclone.parsers.validator` — new `R021_npi_checksum` rule.
|
||
* `cyclone.api_routers.admin` — new `validate-provider` endpoint.
|
||
* `cyclone.cli` — `validate-npi` + `validate-tax-id` subcommands.
|
||
* Tests: `test_npi.py` (27), `test_api_validate_provider.py` (4),
|
||
`test_cli_validate.py` (8), `test_validator.py::test_r021_*` (4) —
|
||
**43 new tests**.
|
||
|
||
## Security hardening (SP19)
|
||
|
||
Three pure-ASGI middlewares sit in front of every FastAPI request.
|
||
They're sized for Cyclone's local-only posture — a misconfigured
|
||
Tailscale / ngrok bind, a buggy cron job uploading a 4 GB file, or a
|
||
port-scraper — not for hostile internet exposure.
|
||
|
||
| Middleware | Default | Override | Reject |
|
||
|------------|---------|----------|--------|
|
||
| `BodySizeLimitMiddleware` | 50 MB | `CYCLONE_MAX_BODY_BYTES` | `413 body_too_large` over Content-Length cap; chunked reads capped too |
|
||
| `RateLimitMiddleware` | 300 req/min/IP | `CYCLONE_RATE_LIMIT_PER_MIN` | `429 rate_limited` over the sliding window; `/api/health` exempt |
|
||
| `SecurityHeadersMiddleware` | always on | n/a | stamps `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: same-origin`, `Permissions-Policy`, `Content-Security-Policy: default-src 'none'; frame-ancestors 'none'` |
|
||
|
||
Every rejection (413 / 429) also writes a tamper-evident
|
||
`api.request_rejected` event into the SP11 audit chain so an
|
||
operator can correlate a misbehaving client with the SP18 JSON logs:
|
||
|
||
```json
|
||
{"event_type":"api.request_rejected","entity_id":"POST /api/parse-837","payload":{"status":413,"reason":"body_too_large","path":"/api/parse-837","method":"POST","ip":"127.0.0.1"}}
|
||
```
|
||
|
||
### Health probe
|
||
|
||
`GET /api/health` now returns a subsystem snapshot:
|
||
|
||
```json
|
||
{
|
||
"status": "ok",
|
||
"version": "0.1.0",
|
||
"db": {"ok": true},
|
||
"scheduler": {"running": true, "interval_s": 60, "sftp_block": "co_medicaid",
|
||
"backup_scheduler_running": false, "backup_interval_hours": 24.0},
|
||
"pubsub": {"parse_completed": 1, "batch_added": 1},
|
||
"batch": {"last_batch_id": 42, "last_batch_kind": "837P",
|
||
"last_batch_at": "2026-06-21T15:30:00.123Z",
|
||
"last_batch_filename": "TP11525703-837P-..."}
|
||
}
|
||
```
|
||
|
||
Returns `"status": "degraded"` if any subsystem reports an error —
|
||
the per-subsystem dict still surfaces so an operator can see which
|
||
one is unhappy. `/api/health` is rate-limit exempt so a load balancer
|
||
hammering the endpoint doesn't trip the limiter.
|
||
|
||
### Files
|
||
|
||
* `cyclone.security` — `BodySizeLimitMiddleware`,
|
||
`RateLimitMiddleware`, `SecurityHeadersMiddleware`, and
|
||
`get_health_snapshot()` (~330 LOC).
|
||
* `cyclone.api_routers.health` rewritten to use `get_health_snapshot()`.
|
||
* `cyclone.pubsub.EventBus.stats()` — new method that returns
|
||
per-kind subscriber counts.
|
||
* `tests/test_security.py` — 13 new tests.
|
||
|
||
## Structured logging (SP18)
|
||
|
||
Cyclone emits newline-delimited JSON to stderr by default — readable
|
||
by `jq`, Loki, Vector, ELK, or any log shipper. Every record carries
|
||
`ts` (ISO 8601 ms UTC), `level`, `logger`, `msg`, and an optional
|
||
`extra` dict for structured fields. Exceptions render as a
|
||
`traceback` string.
|
||
|
||
```
|
||
{"ts":"2026-06-21T15:30:00.123Z","level":"INFO","logger":"cyclone.scheduler","msg":"Processed inbound file","extra":{"input_filename":"ACK_999.x12","parser":"parse_999","claims":3}}
|
||
{"ts":"2026-06-21T15:30:01.456Z","level":"ERROR","logger":"cyclone.api","msg":"Backup create failed","extra":{"reason":"BackupError: passphrase mismatch"},"traceback":"Traceback ..."}
|
||
```
|
||
|
||
### PII scrubbing
|
||
|
||
A `PiiScrubber` filter is attached to the root logger and rewrites
|
||
obvious PHI patterns to `<redacted:npi>` / `<redacted:ssn>` /
|
||
`<redacted:dob>` / `<redacted:patient_name>` before any handler sees
|
||
the record:
|
||
|
||
| Pattern | Replacement |
|
||
|---------|-------------|
|
||
| `\b\d{10}\b` | `<redacted:npi>` |
|
||
| `\b\d{3}-\d{2}-\d{4}\b` or `\b\d{9}\b` at phrase boundary | `<redacted:ssn>` |
|
||
| `(dob\|date_of_birth)[:=]\s*\d{4}-\d{2}-\d{2}` | preserves the key, redacts the date |
|
||
| `patient_name=...` | full chunk redacted |
|
||
| Extras with key `dob`/`ssn`/`npi`/`patient_name`/… | value redacted regardless of shape |
|
||
|
||
The scrubber is conservative — bare ISO dates without a `dob=` prefix
|
||
are **not** scrubbed (they're too often timestamps or batch IDs), and
|
||
11+ digit numbers are left alone (they can't be NPIs). Disable for
|
||
forensic mode with `CYCLONE_LOG_NO_PII_SCRUB=1`.
|
||
|
||
### Knobs
|
||
|
||
| Env var | Default | Meaning |
|
||
|---------|---------|---------|
|
||
| `CYCLONE_LOG_LEVEL` | `INFO` | Root logger level. `DEBUG` for troubleshooting. |
|
||
| `CYCLONE_LOG_FILE` | (none) | Write to this path via `RotatingFileHandler` (10 MB × 5 backups). |
|
||
| `CYCLONE_LOG_JSON` | `true` | `false` uses the dev tabular formatter. |
|
||
| `CYCLONE_LOG_NO_PII_SCRUB` | (none) | `1` disables scrubbing. |
|
||
|
||
CLI:
|
||
|
||
```
|
||
cyclone --log-format=dev parse-837 sample.x12 --output-dir out/ # tabular for tail -f
|
||
cyclone --log-file=/var/log/cyclone.log backup create # JSON to rotating file
|
||
```
|
||
|
||
The `parse-837` / `parse-835` subcommands also accept `--log-level`
|
||
which re-runs `setup_logging()` so the per-invocation level overrides
|
||
the group default.
|
||
|
||
### Files
|
||
|
||
* `cyclone.logging_config` — `JsonFormatter`, `CycloneDevFormatter`,
|
||
`PiiScrubber`, `setup_logging()`.
|
||
* `tests/test_logging_formatter.py` (11), `test_logging_scrubber.py`
|
||
(13), `test_logging_setup.py` (10) — 34 new tests.
|
||
* `cyclone.api` lifespan calls `setup_logging()` first; the CLI
|
||
`main` group does the same.
|
||
|
||
## Encrypted Backups (SP17)
|
||
|
||
The BackupService takes an online consistent snapshot of the live
|
||
SQLite file via SQLite's `.backup()` API, encrypts the bytes with
|
||
AES-256-GCM, and writes a `.bin` + `.meta.json` pair into the backup
|
||
directory (default `~/.local/share/cyclone/backups/`). The encryption
|
||
key is derived from a separate passphrase in the macOS Keychain
|
||
(PBKDF2-HMAC-SHA256, 200,000 iterations, 16-byte salt persisted to
|
||
Keychain) — so a SQLCipher DB-key compromise does not unlock the
|
||
backups, and a backup-passphrase compromise does not unlock the live
|
||
DB. If neither is set, the service refuses (`BackupError`) rather than
|
||
silently writing plaintext.
|
||
|
||
| Method | Path | Purpose |
|
||
| ------ | ---- | ------- |
|
||
| POST | `/api/admin/backup/create` | Take an encrypted backup now. |
|
||
| GET | `/api/admin/backup/list` | List `db_backups` rows (newest first, filterable). |
|
||
| GET | `/api/admin/backup/status` | Counts, disk usage, last-run timestamp, scheduler snapshot. |
|
||
| POST | `/api/admin/backup/{id}/verify` | Decrypt + SHA-256 verify against the sidecar. |
|
||
| POST | `/api/admin/backup/{id}/restore/initiate` | First step: get `restore_token` + preview (fingerprints of backup vs live). |
|
||
| POST | `/api/admin/backup/{id}/restore/confirm` | Second step: dispose engine, copy decrypted DB, rebuild engine. |
|
||
| POST | `/api/admin/backup/prune` | Apply retention policy now. |
|
||
| POST | `/api/admin/backup/scheduler/{start,stop,tick}` | Operate the backup scheduler. |
|
||
|
||
Restore is two-step by design: an idle browser tab can't nuke the
|
||
live DB. The first call returns a one-shot 64-char hex
|
||
`restore_token` plus a side-by-side preview (`backup_db_fingerprint`,
|
||
`backup_table_count`, `current_db_fingerprint`, `current_table_count`).
|
||
The second call swaps the live engine only if the token matches
|
||
within a 5-minute TTL.
|
||
|
||
The scheduler (auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`)
|
||
ticks every `CYCLONE_BACKUP_INTERVAL_HOURS` (default 24), runs
|
||
`create_now` + `prune`, and writes audit events for each outcome
|
||
(`db.backup_created`, `db.backup_failed`, `db.backup_pruned`,
|
||
`db.backup_restored`). The CLI mirrors the API surface:
|
||
|
||
```bash
|
||
cyclone backup init-passphrase # one-time; interactive
|
||
cyclone backup create
|
||
cyclone backup list
|
||
cyclone backup verify <id>
|
||
cyclone backup restore <id> --yes
|
||
cyclone backup prune --yes
|
||
cyclone backup status
|
||
```
|
||
|
||
Retention defaults to 30 days (`CYCLONE_BACKUP_RETENTION_DAYS`). The
|
||
retention policy is best-effort: an operator who runs `cyclone
|
||
backup create` manually retains full control.
|
||
|
||
## SFTP Wire-Up (paramiko)
|
||
|
||
The `clearhouse.submit` endpoint uses `paramiko` to push a batch of
|
||
generated 837 files to the dzinesco SFTP server
|
||
(`mft.gainwelltechnologies.com:22`, path
|
||
`/CO XIX/PROD/coxix_prod_11525703/FromHPE`). The SFTP credential is
|
||
fetched from the macOS Keychain at call time — never read from YAML,
|
||
never logged, never written to disk. The wire-up honors the file-naming
|
||
template stored in the `clearhouse` config:
|
||
|
||
```
|
||
outbound: {tpid}-{tx}-{ts_mt}-1of1.{ext} e.g. 11525703-837P-20260620181814559-1of1.txt
|
||
inbound: TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
|
||
```
|
||
|
||
where `{ts_mt}` is a 17-digit `yyyymmddhhmmssSSS` Mountain Time stamp.
|
||
Inbound filenames are routed by `<FileType>` and `<OrigTx>` to the
|
||
matching parser (`999`, `TA1`, `271`, `277`, `277CA`, `835`).
|
||
|
||
The `SftpClient` interface is the same one the SP9 stub used — the swap
|
||
was a one-file change (`sftp_paramiko.py` replacing `sftp_stub.py`).
|
||
`paramiko` is an optional dependency; the stub remains the default when
|
||
the `paramiko` extras aren't installed so the test suite stays green on
|
||
Linux dev boxes.
|
||
|
||
## Persistence
|
||
|
||
Parsed batches, claims, remittances, matches, 277CA rejections,
|
||
hash-chained audit log entries, and SFTP submission history are stored
|
||
in a SQLite file at `~/.local/share/cyclone/cyclone.db` by default.
|
||
The directory is auto-created on first run. The DB is optionally
|
||
encrypted with SQLCipher — see
|
||
[Encryption at Rest](#encryption-at-rest) for the Keychain-driven
|
||
setup.
|
||
|
||
To use a different location, set `CYCLONE_DB_URL`:
|
||
|
||
```bash
|
||
export CYCLONE_DB_URL=sqlite:///path/to/cyclone.db
|
||
# or
|
||
export CYCLONE_DB_URL=postgresql://user:pass@host:5432/cyclone
|
||
```
|
||
|
||
### Backup
|
||
|
||
```bash
|
||
sqlite3 ~/.local/share/cyclone/cyclone.db ".backup /path/to/backup.db"
|
||
```
|
||
|
||
This is safe to run while the backend is running (uses SQLite's online
|
||
backup API).
|
||
|
||
## Project layout
|
||
|
||
```
|
||
.
|
||
├── backend/
|
||
│ ├── src/cyclone/
|
||
│ │ ├── api.py # FastAPI app, GET + parse routes, /api/{resource}/stream
|
||
│ │ ├── api_helpers.py # NDJSON / content-negotiation / live-tail helpers
|
||
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
|
||
│ │ ├── store.py # CycloneStore, mappers, publish-on-write
|
||
│ │ ├── db.py # SQLAlchemy engine, session factory, ORM models
|
||
│ │ ├── db_migrate.py # PRAGMA user_version migration runner
|
||
│ │ ├── db_crypto.py # optional SQLCipher encryption at rest (SP12)
|
||
│ │ ├── audit_log.py # tamper-evident hash-chained audit_log (SP11)
|
||
│ │ ├── inbox_lanes.py # rejected / payer_rejected / candidates / unmatched / done_today
|
||
│ │ ├── inbox_state.py # 999 envelope reject → claim state transitions
|
||
│ │ ├── inbox_state_277ca.py # 277CA STC A4/A6/A7 → payer_rejected stamp (SP10)
|
||
│ │ ├── providers.py # multi-NPI provider lookups (SP9)
|
||
│ │ ├── payers.py # payer / payer_config lookups (SP9)
|
||
│ │ ├── secrets.py # macOS Keychain-backed secret fetcher
|
||
│ │ ├── reconcile.py # pure-function 835→claim match + line-level match
|
||
│ │ ├── __main__.py # `python -m cyclone serve`
|
||
│ │ ├── cli.py # click CLI
|
||
│ │ └── parsers/ # X12 tokenizer, models, validator, writers, 277CA, 999, TA1, 270, 271
|
||
│ └── tests/
|
||
│ ├── fixtures/ # co_medicaid_*.txt, minimal_*.txt
|
||
│ ├── test_api.py # parse-837/835 round-trip
|
||
│ ├── test_api_gets.py # 6 GET endpoints
|
||
│ ├── test_store.py # store + mappers + iterators
|
||
│ ├── test_api_streaming.py
|
||
│ ├── test_api_stream_live.py # 3 live-tail endpoints + disconnect cleanup
|
||
│ ├── test_pubsub.py # EventBus + subscribe/unsubscribe
|
||
│ ├── test_api_parse_persists.py
|
||
│ ├── test_db.py / test_db_crypto.py / test_db_migrate.py
|
||
│ ├── test_audit_log.py
|
||
│ ├── test_inbox_lanes.py / test_inbox_state.py
|
||
│ ├── test_apply_277ca_rejections.py
|
||
│ ├── test_sftp_stub.py / test_sftp_paramiko.py
|
||
│ └── test_providers_seed.py / test_payer_config_loading.py
|
||
├── src/ # React + Vite + TypeScript UI
|
||
│ ├── components/
|
||
│ │ ├── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, …
|
||
│ │ └── TailStatusPill.tsx # live-tail status badge + reconnect button
|
||
│ ├── pages/ # Claims, Remittances, Providers, Acks, Activity, Upload, Inbox, …
|
||
│ ├── hooks/ # useBatches, useClaims, useRemittances, useProviders, useActivity, useParse
|
||
│ │ # + useTailStream, useMergedTail (live tail)
|
||
│ ├── lib/ # api.ts, format.ts, utils.ts
|
||
│ │ # + tail-stream.ts (NDJSON parser)
|
||
│ ├── store/ # zustand sample-data + parsed-batches store
|
||
│ │ # + tail-store.ts (FIFO-capped live tail slices)
|
||
│ └── types/ # shared TS types
|
||
├── config/
|
||
│ └── payers.yaml # YAML-driven payer + clearhouse config (SP9)
|
||
├── docs/
|
||
│ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes (incl. Keychain setup)
|
||
│ ├── reviews/ # post-SP completeness reviews
|
||
│ ├── superpowers/plans/ # implementation plans
|
||
│ └── superpowers/specs/ # design specs (incl. SP9-SP13)
|
||
├── tailwind.config.js # shimmer, scan, row-flash keyframes
|
||
└── package.json
|
||
```
|
||
|
||
## Roadmap
|
||
|
||
> **Read order for new engineers:**
|
||
> 1. [`docs/REQUIREMENTS.md`](docs/REQUIREMENTS.md) — what Cyclone does (FRs + NFRs + DoD + traceability). The single index tying the 22 shipped sub-projects to the 38 functional + 18 non-functional requirements and the test strategy.
|
||
> 2. [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) — how it fits together (process topology, package layout, data flow, lifecycle, operational concerns).
|
||
> 3. The per-SP spec under [`docs/superpowers/specs/`](docs/superpowers/specs/) for whatever you're touching.
|
||
> 4. The per-SP plan under [`docs/superpowers/plans/`](docs/superpowers/plans/) if you're implementing.
|
||
|
||
Sub-projects 2 through 19 are **shipped**. See the [completeness
|
||
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
|
||
the honest gap analysis against the industry definition of a HIPAA
|
||
clearinghouse — the short version is that the local-only,
|
||
single-operator, single-payer design contract is honored, and the
|
||
items that would be needed to expand that contract (AS2/AS4, SNIP 1–7,
|
||
HITRUST, 276/277 status, 278 referrals, COB) are intentionally out of
|
||
scope.
|
||
|
||
Shipped sub-projects (most recent first):
|
||
|
||
- **Sub-project 22 (shipped) — Pipeline automation agent.** A
|
||
sibling project at `/Users/openclaw/dev/cyclone-pipeline` that
|
||
drives the full 7-phase round-trip (preflight → browser upload →
|
||
parse verify → SFTP submit → TA1 wait → 999 wait → scan + report)
|
||
with crash-safe resume, structured JSON logging, idempotency
|
||
dedup, and a per-run report. Pure Python 3.11+ (httpx, Playwright,
|
||
Click, pydantic v2, structlog). The 835 is not waited for inline —
|
||
it lands the following Monday — and is verified by a separate
|
||
`check-835` subcommand. Embeddable as a library for OpenClaw / Nora
|
||
agent integration. See
|
||
[Pipeline automation agent](#pipeline-automation-agent) above.
|
||
- **Sub-project 19 (shipped) — Security hardening + health probe.**
|
||
Three pure-ASGI middlewares (`BodySizeLimitMiddleware`,
|
||
`RateLimitMiddleware`, `SecurityHeadersMiddleware`) close the
|
||
completeness-review gaps §3.1.4 (no body/rate limits) and §3.1.25
|
||
(no CSP / security headers). 413/429 rejections emit a
|
||
tamper-evident `api.request_rejected` audit event (SP11 chain).
|
||
`/api/health` is now a rich subsystem snapshot — DB connectivity,
|
||
MFT scheduler state, backup scheduler state, live pubsub
|
||
subscriber counts, last batch id + timestamp. See
|
||
[Security hardening (SP19)](#security-hardening-sp19) below.
|
||
- **Sub-project 20 (shipped) — NPI checksum + Tax ID format validation.**
|
||
Pure local validators (`cyclone.npi`) — no NPPES round-trip, no IRS
|
||
e-file lookup. Catches the 99% typo case at parse time. NPI uses
|
||
CMS-published Luhn over `80840 + body` (example: `1234567893` is
|
||
valid). EIN rejects reserved prefixes (`00`, `07`, `80`–`89`).
|
||
Surface: `cyclone validate-npi` / `validate-tax-id` CLI subcommands,
|
||
`GET /api/admin/validate-provider`, new `R021_npi_checksum`
|
||
validator rule (warning, not error — placeholder NPIs in test
|
||
fixtures shouldn't block ingest). See
|
||
[NPI checksum + Tax ID format validation (SP20)](#npi-checksum--tax-id-format-validation-sp20)
|
||
below.
|
||
- **Sub-project 18 (shipped) — Structured JSON logging.** All logs
|
||
emitted by the API, CLI, scheduler tick loop, and backup service
|
||
flow through a `JsonFormatter` (newline-delimited JSON, ISO-8601 ms
|
||
timestamps) by default. A `PiiScrubber` filter redacts obvious PHI
|
||
(NPIs, SSNs, DOBs, patient names) from message + extras — both via
|
||
inline patterns (`npi 1881068062`) and via PHI-keyed extras
|
||
(`extra={"dob": "1980-04-12"}`). Configurable via env vars
|
||
(`CYCLONE_LOG_LEVEL`, `CYCLONE_LOG_FILE`, `CYCLONE_LOG_JSON`,
|
||
`CYCLONE_LOG_NO_PII_SCRUB`) and CLI flags
|
||
(`--log-format=json|dev`, `--log-file=…`); a tabular `CycloneDevFormatter`
|
||
is the opt-out for `tail -f` in dev. See
|
||
[Structured logging](#structured-logging-sp18) below.
|
||
- **Sub-project 17 (shipped) — Encrypted DB backups.** Automated
|
||
encrypted backups via AES-256-GCM (PBKDF2-HMAC-SHA256, 200k iters).
|
||
The operator sets a separate passphrase in the macOS Keychain
|
||
(`cyclone backup init-passphrase`); if missing, the service falls
|
||
back to deriving from the SQLCipher DB key with a WARNING. Online
|
||
backups via SQLite `.backup()`, two-step restore (`initiate` →
|
||
`confirm` with one-shot 64-char hex token), retention pruning with
|
||
a 30-day default, and a tamper-evident audit chain (`db.backup_created`,
|
||
`db.backup_failed`, `db.backup_pruned`, `db.backup_restored`,
|
||
`db.backup_passphrase_set`). Backup scheduler ticks every 24h
|
||
(configurable); auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`.
|
||
Seven admin endpoints + six CLI subcommands. See
|
||
[Encrypted Backups](#encrypted-backups) below.
|
||
- **Sub-project 16 (shipped) — Live MFT polling scheduler.** asyncio
|
||
background loop polls the Gainwell MFT inbound path, downloads
|
||
new files, and routes them through the right parser (999 / 835 /
|
||
277CA / TA1). Idempotent (re-ticks skip already-processed files
|
||
via the new `processed_inbound_files` table). Crash-safe (per-file
|
||
try/except so a bad file doesn't stop the loop). Five admin
|
||
endpoints (`/api/admin/scheduler/{status,start,stop,tick,processed-files}`).
|
||
- **Sub-project 15 (shipped) — SQLCipher key rotation.** In-place
|
||
rotation via `PRAGMA rekey`, serialized through a module-level
|
||
`threading.Lock` and a SQLAlchemy `NullPool` to keep SQLCipher
|
||
thread-affine under FastAPI's per-request threadpool. Writes a
|
||
`db.key_rotated` audit event with old + new key fingerprints and
|
||
post-rotation `table_count`. See
|
||
[Encryption at Rest — Key rotation](#key-rotation).
|
||
- **Sub-project 14 (shipped) — 5-lane Inbox UI.** The Payer-Rejected
|
||
lane is now rendered in the Inbox alongside Rejected / Candidates /
|
||
Unmatched / Done today. New bulk action
|
||
`POST /api/inbox/payer-rejected/acknowledge` drops claims from the
|
||
working surface without erasing the original 277CA rejection event
|
||
(audit log stays intact, SP11).
|
||
- **Sub-project 13 (shipped) — SFTP wire-up.** `paramiko`-backed
|
||
`SftpClient` replaces the SP9 stub. The clearhouse.submit endpoint
|
||
actually pushes to
|
||
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
|
||
SFTP credentials are read from the macOS Keychain at call time.
|
||
- **Sub-project 12 (shipped) — Encryption at rest.** Optional
|
||
SQLCipher AES-256 encryption of the SQLite file, with the key
|
||
fetched from the macOS Keychain. Falls back to plain SQLite when
|
||
the Keychain entry is missing or `sqlcipher3` isn't installed.
|
||
- **Sub-project 11 (shipped) — Tamper-evident audit log.** Every
|
||
`audit_log` row carries a SHA-256 hash chained to the previous row;
|
||
a single walk via `GET /api/admin/audit-log/verify` detects any
|
||
break.
|
||
- **Sub-project 10 (shipped) — 277CA + Payer-Rejected lane.** Inbound
|
||
277CA parser + a new Payer-Rejected inbox lane distinct from the
|
||
999-envelope Rejected lane. The rejection stamp is monotonic.
|
||
- **Sub-project 9 (shipped) — Multi-payer, multi-NPI, SFTP stub.** The
|
||
in-code `PAYER_FACTORIES` dict is replaced by a `config/payers.yaml`
|
||
+ 3 new DB tables (`providers`, `payers`, `payer_configs`) +
|
||
`clearhouse` singleton. Added a `POST /api/clearhouse/submit` stub
|
||
that writes to a local `staging_dir` — swapped for real `paramiko`
|
||
in SP13.
|
||
|
||
- **Sub-project 8 (shipped) — Outbound 837P serializer.** Closes the
|
||
resubmit loop: rejected claims can be regenerated back to an X12 837P
|
||
file (single download from the claim drawer, or ZIP bundle from the
|
||
inbox rejected lane). See the "Outbound 837 Serializer" section
|
||
below for details.
|
||
- **Sub-project 3 (shipped) — More 837P/835 features.**
|
||
- **837P validation rules:** R034 enforces `REF*G1` on frequency-code
|
||
7/8 claims; R035 enforces `BHT06` transaction-type-code is in the
|
||
allowed set per payer config.
|
||
- **835 CAS deep-parsing:** every CAS adjustment now surfaces with its
|
||
CARC reason code + label (e.g. `CO-29: The time limit for filing has
|
||
expired`), surfaced via `GET /api/remittances/{id}` and rendered as
|
||
an expansion row on the Remittances page.
|
||
- **999 ACK transaction set:** full inbound parser + outbound
|
||
serializer. Auto-generated on 837 ingest when `?ack=true` is passed
|
||
to `POST /api/parse-837`. Persisted to the `acks` table, browsable
|
||
on a new `/acks` page, and downloadable as the regenerated raw
|
||
999 text.
|
||
- **270/271 eligibility (API-only):** `POST /api/eligibility/request`
|
||
builds a 270 from a JSON payload (subscriber, provider, payer,
|
||
service type); `POST /api/eligibility/parse-271` ingests the
|
||
response and returns structured `coverage_benefits`.
|
||
- **TA1 interchange ACK:** parser + persistence + API for the
|
||
envelope-level TA1 acknowledgment (separate from the 999
|
||
transaction-set ACK).
|
||
- **Sub-project 4 (shipped) — Frontend features.**
|
||
- **Per-claim detail drawer:** click any row on the Claims page to
|
||
open a side-panel drawer with the full claim context — state +
|
||
amount header, validation panel, service lines, diagnoses, parties
|
||
(billing provider, subscriber, payer), raw X12 segments, matched
|
||
remittance summary, and a vertical state-history timeline. URL is
|
||
synced (`?claim=...`) so links and back-button restore the drawer.
|
||
Keyboard nav: `j`/`k` move between claims, `esc` closes, `?` opens
|
||
the cheatsheet overlay. Skeleton + error + not-found (404) states
|
||
are all distinct. Powered by a new backend endpoint
|
||
`GET /api/claims/{claim_id}` that returns the full drawer payload
|
||
in a single round-trip.
|
||
- **Remit drawer:** mirror of the claim drawer for remittances —
|
||
payer + provider header, financial summary, claim-payments table,
|
||
CAS adjustments panel, parties grid, raw X12 segments. URL is
|
||
synced (`?remit=...`).
|
||
- **Batch diff view:** side-by-side comparison of two batches
|
||
(added / removed / changed claims) reachable from the Batches
|
||
page. Powers the `BatchDetail` and `BatchDiff` views.
|
||
- **Global Cmd-K search:** cross-resource search across claims,
|
||
remittances, and activity; opens from anywhere via the keyboard
|
||
shortcut.
|
||
- **CSV export:** per-page export buttons (claims, remittances, lanes)
|
||
that stream CSV directly from the backend.
|
||
- **Accessibility + print styles:** keyboard navigation focus rings,
|
||
`aria` roles, and `@media print` stylesheets for paper-friendly
|
||
output.
|
||
- **Sub-project 5 (shipped) — Live updates.**
|
||
- The Claims, Remittances, and Activity pages stream new rows in
|
||
real time over Server-Sent Events encoded as newline-delimited
|
||
JSON. The backend publishes a `claim_written` /
|
||
`remittance_written` / `activity_recorded` event on every store
|
||
write; the frontend opens an HTTP stream and dispatches each
|
||
event into a per-resource Zustand store that the page reads on
|
||
top of its base query results.
|
||
- A small status pill in each toolbar surfaces the connection
|
||
state — `live` / `connecting` / `reconnecting` / `stalled` /
|
||
`error` / `closed` — and offers a manual **↻ Reconnect** button
|
||
on `stalled` or `error`. Stale connections (no event for 30s,
|
||
heartbeat included) flip to `stalled` automatically; the
|
||
back-off ladder on errors is `1s → 2s → 4s → 8s → 16s → 30s`
|
||
capped. See the "Live updates" section below for details.
|
||
- **Sub-project 6 (shipped) — Inbox workflow automation.**
|
||
- **Ticker-Tape inbox (`/inbox`):** five lanes ordered by urgency —
|
||
**Rejected** (claims whose 999 rejected them), **Payer-rejected**
|
||
(claims whose 277CA denied them — added in SP10), **Candidates**
|
||
(remits that didn't auto-match a claim), **Unmatched** (claims
|
||
still waiting for a remit), and **Done today** (terminal
|
||
transitions in the last 24 hours). The page subscribes to the
|
||
claim and remittance tail streams so new rejected claims land in
|
||
the Rejected lane within a fraction of a second.
|
||
- **4-field weighted scoring:** every candidate remit is scored on
|
||
patient control number (40), service date (25), charge amount
|
||
(20), and provider NPI (15). Tiers: strong (≥75, Match enabled),
|
||
weak (50–74, dimmed), hidden (<50).
|
||
- **Bulk actions:** multi-select rows in any lane to dismiss
|
||
candidates, resubmit rejected claims, or export the lane as CSV
|
||
in a single round-trip.
|
||
- **Sub-project 7 (shipped) — Per-line adjustment audit.**
|
||
- Every 835 CAS segment is tied back to the specific 837 service
|
||
line it adjudicates, instead of being aggregated to the claim
|
||
level. The match runs **eagerly** at 835 ingest time so the audit
|
||
trail is stable across re-reads.
|
||
- New endpoints surface the per-line projection: `GET
|
||
/api/claims/{id}/line-reconciliation` for the dedicated tab in
|
||
the ClaimDrawer; the existing claim + remit detail endpoints
|
||
gained slim `lineReconciliation[]` / `serviceLinePayments[]` /
|
||
`claimLevelAdjustments[]` fields; the inbox lanes include
|
||
`matched_lines` / `total_lines` badges on matched remittances.
|
||
- The ClaimDrawer gains a **Line Reconciliation** tab (alongside
|
||
Details) that pairs every 837 SV1 line with its 835 SVC composite
|
||
and surfaces per-line CAS reasons + unmatched-line warnings.
|
||
- See the "Per-Line Adjustment Audit" section below for details.
|
||
|
||
### SP3 endpoints
|
||
|
||
- `POST /api/parse-837?ack=true` — existing 837 parse, plus optional
|
||
auto-generated 999 ACK persisted alongside the batch.
|
||
- `POST /api/parse-999` — parse an inbound 999 ACK and persist it.
|
||
- `GET /api/acks` — list ACKs.
|
||
- `GET /api/acks/{id}` — ACK detail, including the regenerated
|
||
`raw_999_text`.
|
||
- `POST /api/parse-ta1` — parse an inbound TA1 envelope ACK and persist it.
|
||
- `GET /api/ta1-acks` — list TA1 acks.
|
||
- `GET /api/ta1-acks/{id}` — TA1 ack detail (envelope control segments
|
||
+ the parser's accept/reject verdict).
|
||
- `POST /api/eligibility/request` — build a 270 from JSON.
|
||
- `POST /api/eligibility/parse-271` — ingest a 271 and return parsed
|
||
coverage benefits.
|
||
|
||
The UI gains a new **Acks** page (sidebar entry) that lists persisted
|
||
ACKs and lets you download the regenerated 999 text.
|
||
|
||
### SP4 endpoints (claim drawer)
|
||
|
||
- `GET /api/claims/{claim_id}` — full claim context for the drawer
|
||
payload: header fields, diagnoses, service lines, parties
|
||
(billing provider / subscriber / payer), validation result, raw
|
||
X12 segments, matched remittance summary (or `null`), and the
|
||
claim's state history (ordered newest-first). 404 with a structured
|
||
`{ "error": "Not found", "detail": "Claim {id} not found" }` body
|
||
when the claim doesn't exist — distinct from a transient fetch
|
||
failure so the UI can render a dedicated not-found state.
|
||
|
||
### SP5 endpoints (live updates)
|
||
|
||
- `GET /api/claims/stream` — NDJSON stream of claim events. Emits a
|
||
snapshot (filtered by the same query params as `GET /api/claims`),
|
||
then `snapshot_end`, then live `claim_written` events as they
|
||
arrive, with a 15s idle heartbeat.
|
||
- `GET /api/remittances/stream` — same shape for remittances
|
||
(subscribes to `remittance_written`, default sort `-received_date`).
|
||
- `GET /api/activity/stream` — same shape for activity events
|
||
(subscribes to `activity_recorded`, default `limit=50`).
|
||
|
||
### SP6 endpoints (inbox)
|
||
|
||
- `GET /api/inbox/lanes` — all five lanes (rejected, payer_rejected,
|
||
candidates, unmatched, done_today) in a single round-trip, with
|
||
row-level scoring and matched-remit context.
|
||
- `POST /api/inbox/candidates/{remit_id}/match` — manual match of a
|
||
candidate remit to one of its scored claims; `409` if the claim
|
||
state moved out from under us.
|
||
- `POST /api/inbox/candidates/dismiss` — `{pairs: [{claim_id,
|
||
remit_id}]}`. Session-scoped, no DB persistence.
|
||
- `POST /api/inbox/rejected/resubmit` — `{claim_ids: [...]}`. `200`
|
||
with `conflicts` for non-rejected claims in the batch.
|
||
- `GET /api/inbox/export.csv?lane=<lane>` — streams CSV of the
|
||
lane's rows.
|
||
|
||
### SP7 endpoints (per-line reconciliation)
|
||
|
||
- `GET /api/claims/{id}/line-reconciliation` — dedicated view for
|
||
the ClaimDrawer's Line Reconciliation tab; returns the slim
|
||
per-line projection (`summary` + `lines[]`).
|
||
- `GET /api/claims/{id}` — now includes a slim `lineReconciliation[]`
|
||
so the ServiceLinesTable can show paid + adjustments columns
|
||
without a second fetch.
|
||
- `GET /api/remittances/{id}` — now includes `serviceLinePayments[]`
|
||
+ `claimLevelAdjustments[]`; the CAS panel renders per-line + claim
|
||
level separately.
|
||
- `GET /api/inbox/lanes` — `matched_remittance` payload gained
|
||
`matched_lines` + `total_lines` so the MatchedRemitCard can badge
|
||
partial matches.
|
||
|
||
### SP8 endpoints (outbound 837P)
|
||
|
||
- `GET /api/claims/{id}/serialize-837` — regenerate the persisted
|
||
claim as a byte-faithful X12 837P file (`text/x12` body,
|
||
`Content-Disposition: attachment; filename="claim-{id}.x12"`).
|
||
404 if the claim doesn't exist, 422 if the stored `raw_json`
|
||
can't be revalidated as a `ClaimOutput`. Drives the **Download 837**
|
||
icon in the ClaimDrawer header.
|
||
- `POST /api/inbox/rejected/resubmit?download=true` — same mass-resubmit
|
||
endpoint as SP6, but the response is a ZIP archive (`application/zip`)
|
||
containing one `claim-{id}.x12` per successfully resubmitted claim.
|
||
Each file uses `serialize_837_for_resubmit` so back-to-back files in
|
||
the bundle get unique ISA13/GS06 control numbers (back-to-back
|
||
resubmits would otherwise all share `000000001`). Conflicts and
|
||
missing ids are deliberately omitted from the ZIP — the user already
|
||
saw them in the JSON response on prior calls. Per-claim regenerate
|
||
failures (rare — usually means `raw_json` is corrupted for one claim)
|
||
are surfaced via the `X-Cyclone-Serialize-Errors` response header
|
||
(JSON-encoded array) so the UI can show "10 resubmitted, 2 couldn't
|
||
be regenerated" without parsing the binary. Drives the **Resubmit +
|
||
Download** button in the Inbox rejected-lane BulkBar (N>1 modal
|
||
prompt).
|
||
|
||
### SP9 endpoints (multi-payer, multi-NPI, SFTP stub)
|
||
|
||
- `GET /api/clearhouse` — the `clearhouse` singleton (name, TPID,
|
||
file-naming block, SFTP block).
|
||
- `POST /api/clearhouse/submit` — push a batch of generated 837 files.
|
||
The SP9 implementation writes to a local `staging_dir`; the SP13
|
||
swap replaces the write with a real `paramiko` SFTP push without
|
||
changing the route shape.
|
||
- `GET /api/config/providers` and `GET /api/config/providers/{npi}` —
|
||
list / fetch providers from the new `providers` table.
|
||
- `GET /api/config/payers` and
|
||
`GET /api/config/payers/{payer_id}/configs` — list payers; for a
|
||
given payer, return the per-transaction-type `payer_configs` rows.
|
||
- `POST /api/admin/reload-config` — re-read `config/payers.yaml` and
|
||
refresh the in-process cache without a server restart.
|
||
|
||
### SP10 endpoints (277CA + Payer-Rejected lane)
|
||
|
||
- `POST /api/parse-277ca` — upload a 277CA file; persist the parsed
|
||
`ClaimStatus` rows and stamp the matching claims with
|
||
`payer_rejected_at` (monotonic, never overwritten by `NULL`).
|
||
- `GET /api/277ca-acks` — list 277CA acks.
|
||
- `GET /api/277ca-acks/{id}` — one 277CA ack with its per-claim
|
||
`ClaimStatus` rows + regenerated text.
|
||
- `GET /api/inbox/lanes` — the response now also carries a
|
||
`payer_rejected` lane populated from
|
||
`Claim.payer_rejected_at IS NOT NULL`.
|
||
|
||
### SP11 endpoints (tamper-evident audit log)
|
||
|
||
- `GET /api/admin/audit-log` — paginated audit log. Each row carries
|
||
`(id, prev_hash, row_hash, event_type, actor, payload_json,
|
||
created_at)` where `row_hash = sha256(prev_hash || canonical_json(payload))`.
|
||
- `GET /api/admin/audit-log/verify` — walk the chain in insertion
|
||
order; return `{ok: true}` or the first `{id, expected, got}`
|
||
mismatch. The walk is O(n) with one indexed lookup per row.
|
||
|
||
### SP12 (encryption at rest — no new routes)
|
||
|
||
SP12 introduces no API routes. The `cyclone.db.key` Keychain entry +
|
||
optional `sqlcipher3` dependency enable AES-256 encryption transparently
|
||
on the next connection. See [Encryption at Rest](#encryption-at-rest)
|
||
and [docs/reference/co-medicaid.md](docs/reference/co-medicaid.md) for
|
||
the one-time setup recipe.
|
||
|
||
### SP13 endpoints (paramiko SFTP)
|
||
|
||
- `POST /api/clearhouse/submit` — same endpoint as SP9; the
|
||
implementation is now a real `paramiko` `SftpClient.write` to
|
||
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
|
||
SFTP credentials are fetched from the macOS Keychain at call time.
|
||
|
||
## License
|
||
|
||
No license file yet; this is internal-use software. Add a `LICENSE` file
|
||
when one is decided.
|