Nora
47 KiB
Cyclone — Technical Architecture
Day-1 read for engineers joining Cyclone. This is the top-level technical design: how the system fits together, what the boundaries are, and why it looks the way it does.
Companion docs:
- Requirements:
docs/REQUIREMENTS.md— what Cyclone does (FRs + NFRs + DoD + traceability).- Specs:
docs/superpowers/specs/— design specs per sub-project (22 specs).- Plans:
docs/superpowers/plans/— TDD-shaped implementation plans per sub-project (27 plans).Scope of this doc: architecture and design. The spec files own per-SP design; the plan files own the task order. This doc explains the whole — the boundaries between modules, the data flow across them, the lifecycle of an inbound file from upload to reconciled claim.
1. System overview
Cyclone is a self-hosted X12 EDI claims-management suite for a single billing office. The first deployment target is Colorado Medicaid (one operator, one trading partner). The system is local-only on purpose: the backend binds to 127.0.0.1 and requires login (bcrypt + HttpOnly session cookie). The threat model is still a stolen or imaged drive, not a remote attacker — SQLCipher at rest and the macOS Keychain handle that. The auth boundary is the HTTP layer; the file-system posture is unchanged. The SP23 product fork changes the threat model to "remote operator on LAN."
Cyclone processes three X12 transaction pairs end-to-end:
| Inbound | Outbound | Use |
|---|---|---|
| 837P (005010X222A1) | 837P (resubmit) | Professional claim submission, acknowledgment tracking |
| 835 (005010X221A1) | — | ERA remittance ingestion, auto-reconciliation |
| 999 / TA1 / 270 / 271 / 277CA | 999 (after submit) | Acknowledgments, eligibility, claim status |
The authoritative store is a single SQLite file at ~/.local/share/cyclone/cyclone.db (or, with sqlcipher3 installed and a Keychain entry, a SQLCipher-encrypted file at the same path). Every claim, remittance, audit event, and reconciliation decision is in that one file. The 6-year HIPAA retention expectation is met by automated daily encrypted backups (SP17) and a tamper-evident hash-chained audit log (SP11).
The whole stack is a single Python process (FastAPI + uvicorn on port 8000) plus a single Node process in dev (Vite on port 5173). No message broker, no separate worker, no database server. The v1 deployment model tolerates this because the operator is one person, the host is one machine, and the failure mode is "open the Activity log and check what broke."
2. Process & network topology
+---------------------+ +-----------------------+
| Browser | HTTP | Vite dev server |
| (localhost:5173) | <----> | (Node 20+) |
| | Vite | |
| React 18 + TanStack | proxy | src/ |
| Query, Zustand, | /api | - pages, components, |
| Radix UI, Tailwind | -----> | hooks, lib |
+---------------------+ +-----------------------+
|
| HTTP (localhost)
v
+----------------------------------------------------------+
| Python process: `python -m cyclone serve` |
| |
| FastAPI app (uvicorn, 127.0.0.1:8000) |
| +-----------------------------------------------------+ |
| | cyclone.api:app (3,548 LOC) | |
| | - routers: claims, remits, providers, acks, | |
| | activity, inbox, upload, download, batches, | |
| | admin, scheduler, health, ta1_acks | |
| | - live-tail: NDJSON pubsub (pubsub.py) | |
| | - lifespan: init db -> seed SP9 -> start SP16 + | |
| | SP17 schedulers -> load PayerConfig cache | |
| +-----------------------------------------------------+ |
| | cyclone.store (facade, 2,423 LOC) | |
| | - add/get/list/manual_match/iter_claims/... | |
| | - delegates to: db.SessionLocal() | |
| +-----------------------------------------------------+ |
| | cyclone.parsers.* (X12 837P/835/999/TA1/270/271/ | |
| | 277CA: tokenize -> segmentize -> model -> validate | |
| | -> write to store) | |
| +-----------------------------------------------------+ |
| | cyclone.reconcile, scoring, batch_diff, audit_log, | |
| | backup_service, scheduler, db_crypto, secrets | |
| +-----------------------------------------------------+ |
+----------------------------------------------------------+
| | |
v v v
+----------------+ +-------------------+ +----------------+
| SQLite / | | macOS Keychain | | Trading |
| SQLCipher | | (keyring) | | partner SFTP |
| cyclone.db | | - SQLCipher key | | (paramiko, SP13)|
| (12 migrations)| | - SFTP password | | |
| | | - backup pass | | |
+----------------+ +-------------------+ +----------------+
One process, one file, three side-channels. The Keychain (secrets) and SFTP (outbound to the payer) are side-channels; the database is the central state.
The cyclone-pipeline/ sibling project (not in this repo) is a separate Python CLI that drives the full 7-phase round-trip: preflight → browser upload → parse verification → SFTP submit → TA1 wait → 999 wait → scan + report. It is not part of the in-process architecture; it sits alongside the backend and uses its API endpoints.
3. Tech stack
3.1 Backend
| Layer | Technology | Version | Notes |
|---|---|---|---|
| Language | Python | 3.11+ (3.13 in dev) | requires-python = ">=3.11" |
| Web framework | FastAPI | ≥0.110, <1 | python -m cyclone serve boots uvicorn |
| ASGI server | uvicorn[standard] | ≥0.27, <1 | --host 127.0.0.1 --port 8000 |
| ORM | SQLAlchemy | ≥2.0, <3 | 2.x style; engine + SessionLocal() function-accessor |
| Validation | Pydantic | ≥2.6, <3 | v2 model_validator + ConfigDict |
| CLI | Click | ≥8.1, <9 | cyclone console script |
| YAML | PyYAML | ≥6.0, <7 | PayerConfig (SP9) |
| Secrets | keyring | ≥25.0, <26 | macOS Keychain on darwin |
| Crypto | cryptography | ≥49.0, <50 | AES-256-GCM for SP17 backups (hard dep) |
| SQLCipher (opt) | sqlcipher3 | ≥0.6, <1 | pip install -e .[sqlcipher] |
| SFTP (opt) | paramiko | ≥3.4, <6 | pip install -e .[sftp] |
| Tests | pytest + pytest-cov + pytest-asyncio + pytest-randomly | latest | 964 tests, 2026-06-23 |
| Test client | httpx | ≥0.27, <1 | dev only |
3.2 Frontend
| Layer | Technology | Version | Notes |
|---|---|---|---|
| Build | Vite | latest | npm run dev (port 5173) |
| Language | TypeScript | latest | tsc -b --noEmit is the typecheck script |
| UI | React | 18.3.1 | function components + hooks |
| Routing | react-router-dom | 6.27.0 | SPA, 11 top-level routes |
| Data | @tanstack/react-query | 5.101+ | server-state cache |
| Client state | Zustand | 4.5.5 | small, ephemeral state (live tail, drill stack) |
| UI primitives | @radix-ui/* | latest | dialog, label, select, slot, tabs |
| Styling | Tailwind CSS | latest | utility-first |
| Icons | lucide-react | 0.453+ | |
| Toasts | sonner | 1.5+ | |
| Test runtime | vitest | 4.1.9 | Node 18+ has Response/fetch natively |
| Test DOM | happy-dom | 20.10+ | not jsdom (NFR-2) |
| Component tests | @testing-library/react | 16.3+ | 73 test files |
3.3 External / operational
- macOS Keychain — secret store for SQLCipher key, SFTP password, backup passphrase. No secrets on disk in plaintext (NFR-4).
- X12 005010 implementation guides — the parsers follow the CMS / X12 published guides. The 837P parser is the most heavily tested; 835 is the second; 999/TA1/270/271/277CA are lighter.
- One trading partner — Colorado Medicaid via the Gainwell SFTP endpoint. Other payers are configured via
cyclone/providers.pyPayerConfigrows (SP9).
4. Backend architecture
4.1 Package layout
The cyclone package is a single namespace rooted at backend/src/cyclone/. 22 top-level modules + 4 subpackages. The store (SP21 in-flight split) is the largest; everything else is small and focused.
backend/src/cyclone/
├── __main__.py — entry point; `python -m cyclone` → CLI, `python -m cyclone serve` → uvicorn
├── __init__.py — `__version__`, package init
├── api.py — FastAPI app, route includes, lifespan (3,548 LOC — the only large file)
├── api_helpers.py — request/response formatters, error → HTTP mapping
├── api_routers/
│ ├── __init__.py
│ ├── acks.py — 999/TA1/271/277CA acknowledgments
│ ├── admin.py — admin-only endpoints (validate-provider, scheduler, backup)
│ ├── health.py — GET /api/health (SP19 deep snapshot)
│ └── ta1_acks.py — TA1-specific (split from acks for the SP3 lifecycle)
├── audit_log.py — SHA-256 hash-chained append-only log (SP11)
├── backup.py — SP17 primitives: derive_key, encrypt, decrypt, BackupFile
├── backup_scheduler.py — SP17 BackupScheduler wrapper
├── backup_service.py — SP17 coordinator: create/list/restore/verify/prune
├── batch_diff.py — 837 ↔ 835 diff (SP4)
├── clearhouse/
│ └── __init__.py — Clearhouse, SftpClient (SP9, SP13)
├── cli.py — Click CLI: `cyclone serve`, `cyclone validate-npi`, `cyclone backup`, etc.
├── db.py — SQLAlchemy engine, SessionLocal, ORM models, init_db, reinit_engine
├── db_crypto.py — SP12 SQLCipher connect-creator + is_encryption_enabled()
├── db_migrate.py — runs the 12 SQL migrations in order
├── edi/
│ └── filenames.py — X12 filename convention helpers (SP9)
├── inbox_lanes.py — 5-lane Inbox computation (SP14)
├── inbox_state.py — claim ↔ inbox state mapping helpers
├── inbox_state_277ca.py — SP10 277CA inbox state integration
├── logging_config.py — SP18 structured JSON + PII scrubber
├── migrations/ — 12 SQL files (0001_initial through 0012_backups)
├── npi.py — SP20 NPI Luhn + Tax ID validation
├── parsers/ — X12 parser subpackage
│ ├── parse_837.py — 837P ingest
│ ├── parse_835.py — 835 ingest
│ ├── parse_999.py / parse_ta1.py
│ ├── parse_270.py / parse_271.py
│ ├── parse_277ca.py — SP10 277CA ingest
│ ├── models*.py — one Pydantic model module per transaction type
│ ├── segments.py — X12 segment tokenizer
│ ├── validator.py — 837P rule registry (R-codes)
│ ├── validator_835.py — 835 validator
│ ├── serialize_837.py — SP8 outbound 837P serializer
│ ├── serialize_270.py / serialize_999.py
│ ├── cas_codes.py — CAS reason-code table
│ ├── batch_ack_builder.py
│ ├── writer.py / writer_835.py
│ └── exceptions.py
├── payers.py — Payer ORM row accessor (SP9)
├── providers.py — Payer / Clearhouse / Provider ORM row DTOs (SP9)
├── pubsub.py — NDJSON live-tail EventBus
├── rebill/ — (SP41 — in-window rebill pipeline)
│ ├── parse_835_svc.py — SVC-level 835 reparse w/ member_id
│ ├── reconcile.py — visit-to-835 join (member, procedure, DOS)
│ ├── carc_filter.py — CARC-aware filter
│ ├── timely_filing.py — 120-day DOS age gate
│ ├── pipeline_a.py — denied/partial → frequency-7
│ ├── pipeline_b.py — NOT_IN_835 → fresh 837Ps by (member, week)
│ ├── summary.py — summary CSV generator
│ ├── run.py — orchestrator (run_rebill)
│ ├── pull_999_acks.py — 999-ack dump + classify
│ └── __init__.py — package surface
├── reconcile.py — 835 → 837 auto-reconciliation engine
├── scheduler.py — SP16 MFT polling scheduler
├── scoring.py — 4-field lane scoring (40/25/20/15, hard-coded — backlog item)
├── secrets.py — Keychain access wrapper (SP9)
├── security.py — security headers, HealthSnapshot, deep health probe (SP19)
└── store.py — CycloneStore facade (2,423 LOC; SP21 in-flight split)
4.2 Module responsibility map (one line each)
| Module | Responsibility |
|---|---|
__main__ |
Dispatch serve vs CLI; honor CYCLONE_PORT / CYCLONE_RELOAD; emit AUTH_DISABLED WARNING if cyclone.auth.deps.AUTH_DISABLED is True at boot |
api |
FastAPI app, lifespan, route includes, 999/270/271 endpoints, exception → HTTP |
auth.* |
HTTP-layer login (bcrypt + HttpOnly session cookie). Submodules: deps (the matrix_gate FastAPI dependency, AUTH_DISABLED flag), routes (/api/auth/login, /api/auth/logout, /api/auth/me), users, sessions, permissions (role matrix), bootstrap (env-var first-admin), cli (cyclone admin create-user / rotate-password) |
api_helpers |
Error → status code mapping, common response shapes |
audit_log |
Append + verify a SHA-256 hash-chained log (SP11) |
backup |
Pure crypto + file I/O for SP17 backup format |
backup_scheduler |
Wrap BackupService in a tick() loop (SP17) |
backup_service |
Create / list / restore / verify / prune (SP17) |
batch_diff |
Pairwise diff of two 837 batches (SP4) |
clearhouse |
Clearhouse + SftpClient (SP9 stub, SP13 paramiko) |
cli |
cyclone console script: serve, validate-npi, backup, list-batches, … |
db |
Engine factory, SessionLocal(), all ORM rows, init_db(), reinit_engine() |
db_crypto |
SQLCipher branch + is_encryption_enabled() (SP12) |
db_migrate |
Walk migrations/*.sql in order, apply pending |
edi.filenames |
Parse / construct X12 file names (SP9) |
inbox_lanes |
Compute the 5 lanes (rejected, payer_rejected, candidates, unmatched, done_today) |
inbox_state |
Pure helpers for state ↔ lane mapping |
inbox_state_277ca |
SP10 277CA state integration (monotonic rule) |
logging_config |
SP18 JSON formatter + PII scrubber |
npi |
SP20 pure validators (Luhn over 80840+body, EIN prefix check) |
parsers.* |
X12 transaction set parsers (one per type) |
payers / providers |
SP9 ORM row DTOs |
pubsub |
In-process NDJSON EventBus for live tail |
reconcile |
835 → 837 auto-match + per-line CAS reconciliation |
scheduler |
SP16 MFT polling scheduler (per-payer tick) |
scoring |
4-field lane scoring weights (40/25/20/15) |
secrets |
Keychain get/set/delete via keyring |
security |
Security headers, deep HealthSnapshot (SP19) |
store |
CycloneStore facade: add / get / list / manual_match / iter_* / recent_activity |
4.3 The CycloneStore facade
The store is the single read/write surface for the database. Every endpoint that mutates state goes through cyclone.store (eventually the cyclone.store.* subpackage after SP21 lands). The facade pattern preserves public API:
store.add(record)— add a parsed batchstore.get(batch_id),get_batch(batch_id),list(limit),all()store.manual_match(claim_id, remit_id)— raisesAlreadyMatchedError(→ 409)store.manual_unmatch(claim_id)— raisesNotMatchedError(→ 409)store.iter_claims(...),iter_remittances(...)store.distinct_providers(),recent_activity(limit)store.list_unmatched(kind="both")store.export_*()— CSV/JSON dumpers
All persistence flows through SQLAlchemy sessions via db.SessionLocal()(). The engine is single-connection-per-request via FastAPI dependency injection. SP21 splits store.py (2,423 LOC) into a cyclone/store/ subpackage with one module per domain (batches, inbox, claim_detail, acks, backups, providers, etc.) and a thin facade that delegates every method to its domain module. After SP21, the public API of cyclone.store is unchanged; the only thing that changes is the file layout.
4.4 The parser pipeline
All inbound files follow the same 5-stage pipeline:
X12 .txt on disk
↓
[1] tokenize (parsers/segments.py: split on ~, then ^, then *:)
↓
[2] segmentize (build Segment records with elements + repeats)
↓
[3] model (parsers/models_*.py: Pydantic models per transaction type)
↓
[4] validate (parsers/validator.py: R-codes; raise ParseError on fatal)
↓
[5] write (store.add(record); record → DB rows)
The 837P validator has the most rules (R020, R021, R034, R035, R200, R210 — see 837P parser spec §2.4 for the registry). The 835 validator is simpler (no R-codes, just a model completeness check + a status_code allowlist). 999/TA1/270/271/277CA have no validators — they trust the upstream.
The writer is responsible for: (a) creating the batches row, (b) creating N claims / remittances / cas_adjustments rows, (c) emitting an activity_events row, (d) for 837P, the synthetic claim-status from the 999 envelope, (e) for 835, triggering reconcile.apply_payment to auto-match. If any step raises, the entire batch is rolled back (single SQLAlchemy transaction).
4.5 The reconciliation engine
cyclone.reconcile is a pure-functions module. The store calls into it for every 835 ingest:
apply_payment(claim, remit, strategy) -> ApplyIntent— happy path; returns the newMatchrow + state transitionapply_reversal(...)— handles a reversal 835 (the original payment + the reversal both stay in the DB; theis_reversalflag on theMatchrow disambiguates)recompute_line_totals(claim)— for SP7's per-line audit, when an 835 carries CAS adjustments
The engine is pure (no DB access). The store orchestrates the SQL writes after a successful apply_* returns. The 4-field scoring (40/25/20/15 for charge amount / NPI / patient ID / service date) lives in cyclone.scoring and is invoked by inbox_lanes.compute_lanes for the candidates and unmatched lanes.
4.6 The audit chain
cyclone.audit_log is a separate, append-only log table (audit_log — distinct from activity_events which is the un-chained /activity feed). Every event type the system can emit has a key in the chain:
claim.submitted,claim.rejected,claim.matched,claim.unmatchedclaim.payer_rejected,claim.payer_rejected_acknowledgedclearhouse.submittedkey.rotated(SP15),backup.created,backup.restored,backup.verified(SP17)api.request_rejected(SP19)
Each row carries the SHA-256 of (prev_hash || kind || actor || payload_json || ts). The first row has prev_hash = 0x00 * 32. audit_log.verify_chain() walks the table top-to-bottom, recomputing each hash, and raises AuditChainError on the first mismatch with the row id and the expected vs. actual hash. SP19's deep health probe calls verify_chain() on every /api/health request so a broken chain surfaces in the operator's status badge within one tick.
5. Frontend architecture
5.1 Tech stack
See §3.2. The frontend is a Vite SPA with TypeScript strict mode, Tailwind utility classes, and Radix primitives. There is no SSR, no Next.js, no router data-loading — every page is a function component that calls useQuery and renders.
5.2 Route map
| Path | Page | Purpose |
|---|---|---|
/ |
Dashboard |
KPI tiles + ticker tape + recent activity |
/claims |
Claims |
Paginated claim list, filters, drill to claim drawer |
/remittances |
Remittances |
Paginated remit list, drill to remit drawer |
/providers |
Providers |
Provider rollup (NPIs, charge totals) |
/activity |
ActivityLog |
Append-only activity_events feed (NDJSON) |
/upload |
Upload |
Drag-drop or click to upload an 837P / 835 / 999 / 277CA |
/reconciliation |
ReconciliationPage |
Cross-claim reconciliation view (4-field score) |
/inbox |
Inbox |
5-lane inbox (SP14) with BulkBar |
/acks |
Acks |
999/TA1/270/271/277CA acknowledgment list |
/batches |
Batches |
Batch list with totals + validation summary |
/batch-diff |
BatchDiff |
Pairwise 837 batch diff |
* |
NotFound |
404 |
5.3 State management
Two layers:
-
Server state —
@tanstack/react-query. Every page uses one or moreuseQuerycalls. The query key shape is["domain", id?, filters?]. Mutations useuseMutationand invalidate the relevant keys. There is no global refetch strategy; each page controls its own stale time. -
Client state —
zustand. Two stores:tail-store.ts— the live-tail connection state (reconnecting, last event id, subscriber counts). Powers the bottom-right TailStatusPill.DrillStackProvider(insrc/components/drill/) — the SP22 universal-drilldown stack. A LIFO of drill contexts (claim → claim line → match → CAS adjustment) so the back button works across drawers.
There is no Redux, no Context for global state, no MobX. The combination of react-query for server state + zustand for client state is intentional — see universal-drilldown spec for the rationale.
5.4 Live tail (NDJSON pubsub)
The Activity feed and the TailStatusPill share a single connection: GET /api/tail?since=<last_event_id>. The server returns Content-Type: application/x-ndjson and streams one JSON object per line. Each line is one of:
{kind: "heartbeat", ts: "..."}— every 5s when no events{kind: "event", id: <int>, event_type: "...", payload: {...}}— when something happens{kind: "error", message: "..."}— server-initiated close (rare)
The client uses fetch + ReadableStream to consume the body and dispatches each line to the relevant useQuery cache via queryClient.setQueryData. The connection auto-reconnects on error with exponential backoff. See live-tail spec for the wire format and reconnection policy.
6. Data model
6.1 Migrations 0001–0014
The schema evolves in ordered SQL files under backend/src/cyclone/migrations/. Each file starts with -- version: N and is applied in order by db_migrate.py. There is no migration framework — db_migrate.py is a 30-line file that checks the schema_version row and applies everything newer. The first 12 ship on main; 13 + 14 are the auth migrations that landed in main on 2026-06-23.
| # | File | What it adds |
|---|---|---|
| 1 | 0001_initial.sql |
batches, claims, remittances, cas_adjustments, matches, activity_events (6 tables) |
| 2 | 0002_acks.sql |
acks table (999/271) |
| 3 | 0003_drop_claims_remits_unique_constraints.sql |
Spec-bug fix: drop the UNIQUE constraints that prevented reversal flows |
| 4 | 0004_rejections_and_state_history.sql |
rejections table + claim_state_history |
| 5 | 0005_create_ta1_acks.sql |
ta1_acks table |
| 6 | 0006_line_reconciliation.sql |
line_reconciliations (SP7) |
| 7 | 0007_providers_payers_clearhouse.sql |
providers, payers, clearhouses (SP9) |
| 8 | 0008_payer_rejected_columns.sql |
claim.payer_rejected_* columns (SP10) |
| 9 | 0009_audit_log.sql |
audit_log hash-chained table (SP11) |
| 10 | 0010_payer_rejected_acknowledged.sql |
claim.payer_rejected_acknowledged_* columns (SP14 hand-off) |
| 11 | 0011_processed_inbound_files.sql |
processed_inbound_files idempotency table (SP16) |
| 12 | 0012_backups.sql |
db_backups table (SP17) |
| 13 | 0013_auth_users_and_sessions.sql |
users + sessions tables, users.password_hash (bcrypt) — landed 2026-06-23 |
| 14 | 0014_audit_log_user_id.sql |
audit_log.user_id FK to users.id (forward reference to 0013) — landed 2026-06-23 |
The SP22 parse-then-decide migrations (drop claims UNIQUE constraint + relax PK to (batch_id, id)) are renumbered to 0015 + 0016 on the claims-unique-fix worktree so the auth migrations stay at the front of the chain; they'll keep those numbers when SP22 lands.
6.2 ERD (high level)
batches (1) ─┬─< (N) claims ─┬─< (N) matches >─ (N) ─┬─ (1) remittances ─< (N) cas_adjustments
│ │ │
│ └─< (N) line_reconciliations (SP7)
│ └─< (N) rejections / claim_state_history
│
├─< (N) acks (999/271)
├─< (N) ta1_acks
└─< (N) activity_events
claims (1) ─< (N) line_reconciliations ─> (1) cas_adjustments
audit_log (no FKs — append-only, hash-chained)
db_backups (no FKs — directory + filename, status row)
processed_inbound_files (no FKs — path + hash, idempotency key)
providers (1) ─< (N) claims
payers (1) ─< (N) claims
clearhouses (1) ─< (N) payers
The batches row is the parent of every parsed X12 file. A batch is the unit of atomicity for parsing: either every claim/remit/ack in the file lands, or the file rolls back. The claims.state column drives the lifecycle (§6.3); the matches table is the join that says "this 835 paid this 837" (with a strategy field for audit: auto vs. manual).
6.3 Claim lifecycle (7 states)
The claims.state column is the canonical state of a claim in the operator's workflow:
submitted ──> accepted ──> paid
│ │ │
│ │ └─> reversed
│ │
│ └─> denied ──> appealed ──> paid
│
└─> rejected (envelope 999)
│
└─> payer_rejected (277CA A4/A6/A7)
submitted— 837P ingested, awaiting 999 ACKaccepted— 999 ACK came back clean; sitting in the trading partner's queuepaid— 835 matched, payment postedreversed— 835 reversal applied (the original payment + the reversal are both preserved; thestatereflects the most recent)denied— 835 came back with a denial status code (CAS group CO)appealed— operator flagged for appeal workflowrejected— envelope-level 999 rejection (syntactic / structural)payer_rejected— semantic 277CA rejection (A4/A6/A7 — claim accepted by envelope but rejected at the line level). SP10 introduced this state and theclaim.payer_rejected_*columns.
State transitions are explicit functions in cyclone.store (e.g. mark_accepted, mark_paid, mark_rejected). Each transition emits an activity_events row and, for the security-relevant ones, an audit_log row.
6.4 Audit chain integrity
The audit_log table is a separate chain from activity_events:
activity_events— un-chained, powers the/activityfeed. The activity feed can be repopulated from the underlying state if rows are missing; integrity is informational.audit_log— SHA-256 hash-chained, powers compliance / forensics. A row cannot be deleted without breaking the chain.audit_log.verify_chain()is the integrity check; the SP19 deep health probe runs it on every/api/healthcall.
The two are intentionally separate because they have different retention and integrity requirements. The convention-vs-trigger tradeoff is documented in the SP11 spec §7.
7. Data flow
7.1 File ingest (manual upload)
Operator ──drag-drop──> /upload (React)
│
│ POST /api/upload (multipart, .txt file)
v
FastAPI route
│
│ save to /tmp/cyclone-uploads/<uuid>.txt
v
parser.dispatch(kind) (auto-detect 837/835/999/TA1/270/271/277CA by ISA segment)
│
│ (the 5-stage pipeline from §4.4)
v
CycloneStore.add(record) (single SQLAlchemy transaction)
│
├─> emit activity_events row
├─> for 835: trigger reconcile.apply_payment
├─> for 999/TA1/277CA: mark the source claim
└─> commit
│
│ emit NDJSON event on the pubsub bus
v
live-tail subscribers (Activity feed, TailStatusPill)
The HTTP response is the new batch_id + a counts summary (e.g. {"batch_id": "...", "claims": 12, "remittances": 0, "validation_issues": 0}). The 999 is the only case where the response carries an ack payload — the operator can read the parsed 999 directly without a second request.
7.2 File ingest (SFTP polling — SP16)
every N seconds (default 30s, per-payer config):
Scheduler.tick()
│
├─> for each clearhouse:
│ ├─> SftpClient.list_inbound() (paramiko, SP13)
│ ├─> for each new .x12 file:
│ │ ├─> check processed_inbound_files (idempotency)
│ │ ├─> SftpClient.read_file(path) (to /tmp/...)
│ │ ├─> parser.dispatch(...)
│ │ ├─> CycloneStore.add(record)
│ │ ├─> INSERT processed_inbound_files (path, sha256, ts)
│ │ └─> emit activity_events row
│ └─> log tick result (count, errors, ms)
└─> heartbeat to /api/health (SP19 reports last tick ts)
The idempotency table is the linchpin: a crash between "read file" and "INSERT processed" results in the same file being processed again on the next tick. The 5-step ordering (SELECT → INSERT side-effects → INSERT idempotency; IntegrityError as the correctness backstop) is detailed in the SP16 spec §3.4.
7.3 File ingest (manual upload, replayed by pipeline)
cyclone-pipeline/ (sibling project) does the same as the operator dragging-and-dropping — it POSTs to /api/upload with the file. There's no separate "pipeline" ingestion path. The pipeline agent is a thin wrapper that adds preflight + SFTP submit + wait-for-999 around the Cyclone API.
7.4 Auto-match on 835 ingest
When an 835 lands, the writer calls reconcile.apply_payment(claim, remit, strategy) for each (claim, remit) pair in the file. The strategy is one of:
auto:pcn_npi_charge— patient_control_number + provider_npi + charge_amount match (the strongest)auto:pcn_charge— patient_control_number + charge_amount (the most common in practice)manual— the operator uses/reconciliationto pair by hand; the strategy is recorded for audit
Each auto-match creates a matches row + transitions the claim state + emits an activity_events row + emits an audit_log row. The match rate is visible in the Dashboard KPI tile.
7.5 Outbound 837P (resubmit)
Two paths:
- Single claim —
GET /api/claims/{id}/serialize-837(SP8). The serializer round-trips the claim throughserialize_837.py, validates with the R-code registry, and returns the X12 text. The Claim drawer's "Download 837" button calls this and triggers a browser download. - Bundle ZIP —
POST /api/claims/resubmit?download=zip(SP8). The serializer rebuilds N 837P files (one per claim in the request), zips them with a manifest, and returns the ZIP. The Inbox BulkBar's "Resubmit selected" calls this.
Both paths are read-only against the DB (they don't mutate the claim state). The operator is expected to download the regenerated file, then submit it manually via SFTP (or via cyclone-pipeline run).
7.6 Acknowledgement ingestion (999, TA1, 277CA)
These are the three "ack" transaction types the operator's trading partner sends back after a submission:
- 999 — functional ack (envelope-level). Triggers
claim.state = accepted | rejected. - TA1 — interchange ack. Triggers an activity_events row; no state change.
- 277CA — claim status (line-level). Triggers
claim.payer_rejectedif any line is A4/A6/A7. Theclaim.payer_rejected_*columns carry the status code + the 277CA's batch id for the audit trail.
All three are ingested via the same /api/upload path; the parser auto-detects the transaction type from the ISA segment. The SP10 spec describes the monotonic rule: a later 277CA cannot clear a previous rejection. This is the regulatory guarantee the SP11 audit chain relies on.
8. API surface
8.1 REST endpoints (by domain)
The full list is in REQUIREMENTS.md §6.3 traceability matrix. The structure is:
/api/health (SP19 deep health probe)
/api/tail?since=<id> (NDJSON live tail)
/api/upload (multipart, any X12 type)
/api/parse-837 (parse + return Pydantic, do not persist — pipeline agent)
/api/batches (list, with totals + validation)
/api/batches/{id} (detail)
/api/batches/{id}/diff?against={other_id} (batch_diff)
/api/batches/{id}/serialize-837 (SP8 — round-trip)
/api/claims (list, paginated, filterable)
/api/claims/{id} (detail, with remits + matches + line_recons)
/api/claims/{id}/serialize-837 (SP8)
/api/claims/resubmit?download=zip (SP8 bulk)
/api/remittances (list)
/api/remittances/{id} (detail)
/api/providers (rollup)
/api/providers/{npi} (single provider, all claims)
/api/activity?since=<id> (un-chained feed, paginated)
/api/inbox (SP14 5-lane view, filterable)
/api/inbox/payer-rejected/acknowledge (SP10 + SP14)
/api/acks (999/271/277CA list)
/api/acks/{id} (detail)
/api/ta1-acks (TA1 list)
/api/admin/validate-provider?npi=&tax_id= (SP20)
/api/admin/db/rotate-key (SP15)
/api/admin/db/fingerprint (SP12)
/api/admin/backup/create (SP17)
/api/admin/backup/list (SP17)
/api/admin/backup/{id}/restore?confirm=true (SP17, two-step)
/api/admin/backup/{id}/verify (SP17)
/api/admin/scheduler/status (SP16)
/api/admin/scheduler/tick (SP16 — manual trigger)
/api/admin/scheduler/start (SP16)
/api/admin/scheduler/stop (SP16)
8.2 Live-tail NDJSON format
GET /api/tail?since=<last_event_id> returns Content-Type: application/x-ndjson and streams:
{"kind":"event","id":1234,"event_type":"claim.submitted","payload":{"claim_id":"...","batch_id":"...","ts":"2026-06-23T..."}}
{"kind":"event","id":1235,"event_type":"claim.matched","payload":{"claim_id":"...","remit_id":"...","strategy":"auto:pcn_npi_charge"}}
{"kind":"heartbeat","ts":"2026-06-23T..."}
{"kind":"event","id":1236,"event_type":"claim.payer_rejected","payload":{"claim_id":"...","status_code":"A7","source_277ca_id":"..."}}
Heartbeats every 5s when no events. The client reconnects on error with exponential backoff capped at 30s. See live-tail spec for the full wire format.
9. Cross-cutting concerns
9.1 Logging (SP18)
cyclone.logging_config configures a single JSON formatter that:
- Emits one JSON object per line to stdout (12-factor)
- Includes
ts,level,logger,msg,module,line,process,thread - Scrubs PHI/PII fields by key name (
npi,member_id,patient_name,ssn_last4,charge_amount,paid_amount,tax_id) — replaced with*** - Redacts secrets by string match (anything matching
password=...&orBearer ...)
The PII scrubber is a best-effort defense-in-depth, not a security boundary. Real PHI protection comes from the SQLCipher layer (SP12) and the Keychain (no secrets in env). The scrubber exists so an operator tailing logs to debug doesn't accidentally paste a log line into a chat.
9.2 Audit chain (SP11)
See §4.6. The chain is checked on every /api/health request and exposes status="degraded" if broken. The operator sees the status in the TailStatusPill and the Dashboard KPI tile.
9.3 PHI/PII handling
- Storage — SQLCipher at rest (opt-in, SP12). Daily encrypted backups (SP17). Tamper-evident audit chain (SP11). 6-year retention via automated backup rotation.
- In transit (SFTP) — TLS to the trading partner; no plaintext EDI over the wire.
- In the UI — no PHI is shown on the Dashboard or Inbox (only counts + KPIs). The Claim drawer and Remit drawer show full PHI; these pages are not in the route map's preview and require an explicit navigation click.
- In logs — scrubbed by key name (see §9.1).
- In analytics — no analytics. No third-party tracking. No telemetry. The frontend makes no requests outside
127.0.0.1:8000.
9.4 Error model
Every API error is a JSON object with {"error": "<code>", "message": "<human>", "detail": <optional>}. The mapping is in api_helpers.py:
| Error class | HTTP status | error code |
|---|---|---|
AlreadyMatchedError |
409 | already_matched |
NotMatchedError |
409 | not_matched |
InvalidStateError |
409 | invalid_state |
ParseError |
422 | parse_error |
ValidationError (R-codes) |
422 | validation_failed (issues in detail) |
FileNotFoundError |
404 | not_found |
ValueError |
400 | bad_request |
PermissionError |
403 | forbidden (raised by matrix_gate when the role is below the endpoint's required role) |
| (anything else) | 500 | internal (with a request id for log correlation) |
The 409-vs-422-vs-400 split is intentional: 409 for state-machine violations (the operator's input is well-formed but conflicts with current state), 422 for parse / validation, 400 for bad input shape.
10. Operational concerns
10.1 Startup order
python -m cyclone serve → cyclone.api.lifespan runs:
db.init_db()— apply pending migrations (0001-0012, plus 0013/0014 on theclaims-unique-fixworktree)- Seed
cyclone.providerswith the SP9 defaults if the table is empty (PayerConfig for Colorado Medicaid + Gainwell) - Configure
BackupService(SP17) — open backup dir, read passphrase from Keychain - Configure
Scheduler(SP16) — wire MFT polling for each clearhouse - If
CYCLONE_SCHEDULER_AUTOSTART=true(default), start both schedulers - Mount the FastAPI app and start uvicorn
The ordering is load-bearing: the store facade cannot be used until init_db finishes, and the schedulers cannot query the DB until the seed step populates the clearhouse rows.
10.2 MFT scheduler (SP16)
cyclone.scheduler.Scheduler runs a per-clearhouse tick on a configurable interval (default 30s, per CYCLONE_SCHEDULER_INTERVAL env var). The tick is described in §7.2. The scheduler is opt-in via CYCLONE_SCHEDULER_AUTOSTART; the operator can start/stop it from the admin endpoints or via POST /api/admin/scheduler/tick for a manual tick.
The 4-step idempotency sequence (SELECT → INSERT side-effects → INSERT idempotency; IntegrityError as the correctness backstop) is the linchpin. The scheduler's _already_processed + _record pair wraps the INSERT to handle the case where two ticks race (an extremely rare event given the 30s interval, but possible during operator-driven manual ticks).
10.3 Backup scheduler (SP17)
cyclone.backup_scheduler.BackupScheduler wraps BackupService and ticks on its own interval (default 24h, per CYCLONE_BACKUP_INTERVAL env var). On each tick: create a backup, prune anything older than retention_days (default 30). Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART.
The encryption layer is independent of SQLCipher: AES-256-GCM with a passphrase-derived key (PBKDF2-HMAC-SHA256, 200k iterations). The passphrase lives in the macOS Keychain. If the passphrase is missing, the backup layer falls back to deriving a key from the SQLCipher key (less ideal but never silently broken — the operator's /api/health reports status="degraded" with a backup.passphrase_missing flag).
10.4 Health probe (SP19)
GET /api/health is a deep snapshot:
- db — open a session,
SELECT 1 - scheduler — last tick ts, is alive
- pubsub — current subscriber counts per event kind
- batch — most recent batch id + ts
- audit_log —
verify_chain()result (deep check on every call; 1ms on a healthy chain, fails loudly on a broken one) - backup — last backup ts, last verify result, passphrase presence
- encryption —
is_encryption_enabled()boolean
Returns status="ok" only when every subsystem is healthy. status="degraded" if any subsystem is unhappy but the API itself is responsive. The status is shown in the TailStatusPill (bottom-right) and the Dashboard KPI tile.
10.5 Shutdown
SIGTERM (or Ctrl-C) → uvicorn drains in-flight requests → cancels the MFT scheduler (in-flight tick completes via asyncio.to_thread) → cancels the backup scheduler (in-flight tick completes) → closes the SQLAlchemy engine → exits 0. There is no checkpoint; the in-flight tick is allowed to complete naturally because the idempotency table makes a partial-tick restart safe.
11. Deployment
11.1 Local dev (default)
The default deployment. Two terminals:
# Terminal 1 — backend on 0.0.0.0:8000 (always binds to all interfaces; firewall is what restricts reachability)
cd backend
.venv/bin/python -m cyclone serve
# Terminal 2 — frontend on localhost:5173
cd ..
npm run dev
The frontend reads its backend URL from VITE_API_BASE_URL (default empty). Create .env.local at the repo root with VITE_API_BASE_URL=http://127.0.0.1:8000. Without it, the UI falls back to its in-memory sample store via the data adapter (parses are disabled).
11.2 Optional Docker (SP23 — product fork)
A Ubuntu + Docker + auth + RBAC + LAN-bind spec exists at docs/superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md. This is a product fork awaiting user decision (per REQUIREMENTS.md §6.2). The v1 single-host single-operator posture assumes local-only; SP23 changes the threat model to "remote operator on LAN."
11.3 Single-host "production"
For a single operator on a single machine, the dev deployment IS the production deployment. There is no separate build / deploy pipeline; the v1 distribution model is git clone && pip install -e '.[dev]' && npm install && python -m cyclone serve. A release-time gate (not yet implemented — backlog item) would run pytest (backend) and npm test (frontend) and fail if any test is red.
12. Out of scope (v1)
The following are explicitly NOT built in v1, by design:
- Multi-user / multi-host — login is required (single-user model today; SP23 adds RBAC). LAN-bind, Docker, reverse proxy still out of scope. (SP23 covers the LAN-bind / Docker / RBAC product fork.)
- Other trading partners — only Colorado Medicaid via Gainwell is configured. The
PayerConfigmechanism (SP9) supports adding more; the spec/plan/CLI work for that is on the backlog. - 837I / 837D / 834 / 820 / 275 / 278 / 276-277 — only 837P and 835 are first-class. Other transaction types are listed in REQUIREMENTS.md §6.2 as backlog items.
- NPPES real-time lookup — only local NPI checksum validation (SP20). The operator who wants real registry lookup has to wire it in later.
- EHR / PM integration — no webhook in/out, no callable
POST /api/parse-837(the pipeline agent uses/api/uploadfor now). - COB / secondary claims — no automatic coordination-of-benefits generator. The operator generates a secondary 837P by hand from the CAS adjustments.
- Real-time eligibility — no SFTP pickup + SOAP + 271 round-trip. The 270/271 parsers exist; the live cycle doesn't.
Each of these is a separate SP. The "v1 done" bar is the 22 shipped SPs (SP1–SP22) on main plus the REQUIREMENTS.md §8 DoD.
13. References
13.1 Top-level docs
- Requirements — FRs, NFRs, DoD, traceability
- README — install + dev + pipeline agent
13.2 Specs (per sub-project)
All in docs/superpowers/specs/:
- SP1 production-readiness:
2026-06-19-cyclone-production-readiness-design.md - SP2 DB + reconciliation:
2026-06-19-cyclone-db-reconciliation-design.md - SP3 EDI features:
2026-06-20-cyclone-edi-features-design.md - SP4 claim drawer:
2026-06-20-cyclone-claim-drawer-design.md - SP5 live tail:
2026-06-20-cyclone-live-tail-design.md - SP7 line reconciliation:
2026-06-20-cyclone-line-reconciliation-design.md - SP8 serialize 837:
2026-06-20-cyclone-serialize-837-design.md - SP9 multi-payer / NPI / SFTP stub:
2026-06-20-cyclone-multi-payer-npi-sftp-design.md - SP10 277CA:
2026-06-23-cyclone-sp10-277ca-payer-rejected-design.md - SP11 audit log:
2026-06-23-cyclone-sp11-hash-chained-audit-design.md - SP12 SQLCipher:
2026-06-23-cyclone-sp12-sqlcipher-encryption-design.md - SP13 SFTP client:
2026-06-23-cyclone-sp13-sftp-client-design.md - SP14 5-lane Inbox:
2026-06-23-cyclone-sp14-inbox-5lane-design.md - SP15 key rotation:
2026-06-23-cyclone-sp15-key-rotation-design.md - SP16 MFT scheduler:
2026-06-23-cyclone-sp16-mft-polling-design.md - SP17 encrypted backup:
2026-06-21-cyclone-encrypted-backup-design.md - SP18 structured logging:
2026-06-21-cyclone-structured-logging-design.md - SP19 security hardening:
2026-06-21-cyclone-security-hardening-design.md - SP20 NPI validation:
2026-06-21-cyclone-npi-validation-design.md - SP21 store split:
2026-06-21-cyclone-store-split-design.md; universal drilldown:2026-06-21-cyclone-universal-drilldown-design.md - SP22 parse-decide:
2026-06-21-cyclone-parse-decide-workflow-design.md; pipeline agent:2026-06-21-cyclone-pipeline-agent-design.md - SP23 (fork) Ubuntu + Docker:
2026-06-22-cyclone-ubuntu-docker-deployment-design.md
13.3 Plans
All in docs/superpowers/plans/. 27 files — one per spec (plus the round-2 backfill for SP9–SP20 in the 2026-06-23-cyclone-sp* naming).
13.4 Reference docs
docs/reference/co-medicaid.md— operator setup for SQLCipher + SFTP + the Gainwell trading-partner details (KP-specific).
13.5 Sibling project
cyclone-pipeline/— round-trip agent (sibling, not in this repo). See README §"Pipeline automation agent" for install + usage.