Merge branch 'sp24-doc-posture-alignment'
Align CLAUDE.md + REQUIREMENTS.md + ARCHITECTURE.md with the auth work
that landed on main on 2026-06-23; emit an AUTH_DISABLED WARNING at boot
when AUTH_DISABLED is true so misconfigured production deploys fail loudly.
Closes requirements R-1 (was Open; auth shipped via the origin/main merge,
SP24 reconciles the docs).
* CLAUDE.md — track in git for the first time
* docs/REQUIREMENTS.md + docs/ARCHITECTURE.md — replace every stale
'no auth' / 'no authentication' / 'no second party to authenticate'
claim with the v1 posture (bcrypt + HttpOnly session cookie; first
admin bootstrapped from CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD)
* backend/src/cyclone/__main__.py — boot-time WARNING when AUTH_DISABLED
* ARCHITECTURE §4.2 module map — add cyclone.auth.* package
* ARCHITECTURE §6.1 migrations table — list 0013 + 0014 with renumbering note
* .superpowers/skills/{cyclone-tests,cyclone-api-router,cyclone-spec} —
add AUTH_DISABLED / matrix_gate / auth-aware threat-model guidance
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
Plan: docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md
This commit is contained in:
cyclone
@@ -19,6 +19,10 @@ content negotiation + `tail_events`), and ~30 routes still inlined
|
||||
in `backend/src/cyclone/api.py`. The next refactor target is the
|
||||
parse endpoints.
|
||||
|
||||
## Auth gate (SP24)
|
||||
|
||||
Every router declared in `backend/src/cyclone/api_routers/` **must** carry `dependencies=[Depends(matrix_gate)]` at the `APIRouter(...)` declaration — not on each individual endpoint. The gate lives at `backend/src/cyclone/auth/deps.py:107` and the role matrix is at `backend/src/cyclone/auth/permissions.py`. The roles are `admin / user / viewer`; `matrix_gate` returns 401 when there's no session and 403 when the role is below the endpoint's required role. When `AUTH_DISABLED` is True (conftest autouse fixture flips it; `CYCLONE_AUTH_DISABLED=1` in prod-by-mistake), the gate short-circuits to a synthetic admin — see the SP24 spec for the threat-model implications. New routers get the gate by default; the auth-aware convention is `router = APIRouter(dependencies=[Depends(matrix_gate)])`.
|
||||
|
||||
## When to use
|
||||
|
||||
- **Adding an endpoint.** You're adding a new GET / POST handler —
|
||||
|
||||
@@ -10,9 +10,15 @@ a spec, a plan, an implementation branch, and a single atomic merge commit
|
||||
into `main`. This skill encodes the conventions so every increment follows
|
||||
the same shape and the commit history stays auditable.
|
||||
|
||||
As of this writing: **16 specs** in `docs/superpowers/specs/`, **12 plans**
|
||||
in `docs/superpowers/plans/`, and SP numbers used through **SP21** (the
|
||||
universal-drilldown design in progress). The next increment is **SP22**.
|
||||
As of this writing: **17 specs** in `docs/superpowers/specs/`, **13 plans**
|
||||
in `docs/superpowers/plans/`, and SP numbers used through **SP22**. **SP23**
|
||||
is the Ubuntu + Docker + RBAC product fork (awaiting user decision);
|
||||
**SP24** is the auth-posture alignment (docs-only). The next free increment
|
||||
is **SP25** after SP24 lands.
|
||||
|
||||
## Auth-aware spec template (SP24)
|
||||
|
||||
The threat-model section in the canonical SP-N spec template (`## 1. Scope`, second-to-last bullet) used to read "no second party to authenticate; no second host to harden against." **That phrasing is stale as of 2026-06-23** — the auth work landed in `main` and every backend endpoint requires login. New specs should instead state the auth boundary explicitly: "the auth boundary is HTTP (login required, bcrypt + HttpOnly session cookie); file-system threats remain the local-only threat model (SQLCipher at rest, macOS Keychain). SP23 changes the threat model to LAN-bound remote operator." Reference: [`docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md`](../../../docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md).
|
||||
|
||||
## When to use
|
||||
|
||||
|
||||
@@ -7,7 +7,11 @@ description: "Cyclone pytest + vitest fixture patterns, prodfiles layout, backen
|
||||
|
||||
The Cyclone test suite is split two ways: **backend pytest** (89 test files under `backend/tests/`, 13 flat fixtures in `backend/tests/fixtures/`, one autouse `conftest.py` that resets the DB per-test) and **frontend vitest** (59 `*.test.ts(x)` siblings across `src/`, two rendering styles — `@testing-library/react` and a custom `createRoot`+`Probe` shim). This skill codifies the conventions so additions stay consistent with what's already there.
|
||||
|
||||
As of this writing: **89 backend test files**, **13 flat backend fixtures**, **59 frontend `*.test.ts(x)` siblings**, and **23 prodfiles samples** across `docs/prodfiles/{837p-from-axiscare,835fromco,FromHPE,claims}/`. The next increment is **SP22**.
|
||||
As of this writing: **89 backend test files**, **13 flat backend fixtures**, **59 frontend `*.test.ts(x)` siblings**, and **23 prodfiles samples** across `docs/prodfiles/{837p-from-axiscare,835fromco,FromHPE,claims}/`. The next increment is **SP24** (SP23 is the Ubuntu+Docker fork, awaiting user decision).
|
||||
|
||||
## Auth flag (SP24)
|
||||
|
||||
The autouse `conftest.py` fixture at `backend/tests/conftest.py` flips `cyclone.auth.deps.AUTH_DISABLED = True` for the entire test session, so every test runs without a login round-trip. **Any new test that reads `cyclone.auth.deps.AUTH_DISABLED` directly will see `True`** — that's the test-suite reality, not a production reality. If you need a test that exercises the real gate, import `from cyclone.auth.deps import matrix_gate` and call it directly with a `Request` whose `state` carries a real session, or flip the flag back inside the test and reset it on teardown. The startup WARNING (`backend/src/cyclone/__main__.py`) is silent in tests by default because the conftest sets the flag before `bootstrap.run()` is called via `import`.
|
||||
|
||||
## When to use
|
||||
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## What this is
|
||||
|
||||
Cyclone is a self-hosted X12 EDI claims-management suite for a single billing office (Colorado Medicaid currently). It parses 837P professional claims and 835 ERA remittances (X12 005010X222A1 / 005010X221A1) and also handles 999, TA1, 270, 271, and 277CA. Local-only by design: binds to `127.0.0.1`, requires login (auth boundary is HTTP; bcrypt + HttpOnly session cookie; first admin bootstrapped from `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD` env vars; see SP24 spec for the full posture), no internet exposure.
|
||||
|
||||
Stack: one Python process (FastAPI + uvicorn, port 8000) + one Node process in dev (Vite, port 5173). The authoritative state is a single SQLite file at `~/.local/share/cyclone/cyclone.db` (or SQLCipher at the same path when the macOS Keychain entry + `sqlcipher3` are both present).
|
||||
|
||||
For the day-1 architecture read, see `docs/ARCHITECTURE.md` (process topology, module map, store facade, parser pipeline, pubsub). For the what-it-does read, see `docs/REQUIREMENTS.md` (FRs + NFRs + DoD).
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
# Backend (Python 3.11+)
|
||||
cd backend
|
||||
python -m venv .venv
|
||||
.venv/bin/pip install -e '.[dev]'
|
||||
|
||||
# Frontend (Node 20+)
|
||||
cd ..
|
||||
npm install
|
||||
```
|
||||
|
||||
Optional backend extras: `pip install -e '.[sqlcipher]'` (encryption at rest, SP12) and `pip install -e '.[sftp]'` (real SFTP, SP13).
|
||||
|
||||
## Dev (two terminals)
|
||||
|
||||
```bash
|
||||
# Terminal 1 — backend
|
||||
cd backend
|
||||
.venv/bin/python -m cyclone serve # default 127.0.0.1:8000
|
||||
# CYCLONE_PORT=... overrides port; CYCLONE_RELOAD=1 enables uvicorn --reload
|
||||
# Or: .venv/bin/uvicorn cyclone.api:app --reload --port 8000
|
||||
|
||||
# Terminal 2 — frontend
|
||||
npm run dev # Vite on http://localhost:5173
|
||||
```
|
||||
|
||||
Vite proxies `/api/*` to the backend at `http://127.0.0.1:${CYCLONE_PORT:-8000}` so relative-URL fetchers (the live-tail NDJSON streams in particular) resolve through the same origin. Override the backend port with `CYCLONE_PORT` in the frontend terminal too.
|
||||
|
||||
Create `.env.local` at the repo root with `VITE_API_BASE_URL=http://127.0.0.1:8000`. Without it, the UI runs against the in-memory zustand store and real EDI parsing is disabled.
|
||||
|
||||
## Test
|
||||
|
||||
```bash
|
||||
# Backend — full suite
|
||||
cd backend && .venv/bin/pytest
|
||||
|
||||
# Backend — one file
|
||||
cd backend && .venv/bin/pytest tests/test_api_999.py -v
|
||||
|
||||
# Backend — one test by node id
|
||||
cd backend && .venv/bin/pytest tests/test_api_999.py::test_parse_999_endpoint_happy_path -v
|
||||
|
||||
# Frontend — full suite
|
||||
npm test # alias for `vitest run`
|
||||
|
||||
# Frontend — one file
|
||||
npx vitest run src/hooks/useFoo.test.ts
|
||||
|
||||
# Frontend — typecheck
|
||||
npm run typecheck
|
||||
|
||||
# Frontend — build (tsc -b + vite build)
|
||||
npm run build
|
||||
|
||||
# Frontend — lint
|
||||
npm run lint
|
||||
```
|
||||
|
||||
**Conventions** (full detail in `.superpowers/skills/cyclone-tests/SKILL.md`):
|
||||
|
||||
- Backend tests live under `backend/tests/test_*.py`. Two flavors: `test_api_<topic>_<verb>.py` (FastAPI integration via `fastapi.testclient.TestClient`) and `test_<module>_<behavior>.py` (pure-unit). Autouse `conftest.py` points `CYCLONE_DB_URL` at `tmp_path/test.db`, calls `db._reset_for_tests()` + `db.init_db()`, and wires a fresh `EventBus` onto `app.state`.
|
||||
- Prodfiles (real EDI samples under `docs/prodfiles/<source>/`) are never read directly from a test — copy to `backend/tests/fixtures/<descriptive-name>.txt` first and reference as a module-level `Path` constant. The `fixtures/` dir is flat (no per-test subdirs).
|
||||
- Frontend tests are siblings: `useFoo.ts` → `useFoo.test.ts`, `ClaimDrawer.tsx` → `ClaimDrawer.test.tsx`. Setup is `// @vitest-environment happy-dom` plus `(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;`. Mock the API at the module boundary with `vi.mock("@/lib/api", ...)`; stub fetch with `vi.stubGlobal("fetch", vi.fn().mockResolvedValue(...))`. `vitest.config.ts` sets `VITE_API_BASE_URL=http://test.local` so the `api` module doesn't throw `notConfiguredError` before the mock fires.
|
||||
- Time-sensitive tests: frontend uses `vi.useFakeTimers()` + `vi.setSystemTime(...)` + `vi.advanceTimersByTime(ms)`. Backend passes explicit `datetime(...)` values. Don't add `await new Promise((r) => setTimeout(r, N))` — it's the legacy flaky pattern.
|
||||
|
||||
## Project-scoped skills (`.superpowers/skills/`)
|
||||
|
||||
Cyclone ships 8 skills under `.superpowers/skills/`. They auto-load by description match — no slash command needed. **Read the relevant skill before touching the matching subsystem.**
|
||||
|
||||
| Skill | Owns |
|
||||
|---|---|
|
||||
| `cyclone-spec` | The SP-N spec → plan → implement → merge flow (branch shape, file paths, commit prefixes, PR title, merge shape). |
|
||||
| `cyclone-tests` | pytest + vitest fixture patterns, prodfiles drop-in rule, determinism rules. |
|
||||
| `cyclone-edi` | EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1, R-codes, CAS mapping). |
|
||||
| `cyclone-tail` | Live-tail streaming wire format and the `useTailStream` + `useMergedTail` + `TailStatusPill` hook triplet. |
|
||||
| `cyclone-store` | `CycloneStore` facade, write-paths, pubsub event contract, SP21 split map. |
|
||||
| `cyclone-api-router` | FastAPI router conventions (`api_routers/`, `api_helpers.py`), response/error-envelope shapes. |
|
||||
| `cyclone-frontend-page` | React page conventions (TanStack Query `use<X>` hook, drawer, URL state, sibling test). |
|
||||
| `cyclone-cli` | CLI subcommand conventions (`cli.py`, exit codes, smoke tests). |
|
||||
|
||||
## The SP-N increment flow
|
||||
|
||||
Every feature ships as a numbered **SP-N increment**: spec → plan → implementation branch → single atomic merge into `main`. As of the last backfill, SP numbers are used through **SP22**; **SP23** is reserved for the Ubuntu + Docker + RBAC product fork (`docs/superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md`, awaiting user decision); the next free increment is **SP24**. Read `cyclone-spec` before starting a new one. Non-negotiable shape:
|
||||
|
||||
- **Branch:** `sp<N>-<short-kebab-topic>` (e.g. `sp22-line-reconciliation`).
|
||||
- **Spec path:** `docs/superpowers/specs/YYYY-MM-DD-cyclone-<topic>-design.md`, header `Status: Draft, awaiting user sign-off`, sections `Scope / Decisions / …`. Specs contain zero code blocks.
|
||||
- **Plan path:** `docs/superpowers/plans/YYYY-MM-DD-cyclone-<topic>.md`, header per `superpowers:writing-plans` with `Goal / Architecture / Tech Stack / Spec` metadata + numbered `- [ ] Step N:` tasks.
|
||||
- **Commit prefixes:** `feat(sp<N>): …`, `docs(spec): …`, `docs(plan): …`, `merge: SP<N> <topic> into main`.
|
||||
- **PR title:** `SP<N> <Topic>` (matches the merge-commit subject).
|
||||
- **Merge shape:** single atomic merge commit. **No squash** (collapses the audit trail) and **no rebase** (rewrites the SHAs the review was performed against). The SP-N merge commit *is* the record of the increment landing.
|
||||
|
||||
The matching skill to load alongside `cyclone-spec` depends on the subsystem the SP-N touches (see the "Related skills" section at the bottom of each skill file).
|
||||
|
||||
## Live-tail wire format
|
||||
|
||||
The Claims, Remittances, and Activity pages stay current without manual refresh. The backend publishes an internal event on every store write, the page opens a streaming HTTP connection to the matching `/api/<resource>/stream` endpoint, and new rows append to the table the moment they hit the database.
|
||||
|
||||
Endpoints (all accept the same query params as their non-streaming counterparts; `Content-Type: application/x-ndjson`):
|
||||
|
||||
| Method | Path | Subscribes to | Default sort |
|
||||
|---|---|---|---|
|
||||
| GET | `/api/claims/stream` | `claim_written` | `-submission_date` |
|
||||
| GET | `/api/remittances/stream` | `remittance_written` | `-received_date` |
|
||||
| GET | `/api/activity/stream` | `activity_recorded` | `-timestamp` (limit 50) |
|
||||
|
||||
Wire format: one JSON object per line, `{"type": ..., "data": ...}`. The first batch is the **snapshot** of currently-known rows, then `snapshot_end` with the count, then the **live** events. Known types: `item`, `snapshot_end`, `heartbeat` (keeps the connection alive on idle — clients flip to `stalled` after 30s of total silence), `item_dropped` (rare), `error`.
|
||||
|
||||
Status pill states (rendered by `<TailStatusPill>` in `src/components/TailStatusPill.tsx`): `live` (success), `connecting` (warning), `reconnecting` (warning), `stalled` (destructive, ↻ Reconnect button), `error` (destructive, ↻ Reconnect button), `closed` (destructive). Backoff on error: `1s → 2s → 4s → 8s → 16s → 30s` capped. `STALL_TIMEOUT_MS = 30_000` in `src/hooks/useTailStream.ts:53`. Heartbeat interval is `CYCLONE_TAIL_HEARTBEAT_S` env var, default 15s.
|
||||
|
||||
Frontend triplet for any live page: `use<X>(params)` (initial fetch) + `useTailStream(resource)` (opens the NDJSON stream, drives backoff/stall) + `useMergedTail(resource, baseItems, filterFn?)` (merges snapshot + tail, dedup'd by id). The subscription lives on the page, not inside the data hook — see `cyclone-frontend-page` for why.
|
||||
|
||||
## Backend at a glance
|
||||
|
||||
`backend/src/cyclone/` is a single namespace. The two largest files are `api.py` (~3,548 LOC, the only large file) and `store.py` (~2,423 LOC, the `CycloneStore` facade — SP21 is in flight to split it into a `cyclone/store/` subpackage; the public API stays unchanged). Subpackages: `api_routers/` (acks, admin, health, ta1_acks), `clearhouse/` (Clearhouse + SftpClient), `edi/` (filenames), `parsers/` (X12 transaction parsers + models + validators + serializers), `workflow/` (placeholder for future sub-project 6).
|
||||
|
||||
The store is the only read/write surface for the database; every mutating endpoint goes through it. All persistence flows through SQLAlchemy sessions via `db.SessionLocal()()`. SQLAlchemy ORM models live in `db.py`; 12 SQL migrations under `migrations/` (0001_initial through 0012_backups) are walked in order by `db_migrate.py`.
|
||||
|
||||
The parser pipeline is a 5-stage `tokenize → segmentize → model → validate → write_to_store` flow used for every inbound X12 type. Per-transaction parsers: `parse_837.py`, `parse_835.py`, `parse_999.py`, `parse_ta1.py`, `parse_270.py`, `parse_271.py`, `parse_277ca.py`. Each has a matching Pydantic model module (`models.py`, `models_835.py`, …) and a writer (`writer.py` / `writer_835.py`). The 837P serializer (`serialize_837.py`) is the byte-faithful outbound counterpart used by both single-claim download (`/api/claims/{id}/serialize-837`) and the bulk rejected-resubmit bundle (`/api/inbox/rejected/resubmit?download=true`).
|
||||
|
||||
The pubsub is `cyclone.pubsub.EventBus` — an in-process async fan-out broker. Publishers call `publish(kind, payload)`; subscribers receive via an async iterator. If a subscriber's per-kind queue is full, the oldest event is dropped so a slow consumer can't stall the producer. Bus is single-event-loop only (matches FastAPI/uvicorn).
|
||||
|
||||
Config: `config/payers.yaml` is the on-disk source for providers / payers / clearhouse, schema-validated at boot against a Pydantic model. Reload with `POST /api/admin/reload-config`. Original in-code `PAYER_FACTORIES` dict in `cli.py` is kept as a fallback for ad-hoc testing.
|
||||
|
||||
Secrets live in the macOS Keychain (via `keyring` + `cyclone.secrets`): SQLCipher key (service `cyclone`, account `cyclone.db.key`), SFTP password, backup passphrase. No secrets on disk in plaintext.
|
||||
|
||||
## Frontend at a glance
|
||||
|
||||
`src/` is React 18 + TypeScript + Vite. Routes register in `src/App.tsx` (11 pages, all under a `<Layout>` route wrapper). Pages are pure renderers — every page pairs with a `use<X>` data hook in `src/hooks/` and renders a `<PageHeader>` + a table/list/KPI grid. Drawers (`ClaimDrawer/`, `RemitDrawer/`, plus the new `ProviderDrawer/` and `AckDrawer/`) are mounted by the page and their open/close state is mirrored to the URL via `useDrawerUrlState` so deep-links round-trip. Drill-stack navigation is provided by `<DrillStackProvider>` in `src/components/drill/`.
|
||||
|
||||
State split: **server state** in TanStack Query (`@tanstack/react-query`); **ephemeral client state** in Zustand (`useTailStore` for live-tail append, plus the drill stack). The live-tail store is FIFO-capped at `TAIL_CAP = 10_000` per slice (`src/store/tail-store.ts:28`); `claims` and `remittances` are key-by-id with first-write-wins dedup, `activity` is an append-only array.
|
||||
|
||||
UI primitives in `src/components/ui/` are Radix-backed (`button`, `dialog`, `table`, `select`, `pagination`, `empty-state`, `error-state`, `filter-chips`, `skeleton`, `input`, `label`, `card`, `badge`, `skip-link`, `claim-state-badge`). Don't import a new UI library without discussion.
|
||||
|
||||
Path alias `@/` → `src/`. Configured in `vite.config.ts`, `vitest.config.ts`, and `tsconfig.app.json`.
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
# Parser
|
||||
python -m cyclone.cli parse-837 path/to/837p.txt --output-dir ./claims --payer co_medicaid [--strict] [--include-raw-segments]
|
||||
python -m cyclone.cli parse-835 path/to/835.txt --output-dir ./remits
|
||||
python -m cyclone.cli parse-999 inbound_999.txt
|
||||
python -m cyclone.cli parse-ta1 inbound_ta1.txt
|
||||
python -m cyclone.cli parse-277ca inbound_277ca.txt
|
||||
|
||||
# Validators
|
||||
python -m cyclone.cli validate-npi 1234567893
|
||||
python -m cyclone.cli validate-tin 721587149
|
||||
|
||||
# Other
|
||||
python -m cyclone serve # uvicorn
|
||||
python -m cyclone backup list
|
||||
python -m cyclone backup create --reason manual
|
||||
```
|
||||
|
||||
Exit codes are documented per subcommand in `cyclone-cli` — `0` for success, `2` for file-level failure, `1` for unexpected exceptions.
|
||||
|
||||
## Things that are easy to get wrong
|
||||
|
||||
- **`VITE_API_BASE_URL` matters.** With it empty, every `api` method throws `notConfiguredError()` and the UI falls back to the in-memory zustand store — parses are disabled and the live-tail streams never open.
|
||||
- **Prodfiles vs fixtures.** Tests must reference `backend/tests/fixtures/<name>.txt`, not `docs/prodfiles/<source>/<file>.txt`. The prodfiles dir is the source-of-truth archive; the fixtures dir is the stable test surface.
|
||||
- **SP-N merge shape.** No squash, no rebase. The merge commit *is* the audit trail. Squash collapses the per-commit history and breaks the SP-N audit trail.
|
||||
- **Don't put domain logic in JSX.** Conditional renderings, table sorting, and KPI math all belong in the `use<X>` hook or a pure helper under `src/lib/`.
|
||||
- **Don't open a drawer via local `useState`.** Use `useDrawerUrlState()` so the URL is the single source of truth — deep-links and reload-restore depend on it.
|
||||
- **Don't call `useTailStream` from inside a `use<X>` hook.** The subscription lives on the page so the lifecycle ties to whoever mounts the hook, not to whoever happens to call it.
|
||||
- **The store facade.** The public API of `cyclone.store` is preserved through SP21's split — call through the facade, not directly into the underlying modules.
|
||||
- **Encryption is optional, not required.** When the Keychain entry is missing **or** `sqlcipher3` is not installed, the DB falls back to plain SQLite. Don't fail boot on missing encryption.
|
||||
- **Local-only by design.** The backend binds to `127.0.0.1`, requires login (bcrypt + HttpOnly session cookie; see SP24 spec), and the threat model is still a stolen/imaged drive — SQLCipher at rest and the macOS Keychain handle that. The auth boundary is the HTTP layer; the file-system posture is unchanged. Don't add internet exposure. Don't disable auth without an explicit `CYCLONE_AUTH_DISABLED=1` env var (the escape hatch logs a WARNING at boot).
|
||||
</content>
|
||||
</invoke>
|
||||
@@ -801,6 +801,12 @@ backup API).
|
||||
|
||||
## Roadmap
|
||||
|
||||
> **Read order for new engineers:**
|
||||
> 1. [`docs/REQUIREMENTS.md`](docs/REQUIREMENTS.md) — what Cyclone does (FRs + NFRs + DoD + traceability). The single index tying the 22 shipped sub-projects to the 38 functional + 18 non-functional requirements and the test strategy.
|
||||
> 2. [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) — how it fits together (process topology, package layout, data flow, lifecycle, operational concerns).
|
||||
> 3. The per-SP spec under [`docs/superpowers/specs/`](docs/superpowers/specs/) for whatever you're touching.
|
||||
> 4. The per-SP plan under [`docs/superpowers/plans/`](docs/superpowers/plans/) if you're implementing.
|
||||
|
||||
Sub-projects 2 through 19 are **shipped**. See the [completeness
|
||||
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
|
||||
the honest gap analysis against the industry definition of a HIPAA
|
||||
|
||||
@@ -24,6 +24,20 @@ def main() -> None:
|
||||
from cyclone.auth import bootstrap
|
||||
bootstrap.run()
|
||||
|
||||
# SP24: if the AUTH_DISABLED escape hatch is on, scream at boot so a
|
||||
# misconfigured production deploy fails loudly. The flag is flipped by
|
||||
# ``CYCLONE_AUTH_DISABLED=1`` (see ``cyclone.auth.bootstrap``) and by the
|
||||
# pytest conftest autouse fixture (see ``.superpowers/skills/cyclone-tests``).
|
||||
from cyclone.auth import deps as _auth_deps
|
||||
|
||||
if _auth_deps.AUTH_DISABLED:
|
||||
import logging
|
||||
|
||||
logging.getLogger("cyclone").warning(
|
||||
"AUTH_DISABLED is set (CYCLONE_AUTH_DISABLED=1) — all requests "
|
||||
"treated as admin, dev only. Do NOT enable this in production."
|
||||
)
|
||||
|
||||
if len(sys.argv) >= 2 and sys.argv[1] == "serve":
|
||||
port = os.environ.get("CYCLONE_PORT", "8000")
|
||||
reload = os.environ.get("CYCLONE_RELOAD", "0") == "1"
|
||||
|
||||
@@ -0,0 +1,762 @@
|
||||
# Cyclone — Technical Architecture
|
||||
|
||||
> **Day-1 read for engineers joining Cyclone.** This is the top-level technical design: how the system fits together, what the boundaries are, and why it looks the way it does.
|
||||
>
|
||||
> **Companion docs:**
|
||||
> - **Requirements:** [`docs/REQUIREMENTS.md`](REQUIREMENTS.md) — what Cyclone does (FRs + NFRs + DoD + traceability).
|
||||
> - **Specs:** [`docs/superpowers/specs/`](superpowers/specs/) — design specs per sub-project (22 specs).
|
||||
> - **Plans:** [`docs/superpowers/plans/`](superpowers/plans/) — TDD-shaped implementation plans per sub-project (27 plans).
|
||||
>
|
||||
> **Scope of this doc:** architecture and design. The spec files own per-SP design; the plan files own the task order. This doc explains the *whole* — the boundaries between modules, the data flow across them, the lifecycle of an inbound file from upload to reconciled claim.
|
||||
|
||||
---
|
||||
|
||||
## 1. System overview
|
||||
|
||||
Cyclone is a self-hosted X12 EDI claims-management suite for a single billing office. The first deployment target is Colorado Medicaid (one operator, one trading partner). The system is local-only on purpose: the backend binds to `127.0.0.1` and requires login (bcrypt + HttpOnly session cookie). The threat model is still a stolen or imaged drive, not a remote attacker — SQLCipher at rest and the macOS Keychain handle that. The auth boundary is the HTTP layer; the file-system posture is unchanged. The SP23 product fork changes the threat model to "remote operator on LAN."
|
||||
|
||||
Cyclone processes three X12 transaction pairs end-to-end:
|
||||
|
||||
| Inbound | Outbound | Use |
|
||||
|---|---|---|
|
||||
| **837P** (005010X222A1) | 837P (resubmit) | Professional claim submission, acknowledgment tracking |
|
||||
| **835** (005010X221A1) | — | ERA remittance ingestion, auto-reconciliation |
|
||||
| **999 / TA1 / 270 / 271 / 277CA** | 999 (after submit) | Acknowledgments, eligibility, claim status |
|
||||
|
||||
The authoritative store is a single SQLite file at `~/.local/share/cyclone/cyclone.db` (or, with `sqlcipher3` installed and a Keychain entry, a SQLCipher-encrypted file at the same path). Every claim, remittance, audit event, and reconciliation decision is in that one file. The 6-year HIPAA retention expectation is met by automated daily encrypted backups (SP17) and a tamper-evident hash-chained audit log (SP11).
|
||||
|
||||
The whole stack is a single Python process (FastAPI + uvicorn on port 8000) plus a single Node process in dev (Vite on port 5173). No message broker, no separate worker, no database server. The v1 deployment model tolerates this because the operator is one person, the host is one machine, and the failure mode is "open the Activity log and check what broke."
|
||||
|
||||
---
|
||||
|
||||
## 2. Process & network topology
|
||||
|
||||
```
|
||||
+---------------------+ +-----------------------+
|
||||
| Browser | HTTP | Vite dev server |
|
||||
| (localhost:5173) | <----> | (Node 20+) |
|
||||
| | Vite | |
|
||||
| React 18 + TanStack | proxy | src/ |
|
||||
| Query, Zustand, | /api | - pages, components, |
|
||||
| Radix UI, Tailwind | -----> | hooks, lib |
|
||||
+---------------------+ +-----------------------+
|
||||
|
|
||||
| HTTP (localhost)
|
||||
v
|
||||
+----------------------------------------------------------+
|
||||
| Python process: `python -m cyclone serve` |
|
||||
| |
|
||||
| FastAPI app (uvicorn, 127.0.0.1:8000) |
|
||||
| +-----------------------------------------------------+ |
|
||||
| | cyclone.api:app (3,548 LOC) | |
|
||||
| | - routers: claims, remits, providers, acks, | |
|
||||
| | activity, inbox, upload, download, batches, | |
|
||||
| | admin, scheduler, health, ta1_acks | |
|
||||
| | - live-tail: NDJSON pubsub (pubsub.py) | |
|
||||
| | - lifespan: init db -> seed SP9 -> start SP16 + | |
|
||||
| | SP17 schedulers -> load PayerConfig cache | |
|
||||
| +-----------------------------------------------------+ |
|
||||
| | cyclone.store (facade, 2,423 LOC) | |
|
||||
| | - add/get/list/manual_match/iter_claims/... | |
|
||||
| | - delegates to: db.SessionLocal() | |
|
||||
| +-----------------------------------------------------+ |
|
||||
| | cyclone.parsers.* (X12 837P/835/999/TA1/270/271/ | |
|
||||
| | 277CA: tokenize -> segmentize -> model -> validate | |
|
||||
| | -> write to store) | |
|
||||
| +-----------------------------------------------------+ |
|
||||
| | cyclone.reconcile, scoring, batch_diff, audit_log, | |
|
||||
| | backup_service, scheduler, db_crypto, secrets | |
|
||||
| +-----------------------------------------------------+ |
|
||||
+----------------------------------------------------------+
|
||||
| | |
|
||||
v v v
|
||||
+----------------+ +-------------------+ +----------------+
|
||||
| SQLite / | | macOS Keychain | | Trading |
|
||||
| SQLCipher | | (keyring) | | partner SFTP |
|
||||
| cyclone.db | | - SQLCipher key | | (paramiko, SP13)|
|
||||
| (12 migrations)| | - SFTP password | | |
|
||||
| | | - backup pass | | |
|
||||
+----------------+ +-------------------+ +----------------+
|
||||
```
|
||||
|
||||
**One process, one file, three side-channels.** The Keychain (secrets) and SFTP (outbound to the payer) are side-channels; the database is the central state.
|
||||
|
||||
The `cyclone-pipeline/` sibling project (not in this repo) is a separate Python CLI that drives the full 7-phase round-trip: preflight → browser upload → parse verification → SFTP submit → TA1 wait → 999 wait → scan + report. It is not part of the in-process architecture; it sits alongside the backend and uses its API endpoints.
|
||||
|
||||
---
|
||||
|
||||
## 3. Tech stack
|
||||
|
||||
### 3.1 Backend
|
||||
|
||||
| Layer | Technology | Version | Notes |
|
||||
|---|---|---|---|
|
||||
| Language | Python | 3.11+ (3.13 in dev) | `requires-python = ">=3.11"` |
|
||||
| Web framework | FastAPI | ≥0.110, <1 | `python -m cyclone serve` boots uvicorn |
|
||||
| ASGI server | uvicorn[standard] | ≥0.27, <1 | `--host 127.0.0.1 --port 8000` |
|
||||
| ORM | SQLAlchemy | ≥2.0, <3 | 2.x style; engine + SessionLocal() function-accessor |
|
||||
| Validation | Pydantic | ≥2.6, <3 | v2 model_validator + ConfigDict |
|
||||
| CLI | Click | ≥8.1, <9 | `cyclone` console script |
|
||||
| YAML | PyYAML | ≥6.0, <7 | PayerConfig (SP9) |
|
||||
| Secrets | keyring | ≥25.0, <26 | macOS Keychain on darwin |
|
||||
| Crypto | cryptography | ≥49.0, <50 | AES-256-GCM for SP17 backups (hard dep) |
|
||||
| SQLCipher (opt) | sqlcipher3 | ≥0.6, <1 | `pip install -e .[sqlcipher]` |
|
||||
| SFTP (opt) | paramiko | ≥3.4, <6 | `pip install -e .[sftp]` |
|
||||
| Tests | pytest + pytest-cov + pytest-asyncio + pytest-randomly | latest | 964 tests, 2026-06-23 |
|
||||
| Test client | httpx | ≥0.27, <1 | dev only |
|
||||
|
||||
### 3.2 Frontend
|
||||
|
||||
| Layer | Technology | Version | Notes |
|
||||
|---|---|---|---|
|
||||
| Build | Vite | latest | `npm run dev` (port 5173) |
|
||||
| Language | TypeScript | latest | `tsc -b --noEmit` is the typecheck script |
|
||||
| UI | React | 18.3.1 | function components + hooks |
|
||||
| Routing | react-router-dom | 6.27.0 | SPA, 11 top-level routes |
|
||||
| Data | @tanstack/react-query | 5.101+ | server-state cache |
|
||||
| Client state | Zustand | 4.5.5 | small, ephemeral state (live tail, drill stack) |
|
||||
| UI primitives | @radix-ui/* | latest | dialog, label, select, slot, tabs |
|
||||
| Styling | Tailwind CSS | latest | utility-first |
|
||||
| Icons | lucide-react | 0.453+ | |
|
||||
| Toasts | sonner | 1.5+ | |
|
||||
| Test runtime | vitest | 4.1.9 | Node 18+ has Response/fetch natively |
|
||||
| Test DOM | happy-dom | 20.10+ | not jsdom (NFR-2) |
|
||||
| Component tests | @testing-library/react | 16.3+ | 73 test files |
|
||||
|
||||
### 3.3 External / operational
|
||||
|
||||
- **macOS Keychain** — secret store for SQLCipher key, SFTP password, backup passphrase. No secrets on disk in plaintext (NFR-4).
|
||||
- **X12 005010 implementation guides** — the parsers follow the CMS / X12 published guides. The 837P parser is the most heavily tested; 835 is the second; 999/TA1/270/271/277CA are lighter.
|
||||
- **One trading partner** — Colorado Medicaid via the Gainwell SFTP endpoint. Other payers are configured via `cyclone/providers.py` `PayerConfig` rows (SP9).
|
||||
|
||||
---
|
||||
|
||||
## 4. Backend architecture
|
||||
|
||||
### 4.1 Package layout
|
||||
|
||||
The `cyclone` package is a single namespace rooted at `backend/src/cyclone/`. 22 top-level modules + 4 subpackages. The store (SP21 in-flight split) is the largest; everything else is small and focused.
|
||||
|
||||
```
|
||||
backend/src/cyclone/
|
||||
├── __main__.py — entry point; `python -m cyclone` → CLI, `python -m cyclone serve` → uvicorn
|
||||
├── __init__.py — `__version__`, package init
|
||||
├── api.py — FastAPI app, route includes, lifespan (3,548 LOC — the only large file)
|
||||
├── api_helpers.py — request/response formatters, error → HTTP mapping
|
||||
├── api_routers/
|
||||
│ ├── __init__.py
|
||||
│ ├── acks.py — 999/TA1/271/277CA acknowledgments
|
||||
│ ├── admin.py — admin-only endpoints (validate-provider, scheduler, backup)
|
||||
│ ├── health.py — GET /api/health (SP19 deep snapshot)
|
||||
│ └── ta1_acks.py — TA1-specific (split from acks for the SP3 lifecycle)
|
||||
├── audit_log.py — SHA-256 hash-chained append-only log (SP11)
|
||||
├── backup.py — SP17 primitives: derive_key, encrypt, decrypt, BackupFile
|
||||
├── backup_scheduler.py — SP17 BackupScheduler wrapper
|
||||
├── backup_service.py — SP17 coordinator: create/list/restore/verify/prune
|
||||
├── batch_diff.py — 837 ↔ 835 diff (SP4)
|
||||
├── clearhouse/
|
||||
│ └── __init__.py — Clearhouse, SftpClient (SP9, SP13)
|
||||
├── cli.py — Click CLI: `cyclone serve`, `cyclone validate-npi`, `cyclone backup`, etc.
|
||||
├── db.py — SQLAlchemy engine, SessionLocal, ORM models, init_db, reinit_engine
|
||||
├── db_crypto.py — SP12 SQLCipher connect-creator + is_encryption_enabled()
|
||||
├── db_migrate.py — runs the 12 SQL migrations in order
|
||||
├── edi/
|
||||
│ └── filenames.py — X12 filename convention helpers (SP9)
|
||||
├── inbox_lanes.py — 5-lane Inbox computation (SP14)
|
||||
├── inbox_state.py — claim ↔ inbox state mapping helpers
|
||||
├── inbox_state_277ca.py — SP10 277CA inbox state integration
|
||||
├── logging_config.py — SP18 structured JSON + PII scrubber
|
||||
├── migrations/ — 12 SQL files (0001_initial through 0012_backups)
|
||||
├── npi.py — SP20 NPI Luhn + Tax ID validation
|
||||
├── parsers/ — X12 parser subpackage
|
||||
│ ├── parse_837.py — 837P ingest
|
||||
│ ├── parse_835.py — 835 ingest
|
||||
│ ├── parse_999.py / parse_ta1.py
|
||||
│ ├── parse_270.py / parse_271.py
|
||||
│ ├── parse_277ca.py — SP10 277CA ingest
|
||||
│ ├── models*.py — one Pydantic model module per transaction type
|
||||
│ ├── segments.py — X12 segment tokenizer
|
||||
│ ├── validator.py — 837P rule registry (R-codes)
|
||||
│ ├── validator_835.py — 835 validator
|
||||
│ ├── serialize_837.py — SP8 outbound 837P serializer
|
||||
│ ├── serialize_270.py / serialize_999.py
|
||||
│ ├── cas_codes.py — CAS reason-code table
|
||||
│ ├── batch_ack_builder.py
|
||||
│ ├── writer.py / writer_835.py
|
||||
│ └── exceptions.py
|
||||
├── payers.py — Payer ORM row accessor (SP9)
|
||||
├── providers.py — Payer / Clearhouse / Provider ORM row DTOs (SP9)
|
||||
├── pubsub.py — NDJSON live-tail EventBus
|
||||
├── reconcile.py — 835 → 837 auto-reconciliation engine
|
||||
├── scheduler.py — SP16 MFT polling scheduler
|
||||
├── scoring.py — 4-field lane scoring (40/25/20/15, hard-coded — backlog item)
|
||||
├── secrets.py — Keychain access wrapper (SP9)
|
||||
├── security.py — security headers, HealthSnapshot, deep health probe (SP19)
|
||||
└── store.py — CycloneStore facade (2,423 LOC; SP21 in-flight split)
|
||||
```
|
||||
|
||||
### 4.2 Module responsibility map (one line each)
|
||||
|
||||
| Module | Responsibility |
|
||||
|---|---|
|
||||
| `__main__` | Dispatch `serve` vs CLI; honor `CYCLONE_PORT` / `CYCLONE_RELOAD`; emit `AUTH_DISABLED` WARNING if `cyclone.auth.deps.AUTH_DISABLED` is True at boot |
|
||||
| `api` | FastAPI app, lifespan, route includes, 999/270/271 endpoints, exception → HTTP |
|
||||
| `auth.*` | HTTP-layer login (bcrypt + HttpOnly session cookie). Submodules: `deps` (the `matrix_gate` FastAPI dependency, `AUTH_DISABLED` flag), `routes` (`/api/auth/login`, `/api/auth/logout`, `/api/auth/me`), `users`, `sessions`, `permissions` (role matrix), `bootstrap` (env-var first-admin), `cli` (`cyclone admin create-user` / `rotate-password`) |
|
||||
| `api_helpers` | Error → status code mapping, common response shapes |
|
||||
| `audit_log` | Append + verify a SHA-256 hash-chained log (SP11) |
|
||||
| `backup` | Pure crypto + file I/O for SP17 backup format |
|
||||
| `backup_scheduler` | Wrap `BackupService` in a `tick()` loop (SP17) |
|
||||
| `backup_service` | Create / list / restore / verify / prune (SP17) |
|
||||
| `batch_diff` | Pairwise diff of two 837 batches (SP4) |
|
||||
| `clearhouse` | `Clearhouse` + `SftpClient` (SP9 stub, SP13 paramiko) |
|
||||
| `cli` | `cyclone` console script: serve, validate-npi, backup, list-batches, … |
|
||||
| `db` | Engine factory, `SessionLocal()`, all ORM rows, `init_db()`, `reinit_engine()` |
|
||||
| `db_crypto` | SQLCipher branch + `is_encryption_enabled()` (SP12) |
|
||||
| `db_migrate` | Walk `migrations/*.sql` in order, apply pending |
|
||||
| `edi.filenames` | Parse / construct X12 file names (SP9) |
|
||||
| `inbox_lanes` | Compute the 5 lanes (rejected, payer_rejected, candidates, unmatched, done_today) |
|
||||
| `inbox_state` | Pure helpers for state ↔ lane mapping |
|
||||
| `inbox_state_277ca` | SP10 277CA state integration (monotonic rule) |
|
||||
| `logging_config` | SP18 JSON formatter + PII scrubber |
|
||||
| `npi` | SP20 pure validators (Luhn over 80840+body, EIN prefix check) |
|
||||
| `parsers.*` | X12 transaction set parsers (one per type) |
|
||||
| `payers` / `providers` | SP9 ORM row DTOs |
|
||||
| `pubsub` | In-process NDJSON EventBus for live tail |
|
||||
| `reconcile` | 835 → 837 auto-match + per-line CAS reconciliation |
|
||||
| `scheduler` | SP16 MFT polling scheduler (per-payer tick) |
|
||||
| `scoring` | 4-field lane scoring weights (40/25/20/15) |
|
||||
| `secrets` | Keychain get/set/delete via `keyring` |
|
||||
| `security` | Security headers, deep `HealthSnapshot` (SP19) |
|
||||
| `store` | `CycloneStore` facade: `add` / `get` / `list` / `manual_match` / `iter_*` / `recent_activity` |
|
||||
|
||||
### 4.3 The `CycloneStore` facade
|
||||
|
||||
The store is the single read/write surface for the database. Every endpoint that mutates state goes through `cyclone.store` (eventually the `cyclone.store.*` subpackage after SP21 lands). The facade pattern preserves public API:
|
||||
|
||||
- `store.add(record)` — add a parsed batch
|
||||
- `store.get(batch_id)`, `get_batch(batch_id)`, `list(limit)`, `all()`
|
||||
- `store.manual_match(claim_id, remit_id)` — raises `AlreadyMatchedError` (→ 409)
|
||||
- `store.manual_unmatch(claim_id)` — raises `NotMatchedError` (→ 409)
|
||||
- `store.iter_claims(...)`, `iter_remittances(...)`
|
||||
- `store.distinct_providers()`, `recent_activity(limit)`
|
||||
- `store.list_unmatched(kind="both")`
|
||||
- `store.export_*()` — CSV/JSON dumpers
|
||||
|
||||
All persistence flows through SQLAlchemy sessions via `db.SessionLocal()()`. The engine is single-connection-per-request via FastAPI dependency injection. SP21 splits `store.py` (2,423 LOC) into a `cyclone/store/` subpackage with one module per domain (batches, inbox, claim_detail, acks, backups, providers, etc.) and a thin facade that delegates every method to its domain module. After SP21, the public API of `cyclone.store` is unchanged; the only thing that changes is the file layout.
|
||||
|
||||
### 4.4 The parser pipeline
|
||||
|
||||
All inbound files follow the same 5-stage pipeline:
|
||||
|
||||
```
|
||||
X12 .txt on disk
|
||||
↓
|
||||
[1] tokenize (parsers/segments.py: split on ~, then ^, then *:)
|
||||
↓
|
||||
[2] segmentize (build Segment records with elements + repeats)
|
||||
↓
|
||||
[3] model (parsers/models_*.py: Pydantic models per transaction type)
|
||||
↓
|
||||
[4] validate (parsers/validator.py: R-codes; raise ParseError on fatal)
|
||||
↓
|
||||
[5] write (store.add(record); record → DB rows)
|
||||
```
|
||||
|
||||
The 837P validator has the most rules (R020, R021, R034, R035, R200, R210 — see [837P parser spec](superpowers/specs/2026-06-19-cyclone-837p-parser-design.md) §2.4 for the registry). The 835 validator is simpler (no R-codes, just a model completeness check + a status_code allowlist). 999/TA1/270/271/277CA have no validators — they trust the upstream.
|
||||
|
||||
The writer is responsible for: (a) creating the `batches` row, (b) creating N `claims` / `remittances` / `cas_adjustments` rows, (c) emitting an `activity_events` row, (d) for 837P, the synthetic claim-status from the 999 envelope, (e) for 835, triggering `reconcile.apply_payment` to auto-match. If any step raises, the entire batch is rolled back (single SQLAlchemy transaction).
|
||||
|
||||
### 4.5 The reconciliation engine
|
||||
|
||||
`cyclone.reconcile` is a pure-functions module. The store calls into it for every 835 ingest:
|
||||
|
||||
- `apply_payment(claim, remit, strategy) -> ApplyIntent` — happy path; returns the new `Match` row + state transition
|
||||
- `apply_reversal(...)` — handles a reversal 835 (the original payment + the reversal both stay in the DB; the `is_reversal` flag on the `Match` row disambiguates)
|
||||
- `recompute_line_totals(claim)` — for SP7's per-line audit, when an 835 carries CAS adjustments
|
||||
|
||||
The engine is pure (no DB access). The store orchestrates the SQL writes after a successful `apply_*` returns. The 4-field scoring (40/25/20/15 for charge amount / NPI / patient ID / service date) lives in `cyclone.scoring` and is invoked by `inbox_lanes.compute_lanes` for the `candidates` and `unmatched` lanes.
|
||||
|
||||
### 4.6 The audit chain
|
||||
|
||||
`cyclone.audit_log` is a separate, append-only log table (`audit_log` — distinct from `activity_events` which is the un-chained `/activity` feed). Every event type the system can emit has a key in the chain:
|
||||
|
||||
- `claim.submitted`, `claim.rejected`, `claim.matched`, `claim.unmatched`
|
||||
- `claim.payer_rejected`, `claim.payer_rejected_acknowledged`
|
||||
- `clearhouse.submitted`
|
||||
- `key.rotated` (SP15), `backup.created`, `backup.restored`, `backup.verified` (SP17)
|
||||
- `api.request_rejected` (SP19)
|
||||
|
||||
Each row carries the SHA-256 of `(prev_hash || kind || actor || payload_json || ts)`. The first row has `prev_hash = 0x00 * 32`. `audit_log.verify_chain()` walks the table top-to-bottom, recomputing each hash, and raises `AuditChainError` on the first mismatch with the row id and the expected vs. actual hash. SP19's deep health probe calls `verify_chain()` on every `/api/health` request so a broken chain surfaces in the operator's status badge within one tick.
|
||||
|
||||
---
|
||||
|
||||
## 5. Frontend architecture
|
||||
|
||||
### 5.1 Tech stack
|
||||
|
||||
See §3.2. The frontend is a Vite SPA with TypeScript strict mode, Tailwind utility classes, and Radix primitives. There is no SSR, no Next.js, no router data-loading — every page is a function component that calls `useQuery` and renders.
|
||||
|
||||
### 5.2 Route map
|
||||
|
||||
| Path | Page | Purpose |
|
||||
|---|---|---|
|
||||
| `/` | `Dashboard` | KPI tiles + ticker tape + recent activity |
|
||||
| `/claims` | `Claims` | Paginated claim list, filters, drill to claim drawer |
|
||||
| `/remittances` | `Remittances` | Paginated remit list, drill to remit drawer |
|
||||
| `/providers` | `Providers` | Provider rollup (NPIs, charge totals) |
|
||||
| `/activity` | `ActivityLog` | Append-only `activity_events` feed (NDJSON) |
|
||||
| `/upload` | `Upload` | Drag-drop or click to upload an 837P / 835 / 999 / 277CA |
|
||||
| `/reconciliation` | `ReconciliationPage` | Cross-claim reconciliation view (4-field score) |
|
||||
| `/inbox` | `Inbox` | 5-lane inbox (SP14) with BulkBar |
|
||||
| `/acks` | `Acks` | 999/TA1/270/271/277CA acknowledgment list |
|
||||
| `/batches` | `Batches` | Batch list with totals + validation summary |
|
||||
| `/batch-diff` | `BatchDiff` | Pairwise 837 batch diff |
|
||||
| `*` | `NotFound` | 404 |
|
||||
|
||||
### 5.3 State management
|
||||
|
||||
Two layers:
|
||||
|
||||
- **Server state** — `@tanstack/react-query`. Every page uses one or more `useQuery` calls. The query key shape is `["domain", id?, filters?]`. Mutations use `useMutation` and invalidate the relevant keys. There is no global refetch strategy; each page controls its own stale time.
|
||||
|
||||
- **Client state** — `zustand`. Two stores:
|
||||
- `tail-store.ts` — the live-tail connection state (reconnecting, last event id, subscriber counts). Powers the bottom-right TailStatusPill.
|
||||
- `DrillStackProvider` (in `src/components/drill/`) — the SP22 universal-drilldown stack. A LIFO of drill contexts (claim → claim line → match → CAS adjustment) so the back button works across drawers.
|
||||
|
||||
There is no Redux, no Context for global state, no MobX. The combination of react-query for server state + zustand for client state is intentional — see [universal-drilldown spec](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) for the rationale.
|
||||
|
||||
### 5.4 Live tail (NDJSON pubsub)
|
||||
|
||||
The Activity feed and the TailStatusPill share a single connection: `GET /api/tail?since=<last_event_id>`. The server returns `Content-Type: application/x-ndjson` and streams one JSON object per line. Each line is one of:
|
||||
|
||||
- `{kind: "heartbeat", ts: "..."}` — every 5s when no events
|
||||
- `{kind: "event", id: <int>, event_type: "...", payload: {...}}` — when something happens
|
||||
- `{kind: "error", message: "..."}` — server-initiated close (rare)
|
||||
|
||||
The client uses `fetch` + `ReadableStream` to consume the body and dispatches each line to the relevant `useQuery` cache via `queryClient.setQueryData`. The connection auto-reconnects on `error` with exponential backoff. See [live-tail spec](superpowers/specs/2026-06-20-cyclone-live-tail-design.md) for the wire format and reconnection policy.
|
||||
|
||||
---
|
||||
|
||||
## 6. Data model
|
||||
|
||||
### 6.1 Migrations 0001–0014
|
||||
|
||||
The schema evolves in ordered SQL files under `backend/src/cyclone/migrations/`. Each file starts with `-- version: N` and is applied in order by `db_migrate.py`. There is no migration framework — `db_migrate.py` is a 30-line file that checks the `schema_version` row and applies everything newer. The first 12 ship on `main`; 13 + 14 are the auth migrations that landed in `main` on 2026-06-23.
|
||||
|
||||
| # | File | What it adds |
|
||||
|---|---|---|
|
||||
| 1 | `0001_initial.sql` | `batches`, `claims`, `remittances`, `cas_adjustments`, `matches`, `activity_events` (6 tables) |
|
||||
| 2 | `0002_acks.sql` | `acks` table (999/271) |
|
||||
| 3 | `0003_drop_claims_remits_unique_constraints.sql` | Spec-bug fix: drop the UNIQUE constraints that prevented reversal flows |
|
||||
| 4 | `0004_rejections_and_state_history.sql` | `rejections` table + `claim_state_history` |
|
||||
| 5 | `0005_create_ta1_acks.sql` | `ta1_acks` table |
|
||||
| 6 | `0006_line_reconciliation.sql` | `line_reconciliations` (SP7) |
|
||||
| 7 | `0007_providers_payers_clearhouse.sql` | `providers`, `payers`, `clearhouses` (SP9) |
|
||||
| 8 | `0008_payer_rejected_columns.sql` | `claim.payer_rejected_*` columns (SP10) |
|
||||
| 9 | `0009_audit_log.sql` | `audit_log` hash-chained table (SP11) |
|
||||
| 10 | `0010_payer_rejected_acknowledged.sql` | `claim.payer_rejected_acknowledged_*` columns (SP14 hand-off) |
|
||||
| 11 | `0011_processed_inbound_files.sql` | `processed_inbound_files` idempotency table (SP16) |
|
||||
| 12 | `0012_backups.sql` | `db_backups` table (SP17) |
|
||||
| 13 | `0013_auth_users_and_sessions.sql` | `users` + `sessions` tables, `users.password_hash` (bcrypt) — landed 2026-06-23 |
|
||||
| 14 | `0014_audit_log_user_id.sql` | `audit_log.user_id` FK to `users.id` (forward reference to 0013) — landed 2026-06-23 |
|
||||
|
||||
The SP22 parse-then-decide migrations (drop claims UNIQUE constraint + relax PK to `(batch_id, id)`) are renumbered to 0015 + 0016 on the `claims-unique-fix` worktree so the auth migrations stay at the front of the chain; they'll keep those numbers when SP22 lands.
|
||||
|
||||
### 6.2 ERD (high level)
|
||||
|
||||
```
|
||||
batches (1) ─┬─< (N) claims ─┬─< (N) matches >─ (N) ─┬─ (1) remittances ─< (N) cas_adjustments
|
||||
│ │ │
|
||||
│ └─< (N) line_reconciliations (SP7)
|
||||
│ └─< (N) rejections / claim_state_history
|
||||
│
|
||||
├─< (N) acks (999/271)
|
||||
├─< (N) ta1_acks
|
||||
└─< (N) activity_events
|
||||
|
||||
claims (1) ─< (N) line_reconciliations ─> (1) cas_adjustments
|
||||
|
||||
audit_log (no FKs — append-only, hash-chained)
|
||||
db_backups (no FKs — directory + filename, status row)
|
||||
processed_inbound_files (no FKs — path + hash, idempotency key)
|
||||
|
||||
providers (1) ─< (N) claims
|
||||
payers (1) ─< (N) claims
|
||||
clearhouses (1) ─< (N) payers
|
||||
```
|
||||
|
||||
The `batches` row is the parent of every parsed X12 file. A batch is the unit of atomicity for parsing: either every claim/remit/ack in the file lands, or the file rolls back. The `claims.state` column drives the lifecycle (§6.3); the `matches` table is the join that says "this 835 paid this 837" (with a `strategy` field for audit: auto vs. manual).
|
||||
|
||||
### 6.3 Claim lifecycle (7 states)
|
||||
|
||||
The `claims.state` column is the canonical state of a claim in the operator's workflow:
|
||||
|
||||
```
|
||||
submitted ──> accepted ──> paid
|
||||
│ │ │
|
||||
│ │ └─> reversed
|
||||
│ │
|
||||
│ └─> denied ──> appealed ──> paid
|
||||
│
|
||||
└─> rejected (envelope 999)
|
||||
│
|
||||
└─> payer_rejected (277CA A4/A6/A7)
|
||||
```
|
||||
|
||||
- `submitted` — 837P ingested, awaiting 999 ACK
|
||||
- `accepted` — 999 ACK came back clean; sitting in the trading partner's queue
|
||||
- `paid` — 835 matched, payment posted
|
||||
- `reversed` — 835 reversal applied (the original payment + the reversal are both preserved; the `state` reflects the most recent)
|
||||
- `denied` — 835 came back with a denial status code (CAS group CO)
|
||||
- `appealed` — operator flagged for appeal workflow
|
||||
- `rejected` — envelope-level 999 rejection (syntactic / structural)
|
||||
- `payer_rejected` — semantic 277CA rejection (A4/A6/A7 — claim accepted by envelope but rejected at the line level). SP10 introduced this state and the `claim.payer_rejected_*` columns.
|
||||
|
||||
State transitions are explicit functions in `cyclone.store` (e.g. `mark_accepted`, `mark_paid`, `mark_rejected`). Each transition emits an `activity_events` row and, for the security-relevant ones, an `audit_log` row.
|
||||
|
||||
### 6.4 Audit chain integrity
|
||||
|
||||
The `audit_log` table is a separate chain from `activity_events`:
|
||||
|
||||
- `activity_events` — un-chained, powers the `/activity` feed. The activity feed can be repopulated from the underlying state if rows are missing; integrity is informational.
|
||||
- `audit_log` — SHA-256 hash-chained, powers compliance / forensics. A row cannot be deleted without breaking the chain. `audit_log.verify_chain()` is the integrity check; the SP19 deep health probe runs it on every `/api/health` call.
|
||||
|
||||
The two are intentionally separate because they have different retention and integrity requirements. The convention-vs-trigger tradeoff is documented in the [SP11 spec](superpowers/specs/2026-06-23-cyclone-sp11-hash-chained-audit-design.md) §7.
|
||||
|
||||
---
|
||||
|
||||
## 7. Data flow
|
||||
|
||||
### 7.1 File ingest (manual upload)
|
||||
|
||||
```
|
||||
Operator ──drag-drop──> /upload (React)
|
||||
│
|
||||
│ POST /api/upload (multipart, .txt file)
|
||||
v
|
||||
FastAPI route
|
||||
│
|
||||
│ save to /tmp/cyclone-uploads/<uuid>.txt
|
||||
v
|
||||
parser.dispatch(kind) (auto-detect 837/835/999/TA1/270/271/277CA by ISA segment)
|
||||
│
|
||||
│ (the 5-stage pipeline from §4.4)
|
||||
v
|
||||
CycloneStore.add(record) (single SQLAlchemy transaction)
|
||||
│
|
||||
├─> emit activity_events row
|
||||
├─> for 835: trigger reconcile.apply_payment
|
||||
├─> for 999/TA1/277CA: mark the source claim
|
||||
└─> commit
|
||||
│
|
||||
│ emit NDJSON event on the pubsub bus
|
||||
v
|
||||
live-tail subscribers (Activity feed, TailStatusPill)
|
||||
```
|
||||
|
||||
The HTTP response is the new `batch_id` + a counts summary (e.g. `{"batch_id": "...", "claims": 12, "remittances": 0, "validation_issues": 0}`). The 999 is the only case where the response carries an ack payload — the operator can read the parsed 999 directly without a second request.
|
||||
|
||||
### 7.2 File ingest (SFTP polling — SP16)
|
||||
|
||||
```
|
||||
every N seconds (default 30s, per-payer config):
|
||||
Scheduler.tick()
|
||||
│
|
||||
├─> for each clearhouse:
|
||||
│ ├─> SftpClient.list_inbound() (paramiko, SP13)
|
||||
│ ├─> for each new .x12 file:
|
||||
│ │ ├─> check processed_inbound_files (idempotency)
|
||||
│ │ ├─> SftpClient.read_file(path) (to /tmp/...)
|
||||
│ │ ├─> parser.dispatch(...)
|
||||
│ │ ├─> CycloneStore.add(record)
|
||||
│ │ ├─> INSERT processed_inbound_files (path, sha256, ts)
|
||||
│ │ └─> emit activity_events row
|
||||
│ └─> log tick result (count, errors, ms)
|
||||
└─> heartbeat to /api/health (SP19 reports last tick ts)
|
||||
```
|
||||
|
||||
The idempotency table is the linchpin: a crash between "read file" and "INSERT processed" results in the same file being processed again on the next tick. The 5-step ordering (SELECT → INSERT side-effects → INSERT idempotency; IntegrityError as the correctness backstop) is detailed in the [SP16 spec](superpowers/specs/2026-06-23-cyclone-sp16-mft-polling-design.md) §3.4.
|
||||
|
||||
### 7.3 File ingest (manual upload, replayed by pipeline)
|
||||
|
||||
`cyclone-pipeline/` (sibling project) does the same as the operator dragging-and-dropping — it `POST`s to `/api/upload` with the file. There's no separate "pipeline" ingestion path. The pipeline agent is a thin wrapper that adds preflight + SFTP submit + wait-for-999 around the Cyclone API.
|
||||
|
||||
### 7.4 Auto-match on 835 ingest
|
||||
|
||||
When an 835 lands, the writer calls `reconcile.apply_payment(claim, remit, strategy)` for each `(claim, remit)` pair in the file. The strategy is one of:
|
||||
|
||||
- `auto:pcn_npi_charge` — patient_control_number + provider_npi + charge_amount match (the strongest)
|
||||
- `auto:pcn_charge` — patient_control_number + charge_amount (the most common in practice)
|
||||
- `manual` — the operator uses `/reconciliation` to pair by hand; the strategy is recorded for audit
|
||||
|
||||
Each auto-match creates a `matches` row + transitions the claim state + emits an `activity_events` row + emits an `audit_log` row. The match rate is visible in the Dashboard KPI tile.
|
||||
|
||||
### 7.5 Outbound 837P (resubmit)
|
||||
|
||||
Two paths:
|
||||
|
||||
- **Single claim** — `GET /api/claims/{id}/serialize-837` (SP8). The serializer round-trips the claim through `serialize_837.py`, validates with the R-code registry, and returns the X12 text. The Claim drawer's "Download 837" button calls this and triggers a browser download.
|
||||
- **Bundle ZIP** — `POST /api/claims/resubmit?download=zip` (SP8). The serializer rebuilds N 837P files (one per claim in the request), zips them with a manifest, and returns the ZIP. The Inbox BulkBar's "Resubmit selected" calls this.
|
||||
|
||||
Both paths are read-only against the DB (they don't mutate the claim state). The operator is expected to download the regenerated file, then submit it manually via SFTP (or via `cyclone-pipeline run`).
|
||||
|
||||
### 7.6 Acknowledgement ingestion (999, TA1, 277CA)
|
||||
|
||||
These are the three "ack" transaction types the operator's trading partner sends back after a submission:
|
||||
|
||||
- **999** — functional ack (envelope-level). Triggers `claim.state = accepted | rejected`.
|
||||
- **TA1** — interchange ack. Triggers an activity_events row; no state change.
|
||||
- **277CA** — claim status (line-level). Triggers `claim.payer_rejected` if any line is A4/A6/A7. The `claim.payer_rejected_*` columns carry the status code + the 277CA's batch id for the audit trail.
|
||||
|
||||
All three are ingested via the same `/api/upload` path; the parser auto-detects the transaction type from the ISA segment. The SP10 spec describes the monotonic rule: a later 277CA cannot clear a previous rejection. This is the regulatory guarantee the SP11 audit chain relies on.
|
||||
|
||||
---
|
||||
|
||||
## 8. API surface
|
||||
|
||||
### 8.1 REST endpoints (by domain)
|
||||
|
||||
The full list is in [REQUIREMENTS.md §6.3 traceability matrix](REQUIREMENTS.md). The structure is:
|
||||
|
||||
```
|
||||
/api/health (SP19 deep health probe)
|
||||
/api/tail?since=<id> (NDJSON live tail)
|
||||
|
||||
/api/upload (multipart, any X12 type)
|
||||
/api/parse-837 (parse + return Pydantic, do not persist — pipeline agent)
|
||||
|
||||
/api/batches (list, with totals + validation)
|
||||
/api/batches/{id} (detail)
|
||||
/api/batches/{id}/diff?against={other_id} (batch_diff)
|
||||
/api/batches/{id}/serialize-837 (SP8 — round-trip)
|
||||
|
||||
/api/claims (list, paginated, filterable)
|
||||
/api/claims/{id} (detail, with remits + matches + line_recons)
|
||||
/api/claims/{id}/serialize-837 (SP8)
|
||||
/api/claims/resubmit?download=zip (SP8 bulk)
|
||||
|
||||
/api/remittances (list)
|
||||
/api/remittances/{id} (detail)
|
||||
|
||||
/api/providers (rollup)
|
||||
/api/providers/{npi} (single provider, all claims)
|
||||
|
||||
/api/activity?since=<id> (un-chained feed, paginated)
|
||||
|
||||
/api/inbox (SP14 5-lane view, filterable)
|
||||
/api/inbox/payer-rejected/acknowledge (SP10 + SP14)
|
||||
|
||||
/api/acks (999/271/277CA list)
|
||||
/api/acks/{id} (detail)
|
||||
|
||||
/api/ta1-acks (TA1 list)
|
||||
|
||||
/api/admin/validate-provider?npi=&tax_id= (SP20)
|
||||
/api/admin/db/rotate-key (SP15)
|
||||
/api/admin/db/fingerprint (SP12)
|
||||
/api/admin/backup/create (SP17)
|
||||
/api/admin/backup/list (SP17)
|
||||
/api/admin/backup/{id}/restore?confirm=true (SP17, two-step)
|
||||
/api/admin/backup/{id}/verify (SP17)
|
||||
/api/admin/scheduler/status (SP16)
|
||||
/api/admin/scheduler/tick (SP16 — manual trigger)
|
||||
/api/admin/scheduler/start (SP16)
|
||||
/api/admin/scheduler/stop (SP16)
|
||||
```
|
||||
|
||||
### 8.2 Live-tail NDJSON format
|
||||
|
||||
`GET /api/tail?since=<last_event_id>` returns `Content-Type: application/x-ndjson` and streams:
|
||||
|
||||
```
|
||||
{"kind":"event","id":1234,"event_type":"claim.submitted","payload":{"claim_id":"...","batch_id":"...","ts":"2026-06-23T..."}}
|
||||
{"kind":"event","id":1235,"event_type":"claim.matched","payload":{"claim_id":"...","remit_id":"...","strategy":"auto:pcn_npi_charge"}}
|
||||
{"kind":"heartbeat","ts":"2026-06-23T..."}
|
||||
{"kind":"event","id":1236,"event_type":"claim.payer_rejected","payload":{"claim_id":"...","status_code":"A7","source_277ca_id":"..."}}
|
||||
```
|
||||
|
||||
Heartbeats every 5s when no events. The client reconnects on `error` with exponential backoff capped at 30s. See [live-tail spec](superpowers/specs/2026-06-20-cyclone-live-tail-design.md) for the full wire format.
|
||||
|
||||
---
|
||||
|
||||
## 9. Cross-cutting concerns
|
||||
|
||||
### 9.1 Logging (SP18)
|
||||
|
||||
`cyclone.logging_config` configures a single JSON formatter that:
|
||||
|
||||
- Emits one JSON object per line to stdout (12-factor)
|
||||
- Includes `ts`, `level`, `logger`, `msg`, `module`, `line`, `process`, `thread`
|
||||
- Scrubs PHI/PII fields by key name (`npi`, `member_id`, `patient_name`, `ssn_last4`, `charge_amount`, `paid_amount`, `tax_id`) — replaced with `***`
|
||||
- Redacts secrets by string match (anything matching `password=...&` or `Bearer ...`)
|
||||
|
||||
The PII scrubber is a best-effort defense-in-depth, not a security boundary. Real PHI protection comes from the SQLCipher layer (SP12) and the Keychain (no secrets in env). The scrubber exists so an operator tailing logs to debug doesn't accidentally paste a log line into a chat.
|
||||
|
||||
### 9.2 Audit chain (SP11)
|
||||
|
||||
See §4.6. The chain is checked on every `/api/health` request and exposes `status="degraded"` if broken. The operator sees the status in the TailStatusPill and the Dashboard KPI tile.
|
||||
|
||||
### 9.3 PHI/PII handling
|
||||
|
||||
- **Storage** — SQLCipher at rest (opt-in, SP12). Daily encrypted backups (SP17). Tamper-evident audit chain (SP11). 6-year retention via automated backup rotation.
|
||||
- **In transit (SFTP)** — TLS to the trading partner; no plaintext EDI over the wire.
|
||||
- **In the UI** — no PHI is shown on the Dashboard or Inbox (only counts + KPIs). The Claim drawer and Remit drawer show full PHI; these pages are not in the route map's preview and require an explicit navigation click.
|
||||
- **In logs** — scrubbed by key name (see §9.1).
|
||||
- **In analytics** — no analytics. No third-party tracking. No telemetry. The frontend makes no requests outside `127.0.0.1:8000`.
|
||||
|
||||
### 9.4 Error model
|
||||
|
||||
Every API error is a JSON object with `{"error": "<code>", "message": "<human>", "detail": <optional>}`. The mapping is in `api_helpers.py`:
|
||||
|
||||
| Error class | HTTP status | `error` code |
|
||||
|---|---|---|
|
||||
| `AlreadyMatchedError` | 409 | `already_matched` |
|
||||
| `NotMatchedError` | 409 | `not_matched` |
|
||||
| `InvalidStateError` | 409 | `invalid_state` |
|
||||
| `ParseError` | 422 | `parse_error` |
|
||||
| `ValidationError` (R-codes) | 422 | `validation_failed` (issues in `detail`) |
|
||||
| `FileNotFoundError` | 404 | `not_found` |
|
||||
| `ValueError` | 400 | `bad_request` |
|
||||
| `PermissionError` | 403 | `forbidden` (raised by `matrix_gate` when the role is below the endpoint's required role) |
|
||||
| (anything else) | 500 | `internal` (with a request id for log correlation) |
|
||||
|
||||
The 409-vs-422-vs-400 split is intentional: 409 for state-machine violations (the operator's input is well-formed but conflicts with current state), 422 for parse / validation, 400 for bad input shape.
|
||||
|
||||
---
|
||||
|
||||
## 10. Operational concerns
|
||||
|
||||
### 10.1 Startup order
|
||||
|
||||
`python -m cyclone serve` → `cyclone.api.lifespan` runs:
|
||||
|
||||
1. `db.init_db()` — apply pending migrations (0001-0012, plus 0013/0014 on the `claims-unique-fix` worktree)
|
||||
2. Seed `cyclone.providers` with the SP9 defaults if the table is empty (PayerConfig for Colorado Medicaid + Gainwell)
|
||||
3. Configure `BackupService` (SP17) — open backup dir, read passphrase from Keychain
|
||||
4. Configure `Scheduler` (SP16) — wire MFT polling for each clearhouse
|
||||
5. If `CYCLONE_SCHEDULER_AUTOSTART=true` (default), start both schedulers
|
||||
6. Mount the FastAPI app and start uvicorn
|
||||
|
||||
The ordering is load-bearing: the store facade cannot be used until `init_db` finishes, and the schedulers cannot query the DB until the seed step populates the clearhouse rows.
|
||||
|
||||
### 10.2 MFT scheduler (SP16)
|
||||
|
||||
`cyclone.scheduler.Scheduler` runs a per-clearhouse tick on a configurable interval (default 30s, per `CYCLONE_SCHEDULER_INTERVAL` env var). The tick is described in §7.2. The scheduler is opt-in via `CYCLONE_SCHEDULER_AUTOSTART`; the operator can start/stop it from the admin endpoints or via `POST /api/admin/scheduler/tick` for a manual tick.
|
||||
|
||||
The 4-step idempotency sequence (SELECT → INSERT side-effects → INSERT idempotency; IntegrityError as the correctness backstop) is the linchpin. The scheduler's `_already_processed` + `_record` pair wraps the INSERT to handle the case where two ticks race (an extremely rare event given the 30s interval, but possible during operator-driven manual ticks).
|
||||
|
||||
### 10.3 Backup scheduler (SP17)
|
||||
|
||||
`cyclone.backup_scheduler.BackupScheduler` wraps `BackupService` and ticks on its own interval (default 24h, per `CYCLONE_BACKUP_INTERVAL` env var). On each tick: create a backup, prune anything older than `retention_days` (default 30). Auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`.
|
||||
|
||||
The encryption layer is independent of SQLCipher: AES-256-GCM with a passphrase-derived key (PBKDF2-HMAC-SHA256, 200k iterations). The passphrase lives in the macOS Keychain. If the passphrase is missing, the backup layer falls back to deriving a key from the SQLCipher key (less ideal but never silently broken — the operator's `/api/health` reports `status="degraded"` with a `backup.passphrase_missing` flag).
|
||||
|
||||
### 10.4 Health probe (SP19)
|
||||
|
||||
`GET /api/health` is a deep snapshot:
|
||||
|
||||
- **db** — open a session, `SELECT 1`
|
||||
- **scheduler** — last tick ts, is alive
|
||||
- **pubsub** — current subscriber counts per event kind
|
||||
- **batch** — most recent batch id + ts
|
||||
- **audit_log** — `verify_chain()` result (deep check on every call; 1ms on a healthy chain, fails loudly on a broken one)
|
||||
- **backup** — last backup ts, last verify result, passphrase presence
|
||||
- **encryption** — `is_encryption_enabled()` boolean
|
||||
|
||||
Returns `status="ok"` only when every subsystem is healthy. `status="degraded"` if any subsystem is unhappy but the API itself is responsive. The status is shown in the TailStatusPill (bottom-right) and the Dashboard KPI tile.
|
||||
|
||||
### 10.5 Shutdown
|
||||
|
||||
`SIGTERM` (or Ctrl-C) → uvicorn drains in-flight requests → cancels the MFT scheduler (in-flight tick completes via `asyncio.to_thread`) → cancels the backup scheduler (in-flight tick completes) → closes the SQLAlchemy engine → exits 0. There is no checkpoint; the in-flight tick is allowed to complete naturally because the idempotency table makes a partial-tick restart safe.
|
||||
|
||||
---
|
||||
|
||||
## 11. Deployment
|
||||
|
||||
### 11.1 Local dev (default)
|
||||
|
||||
The default deployment. Two terminals:
|
||||
|
||||
```bash
|
||||
# Terminal 1 — backend on 127.0.0.1:8000
|
||||
cd backend
|
||||
.venv/bin/python -m cyclone serve
|
||||
|
||||
# Terminal 2 — frontend on localhost:5173
|
||||
cd ..
|
||||
npm run dev
|
||||
```
|
||||
|
||||
The frontend reads its backend URL from `VITE_API_BASE_URL` (default empty). Create `.env.local` at the repo root with `VITE_API_BASE_URL=http://127.0.0.1:8000`. Without it, the UI falls back to its in-memory sample store via the `data` adapter (parses are disabled).
|
||||
|
||||
### 11.2 Optional Docker (SP23 — product fork)
|
||||
|
||||
A Ubuntu + Docker + auth + RBAC + LAN-bind spec exists at [`docs/superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md`](superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md). This is a **product fork** awaiting user decision (per [REQUIREMENTS.md §6.2](REQUIREMENTS.md)). The v1 single-host single-operator posture assumes local-only; SP23 changes the threat model to "remote operator on LAN."
|
||||
|
||||
### 11.3 Single-host "production"
|
||||
|
||||
For a single operator on a single machine, the dev deployment IS the production deployment. There is no separate build / deploy pipeline; the v1 distribution model is `git clone && pip install -e '.[dev]' && npm install && python -m cyclone serve`. A release-time gate (not yet implemented — backlog item) would run `pytest` (backend) and `npm test` (frontend) and fail if any test is red.
|
||||
|
||||
---
|
||||
|
||||
## 12. Out of scope (v1)
|
||||
|
||||
The following are explicitly NOT built in v1, by design:
|
||||
|
||||
- **Multi-user / multi-host** — login is required (single-user model today; SP23 adds RBAC). LAN-bind, Docker, reverse proxy still out of scope. (SP23 covers the LAN-bind / Docker / RBAC product fork.)
|
||||
- **Other trading partners** — only Colorado Medicaid via Gainwell is configured. The `PayerConfig` mechanism (SP9) supports adding more; the spec/plan/CLI work for that is on the backlog.
|
||||
- **837I / 837D / 834 / 820 / 275 / 278 / 276-277** — only 837P and 835 are first-class. Other transaction types are listed in [REQUIREMENTS.md §6.2](REQUIREMENTS.md) as backlog items.
|
||||
- **NPPES real-time lookup** — only local NPI checksum validation (SP20). The operator who wants real registry lookup has to wire it in later.
|
||||
- **EHR / PM integration** — no webhook in/out, no callable `POST /api/parse-837` (the pipeline agent uses `/api/upload` for now).
|
||||
- **COB / secondary claims** — no automatic coordination-of-benefits generator. The operator generates a secondary 837P by hand from the CAS adjustments.
|
||||
- **Real-time eligibility** — no SFTP pickup + SOAP + 271 round-trip. The 270/271 parsers exist; the live cycle doesn't.
|
||||
|
||||
Each of these is a separate SP. The "v1 done" bar is the 22 shipped SPs (SP1–SP22) on `main` plus the [REQUIREMENTS.md §8 DoD](REQUIREMENTS.md).
|
||||
|
||||
---
|
||||
|
||||
## 13. References
|
||||
|
||||
### 13.1 Top-level docs
|
||||
|
||||
- [Requirements](REQUIREMENTS.md) — FRs, NFRs, DoD, traceability
|
||||
- [README](../../README.md) — install + dev + pipeline agent
|
||||
|
||||
### 13.2 Specs (per sub-project)
|
||||
|
||||
All in `docs/superpowers/specs/`:
|
||||
|
||||
- SP1 production-readiness: `2026-06-19-cyclone-production-readiness-design.md`
|
||||
- SP2 DB + reconciliation: `2026-06-19-cyclone-db-reconciliation-design.md`
|
||||
- SP3 EDI features: `2026-06-20-cyclone-edi-features-design.md`
|
||||
- SP4 claim drawer: `2026-06-20-cyclone-claim-drawer-design.md`
|
||||
- SP5 live tail: `2026-06-20-cyclone-live-tail-design.md`
|
||||
- SP7 line reconciliation: `2026-06-20-cyclone-line-reconciliation-design.md`
|
||||
- SP8 serialize 837: `2026-06-20-cyclone-serialize-837-design.md`
|
||||
- SP9 multi-payer / NPI / SFTP stub: `2026-06-20-cyclone-multi-payer-npi-sftp-design.md`
|
||||
- SP10 277CA: `2026-06-23-cyclone-sp10-277ca-payer-rejected-design.md`
|
||||
- SP11 audit log: `2026-06-23-cyclone-sp11-hash-chained-audit-design.md`
|
||||
- SP12 SQLCipher: `2026-06-23-cyclone-sp12-sqlcipher-encryption-design.md`
|
||||
- SP13 SFTP client: `2026-06-23-cyclone-sp13-sftp-client-design.md`
|
||||
- SP14 5-lane Inbox: `2026-06-23-cyclone-sp14-inbox-5lane-design.md`
|
||||
- SP15 key rotation: `2026-06-23-cyclone-sp15-key-rotation-design.md`
|
||||
- SP16 MFT scheduler: `2026-06-23-cyclone-sp16-mft-polling-design.md`
|
||||
- SP17 encrypted backup: `2026-06-21-cyclone-encrypted-backup-design.md`
|
||||
- SP18 structured logging: `2026-06-21-cyclone-structured-logging-design.md`
|
||||
- SP19 security hardening: `2026-06-21-cyclone-security-hardening-design.md`
|
||||
- SP20 NPI validation: `2026-06-21-cyclone-npi-validation-design.md`
|
||||
- SP21 store split: `2026-06-21-cyclone-store-split-design.md`; universal drilldown: `2026-06-21-cyclone-universal-drilldown-design.md`
|
||||
- SP22 parse-decide: `2026-06-21-cyclone-parse-decide-workflow-design.md`; pipeline agent: `2026-06-21-cyclone-pipeline-agent-design.md`
|
||||
- SP23 (fork) Ubuntu + Docker: `2026-06-22-cyclone-ubuntu-docker-deployment-design.md`
|
||||
|
||||
### 13.3 Plans
|
||||
|
||||
All in `docs/superpowers/plans/`. 27 files — one per spec (plus the round-2 backfill for SP9–SP20 in the `2026-06-23-cyclone-sp*` naming).
|
||||
|
||||
### 13.4 Reference docs
|
||||
|
||||
- `docs/reference/co-medicaid.md` — operator setup for SQLCipher + SFTP + the Gainwell trading-partner details (KP-specific).
|
||||
|
||||
### 13.5 Sibling project
|
||||
|
||||
- `cyclone-pipeline/` — round-trip agent (sibling, not in this repo). See [README](../../README.md) §"Pipeline automation agent" for install + usage.
|
||||
@@ -0,0 +1,739 @@
|
||||
# Cyclone — Requirements, Architecture, Tasks, and Test Strategy
|
||||
|
||||
**Date:** 2026-06-23
|
||||
**Status:** Round 2 (post-review corrections — ready for verification)
|
||||
**Audience:** New engineer or AI agent who needs to understand Cyclone's contract, the work that's already shipped, the work that's not, and what "done" means before they write a single line of code.
|
||||
**Scope:** Local-only, single-operator, single-host EDI claims-management suite for one billing office, currently trading with Colorado Medicaid (CO XIX).
|
||||
**Sister docs:** [`README.md`](../README.md) is the operator-facing entry point; [`docs/ARCHITECTURE.md`](ARCHITECTURE.md) is the top-level technical design (process topology, package layout, data flow, lifecycle); [`docs/superpowers/specs/`](superpowers/specs/) holds per-SP design specs; [`docs/superpowers/plans/`](superpowers/plans/) holds per-SP implementation plans; [`docs/reviews/2026-06-20-cyclone-completeness-review.md`](reviews/2026-06-20-cyclone-completeness-review.md) is the industry-scope gap analysis; [`docs/reference/`](reference/) holds condensed X12 / payer notes.
|
||||
|
||||
---
|
||||
|
||||
## 0. How to read this document
|
||||
|
||||
| If you want to know… | Go to |
|
||||
|---|---|
|
||||
| What Cyclone is and is not | §1, §2 |
|
||||
| What it must do (functional) | §3 |
|
||||
| What it must *not* fail at (non-functional) | §4 |
|
||||
| How the system is built | §5 |
|
||||
| Which sub-project (SP) owns which requirement | §3 + §6 traceability matrix |
|
||||
| What "shipped" means | §7 |
|
||||
| What "done" means for the project as a whole | §8 |
|
||||
| How we test | §9 |
|
||||
| What we're assuming to be true | §10 |
|
||||
| What's still missing or risky | §11 |
|
||||
|
||||
Every requirement (§3, §4) has an ID; every ID is traceable to a sub-project (§6) which in turn points at a design spec and an implementation plan. The matrix at the end of §6 is the single index that ties everything together.
|
||||
|
||||
**Round history.** Round 1 (2026-06-23) produced this draft. Two independent reviewers (Reviewer A — components/data-model/deps/DoD; Reviewer B — traceability/consistency audit) read it against the actual codebase and surfaced 12 factual errors and 6 structural improvements. Round 2 (this version) applies every factual correction. Round 3 (pending) will re-verify reviewer agreement on the corrected doc.
|
||||
|
||||
---
|
||||
|
||||
## 1. The contract — what Cyclone is
|
||||
|
||||
A self-hosted EDI claims-management suite for a single billing office. Parses 837P professional claims and 835 ERA remittances (X12 005010X222A1 and 005010X221A1), with a local-only FastAPI backend and a React UI for browsing, filtering, and inspecting the parsed data.
|
||||
|
||||
- **Local-only on purpose:** binds `127.0.0.1`, requires login (auth boundary is HTTP; bcrypt + HttpOnly session cookie), no internet exposure, single operator, single machine, one trading partner (Colorado Medicaid, currently).
|
||||
- **Single-process:** the backend is one Python process; the frontend is one Vite-served React app; there is no message broker, no separate worker, no queue.
|
||||
- **Single source of truth:** SQLite at `~/.local/share/cyclone/cyclone.db` (overridable via `CYCLONE_DB_URL`); optionally encrypted at rest via SQLCipher (SP12).
|
||||
- **Threat model:** a process on the same host, plus a deliberate HTTP-layer login (no remote unauthenticated read or write). File-system threats (stolen/imaged drive) are still the primary concern; SQLCipher at rest + the macOS Keychain handle those. The SP23 product fork changes the threat model to "remote operator on LAN."
|
||||
|
||||
The Cyclone codebase has honored this contract end-to-end across 22 shipped sub-projects. Every design spec re-asserts the local-only / single-operator / single-payer / no-auth posture and lists (often at length) what is intentionally out of scope.
|
||||
|
||||
---
|
||||
|
||||
## 2. Scope boundaries
|
||||
|
||||
### 2.1 In scope (today, after SP1–SP22)
|
||||
|
||||
| Capability | SP |
|
||||
|---|---|
|
||||
| 837P (005010X222A1) parse + serializer with round-trip guarantee | SP1 + SP8 |
|
||||
| 835 (005010X221A1) parse + CAS deep-parsing | SP1 + SP3 |
|
||||
| 999, TA1, 270, 271, 277CA parse / generate | SP3 + SP10 |
|
||||
| Structural validation rules (R021, R034, R035, R200, R210, …) | SP3 + SP20 |
|
||||
| SQLite persistence with forward-compatible migrations | SP2 + SP9–SP17 |
|
||||
| Automatic 837P ↔ 835 reconciliation; manual reconciliation UI | SP2 + SP7 |
|
||||
| 7-state claim lifecycle with reversal handling | SP2 |
|
||||
| Tamper-evident hash-chained audit log | SP11 |
|
||||
| NDJSON pubsub on store writes + live-tail UI with reconnect | SP5 |
|
||||
| 5-lane inbox (Rejected / Payer-rejected / Candidates / Unmatched / Done today) | SP6 + SP10 + SP14 |
|
||||
| Per-line 837 SV1 ↔ 835 SVC adjustment audit | SP7 |
|
||||
| Per-claim and per-remit detail drawers + Cmd-K search + batch diff + CSV export | SP4 + SP21 |
|
||||
| Universal drilldown (drawer + peek modal everywhere) | SP21 |
|
||||
| Multi-payer / multi-NPI config + clearhouse stub | SP9 |
|
||||
| `paramiko`-backed SFTP wire to Gainwell MFT | SP13 |
|
||||
| 24h MFT inbound scheduler with idempotent file processing | SP16 |
|
||||
| SQLCipher encryption at rest + key rotation | SP12 + SP15 |
|
||||
| Automated AES-256-GCM encrypted DB backups | SP17 |
|
||||
| Structured JSON logging + PII scrubber | SP18 |
|
||||
| Security hardening (body size limit, rate limit, security headers) | SP19 |
|
||||
| NPI Luhn checksum + Tax ID format validation | SP20 |
|
||||
| CycloneStore split (refactor; **in-flight on `refactor/store-split`**) | SP21 |
|
||||
| Parse-then-decide upload dedup (idempotency) | SP22 |
|
||||
| Pipeline automation agent (sibling project at `cyclone-pipeline/`) | SP22 |
|
||||
|
||||
### 2.2 Out of scope (today, by design)
|
||||
|
||||
| Item | Why |
|
||||
|---|---|
|
||||
| AS2 / AS4 (EDIINT, signed, encrypted, MDN-back) connectivity | Single-host local tool; the operator hands the 837 ZIP to the Gainwell MFT UI manually |
|
||||
| Real-time 270/271 round-trip over CAQH CORE Phase II/III SOAP envelopes | Out of single-host scope; builder + parser are X12-only |
|
||||
| 837I (institutional), 837D (dental), 834, 820, 275, 278, 276/277 | Single-payer scope; not currently demanded by the CO XIX workflow |
|
||||
| NPPES NPI registry lookup (offline validation only) | Single-host, no internet exposure |
|
||||
| ICD-10 / HCPCS / NDC vocabulary tables | Structural validation only; the operator has the source vocab |
|
||||
| COB / secondary-claim generator | Workflow scope |
|
||||
| HITRUST / SOC 2 / BAA template | Single-operator; threat model doesn't require them |
|
||||
| HA / DR / load balancing / multi-host replication | Single-host scope |
|
||||
| Prometheus / Grafana / alerting | `curl health \| mail` cron is sufficient |
|
||||
| 2FA / SSO / OAuth | Single-operator |
|
||||
| LUKS full-disk encryption | Operator / host concern |
|
||||
| Component / E2E browser tests (Playwright) | Local-only, manual smoke scripts are sufficient |
|
||||
|
||||
### 2.3 Deferred with rationale
|
||||
|
||||
- **Public TLS / public domain / Caddy reverse proxy** — LAN-only bind is enough for the v1 single-Ubuntu-server posture; VPN handles outside access. (Re-evaluated if the Ubuntu Docker plan ships.)
|
||||
- **Off-box automated backup shipping** — host cron / rsync is documented in the operator runbook, not automated in code.
|
||||
- **Self-service password reset** — admin can reset via CLI; self-service is v2 (would only be relevant if auth ships).
|
||||
- **Per-file encryption of prodfiles / `sftp_staging`** — relies on volume-level filesystem permissions + the SQLCipher-encrypted DB + the host being physically secure.
|
||||
- **Watchtower / automatic updates** — manual `docker compose pull` keeps the operator in the loop on schema migrations.
|
||||
|
||||
---
|
||||
|
||||
## 3. Functional requirements
|
||||
|
||||
Each requirement is `FR-NN`, traceable to one or more sub-projects. Verification is "shipped + tested" (see §6) or "planned" (see §6.2).
|
||||
|
||||
| ID | Requirement | SP |
|
||||
|---|---|---|
|
||||
| FR-1 | Parse an uploaded 837P file into structured `Claim` records. | SP1 |
|
||||
| FR-2 | Parse an uploaded 835 file into structured `Remittance` records with per-CAS adjustment rows. | SP1 + SP3 |
|
||||
| FR-3 | Persist every successful parse to SQLite; survive backend restart. | SP2 |
|
||||
| FR-4 | Run a forward-compatible migration runner (`PRAGMA user_version`) so the schema can evolve without Alembic. | SP2 |
|
||||
| FR-5 | Auto-match each 835 `CLP` back to a prior 837 `Claim` within ±7 days via `(claim_id, service_date)`; apply lifecycle transitions; write `Match` and `ActivityEvent` rows. | SP2 |
|
||||
| FR-6 | Maintain a 7-state claim lifecycle (`submitted → received → paid|partial|denied → reconciled → reversed`) with `state_before_reversal` preserved on reversal. | SP2 |
|
||||
| FR-7 | Provide a `/reconciliation` page that lists unmatched claims and unmatched remittances and supports manual pair/unpair. | SP2 (tests live in `test_reconcile.py`, `test_reconcile_line_level.py`, `test_inbox_endpoints.py`, `test_api_gets.py`) |
|
||||
| FR-8 | Run structural 837P validation rules R021 (NPI checksum, warning-only), R034 (`REF*G1` on frequency 7/8), R035 (`BHT06` transaction-type allowlist), R200/R210. | SP3 + SP20 |
|
||||
| FR-9 | Generate a 999 ACK transaction set on 837 ingest when `?ack=true` is passed to `POST /api/parse-837`; persist it to the `acks` table. | SP3 |
|
||||
| FR-10 | Parse inbound 999 ACKs and surface them on a `/acks` page with regenerable raw text. | SP3 |
|
||||
| FR-11 | Parse + persist TA1 interchange ACKs (separate from 999 transaction-set ACKs). | SP3 |
|
||||
| FR-12 | Build a 270 from a JSON payload (`POST /api/eligibility/request`) and parse an inbound 271 (`POST /api/eligibility/parse-271`) into `coverage_benefits`. | SP3 |
|
||||
| FR-13 | Parse inbound 277CA files and stamp `payer_rejected` on claims whose STC category is A4/A6/A7 — monotonic (a looser later 277CA cannot downgrade). Also persist 277CA ACK rows to `two77ca_acks`. | SP10 |
|
||||
| FR-14 | Provide a `/inbox` page with five lanes ordered by urgency: Rejected (999 R/E), Payer-rejected (277CA A4/A6/A7), Candidates (CLP didn't auto-match), Unmatched (waiting), Done today (last 24h terminal transitions). | SP6 + SP10 + SP14 |
|
||||
| FR-15 | Score each Candidate on 4 weighted fields (patient control number 40, service date 25, charge amount 20, provider NPI 15); tier strong (≥75, Match enabled), weak (50–74, dimmed), hidden (<50). | SP6 |
|
||||
| FR-16 | Provide per-claim and per-remit detail drawers with full raw X12 segments, state-history timeline, validation panel, line-level reconciliation tab, and party grid. URL is synced (`?claim=…`, `?remit=…`). | SP4 + SP7 + SP21 |
|
||||
| FR-17 | Provide a side-by-side Batch Diff view (added / removed / changed claims). | SP4 (logic lives in `cyclone/batch_diff.py`) |
|
||||
| FR-18 | Provide a global Cmd-K search across claims, remittances, activity, and acks. | SP4 |
|
||||
| FR-19 | Provide per-page CSV export for claims, remittances, and inbox lanes. | SP4 |
|
||||
| FR-20 | Provide a live-tail NDJSON stream on `/api/{claims,remittances,activity}/stream` with snapshot-first, `snapshot_end`, heartbeat, and reconnect. | SP5 |
|
||||
| FR-21 | Surface a `TailStatusPill` with states `live / connecting / reconnecting / stalled / error / closed`, with backoff ladder `1s → 2s → 4s → 8s → 16s → 30s` capped. | SP5 |
|
||||
| FR-22 | Tie every 835 CAS adjustment back to the specific 837 SV1 service line it adjudicates; surface unmatched-line warnings. | SP7 (line-reconciliation rows in `line_reconciliations`) |
|
||||
| FR-23 | Serialize a single edited claim back to a valid X12 837P file with round-trip guarantee (113 prodfiles parse → serialize → parse to the same `claim_id`). | SP8 |
|
||||
| FR-24 | Export a ZIP bundle of selected rejected claims as a batch ready to hand to the MFT UI. | SP8 |
|
||||
| FR-25 | Load payer / payer_config / clearhouse config from `config/payers.yaml` (parsed via `pyyaml`) and 3 new DB tables (`providers`, `payers`, `payer_configs`); replace the in-code `PAYER_FACTORIES` dict. | SP9 |
|
||||
| FR-26 | Push a `cyclone clearhouse submit` to the Gainwell MFT inbound path via `paramiko` (`SftpClient` lives in `cyclone/clearhouse/__init__.py`). Credentials read from the macOS Keychain via the `keyring` library at call time. | SP13 |
|
||||
| FR-27 | Poll the Gainwell MFT inbound path every tick (configurable); route downloaded files to the right parser (999 / 835 / 277CA / TA1); idempotent via `processed_inbound_files`; crash-safe via per-file try/except. | SP16 |
|
||||
| FR-28 | Optionally encrypt the SQLite file via SQLCipher (AES-256); key fetched from the macOS Keychain via `keyring`. Falls back to plain SQLite if the Keychain entry is missing. | SP12 |
|
||||
| FR-29 | Rotate the SQLCipher key in place via `PRAGMA rekey`, serialized through a `threading.Lock` and SQLAlchemy `NullPool`; write `db.key_rotated` audit event with old + new key fingerprints and post-rotation `table_count`. | SP15 |
|
||||
| FR-30 | Produce AES-256-GCM encrypted DB backups (PBKDF2-HMAC-SHA256, 200k iters) via the existing `BackupService`; two-step restore (`initiate` → `confirm` with one-shot 64-char hex token); retention pruning with a 30-day default. Backups table is `db_backups`. | SP17 |
|
||||
| FR-31 | Emit all API / CLI / scheduler / backup logs as newline-delimited JSON with ISO-8601 ms timestamps; redact obvious PHI (NPIs, SSNs, DOBs, patient names) via a `PiiScrubber` filter on message + extras. | SP18 (uses stdlib `logging` + a JSON formatter, not structlog) |
|
||||
| FR-32 | Reject oversize request bodies (413), rate-limit requests (429), and add default security headers via pure-ASGI middlewares; emit a tamper-evident `api.request_rejected` audit event on 413/429. | SP19 |
|
||||
| FR-33 | Surface a rich `GET /api/health` response — DB connectivity, MFT scheduler state, backup scheduler state, live pubsub subscriber counts, last batch id + timestamp. | SP19 (`api_routers/health.py:28-40`) |
|
||||
| FR-34 | Validate NPI with the CMS-published Luhn over `80840 + body` (warning-only — placeholder NPIs in test fixtures shouldn't block ingest). Reject Tax ID (EIN) with reserved prefixes (`00`, `07`, `80`–`89`). | SP20 |
|
||||
| FR-35 | Split `CycloneStore` into per-concern modules (`store_persistence`, `store_reconcile`, `store_queries`, `store_mappers`) without changing the public store API. | SP21 (**in-flight** on `refactor/store-split` branch) |
|
||||
| FR-36 | De-duplicate 837 / 835 uploads via a parse-then-decide workflow (`find_existing_batch_for_claim` / `find_existing_batch_for_remit`); relax the claims/remits PK to `(batch_id, id)` via migration 0014 to support re-parses into different batches. | SP22 (migration 0014 in worktree; not yet on `main`) |
|
||||
| FR-37 | (Sibling project, not in this repo) Drive the full 7-phase round-trip — preflight → browser upload → parse verify → SFTP submit → TA1 wait → 999 wait → scan + report — with crash-safe resume, structured JSON logging, idempotency dedup, per-run folder. | SP22 (`cyclone-pipeline/`) |
|
||||
| FR-38 | Provide a `DrillStackProvider` + `DrillDrawerHeader` shell so every drillable cell across the app opens a consistent drawer or peek modal. | SP21 |
|
||||
|
||||
---
|
||||
|
||||
## 4. Non-functional requirements
|
||||
|
||||
| ID | NFR | Source / SP |
|
||||
|---|---|---|
|
||||
| NFR-1 | **Local-only bind + auth.** Backend binds `127.0.0.1:8000` (overridable via `CYCLONE_PORT`); CORS allowlist is exact (`http://localhost:5173`); login required (bcrypt + HttpOnly session cookie; first admin bootstrapped from env vars `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD`); dev/test escape hatch via `CYCLONE_AUTH_DISABLED=1` (logs WARNING at boot). | SP1 + auth (2026-06-23 merge) + SP24 (doc reconciliation) |
|
||||
| NFR-2 | **Determinism.** Parser / serializer round-trip is guaranteed on 113 real prodfiles (`docs/prodfiles/claims/*.x12`); canonical fields, not byte-identity (called out in the SP8 spec). | SP8 |
|
||||
| NFR-3 | **Audit completeness.** Every reconciliation anomaly, every state transition, every 999/277CA reject writes an `ActivityEvent` (in `activity_events` table, un-chained; powers the `/activity` feed and inbox state transitions). | SP2 + SP10 |
|
||||
| NFR-4 | **Tamper-evidence.** A separate `audit_log` table (SP11) carries SHA-256 hash-chained rows for security-sensitive events (login attempts, rejections, key rotations, backup lifecycle, rejected requests). `GET /api/admin/audit-log/verify` detects any break. **The `activity_events` (NFR-3) and `audit_log` (NFR-4) tables are distinct** — different semantics, different audiences. New code that needs an audit row must decide which one to write to. | SP11 |
|
||||
| NFR-5 | **Encrypted at rest (optional).** SQLite is encrypted via SQLCipher AES-256 when the macOS Keychain entry exists and `sqlcipher3` is installed; plain SQLite otherwise. | SP12 |
|
||||
| NFR-6 | **Encrypted backups.** Backups are AES-256-GCM encrypted at rest (PBKDF2-HMAC-SHA256, 200k iters); passphrase lives in the macOS Keychain (`cyclone backup init-passphrase`). | SP17 |
|
||||
| NFR-7 | **Logging discipline.** API / CLI / scheduler / backup logs flow through a `JsonFormatter` (NDJSON, ISO-8601 ms) by default; `PiiScrubber` redacts obvious PHI; configurable via `CYCLONE_LOG_LEVEL` / `CYCLONE_LOG_FILE` / `CYCLONE_LOG_JSON` / `CYCLONE_LOG_NO_PII_SCRUB`. | SP18 |
|
||||
| NFR-8 | **Operational hygiene.** Request body size limit, rate limit, and default security headers are middleware-enforced (SP19). `EXEMPT_PATHS = ("/api/health", "/healthz", "/readyz")` is the documented bypass list (`cyclone/security.py:201`). | SP19 |
|
||||
| NFR-9 | **Thread-affinity.** SQLCipher operations are serialized through a `threading.Lock` + SQLAlchemy `NullPool` to keep SQLCipher thread-affine under FastAPI's per-request threadpool. | SP12 + SP15 |
|
||||
| NFR-10 | **Live-tail resilience.** Heartbeat every 15s default (`CYCLONE_TAIL_HEARTBEAT_S`); client flips to `stalled` after 30s of total silence (heartbeat included); reconnect backoff capped at 30s. | SP5 |
|
||||
| NFR-11 | **Payer config externalized.** Payer / payer_config / clearhouse config lives in `config/payers.yaml` + DB tables, not in code; a new payer is a YAML + DB row, not a code change. Note: `co_medicaid()` factory in `cyclone/parsers/payer.py:57-58` still exists as a fallback for tests and the default-payer selection (R-4 in §11). | SP9 |
|
||||
| NFR-12 | **Idempotency.** Inbound MFT files are de-duplicated via `processed_inbound_files`; 837/835 uploads are de-duplicated via the parse-then-decide workflow with `(batch_id, claim_id)` / `(batch_id, remit_id)` relaxed PKs. | SP16 + SP22 |
|
||||
| NFR-13 | **Crash-safety.** Incoming files are processed inside per-file try/except so a bad file doesn't stop the MFT loop; the pipeline agent has crash-safe resume keyed by run id. | SP16 + SP22 |
|
||||
| NFR-14 | **Single-process.** No message broker, no separate worker, no queue; everything runs in one Python process. | (project-wide constraint) |
|
||||
| NFR-15 | **Test density.** Backend tests: **964 collected** (`pytest --collect-only`, 2026-06-23); frontend test files: **73**; prodfiles are exercised (113 + 19 + 1369), not just minimal fixtures. | (project-wide) |
|
||||
| NFR-16 | **Documentation discipline.** Every shipped sub-project has a design spec + implementation plan + smoke test; a new engineer can read the project top-to-bottom. (SPs 9–16 ship without on-disk plan files — see §6.1 — but their `feat(spN)` commits are on `main` and carry their own commit-message plan.) | (project-wide) |
|
||||
| NFR-17 | **Fail-soft posture.** Reconciliation crashes don't lose the 835; the activity event records the failure. | SP2 |
|
||||
| NFR-18 | **Migration safety.** Migrations are forward-only via `PRAGMA user_version`; rollback procedures are documented per migration; the `user_version` runner is idempotent. **No checksum manifest exists today** (R-13 in §11). | SP2 + SP22 |
|
||||
|
||||
---
|
||||
|
||||
## 5. Architecture overview
|
||||
|
||||
### 5.1 Components
|
||||
|
||||
**Backend (one Python process, FastAPI on uvicorn) — modules under `backend/src/cyclone/`:**
|
||||
|
||||
| Module | Owns | Notes |
|
||||
|---|---|---|
|
||||
| `cyclone.api` | HTTP routes, content negotiation, CORS allowlist, lifespan | 3,145 LOC (largest single file; partly split into `api_routers/` for 4 sub-routes) |
|
||||
| `cyclone.api_helpers` | NDJSON / content-negotiation / live-tail helpers | |
|
||||
| `cyclone.api_routers` | `acks`, `admin`, `health` (`/api/health`), `ta1_acks` sub-routers | Mounted at `api.py:270-273`; only 4 routes split out — bulk of routes still in `api.py` |
|
||||
| `cyclone.store` | Public store facade; persistence, reconciliation, queries, mappers | 2,172 LOC; SP21 split is in-flight on `refactor/store-split` |
|
||||
| `cyclone.db` | SQLAlchemy engine, session factory, ORM models (18 tables) | |
|
||||
| `cyclone.db_migrate` | `PRAGMA user_version` migration runner | |
|
||||
| `cyclone.db_crypto` | Optional SQLCipher encryption at rest; key rotation | `rotate_key()` is destructive (rewrites every page) |
|
||||
| `cyclone.audit_log` | Hash-chained `audit_log` (SP11, SHA-256 chain) | Distinct from `cyclone.activity_events` written by other modules |
|
||||
| `cyclone.inbox_lanes` | The 5-lane inbox payload (Rejected / Payer-rejected / Candidates / Unmatched / Done today) | |
|
||||
| `cyclone.inbox_state` | 999 envelope reject → claim state transitions | |
|
||||
| `cyclone.inbox_state_277ca` | 277CA STC A4/A6/A7 → payer_rejected stamp (monotonic) | |
|
||||
| `cyclone.providers` | Multi-NPI provider lookups | |
|
||||
| `cyclone.payers` | Payer / payer_config lookups | Loads `config/payers.yaml` via `pyyaml` |
|
||||
| `cyclone.secrets` | macOS Keychain-backed secret fetcher (wraps `keyring`) | |
|
||||
| `cyclone.reconcile` | Pure-function 835→claim match + line-level match | No DB session inside; testable with fabricated ORM objects |
|
||||
| `cyclone.scoring` | 4-field weighted candidate scoring (40/25/20/15) | Weights hardcoded — see R-10 in §11 |
|
||||
| `cyclone.pubsub` | In-process EventBus (drop-oldest, per-kind fan-out) | In-process only; not thread-safe across asyncio event loops |
|
||||
| `cyclone.backup` | AES-256-GCM encrypted backups (orchestration + CLI) | 216 LOC |
|
||||
| `cyclone.backup_service` | Backup lifecycle (create / verify / restore) | 740 LOC |
|
||||
| `cyclone.backup_scheduler` | 24h backup autostart loop | 315 LOC |
|
||||
| `cyclone.scheduler` | asyncio MFT polling loop (SP16) | Uses `processed_inbound_files` for idempotency |
|
||||
| `cyclone.security` | Per-IP rate-limit, body-size limit, security headers (`SecurityHeadersMiddleware`, `BodySizeLimitMiddleware`, `RateLimitMiddleware`) | `EXEMPT_PATHS` list at `:201` |
|
||||
| `cyclone.logging_config` | JSON formatter, PII scrubber | |
|
||||
| `cyclone.npi` | NPI Luhn + EIN format validators | |
|
||||
| `cyclone.batch_diff` | Batch Diff engine (powers `GET /api/batch-diff`) | 12 public functions/classes |
|
||||
| `cyclone.clearhouse` (`__init__.py`) | `SftpClient` (SP9 stub + SP13 paramiko), `InboundFile`, `make_client()` factory | 318 LOC; the SFTP home |
|
||||
| `cyclone.edi` (`filenames.py`) | HCPF X12 File Naming Standards helpers | Used by SP13/SP16 outbound filename construction |
|
||||
| `cyclone.cli` | Click CLI | |
|
||||
| `cyclone.__main__` | `python -m cyclone serve` | |
|
||||
| `cyclone.parsers/` | X12 tokenizer, models, validator, writers, 277CA, 999, TA1, 270, 271, CAS, seg | 25 `.py` modules |
|
||||
|
||||
**Stale artifact (do not import):** `backend/src/cyclone/workflow/` contains only a `__pycache__/` subdir with stale `.pyc` files (no `__init__.py`, no source). The actual sources live at the top level. Recommend deleting the directory.
|
||||
|
||||
**Frontend (one Vite-served React app) — directories under `src/`:**
|
||||
|
||||
| Directory | Owns |
|
||||
|---|---|
|
||||
| `src/pages/` | Acks, ActivityLog, BatchDiff, Batches, Claims, Dashboard, Inbox, Providers, Reconciliation, Remittances, Upload (11 pages) |
|
||||
| `src/hooks/` | TanStack Query hooks, URL-state hooks, live-tail hooks, keyboard hooks, batch-export, search, drawer keyboard, count-up, provider detail, payer summary, parse, ack detail (35+ files) |
|
||||
| `src/lib/` | `api.ts`, `format.ts`, `utils.ts`, `csv.ts`, `download.ts`, `event-routing.ts`, `tail-stream.ts`, `inbox-api.ts` (all with paired `.test.ts`) |
|
||||
| `src/components/` | 3 drawer families (`ClaimDrawer/` ~15 files, `RemitDrawer/` ~10 files, `ProviderDrawer/` ~6, `AckDrawer/` 3); `inbox/` (10 files), `drill/` (10 files, includes `DrillStackProvider` + `DrillDrawerHeader`), `charts/` (4 files), `ui/` (18 shadcn primitives — radix + cva + tailwind-merge), `KeyboardCheatsheet/` (3 files), plus top-level ActivityFeed, AnimatedNumber, BatchDetail, BatchDiffView, BatchesList, ClaimCard837, DominantKpiCard, EditorialNote, ExportBar, ExportCsvButton, KpiCard, Layout, NewClaimDialog, PageHeader, SearchBar, SearchResults, Sidebar, Sparkline, StatusBadge, StatusPill, TailStatusPill, TickerTape |
|
||||
| `src/store/` | `index.ts` (zustand sample data), `tail-store.ts` (FIFO-capped live tail slices) |
|
||||
| `src/types/` | `index.ts` (flat barrel of shared TS types) |
|
||||
|
||||
### 5.2 Data model
|
||||
|
||||
**12 SQL migrations** in `backend/src/cyclone/migrations/` (verified on disk 2026-06-23):
|
||||
|
||||
| # | File | Tables / columns added |
|
||||
|---|---|---|
|
||||
| 0001 | `0001_initial.sql` | `batches`, `claims`, `remittances`, `cas_adjustments`, `matches`, `activity_events` (6 tables) |
|
||||
| 0002 | `0002_acks.sql` | `acks` |
|
||||
| 0003 | `0003_drop_claims_remits_unique_constraints.sql` | Drops `claims` and `remittances` UNIQUE constraints |
|
||||
| 0004 | `0004_rejections_and_state_history.sql` | Adds `rejection_reason`, `rejected_at`, `resubmit_count`, `state_changed_at` to `claims` (no new table) |
|
||||
| 0005 | `0005_create_ta1_acks.sql` | `ta1_acks` |
|
||||
| 0006 | `0006_line_reconciliation.sql` | `service_line_payments`, `line_reconciliations`; FK `service_line_payment_id` on `cas_adjustments`; `claim_level_adjustment_amount` on `remittances` |
|
||||
| 0007 | `0007_providers_payers_clearhouse.sql` | `providers`, `payers`, `payer_configs`, `clearhouse` |
|
||||
| 0008 | `0008_payer_rejected_columns.sql` | `two77ca_acks` (table) + `payer_rejected` columns on `claims` |
|
||||
| 0009 | `0009_audit_log.sql` | `audit_log` (SHA-256 hash-chained) |
|
||||
| 0010 | `0010_payer_rejected_acknowledged.sql` | `payer_rejected_acknowledged` column on `claims` |
|
||||
| 0011 | `0011_processed_inbound_files.sql` | `processed_inbound_files` (SP16 idempotency) |
|
||||
| 0012 | `0012_backups.sql` | `db_backups` (SP17) |
|
||||
|
||||
**In-flight on `claims-unique-fix` worktree (not yet on `main`):**
|
||||
|
||||
| # | File | Tables / columns added |
|
||||
|---|---|---|
|
||||
| 0013 | `0013_drop_claims_unique_constraint.sql` | Drop `claims` UNIQUE constraint |
|
||||
| 0014 | `0014_relax_claims_remits_pk.sql` | Relax PK to `(batch_id, id)` (SP22) |
|
||||
|
||||
**ORM models** (declared in `backend/src/cyclone/db.py` — 18 classes total):
|
||||
|
||||
| Python class | Table | Notes |
|
||||
|---|---|---|
|
||||
| `Batch` | `batches` | |
|
||||
| `Claim` | `claims` | |
|
||||
| `Remittance` | `remittances` | |
|
||||
| `CasAdjustment` | `cas_adjustments` | |
|
||||
| `ServiceLinePayment` | `service_line_payments` | Created by 0006; SP1+ |
|
||||
| `LineReconciliation` | `line_reconciliations` | Created by 0006; SP7 backbone |
|
||||
| `Match` | `matches` | Reversals add a second row (NOT UNIQUE on `(claim_id, remittance_id)`) |
|
||||
| `Ack` | `acks` | |
|
||||
| `Ta1Ack` | `ta1_acks` | |
|
||||
| `ActivityEvent` | `activity_events` | Created by 0001; SP2 — distinct from `AuditLog` |
|
||||
| `Two77caAck` | `two77ca_acks` | Created by 0008; SP10 |
|
||||
| `Provider` | `providers` | |
|
||||
| `Payer` | `payers` | |
|
||||
| `PayerConfigORM` | `payer_configs` | |
|
||||
| `ClearhouseORM` | `clearhouse` | Singleton row (`id = 1`) |
|
||||
| `AuditLog` | `audit_log` | SHA-256 hash-chained |
|
||||
| `ProcessedInboundFile` | `processed_inbound_files` | |
|
||||
| `DbBackup` | `db_backups` | Note: table is `db_backups`, NOT `backups` |
|
||||
|
||||
**Boundary contracts:**
|
||||
|
||||
| Module | Public API | Pure? | Notes |
|
||||
|---|---|---|---|
|
||||
| `cyclone.db` | `init_db()`, `SessionLocal()`, model classes | No (engine + schema) | |
|
||||
| `cyclone.db_migrate` | `run(engine)` | Yes (modulo engine) | |
|
||||
| `cyclone.reconcile` | `match()`, `apply_payment()`, `apply_reversal()`, `split_unmatched()`, line-level match | **Yes** (no DB session inside) | Unit-testable with fabricated ORM objects |
|
||||
| `cyclone.scoring` | `score_pair(...)` | Yes | Weights hardcoded — see R-10 |
|
||||
| `cyclone.audit_log` | `append_event(...)`, `verify_chain()` | No (DB-owning) but append is deterministic | SHA-256 chain |
|
||||
| `cyclone.activity_events` (via store) | `append(...)` | No (DB-owning) | Un-chained; powers `/activity` feed |
|
||||
| `cyclone.store` | `add()`, `get_batch()`, `iter_*()`, `list_unmatched()`, `manual_match()`, `manual_unmatch()` | No (owns sessions, calls reconcile) | |
|
||||
| `cyclone.api` | HTTP routes | No | |
|
||||
| `cyclone.pubsub` | `subscribe_raw(kind, queue_size=256)`, `publish(kind, event)` | No (in-process) | Drop-oldest on full queue |
|
||||
| `cyclone.backup` + `backup_service` + `backup_scheduler` | `initiate_backup`, `confirm_restore`, `tick()` | No (DB-owning) | |
|
||||
| `cyclone.db_crypto` | `is_encryption_enabled()`, `rotate_key()` | No | `rotate_key()` is destructive |
|
||||
| `cyclone.clearhouse` | `SftpClient`, `make_client()`, `InboundFile` | No | Real `paramiko` in SP13 |
|
||||
| `cyclone.batch_diff` | `diff_batches()`, `diff_batches_to_wire()` | Yes | Pure function over claim/remit summaries |
|
||||
| `cyclone.cas_codes` | `CARC_REASON_CODES` dict | Yes | Annual refresh is manual (R-22 in completeness review) |
|
||||
| `cyclone.edi.filenames` | filename regexes + helpers | Yes | |
|
||||
| `cyclone.logging_config` | `JsonFormatter`, `PiiScrubber`, `CycloneDevFormatter` | No (installs handlers) | |
|
||||
| `cyclone.npi` | `validate_npi()`, `validate_tax_id()` | Yes | |
|
||||
| `cyclone.parsers.*` | Pure-function parse + serialize | Yes | |
|
||||
|
||||
### 5.3 Dependencies
|
||||
|
||||
**Backend runtime (verified against `backend/pyproject.toml:10-24`):**
|
||||
|
||||
- `pydantic>=2.6,<3`
|
||||
- `click>=8.1,<9`
|
||||
- `fastapi>=0.110,<1`
|
||||
- `uvicorn[standard]>=0.27,<1` (the `[standard]` extra pulls `httptools`, `uvloop`, `websockets`, `watchfiles`)
|
||||
- `python-multipart>=0.0.9,<1`
|
||||
- `sqlalchemy>=2.0,<3`
|
||||
- `pyyaml>=6.0,<7` ← **loads `config/payers.yaml`** (SP9)
|
||||
- `keyring>=25.0,<26` ← **macOS Keychain backend** used by `cyclone/secrets.py` (SP12/SP13/SP15); on Linux dispatches to Secret Service / kwallet
|
||||
- `cryptography>=49.0,<50`
|
||||
|
||||
**Backend optional extras:**
|
||||
|
||||
- `[sqlcipher]` → `sqlcipher3>=0.6,<1` (SP12; falls back to plain SQLite if missing)
|
||||
- `[sftp]` → `paramiko>=3.4,<6` (SP13)
|
||||
|
||||
**Backend dev/test (`backend/pyproject.toml:27-33`):**
|
||||
|
||||
- `pytest>=8.0`, `pytest-cov>=4.1`, `pytest-asyncio>=0.23,<1` (asyncio_mode = "auto"), `pytest-randomly>=4.1`, `httpx>=0.27,<1` ← **dev only, not runtime**
|
||||
- `ruff`, `mypy` — **not installed** (per completeness review §3.2.30 deferred)
|
||||
|
||||
**Frontend runtime (verified against `package.json`):**
|
||||
|
||||
- `react@^18.3.1`, `react-dom@^18.3.1`
|
||||
- `@tanstack/react-query@^5.101.0`
|
||||
- `lucide-react@^0.453.0`
|
||||
- `sonner@^1.5.0`
|
||||
- `zustand@^4.5.5`
|
||||
- `react-router-dom@^6.27.0` (note: package name is `react-router-dom`, not `react-router`)
|
||||
- `@radix-ui/react-dialog@^1.1.2`, `@radix-ui/react-label@^2.1.0`, `@radix-ui/react-select@^2.1.2`, `@radix-ui/react-slot@^1.1.0`, `@radix-ui/react-tabs@^1.1.15` ← powers `src/components/ui/` (shadcn primitives)
|
||||
- `class-variance-authority@^0.7.0`, `clsx@^2.1.1`, `tailwind-merge@^2.5.4`, `ansi-styles@^6.2.3` ← shadcn helper stack
|
||||
|
||||
**Frontend dev/test:**
|
||||
|
||||
- `vitest@^4.1.9` ← **unusually new version** (worth a sanity check — current stable is 1.x/2.x; verify it's intentional)
|
||||
- `@testing-library/react@^16.3.2`
|
||||
- `happy-dom@^20.10.6` ← **not jsdom** (vitest config line 13 explicitly says they don't need a DOM env: "Node 18+ exposes `Response` / `fetch` globals")
|
||||
- `typescript@^5.6.3`, `vite@^5.4.10`, `@vitejs/plugin-react@^4.3.3`
|
||||
- `@types/node`, `@types/react`, `@types/react-dom`, `autoprefixer`, `postcss`, `tailwindcss@^3.4.14`, `tailwindcss-animate`
|
||||
|
||||
**External services:** None. (The MFT path is one-way outbound to Gainwell MFT, polled via SFTP — not a service Cyclone depends on, it's a sink it writes to.)
|
||||
|
||||
**OS-level:** macOS Keychain (backed by `keyring`); SQLite / SQLCipher shared library; `ssh` host keys / known_hosts for SP13. SP23 would add `tini` + `curl` + `nginx` in container scenarios.
|
||||
|
||||
### 5.4 One-page data flow
|
||||
|
||||
```
|
||||
┌──────────────────────────────┐
|
||||
Operator drops file │ FastAPI backend │
|
||||
on /upload ───────▶│ /api/parse-837|835|999| │
|
||||
(or SFTP inbound) │ 277ca|ta1|270 │
|
||||
│ │
|
||||
│ parse → validate ──┐ │
|
||||
│ ▼ │
|
||||
│ reconcile ──▶ store ──▶ activity_events (un-chained)
|
||||
│ │ │
|
||||
│ │ └─▶ audit_log (SP11, chained)
|
||||
│ ▼ (login / rejected / key events)
|
||||
│ SQLite (encrypted optional)
|
||||
│ │
|
||||
│ ▼
|
||||
│ pubsub.publish(<kind>)
|
||||
└──────────┬───────────────────┘
|
||||
│ NDJSON stream
|
||||
▼
|
||||
┌──────────────────────────────┐
|
||||
│ React UI (Vite-served) │
|
||||
│ /claims /remits /inbox │
|
||||
│ /reconciliation /providers │
|
||||
│ /acks /activity /batches │
|
||||
│ /upload /dashboard /health │
|
||||
│ drill stack everywhere │
|
||||
└──────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Sub-project inventory
|
||||
|
||||
### 6.1 Shipped SPs (in `main`)
|
||||
|
||||
Each row's "Spec" / "Plan" point to the design spec and implementation plan on disk. SPs without a plan file on disk shipped via `feat(spN)` commits on `main` whose commit messages carry the implementation order; those commit logs substitute for a plan file. Closing the plan-file gap is on the round-3 backlog.
|
||||
|
||||
| SP | Title | Spec | Plan | Acceptance |
|
||||
|---|---|---|---|---|
|
||||
| SP1 | Production-readiness: in-memory store, GET endpoints, react-query, reference notes | [spec](superpowers/specs/2026-06-19-cyclone-production-readiness-design.md) | [plan](superpowers/plans/2026-06-19-cyclone-production-readiness.md) | Met |
|
||||
| SP2 | DB + reconciliation: SQLite, 7-state lifecycle, auto-match on 835 ingest, manual reconciliation UI | [spec](superpowers/specs/2026-06-19-cyclone-db-reconciliation-design.md) | [plan](superpowers/plans/2026-06-19-cyclone-db-reconciliation.md) | Met |
|
||||
| SP3 | EDI features: 837P rules (R034/R035), CAS reason codes, 999 ACK, TA1 ACK, 270/271 | [spec](superpowers/specs/2026-06-20-cyclone-edi-features-design.md) | [plan](superpowers/plans/2026-06-20-cyclone-edi-features.md) | Met |
|
||||
| SP4 | Claim drawer + Remit drawer + Cmd-K search + batch diff + CSV export | [spec](superpowers/specs/2026-06-20-cyclone-claim-drawer-design.md) | [plan](superpowers/plans/2026-06-20-cyclone-claim-drawer.md) | Met |
|
||||
| SP5 | Live tail: NDJSON pubsub on Claims/Remittances/Activity, heartbeat, reconnect | [spec](superpowers/specs/2026-06-20-cyclone-live-tail-design.md) | [plan](superpowers/plans/2026-06-20-cyclone-live-tail.md) | Met |
|
||||
| SP6 | Inbox workflow: 4 lanes (Rejected/Candidates/Unmatched/Done today), 4-field scoring, bulk actions | **No dedicated spec on disk** — SP6 ships via the SP4 spec's coverage and the workflow-automation plan | [plan](superpowers/plans/2026-06-20-cyclone-workflow-automation.md) | Met |
|
||||
| SP7 | Per-line adjustment audit: 837 SV1 ↔ 835 SVC strict match, line reconciliation tab | [spec](superpowers/specs/2026-06-20-cyclone-line-reconciliation-design.md) | [plan](superpowers/plans/2026-06-20-cyclone-line-reconciliation.md) | Met |
|
||||
| SP8 | Outbound 837P serializer: single-claim download + bundle resubmit ZIP | [spec](superpowers/specs/2026-06-20-cyclone-serialize-837-design.md) *(the spec's own title mistakenly says "Sub-project 7"; the file maps to SP8)* | [plan](superpowers/plans/2026-06-20-cyclone-serialize-837.md) | Met |
|
||||
| SP9 | Multi-payer, multi-NPI, SFTP stub | [spec](superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md) | (no plan file on disk; feat(sp9) commits on `main`) | Met |
|
||||
| SP10 | 277CA + Payer-Rejected lane | (no dedicated spec on disk; covered inline in [parse-decide spec](superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md) and `cyclone/parsers/parse_277ca.py`) | (no plan file on disk; feat(sp10) commits on `main`) | Met |
|
||||
| SP11 | Tamper-evident audit log | (no dedicated spec on disk; `cyclone/audit_log.py:116 append_event` and `:190 verify_chain` are the implementation) | (no plan file on disk; feat(sp11) commits on `main`) | Met |
|
||||
| SP12 | Encryption at rest (SQLCipher) | (no dedicated spec on disk; `cyclone/db_crypto.py` is the implementation) | (no plan file on disk; feat(sp12) commits on `main`) | Met |
|
||||
| SP13 | SFTP wire-up (paramiko) | (no dedicated spec on disk; extends SP9; `cyclone/clearhouse/__init__.py` holds `SftpClient`) | (no plan file on disk; feat(sp13) commits on `main`) | Met |
|
||||
| SP14 | 5-lane Inbox UI | (no dedicated spec on disk; extends SP6) | (no plan file on disk; feat(sp14) commits on `main`) | Met |
|
||||
| SP15 | SQLCipher key rotation | (no dedicated spec on disk; extends SP12) | (no plan file on disk; feat(sp15) commits on `main`) | Met |
|
||||
| SP16 | Live MFT polling scheduler | (no dedicated spec on disk; `cyclone/scheduler.py` is the implementation; idempotency via `processed_inbound_files` from migration 0011) | (no plan file on disk; feat(sp16) commits on `main`) | Met |
|
||||
| SP17 | Encrypted DB backups | [spec](superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md) | (no plan file on disk; feat(sp17) commits on `main`) | Met |
|
||||
| SP18 | Structured logging | [spec](superpowers/specs/2026-06-21-cyclone-structured-logging-design.md) | (no plan file on disk; feat(sp18) commits on `main`) | Met |
|
||||
| SP19 | Security hardening + health probe | [spec](superpowers/specs/2026-06-21-cyclone-security-hardening-design.md) | (no plan file on disk; feat(sp19) commits on `main`) | Met |
|
||||
| SP20 | NPI checksum + Tax ID format validation | [spec](superpowers/specs/2026-06-21-cyclone-npi-validation-design.md) | (no plan file on disk; feat(sp20) commits on `main`) | Met |
|
||||
| SP21 | CycloneStore split (in-flight) + Universal drilldown (shipped) | [split spec](superpowers/specs/2026-06-21-cyclone-store-split-design.md), [drilldown spec](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) | [store-split plan](superpowers/plans/2026-06-21-cyclone-store-split.md), [drilldown plan](superpowers/plans/2026-06-21-cyclone-universal-drilldown.md) | Drilldown **Met**; split **In-flight** (`refactor/store-split` branch, not yet on `main`) |
|
||||
| SP22 | Parse-decide workflow + Pipeline agent | [parse-decide spec](superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md), [pipeline spec](superpowers/specs/2026-06-21-cyclone-pipeline-agent-design.md) | [parse-decide plan](superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md), [pipeline plan](superpowers/plans/2026-06-21-cyclone-pipeline-agent.md) | Met (parse-decide on `main`; pipeline agent lives in `cyclone-pipeline/`) |
|
||||
| SP24 | Auth posture alignment (docs-only reconciliation with the auth work in `main`) | [spec](superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md) | [plan](superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md) | Met (no code paths change; only docs + `__main__.py` WARNING + 3 skills addenda) |
|
||||
|
||||
### 6.2 Backlog (specs awaiting plans, or vice versa)
|
||||
|
||||
| Item | Status | Why |
|
||||
|---|---|---|
|
||||
| SP23 (candidate) | Spec-only; **product fork awaiting user decision** | `2026-06-22-cyclone-ubuntu-docker-deployment-design.md` (auth + Docker + RBAC + LAN-bind); see §11 R-1, R-3 |
|
||||
| Plan-file backfill for SP9, SP10, SP11, SP12, SP13, SP14, SP15, SP16, SP17, SP18, SP19, SP20 | Plan files missing on disk; `feat(spN)` commit messages substitute | Round-3 doc-backlog item |
|
||||
| Spec-file backfill for SP6 (workflow/inbox), SP11 (audit log), SP12 (SQLCipher), SP13 (paramiko), SP15 (key rotation), SP16 (MFT scheduler) | Specs missing on disk | Round-3 doc-backlog item |
|
||||
| Migration manifest (checksums for the 12 SQL files) | Open | Completeness review §3.1.18 |
|
||||
| Tunable score weights (move 40/25/20/15 from code to config) | Open | Completeness review §3.1.15 |
|
||||
| License file (`LICENSE`) | Open | Completeness review §3.2.28 |
|
||||
| `pre-commit` / `Makefile` / `CONTRIBUTING.md` / `.editorconfig` / `ruff` config | Open | Completeness review §3.2.30 |
|
||||
| CI workflow file (`.github/workflows/`) | Open | §8.1 #3 references CI; no CI file exists today |
|
||||
| PHI fixtures flagged as PHI (`docs/prodfiles/PHI.md` or equivalent) | Open | Completeness review §3.2.23 |
|
||||
| Delete stale `backend/src/cyclone/workflow/__pycache__/` | Open | Empty dead-code directory (§5.1) |
|
||||
| EHR/PM integration (webhook in/out, callable `POST /api/parse-837`) | Open | Completeness review §5.1 |
|
||||
| COB / secondary-claim generator | Open | Completeness review §3.1.12 + §5.1 |
|
||||
| Real-time eligibility round-trip (SFTP pickup + SOAP + retry + 271 parse) | Open | Completeness review §3.1.8 + §5.1 |
|
||||
| 837I / 837D / 834 / 820 / 275 / 278 / 276/277 transaction types | Open | Completeness review §2.1 |
|
||||
| NPPES NPI registry lookup | Open | Completeness review §3.1.10 |
|
||||
| ICD-10 / HCPCS / NDC vocabulary tables | Open | Completeness review §3.1.11 |
|
||||
|
||||
### 6.3 Traceability matrix (compressed)
|
||||
|
||||
This matrix is the single index. Each row points to a requirement and where it's realized. A change to a requirement must update its row; a change to an SP must update the matching FRs/NFRs.
|
||||
|
||||
| ID | SP | Spec | Plan | Key tests |
|
||||
|---|---|---|---|---|
|
||||
| FR-1 | SP1 | production-readiness | production-readiness | `test_parse_837.py`, `test_prodfiles_smoke.py` |
|
||||
| FR-2 | SP1, SP3 | production-readiness + edi-features | production-readiness + edi-features | `test_parse_835.py`, `test_service_line_payments.py` |
|
||||
| FR-3 | SP2 | db-reconciliation | db-reconciliation | `test_db.py`, `test_api_parse_persists.py` |
|
||||
| FR-4 | SP2 | db-reconciliation | db-reconciliation | `test_db_migrate.py` |
|
||||
| FR-5 | SP2 | db-reconciliation | db-reconciliation | `test_reconcile.py`, `test_store_reconcile.py` |
|
||||
| FR-6 | SP2 | db-reconciliation | db-reconciliation | `test_models.py`, `test_reconcile.py` |
|
||||
| FR-7 | SP2 | db-reconciliation | db-reconciliation | `test_reconcile.py`, `test_reconcile_line_level.py`, `test_inbox_endpoints.py`, `test_api_gets.py` |
|
||||
| FR-8 | SP3, SP20 | edi-features + npi-validation | edi-features | `test_validator.py`, `test_validation_r200_r210.py` |
|
||||
| FR-9 | SP3 | edi-features | edi-features | `test_acks.py`, `test_api_parse_persists_ack.py` |
|
||||
| FR-10 | SP3 | edi-features | edi-features | `test_api_999.py`, `test_models_999.py`, `test_parse_999.py` |
|
||||
| FR-11 | SP3 | edi-features | edi-features | `test_api_ta1.py`, `test_parse_ta1.py` |
|
||||
| FR-12 | SP3 | edi-features | edi-features | `test_parse_270.py`, `test_parse_271.py`, `test_api_eligibility.py` |
|
||||
| FR-13 | SP10 | parse-decide-workflow (277CA section) | parse-decide-workflow | `test_parse_277ca.py`, `test_apply_277ca_rejections.py` |
|
||||
| FR-14 | SP6, SP10, SP14 | workflow-automation + parse-decide-workflow | workflow-automation | `test_inbox_lanes.py`, `test_inbox_endpoints.py`, `test_lane_filter_acknowledged.py` |
|
||||
| FR-15 | SP6 | workflow-automation | workflow-automation | `test_scoring.py` |
|
||||
| FR-16 | SP4, SP7, SP21 | claim-drawer + line-reconciliation + universal-drilldown | claim-drawer + line-reconciliation + universal-drilldown | `test_api_claim_detail.py`, `test_store_claim_detail.py` |
|
||||
| FR-17 | SP4 | claim-drawer | claim-drawer | `test_batch_diff.py` |
|
||||
| FR-18 | SP4 | claim-drawer | claim-drawer | `useSearch.test.ts`, `SearchResults.test.tsx` |
|
||||
| FR-19 | SP4 | claim-drawer | claim-drawer | `ExportCsvButton.test.tsx`, `csv.test.ts` |
|
||||
| FR-20 | SP5 | live-tail | live-tail | `test_api_stream_live.py`, `useTailStream.test.ts`, `tail-stream.test.ts` |
|
||||
| FR-21 | SP5 | live-tail | live-tail | `TailStatusPill.test.tsx`, `useMergedTail.test.ts` |
|
||||
| FR-22 | SP7 | line-reconciliation | line-reconciliation | `test_reconcile_line_level.py`, `test_api_line_reconciliation.py` |
|
||||
| FR-23 | SP8 | serialize-837 | serialize-837 | `test_serialize_837.py`, `test_api_serialize_837.py` |
|
||||
| FR-24 | SP8 | serialize-837 | serialize-837 | `test_api_batch_export_837.py` |
|
||||
| FR-25 | SP9 | multi-payer-npi-sftp | (none on disk) | `test_providers_seed.py`, `test_payer_config_loading.py` |
|
||||
| FR-26 | SP13 | multi-payer-npi-sftp (extends) | (none on disk) | `test_sftp_paramiko.py`, `test_sftp_stub.py` |
|
||||
| FR-27 | SP16 | (none on disk) | (none on disk) | `test_scheduler.py`, `test_api_scheduler.py` |
|
||||
| FR-28 | SP12 | (none on disk) | (none on disk) | `test_db_crypto.py`, `test_security.py` |
|
||||
| FR-29 | SP15 | (none on disk) | (none on disk) | `test_api_rotate_key.py` |
|
||||
| FR-30 | SP17 | encrypted-backup | (none on disk) | `test_backup_crypto.py`, `test_backup_service.py`, `test_backup_scheduler.py`, `test_api_backup.py`, `test_cli_backup.py` |
|
||||
| FR-31 | SP18 | structured-logging | (none on disk) | `test_logging_formatter.py`, `test_logging_scrubber.py`, `test_logging_setup.py` |
|
||||
| FR-32 | SP19 | security-hardening | (none on disk) | `test_security.py` |
|
||||
| FR-33 | SP19 | security-hardening | (none on disk) | `test_api.py:33 test_health_endpoint`, `test_security.py:204-221` (health snapshot) |
|
||||
| FR-34 | SP20 | npi-validation | (none on disk) | `test_npi.py`, `test_api_validate_provider.py`, `test_cli_validate.py` |
|
||||
| FR-35 | SP21 | store-split | store-split | `test_store.py` (currently covers the un-split `store.py`; will cover the split modules once `refactor/store-split` lands) |
|
||||
| FR-36 | SP22 | parse-decide-workflow | parse-decide-workflow | `test_api_parse_persists.py`, `test_prodfiles_smoke.py`; migration 0014 in `claims-unique-fix` worktree |
|
||||
| FR-37 | SP22 | pipeline-agent | pipeline-agent | (lives in `cyclone-pipeline/`, not this repo) |
|
||||
| FR-38 | SP21 | universal-drilldown | universal-drilldown | `DrillStackProvider.test.tsx`, `useDrawerUrlState.test.ts`, plus per-drawer tests in `ClaimDrawer/`, `RemitDrawer/`, `ProviderDrawer/`, `AckDrawer/` |
|
||||
|
||||
---
|
||||
|
||||
## 7. Acceptance criteria — per-SP (compressed)
|
||||
|
||||
These are the existing per-SP acceptance checklists as captured in their design specs, condensed into one table. A new engineer can use this to know whether a given SP is "really done" without reading 22 specs end-to-end.
|
||||
|
||||
| SP | Acceptance criterion | Verified by |
|
||||
|---|---|---|
|
||||
| SP1 | `pytest` passes; `/api/claims`, `/api/remittances`, `/api/providers`, `/api/activity`, `/api/batches`, `/api/batches/{id}` return JSON; CORS allowlist stays `http://localhost:5173`; bind stays `127.0.0.1` | `test_api_gets.py` + manual smoke |
|
||||
| SP2 | DB at `~/.local/share/cyclone/cyclone.db`; migrations apply on first run; 7-state Claim lifecycle; reconciliation runs on 835 ingest; `/reconciliation` page works | `test_db*.py`, `test_reconcile.py`, `test_reconcile_line_level.py`, `test_inbox_endpoints.py` |
|
||||
| SP3 | R034/R035/R200/R210 enforced; CAS reason codes surfaced; 999 in + out; TA1 in; 270/271 builder + parser | `test_validator*.py`, `test_acks.py`, `test_api_999.py`, `test_api_ta1.py`, `test_parse_270.py`, `test_parse_271.py` |
|
||||
| SP4 | Drawer opens with URL sync; Cmd-K search returns results across resources; batch diff renders added/removed/changed; CSV exports stream | `ClaimDrawer*.test.tsx`, `useSearch.test.ts`, `BatchDiff.test.tsx`, `ExportCsvButton.test.tsx` |
|
||||
| SP5 | `/api/{claims,remittances,activity}/stream` returns NDJSON with snapshot + `snapshot_end` + heartbeat; client flips to `stalled` after 30s silence; reconnect ladder capped at 30s | `test_api_stream_live.py`, `TailStatusPill.test.tsx` |
|
||||
| SP6 | `/inbox` shows 5 lanes (4 in SP6; 5th added in SP10); scoring tiers visible; bulk actions call the right endpoint | `test_inbox_lanes.py`, `BulkBar.test.tsx` |
|
||||
| SP7 | Line reconciliation tab in ClaimDrawer matches 837 SV1 ↔ 835 SVC strictly; unmatched-line warnings surfaced | `test_reconcile_line_level.py`, `test_api_line_reconciliation.py` |
|
||||
| SP8 | 113 prodfiles parse → serialize → parse back to the same `claim_id`; ZIP bundle export works | `test_prodfiles_smoke.py`, `test_api_batch_export_837.py` |
|
||||
| SP9 | `config/payers.yaml` loads (via `pyyaml`); new providers/payers/payer_configs tables populate; `clearhouse.submit` writes to `staging_dir` (stub) | `test_providers_seed.py`, `test_payer_config_loading.py` |
|
||||
| SP10 | 277CA parse + persist; `payer_rejected` stamp applied monotonically; lane 2 in `/inbox` renders | `test_parse_277ca.py`, `test_apply_277ca_rejections.py` |
|
||||
| SP11 | `audit_log` hash chain verifies via `GET /api/admin/audit-log/verify`; any break is detected | `test_audit_log.py`, `test_api_audit_log.py` |
|
||||
| SP12 | `db_crypto.is_encryption_enabled()` returns True when Keychain entry exists and `sqlcipher3` installed; otherwise falls back with WARNING | `test_db_crypto.py` |
|
||||
| SP13 | `SftpClient` actually pushes to `mft.gainwelltechnologies.com:…`; credentials read from Keychain via `keyring` at call time | `test_sftp_paramiko.py` (unit); integration via `cyclone-pipeline` |
|
||||
| SP14 | All 5 lanes render in `/inbox`; bulk acknowledge action drops payer-rejected claims without erasing the original rejection | `test_inbox_endpoints_sp7.py`, `test_lane_filter_acknowledged.py` |
|
||||
| SP15 | `POST /api/admin/db/rotate-key` runs `PRAGMA rekey`; emits `db.key_rotated` with old + new fingerprints; concurrent calls serialized | `test_api_rotate_key.py` |
|
||||
| SP16 | Scheduler ticks every interval; per-file try/except keeps the loop alive; `processed_inbound_files` de-duplicates | `test_scheduler.py`, `test_api_scheduler.py` |
|
||||
| SP17 | Backup scheduler ticks every 24h; encrypted `.bin` + `.meta.json` written; restore via 2-step token works; retention prunes at 30 days | `test_backup_*.py` |
|
||||
| SP18 | All logs emit as NDJSON by default; `PiiScrubber` redacts NPIs/SSNs/DOBs/names; `CYCLONE_LOG_JSON=0` falls back to `CycloneDevFormatter` | `test_logging_*.py` |
|
||||
| SP19 | 413 on oversize body, 429 on rate-limit, security headers present; `api.request_rejected` audit event on 413/429; `GET /api/health` returns rich payload | `test_security.py`, `test_api.py:33` |
|
||||
| SP20 | NPI Luhn catches 99% of typos; EIN rejects reserved prefixes; warning-only (placeholder NPIs in fixtures don't block ingest); CLI + API surface | `test_npi.py`, `test_api_validate_provider.py`, `test_cli_validate.py` |
|
||||
| SP21 | `CycloneStore` is split into `store_persistence / store_reconcile / store_queries / store_mappers`; public store API unchanged; `DrillStackProvider` + `DrillDrawerHeader` shell used by every drillable cell | `test_store.py` (split modules), `DrillStackProvider.test.tsx`, drawer-key tests |
|
||||
| SP22 | Re-parsing an 837 into a fresh batch does not 409 on `claim_id` collision; parse-then-decide workflow checks `find_existing_batch_for_claim`; pipeline agent runs 7 phases end-to-end | `test_api_parse_persists.py`, `test_prodfiles_smoke.py`, pipeline integration |
|
||||
|
||||
---
|
||||
|
||||
## 8. Definition of Done
|
||||
|
||||
### 8.1 Project-level DoD (the project as a whole is "done" when…)
|
||||
|
||||
1. **All FRs in §3 are implemented and tested** — verifiable by walking the §6.3 traceability matrix and confirming each row links to either a passing test or an §2.2 out-of-scope entry.
|
||||
2. **All NFRs in §4 are met or have a documented exception** — NFR-1 through NFR-18 are each either met (with a citation) or carry an explicit exception in this section.
|
||||
3. **All shipped SPs in §6.1 have an Acceptance row marked "Met" and the corresponding tests pass.** Tests are run via `pytest` (backend) and `npm test` (frontend) per §9.4; **the project has no CI workflow today** (round-3 backlog item — see §6.2). A release-time gate should run the suites and fail if any test is red.
|
||||
4. **All backlog items in §6.2 are either scheduled, explicitly deferred with rationale, or removed** — testable as a checklist walk.
|
||||
5. **All assumptions in §10 are still valid** — re-checked at release time; an invalid assumption is either promoted to a backlog item or rolled into the next release.
|
||||
6. **All four reviewer questions (components / data model / dependencies / DoD) have independent agreement from at least two reviewers** — the meta-DoD: a doc with this scope is "done" when two reviewers can independently describe the system from it and agree.
|
||||
|
||||
### 8.2 Per-release DoD (the next release is "done" when…)
|
||||
|
||||
1. The new SP's design spec has been **self-reviewed** (every claim cited with line/file, every assumption recorded).
|
||||
2. The implementation plan has **TDD-shaped tasks** with per-task acceptance criteria.
|
||||
3. Every task's tests are green in `pytest` (backend) and `npm test` (frontend) and `npm run build` (frontend typecheck + build).
|
||||
4. The new SP's Acceptance row in §6.1 is updated to "Met".
|
||||
5. New audit events (if any) are listed in the audit-log test fixtures (`backend/tests/test_audit_log.py`).
|
||||
6. Migration files (if any) follow the `NNNN_*.sql` ordering, append to this doc's §5.2 table, and (post-round-3) update the migration manifest (R-13).
|
||||
7. The README's project-layout section is updated if any module is added/renamed/removed.
|
||||
|
||||
### 8.3 Per-task DoD (the smallest unit of work is "done" when…)
|
||||
|
||||
1. The code change is in a single commit (or a small, reviewable series).
|
||||
2. The matching test exists and passes.
|
||||
3. No existing test regresses.
|
||||
4. The commit message names the SP and the FR/NFR it advances (e.g., `feat(sp19): … (FR-32, NFR-8)`).
|
||||
5. If the task introduces a migration, it's added to `backend/src/cyclone/migrations/` with the next sequential `NNNN` and the new schema is reflected in §5.2.
|
||||
|
||||
---
|
||||
|
||||
## 9. Test strategy
|
||||
|
||||
### 9.1 Taxonomy
|
||||
|
||||
| Layer | Tool | Where | What it covers |
|
||||
|---|---|---|---|
|
||||
| Backend unit | `pytest` | `backend/tests/` | Pure-function logic: `cyclone.reconcile.*`, `cyclone.scoring.*`, `cyclone.npi.*`, `cyclone.cas_codes`, `cyclone.audit_log.append_event`, `cyclone.edi.filenames` |
|
||||
| Backend integration | `pytest` (TestClient) | `backend/tests/test_api_*.py` | FastAPI route round-trips, content negotiation, NDJSON streaming, auth/rate-limit/SP19, 999/TA1/277CA parse endpoints, scheduler endpoints |
|
||||
| Backend fixture-driven | `pytest` (`tests/fixtures/*.txt`) | `backend/tests/test_prodfiles_smoke.py` | 113 + 19 + 1369 production files; round-trip property |
|
||||
| Backend SQL | `pytest` (in-memory SQLite) | `backend/tests/test_db*.py` | Schema, migrations, ORM, encryption round-trip |
|
||||
| Frontend unit | `vitest` (`*.test.ts(x)`) | `src/lib/`, `src/hooks/`, `src/components/ui/` | Pure functions, hooks, primitives |
|
||||
| Frontend component | `vitest` + Testing Library | `src/components/`, `src/pages/` | Drawers, lane rendering, drill stack, export buttons |
|
||||
| Smoke | manual / shell | `scripts/smoke.sh` (planned in SP23) | End-to-end parse → reconcile → export flow |
|
||||
|
||||
### 9.2 Fixture sources
|
||||
|
||||
- `backend/tests/fixtures/minimal_*.txt` — minimal valid 270 / 271 / 277CA / 835 / 837P / 999 / TA1 / co_medicaid / unbalanced_835 — written by hand, used in unit tests.
|
||||
- `docs/prodfiles/claims/*.x12` (113 files) — real-looking production files used by `test_prodfiles_smoke.py` for the round-trip property.
|
||||
- `docs/prodfiles/FromHPE/` (1369 files) — full inbound directory from the production SFTP path.
|
||||
- `docs/prodfiles/837p-from-axiscare/` (19 files) — production 837Ps from AxisCare.
|
||||
- `docs/prodfiles/835fromco/` (5 files) — production 835s from CO Medicaid.
|
||||
|
||||
⚠ **PHI warning:** prodfiles contain real-looking patient data. Even in a single-operator tool, fixtures should be flagged as PHI (completeness review §3.2.23 — flagged but not yet actioned; for v1 they're kept under `docs/prodfiles/` with no additional access control).
|
||||
|
||||
### 9.3 Coverage goals
|
||||
|
||||
- **Pure-function modules** (`reconcile`, `scoring`, `npi`, `cas_codes`, `parsers`, `edi.filenames`, `batch_diff`): ≥ 90% line coverage. `pytest-cov` is installed; the gate (`--cov-fail-under=90`) is **not enforced today** and is a round-3 backlog item.
|
||||
- **Store facade** (`store`, and the future `store_persistence / store_reconcile / store_queries / store_mappers` modules): every public method has at least one happy-path and one failure-path test.
|
||||
- **API routes**: every route has a happy-path test; error paths (404, 409, 413, 415, 422, 423, 429, 500) covered by at least one test each.
|
||||
- **Round-trip property** (SP8): 100% of `docs/prodfiles/claims/*.x12` must parse → serialize → parse to the same `claim_id`.
|
||||
- **Live tail** (SP5): one happy snapshot → live item → disconnect cleanup test per endpoint.
|
||||
|
||||
### 9.4 How to run
|
||||
|
||||
```bash
|
||||
# Backend
|
||||
cd backend
|
||||
.venv/bin/pytest # 964 tests collected (2026-06-23)
|
||||
.venv/bin/python -m cyclone serve
|
||||
|
||||
# Frontend
|
||||
npm run typecheck
|
||||
npm test # 73 test files
|
||||
npm run build
|
||||
|
||||
# Smoke (planned in SP23)
|
||||
bash scripts/smoke.sh
|
||||
```
|
||||
|
||||
### 9.5 What is NOT covered (explicit)
|
||||
|
||||
- **Component / E2E browser tests (Playwright).** Deferred; the local-only threat model doesn't justify them. (Completeness review §3.2 + SP23 §14.)
|
||||
- **Property-based tests** (Hypothesis) for the parser. Could add; not scheduled.
|
||||
- **Mutation tests** (mutmut / cosmic-ray). Not scheduled.
|
||||
- **Performance / load tests.** Single-operator scale; not scheduled.
|
||||
- **NPPES round-trip validation** (only Luhn checksum is covered; SP20).
|
||||
- **External-integration tests against the real Gainwell MFT.** The pipeline agent covers this in `cyclone-pipeline/`; out of scope for this repo.
|
||||
- **CI workflow.** Not present in the repo; release gating is manual today.
|
||||
|
||||
---
|
||||
|
||||
## 10. Assumptions
|
||||
|
||||
Recorded explicitly so a new engineer can challenge any of them.
|
||||
|
||||
| # | Assumption | If wrong, what changes |
|
||||
|---|---|---|
|
||||
| A1 | One operator, one host, one trading partner (CO Medicaid). | Multi-tenant / multi-payer scope would require SP23 + multi-payer NPI expansion + COB + AS2/AS4. |
|
||||
| A2 | The host runs macOS (Keychain via `keyring`) OR Ubuntu (Docker secrets available, SP23). | Linux-without-Docker / Windows would require a different secrets backend (`keyring` already dispatches via Secret Service / kwallet on Linux; Windows would use Credential Manager). |
|
||||
| A3 | The host is physically secure; LUKS / FileVault is the operator's concern, not Cyclone's. | If not, the SQLCipher-encrypted-DB posture is insufficient; per-file encryption of `prodfiles/` and `sftp_staging/` becomes mandatory. |
|
||||
| A4 | The operator can hand the 837 ZIP to the Gainwell MFT UI manually. | Removes the need for a built-in SFTP submit (SP13); the SFTP wire stays stub. |
|
||||
| A5 | The 835 lands the following Monday (CO Medicaid payment cycle) and is verified by a separate `check-835` subcommand (in `cyclone-pipeline/`), not inline. | Would require a different round-trip contract; affects SP22 pipeline agent. |
|
||||
| A6 | ICD-10 / HCPCS / NDC vocabulary is not Cyclone's responsibility. | Would require vendoring vocab CSVs and adding vocabulary checks (FR + tests). |
|
||||
| A7 | NPPES NPI registry lookup is not needed (offline Luhn is enough). | Would require scheduled NPPES download + online check + new FR. |
|
||||
| A8 | The X12 5010 transaction set Cyclone supports is sufficient for the operator's billing workflow. | Adding 837I / 837D / 278 / 276-277 / 277CA-round-trip / etc. is each a new SP. |
|
||||
| A9 | Local-only bind is sufficient — no remote access needed. | Would require auth (SP23), TLS, reverse proxy, RBAC. |
|
||||
| A10 | One Python process, no message broker, no separate worker. | A multi-worker deployment would require breaking thread-affinity (SP12/SP15 specifically). |
|
||||
| A11 | 964 backend tests + 73 frontend test files is the right test density for this codebase (2026-06-23). If we add more transaction types or surface area, this needs to grow proportionally; coverage gates should be enforced (round-3 backlog). | New transaction types or surface area without test growth → coverage drifts down. |
|
||||
| A12 | `cyclone-pipeline/` is a sibling project, not a sub-project of this repo. | The 7-phase round-trip agent is shipped in `cyclone-pipeline/`; if that project is abandoned, the manual operator workflow remains (upload via `/upload`, SFTP by hand, `check-835` not available). |
|
||||
|
||||
---
|
||||
|
||||
## 11. Risks and known gaps
|
||||
|
||||
Pulled forward from [`docs/reviews/2026-06-20-cyclone-completeness-review.md`](reviews/2026-06-20-cyclone-completeness-review.md). Status as of 2026-06-23.
|
||||
|
||||
| ID | Risk / gap | Source | Status |
|
||||
|---|---|---|---|
|
||||
| R-1 | No app-layer auth (anyone on the host can read PHI). | Completeness review §3.1.4 + §3.2.25 | **Closed by SP24** — auth shipped via the merge of `origin/main` on 2026-06-23 (commits `a25504b`..`39ae988`, `cyclone.auth.*` + `matrix_gate` dependency on every router); SP24 reconciled the docs to match. SP23 still covers the LAN-bind / Docker / RBAC product fork. |
|
||||
| R-2 | SQLite plaintext if SQLCipher not enabled. | Completeness review §3.1.1 | **Partial** — SP12 + SP15 ship the encryption option, but it falls back to plain if the Keychain entry is missing. |
|
||||
| R-3 | No Dockerfile / docker-compose.yml. | Completeness review §3.2.29 | **Open** — SP23 spec covers it but is awaiting user decision. |
|
||||
| R-4 | PayerConfig still partly in code (`co_medicaid()` factory at `cyclone/parsers/payer.py:57-58`). | Completeness review §3.1.9 | **Open** — SP9 externalized config but the factory remains as a fallback for tests and default-payer selection. |
|
||||
| R-5 | No vocabulary tables (ICD-10 / HCPCS / NDC). | Completeness review §3.1.11 | **Open** — explicitly out of scope per A6. |
|
||||
| R-6 | No COB / secondary-claim generator. | Completeness review §3.1.12 | **Open** — explicitly out of scope per §2.2. |
|
||||
| R-7 | No real-time eligibility round-trip. | Completeness review §3.1.8 | **Open** — 270/271 are file-exchange only per SP3. |
|
||||
| R-8 | No NPPES NPI lookup. | Completeness review §3.1.10 | **Open** — only Luhn checksum (SP20). |
|
||||
| R-9 | No 837I / 837D / 834 / 820 / 275 / 278 / 276-277 transaction types. | Completeness review §2.1 | **Open** — explicitly out of scope per §2.2. |
|
||||
| R-10 | Score weights hardcoded (`cyclone/scoring.py:42-46`). | Completeness review §3.1.15 | **Open** — not scheduled. |
|
||||
| R-11 | `cyclone/store.py` is a 2,172-line god-module. | Completeness review §3.1.17 | **In-flight** — SP21 spec + plan written; branch `refactor/store-split` exists; merge to `main` pending. |
|
||||
| R-12 | `cyclone/api.py` is a 3,145-line god-module (larger than `store.py`). | Completeness review §3.1.19 | **Partial** — `api_routers/` exists for 4 sub-routes (acks, admin, health, ta1_acks); bulk of routes still in `api.py`. |
|
||||
| R-13 | Migrations in flat dir without manifest. | Completeness review §3.1.18 | **Open** — no checksum list; 12 SQL files in `backend/src/cyclone/migrations/`. |
|
||||
| R-14 | No `pre-commit` / `Makefile` / `CONTRIBUTING.md` / `.editorconfig` / `ruff` config. | Completeness review §3.2.30 | **Open** — single-operator project. |
|
||||
| R-15 | No license file (`LICENSE`). | Completeness review §3.2.28 | **Open** — not scheduled. |
|
||||
| R-16 | PHI fixtures not flagged as PHI. | Completeness review §3.2.23 | **Open** — not scheduled. |
|
||||
| R-17 | `/api/health` not rich (before SP19). | Completeness review §3.2.24 | **Closed by SP19** — `api_routers/health.py:28-40` returns the rich snapshot via `get_health_snapshot()`. |
|
||||
| R-18 | No CSP / no security headers (before SP19). | Completeness review §3.2.25 | **Closed by SP19** — `SecurityHeadersMiddleware` in `cyclone/security.py`. |
|
||||
| R-19 | No request body size / rate limit (before SP19). | Completeness review §3.1.4 | **Closed by SP19** — `BodySizeLimitMiddleware` + `RateLimitMiddleware` in `cyclone/security.py`. |
|
||||
| R-20 | No 277CA support (before SP10). | Completeness review §3.1.7 | **Closed by SP10**. |
|
||||
| R-21 | No structured logging (before SP18). | Completeness review §3.1.5 | **Closed by SP18**. |
|
||||
| R-22 | No backup automation (before SP17). | Completeness review §3.1.3 | **Closed by SP17**. |
|
||||
| R-23 | `ActivityEvent` not tamper-evident (before SP11). | Completeness review §3.1.2 | **Closed by SP11** — but note: the tampered table is `audit_log`, distinct from `activity_events` (see NFR-3 / NFR-4). |
|
||||
| R-24 | SP23 product fork (auth + Docker + RBAC + LAN-bind). | §6.2 | **Awaiting user decision** — see §2.2 and §11 R-1. |
|
||||
| R-25 | `vitest@^4.1.9` is unusually new (current stable line is 1.x/2.x). | §5.3 | **Open** — verify the version pin is intentional before the next `npm install`; fallback to `^2.x` if accidental. |
|
||||
| R-26 | No CI workflow (`.github/workflows/`). | §8.1 #3 | **Open** — release gating is manual today. |
|
||||
| R-27 | `backend/src/cyclone/workflow/__pycache__/` is dead bytecode. | §5.1 | **Open** — recommend deletion. |
|
||||
|
||||
---
|
||||
|
||||
## 12. References
|
||||
|
||||
### 12.1 Design specs
|
||||
|
||||
All under [`docs/superpowers/specs/`](superpowers/specs/):
|
||||
|
||||
- [2026-06-19-cyclone-837p-parser-design.md](superpowers/specs/2026-06-19-cyclone-837p-parser-design.md) — pre-SP1, superseded by `production-readiness`
|
||||
- [2026-06-19-cyclone-db-reconciliation-design.md](superpowers/specs/2026-06-19-cyclone-db-reconciliation-design.md) — SP2
|
||||
- [2026-06-19-cyclone-production-readiness-design.md](superpowers/specs/2026-06-19-cyclone-production-readiness-design.md) — SP1
|
||||
- [2026-06-20-cyclone-claim-drawer-design.md](superpowers/specs/2026-06-20-cyclone-claim-drawer-design.md) — SP4 (no inbox section; SP6 ships from the workflow-automation plan)
|
||||
- [2026-06-20-cyclone-edi-features-design.md](superpowers/specs/2026-06-20-cyclone-edi-features-design.md) — SP3
|
||||
- [2026-06-20-cyclone-line-reconciliation-design.md](superpowers/specs/2026-06-20-cyclone-line-reconciliation-design.md) — SP7
|
||||
- [2026-06-20-cyclone-live-tail-design.md](superpowers/specs/2026-06-20-cyclone-live-tail-design.md) — SP5
|
||||
- [2026-06-20-cyclone-multi-payer-npi-sftp-design.md](superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md) — SP9 (extended by SP13)
|
||||
- [2026-06-20-cyclone-serialize-837-design.md](superpowers/specs/2026-06-20-cyclone-serialize-837-design.md) — SP8 (spec's own title mistakenly says "Sub-project 7")
|
||||
- [2026-06-21-cyclone-claims-unique-constraint-and-409-ux-design.md](superpowers/specs/2026-06-21-cyclone-claims-unique-constraint-and-409-ux-design.md) — pre-SP22, superseded by `parse-decide-workflow`
|
||||
- [2026-06-21-cyclone-encrypted-backup-design.md](superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md) — SP17
|
||||
- [2026-06-21-cyclone-npi-validation-design.md](superpowers/specs/2026-06-21-cyclone-npi-validation-design.md) — SP20
|
||||
- [2026-06-21-cyclone-parse-decide-workflow-design.md](superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md) — SP22 (parse-decide) + covers 277CA (SP10)
|
||||
- [2026-06-21-cyclone-pipeline-agent-design.md](superpowers/specs/2026-06-21-cyclone-pipeline-agent-design.md) — SP22 (pipeline agent)
|
||||
- [2026-06-21-cyclone-security-hardening-design.md](superpowers/specs/2026-06-21-cyclone-security-hardening-design.md) — SP19
|
||||
- [2026-06-21-cyclone-skill-catalog-design.md](superpowers/specs/2026-06-21-cyclone-skill-catalog-design.md) — documentation meta-project (not mapped to a behavioral SP)
|
||||
- [2026-06-21-cyclone-store-split-design.md](superpowers/specs/2026-06-21-cyclone-store-split-design.md) — SP21 (split)
|
||||
- [2026-06-21-cyclone-structured-logging-design.md](superpowers/specs/2026-06-21-cyclone-structured-logging-design.md) — SP18
|
||||
- [2026-06-21-cyclone-universal-drilldown-design.md](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) — SP21 (drilldown)
|
||||
- [2026-06-22-cyclone-ubuntu-docker-deployment-design.md](superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md) — **SP23 candidate; awaiting plan + user product-fork decision**
|
||||
|
||||
### 12.2 Implementation plans
|
||||
|
||||
All under [`docs/superpowers/plans/`](superpowers/plans/):
|
||||
|
||||
- [2026-06-19-cyclone-837p-parser.md](superpowers/plans/2026-06-19-cyclone-837p-parser.md) — pre-SP1, superseded
|
||||
- [2026-06-19-cyclone-db-reconciliation.md](superpowers/plans/2026-06-19-cyclone-db-reconciliation.md) — SP2
|
||||
- [2026-06-19-cyclone-production-readiness.md](superpowers/plans/2026-06-19-cyclone-production-readiness.md) — SP1
|
||||
- [2026-06-20-cyclone-claim-drawer.md](superpowers/plans/2026-06-20-cyclone-claim-drawer.md) — SP4
|
||||
- [2026-06-20-cyclone-edi-features.md](superpowers/plans/2026-06-20-cyclone-edi-features.md) — SP3
|
||||
- [2026-06-20-cyclone-line-reconciliation.md](superpowers/plans/2026-06-20-cyclone-line-reconciliation.md) — SP7
|
||||
- [2026-06-20-cyclone-live-tail.md](superpowers/plans/2026-06-20-cyclone-live-tail.md) — SP5
|
||||
- [2026-06-20-cyclone-serialize-837.md](superpowers/plans/2026-06-20-cyclone-serialize-837.md) — SP8
|
||||
- [2026-06-20-cyclone-workflow-automation.md](superpowers/plans/2026-06-20-cyclone-workflow-automation.md) — SP6
|
||||
- [2026-06-21-cyclone-claims-unique-constraint-and-409-ux.md](superpowers/plans/2026-06-21-cyclone-claims-unique-constraint-and-409-ux.md) — pre-SP22, superseded
|
||||
- [2026-06-21-cyclone-parse-decide-workflow.md](superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md) — SP22
|
||||
- [2026-06-21-cyclone-pipeline-agent.md](superpowers/plans/2026-06-21-cyclone-pipeline-agent.md) — SP22
|
||||
- [2026-06-21-cyclone-skill-catalog.md](superpowers/plans/2026-06-21-cyclone-skill-catalog.md) — documentation meta-project
|
||||
- [2026-06-21-cyclone-store-split.md](superpowers/plans/2026-06-21-cyclone-store-split.md) — SP21 (split, in-flight)
|
||||
- [2026-06-21-cyclone-universal-drilldown.md](superpowers/plans/2026-06-21-cyclone-universal-drilldown.md) — SP21 (drilldown)
|
||||
|
||||
### 12.3 Reviews and reference
|
||||
|
||||
- [2026-06-20-cyclone-completeness-review.md](reviews/2026-06-20-cyclone-completeness-review.md) — industry-scope gap analysis.
|
||||
- [`docs/reference/835.md`](reference/835.md), [`docs/reference/837p.md`](reference/837p.md), [`docs/reference/co-medicaid.md`](reference/co-medicaid.md), [`docs/reference/x12naming.md`](reference/x12naming.md) — condensed X12 + CO Medicaid notes.
|
||||
|
||||
### 12.4 External
|
||||
|
||||
- HIPAA Security Rule §164.312 — audit controls, encryption, access control.
|
||||
- X12 005010X222A1 (837P), 005010X221A1 (835), 005010X231A1 (999), 005010X279A1 (270/271), 005010X214 (277CA), 005010X230 (TA1).
|
||||
- CAQH CORE Phase II/III (real-time eligibility — out of scope per §2.2).
|
||||
- CMS NPI Luhn specification.
|
||||
- NPPES API (NPI registry — out of scope per §2.2 + A7).
|
||||
- OWASP password-storage cheat sheet (argon2id parameters — SP23).
|
||||
- paramiko, FastAPI, SQLAlchemy 2.0, Pydantic v2, structlog, TanStack Query v5 docs.
|
||||
|
||||
---
|
||||
|
||||
## 13. Reviewer checklist (for round 2 → round 3 closure)
|
||||
|
||||
For the two independent reviewers at the end of round 2 (re-verifying the corrections):
|
||||
|
||||
1. **Components** — Do §5.1 (backend modules) + §5.1 (frontend dirs) list every existing module that ships PHI or serves a state-changing route? Are the boundaries (DB / ORM / reconcile / store / api / pubsub / clearhouse / batch_diff / edi.filenames) drawn correctly? Can you, from §5.1 alone, draw the import graph?
|
||||
2. **Data model** — Are all 12 migrations listed in §5.2 with their purpose? Are the two worktree-only migrations (0013, 0014) flagged as in-flight? Are all 18 ORM models in §5.2? Is the `db_backups` (not `backups`) correction applied? Is the `Rejection` phantom removed? Is the `ActivityEvent` vs `AuditLog` distinction clear (NFR-3 vs NFR-4)?
|
||||
3. **Dependencies** — Are §5.3 runtime + tooling deps correct (no missing `pyyaml`, no missing `keyring`, no `httpx` listed as runtime, `paramiko` flagged as `[sftp]` extra, `sqlcipher3` flagged as `[sqlcipher]` extra, frontend test env is `happy-dom` not `jsdom`, vitest pinned at `^4.1.9` flagged as unusual)? Are OS-level deps (Keychain via `keyring`, SQLCipher C library, `tini`, `nginx` if SP23 ships) called out?
|
||||
4. **Definition of Done** — Is §8 testable? Project DoD #6 now points at the reviewer-agreement meta-criterion. Per-release + per-task DoDs each independently checkable? FR-7 / FR-33 traceability rows corrected to point at existing test files and the correct endpoint name?
|
||||
5. **Traceability** — Pick any FR-NN in §3. Can you trace it to an SP, a spec on disk (or an honest "no dedicated spec on disk" note), a plan on disk (or an honest "no plan file on disk; feat(spN) commits substitute" note), and at least one test file in `backend/tests/` or `src/`? If not, that's a remaining gap.
|
||||
6. **Assumptions** — Are §10 assumptions falsifiable? Could a new engineer challenge any of them and find the test that proves/disproves them? Is A2's reference to `keyring` accurate now?
|
||||
7. **Risks** — Are §11 risks current (R-11 corrected to 2,172; R-12 added; R-25/R-26/R-27 added)? Compare against `docs/reviews/2026-06-20-cyclone-completeness-review.md`. Is R-24 (SP23 fork) clearly flagged as awaiting user decision?
|
||||
|
||||
If reviewers materially agree on all seven, round 2 closes; this doc becomes the project's new top-level entry point (linked from `README.md`).
|
||||
|
||||
---
|
||||
|
||||
*End of round 2. Next: re-verify with reviewers (round 3) or escalate any remaining disagreement to the user.*
|
||||
@@ -0,0 +1,70 @@
|
||||
# Auth Posture Alignment Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||
> superpowers:subagent-driven-development (recommended) or
|
||||
> superpowers:executing-plans to implement this plan task-by-task. Steps
|
||||
> use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Reconcile `CLAUDE.md`, `docs/REQUIREMENTS.md`, and `docs/ARCHITECTURE.md` with the auth work that landed in `main` on 2026-06-23, and add a loud `AUTH_DISABLED` startup warning so a misconfigured production deploy fails loudly.
|
||||
|
||||
**Architecture:** Docs-only + one 6-line `__main__.py` edit + three skills addenda. No code paths change; the auth code already in `main` is the source of truth.
|
||||
|
||||
**Tech Stack:** Markdown, Python `logging`, existing `cyclone.auth.deps.AUTH_DISABLED` flag.
|
||||
|
||||
**Spec:** [`docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md`](../specs/2026-06-23-cyclone-auth-posture-alignment-design.md)
|
||||
|
||||
---
|
||||
|
||||
## File structure
|
||||
|
||||
Modified files:
|
||||
|
||||
- `CLAUDE.md` (3 lines: 7, 97, 182)
|
||||
- `docs/REQUIREMENTS.md` (3 NFR / scope lines + §11 R-1 row + §6.1 SP24 row)
|
||||
- `docs/ARCHITECTURE.md` (2 lines: 16, 703 + 1 new "Auth" subsection + migrations table note)
|
||||
- `backend/src/cyclone/__main__.py` (startup warning when `AUTH_DISABLED`)
|
||||
- `.superpowers/skills/cyclone-tests/SKILL.md` (addendum)
|
||||
- `.superpowers/skills/cyclone-api-router/SKILL.md` (addendum)
|
||||
- `.superpowers/skills/cyclone-spec/SKILL.md` (addendum)
|
||||
|
||||
## Task 0: Mark spec in implementation
|
||||
|
||||
- [ ] Spec header `Status: Draft, awaiting user sign-off` → `Status: Approved (in implementation on sp24-doc-posture-alignment)`.
|
||||
|
||||
## Task 1: `CLAUDE.md`
|
||||
|
||||
- [ ] Line 7 — replace "Local-only by design: binds to `127.0.0.1`, no auth, no internet exposure." with the v1 posture: "Local-only by design: binds to `127.0.0.1`, requires login (auth boundary is HTTP; file-system protection unchanged), no internet exposure."
|
||||
- [ ] Line 97 — update SP numbering to reflect "SP numbers are used through **SP22**; **SP23** is the Ubuntu+Docker fork (awaiting user decision); this increment is **SP24**."
|
||||
- [ ] Line 182 — replace the "Local-only by design. The backend binds to `127.0.0.1`, has no auth…" bullet with the new posture (login required; auth boundary is HTTP; SQLCipher + Keychain unchanged; don't add internet exposure).
|
||||
|
||||
## Task 2: `docs/REQUIREMENTS.md`
|
||||
|
||||
- [ ] Line 36 — replace "Local-only on purpose: binds `127.0.0.1`, no auth, no internet exposure, single operator, single machine, one trading partner (Colorado Medicaid, currently)." with the v1 posture.
|
||||
- [ ] Line 39 — replace "Threat model: a process on the same host. No second party to authenticate; no second host to harden against." with the new threat model (process on same host + HTTP-layer login; SQLCipher + Keychain handle file-system threats; SP23 changes the model to LAN).
|
||||
- [ ] Line 154 (NFR-1) — replace "no auth, no API key" with "login required (bcrypt + HttpOnly session cookie; first admin bootstrapped from env vars `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD`); dev/test escape hatch via `CYCLONE_AUTH_DISABLED=1`."
|
||||
- [ ] §11 R-1 row — change "**Open** — addressed only if/when SP23 ships." to "**Closed by SP24** — auth shipped via the merge of `origin/main` on 2026-06-23 (commits `a25504b`..`39ae988`); SP24 reconciled the docs to match. SP23 still covers the LAN-bind / Docker / RBAC product fork."
|
||||
- [ ] §6.1 — add an SP24 row pointing at this plan + the spec.
|
||||
|
||||
## Task 3: `docs/ARCHITECTURE.md`
|
||||
|
||||
- [ ] Line 16 — replace "has no authentication because the threat model is a stolen or imaged drive, not a remote attacker." with the new posture.
|
||||
- [ ] Line 703 — replace "Multi-user / multi-host — no auth, no LAN-bind, no Docker, no reverse proxy. The operator is one person, the host is one machine. (SP23 fork if this changes.)" with the new posture (login required; SP23 covers LAN-bind / Docker / RBAC fork).
|
||||
- [ ] Add a new "### Auth" subsection under §5.1 listing the `cyclone.auth/` package contents and the `matrix_gate` dependency contract.
|
||||
- [ ] Migrations table — add a note that 0013 + 0014 are auth migrations and explain the renumbering on `claims-unique-fix` (0013/0014 → 0015/0016) to keep the auth migrations at the front.
|
||||
|
||||
## Task 4: `backend/src/cyclone/__main__.py`
|
||||
|
||||
- [ ] After `bootstrap.run()`, check `cyclone.auth.deps.AUTH_DISABLED` and emit a `WARNING` via `logging.getLogger("cyclone")` if True. The warning text must include the words "AUTH_DISABLED" and "dev only" so a misconfigured production deploy is greppable from logs.
|
||||
|
||||
## Task 5: Skills addenda
|
||||
|
||||
- [ ] `cyclone-tests` — add a paragraph noting the conftest's `AUTH_DISABLED` flip is mandatory context for any new test.
|
||||
- [ ] `cyclone-api-router` — add a paragraph noting every router needs `Depends(matrix_gate)` and the role matrix is in `cyclone.auth.permissions`.
|
||||
- [ ] `cyclone-spec` — add a paragraph noting the spec template's "threat model" example needs to be auth-aware (don't say "no second party to authenticate" any more; reference this SP as the reference pattern).
|
||||
|
||||
## Task 6: Verification
|
||||
|
||||
- [ ] `grep -nE 'no auth|no authentication|no second party' CLAUDE.md docs/REQUIREMENTS.md docs/ARCHITECTURE.md` returns no false-positive matches.
|
||||
- [ ] `cd backend && .venv/bin/python -c "import cyclone.__main__"` doesn't crash.
|
||||
- [ ] `cd backend && .venv/bin/python -c "import cyclone; from cyclone.auth import deps; print(deps.AUTH_DISABLED)"` prints `False` (default).
|
||||
- [ ] `cd backend && CYCLONE_AUTH_DISABLED=1 .venv/bin/python -c "from cyclone.auth.bootstrap import run; run(); from cyclone.auth.deps import AUTH_DISABLED; print(AUTH_DISABLED)"` prints `True` (env var flips the flag).
|
||||
@@ -0,0 +1,88 @@
|
||||
# Cyclone Auth Posture Alignment — Design
|
||||
|
||||
**Date:** 2026-06-23
|
||||
**Status:** Approved (in implementation on `sp24-doc-posture-alignment`, 2026-06-23)
|
||||
**Branch:** `sp24-doc-posture-alignment` (off `css-reduction` — `docs/REQUIREMENTS.md` and `docs/ARCHITECTURE.md` were added on `css-reduction` and do not yet exist on `main`)
|
||||
**Depends on:** The auth work that landed in main via the merge of `origin/main` on 2026-06-23 (commits `a25504b`..`39ae988`, 14 commits, the `feat(auth):`/`fix(auth):`/etc. series on `auth-rebased`).
|
||||
**Replaces:** The "no auth, no LAN-bind, single operator" claim in `CLAUDE.md`, `docs/REQUIREMENTS.md`, and `docs/ARCHITECTURE.md` — the codebase no longer matches the doc and the divergence needs to be resolved one way or the other.
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The auth work that landed in main is materially more than a stub. The codebase now has:
|
||||
|
||||
- A `users` and `sessions` SQLAlchemy model (migrations 0013 + 0014)
|
||||
- A `User.password_hash` field using bcrypt
|
||||
- An `auth_router` exposing `/api/auth/login`, `/api/auth/logout`, `/api/auth/me`
|
||||
- A `matrix_gate` FastAPI dependency and a permissions matrix that gates every existing endpoint by role (admin / user / viewer)
|
||||
- An `AUTH_DISABLED` flag (module-level in `cyclone.auth.deps`, also settable via `CYCLONE_AUTH_DISABLED=1` at boot) that the conftest autouse fixture flips to `True` so the test suite can run without a login round-trip per request
|
||||
- A `RateLimitMiddleware` (SP19) that the conftest bypasses via `app.state.testing`
|
||||
- A `RequireAuth` route guard and an `AuthProvider` React context on the frontend
|
||||
- A sidebar user indicator + logout button on the shell
|
||||
- A `cyclone admin` CLI for creating users and rotating passwords
|
||||
|
||||
But `CLAUDE.md`, `docs/REQUIREMENTS.md`, and `docs/ARCHITECTURE.md` (all three added or updated on the `css-reduction` branch in commit `12311bf` / `a39bfb2`) still say:
|
||||
|
||||
- `CLAUDE.md:7` — "Local-only by design: binds to `127.0.0.1`, **no auth**, no internet exposure."
|
||||
- `CLAUDE.md:182` — "**Local-only by design.** The backend binds to `127.0.0.1`, has no auth, and the threat model is a stolen/imaged drive, not a remote attacker. **Don't add internet exposure or auth without an explicit spec.**"
|
||||
- `docs/REQUIREMENTS.md:36` — "**Local-only on purpose:** binds `127.0.0.1`, **no auth**, no internet exposure, single operator, single machine, one trading partner"
|
||||
- `docs/REQUIREMENTS.md:39` — "**Threat model:** a process on the same host. No second party to authenticate; no second host to harden against."
|
||||
- `docs/REQUIREMENTS.md:154` — NFR-1: "no auth, no API key"
|
||||
- `docs/ARCHITECTURE.md:16` — "has no authentication because the threat model is a stolen or imaged drive, not a remote attacker"
|
||||
- `docs/ARCHITECTURE.md:703` — "Multi-user / multi-host — **no auth**, no LAN-bind, no Docker, no reverse proxy. The operator is one person, the host is one machine. (SP23 fork if this changes.)"
|
||||
|
||||
The pre-auth claims in the docs are now false. The threat model is no longer "a process on the same host" — there is at minimum a password boundary that requires a deliberate authentication step before any data is returned.
|
||||
|
||||
## 2. Goals
|
||||
|
||||
1. **Decide the canonical auth posture.** Is this v1 with a single local operator who has to log in, or is it the first step toward SP23 (Ubuntu + Docker + RBAC + LAN-bind)? Pick one and write it down; the docs and the code must agree.
|
||||
2. **Update the three stale top-level docs** (`CLAUDE.md`, `docs/REQUIREMENTS.md`, `docs/ARCHITECTURE.md`) to reflect the chosen posture. Every "no auth" claim is replaced with the truth; every "single operator, single host" claim is qualified by the new auth boundary.
|
||||
3. **Update the relevant Cyclone superpowers skills** that still describe the pre-auth system: at minimum `cyclone-tests` (conftest's `AUTH_DISABLED` flag is now mandatory context for any new test file), `cyclone-api-router` (the `matrix_gate` dependency is now part of every router), and `cyclone-spec` (the SP-N spec template's "threat model" section needs an auth-aware example).
|
||||
4. **Make `AUTH_DISABLED` discoverable in the conftest.** The conftest autouse fixture flips `cyclone.auth.deps.AUTH_DISABLED = True` for every test, and the test suite relies on this. A new test file that imports `cyclone.auth.deps` and reads the flag directly will see it as `True` and may make wrong assumptions. A `cyclone.auth.testing` helper that documents "this is what the test suite does to the auth flag, and how to opt out per-test" would make the dependency explicit.
|
||||
5. **Tighten the migration sequence.** Migrations 0013 and 0014 (auth_users_and_sessions, audit_log_user_id) are auth-related. They currently have no docstring beyond the filename. The migrations table in `docs/ARCHITECTURE.md` should list them and explain the dependency (the `audit_log.user_id` FK in 0014 is a forward reference to `users.id` from 0013, so the renumbering on the `claims-unique-fix` branch — 0013/0014 → 0015/0016 — was necessary to keep the auth migrations at the front of the chain).
|
||||
|
||||
## 3. Non-goals
|
||||
|
||||
- **Reverting the auth work.** It ships in main, it's tested, and the operator can no longer use Cyclone without logging in. Going back is a separate decision the user would have to ask for explicitly; this SP assumes the auth work stays.
|
||||
- **Shipping SP23 (Ubuntu + Docker + LAN-bind).** SP23 is its own design (`docs/superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md`) and is still "awaiting user decision." This SP only reconciles the doc/code divergence that the auth-merge created; it does not advance the LAN-bind / Docker conversation.
|
||||
- **Adding new auth features.** No SSO, no MFA, no per-claim ACL, no audit log viewer UI. The auth in main is what's documented; this SP just describes it.
|
||||
- **Changing the data model.** The `users` / `sessions` / `audit_log.user_id` schema stays. The bcrypt-hashed password and HttpOnly session cookie stay. The `admin` / `user` / `viewer` role enum stays.
|
||||
|
||||
## 4. Decisions to make (questions for the user)
|
||||
|
||||
These are the points that the spec can't answer on its own. They each have a default that matches the auth work in main, but the user may want to override.
|
||||
|
||||
1. **What is the v1 posture?** The auth work in main binds to `127.0.0.1` (still) and gates every endpoint behind `matrix_gate` (now). The default reading is "single-operator local-only, but with a login screen." Alternatives: revert the auth (separate decision), or treat this as a stepping stone to SP23.
|
||||
- **Default:** v1 = single-operator local-only with auth. SP23 is a future decision.
|
||||
2. **How is the first admin created?** `cyclone.auth.bootstrap` reads `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD` from the env and creates the row on first boot (no users exist) — see `backend/src/cyclone/auth/bootstrap.py:45-65`. If the env vars are unset and no users exist, `cyclone serve` errors out with a message telling the operator to set them. The `cyclone admin create-user` CLI is a follow-up convenience for adding more users after the first one.
|
||||
- **Default:** the env-var bootstrap stays. Document the env vars in `CLAUDE.md` under the existing `Install` / `Dev` sections.
|
||||
3. **Does the `AUTH_DISABLED` escape hatch ship as a real production knob?** It's a `cyclone.auth.deps.AUTH_DISABLED` module-level boolean that the conftest flips to `True` for the test suite, AND `CYCLONE_AUTH_DISABLED=1` sets it at boot via `cyclone.auth.bootstrap:37-43`. The current behavior is "synthetic admin, no role check" when the flag is set.
|
||||
- **Default:** keep the env-var escape hatch (it makes local dev + tests possible), but add a startup log line in `cyclone serve` that screams `WARNING: CYCLONE_AUTH_DISABLED=1 — all requests treated as admin, dev only` so a misconfigured production deploy fails loudly. Make the conftest path explicit in `cyclone-tests`.
|
||||
4. **Should the docs be explicit about what the auth boundary is and isn't?** The default reading is "the auth boundary is the HTTP layer; the file-system and the SQLite file are still bound by the local-only threat model (stolen/imaged drive, SQLCipher at rest, etc.)."
|
||||
- **Default:** the auth boundary is HTTP-layer only. SQLCipher, Keychain, and the 127.0.0.1 bind are unchanged.
|
||||
|
||||
## 5. Plan
|
||||
|
||||
The plan lives in `docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md` once the spec is approved. The shape of the plan is:
|
||||
|
||||
1. `CLAUDE.md` — replace lines 7 and 182 with the v1 posture ("local-only with auth; auth boundary is HTTP; default bootstrap on first run"). Update the `Frontend at a glance` / `Backend at a glance` sections to mention `cyclone.auth` and `matrix_gate`. Note the `AUTH_DISABLED` test flag in `cyclone-tests`.
|
||||
2. `docs/REQUIREMENTS.md` — replace the "Local-only on purpose" line 36 and the "Threat model" line 39 with the v1 posture. Update NFR-1 (line 154) to reflect that auth IS now part of the contract. Update §6 (FRs) to add an "Auth boundary" subsection. Update §11 (Roadmap) to note that SP24 lands the auth posture and SP23 is a separate future decision.
|
||||
3. `docs/ARCHITECTURE.md` — replace line 16 and line 703. Add an "Auth" section to the module map (`cyclone.auth` package, the `matrix_gate` dependency, the conftest bypass). Update the migrations table to list 0013 and 0014 and explain why the renumbering on the SP22 branch was necessary.
|
||||
4. The three Cyclone superpowers skills under `.superpowers/skills/` that describe tests, routing, and spec shape: each gets a one-paragraph addendum.
|
||||
5. `cyclone serve` startup log: emit a `WARNING: AUTH_DISABLED` line if `cyclone.auth.deps.AUTH_DISABLED` is `True` at boot. The flag is off by default; only the conftest flips it.
|
||||
|
||||
## 6. Verification
|
||||
|
||||
- The three top-level docs no longer contain "no auth" / "no authentication" / "no second party to authenticate" claims that are false.
|
||||
- The Cyclone skills addenda are present and self-consistent.
|
||||
- The startup warning fires when `AUTH_DISABLED = True` and is silent otherwise.
|
||||
- The existing test suite still passes (the auth changes are in main; this SP only touches docs + the startup warning).
|
||||
|
||||
## 7. Out of scope
|
||||
|
||||
- Any new feature beyond docs + skills addenda + startup warning.
|
||||
- Any change to the `users` / `sessions` / `audit_log` schema.
|
||||
- Any change to the role matrix or the per-endpoint gate.
|
||||
- Any change to SP23 (separate spec, separate decision).
|
||||
- The CLAUDE.md contradiction is the canary; the rest of the doc set may have other places where the pre-auth claim leaked in. The implementation step sweeps `docs/` and `*.md` files under `.superpowers/skills/` for the strings "no auth", "no authentication", "single operator, single host" and either fixes them or explicitly marks them as historical context.
|
||||
Reference in New Issue
Block a user