tyler/cyclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(sp42): REQUIREMENTS.md — refresh store/api LOC, migrations 12→23, SP23 shipped, FR-35/36 met, risks closed

Browse Source
This commit is contained in:
Nora
2026-07-08 23:13:02 -06:00
parent d8d03d631f
commit 2bb10f2a39
1 changed files with 76 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/REQUIREMENTS.md
+76 -51
View File
@@ -25,7 +25,7 @@
Every requirement (§3, §4) has an ID; every ID is traceable to a sub-project (§6) which in turn points at a design spec and an implementation plan. The matrix at the end of §6 is the single index that ties everything together.
**Round history.** Round 1 (2026-06-23) produced this draft. Two independent reviewers (Reviewer A — components/data-model/deps/DoD; Reviewer B — traceability/consistency audit) read it against the actual codebase and surfaced 12 factual errors and 6 structural improvements. Round 2 (this version) applies every factual correction. Round 3 (pending) will re-verify reviewer agreement on the corrected doc.
**Round history.** Round 1 (2026-06-23) produced this draft. Two independent reviewers (Reviewer A — components/data-model/deps/DoD; Reviewer B — traceability/consistency audit) read it against the actual codebase and surfaced 12 factual errors and 6 structural improvements. Round 2 (this version, dated 2026-07-08) applies every factual correction through SP41, including the post-SP23 doc alignment (LAN-bind + auth posture) and the SP21 store split + SP36 api split + SP41 rebill additions. Round 3 (pending) will re-verify reviewer agreement on the corrected doc.
---
@@ -33,18 +33,18 @@ Every requirement (§3, §4) has an ID; every ID is traceable to a sub-project (
A self-hosted EDI claims-management suite for a single billing office. Parses 837P professional claims and 835 ERA remittances (X12 005010X222A1 and 005010X221A1), with a local-only FastAPI backend and a React UI for browsing, filtering, and inspecting the parsed data.
- **Local-only on purpose:** binds `127.0.0.1`, requires login (auth boundary is HTTP; bcrypt + HttpOnly session cookie), no internet exposure, single operator, single machine, one trading partner (Colorado Medicaid, currently).
- **LAN-only by design:** binds `0.0.0.0:8000`, requires login (auth boundary is HTTP; bcrypt + HttpOnly session cookie; first admin bootstrapped from `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD` env vars), reachability controlled by the host firewall / compose port publishing, single operator, one trading partner (Colorado Medicaid, currently). The dev/test escape hatch is `CYCLONE_AUTH_DISABLED=1`, which logs a WARNING at boot.
- **Single-process:** the backend is one Python process; the frontend is one Vite-served React app; there is no message broker, no separate worker, no queue.
- **Single source of truth:** SQLite at `~/.local/share/cyclone/cyclone.db` (overridable via `CYCLONE_DB_URL`); optionally encrypted at rest via SQLCipher (SP12).
- **Threat model:** a process on the same host, plus a deliberate HTTP-layer login (no remote unauthenticated read or write). File-system threats (stolen/imaged drive) are still the primary concern; SQLCipher at rest + the macOS Keychain handle those. The SP23 product fork changes the threat model to "remote operator on LAN."
- **Threat model:** a process on the same host, plus a deliberate HTTP-layer login (no remote unauthenticated read or write). File-system threats (stolen/imaged drive) are still the primary concern; SQLCipher at rest + the macOS Keychain handle those. SP23 expanded the threat model to "remote operator on LAN" by adding login + RBAC + Docker + Ubuntu packaging; the LAN-bind posture remains.
The Cyclone codebase has honored this contract end-to-end across 22 shipped sub-projects. Every design spec re-asserts the local-only / single-operator / single-payer / no-auth posture and lists (often at length) what is intentionally out of scope.
The Cyclone codebase has honored this contract end-to-end across 41 shipped sub-projects. Every design spec re-asserts the LAN-only / single-operator / single-payer posture and lists (often at length) what is intentionally out of scope.
---
## 2. Scope boundaries
### 2.1 In scope (today, after SP1–SP22)
### 2.1 In scope (today, after SP1–SP41)
| Capability | SP |
|---|---|
@@ -69,9 +69,16 @@ The Cyclone codebase has honored this contract end-to-end across 22 shipped sub-
| Structured JSON logging + PII scrubber | SP18 |
| Security hardening (body size limit, rate limit, security headers) | SP19 |
| NPI Luhn checksum + Tax ID format validation | SP20 |
| CycloneStore split (refactor; **in-flight on `refactor/store-split`**) | SP21 |
| CycloneStore split (16-module subpackage; public API preserved via `cyclone.store` facade) | SP21 |
| Parse-then-decide upload dedup (idempotency) | SP22 |
| Pipeline automation agent (sibling project at `cyclone-pipeline/`) | SP22 |
| HTTP-layer auth (bcrypt + HttpOnly session cookie) + RBAC roles + `CYCLONE_AUTH_DISABLED` escape hatch | SP23 |
| Ubuntu + Docker + nginx + tini packaging | SP23 |
| Audit-log `user_id` (every audit event records the acting user) | SP-auth |
| `submission_dedup` table + claim-id dedup at SFTP pre-flight | SP22 + SP41 |
| Resubmissions table + bulk ingest helpers | SP21 (resubmissions) |
| In-window rebill pipeline (visit↔835 reconciliation + Pipeline A/B + summary CSV) | SP41 |
| Live-tail NDJSON streams for `/api/acks/stream` and `/api/ta1-acks/stream` | SP-acks-tail |
### 2.2 Out of scope (today, by design)
@@ -140,8 +147,8 @@ Each requirement is `FR-NN`, traceable to one or more sub-projects. Verification
| FR-32 | Reject oversize request bodies (413), rate-limit requests (429), and add default security headers via pure-ASGI middlewares; emit a tamper-evident `api.request_rejected` audit event on 413/429. | SP19 |
| FR-33 | Surface a rich `GET /api/health` response — DB connectivity, MFT scheduler state, backup scheduler state, live pubsub subscriber counts, last batch id + timestamp. | SP19 (`api_routers/health.py:28-40`) |
| FR-34 | Validate NPI with the CMS-published Luhn over `80840 + body` (warning-only — placeholder NPIs in test fixtures shouldn't block ingest). Reject Tax ID (EIN) with reserved prefixes (`00`, `07`, `80`–`89`). | SP20 |
| FR-35 | Split `CycloneStore` into per-concern modules (`store_persistence`, `store_reconcile`, `store_queries`, `store_mappers`) without changing the public store API. | SP21 (**in-flight** on `refactor/store-split` branch) |
| FR-36 | De-duplicate 837 / 835 uploads via a parse-then-decide workflow (`find_existing_batch_for_claim` / `find_existing_batch_for_remit`); relax the claims/remits PK to `(batch_id, id)` via migration 0014 to support re-parses into different batches. | SP22 (migration 0014 in worktree; not yet on `main`) |
| FR-35 | Split `CycloneStore` into per-concern modules under `cyclone/store/` (16 modules, ~5,400 LOC total) without changing the public store API; the `cyclone.store` import surface still resolves to the same names. | SP21 (**Met**; merged on `main`; module list in §5.1) |
| FR-36 | De-duplicate 837 / 835 uploads via a parse-then-decide workflow (`find_existing_batch_for_claim` / `find_existing_batch_for_remit`); relax the claims unique constraint via migration 0015 (a defensive drop of an inline UNIQUE that was a no-op against the current schema) to support re-parses into different batches. | SP22 (**Met**; migration 0015 on `main`) |
| FR-37 | (Sibling project, not in this repo) Drive the full 7-phase round-trip — preflight → browser upload → parse verify → SFTP submit → TA1 wait → 999 wait → scan + report — with crash-safe resume, structured JSON logging, idempotency dedup, per-run folder. | SP22 (`cyclone-pipeline/`) |
| FR-38 | Provide a `DrillStackProvider` + `DrillDrawerHeader` shell so every drillable cell across the app opens a consistent drawer or peek modal. | SP21 |
@@ -164,7 +171,7 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re
| ID | NFR | Source / SP |
|---|---|---|
| NFR-1 | **Always bind 0.0.0.0 + auth.** Backend binds `0.0.0.0:8000` (overridable via `CYCLONE_HOST` / `CYCLONE_PORT`); CORS allowlist is exact (`http://localhost:5173`); login required (bcrypt + HttpOnly session cookie; first admin bootstrapped from env vars `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD`); dev/test escape hatch via `CYCLONE_AUTH_DISABLED=1` (logs WARNING at boot). Reachability is controlled by the host firewall / compose port publishing, not the bind address. | SP1 + auth (2026-06-23 merge) + SP24 (doc reconciliation) |
| NFR-1 | **Always bind 0.0.0.0 + auth.** Backend binds `0.0.0.0:8000` (overridable via `CYCLONE_HOST` / `CYCLONE_PORT`); CORS allowlist is exact (`http://localhost:5173`); login required (bcrypt + HttpOnly session cookie; first admin bootstrapped from env vars `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD`); every API router declares `Depends(matrix_gate)` (`cyclone/auth/deps.py:matrix_gate`) so requests without a valid session are rejected; dev/test escape hatch via `CYCLONE_AUTH_DISABLED=1` (logs WARNING at boot). Reachability is controlled by the host firewall / compose port publishing, not the bind address. | SP1 + SP23 (auth + RBAC) |
| NFR-2 | **Determinism.** Parser / serializer round-trip is guaranteed on 113 real prodfiles (`docs/prodfiles/claims/*.x12`); canonical fields, not byte-identity (called out in the SP8 spec). | SP8 |
| NFR-3 | **Audit completeness.** Every reconciliation anomaly, every state transition, every 999/277CA reject writes an `ActivityEvent` (in `activity_events` table, un-chained; powers the `/activity` feed and inbox state transitions). | SP2 + SP10 |
| NFR-4 | **Tamper-evidence.** A separate `audit_log` table (SP11) carries SHA-256 hash-chained rows for security-sensitive events (login attempts, rejections, key rotations, backup lifecycle, rejected requests). `GET /api/admin/audit-log/verify` detects any break. **The `activity_events` (NFR-3) and `audit_log` (NFR-4) tables are distinct** — different semantics, different audiences. New code that needs an audit row must decide which one to write to. | SP11 |
@@ -178,7 +185,7 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re
| NFR-12 | **Idempotency.** Inbound MFT files are de-duplicated via `processed_inbound_files`; 837/835 uploads are de-duplicated via the parse-then-decide workflow with `(batch_id, claim_id)` / `(batch_id, remit_id)` relaxed PKs. | SP16 + SP22 |
| NFR-13 | **Crash-safety.** Incoming files are processed inside per-file try/except so a bad file doesn't stop the MFT loop; the pipeline agent has crash-safe resume keyed by run id. | SP16 + SP22 |
| NFR-14 | **Single-process.** No message broker, no separate worker, no queue; everything runs in one Python process. | (project-wide constraint) |
| NFR-15 | **Test density.** Backend tests: **964 collected** (`pytest --collect-only`, 2026-06-23); frontend test files: **73**; prodfiles are exercised (113 + 19 + 1369), not just minimal fixtures. | (project-wide) |
| NFR-15 | **Test density.** Backend tests: **1,624 collected** (`pytest --collect-only`, 2026-07-08); frontend test files: **85**; prodfiles are exercised (113 + 19 + 1369), not just minimal fixtures. | (project-wide) |
| NFR-16 | **Documentation discipline.** Every shipped sub-project has a design spec + implementation plan + smoke test; a new engineer can read the project top-to-bottom. (SPs 9–16 ship without on-disk plan files — see §6.1 — but their `feat(spN)` commits are on `main` and carry their own commit-message plan.) | (project-wide) |
| NFR-17 | **Fail-soft posture.** Reconciliation crashes don't lose the 835; the activity event records the failure. | SP2 |
| NFR-18 | **Migration safety.** Migrations are forward-only via `PRAGMA user_version`; rollback procedures are documented per migration; the `user_version` runner is idempotent. **No checksum manifest exists today** (R-13 in §11). | SP2 + SP22 |
@@ -193,14 +200,16 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re
| Module | Owns | Notes |
|---|---|---|
| `cyclone.api` | HTTP routes, content negotiation, CORS allowlist, lifespan | 3,145 LOC (largest single file; partly split into `api_routers/` for 4 sub-routes) |
| `cyclone.api` | HTTP routes, content negotiation, CORS allowlist, lifespan | 378 LOC; routes are mounted from `api_routers/` (22 sub-routers — see below) |
| `cyclone.api_helpers` | NDJSON / content-negotiation / live-tail helpers | |
| `cyclone.api_routers` | `acks`, `admin`, `health` (`/api/health`), `ta1_acks` sub-routers | Mounted at `api.py:270-273`; only 4 routes split out — bulk of routes still in `api.py` |
| `cyclone.store` | Public store facade; persistence, reconciliation, queries, mappers | 2,172 LOC; SP21 split is in-flight on `refactor/store-split` |
| `cyclone.api_routers` | Per-resource sub-routers mounted under `/api/...` | 22 modules: `acks`, `activity`, `admin`, `batches`, `claim_acks`, `claims`, `clearhouse`, `config`, `dashboard`, `eligibility`, `health`, `inbox`, `parse`, `payers`, `providers`, `rebill`, `reconciliation`, `remittances`, `submission`, `ta1_acks` (+ `_shared.py`). Every router declares `Depends(matrix_gate)` for auth. |
| `cyclone.auth` | Login, logout, session cookie, RBAC roles, `matrix_gate` dependency | New in SP23; password hashing, session table, user roles |
| `cyclone.handlers` | Shared FastAPI handlers (auth-aware JSON responses, audit-log injection, etc.) | Cross-cutting request glue |
| `cyclone.store` (subpackage) | Public store facade; the split modules behind it | 16 modules (`__init__` facade + `acks`, `backfill`, `backups`, `batches`, `claim_acks`, `claim_detail`, `exceptions`, `inbox`, `kpis`, `orm_builders`, `providers`, `records`, `resubmissions`, `submission_dedup`, `ui`, `write`); public API re-exported from `cyclone.store` so callers see no breaking change |
| `cyclone.db` | SQLAlchemy engine, session factory, ORM models (18 tables) | |
| `cyclone.db_migrate` | `PRAGMA user_version` migration runner | |
| `cyclone.db_crypto` | Optional SQLCipher encryption at rest; key rotation | `rotate_key()` is destructive (rewrites every page) |
| `cyclone.audit_log` | Hash-chained `audit_log` (SP11, SHA-256 chain) | Distinct from `cyclone.activity_events` written by other modules |
| `cyclone.audit_log` | Hash-chained `audit_log` (SP11, SHA-256 chain) | Distinct from `cyclone.activity_events` written by other modules; post-SP23 records `user_id` of the actor |
| `cyclone.inbox_lanes` | The 5-lane inbox payload (Rejected / Payer-rejected / Candidates / Unmatched / Done today) | |
| `cyclone.inbox_state` | 999 envelope reject → claim state transitions | |
| `cyclone.inbox_state_277ca` | 277CA STC A4/A6/A7 → payer_rejected stamp (monotonic) | |
@@ -210,16 +219,19 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re
| `cyclone.reconcile` | Pure-function 835→claim match + line-level match | No DB session inside; testable with fabricated ORM objects |
| `cyclone.scoring` | 4-field weighted candidate scoring (40/25/20/15) | Weights hardcoded — see R-10 in §11 |
| `cyclone.pubsub` | In-process EventBus (drop-oldest, per-kind fan-out) | In-process only; not thread-safe across asyncio event loops |
| `cyclone.backup` | AES-256-GCM encrypted backups (orchestration + CLI) | 216 LOC |
| `cyclone.backup_service` | Backup lifecycle (create / verify / restore) | 740 LOC |
| `cyclone.backup_scheduler` | 24h backup autostart loop | 315 LOC |
| `cyclone.backup` | AES-256-GCM encrypted backups (orchestration + CLI) | |
| `cyclone.backup_service` | Backup lifecycle (create / verify / restore) | |
| `cyclone.backup_scheduler` | 24h backup autostart loop | |
| `cyclone.scheduler` | asyncio MFT polling loop (SP16) | Uses `processed_inbound_files` for idempotency |
| `cyclone.security` | Per-IP rate-limit, body-size limit, security headers (`SecurityHeadersMiddleware`, `BodySizeLimitMiddleware`, `RateLimitMiddleware`) | `EXEMPT_PATHS` list at `:201` |
| `cyclone.logging_config` | JSON formatter, PII scrubber | |
| `cyclone.npi` | NPI Luhn + EIN format validators | |
| `cyclone.batch_diff` | Batch Diff engine (powers `GET /api/batch-diff`) | 12 public functions/classes |
| `cyclone.clearhouse` (`__init__.py`) | `SftpClient` (SP9 stub + SP13 paramiko), `InboundFile`, `make_client()` factory | 318 LOC; the SFTP home |
| `cyclone.clearhouse` (`__init__.py`) | `SftpClient` (SP9 stub + SP13 paramiko), `InboundFile`, `make_client()` factory | The SFTP home |
| `cyclone.edi` (`filenames.py`) | HCPF X12 File Naming Standards helpers | Used by SP13/SP16 outbound filename construction |
| `cyclone.rebill` | In-window rebill pipeline: visit↔835 reconciliation, CARC filter, timely-filing gate, Pipeline A/B, summary CSV | New in SP41 |
| `cyclone.reissue` | Reissue-claims pipeline (revised-claim regeneration + RUNBOOK entry) | SP24 |
| `cyclone.submission` | Canonical `submit_file` helper shared by `cyclone submit-batch` CLI and `POST /api/submit-batch`; parse → DB-write → SFTP-upload → audit per file | SP37 |
| `cyclone.cli` | Click CLI | |
| `cyclone.__main__` | `python -m cyclone serve` | |
| `cyclone.parsers/` | X12 tokenizer, models, validator, writers, 277CA, 999, TA1, 270, 271, CAS, seg | 25 `.py` modules |
@@ -230,7 +242,7 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re
| Directory | Owns |
|---|---|
| `src/pages/` | Acks, ActivityLog, BatchDiff, Batches, Claims, Dashboard, Inbox, Providers, Reconciliation, Remittances, Upload (11 pages) |
| `src/pages/` | Acks, ActivityLog, BatchDiff, Batches, Claims, Dashboard, Inbox, Login, Providers, Reconciliation, Remittances, Upload (12 pages — Login added in SP23) |
| `src/hooks/` | TanStack Query hooks, URL-state hooks, live-tail hooks, keyboard hooks, batch-export, search, drawer keyboard, count-up, provider detail, payer summary, parse, ack detail (35+ files) |
| `src/lib/` | `api.ts`, `format.ts`, `utils.ts`, `csv.ts`, `download.ts`, `event-routing.ts`, `tail-stream.ts`, `inbox-api.ts` (all with paired `.test.ts`) |
| `src/components/` | 3 drawer families (`ClaimDrawer/` ~15 files, `RemitDrawer/` ~10 files, `ProviderDrawer/` ~6, `AckDrawer/` 3); `inbox/` (10 files), `drill/` (10 files, includes `DrillStackProvider` + `DrillDrawerHeader`), `charts/` (4 files), `ui/` (18 shadcn primitives — radix + cva + tailwind-merge), `KeyboardCheatsheet/` (3 files), plus top-level ActivityFeed, AnimatedNumber, BatchDetail, BatchDiffView, BatchesList, ClaimCard837, DominantKpiCard, EditorialNote, ExportBar, ExportCsvButton, KpiCard, Layout, NewClaimDialog, PageHeader, SearchBar, SearchResults, Sidebar, Sparkline, StatusBadge, StatusPill, TailStatusPill, TickerTape |
@@ -239,7 +251,7 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re
### 5.2 Data model
**12 SQL migrations** in `backend/src/cyclone/migrations/` (verified on disk 2026-06-23):
**23 SQL migrations** in `backend/src/cyclone/migrations/` (verified on disk 2026-07-08):
| # | File | Tables / columns added |
|---|---|---|
@@ -255,13 +267,17 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re
| 0010 | `0010_payer_rejected_acknowledged.sql` | `payer_rejected_acknowledged` column on `claims` |
| 0011 | `0011_processed_inbound_files.sql` | `processed_inbound_files` (SP16 idempotency) |
| 0012 | `0012_backups.sql` | `db_backups` (SP17) |
**In-flight on `claims-unique-fix` worktree (not yet on `main`):**
| # | File | Tables / columns added |
|---|---|---|
| 0013 | `0013_drop_claims_unique_constraint.sql` | Drop `claims` UNIQUE constraint |
| 0014 | `0014_relax_claims_remits_pk.sql` | Relax PK to `(batch_id, id)` (SP22) |
| 0013 | `0013_auth_users_and_sessions.sql` | `users`, `sessions`, RBAC role columns (SP23 auth) |
| 0014 | `0014_audit_log_user_id.sql` | `audit_log.user_id` column + index (records acting user on every audit event) |
| 0015 | `0015_drop_claims_unique_constraint.sql` | Defensive drop of the inline `UNIQUE(batch_id, patient_control_number)` on `claims` (SP22; table-recreate dance — no-op against current schema) |
| 0016 | `0016_claims_matched_remittance_id_index.sql` | Index on `claims.matched_remittance_id` to speed inbox joins |
| 0017 | `0017_backfill_claim_patient_control_number.sql` | Backfill `patient_control_number` from X12 CLM01 for legacy rows |
| 0018 | `0018_claim_acks.sql` | `claim_acks` table (claim-level ACK metadata) |
| 0019 | `0019_add_rendering_and_service_provider_npis.sql` | Rendering + service-provider NPI columns on `claims` |
| 0020 | `0020_add_batch_txn_set_control_number.sql` | `batches.transaction_set_control_number` (ST/SE control number persistence) |
| 0021 | `0021_resubmissions.sql` | `resubmissions` table (resubmission tracking + bulk-ingest helpers) |
| 0022 | `0022_submission_dedup.sql` | `submission_dedup` table (claim-id dedup window at SFTP pre-flight, SP41) |
| 0023 | `0023_visits.sql` | `visits` table (SP41 in-window rebill pipeline input) |
**ORM models** (declared in `backend/src/cyclone/db.py` — 18 classes total):
@@ -417,18 +433,20 @@ Each row's "Spec" / "Plan" point to the design spec and implementation plan on d
| SP18 | Structured logging | [spec](superpowers/specs/2026-06-21-cyclone-structured-logging-design.md) | (no plan file on disk; feat(sp18) commits on `main`) | Met |
| SP19 | Security hardening + health probe | [spec](superpowers/specs/2026-06-21-cyclone-security-hardening-design.md) | (no plan file on disk; feat(sp19) commits on `main`) | Met |
| SP20 | NPI checksum + Tax ID format validation | [spec](superpowers/specs/2026-06-21-cyclone-npi-validation-design.md) | (no plan file on disk; feat(sp20) commits on `main`) | Met |
| SP21 | CycloneStore split (in-flight) + Universal drilldown (shipped) | [split spec](superpowers/specs/2026-06-21-cyclone-store-split-design.md), [drilldown spec](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) | [store-split plan](superpowers/plans/2026-06-21-cyclone-store-split.md), [drilldown plan](superpowers/plans/2026-06-21-cyclone-universal-drilldown.md) | Drilldown **Met**; split **In-flight** (`refactor/store-split` branch, not yet on `main`) |
| SP21 | CycloneStore split (16-module subpackage) + Universal drilldown | [split spec](superpowers/specs/2026-06-21-cyclone-store-split-design.md), [drilldown spec](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) | [store-split plan](superpowers/plans/2026-06-21-cyclone-store-split.md), [drilldown plan](superpowers/plans/2026-06-21-cyclone-universal-drilldown.md) | Met (split landed on `main`; public API re-exported from `cyclone.store` facade; drilldown used by every drawer) |
| SP22 | Parse-decide workflow + Pipeline agent | [parse-decide spec](superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md), [pipeline spec](superpowers/specs/2026-06-21-cyclone-pipeline-agent-design.md) | [parse-decide plan](superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md), [pipeline plan](superpowers/plans/2026-06-21-cyclone-pipeline-agent.md) | Met (parse-decide on `main`; pipeline agent lives in `cyclone-pipeline/`) |
| SP24 | Auth posture alignment (docs-only reconciliation with the auth work in `main`) | [spec](superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md) | [plan](superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md) | Met (no code paths change; only docs + `__main__.py` WARNING + 3 skills addenda) |
| SP23 | Ubuntu + Docker + nginx + tini + auth + RBAC (LAN-bind product fork) | [spec](superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md) | (plan in the same docs directory; auth is the dominant change) | Met (merges `cyclone.auth.*`, every API router gated by `matrix_gate`, Login page added to the React app) |
| SP24 | Reissue-claims (revised-claim regeneration + deprecation shim) | [spec](superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md) (docs alignment + reissue) | [plan](superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md) | Met (merge commit `c9588f7`; `cyclone.reissue` shipped) |
| SP25–SP40 | Sub-projects landed between SP24 and SP41 (covering resubmissions, `claim_acks`, rendering/service NPI columns, transaction-set control numbers, additional live-tail streams, dashboards, rebill prerequisites) | (per-SP design specs + plans under `docs/superpowers/`) | (per-SP plans under `docs/superpowers/plans/`) | Met — see §6.1 acceptance table below |
| SP41 | In-window rebill pipeline (`cyclone.rebill.*`) | [spec](superpowers/specs/2026-07-07-cyclone-inwindow-rebill-pipeline-design.md) | [plan](superpowers/plans/2026-07-07-cyclone-inwindow-rebill-pipeline.md) | Met (merge commit `309e6c0`; visit↔835 reconciliation, Pipeline A/B, summary CSV, claim-id dedup at SFTP pre-flight) |
### 6.2 Backlog (specs awaiting plans, or vice versa)
| Item | Status | Why |
|---|---|---|
| SP23 (candidate) | Spec-only; **product fork awaiting user decision** | `2026-06-22-cyclone-ubuntu-docker-deployment-design.md` (auth + Docker + RBAC + LAN-bind); see §11 R-1, R-3 |
| Plan-file backfill for SP9, SP10, SP11, SP12, SP13, SP14, SP15, SP16, SP17, SP18, SP19, SP20 | Plan files missing on disk; `feat(spN)` commit messages substitute | Round-3 doc-backlog item |
| Spec-file backfill for SP6 (workflow/inbox), SP11 (audit log), SP12 (SQLCipher), SP13 (paramiko), SP15 (key rotation), SP16 (MFT scheduler) | Specs missing on disk | Round-3 doc-backlog item |
| Migration manifest (checksums for the 12 SQL files) | Open | Completeness review §3.1.18 |
| Migration manifest (checksums for the 23 SQL files) | Open | Completeness review §3.1.18 |
| Tunable score weights (move 40/25/20/15 from code to config) | Open | Completeness review §3.1.15 |
| License file (`LICENSE`) | Open | Completeness review §3.2.28 |
| `pre-commit` / `Makefile` / `CONTRIBUTING.md` / `.editorconfig` / `ruff` config | Open | Completeness review §3.2.30 |
@@ -482,8 +500,8 @@ This matrix is the single index. Each row points to a requirement and where it's
| FR-32 | SP19 | security-hardening | (none on disk) | `test_security.py` |
| FR-33 | SP19 | security-hardening | (none on disk) | `test_api.py:33 test_health_endpoint`, `test_security.py:204-221` (health snapshot) |
| FR-34 | SP20 | npi-validation | (none on disk) | `test_npi.py`, `test_api_validate_provider.py`, `test_cli_validate.py` |
| FR-35 | SP21 | store-split | store-split | `test_store.py` (currently covers the un-split `store.py`; will cover the split modules once `refactor/store-split` lands) |
| FR-36 | SP22 | parse-decide-workflow | parse-decide-workflow | `test_api_parse_persists.py`, `test_prodfiles_smoke.py`; migration 0014 in `claims-unique-fix` worktree |
| FR-35 | SP21 | store-split | store-split | `tests/store/test_*.py` (per-module coverage of the split subpackage); `test_store.py` continues to exercise the public `cyclone.store` facade |
| FR-36 | SP22 | parse-decide-workflow | parse-decide-workflow | `test_api_parse_persists.py`, `test_prodfiles_smoke.py`, migration 0015 (defensive drop of inline UNIQUE on `claims(batch_id, patient_control_number)`) |
| FR-37 | SP22 | pipeline-agent | pipeline-agent | (lives in `cyclone-pipeline/`, not this repo) |
| FR-38 | SP21 | universal-drilldown | universal-drilldown | `DrillStackProvider.test.tsx`, `useDrawerUrlState.test.ts`, plus per-drawer tests in `ClaimDrawer/`, `RemitDrawer/`, `ProviderDrawer/`, `AckDrawer/` |
@@ -588,12 +606,12 @@ These are the existing per-SP acceptance checklists as captured in their design
```bash
# Backend
cd backend
.venv/bin/pytest # 964 tests collected (2026-06-23)
.venv/bin/pytest # 1,624 tests collected (2026-07-08)
.venv/bin/python -m cyclone serve
# Frontend
npm run typecheck
npm test # 73 test files
npm test # 85 test files
npm run build
# Smoke (planned in SP23)
@@ -628,7 +646,7 @@ Recorded explicitly so a new engineer can challenge any of them.
| A8 | The X12 5010 transaction set Cyclone supports is sufficient for the operator's billing workflow. | Adding 837I / 837D / 278 / 276-277 / 277CA-round-trip / etc. is each a new SP. |
| A9 | Local-only bind is sufficient — no remote access needed. | Would require auth (SP23), TLS, reverse proxy, RBAC. |
| A10 | One Python process, no message broker, no separate worker. | A multi-worker deployment would require breaking thread-affinity (SP12/SP15 specifically). |
| A11 | 964 backend tests + 73 frontend test files is the right test density for this codebase (2026-06-23). If we add more transaction types or surface area, this needs to grow proportionally; coverage gates should be enforced (round-3 backlog). | New transaction types or surface area without test growth → coverage drifts down. |
| A11 | 1,624 backend tests + 85 frontend test files is the right test density for this codebase (2026-07-08). If we add more transaction types or surface area, this needs to grow proportionally; coverage gates should be enforced (round-3 backlog). | New transaction types or surface area without test growth → coverage drifts down. |
| A12 | `cyclone-pipeline/` is a sibling project, not a sub-project of this repo. | The 7-phase round-trip agent is shipped in `cyclone-pipeline/`; if that project is abandoned, the manual operator workflow remains (upload via `/upload`, SFTP by hand, `check-835` not available). |
---
@@ -639,9 +657,9 @@ Pulled forward from [`docs/reviews/2026-06-20-cyclone-completeness-review.md`](r
| ID | Risk / gap | Source | Status |
|---|---|---|---|
| R-1 | No app-layer auth (anyone on the host can read PHI). | Completeness review §3.1.4 + §3.2.25 | **Closed by SP24** — auth shipped via the merge of `origin/main` on 2026-06-23 (commits `a25504b`..`39ae988`, `cyclone.auth.*` + `matrix_gate` dependency on every router); SP24 reconciled the docs to match. SP23 still covers the LAN-bind / Docker / RBAC product fork. |
| R-1 | No app-layer auth (anyone on the host can read PHI). | Completeness review §3.1.4 + §3.2.25 | **Closed by SP23** — auth shipped via the merge of `origin/main` on 2026-06-23 (commits `a25504b`..`39ae988`, `cyclone.auth.*` + `matrix_gate` dependency on every router); `users` / `sessions` tables live in migration 0013; every audit event records the acting `user_id` (migration 0014). |
| R-2 | SQLite plaintext if SQLCipher not enabled. | Completeness review §3.1.1 | **Partial** — SP12 + SP15 ship the encryption option, but it falls back to plain if the Keychain entry is missing. |
| R-3 | No Dockerfile / docker-compose.yml. | Completeness review §3.2.29 | **Open** — SP23 spec covers it but is awaiting user decision. |
| R-3 | No Dockerfile / docker-compose.yml. | Completeness review §3.2.29 | **Closed by SP23** — Docker packaging is part of the LAN-bind product fork (Ubuntu base, `tini`, nginx reverse proxy, `docker-compose.yml`). |
| R-4 | PayerConfig still partly in code (`co_medicaid()` factory at `cyclone/parsers/payer.py:57-58`). | Completeness review §3.1.9 | **Open** — SP9 externalized config but the factory remains as a fallback for tests and default-payer selection. |
| R-5 | No vocabulary tables (ICD-10 / HCPCS / NDC). | Completeness review §3.1.11 | **Open** — explicitly out of scope per A6. |
| R-6 | No COB / secondary-claim generator. | Completeness review §3.1.12 | **Open** — explicitly out of scope per §2.2. |
@@ -649,9 +667,9 @@ Pulled forward from [`docs/reviews/2026-06-20-cyclone-completeness-review.md`](r
| R-8 | No NPPES NPI lookup. | Completeness review §3.1.10 | **Open** — only Luhn checksum (SP20). |
| R-9 | No 837I / 837D / 834 / 820 / 275 / 278 / 276-277 transaction types. | Completeness review §2.1 | **Open** — explicitly out of scope per §2.2. |
| R-10 | Score weights hardcoded (`cyclone/scoring.py:42-46`). | Completeness review §3.1.15 | **Open** — not scheduled. |
| R-11 | `cyclone/store.py` is a 2,172-line god-module. | Completeness review §3.1.17 | **In-flight** — SP21 spec + plan written; branch `refactor/store-split` exists; merge to `main` pending. |
| R-12 | `cyclone/api.py` is a 3,145-line god-module (larger than `store.py`). | Completeness review §3.1.19 | **Partial** — `api_routers/` exists for 4 sub-routes (acks, admin, health, ta1_acks); bulk of routes still in `api.py`. |
| R-13 | Migrations in flat dir without manifest. | Completeness review §3.1.18 | **Open** — no checksum list; 12 SQL files in `backend/src/cyclone/migrations/`. |
| R-11 | `cyclone/store.py` is a 2,172-line god-module. | Completeness review §3.1.17 | **Closed by SP21** — `cyclone.store` is now a 16-module subpackage (`acks`, `backfill`, `backups`, `batches`, `claim_acks`, `claim_detail`, `exceptions`, `inbox`, `kpis`, `orm_builders`, `providers`, `records`, `resubmissions`, `submission_dedup`, `ui`, `write` + `__init__` facade); public API re-exported from `cyclone.store` so callers see no breaking change. |
| R-12 | `cyclone/api.py` is a 3,145-line god-module (larger than `store.py`). | Completeness review §3.1.19 | **Closed by SP36** — `api.py` is now 378 LOC; 22 sub-routers live under `cyclone/api_routers/` (every router gated by `Depends(matrix_gate)`). |
| R-13 | Migrations in flat dir without manifest. | Completeness review §3.1.18 | **Open** — no checksum list; 23 SQL files in `backend/src/cyclone/migrations/`. |
| R-14 | No `pre-commit` / `Makefile` / `CONTRIBUTING.md` / `.editorconfig` / `ruff` config. | Completeness review §3.2.30 | **Open** — single-operator project. |
| R-15 | No license file (`LICENSE`). | Completeness review §3.2.28 | **Open** — not scheduled. |
| R-16 | PHI fixtures not flagged as PHI. | Completeness review §3.2.23 | **Open** — not scheduled. |
@@ -662,10 +680,11 @@ Pulled forward from [`docs/reviews/2026-06-20-cyclone-completeness-review.md`](r
| R-21 | No structured logging (before SP18). | Completeness review §3.1.5 | **Closed by SP18**. |
| R-22 | No backup automation (before SP17). | Completeness review §3.1.3 | **Closed by SP17**. |
| R-23 | `ActivityEvent` not tamper-evident (before SP11). | Completeness review §3.1.2 | **Closed by SP11** — but note: the tampered table is `audit_log`, distinct from `activity_events` (see NFR-3 / NFR-4). |
| R-24 | SP23 product fork (auth + Docker + RBAC + LAN-bind). | §6.2 | **Awaiting user decision** — see §2.2 and §11 R-1. |
| R-24 | SP23 product fork (auth + Docker + RBAC + LAN-bind). | §6.1 | **Closed by SP23** — auth, RBAC, Docker, nginx, tini, LAN-bind all shipped on `main`. |
| R-25 | `vitest@^4.1.9` is unusually new (current stable line is 1.x/2.x). | §5.3 | **Open** — verify the version pin is intentional before the next `npm install`; fallback to `^2.x` if accidental. |
| R-26 | No CI workflow (`.github/workflows/`). | §8.1 #3 | **Open** — release gating is manual today. |
| R-27 | `backend/src/cyclone/workflow/__pycache__/` is dead bytecode. | §5.1 | **Open** — recommend deletion. |
| R-28 | Test density drifted from 964 → 1,624 backend tests and 73 → 85 frontend test files. | §4 NFR-15 | **Open** — coverage gates (`--cov-fail-under=90`) still not enforced; counts tracked in NFR-15. |
---
@@ -694,7 +713,10 @@ All under [`docs/superpowers/specs/`](superpowers/specs/):
- [2026-06-21-cyclone-store-split-design.md](superpowers/specs/2026-06-21-cyclone-store-split-design.md) — SP21 (split)
- [2026-06-21-cyclone-structured-logging-design.md](superpowers/specs/2026-06-21-cyclone-structured-logging-design.md) — SP18
- [2026-06-21-cyclone-universal-drilldown-design.md](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) — SP21 (drilldown)
- [2026-06-22-cyclone-ubuntu-docker-deployment-design.md](superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md) — **SP23 candidate; awaiting plan + user product-fork decision**
- [2026-06-22-cyclone-ubuntu-docker-deployment-design.md](superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md) — **SP23 (shipped)**: Ubuntu + Docker + nginx + tini + auth + RBAC LAN-bind product fork
- [2026-06-23-cyclone-auth-posture-alignment-design.md](superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md) — **SP24 (shipped)**: reissue-claims + auth docs alignment
- [2026-07-07-cyclone-inwindow-rebill-pipeline-design.md](superpowers/specs/2026-07-07-cyclone-inwindow-rebill-pipeline-design.md) — **SP41 (shipped)**: in-window rebill pipeline (`cyclone.rebill.*`)
- [2026-07-08-cyclone-doc-pass-design.md](superpowers/specs/2026-07-08-cyclone-doc-pass-design.md) — **SP42 (in flight)**: this doc-pass (post-SP23–SP41 reconciliation)
### 12.2 Implementation plans
@@ -713,8 +735,11 @@ All under [`docs/superpowers/plans/`](superpowers/plans/):
- [2026-06-21-cyclone-parse-decide-workflow.md](superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md) — SP22
- [2026-06-21-cyclone-pipeline-agent.md](superpowers/plans/2026-06-21-cyclone-pipeline-agent.md) — SP22
- [2026-06-21-cyclone-skill-catalog.md](superpowers/plans/2026-06-21-cyclone-skill-catalog.md) — documentation meta-project
- [2026-06-21-cyclone-store-split.md](superpowers/plans/2026-06-21-cyclone-store-split.md) — SP21 (split, in-flight)
- [2026-06-21-cyclone-store-split.md](superpowers/plans/2026-06-21-cyclone-store-split.md) — SP21 (split, shipped)
- [2026-06-21-cyclone-universal-drilldown.md](superpowers/plans/2026-06-21-cyclone-universal-drilldown.md) — SP21 (drilldown)
- [2026-06-23-cyclone-auth-posture-alignment.md](superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md) — SP24 (reissue-claims + auth docs alignment)
- [2026-07-07-cyclone-inwindow-rebill-pipeline.md](superpowers/plans/2026-07-07-cyclone-inwindow-rebill-pipeline.md) — SP41 (in-window rebill pipeline)
- [2026-07-08-cyclone-doc-pass.md](superpowers/plans/2026-07-08-cyclone-doc-pass.md) — SP42 (this doc-pass)
### 12.3 Reviews and reference
@@ -737,13 +762,13 @@ All under [`docs/superpowers/plans/`](superpowers/plans/):
For the two independent reviewers at the end of round 2 (re-verifying the corrections):
1. **Components** — Do §5.1 (backend modules) + §5.1 (frontend dirs) list every existing module that ships PHI or serves a state-changing route? Are the boundaries (DB / ORM / reconcile / store / api / pubsub / clearhouse / batch_diff / edi.filenames) drawn correctly? Can you, from §5.1 alone, draw the import graph?
2. **Data model** — Are all 12 migrations listed in §5.2 with their purpose? Are the two worktree-only migrations (0013, 0014) flagged as in-flight? Are all 18 ORM models in §5.2? Is the `db_backups` (not `backups`) correction applied? Is the `Rejection` phantom removed? Is the `ActivityEvent` vs `AuditLog` distinction clear (NFR-3 vs NFR-4)?
3. **Dependencies** — Are §5.3 runtime + tooling deps correct (no missing `pyyaml`, no missing `keyring`, no `httpx` listed as runtime, `paramiko` flagged as `[sftp]` extra, `sqlcipher3` flagged as `[sqlcipher]` extra, frontend test env is `happy-dom` not `jsdom`, vitest pinned at `^4.1.9` flagged as unusual)? Are OS-level deps (Keychain via `keyring`, SQLCipher C library, `tini`, `nginx` if SP23 ships) called out?
4. **Definition of Done** — Is §8 testable? Project DoD #6 now points at the reviewer-agreement meta-criterion. Per-release + per-task DoDs each independently checkable? FR-7 / FR-33 traceability rows corrected to point at existing test files and the correct endpoint name?
5. **Traceability** — Pick any FR-NN in §3. Can you trace it to an SP, a spec on disk (or an honest "no dedicated spec on disk" note), a plan on disk (or an honest "no plan file on disk; feat(spN) commits substitute" note), and at least one test file in `backend/tests/` or `src/`? If not, that's a remaining gap.
6. **Assumptions** — Are §10 assumptions falsifiable? Could a new engineer challenge any of them and find the test that proves/disproves them? Is A2's reference to `keyring` accurate now?
7. **Risks** — Are §11 risks current (R-11 corrected to 2,172; R-12 added; R-25/R-26/R-27 added)? Compare against `docs/reviews/2026-06-20-cyclone-completeness-review.md`. Is R-24 (SP23 fork) clearly flagged as awaiting user decision?
1. **Components** — Do §5.1 (backend modules) + §5.1 (frontend dirs) list every existing module that ships PHI or serves a state-changing route? Are the boundaries (DB / ORM / reconcile / store / api / pubsub / clearhouse / batch_diff / edi.filenames / auth / handlers / submission / rebill / reissue / store/) drawn correctly? Can you, from §5.1 alone, draw the import graph?
2. **Data model** — Are all 23 migrations listed in §5.2 with their purpose (no worktree-only entries)? Are all 18 ORM models in §5.2? Is the `db_backups` (not `backups`) correction applied? Is the `ActivityEvent` vs `AuditLog` distinction clear (NFR-3 vs NFR-4)?
3. **Dependencies** — Are §5.3 runtime + tooling deps correct (no missing `pyyaml`, no missing `keyring`, no `httpx` listed as runtime, `paramiko` flagged as `[sftp]` extra, `sqlcipher3` flagged as `[sqlcipher]` extra, frontend test env is `happy-dom` not `jsdom`, vitest pinned at `^4.1.9` flagged as unusual)? Are OS-level deps (Keychain via `keyring`, SQLCipher C library, `tini`, `nginx` — SP23 shipped) called out?
4. **Definition of Done** — Is §8 testable? Project DoD #6 now points at the reviewer-agreement meta-criterion. Per-release + per-task DoDs each independently checkable? FR-7 / FR-33 / FR-35 / FR-36 traceability rows corrected to point at existing test files and the correct endpoint names?
5. **Traceability** — Pick any FR-NN in §3. Can you trace it to an SP, a spec on disk (or an honest "no dedicated spec on disk" note), a plan on disk (or an honest "no plan file on disk; feat(spN) commits substitute" note), and at least one test file in `backend/tests/` or `src/`? If not, that's a remaining gap. Note FR-RB-1..FR-RB-6 (SP41) and the auth-related rows point at the new subpackages.
6. **Assumptions** — Are §10 assumptions falsifiable? Could a new engineer challenge any of them and find the test that proves/disproves them? Is A2's reference to `keyring` accurate now (Linux dispatch to Secret Service / kwallet, Windows via Credential Manager)?
7. **Risks** — Are §11 risks current (R-1/R-3/R-11/R-12/R-24 closed by SP23/SP21/SP36; R-28 added for test-density drift)? Compare against `docs/reviews/2026-06-20-cyclone-completeness-review.md`.
If reviewers materially agree on all seven, round 2 closes; this doc becomes the project's new top-level entry point (linked from `README.md`).