From 2bb10f2a39997df57a9a27aad308643bf3078949 Mon Sep 17 00:00:00 2001 From: Nora Date: Wed, 8 Jul 2026 23:13:02 -0600 Subject: [PATCH] =?UTF-8?q?docs(sp42):=20REQUIREMENTS.md=20=E2=80=94=20ref?= =?UTF-8?q?resh=20store/api=20LOC,=20migrations=2012=E2=86=9223,=20SP23=20?= =?UTF-8?q?shipped,=20FR-35/36=20met,=20risks=20closed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/REQUIREMENTS.md | 127 ++++++++++++++++++++++++++----------------- 1 file changed, 76 insertions(+), 51 deletions(-) diff --git a/docs/REQUIREMENTS.md b/docs/REQUIREMENTS.md index 85a900f..2033521 100644 --- a/docs/REQUIREMENTS.md +++ b/docs/REQUIREMENTS.md @@ -25,7 +25,7 @@ Every requirement (§3, §4) has an ID; every ID is traceable to a sub-project (§6) which in turn points at a design spec and an implementation plan. The matrix at the end of §6 is the single index that ties everything together. -**Round history.** Round 1 (2026-06-23) produced this draft. Two independent reviewers (Reviewer A — components/data-model/deps/DoD; Reviewer B — traceability/consistency audit) read it against the actual codebase and surfaced 12 factual errors and 6 structural improvements. Round 2 (this version) applies every factual correction. Round 3 (pending) will re-verify reviewer agreement on the corrected doc. +**Round history.** Round 1 (2026-06-23) produced this draft. Two independent reviewers (Reviewer A — components/data-model/deps/DoD; Reviewer B — traceability/consistency audit) read it against the actual codebase and surfaced 12 factual errors and 6 structural improvements. Round 2 (this version, dated 2026-07-08) applies every factual correction through SP41, including the post-SP23 doc alignment (LAN-bind + auth posture) and the SP21 store split + SP36 api split + SP41 rebill additions. Round 3 (pending) will re-verify reviewer agreement on the corrected doc. --- @@ -33,18 +33,18 @@ Every requirement (§3, §4) has an ID; every ID is traceable to a sub-project ( A self-hosted EDI claims-management suite for a single billing office. Parses 837P professional claims and 835 ERA remittances (X12 005010X222A1 and 005010X221A1), with a local-only FastAPI backend and a React UI for browsing, filtering, and inspecting the parsed data. -- **Local-only on purpose:** binds `127.0.0.1`, requires login (auth boundary is HTTP; bcrypt + HttpOnly session cookie), no internet exposure, single operator, single machine, one trading partner (Colorado Medicaid, currently). +- **LAN-only by design:** binds `0.0.0.0:8000`, requires login (auth boundary is HTTP; bcrypt + HttpOnly session cookie; first admin bootstrapped from `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD` env vars), reachability controlled by the host firewall / compose port publishing, single operator, one trading partner (Colorado Medicaid, currently). The dev/test escape hatch is `CYCLONE_AUTH_DISABLED=1`, which logs a WARNING at boot. - **Single-process:** the backend is one Python process; the frontend is one Vite-served React app; there is no message broker, no separate worker, no queue. - **Single source of truth:** SQLite at `~/.local/share/cyclone/cyclone.db` (overridable via `CYCLONE_DB_URL`); optionally encrypted at rest via SQLCipher (SP12). -- **Threat model:** a process on the same host, plus a deliberate HTTP-layer login (no remote unauthenticated read or write). File-system threats (stolen/imaged drive) are still the primary concern; SQLCipher at rest + the macOS Keychain handle those. The SP23 product fork changes the threat model to "remote operator on LAN." +- **Threat model:** a process on the same host, plus a deliberate HTTP-layer login (no remote unauthenticated read or write). File-system threats (stolen/imaged drive) are still the primary concern; SQLCipher at rest + the macOS Keychain handle those. SP23 expanded the threat model to "remote operator on LAN" by adding login + RBAC + Docker + Ubuntu packaging; the LAN-bind posture remains. -The Cyclone codebase has honored this contract end-to-end across 22 shipped sub-projects. Every design spec re-asserts the local-only / single-operator / single-payer / no-auth posture and lists (often at length) what is intentionally out of scope. +The Cyclone codebase has honored this contract end-to-end across 41 shipped sub-projects. Every design spec re-asserts the LAN-only / single-operator / single-payer posture and lists (often at length) what is intentionally out of scope. --- ## 2. Scope boundaries -### 2.1 In scope (today, after SP1–SP22) +### 2.1 In scope (today, after SP1–SP41) | Capability | SP | |---|---| @@ -69,9 +69,16 @@ The Cyclone codebase has honored this contract end-to-end across 22 shipped sub- | Structured JSON logging + PII scrubber | SP18 | | Security hardening (body size limit, rate limit, security headers) | SP19 | | NPI Luhn checksum + Tax ID format validation | SP20 | -| CycloneStore split (refactor; **in-flight on `refactor/store-split`**) | SP21 | +| CycloneStore split (16-module subpackage; public API preserved via `cyclone.store` facade) | SP21 | | Parse-then-decide upload dedup (idempotency) | SP22 | | Pipeline automation agent (sibling project at `cyclone-pipeline/`) | SP22 | +| HTTP-layer auth (bcrypt + HttpOnly session cookie) + RBAC roles + `CYCLONE_AUTH_DISABLED` escape hatch | SP23 | +| Ubuntu + Docker + nginx + tini packaging | SP23 | +| Audit-log `user_id` (every audit event records the acting user) | SP-auth | +| `submission_dedup` table + claim-id dedup at SFTP pre-flight | SP22 + SP41 | +| Resubmissions table + bulk ingest helpers | SP21 (resubmissions) | +| In-window rebill pipeline (visit↔835 reconciliation + Pipeline A/B + summary CSV) | SP41 | +| Live-tail NDJSON streams for `/api/acks/stream` and `/api/ta1-acks/stream` | SP-acks-tail | ### 2.2 Out of scope (today, by design) @@ -140,8 +147,8 @@ Each requirement is `FR-NN`, traceable to one or more sub-projects. Verification | FR-32 | Reject oversize request bodies (413), rate-limit requests (429), and add default security headers via pure-ASGI middlewares; emit a tamper-evident `api.request_rejected` audit event on 413/429. | SP19 | | FR-33 | Surface a rich `GET /api/health` response — DB connectivity, MFT scheduler state, backup scheduler state, live pubsub subscriber counts, last batch id + timestamp. | SP19 (`api_routers/health.py:28-40`) | | FR-34 | Validate NPI with the CMS-published Luhn over `80840 + body` (warning-only — placeholder NPIs in test fixtures shouldn't block ingest). Reject Tax ID (EIN) with reserved prefixes (`00`, `07`, `80`–`89`). | SP20 | -| FR-35 | Split `CycloneStore` into per-concern modules (`store_persistence`, `store_reconcile`, `store_queries`, `store_mappers`) without changing the public store API. | SP21 (**in-flight** on `refactor/store-split` branch) | -| FR-36 | De-duplicate 837 / 835 uploads via a parse-then-decide workflow (`find_existing_batch_for_claim` / `find_existing_batch_for_remit`); relax the claims/remits PK to `(batch_id, id)` via migration 0014 to support re-parses into different batches. | SP22 (migration 0014 in worktree; not yet on `main`) | +| FR-35 | Split `CycloneStore` into per-concern modules under `cyclone/store/` (16 modules, ~5,400 LOC total) without changing the public store API; the `cyclone.store` import surface still resolves to the same names. | SP21 (**Met**; merged on `main`; module list in §5.1) | +| FR-36 | De-duplicate 837 / 835 uploads via a parse-then-decide workflow (`find_existing_batch_for_claim` / `find_existing_batch_for_remit`); relax the claims unique constraint via migration 0015 (a defensive drop of an inline UNIQUE that was a no-op against the current schema) to support re-parses into different batches. | SP22 (**Met**; migration 0015 on `main`) | | FR-37 | (Sibling project, not in this repo) Drive the full 7-phase round-trip — preflight → browser upload → parse verify → SFTP submit → TA1 wait → 999 wait → scan + report — with crash-safe resume, structured JSON logging, idempotency dedup, per-run folder. | SP22 (`cyclone-pipeline/`) | | FR-38 | Provide a `DrillStackProvider` + `DrillDrawerHeader` shell so every drillable cell across the app opens a consistent drawer or peek modal. | SP21 | @@ -164,7 +171,7 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re | ID | NFR | Source / SP | |---|---|---| -| NFR-1 | **Always bind 0.0.0.0 + auth.** Backend binds `0.0.0.0:8000` (overridable via `CYCLONE_HOST` / `CYCLONE_PORT`); CORS allowlist is exact (`http://localhost:5173`); login required (bcrypt + HttpOnly session cookie; first admin bootstrapped from env vars `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD`); dev/test escape hatch via `CYCLONE_AUTH_DISABLED=1` (logs WARNING at boot). Reachability is controlled by the host firewall / compose port publishing, not the bind address. | SP1 + auth (2026-06-23 merge) + SP24 (doc reconciliation) | +| NFR-1 | **Always bind 0.0.0.0 + auth.** Backend binds `0.0.0.0:8000` (overridable via `CYCLONE_HOST` / `CYCLONE_PORT`); CORS allowlist is exact (`http://localhost:5173`); login required (bcrypt + HttpOnly session cookie; first admin bootstrapped from env vars `CYCLONE_ADMIN_USERNAME` + `CYCLONE_ADMIN_PASSWORD`); every API router declares `Depends(matrix_gate)` (`cyclone/auth/deps.py:matrix_gate`) so requests without a valid session are rejected; dev/test escape hatch via `CYCLONE_AUTH_DISABLED=1` (logs WARNING at boot). Reachability is controlled by the host firewall / compose port publishing, not the bind address. | SP1 + SP23 (auth + RBAC) | | NFR-2 | **Determinism.** Parser / serializer round-trip is guaranteed on 113 real prodfiles (`docs/prodfiles/claims/*.x12`); canonical fields, not byte-identity (called out in the SP8 spec). | SP8 | | NFR-3 | **Audit completeness.** Every reconciliation anomaly, every state transition, every 999/277CA reject writes an `ActivityEvent` (in `activity_events` table, un-chained; powers the `/activity` feed and inbox state transitions). | SP2 + SP10 | | NFR-4 | **Tamper-evidence.** A separate `audit_log` table (SP11) carries SHA-256 hash-chained rows for security-sensitive events (login attempts, rejections, key rotations, backup lifecycle, rejected requests). `GET /api/admin/audit-log/verify` detects any break. **The `activity_events` (NFR-3) and `audit_log` (NFR-4) tables are distinct** — different semantics, different audiences. New code that needs an audit row must decide which one to write to. | SP11 | @@ -178,7 +185,7 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re | NFR-12 | **Idempotency.** Inbound MFT files are de-duplicated via `processed_inbound_files`; 837/835 uploads are de-duplicated via the parse-then-decide workflow with `(batch_id, claim_id)` / `(batch_id, remit_id)` relaxed PKs. | SP16 + SP22 | | NFR-13 | **Crash-safety.** Incoming files are processed inside per-file try/except so a bad file doesn't stop the MFT loop; the pipeline agent has crash-safe resume keyed by run id. | SP16 + SP22 | | NFR-14 | **Single-process.** No message broker, no separate worker, no queue; everything runs in one Python process. | (project-wide constraint) | -| NFR-15 | **Test density.** Backend tests: **964 collected** (`pytest --collect-only`, 2026-06-23); frontend test files: **73**; prodfiles are exercised (113 + 19 + 1369), not just minimal fixtures. | (project-wide) | +| NFR-15 | **Test density.** Backend tests: **1,624 collected** (`pytest --collect-only`, 2026-07-08); frontend test files: **85**; prodfiles are exercised (113 + 19 + 1369), not just minimal fixtures. | (project-wide) | | NFR-16 | **Documentation discipline.** Every shipped sub-project has a design spec + implementation plan + smoke test; a new engineer can read the project top-to-bottom. (SPs 9–16 ship without on-disk plan files — see §6.1 — but their `feat(spN)` commits are on `main` and carry their own commit-message plan.) | (project-wide) | | NFR-17 | **Fail-soft posture.** Reconciliation crashes don't lose the 835; the activity event records the failure. | SP2 | | NFR-18 | **Migration safety.** Migrations are forward-only via `PRAGMA user_version`; rollback procedures are documented per migration; the `user_version` runner is idempotent. **No checksum manifest exists today** (R-13 in §11). | SP2 + SP22 | @@ -193,14 +200,16 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re | Module | Owns | Notes | |---|---|---| -| `cyclone.api` | HTTP routes, content negotiation, CORS allowlist, lifespan | 3,145 LOC (largest single file; partly split into `api_routers/` for 4 sub-routes) | +| `cyclone.api` | HTTP routes, content negotiation, CORS allowlist, lifespan | 378 LOC; routes are mounted from `api_routers/` (22 sub-routers — see below) | | `cyclone.api_helpers` | NDJSON / content-negotiation / live-tail helpers | | -| `cyclone.api_routers` | `acks`, `admin`, `health` (`/api/health`), `ta1_acks` sub-routers | Mounted at `api.py:270-273`; only 4 routes split out — bulk of routes still in `api.py` | -| `cyclone.store` | Public store facade; persistence, reconciliation, queries, mappers | 2,172 LOC; SP21 split is in-flight on `refactor/store-split` | +| `cyclone.api_routers` | Per-resource sub-routers mounted under `/api/...` | 22 modules: `acks`, `activity`, `admin`, `batches`, `claim_acks`, `claims`, `clearhouse`, `config`, `dashboard`, `eligibility`, `health`, `inbox`, `parse`, `payers`, `providers`, `rebill`, `reconciliation`, `remittances`, `submission`, `ta1_acks` (+ `_shared.py`). Every router declares `Depends(matrix_gate)` for auth. | +| `cyclone.auth` | Login, logout, session cookie, RBAC roles, `matrix_gate` dependency | New in SP23; password hashing, session table, user roles | +| `cyclone.handlers` | Shared FastAPI handlers (auth-aware JSON responses, audit-log injection, etc.) | Cross-cutting request glue | +| `cyclone.store` (subpackage) | Public store facade; the split modules behind it | 16 modules (`__init__` facade + `acks`, `backfill`, `backups`, `batches`, `claim_acks`, `claim_detail`, `exceptions`, `inbox`, `kpis`, `orm_builders`, `providers`, `records`, `resubmissions`, `submission_dedup`, `ui`, `write`); public API re-exported from `cyclone.store` so callers see no breaking change | | `cyclone.db` | SQLAlchemy engine, session factory, ORM models (18 tables) | | | `cyclone.db_migrate` | `PRAGMA user_version` migration runner | | | `cyclone.db_crypto` | Optional SQLCipher encryption at rest; key rotation | `rotate_key()` is destructive (rewrites every page) | -| `cyclone.audit_log` | Hash-chained `audit_log` (SP11, SHA-256 chain) | Distinct from `cyclone.activity_events` written by other modules | +| `cyclone.audit_log` | Hash-chained `audit_log` (SP11, SHA-256 chain) | Distinct from `cyclone.activity_events` written by other modules; post-SP23 records `user_id` of the actor | | `cyclone.inbox_lanes` | The 5-lane inbox payload (Rejected / Payer-rejected / Candidates / Unmatched / Done today) | | | `cyclone.inbox_state` | 999 envelope reject → claim state transitions | | | `cyclone.inbox_state_277ca` | 277CA STC A4/A6/A7 → payer_rejected stamp (monotonic) | | @@ -210,16 +219,19 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re | `cyclone.reconcile` | Pure-function 835→claim match + line-level match | No DB session inside; testable with fabricated ORM objects | | `cyclone.scoring` | 4-field weighted candidate scoring (40/25/20/15) | Weights hardcoded — see R-10 in §11 | | `cyclone.pubsub` | In-process EventBus (drop-oldest, per-kind fan-out) | In-process only; not thread-safe across asyncio event loops | -| `cyclone.backup` | AES-256-GCM encrypted backups (orchestration + CLI) | 216 LOC | -| `cyclone.backup_service` | Backup lifecycle (create / verify / restore) | 740 LOC | -| `cyclone.backup_scheduler` | 24h backup autostart loop | 315 LOC | +| `cyclone.backup` | AES-256-GCM encrypted backups (orchestration + CLI) | | +| `cyclone.backup_service` | Backup lifecycle (create / verify / restore) | | +| `cyclone.backup_scheduler` | 24h backup autostart loop | | | `cyclone.scheduler` | asyncio MFT polling loop (SP16) | Uses `processed_inbound_files` for idempotency | | `cyclone.security` | Per-IP rate-limit, body-size limit, security headers (`SecurityHeadersMiddleware`, `BodySizeLimitMiddleware`, `RateLimitMiddleware`) | `EXEMPT_PATHS` list at `:201` | | `cyclone.logging_config` | JSON formatter, PII scrubber | | | `cyclone.npi` | NPI Luhn + EIN format validators | | | `cyclone.batch_diff` | Batch Diff engine (powers `GET /api/batch-diff`) | 12 public functions/classes | -| `cyclone.clearhouse` (`__init__.py`) | `SftpClient` (SP9 stub + SP13 paramiko), `InboundFile`, `make_client()` factory | 318 LOC; the SFTP home | +| `cyclone.clearhouse` (`__init__.py`) | `SftpClient` (SP9 stub + SP13 paramiko), `InboundFile`, `make_client()` factory | The SFTP home | | `cyclone.edi` (`filenames.py`) | HCPF X12 File Naming Standards helpers | Used by SP13/SP16 outbound filename construction | +| `cyclone.rebill` | In-window rebill pipeline: visit↔835 reconciliation, CARC filter, timely-filing gate, Pipeline A/B, summary CSV | New in SP41 | +| `cyclone.reissue` | Reissue-claims pipeline (revised-claim regeneration + RUNBOOK entry) | SP24 | +| `cyclone.submission` | Canonical `submit_file` helper shared by `cyclone submit-batch` CLI and `POST /api/submit-batch`; parse → DB-write → SFTP-upload → audit per file | SP37 | | `cyclone.cli` | Click CLI | | | `cyclone.__main__` | `python -m cyclone serve` | | | `cyclone.parsers/` | X12 tokenizer, models, validator, writers, 277CA, 999, TA1, 270, 271, CAS, seg | 25 `.py` modules | @@ -230,7 +242,7 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re | Directory | Owns | |---|---| -| `src/pages/` | Acks, ActivityLog, BatchDiff, Batches, Claims, Dashboard, Inbox, Providers, Reconciliation, Remittances, Upload (11 pages) | +| `src/pages/` | Acks, ActivityLog, BatchDiff, Batches, Claims, Dashboard, Inbox, Login, Providers, Reconciliation, Remittances, Upload (12 pages — Login added in SP23) | | `src/hooks/` | TanStack Query hooks, URL-state hooks, live-tail hooks, keyboard hooks, batch-export, search, drawer keyboard, count-up, provider detail, payer summary, parse, ack detail (35+ files) | | `src/lib/` | `api.ts`, `format.ts`, `utils.ts`, `csv.ts`, `download.ts`, `event-routing.ts`, `tail-stream.ts`, `inbox-api.ts` (all with paired `.test.ts`) | | `src/components/` | 3 drawer families (`ClaimDrawer/` ~15 files, `RemitDrawer/` ~10 files, `ProviderDrawer/` ~6, `AckDrawer/` 3); `inbox/` (10 files), `drill/` (10 files, includes `DrillStackProvider` + `DrillDrawerHeader`), `charts/` (4 files), `ui/` (18 shadcn primitives — radix + cva + tailwind-merge), `KeyboardCheatsheet/` (3 files), plus top-level ActivityFeed, AnimatedNumber, BatchDetail, BatchDiffView, BatchesList, ClaimCard837, DominantKpiCard, EditorialNote, ExportBar, ExportCsvButton, KpiCard, Layout, NewClaimDialog, PageHeader, SearchBar, SearchResults, Sidebar, Sparkline, StatusBadge, StatusPill, TailStatusPill, TickerTape | @@ -239,7 +251,7 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re ### 5.2 Data model -**12 SQL migrations** in `backend/src/cyclone/migrations/` (verified on disk 2026-06-23): +**23 SQL migrations** in `backend/src/cyclone/migrations/` (verified on disk 2026-07-08): | # | File | Tables / columns added | |---|---|---| @@ -255,13 +267,17 @@ The `FR-RB-*` IDs are SP41-specific functional requirements for the in-window re | 0010 | `0010_payer_rejected_acknowledged.sql` | `payer_rejected_acknowledged` column on `claims` | | 0011 | `0011_processed_inbound_files.sql` | `processed_inbound_files` (SP16 idempotency) | | 0012 | `0012_backups.sql` | `db_backups` (SP17) | - -**In-flight on `claims-unique-fix` worktree (not yet on `main`):** - -| # | File | Tables / columns added | -|---|---|---| -| 0013 | `0013_drop_claims_unique_constraint.sql` | Drop `claims` UNIQUE constraint | -| 0014 | `0014_relax_claims_remits_pk.sql` | Relax PK to `(batch_id, id)` (SP22) | +| 0013 | `0013_auth_users_and_sessions.sql` | `users`, `sessions`, RBAC role columns (SP23 auth) | +| 0014 | `0014_audit_log_user_id.sql` | `audit_log.user_id` column + index (records acting user on every audit event) | +| 0015 | `0015_drop_claims_unique_constraint.sql` | Defensive drop of the inline `UNIQUE(batch_id, patient_control_number)` on `claims` (SP22; table-recreate dance — no-op against current schema) | +| 0016 | `0016_claims_matched_remittance_id_index.sql` | Index on `claims.matched_remittance_id` to speed inbox joins | +| 0017 | `0017_backfill_claim_patient_control_number.sql` | Backfill `patient_control_number` from X12 CLM01 for legacy rows | +| 0018 | `0018_claim_acks.sql` | `claim_acks` table (claim-level ACK metadata) | +| 0019 | `0019_add_rendering_and_service_provider_npis.sql` | Rendering + service-provider NPI columns on `claims` | +| 0020 | `0020_add_batch_txn_set_control_number.sql` | `batches.transaction_set_control_number` (ST/SE control number persistence) | +| 0021 | `0021_resubmissions.sql` | `resubmissions` table (resubmission tracking + bulk-ingest helpers) | +| 0022 | `0022_submission_dedup.sql` | `submission_dedup` table (claim-id dedup window at SFTP pre-flight, SP41) | +| 0023 | `0023_visits.sql` | `visits` table (SP41 in-window rebill pipeline input) | **ORM models** (declared in `backend/src/cyclone/db.py` — 18 classes total): @@ -417,18 +433,20 @@ Each row's "Spec" / "Plan" point to the design spec and implementation plan on d | SP18 | Structured logging | [spec](superpowers/specs/2026-06-21-cyclone-structured-logging-design.md) | (no plan file on disk; feat(sp18) commits on `main`) | Met | | SP19 | Security hardening + health probe | [spec](superpowers/specs/2026-06-21-cyclone-security-hardening-design.md) | (no plan file on disk; feat(sp19) commits on `main`) | Met | | SP20 | NPI checksum + Tax ID format validation | [spec](superpowers/specs/2026-06-21-cyclone-npi-validation-design.md) | (no plan file on disk; feat(sp20) commits on `main`) | Met | -| SP21 | CycloneStore split (in-flight) + Universal drilldown (shipped) | [split spec](superpowers/specs/2026-06-21-cyclone-store-split-design.md), [drilldown spec](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) | [store-split plan](superpowers/plans/2026-06-21-cyclone-store-split.md), [drilldown plan](superpowers/plans/2026-06-21-cyclone-universal-drilldown.md) | Drilldown **Met**; split **In-flight** (`refactor/store-split` branch, not yet on `main`) | +| SP21 | CycloneStore split (16-module subpackage) + Universal drilldown | [split spec](superpowers/specs/2026-06-21-cyclone-store-split-design.md), [drilldown spec](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) | [store-split plan](superpowers/plans/2026-06-21-cyclone-store-split.md), [drilldown plan](superpowers/plans/2026-06-21-cyclone-universal-drilldown.md) | Met (split landed on `main`; public API re-exported from `cyclone.store` facade; drilldown used by every drawer) | | SP22 | Parse-decide workflow + Pipeline agent | [parse-decide spec](superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md), [pipeline spec](superpowers/specs/2026-06-21-cyclone-pipeline-agent-design.md) | [parse-decide plan](superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md), [pipeline plan](superpowers/plans/2026-06-21-cyclone-pipeline-agent.md) | Met (parse-decide on `main`; pipeline agent lives in `cyclone-pipeline/`) | -| SP24 | Auth posture alignment (docs-only reconciliation with the auth work in `main`) | [spec](superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md) | [plan](superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md) | Met (no code paths change; only docs + `__main__.py` WARNING + 3 skills addenda) | +| SP23 | Ubuntu + Docker + nginx + tini + auth + RBAC (LAN-bind product fork) | [spec](superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md) | (plan in the same docs directory; auth is the dominant change) | Met (merges `cyclone.auth.*`, every API router gated by `matrix_gate`, Login page added to the React app) | +| SP24 | Reissue-claims (revised-claim regeneration + deprecation shim) | [spec](superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md) (docs alignment + reissue) | [plan](superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md) | Met (merge commit `c9588f7`; `cyclone.reissue` shipped) | +| SP25–SP40 | Sub-projects landed between SP24 and SP41 (covering resubmissions, `claim_acks`, rendering/service NPI columns, transaction-set control numbers, additional live-tail streams, dashboards, rebill prerequisites) | (per-SP design specs + plans under `docs/superpowers/`) | (per-SP plans under `docs/superpowers/plans/`) | Met — see §6.1 acceptance table below | +| SP41 | In-window rebill pipeline (`cyclone.rebill.*`) | [spec](superpowers/specs/2026-07-07-cyclone-inwindow-rebill-pipeline-design.md) | [plan](superpowers/plans/2026-07-07-cyclone-inwindow-rebill-pipeline.md) | Met (merge commit `309e6c0`; visit↔835 reconciliation, Pipeline A/B, summary CSV, claim-id dedup at SFTP pre-flight) | ### 6.2 Backlog (specs awaiting plans, or vice versa) | Item | Status | Why | |---|---|---| -| SP23 (candidate) | Spec-only; **product fork awaiting user decision** | `2026-06-22-cyclone-ubuntu-docker-deployment-design.md` (auth + Docker + RBAC + LAN-bind); see §11 R-1, R-3 | | Plan-file backfill for SP9, SP10, SP11, SP12, SP13, SP14, SP15, SP16, SP17, SP18, SP19, SP20 | Plan files missing on disk; `feat(spN)` commit messages substitute | Round-3 doc-backlog item | | Spec-file backfill for SP6 (workflow/inbox), SP11 (audit log), SP12 (SQLCipher), SP13 (paramiko), SP15 (key rotation), SP16 (MFT scheduler) | Specs missing on disk | Round-3 doc-backlog item | -| Migration manifest (checksums for the 12 SQL files) | Open | Completeness review §3.1.18 | +| Migration manifest (checksums for the 23 SQL files) | Open | Completeness review §3.1.18 | | Tunable score weights (move 40/25/20/15 from code to config) | Open | Completeness review §3.1.15 | | License file (`LICENSE`) | Open | Completeness review §3.2.28 | | `pre-commit` / `Makefile` / `CONTRIBUTING.md` / `.editorconfig` / `ruff` config | Open | Completeness review §3.2.30 | @@ -482,8 +500,8 @@ This matrix is the single index. Each row points to a requirement and where it's | FR-32 | SP19 | security-hardening | (none on disk) | `test_security.py` | | FR-33 | SP19 | security-hardening | (none on disk) | `test_api.py:33 test_health_endpoint`, `test_security.py:204-221` (health snapshot) | | FR-34 | SP20 | npi-validation | (none on disk) | `test_npi.py`, `test_api_validate_provider.py`, `test_cli_validate.py` | -| FR-35 | SP21 | store-split | store-split | `test_store.py` (currently covers the un-split `store.py`; will cover the split modules once `refactor/store-split` lands) | -| FR-36 | SP22 | parse-decide-workflow | parse-decide-workflow | `test_api_parse_persists.py`, `test_prodfiles_smoke.py`; migration 0014 in `claims-unique-fix` worktree | +| FR-35 | SP21 | store-split | store-split | `tests/store/test_*.py` (per-module coverage of the split subpackage); `test_store.py` continues to exercise the public `cyclone.store` facade | +| FR-36 | SP22 | parse-decide-workflow | parse-decide-workflow | `test_api_parse_persists.py`, `test_prodfiles_smoke.py`, migration 0015 (defensive drop of inline UNIQUE on `claims(batch_id, patient_control_number)`) | | FR-37 | SP22 | pipeline-agent | pipeline-agent | (lives in `cyclone-pipeline/`, not this repo) | | FR-38 | SP21 | universal-drilldown | universal-drilldown | `DrillStackProvider.test.tsx`, `useDrawerUrlState.test.ts`, plus per-drawer tests in `ClaimDrawer/`, `RemitDrawer/`, `ProviderDrawer/`, `AckDrawer/` | @@ -588,12 +606,12 @@ These are the existing per-SP acceptance checklists as captured in their design ```bash # Backend cd backend -.venv/bin/pytest # 964 tests collected (2026-06-23) +.venv/bin/pytest # 1,624 tests collected (2026-07-08) .venv/bin/python -m cyclone serve # Frontend npm run typecheck -npm test # 73 test files +npm test # 85 test files npm run build # Smoke (planned in SP23) @@ -628,7 +646,7 @@ Recorded explicitly so a new engineer can challenge any of them. | A8 | The X12 5010 transaction set Cyclone supports is sufficient for the operator's billing workflow. | Adding 837I / 837D / 278 / 276-277 / 277CA-round-trip / etc. is each a new SP. | | A9 | Local-only bind is sufficient — no remote access needed. | Would require auth (SP23), TLS, reverse proxy, RBAC. | | A10 | One Python process, no message broker, no separate worker. | A multi-worker deployment would require breaking thread-affinity (SP12/SP15 specifically). | -| A11 | 964 backend tests + 73 frontend test files is the right test density for this codebase (2026-06-23). If we add more transaction types or surface area, this needs to grow proportionally; coverage gates should be enforced (round-3 backlog). | New transaction types or surface area without test growth → coverage drifts down. | +| A11 | 1,624 backend tests + 85 frontend test files is the right test density for this codebase (2026-07-08). If we add more transaction types or surface area, this needs to grow proportionally; coverage gates should be enforced (round-3 backlog). | New transaction types or surface area without test growth → coverage drifts down. | | A12 | `cyclone-pipeline/` is a sibling project, not a sub-project of this repo. | The 7-phase round-trip agent is shipped in `cyclone-pipeline/`; if that project is abandoned, the manual operator workflow remains (upload via `/upload`, SFTP by hand, `check-835` not available). | --- @@ -639,9 +657,9 @@ Pulled forward from [`docs/reviews/2026-06-20-cyclone-completeness-review.md`](r | ID | Risk / gap | Source | Status | |---|---|---|---| -| R-1 | No app-layer auth (anyone on the host can read PHI). | Completeness review §3.1.4 + §3.2.25 | **Closed by SP24** — auth shipped via the merge of `origin/main` on 2026-06-23 (commits `a25504b`..`39ae988`, `cyclone.auth.*` + `matrix_gate` dependency on every router); SP24 reconciled the docs to match. SP23 still covers the LAN-bind / Docker / RBAC product fork. | +| R-1 | No app-layer auth (anyone on the host can read PHI). | Completeness review §3.1.4 + §3.2.25 | **Closed by SP23** — auth shipped via the merge of `origin/main` on 2026-06-23 (commits `a25504b`..`39ae988`, `cyclone.auth.*` + `matrix_gate` dependency on every router); `users` / `sessions` tables live in migration 0013; every audit event records the acting `user_id` (migration 0014). | | R-2 | SQLite plaintext if SQLCipher not enabled. | Completeness review §3.1.1 | **Partial** — SP12 + SP15 ship the encryption option, but it falls back to plain if the Keychain entry is missing. | -| R-3 | No Dockerfile / docker-compose.yml. | Completeness review §3.2.29 | **Open** — SP23 spec covers it but is awaiting user decision. | +| R-3 | No Dockerfile / docker-compose.yml. | Completeness review §3.2.29 | **Closed by SP23** — Docker packaging is part of the LAN-bind product fork (Ubuntu base, `tini`, nginx reverse proxy, `docker-compose.yml`). | | R-4 | PayerConfig still partly in code (`co_medicaid()` factory at `cyclone/parsers/payer.py:57-58`). | Completeness review §3.1.9 | **Open** — SP9 externalized config but the factory remains as a fallback for tests and default-payer selection. | | R-5 | No vocabulary tables (ICD-10 / HCPCS / NDC). | Completeness review §3.1.11 | **Open** — explicitly out of scope per A6. | | R-6 | No COB / secondary-claim generator. | Completeness review §3.1.12 | **Open** — explicitly out of scope per §2.2. | @@ -649,9 +667,9 @@ Pulled forward from [`docs/reviews/2026-06-20-cyclone-completeness-review.md`](r | R-8 | No NPPES NPI lookup. | Completeness review §3.1.10 | **Open** — only Luhn checksum (SP20). | | R-9 | No 837I / 837D / 834 / 820 / 275 / 278 / 276-277 transaction types. | Completeness review §2.1 | **Open** — explicitly out of scope per §2.2. | | R-10 | Score weights hardcoded (`cyclone/scoring.py:42-46`). | Completeness review §3.1.15 | **Open** — not scheduled. | -| R-11 | `cyclone/store.py` is a 2,172-line god-module. | Completeness review §3.1.17 | **In-flight** — SP21 spec + plan written; branch `refactor/store-split` exists; merge to `main` pending. | -| R-12 | `cyclone/api.py` is a 3,145-line god-module (larger than `store.py`). | Completeness review §3.1.19 | **Partial** — `api_routers/` exists for 4 sub-routes (acks, admin, health, ta1_acks); bulk of routes still in `api.py`. | -| R-13 | Migrations in flat dir without manifest. | Completeness review §3.1.18 | **Open** — no checksum list; 12 SQL files in `backend/src/cyclone/migrations/`. | +| R-11 | `cyclone/store.py` is a 2,172-line god-module. | Completeness review §3.1.17 | **Closed by SP21** — `cyclone.store` is now a 16-module subpackage (`acks`, `backfill`, `backups`, `batches`, `claim_acks`, `claim_detail`, `exceptions`, `inbox`, `kpis`, `orm_builders`, `providers`, `records`, `resubmissions`, `submission_dedup`, `ui`, `write` + `__init__` facade); public API re-exported from `cyclone.store` so callers see no breaking change. | +| R-12 | `cyclone/api.py` is a 3,145-line god-module (larger than `store.py`). | Completeness review §3.1.19 | **Closed by SP36** — `api.py` is now 378 LOC; 22 sub-routers live under `cyclone/api_routers/` (every router gated by `Depends(matrix_gate)`). | +| R-13 | Migrations in flat dir without manifest. | Completeness review §3.1.18 | **Open** — no checksum list; 23 SQL files in `backend/src/cyclone/migrations/`. | | R-14 | No `pre-commit` / `Makefile` / `CONTRIBUTING.md` / `.editorconfig` / `ruff` config. | Completeness review §3.2.30 | **Open** — single-operator project. | | R-15 | No license file (`LICENSE`). | Completeness review §3.2.28 | **Open** — not scheduled. | | R-16 | PHI fixtures not flagged as PHI. | Completeness review §3.2.23 | **Open** — not scheduled. | @@ -662,10 +680,11 @@ Pulled forward from [`docs/reviews/2026-06-20-cyclone-completeness-review.md`](r | R-21 | No structured logging (before SP18). | Completeness review §3.1.5 | **Closed by SP18**. | | R-22 | No backup automation (before SP17). | Completeness review §3.1.3 | **Closed by SP17**. | | R-23 | `ActivityEvent` not tamper-evident (before SP11). | Completeness review §3.1.2 | **Closed by SP11** — but note: the tampered table is `audit_log`, distinct from `activity_events` (see NFR-3 / NFR-4). | -| R-24 | SP23 product fork (auth + Docker + RBAC + LAN-bind). | §6.2 | **Awaiting user decision** — see §2.2 and §11 R-1. | +| R-24 | SP23 product fork (auth + Docker + RBAC + LAN-bind). | §6.1 | **Closed by SP23** — auth, RBAC, Docker, nginx, tini, LAN-bind all shipped on `main`. | | R-25 | `vitest@^4.1.9` is unusually new (current stable line is 1.x/2.x). | §5.3 | **Open** — verify the version pin is intentional before the next `npm install`; fallback to `^2.x` if accidental. | | R-26 | No CI workflow (`.github/workflows/`). | §8.1 #3 | **Open** — release gating is manual today. | | R-27 | `backend/src/cyclone/workflow/__pycache__/` is dead bytecode. | §5.1 | **Open** — recommend deletion. | +| R-28 | Test density drifted from 964 → 1,624 backend tests and 73 → 85 frontend test files. | §4 NFR-15 | **Open** — coverage gates (`--cov-fail-under=90`) still not enforced; counts tracked in NFR-15. | --- @@ -694,7 +713,10 @@ All under [`docs/superpowers/specs/`](superpowers/specs/): - [2026-06-21-cyclone-store-split-design.md](superpowers/specs/2026-06-21-cyclone-store-split-design.md) — SP21 (split) - [2026-06-21-cyclone-structured-logging-design.md](superpowers/specs/2026-06-21-cyclone-structured-logging-design.md) — SP18 - [2026-06-21-cyclone-universal-drilldown-design.md](superpowers/specs/2026-06-21-cyclone-universal-drilldown-design.md) — SP21 (drilldown) -- [2026-06-22-cyclone-ubuntu-docker-deployment-design.md](superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md) — **SP23 candidate; awaiting plan + user product-fork decision** +- [2026-06-22-cyclone-ubuntu-docker-deployment-design.md](superpowers/specs/2026-06-22-cyclone-ubuntu-docker-deployment-design.md) — **SP23 (shipped)**: Ubuntu + Docker + nginx + tini + auth + RBAC LAN-bind product fork +- [2026-06-23-cyclone-auth-posture-alignment-design.md](superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md) — **SP24 (shipped)**: reissue-claims + auth docs alignment +- [2026-07-07-cyclone-inwindow-rebill-pipeline-design.md](superpowers/specs/2026-07-07-cyclone-inwindow-rebill-pipeline-design.md) — **SP41 (shipped)**: in-window rebill pipeline (`cyclone.rebill.*`) +- [2026-07-08-cyclone-doc-pass-design.md](superpowers/specs/2026-07-08-cyclone-doc-pass-design.md) — **SP42 (in flight)**: this doc-pass (post-SP23–SP41 reconciliation) ### 12.2 Implementation plans @@ -713,8 +735,11 @@ All under [`docs/superpowers/plans/`](superpowers/plans/): - [2026-06-21-cyclone-parse-decide-workflow.md](superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md) — SP22 - [2026-06-21-cyclone-pipeline-agent.md](superpowers/plans/2026-06-21-cyclone-pipeline-agent.md) — SP22 - [2026-06-21-cyclone-skill-catalog.md](superpowers/plans/2026-06-21-cyclone-skill-catalog.md) — documentation meta-project -- [2026-06-21-cyclone-store-split.md](superpowers/plans/2026-06-21-cyclone-store-split.md) — SP21 (split, in-flight) +- [2026-06-21-cyclone-store-split.md](superpowers/plans/2026-06-21-cyclone-store-split.md) — SP21 (split, shipped) - [2026-06-21-cyclone-universal-drilldown.md](superpowers/plans/2026-06-21-cyclone-universal-drilldown.md) — SP21 (drilldown) +- [2026-06-23-cyclone-auth-posture-alignment.md](superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md) — SP24 (reissue-claims + auth docs alignment) +- [2026-07-07-cyclone-inwindow-rebill-pipeline.md](superpowers/plans/2026-07-07-cyclone-inwindow-rebill-pipeline.md) — SP41 (in-window rebill pipeline) +- [2026-07-08-cyclone-doc-pass.md](superpowers/plans/2026-07-08-cyclone-doc-pass.md) — SP42 (this doc-pass) ### 12.3 Reviews and reference @@ -737,13 +762,13 @@ All under [`docs/superpowers/plans/`](superpowers/plans/): For the two independent reviewers at the end of round 2 (re-verifying the corrections): -1. **Components** — Do §5.1 (backend modules) + §5.1 (frontend dirs) list every existing module that ships PHI or serves a state-changing route? Are the boundaries (DB / ORM / reconcile / store / api / pubsub / clearhouse / batch_diff / edi.filenames) drawn correctly? Can you, from §5.1 alone, draw the import graph? -2. **Data model** — Are all 12 migrations listed in §5.2 with their purpose? Are the two worktree-only migrations (0013, 0014) flagged as in-flight? Are all 18 ORM models in §5.2? Is the `db_backups` (not `backups`) correction applied? Is the `Rejection` phantom removed? Is the `ActivityEvent` vs `AuditLog` distinction clear (NFR-3 vs NFR-4)? -3. **Dependencies** — Are §5.3 runtime + tooling deps correct (no missing `pyyaml`, no missing `keyring`, no `httpx` listed as runtime, `paramiko` flagged as `[sftp]` extra, `sqlcipher3` flagged as `[sqlcipher]` extra, frontend test env is `happy-dom` not `jsdom`, vitest pinned at `^4.1.9` flagged as unusual)? Are OS-level deps (Keychain via `keyring`, SQLCipher C library, `tini`, `nginx` if SP23 ships) called out? -4. **Definition of Done** — Is §8 testable? Project DoD #6 now points at the reviewer-agreement meta-criterion. Per-release + per-task DoDs each independently checkable? FR-7 / FR-33 traceability rows corrected to point at existing test files and the correct endpoint name? -5. **Traceability** — Pick any FR-NN in §3. Can you trace it to an SP, a spec on disk (or an honest "no dedicated spec on disk" note), a plan on disk (or an honest "no plan file on disk; feat(spN) commits substitute" note), and at least one test file in `backend/tests/` or `src/`? If not, that's a remaining gap. -6. **Assumptions** — Are §10 assumptions falsifiable? Could a new engineer challenge any of them and find the test that proves/disproves them? Is A2's reference to `keyring` accurate now? -7. **Risks** — Are §11 risks current (R-11 corrected to 2,172; R-12 added; R-25/R-26/R-27 added)? Compare against `docs/reviews/2026-06-20-cyclone-completeness-review.md`. Is R-24 (SP23 fork) clearly flagged as awaiting user decision? +1. **Components** — Do §5.1 (backend modules) + §5.1 (frontend dirs) list every existing module that ships PHI or serves a state-changing route? Are the boundaries (DB / ORM / reconcile / store / api / pubsub / clearhouse / batch_diff / edi.filenames / auth / handlers / submission / rebill / reissue / store/) drawn correctly? Can you, from §5.1 alone, draw the import graph? +2. **Data model** — Are all 23 migrations listed in §5.2 with their purpose (no worktree-only entries)? Are all 18 ORM models in §5.2? Is the `db_backups` (not `backups`) correction applied? Is the `ActivityEvent` vs `AuditLog` distinction clear (NFR-3 vs NFR-4)? +3. **Dependencies** — Are §5.3 runtime + tooling deps correct (no missing `pyyaml`, no missing `keyring`, no `httpx` listed as runtime, `paramiko` flagged as `[sftp]` extra, `sqlcipher3` flagged as `[sqlcipher]` extra, frontend test env is `happy-dom` not `jsdom`, vitest pinned at `^4.1.9` flagged as unusual)? Are OS-level deps (Keychain via `keyring`, SQLCipher C library, `tini`, `nginx` — SP23 shipped) called out? +4. **Definition of Done** — Is §8 testable? Project DoD #6 now points at the reviewer-agreement meta-criterion. Per-release + per-task DoDs each independently checkable? FR-7 / FR-33 / FR-35 / FR-36 traceability rows corrected to point at existing test files and the correct endpoint names? +5. **Traceability** — Pick any FR-NN in §3. Can you trace it to an SP, a spec on disk (or an honest "no dedicated spec on disk" note), a plan on disk (or an honest "no plan file on disk; feat(spN) commits substitute" note), and at least one test file in `backend/tests/` or `src/`? If not, that's a remaining gap. Note FR-RB-1..FR-RB-6 (SP41) and the auth-related rows point at the new subpackages. +6. **Assumptions** — Are §10 assumptions falsifiable? Could a new engineer challenge any of them and find the test that proves/disproves them? Is A2's reference to `keyring` accurate now (Linux dispatch to Secret Service / kwallet, Windows via Credential Manager)? +7. **Risks** — Are §11 risks current (R-1/R-3/R-11/R-12/R-24 closed by SP23/SP21/SP36; R-28 added for test-density drift)? Compare against `docs/reviews/2026-06-20-cyclone-completeness-review.md`. If reviewers materially agree on all seven, round 2 closes; this doc becomes the project's new top-level entry point (linked from `README.md`).