Align CLAUDE.md + REQUIREMENTS.md + ARCHITECTURE.md with the auth work
that landed on main on 2026-06-23; emit an AUTH_DISABLED WARNING at boot
when AUTH_DISABLED is true so misconfigured production deploys fail loudly.
Closes requirements R-1 (was Open; auth shipped via the origin/main merge,
SP24 reconciles the docs).
* CLAUDE.md — track in git for the first time
* docs/REQUIREMENTS.md + docs/ARCHITECTURE.md — replace every stale
'no auth' / 'no authentication' / 'no second party to authenticate'
claim with the v1 posture (bcrypt + HttpOnly session cookie; first
admin bootstrapped from CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD)
* backend/src/cyclone/__main__.py — boot-time WARNING when AUTH_DISABLED
* ARCHITECTURE §4.2 module map — add cyclone.auth.* package
* ARCHITECTURE §6.1 migrations table — list 0013 + 0014 with renumbering note
* .superpowers/skills/{cyclone-tests,cyclone-api-router,cyclone-spec} —
add AUTH_DISABLED / matrix_gate / auth-aware threat-model guidance
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
Plan: docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md
- test_migration_latest_idempotent_on_fresh_db now uses the pytest
tmp_path fixture instead of a literal /tmp path that did not exist,
so sqlite can create the test DB.
- test_drop_claims_unique_constraint_migration now inserts the NOT
NULL batches.kind / input_filename / parsed_at columns when seeding
the parent row for the two same-PCN claims.
Adds migration 0015_drop_claims_unique_constraint.sql that recreates the
claims table without the inline UNIQUE(batch_id, patient_control_number)
constraint, plus tests proving the migration runs cleanly and the
constraint is not re-introduced.
Replaces every stale 'no auth' / 'no authentication' / 'no second party to
authenticate' claim in the three top-level docs with the v1 posture: the
auth boundary is HTTP (bcrypt + HttpOnly session cookie; first admin
bootstrapped from CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD env
vars); the file-system posture (SQLCipher at rest, macOS Keychain) is
unchanged; the threat model is still a stolen/imaged drive.
Closes requirements §11 R-1 (was Open; now Closed by SP24 — auth shipped
via the origin/main merge on 2026-06-23, SP24 reconciled the docs).
Also:
- backend/src/cyclone/__main__.py — emit WARNING when AUTH_DISABLED is
True at boot so a misconfigured production deploy fails loudly
- §6.1 migrations table in ARCHITECTURE.md — list 0013 (auth_users_and_
sessions) and 0014 (audit_log_user_id) with the renumbering note
- §4.2 module map — add the cyclone.auth.* package
- Three skills addenda: cyclone-tests (AUTH_DISABLED conftest bypass is
mandatory context), cyclone-api-router (every router needs
Depends(matrix_gate)), cyclone-spec (spec template threat-model is now
auth-aware)
This also tracks CLAUDE.md in git for the first time (was previously
untracked; the SP24 doc updates are in scope for the increment).
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
Plan: docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md
Two bugs surfaced as 403/500 storms in the live UI:
1. PERMISSIONS matrix only had '/api/reconcile' as a prefix but the
client hits '/api/reconciliation/*' (dashboard reconciliation tab,
manual match endpoint). Every call returned 403 even for admin.
Add ('GET', '/api/reconciliation') and ('POST', '/api/reconciliation')
alongside the existing /api/reconcile entries.
2. The seed CLI built billing_provider.address as a flat string but
to_ui_claim_detail._address_to_ui expects {line1,line2,city,state,zip},
so opening any seeded claim in the detail drawer threw
'str object has no attribute get' → 500. Build the dict from the
provider fields instead.
Before this change, every claim — matched or not — came back from
`/api/claims` with `receivedAmount: 0.0`. `to_ui_claim_from_orm`
hardcoded the value, so the Dashboard's 'Received' KPI and the
claim detail drawer's paid amount always read $0 regardless of
whether the claim had been paired with a paying remittance.
The fix threads the real value through the read path:
- `to_ui_claim_from_orm` accepts a new `received_total: float`
kwarg (default 0.0 for callers that don't have a remittance in
scope).
- `iter_claims` bulk-loads `Remittance.total_paid` for every
matched claim id in the result set (single SQL roundtrip via
`IN` filter — no N+1) and stamps the sum onto each claim dict.
- `manual_match` passes `float(remit.total_paid)` from the remit
it's already holding.
- `manual_unmatch` reads `paired_remit.total_paid` before clearing
the FK so the response still reports what was paid pre-unpair.
- `add` (write path) and `list_unmatched` (filters matched-remit
is NULL) pass `0.0` explicitly for readability.
The dev seed CLI now also generates matched Remittance rows for
every PAID/PARTIAL claim — `matched_remittance_id` is set on the
claim, and the remittance's `total_paid` is derived from the
billed amount using the same ratios the frontend's old sample
fixtures used (60–100% for PAID, 20–50% for PARTIAL). That gives
the Dashboard a non-zero 'Received' KPI on a fresh dev DB.
New tests:
- tests/test_iter_claims_received.py — 4 cases covering matched,
unmatched, orphan FK, and bulk-load paths. Catches regressions
if anyone re-introduces the hardcoded 0.0 or breaks the bulk
query.
Now that Dashboard reads live API hooks instead of the in-memory
sample store, a fresh dev DB renders as $0 KPIs and 'No activity
yet.' for a long time. `python -m cyclone seed` inserts a
deterministic batch of 96 claims + 28 activity events so the
Dashboard / Claims / Activity Log pages have something to render
without going through the Inbox / EDI parser pipeline.
python -m cyclone seed # insert (no-op if seeded)
python -m cyclone seed --reset # wipe + re-insert
python -m cyclone seed --count 10
python -m cyclone seed --status # counts only
The seed mirrors the frontend `sampleData.ts` fixtures (3 TX
providers, 7 payers, 8 CPTs, 96 claims spread across the last 200
days) so dev dashboards look identical to the previous in-memory
fixture mode. `raw_json` and `payload_json` are populated in the
shape that `store.to_ui_claim_from_orm` / `recent_activity` parse
back into the UI wire format — wire parity, not shortcuts.
`--reset` cleans by id prefix (`SEED-` / `CLM-S`) rather than
relying on SQLite's `ON DELETE CASCADE`, which the dev session
doesn't enforce (no `PRAGMA foreign_keys = ON` at the SQLAlchemy
session layer). Safer than silent orphans.
Verified end-to-end: 986 backend tests pass, dashboard renders
$75,833 / 96 claims / 11.5% denial rate / 10 activity events /
top-3 providers ranked by claim count.
Two bootability gaps caught by running the live stack:
1. cyclone.auth.bootstrap.run() called SessionLocal() before db.init_db()
had run on a fresh DB, so 'python -m cyclone serve' / 'users create'
/ '--help' all crashed with 'db.init_db() has not been called'.
Add init_db() inside bootstrap (idempotent — no-op once the schema
is current) so first-boot works without manual prep.
2. PERMISSIONS matrix didn't register the inbox endpoints that were
added in SP10 (export.csv) and SP14 (candidates/dismiss,
rejected/resubmit, payer-rejected/acknowledge). The matrix's
longest-prefix match meant /api/inbox/candidates/dismiss matched
the '/api/inbox' prefix but only because that single entry
existed; adding the dedicated write-prefix entries makes the
intent explicit and keeps viewer role out of every inbox write
even if the broader /api/inbox prefix is later broadened.
Verified live:
- admin can login, GET /api/inbox/lanes (200), GET
/api/inbox/export.csv (200), POST /api/inbox/candidates/dismiss
(200).
- viewer can login, GET /api/inbox/lanes (200), GET
/api/inbox/export.csv (200), but POST /api/inbox/candidates/dismiss
and POST /api/inbox/rejected/resubmit both return 403.
- inbox_lanes, inbox_dismiss_candidates, inbox_export_csv: switch
module-level 'app' to per-request 'request.app' so per-endpoint
state stays consistent with the test's TestClient target across
importlib.reload() (test_api.py::test_cors_extra_origins_via_env
reloads the api module to mutate CORS allow-lists; pre-reload
endpoints then mutate the wrong app instance).
- _http_exc_handler: when HTTPException.detail is a dict, wrap under
'detail' so the standard envelope stays stable and callers can
branch on body['detail']['error'].
- conftest._auto_init_db: re-resolve cyclone.api.app each fixture
invocation (instead of caching at module load) so the reload pattern
doesn't leave event_bus set on a stale app.
- test_acks.test_migration_latest_idempotent_on_fresh_db: bump
user_version assertion from 12 to 14 after auth migration renumber
(0013 users+sessions, 0014 audit_log.user_id).
Also installs sqlcipher3 + paramiko into the backend venv so the
capability tests can run; both modules were optional deps that
test_db_crypto and test_sftp_paramiko assume are present.
Backend test results: 1008 passed, 9 skipped (gitignored prodfile
fixtures), 0 failed.
The auth cherry-picks brought in 0010_auth_users_and_sessions.sql
(version: 10) and 0011_audit_log_user_id.sql (version: 11), which
collide with main's 0010_payer_rejected_acknowledged.sql (version: 10)
and 0011_processed_inbound_files.sql (version: 11). The migration
runner is filename-sorted but uses the -- version: N header to gate
'application vs skip', so the second file at each version gets
silently skipped — claims.payer_rejected_acknowledged_at never
got ALTERed, breaking every test that touches the claims table.
Renumber to 0013 and 0014 (and bump the -- version headers to
match) so they apply after 0012_backups.sql and main's existing
SP14 / SP15 migrations.
- New test_existing_endpoints_require_auth.py: spot-check that existing
/api/* endpoints now require auth (gated via Depends(matrix_gate)) when
AUTH_DISABLED is False. Health remains public.
- conftest.py: flip AUTH_DISABLED=True for the suite so the legacy
pre-auth tests keep passing without login. Auth tests flip it back
off via their own autouse fixture (now patched to use monkeypatch
for cleanup).
Verified: 53 auth tests pass; 222 pre-existing non-auth failures are
unchanged.
Replaces the skipped 63ae0d2 commit: adds matrix_gate to deps.py and
applies it as a dependency on every authenticated FastAPI route, including
extracted routers (acks, ta1_acks, admin) and inline endpoints in api.py.
The PERMISSIONS matrix in auth/permissions.py controls which roles can
hit which (method, path) combinations; matrix_gate is fail-closed by
default (any endpoint not in the matrix returns 403).
Follow-up to 7c1be58. pytest-randomly was added to a PEP 735
[dependency-groups] dev block by 'uv add --dev', leaving the
project with two 'dev' groups (the legacy [project.optional-
dependencies] dev and the new [dependency-groups] dev). To get
the full dev env someone had to run 'uv sync --extra dev --group
dev', which is easy to miss.
This commit moves pytest-randomly>=4.1 into the
[project.optional-dependencies] dev list (where pytest,
pytest-cov, pytest-asyncio, and httpx already live) and drops
the [dependency-groups] block. 'uv sync --extra dev' now
installs the full dev toolchain. uv.lock updated accordingly:
the [package.dev-dependencies] and [package.metadata.requires-
dev] sections are gone, and pytest-randomly is part of the
standard dev extra.
Two follow-ups to the v0.2.0 release:
1. Add cryptography>=49.0,<50 to the [project.dependencies] list.
cyclone.backup / cyclone.backup_service import it at module top
level, so it has to be a hard dep — not an extra — or the test
suite fails to collect on a fresh 'uv sync'. A clean install
(without this commit) gets 28 collection errors with no tracked
file changes. This was latent because the original 99M venv
had cryptography installed out-of-band and masked the missing
dep declaration.
2. Add pytest-randomly>=4.1.0 to the dev dep group.
Used to characterize the test suite's order-dependence: under
alphabetic order the suite shows 7 failures; under seed=12345
it shows 34. That variance is evidence of test-order state
interactions, not venv issues. Keeping pytest-randomly in
the dev deps so this can be re-verified on demand.
Note: pytest-randomly was added with 'uv add --dev', which uses
the PEP 735 [dependency-groups] syntax. pyproject.toml now has
two 'dev' groups — the legacy [project.optional-dependencies] dev
= [pytest, pytest-cov, pytest-asyncio, httpx] and the new
[dependency-groups] dev = [pytest-randomly]. To get the full dev
env, run: uv sync --extra dev --group dev. Worth consolidating
in a follow-up.
Backend:
- New POST /api/batches/{id}/export-837: regenerate X12 837 files
for a list of claim_ids into a ZIP using HCPF file naming standards,
with a unique interchange/group control number per export. Wire
the clearhouse Loop 1000A (NM1*41 + PER) and per-payer receiver
(NM1*40) blocks so the serializer no longer falls back to
CYCLONE / RECEIVER placeholders.
- /api/parse-837 and /api/parse-835 now surface the server-side
batch_id in both JSON and NDJSON response shapes so the frontend
can hit batch-scoped endpoints without an extra listBatches
round-trip.
- Filename helpers and the 837 serializer updated to match the new
HCPF envelope; tests cover batch export, parse batch_id, and the
serializer's control-number uniqueness guarantee.
Frontend:
- New shared components: ClaimCard, ClaimCard837, DominantKpiCard,
EditorialNote, ExportBar, TickerTape, and a charts/ set
(BarChart, HBarChart, SegmentedBar, AgingBars).
- New useBatchExport hook driving ExportBar's download flow against
the new endpoint.
- ClaimDrawer, Lane, and Layout migrated from raw CSS-variable
colors to Tailwind theme tokens (bg-card, text-foreground,
border/60, etc.) for consistency with the rest of the instrument
chrome; the active tab indicator gains a subtle accent glow.
- Upload, Inbox, Batches, BatchDiff, Reconciliation, and Acks pages
reworked to compose the new shared components and consume the new
batch-scoped API surface (notably ExportBar wired into Batches).
Tooling / Docs:
- Add audit-uiux.mjs and a docs/goodclaim.x12 sample fixture.
- Update ClaimDrawer testids and add coverage for the new
components and the useBatchExport hook.
Rolls up into the v0.2.0 release tag.
SP21 Task 2.5 — the Dashboard's 'Recent activity' card now routes clicks
to the matching entity drawer / page by event kind:
- claim_* → /claims?claim=<id> (drawer in Phase 5)
- provider_added → /providers?provider=<npi> (ProviderDrawer)
- remit_received → toast 'coming in a later phase' (RemitDrawer in Phase 4)
- anything else → toast (manual_match, unknown kinds)
Implementation:
- New src/lib/event-routing.ts with the eventKindToUrl() helper,
plus a unit test covering all 6 + default branches.
- src/components/ActivityFeed.tsx gains an optional onItemClick
prop; when set, each row gets role='button', tabIndex=0, the
drillable hover affordance (chevron + tint), and an Enter/Space
keybinding. e.stopPropagation() is called before the handler so a
parent row click can't double-fire (same fix as Task 2.4).
- src/pages/Dashboard.tsx wires the handler on the 'Recent activity'
card via eventKindToUrl + sonner toast for unhandled kinds.
- Backend: CycloneStore.recent_activity() now exposes claimId and
remittanceId on each row (read from ActivityEvent.claim_id /
remittance_id) so the routing helper has the entity ids it needs.
- The frontend Activity interface gains optional claimId /
remittanceId fields; the in-memory sample data and the
addClaim store action populate them so the dashboard works in
both API-configured and sample-data modes.
Three independent improvements that fix real browser-facing bugs:
1. CORS: allow 127.0.0.1:5173 in addition to localhost:5173. Both
resolve to the same Vite dev server but CORS treats them as distinct
origins, so tabs opened via the IP form silently break.
2. CORS: support CYCLONE_ALLOWED_ORIGINS env var (comma-separated) for
LAN / staging hosts. The middleware reads it at module import.
3. Catch-all exception handler: returns JSON 500 with CORS headers
instead of a bare Uvicorn text/plain response. Without this, any
unhandled exception is misreported by browsers as a CORS error
because the body can't be read without the allow-origin header.
4. IntegrityError → 409: when (batch_id, patient_control_number) is
UNIQUE-constrained and a duplicate collides, return 409 with the
batch id instead of letting the exception 500. Same problem as (3)
for the most common ingest failure mode.
Tests added:
- test_cors_headers_present_for_loopback_ip
- test_cors_extra_origins_via_env (uses importlib.reload because the
allow-list is built at module import)
Adds pure local validators for the 10-digit NPI Luhn checksum (CMS-
published algorithm with the '80840' NPPES prefix) and 9-digit EIN
format (rejects reserved prefixes 00/07/80-89). No NPPES round-trip,
no IRS e-file lookup — catches the 99% typo case at parse time.
Surface:
- cyclone.npi.is_valid_npi / is_valid_tax_id / normalize_tax_id
- CLI: 'cyclone validate-npi <npi>' and 'cyclone validate-tax-id <ein>'
- API: GET /api/admin/validate-provider?npi=&tax_id=
- Parser validator: new R021_npi_checksum rule (warning, not error,
to keep test fixtures with placeholder NPIs ingestible)
- minimal_837p.txt fixture NPI updated from '1234567890' to the
Luhn-valid '1993999998' so strict-mode CLI parses still pass
Tests:
- test_npi.py — 27 cases (Luhn math, valid/invalid NPIs, EIN cases,
normalize_tax_id edge cases)
- test_api_validate_provider.py — 4 cases (both valid, both invalid,
omitted NPI, omitted tax_id)
- test_cli_validate.py — 8 cases (valid/invalid for both subcommands,
exit codes, malformed inputs)
- test_validator.py — 4 new R021 cases (valid Luhn silent, bad Luhn
warning, skipped when format bad, skipped when NPI missing)
Total: 923 tests pass.
Three pure-ASGI middlewares close completeness-review gaps §3.1.4
(no body/rate limits) and §3.1.25 (no security headers):
- BodySizeLimitMiddleware — rejects oversized uploads (50 MB
default, CYCLONE_MAX_BODY_BYTES override). 413 on over-cap
Content-Length and on chunked reads that cross the cap.
- RateLimitMiddleware — sliding-window per-IP limiter (300/min
default, CYCLONE_RATE_LIMIT_PER_MIN override). 429 over the
window. /api/health is exempt.
- SecurityHeadersMiddleware — stamps X-Content-Type-Options,
X-Frame-Options, Referrer-Policy, Permissions-Policy, and a
strict Content-Security-Policy on every response.
Every 413/429 also writes a tamper-evident api.request_rejected
event into the SP11 audit chain so an operator can correlate
rejections with the SP18 JSON logs.
GET /api/health is rewritten to return a subsystem snapshot:
DB connectivity (SELECT 1), MFT scheduler state, backup scheduler
state, live pubsub subscriber counts, last batch id + timestamp.
Returns status='degraded' if any subsystem is unhappy; per-subsystem
errors surfaced in the respective dict.
Cyclone.pubsub.EventBus.stats() — new method for live subscriber
counts.
13 new tests (test_security.py) + 1 updated (test_api.py health
endpoint). All 883 backend tests pass.
Cyclone previously emitted stdlib default-formatted log lines that
operators couldn't parse with anything beyond grep. SP18 replaces
that with:
- JsonFormatter: newline-delimited JSON, ISO-8601 ms timestamps,
structured "extra" dict, exception tracebacks serialized as a
single string.
- CycloneDevFormatter: tabular format for tail -f in dev.
- PiiScrubber: logging.Filter that redacts NPIs, SSNs, DOBs,
patient names — both inline in the message ("npi 1881068062")
and via PHI-keyed extras ({"dob": "1980-04-12"}).
- setup_logging(): idempotent entry point used by the API lifespan
and CLI main; respects CYCLONE_LOG_LEVEL/FILE/JSON/NO_PII_SCRUB.
- CLI --log-format=json|dev + --log-file=… flags.
- Migrate 9 highest-value log sites in scheduler/backup_scheduler/
backup_service to use extra={...} (input_filename, claims, parser,
backup_id, db_fingerprint, etc.).
34 new tests (test_logging_formatter + test_logging_scrubber +
test_logging_setup). All 867 backend tests pass.
Adds automated encrypted backups of the live SQLite file. Closes the
'no backup automation' gap called out in the completeness review
(docs/reviews/2026-06-20-cyclone-completeness-review.md §3.1 #3) and
gives the SP16 MFT scheduler a recovery path when the MFT pipeline
loses days of inbound 999/277CA work in a single crash.
Architecture
------------
- AES-256-GCM with PBKDF2-HMAC-SHA256 (200,000 iters, 16-byte salt)
- Online backups via SQLite's .backup() API — no app downtime
- Salt + passphrase persisted to macOS Keychain (separate accounts
backup.passphrase + backup.salt) so the key is reproducible across
processes
- Two-step restore (initiate → confirm) with a one-shot 64-char hex
token; the second call disposes + rebuilds the engine only if the
token matches within a 5-minute TTL
- Tamper-evident audit chain (SP11) — db.backup_created,
db.backup_failed, db.backup_pruned, db.backup_restored,
db.backup_passphrase_set
- BackupService + BackupScheduler + module-level singletons
- 8 admin endpoints + 6 CLI subcommands
- Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART=true; default
interval 24h, default retention 30 days
- Fallback posture: if no separate passphrase is set and SQLCipher
is enabled, the key is derived from the SQLCipher DB key with a
fixed salt + WARNING log (degraded but never plaintext)
New modules
-----------
- cyclone.backup — PBKDF2, AES-GCM, sidecar format
- cyclone.backup_service — create_now / list / verify / restore / prune / status
- cyclone.backup_scheduler — async tick loop with audit hooks
New surface
-----------
- 8 admin endpoints under /api/admin/backup/*
- 6 CLI subcommands under cyclone backup (init-passphrase, create,
list, verify, restore, prune, status)
- Migration 0012_backups.sql + DbBackup ORM
- store.add_backup_pending()
Tests
-----
- 14 unit tests in test_backup_crypto.py (key derivation, encrypt/
decrypt round-trip, tampered ciphertext, wrong passphrase, sidecar
round-trip, filename format)
- 19 tests in test_backup_service.py (create/list/verify/restore/
prune/status, error handling, fallback key, module singleton)
- 14 API tests in test_api_backup.py (all 8 endpoints + scheduler
endpoints, two-step restore, error responses)
- 10 tests in test_backup_scheduler.py (tick / start / stop /
audit / coalescing / module singleton)
- 5 CLI tests in test_cli_backup.py (create / list / verify /
restore confirm prompt / prune confirm prompt /
init-passphrase minimum-length check)
Total new tests: 62. All pass. Full backend suite: 833 passed,
9 skipped (gitignored prodfiles), 1 warning.
Design doc: docs/superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md
README: new 'Encrypted Backups (SP17)' section, SP17 entry in
Roadmap, retention default documented in §Project layout.
Adds an asyncio-based background scheduler that polls the Gainwell
MFT inbound path, downloads new files, and routes them through the
appropriate parser (999 / 835 / 277CA / TA1). Idempotent (re-ticks
and restarts skip already-processed files via the new
processed_inbound_files table). Crash-safe (per-file try/except so
one bad file doesn't stop the loop).
Lifespan auto-configures from the seeded dzinesco clearhouse's SFTP
block; auto-start is opt-in via CYCLONE_SCHEDULER_AUTOSTART.
Five admin endpoints added:
GET /api/admin/scheduler/status
POST /api/admin/scheduler/start
POST /api/admin/scheduler/stop
POST /api/admin/scheduler/tick
GET /api/admin/scheduler/processed-files?status=&limit=
20 new tests (15 unit + 5 API).
Self-review nits from the router-split commit:
- All four new files lacked a trailing newline (PEP 8 / POSIX).
- acks.py was lazily importing ParseResult999 / serialize_999 inside
get_ack_endpoint. Hoist to module-level — there's no import cycle
(acks.py does not import from cyclone.api), so the imports are safe
to do once.
No behavior change. Targeted tests (test_acks + test_health + test_api_gets)
still pass 41/41.
Step 2 (first half) of the architecture satisfaction loop. api.py
shrank from 2595 to 2452 lines (-143) by extracting three read-only
resource groups into cyclone.api_routers:
- health.py: GET /api/health (1 endpoint)
- acks.py: GET /api/acks, GET /api/acks/{ack_id} (2 endpoints)
- ta1_acks.py: GET /api/ta1-acks, GET /api/ta1-acks/{ack_id} (2 endpoints)
Each router owns its endpoint bodies + the small UI-shape helper that
goes with them (_ack_to_ui, _ta1_to_ui, _serialize_ta1_from_row). The
helpers stay in the router file rather than being shared because each
is only used by its own endpoints.
api.py now ends the app-wiring section with three include_router()
calls. The new package is named cyclone.api_routers (not
cyclone.api.routers) to avoid the Python package-vs-same-named-module
ambiguity that would shadow the existing cyclone.api module.
Verifies: 41 targeted tests (test_acks, test_health, test_api_gets)
pass, full pytest is 8 failed / 735 passed / 16 skipped — identical
to clean main baseline. Live curl against the running server:
GET /api/health -> 200, GET /api/acks -> 200, GET /api/ta1-acks -> 200.
See /tmp/refactor-cyclone.md for the full plan.
Adds in-place key rotation for the encrypted DB at rest (HIPAA
sec.164.308(a)(4) - periodic key rotation).
- db_crypto.rotate_db_key(): opens with old key, issues PRAGMA rekey,
reopens with new key, verifies schema survived (table-count sanity).
- db_crypto.generate_db_key(): fresh 256-bit CSPRNG hex key.
- db_crypto.fingerprint(): SHA-256[:8] of a key, for the operator to
compare across rotations.
- db.dispose_engine() + db.reinit_engine(): SP15 plumbing. The
rotation endpoint disposes the pooled connections (SQLCipher
refuses to rekey while another connection holds the file), runs
the rekey, then rebuilds the engine with the new key from the
Keychain.
- API: POST /api/admin/db/rotate-key with module-level threading.Lock
to serialize rotations. 400 when encryption not enabled, 409 when
a rotation is already in flight, 503 on rekey or Keychain failure
with a reason that tells the operator what to do next.
- Engine uses NullPool when SQLCipher is enabled: the default
QueuePool returns connections to a shared queue that any thread
can pull from, which breaks SQLCipher's thread affinity. NullPool
trades connection reuse for thread safety, the only correct
behavior under FastAPI's per-request threadpool.
- Audit event db.key_rotated with old/new fingerprints and
table_count, written after the engine is rebuilt so the new key
proves it can take new writes.
- previous key is stashed to a second Keychain account so the
operator can roll back if the new key turns out to be broken.
Tests:
- test_db_crypto.py: 12 new tests for generate/fingerprint/rekey
mechanics (5 require SQLCipher at runtime; skipped otherwise).
- test_api_rotate_key.py: 6 new tests for endpoint wiring
(encryption-required, Keychain update, audit event, rekey-failure
rollback, Keychain-write-failure 503, concurrent-rotation 409).
First checkpoint of the architecture satisfaction loop. Cyclone's api.py
was a 2281-line god-module with 14 cross-cutting helpers inlined next
to the @app route declarations. This commit moves them to a dedicated
cyclone.api_helpers module:
- NDJSON wire format: ndjson_line, ndjson_stream_837, ndjson_stream_835,
ndjson_stream_list.
- Content negotiation: client_wants_json, wants_ndjson.
- Strict / raw_segments rewrites: strict_rewrite_837, strict_rewrite_835,
drop_raw_segments_837, drop_raw_segments_835.
- Validation probes: has_claim_validation_errors, has_835_validation_errors.
- Live-tail generator: tail_events, heartbeat_seconds, utcnow.
api.py re-imports them under the original underscore-prefixed names so
every route call site stays unchanged. claims_stream, remittances_stream,
and activity_stream remain exposed at cyclone.api (test_api_stream_live
imports them directly).
Verifies byte-identical NDJSON wire format, content negotiation rules,
and the tail_events async-generator semantics (deliberately polls the
EventBus queue rather than awaiting its async iterator, so heartbeats
don't poison the bus subscription).
Live-tested: GET /api/health, /api/claims, /api/remittances,
/api/activity, /api/acks, /api/providers, /api/inbox/lanes,
/api/inbox/payer-rejected/acknowledge, and the /api/claims/stream
NDJSON tail all return expected codes / payload.
Backend pytest: 29 failures identical to baseline (pre-existing
secrets env, serialize_837, db_crypto env, prodfile env failures),
700 passed. Frontend npm test: 357/357 passing.
See /tmp/refactor-cyclone.md for the full checkpoint log and the plan
for the next step (splitting api.py routes into FastAPI APIRouters).
Autoreview: /tmp/grok-review-local.md (0 bugs, 1 suggestion, 4 nits —
all addressed: dead asyncio/os imports removed, dead Any import
removed, duplicate utcnow import dropped, trailing newline added).
Closes the gap between the SP10 backend (5 lanes) and the SP6
frontend (4 lanes). The Payer-Rejected lane (277CA STC A4/A6/A7)
is now rendered alongside Rejected/Candidates/Unmatched/Done,
with an Acknowledge bulk action that drops claims from the
working surface without erasing the original 277CA rejection
event (audit log stays intact, SP11).
Backend:
* Migration 0010: add payer_rejected_acknowledged_at +
payer_rejected_acknowledged_actor columns + partial index.
* db.py: surface the two new columns on the Claim model.
* inbox_lanes.py: filter acknowledged claims out of the
payer_rejected lane; expose the new fields on the row payload
for forward-compat (e.g. a future 'Recently acknowledged' view).
* api.py:
- POST /api/inbox/payer-rejected/acknowledge
Bulk-acknowledge. Idempotent. Returns transitioned /
already_acked / not_found / not_rejected counts so the UI
can show '3 of 5 were already acknowledged' on a noop bulk.
Writes a 'claim.payer_rejected_acknowledged' event to the
SP11 hash-chained audit log.
- GET /api/inbox/export.csv: accept 'payer_rejected' lane.
* test_acks.py: bump user_version assertion to 10.
* test_lane_filter_acknowledged.py: 4 tests for the lane filter
and forward-compat row payload.
* test_payer_rejected_acknowledge.py: 6 tests for the endpoint
(happy path, idempotency, no-op on non-rejected, missing
ids, 400 on empty, audit-log wiring + chain integrity).
Frontend:
* lib/inbox-api.ts: add payer_rejected to InboxLanes, add
acknowledgePayerRejected(), update exportInboxCsvUrl union.
* hooks/useInboxLanes.ts: add payer_rejected to initial state.
* hooks/useInboxLanes.test.ts: add payer_rejected to mocks.
* components/inbox/BulkBar.tsx: add 'payer_rejected' lane with
Acknowledge action (no Resubmit, no Dismiss — payer-rejected
is not eligible for either).
* components/inbox/BulkBar.test.tsx: add payer_rejected test.
* pages/Inbox.tsx: render the 5th lane, hook up onAcknowledge,
include payer_rejected in the needEyes count.
* pages/Inbox.test.tsx: 3 new tests (5-lane render, need-eyes
count, acknowledge action hits the right endpoint).
* components/inbox/InboxHeader.tsx: doc comment now explains
why payer_rejected rolls up into need-eyes.
Pre-existing typecheck warnings in BulkBar.test.tsx / InboxRow
.test.tsx / Lane.tsx / download.test.ts are unchanged from
main — not touched here.
Test counts: backend 724 -> 734 (+10). Frontend 350 -> 354 (+4).
Replace SftpClient stub write_file/list_inbound/read_file
implementations with real paramiko SSHClient + SFTPClient
calls. The public API (SftpClient.write_file, list_inbound,
read_file, get_secret) is unchanged from SP9 — same signature,
same return types — so the API layer needs no changes.
Real-mode behavior:
* _connect() returns a context manager yielding (ssh, sftp);
closes both on exit. Lazy-imports paramiko so the stub-only
test path doesn't need the dependency.
* Auth resolves from SftpBlock.auth: password_keychain_account
(MFT model) or key_file + optional key_passphrase_keychain_account.
Missing Keychain entries fail loud (RuntimeError) rather than
silently attempting empty-password auth.
* write_file: opens sftp.open(remote, 'wb') and writes bytes;
mkdirs the parent dir (idempotent — MFT pre-creates FromHPE/ToHPE).
* list_inbound: listdir_attr + per-file download into local
staging cache; skips directory entries (0o040000 mask).
* read_file: download via shutil.copyfileobj into BytesIO.
Stub mode is unchanged. AutoAddPolicy for first-time MFT host
fingerprint; operator should pin the key for production.
Adds tests/test_sftp_paramiko.py: 9 tests covering
* stub still works
* real-mode connect builds correct paramiko call from
password_keychain_account, raises on missing Keychain,
raises on missing auth config, raises on STUB_SECRET
* write_file opens 'wb' on the right path and writes bytes
* list_inbound translates attrs into InboundFile records and
caches files locally; skips dirs
Removes 2 obsolete tests in test_sftp_stub.py that expected
SP13-mode to raise NotImplementedError.
pyproject.toml: new optional 'sftp' extra (paramiko>=3.4,<6).
- New cyclone.db_crypto module:
* is_sqlcipher_available() — capability check
* is_encryption_enabled() — Keychain key + sqlcipher3 present
* get_db_key() — reads 'cyclone.db.key' from Keychain
* make_sqlcipher_connect_creator(url, key) — SQLAlchemy creator
- db._make_engine() now switches to SQLCipher when key is present
- pyproject.toml: optional 'sqlcipher' extra (sqlcipher3>=0.6,<1)
- Fallback: without Keychain key, DB stays plain SQLite (no surprise
behavior for operators who haven't set up encryption yet)
- Verified: encrypted file is unreadable as plain SQLite, wrong key
raises on first query, migrations + ORM work transparently
- HIPAA §164.312(a)(2)(iv) compliance note in docs
Tests: 705 -> 717 (12 new for SQLCipher). All 717 backend tests pass.
Add an optional ``?download=true`` query param to
``POST /api/inbox/rejected/resubmit`` that returns the same operation
result as a ZIP archive of regenerated 837P files (one
``claim-{id}.x12`` per successfully resubmitted claim) rather than the
JSON envelope.
Why: operators who mass-resubmit rejected claims want to hand the
files straight to their clearinghouse; round-tripping through copy-
paste is error-prone.
Implementation notes:
* Uses ``serialize_837_for_resubmit`` so each X12 file in the bundle
gets a unique interchange/group control number (back-to-back
resubmits would otherwise collide on ISA13/GS06 = "000000001").
* Conflicts and missing ids are deliberately excluded from the ZIP
— the user already saw them in the JSON path on prior calls; the
download is the "give me the files I asked for" payload.
* Empty resubmit + download returns 200 with an empty ZIP so the UI
can still hand the user a downloadable artifact.
Tests (test_inbox_endpoints.py): 2 new tests covering the success
shape (one .x12 per accepted claim) and the conflict-exclusion
contract.
X12 835 SVC segment:
SVC01 = composite procedure
SVC02 = charge
SVC03 = payment
SVC04 = Unit or Basis for Measurement Code (UN, MJ, DA, ...)
SVC05 = Service Unit Count
The parser previously read SVC04 as the units count and SVC05 as the
unit type — backwards. On real 835s (and the canonical minimal
fixture), SVC04 carries the code 'UN' which fails Decimal parsing, so
the units always came out as None and the code string was assigned to
unit_type. SP7's line-level matcher couldn't compare units on the SVC
side against the claim side because of this.
- _consume_service_payment: SVC04 → unit_type, SVC05 → units count
- Default unit_type to 'UN' when only the count is present
- minimal_835.txt + unbalanced_835.txt: swap positions to match spec
- Add 2 regression tests (units-and-unit-type, default-unit-type-to-UN)
cyclone
Tyler
Nora
sp15-bot