CycloneStore split precedent (SP21) — lift three helpers from
scheduler.py + api.py into one module. Both callers will switch
to this in Task 6. The new package also defines HandleResult
and the HANDLERS registry; handle_999 / handle_ta1 / handle_277ca
/ handle_835 fill in over Tasks 2-5.
The registry is lazy + best-effort: register_handlers() catches
broad Exception so a partial-modification SyntaxError in one
in-flight handler module can't break scheduler or API import.
11 tests added; existing suite unchanged (1 failed + 2 errors
pre-existing in test_provider_extended_response.py are not
introduced by this commit).
Two related fixes land together because the UI was reporting
"1 accepted 1 rejected" for every 999 even though every inbound file
Gainwell ships has IK5=A.
1. Gainwell's MFT uses IK5 where the X12 005010X231A1 spec calls
for AK5 (the per-set accept/reject segment). The parser only
recognized AK5, so set_responses[0].set_accept_reject.code
defaulted to 'R' and the count summary showed all rejections.
_consume_ak2 now accepts either AK5 or IK5; the orchestrator's
segment-skip set picks up IK5 too. A new fixture
(minimal_999_ik5_gainwell.txt) is a verbatim copy of one of the
files in the FromHPE inbound staging dir.
2. _ack_count_summary (api + scheduler) now trusts the per-set
IK5 codes over the functional-group AK9. Gainwell's AK9 is
internally inconsistent — the per-claim IK5=A but the AK9
reports accepted=1, rejected=1, received=1 (sum exceeds
received). Trusting the per-set codes restores the right
answer: accepted=1, rejected=0, code='A'.
3. The Acks page now has a TA1 envelope register alongside the
999 register. TA1s are the lower-level sibling of the 999
(one row per inbound ISA/IEA). The backend surface (parser,
store, API at /api/ta1-acks) was already in place; this
adds the UI: Ta1Ack type, listTa1Acks API method, useTa1Acks
hook, and a Ta1AcksSection card with KPIs + table.
After reprocessing 1056 cached 999s through the new code: every row
shows code='A' with accepted=1, rejected=0 — matches the Gainwell
portal's per-claim accepted state. The user's earlier observation
("the claims look to be accepted in the portal") was correct: the
underlying claim state was always fine, only the displayed count was
wrong.
- backend/src/cyclone/parsers/parse_999.py | 21 ++-
- backend/src/cyclone/api.py | 11 +-
- backend/src/cyclone/scheduler.py | 13 +-
- backend/tests/test_parse_999.py | 32 ++++
- backend/tests/fixtures/minimal_999_ik5_gainwell.txt
- src/types/index.ts | 32 ++++
- src/lib/api.ts | 62 +++++-
- src/hooks/useTa1Acks.ts | 26 +++ (new)
- src/pages/Acks.tsx | 209 +++++++++++++++++++-
Gainwell's MFT ships every 999 with the same ISA interchange
control number (`000000001`) and one 999 ack covers a whole
batch, not a single claim — so the AK2 set_control_number
(patient_control_number) is the same for the ~96 999s in a
batch. With the old synthetic-id formula (`999-{icn}`), all 385
daily acks collapsed onto a single row the operator couldn't
distinguish.
The new formula is `999-{pcn}-{filename_hash8}` (or
`999-{icn}-{filename_hash8}` for envelope-only 999s without an
AK2). The PCN gives the operator a human-readable handle to the
claim batch; the 8-char hash of the inbound filename guarantees
uniqueness within a batch. Fits in the VARCHAR(32) source_batch_id
column (max 22 chars).
Also surface `patient_control_number` in /api/acks list response
(extracted from raw_json's set_responses[0].set_control_number)
and in the Acks UI as the primary label, with the synthetic id
shown dimmed after a middle dot. The detail endpoint already
exposed raw_json for the full 999 parse tree.
Three changes that unblock the daily inbound pull from Gainwell's
FromHPE MFT path:
1. INBOUND_RE now accepts both 'TP' and 'tp' prefixes via inline
(?i:TP) scoping — case-folding the whole pattern would let
lowercase '837p' / tracking IDs through, which is invalid HCPF.
The rest of the pattern is still case-sensitive.
2. _list_inbound_paramiko skips *_warn.txt entries. Gainwell's MFT
drops ~583 advisory text-format notes in the same inbound dir;
they come first alphabetically and were padding every poll with
~80 min of pointless downloads.
3. New SftpClient.list_inbound_names() + download_inbound() pair
gives a metadata-only listing and on-demand fetch. The
scheduler's existing full-listing path still works (now without
the warn padding); the new path is what the new
/api/admin/scheduler/pull-inbound endpoint and the
'cyclone pull-inbound' CLI use to fast-target a date range
without paying the cost of a full ~6000-file download.
Scheduler.process_inbound_files() runs the same per-file pipeline
as a regular tick on the pre-fetched list, so dedup via
processed_inbound_files still applies.
Tests added in test_filenames.py (lowercase + mixed-case cases),
test_sftp_paramiko.py (warn skip + no-download listing), and
test_scheduler.py (process_inbound_files idempotency).
With this, the daily 385-file pull for 20260624 completes in
seconds via 'docker exec cyclone-backend-1 python -m cyclone
pull-inbound --date 20260624 --block dzinesco' (or the equivalent
POST to /api/admin/scheduler/pull-inbound?date=20260624).
In real (paramiko) mode, `_download_and_parse` called
`client.read_file(f.name)` with a bare filename. paramiko's
`sftp.open(f.name)` opens at the SFTP root, not at `paths.inbound`
(FromHPE) — so the scheduler would fail to download any file in real
mode even with the path swap from the previous commit.
But this round-trip is also unnecessary: `_list_inbound_paramiko`
already downloads each entry into the local cache (cache_path) and
returns it as `InboundFile.local_path` as part of the listing pass.
Reading from disk is faster than re-fetching and avoids the path bug.
Stub mode was already reading from `f.local_path`. Now both modes do,
which is the simpler invariant.
Verified: 36 tests pass (test_scheduler, test_api_scheduler, test_sftp_stub,
test_sftp_paramiko).
The dzinesco SP9 seed had `paths.inbound` and `paths.outbound` mapped to
the wrong Gainwell MFT directories:
- paths.outbound was FromHPE/ (HPE sends files FROM here TO us — inbound)
- paths.inbound was ToHPE/ (we send files TO here — outbound)
So `/api/clearhouse/submit` was writing 837P claims to FromHPE (where
HPE puts acks/835s) and the SP16 scheduler was polling ToHPE (where our
claims go). 999 / TA1 / 835 files in the real FromHPE inbox were
unreachable.
Semantics per operator (2026-06-24):
- FromHPE = HPE/Gainwell → us = 999, TA1, 835 (inbound)
- ToHPE = us → HPE/Gainwell = 837P claims (outbound)
**Runtime code (the actual fix):**
- backend/src/cyclone/store.py — SP9 seed paths flipped
- backend/src/cyclone/edi/filenames.py — docstring corrected
**Test fixtures + assertions (would otherwise fail on the new seed):**
- backend/tests/test_clearhouse_api.py
- backend/tests/test_providers_seed.py
- backend/tests/test_sftp_stub.py — incl. inbound dir paths
- backend/tests/test_sftp_paramiko.py
- backend/tests/test_store_update_clearhouse.py
- backend/tests/test_api_clearhouse_patch.py
- backend/tests/test_scheduler.py
- backend/tests/test_api_scheduler.py
**Docs (text-only — keeps the codebase self-consistent):**
- README.md
- docs/reference/co-medicaid.md
- docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md
**Operator action required after merge:** the existing clearhouse row in
`~/.local/share/cyclone/cyclone.db` was seeded with the old wrong
paths. Easiest recovery:
1. `rm ~/.local/share/cyclone/cyclone.db` and let `ensure_clearhouse_seeded` re-run on next boot, OR
2. PATCH /api/clearhouse with the new `paths` block (the SP25
reconfigure hook picks it up live, including by the running scheduler).
**Verification:**
- 82 tests in the affected files pass (test_clearhouse_api, test_providers_seed,
test_sftp_stub, test_sftp_paramiko, test_store_update_clearhouse,
test_api_clearhouse_patch, test_scheduler, test_api_scheduler, test_filenames).
- Full backend suite: 1029 pass + 36 pre-existing order-dependent flakes
unrelated to this change (verified by running the same tests in isolation).
Bring the SP25 SFTP Polling Enablement (PATCH /api/clearhouse,
scheduler hot-reload, env-var-first secret lookup) and SP26
SFTP Password File Companion (Docker secret _FILE tier) onto
the v1.0.0 release branch so the production stack can actually
flip the Gainwell MFT poll from stub → real.
Conflict resolved in docker-compose.yml: kept both the SP26
CYCLONE_SFTP_PASSWORD_FILE env var and the new 'Always bind
to 0.0.0.0' comment.
The matrix_gate dependency was added to ~50 routes (clearhouse, all
/api/admin/*, /api/config/*, /api/eligibility/*, /api/parse-999/ta1/277ca,
/api/batches/{id}/export-837, /api/payer-rejected/acknowledge, etc.)
but the PERMISSIONS matrix was only populated for ~25 of them. Default-
deny therefore returned 403 to every authenticated role on the rest,
including the SFTP admin endpoints needed for MFT polling.
Roles:
- Admin-only: /api/clearhouse* (SFTP creds + dzinesco identity), all
/api/admin/* (audit-log, backup, scheduler, db rotate, reload-config,
validate-provider)
- All authenticated: /api/batch-diff, /api/config/*, /api/payers/*
- Write (admin + user, no viewer): /api/parse-999/ta1/277ca, the
/api/batches POST prefix (regenerates X12 from DB rows), and
/api/eligibility/* (270 build / 271 parse)
Unblocks /api/clearhouse, /api/admin/scheduler/*, /api/admin/backup/*,
and the rest of the admin surface so MFT polling and live verification
can proceed.
The PERMISSIONS matrix only listed the POST /api/acks entry (parse-999
ingest). The GET list + detail endpoints for 999 / TA1 / 277CA acks
were missing, so the matrix_gate (default-deny) returned 403 to every
authenticated role, breaking the 999 ACKs / TA1 / 277CA inbox pages.
Add the three GET entries as ALL_ROLES — they're read-only metadata
surfaces that every authenticated operator needs to see. Add a
regression test that logs in as admin via the public route and
exercises the matrix; the existing test_existing_endpoints_require_auth
suite only checks the unauthenticated 401 path, which is why this slipped
through.
Fixes the live-verification 'Couldn't load ACKs from the backend /
forbidden' error reported against the UI.
Reachability is controlled by the host firewall / compose port
publishing, not the bind address. Backend, compose, and docs all
now default to 0.0.0.0 so the API is reachable from the frontend
container, the host, and the LAN without per-env overrides.
Files:
- backend/src/cyclone/__main__.py: default host = 0.0.0.0
- docker-compose.yml: refresh the comment to match the new posture
- CLAUDE.md / README.md / docs/ARCHITECTURE.md / docs/REQUIREMENTS.md:
reframe the bind note accordingly
Add a second tab to the Upload page that surfaces the persisted batch
archive and lets the user re-download any 837P batch as a ZIP without
re-parsing the original file.
- Backend: /api/batches now carries per-row claimIds (837P only).
835 batches return an empty list, which the UI uses as the signal
to hide the Re-export button on those rows. Avoids an extra
round-trip to /api/batches/{id} per row.
- Frontend: BatchSummary.claimIds added to the list-endpoint type.
- Upload page: page body wrapped in Tabs.Root with a History trigger
that mirrors ?tab= in the URL for deep-link round-trip. The
History tab renders UploadHistory → HistoryTable → HistoryRow with
a one-click Re-export ZIP button per 837P row. The button calls
POST /api/batches/{id}/export-837 with the row's claim ids and
downloads the ZIP via downloadBlob. Falls back to the in-memory
parsedBatches store when the backend returns no rows so the tab
stays useful in sample-data mode.
- Backend tests: claimIds present on 837P rows, empty on 835 rows.
- Frontend tests: 13 tests covering tab switching, URL deep-link,
loading/error/empty states, the 837P-vs-835 button visibility
split, the Re-export happy path, and the failure toast.
The dev-server allow-list (VITE_DEV_ORIGINS = localhost:5173 +
127.0.0.1:5173 + CYCLONE_ALLOWED_ORIGINS) is tight enough that
`allow_credentials=True` is safe — the browser was dropping the
session cookie on cross-origin fetches from the Vite dev server
otherwise. Tight allow-list + credentials is the standard setup.
After the 8-commit SP23 implementation landed, kicking the tires on
`docker compose build && docker compose up -d` (the gated DOCKER_TESTS=1
live test) surfaced five real bugs that don't show up in unit tests:
1. Backend wheel was built from the stub `__init__.py`, not the real
source. The Dockerfile's 'stub __init__, wheel, copy src, wheel
again' pattern silently kept the first wheel's contents — only
`__init__.py` got re-stubbed. The installed package had an empty
`__init__.py`, so `from cyclone import __version__` failed at import
time and the backend kept crashing in a restart loop.
Fix: single `COPY src/` + single `pip wheel`. Comment explains
why the stub trick is gone for good.
2. Backend binds to 127.0.0.1 (intentional — local-only by design,
see CLAUDE.md). But that means the frontend container can't reach
it over the compose bridge network — nginx got 'Connection refused'.
Fix: `CYCLONE_HOST` env var, defaults to 127.0.0.1 (preserves local
posture for non-Docker runs), set to 0.0.0.0 by the docker-compose
backend service. Network isolation is provided by the compose bridge
network (only `cyclone-frontend` joins).
3. Healthcheck probed `/api/healthz` (404 — the route is `/api/health`).
Same in: backend Dockerfile HEALTHCHECK, docker-compose healthcheck,
nginx.conf doesn't have one (frontend proxies through), RUNBOOK.md,
scripts/post-deploy.sh, scripts/smoke.sh.
Fix: `/api/healthz` → `/api/health` everywhere SP23 owns.
4. The auth matrix in `cyclone.auth.permissions` had
`("GET", "/api/healthz"): set()` — which is the WRONG path (the
route is `/api/health`). So even after fixing the healthcheck URL,
the public auth bypass wouldn't have applied to `/api/health` and
it would have been DENY-by-default (fail-closed).
Fix: matrix entry updated to `/api/health`.
5. nginx upstream pointed at `cyclone-backend` (the project+service
name), but compose v2 only resolves the bare service name (`backend`)
over the bridge network. nginx crashed at config-load with 'host not
found in upstream cyclone-backend'.
Fix: `cyclone-backend:8000` → `backend:8000` in nginx.conf + spec
+ plan.
6. Frontend HEALTHCHECK used `http://localhost:8080/`. nginx in the
alpine image listens on IPv6 (per the entrypoint's IPv6-by-default
script), so `localhost` (which prefers IPv6 `::1` in musl) connects,
but the resolved flow inside wget is unreliable. `127.0.0.1` works.
Fix: HEALTHCHECK uses `http://127.0.0.1:8080/`.
Also moves `frontend/Dockerfile` → `Dockerfile.frontend` and
`frontend/nginx.conf` → `nginx.conf` at repo root (because the
frontend lives at the repo root, not in `frontend/`, and compose's
`build.context: .` needs them at the same root as compose.yml).
The frontend's pre-existing `.dockerignore` was empty/unused, so it's
dropped — the root `.dockerignore` covers it.
Adds `docker-compose.override.yml` for local bring-up testing on a
host without sudo. Production uses `/etc/cyclone/secrets/` directly.
Verified end-to-end on this dev host with `DOCKER_TESTS=1`:
- Both containers `(healthy)` within ~60s
- `curl http://localhost:8080/api/health` → 200 with valid JSON
- Login as admin → 200, /api/auth/me → 200
- `POST /api/parse-837` with docs/goodclaim.x12 → 200, batch created
- Full backend test suite: 1014 passed, 9 skipped (prodfiles gitignored)
Add _read_secret() helper that prefers *_FILE env vars (the standard Docker-secret pattern) over bare env vars. Strips trailing whitespace from file contents so printf/echo newlines don't break bcrypt verify. Existing CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD still work for non-Docker deployments. Add tests/test_auth_bootstrap_file.py covering: file-path read, file-overrides-bare, bare-fallback, whitespace-stripping, missing-file raises.
Align CLAUDE.md + REQUIREMENTS.md + ARCHITECTURE.md with the auth work
that landed on main on 2026-06-23; emit an AUTH_DISABLED WARNING at boot
when AUTH_DISABLED is true so misconfigured production deploys fail loudly.
Closes requirements R-1 (was Open; auth shipped via the origin/main merge,
SP24 reconciles the docs).
* CLAUDE.md — track in git for the first time
* docs/REQUIREMENTS.md + docs/ARCHITECTURE.md — replace every stale
'no auth' / 'no authentication' / 'no second party to authenticate'
claim with the v1 posture (bcrypt + HttpOnly session cookie; first
admin bootstrapped from CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD)
* backend/src/cyclone/__main__.py — boot-time WARNING when AUTH_DISABLED
* ARCHITECTURE §4.2 module map — add cyclone.auth.* package
* ARCHITECTURE §6.1 migrations table — list 0013 + 0014 with renumbering note
* .superpowers/skills/{cyclone-tests,cyclone-api-router,cyclone-spec} —
add AUTH_DISABLED / matrix_gate / auth-aware threat-model guidance
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
Plan: docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md
Adds migration 0015_drop_claims_unique_constraint.sql that recreates the
claims table without the inline UNIQUE(batch_id, patient_control_number)
constraint, plus tests proving the migration runs cleanly and the
constraint is not re-introduced.
Replaces every stale 'no auth' / 'no authentication' / 'no second party to
authenticate' claim in the three top-level docs with the v1 posture: the
auth boundary is HTTP (bcrypt + HttpOnly session cookie; first admin
bootstrapped from CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD env
vars); the file-system posture (SQLCipher at rest, macOS Keychain) is
unchanged; the threat model is still a stolen/imaged drive.
Closes requirements §11 R-1 (was Open; now Closed by SP24 — auth shipped
via the origin/main merge on 2026-06-23, SP24 reconciled the docs).
Also:
- backend/src/cyclone/__main__.py — emit WARNING when AUTH_DISABLED is
True at boot so a misconfigured production deploy fails loudly
- §6.1 migrations table in ARCHITECTURE.md — list 0013 (auth_users_and_
sessions) and 0014 (audit_log_user_id) with the renumbering note
- §4.2 module map — add the cyclone.auth.* package
- Three skills addenda: cyclone-tests (AUTH_DISABLED conftest bypass is
mandatory context), cyclone-api-router (every router needs
Depends(matrix_gate)), cyclone-spec (spec template threat-model is now
auth-aware)
This also tracks CLAUDE.md in git for the first time (was previously
untracked; the SP24 doc updates are in scope for the increment).
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
Plan: docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md
Two bugs surfaced as 403/500 storms in the live UI:
1. PERMISSIONS matrix only had '/api/reconcile' as a prefix but the
client hits '/api/reconciliation/*' (dashboard reconciliation tab,
manual match endpoint). Every call returned 403 even for admin.
Add ('GET', '/api/reconciliation') and ('POST', '/api/reconciliation')
alongside the existing /api/reconcile entries.
2. The seed CLI built billing_provider.address as a flat string but
to_ui_claim_detail._address_to_ui expects {line1,line2,city,state,zip},
so opening any seeded claim in the detail drawer threw
'str object has no attribute get' → 500. Build the dict from the
provider fields instead.
Before this change, every claim — matched or not — came back from
`/api/claims` with `receivedAmount: 0.0`. `to_ui_claim_from_orm`
hardcoded the value, so the Dashboard's 'Received' KPI and the
claim detail drawer's paid amount always read $0 regardless of
whether the claim had been paired with a paying remittance.
The fix threads the real value through the read path:
- `to_ui_claim_from_orm` accepts a new `received_total: float`
kwarg (default 0.0 for callers that don't have a remittance in
scope).
- `iter_claims` bulk-loads `Remittance.total_paid` for every
matched claim id in the result set (single SQL roundtrip via
`IN` filter — no N+1) and stamps the sum onto each claim dict.
- `manual_match` passes `float(remit.total_paid)` from the remit
it's already holding.
- `manual_unmatch` reads `paired_remit.total_paid` before clearing
the FK so the response still reports what was paid pre-unpair.
- `add` (write path) and `list_unmatched` (filters matched-remit
is NULL) pass `0.0` explicitly for readability.
The dev seed CLI now also generates matched Remittance rows for
every PAID/PARTIAL claim — `matched_remittance_id` is set on the
claim, and the remittance's `total_paid` is derived from the
billed amount using the same ratios the frontend's old sample
fixtures used (60–100% for PAID, 20–50% for PARTIAL). That gives
the Dashboard a non-zero 'Received' KPI on a fresh dev DB.
New tests:
- tests/test_iter_claims_received.py — 4 cases covering matched,
unmatched, orphan FK, and bulk-load paths. Catches regressions
if anyone re-introduces the hardcoded 0.0 or breaks the bulk
query.
Now that Dashboard reads live API hooks instead of the in-memory
sample store, a fresh dev DB renders as $0 KPIs and 'No activity
yet.' for a long time. `python -m cyclone seed` inserts a
deterministic batch of 96 claims + 28 activity events so the
Dashboard / Claims / Activity Log pages have something to render
without going through the Inbox / EDI parser pipeline.
python -m cyclone seed # insert (no-op if seeded)
python -m cyclone seed --reset # wipe + re-insert
python -m cyclone seed --count 10
python -m cyclone seed --status # counts only
The seed mirrors the frontend `sampleData.ts` fixtures (3 TX
providers, 7 payers, 8 CPTs, 96 claims spread across the last 200
days) so dev dashboards look identical to the previous in-memory
fixture mode. `raw_json` and `payload_json` are populated in the
shape that `store.to_ui_claim_from_orm` / `recent_activity` parse
back into the UI wire format — wire parity, not shortcuts.
`--reset` cleans by id prefix (`SEED-` / `CLM-S`) rather than
relying on SQLite's `ON DELETE CASCADE`, which the dev session
doesn't enforce (no `PRAGMA foreign_keys = ON` at the SQLAlchemy
session layer). Safer than silent orphans.
Verified end-to-end: 986 backend tests pass, dashboard renders
$75,833 / 96 claims / 11.5% denial rate / 10 activity events /
top-3 providers ranked by claim count.
Two bootability gaps caught by running the live stack:
1. cyclone.auth.bootstrap.run() called SessionLocal() before db.init_db()
had run on a fresh DB, so 'python -m cyclone serve' / 'users create'
/ '--help' all crashed with 'db.init_db() has not been called'.
Add init_db() inside bootstrap (idempotent — no-op once the schema
is current) so first-boot works without manual prep.
2. PERMISSIONS matrix didn't register the inbox endpoints that were
added in SP10 (export.csv) and SP14 (candidates/dismiss,
rejected/resubmit, payer-rejected/acknowledge). The matrix's
longest-prefix match meant /api/inbox/candidates/dismiss matched
the '/api/inbox' prefix but only because that single entry
existed; adding the dedicated write-prefix entries makes the
intent explicit and keeps viewer role out of every inbox write
even if the broader /api/inbox prefix is later broadened.
Verified live:
- admin can login, GET /api/inbox/lanes (200), GET
/api/inbox/export.csv (200), POST /api/inbox/candidates/dismiss
(200).
- viewer can login, GET /api/inbox/lanes (200), GET
/api/inbox/export.csv (200), but POST /api/inbox/candidates/dismiss
and POST /api/inbox/rejected/resubmit both return 403.
- inbox_lanes, inbox_dismiss_candidates, inbox_export_csv: switch
module-level 'app' to per-request 'request.app' so per-endpoint
state stays consistent with the test's TestClient target across
importlib.reload() (test_api.py::test_cors_extra_origins_via_env
reloads the api module to mutate CORS allow-lists; pre-reload
endpoints then mutate the wrong app instance).
- _http_exc_handler: when HTTPException.detail is a dict, wrap under
'detail' so the standard envelope stays stable and callers can
branch on body['detail']['error'].
- conftest._auto_init_db: re-resolve cyclone.api.app each fixture
invocation (instead of caching at module load) so the reload pattern
doesn't leave event_bus set on a stale app.
- test_acks.test_migration_latest_idempotent_on_fresh_db: bump
user_version assertion from 12 to 14 after auth migration renumber
(0013 users+sessions, 0014 audit_log.user_id).
Also installs sqlcipher3 + paramiko into the backend venv so the
capability tests can run; both modules were optional deps that
test_db_crypto and test_sftp_paramiko assume are present.
Backend test results: 1008 passed, 9 skipped (gitignored prodfile
fixtures), 0 failed.
The auth cherry-picks brought in 0010_auth_users_and_sessions.sql
(version: 10) and 0011_audit_log_user_id.sql (version: 11), which
collide with main's 0010_payer_rejected_acknowledged.sql (version: 10)
and 0011_processed_inbound_files.sql (version: 11). The migration
runner is filename-sorted but uses the -- version: N header to gate
'application vs skip', so the second file at each version gets
silently skipped — claims.payer_rejected_acknowledged_at never
got ALTERed, breaking every test that touches the claims table.
Renumber to 0013 and 0014 (and bump the -- version headers to
match) so they apply after 0012_backups.sql and main's existing
SP14 / SP15 migrations.
Replaces the skipped 63ae0d2 commit: adds matrix_gate to deps.py and
applies it as a dependency on every authenticated FastAPI route, including
extracted routers (acks, ta1_acks, admin) and inline endpoints in api.py.
The PERMISSIONS matrix in auth/permissions.py controls which roles can
hit which (method, path) combinations; matrix_gate is fail-closed by
default (any endpoint not in the matrix returns 403).
Backend:
- New POST /api/batches/{id}/export-837: regenerate X12 837 files
for a list of claim_ids into a ZIP using HCPF file naming standards,
with a unique interchange/group control number per export. Wire
the clearhouse Loop 1000A (NM1*41 + PER) and per-payer receiver
(NM1*40) blocks so the serializer no longer falls back to
CYCLONE / RECEIVER placeholders.
- /api/parse-837 and /api/parse-835 now surface the server-side
batch_id in both JSON and NDJSON response shapes so the frontend
can hit batch-scoped endpoints without an extra listBatches
round-trip.
- Filename helpers and the 837 serializer updated to match the new
HCPF envelope; tests cover batch export, parse batch_id, and the
serializer's control-number uniqueness guarantee.
Frontend:
- New shared components: ClaimCard, ClaimCard837, DominantKpiCard,
EditorialNote, ExportBar, TickerTape, and a charts/ set
(BarChart, HBarChart, SegmentedBar, AgingBars).
- New useBatchExport hook driving ExportBar's download flow against
the new endpoint.
- ClaimDrawer, Lane, and Layout migrated from raw CSS-variable
colors to Tailwind theme tokens (bg-card, text-foreground,
border/60, etc.) for consistency with the rest of the instrument
chrome; the active tab indicator gains a subtle accent glow.
- Upload, Inbox, Batches, BatchDiff, Reconciliation, and Acks pages
reworked to compose the new shared components and consume the new
batch-scoped API surface (notably ExportBar wired into Batches).
Tooling / Docs:
- Add audit-uiux.mjs and a docs/goodclaim.x12 sample fixture.
- Update ClaimDrawer testids and add coverage for the new
components and the useBatchExport hook.
Rolls up into the v0.2.0 release tag.
SP21 Task 2.5 — the Dashboard's 'Recent activity' card now routes clicks
to the matching entity drawer / page by event kind:
- claim_* → /claims?claim=<id> (drawer in Phase 5)
- provider_added → /providers?provider=<npi> (ProviderDrawer)
- remit_received → toast 'coming in a later phase' (RemitDrawer in Phase 4)
- anything else → toast (manual_match, unknown kinds)
Implementation:
- New src/lib/event-routing.ts with the eventKindToUrl() helper,
plus a unit test covering all 6 + default branches.
- src/components/ActivityFeed.tsx gains an optional onItemClick
prop; when set, each row gets role='button', tabIndex=0, the
drillable hover affordance (chevron + tint), and an Enter/Space
keybinding. e.stopPropagation() is called before the handler so a
parent row click can't double-fire (same fix as Task 2.4).
- src/pages/Dashboard.tsx wires the handler on the 'Recent activity'
card via eventKindToUrl + sonner toast for unhandled kinds.
- Backend: CycloneStore.recent_activity() now exposes claimId and
remittanceId on each row (read from ActivityEvent.claim_id /
remittance_id) so the routing helper has the entity ids it needs.
- The frontend Activity interface gains optional claimId /
remittanceId fields; the in-memory sample data and the
addClaim store action populate them so the dashboard works in
both API-configured and sample-data modes.
Three independent improvements that fix real browser-facing bugs:
1. CORS: allow 127.0.0.1:5173 in addition to localhost:5173. Both
resolve to the same Vite dev server but CORS treats them as distinct
origins, so tabs opened via the IP form silently break.
2. CORS: support CYCLONE_ALLOWED_ORIGINS env var (comma-separated) for
LAN / staging hosts. The middleware reads it at module import.
3. Catch-all exception handler: returns JSON 500 with CORS headers
instead of a bare Uvicorn text/plain response. Without this, any
unhandled exception is misreported by browsers as a CORS error
because the body can't be read without the allow-origin header.
4. IntegrityError → 409: when (batch_id, patient_control_number) is
UNIQUE-constrained and a duplicate collides, return 409 with the
batch id instead of letting the exception 500. Same problem as (3)
for the most common ingest failure mode.
Tests added:
- test_cors_headers_present_for_loopback_ip
- test_cors_extra_origins_via_env (uses importlib.reload because the
allow-list is built at module import)
Adds pure local validators for the 10-digit NPI Luhn checksum (CMS-
published algorithm with the '80840' NPPES prefix) and 9-digit EIN
format (rejects reserved prefixes 00/07/80-89). No NPPES round-trip,
no IRS e-file lookup — catches the 99% typo case at parse time.
Surface:
- cyclone.npi.is_valid_npi / is_valid_tax_id / normalize_tax_id
- CLI: 'cyclone validate-npi <npi>' and 'cyclone validate-tax-id <ein>'
- API: GET /api/admin/validate-provider?npi=&tax_id=
- Parser validator: new R021_npi_checksum rule (warning, not error,
to keep test fixtures with placeholder NPIs ingestible)
- minimal_837p.txt fixture NPI updated from '1234567890' to the
Luhn-valid '1993999998' so strict-mode CLI parses still pass
Tests:
- test_npi.py — 27 cases (Luhn math, valid/invalid NPIs, EIN cases,
normalize_tax_id edge cases)
- test_api_validate_provider.py — 4 cases (both valid, both invalid,
omitted NPI, omitted tax_id)
- test_cli_validate.py — 8 cases (valid/invalid for both subcommands,
exit codes, malformed inputs)
- test_validator.py — 4 new R021 cases (valid Luhn silent, bad Luhn
warning, skipped when format bad, skipped when NPI missing)
Total: 923 tests pass.
Three pure-ASGI middlewares close completeness-review gaps §3.1.4
(no body/rate limits) and §3.1.25 (no security headers):
- BodySizeLimitMiddleware — rejects oversized uploads (50 MB
default, CYCLONE_MAX_BODY_BYTES override). 413 on over-cap
Content-Length and on chunked reads that cross the cap.
- RateLimitMiddleware — sliding-window per-IP limiter (300/min
default, CYCLONE_RATE_LIMIT_PER_MIN override). 429 over the
window. /api/health is exempt.
- SecurityHeadersMiddleware — stamps X-Content-Type-Options,
X-Frame-Options, Referrer-Policy, Permissions-Policy, and a
strict Content-Security-Policy on every response.
Every 413/429 also writes a tamper-evident api.request_rejected
event into the SP11 audit chain so an operator can correlate
rejections with the SP18 JSON logs.
GET /api/health is rewritten to return a subsystem snapshot:
DB connectivity (SELECT 1), MFT scheduler state, backup scheduler
state, live pubsub subscriber counts, last batch id + timestamp.
Returns status='degraded' if any subsystem is unhappy; per-subsystem
errors surfaced in the respective dict.
Cyclone.pubsub.EventBus.stats() — new method for live subscriber
counts.
13 new tests (test_security.py) + 1 updated (test_api.py health
endpoint). All 883 backend tests pass.
Cyclone previously emitted stdlib default-formatted log lines that
operators couldn't parse with anything beyond grep. SP18 replaces
that with:
- JsonFormatter: newline-delimited JSON, ISO-8601 ms timestamps,
structured "extra" dict, exception tracebacks serialized as a
single string.
- CycloneDevFormatter: tabular format for tail -f in dev.
- PiiScrubber: logging.Filter that redacts NPIs, SSNs, DOBs,
patient names — both inline in the message ("npi 1881068062")
and via PHI-keyed extras ({"dob": "1980-04-12"}).
- setup_logging(): idempotent entry point used by the API lifespan
and CLI main; respects CYCLONE_LOG_LEVEL/FILE/JSON/NO_PII_SCRUB.
- CLI --log-format=json|dev + --log-file=… flags.
- Migrate 9 highest-value log sites in scheduler/backup_scheduler/
backup_service to use extra={...} (input_filename, claims, parser,
backup_id, db_fingerprint, etc.).
34 new tests (test_logging_formatter + test_logging_scrubber +
test_logging_setup). All 867 backend tests pass.
Adds automated encrypted backups of the live SQLite file. Closes the
'no backup automation' gap called out in the completeness review
(docs/reviews/2026-06-20-cyclone-completeness-review.md §3.1 #3) and
gives the SP16 MFT scheduler a recovery path when the MFT pipeline
loses days of inbound 999/277CA work in a single crash.
Architecture
------------
- AES-256-GCM with PBKDF2-HMAC-SHA256 (200,000 iters, 16-byte salt)
- Online backups via SQLite's .backup() API — no app downtime
- Salt + passphrase persisted to macOS Keychain (separate accounts
backup.passphrase + backup.salt) so the key is reproducible across
processes
- Two-step restore (initiate → confirm) with a one-shot 64-char hex
token; the second call disposes + rebuilds the engine only if the
token matches within a 5-minute TTL
- Tamper-evident audit chain (SP11) — db.backup_created,
db.backup_failed, db.backup_pruned, db.backup_restored,
db.backup_passphrase_set
- BackupService + BackupScheduler + module-level singletons
- 8 admin endpoints + 6 CLI subcommands
- Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART=true; default
interval 24h, default retention 30 days
- Fallback posture: if no separate passphrase is set and SQLCipher
is enabled, the key is derived from the SQLCipher DB key with a
fixed salt + WARNING log (degraded but never plaintext)
New modules
-----------
- cyclone.backup — PBKDF2, AES-GCM, sidecar format
- cyclone.backup_service — create_now / list / verify / restore / prune / status
- cyclone.backup_scheduler — async tick loop with audit hooks
New surface
-----------
- 8 admin endpoints under /api/admin/backup/*
- 6 CLI subcommands under cyclone backup (init-passphrase, create,
list, verify, restore, prune, status)
- Migration 0012_backups.sql + DbBackup ORM
- store.add_backup_pending()
Tests
-----
- 14 unit tests in test_backup_crypto.py (key derivation, encrypt/
decrypt round-trip, tampered ciphertext, wrong passphrase, sidecar
round-trip, filename format)
- 19 tests in test_backup_service.py (create/list/verify/restore/
prune/status, error handling, fallback key, module singleton)
- 14 API tests in test_api_backup.py (all 8 endpoints + scheduler
endpoints, two-step restore, error responses)
- 10 tests in test_backup_scheduler.py (tick / start / stop /
audit / coalescing / module singleton)
- 5 CLI tests in test_cli_backup.py (create / list / verify /
restore confirm prompt / prune confirm prompt /
init-passphrase minimum-length check)
Total new tests: 62. All pass. Full backend suite: 833 passed,
9 skipped (gitignored prodfiles), 1 warning.
Design doc: docs/superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md
README: new 'Encrypted Backups (SP17)' section, SP17 entry in
Roadmap, retention default documented in §Project layout.
Adds an asyncio-based background scheduler that polls the Gainwell
MFT inbound path, downloads new files, and routes them through the
appropriate parser (999 / 835 / 277CA / TA1). Idempotent (re-ticks
and restarts skip already-processed files via the new
processed_inbound_files table). Crash-safe (per-file try/except so
one bad file doesn't stop the loop).
Lifespan auto-configures from the seeded dzinesco clearhouse's SFTP
block; auto-start is opt-in via CYCLONE_SCHEDULER_AUTOSTART.
Five admin endpoints added:
GET /api/admin/scheduler/status
POST /api/admin/scheduler/start
POST /api/admin/scheduler/stop
POST /api/admin/scheduler/tick
GET /api/admin/scheduler/processed-files?status=&limit=
20 new tests (15 unit + 5 API).
Self-review nits from the router-split commit:
- All four new files lacked a trailing newline (PEP 8 / POSIX).
- acks.py was lazily importing ParseResult999 / serialize_999 inside
get_ack_endpoint. Hoist to module-level — there's no import cycle
(acks.py does not import from cyclone.api), so the imports are safe
to do once.
No behavior change. Targeted tests (test_acks + test_health + test_api_gets)
still pass 41/41.
Step 2 (first half) of the architecture satisfaction loop. api.py
shrank from 2595 to 2452 lines (-143) by extracting three read-only
resource groups into cyclone.api_routers:
- health.py: GET /api/health (1 endpoint)
- acks.py: GET /api/acks, GET /api/acks/{ack_id} (2 endpoints)
- ta1_acks.py: GET /api/ta1-acks, GET /api/ta1-acks/{ack_id} (2 endpoints)
Each router owns its endpoint bodies + the small UI-shape helper that
goes with them (_ack_to_ui, _ta1_to_ui, _serialize_ta1_from_row). The
helpers stay in the router file rather than being shared because each
is only used by its own endpoints.
api.py now ends the app-wiring section with three include_router()
calls. The new package is named cyclone.api_routers (not
cyclone.api.routers) to avoid the Python package-vs-same-named-module
ambiguity that would shadow the existing cyclone.api module.
Verifies: 41 targeted tests (test_acks, test_health, test_api_gets)
pass, full pytest is 8 failed / 735 passed / 16 skipped — identical
to clean main baseline. Live curl against the running server:
GET /api/health -> 200, GET /api/acks -> 200, GET /api/ta1-acks -> 200.
See /tmp/refactor-cyclone.md for the full plan.
Nora
tyler
tyler
Cyclone
Tyler