Add check_matched_pair_drift() at module level in store.py — read-only
audit of the Claim.matched_remittance_id ↔ Remittance.claim_id FK pair.
Logs WARNING on drift with up to 5 examples of each of two cases
(claim-side and remit-side), returns the count of drifted rows. Wired
into api.py::lifespan right after db.init_db() and ensure_clearhouse_seeded(),
wrapped in try/except so a query failure logs but doesn't crash boot.
The symmetric-write + symmetric-clear invariants on
manual_match / manual_unmatch were already correct; this commit pins
them with focused regression tests in test_store_match_invariant.py
(8 tests covering both directions, both happy-path, rollback pin, and
drift-check unit behavior). PCN asymmetry in the fixture prevents the
auto-match inside CycloneStore.add (Task 10) from pre-pairing, so
manual_match is the only writer.
Migration 0016 adds the missing ix_claims_matched_remittance_id index —
the drift check scans WHERE matched_remittance_id IS NOT NULL on every
boot and would become a full-table scan past ~10k claims. Symmetric
with ix_remittances_claim_id (added in 0007). Migration tests bumped
from head=15 to head=16.
The Acks page silently capped at 100 of 1056 rows in the operator's
DB: the eyebrow read `${items.length} on file` (not the server's
`total`), the KPI strip summed from the 100 visible items, and the
endpoint accepted no `offset` — so the user had no signal that 956
more acks existed. Same class of bug on the Activity page (cap=200,
no 'X of Y' hint).
Backend
- /api/acks (acks.py): add `offset`, bump `le=1000`→`le=5000`,
slice `rows[offset:offset+limit]`, return server-side
`aggregates` (accepted/rejected/received summed over the full
row set, not the page) so the KPI strip reflects every persisted
999 instead of just the visible 50.
- /api/activity list + stream (api.py): bump `le=500`→`le=5000` so
the page can ask for a denser snapshot.
- /api/277ca-acks (api.py): bump `le=1000`→`le=5000` for
consistency.
- /api/ta1-acks: left at `le=1000` — TA1s aren't shipped today and
the structural fix (offset + aggregates) wasn't applied, so a
larger cap would just make the same latent silent-failure easier
to hit. (TODO: fold in the same shape when Gainwell starts
shipping TA1s.)
Frontend
- listAcks (api.ts): accept `offset`, surface `aggregates`,
adapt wire `*_count` keys to the in-page
`accepted`/`rejected`/`received` shape so the page can use
`data.aggregates` as a drop-in for the page-local fallback
accumulator.
- useAcks (hooks): pass `offset` through; return type carries
`aggregates`.
- Acks.tsx: add `page` state (PAGE_SIZE=50), use `data.total`
for eyebrow + watermark (not `items.length`), use
`data.aggregates` for the KPI strip (with in-page fallback
accumulator on first paint), render `<Pagination>` when
`totalCount > PAGE_SIZE`. Footer row reads "N rows on file"
instead of "N rows".
- ActivityLog.tsx: bump `limit: 200`→`limit: 500`, eyebrow reads
"Activity · showing N most recent" to make the bounded-window
semantics honest (the endpoint doesn't expose a true total — it
reports events matching the current kind/since filter, capped at
the request limit).
Tests
- test_acks.py: 4 new tests pin the fix:
1. `offset` walks the full set; `has_more` flips at the
boundary.
2. `aggregates` reflects the full row set, not the page (the
silent-failure pin) — and stays stable across page slices.
3. `limit` cap of 5000 is enforced (422 above it).
4. `offset` past the end returns an empty page with stable
aggregates (a stale UI page state across a row count change
must not 500 or zero the KPIs).
Live smoke-verified: /api/acks?limit=2&offset=0 vs ?offset=2 return
the expected row slices, aggregates stable at 15/10/15 for 5 seeded
rows, /api/acks?limit=10000 rejected with 422.
Triage note: the TA1 section (`Ta1AcksSection`, lines 609-616 of
Acks.tsx) has the same latent silent-failure pattern (page-sums
KPIs, no offset on /api/ta1-acks). Left untouched because the
empty-state copy says Gainwell doesn't ship TA1s today and the
larger structural fix belongs in a follow-up.
Adds migration 0015_drop_claims_unique_constraint.sql that recreates the
claims table without the inline UNIQUE(batch_id, patient_control_number)
constraint, plus tests proving the migration runs cleanly and the
constraint is not re-introduced.
- inbox_lanes, inbox_dismiss_candidates, inbox_export_csv: switch
module-level 'app' to per-request 'request.app' so per-endpoint
state stays consistent with the test's TestClient target across
importlib.reload() (test_api.py::test_cors_extra_origins_via_env
reloads the api module to mutate CORS allow-lists; pre-reload
endpoints then mutate the wrong app instance).
- _http_exc_handler: when HTTPException.detail is a dict, wrap under
'detail' so the standard envelope stays stable and callers can
branch on body['detail']['error'].
- conftest._auto_init_db: re-resolve cyclone.api.app each fixture
invocation (instead of caching at module load) so the reload pattern
doesn't leave event_bus set on a stale app.
- test_acks.test_migration_latest_idempotent_on_fresh_db: bump
user_version assertion from 12 to 14 after auth migration renumber
(0013 users+sessions, 0014 audit_log.user_id).
Also installs sqlcipher3 + paramiko into the backend venv so the
capability tests can run; both modules were optional deps that
test_db_crypto and test_sftp_paramiko assume are present.
Backend test results: 1008 passed, 9 skipped (gitignored prodfile
fixtures), 0 failed.
Adds automated encrypted backups of the live SQLite file. Closes the
'no backup automation' gap called out in the completeness review
(docs/reviews/2026-06-20-cyclone-completeness-review.md §3.1 #3) and
gives the SP16 MFT scheduler a recovery path when the MFT pipeline
loses days of inbound 999/277CA work in a single crash.
Architecture
------------
- AES-256-GCM with PBKDF2-HMAC-SHA256 (200,000 iters, 16-byte salt)
- Online backups via SQLite's .backup() API — no app downtime
- Salt + passphrase persisted to macOS Keychain (separate accounts
backup.passphrase + backup.salt) so the key is reproducible across
processes
- Two-step restore (initiate → confirm) with a one-shot 64-char hex
token; the second call disposes + rebuilds the engine only if the
token matches within a 5-minute TTL
- Tamper-evident audit chain (SP11) — db.backup_created,
db.backup_failed, db.backup_pruned, db.backup_restored,
db.backup_passphrase_set
- BackupService + BackupScheduler + module-level singletons
- 8 admin endpoints + 6 CLI subcommands
- Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART=true; default
interval 24h, default retention 30 days
- Fallback posture: if no separate passphrase is set and SQLCipher
is enabled, the key is derived from the SQLCipher DB key with a
fixed salt + WARNING log (degraded but never plaintext)
New modules
-----------
- cyclone.backup — PBKDF2, AES-GCM, sidecar format
- cyclone.backup_service — create_now / list / verify / restore / prune / status
- cyclone.backup_scheduler — async tick loop with audit hooks
New surface
-----------
- 8 admin endpoints under /api/admin/backup/*
- 6 CLI subcommands under cyclone backup (init-passphrase, create,
list, verify, restore, prune, status)
- Migration 0012_backups.sql + DbBackup ORM
- store.add_backup_pending()
Tests
-----
- 14 unit tests in test_backup_crypto.py (key derivation, encrypt/
decrypt round-trip, tampered ciphertext, wrong passphrase, sidecar
round-trip, filename format)
- 19 tests in test_backup_service.py (create/list/verify/restore/
prune/status, error handling, fallback key, module singleton)
- 14 API tests in test_api_backup.py (all 8 endpoints + scheduler
endpoints, two-step restore, error responses)
- 10 tests in test_backup_scheduler.py (tick / start / stop /
audit / coalescing / module singleton)
- 5 CLI tests in test_cli_backup.py (create / list / verify /
restore confirm prompt / prune confirm prompt /
init-passphrase minimum-length check)
Total new tests: 62. All pass. Full backend suite: 833 passed,
9 skipped (gitignored prodfiles), 1 warning.
Design doc: docs/superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md
README: new 'Encrypted Backups (SP17)' section, SP17 entry in
Roadmap, retention default documented in §Project layout.
Adds an asyncio-based background scheduler that polls the Gainwell
MFT inbound path, downloads new files, and routes them through the
appropriate parser (999 / 835 / 277CA / TA1). Idempotent (re-ticks
and restarts skip already-processed files via the new
processed_inbound_files table). Crash-safe (per-file try/except so
one bad file doesn't stop the loop).
Lifespan auto-configures from the seeded dzinesco clearhouse's SFTP
block; auto-start is opt-in via CYCLONE_SCHEDULER_AUTOSTART.
Five admin endpoints added:
GET /api/admin/scheduler/status
POST /api/admin/scheduler/start
POST /api/admin/scheduler/stop
POST /api/admin/scheduler/tick
GET /api/admin/scheduler/processed-files?status=&limit=
20 new tests (15 unit + 5 API).
Closes the gap between the SP10 backend (5 lanes) and the SP6
frontend (4 lanes). The Payer-Rejected lane (277CA STC A4/A6/A7)
is now rendered alongside Rejected/Candidates/Unmatched/Done,
with an Acknowledge bulk action that drops claims from the
working surface without erasing the original 277CA rejection
event (audit log stays intact, SP11).
Backend:
* Migration 0010: add payer_rejected_acknowledged_at +
payer_rejected_acknowledged_actor columns + partial index.
* db.py: surface the two new columns on the Claim model.
* inbox_lanes.py: filter acknowledged claims out of the
payer_rejected lane; expose the new fields on the row payload
for forward-compat (e.g. a future 'Recently acknowledged' view).
* api.py:
- POST /api/inbox/payer-rejected/acknowledge
Bulk-acknowledge. Idempotent. Returns transitioned /
already_acked / not_found / not_rejected counts so the UI
can show '3 of 5 were already acknowledged' on a noop bulk.
Writes a 'claim.payer_rejected_acknowledged' event to the
SP11 hash-chained audit log.
- GET /api/inbox/export.csv: accept 'payer_rejected' lane.
* test_acks.py: bump user_version assertion to 10.
* test_lane_filter_acknowledged.py: 4 tests for the lane filter
and forward-compat row payload.
* test_payer_rejected_acknowledge.py: 6 tests for the endpoint
(happy path, idempotency, no-op on non-rejected, missing
ids, 400 on empty, audit-log wiring + chain integrity).
Frontend:
* lib/inbox-api.ts: add payer_rejected to InboxLanes, add
acknowledgePayerRejected(), update exportInboxCsvUrl union.
* hooks/useInboxLanes.ts: add payer_rejected to initial state.
* hooks/useInboxLanes.test.ts: add payer_rejected to mocks.
* components/inbox/BulkBar.tsx: add 'payer_rejected' lane with
Acknowledge action (no Resubmit, no Dismiss — payer-rejected
is not eligible for either).
* components/inbox/BulkBar.test.tsx: add payer_rejected test.
* pages/Inbox.tsx: render the 5th lane, hook up onAcknowledge,
include payer_rejected in the needEyes count.
* pages/Inbox.test.tsx: 3 new tests (5-lane render, need-eyes
count, acknowledge action hits the right endpoint).
* components/inbox/InboxHeader.tsx: doc comment now explains
why payer_rejected rolls up into need-eyes.
Pre-existing typecheck warnings in BulkBar.test.tsx / InboxRow
.test.tsx / Lane.tsx / download.test.ts are unchanged from
main — not touched here.
Test counts: backend 724 -> 734 (+10). Frontend 350 -> 354 (+4).
Adds:
- claims.rejection_reason, claims.rejected_at, claims.resubmit_count
- claims.state_changed_at (was missing, needed for Done-today lane)
- ix_claims_state_changed_at composite index
Also fixes the ClaimState count assertion in test_db_models.py
(7 → 8) to match the REJECTED enum value added in the previous
commit, and bumps the user_version expectation in test_acks.py
(3 → 4) for the same reason.
Three related changes for real CO Medicaid data:
1. Drop UNIQUE(batch_id, patient_control_number) on claims and
UNIQUE(batch_id, payer_claim_control_number) on remittances. The X12
spec allows multiple CLM segments per 2000B subscriber loop and 835
ERAs can repeat a payer_claim_control_number for reversals. Claim/
remittance identity is provided by the primary key (claims.id = CLM01,
remittances.id = CLP01).
2. Add validator rule R835_MULTI_BPR warning for files with multiple BPR
segments (CO Medicaid split-payment pattern). The parser already sums
BPR02 paid_amounts; this surfaces the non-standard data to operators.
3. Skip R835_BAL_BPR_vs_CLP04 when BPR01='I' (Information Only 835).
In that mode BPR02 is informational and the per-claim CLP04 totals
are authoritative — a diff is expected, not an error.
Migration 0003 handles the drop with IF EXISTS so fresh DBs skip cleanly.
Updates affected tests to reflect new schema (no UNIQUE constraint on
batch_id + patient_control_number / payer_claim_control_number).
Fixes test_api_835::test_prodfile_round_trip_persists_separately which
was failing on real production data.
Nora
Tyler