Adds automated encrypted backups of the live SQLite file. Closes the
'no backup automation' gap called out in the completeness review
(docs/reviews/2026-06-20-cyclone-completeness-review.md §3.1 #3) and
gives the SP16 MFT scheduler a recovery path when the MFT pipeline
loses days of inbound 999/277CA work in a single crash.
Architecture
------------
- AES-256-GCM with PBKDF2-HMAC-SHA256 (200,000 iters, 16-byte salt)
- Online backups via SQLite's .backup() API — no app downtime
- Salt + passphrase persisted to macOS Keychain (separate accounts
backup.passphrase + backup.salt) so the key is reproducible across
processes
- Two-step restore (initiate → confirm) with a one-shot 64-char hex
token; the second call disposes + rebuilds the engine only if the
token matches within a 5-minute TTL
- Tamper-evident audit chain (SP11) — db.backup_created,
db.backup_failed, db.backup_pruned, db.backup_restored,
db.backup_passphrase_set
- BackupService + BackupScheduler + module-level singletons
- 8 admin endpoints + 6 CLI subcommands
- Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART=true; default
interval 24h, default retention 30 days
- Fallback posture: if no separate passphrase is set and SQLCipher
is enabled, the key is derived from the SQLCipher DB key with a
fixed salt + WARNING log (degraded but never plaintext)
New modules
-----------
- cyclone.backup — PBKDF2, AES-GCM, sidecar format
- cyclone.backup_service — create_now / list / verify / restore / prune / status
- cyclone.backup_scheduler — async tick loop with audit hooks
New surface
-----------
- 8 admin endpoints under /api/admin/backup/*
- 6 CLI subcommands under cyclone backup (init-passphrase, create,
list, verify, restore, prune, status)
- Migration 0012_backups.sql + DbBackup ORM
- store.add_backup_pending()
Tests
-----
- 14 unit tests in test_backup_crypto.py (key derivation, encrypt/
decrypt round-trip, tampered ciphertext, wrong passphrase, sidecar
round-trip, filename format)
- 19 tests in test_backup_service.py (create/list/verify/restore/
prune/status, error handling, fallback key, module singleton)
- 14 API tests in test_api_backup.py (all 8 endpoints + scheduler
endpoints, two-step restore, error responses)
- 10 tests in test_backup_scheduler.py (tick / start / stop /
audit / coalescing / module singleton)
- 5 CLI tests in test_cli_backup.py (create / list / verify /
restore confirm prompt / prune confirm prompt /
init-passphrase minimum-length check)
Total new tests: 62. All pass. Full backend suite: 833 passed,
9 skipped (gitignored prodfiles), 1 warning.
Design doc: docs/superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md
README: new 'Encrypted Backups (SP17)' section, SP17 entry in
Roadmap, retention default documented in §Project layout.
manual_match previously only flipped the claim↔remit FK and the
claim state — it never ran line-level reconciliation, so manually-
paired claims surfaced empty line-reconciliation rows to the UI
and skipped CLP-level CAS aggregate recompute.
Refactor reconcile.run() to call a new per-pair helper
_reconcile_pair(session, claim, remittance) that:
- clears any existing LineReconciliation rows for the claim
(idempotent re-run; safe across manual_unmatch + manual_rematch
cycles that may pair the claim with a different remittance),
- reads 837 SV1 lines from Claim.raw_json and 835 SVC rows from
ServiceLinePayment,
- runs match_service_lines() and persists a LineReconciliation row
per side,
- recomputes Remittance.claim_level_adjustment_amount (CLP-level
CAS) and Remittance.adjustment_amount (total CAS).
manual_match now calls this helper after the FK is set and before
commit, mirroring the auto-match path. Reversals are skipped (they
don't have SV1↔SVC line pairs; per §7.3).
Tests (test_store_reconcile.py):
- test_manual_match_populates_line_reconciliation_rows: end-to-end
check that a manual pair writes the expected matched +
unmatched_837_only rows plus zero CAS aggregates.
- test_manual_match_idempotent_line_reconciliation: after
manual_unmatch + manual_rematch + inserting a CLP-level CAS row
directly, the claim has exactly two fresh LineReconciliation rows
(no duplicates) and the remittance aggregate reflects the new CAS.
Smoke tested end-to-end via TestClient: parse co_medicaid_837p.txt
+ co_medicaid_835.txt, auto-matcher skips (PCNs differ), manual
match via POST /api/reconciliation/match, then
GET /api/claims/{id}/line-reconciliation returns 2 rows and
GET /api/inbox/lanes shows matched_remittance.total_lines=2.
Note: matched_lines may be 0 on real 835 fixtures due to a
pre-existing parser bug in _consume_service_payment that swaps
SVC04 (units) and SVC05 (unit-of-measure) when reading units from
the segment. The SP7 strict-match criterion requires units parity,
so the misread produces None on the SVC side and strict-match
never succeeds. Surfaced as a follow-up; the SP7 work itself is
correct (line counts are computed and persisted).
The P2 subagent shipped a UI chevron+expansion in Remittances.tsx that
read adjustments from the list payload, but iter_remittances() didn't
include the CasAdjustment rows. Result: the expansion was always empty
when a remittance was opened from the list (the detail endpoint worked).
Fix: bulk-fetch all CasAdjustment rows for the remittance_ids in one
query, then attach them as the 'adjustments' array. N+1-free.
Replace the in-memory InMemoryStore with a SQLAlchemy-backed
CycloneStore that persists Batch, Claim, Remittance, Match, and
ActivityEvent rows to a configurable DB engine (sqlite by default,
overridable via CYCLONE_DB_URL).
Public API preserved: add / get_batch / iter_claims / iter_remittances
/ distinct_providers / recent_activity. New T10/T12 stubs:
list_unmatched, manual_match, manual_unmatch.
Backward-compat shims for tests that called ._batches.clear() or
acquired ._lock as a context manager.
Idempotency: add() does a per-row session.get(Claim, c.id) (and
s.get(Remittance, ...)) check before each insert; duplicates are
skipped with a warning. This makes re-uploading the same fixture
idempotent instead of raising IntegrityError on the PK.
Auto-init DB fixture (backend/tests/conftest.py) sets
CYCLONE_DB_URL to a per-test sqlite file and calls db.init_db()
once per test, replacing the old module-scoped in-memory fixture.
- iter_claims/iter_remittances now return list[dict] (snapshot under
the lock, return outside) so slow consumers no longer hold the
write lock during iteration
- iter_claims gains payer (case-insensitive substring) and
date_from/date_to filters via the new _date_in_bounds helper
- iter_remittances wires date_from/date_to and payer
- BatchRecord split into BatchRecord837 / BatchRecord835 with
__new__ dispatch so isinstance narrows result correctly,
eliminating the four 'type: ignore[assignment]' band-aids
- BatchRecord now rejects naive parsed_at via model_validator
- to_ui_remittance's adjustmentAmount=0.0 is documented as a
TODO for sub-project 2 reconciliation
151 passed, 1 skipped (no regressions; +6 tests for new filters)
Tyler