Cyclone previously emitted stdlib default-formatted log lines that
operators couldn't parse with anything beyond grep. SP18 replaces
that with:
- JsonFormatter: newline-delimited JSON, ISO-8601 ms timestamps,
structured "extra" dict, exception tracebacks serialized as a
single string.
- CycloneDevFormatter: tabular format for tail -f in dev.
- PiiScrubber: logging.Filter that redacts NPIs, SSNs, DOBs,
patient names — both inline in the message ("npi 1881068062")
and via PHI-keyed extras ({"dob": "1980-04-12"}).
- setup_logging(): idempotent entry point used by the API lifespan
and CLI main; respects CYCLONE_LOG_LEVEL/FILE/JSON/NO_PII_SCRUB.
- CLI --log-format=json|dev + --log-file=… flags.
- Migrate 9 highest-value log sites in scheduler/backup_scheduler/
backup_service to use extra={...} (input_filename, claims, parser,
backup_id, db_fingerprint, etc.).
34 new tests (test_logging_formatter + test_logging_scrubber +
test_logging_setup). All 867 backend tests pass.
Adds automated encrypted backups of the live SQLite file. Closes the
'no backup automation' gap called out in the completeness review
(docs/reviews/2026-06-20-cyclone-completeness-review.md §3.1 #3) and
gives the SP16 MFT scheduler a recovery path when the MFT pipeline
loses days of inbound 999/277CA work in a single crash.
Architecture
------------
- AES-256-GCM with PBKDF2-HMAC-SHA256 (200,000 iters, 16-byte salt)
- Online backups via SQLite's .backup() API — no app downtime
- Salt + passphrase persisted to macOS Keychain (separate accounts
backup.passphrase + backup.salt) so the key is reproducible across
processes
- Two-step restore (initiate → confirm) with a one-shot 64-char hex
token; the second call disposes + rebuilds the engine only if the
token matches within a 5-minute TTL
- Tamper-evident audit chain (SP11) — db.backup_created,
db.backup_failed, db.backup_pruned, db.backup_restored,
db.backup_passphrase_set
- BackupService + BackupScheduler + module-level singletons
- 8 admin endpoints + 6 CLI subcommands
- Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART=true; default
interval 24h, default retention 30 days
- Fallback posture: if no separate passphrase is set and SQLCipher
is enabled, the key is derived from the SQLCipher DB key with a
fixed salt + WARNING log (degraded but never plaintext)
New modules
-----------
- cyclone.backup — PBKDF2, AES-GCM, sidecar format
- cyclone.backup_service — create_now / list / verify / restore / prune / status
- cyclone.backup_scheduler — async tick loop with audit hooks
New surface
-----------
- 8 admin endpoints under /api/admin/backup/*
- 6 CLI subcommands under cyclone backup (init-passphrase, create,
list, verify, restore, prune, status)
- Migration 0012_backups.sql + DbBackup ORM
- store.add_backup_pending()
Tests
-----
- 14 unit tests in test_backup_crypto.py (key derivation, encrypt/
decrypt round-trip, tampered ciphertext, wrong passphrase, sidecar
round-trip, filename format)
- 19 tests in test_backup_service.py (create/list/verify/restore/
prune/status, error handling, fallback key, module singleton)
- 14 API tests in test_api_backup.py (all 8 endpoints + scheduler
endpoints, two-step restore, error responses)
- 10 tests in test_backup_scheduler.py (tick / start / stop /
audit / coalescing / module singleton)
- 5 CLI tests in test_cli_backup.py (create / list / verify /
restore confirm prompt / prune confirm prompt /
init-passphrase minimum-length check)
Total new tests: 62. All pass. Full backend suite: 833 passed,
9 skipped (gitignored prodfiles), 1 warning.
Design doc: docs/superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md
README: new 'Encrypted Backups (SP17)' section, SP17 entry in
Roadmap, retention default documented in §Project layout.
- backend/src/cyclone/parsers/serialize_837.py — full-rebuild 837P serializer.
Emits envelope (ISA/GS/ST/SE/GE/IEA + BHT) + submitter/receiver/billing
provider/subscriber/payer hierarchy + editable segments (CLM/REF*G1/HI)
+ per-service-line LX/SV1/DTP*472/REF*6R — all from canonical
ClaimOutput fields.
Pivoted from spec §3.1 hybrid to full rebuild because ClaimOutput.raw_segments
only captures post-CLM segments (CLM, REF*G1, HI, LX, SV1 pairs) — not the
envelope or hierarchies. A pass-through approach cannot regenerate those
without expanding raw_segments in parse_837.py (out of scope for this SP).
- backend/tests/test_serialize_837.py — 36 tests covering envelope shape,
hierarchy segments, claim-level builders, service-line builders, edited-field
propagation, round-trip, custom sender/receiver IDs, and resubmit helper.
- backend/tests/test_prodfiles_smoke.py::test_claims_prodfile_round_trip —
every file in docs/prodfiles/claims/ (113 files) round-trips through
serialize_837 → parse_837_text with deep-equal ClaimOutput (modulo
validation, which is recomputed by the parser).
- docs/superpowers/plans/2026-06-20-cyclone-serialize-837.md — full plan
with amendment note documenting the Approach A pivot.
Per session convention, plan note about unrelated modifications to
parse_835.py / fixtures stashed separately.
- Mark SP4 as fully shipped (batch diff, search, CSV, a11y all landed)
- Add SP6 (Inbox) and SP7 (Per-line reconciliation) sections
- Add SP6 + SP7 endpoint inventories
- Note next up: outbound 837P serializer
The workflow-automation plan was authored but never committed; the
features it specified have shipped, so commit it for the historical
record alongside this README refresh.
Sub-project 7 adds serialize_837.py so a parsed ClaimOutput can be
regenerated as a complete X12 837P file (the shape of
docs/prodfiles/claims/). Unblocks the SP6 resubmit lane end-to-end
and adds a 'Download 837' affordance on the claim drawer.
Approach: hybrid (fresh envelope + selective body rebuild). Stable
segments (provider/subscriber/payer hierarchies) pass through from
claim.raw_segments; editable segments (CLM, REF*G1, HI, SV1, DTP*472)
are rebuilt from canonical ClaimOutput fields so post-parse edits
propagate to the output.
Out of scope: 997 ACK, claim <-> TA1/999 linking, in-app claim
editing UI, multi-CLM envelope batching, Inbox/drawer redesign.
Brings the SP5 live-tail implementation into main:
Backend
* EventBus (cyclone.pubsub): per-kind fan-out with drop-oldest overflow
* FastAPI lifespan initializes the bus + db once per process
* store.add() publishes claim_written / remittance_written /
activity_recorded events on every batch write
* GET /api/{claims,remittances,activity}/stream: NDJSON snapshot +
live subscription + 15s idle heartbeat
* EventBus.unsubscribe() lets the tail loop release its queue on
client disconnect (no queue leak per open stream)
Frontend
* src/lib/tail-stream.ts: streamTail() async-generator over fetch
* src/store/tail-store.ts: zustand with FIFO cap 10k per slice
* src/hooks/useTailStream.ts: connecting/live/reconnecting/stalled/error/closed
state machine with 1→2→4→8→16→30s backoff
* src/hooks/useMergedTail.ts: base + tail merge with filter
* src/components/TailStatusPill.tsx: badge + Reconnect button
* Claims, Remittances, ActivityLog pages wired to the tail
Tests
* 437 backend tests pass (was 418 before SP5)
* 154 frontend tests pass (was 124)
* npm run typecheck clean
* end-to-end smoke: open /api/claims/stream, POST 837, see new claims
arrive in real time without refresh
# Conflicts:
# src/pages/ActivityLog.tsx
# src/pages/Remittances.test.tsx
# src/pages/Remittances.tsx
Replaces the in-memory store with SQLite via SQLAlchemy 2.0; adds
automatic 837P ↔ 835 reconciliation on every 835 parse; ships a
manual /reconciliation page; extends the claim lifecycle to a
7-state model with reversal handling.
Key decisions (locked from brainstorming):
- SQLite at ~/.local/share/cyclone/cyclone.db, overridable via
CYCLONE_DB_URL (Postgres escape hatch)
- SQLAlchemy 2.0 sync ORM; PRAGMA user_version migration runner
- Strict auto-match on patient_control_number (CLM01 == CLP01)
within +/-7 days of service_date
- 7-state claim model with reversal that sets Claim.state=REVERSED
and preserves prior state on Match.prior_claim_state
- Reconciliation triggered inside store.add() after ERA persist;
fail-soft (logged + activity event, never blocks parse)
- 3 new API endpoints; new /reconciliation page; same palette
and aesthetic voice as sub-project 1
Spec is approved-section-by-section and self-reviewed.
- Approved design for the Python 837P parser module under backend/
- Pydantic v2, click CLI, structural + CO Medicaid validation
- One JSON file per claim, summary.json, continue-on-failure
- Includes .gitignore for the Python + Node stack
Tyler
Cyclone