The README only documented up to SP8 even though SP9-SP13 (multi-payer
+ multi-NPI + SFTP stub, 277CA + Payer-Rejected lane, tamper-evident
audit log, optional SQLCipher encryption, paramiko SFTP wire-up) are
all merged into main.
This brings the README back in line with the codebase:
- Inbox section: bump lane count from four to five, add the
payer_rejected lane (and its monotonic stamping contract).
- Add a Multi-Payer, Multi-NPI & Clearhouse section (SP9): the
providers / payers / payer_configs / clearhouse tables, the
config/payers.yaml loader, the new config + clearhouse endpoints.
- Add a 277CA Claim Acknowledgment section (SP10): parse-277ca +
277ca-acks, the Payer-Rejected stamp semantics.
- Add a Tamper-Evident Audit Log section (SP11): hash-chained
audit_log, the verify endpoint.
- Add an Encryption at Rest section (SP12): Keychain-stored
SQLCipher key, transparent fallback to plain SQLite.
- Add an SFTP Wire-Up (paramiko) section (SP13): the production
submit path and the file-naming template.
- Persistence section: link to the Encryption at Rest section.
- Roadmap: bump shipped sub-projects to 2-13, add a one-line
pointer to the completeness review for the honest industry-gap
accounting.
- Project layout: add api_helpers, db_crypto, audit_log, the new
inbox_* modules, providers/payers/secrets, the new test files,
and the config/ + docs/reviews/ + docs/superpowers/specs/ tree.
- backend/README: refresh the endpoint table to include the
parse-999/parse-ta1/parse-277ca/eligibility/clearhouse/admin
surface; point at the root README for the full route list.
Replace SftpClient stub write_file/list_inbound/read_file
implementations with real paramiko SSHClient + SFTPClient
calls. The public API (SftpClient.write_file, list_inbound,
read_file, get_secret) is unchanged from SP9 — same signature,
same return types — so the API layer needs no changes.
Real-mode behavior:
* _connect() returns a context manager yielding (ssh, sftp);
closes both on exit. Lazy-imports paramiko so the stub-only
test path doesn't need the dependency.
* Auth resolves from SftpBlock.auth: password_keychain_account
(MFT model) or key_file + optional key_passphrase_keychain_account.
Missing Keychain entries fail loud (RuntimeError) rather than
silently attempting empty-password auth.
* write_file: opens sftp.open(remote, 'wb') and writes bytes;
mkdirs the parent dir (idempotent — MFT pre-creates FromHPE/ToHPE).
* list_inbound: listdir_attr + per-file download into local
staging cache; skips directory entries (0o040000 mask).
* read_file: download via shutil.copyfileobj into BytesIO.
Stub mode is unchanged. AutoAddPolicy for first-time MFT host
fingerprint; operator should pin the key for production.
Adds tests/test_sftp_paramiko.py: 9 tests covering
* stub still works
* real-mode connect builds correct paramiko call from
password_keychain_account, raises on missing Keychain,
raises on missing auth config, raises on STUB_SECRET
* write_file opens 'wb' on the right path and writes bytes
* list_inbound translates attrs into InboundFile records and
caches files locally; skips dirs
Removes 2 obsolete tests in test_sftp_stub.py that expected
SP13-mode to raise NotImplementedError.
pyproject.toml: new optional 'sftp' extra (paramiko>=3.4,<6).
- New cyclone.db_crypto module:
* is_sqlcipher_available() — capability check
* is_encryption_enabled() — Keychain key + sqlcipher3 present
* get_db_key() — reads 'cyclone.db.key' from Keychain
* make_sqlcipher_connect_creator(url, key) — SQLAlchemy creator
- db._make_engine() now switches to SQLCipher when key is present
- pyproject.toml: optional 'sqlcipher' extra (sqlcipher3>=0.6,<1)
- Fallback: without Keychain key, DB stays plain SQLite (no surprise
behavior for operators who haven't set up encryption yet)
- Verified: encrypted file is unreadable as plain SQLite, wrong key
raises on first query, migrations + ORM work transparently
- HIPAA §164.312(a)(2)(iv) compliance note in docs
Tests: 705 -> 717 (12 new for SQLCipher). All 717 backend tests pass.
- Mark SP8 as shipped (Sub-projects 2 through 8)
- Drop the 'Next up: outbound 837P serializer' trailer
- New 'Outbound 837 Serializer' section above 'Per-Line Adjustment
Audit': design rationale (Approach A full rebuild vs spec §3.1
hybrid), two-surface UX (single-claim drawer download + multi-claim
inbox ZIP bundle), pointers to the serializer module, the two API
endpoints, the frontend helpers, and the UI entry points.
- New 'SP8 endpoints' subsection in the endpoint inventory.
- inbox-api.test.ts: pin ?download=true POST contract; blob + filename +
X-Cyclone-Serialize-Errors parsing; non-2xx error surfacing.
- Inbox.test.tsx: end-to-end multi-select → Resubmit + Download path
verifies api call args and downloadBlob wiring.
- inbox-api.ts: drop redundant isConfigured short-circuit in download
variant (backend has its own auth gate) and switch error reading to
res.text() to match Blob response shape.
Wire the new ``?download=true`` resubmit endpoint into the Inbox
page. Operators can now ask the backend for a ZIP of regenerated
837s straight from the rejected-claims bulk action, with the
``X-Cyclone-Serialize-Errors`` header surfaced as a non-blocking
warning so partial successes don't swallow per-claim failures.
* ``src/lib/inbox-api.ts``: new ``resubmitRejectedWithDownload``
helper returning ``{blob, filename, serializeErrors}`` so callers
can hand the bundle to the new ``downloadBlob`` utility without
re-parsing headers.
* ``src/lib/download.ts``: new ``downloadBlob(blob, filename)`` plus
a test covering the extension/content-type mapping and the
"use the suggested filename when present" rule.
* ``src/pages/Inbox.tsx``: rejected-claims bulk action now exposes
a "Resubmit & download" button next to the existing JSON path,
wired through the helper. Conflicts and per-claim serialize
errors render in the existing toast/result surface.
Tests: 4 new download.ts tests, 5 inbox-api tests (including
serialize-errors header parsing).
Add an optional ``?download=true`` query param to
``POST /api/inbox/rejected/resubmit`` that returns the same operation
result as a ZIP archive of regenerated 837P files (one
``claim-{id}.x12`` per successfully resubmitted claim) rather than the
JSON envelope.
Why: operators who mass-resubmit rejected claims want to hand the
files straight to their clearinghouse; round-tripping through copy-
paste is error-prone.
Implementation notes:
* Uses ``serialize_837_for_resubmit`` so each X12 file in the bundle
gets a unique interchange/group control number (back-to-back
resubmits would otherwise collide on ISA13/GS06 = "000000001").
* Conflicts and missing ids are deliberately excluded from the ZIP
— the user already saw them in the JSON path on prior calls; the
download is the "give me the files I asked for" payload.
* Empty resubmit + download returns 200 with an empty ZIP so the UI
can still hand the user a downloadable artifact.
Tests (test_inbox_endpoints.py): 2 new tests covering the success
shape (one .x12 per accepted claim) and the conflict-exclusion
contract.
Three pieces:
- src/lib/download.ts: generic downloadTextFile(filename, mime, text)
helper. Mirrors csv.ts:downloadCsv but takes an explicit MIME type and
drops the BOM prepend (which would corrupt the ISA segment).
- src/lib/api.ts: serializeClaim837(id) → {text, filename}. Fetches
GET /api/claims/{id}/serialize-837, pulls the suggested filename from
Content-Disposition (falls back to claim-{id}.x12 if the header is
missing). Throws ApiError on non-2xx so callers can branch on .status.
- ClaimDrawerHeader: Download icon button between the amount and the
close button. Click → api.serializeClaim837 → downloadTextFile.
Disabled + 'Downloading 837 file' aria-label while the fetch is in
flight so the click feels responsive. Optional onError prop surfaces
fetch failures; defaults to a no-op so existing callers stay clean.
Tests: 3 download.test.ts, 3 api.test.ts, 2 header.test.ts (happy
path + error path). Frontend: 350 passing (+8 from 342).
X12 835 SVC segment:
SVC01 = composite procedure
SVC02 = charge
SVC03 = payment
SVC04 = Unit or Basis for Measurement Code (UN, MJ, DA, ...)
SVC05 = Service Unit Count
The parser previously read SVC04 as the units count and SVC05 as the
unit type — backwards. On real 835s (and the canonical minimal
fixture), SVC04 carries the code 'UN' which fails Decimal parsing, so
the units always came out as None and the code string was assigned to
unit_type. SP7's line-level matcher couldn't compare units on the SVC
side against the claim side because of this.
- _consume_service_payment: SVC04 → unit_type, SVC05 → units count
- Default unit_type to 'UN' when only the count is present
- minimal_835.txt + unbalanced_835.txt: swap positions to match spec
- Add 2 regression tests (units-and-unit-type, default-unit-type-to-UN)
Returns the persisted ClaimOutput as a regenerated X12 837P file via
the new outbound serializer (Approach A — full rebuild from canonical
fields). 404 on missing claim, 422 if the stored raw_json cannot be
validated as a ClaimOutput. text/x12 content-type, attachment
disposition with the claim id as the filename.
3 tests:
- endpoint returns text/x12 attachment starting with ISA*
- 404 for missing claim id
- regenerated text round-trips back through parse()
- backend/src/cyclone/parsers/serialize_837.py — full-rebuild 837P serializer.
Emits envelope (ISA/GS/ST/SE/GE/IEA + BHT) + submitter/receiver/billing
provider/subscriber/payer hierarchy + editable segments (CLM/REF*G1/HI)
+ per-service-line LX/SV1/DTP*472/REF*6R — all from canonical
ClaimOutput fields.
Pivoted from spec §3.1 hybrid to full rebuild because ClaimOutput.raw_segments
only captures post-CLM segments (CLM, REF*G1, HI, LX, SV1 pairs) — not the
envelope or hierarchies. A pass-through approach cannot regenerate those
without expanding raw_segments in parse_837.py (out of scope for this SP).
- backend/tests/test_serialize_837.py — 36 tests covering envelope shape,
hierarchy segments, claim-level builders, service-line builders, edited-field
propagation, round-trip, custom sender/receiver IDs, and resubmit helper.
- backend/tests/test_prodfiles_smoke.py::test_claims_prodfile_round_trip —
every file in docs/prodfiles/claims/ (113 files) round-trips through
serialize_837 → parse_837_text with deep-equal ClaimOutput (modulo
validation, which is recomputed by the parser).
- docs/superpowers/plans/2026-06-20-cyclone-serialize-837.md — full plan
with amendment note documenting the Approach A pivot.
Per session convention, plan note about unrelated modifications to
parse_835.py / fixtures stashed separately.
Existing test fixtures (ClaimDrawer, ClaimDrawerHeader, Claims pages)
construct partial ClaimDetail objects without the new slim
lineReconciliation projection. Other SP7 fields follow the optional
pattern (matchedLines?, serviceLinePayments?, claimLevelAdjustments?),
so apply the same to lineReconciliation.
Backend always emits it for matched claims; UI treats absent as
"no per-line audit yet".
manual_match previously only flipped the claim↔remit FK and the
claim state — it never ran line-level reconciliation, so manually-
paired claims surfaced empty line-reconciliation rows to the UI
and skipped CLP-level CAS aggregate recompute.
Refactor reconcile.run() to call a new per-pair helper
_reconcile_pair(session, claim, remittance) that:
- clears any existing LineReconciliation rows for the claim
(idempotent re-run; safe across manual_unmatch + manual_rematch
cycles that may pair the claim with a different remittance),
- reads 837 SV1 lines from Claim.raw_json and 835 SVC rows from
ServiceLinePayment,
- runs match_service_lines() and persists a LineReconciliation row
per side,
- recomputes Remittance.claim_level_adjustment_amount (CLP-level
CAS) and Remittance.adjustment_amount (total CAS).
manual_match now calls this helper after the FK is set and before
commit, mirroring the auto-match path. Reversals are skipped (they
don't have SV1↔SVC line pairs; per §7.3).
Tests (test_store_reconcile.py):
- test_manual_match_populates_line_reconciliation_rows: end-to-end
check that a manual pair writes the expected matched +
unmatched_837_only rows plus zero CAS aggregates.
- test_manual_match_idempotent_line_reconciliation: after
manual_unmatch + manual_rematch + inserting a CLP-level CAS row
directly, the claim has exactly two fresh LineReconciliation rows
(no duplicates) and the remittance aggregate reflects the new CAS.
Smoke tested end-to-end via TestClient: parse co_medicaid_837p.txt
+ co_medicaid_835.txt, auto-matcher skips (PCNs differ), manual
match via POST /api/reconciliation/match, then
GET /api/claims/{id}/line-reconciliation returns 2 rows and
GET /api/inbox/lanes shows matched_remittance.total_lines=2.
Note: matched_lines may be 0 on real 835 fixtures due to a
pre-existing parser bug in _consume_service_payment that swaps
SVC04 (units) and SVC05 (unit-of-measure) when reading units from
the segment. The SP7 strict-match criterion requires units parity,
so the misread produces None on the SVC side and strict-match
never succeeds. Surfaced as a follow-up; the SP7 work itself is
correct (line counts are computed and persisted).
- Mark SP4 as fully shipped (batch diff, search, CSV, a11y all landed)
- Add SP6 (Inbox) and SP7 (Per-line reconciliation) sections
- Add SP6 + SP7 endpoint inventories
- Note next up: outbound 837P serializer
The workflow-automation plan was authored but never committed; the
features it specified have shipped, so commit it for the historical
record alongside this README refresh.
Nora
Tyler