cyclone/docs/superpowers/specs/2026-06-20-cyclone-serialize-837-design.md

Sub-project 7 — Outbound 837P Serializer: Design Spec

Date: 2026-06-20 Status: Approved (pending user review of this doc) Branch: sp7-serialize-837 Aesthetic direction: No UI change in this sub-project. Backend + minimal drawer button.

1. Scope

Add an outbound X12 837P serializer to Cyclone so a parsed ClaimOutput can be regenerated as a complete, round-trippable X12 837P file — the same shape as one file in docs/prodfiles/claims/. Unblocks the SP6 "resubmit" lane end-to-end (the existing /api/inbox/rejected/resubmit currently flips DB state but emits no file): ?download=true returns the regenerated 837 files in the response so the Inbox can offer a "Resubmit + download bundle" action. Adds a "Download 837" affordance on the claim drawer for single-claim exports.

Out of scope (deferred):

• 997 ACK transaction set — TA1's functional-group sibling, deferred.
• Claim ↔ TA1/999 linking — separate sub-project.
• In-app claim editing UI — a future SP. The serializer accepts already-edited ClaimOutput rows but does not provide the form.
• Batch (multi-CLM) envelope serialization — v1 is one CLM per file. The parse_837.py batch ingestion already handles envelopes; the symmetric serializer is a separate task.
• Frontend redesign of the Inbox or drawer — same drawer, one extra button.

2. Goals

1. One public function. cyclone.parsers.serialize_837.serialize_837(claim: ClaimOutput, *, interchange_control_number=..., group_control_number=...) -> str returns a complete X12 837P text. Round-trips through cyclone.parsers.parse_837.parse_837_text for the same claim.
2. Edits propagate. If a claim's editable fields were changed after parse (charge, prior_auth, diagnoses, service lines), the serialized output reflects the edits — not the original inbound file.
3. Stable segments preserved verbatim. Billing-provider hierarchy, subscriber hierarchy, payer info, submitter/receiver info, and line provider references come through byte-identical from claim.raw_segments. The serializer does not re-emit them.
4. Resubmit lane completes the loop. POST /api/inbox/rejected/resubmit?download=true returns a downloadable bundle of regenerated 837 files for the resubmitted claims. Default behavior unchanged (state flip only).
5. Drawer affordance. Claim drawer header gains a "Download 837" button that calls the serializer endpoint.

3. Locked decisions

3.1 Approach C — hybrid (fresh envelope + selective body rebuild)

Three candidate approaches were considered (full rebuild, raw-segments splice, hybrid). Hybrid wins because:

• Envelope splice alone (Approach B) doesn't propagate edits — a charge change in the DB would never reach the output.
• Full rebuild alone (Approach A) is ~300 LOC of NM1/N3/N4/PRV/HL/SBR emitter code that already exists implicitly inside the parser; re-emitting it invites drift from the parser.
• Hybrid uses the parser's own captured segments for the parts that don't change (provider/subscriber/payer/submitter hierarchies) and rebuilds only the editable segments (CLM, REFG1, HI, SV1, DTP472) from the canonical ClaimOutput fields. ~150 LOC. Drift-resistant. Edits propagate.

3.2 Editable vs stable segment classification

Segment	Stable / Editable	Reason
ISA, GS, ST, SE, GE, IEA	Fresh	Always rebuilt; carries the new interchange/group control numbers
BHT	Rebuilt	Small + deterministic; rebuild from Envelope so BHT06 transaction-type-code edits propagate
NM1*41 (submitter)	Stable	Submitter name/ID doesn't change per claim
PER	Stable	Submitter contact doesn't change per claim
NM1*40 (receiver)	Stable	Payer-as-receiver doesn't change per claim
HL*1, PRV, NM1*85	Stable	Billing provider identity
Billing provider N3, N4	Stable	Address
REF*EI	Stable	Tax ID
HL*2, SBR	Stable	Subscriber hierarchy + payer sequence
NM1*IL	Stable	Subscriber name
Subscriber N3, N4, DMG	Stable	Subscriber address + demographics
NM1*PR	Stable	Payer identity
CLM	Editable	Charge, frequency, POS, facility code, signature/assignment flags all live on claim.claim
REF*G1	Editable	Prior auth — only emitted when claim.claim.prior_auth is set
HI	Editable	Diagnoses — rebuilt from claim.diagnoses (multiple qualifiers supported)
LX	Stable	Service-line counter — regenerated for each line, but the value is positional
SV1	Editable	Procedure, modifiers, charge, units, unit_type
DTP*472	Editable	Service date
REF*6R	Stable	Line provider reference — passes through from the matching raw_segment

The "editable" segments are looked up by (segment_id, qualifier) in claim.raw_segments and replaced. Everything else flows through in original order.

3.3 No envelope-level batching

v1 emits exactly one CLM per output file. A future sub-project can promote the serializer to emit one ST/SE envelope per N claims; the v1 design keeps that door open (the body rebuilder iterates service_lines and diagnoses cleanly) but does not implement it.

3.4 Resubmit integration: opt-in ?download=true

POST /api/inbox/rejected/resubmit keeps its current behavior by default — flips ClaimState.REJECTED → SUBMITTED, increments resubmit_count, returns {ok, resubmitted, conflicts}. A new query param download=true switches the response to a multipart-style payload:

{
  "ok": true,
  "resubmitted": ["CLM-1", "CLM-2"],
  "conflicts": [],
  "files": [
    {"claim_id": "CLM-1", "filename": "CLM-1.x12", "x12_text": "ISA*..."},
    {"claim_id": "CLM-2", "filename": "CLM-2.x12", "x12_text": "ISA*..."}
  ]
}

Control-number rule for the bundle: each file in files[] is assigned a unique 9-digit interchange control number and a unique group control number. The numbers are derived deterministically from the source batch id + the claim's position in the bundle (e.g. 000000101, 000000102, …) so two back-to-back resubmit calls produce different envelopes for the same set of claims — required by X12 and matches what CO Medicaid expects. The 837 serializer gains a small serialize_837_for_resubmit(claim, *, interchange_index: int) -> str helper that builds the same body with the indexed control number; the default serialize_837(claim) keeps "000000001" for single-claim draws from the drawer.

The frontend (Phase 4) collects files, builds a zip client-side, and triggers a browser download. v1 does not stream a server-side zip — keeps the backend simple and avoids a new zipfile dependency on the response path.

3.5 Drawer download

src/components/ClaimDrawer/ClaimDrawerHeader.tsx gains one button: "Download 837". On click: api.getClaim837(claimId) → Blob → object URL → anchor download. No new drawer state, no new dependency.

4. Schemas

4.1 serialize_837.py — internal helpers

def _build_isa(sender_id: str, receiver_id: str, interchange_control_number: str) -> str
def _build_gs(sender_id: str, receiver_id: str, group_control_number: str) -> str
def _build_st(control_number: str, implementation_guide: str) -> str
def _build_bht(envelope: Envelope) -> str           # NEW: pulls BHT06 from envelope
def _build_clm(claim: ClaimHeader) -> str           # CLM01..CLM07
def _build_ref_g1(prior_auth: str | None) -> str    # "" if None
def _build_hi(diagnoses: list[Diagnosis]) -> str    # one HI segment
def _build_sv1(line: ServiceLine) -> str
def _build_dtp_472(service_date: date | None) -> str
def _build_se(count: int, control_number: str) -> str

def _pass_through_stable_segments(
    raw_segments: list[list[str]],
    *,
    drop: set[tuple[str, str | None]],
) -> list[str]:
    """Yield raw_segments with the matching editable segments removed,
    preserving original order of the remaining stable segments."""

def serialize_837(
    claim: ClaimOutput,
    *,
    interchange_control_number: str = "000000001",
    group_control_number: str = "1",
) -> str

_pass_through_stable_segments is the disambiguator: it knows the editable segment kinds (by (seg_id, qualifier_or_None)) and removes them from claim.raw_segments. The body emitter inserts the fresh ones in canonical positions:

1. BHT (from envelope) — first
2. Submitter + receiver block (stable)
3. Billing provider hierarchy (stable, including REF*EI)
4. Subscriber hierarchy (stable)
5. Payer (NM1*PR, stable)
6. CLM (rebuilt, editable)
7. REF*G1 (rebuilt if prior_auth)
8. HI (rebuilt)
9. For each service line: LX (stable, from raw) + SV1 (rebuilt) + DTP*472 (rebuilt) + REF*6R (stable)

If claim.raw_segments is empty or missing, serialize_837 raises SerializeError("claim has no raw_segments; cannot serialize from-scratch"). v1 does not implement a full from-scratch emitter (Approach A in §3.1) — that's a future sub-project.

4.2 API response additions

Method	Path	Returns
GET	/api/claims/{claim_id}/serialize-837	text/x12 body, Content-Disposition: attachment; filename="claim-{claim_id}.x12". 404 if claim missing. 422 if no raw_json/raw_segments.
POST	/api/inbox/rejected/resubmit?download=true	existing JSON, plus files: [{claim_id, filename, x12_text}] array when download=true.

GET /api/claims/{claim_id}/serialize-837 is mounted only in v1 — no streaming, no cache headers. The 837 text is small (~2-4 KB).

5. UI additions

• src/lib/api.ts: add api.getClaim837(claimId: string): Promise<Blob> — does the GET, returns the blob. Add api.resubmitRejected(claimIds, { download: true }): Promise<{ ok, resubmitted, conflicts, files }> for the bundle path.
• src/components/ClaimDrawer/ClaimDrawerHeader.tsx: add a Button labeled "Download 837" with Download icon, onClick → api.getClaim837(claim.id) → object URL → anchor click.
• Resubmit download UX (P4): the Inbox page's "Resubmit" action gains a small modal — "Resubmit and download bundle?" with two buttons. On confirm: api.resubmitRejected(ids, { download: true }) → take files[] → build a zip in-browser (using JSZip via CDN script tag — no new npm dep) → trigger anchor download. On decline: existing path (state flip only).
• No state changes in the drawer. No new tests beyond the existing drawer suite.

6. Migration / rollout

No schema migration. No new dependencies.

The serializer is purely additive — parse_837.py, serialize_270.py, serialize_999.py are unchanged. The new endpoint is additive; existing callers of /api/inbox/rejected/resubmit see no behavior change unless they opt into ?download=true.

7. Test coverage targets

Phase	New tests	Total target
P1 — serializer module	9 backend	517 backend
P2 — serialize-837 endpoint	3 backend	520 backend
P3 — drawer button	1 frontend	13 frontend
P4 — resubmit ?download=true + bundle modal	2 backend + 1 frontend	522 backend / 14 frontend
P5 — prodfile round-trip	1 backend	523 backend

7.1 Test details

backend/tests/test_serialize_837.py (9 tests):

• test_round_trip_unchanged_claim_parses_back_equivalent — serialize → parse → assert ClaimOutput deep-equal (modulo validation which is recomputed)
• test_round_trip_edited_charge_propagates — mutate claim.claim.total_charge → serialize → parse → assert new charge
• test_round_trip_edited_frequency_propagates — same pattern for frequency_code
• test_round_trip_edited_prior_auth_propagates — REF*G1 round-trip
• test_round_trip_added_diagnosis_propagates — append to claim.diagnoses → HI segment count grows
• test_round_trip_edited_service_line_charge_propagates — SV1 charge round-trip
• test_round_trip_edited_service_date_propagates — DTP*472 round-trip
• test_no_raw_segments_raises_serialize_error — empty raw_segments raises SerializeError
• test_envelope_is_106_char_isa_with_standard_delimiters — ISA shape check

backend/tests/test_api_serialize_837.py (3 tests):

• test_endpoint_returns_x12_attachment — happy path, checks Content-Type and Content-Disposition
• test_endpoint_404_for_missing_claim
• test_endpoint_422_for_claim_without_raw_json

backend/tests/test_inbox_resubmit_download.py (2 tests):

• test_resubmit_default_returns_no_files — backwards compat
• test_resubmit_download_true_returns_x12_per_resubmitted_claim

backend/tests/test_prodfiles_smoke.py (1 new test):

• test_claims_prodfile_round_trip — every file in docs/prodfiles/claims/ (113 files) round-trips through serialize_837 → parse_837_text with deep-equal ClaimOutput. Counts as 1 test for the suite total (parametrization kept internal).

src/components/ClaimDrawer/ClaimDrawerHeader.test.tsx (1 test):

• test_download_button_calls_api_get_claim_837

src/pages/Inbox.test.tsx (1 new test):

• test_resubmit_with_download_modal_zips_and_downloads — verifies the modal flow calls the API with download=true and triggers a zip+download on confirm.

8. Risk areas

1. Segment-position drift. claim.raw_segments is captured at parse time; if the parser ever reorders or drops a segment, the pass-through would shift the body. Mitigation: round-trip tests on every prodfile catch this immediately.
2. Editable vs stable classification drift. If a payer config adds a new editable segment kind (e.g. K3 notes), the serializer won't know to rebuild it. Mitigation: keep _EDITABLE_SEGMENT_KINDS as a single named tuple constant near the top of serialize_837.py so it's easy to extend.
3. Refusal to serialize a no-raw claim. v1 raises rather than producing broken output. The drawer button should disable itself when raw_json is absent. UI: claim.raw_segments.length > 0 check in the button's disabled prop.
4. Bundle size of ?download=true response. 100 resubmitted claims × ~3 KB = ~300 KB JSON. Acceptable for a local-only tool. If the resubmit lane ever scales beyond that, switch to NDJSON streaming (same pattern as SP5 tails).

9. Acceptance checklist

• Every docs/prodfiles/claims/*.x12 file (113 files) round-trips through serialize_837 → parse_837_text and produces a ClaimOutput deep-equal to the source.
• GET /api/claims/{claim_id}/serialize-837 returns the regenerated 837 text with the right Content-Type and Content-Disposition.
• The Claim drawer "Download 837" button produces the same byte sequence as the round-trip test.
• POST /api/inbox/rejected/resubmit (default) behavior unchanged.
• POST /api/inbox/rejected/resubmit?download=true returns the files array and the frontend successfully zips + downloads.
• A claim with raw_segments=[] raises SerializeError and the API returns 422.
• Full backend suite: ≥ 523 passing (508 + 15 new). Frontend suite: ≥ 14 passing.
• No new pip or npm dependencies. JSZip is loaded via <script> tag from a CDN at runtime — no package.json change.

10. Out of scope (explicit)

• 997 ACK — TA1's functional-group sibling. Deferred.
• Claim ↔ TA1/999 linking — separate sub-project.
• In-app claim editing UI — the serializer accepts edited claims but does not provide the editor.
• Batch (multi-CLM) envelope serialization — v1 is one CLM per file.
• Frontend redesign of the Inbox or drawer — same drawer, one extra button.

11. References

• backend/src/cyclone/parsers/serialize_270.py — pattern reference (one-inquiry 270 serializer)
• backend/src/cyclone/parsers/serialize_999.py — pattern reference (one-functional-group 999 serializer)
• backend/src/cyclone/parsers/parse_837.py — inverse; claim.raw_segments capture
• docs/prodfiles/claims/ — 113 sample output files (the shape we emit)
• backend/src/cyclone/api.py:759-783 — existing inbox_resubmit_rejected endpoint to extend
• src/components/ClaimDrawer/ClaimDrawerHeader.tsx — drawer header to extend