cyclone/docs/superpowers/specs/2026-06-20-cyclone-serialize-837-design.md

# Sub-project 7 — Outbound 837P Serializer: Design Spec

**Date:** 2026-06-20
**Status:** Approved (pending user review of this doc)
**Branch:** `sp7-serialize-837`
**Aesthetic direction:** No UI change in this sub-project. Backend + minimal drawer button.

## 1. Scope

Add an outbound X12 837P serializer to Cyclone so a parsed `ClaimOutput` can be regenerated as a complete, round-trippable X12 837P file — the same shape as one file in `docs/prodfiles/claims/`. Unblocks the SP6 "resubmit" lane end-to-end (the existing `/api/inbox/rejected/resubmit` currently flips DB state but emits no file): `?download=true` returns the regenerated 837 files in the response so the Inbox can offer a "Resubmit + download bundle" action. Adds a "Download 837" affordance on the claim drawer for single-claim exports.

**Out of scope** (deferred):

- **997 ACK transaction set** — TA1's functional-group sibling, deferred.
- **Claim ↔ TA1/999 linking** — separate sub-project.
- **In-app claim editing UI** — a future SP. The serializer accepts already-edited `ClaimOutput` rows but does not provide the form.
- **Batch (multi-CLM) envelope serialization** — v1 is one CLM per file. The `parse_837.py` batch ingestion already handles envelopes; the symmetric serializer is a separate task.
- **Frontend redesign of the Inbox or drawer** — same drawer, one extra button.

## 2. Goals

1. **One public function.** `cyclone.parsers.serialize_837.serialize_837(claim: ClaimOutput, *, interchange_control_number=..., group_control_number=...) -> str` returns a complete X12 837P text. Round-trips through `cyclone.parsers.parse_837.parse_837_text` for the same claim.
2. **Edits propagate.** If a claim's editable fields were changed after parse (charge, prior_auth, diagnoses, service lines), the serialized output reflects the edits — not the original inbound file.
3. **Stable segments preserved verbatim.** Billing-provider hierarchy, subscriber hierarchy, payer info, submitter/receiver info, and line provider references come through byte-identical from `claim.raw_segments`. The serializer does not re-emit them.
4. **Resubmit lane completes the loop.** `POST /api/inbox/rejected/resubmit?download=true` returns a downloadable bundle of regenerated 837 files for the resubmitted claims. Default behavior unchanged (state flip only).
5. **Drawer affordance.** Claim drawer header gains a "Download 837" button that calls the serializer endpoint.

## 3. Locked decisions

### 3.1 Approach C — hybrid (fresh envelope + selective body rebuild)

Three candidate approaches were considered (full rebuild, raw-segments splice, hybrid). Hybrid wins because:

- **Envelope splice** alone (Approach B) doesn't propagate edits — a charge change in the DB would never reach the output.
- **Full rebuild** alone (Approach A) is ~300 LOC of NM1/N3/N4/PRV/HL/SBR emitter code that already exists implicitly inside the parser; re-emitting it invites drift from the parser.
- **Hybrid** uses the parser's own captured segments for the parts that don't change (provider/subscriber/payer/submitter hierarchies) and rebuilds only the editable segments (CLM, REF*G1, HI, SV1, DTP*472) from the canonical `ClaimOutput` fields. ~150 LOC. Drift-resistant. Edits propagate.

### 3.2 Editable vs stable segment classification

| Segment | Stable / Editable | Reason |
|---------|-------------------|--------|
| `ISA`, `GS`, `ST`, `SE`, `GE`, `IEA` | **Fresh** | Always rebuilt; carries the new interchange/group control numbers |
| `BHT` | **Rebuilt** | Small + deterministic; rebuild from `Envelope` so BHT06 transaction-type-code edits propagate |
| `NM1*41` (submitter) | Stable | Submitter name/ID doesn't change per claim |
| `PER` | Stable | Submitter contact doesn't change per claim |
| `NM1*40` (receiver) | Stable | Payer-as-receiver doesn't change per claim |
| `HL*1`, `PRV`, `NM1*85` | Stable | Billing provider identity |
| Billing provider `N3`, `N4` | Stable | Address |
| `REF*EI` | Stable | Tax ID |
| `HL*2`, `SBR` | Stable | Subscriber hierarchy + payer sequence |
| `NM1*IL` | Stable | Subscriber name |
| Subscriber `N3`, `N4`, `DMG` | Stable | Subscriber address + demographics |
| `NM1*PR` | Stable | Payer identity |
| **`CLM`** | **Editable** | Charge, frequency, POS, facility code, signature/assignment flags all live on `claim.claim` |
| **`REF*G1`** | **Editable** | Prior auth — only emitted when `claim.claim.prior_auth` is set |
| **`HI`** | **Editable** | Diagnoses — rebuilt from `claim.diagnoses` (multiple qualifiers supported) |
| `LX` | Stable | Service-line counter — regenerated for each line, but the value is positional |
| **`SV1`** | **Editable** | Procedure, modifiers, charge, units, unit_type |
| **`DTP*472`** | **Editable** | Service date |
| `REF*6R` | Stable | Line provider reference — passes through from the matching raw_segment |

The "editable" segments are looked up by `(segment_id, qualifier)` in `claim.raw_segments` and replaced. Everything else flows through in original order.

### 3.3 No envelope-level batching

v1 emits exactly one CLM per output file. A future sub-project can promote the serializer to emit one ST/SE envelope per N claims; the v1 design keeps that door open (the body rebuilder iterates `service_lines` and `diagnoses` cleanly) but does not implement it.

### 3.4 Resubmit integration: opt-in `?download=true`

`POST /api/inbox/rejected/resubmit` keeps its current behavior by default — flips `ClaimState.REJECTED → SUBMITTED`, increments `resubmit_count`, returns `{ok, resubmitted, conflicts}`. A new query param `download=true` switches the response to a multipart-style payload:

```json
{
  "ok": true,
  "resubmitted": ["CLM-1", "CLM-2"],
  "conflicts": [],
  "files": [
    {"claim_id": "CLM-1", "filename": "CLM-1.x12", "x12_text": "ISA*..."},
    {"claim_id": "CLM-2", "filename": "CLM-2.x12", "x12_text": "ISA*..."}
  ]
}
```

**Control-number rule for the bundle:** each file in `files[]` is assigned a unique 9-digit interchange control number and a unique group control number. The numbers are derived deterministically from the source batch id + the claim's position in the bundle (e.g. `000000101`, `000000102`, …) so two back-to-back resubmit calls produce different envelopes for the same set of claims — required by X12 and matches what CO Medicaid expects. The 837 serializer gains a small `serialize_837_for_resubmit(claim, *, interchange_index: int) -> str` helper that builds the same body with the indexed control number; the default `serialize_837(claim)` keeps `"000000001"` for single-claim draws from the drawer.

The frontend (Phase 4) collects `files`, builds a zip client-side, and triggers a browser download. v1 does not stream a server-side zip — keeps the backend simple and avoids a new `zipfile` dependency on the response path.

### 3.5 Drawer download

`src/components/ClaimDrawer/ClaimDrawerHeader.tsx` gains one button: "Download 837". On click: `api.getClaim837(claimId)` → `Blob` → object URL → anchor download. No new drawer state, no new dependency.

## 4. Schemas

### 4.1 `serialize_837.py` — internal helpers

```python
def _build_isa(sender_id: str, receiver_id: str, interchange_control_number: str) -> str
def _build_gs(sender_id: str, receiver_id: str, group_control_number: str) -> str
def _build_st(control_number: str, implementation_guide: str) -> str
def _build_bht(envelope: Envelope) -> str           # NEW: pulls BHT06 from envelope
def _build_clm(claim: ClaimHeader) -> str           # CLM01..CLM07
def _build_ref_g1(prior_auth: str | None) -> str    # "" if None
def _build_hi(diagnoses: list[Diagnosis]) -> str    # one HI segment
def _build_sv1(line: ServiceLine) -> str
def _build_dtp_472(service_date: date | None) -> str
def _build_se(count: int, control_number: str) -> str

def _pass_through_stable_segments(
    raw_segments: list[list[str]],
    *,
    drop: set[tuple[str, str | None]],
) -> list[str]:
    """Yield raw_segments with the matching editable segments removed,
    preserving original order of the remaining stable segments."""

def serialize_837(
    claim: ClaimOutput,
    *,
    interchange_control_number: str = "000000001",
    group_control_number: str = "1",
) -> str
```

`_pass_through_stable_segments` is the disambiguator: it knows the editable segment kinds (by `(seg_id, qualifier_or_None)`) and removes them from `claim.raw_segments`. The body emitter inserts the fresh ones in canonical positions:
1. `BHT` (from envelope) — first
2. Submitter + receiver block (stable)
3. Billing provider hierarchy (stable, including `REF*EI`)
4. Subscriber hierarchy (stable)
5. Payer (`NM1*PR`, stable)
6. `CLM` (rebuilt, editable)
7. `REF*G1` (rebuilt if prior_auth)
8. `HI` (rebuilt)
9. For each service line: `LX` (stable, from raw) + `SV1` (rebuilt) + `DTP*472` (rebuilt) + `REF*6R` (stable)

If `claim.raw_segments` is empty or missing, `serialize_837` raises `SerializeError("claim has no raw_segments; cannot serialize from-scratch")`. v1 does not implement a full from-scratch emitter (Approach A in §3.1) — that's a future sub-project.

### 4.2 API response additions

| Method | Path | Returns |
|--------|------|---------|
| GET | `/api/claims/{claim_id}/serialize-837` | `text/x12` body, `Content-Disposition: attachment; filename="claim-{claim_id}.x12"`. 404 if claim missing. 422 if no `raw_json`/`raw_segments`. |
| POST | `/api/inbox/rejected/resubmit?download=true` | existing JSON, plus `files: [{claim_id, filename, x12_text}]` array when `download=true`. |

`GET /api/claims/{claim_id}/serialize-837` is mounted only in v1 — no streaming, no cache headers. The 837 text is small (~2-4 KB).

## 5. UI additions

- `src/lib/api.ts`: add `api.getClaim837(claimId: string): Promise<Blob>` — does the GET, returns the blob. Add `api.resubmitRejected(claimIds, { download: true }): Promise<{ ok, resubmitted, conflicts, files }>` for the bundle path.
- `src/components/ClaimDrawer/ClaimDrawerHeader.tsx`: add a `Button` labeled "Download 837" with `Download` icon, onClick → `api.getClaim837(claim.id)` → object URL → anchor click.
- **Resubmit download UX** (P4): the Inbox page's "Resubmit" action gains a small modal — "Resubmit and download bundle?" with two buttons. On confirm: `api.resubmitRejected(ids, { download: true })` → take `files[]` → build a zip in-browser (using `JSZip` via CDN script tag — no new npm dep) → trigger anchor download. On decline: existing path (state flip only).
- No state changes in the drawer. No new tests beyond the existing drawer suite.

## 6. Migration / rollout

No schema migration. No new dependencies.

The serializer is purely additive — `parse_837.py`, `serialize_270.py`, `serialize_999.py` are unchanged. The new endpoint is additive; existing callers of `/api/inbox/rejected/resubmit` see no behavior change unless they opt into `?download=true`.

## 7. Test coverage targets

| Phase | New tests | Total target |
|-------|-----------|--------------|
| P1 — serializer module | 9 backend | 517 backend |
| P2 — `serialize-837` endpoint | 3 backend | 520 backend |
| P3 — drawer button | 1 frontend | 13 frontend |
| P4 — resubmit `?download=true` + bundle modal | 2 backend + 1 frontend | 522 backend / 14 frontend |
| P5 — prodfile round-trip | 1 backend | 523 backend |

### 7.1 Test details

`backend/tests/test_serialize_837.py` (9 tests):

- `test_round_trip_unchanged_claim_parses_back_equivalent` — serialize → parse → assert `ClaimOutput` deep-equal (modulo `validation` which is recomputed)
- `test_round_trip_edited_charge_propagates` — mutate `claim.claim.total_charge` → serialize → parse → assert new charge
- `test_round_trip_edited_frequency_propagates` — same pattern for frequency_code
- `test_round_trip_edited_prior_auth_propagates` — REF*G1 round-trip
- `test_round_trip_added_diagnosis_propagates` — append to `claim.diagnoses` → HI segment count grows
- `test_round_trip_edited_service_line_charge_propagates` — SV1 charge round-trip
- `test_round_trip_edited_service_date_propagates` — DTP*472 round-trip
- `test_no_raw_segments_raises_serialize_error` — empty `raw_segments` raises `SerializeError`
- `test_envelope_is_106_char_isa_with_standard_delimiters` — ISA shape check

`backend/tests/test_api_serialize_837.py` (3 tests):

- `test_endpoint_returns_x12_attachment` — happy path, checks `Content-Type` and `Content-Disposition`
- `test_endpoint_404_for_missing_claim`
- `test_endpoint_422_for_claim_without_raw_json`

`backend/tests/test_inbox_resubmit_download.py` (2 tests):

- `test_resubmit_default_returns_no_files` — backwards compat
- `test_resubmit_download_true_returns_x12_per_resubmitted_claim`

`backend/tests/test_prodfiles_smoke.py` (1 new test):

- `test_claims_prodfile_round_trip` — every file in `docs/prodfiles/claims/` (113 files) round-trips through `serialize_837` → `parse_837_text` with deep-equal `ClaimOutput`. Counts as 1 test for the suite total (parametrization kept internal).

`src/components/ClaimDrawer/ClaimDrawerHeader.test.tsx` (1 test):

- `test_download_button_calls_api_get_claim_837`

`src/pages/Inbox.test.tsx` (1 new test):

- `test_resubmit_with_download_modal_zips_and_downloads` — verifies the modal flow calls the API with `download=true` and triggers a zip+download on confirm.

## 8. Risk areas

1. **Segment-position drift.** `claim.raw_segments` is captured at parse time; if the parser ever reorders or drops a segment, the pass-through would shift the body. Mitigation: round-trip tests on every prodfile catch this immediately.
2. **Editable vs stable classification drift.** If a payer config adds a new editable segment kind (e.g. K3 notes), the serializer won't know to rebuild it. Mitigation: keep `_EDITABLE_SEGMENT_KINDS` as a single named tuple constant near the top of `serialize_837.py` so it's easy to extend.
3. **Refusal to serialize a no-raw claim.** v1 raises rather than producing broken output. The drawer button should disable itself when raw_json is absent. UI: `claim.raw_segments.length > 0` check in the button's `disabled` prop.
4. **Bundle size of `?download=true` response.** 100 resubmitted claims × ~3 KB = ~300 KB JSON. Acceptable for a local-only tool. If the resubmit lane ever scales beyond that, switch to NDJSON streaming (same pattern as SP5 tails).

## 9. Acceptance checklist

- [ ] Every `docs/prodfiles/claims/*.x12` file (113 files) round-trips through `serialize_837` → `parse_837_text` and produces a `ClaimOutput` deep-equal to the source.
- [ ] `GET /api/claims/{claim_id}/serialize-837` returns the regenerated 837 text with the right `Content-Type` and `Content-Disposition`.
- [ ] The Claim drawer "Download 837" button produces the same byte sequence as the round-trip test.
- [ ] `POST /api/inbox/rejected/resubmit` (default) behavior unchanged.
- [ ] `POST /api/inbox/rejected/resubmit?download=true` returns the `files` array and the frontend successfully zips + downloads.
- [ ] A claim with `raw_segments=[]` raises `SerializeError` and the API returns 422.
- [ ] Full backend suite: ≥ 523 passing (508 + 15 new). Frontend suite: ≥ 14 passing.
- [ ] No new pip or npm dependencies. JSZip is loaded via `<script>` tag from a CDN at runtime — no `package.json` change.

## 10. Out of scope (explicit)

- **997 ACK** — TA1's functional-group sibling. Deferred.
- **Claim ↔ TA1/999 linking** — separate sub-project.
- **In-app claim editing UI** — the serializer accepts edited claims but does not provide the editor.
- **Batch (multi-CLM) envelope serialization** — v1 is one CLM per file.
- **Frontend redesign of the Inbox or drawer** — same drawer, one extra button.

## 11. References

- `backend/src/cyclone/parsers/serialize_270.py` — pattern reference (one-inquiry 270 serializer)
- `backend/src/cyclone/parsers/serialize_999.py` — pattern reference (one-functional-group 999 serializer)
- `backend/src/cyclone/parsers/parse_837.py` — inverse; `claim.raw_segments` capture
- `docs/prodfiles/claims/` — 113 sample output files (the shape we emit)
- `backend/src/cyclone/api.py:759-783` — existing `inbox_resubmit_rejected` endpoint to extend
- `src/components/ClaimDrawer/ClaimDrawerHeader.tsx` — drawer header to extend