diff --git a/docs/superpowers/specs/2026-06-20-cyclone-serialize-837-design.md b/docs/superpowers/specs/2026-06-20-cyclone-serialize-837-design.md new file mode 100644 index 0000000..812a3b9 --- /dev/null +++ b/docs/superpowers/specs/2026-06-20-cyclone-serialize-837-design.md @@ -0,0 +1,239 @@ +# Sub-project 7 — Outbound 837P Serializer: Design Spec + +**Date:** 2026-06-20 +**Status:** Approved (pending user review of this doc) +**Branch:** `sp7-serialize-837` +**Aesthetic direction:** No UI change in this sub-project. Backend + minimal drawer button. + +## 1. Scope + +Add an outbound X12 837P serializer to Cyclone so a parsed `ClaimOutput` can be regenerated as a complete, round-trippable X12 837P file — the same shape as one file in `docs/prodfiles/claims/`. Unblocks the SP6 "resubmit" lane end-to-end (the existing `/api/inbox/rejected/resubmit` currently flips DB state but emits no file): `?download=true` returns the regenerated 837 files in the response so the Inbox can offer a "Resubmit + download bundle" action. Adds a "Download 837" affordance on the claim drawer for single-claim exports. + +**Out of scope** (deferred): + +- **997 ACK transaction set** — TA1's functional-group sibling, deferred. +- **Claim ↔ TA1/999 linking** — separate sub-project. +- **In-app claim editing UI** — a future SP. The serializer accepts already-edited `ClaimOutput` rows but does not provide the form. +- **Batch (multi-CLM) envelope serialization** — v1 is one CLM per file. The `parse_837.py` batch ingestion already handles envelopes; the symmetric serializer is a separate task. +- **Frontend redesign of the Inbox or drawer** — same drawer, one extra button. + +## 2. Goals + +1. **One public function.** `cyclone.parsers.serialize_837.serialize_837(claim: ClaimOutput, *, interchange_control_number=..., group_control_number=...) -> str` returns a complete X12 837P text. Round-trips through `cyclone.parsers.parse_837.parse_837_text` for the same claim. +2. **Edits propagate.** If a claim's editable fields were changed after parse (charge, prior_auth, diagnoses, service lines), the serialized output reflects the edits — not the original inbound file. +3. **Stable segments preserved verbatim.** Billing-provider hierarchy, subscriber hierarchy, payer info, submitter/receiver info, and line provider references come through byte-identical from `claim.raw_segments`. The serializer does not re-emit them. +4. **Resubmit lane completes the loop.** `POST /api/inbox/rejected/resubmit?download=true` returns a downloadable bundle of regenerated 837 files for the resubmitted claims. Default behavior unchanged (state flip only). +5. **Drawer affordance.** Claim drawer header gains a "Download 837" button that calls the serializer endpoint. + +## 3. Locked decisions + +### 3.1 Approach C — hybrid (fresh envelope + selective body rebuild) + +Three candidate approaches were considered (full rebuild, raw-segments splice, hybrid). Hybrid wins because: + +- **Envelope splice** alone (Approach B) doesn't propagate edits — a charge change in the DB would never reach the output. +- **Full rebuild** alone (Approach A) is ~300 LOC of NM1/N3/N4/PRV/HL/SBR emitter code that already exists implicitly inside the parser; re-emitting it invites drift from the parser. +- **Hybrid** uses the parser's own captured segments for the parts that don't change (provider/subscriber/payer/submitter hierarchies) and rebuilds only the editable segments (CLM, REF*G1, HI, SV1, DTP*472) from the canonical `ClaimOutput` fields. ~150 LOC. Drift-resistant. Edits propagate. + +### 3.2 Editable vs stable segment classification + +| Segment | Stable / Editable | Reason | +|---------|-------------------|--------| +| `ISA`, `GS`, `ST`, `SE`, `GE`, `IEA` | **Fresh** | Always rebuilt; carries the new interchange/group control numbers | +| `BHT` | **Rebuilt** | Small + deterministic; rebuild from `Envelope` so BHT06 transaction-type-code edits propagate | +| `NM1*41` (submitter) | Stable | Submitter name/ID doesn't change per claim | +| `PER` | Stable | Submitter contact doesn't change per claim | +| `NM1*40` (receiver) | Stable | Payer-as-receiver doesn't change per claim | +| `HL*1`, `PRV`, `NM1*85` | Stable | Billing provider identity | +| Billing provider `N3`, `N4` | Stable | Address | +| `REF*EI` | Stable | Tax ID | +| `HL*2`, `SBR` | Stable | Subscriber hierarchy + payer sequence | +| `NM1*IL` | Stable | Subscriber name | +| Subscriber `N3`, `N4`, `DMG` | Stable | Subscriber address + demographics | +| `NM1*PR` | Stable | Payer identity | +| **`CLM`** | **Editable** | Charge, frequency, POS, facility code, signature/assignment flags all live on `claim.claim` | +| **`REF*G1`** | **Editable** | Prior auth — only emitted when `claim.claim.prior_auth` is set | +| **`HI`** | **Editable** | Diagnoses — rebuilt from `claim.diagnoses` (multiple qualifiers supported) | +| `LX` | Stable | Service-line counter — regenerated for each line, but the value is positional | +| **`SV1`** | **Editable** | Procedure, modifiers, charge, units, unit_type | +| **`DTP*472`** | **Editable** | Service date | +| `REF*6R` | Stable | Line provider reference — passes through from the matching raw_segment | + +The "editable" segments are looked up by `(segment_id, qualifier)` in `claim.raw_segments` and replaced. Everything else flows through in original order. + +### 3.3 No envelope-level batching + +v1 emits exactly one CLM per output file. A future sub-project can promote the serializer to emit one ST/SE envelope per N claims; the v1 design keeps that door open (the body rebuilder iterates `service_lines` and `diagnoses` cleanly) but does not implement it. + +### 3.4 Resubmit integration: opt-in `?download=true` + +`POST /api/inbox/rejected/resubmit` keeps its current behavior by default — flips `ClaimState.REJECTED → SUBMITTED`, increments `resubmit_count`, returns `{ok, resubmitted, conflicts}`. A new query param `download=true` switches the response to a multipart-style payload: + +```json +{ + "ok": true, + "resubmitted": ["CLM-1", "CLM-2"], + "conflicts": [], + "files": [ + {"claim_id": "CLM-1", "filename": "CLM-1.x12", "x12_text": "ISA*..."}, + {"claim_id": "CLM-2", "filename": "CLM-2.x12", "x12_text": "ISA*..."} + ] +} +``` + +**Control-number rule for the bundle:** each file in `files[]` is assigned a unique 9-digit interchange control number and a unique group control number. The numbers are derived deterministically from the source batch id + the claim's position in the bundle (e.g. `000000101`, `000000102`, …) so two back-to-back resubmit calls produce different envelopes for the same set of claims — required by X12 and matches what CO Medicaid expects. The 837 serializer gains a small `serialize_837_for_resubmit(claim, *, interchange_index: int) -> str` helper that builds the same body with the indexed control number; the default `serialize_837(claim)` keeps `"000000001"` for single-claim draws from the drawer. + +The frontend (Phase 4) collects `files`, builds a zip client-side, and triggers a browser download. v1 does not stream a server-side zip — keeps the backend simple and avoids a new `zipfile` dependency on the response path. + +### 3.5 Drawer download + +`src/components/ClaimDrawer/ClaimDrawerHeader.tsx` gains one button: "Download 837". On click: `api.getClaim837(claimId)` → `Blob` → object URL → anchor download. No new drawer state, no new dependency. + +## 4. Schemas + +### 4.1 `serialize_837.py` — internal helpers + +```python +def _build_isa(sender_id: str, receiver_id: str, interchange_control_number: str) -> str +def _build_gs(sender_id: str, receiver_id: str, group_control_number: str) -> str +def _build_st(control_number: str, implementation_guide: str) -> str +def _build_bht(envelope: Envelope) -> str # NEW: pulls BHT06 from envelope +def _build_clm(claim: ClaimHeader) -> str # CLM01..CLM07 +def _build_ref_g1(prior_auth: str | None) -> str # "" if None +def _build_hi(diagnoses: list[Diagnosis]) -> str # one HI segment +def _build_sv1(line: ServiceLine) -> str +def _build_dtp_472(service_date: date | None) -> str +def _build_se(count: int, control_number: str) -> str + +def _pass_through_stable_segments( + raw_segments: list[list[str]], + *, + drop: set[tuple[str, str | None]], +) -> list[str]: + """Yield raw_segments with the matching editable segments removed, + preserving original order of the remaining stable segments.""" + +def serialize_837( + claim: ClaimOutput, + *, + interchange_control_number: str = "000000001", + group_control_number: str = "1", +) -> str +``` + +`_pass_through_stable_segments` is the disambiguator: it knows the editable segment kinds (by `(seg_id, qualifier_or_None)`) and removes them from `claim.raw_segments`. The body emitter inserts the fresh ones in canonical positions: +1. `BHT` (from envelope) — first +2. Submitter + receiver block (stable) +3. Billing provider hierarchy (stable, including `REF*EI`) +4. Subscriber hierarchy (stable) +5. Payer (`NM1*PR`, stable) +6. `CLM` (rebuilt, editable) +7. `REF*G1` (rebuilt if prior_auth) +8. `HI` (rebuilt) +9. For each service line: `LX` (stable, from raw) + `SV1` (rebuilt) + `DTP*472` (rebuilt) + `REF*6R` (stable) + +If `claim.raw_segments` is empty or missing, `serialize_837` raises `SerializeError("claim has no raw_segments; cannot serialize from-scratch")`. v1 does not implement a full from-scratch emitter (Approach A in §3.1) — that's a future sub-project. + +### 4.2 API response additions + +| Method | Path | Returns | +|--------|------|---------| +| GET | `/api/claims/{claim_id}/serialize-837` | `text/x12` body, `Content-Disposition: attachment; filename="claim-{claim_id}.x12"`. 404 if claim missing. 422 if no `raw_json`/`raw_segments`. | +| POST | `/api/inbox/rejected/resubmit?download=true` | existing JSON, plus `files: [{claim_id, filename, x12_text}]` array when `download=true`. | + +`GET /api/claims/{claim_id}/serialize-837` is mounted only in v1 — no streaming, no cache headers. The 837 text is small (~2-4 KB). + +## 5. UI additions + +- `src/lib/api.ts`: add `api.getClaim837(claimId: string): Promise` — does the GET, returns the blob. Add `api.resubmitRejected(claimIds, { download: true }): Promise<{ ok, resubmitted, conflicts, files }>` for the bundle path. +- `src/components/ClaimDrawer/ClaimDrawerHeader.tsx`: add a `Button` labeled "Download 837" with `Download` icon, onClick → `api.getClaim837(claim.id)` → object URL → anchor click. +- **Resubmit download UX** (P4): the Inbox page's "Resubmit" action gains a small modal — "Resubmit and download bundle?" with two buttons. On confirm: `api.resubmitRejected(ids, { download: true })` → take `files[]` → build a zip in-browser (using `JSZip` via CDN script tag — no new npm dep) → trigger anchor download. On decline: existing path (state flip only). +- No state changes in the drawer. No new tests beyond the existing drawer suite. + +## 6. Migration / rollout + +No schema migration. No new dependencies. + +The serializer is purely additive — `parse_837.py`, `serialize_270.py`, `serialize_999.py` are unchanged. The new endpoint is additive; existing callers of `/api/inbox/rejected/resubmit` see no behavior change unless they opt into `?download=true`. + +## 7. Test coverage targets + +| Phase | New tests | Total target | +|-------|-----------|--------------| +| P1 — serializer module | 9 backend | 517 backend | +| P2 — `serialize-837` endpoint | 3 backend | 520 backend | +| P3 — drawer button | 1 frontend | 13 frontend | +| P4 — resubmit `?download=true` + bundle modal | 2 backend + 1 frontend | 522 backend / 14 frontend | +| P5 — prodfile round-trip | 1 backend | 523 backend | + +### 7.1 Test details + +`backend/tests/test_serialize_837.py` (9 tests): + +- `test_round_trip_unchanged_claim_parses_back_equivalent` — serialize → parse → assert `ClaimOutput` deep-equal (modulo `validation` which is recomputed) +- `test_round_trip_edited_charge_propagates` — mutate `claim.claim.total_charge` → serialize → parse → assert new charge +- `test_round_trip_edited_frequency_propagates` — same pattern for frequency_code +- `test_round_trip_edited_prior_auth_propagates` — REF*G1 round-trip +- `test_round_trip_added_diagnosis_propagates` — append to `claim.diagnoses` → HI segment count grows +- `test_round_trip_edited_service_line_charge_propagates` — SV1 charge round-trip +- `test_round_trip_edited_service_date_propagates` — DTP*472 round-trip +- `test_no_raw_segments_raises_serialize_error` — empty `raw_segments` raises `SerializeError` +- `test_envelope_is_106_char_isa_with_standard_delimiters` — ISA shape check + +`backend/tests/test_api_serialize_837.py` (3 tests): + +- `test_endpoint_returns_x12_attachment` — happy path, checks `Content-Type` and `Content-Disposition` +- `test_endpoint_404_for_missing_claim` +- `test_endpoint_422_for_claim_without_raw_json` + +`backend/tests/test_inbox_resubmit_download.py` (2 tests): + +- `test_resubmit_default_returns_no_files` — backwards compat +- `test_resubmit_download_true_returns_x12_per_resubmitted_claim` + +`backend/tests/test_prodfiles_smoke.py` (1 new test): + +- `test_claims_prodfile_round_trip` — every file in `docs/prodfiles/claims/` (113 files) round-trips through `serialize_837` → `parse_837_text` with deep-equal `ClaimOutput`. Counts as 1 test for the suite total (parametrization kept internal). + +`src/components/ClaimDrawer/ClaimDrawerHeader.test.tsx` (1 test): + +- `test_download_button_calls_api_get_claim_837` + +`src/pages/Inbox.test.tsx` (1 new test): + +- `test_resubmit_with_download_modal_zips_and_downloads` — verifies the modal flow calls the API with `download=true` and triggers a zip+download on confirm. + +## 8. Risk areas + +1. **Segment-position drift.** `claim.raw_segments` is captured at parse time; if the parser ever reorders or drops a segment, the pass-through would shift the body. Mitigation: round-trip tests on every prodfile catch this immediately. +2. **Editable vs stable classification drift.** If a payer config adds a new editable segment kind (e.g. K3 notes), the serializer won't know to rebuild it. Mitigation: keep `_EDITABLE_SEGMENT_KINDS` as a single named tuple constant near the top of `serialize_837.py` so it's easy to extend. +3. **Refusal to serialize a no-raw claim.** v1 raises rather than producing broken output. The drawer button should disable itself when raw_json is absent. UI: `claim.raw_segments.length > 0` check in the button's `disabled` prop. +4. **Bundle size of `?download=true` response.** 100 resubmitted claims × ~3 KB = ~300 KB JSON. Acceptable for a local-only tool. If the resubmit lane ever scales beyond that, switch to NDJSON streaming (same pattern as SP5 tails). + +## 9. Acceptance checklist + +- [ ] Every `docs/prodfiles/claims/*.x12` file (113 files) round-trips through `serialize_837` → `parse_837_text` and produces a `ClaimOutput` deep-equal to the source. +- [ ] `GET /api/claims/{claim_id}/serialize-837` returns the regenerated 837 text with the right `Content-Type` and `Content-Disposition`. +- [ ] The Claim drawer "Download 837" button produces the same byte sequence as the round-trip test. +- [ ] `POST /api/inbox/rejected/resubmit` (default) behavior unchanged. +- [ ] `POST /api/inbox/rejected/resubmit?download=true` returns the `files` array and the frontend successfully zips + downloads. +- [ ] A claim with `raw_segments=[]` raises `SerializeError` and the API returns 422. +- [ ] Full backend suite: ≥ 523 passing (508 + 15 new). Frontend suite: ≥ 14 passing. +- [ ] No new pip or npm dependencies. JSZip is loaded via `