docs(sp7): outbound 837P serializer design spec
Sub-project 7 adds serialize_837.py so a parsed ClaimOutput can be regenerated as a complete X12 837P file (the shape of docs/prodfiles/claims/). Unblocks the SP6 resubmit lane end-to-end and adds a 'Download 837' affordance on the claim drawer. Approach: hybrid (fresh envelope + selective body rebuild). Stable segments (provider/subscriber/payer hierarchies) pass through from claim.raw_segments; editable segments (CLM, REF*G1, HI, SV1, DTP*472) are rebuilt from canonical ClaimOutput fields so post-parse edits propagate to the output. Out of scope: 997 ACK, claim <-> TA1/999 linking, in-app claim editing UI, multi-CLM envelope batching, Inbox/drawer redesign.
This commit is contained in:
Tyler
@@ -0,0 +1,239 @@
|
|||||||
|
# Sub-project 7 — Outbound 837P Serializer: Design Spec
|
||||||
|
|
||||||
|
**Date:** 2026-06-20
|
||||||
|
**Status:** Approved (pending user review of this doc)
|
||||||
|
**Branch:** `sp7-serialize-837`
|
||||||
|
**Aesthetic direction:** No UI change in this sub-project. Backend + minimal drawer button.
|
||||||
|
|
||||||
|
## 1. Scope
|
||||||
|
|
||||||
|
Add an outbound X12 837P serializer to Cyclone so a parsed `ClaimOutput` can be regenerated as a complete, round-trippable X12 837P file — the same shape as one file in `docs/prodfiles/claims/`. Unblocks the SP6 "resubmit" lane end-to-end (the existing `/api/inbox/rejected/resubmit` currently flips DB state but emits no file): `?download=true` returns the regenerated 837 files in the response so the Inbox can offer a "Resubmit + download bundle" action. Adds a "Download 837" affordance on the claim drawer for single-claim exports.
|
||||||
|
|
||||||
|
**Out of scope** (deferred):
|
||||||
|
|
||||||
|
- **997 ACK transaction set** — TA1's functional-group sibling, deferred.
|
||||||
|
- **Claim ↔ TA1/999 linking** — separate sub-project.
|
||||||
|
- **In-app claim editing UI** — a future SP. The serializer accepts already-edited `ClaimOutput` rows but does not provide the form.
|
||||||
|
- **Batch (multi-CLM) envelope serialization** — v1 is one CLM per file. The `parse_837.py` batch ingestion already handles envelopes; the symmetric serializer is a separate task.
|
||||||
|
- **Frontend redesign of the Inbox or drawer** — same drawer, one extra button.
|
||||||
|
|
||||||
|
## 2. Goals
|
||||||
|
|
||||||
|
1. **One public function.** `cyclone.parsers.serialize_837.serialize_837(claim: ClaimOutput, *, interchange_control_number=..., group_control_number=...) -> str` returns a complete X12 837P text. Round-trips through `cyclone.parsers.parse_837.parse_837_text` for the same claim.
|
||||||
|
2. **Edits propagate.** If a claim's editable fields were changed after parse (charge, prior_auth, diagnoses, service lines), the serialized output reflects the edits — not the original inbound file.
|
||||||
|
3. **Stable segments preserved verbatim.** Billing-provider hierarchy, subscriber hierarchy, payer info, submitter/receiver info, and line provider references come through byte-identical from `claim.raw_segments`. The serializer does not re-emit them.
|
||||||
|
4. **Resubmit lane completes the loop.** `POST /api/inbox/rejected/resubmit?download=true` returns a downloadable bundle of regenerated 837 files for the resubmitted claims. Default behavior unchanged (state flip only).
|
||||||
|
5. **Drawer affordance.** Claim drawer header gains a "Download 837" button that calls the serializer endpoint.
|
||||||
|
|
||||||
|
## 3. Locked decisions
|
||||||
|
|
||||||
|
### 3.1 Approach C — hybrid (fresh envelope + selective body rebuild)
|
||||||
|
|
||||||
|
Three candidate approaches were considered (full rebuild, raw-segments splice, hybrid). Hybrid wins because:
|
||||||
|
|
||||||
|
- **Envelope splice** alone (Approach B) doesn't propagate edits — a charge change in the DB would never reach the output.
|
||||||
|
- **Full rebuild** alone (Approach A) is ~300 LOC of NM1/N3/N4/PRV/HL/SBR emitter code that already exists implicitly inside the parser; re-emitting it invites drift from the parser.
|
||||||
|
- **Hybrid** uses the parser's own captured segments for the parts that don't change (provider/subscriber/payer/submitter hierarchies) and rebuilds only the editable segments (CLM, REF*G1, HI, SV1, DTP*472) from the canonical `ClaimOutput` fields. ~150 LOC. Drift-resistant. Edits propagate.
|
||||||
|
|
||||||
|
### 3.2 Editable vs stable segment classification
|
||||||
|
|
||||||
|
| Segment | Stable / Editable | Reason |
|
||||||
|
|---------|-------------------|--------|
|
||||||
|
| `ISA`, `GS`, `ST`, `SE`, `GE`, `IEA` | **Fresh** | Always rebuilt; carries the new interchange/group control numbers |
|
||||||
|
| `BHT` | **Rebuilt** | Small + deterministic; rebuild from `Envelope` so BHT06 transaction-type-code edits propagate |
|
||||||
|
| `NM1*41` (submitter) | Stable | Submitter name/ID doesn't change per claim |
|
||||||
|
| `PER` | Stable | Submitter contact doesn't change per claim |
|
||||||
|
| `NM1*40` (receiver) | Stable | Payer-as-receiver doesn't change per claim |
|
||||||
|
| `HL*1`, `PRV`, `NM1*85` | Stable | Billing provider identity |
|
||||||
|
| Billing provider `N3`, `N4` | Stable | Address |
|
||||||
|
| `REF*EI` | Stable | Tax ID |
|
||||||
|
| `HL*2`, `SBR` | Stable | Subscriber hierarchy + payer sequence |
|
||||||
|
| `NM1*IL` | Stable | Subscriber name |
|
||||||
|
| Subscriber `N3`, `N4`, `DMG` | Stable | Subscriber address + demographics |
|
||||||
|
| `NM1*PR` | Stable | Payer identity |
|
||||||
|
| **`CLM`** | **Editable** | Charge, frequency, POS, facility code, signature/assignment flags all live on `claim.claim` |
|
||||||
|
| **`REF*G1`** | **Editable** | Prior auth — only emitted when `claim.claim.prior_auth` is set |
|
||||||
|
| **`HI`** | **Editable** | Diagnoses — rebuilt from `claim.diagnoses` (multiple qualifiers supported) |
|
||||||
|
| `LX` | Stable | Service-line counter — regenerated for each line, but the value is positional |
|
||||||
|
| **`SV1`** | **Editable** | Procedure, modifiers, charge, units, unit_type |
|
||||||
|
| **`DTP*472`** | **Editable** | Service date |
|
||||||
|
| `REF*6R` | Stable | Line provider reference — passes through from the matching raw_segment |
|
||||||
|
|
||||||
|
The "editable" segments are looked up by `(segment_id, qualifier)` in `claim.raw_segments` and replaced. Everything else flows through in original order.
|
||||||
|
|
||||||
|
### 3.3 No envelope-level batching
|
||||||
|
|
||||||
|
v1 emits exactly one CLM per output file. A future sub-project can promote the serializer to emit one ST/SE envelope per N claims; the v1 design keeps that door open (the body rebuilder iterates `service_lines` and `diagnoses` cleanly) but does not implement it.
|
||||||
|
|
||||||
|
### 3.4 Resubmit integration: opt-in `?download=true`
|
||||||
|
|
||||||
|
`POST /api/inbox/rejected/resubmit` keeps its current behavior by default — flips `ClaimState.REJECTED → SUBMITTED`, increments `resubmit_count`, returns `{ok, resubmitted, conflicts}`. A new query param `download=true` switches the response to a multipart-style payload:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"ok": true,
|
||||||
|
"resubmitted": ["CLM-1", "CLM-2"],
|
||||||
|
"conflicts": [],
|
||||||
|
"files": [
|
||||||
|
{"claim_id": "CLM-1", "filename": "CLM-1.x12", "x12_text": "ISA*..."},
|
||||||
|
{"claim_id": "CLM-2", "filename": "CLM-2.x12", "x12_text": "ISA*..."}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Control-number rule for the bundle:** each file in `files[]` is assigned a unique 9-digit interchange control number and a unique group control number. The numbers are derived deterministically from the source batch id + the claim's position in the bundle (e.g. `000000101`, `000000102`, …) so two back-to-back resubmit calls produce different envelopes for the same set of claims — required by X12 and matches what CO Medicaid expects. The 837 serializer gains a small `serialize_837_for_resubmit(claim, *, interchange_index: int) -> str` helper that builds the same body with the indexed control number; the default `serialize_837(claim)` keeps `"000000001"` for single-claim draws from the drawer.
|
||||||
|
|
||||||
|
The frontend (Phase 4) collects `files`, builds a zip client-side, and triggers a browser download. v1 does not stream a server-side zip — keeps the backend simple and avoids a new `zipfile` dependency on the response path.
|
||||||
|
|
||||||
|
### 3.5 Drawer download
|
||||||
|
|
||||||
|
`src/components/ClaimDrawer/ClaimDrawerHeader.tsx` gains one button: "Download 837". On click: `api.getClaim837(claimId)` → `Blob` → object URL → anchor download. No new drawer state, no new dependency.
|
||||||
|
|
||||||
|
## 4. Schemas
|
||||||
|
|
||||||
|
### 4.1 `serialize_837.py` — internal helpers
|
||||||
|
|
||||||
|
```python
|
||||||
|
def _build_isa(sender_id: str, receiver_id: str, interchange_control_number: str) -> str
|
||||||
|
def _build_gs(sender_id: str, receiver_id: str, group_control_number: str) -> str
|
||||||
|
def _build_st(control_number: str, implementation_guide: str) -> str
|
||||||
|
def _build_bht(envelope: Envelope) -> str # NEW: pulls BHT06 from envelope
|
||||||
|
def _build_clm(claim: ClaimHeader) -> str # CLM01..CLM07
|
||||||
|
def _build_ref_g1(prior_auth: str | None) -> str # "" if None
|
||||||
|
def _build_hi(diagnoses: list[Diagnosis]) -> str # one HI segment
|
||||||
|
def _build_sv1(line: ServiceLine) -> str
|
||||||
|
def _build_dtp_472(service_date: date | None) -> str
|
||||||
|
def _build_se(count: int, control_number: str) -> str
|
||||||
|
|
||||||
|
def _pass_through_stable_segments(
|
||||||
|
raw_segments: list[list[str]],
|
||||||
|
*,
|
||||||
|
drop: set[tuple[str, str | None]],
|
||||||
|
) -> list[str]:
|
||||||
|
"""Yield raw_segments with the matching editable segments removed,
|
||||||
|
preserving original order of the remaining stable segments."""
|
||||||
|
|
||||||
|
def serialize_837(
|
||||||
|
claim: ClaimOutput,
|
||||||
|
*,
|
||||||
|
interchange_control_number: str = "000000001",
|
||||||
|
group_control_number: str = "1",
|
||||||
|
) -> str
|
||||||
|
```
|
||||||
|
|
||||||
|
`_pass_through_stable_segments` is the disambiguator: it knows the editable segment kinds (by `(seg_id, qualifier_or_None)`) and removes them from `claim.raw_segments`. The body emitter inserts the fresh ones in canonical positions:
|
||||||
|
1. `BHT` (from envelope) — first
|
||||||
|
2. Submitter + receiver block (stable)
|
||||||
|
3. Billing provider hierarchy (stable, including `REF*EI`)
|
||||||
|
4. Subscriber hierarchy (stable)
|
||||||
|
5. Payer (`NM1*PR`, stable)
|
||||||
|
6. `CLM` (rebuilt, editable)
|
||||||
|
7. `REF*G1` (rebuilt if prior_auth)
|
||||||
|
8. `HI` (rebuilt)
|
||||||
|
9. For each service line: `LX` (stable, from raw) + `SV1` (rebuilt) + `DTP*472` (rebuilt) + `REF*6R` (stable)
|
||||||
|
|
||||||
|
If `claim.raw_segments` is empty or missing, `serialize_837` raises `SerializeError("claim has no raw_segments; cannot serialize from-scratch")`. v1 does not implement a full from-scratch emitter (Approach A in §3.1) — that's a future sub-project.
|
||||||
|
|
||||||
|
### 4.2 API response additions
|
||||||
|
|
||||||
|
| Method | Path | Returns |
|
||||||
|
|--------|------|---------|
|
||||||
|
| GET | `/api/claims/{claim_id}/serialize-837` | `text/x12` body, `Content-Disposition: attachment; filename="claim-{claim_id}.x12"`. 404 if claim missing. 422 if no `raw_json`/`raw_segments`. |
|
||||||
|
| POST | `/api/inbox/rejected/resubmit?download=true` | existing JSON, plus `files: [{claim_id, filename, x12_text}]` array when `download=true`. |
|
||||||
|
|
||||||
|
`GET /api/claims/{claim_id}/serialize-837` is mounted only in v1 — no streaming, no cache headers. The 837 text is small (~2-4 KB).
|
||||||
|
|
||||||
|
## 5. UI additions
|
||||||
|
|
||||||
|
- `src/lib/api.ts`: add `api.getClaim837(claimId: string): Promise<Blob>` — does the GET, returns the blob. Add `api.resubmitRejected(claimIds, { download: true }): Promise<{ ok, resubmitted, conflicts, files }>` for the bundle path.
|
||||||
|
- `src/components/ClaimDrawer/ClaimDrawerHeader.tsx`: add a `Button` labeled "Download 837" with `Download` icon, onClick → `api.getClaim837(claim.id)` → object URL → anchor click.
|
||||||
|
- **Resubmit download UX** (P4): the Inbox page's "Resubmit" action gains a small modal — "Resubmit and download bundle?" with two buttons. On confirm: `api.resubmitRejected(ids, { download: true })` → take `files[]` → build a zip in-browser (using `JSZip` via CDN script tag — no new npm dep) → trigger anchor download. On decline: existing path (state flip only).
|
||||||
|
- No state changes in the drawer. No new tests beyond the existing drawer suite.
|
||||||
|
|
||||||
|
## 6. Migration / rollout
|
||||||
|
|
||||||
|
No schema migration. No new dependencies.
|
||||||
|
|
||||||
|
The serializer is purely additive — `parse_837.py`, `serialize_270.py`, `serialize_999.py` are unchanged. The new endpoint is additive; existing callers of `/api/inbox/rejected/resubmit` see no behavior change unless they opt into `?download=true`.
|
||||||
|
|
||||||
|
## 7. Test coverage targets
|
||||||
|
|
||||||
|
| Phase | New tests | Total target |
|
||||||
|
|-------|-----------|--------------|
|
||||||
|
| P1 — serializer module | 9 backend | 517 backend |
|
||||||
|
| P2 — `serialize-837` endpoint | 3 backend | 520 backend |
|
||||||
|
| P3 — drawer button | 1 frontend | 13 frontend |
|
||||||
|
| P4 — resubmit `?download=true` + bundle modal | 2 backend + 1 frontend | 522 backend / 14 frontend |
|
||||||
|
| P5 — prodfile round-trip | 1 backend | 523 backend |
|
||||||
|
|
||||||
|
### 7.1 Test details
|
||||||
|
|
||||||
|
`backend/tests/test_serialize_837.py` (9 tests):
|
||||||
|
|
||||||
|
- `test_round_trip_unchanged_claim_parses_back_equivalent` — serialize → parse → assert `ClaimOutput` deep-equal (modulo `validation` which is recomputed)
|
||||||
|
- `test_round_trip_edited_charge_propagates` — mutate `claim.claim.total_charge` → serialize → parse → assert new charge
|
||||||
|
- `test_round_trip_edited_frequency_propagates` — same pattern for frequency_code
|
||||||
|
- `test_round_trip_edited_prior_auth_propagates` — REF*G1 round-trip
|
||||||
|
- `test_round_trip_added_diagnosis_propagates` — append to `claim.diagnoses` → HI segment count grows
|
||||||
|
- `test_round_trip_edited_service_line_charge_propagates` — SV1 charge round-trip
|
||||||
|
- `test_round_trip_edited_service_date_propagates` — DTP*472 round-trip
|
||||||
|
- `test_no_raw_segments_raises_serialize_error` — empty `raw_segments` raises `SerializeError`
|
||||||
|
- `test_envelope_is_106_char_isa_with_standard_delimiters` — ISA shape check
|
||||||
|
|
||||||
|
`backend/tests/test_api_serialize_837.py` (3 tests):
|
||||||
|
|
||||||
|
- `test_endpoint_returns_x12_attachment` — happy path, checks `Content-Type` and `Content-Disposition`
|
||||||
|
- `test_endpoint_404_for_missing_claim`
|
||||||
|
- `test_endpoint_422_for_claim_without_raw_json`
|
||||||
|
|
||||||
|
`backend/tests/test_inbox_resubmit_download.py` (2 tests):
|
||||||
|
|
||||||
|
- `test_resubmit_default_returns_no_files` — backwards compat
|
||||||
|
- `test_resubmit_download_true_returns_x12_per_resubmitted_claim`
|
||||||
|
|
||||||
|
`backend/tests/test_prodfiles_smoke.py` (1 new test):
|
||||||
|
|
||||||
|
- `test_claims_prodfile_round_trip` — every file in `docs/prodfiles/claims/` (113 files) round-trips through `serialize_837` → `parse_837_text` with deep-equal `ClaimOutput`. Counts as 1 test for the suite total (parametrization kept internal).
|
||||||
|
|
||||||
|
`src/components/ClaimDrawer/ClaimDrawerHeader.test.tsx` (1 test):
|
||||||
|
|
||||||
|
- `test_download_button_calls_api_get_claim_837`
|
||||||
|
|
||||||
|
`src/pages/Inbox.test.tsx` (1 new test):
|
||||||
|
|
||||||
|
- `test_resubmit_with_download_modal_zips_and_downloads` — verifies the modal flow calls the API with `download=true` and triggers a zip+download on confirm.
|
||||||
|
|
||||||
|
## 8. Risk areas
|
||||||
|
|
||||||
|
1. **Segment-position drift.** `claim.raw_segments` is captured at parse time; if the parser ever reorders or drops a segment, the pass-through would shift the body. Mitigation: round-trip tests on every prodfile catch this immediately.
|
||||||
|
2. **Editable vs stable classification drift.** If a payer config adds a new editable segment kind (e.g. K3 notes), the serializer won't know to rebuild it. Mitigation: keep `_EDITABLE_SEGMENT_KINDS` as a single named tuple constant near the top of `serialize_837.py` so it's easy to extend.
|
||||||
|
3. **Refusal to serialize a no-raw claim.** v1 raises rather than producing broken output. The drawer button should disable itself when raw_json is absent. UI: `claim.raw_segments.length > 0` check in the button's `disabled` prop.
|
||||||
|
4. **Bundle size of `?download=true` response.** 100 resubmitted claims × ~3 KB = ~300 KB JSON. Acceptable for a local-only tool. If the resubmit lane ever scales beyond that, switch to NDJSON streaming (same pattern as SP5 tails).
|
||||||
|
|
||||||
|
## 9. Acceptance checklist
|
||||||
|
|
||||||
|
- [ ] Every `docs/prodfiles/claims/*.x12` file (113 files) round-trips through `serialize_837` → `parse_837_text` and produces a `ClaimOutput` deep-equal to the source.
|
||||||
|
- [ ] `GET /api/claims/{claim_id}/serialize-837` returns the regenerated 837 text with the right `Content-Type` and `Content-Disposition`.
|
||||||
|
- [ ] The Claim drawer "Download 837" button produces the same byte sequence as the round-trip test.
|
||||||
|
- [ ] `POST /api/inbox/rejected/resubmit` (default) behavior unchanged.
|
||||||
|
- [ ] `POST /api/inbox/rejected/resubmit?download=true` returns the `files` array and the frontend successfully zips + downloads.
|
||||||
|
- [ ] A claim with `raw_segments=[]` raises `SerializeError` and the API returns 422.
|
||||||
|
- [ ] Full backend suite: ≥ 523 passing (508 + 15 new). Frontend suite: ≥ 14 passing.
|
||||||
|
- [ ] No new pip or npm dependencies. JSZip is loaded via `<script>` tag from a CDN at runtime — no `package.json` change.
|
||||||
|
|
||||||
|
## 10. Out of scope (explicit)
|
||||||
|
|
||||||
|
- **997 ACK** — TA1's functional-group sibling. Deferred.
|
||||||
|
- **Claim ↔ TA1/999 linking** — separate sub-project.
|
||||||
|
- **In-app claim editing UI** — the serializer accepts edited claims but does not provide the editor.
|
||||||
|
- **Batch (multi-CLM) envelope serialization** — v1 is one CLM per file.
|
||||||
|
- **Frontend redesign of the Inbox or drawer** — same drawer, one extra button.
|
||||||
|
|
||||||
|
## 11. References
|
||||||
|
|
||||||
|
- `backend/src/cyclone/parsers/serialize_270.py` — pattern reference (one-inquiry 270 serializer)
|
||||||
|
- `backend/src/cyclone/parsers/serialize_999.py` — pattern reference (one-functional-group 999 serializer)
|
||||||
|
- `backend/src/cyclone/parsers/parse_837.py` — inverse; `claim.raw_segments` capture
|
||||||
|
- `docs/prodfiles/claims/` — 113 sample output files (the shape we emit)
|
||||||
|
- `backend/src/cyclone/api.py:759-783` — existing `inbox_resubmit_rejected` endpoint to extend
|
||||||
|
- `src/components/ClaimDrawer/ClaimDrawerHeader.tsx` — drawer header to extend
|
||||||
Reference in New Issue
Block a user