# Cyclone 837P Parser — Design
**Date:** 2026-06-19
**Status:** Approved (pending user review of this doc)
**Scope:** v1 — Python module + CLI for parsing X12 837P professional claim files into per-claim JSON, with structural + Colorado-Medicaid-specific validation.
---
## 1. Overview
The Cyclone EDI suite (CuNtx) needs a Python module that ingests X12 837P files exported by AxisCare and produces one validated JSON file per claim. The output drives the existing Vite/React frontend (Cyclone) through a future FastAPI service that does not exist yet.
This design covers the parser, the data model, validation, and the CLI. It does not cover the HTTP service, database, or the 835 ERA parser — those are separate, future work items that this module is structured to enable.
## 2. Goals
1. Parse a real X12 837P file (e.g. `docs/prodfiles/837p-from-axiscare/tp11525703-837P-20260618153358831-1of1.txt`) into one structured JSON file per `CLM` segment.
2. Validate each claim against structural rules and Colorado Medical Assistance Program companion-guide rules, collect errors, and continue processing the rest of the batch.
3. Preserve the original raw segments inside each JSON for debugging and round-tripping.
4. Provide a CLI: `python -m cyclone.cli parse-837 --output-dir `.
5. Be configurable so a future 835 parser can reuse `segments`, `validator`, and `models` with a different `PayerConfig`.
## 3. Non-goals (v1)
- FastAPI server / HTTP endpoints. The existing `src/lib/api.ts` is a stub; it stays a stub.
- Database persistence.
- Automatic claim repair (e.g. synthesizing a missing NPI). A `--max-retries` flag exists for v2 hook integration but no fixers ship in v1.
- 835 ERA parsing (separate spec).
- Code-set validation (CPT/HCPCS/ICD-10 vocab). Only structural shape checks.
- UI work. The frontend reads JSON files or — later — API responses; it does not change in this spec.
## 4. Stack
- **Python:** 3.11+ (uses `match`, `tomllib`, `Self` from typing).
- **Pydantic:** v2 (`BaseModel`, `field_validator`, `model_config = ConfigDict(frozen=True, extra="forbid")` for immutable input models).
- **CLI:** `click` 8.x.
- **Tests:** `pytest` 8.x.
- **Linting (optional, not blocking v1):** `ruff`.
No third-party EDI libraries. The X12 format is regular enough to tokenize in ~50 lines.
## 5. Directory layout
```
cyclone/backend/
├── pyproject.toml # PEP 621 metadata, src layout, click + pydantic + pytest deps
├── README.md # install + usage
├── requirements.txt # pinned for non-pipenv installs
├── .gitignore # __pycache__, .pytest_cache, .venv, *.egg-info
├── src/cyclone/
│ ├── __init__.py
│ ├── cli.py # click entrypoint: `python -m cyclone.cli parse-837 ...`
│ └── parsers/
│ ├── __init__.py # public re-exports: parse_837, Claim, PayerConfig, ...
│ ├── exceptions.py # CycloneParseError, CycloneValidationError
│ ├── segments.py # tokenize, parse_isa_delimiters, EDIStream iterator
│ ├── models.py # all Pydantic models
│ ├── payer.py # PayerConfig + co_medicaid() factory
│ ├── validator.py # structural + CO rules, returns ValidationReport
│ └── parse_837.py # orchestrator: text -> list[Claim]
└── tests/
├── __init__.py
├── conftest.py # shared fixtures: minimal 837P, CO Medicaid 837P
├── fixtures/
│ ├── minimal_837p.txt
│ └── co_medicaid_837p.txt
├── test_segments.py
├── test_parse_837.py
├── test_validator.py
└── test_cli.py # CLI smoke test; skips prodfile integration if absent
```
## 6. Data flow
```
input file (text)
│
▼
segments.tokenize(text)
│ detects ISA delimiters, returns list[list[str]]
│ raises CycloneParseError on malformed ISA
▼
parse_837.parse(stream, payer_config)
│ walks HL hierarchy, groups 2400 service lines under 2300 claims,
│ emits one Claim per CLM segment
│ collect errors; do not raise on per-claim failure
▼
validator.validate(claim, payer_config) → ValidationReport
│ append-only; never raises
▼
list[ClaimOutput] (Pydantic)
│
▼
cli writes:
./claims/claim-{ISA13}-{BHT04}.json (per claim, even if failed)
./claims/summary.json (batch summary)
```
## 7. Data model
All models live in `parsers/models.py`. Models are Pydantic v2 `BaseModel`s with `model_config = ConfigDict(extra="ignore", str_strip_whitespace=True)`.
### Top-level
| Model | Notes |
|---|---|
| `ParseResult` | The orchestrator's return type: `claims: list[ClaimOutput]`, `summary: BatchSummary`, `envelope: Envelope`. |
| `Envelope` | ISA / GS / ST / BHT, used to write filenames. Fields: `sender_id`, `receiver_id`, `control_number` (ISA13), `transaction_date` (BHT04), `transaction_time` (BHT05), `implementation_guide` (ST03). |
| `BatchSummary` | `total_claims: int`, `passed: int`, `failed: int`, `failed_claim_ids: list[str]`, `output_dir: str`. |
### `ClaimOutput`
| Field | Type | Source |
|---|---|---|
| `claim_id` | `str` | CLM01 |
| `control_number` | `str` | ISA13 |
| `transaction_date` | `str` (ISO date) | BHT04 |
| `billing_provider` | `BillingProvider` | Loop 2010AA (NM1*85, N3, N4, REF*EI) |
| `subscriber` | `Subscriber` | Loop 2010BA (NM1*IL, N3, N4, DMG) |
| `payer` | `Payer` | Loop 2010BB (NM1*PR) |
| `claim` | `ClaimHeader` | CLM, REF*G1 |
| `diagnoses` | `list[Diagnosis]` | HI segment |
| `service_lines` | `list[ServiceLine]` | Loop 2400 (LX → SV1 → DTP*472 → REF*6R) |
| `validation` | `ValidationReport` | `passed: bool`, `errors: list[ValidationIssue]`, `warnings: list[ValidationIssue]` |
| `raw_segments` | `list[list[str]]` | Original tokenized segments; **always populated** so JSON files are debuggable |
### Sub-models
- `BillingProvider`: `npi`, `name`, `tax_id`, `address: Address`
- `Subscriber`: `first_name`, `last_name` (split from NM103), `member_id`, `dob` (ISO), `gender` ("M"|"F"|"U"), `address: Address`
- `Address`: `line1`, `line2?`, `city`, `state`, `zip`
- `Payer`: `name`, `id`
- `ClaimHeader`: `claim_id` (CLM01), `total_charge` (CLM02, `Decimal`), `place_of_service` (CLM05-1), `frequency_code` (CLM05-3), `provider_signature` (CLM06), `assignment` (CLM07), `release_of_info` (CLM09), `prior_auth?` (REF*G1, optional — see assumption #5 in §14)
- `Diagnosis`: `code`, `qualifier`
- `ServiceLine`: `line_number` (LX01), `procedure: Procedure`, `charge` (SV102, `Decimal`), `unit_type?` (SV103), `units?` (SV104, `Decimal`), `place_of_service?` (SV105), `service_date?` (ISO, from DTP*472), `provider_reference?` (REF*6R)
- `Procedure`: `qualifier` (SV101-1), `code` (SV101-2), `modifiers: list[str]` (SV101-3..6)
- `ValidationIssue`: `rule: str`, `severity: Literal["error","warning"]`, `message: str`, `segment_index?: int`
- `ValidationReport`: `passed: bool`, `errors: list[ValidationIssue]`, `warnings: list[ValidationIssue]`
### `PayerConfig` (`payer.py`)
```python
class PayerConfig(BaseModel):
name: str
sbr09_claim_filing: str # "MC" for CO Medicaid
allowed_claim_frequencies: set[int] # {1, 7, 8} for CO
require_ref_g1_for_adjustments: bool
allowed_bht06: set[str] # {"CH"} for FFS CO; {"RP"} for encounter-only
payer_id: str # "SKCO0"
payer_name: str # "COHCPF"
no_patient_loop: bool # True for CO
encounter_claim_in_same_batch: bool # False for CO (CH and RP must be separate)
```
Built-ins:
- `PayerConfig.co_medicaid()` — defaults for Colorado Medical Assistance Program, derived from `docs/companionguides/837p.md`.
- `PayerConfig.generic_837p()` — relaxed defaults for unknown payers; only structural rules apply.
## 8. Parser internals (`parse_837.py`)
A single `parse(text: str, payer_config: PayerConfig) -> ParseResult` function:
1. Tokenize via `segments.tokenize`.
2. Validate the envelope: ISA / IEA, GS / GE, ST / SE. Mismatched control numbers are reported as a single fatal error on the `ParseResult` (the function never raises for envelope problems; the CLI decides what to do with `summary.total_claims == 0`).
3. Find the billing-provider HL (`HL*1**20*1`). Read the provider from 2010AA.
4. For each subsequent HL (`HL*n*1*22*0`):
- Read subscriber from 2010BA.
- Read payer from 2010BB.
- Scan forward until the next HL or `SE`. For each `CLM`:
- Start a new `ClaimOutput`.
- Capture `REF*G1` (prior auth / Payer Claim Control Number).
- Capture `HI` (diagnoses).
- For each subsequent `LX`/`SV1` block, append a `ServiceLine`.
- Hand the populated `ClaimOutput` to `validator.validate`; store the report back on the model.
5. Build `BatchSummary`.
6. Return `ParseResult`.
Per-claim failures (e.g. malformed `CLM` line) are caught with a `try/except` and recorded as a single error inside that claim's `validation.errors`. The parser never aborts mid-batch.
## 9. Validation rules (`validator.py`)
A list of pure functions `Rule = Callable[[ClaimOutput, PayerConfig], Iterable[ValidationIssue]]`. The validator runs all rules and concatenates results.
| Rule ID | Severity | Description |
|---|---|---|
| `R001_envelope_balanced` | error | ISA/IEA, GS/GE, ST/SE control numbers match |
| `R010_clm01_present` | error | `CLM01` non-empty |
| `R011_total_charge_positive` | error | `CLM02 > 0` |
| `R020_npi_format` | error | All `XX`-qualified NPIs match `^\d{10}$` |
| `R030_frequency_allowed` | error | `CLM05-3 ∈ payer_config.allowed_claim_frequencies` |
| `R031_ref_g1_optional` | warning | REF\*G1 is captured but **not enforced** in v1. The companion guide describes G1 as the encounter TCN, but the production data shows G1 used on every fee-for-service claim with values that look like prior-auth numbers. The rule exists as a placeholder for the payer to tighten the policy in a follow-up; for now it surfaces as informational. |
| `R040_sbr09_matches_payer` | error | `SBR09 == payer_config.sbr09_claim_filing` |
| `R050_diagnosis_present` | error | At least one `HI` diagnosis present |
| `R060_service_dates_present` | error | Every `ServiceLine` has a `service_date` |
| `R070_charges_sum` | warning | `abs(sum(SV1.charge) - CLM02) < 0.01` |
| `R080_no_patient_loop` | error | When `payer_config.no_patient_loop`, fail if a Loop 2000C (patient) HL appears |
| `R090_bht06_allowed` | error | `BHT06 ∈ payer_config.allowed_bht06` |
| `R100_payer_id_matches` | warning | `Payer.id == payer_config.payer_id` (mismatch is non-fatal; informational) |
`--strict` upgrades warnings to errors at the CLI level (post-validation, not inside the rules).
## 10. Output files
The CLI writes to `--output-dir`:
```
/
├── claim-991102977-20260611.json # one per CLM, named -
├── claim-991102977-20260611.json
├── ...
└── summary.json
```
`summary.json` shape:
```json
{
"input_file": "/abs/path/to/input.txt",
"control_number": "991102977",
"transaction_date": "2026-06-11",
"total_claims": 12,
"passed": 11,
"failed": 1,
"failed_claim_ids": ["t991102977o1c3d"],
"issues_by_rule": {"R070_charges_sum": 1}
}
```
If two claims in the same file share `ISA13` and `BHT04` (unusual but legal), the second file gets a numeric suffix: `claim-991102977-20260611-2.json`. Filenames are sanitized; control characters are stripped.
## 11. CLI
`cli.py` exposes a single command group:
```bash
python -m cyclone.cli parse-837 \
--output-dir # required
[--payer co_medicaid] # default: co_medicaid
[--strict] # warnings -> errors
[--max-retries N] # default 0; re-runs validation, does not patch
[--include-raw-segments / --no-raw-segments] # default: include
[--log-level INFO]
```
Exit codes:
- `0` — every claim parsed (failures are written to JSON; this is by design)
- `2` — envelope-level failure (e.g. ISA not found, no `CLM` segments in the file)
- `1` — unexpected exception (bug); message printed to stderr
The CLI prints a one-line summary table to stdout:
```
parsed=12 passed=11 failed=1 output=./claims
```
## 12. Testing
| Test file | Coverage |
|---|---|
| `test_segments.py` | ISA delimiter detection, edge cases (long ISA IDs, missing terminator), `EDIStream` iteration, raises on malformed input |
| `test_parse_837.py` | Parses `fixtures/minimal_837p.txt` and `fixtures/co_medicaid_837p.txt`; asserts claim count, model field values, raw segments preserved, errors collected (not raised) for known-bad input |
| `test_validator.py` | Each `R0xx` rule, both pass and fail cases; `PayerConfig.co_medicaid()` vs `generic_837p()` |
| `test_cli.py` | `CliRunner` invokes `parse-837` on the synthetic fixture and asserts files written. An additional test reads one file from `docs/prodfiles/837p-from-axiscare/` if any are present and is `pytest.skip`-ed otherwise. |
Fixtures:
- `minimal_837p.txt` — hand-written, smallest valid 837P (one subscriber, one claim, one service line). ≤ 30 segments.
- `co_medicaid_837p.txt` — generated from a sanitized subset of a production file. Names are randomized, member IDs are stubbed, addresses use the same ZIP code with a generic street.
## 13. Out of scope / future work
- **FastAPI server** — separate spec. The `ClaimOutput` schema is the contract; the API will serialize the same models.
- **835 ERA parser** — separate spec. Will share `segments.py`, `validator.py` (with a different rule list), and parts of `models.py`.
- **Database persistence** — separate spec.
- **Auto-fix / claim repair** — a `--max-retries` flag exists in v1 to test the wiring, but no fixers ship.
- **Code-set lookups** — CPT/HCPCS/ICD-10 vocab validation deferred; out of scope for v1.
## 14. Open assumptions (will be verified during implementation)
1. Each input file contains exactly one ISA / GS / ST envelope. **Likely true** for AxisCare exports but should be asserted.
2. Subscriber = patient (no 2000C loop) is enforced for CO Medicaid via the companion guide and the production file; this is captured by `R080_no_patient_loop`.
3. The production file uses the standard `*` element and `~` segment delimiters. **Confirmed** in the first 200 lines of `tp11525703-837P-20260618153358831-1of1.txt`.
4. `REF*6R` and `REF*G1` are the only REF qualifiers we care about in v1. Other qualifiers (D9 for ICN, etc.) are captured as opaque strings in `raw_segments` for now.
5. **REF*G1 semantic ambiguity** — the CO companion guide describes G1 as the encounter TCN, but the production data shows G1 used on every FFS claim with small integer values that look like prior-auth numbers. v1 captures the value into `claim.prior_auth` and surfaces its presence as a warning; the rule is intentionally lenient and documented as a follow-up.