docs: add EDI features spec + implementation plan (sub-project 3, 30 tasks)

This commit is contained in:
Tyler
2026-06-20 07:14:14 -06:00
parent 8a289b8e80
commit b47818883a
2 changed files with 595 additions and 0 deletions
@@ -0,0 +1,379 @@
# Sub-project 3 — Backend EDI Features: Implementation Plan
**Date:** 2026-06-20
**Status:** Approved
**Branch:** `edi-features`
**Spec:** `docs/superpowers/specs/2026-06-20-cyclone-edi-features-design.md`
30 tasks across 5 phases. Follows the same task pattern as the SP2 plan (`[ ] Step 1 → Step N → commit`).
## Phase 1 — 837P extra validation rules
### Task 1: REF*G1 enforcement rule (R032)
**Files:**
- Modify: `backend/src/cyclone/parsers/validator.py`
- [ ] **Step 1: Add failing tests**
Append to `backend/tests/test_validator.py`:
```python
def test_r032_ref_g1_required_when_freq_7_no_ref_g1_emits_error(co_medicaid_payer_with_ref_g1_required):
# ... claim with frequency_code=7, no REF*G1
# expect validation error
def test_r032_ref_g1_required_when_freq_8_no_ref_g1_emits_error(co_medicaid_payer_with_ref_g1_required):
# ... claim with frequency_code=8, no REF*G1
# expect validation error
def test_r032_ref_g1_required_when_freq_7_with_ref_g1_passes(co_medicaid_payer_with_ref_g1_required):
# ... claim with frequency_code=7, REF*G1 present
# expect no error
def test_r032_ref_g1_required_when_freq_1_no_ref_g1_passes(co_medicaid_payer_with_ref_g1_required):
# ... claim with frequency_code=1, no REF*G1
# expect no error (rule only fires on 7/8)
```
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Implement `_r032_ref_g1_required` in `validator.py`**
- [ ] **Step 4: Register in `_RULES` list**
- [ ] **Step 5: Commit**
### Task 2: BHT06 transaction-set-purpose-code rule (R033)
**Files:**
- Modify: `backend/src/cyclone/parsers/validator.py`
- [ ] **Step 1: Add failing tests**
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Implement `_r033_bht06_allowed` in `validator.py`**
- [ ] **Step 4: Register in `_RULES` list**
- [ ] **Step 5: Commit**
### Task 3: Verify existing R031 (REF*G1 optional) keeps warning behavior
**Files:**
- Verify: `backend/src/cyclone/parsers/validator.py:54-57`
- [ ] **Step 1: Read current `_r031_ref_g1_optional`**
- [ ] **Step 2: Confirm it stays as a warning for default configs**
- [ ] **Step 3: (No commit — verification task)**
### Task 4: Wire `require_ref_g1_for_adjustments` into CO Medicaid factory
**Files:**
- Verify: `backend/src/cyclone/parsers/payer.py:67`
- [ ] **Step 1: Confirm factory sets the flag for CO**
- [ ] **Step 2: Run existing co_medicaid fixture through new R032**
- [ ] **Step 3: Adjust fixture if false positives appear**
- [ ] **Step 4: Commit if any change**
### Task 5: Phase 1 commit + full suite
```bash
.venv/bin/pytest -q
```
Expected: 205 + 6 = ~211 pass.
---
## Phase 2 — 835 CAS deep-parsing (reason-code lookup)
### Task 6: `parsers/cas_codes.py` with CARC dict
**Files:**
- Create: `backend/src/cyclone/parsers/cas_codes.py`
- Create: `backend/tests/test_cas_codes.py`
- [ ] **Step 1: Add failing tests** for known/unknown lookup, count > 100
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Implement `reason_label` + `all_known_codes` with ~150 CARC codes**
- [ ] **Step 4: Run tests to verify they pass**
- [ ] **Step 5: Commit**
### Task 7: Extend `GET /api/remittances/{id}` to surface `adjustments`
**Files:**
- Modify: `backend/src/cyclone/store.py`
- Modify: `backend/src/cyclone/api.py`
- Modify: `backend/tests/test_api_gets.py`
- [ ] **Step 1: Add failing tests** (empty adjustments returns `[]`, populated returns `{group, reason, label, amount, quantity}`)
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Add a `get_remit_with_adjustments` method to `CycloneStore`**
- [ ] **Step 4: Update `to_ui_remittance` (or add `to_ui_remittance_detail`) to include adjustments array**
- [ ] **Step 5: Update endpoint to use the new mapper**
- [ ] **Step 6: Run tests to verify they pass**
- [ ] **Step 7: Commit**
### Task 8: Frontend Remit type + Remittances page adjustments expansion
**Files:**
- Modify: `src/types/index.ts`
- Modify: `src/pages/Remittances.tsx`
- [ ] **Step 1: Add `adjustments?: { group, reason, label, amount, quantity }[]` to `Remittance`**
- [ ] **Step 2: Update Remittances page to render adjustments expansion**
- [ ] **Step 3: Add test** in Remittances.test.tsx
- [ ] **Step 4: Verify build + tests pass**
- [ ] **Step 5: Commit**
### Task 9: Phase 2 commit + full suite
```bash
.venv/bin/pytest -q && npm test -- --run
```
Expected: ~215 backend, ~9 frontend.
---
## Phase 3 — 999 ACK transaction set
### Task 10: New Pydantic models for 999
**Files:**
- Create: `backend/src/cyclone/parsers/models_999.py`
- Modify: `backend/src/cyclone/parsers/__init__.py`
- [ ] **Step 1: Implement model classes per spec §4.1**
- [ ] **Step 2: Wire lazy re-exports**
- [ ] **Step 3: Verify import + simple instantiation works** (1 sanity test)
- [ ] **Step 4: Commit**
### Task 11: 999 parser
**Files:**
- Create: `backend/src/cyclone/parsers/parse_999.py`
- [ ] **Step 1: Add failing tests** with minimal_999.txt fixture (basic happy path)
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Implement orchestrator** following `parse_835.py` pattern
- [ ] **Step 4: Run tests to verify they pass**
- [ ] **Step 5: Commit**
### Task 12: 999 serializer (outbound)
**Files:**
- Create: `backend/src/cyclone/parsers/serialize_999.py`
- [ ] **Step 1: Add failing round-trip test** (parse → serialize → parse → assert equal)
- [ ] **Step 2: Run test to verify it fails**
- [ ] **Step 3: Implement `serialize_999`** (ISA/GS/ST/AK1/[AK2/AK3/AK4/AK5*]/AK9/SE/GE/IEA)
- [ ] **Step 4: Run test to verify it passes**
- [ ] **Step 5: Commit**
### Task 13: `acks` table + ORM + migration
**Files:**
- Create: `backend/src/cyclone/migrations/0002_acks.sql`
- Modify: `backend/src/cyclone/db.py`
- [ ] **Step 1: Add migration SQL** per spec §4.4
- [ ] **Step 2: Add ORM model `Ack` to `db.py`**
- [ ] **Step 3: Add `CycloneStore.add_ack(...)` method**
- [ ] **Step 4: Run migration on a fresh DB; verify table exists**
- [ ] **Step 5: Commit**
### Task 14: `POST /api/parse-999` endpoint
**Files:**
- Modify: `backend/src/cyclone/api.py`
- [ ] **Step 1: Add failing tests** (parse + persist, returns summary)
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Add endpoint** mirroring `parse-835` JSON path
- [ ] **Step 4: Run tests to verify they pass**
- [ ] **Step 5: Commit**
### Task 15: Auto-generate 999 on 837 ingest via `?ack=true`
**Files:**
- Modify: `backend/src/cyclone/api.py`
- Modify: `backend/tests/test_api_parse_persists.py`
- [ ] **Step 1: Add 3 failing tests** (default no ack, ack=true happy path, ack=true with rejected claims)
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Add `ack: bool = Query(False)` param to `parse_837_endpoint`**
- [ ] **Step 4: Build `BatchAckBuilder` that converts validation report to 999**
- [ ] **Step 5: Persist `Ack` row, attach to response body**
- [ ] **Step 6: Run tests to verify they pass**
- [ ] **Step 7: Commit**
### Task 16: Frontend `/acks` page + sidebar entry
**Files:**
- Modify: `src/lib/api.ts`
- Modify: `src/hooks/` (new `useAcks.ts`)
- Modify: `src/types/index.ts`
- Create: `src/pages/Acks.tsx`
- Create: `src/pages/Acks.test.tsx`
- Modify: `src/components/Sidebar.tsx`
- [ ] **Step 1: Add `Ack` type to `src/types/index.ts`**
- [ ] **Step 2: Add `api.listAcks()` + `api.getAck(id)` to `src/lib/api.ts`**
- [ ] **Step 3: Create `useAcks` hook**
- [ ] **Step 4: Create `Acks.tsx` page** with empty state, populated list, download button
- [ ] **Step 5: Add sidebar nav entry with `CheckCircle2` icon**
- [ ] **Step 6: Add tests** (1-2 cases: empty + populated)
- [ ] **Step 7: Verify typecheck + build + tests pass**
- [ ] **Step 8: Commit**
### Task 17: Phase 3 commit + full suite
```bash
.venv/bin/pytest -q && npm test -- --run
```
Expected: ~225 backend, ~10 frontend.
---
## Phase 4 — 270/271 eligibility
### Task 18: 270 request models
**Files:**
- Create: `backend/src/cyclone/parsers/models_270.py`
- [ ] **Step 1: Implement model classes per spec §4.2**
- [ ] **Step 2: Wire lazy re-exports**
- [ ] **Step 3: Sanity test**
- [ ] **Step 4: Commit**
### Task 19: 271 response models + service-type-code dict
**Files:**
- Create: `backend/src/cyclone/parsers/models_271.py`
- [ ] **Step 1: Implement model classes per spec §4.3**
- [ ] **Step 2: Add inline `SERVICE_TYPE_CODES` dict** (~50 codes)
- [ ] **Step 3: Add `service_type_description(code)` helper**
- [ ] **Step 4: Sanity test**
- [ ] **Step 5: Commit**
### Task 20: 270 request parser
**Files:**
- Create: `backend/src/cyclone/parsers/parse_270.py`
- [ ] **Step 1: Add failing tests** with minimal_270.txt fixture
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Implement orchestrator**
- [ ] **Step 4: Run tests to verify they pass**
- [ ] **Step 5: Commit**
### Task 21: 271 response parser
**Files:**
- Create: `backend/src/cyclone/parsers/parse_271.py`
- [ ] **Step 1: Add failing tests** with minimal_271.txt fixture (2 EB segments)
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Implement orchestrator** with EB segment handling
- [ ] **Step 4: Run tests to verify they pass**
- [ ] **Step 5: Commit**
### Task 22: 270 request serializer (outbound)
**Files:**
- Create: `backend/src/cyclone/parsers/serialize_270.py`
- [ ] **Step 1: Add failing round-trip test**
- [ ] **Step 2: Run test to verify it fails**
- [ ] **Step 3: Implement `serialize_270`**
- [ ] **Step 4: Run test to verify it passes**
- [ ] **Step 5: Commit**
### Task 23: `POST /api/eligibility/request` endpoint
**Files:**
- Modify: `backend/src/cyclone/api.py`
- Create: `backend/tests/test_api_eligibility.py`
- [ ] **Step 1: Add failing test**
- [ ] **Step 2: Run test to verify it fails**
- [ ] **Step 3: Implement endpoint**
- [ ] **Step 4: Run test to verify it passes**
- [ ] **Step 5: Commit**
### Task 24: `POST /api/eligibility/parse-271` endpoint
**Files:**
- Modify: `backend/src/cyclone/api.py`
- [ ] **Step 1: Add 2 failing tests** (happy path + parse error)
- [ ] **Step 2: Run tests to verify they fail**
- [ ] **Step 3: Implement endpoint**
- [ ] **Step 4: Run tests to verify they pass**
- [ ] **Step 5: Commit**
### Task 25: Fixtures + cross-parser round-trip test
**Files:**
- Create: `backend/tests/fixtures/minimal_270.txt`
- Create: `backend/tests/fixtures/minimal_271.txt`
- [ ] **Step 1: Write minimal_270.txt fixture** (small valid 270 request)
- [ ] **Step 2: Write minimal_271.txt fixture** (small valid 271 response with 2 EB segments)
- [ ] **Step 3: Add round-trip integration test** (parse 270 → serialize → parse → assert equal)
- [ ] **Step 4: Verify tests pass**
- [ ] **Step 5: Commit**
### Task 26: Phase 4 commit + full suite
```bash
.venv/bin/pytest -q
```
Expected: ~229 backend, ~10 frontend.
---
## Phase 5 — Docs + smoke
### Task 27: Spec doc already committed (T0)
Verify `docs/superpowers/specs/2026-06-20-cyclone-edi-features-design.md` is on branch.
### Task 28: Update root README
**Files:**
- Modify: `README.md`
- [ ] **Step 1: Update Roadmap section** to mark SP3 shipped and describe 4 areas
- [ ] **Step 2: Commit**
### Task 29: End-to-end smoke
- [ ] **Step 1: Clean DB**, start server, parse 837 with `?ack=true`
- [ ] **Step 2: Confirm 999 in response + `acks` row in DB**
- [ ] **Step 3: Parse 999 fixture**, confirm endpoint round-trips
- [ ] **Step 4: Build 270 request** via API, paste into parser
- [ ] **Step 5: Commit empty** `smoke: end-to-end SP3 flow passes`
### Task 30: Final cleanup + merge worktree to main
- [ ] **Step 1: Verify worktree is clean**
- [ ] **Step 2: Discard main's stale uncommitted changes** (if any)
- [ ] **Step 3: Fast-forward merge `edi-features` into `main`**
- [ ] **Step 4: Remove worktree + delete branch**
---
## Test count targets
| Phase | New backend | New frontend | Running total |
|-------|-------------|--------------|---------------|
| Start | 205 | 8 | — |
| P1 end | +6 | 0 | 211 / 8 |
| P2 end | +4 | +1 | 215 / 9 |
| P3 end | +10 | +2 | 225 / 11 |
| P4 end | +4 | 0 | 229 / 11 |
## Risk reminders
- 270/271 scope creep — keep tight to EQ/EB surface.
- 999 auto-gen semantics — "rejected" = validation errors.
- CARC codes drift — `LAST_UPDATED` constant for tracking.
- Migration ordering — `0002_acks.sql` after `0001_initial.sql`, idempotent.
@@ -0,0 +1,216 @@
# Sub-project 3 — Backend EDI Features: Design Spec
**Date:** 2026-06-20
**Status:** Approved
**Branch:** `edi-features`
## 1. Scope
Four backend EDI feature areas extending cyclone beyond the 837P parser and 835 ERA parser already shipped:
1. **Additional 837P validation rules** — REF\*G1 enforcement for adjustment-frequency claims, BHT06 transaction-set-purpose-code allowlist.
2. **835 CAS deep-parsing** — surface CARC/RARC reason-code labels alongside the existing CAS persistence.
3. **999 ACK** — parse inbound 999s, serialize outbound 999s, optionally auto-generate a 999 in response to an inbound 837P.
4. **270/271 eligibility** — parse 270 requests and 271 responses, serialize 270 requests.
## 2. Out of scope (deferred to future sub-projects)
- TA1 / 997 transaction sets (Cyclone doesn't generate them today; deferred)
- Real-time payer eligibility round-trip (would require an outbound HTTP client + payer credentials)
- 999 ACK UI drilldown beyond a list page
- Per-claim ACK row linking in the DB
## 3. Locked decisions
### 3.1 CAS reason-code lookup = inline Python dict
A new `parsers/cas_codes.py` module with ~150 CARC codes (the most common in Colorado Medicaid remits: CO-45, CO-97, PR-1, PR-2, PR-3, CO-16, CO-29, CO-50, CO-109, CO-119, etc.). No external CSV dependency — same shape as `claim_status_label(code)` at `parsers/models_835.py:55-71`.
```python
def reason_label(group_code: str, reason_code: str) -> str
def all_known_codes() -> list[tuple[str, str, str]] # (group, reason, label)
```
Includes a `LAST_UPDATED = "2026-06-20"` module constant for refresh tracking.
### 3.2 999 ACKs get a dedicated `acks` table
A new migration `0002_acks.sql` adds the table rather than reusing `batches` with a `kind="999"` + `parent_batch_id` FK. Cleaner symmetry with the `cas_adjustments` / `matches` patterns. The `acks` table holds the *response*, not the source batch.
```sql
CREATE TABLE acks (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source_batch_id TEXT NOT NULL REFERENCES batches(id) ON DELETE CASCADE,
accepted_count INTEGER NOT NULL DEFAULT 0,
rejected_count INTEGER NOT NULL DEFAULT 0,
received_count INTEGER NOT NULL DEFAULT 0,
ack_code TEXT NOT NULL,
parsed_at DATETIME NOT NULL,
raw_json TEXT
);
CREATE INDEX ix_acks_source_batch_id ON acks(source_batch_id);
```
### 3.3 Auto-999 generation gated by `?ack=true`
`POST /api/parse-837` gains a new `ack: bool = Query(False)` parameter. When True, after `store.add(record)`, build a 999 from the validation report:
- For each claim: AK2 → AK5 with `A` (accepted) for passed validation, `R` (rejected) for failed, `E` (accepted-with-errors) for warnings.
- Serialize via `serialize_999`, persist the resulting `Ack` row.
- Add `body["ack"] = { id, accepted_count, rejected_count, raw_999_text }` to the JSON response.
Default behavior unchanged (no auto-ACK).
### 3.4 270/271 = API-only, no UI, no DB persistence
Ephemeral request/response round-trip. The operator pastes a 270 query, gets a printable 270 file, submits manually, pastes the 271 back. v1 captures the minimum useful surface (EQ inquiry + EB coverage-benefit response); anything more is a separate sub-project.
## 4. Schemas
### 4.1 999 models (`parsers/models_999.py`)
```python
class ParseResult999(BaseModel):
envelope: Envelope
functional_groups: list[FunctionalGroupResponse] # AK1/AK2/AK9
class FunctionalGroupResponse(BaseModel):
ak1: AcknowledgmentHeader # identifies the source functional group
sets: list[SetFunctionalGroupResponse] # one per ST/SE set
class AcknowledgmentHeader(BaseModel):
functional_id_code: str # e.g. "HC"
group_control_number: str # from GS06
class SetFunctionalGroupResponse(BaseModel):
ak2: SetHeader | None
segment_errors: list[SegmentError]
element_errors: list[ElementError]
ak5: SegmentAcceptReject # A / R / E / W
class SegmentError(BaseModel):
ak3: SegmentContext
loop_id: str | None
segment_position: int
class ElementError(BaseModel):
ak3_ref: int # FK by row id
element_position: int
class SegmentAcceptReject(BaseModel):
code: str # A=accepted, R=rejected, E=accepted-with-errors, W=warning, X=rejected-all
class FunctionalGroupAck(BaseModel):
ak1: AcknowledgmentHeader
received_count: int # AK905
accepted_count: int # AK906
rejected_count: int # AK907
ack_code: str # AK909
```
### 4.2 270 request models (`parsers/models_270.py`)
```python
class ParseResult270(BaseModel):
envelope: Envelope
information_source: InformationSource270 # 2100A — payer
information_receiver: InformationReceiver270 # 2100B — receiver
subscriber: Subscriber270 # 2000C
patient: Patient270 | None # 2100C
inquiries: list[EligibilityBenefitInquiry] # EQ segments
class EligibilityBenefitInquiry(BaseModel):
service_type_code: str # EQ01
coverage_info: str | None
class Patient270(BaseModel):
first_name: str | None
last_name: str | None
member_id: str | None
dob: date | None
gender: str | None
relationship: str | None
```
### 4.3 271 response models (`parsers/models_271.py`)
```python
class ParseResult271(BaseModel):
envelope: Envelope
information_source: InformationSource271
information_receiver: InformationReceiver271
subscriber: Subscriber271
patient: Patient271 | None
coverage_benefits: list[CoverageBenefit] # EB segments, the heart of 271
class CoverageBenefit(BaseModel):
service_type_code: str # EB01
coverage_level: str # EB02 (INDividual, FAMily, etc.)
service_type_description: str | None # resolved from inline dict
benefit_amount: Decimal | None
in_plan_network: bool | None
authorization_required: bool | None
plan_coverage_description: str | None
time_qualifier: str | None
benefit_date: date | None
additional_info: list[BenefitAdditionalInfo]
```
### 4.4 `acks` ORM model
```python
class Ack(Base):
__tablename__ = "acks"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
source_batch_id: Mapped[str] = mapped_column(
String(64), ForeignKey("batches.id", ondelete="CASCADE"), nullable=False,
)
accepted_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
rejected_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
received_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
ack_code: Mapped[str] = mapped_column(String(8), nullable=False)
parsed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
raw_json: Mapped[Optional[dict]] = mapped_column(JSONText, nullable=True)
```
## 5. API additions
| Method | Path | Body / Query | Returns |
|--------|------|--------------|---------|
| POST | `/api/parse-999` | file upload | `ParseResult999` + persisted `acks` row |
| POST | `/api/parse-837?ack=true` | file upload | existing response + `body["ack"]` |
| GET | `/api/acks` | — | list of `Ack` records |
| GET | `/api/acks/{id}` | — | `Ack` + parsed 999 detail |
| POST | `/api/eligibility/request` | JSON body | raw 270 text |
| POST | `/api/eligibility/parse-271` | text body or file upload | `ParseResult271` summary |
## 6. UI additions
- New `/acks` page (`src/pages/Acks.tsx`) — list view grouped by source batch with "Download 999" button.
- `useAcks` hook (`src/hooks/useAcks.ts`).
- `api.listAcks()` + `api.getAck(id)` in `src/lib/api.ts`.
- Sidebar nav entry with `CheckCircle2` icon.
- Remittances page gains an "Adjustments" expansion row that lists CAS rows with reason-code labels.
No new UI for 270/271 (API-only per §3.4).
## 7. Migration ordering
`0002_acks.sql` runs after `0001_initial.sql`. `db_migrate.run()` is idempotent — re-running on a fresh DB is a no-op. `create_all` is also a no-op when migration is current (per SP2 pattern). No data backfill needed (acks table is new).
## 8. Test coverage targets
| Phase | New tests | Total target |
|-------|-----------|--------------|
| P1 — 837P rules | ~6 | ~211 backend |
| P2 — CAS deep-parsing | ~4 | ~215 backend |
| P3 — 999 ACK | ~10 backend + ~2 frontend | ~225 backend / ~10 frontend |
| P4 — 270/271 | ~4 backend | ~229 backend / ~10 frontend |
| P5 — smoke | — | (verification) |
## 9. Risk areas
1. **270/271 scope creep.** X12 specs are large; we capture only the minimum useful surface.
2. **999 auto-gen semantics.** "Rejected" is loosely defined; we use validation errors as the rejection signal. Document this in the OpenAPI/scalar docs.
3. **CARC codes drift.** Inline dict is a snapshot dated 2026-06-20. Plan to refresh annually.
4. **Schema migration ordering.** Idempotent on fresh DBs; no data backfill. Confirmed safe.