docs(plan): SP33 Co TXIX payer fix implementation
This commit is contained in:
@@ -0,0 +1,768 @@
|
|||||||
|
# SP33 — Co TXIX Payer Fix Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use `superpowers:executing-plans` (Tyler picked inline execution for SP33, single-operator night-shift). Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Make dzinesco's CO Medicaid 837P submissions emit `NM1*PR*...*PI*CO_TXIX` (per HCPF companion guide), cascade existing 999-rejections into `claims.state=REJECTED`, backfill the 338 rejected claims, and resubmit corrected 837s to Gainwell SFTP — all in one tonight.
|
||||||
|
|
||||||
|
**Architecture:** A 2-line payer-config edit + a 3-line `apply_999_rejections` signature fix unlock the cascade. Two new `cyclone.cli` subcommands do the backfill and the resubmit. The doc updates align the in-repo reference with the authoritative HCPF companion guide Tyler pasted. Atomic single merge into `main`.
|
||||||
|
|
||||||
|
**Tech Stack:** Python 3.11, SQLAlchemy + SQLite/SQLCipher, paramiko (SFTP), pytest. Frontend untouched.
|
||||||
|
|
||||||
|
**Spec:** [`docs/superpowers/specs/2026-07-02-cyclone-co-txix-payer-fix-design.md`](../specs/2026-07-02-cyclone-co-txix-payer-fix-design.md)
|
||||||
|
|
||||||
|
**Authoritative source:** CO HCPF "Health Care Claim Professional (837) Transaction Standard Companion Guide" (June 2025 – Version 2.5). Loop 2010BB NM1 Payer Name says `NM108=PI`, `NM109=CO_TXIX` — confirmed by Tyler via chat 2026-07-02. In-repo reference for shape: [`docs/goodclaim.x12`](../../goodclaim.x12).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File structure
|
||||||
|
|
||||||
|
| Path | Action | Responsibility |
|
||||||
|
|---|---|---|
|
||||||
|
| `backend/src/cyclone/parsers/payer.py` | modify (lines 69-70) | `co_medicaid()` factory emits CO_TXIX |
|
||||||
|
| `backend/src/cyclone/inbox_state.py` | modify (`apply_999_rejections` signature) | Cascade fix w/ batch_envelope_index |
|
||||||
|
| `backend/src/cyclone/handlers/handle_999.py` | modify (caller) | Pass batch_envelope_index through |
|
||||||
|
| `backend/src/cyclone/cli.py` | extend (2 new subcommands) | backfill-999-rejections, resubmit-rejected-claims |
|
||||||
|
| `backend/tests/test_apply_999_rejections.py` | create | Cascade-fix coverage |
|
||||||
|
| `backend/tests/test_serialize_837.py` | extend | Round-trip with CO_TXIX |
|
||||||
|
| `docs/reference/837p.md` | modify (line 48) | Doc update |
|
||||||
|
| `docs/reference/co-medicaid.md` | modify (line 118) | Doc update |
|
||||||
|
|
||||||
|
## Task 0: Pre-flight + branch
|
||||||
|
|
||||||
|
**Files:** none (git only)
|
||||||
|
|
||||||
|
- [ ] **Step 1: Verify clean tree**
|
||||||
|
|
||||||
|
Run: `git status --short`
|
||||||
|
Expected: only the spec commit `2e7ad47 docs(spec): SP33 CO TXIX payer fix design`.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Create branch off main**
|
||||||
|
|
||||||
|
Run: `git checkout -b sp33-co-txix-payer-fix main`
|
||||||
|
Expected: `Switched to a new branch 'sp33-co-txix-payer-fix'` (already on `main` at `2e7ad47` or `dfd6542+`).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Confirm Docker backend is up**
|
||||||
|
|
||||||
|
Run: `docker ps --filter name=cyclone-backend-1 --format '{{.Names}} {{.Status}}'`
|
||||||
|
Expected: `cyclone-backend-1 Up X minutes (healthy)`.
|
||||||
|
|
||||||
|
## Task 1: Fix `co_medicaid()` factory
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `backend/src/cyclone/parsers/payer.py:69-70`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Read the current block (sanity)**
|
||||||
|
|
||||||
|
Run: `sed -n '67,72p' backend/src/cyclone/parsers/payer.py`
|
||||||
|
Expected output:
|
||||||
|
|
||||||
|
```
|
||||||
|
allowed_bht06={"CH"},
|
||||||
|
payer_id="SKCO0",
|
||||||
|
payer_name="COHCPF",
|
||||||
|
no_patient_loop=True,
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Apply the fix (2 lines)**
|
||||||
|
|
||||||
|
`search_replace` exact:
|
||||||
|
|
||||||
|
OLD:
|
||||||
|
```
|
||||||
|
payer_id="SKCO0",
|
||||||
|
payer_name="COHCPF",
|
||||||
|
```
|
||||||
|
|
||||||
|
NEW:
|
||||||
|
```
|
||||||
|
payer_id="CO_TXIX",
|
||||||
|
payer_name="CO_TXIX",
|
||||||
|
```
|
||||||
|
|
||||||
|
(Drop the bad stale-comment line above it too — the `co_medicaid()` docstring still says "Source: `docs/companionguides/837p.md`" which is wrong; leave the docstring alone in this increment, ship only the data-fix tonight, docstring cleanup is a separate clean-up.)
|
||||||
|
|
||||||
|
- [ ] **Step 3: Verify the diff**
|
||||||
|
|
||||||
|
Run: `git diff backend/src/cyclone/parsers/payer.py`
|
||||||
|
Expected: a 2-line change, exactly the two `payer_id`/`payer_name` lines.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Add test**
|
||||||
|
|
||||||
|
Create: `backend/tests/test_co_medicaid_payer_id_emits_CO_TXIX.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""SP33 regression test: ``PayerConfig.co_medicaid()`` emits CO_TXIX."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
def test_co_medicaid_factory_emits_CO_TXIX():
|
||||||
|
"""Per HCPF 837P Companion Guide, NM1*PR NM109 must equal CO_TXIX."""
|
||||||
|
from cyclone.parsers.payer import PayerConfig
|
||||||
|
cfg = PayerConfig.co_medicaid()
|
||||||
|
assert cfg.payer_id == "CO_TXIX", (
|
||||||
|
f"co_medicaid().payer_id must be 'CO_TXIX' (HCPF guide), got {cfg.payer_id!r}"
|
||||||
|
)
|
||||||
|
assert cfg.payer_name == "CO_TXIX", (
|
||||||
|
f"co_medicaid().payer_name must be 'CO_TXIX' (matches docs/goodclaim.x12), got {cfg.payer_name!r}"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run the test in the container (since the venv on host isn't trustworthy)**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
docker exec cyclone-backend-1 bash -c "cd /var/lib/cyclone && .venv/bin/pytest tests/test_co_medicaid_payer_id_emits_CO_TXIX.py -v"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: 1 passed.
|
||||||
|
|
||||||
|
- [ ] **Step 6: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add backend/src/cyclone/parsers/payer.py backend/tests/test_co_medicaid_payer_id_emits_CO_TXIX.py
|
||||||
|
git commit -m "feat(sp33): PayerConfig.co_medicaid() emits CO_TXIX"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task 2: Fix `apply_999_rejections` cascade
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `backend/src/cyclone/inbox_state.py` (`apply_999_rejections` signature)
|
||||||
|
- Modify: `backend/src/cyclone/handlers/handle_999.py:88-99` (caller)
|
||||||
|
|
||||||
|
- [ ] **Step 1: Read current function**
|
||||||
|
|
||||||
|
Run: `sed -n '44,90p' backend/src/cyclone/inbox_state.py`
|
||||||
|
Expected: see `claim = claim_lookup(sr.set_control_number)` at ~line 56.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Apply signature + body change**
|
||||||
|
|
||||||
|
OLD (the full function body, lines 44-69):
|
||||||
|
|
||||||
|
```python
|
||||||
|
def apply_999_rejections(
|
||||||
|
session: Session,
|
||||||
|
parsed_999,
|
||||||
|
*,
|
||||||
|
claim_lookup: Callable[[str], Claim | None],
|
||||||
|
) -> Apply999Result:
|
||||||
|
"""For each set response with code R or E, look up the matching claim and
|
||||||
|
move it to REJECTED. Idempotent on already-rejected claims.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
session: SQLAlchemy session.
|
||||||
|
parsed_999: a ParseResult999 (or any object with .set_responses).
|
||||||
|
claim_lookup: callable from patient_control_number → Claim or None.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Apply999Result with lists of matched claim ids and orphan PCNs.
|
||||||
|
"""
|
||||||
|
result = Apply999Result()
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
for sr in parsed_999.set_responses:
|
||||||
|
code = sr.set_accept_reject.code
|
||||||
|
if code not in ("R", "E", "X"):
|
||||||
|
continue
|
||||||
|
|
||||||
|
claim = claim_lookup(sr.set_control_number)
|
||||||
|
if claim is None:
|
||||||
|
result.orphans.append(sr.set_control_number)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if claim.state == ClaimState.REJECTED:
|
||||||
|
# Idempotent: don't double-mutate.
|
||||||
|
continue
|
||||||
|
|
||||||
|
claim.state = ClaimState.REJECTED
|
||||||
|
claim.state_changed_at = now
|
||||||
|
claim.rejected_at = now
|
||||||
|
claim.rejection_reason = _build_reason(
|
||||||
|
code, len(sr.segment_errors or [])
|
||||||
|
)
|
||||||
|
result.matched.append(claim.id)
|
||||||
|
|
||||||
|
if result.matched or result.orphans:
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
return result
|
||||||
|
```
|
||||||
|
|
||||||
|
NEW:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def apply_999_rejections(
|
||||||
|
session: Session,
|
||||||
|
parsed_999,
|
||||||
|
*,
|
||||||
|
claim_lookup: Callable[[str], Claim | None],
|
||||||
|
batch_envelope_index: dict[str, list[str]] | None = None,
|
||||||
|
) -> Apply999Result:
|
||||||
|
"""For each set response with code R or E, look up the matching claim and
|
||||||
|
move it to REJECTED. Idempotent on already-rejected claims.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
session: SQLAlchemy session.
|
||||||
|
parsed_999: a ParseResult999 (or any object with .set_responses).
|
||||||
|
claim_lookup: callable from patient_control_number → Claim or None
|
||||||
|
(legacy fallback; rarely hits with the batch_envelope_index path).
|
||||||
|
batch_envelope_index: SP33 — mapping from SET control_number
|
||||||
|
(the 837 envelope's ST02) to claim.id list for claims in that
|
||||||
|
SET. Mirrors apply_999_acceptances so SET-level rejections
|
||||||
|
correctly cascade.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Apply999Result with lists of matched claim ids and orphan PCNs.
|
||||||
|
"""
|
||||||
|
result = Apply999Result()
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
index = batch_envelope_index or {}
|
||||||
|
|
||||||
|
for sr in parsed_999.set_responses:
|
||||||
|
code = sr.set_accept_reject.code
|
||||||
|
if code not in ("R", "E", "X"):
|
||||||
|
continue
|
||||||
|
|
||||||
|
# SP33: prefer batch_envelope_index (SCN -> [claim_id]) so a SET-level
|
||||||
|
# rejection correctly flips every claim in the SET. Fall back to
|
||||||
|
# the legacy claim_lookup when the index is empty for this SCN.
|
||||||
|
candidate_ids = index.get(sr.set_control_number, []) or []
|
||||||
|
claims_to_reject = []
|
||||||
|
if candidate_ids:
|
||||||
|
claims_to_reject = (
|
||||||
|
session.query(Claim)
|
||||||
|
.filter(Claim.id.in_(candidate_ids))
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
legacy = claim_lookup(sr.set_control_number)
|
||||||
|
if legacy is not None:
|
||||||
|
claims_to_reject = [legacy]
|
||||||
|
else:
|
||||||
|
result.orphans.append(sr.set_control_number)
|
||||||
|
continue
|
||||||
|
|
||||||
|
for claim in claims_to_reject:
|
||||||
|
if claim.state == ClaimState.REJECTED:
|
||||||
|
# Idempotent: don't double-mutate.
|
||||||
|
continue
|
||||||
|
claim.state = ClaimState.REJECTED
|
||||||
|
claim.state_changed_at = now
|
||||||
|
claim.rejected_at = now
|
||||||
|
claim.rejection_reason = _build_reason(
|
||||||
|
code, len(sr.segment_errors or [])
|
||||||
|
)
|
||||||
|
result.matched.append(claim.id)
|
||||||
|
|
||||||
|
if result.matched or result.orphans:
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
return result
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Update the caller in handle_999.py**
|
||||||
|
|
||||||
|
OLD (line 88):
|
||||||
|
|
||||||
|
```python
|
||||||
|
rejection_result = apply_999_rejections(
|
||||||
|
session, result, claim_lookup=_lookup,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
NEW:
|
||||||
|
|
||||||
|
```python
|
||||||
|
rejection_result = apply_999_rejections(
|
||||||
|
session, result,
|
||||||
|
claim_lookup=_lookup,
|
||||||
|
batch_envelope_index=batch_index,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Create the cascade test**
|
||||||
|
|
||||||
|
Create: `backend/tests/test_apply_999_rejections.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""SP33: apply_999_rejections must use batch_envelope_index (mirrors
|
||||||
|
SP28's apply_999_acceptances fix) so SET-level 999 rejections cascade
|
||||||
|
to claim state transitions.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.db import Claim, ClaimState
|
||||||
|
from cyclone.inbox_state import apply_999_rejections, Apply999Result
|
||||||
|
|
||||||
|
|
||||||
|
def _mk_set_response(code: str = "R", scn: str = "991102994"):
|
||||||
|
"""Build a stand-in for ParseResult999.set_responses[0]."""
|
||||||
|
return SimpleNamespace(
|
||||||
|
set_accept_reject=SimpleNamespace(code=code),
|
||||||
|
set_control_number=scn,
|
||||||
|
segment_errors=[],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _mk_claim(state: str = "SUBMITTED"):
|
||||||
|
c = MagicMock(spec=Claim)
|
||||||
|
c.id = "claim-" + state.lower()
|
||||||
|
c.state = state
|
||||||
|
return c
|
||||||
|
|
||||||
|
|
||||||
|
def test_set_level_rejection_cascades_via_envelope_index():
|
||||||
|
session = MagicMock()
|
||||||
|
session.query.return_value.filter.return_value.all.return_value = [_mk_claim()]
|
||||||
|
|
||||||
|
parsed = SimpleNamespace(
|
||||||
|
set_responses=[_mk_set_response(code="R", scn="991102994")]
|
||||||
|
)
|
||||||
|
index = {"991102994": ["claim-submitted"]}
|
||||||
|
|
||||||
|
result = apply_999_rejections(
|
||||||
|
session, parsed,
|
||||||
|
claim_lookup=lambda _pcn: None,
|
||||||
|
batch_envelope_index=index,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(result.matched) == 1
|
||||||
|
assert result.matched[0] == "claim-submitted"
|
||||||
|
assert result.orphans == []
|
||||||
|
session.commit.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
|
def test_set_level_rejection_without_index_still_uses_legacy_lookup():
|
||||||
|
"""Backwards compat: when batch_envelope_index is None, the function
|
||||||
|
must fall back to claim_lookup (existing behavior preserved)."""
|
||||||
|
session = MagicMock()
|
||||||
|
parsed = SimpleNamespace(
|
||||||
|
set_responses=[_mk_set_response(code="R", scn="991102994")]
|
||||||
|
)
|
||||||
|
legacy_claim = _mk_claim()
|
||||||
|
result = apply_999_rejections(
|
||||||
|
session, parsed,
|
||||||
|
claim_lookup=lambda scn: legacy_claim if scn == "991102994" else None,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.matched == ["claim-submitted"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_envelope_index_no_match_becomes_orphan():
|
||||||
|
session = MagicMock()
|
||||||
|
parsed = SimpleNamespace(
|
||||||
|
set_responses=[_mk_set_response(code="R", scn="991102999")]
|
||||||
|
)
|
||||||
|
result = apply_999_rejections(
|
||||||
|
session, parsed,
|
||||||
|
claim_lookup=lambda _pcn: None,
|
||||||
|
batch_envelope_index={}, # empty index
|
||||||
|
)
|
||||||
|
assert result.matched == []
|
||||||
|
assert "991102999" in result.orphans
|
||||||
|
|
||||||
|
|
||||||
|
def test_accepted_999_does_not_reject():
|
||||||
|
session = MagicMock()
|
||||||
|
parsed = SimpleNamespace(
|
||||||
|
set_responses=[_mk_set_response(code="A", scn="991102994")]
|
||||||
|
)
|
||||||
|
result = apply_999_rejections(
|
||||||
|
session, parsed,
|
||||||
|
claim_lookup=lambda _pcn: None,
|
||||||
|
batch_envelope_index={"991102994": ["claim-x"]},
|
||||||
|
)
|
||||||
|
assert result.matched == []
|
||||||
|
session.commit.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
def test_already_rejected_is_idempotent():
|
||||||
|
session = MagicMock()
|
||||||
|
already_rejected = _mk_claim(state="REJECTED")
|
||||||
|
session.query.return_value.filter.return_value.all.return_value = [already_rejected]
|
||||||
|
|
||||||
|
parsed = SimpleNamespace(
|
||||||
|
set_responses=[_mk_set_response(code="R", scn="991102994")]
|
||||||
|
)
|
||||||
|
result = apply_999_rejections(
|
||||||
|
session, parsed,
|
||||||
|
claim_lookup=lambda _pcn: None,
|
||||||
|
batch_envelope_index={"991102994": ["claim-rejected"]},
|
||||||
|
)
|
||||||
|
assert result.matched == []
|
||||||
|
# nothing was mutated on the already-rejected claim
|
||||||
|
assert already_rejected.state == "REJECTED"
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run both test files**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
docker exec cyclone-backend-1 bash -c "cd /var/lib/cyclone && .venv/bin/pytest tests/test_apply_999_rejections.py tests/test_co_medicaid_payer_id_emits_CO_TXIX.py -v"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: 6 passed (1 from Task 1 + 5 from Task 2).
|
||||||
|
|
||||||
|
- [ ] **Step 6: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add backend/src/cyclone/inbox_state.py backend/src/cyclone/handlers/handle_999.py backend/tests/test_apply_999_rejections.py
|
||||||
|
git commit -m "feat(sp33): apply_999_rejections uses batch_envelope_index"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task 3: CLI subcommand `backfill-999-rejections`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `backend/src/cyclone/cli.py` (add subcommand)
|
||||||
|
|
||||||
|
- [ ] **Step 1: Locate the cli.py entry point**
|
||||||
|
|
||||||
|
Run: `grep -n '@click.command\|def cli\|^@' backend/src/cyclone/cli.py | head -20`
|
||||||
|
|
||||||
|
Find an existing subcommand (e.g. `parse-837`) and copy its structure.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Add `backfill-999-rejections` subcommand**
|
||||||
|
|
||||||
|
Insert after an existing subcommand. The exact insertion point depends on the layout — pick a sensible spot. Use this implementation:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@click.command(name="backfill-999-rejections")
|
||||||
|
@click.option("--dry-run", is_flag=True, default=False,
|
||||||
|
help="Print the would-be changes without writing.")
|
||||||
|
@click.option("--actor", default="sp33-backfill",
|
||||||
|
help="Audit log actor for the claim.rejected events.")
|
||||||
|
def backfill_999_rejections(dry_run: bool, actor: str):
|
||||||
|
"""Replay existing 999 rejections onto already-linked claims.
|
||||||
|
|
||||||
|
SP33. Idempotent — claims already in REJECTED state are skipped.
|
||||||
|
|
||||||
|
Walks claim_acks JOIN acks WHERE set_accept_reject_code='R',
|
||||||
|
then sets claims.state='REJECTED' and fills payer_rejected_* fields.
|
||||||
|
"""
|
||||||
|
from sqlalchemy import select
|
||||||
|
from cyclone import db as cycl_db
|
||||||
|
from cyclone.audit_log import AuditEvent, append_event
|
||||||
|
from cyclone.models import Claim as ClaimModel, ClaimAck, Ack
|
||||||
|
|
||||||
|
cycl_db.init_db()
|
||||||
|
with cycl_db.SessionLocal()() as session:
|
||||||
|
rows = session.execute(
|
||||||
|
select(ClaimAck.claim_id, Claim.ack_code, ClaimAck.set_control_number,
|
||||||
|
ClaimAck.set_accept_reject_code, ClaimAck.ak2_index)
|
||||||
|
.join(Ack, Ack.id == ClaimAck.ack_id)
|
||||||
|
.join(ClaimModel, ClaimModel.id == ClaimAck.claim_id)
|
||||||
|
.where(ClaimAck.set_accept_reject_code == "R")
|
||||||
|
).all()
|
||||||
|
matched = already = errors = 0
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
for (claim_id, ack_code, scn, arc, ak2_idx) in rows:
|
||||||
|
claim = session.get(ClaimModel, claim_id)
|
||||||
|
if claim is None:
|
||||||
|
errors += 1
|
||||||
|
continue
|
||||||
|
if claim.state == "REJECTED":
|
||||||
|
already += 1
|
||||||
|
continue
|
||||||
|
if not dry_run:
|
||||||
|
claim.state = "REJECTED"
|
||||||
|
claim.state_changed_at = now
|
||||||
|
claim.rejected_at = now
|
||||||
|
claim.rejection_reason = f"999 AK5={ack_code or 'R'} SCN={scn} ak2={ak2_idx}"
|
||||||
|
claim.payer_rejected_at = now
|
||||||
|
claim.payer_rejected_reason = f"999 SET-level reject at SCN={scn}"
|
||||||
|
claim.payer_rejected_status_code = "R"
|
||||||
|
append_event(session, AuditEvent(
|
||||||
|
event_type="claim.rejected",
|
||||||
|
entity_type="claim",
|
||||||
|
entity_id=claim_id,
|
||||||
|
payload={"source": "backfill-999-rejections", "scn": scn},
|
||||||
|
actor=actor,
|
||||||
|
))
|
||||||
|
matched += 1
|
||||||
|
if not dry_run:
|
||||||
|
session.commit()
|
||||||
|
click.echo(
|
||||||
|
f"matched={matched} already_rejected={already} errors={errors} dry_run={dry_run}"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
The exact model class names (`Claim`, `ClaimAck`, `Ack`) and module names (`cyclone.models`, `cyclone.db`, `cyclone.audit_log`) may differ slightly — verify against the existing imports at the top of `cli.py` and adjust.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Sanity check (dry-run)**
|
||||||
|
|
||||||
|
Run: `docker exec cyclone-backend-1 bash -c "cd /var/lib/cyclone && .venv/bin/python -m cyclone.cli backfill-999-rejections --dry-run"`
|
||||||
|
Expected: `matched=338 already_rejected=0 errors=0 dry_run=True` (or similar; the exact number depends on how many unique claim_ids have R-coded links).
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run for real**
|
||||||
|
|
||||||
|
Run: `docker exec cyclone-backend-1 bash -c "cd /var/lib/cyclone && .venv/bin/python -m cyclone.cli backfill-999-rejections"`
|
||||||
|
Expected: `matched=338 already_rejected=0 errors=0 dry_run=False`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Verify claim states flipped in the DB**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
docker exec cyclone-backend-1 python3 -c "
|
||||||
|
import sqlite3
|
||||||
|
conn = sqlite3.connect('/var/lib/cyclone/db/cyclone.db')
|
||||||
|
cur = conn.cursor()
|
||||||
|
cur.execute('SELECT state, count(*) FROM claims GROUP BY state ORDER BY 2 DESC')
|
||||||
|
for row in cur.fetchall(): print(row)
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `('REJECTED', 341)` and `('SUBMITTED', 334)` (338 newly-rejected + the 3 pre-existing rejected = 341; original 672 minus 338 = 334).
|
||||||
|
|
||||||
|
- [ ] **Step 6: Commit (CLI only — backfill itself is a live op, no commit for that)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add backend/src/cyclone/cli.py
|
||||||
|
git commit -m "feat(sp33): cli backfill-999-rejections"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task 4: CLI subcommand `resubmit-rejected-claims`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `backend/src/cyclone/cli.py` (add subcommand)
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add `resubmit-rejected-claims` subcommand**
|
||||||
|
|
||||||
|
```python
|
||||||
|
@click.command(name="resubmit-rejected-claims")
|
||||||
|
@click.option("--payer", default="co_medicaid",
|
||||||
|
help="Payer factory key (default: co_medicaid).")
|
||||||
|
@click.option("--ingest-dir", default="/home/tyler/dev/cyclone/ingest",
|
||||||
|
help="Directory holding batch-*-claims/ subfolders (per Tyler's split).")
|
||||||
|
@click.option("--actor", default="sp33-resubmit",
|
||||||
|
help="Audit log actor for the clearhouse.submitted events.")
|
||||||
|
@click.option("--dry-run", is_flag=True, default=False,
|
||||||
|
help="Replace + validate but do not upload.")
|
||||||
|
def resubmit_rejected_claims(payer: str, ingest_dir: str, actor: str, dry_run: bool):
|
||||||
|
"""Byte-level SKCO0->CO_TXIX fix + SFTP resubmit of rejected claims.
|
||||||
|
|
||||||
|
SP33. For each claims row in REJECTED state, locate the matching single-
|
||||||
|
claim 837 file in ingest_dir/batch-<batch_id>-<N>-claims/*.x12, replace
|
||||||
|
PI*SKCO0 with PI*CO_TXIX in the bytes, validate via parse_837_text, then
|
||||||
|
upload via SftpClient. Idempotent per claim (resubmit_count gates re-run).
|
||||||
|
"""
|
||||||
|
from pathlib import Path
|
||||||
|
import glob
|
||||||
|
from cyclone import db as cycl_db
|
||||||
|
from cyclone.audit_log import AuditEvent, append_event
|
||||||
|
from cyclone.clearhouse import SftpClient
|
||||||
|
from cyclone.parsers.parse_837 import parse_837_text
|
||||||
|
|
||||||
|
cycl_db.init_db()
|
||||||
|
upload_root = "/CO XIX/PROD/coxix_prod_11525703/ToHPE"
|
||||||
|
|
||||||
|
# 1. collect rejected claims
|
||||||
|
with cycl_db.SessionLocal()() as session:
|
||||||
|
from sqlalchemy import select
|
||||||
|
from cyclone.models import Claim as ClaimModel
|
||||||
|
rows = session.execute(
|
||||||
|
select(ClaimModel.id, ClaimModel.batch_id, ClaimModel.patient_control_number,
|
||||||
|
ClaimModel.resubmit_count, ClaimModel.charge_amount)
|
||||||
|
.where(ClaimModel.state == "REJECTED")
|
||||||
|
.order_by(ClaimModel.batch_id, ClaimModel.id)
|
||||||
|
).all()
|
||||||
|
|
||||||
|
# 2. discover candidate files
|
||||||
|
files_by_batch = {}
|
||||||
|
for batch_dir in glob.glob(os.path.join(ingest_dir, "batch-*-*-claims")):
|
||||||
|
batch_id = os.path.basename(batch_dir).split("-")[1] # batch-<id>-<N>-claims
|
||||||
|
files_by_batch[batch_id] = sorted(glob.glob(os.path.join(batch_dir, "*.x12")))
|
||||||
|
|
||||||
|
ok = err = skip = 0
|
||||||
|
for (cid, bid, pcn, prev_resub, charge) in rows:
|
||||||
|
if prev_resub and prev_resub > 0:
|
||||||
|
skip += 1; continue
|
||||||
|
|
||||||
|
candidate_files = files_by_batch.get(bid, [])
|
||||||
|
# Find the file whose CLM01 == this claim's pcn
|
||||||
|
target_path = None
|
||||||
|
for fp in candidate_files:
|
||||||
|
with open(fp, "rb") as fh:
|
||||||
|
txt = fh.read().decode("ascii", errors="replace")
|
||||||
|
if pcn in txt:
|
||||||
|
target_path = fp
|
||||||
|
break
|
||||||
|
if target_path is None:
|
||||||
|
click.echo(f" no-file claim={cid} pcn={pcn}", err=True)
|
||||||
|
err += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Byte-level fix
|
||||||
|
with open(target_path, "rb") as fh:
|
||||||
|
raw = fh.read()
|
||||||
|
if b"PI*SKCO0" not in raw:
|
||||||
|
click.echo(f" no-replace claim={cid} file={os.path.basename(target_path)}", err=True)
|
||||||
|
err += 1; continue
|
||||||
|
fixed = raw.replace(b"PI*SKCO0", b"PI*CO_TXIX")
|
||||||
|
|
||||||
|
# Validate
|
||||||
|
try:
|
||||||
|
parsed = parse_837_text(fixed.decode("ascii"))
|
||||||
|
except Exception as exc:
|
||||||
|
click.echo(f" parse-fail claim={cid} {exc.__class__.__name__}: {exc}", err=True)
|
||||||
|
err += 1; continue
|
||||||
|
|
||||||
|
# Upload
|
||||||
|
remote_name = os.path.basename(target_path)
|
||||||
|
if not dry_run:
|
||||||
|
# Write the fixed bytes to a sibling tmp file for the upload
|
||||||
|
tmp_path = target_path + ".fixed"
|
||||||
|
with open(tmp_path, "wb") as fh:
|
||||||
|
fh.write(fixed)
|
||||||
|
try:
|
||||||
|
SftpClient().put(tmp_path, f"{upload_root}/{remote_name}")
|
||||||
|
finally:
|
||||||
|
os.unlink(tmp_path)
|
||||||
|
claim = session.get(ClaimModel, cid)
|
||||||
|
claim.resubmit_count = (claim.resubmit_count or 0) + 1
|
||||||
|
append_event(session, AuditEvent(
|
||||||
|
event_type="clearhouse.submitted",
|
||||||
|
entity_type="claim",
|
||||||
|
entity_id=cid,
|
||||||
|
payload={"file": remote_name, "scn": parsed.envelope.control_number},
|
||||||
|
actor=actor,
|
||||||
|
))
|
||||||
|
session.commit()
|
||||||
|
click.echo(f" ok claim={cid} file={remote_name}")
|
||||||
|
ok += 1
|
||||||
|
|
||||||
|
click.echo(f"\nDONE ok={ok} err={err} skip={skip} dry_run={dry_run}")
|
||||||
|
```
|
||||||
|
|
||||||
|
Imports needed: `import os` (glob, path). Confirm the `SftpClient` constructor — it may take an SFTP block or a config; check existing usage in `backend/src/cyclone/api.py` (the `/api/clearhouse/submit` endpoint) and mirror it.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run with --dry-run first**
|
||||||
|
|
||||||
|
Run: `docker exec cyclone-backend-1 bash -c "cd /var/lib/cyclone && CYCLONE_INGEST_DIR=/tmp/ingest_passthrough .venv/bin/python -m cyclone.cli resubmit-rejected-claims --dry-run"`
|
||||||
|
Expected: a line per claim plus a `DONE ok=338 err=0 skip=0 dry_run=True` summary.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run for real (upload)**
|
||||||
|
|
||||||
|
Mount /home/tyler/dev/cyclone/ingest into the container so the `--ingest-dir` resolves correctly. Easiest: copy the batch-* subdirs into the container:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec cyclone-backend-1 mkdir -p /tmp/ingest_split
|
||||||
|
tar -C /home/tyler/dev/cyclone/ingest -cf - 'batch-*-*-claims' | docker exec -i cyclone-backend-1 tar -xf - -C /tmp/ingest_split
|
||||||
|
```
|
||||||
|
|
||||||
|
Then: `docker exec cyclone-backend-1 bash -c "cd /var/lib/cyclone && .venv/bin/python -m cyclone.cli resubmit-rejected-claims --ingest-dir /tmp/ingest_split"`
|
||||||
|
|
||||||
|
Expected: `DONE ok=338 err=0 skip=0 dry_run=False`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify the upload landed**
|
||||||
|
|
||||||
|
Connect to SFTP and check the latest 338 files in `/ToHPE/`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec cyclone-backend-1 bash -c "
|
||||||
|
import paramiko, time
|
||||||
|
ssh = paramiko.SSHClient(); ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('mft.gainwelltechnologies.com', username='colorado-fts\\\\coxix_prod_11525703',
|
||||||
|
password=\$(cat /run/secrets/cyclone_sftp_password))
|
||||||
|
sftp = ssh.open_sftp()
|
||||||
|
files = sorted(sftp.listdir_attr('/CO XIX/PROD/coxix_prod_11525703/ToHPE'),
|
||||||
|
key=lambda a: a.st_mtime)[-10:]
|
||||||
|
for f in files: print(f.filename, time.ctime(f.st_mtime))
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: ~10 837P files dated 2026-07-02 (or now).
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add backend/src/cyclone/cli.py
|
||||||
|
git commit -m "feat(sp33): cli resubmit-rejected-claims"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task 5: Doc updates
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `docs/reference/837p.md:48`
|
||||||
|
- Modify: `docs/reference/co-medicaid.md:118`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update `docs/reference/837p.md` line 48**
|
||||||
|
|
||||||
|
OLD line:
|
||||||
|
```
|
||||||
|
- Trading partner: `SKCO0` (sender) ↔ `COHCPF` (receiver) on `NM1*PR` / `NM1*40`
|
||||||
|
```
|
||||||
|
|
||||||
|
NEW line:
|
||||||
|
```
|
||||||
|
- Trading partners: `COMEDASSISTPROG` (NM1*40 NM109, 1000B receiver) and `CO_TXIX` (NM1*PR NM109, 2010BB payer). See `docs/goodclaim.x12` for a canonical example and the HCPF 837P Companion Guide for the full segment table.
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Update `docs/reference/co-medicaid.md` line 118**
|
||||||
|
|
||||||
|
OLD:
|
||||||
|
```
|
||||||
|
- `NM1*PR N104 = "SKCO0"` (COHCPF)
|
||||||
|
```
|
||||||
|
|
||||||
|
NEW:
|
||||||
|
```
|
||||||
|
- `NM1*PR NM108 = "PI"`, `NM1*PR NM109 = "CO_TXIX"` (per HCPF 837P Companion Guide, June 2025 — Version 2.5). dzinesco submits against this code under dzinesco TPID `11525703`. The legacy trading-partner ID `SKCO0` is no longer accepted as the payer identifier.
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add docs/reference/837p.md docs/reference/co-medicaid.md
|
||||||
|
git commit -m "docs(sp33): update reference docs to CO_TXIX (per HCPF 837P Companion Guide)"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task 6: Verify + final test run
|
||||||
|
|
||||||
|
**Files:** none (verification only)
|
||||||
|
|
||||||
|
- [ ] **Step 1: Full test suite**
|
||||||
|
|
||||||
|
Run: `docker exec cyclone-backend-1 bash -c "cd /var/lib/cyclone && .venv/bin/pytest -x -q"`
|
||||||
|
Expected: all tests pass. The 4 affected test files (this plan + Spec §6 list) all green.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Smoke-test the live DB dashboard widget**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
curl -fsS http://localhost:8000/api/batches | python3 -m json.tool | head -30
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: the dashboard reflects the freshly-REJECTED batches (acceptedCount=0 for the 4 affected batches — this is now correct! They were rejected by Gainwell, and the cascade now shows that).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Atomic merge**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git checkout main
|
||||||
|
git merge --no-ff sp33-co-txix-payer-fix -m "merge: SP33 Co TXIX payer fix + 999 cascade repair into main"
|
||||||
|
git log --oneline -3
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: a single merge commit `merge: SP33 ...` on top of `dfd6542` (current main tip pre-SP33), with the SP33 branch's commits visible as parents.
|
||||||
|
|
||||||
|
## Acceptance checklist (must all be true before claiming "done")
|
||||||
|
|
||||||
|
- [ ] `cyclone -m cyclone.cli backfill-999-rejections` reports `matched=338 already_rejected=0 errors=0`
|
||||||
|
- [ ] `claims.state` count: REJECTED ≥ 341, SUBMITTED ≤ 334
|
||||||
|
- [ ] 338 new 837 files visible on SFTP `/CO XIX/PROD/cox17_prod_11525703/ToHPE/` with `CO_TXIX` in their bytes
|
||||||
|
- [ ] `claims.resubmit_count > 0` for the 338 claims
|
||||||
|
- [ ] `audit_log` shows 338 `claim.rejected` + 338 `clearhouse.submitted` events with actor=`sp33-*`
|
||||||
|
- [ ] Dashboard "Recent batches" widget for the 4 affected batches shows the same `acceptedCount` it was showing — but now it's accurate (0 rejected by Gainwell, not 0 accepted by Gainwell)
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
If anything breaks and SP33 needs to back out:
|
||||||
|
|
||||||
|
1. `git revert -m 1 <merge-sha>` on `main` — produces a fresh revert commit, preserves history.
|
||||||
|
2. The 2 cli subcommands revert cleanly (pure additions).
|
||||||
|
3. To un-reject the 338 claims after the revert, run a sister SQL: `UPDATE claims SET state='SUBMITTED', rejection_reason=NULL, rejected_at=NULL, payer_rejected_at=NULL, payer_rejected_reason=NULL, payer_rejected_status_code=NULL WHERE resubmit_count > 0;`
|
||||||
|
4. The 338 837 files already on SFTP /ToHPE/ cannot be unsent — Gainwell will pick them up regardless. Discretion required.
|
||||||
Reference in New Issue
Block a user