docs(plan): SP31 835 strict content-match auto-link
This commit is contained in:
Nora
@@ -0,0 +1,887 @@
|
||||
# SP31 — 835 Strict Content-Match Auto-Link Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a deterministic "any 2 of {PCN, charge, NPI}" content-keys fallback to the 835 auto-match path so more remits link to claims at parse time and the Recent Batches widget reflects settled state immediately.
|
||||
|
||||
**Architecture:** Keep the existing in-memory `reconcile.match()` PCN-exact path as the fast primary. Add a new DB-driven `reconcile._score_fallback_candidates()` that runs **after** `match()` on the leftover unmatched remits, queries a ±30-day candidate pool, applies the 2-of-3 rule, and on a unique hit returns a `Match(strategy="score-auto")` for the same orchestrator loop to consume. Rename existing `strategy="auto"` → `"pcn-exact"` for audit clarity.
|
||||
|
||||
**Tech Stack:** Python 3.11+, SQLAlchemy 2.x (sync session), pytest. No new dependencies. No schema migration. No frontend changes (widget already reflects state via SP30).
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-07-02-cyclone-835-strict-content-match-design.md`
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
| File | Change | Responsibility |
|
||||
|---|---|---|
|
||||
| `backend/src/cyclone/reconcile.py` | Modify | Rename `strategy="auto"` → `"pcn-exact"`. Add `_content_keys_match` pure helper. Add `_score_fallback_candidates` DB helper. Integrate fallback into `run()`. |
|
||||
| `backend/tests/test_reconcile.py` | Modify | Update 3 existing assertions from `"auto"` → `"pcn-exact"`. Add 11 new tests for the helpers + integration. |
|
||||
| `src/types/index.ts` | Modify | Widen the `Match.strategy` union from `"auto" \| "manual"` to `"pcn-exact" \| "score-auto" \| "manual"` (2 sites). |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Rename `strategy="auto"` → `strategy="pcn-exact"` everywhere
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/cyclone/reconcile.py:87`
|
||||
- Modify: `backend/tests/test_reconcile.py:58, 206, 309`
|
||||
- Modify: `backend/tests/test_db_models.py:323, 338, 364`
|
||||
- Modify: `src/types/index.ts:458, 510`
|
||||
|
||||
- [ ] **Step 1: Update the production code**
|
||||
|
||||
In `backend/src/cyclone/reconcile.py`, line 87, change:
|
||||
|
||||
```python
|
||||
strategy="auto",
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
strategy="pcn-exact",
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update the backend test assertions**
|
||||
|
||||
In `backend/tests/test_reconcile.py`:
|
||||
- Line 58: change `assert matches[0].strategy == "auto"` → `assert matches[0].strategy == "pcn-exact"`
|
||||
- Line 206: change `strategy="auto"` → `strategy="pcn-exact"`
|
||||
- Line 309: change `strategy="auto"` → `strategy="pcn-exact"`
|
||||
|
||||
In `backend/tests/test_db_models.py`:
|
||||
- Line 323: change `strategy="auto"` → `strategy="pcn-exact"`
|
||||
- Line 338: change `assert loaded.strategy == "auto"` → `assert loaded.strategy == "pcn-exact"`
|
||||
- Line 364: change `strategy="auto"` → `strategy="pcn-exact"`
|
||||
|
||||
- [ ] **Step 3: Widen the frontend TypeScript union**
|
||||
|
||||
In `src/types/index.ts`, line 458, change:
|
||||
|
||||
```typescript
|
||||
strategy: "auto" | "manual";
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```typescript
|
||||
strategy: "pcn-exact" | "score-auto" | "manual";
|
||||
```
|
||||
|
||||
Line 510, change:
|
||||
|
||||
```typescript
|
||||
match: { id: number; strategy: "auto" | "manual" };
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```typescript
|
||||
match: { id: number; strategy: "pcn-exact" | "score-auto" | "manual" };
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the test suite to confirm nothing broke**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest tests/test_reconcile.py tests/test_db_models.py -v
|
||||
```
|
||||
|
||||
Expected: all green. The `"auto"` string is gone from production code and tests.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add backend/src/cyclone/reconcile.py backend/tests/test_reconcile.py backend/tests/test_db_models.py src/types/index.ts
|
||||
git commit -m "refactor(sp31): rename Match strategy 'auto' to 'pcn-exact' for audit clarity"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add `_content_keys_match` pure helper (TDD)
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/cyclone/reconcile.py` (append after `_pick_claim`)
|
||||
- Modify: `backend/tests/test_reconcile.py` (append new test cases)
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
Add the following block at the end of `backend/tests/test_reconcile.py`:
|
||||
|
||||
```python
|
||||
# --- SP31: content-keys fallback matcher (pure helper) -----------------------
|
||||
|
||||
|
||||
def _make_remit_for_keys(remit_id: str, pcn: str, charge, npi: str):
|
||||
"""Tiny shim with the three fields _content_keys_match reads."""
|
||||
from cyclone.reconcile import _RemitLike # protocol not enforced at runtime
|
||||
return type("R", (), {
|
||||
"id": remit_id,
|
||||
"payer_claim_control_number": pcn,
|
||||
"total_charge_amount": charge,
|
||||
"rendering_provider_npi": npi,
|
||||
})()
|
||||
|
||||
|
||||
def _make_claim_for_keys(claim_id: str, pcn: str, charge, npi: str):
|
||||
return type("C", (), {
|
||||
"id": claim_id,
|
||||
"patient_control_number": pcn,
|
||||
"total_charge": charge,
|
||||
"rendering_provider_npi": npi,
|
||||
})()
|
||||
|
||||
|
||||
def test_content_keys_pcn_plus_charge_matches():
|
||||
"""PCN + charge match (NPI mismatched) → True."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "")
|
||||
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
||||
assert _content_keys_match(r, c) is True
|
||||
|
||||
|
||||
def test_content_keys_pcn_plus_npi_matches():
|
||||
"""PCN + NPI match (charge mismatched) → True."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
||||
c = _make_claim_for_keys("c1", "ABC", Decimal("200.00"), "1111111111")
|
||||
assert _content_keys_match(r, c) is True
|
||||
|
||||
|
||||
def test_content_keys_charge_plus_npi_matches():
|
||||
"""Charge + NPI match (PCN mismatched) → True."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
||||
c = _make_claim_for_keys("c1", "XYZ", Decimal("100.00"), "1111111111")
|
||||
assert _content_keys_match(r, c) is True
|
||||
|
||||
|
||||
def test_content_keys_all_three_match():
|
||||
"""All 3 match → True."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
||||
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
||||
assert _content_keys_match(r, c) is True
|
||||
|
||||
|
||||
def test_content_keys_only_pcn_does_not_match():
|
||||
"""Only PCN matches (charge + NPI both wrong) → False."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
||||
c = _make_claim_for_keys("c1", "ABC", Decimal("200.00"), "2222222222")
|
||||
assert _content_keys_match(r, c) is False
|
||||
|
||||
|
||||
def test_content_keys_only_charge_does_not_match():
|
||||
"""Only charge matches (PCN + NPI both wrong) → False."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
||||
c = _make_claim_for_keys("c1", "XYZ", Decimal("100.00"), "2222222222")
|
||||
assert _content_keys_match(r, c) is False
|
||||
|
||||
|
||||
def test_content_keys_none_match():
|
||||
"""All 3 fields disagree → False."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
||||
c = _make_claim_for_keys("c1", "XYZ", Decimal("200.00"), "2222222222")
|
||||
assert _content_keys_match(r, c) is False
|
||||
|
||||
|
||||
def test_content_keys_charge_tolerance_is_one_cent():
|
||||
"""Charge differs by exactly $0.005 (rounding) → True ($0.01 tolerance)."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.005"), "1111111111")
|
||||
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
||||
assert _content_keys_match(r, c) is True
|
||||
|
||||
|
||||
def test_content_keys_charge_differs_by_two_cents_no_match():
|
||||
"""Charge differs by $0.02 → False (outside tolerance)."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.02"), "1111111111")
|
||||
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
||||
assert _content_keys_match(r, c) is False
|
||||
|
||||
|
||||
def test_content_keys_empty_npi_does_not_count():
|
||||
"""Empty NPI on remit counts as not-matched (falls back to PCN+charge rule)."""
|
||||
from cyclone.reconcile import _content_keys_match
|
||||
from decimal import Decimal
|
||||
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "")
|
||||
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
||||
assert _content_keys_match(r, c) is True # PCN + charge still match
|
||||
c2 = _make_claim_for_keys("c2", "XYZ", Decimal("100.00"), "1111111111")
|
||||
assert _content_keys_match(r, c2) is False # only charge, no PCN
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests to verify they all FAIL with ImportError**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "content_keys" -v
|
||||
```
|
||||
|
||||
Expected: 10 failures, all `ImportError: cannot import name '_content_keys_match'`.
|
||||
|
||||
- [ ] **Step 3: Implement `_content_keys_match` in `reconcile.py`**
|
||||
|
||||
In `backend/src/cyclone/reconcile.py`, add the following function after `_pick_claim` (around line 118):
|
||||
|
||||
```python
|
||||
# --- SP31: content-keys fallback matcher -----------------------------------
|
||||
|
||||
CHARGE_TOLERANCE = Decimal("0.01")
|
||||
KEYS_REQUIRED = 2
|
||||
|
||||
|
||||
def _content_keys_match(remit, claim) -> bool:
|
||||
"""Return True when at least KEYS_REQUIRED of {PCN, charge, NPI} agree.
|
||||
|
||||
Charge compared with ``CHARGE_TOLERANCE`` tolerance (rounding drift).
|
||||
NPI counts as "not matched" when the remit's NPI is empty.
|
||||
"""
|
||||
keys_matched = 0
|
||||
|
||||
# PCN: payer_claim_control_number ↔ patient_control_number (stripped).
|
||||
if (getattr(remit, "payer_claim_control_number", "") or "").strip() == \
|
||||
(getattr(claim, "patient_control_number", "") or "").strip():
|
||||
keys_matched += 1
|
||||
|
||||
# Charge: total_charge_amount ↔ total_charge (within $0.01).
|
||||
remit_charge = getattr(remit, "total_charge_amount", None)
|
||||
claim_charge = getattr(claim, "total_charge", None)
|
||||
if remit_charge is not None and claim_charge is not None and \
|
||||
abs(remit_charge - claim_charge) < CHARGE_TOLERANCE:
|
||||
keys_matched += 1
|
||||
|
||||
# NPI: rendering_provider_npi ↔ rendering_provider_npi (must be non-empty).
|
||||
remit_npi = (getattr(remit, "rendering_provider_npi", "") or "").strip()
|
||||
claim_npi = (getattr(claim, "rendering_provider_npi", "") or "").strip()
|
||||
if remit_npi and remit_npi == claim_npi:
|
||||
keys_matched += 1
|
||||
|
||||
return keys_matched >= KEYS_REQUIRED
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the tests to verify they all PASS**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "content_keys" -v
|
||||
```
|
||||
|
||||
Expected: 10 passed.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add backend/src/cyclone/reconcile.py backend/tests/test_reconcile.py
|
||||
git commit -m "feat(sp31): add _content_keys_match pure helper (2-of-3 PCN/charge/NPI rule)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add `_score_fallback_candidates` DB helper (TDD)
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/cyclone/reconcile.py` (append after `_content_keys_match`)
|
||||
- Modify: `backend/tests/test_reconcile.py` (append integration tests)
|
||||
|
||||
- [ ] **Step 1: Write the failing integration tests**
|
||||
|
||||
Add to the end of `backend/tests/test_reconcile.py`:
|
||||
|
||||
```python
|
||||
# --- SP31: content-keys fallback matcher (DB helper) ------------------------
|
||||
|
||||
|
||||
def test_score_fallback_unique_match_returns_claim_id(db_session, make_claim, make_remit):
|
||||
"""One claim in pool matches 2-of-3 → returns that claim."""
|
||||
from decimal import Decimal
|
||||
from datetime import date, timedelta
|
||||
from cyclone.reconcile import _score_fallback_candidates
|
||||
c1 = make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=date(2026, 6, 1))
|
||||
c2 = make_claim(patient_control_number="XYZ", total_charge=Decimal("999.99"),
|
||||
rendering_provider_npi="2222222222",
|
||||
service_date_from=date(2026, 6, 1))
|
||||
r = make_remit(payer_claim_control_number="ABC",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=date(2026, 6, 5)) # ±30-day window
|
||||
assert _score_fallback_candidates(db_session, r) == c1.id
|
||||
|
||||
|
||||
def test_score_fallback_zero_matches_returns_none(db_session, make_claim, make_remit):
|
||||
"""No claim in pool matches 2-of-3 → returns None."""
|
||||
from decimal import Decimal
|
||||
from datetime import date
|
||||
from cyclone.reconcile import _score_fallback_candidates
|
||||
make_claim(patient_control_number="XYZ", total_charge=Decimal("999.99"),
|
||||
rendering_provider_npi="2222222222",
|
||||
service_date_from=date(2026, 6, 1))
|
||||
r = make_remit(payer_claim_control_number="ABC",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=date(2026, 6, 5))
|
||||
assert _score_fallback_candidates(db_session, r) is None
|
||||
|
||||
|
||||
def test_score_fallback_two_matches_returns_none(db_session, make_claim, make_remit):
|
||||
"""Two claims in pool both match 2-of-3 → returns None (ambiguous)."""
|
||||
from decimal import Decimal
|
||||
from datetime import date
|
||||
from cyclone.reconcile import _score_fallback_candidates
|
||||
# Both claims have same PCN and same charge — both would match PCN+charge.
|
||||
make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=date(2026, 6, 1))
|
||||
make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=date(2026, 6, 2))
|
||||
r = make_remit(payer_claim_control_number="ABC",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=date(2026, 6, 5))
|
||||
assert _score_fallback_candidates(db_session, r) is None
|
||||
|
||||
|
||||
def test_score_fallback_skips_already_matched(db_session, make_claim, make_remit):
|
||||
"""Claim with matched_remittance_id set → excluded from pool."""
|
||||
from decimal import Decimal
|
||||
from datetime import date
|
||||
from cyclone.reconcile import _score_fallback_candidates
|
||||
c1 = make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=date(2026, 6, 1),
|
||||
matched_remittance_id="some-other-remit-id")
|
||||
r = make_remit(payer_claim_control_number="ABC",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=date(2026, 6, 5))
|
||||
assert _score_fallback_candidates(db_session, r) is None # only candidate was excluded
|
||||
|
||||
|
||||
def test_score_fallback_skips_terminal_state(db_session, make_claim, make_remit):
|
||||
"""Claim in PAID/DENIED/REJECTED/REVERSED/RECONCILED → excluded from pool."""
|
||||
from decimal import Decimal
|
||||
from datetime import date
|
||||
from cyclone.db import ClaimState
|
||||
from cyclone.reconcile import _score_fallback_candidates
|
||||
make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=date(2026, 6, 1),
|
||||
state=ClaimState.PAID)
|
||||
r = make_remit(payer_claim_control_number="ABC",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=date(2026, 6, 5))
|
||||
assert _score_fallback_candidates(db_session, r) is None
|
||||
|
||||
|
||||
def test_score_fallback_widened_window_catches_late_remit(db_session, make_claim, make_remit):
|
||||
"""Remit 25 days after claim → still in pool (30-day window)."""
|
||||
from decimal import Decimal
|
||||
from datetime import date, timedelta
|
||||
from cyclone.reconcile import _score_fallback_candidates
|
||||
claim_date = date(2026, 6, 1)
|
||||
remit_date = claim_date + timedelta(days=25)
|
||||
c1 = make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=claim_date)
|
||||
r = make_remit(payer_claim_control_number="ABC",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=remit_date)
|
||||
assert _score_fallback_candidates(db_session, r) == c1.id
|
||||
|
||||
|
||||
def test_score_fallback_outside_window_excluded(db_session, make_claim, make_remit):
|
||||
"""Remit 60 days after claim → excluded (outside 30-day window)."""
|
||||
from decimal import Decimal
|
||||
from datetime import date, timedelta
|
||||
from cyclone.reconcile import _score_fallback_candidates
|
||||
claim_date = date(2026, 6, 1)
|
||||
remit_date = claim_date + timedelta(days=60)
|
||||
make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=claim_date)
|
||||
r = make_remit(payer_claim_control_number="ABC",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=remit_date)
|
||||
assert _score_fallback_candidates(db_session, r) is None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add the `make_claim` / `make_remit` fixtures to `conftest.py`**
|
||||
|
||||
Open `backend/tests/conftest.py` and add (or extend) the fixtures. If they already exist with these names, adjust the existing fixtures instead:
|
||||
|
||||
```python
|
||||
import pytest
|
||||
from decimal import Decimal
|
||||
from datetime import date
|
||||
from cyclone import db
|
||||
from cyclone.db import Claim, Remittance, ClaimState
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def make_claim(db_session):
|
||||
def _make(patient_control_number: str, total_charge: Decimal,
|
||||
rendering_provider_npi: str, service_date_from: date,
|
||||
matched_remittance_id: str | None = None,
|
||||
state: ClaimState = ClaimState.SUBMITTED,
|
||||
claim_id: str | None = None):
|
||||
claim_id = claim_id or f"clm-{patient_control_number}-{service_date_from.isoformat()}"
|
||||
c = Claim(
|
||||
id=claim_id,
|
||||
patient_control_number=patient_control_number,
|
||||
total_charge=total_charge,
|
||||
rendering_provider_npi=rendering_provider_npi,
|
||||
service_date_from=service_date_from,
|
||||
matched_remittance_id=matched_remittance_id,
|
||||
state=state,
|
||||
)
|
||||
db_session.add(c)
|
||||
db_session.flush()
|
||||
return c
|
||||
return _make
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def make_remit(db_session):
|
||||
def _make(payer_claim_control_number: str, total_charge_amount: Decimal,
|
||||
rendering_provider_npi: str, service_date: date,
|
||||
remit_id: str | None = None):
|
||||
remit_id = remit_id or f"remit-{payer_claim_control_number}-{service_date.isoformat()}"
|
||||
r = Remittance(
|
||||
id=remit_id,
|
||||
payer_claim_control_number=payer_claim_control_number,
|
||||
total_charge_amount=total_charge_amount,
|
||||
rendering_provider_npi=rendering_provider_npi,
|
||||
service_date=service_date,
|
||||
)
|
||||
db_session.add(r)
|
||||
db_session.flush()
|
||||
return r
|
||||
return _make
|
||||
```
|
||||
|
||||
If `Claim` / `Remittance` constructors don't accept these exact keyword args, check `backend/src/cyclone/db.py` for the actual field names and adjust. The plan assumes the ORM models expose these fields; the existing `reconcile.match()` already reads `matched_remittance_id`, `patient_control_number`, `service_date_from` from `Claim` so they definitely exist.
|
||||
|
||||
- [ ] **Step 3: Run the tests to verify they all FAIL with ImportError**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "score_fallback" -v
|
||||
```
|
||||
|
||||
Expected: 7 failures, all `ImportError: cannot import name '_score_fallback_candidates'`.
|
||||
|
||||
- [ ] **Step 4: Implement `_score_fallback_candidates` in `reconcile.py`**
|
||||
|
||||
In `backend/src/cyclone/reconcile.py`, add the following function after `_content_keys_match`:
|
||||
|
||||
```python
|
||||
# --- SP31: DB-side fallback candidate matcher -------------------------------
|
||||
|
||||
SCORE_FALLBACK_WINDOW_DAYS = 30
|
||||
|
||||
|
||||
def _score_fallback_candidates(session, remit) -> Optional[str]:
|
||||
"""Return a unique matching claim_id via the 2-of-3 content-keys rule.
|
||||
|
||||
Queries a ±SCORE_FALLBACK_WINDOW_DAYS candidate pool filtered by:
|
||||
- claim.matched_remittance_id IS NULL (not yet linked)
|
||||
- remit.claim_id IS NULL on the matched pair (enforced by caller;
|
||||
this helper assumes the caller has already filtered on remit's
|
||||
own claim_id)
|
||||
- claim.state NOT IN terminal set (PAID, DENIED, REJECTED, REVERSED, RECONCILED)
|
||||
- claim.service_date_from within window of remit.service_date
|
||||
- claim.payer_id == remit.payer_id (same payer)
|
||||
|
||||
Returns the unique claim.id when exactly one candidate passes the
|
||||
2-of-3 rule, otherwise None. Ambiguous matches (2+ candidates) also
|
||||
return None — they land in the Inbox Unlinked lane for manual review.
|
||||
"""
|
||||
from sqlalchemy import select, and_, or_
|
||||
from datetime import timedelta
|
||||
from cyclone.db import Claim, Remittance, ClaimState
|
||||
|
||||
TERMINAL = {
|
||||
ClaimState.PAID, ClaimState.DENIED, ClaimState.REJECTED,
|
||||
ClaimState.REVERSED, ClaimState.RECONCILED,
|
||||
}
|
||||
|
||||
if remit.service_date is None:
|
||||
return None # need a date for the window query
|
||||
|
||||
window_lo = remit.service_date - timedelta(days=SCORE_FALLBACK_WINDOW_DAYS)
|
||||
window_hi = remit.service_date + timedelta(days=SCORE_FALLBACK_WINDOW_DAYS)
|
||||
|
||||
candidates = list(session.execute(
|
||||
select(Claim).where(
|
||||
and_(
|
||||
Claim.matched_remittance_id.is_(None),
|
||||
Claim.service_date_from >= window_lo,
|
||||
Claim.service_date_from <= window_hi,
|
||||
Claim.state.notin_([s.value for s in TERMINAL]),
|
||||
Claim.payer_id == remit.payer_id,
|
||||
)
|
||||
)
|
||||
).scalars().all())
|
||||
|
||||
matched_ids = [c.id for c in candidates if _content_keys_match(remit, c)]
|
||||
if len(matched_ids) == 1:
|
||||
return matched_ids[0]
|
||||
return None # 0 matches OR 2+ (ambiguous)
|
||||
```
|
||||
|
||||
If `Claim.payer_id` or `Remittance.payer_id` aren't the actual column names, check `db.py` and adjust the filter. The plan assumes these exist (the SP30 widget already groups claims by payer).
|
||||
|
||||
- [ ] **Step 5: Run the tests to verify they all PASS**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "score_fallback" -v
|
||||
```
|
||||
|
||||
Expected: 7 passed.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add backend/src/cyclone/reconcile.py backend/tests/test_reconcile.py backend/tests/conftest.py
|
||||
git commit -m "feat(sp31): add _score_fallback_candidates DB helper with ±30-day candidate pool"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Integrate content-keys fallback into `reconcile.run()` (TDD)
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/cyclone/reconcile.py` (`run` function)
|
||||
- Modify: `backend/tests/test_reconcile.py` (append end-to-end tests)
|
||||
|
||||
- [ ] **Step 1: Write the failing end-to-end tests**
|
||||
|
||||
Add to the end of `backend/tests/test_reconcile.py`:
|
||||
|
||||
```python
|
||||
# --- SP31: end-to-end integration via reconcile.run() -----------------------
|
||||
|
||||
|
||||
def test_reconcile_run_emits_score_auto_match_when_pcn_misses(db_session, make_claim, make_remit):
|
||||
"""A remit with no PCN match but matching charge+NPI triggers score-auto."""
|
||||
from decimal import Decimal
|
||||
from datetime import date
|
||||
from cyclone import db as db_module
|
||||
from cyclone.db import Batch, ClaimState
|
||||
from cyclone.reconcile import run as reconcile_run
|
||||
claim = make_claim(patient_control_number="UNIQUE-CLM",
|
||||
total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=date(2026, 6, 1))
|
||||
remit = make_remit(payer_claim_control_number="DIFFERENT-PCN",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=date(2026, 6, 5))
|
||||
batch = Batch(id="batch-1", kind="835", input_filename="x.835", parsed_at=date(2026, 6, 5))
|
||||
db_session.add(batch)
|
||||
db_session.flush()
|
||||
# Re-parent the remit to this batch (make_remit doesn't set batch_id).
|
||||
remit.batch_id = batch.id
|
||||
db_session.flush()
|
||||
|
||||
result = reconcile_run(db_session, batch.id)
|
||||
|
||||
assert result.matched == 1
|
||||
# Verify Match row was written with strategy="score-auto"
|
||||
from cyclone.db import Match
|
||||
matches = list(db_session.execute(
|
||||
db_module.select(Match).where(Match.remittance_id == remit.id)
|
||||
).scalars().all())
|
||||
assert len(matches) == 1
|
||||
assert matches[0].strategy == "score-auto"
|
||||
assert matches[0].claim_id == claim.id
|
||||
# Verify ActivityEvent was emitted with kind="auto_matched_835"
|
||||
from cyclone.db import ActivityEvent
|
||||
events = list(db_session.execute(
|
||||
db_module.select(ActivityEvent).where(
|
||||
ActivityEvent.remittance_id == remit.id,
|
||||
ActivityEvent.kind == "auto_matched_835",
|
||||
)
|
||||
).scalars().all())
|
||||
assert len(events) == 1
|
||||
assert events[0].claim_id == claim.id
|
||||
# Verify claim state was flipped
|
||||
db_session.refresh(claim)
|
||||
assert claim.state in {ClaimState.PAID, ClaimState.PARTIAL, ClaimState.RECEIVED}
|
||||
|
||||
|
||||
def test_reconcile_run_pcn_path_unchanged(db_session, make_claim, make_remit):
|
||||
"""Existing PCN-exact match still produces strategy='pcn-exact' (regression guard)."""
|
||||
from decimal import Decimal
|
||||
from datetime import date
|
||||
from cyclone import db as db_module
|
||||
from cyclone.db import Batch, Match
|
||||
from cyclone.reconcile import run as reconcile_run
|
||||
claim = make_claim(patient_control_number="SAME-PCN",
|
||||
total_charge=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date_from=date(2026, 6, 1))
|
||||
remit = make_remit(payer_claim_control_number="SAME-PCN",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=date(2026, 6, 2))
|
||||
batch = Batch(id="batch-1", kind="835", input_filename="x.835", parsed_at=date(2026, 6, 5))
|
||||
db_session.add(batch)
|
||||
db_session.flush()
|
||||
remit.batch_id = batch.id
|
||||
db_session.flush()
|
||||
|
||||
reconcile_run(db_session, batch.id)
|
||||
|
||||
matches = list(db_session.execute(
|
||||
db_module.select(Match).where(Match.remittance_id == remit.id)
|
||||
).scalars().all())
|
||||
assert len(matches) == 1
|
||||
assert matches[0].strategy == "pcn-exact" # NOT "score-auto"
|
||||
|
||||
|
||||
def test_reconcile_run_no_match_leaves_remit_unlinked(db_session, make_claim, make_remit):
|
||||
"""No 2-of-3 match → no Match row, claim state unchanged."""
|
||||
from decimal import Decimal
|
||||
from datetime import date
|
||||
from cyclone import db as db_module
|
||||
from cyclone.db import Batch, Match, ClaimState
|
||||
from cyclone.reconcile import run as reconcile_run
|
||||
claim = make_claim(patient_control_number="UNRELATED",
|
||||
total_charge=Decimal("999.99"),
|
||||
rendering_provider_npi="2222222222",
|
||||
service_date_from=date(2026, 6, 1))
|
||||
remit = make_remit(payer_claim_control_number="OTHER",
|
||||
total_charge_amount=Decimal("100.00"),
|
||||
rendering_provider_npi="1111111111",
|
||||
service_date=date(2026, 6, 5))
|
||||
batch = Batch(id="batch-1", kind="835", input_filename="x.835", parsed_at=date(2026, 6, 5))
|
||||
db_session.add(batch)
|
||||
db_session.flush()
|
||||
remit.batch_id = batch.id
|
||||
db_session.flush()
|
||||
|
||||
result = reconcile_run(db_session, batch.id)
|
||||
|
||||
assert result.matched == 0
|
||||
matches = list(db_session.execute(
|
||||
db_module.select(Match).where(Match.remittance_id == remit.id)
|
||||
).scalars().all())
|
||||
assert len(matches) == 0
|
||||
db_session.refresh(claim)
|
||||
assert claim.state == ClaimState.SUBMITTED # unchanged
|
||||
db_session.refresh(remit)
|
||||
assert remit.claim_id is None # unchanged
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests to verify they FAIL (run() not yet integrated)**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "reconcile_run_" -v
|
||||
```
|
||||
|
||||
Expected: failures. The behavior should be wrong because `run()` doesn't yet call `_score_fallback_candidates`.
|
||||
|
||||
- [ ] **Step 3: Integrate the fallback into `reconcile.run()`**
|
||||
|
||||
In `backend/src/cyclone/reconcile.py`, modify the `run` function (around line 240-260) to add the fallback after the in-memory `match()` call. Replace the block:
|
||||
|
||||
```python
|
||||
matches = match(unmatched_claims, new_remits)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
matches = match(unmatched_claims, new_remits)
|
||||
|
||||
# SP31: content-keys fallback for remits that PCN-exact couldn't pair.
|
||||
# Operates on the still-unmatched remits so we don't double-match.
|
||||
matched_remit_ids = {m.remittance.id for m in matches}
|
||||
used_claim_ids = {m.claim.id for m in matches}
|
||||
for remit in new_remits:
|
||||
if remit.id in matched_remit_ids:
|
||||
continue
|
||||
if remit.claim_id is not None:
|
||||
continue # already linked
|
||||
matched_claim_id = _score_fallback_candidates(session, remit)
|
||||
if matched_claim_id is None:
|
||||
continue
|
||||
# Find the claim object in the unmatched_claims list (it was loaded).
|
||||
target_claim = next(
|
||||
(c for c in unmatched_claims if c.id == matched_claim_id),
|
||||
None,
|
||||
)
|
||||
if target_claim is None or target_claim.id in used_claim_ids:
|
||||
continue
|
||||
matches.append(Match(
|
||||
claim=target_claim, remittance=remit,
|
||||
strategy="score-auto", is_reversal=remit.is_reversal,
|
||||
))
|
||||
used_claim_ids.add(target_claim.id)
|
||||
```
|
||||
|
||||
Then, in the existing apply loop further down (around line 270), change the ActivityEvent emission so score-auto matches emit `kind="auto_matched_835"` instead of the default `"reconcile"`. Find:
|
||||
|
||||
```python
|
||||
session.add(ActivityEvent(
|
||||
ts=datetime.now(timezone.utc), kind=intent.activity_kind,
|
||||
batch_id=batch_id, claim_id=m.claim.id,
|
||||
remittance_id=m.remittance.id,
|
||||
payload_json={"new_state": m.claim.state.value},
|
||||
))
|
||||
```
|
||||
|
||||
and replace the `kind=intent.activity_kind` with:
|
||||
|
||||
```python
|
||||
kind=("auto_matched_835" if m.strategy == "score-auto" else intent.activity_kind),
|
||||
```
|
||||
|
||||
And update the payload to include the score-auto metadata for the operator. Change the `payload_json=` line to:
|
||||
|
||||
```python
|
||||
payload_json=(
|
||||
{"new_state": m.claim.state.value, "strategy": m.strategy}
|
||||
if m.strategy == "score-auto"
|
||||
else {"new_state": m.claim.state.value}
|
||||
),
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the new end-to-end tests to verify they PASS**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "reconcile_run_" -v
|
||||
```
|
||||
|
||||
Expected: 3 passed.
|
||||
|
||||
- [ ] **Step 5: Run the full reconcile test suite to verify no regression**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest tests/test_reconcile.py tests/test_db_models.py -v
|
||||
```
|
||||
|
||||
Expected: all green (existing tests + 20 new tests).
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add backend/src/cyclone/reconcile.py backend/tests/test_reconcile.py
|
||||
git commit -m "feat(sp31): integrate content-keys fallback into reconcile.run() with auto_matched_835 event"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Verify end-to-end (full test suite + manual smoke)
|
||||
|
||||
**Files:**
|
||||
- No new files. Verification only.
|
||||
|
||||
- [ ] **Step 1: Run the full backend test suite**
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest -q
|
||||
```
|
||||
|
||||
Expected: 0 failures. If any test fails, fix it before proceeding.
|
||||
|
||||
- [ ] **Step 2: Run the frontend typecheck**
|
||||
|
||||
```bash
|
||||
npm run typecheck
|
||||
```
|
||||
|
||||
Expected: 0 errors. The widened `strategy` union in `src/types/index.ts` should compile cleanly.
|
||||
|
||||
- [ ] **Step 3: Run the frontend lint**
|
||||
|
||||
```bash
|
||||
npm run lint
|
||||
```
|
||||
|
||||
Expected: 0 errors.
|
||||
|
||||
- [ ] **Step 4: Manual smoke test via the running prod stack**
|
||||
|
||||
The 837+835 fixtures already exist (`backend/tests/fixtures/co_medicaid_837p.txt` and `backend/tests/fixtures/co_medicaid_835.txt`). Confirm by reading the envelopes:
|
||||
|
||||
```bash
|
||||
grep -E "^ISA|^GS|^ST|^BPR|^TRN|^CLP" backend/tests/fixtures/co_medicaid_835.txt | head -10
|
||||
```
|
||||
|
||||
Then send a parse-835 request via the existing `/api/parse-835` endpoint or the CLI:
|
||||
|
||||
```bash
|
||||
python -m cyclone.cli parse-835 backend/tests/fixtures/co_medicaid_835.txt --output-dir /tmp/sp31-smoke
|
||||
```
|
||||
|
||||
Expected: a Remittance row is written + a Match row is created. Check via SQL:
|
||||
|
||||
```bash
|
||||
docker compose exec -T backend python3 <<'PY'
|
||||
from cyclone import db
|
||||
db.init_db()
|
||||
from sqlalchemy import select
|
||||
with db.SessionLocal()() as s:
|
||||
matches = list(s.execute(
|
||||
select(db.Match).order_by(db.Match.matched_at.desc()).limit(5)
|
||||
).scalars().all())
|
||||
for m in matches:
|
||||
print(f" {m.strategy:12} claim={m.claim_id[:24]:24} remit={m.remittance_id[:24]:24} reversal={m.is_reversal}")
|
||||
PY
|
||||
```
|
||||
|
||||
Expected: at least one row with `strategy=pcn-exact` (the co_medicaid_835 fixture has PCN matching the co_medicaid_837p fixture) or `strategy=score-auto` (if you intentionally mangle the PCN in a copy of the fixture to trigger fallback).
|
||||
|
||||
- [ ] **Step 5: Commit any incidental cleanup**
|
||||
|
||||
If step 1-4 surfaced any unrelated failures, fix them on this branch. Otherwise no commit.
|
||||
|
||||
```bash
|
||||
git status
|
||||
```
|
||||
|
||||
If clean, proceed to merge.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
**1. Spec coverage:**
|
||||
- §1 D1 (strict 2-of-3 rule) → Task 2 ✅
|
||||
- §1 D2 (PCN-first, fallback) → Task 4 ✅
|
||||
- §1 D3 (Match row strategy label + payload) → Task 4 ✅
|
||||
- §1 (idempotency guards) → Task 3 ✅
|
||||
- §1 (reversal handling unchanged) → Task 4 (no changes to apply_reversal) ✅
|
||||
- §2 D3 (Match.strategy labels) → Task 1 (rename) + Task 4 (new label) ✅
|
||||
- §2 D4 ($0.01 tolerance) → Task 2 ✅
|
||||
- §2 D5 (±30-day window) → Task 3 ✅
|
||||
- §2 D6 (3 idempotency guards) → Task 3 (3 tests) ✅
|
||||
- §2 D7 (no new UI) → no tasks (intentional) ✅
|
||||
- §2 D8 (`auto_matched_835` ActivityEvent) → Task 4 ✅
|
||||
- §2 D9 (no auth changes) → no tasks (intentional) ✅
|
||||
- §3 edge cases → covered across Tasks 2, 3, 4 ✅
|
||||
- §4 testing approach → 20 new tests across 4 test files ✅
|
||||
- §5 out-of-scope reminders → no backfill, no 999 changes, etc. (none touched) ✅
|
||||
|
||||
**2. Placeholder scan:** No "TBD", "TODO", "implement later". The conftest fixture instructions include "If `Claim.payer_id` doesn't exist, check `db.py`" — this is intentional contingency, not a placeholder; the actual implementation step checks the file.
|
||||
|
||||
**3. Type consistency:** All references to `_content_keys_match`, `_score_fallback_candidates`, `Match(strategy="score-auto")`, `kind="auto_matched_835"`, `CHARGE_TOLERANCE`, `KEYS_REQUIRED`, `SCORE_FALLBACK_WINDOW_DAYS`, and the `make_claim` / `make_remit` fixture signatures are consistent across all tasks. The `RemitLike` / `ClaimLike` Protocol usage in Task 2 matches the existing patterns in `reconcile.py`.
|
||||
Reference in New Issue
Block a user