Nora
887 lines
34 KiB
Markdown
887 lines
34 KiB
Markdown
# SP31 — 835 Strict Content-Match Auto-Link Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Add a deterministic "any 2 of {PCN, charge, NPI}" content-keys fallback to the 835 auto-match path so more remits link to claims at parse time and the Recent Batches widget reflects settled state immediately.
|
|
|
|
**Architecture:** Keep the existing in-memory `reconcile.match()` PCN-exact path as the fast primary. Add a new DB-driven `reconcile._score_fallback_candidates()` that runs **after** `match()` on the leftover unmatched remits, queries a ±30-day candidate pool, applies the 2-of-3 rule, and on a unique hit returns a `Match(strategy="score-auto")` for the same orchestrator loop to consume. Rename existing `strategy="auto"` → `"pcn-exact"` for audit clarity.
|
|
|
|
**Tech Stack:** Python 3.11+, SQLAlchemy 2.x (sync session), pytest. No new dependencies. No schema migration. No frontend changes (widget already reflects state via SP30).
|
|
|
|
**Spec:** `docs/superpowers/specs/2026-07-02-cyclone-835-strict-content-match-design.md`
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
| File | Change | Responsibility |
|
|
|---|---|---|
|
|
| `backend/src/cyclone/reconcile.py` | Modify | Rename `strategy="auto"` → `"pcn-exact"`. Add `_content_keys_match` pure helper. Add `_score_fallback_candidates` DB helper. Integrate fallback into `run()`. |
|
|
| `backend/tests/test_reconcile.py` | Modify | Update 3 existing assertions from `"auto"` → `"pcn-exact"`. Add 11 new tests for the helpers + integration. |
|
|
| `src/types/index.ts` | Modify | Widen the `Match.strategy` union from `"auto" \| "manual"` to `"pcn-exact" \| "score-auto" \| "manual"` (2 sites). |
|
|
|
|
---
|
|
|
|
## Task 1: Rename `strategy="auto"` → `strategy="pcn-exact"` everywhere
|
|
|
|
**Files:**
|
|
- Modify: `backend/src/cyclone/reconcile.py:87`
|
|
- Modify: `backend/tests/test_reconcile.py:58, 206, 309`
|
|
- Modify: `backend/tests/test_db_models.py:323, 338, 364`
|
|
- Modify: `src/types/index.ts:458, 510`
|
|
|
|
- [ ] **Step 1: Update the production code**
|
|
|
|
In `backend/src/cyclone/reconcile.py`, line 87, change:
|
|
|
|
```python
|
|
strategy="auto",
|
|
```
|
|
|
|
to:
|
|
|
|
```python
|
|
strategy="pcn-exact",
|
|
```
|
|
|
|
- [ ] **Step 2: Update the backend test assertions**
|
|
|
|
In `backend/tests/test_reconcile.py`:
|
|
- Line 58: change `assert matches[0].strategy == "auto"` → `assert matches[0].strategy == "pcn-exact"`
|
|
- Line 206: change `strategy="auto"` → `strategy="pcn-exact"`
|
|
- Line 309: change `strategy="auto"` → `strategy="pcn-exact"`
|
|
|
|
In `backend/tests/test_db_models.py`:
|
|
- Line 323: change `strategy="auto"` → `strategy="pcn-exact"`
|
|
- Line 338: change `assert loaded.strategy == "auto"` → `assert loaded.strategy == "pcn-exact"`
|
|
- Line 364: change `strategy="auto"` → `strategy="pcn-exact"`
|
|
|
|
- [ ] **Step 3: Widen the frontend TypeScript union**
|
|
|
|
In `src/types/index.ts`, line 458, change:
|
|
|
|
```typescript
|
|
strategy: "auto" | "manual";
|
|
```
|
|
|
|
to:
|
|
|
|
```typescript
|
|
strategy: "pcn-exact" | "score-auto" | "manual";
|
|
```
|
|
|
|
Line 510, change:
|
|
|
|
```typescript
|
|
match: { id: number; strategy: "auto" | "manual" };
|
|
```
|
|
|
|
to:
|
|
|
|
```typescript
|
|
match: { id: number; strategy: "pcn-exact" | "score-auto" | "manual" };
|
|
```
|
|
|
|
- [ ] **Step 4: Run the test suite to confirm nothing broke**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest tests/test_reconcile.py tests/test_db_models.py -v
|
|
```
|
|
|
|
Expected: all green. The `"auto"` string is gone from production code and tests.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add backend/src/cyclone/reconcile.py backend/tests/test_reconcile.py backend/tests/test_db_models.py src/types/index.ts
|
|
git commit -m "refactor(sp31): rename Match strategy 'auto' to 'pcn-exact' for audit clarity"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 2: Add `_content_keys_match` pure helper (TDD)
|
|
|
|
**Files:**
|
|
- Modify: `backend/src/cyclone/reconcile.py` (append after `_pick_claim`)
|
|
- Modify: `backend/tests/test_reconcile.py` (append new test cases)
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
Add the following block at the end of `backend/tests/test_reconcile.py`:
|
|
|
|
```python
|
|
# --- SP31: content-keys fallback matcher (pure helper) -----------------------
|
|
|
|
|
|
def _make_remit_for_keys(remit_id: str, pcn: str, charge, npi: str):
|
|
"""Tiny shim with the three fields _content_keys_match reads."""
|
|
from cyclone.reconcile import _RemitLike # protocol not enforced at runtime
|
|
return type("R", (), {
|
|
"id": remit_id,
|
|
"payer_claim_control_number": pcn,
|
|
"total_charge_amount": charge,
|
|
"rendering_provider_npi": npi,
|
|
})()
|
|
|
|
|
|
def _make_claim_for_keys(claim_id: str, pcn: str, charge, npi: str):
|
|
return type("C", (), {
|
|
"id": claim_id,
|
|
"patient_control_number": pcn,
|
|
"total_charge": charge,
|
|
"rendering_provider_npi": npi,
|
|
})()
|
|
|
|
|
|
def test_content_keys_pcn_plus_charge_matches():
|
|
"""PCN + charge match (NPI mismatched) → True."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "")
|
|
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
|
assert _content_keys_match(r, c) is True
|
|
|
|
|
|
def test_content_keys_pcn_plus_npi_matches():
|
|
"""PCN + NPI match (charge mismatched) → True."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
|
c = _make_claim_for_keys("c1", "ABC", Decimal("200.00"), "1111111111")
|
|
assert _content_keys_match(r, c) is True
|
|
|
|
|
|
def test_content_keys_charge_plus_npi_matches():
|
|
"""Charge + NPI match (PCN mismatched) → True."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
|
c = _make_claim_for_keys("c1", "XYZ", Decimal("100.00"), "1111111111")
|
|
assert _content_keys_match(r, c) is True
|
|
|
|
|
|
def test_content_keys_all_three_match():
|
|
"""All 3 match → True."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
|
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
|
assert _content_keys_match(r, c) is True
|
|
|
|
|
|
def test_content_keys_only_pcn_does_not_match():
|
|
"""Only PCN matches (charge + NPI both wrong) → False."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
|
c = _make_claim_for_keys("c1", "ABC", Decimal("200.00"), "2222222222")
|
|
assert _content_keys_match(r, c) is False
|
|
|
|
|
|
def test_content_keys_only_charge_does_not_match():
|
|
"""Only charge matches (PCN + NPI both wrong) → False."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
|
c = _make_claim_for_keys("c1", "XYZ", Decimal("100.00"), "2222222222")
|
|
assert _content_keys_match(r, c) is False
|
|
|
|
|
|
def test_content_keys_none_match():
|
|
"""All 3 fields disagree → False."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "1111111111")
|
|
c = _make_claim_for_keys("c1", "XYZ", Decimal("200.00"), "2222222222")
|
|
assert _content_keys_match(r, c) is False
|
|
|
|
|
|
def test_content_keys_charge_tolerance_is_one_cent():
|
|
"""Charge differs by exactly $0.005 (rounding) → True ($0.01 tolerance)."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.005"), "1111111111")
|
|
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
|
assert _content_keys_match(r, c) is True
|
|
|
|
|
|
def test_content_keys_charge_differs_by_two_cents_no_match():
|
|
"""Charge differs by $0.02 → False (outside tolerance)."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.02"), "1111111111")
|
|
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
|
assert _content_keys_match(r, c) is False
|
|
|
|
|
|
def test_content_keys_empty_npi_does_not_count():
|
|
"""Empty NPI on remit counts as not-matched (falls back to PCN+charge rule)."""
|
|
from cyclone.reconcile import _content_keys_match
|
|
from decimal import Decimal
|
|
r = _make_remit_for_keys("r1", "ABC", Decimal("100.00"), "")
|
|
c = _make_claim_for_keys("c1", "ABC", Decimal("100.00"), "1111111111")
|
|
assert _content_keys_match(r, c) is True # PCN + charge still match
|
|
c2 = _make_claim_for_keys("c2", "XYZ", Decimal("100.00"), "1111111111")
|
|
assert _content_keys_match(r, c2) is False # only charge, no PCN
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they all FAIL with ImportError**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "content_keys" -v
|
|
```
|
|
|
|
Expected: 10 failures, all `ImportError: cannot import name '_content_keys_match'`.
|
|
|
|
- [ ] **Step 3: Implement `_content_keys_match` in `reconcile.py`**
|
|
|
|
In `backend/src/cyclone/reconcile.py`, add the following function after `_pick_claim` (around line 118):
|
|
|
|
```python
|
|
# --- SP31: content-keys fallback matcher -----------------------------------
|
|
|
|
CHARGE_TOLERANCE = Decimal("0.01")
|
|
KEYS_REQUIRED = 2
|
|
|
|
|
|
def _content_keys_match(remit, claim) -> bool:
|
|
"""Return True when at least KEYS_REQUIRED of {PCN, charge, NPI} agree.
|
|
|
|
Charge compared with ``CHARGE_TOLERANCE`` tolerance (rounding drift).
|
|
NPI counts as "not matched" when the remit's NPI is empty.
|
|
"""
|
|
keys_matched = 0
|
|
|
|
# PCN: payer_claim_control_number ↔ patient_control_number (stripped).
|
|
if (getattr(remit, "payer_claim_control_number", "") or "").strip() == \
|
|
(getattr(claim, "patient_control_number", "") or "").strip():
|
|
keys_matched += 1
|
|
|
|
# Charge: total_charge_amount ↔ total_charge (within $0.01).
|
|
remit_charge = getattr(remit, "total_charge_amount", None)
|
|
claim_charge = getattr(claim, "total_charge", None)
|
|
if remit_charge is not None and claim_charge is not None and \
|
|
abs(remit_charge - claim_charge) < CHARGE_TOLERANCE:
|
|
keys_matched += 1
|
|
|
|
# NPI: rendering_provider_npi ↔ rendering_provider_npi (must be non-empty).
|
|
remit_npi = (getattr(remit, "rendering_provider_npi", "") or "").strip()
|
|
claim_npi = (getattr(claim, "rendering_provider_npi", "") or "").strip()
|
|
if remit_npi and remit_npi == claim_npi:
|
|
keys_matched += 1
|
|
|
|
return keys_matched >= KEYS_REQUIRED
|
|
```
|
|
|
|
- [ ] **Step 4: Run the tests to verify they all PASS**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "content_keys" -v
|
|
```
|
|
|
|
Expected: 10 passed.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add backend/src/cyclone/reconcile.py backend/tests/test_reconcile.py
|
|
git commit -m "feat(sp31): add _content_keys_match pure helper (2-of-3 PCN/charge/NPI rule)"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 3: Add `_score_fallback_candidates` DB helper (TDD)
|
|
|
|
**Files:**
|
|
- Modify: `backend/src/cyclone/reconcile.py` (append after `_content_keys_match`)
|
|
- Modify: `backend/tests/test_reconcile.py` (append integration tests)
|
|
|
|
- [ ] **Step 1: Write the failing integration tests**
|
|
|
|
Add to the end of `backend/tests/test_reconcile.py`:
|
|
|
|
```python
|
|
# --- SP31: content-keys fallback matcher (DB helper) ------------------------
|
|
|
|
|
|
def test_score_fallback_unique_match_returns_claim_id(db_session, make_claim, make_remit):
|
|
"""One claim in pool matches 2-of-3 → returns that claim."""
|
|
from decimal import Decimal
|
|
from datetime import date, timedelta
|
|
from cyclone.reconcile import _score_fallback_candidates
|
|
c1 = make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=date(2026, 6, 1))
|
|
c2 = make_claim(patient_control_number="XYZ", total_charge=Decimal("999.99"),
|
|
rendering_provider_npi="2222222222",
|
|
service_date_from=date(2026, 6, 1))
|
|
r = make_remit(payer_claim_control_number="ABC",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=date(2026, 6, 5)) # ±30-day window
|
|
assert _score_fallback_candidates(db_session, r) == c1.id
|
|
|
|
|
|
def test_score_fallback_zero_matches_returns_none(db_session, make_claim, make_remit):
|
|
"""No claim in pool matches 2-of-3 → returns None."""
|
|
from decimal import Decimal
|
|
from datetime import date
|
|
from cyclone.reconcile import _score_fallback_candidates
|
|
make_claim(patient_control_number="XYZ", total_charge=Decimal("999.99"),
|
|
rendering_provider_npi="2222222222",
|
|
service_date_from=date(2026, 6, 1))
|
|
r = make_remit(payer_claim_control_number="ABC",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=date(2026, 6, 5))
|
|
assert _score_fallback_candidates(db_session, r) is None
|
|
|
|
|
|
def test_score_fallback_two_matches_returns_none(db_session, make_claim, make_remit):
|
|
"""Two claims in pool both match 2-of-3 → returns None (ambiguous)."""
|
|
from decimal import Decimal
|
|
from datetime import date
|
|
from cyclone.reconcile import _score_fallback_candidates
|
|
# Both claims have same PCN and same charge — both would match PCN+charge.
|
|
make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=date(2026, 6, 1))
|
|
make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=date(2026, 6, 2))
|
|
r = make_remit(payer_claim_control_number="ABC",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=date(2026, 6, 5))
|
|
assert _score_fallback_candidates(db_session, r) is None
|
|
|
|
|
|
def test_score_fallback_skips_already_matched(db_session, make_claim, make_remit):
|
|
"""Claim with matched_remittance_id set → excluded from pool."""
|
|
from decimal import Decimal
|
|
from datetime import date
|
|
from cyclone.reconcile import _score_fallback_candidates
|
|
c1 = make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=date(2026, 6, 1),
|
|
matched_remittance_id="some-other-remit-id")
|
|
r = make_remit(payer_claim_control_number="ABC",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=date(2026, 6, 5))
|
|
assert _score_fallback_candidates(db_session, r) is None # only candidate was excluded
|
|
|
|
|
|
def test_score_fallback_skips_terminal_state(db_session, make_claim, make_remit):
|
|
"""Claim in PAID/DENIED/REJECTED/REVERSED/RECONCILED → excluded from pool."""
|
|
from decimal import Decimal
|
|
from datetime import date
|
|
from cyclone.db import ClaimState
|
|
from cyclone.reconcile import _score_fallback_candidates
|
|
make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=date(2026, 6, 1),
|
|
state=ClaimState.PAID)
|
|
r = make_remit(payer_claim_control_number="ABC",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=date(2026, 6, 5))
|
|
assert _score_fallback_candidates(db_session, r) is None
|
|
|
|
|
|
def test_score_fallback_widened_window_catches_late_remit(db_session, make_claim, make_remit):
|
|
"""Remit 25 days after claim → still in pool (30-day window)."""
|
|
from decimal import Decimal
|
|
from datetime import date, timedelta
|
|
from cyclone.reconcile import _score_fallback_candidates
|
|
claim_date = date(2026, 6, 1)
|
|
remit_date = claim_date + timedelta(days=25)
|
|
c1 = make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=claim_date)
|
|
r = make_remit(payer_claim_control_number="ABC",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=remit_date)
|
|
assert _score_fallback_candidates(db_session, r) == c1.id
|
|
|
|
|
|
def test_score_fallback_outside_window_excluded(db_session, make_claim, make_remit):
|
|
"""Remit 60 days after claim → excluded (outside 30-day window)."""
|
|
from decimal import Decimal
|
|
from datetime import date, timedelta
|
|
from cyclone.reconcile import _score_fallback_candidates
|
|
claim_date = date(2026, 6, 1)
|
|
remit_date = claim_date + timedelta(days=60)
|
|
make_claim(patient_control_number="ABC", total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=claim_date)
|
|
r = make_remit(payer_claim_control_number="ABC",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=remit_date)
|
|
assert _score_fallback_candidates(db_session, r) is None
|
|
```
|
|
|
|
- [ ] **Step 2: Add the `make_claim` / `make_remit` fixtures to `conftest.py`**
|
|
|
|
Open `backend/tests/conftest.py` and add (or extend) the fixtures. If they already exist with these names, adjust the existing fixtures instead:
|
|
|
|
```python
|
|
import pytest
|
|
from decimal import Decimal
|
|
from datetime import date
|
|
from cyclone import db
|
|
from cyclone.db import Claim, Remittance, ClaimState
|
|
|
|
|
|
@pytest.fixture
|
|
def make_claim(db_session):
|
|
def _make(patient_control_number: str, total_charge: Decimal,
|
|
rendering_provider_npi: str, service_date_from: date,
|
|
matched_remittance_id: str | None = None,
|
|
state: ClaimState = ClaimState.SUBMITTED,
|
|
claim_id: str | None = None):
|
|
claim_id = claim_id or f"clm-{patient_control_number}-{service_date_from.isoformat()}"
|
|
c = Claim(
|
|
id=claim_id,
|
|
patient_control_number=patient_control_number,
|
|
total_charge=total_charge,
|
|
rendering_provider_npi=rendering_provider_npi,
|
|
service_date_from=service_date_from,
|
|
matched_remittance_id=matched_remittance_id,
|
|
state=state,
|
|
)
|
|
db_session.add(c)
|
|
db_session.flush()
|
|
return c
|
|
return _make
|
|
|
|
|
|
@pytest.fixture
|
|
def make_remit(db_session):
|
|
def _make(payer_claim_control_number: str, total_charge_amount: Decimal,
|
|
rendering_provider_npi: str, service_date: date,
|
|
remit_id: str | None = None):
|
|
remit_id = remit_id or f"remit-{payer_claim_control_number}-{service_date.isoformat()}"
|
|
r = Remittance(
|
|
id=remit_id,
|
|
payer_claim_control_number=payer_claim_control_number,
|
|
total_charge_amount=total_charge_amount,
|
|
rendering_provider_npi=rendering_provider_npi,
|
|
service_date=service_date,
|
|
)
|
|
db_session.add(r)
|
|
db_session.flush()
|
|
return r
|
|
return _make
|
|
```
|
|
|
|
If `Claim` / `Remittance` constructors don't accept these exact keyword args, check `backend/src/cyclone/db.py` for the actual field names and adjust. The plan assumes the ORM models expose these fields; the existing `reconcile.match()` already reads `matched_remittance_id`, `patient_control_number`, `service_date_from` from `Claim` so they definitely exist.
|
|
|
|
- [ ] **Step 3: Run the tests to verify they all FAIL with ImportError**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "score_fallback" -v
|
|
```
|
|
|
|
Expected: 7 failures, all `ImportError: cannot import name '_score_fallback_candidates'`.
|
|
|
|
- [ ] **Step 4: Implement `_score_fallback_candidates` in `reconcile.py`**
|
|
|
|
In `backend/src/cyclone/reconcile.py`, add the following function after `_content_keys_match`:
|
|
|
|
```python
|
|
# --- SP31: DB-side fallback candidate matcher -------------------------------
|
|
|
|
SCORE_FALLBACK_WINDOW_DAYS = 30
|
|
|
|
|
|
def _score_fallback_candidates(session, remit) -> Optional[str]:
|
|
"""Return a unique matching claim_id via the 2-of-3 content-keys rule.
|
|
|
|
Queries a ±SCORE_FALLBACK_WINDOW_DAYS candidate pool filtered by:
|
|
- claim.matched_remittance_id IS NULL (not yet linked)
|
|
- remit.claim_id IS NULL on the matched pair (enforced by caller;
|
|
this helper assumes the caller has already filtered on remit's
|
|
own claim_id)
|
|
- claim.state NOT IN terminal set (PAID, DENIED, REJECTED, REVERSED, RECONCILED)
|
|
- claim.service_date_from within window of remit.service_date
|
|
- claim.payer_id == remit.payer_id (same payer)
|
|
|
|
Returns the unique claim.id when exactly one candidate passes the
|
|
2-of-3 rule, otherwise None. Ambiguous matches (2+ candidates) also
|
|
return None — they land in the Inbox Unlinked lane for manual review.
|
|
"""
|
|
from sqlalchemy import select, and_, or_
|
|
from datetime import timedelta
|
|
from cyclone.db import Claim, Remittance, ClaimState
|
|
|
|
TERMINAL = {
|
|
ClaimState.PAID, ClaimState.DENIED, ClaimState.REJECTED,
|
|
ClaimState.REVERSED, ClaimState.RECONCILED,
|
|
}
|
|
|
|
if remit.service_date is None:
|
|
return None # need a date for the window query
|
|
|
|
window_lo = remit.service_date - timedelta(days=SCORE_FALLBACK_WINDOW_DAYS)
|
|
window_hi = remit.service_date + timedelta(days=SCORE_FALLBACK_WINDOW_DAYS)
|
|
|
|
candidates = list(session.execute(
|
|
select(Claim).where(
|
|
and_(
|
|
Claim.matched_remittance_id.is_(None),
|
|
Claim.service_date_from >= window_lo,
|
|
Claim.service_date_from <= window_hi,
|
|
Claim.state.notin_([s.value for s in TERMINAL]),
|
|
Claim.payer_id == remit.payer_id,
|
|
)
|
|
)
|
|
).scalars().all())
|
|
|
|
matched_ids = [c.id for c in candidates if _content_keys_match(remit, c)]
|
|
if len(matched_ids) == 1:
|
|
return matched_ids[0]
|
|
return None # 0 matches OR 2+ (ambiguous)
|
|
```
|
|
|
|
If `Claim.payer_id` or `Remittance.payer_id` aren't the actual column names, check `db.py` and adjust the filter. The plan assumes these exist (the SP30 widget already groups claims by payer).
|
|
|
|
- [ ] **Step 5: Run the tests to verify they all PASS**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "score_fallback" -v
|
|
```
|
|
|
|
Expected: 7 passed.
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add backend/src/cyclone/reconcile.py backend/tests/test_reconcile.py backend/tests/conftest.py
|
|
git commit -m "feat(sp31): add _score_fallback_candidates DB helper with ±30-day candidate pool"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 4: Integrate content-keys fallback into `reconcile.run()` (TDD)
|
|
|
|
**Files:**
|
|
- Modify: `backend/src/cyclone/reconcile.py` (`run` function)
|
|
- Modify: `backend/tests/test_reconcile.py` (append end-to-end tests)
|
|
|
|
- [ ] **Step 1: Write the failing end-to-end tests**
|
|
|
|
Add to the end of `backend/tests/test_reconcile.py`:
|
|
|
|
```python
|
|
# --- SP31: end-to-end integration via reconcile.run() -----------------------
|
|
|
|
|
|
def test_reconcile_run_emits_score_auto_match_when_pcn_misses(db_session, make_claim, make_remit):
|
|
"""A remit with no PCN match but matching charge+NPI triggers score-auto."""
|
|
from decimal import Decimal
|
|
from datetime import date
|
|
from cyclone import db as db_module
|
|
from cyclone.db import Batch, ClaimState
|
|
from cyclone.reconcile import run as reconcile_run
|
|
claim = make_claim(patient_control_number="UNIQUE-CLM",
|
|
total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=date(2026, 6, 1))
|
|
remit = make_remit(payer_claim_control_number="DIFFERENT-PCN",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=date(2026, 6, 5))
|
|
batch = Batch(id="batch-1", kind="835", input_filename="x.835", parsed_at=date(2026, 6, 5))
|
|
db_session.add(batch)
|
|
db_session.flush()
|
|
# Re-parent the remit to this batch (make_remit doesn't set batch_id).
|
|
remit.batch_id = batch.id
|
|
db_session.flush()
|
|
|
|
result = reconcile_run(db_session, batch.id)
|
|
|
|
assert result.matched == 1
|
|
# Verify Match row was written with strategy="score-auto"
|
|
from cyclone.db import Match
|
|
matches = list(db_session.execute(
|
|
db_module.select(Match).where(Match.remittance_id == remit.id)
|
|
).scalars().all())
|
|
assert len(matches) == 1
|
|
assert matches[0].strategy == "score-auto"
|
|
assert matches[0].claim_id == claim.id
|
|
# Verify ActivityEvent was emitted with kind="auto_matched_835"
|
|
from cyclone.db import ActivityEvent
|
|
events = list(db_session.execute(
|
|
db_module.select(ActivityEvent).where(
|
|
ActivityEvent.remittance_id == remit.id,
|
|
ActivityEvent.kind == "auto_matched_835",
|
|
)
|
|
).scalars().all())
|
|
assert len(events) == 1
|
|
assert events[0].claim_id == claim.id
|
|
# Verify claim state was flipped
|
|
db_session.refresh(claim)
|
|
assert claim.state in {ClaimState.PAID, ClaimState.PARTIAL, ClaimState.RECEIVED}
|
|
|
|
|
|
def test_reconcile_run_pcn_path_unchanged(db_session, make_claim, make_remit):
|
|
"""Existing PCN-exact match still produces strategy='pcn-exact' (regression guard)."""
|
|
from decimal import Decimal
|
|
from datetime import date
|
|
from cyclone import db as db_module
|
|
from cyclone.db import Batch, Match
|
|
from cyclone.reconcile import run as reconcile_run
|
|
claim = make_claim(patient_control_number="SAME-PCN",
|
|
total_charge=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date_from=date(2026, 6, 1))
|
|
remit = make_remit(payer_claim_control_number="SAME-PCN",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=date(2026, 6, 2))
|
|
batch = Batch(id="batch-1", kind="835", input_filename="x.835", parsed_at=date(2026, 6, 5))
|
|
db_session.add(batch)
|
|
db_session.flush()
|
|
remit.batch_id = batch.id
|
|
db_session.flush()
|
|
|
|
reconcile_run(db_session, batch.id)
|
|
|
|
matches = list(db_session.execute(
|
|
db_module.select(Match).where(Match.remittance_id == remit.id)
|
|
).scalars().all())
|
|
assert len(matches) == 1
|
|
assert matches[0].strategy == "pcn-exact" # NOT "score-auto"
|
|
|
|
|
|
def test_reconcile_run_no_match_leaves_remit_unlinked(db_session, make_claim, make_remit):
|
|
"""No 2-of-3 match → no Match row, claim state unchanged."""
|
|
from decimal import Decimal
|
|
from datetime import date
|
|
from cyclone import db as db_module
|
|
from cyclone.db import Batch, Match, ClaimState
|
|
from cyclone.reconcile import run as reconcile_run
|
|
claim = make_claim(patient_control_number="UNRELATED",
|
|
total_charge=Decimal("999.99"),
|
|
rendering_provider_npi="2222222222",
|
|
service_date_from=date(2026, 6, 1))
|
|
remit = make_remit(payer_claim_control_number="OTHER",
|
|
total_charge_amount=Decimal("100.00"),
|
|
rendering_provider_npi="1111111111",
|
|
service_date=date(2026, 6, 5))
|
|
batch = Batch(id="batch-1", kind="835", input_filename="x.835", parsed_at=date(2026, 6, 5))
|
|
db_session.add(batch)
|
|
db_session.flush()
|
|
remit.batch_id = batch.id
|
|
db_session.flush()
|
|
|
|
result = reconcile_run(db_session, batch.id)
|
|
|
|
assert result.matched == 0
|
|
matches = list(db_session.execute(
|
|
db_module.select(Match).where(Match.remittance_id == remit.id)
|
|
).scalars().all())
|
|
assert len(matches) == 0
|
|
db_session.refresh(claim)
|
|
assert claim.state == ClaimState.SUBMITTED # unchanged
|
|
db_session.refresh(remit)
|
|
assert remit.claim_id is None # unchanged
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they FAIL (run() not yet integrated)**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "reconcile_run_" -v
|
|
```
|
|
|
|
Expected: failures. The behavior should be wrong because `run()` doesn't yet call `_score_fallback_candidates`.
|
|
|
|
- [ ] **Step 3: Integrate the fallback into `reconcile.run()`**
|
|
|
|
In `backend/src/cyclone/reconcile.py`, modify the `run` function (around line 240-260) to add the fallback after the in-memory `match()` call. Replace the block:
|
|
|
|
```python
|
|
matches = match(unmatched_claims, new_remits)
|
|
```
|
|
|
|
with:
|
|
|
|
```python
|
|
matches = match(unmatched_claims, new_remits)
|
|
|
|
# SP31: content-keys fallback for remits that PCN-exact couldn't pair.
|
|
# Operates on the still-unmatched remits so we don't double-match.
|
|
matched_remit_ids = {m.remittance.id for m in matches}
|
|
used_claim_ids = {m.claim.id for m in matches}
|
|
for remit in new_remits:
|
|
if remit.id in matched_remit_ids:
|
|
continue
|
|
if remit.claim_id is not None:
|
|
continue # already linked
|
|
matched_claim_id = _score_fallback_candidates(session, remit)
|
|
if matched_claim_id is None:
|
|
continue
|
|
# Find the claim object in the unmatched_claims list (it was loaded).
|
|
target_claim = next(
|
|
(c for c in unmatched_claims if c.id == matched_claim_id),
|
|
None,
|
|
)
|
|
if target_claim is None or target_claim.id in used_claim_ids:
|
|
continue
|
|
matches.append(Match(
|
|
claim=target_claim, remittance=remit,
|
|
strategy="score-auto", is_reversal=remit.is_reversal,
|
|
))
|
|
used_claim_ids.add(target_claim.id)
|
|
```
|
|
|
|
Then, in the existing apply loop further down (around line 270), change the ActivityEvent emission so score-auto matches emit `kind="auto_matched_835"` instead of the default `"reconcile"`. Find:
|
|
|
|
```python
|
|
session.add(ActivityEvent(
|
|
ts=datetime.now(timezone.utc), kind=intent.activity_kind,
|
|
batch_id=batch_id, claim_id=m.claim.id,
|
|
remittance_id=m.remittance.id,
|
|
payload_json={"new_state": m.claim.state.value},
|
|
))
|
|
```
|
|
|
|
and replace the `kind=intent.activity_kind` with:
|
|
|
|
```python
|
|
kind=("auto_matched_835" if m.strategy == "score-auto" else intent.activity_kind),
|
|
```
|
|
|
|
And update the payload to include the score-auto metadata for the operator. Change the `payload_json=` line to:
|
|
|
|
```python
|
|
payload_json=(
|
|
{"new_state": m.claim.state.value, "strategy": m.strategy}
|
|
if m.strategy == "score-auto"
|
|
else {"new_state": m.claim.state.value}
|
|
),
|
|
```
|
|
|
|
- [ ] **Step 4: Run the new end-to-end tests to verify they PASS**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest tests/test_reconcile.py -k "reconcile_run_" -v
|
|
```
|
|
|
|
Expected: 3 passed.
|
|
|
|
- [ ] **Step 5: Run the full reconcile test suite to verify no regression**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest tests/test_reconcile.py tests/test_db_models.py -v
|
|
```
|
|
|
|
Expected: all green (existing tests + 20 new tests).
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add backend/src/cyclone/reconcile.py backend/tests/test_reconcile.py
|
|
git commit -m "feat(sp31): integrate content-keys fallback into reconcile.run() with auto_matched_835 event"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 5: Verify end-to-end (full test suite + manual smoke)
|
|
|
|
**Files:**
|
|
- No new files. Verification only.
|
|
|
|
- [ ] **Step 1: Run the full backend test suite**
|
|
|
|
```bash
|
|
cd backend && .venv/bin/pytest -q
|
|
```
|
|
|
|
Expected: 0 failures. If any test fails, fix it before proceeding.
|
|
|
|
- [ ] **Step 2: Run the frontend typecheck**
|
|
|
|
```bash
|
|
npm run typecheck
|
|
```
|
|
|
|
Expected: 0 errors. The widened `strategy` union in `src/types/index.ts` should compile cleanly.
|
|
|
|
- [ ] **Step 3: Run the frontend lint**
|
|
|
|
```bash
|
|
npm run lint
|
|
```
|
|
|
|
Expected: 0 errors.
|
|
|
|
- [ ] **Step 4: Manual smoke test via the running prod stack**
|
|
|
|
The 837+835 fixtures already exist (`backend/tests/fixtures/co_medicaid_837p.txt` and `backend/tests/fixtures/co_medicaid_835.txt`). Confirm by reading the envelopes:
|
|
|
|
```bash
|
|
grep -E "^ISA|^GS|^ST|^BPR|^TRN|^CLP" backend/tests/fixtures/co_medicaid_835.txt | head -10
|
|
```
|
|
|
|
Then send a parse-835 request via the existing `/api/parse-835` endpoint or the CLI:
|
|
|
|
```bash
|
|
python -m cyclone.cli parse-835 backend/tests/fixtures/co_medicaid_835.txt --output-dir /tmp/sp31-smoke
|
|
```
|
|
|
|
Expected: a Remittance row is written + a Match row is created. Check via SQL:
|
|
|
|
```bash
|
|
docker compose exec -T backend python3 <<'PY'
|
|
from cyclone import db
|
|
db.init_db()
|
|
from sqlalchemy import select
|
|
with db.SessionLocal()() as s:
|
|
matches = list(s.execute(
|
|
select(db.Match).order_by(db.Match.matched_at.desc()).limit(5)
|
|
).scalars().all())
|
|
for m in matches:
|
|
print(f" {m.strategy:12} claim={m.claim_id[:24]:24} remit={m.remittance_id[:24]:24} reversal={m.is_reversal}")
|
|
PY
|
|
```
|
|
|
|
Expected: at least one row with `strategy=pcn-exact` (the co_medicaid_835 fixture has PCN matching the co_medicaid_837p fixture) or `strategy=score-auto` (if you intentionally mangle the PCN in a copy of the fixture to trigger fallback).
|
|
|
|
- [ ] **Step 5: Commit any incidental cleanup**
|
|
|
|
If step 1-4 surfaced any unrelated failures, fix them on this branch. Otherwise no commit.
|
|
|
|
```bash
|
|
git status
|
|
```
|
|
|
|
If clean, proceed to merge.
|
|
|
|
---
|
|
|
|
## Self-Review
|
|
|
|
**1. Spec coverage:**
|
|
- §1 D1 (strict 2-of-3 rule) → Task 2 ✅
|
|
- §1 D2 (PCN-first, fallback) → Task 4 ✅
|
|
- §1 D3 (Match row strategy label + payload) → Task 4 ✅
|
|
- §1 (idempotency guards) → Task 3 ✅
|
|
- §1 (reversal handling unchanged) → Task 4 (no changes to apply_reversal) ✅
|
|
- §2 D3 (Match.strategy labels) → Task 1 (rename) + Task 4 (new label) ✅
|
|
- §2 D4 ($0.01 tolerance) → Task 2 ✅
|
|
- §2 D5 (±30-day window) → Task 3 ✅
|
|
- §2 D6 (3 idempotency guards) → Task 3 (3 tests) ✅
|
|
- §2 D7 (no new UI) → no tasks (intentional) ✅
|
|
- §2 D8 (`auto_matched_835` ActivityEvent) → Task 4 ✅
|
|
- §2 D9 (no auth changes) → no tasks (intentional) ✅
|
|
- §3 edge cases → covered across Tasks 2, 3, 4 ✅
|
|
- §4 testing approach → 20 new tests across 4 test files ✅
|
|
- §5 out-of-scope reminders → no backfill, no 999 changes, etc. (none touched) ✅
|
|
|
|
**2. Placeholder scan:** No "TBD", "TODO", "implement later". The conftest fixture instructions include "If `Claim.payer_id` doesn't exist, check `db.py`" — this is intentional contingency, not a placeholder; the actual implementation step checks the file.
|
|
|
|
**3. Type consistency:** All references to `_content_keys_match`, `_score_fallback_candidates`, `Match(strategy="score-auto")`, `kind="auto_matched_835"`, `CHARGE_TOLERANCE`, `KEYS_REQUIRED`, `SCORE_FALLBACK_WINDOW_DAYS`, and the `make_claim` / `make_remit` fixture signatures are consistent across all tasks. The `RemitLike` / `ClaimLike` Protocol usage in Task 2 matches the existing patterns in `reconcile.py`. |