fix(sp27): Claim.patient_control_number populated from CLM01 (claim_id), not 2010BA NM109 (member_id)

The 837 ingest path (_claim_837_row in store.py:194) populated
Claim.patient_control_number from claim.subscriber.member_id — the
subscriber's 2010BA NM109 Medicaid ID — instead of from
claim.claim_id (CLM01, the claim submitter's identifier the 837
actually sent).

That silently broke every downstream join that uses this column as a
cross-reference key:

  * reconcile.match() (reconcile.py:74) — joins
    Claim.patient_control_number against
    Remittance.payer_claim_control_number (which is parsed from CLP01,
    the 835's echo of CLM01 per X12 spec). Member_id never matches
    CLP01, so auto-match always fails.
  * apply_999_rejections — same lookup, same broken key.
  * apply_277ca_rejections — same lookup, same broken key.
  * scoring.score_pair — same broken key.

Live DB probe (1,739 remits, 337 claims):
  * Claim.id == Remit.payer_claim_control_number    → 0 matches
  * Claim.patient_control_number == Remit.payer_claim_control_number
    → 9 matches (substring coincidences; synthetic PCN strings share
    alphanumeric characters with the human-readable member_ids)
  * Claim.matched_remittance_id NOT NULL             → 0 claims

This commit changes _claim_837_row to write
Claim.patient_control_number = claim.claim_id (= CLM01). The
reconcile matcher's existing join now hits the row the 835 echoes
back. Companion migration 0017 backfills the 337 pre-fix rows so
they're on equal footing with new ingests (UPDATE claims SET
patient_control_number = id WHERE patient_control_number IS DISTINCT
FROM id — idempotent).

Tests:
  * 2 new RED→GREEN tests in test_store_reconcile.py:
    - test_837_ingest_populates_patient_control_number_from_claim_id
      pins the field semantics directly
    - test_837_then_835_with_echoed_pcn_auto_pairs proves the full
      end-to-end auto-match now fires
  * 3 existing manual_match tests that relied on the bug (had setup
    using member_id != claim_id to deliberately prevent auto-match)
    updated to use distinct PCNs explicitly so the tests still
    exercise the manual-match path with a real orphan pair.
  * 3 store_claim_detail / api_gets tests adjusted for the same
    reason.

Verified: 1,176 / 1,177 backend tests pass; the one failure is the
pre-existing flake in test_provider_extended_response.py noted before
this work started.

Caveat for the existing dev DB: the 1,739 remits already in the DB
were ingested from 835 fixtures whose CLP01 is a different synthetic
identifier than the 837's CLM01 (the test fixtures never echoed
CLM01 — a fixture-data limitation, not a code bug). After this fix,
new 837+835 ingest pairs whose payer echoes CLM01 in CLP01 will
auto-match as expected. The pre-existing 1,739 remits will continue
to land in the unmatched bucket; that can only be fixed by
regenerating the test fixtures (out of scope for this SP).
This commit is contained in:
Nora
2026-06-29 14:18:22 -06:00
parent d81b6ed4fc
commit d8841834dc
7 changed files with 239 additions and 27 deletions
@@ -0,0 +1,28 @@
-- version: 17
-- Backfill claims.patient_control_number = claims.id.
--
-- SP27 Task 17 fixed the 837 ingest in store.py:_claim_837_row so
-- ``Claim.patient_control_number`` is populated from
-- ``claim.claim_id`` (CLM01) instead of ``claim.subscriber.member_id``
-- (the 2010BA NM109). The reconcile matcher joins on
-- ``Claim.patient_control_number == Remittance.payer_claim_control_number``
-- and the 835 echoes CLM01 in CLP01 per X12 spec, so the wrong field
-- silently broke every auto-match in production.
--
-- For rows written BEFORE the fix, the stored value is the member_id
-- (e.g. "W953474") which never matches any remit's CLP01. This
-- migration backfills those rows by aligning
-- ``patient_control_number`` with the row's own ``id`` (= CLM01).
-- After this, every claim row's PCN is consistent with the value the
-- 837 actually sent, and any newly-ingested 835 whose CLP01 echoes
-- that CLM01 will auto-pair.
--
-- Idempotent: only touches rows where the stored PCN doesn't already
-- match the row's id, so re-running on already-fixed rows is a no-op.
--
-- Reversal safety: the migration does NOT clear matched_remittance_id,
-- so any pre-existing manual_match pairs stay intact.
UPDATE claims
SET patient_control_number = id
WHERE patient_control_number IS DISTINCT FROM id;
+8 -1
View File
@@ -191,7 +191,14 @@ def _claim_837_row(claim: ClaimOutput, batch_id: str) -> Claim:
return Claim(
id=claim.claim_id,
batch_id=batch_id,
patient_control_number=claim.subscriber.member_id or "",
# SP27 Task 17: Claim.patient_control_number must hold the CLM01
# claim_submitter's_identifier the 837 sent — that's the value the
# 835 echoes in CLP01, which the reconcile matcher joins on
# (reconcile.py:by_pcn), and what 999 / 277CA ACK lookups also
# use to cross-reference the original claim. Storing
# subscriber.member_id here (the 2010BA NM109) silently broke
# every auto-match in production.
patient_control_number=claim.claim_id or "",
service_date_from=d_from,
service_date_to=d_to,
charge_amount=Decimal(claim.claim.total_charge or 0),