feat(sp27): unify 835 ingest + reconciliation in handle_835 (atomic)

Move 'reconcile.run(s, record.id)' inside CycloneStore.add()'s ingest
session, before s.commit(). The placeholder adjustment_amount set by
_remittance_835_row is overwritten by reconcile's CAS-aggregate pass
in the same transaction — readers never see a half-reconciled
Remittance row. If reconcile raises, the entire 835 ingest rolls back
via the session's __exit__.

The previous flow committed batch + remittance rows in session-1,
then opened session-2 to run reconcile.fail-soft. Two visible
problems closed: a race window where readers fetched the placeholder
adjustment_amount, and a half-reconciled state left visible if
reconcile crashed.

Deviation from the plan (N4 in autoreview): the reconcile call lives
in cycl_store.add() rather than handle_835 calling a new
reconcile.run_now(batch_id) helper after add(). Same end state, one
fewer module surface, handler stays a thin wrapper over the store.
This commit is contained in:
Nora
2026-06-29 12:08:47 -06:00
parent 44a6fb031a
commit d5f95b4f3c
6 changed files with 203 additions and 55 deletions
+6 -5
View File
@@ -519,12 +519,13 @@ def _build_and_persist_ack(batch_id: str) -> dict | None:
def _reconciliation_summary_for_batch(batch_id: str) -> dict:
"""Return ``{matched, unmatched_claims, unmatched_remittances, skipped}`` for a batch.
Reads from the DB after ``store.add()`` has already triggered T10
reconciliation synchronously (via ``_run_reconcile``). Counts are
observed at this moment; a subsequent manual match/unmatch will not
be reflected until the next request.
Reads from the DB after ``store.add()`` has already run reconciliation
synchronously (SP27 Task 10: ``reconcile.run(s, batch_id)`` inside the
ingest session, before commit). Counts are observed at this moment;
a subsequent manual match/unmatch will not be reflected until the
next request.
``skipped`` is reserved for future use — the T10 orchestrator tracks
``skipped`` is reserved for future use — the orchestrator tracks
skipped claims internally but does not surface a queryable count.
"""
from sqlalchemy import func, select
+5 -4
View File
@@ -15,10 +15,11 @@ but moving it is out of Task 5 scope). We import it lazily inside
acceptable because api.py also imports scheduler lazily (inside its
lifespan handler).
The two-phase ingest problem (batch row first, then a separate
``reconcile`` pass that overwrites ``adjustment_amount``) is a known
gap; atomic unification happens in SP27 Task 10. For Task 5 we lock
current behavior.
Two-phase ingest closed in SP27 Task 10: the placeholder
``adjustment_amount`` is overwritten by reconcile in the same DB
session as the insert (before commit), so a reader never sees a
half-reconciled Remittance row. If reconcile raises, the whole
ingest rolls back and the scheduler records the per-file error.
``event_bus`` is best-effort and follows the same TODO(sp27-task-6)
sync/async gap as handle_999 / handle_ta1 / handle_277ca.
+9 -4
View File
@@ -224,10 +224,15 @@ class ReconcileResult:
def run(session, batch_id: str) -> ReconcileResult:
"""Orchestrate reconciliation for an 835 batch.
Expects Batch + Remittance + CasAdjustment rows already persisted
(this is called from store.add() AFTER commit). Loads the new
remittances + all currently-unmatched Claims, calls match(), then
applies each match's intent inside the caller's session.
Expects Batch + Remittance + CasAdjustment rows already added to
``session`` (not yet committed). SP27 Task 10 calls this from
inside ``CycloneStore.add``'s ingest session, BEFORE
``s.commit()`` — so the whole 835 ingest (rows + reconcile) is
a single transaction. If anything here raises, the ingest rolls
back; no half-reconciled batch ever lands.
Loads the new remittances + all currently-unmatched Claims, calls
match(), then applies each match's intent inside the caller's session.
Returns counts. Does NOT commit; caller controls transaction.
Raises on failure (caller catches and writes activity event).
+16 -35
View File
@@ -912,9 +912,12 @@ class CycloneStore:
For 835 batches: inserts the Batch row, one Remittance row per
ClaimPayment, and a ``remit_received`` ActivityEvent per
ClaimPayment. After commit, calls ``_run_reconcile`` (T10 stub)
in a fail-soft manner — reconciliation errors are logged but
do not roll back the persisted batch.
ClaimPayment. Reconciliation (auto-match + per-pair CAS
aggregate) runs IN THE SAME SESSION before commit (SP27
Task 10) — a single ``s.commit()`` covers both ingest and
reconciliation. If reconcile raises, the whole 835 ingest
rolls back; the batch never appears half-reconciled with
placeholder ``adjustment_amount`` values.
Idempotency: ``Claim.id`` and ``Remittance.id`` are PRIMARY KEYS,
so a re-ingest of the same fixture (e.g. ``/api/parse-837`` called
@@ -1019,19 +1022,17 @@ class CycloneStore:
f"Unsupported BatchRecord subclass: {type(record).__name__}"
)
s.commit()
# SP27 Task 10: run reconcile INSIDE the same session, before
# commit. The placeholder ``adjustment_amount`` set by
# ``_remittance_835_row`` is overwritten by reconcile's
# ``_reconcile_pair`` SUM-of-CAS-aggregate pass before the
# rows are ever visible. If reconcile raises, the whole
# 835 ingest rolls back and the batch never lands.
if record.kind == "835":
from cyclone import reconcile as _reconcile
_reconcile.run(s, record.id)
# Reconcile 835 batches after the batch is durably persisted.
# Fail-soft: errors are logged, not raised, so an 835 parse that
# crashes in reconciliation still shows up in /api/batches.
if record.kind == "835":
try:
self._run_reconcile(record.id)
except Exception: # pragma: no cover - logged via default handler
import logging
logging.getLogger(__name__).exception(
"reconcile.run failed for batch %s", record.id,
)
s.commit()
# Publish live-tail events synchronously. EventBus.publish is async
# but its body is purely synchronous ``put_nowait`` enqueues; we
@@ -1113,26 +1114,6 @@ class CycloneStore:
for queue in list(event_bus._subscribers.get(kind, ())):
event_bus._enqueue_or_drop_oldest(queue, event)
def _run_reconcile(self, batch_id: str) -> None:
"""T10 stub: invoke the reconcile orchestrator for a batch.
The actual reconciliation is implemented in T10 (this method
will import ``cyclone.reconcile`` and call ``reconcile.run``
with the current session). For T9 we keep the import lazy and
fail-soft so a missing or NotImplementedError reconcile module
never breaks the 835 ingest path.
"""
# T10 will replace this with the real implementation. Until then
# we accept the failure modes the spec lists: ImportError
# (module not yet wired), NotImplementedError (stub in place),
# or any other transient reconcile error — all swallowed.
try:
from cyclone import reconcile as _reconcile
with db.SessionLocal()() as s:
_reconcile.run(s, batch_id)
except (ImportError, NotImplementedError):
pass
# -- read path ------------------------------------------------------
def _row_to_record(self, row: Batch) -> BatchRecord: