feat(sp27): unify 835 ingest + reconciliation in handle_835 (atomic)
Move 'reconcile.run(s, record.id)' inside CycloneStore.add()'s ingest session, before s.commit(). The placeholder adjustment_amount set by _remittance_835_row is overwritten by reconcile's CAS-aggregate pass in the same transaction — readers never see a half-reconciled Remittance row. If reconcile raises, the entire 835 ingest rolls back via the session's __exit__. The previous flow committed batch + remittance rows in session-1, then opened session-2 to run reconcile.fail-soft. Two visible problems closed: a race window where readers fetched the placeholder adjustment_amount, and a half-reconciled state left visible if reconcile crashed. Deviation from the plan (N4 in autoreview): the reconcile call lives in cycl_store.add() rather than handle_835 calling a new reconcile.run_now(batch_id) helper after add(). Same end state, one fewer module surface, handler stays a thin wrapper over the store.
This commit is contained in:
@@ -519,12 +519,13 @@ def _build_and_persist_ack(batch_id: str) -> dict | None:
|
||||
def _reconciliation_summary_for_batch(batch_id: str) -> dict:
|
||||
"""Return ``{matched, unmatched_claims, unmatched_remittances, skipped}`` for a batch.
|
||||
|
||||
Reads from the DB after ``store.add()`` has already triggered T10
|
||||
reconciliation synchronously (via ``_run_reconcile``). Counts are
|
||||
observed at this moment; a subsequent manual match/unmatch will not
|
||||
be reflected until the next request.
|
||||
Reads from the DB after ``store.add()`` has already run reconciliation
|
||||
synchronously (SP27 Task 10: ``reconcile.run(s, batch_id)`` inside the
|
||||
ingest session, before commit). Counts are observed at this moment;
|
||||
a subsequent manual match/unmatch will not be reflected until the
|
||||
next request.
|
||||
|
||||
``skipped`` is reserved for future use — the T10 orchestrator tracks
|
||||
``skipped`` is reserved for future use — the orchestrator tracks
|
||||
skipped claims internally but does not surface a queryable count.
|
||||
"""
|
||||
from sqlalchemy import func, select
|
||||
|
||||
@@ -15,10 +15,11 @@ but moving it is out of Task 5 scope). We import it lazily inside
|
||||
acceptable because api.py also imports scheduler lazily (inside its
|
||||
lifespan handler).
|
||||
|
||||
The two-phase ingest problem (batch row first, then a separate
|
||||
``reconcile`` pass that overwrites ``adjustment_amount``) is a known
|
||||
gap; atomic unification happens in SP27 Task 10. For Task 5 we lock
|
||||
current behavior.
|
||||
Two-phase ingest closed in SP27 Task 10: the placeholder
|
||||
``adjustment_amount`` is overwritten by reconcile in the same DB
|
||||
session as the insert (before commit), so a reader never sees a
|
||||
half-reconciled Remittance row. If reconcile raises, the whole
|
||||
ingest rolls back and the scheduler records the per-file error.
|
||||
|
||||
``event_bus`` is best-effort and follows the same TODO(sp27-task-6)
|
||||
sync/async gap as handle_999 / handle_ta1 / handle_277ca.
|
||||
|
||||
@@ -224,10 +224,15 @@ class ReconcileResult:
|
||||
def run(session, batch_id: str) -> ReconcileResult:
|
||||
"""Orchestrate reconciliation for an 835 batch.
|
||||
|
||||
Expects Batch + Remittance + CasAdjustment rows already persisted
|
||||
(this is called from store.add() AFTER commit). Loads the new
|
||||
remittances + all currently-unmatched Claims, calls match(), then
|
||||
applies each match's intent inside the caller's session.
|
||||
Expects Batch + Remittance + CasAdjustment rows already added to
|
||||
``session`` (not yet committed). SP27 Task 10 calls this from
|
||||
inside ``CycloneStore.add``'s ingest session, BEFORE
|
||||
``s.commit()`` — so the whole 835 ingest (rows + reconcile) is
|
||||
a single transaction. If anything here raises, the ingest rolls
|
||||
back; no half-reconciled batch ever lands.
|
||||
|
||||
Loads the new remittances + all currently-unmatched Claims, calls
|
||||
match(), then applies each match's intent inside the caller's session.
|
||||
|
||||
Returns counts. Does NOT commit; caller controls transaction.
|
||||
Raises on failure (caller catches and writes activity event).
|
||||
|
||||
@@ -912,9 +912,12 @@ class CycloneStore:
|
||||
|
||||
For 835 batches: inserts the Batch row, one Remittance row per
|
||||
ClaimPayment, and a ``remit_received`` ActivityEvent per
|
||||
ClaimPayment. After commit, calls ``_run_reconcile`` (T10 stub)
|
||||
in a fail-soft manner — reconciliation errors are logged but
|
||||
do not roll back the persisted batch.
|
||||
ClaimPayment. Reconciliation (auto-match + per-pair CAS
|
||||
aggregate) runs IN THE SAME SESSION before commit (SP27
|
||||
Task 10) — a single ``s.commit()`` covers both ingest and
|
||||
reconciliation. If reconcile raises, the whole 835 ingest
|
||||
rolls back; the batch never appears half-reconciled with
|
||||
placeholder ``adjustment_amount`` values.
|
||||
|
||||
Idempotency: ``Claim.id`` and ``Remittance.id`` are PRIMARY KEYS,
|
||||
so a re-ingest of the same fixture (e.g. ``/api/parse-837`` called
|
||||
@@ -1019,19 +1022,17 @@ class CycloneStore:
|
||||
f"Unsupported BatchRecord subclass: {type(record).__name__}"
|
||||
)
|
||||
|
||||
s.commit()
|
||||
# SP27 Task 10: run reconcile INSIDE the same session, before
|
||||
# commit. The placeholder ``adjustment_amount`` set by
|
||||
# ``_remittance_835_row`` is overwritten by reconcile's
|
||||
# ``_reconcile_pair`` SUM-of-CAS-aggregate pass before the
|
||||
# rows are ever visible. If reconcile raises, the whole
|
||||
# 835 ingest rolls back and the batch never lands.
|
||||
if record.kind == "835":
|
||||
from cyclone import reconcile as _reconcile
|
||||
_reconcile.run(s, record.id)
|
||||
|
||||
# Reconcile 835 batches after the batch is durably persisted.
|
||||
# Fail-soft: errors are logged, not raised, so an 835 parse that
|
||||
# crashes in reconciliation still shows up in /api/batches.
|
||||
if record.kind == "835":
|
||||
try:
|
||||
self._run_reconcile(record.id)
|
||||
except Exception: # pragma: no cover - logged via default handler
|
||||
import logging
|
||||
logging.getLogger(__name__).exception(
|
||||
"reconcile.run failed for batch %s", record.id,
|
||||
)
|
||||
s.commit()
|
||||
|
||||
# Publish live-tail events synchronously. EventBus.publish is async
|
||||
# but its body is purely synchronous ``put_nowait`` enqueues; we
|
||||
@@ -1113,26 +1114,6 @@ class CycloneStore:
|
||||
for queue in list(event_bus._subscribers.get(kind, ())):
|
||||
event_bus._enqueue_or_drop_oldest(queue, event)
|
||||
|
||||
def _run_reconcile(self, batch_id: str) -> None:
|
||||
"""T10 stub: invoke the reconcile orchestrator for a batch.
|
||||
|
||||
The actual reconciliation is implemented in T10 (this method
|
||||
will import ``cyclone.reconcile`` and call ``reconcile.run``
|
||||
with the current session). For T9 we keep the import lazy and
|
||||
fail-soft so a missing or NotImplementedError reconcile module
|
||||
never breaks the 835 ingest path.
|
||||
"""
|
||||
# T10 will replace this with the real implementation. Until then
|
||||
# we accept the failure modes the spec lists: ImportError
|
||||
# (module not yet wired), NotImplementedError (stub in place),
|
||||
# or any other transient reconcile error — all swallowed.
|
||||
try:
|
||||
from cyclone import reconcile as _reconcile
|
||||
with db.SessionLocal()() as s:
|
||||
_reconcile.run(s, batch_id)
|
||||
except (ImportError, NotImplementedError):
|
||||
pass
|
||||
|
||||
# -- read path ------------------------------------------------------
|
||||
|
||||
def _row_to_record(self, row: Batch) -> BatchRecord:
|
||||
|
||||
Reference in New Issue
Block a user