Three related changes for real CO Medicaid data:
1. Drop UNIQUE(batch_id, patient_control_number) on claims and
UNIQUE(batch_id, payer_claim_control_number) on remittances. The X12
spec allows multiple CLM segments per 2000B subscriber loop and 835
ERAs can repeat a payer_claim_control_number for reversals. Claim/
remittance identity is provided by the primary key (claims.id = CLM01,
remittances.id = CLP01).
2. Add validator rule R835_MULTI_BPR warning for files with multiple BPR
segments (CO Medicaid split-payment pattern). The parser already sums
BPR02 paid_amounts; this surfaces the non-standard data to operators.
3. Skip R835_BAL_BPR_vs_CLP04 when BPR01='I' (Information Only 835).
In that mode BPR02 is informational and the per-claim CLP04 totals
are authoritative — a diff is expected, not an error.
Migration 0003 handles the drop with IF EXISTS so fresh DBs skip cleanly.
Updates affected tests to reflect new schema (no UNIQUE constraint on
batch_id + patient_control_number / payer_claim_control_number).
Fixes test_api_835::test_prodfile_round_trip_persists_separately which
was failing on real production data.
Adds three live-tail streaming endpoints that emit an NDJSON snapshot
then forward new event-bus events as they arrive, with a 15s idle
heartbeat (overridable via CYCLONE_TAIL_HEARTBEAT_S for tests).
Each endpoint:
1. yields a snapshot of existing rows as {"type":"item","data":<row>}
2. terminates the snapshot with {"type":"snapshot_end","data":{"count":N}}
3. subscribes to its event kind and forwards each new event as an
{"type":"item","data":<event>} line
4. emits a {"type":"heartbeat","data":{"ts":<iso>}} line every
CYCLONE_TAIL_HEARTBEAT_S seconds when idle
5. checks request.is_disconnected() before each yield and unsubscribes
from the bus on cleanup so a closed stream releases its queue
The shared tail loop lives in api._tail_events, which polls
bus.subscribe_raw()'s queue directly instead of using the bus's
async-iterator wrapper — wait_for on an async generator cancels the
inner future on timeout, which poisons subsequent __anext__ calls with
StopAsyncIteration. Queue.get() is idempotent under cancellation, so
heartbeats don't break the subscription.
EventBus gains an unsubscribe(queue, kinds) method (idempotent) so
the tail loop can release its queue in a try/finally. The disconnect
test asserts the subscriber list is empty after the body iterator is
closed, validating no queue leak per open stream.
Tests in test_api_stream_live.py: 8 tests covering snapshot shape,
post-snapshot publish, heartbeat timing, multi-item snapshots, and
client disconnect cleanup. Plus 2 tests in test_pubsub.py for the
new unsubscribe method.
- api.parse_837 / parse_835: pass request.app.state.event_bus into store.add()
- conftest: autouse fixture wires a fresh EventBus onto app.state for every
test, since TestClient does not invoke the FastAPI lifespan handler
unless used as a context manager
- test_pubsub: split get_event_bus coverage into a raises-when-missing test
and a returns-attached-bus test, both save/restore app.state.event_bus
around the assertion so the autouse fixture's bus is preserved
Phase 2 complete: db init moved to lifespan, EventBus is the process-wide
publish point, and the two ingest endpoints publish claim_written /
remittance_written / activity_recorded events on every store.add().
The P3 subagent shipped a Download button on /acks that fell back to
'no raw text' because the detail endpoint returned raw_json (the parsed
model dump) but never the regenerated X12.
Fix: detail endpoint now re-serializes raw_json via serialize_999 and
returns it as raw_999_text, so the operator can actually download the
999 file. List endpoint unchanged (keeps payload small).
The P2 subagent shipped a UI chevron+expansion in Remittances.tsx that
read adjustments from the list payload, but iter_remittances() didn't
include the CasAdjustment rows. Result: the expansion was always empty
when a remittance was opened from the list (the detail endpoint worked).
Fix: bulk-fetch all CasAdjustment rows for the remittance_ids in one
query, then attach them as the 'adjustments' array. N+1-free.
- R034 _r034_ref_g1_required: when payer opts in (require_ref_g1_for_adjustments),
emit error if frequency_code is 7/8 and no REF*G1 segment appears in raw_segments.
CO Medicaid stays lenient in v1 (gate is False by default).
- R035 _r035_bht06_allowed: BHT06 transaction type code must be in cfg.allowed_bht06.
Skipped silently when transaction_type_code is None.
- Add transaction_type_code (str|None) to Envelope and ClaimOutput.
- parse_837: read BHT06 (seg[6]) in _build_envelope; mirror onto each ClaimOutput
via the existing model_copy(update={...}) call site.
- 9 new tests in test_validator.py: 5 R034 (incl. lenient no-op) + 4 R035.
213 passed, 1 skipped (prodfile corpus present-conditional).
T12's list_unmatched filters Remittance by claim_id IS NULL. T10's
reconcile.run was only setting Claim.matched_remittance_id, leaving
auto-matched remits visible in the orphan bucket. Mirror the FK on
both sides so both list_unmatched filters produce consistent results.
T12 implementer flagged this as out-of-scope; addressed here as a
small follow-up so the Reconciliation page works correctly end-to-end.
Three spec-bug fixes from T10 implementation:
- Match.claim_id: drop UNIQUE constraint (T4 schema error) — reversals add a 2nd row per claim (audit trail). Add explicit non-unique index ix_matches_claim_id to preserve query performance. Mirrored in ORM and migration 0001_initial.sql.
- test_run_orphan_remit_leaves_claim_unmatched: claim PCN-A does not match remit PCN-NEW, so unmatched_claims must be 1, not 0.
- test_run_reversal_flips_paid_to_reversed: claim service_date_from was 9 days before reversal remit service_date (outside default 7-day window) so no match occurred; changed to 5 days apart so match() picks the claim.
- test_match_unique_per_claim: in test_db_models.py asserted the now-removed UNIQUE behavior; renamed and inverted to assert the audit-trail design (two Match rows per claim allowed).
Replace the in-memory InMemoryStore with a SQLAlchemy-backed
CycloneStore that persists Batch, Claim, Remittance, Match, and
ActivityEvent rows to a configurable DB engine (sqlite by default,
overridable via CYCLONE_DB_URL).
Public API preserved: add / get_batch / iter_claims / iter_remittances
/ distinct_providers / recent_activity. New T10/T12 stubs:
list_unmatched, manual_match, manual_unmatch.
Backward-compat shims for tests that called ._batches.clear() or
acquired ._lock as a context manager.
Idempotency: add() does a per-row session.get(Claim, c.id) (and
s.get(Remittance, ...)) check before each insert; duplicates are
skipped with a warning. This makes re-uploading the same fixture
idempotent instead of raising IntegrityError on the PK.
Auto-init DB fixture (backend/tests/conftest.py) sets
CYCLONE_DB_URL to a per-test sqlite file and calls db.init_db()
once per test, replacing the old module-scoped in-memory fixture.
Spec 6.2 says: 'Default JSON response wraps the same data in a
{items, total, returned, has_more} envelope so the frontend can
paginate uniformly.' Previously the GET list endpoints defaulted
to NDJSON (consistent with the parse endpoints), which forced
every curl call to set Accept: application/json.
Now NDJSON is strictly opt-in via Accept: application/x-ndjson;
JSON is the default. Parse endpoints keep their existing NDJSON
default (browser uploads).
T8 test 'test_claims_default_accept_returns_ndjson' updated to
assert the new default.
Tyler