Adds three live-tail streaming endpoints that emit an NDJSON snapshot
then forward new event-bus events as they arrive, with a 15s idle
heartbeat (overridable via CYCLONE_TAIL_HEARTBEAT_S for tests).
Each endpoint:
1. yields a snapshot of existing rows as {"type":"item","data":<row>}
2. terminates the snapshot with {"type":"snapshot_end","data":{"count":N}}
3. subscribes to its event kind and forwards each new event as an
{"type":"item","data":<event>} line
4. emits a {"type":"heartbeat","data":{"ts":<iso>}} line every
CYCLONE_TAIL_HEARTBEAT_S seconds when idle
5. checks request.is_disconnected() before each yield and unsubscribes
from the bus on cleanup so a closed stream releases its queue
The shared tail loop lives in api._tail_events, which polls
bus.subscribe_raw()'s queue directly instead of using the bus's
async-iterator wrapper — wait_for on an async generator cancels the
inner future on timeout, which poisons subsequent __anext__ calls with
StopAsyncIteration. Queue.get() is idempotent under cancellation, so
heartbeats don't break the subscription.
EventBus gains an unsubscribe(queue, kinds) method (idempotent) so
the tail loop can release its queue in a try/finally. The disconnect
test asserts the subscriber list is empty after the body iterator is
closed, validating no queue leak per open stream.
Tests in test_api_stream_live.py: 8 tests covering snapshot shape,
post-snapshot publish, heartbeat timing, multi-item snapshots, and
client disconnect cleanup. Plus 2 tests in test_pubsub.py for the
new unsubscribe method.
- api.parse_837 / parse_835: pass request.app.state.event_bus into store.add()
- conftest: autouse fixture wires a fresh EventBus onto app.state for every
test, since TestClient does not invoke the FastAPI lifespan handler
unless used as a context manager
- test_pubsub: split get_event_bus coverage into a raises-when-missing test
and a returns-attached-bus test, both save/restore app.state.event_bus
around the assertion so the autouse fixture's bus is preserved
Phase 2 complete: db init moved to lifespan, EventBus is the process-wide
publish point, and the two ingest endpoints publish claim_written /
remittance_written / activity_recorded events on every store.add().
The P3 subagent shipped a Download button on /acks that fell back to
'no raw text' because the detail endpoint returned raw_json (the parsed
model dump) but never the regenerated X12.
Fix: detail endpoint now re-serializes raw_json via serialize_999 and
returns it as raw_999_text, so the operator can actually download the
999 file. List endpoint unchanged (keeps payload small).
The P2 subagent shipped a UI chevron+expansion in Remittances.tsx that
read adjustments from the list payload, but iter_remittances() didn't
include the CasAdjustment rows. Result: the expansion was always empty
when a remittance was opened from the list (the detail endpoint worked).
Fix: bulk-fetch all CasAdjustment rows for the remittance_ids in one
query, then attach them as the 'adjustments' array. N+1-free.
- R034 _r034_ref_g1_required: when payer opts in (require_ref_g1_for_adjustments),
emit error if frequency_code is 7/8 and no REF*G1 segment appears in raw_segments.
CO Medicaid stays lenient in v1 (gate is False by default).
- R035 _r035_bht06_allowed: BHT06 transaction type code must be in cfg.allowed_bht06.
Skipped silently when transaction_type_code is None.
- Add transaction_type_code (str|None) to Envelope and ClaimOutput.
- parse_837: read BHT06 (seg[6]) in _build_envelope; mirror onto each ClaimOutput
via the existing model_copy(update={...}) call site.
- 9 new tests in test_validator.py: 5 R034 (incl. lenient no-op) + 4 R035.
213 passed, 1 skipped (prodfile corpus present-conditional).
Three spec-bug fixes from T10 implementation:
- Match.claim_id: drop UNIQUE constraint (T4 schema error) — reversals add a 2nd row per claim (audit trail). Add explicit non-unique index ix_matches_claim_id to preserve query performance. Mirrored in ORM and migration 0001_initial.sql.
- test_run_orphan_remit_leaves_claim_unmatched: claim PCN-A does not match remit PCN-NEW, so unmatched_claims must be 1, not 0.
- test_run_reversal_flips_paid_to_reversed: claim service_date_from was 9 days before reversal remit service_date (outside default 7-day window) so no match occurred; changed to 5 days apart so match() picks the claim.
- test_match_unique_per_claim: in test_db_models.py asserted the now-removed UNIQUE behavior; renamed and inverted to assert the audit-trail design (two Match rows per claim allowed).
Replace the in-memory InMemoryStore with a SQLAlchemy-backed
CycloneStore that persists Batch, Claim, Remittance, Match, and
ActivityEvent rows to a configurable DB engine (sqlite by default,
overridable via CYCLONE_DB_URL).
Public API preserved: add / get_batch / iter_claims / iter_remittances
/ distinct_providers / recent_activity. New T10/T12 stubs:
list_unmatched, manual_match, manual_unmatch.
Backward-compat shims for tests that called ._batches.clear() or
acquired ._lock as a context manager.
Idempotency: add() does a per-row session.get(Claim, c.id) (and
s.get(Remittance, ...)) check before each insert; duplicates are
skipped with a warning. This makes re-uploading the same fixture
idempotent instead of raising IntegrityError on the PK.
Auto-init DB fixture (backend/tests/conftest.py) sets
CYCLONE_DB_URL to a per-test sqlite file and calls db.init_db()
once per test, replacing the old module-scoped in-memory fixture.
Spec 6.2 says: 'Default JSON response wraps the same data in a
{items, total, returned, has_more} envelope so the frontend can
paginate uniformly.' Previously the GET list endpoints defaulted
to NDJSON (consistent with the parse endpoints), which forced
every curl call to set Accept: application/json.
Now NDJSON is strictly opt-in via Accept: application/x-ndjson;
JSON is the default. Parse endpoints keep their existing NDJSON
default (browser uploads).
T8 test 'test_claims_default_accept_returns_ndjson' updated to
assert the new default.
- iter_claims/iter_remittances now return list[dict] (snapshot under
the lock, return outside) so slow consumers no longer hold the
write lock during iteration
- iter_claims gains payer (case-insensitive substring) and
date_from/date_to filters via the new _date_in_bounds helper
- iter_remittances wires date_from/date_to and payer
- BatchRecord split into BatchRecord837 / BatchRecord835 with
__new__ dispatch so isinstance narrows result correctly,
eliminating the four 'type: ignore[assignment]' band-aids
- BatchRecord now rejects naive parsed_at via model_validator
- to_ui_remittance's adjustmentAmount=0.0 is documented as a
TODO for sub-project 2 reconciliation
151 passed, 1 skipped (no regressions; +6 tests for new filters)
Tyler