Add an optional ``?download=true`` query param to
``POST /api/inbox/rejected/resubmit`` that returns the same operation
result as a ZIP archive of regenerated 837P files (one
``claim-{id}.x12`` per successfully resubmitted claim) rather than the
JSON envelope.
Why: operators who mass-resubmit rejected claims want to hand the
files straight to their clearinghouse; round-tripping through copy-
paste is error-prone.
Implementation notes:
* Uses ``serialize_837_for_resubmit`` so each X12 file in the bundle
gets a unique interchange/group control number (back-to-back
resubmits would otherwise collide on ISA13/GS06 = "000000001").
* Conflicts and missing ids are deliberately excluded from the ZIP
— the user already saw them in the JSON path on prior calls; the
download is the "give me the files I asked for" payload.
* Empty resubmit + download returns 200 with an empty ZIP so the UI
can still hand the user a downloadable artifact.
Tests (test_inbox_endpoints.py): 2 new tests covering the success
shape (one .x12 per accepted claim) and the conflict-exclusion
contract.
X12 835 SVC segment:
SVC01 = composite procedure
SVC02 = charge
SVC03 = payment
SVC04 = Unit or Basis for Measurement Code (UN, MJ, DA, ...)
SVC05 = Service Unit Count
The parser previously read SVC04 as the units count and SVC05 as the
unit type — backwards. On real 835s (and the canonical minimal
fixture), SVC04 carries the code 'UN' which fails Decimal parsing, so
the units always came out as None and the code string was assigned to
unit_type. SP7's line-level matcher couldn't compare units on the SVC
side against the claim side because of this.
- _consume_service_payment: SVC04 → unit_type, SVC05 → units count
- Default unit_type to 'UN' when only the count is present
- minimal_835.txt + unbalanced_835.txt: swap positions to match spec
- Add 2 regression tests (units-and-unit-type, default-unit-type-to-UN)
Returns the persisted ClaimOutput as a regenerated X12 837P file via
the new outbound serializer (Approach A — full rebuild from canonical
fields). 404 on missing claim, 422 if the stored raw_json cannot be
validated as a ClaimOutput. text/x12 content-type, attachment
disposition with the claim id as the filename.
3 tests:
- endpoint returns text/x12 attachment starting with ISA*
- 404 for missing claim id
- regenerated text round-trips back through parse()
- backend/src/cyclone/parsers/serialize_837.py — full-rebuild 837P serializer.
Emits envelope (ISA/GS/ST/SE/GE/IEA + BHT) + submitter/receiver/billing
provider/subscriber/payer hierarchy + editable segments (CLM/REF*G1/HI)
+ per-service-line LX/SV1/DTP*472/REF*6R — all from canonical
ClaimOutput fields.
Pivoted from spec §3.1 hybrid to full rebuild because ClaimOutput.raw_segments
only captures post-CLM segments (CLM, REF*G1, HI, LX, SV1 pairs) — not the
envelope or hierarchies. A pass-through approach cannot regenerate those
without expanding raw_segments in parse_837.py (out of scope for this SP).
- backend/tests/test_serialize_837.py — 36 tests covering envelope shape,
hierarchy segments, claim-level builders, service-line builders, edited-field
propagation, round-trip, custom sender/receiver IDs, and resubmit helper.
- backend/tests/test_prodfiles_smoke.py::test_claims_prodfile_round_trip —
every file in docs/prodfiles/claims/ (113 files) round-trips through
serialize_837 → parse_837_text with deep-equal ClaimOutput (modulo
validation, which is recomputed by the parser).
- docs/superpowers/plans/2026-06-20-cyclone-serialize-837.md — full plan
with amendment note documenting the Approach A pivot.
Per session convention, plan note about unrelated modifications to
parse_835.py / fixtures stashed separately.
manual_match previously only flipped the claim↔remit FK and the
claim state — it never ran line-level reconciliation, so manually-
paired claims surfaced empty line-reconciliation rows to the UI
and skipped CLP-level CAS aggregate recompute.
Refactor reconcile.run() to call a new per-pair helper
_reconcile_pair(session, claim, remittance) that:
- clears any existing LineReconciliation rows for the claim
(idempotent re-run; safe across manual_unmatch + manual_rematch
cycles that may pair the claim with a different remittance),
- reads 837 SV1 lines from Claim.raw_json and 835 SVC rows from
ServiceLinePayment,
- runs match_service_lines() and persists a LineReconciliation row
per side,
- recomputes Remittance.claim_level_adjustment_amount (CLP-level
CAS) and Remittance.adjustment_amount (total CAS).
manual_match now calls this helper after the FK is set and before
commit, mirroring the auto-match path. Reversals are skipped (they
don't have SV1↔SVC line pairs; per §7.3).
Tests (test_store_reconcile.py):
- test_manual_match_populates_line_reconciliation_rows: end-to-end
check that a manual pair writes the expected matched +
unmatched_837_only rows plus zero CAS aggregates.
- test_manual_match_idempotent_line_reconciliation: after
manual_unmatch + manual_rematch + inserting a CLP-level CAS row
directly, the claim has exactly two fresh LineReconciliation rows
(no duplicates) and the remittance aggregate reflects the new CAS.
Smoke tested end-to-end via TestClient: parse co_medicaid_837p.txt
+ co_medicaid_835.txt, auto-matcher skips (PCNs differ), manual
match via POST /api/reconciliation/match, then
GET /api/claims/{id}/line-reconciliation returns 2 rows and
GET /api/inbox/lanes shows matched_remittance.total_lines=2.
Note: matched_lines may be 0 on real 835 fixtures due to a
pre-existing parser bug in _consume_service_payment that swaps
SVC04 (units) and SVC05 (unit-of-measure) when reading units from
the segment. The SP7 strict-match criterion requires units parity,
so the misread produces None on the SVC side and strict-match
never succeeds. Surfaced as a follow-up; the SP7 work itself is
correct (line counts are computed and persisted).
T7-T10 combined (single edit: all endpoints share a section).
- GET /api/inbox/lanes — four lanes in one call
- POST /api/inbox/candidates/{remit_id}/match — 409 on conflict
- POST /api/inbox/candidates/dismiss — session-scoped dismissed set
- POST /api/inbox/rejected/resubmit — 200 with conflicts list
- GET /api/inbox/export.csv?lane=<lane> — streams CSV
Also adds module-level imports (db, Claim, ClaimState, Remittance,
csv, io, datetime) that the new endpoints need; cleans up the
duplicated local imports in the parse-999 SP6 T4 block.
T4 — calls apply_999_rejections inside /api/parse-999, then
publishes a claim.rejected event for each transitioned claim so
the Inbox live-tail refetches.
T3 — moves claims to ClaimState.REJECTED on 999 AK5 R/E/X.
- New module: src/cyclone/inbox_state.py
- New tests: tests/test_inbox_state.py (4 cases)
- Adds ORM mappings for rejection_reason / rejected_at / resubmit_count
/ state_changed_at on Claim (db.py) — these were added to the
schema by migration 0004 but not yet exposed to the ORM.
Adds:
- claims.rejection_reason, claims.rejected_at, claims.resubmit_count
- claims.state_changed_at (was missing, needed for Done-today lane)
- ix_claims_state_changed_at composite index
Also fixes the ClaimState count assertion in test_db_models.py
(7 → 8) to match the REJECTED enum value added in the previous
commit, and bumps the user_version expectation in test_acks.py
(3 → 4) for the same reason.
The store dedupes remittances by PCN. Refine the assertions:
- total_clps counts the raw CLP segments across all files (3374)
- unique_pcns counts distinct PCNs (persisted row count)
- Read claimId via the API response shape, not the SQLAlchemy attribute
Three related changes for real CO Medicaid data:
1. Drop UNIQUE(batch_id, patient_control_number) on claims and
UNIQUE(batch_id, payer_claim_control_number) on remittances. The X12
spec allows multiple CLM segments per 2000B subscriber loop and 835
ERAs can repeat a payer_claim_control_number for reversals. Claim/
remittance identity is provided by the primary key (claims.id = CLM01,
remittances.id = CLP01).
2. Add validator rule R835_MULTI_BPR warning for files with multiple BPR
segments (CO Medicaid split-payment pattern). The parser already sums
BPR02 paid_amounts; this surfaces the non-standard data to operators.
3. Skip R835_BAL_BPR_vs_CLP04 when BPR01='I' (Information Only 835).
In that mode BPR02 is informational and the per-claim CLP04 totals
are authoritative — a diff is expected, not an error.
Migration 0003 handles the drop with IF EXISTS so fresh DBs skip cleanly.
Updates affected tests to reflect new schema (no UNIQUE constraint on
batch_id + patient_control_number / payer_claim_control_number).
Fixes test_api_835::test_prodfile_round_trip_persists_separately which
was failing on real production data.
Adds three live-tail streaming endpoints that emit an NDJSON snapshot
then forward new event-bus events as they arrive, with a 15s idle
heartbeat (overridable via CYCLONE_TAIL_HEARTBEAT_S for tests).
Each endpoint:
1. yields a snapshot of existing rows as {"type":"item","data":<row>}
2. terminates the snapshot with {"type":"snapshot_end","data":{"count":N}}
3. subscribes to its event kind and forwards each new event as an
{"type":"item","data":<event>} line
4. emits a {"type":"heartbeat","data":{"ts":<iso>}} line every
CYCLONE_TAIL_HEARTBEAT_S seconds when idle
5. checks request.is_disconnected() before each yield and unsubscribes
from the bus on cleanup so a closed stream releases its queue
The shared tail loop lives in api._tail_events, which polls
bus.subscribe_raw()'s queue directly instead of using the bus's
async-iterator wrapper — wait_for on an async generator cancels the
inner future on timeout, which poisons subsequent __anext__ calls with
StopAsyncIteration. Queue.get() is idempotent under cancellation, so
heartbeats don't break the subscription.
EventBus gains an unsubscribe(queue, kinds) method (idempotent) so
the tail loop can release its queue in a try/finally. The disconnect
test asserts the subscriber list is empty after the body iterator is
closed, validating no queue leak per open stream.
Tests in test_api_stream_live.py: 8 tests covering snapshot shape,
post-snapshot publish, heartbeat timing, multi-item snapshots, and
client disconnect cleanup. Plus 2 tests in test_pubsub.py for the
new unsubscribe method.
- api.parse_837 / parse_835: pass request.app.state.event_bus into store.add()
- conftest: autouse fixture wires a fresh EventBus onto app.state for every
test, since TestClient does not invoke the FastAPI lifespan handler
unless used as a context manager
- test_pubsub: split get_event_bus coverage into a raises-when-missing test
and a returns-attached-bus test, both save/restore app.state.event_bus
around the assertion so the autouse fixture's bus is preserved
Phase 2 complete: db init moved to lifespan, EventBus is the process-wide
publish point, and the two ingest endpoints publish claim_written /
remittance_written / activity_recorded events on every store.add().
The P3 subagent shipped a Download button on /acks that fell back to
'no raw text' because the detail endpoint returned raw_json (the parsed
model dump) but never the regenerated X12.
Fix: detail endpoint now re-serializes raw_json via serialize_999 and
returns it as raw_999_text, so the operator can actually download the
999 file. List endpoint unchanged (keeps payload small).
The P2 subagent shipped a UI chevron+expansion in Remittances.tsx that
read adjustments from the list payload, but iter_remittances() didn't
include the CasAdjustment rows. Result: the expansion was always empty
when a remittance was opened from the list (the detail endpoint worked).
Fix: bulk-fetch all CasAdjustment rows for the remittance_ids in one
query, then attach them as the 'adjustments' array. N+1-free.
- R034 _r034_ref_g1_required: when payer opts in (require_ref_g1_for_adjustments),
emit error if frequency_code is 7/8 and no REF*G1 segment appears in raw_segments.
CO Medicaid stays lenient in v1 (gate is False by default).
- R035 _r035_bht06_allowed: BHT06 transaction type code must be in cfg.allowed_bht06.
Skipped silently when transaction_type_code is None.
- Add transaction_type_code (str|None) to Envelope and ClaimOutput.
- parse_837: read BHT06 (seg[6]) in _build_envelope; mirror onto each ClaimOutput
via the existing model_copy(update={...}) call site.
- 9 new tests in test_validator.py: 5 R034 (incl. lenient no-op) + 4 R035.
213 passed, 1 skipped (prodfile corpus present-conditional).
Three spec-bug fixes from T10 implementation:
- Match.claim_id: drop UNIQUE constraint (T4 schema error) — reversals add a 2nd row per claim (audit trail). Add explicit non-unique index ix_matches_claim_id to preserve query performance. Mirrored in ORM and migration 0001_initial.sql.
- test_run_orphan_remit_leaves_claim_unmatched: claim PCN-A does not match remit PCN-NEW, so unmatched_claims must be 1, not 0.
- test_run_reversal_flips_paid_to_reversed: claim service_date_from was 9 days before reversal remit service_date (outside default 7-day window) so no match occurred; changed to 5 days apart so match() picks the claim.
- test_match_unique_per_claim: in test_db_models.py asserted the now-removed UNIQUE behavior; renamed and inverted to assert the audit-trail design (two Match rows per claim allowed).
Replace the in-memory InMemoryStore with a SQLAlchemy-backed
CycloneStore that persists Batch, Claim, Remittance, Match, and
ActivityEvent rows to a configurable DB engine (sqlite by default,
overridable via CYCLONE_DB_URL).
Public API preserved: add / get_batch / iter_claims / iter_remittances
/ distinct_providers / recent_activity. New T10/T12 stubs:
list_unmatched, manual_match, manual_unmatch.
Backward-compat shims for tests that called ._batches.clear() or
acquired ._lock as a context manager.
Idempotency: add() does a per-row session.get(Claim, c.id) (and
s.get(Remittance, ...)) check before each insert; duplicates are
skipped with a warning. This makes re-uploading the same fixture
idempotent instead of raising IntegrityError on the PK.
Auto-init DB fixture (backend/tests/conftest.py) sets
CYCLONE_DB_URL to a per-test sqlite file and calls db.init_db()
once per test, replacing the old module-scoped in-memory fixture.
Tyler