Followup #3 from the SP37 final-state tracker. The router's
per-file try/except previously coerced every unexpected exception
to SubmitOutcome.SFTP_FAILED — misleading because a RuntimeError
from cycl_store.add has nothing to do with SFTP.
Changes:
* submission/result.py — add UNEXPECTED_ERROR = 'unexpected_error'
enum value. Docstring distinguishes it from the typed SFTP_FAILED.
* api_routers/submission.py — exception handler now returns
UNEXPECTED_ERROR (was SFTP_FAILED).
* test_api_submit_batch.py::test_submit_batch_unexpected_exception_recorded_in_results
— updated assertion (was sftp_failed, now unexpected_error).
* test_unexpected_error_outcome.py (new, 4 tests):
- enum value exists
- distinct from typed failure values
- API surfaces 'unexpected_error' for uncaught exceptions
- typed SFTP_FAILED path still surfaces 'sftp_failed' (regression lock)
Tests: 40/40 pass in 1.41s (full SP37 chain + new file).
Live curl: POST /api/submit-batch in stub mode → HTTP 409 (unchanged
behavior, no regression on the config-level guards).
Followup #2 from the SP37 final-state tracker. Replaces the
subprocess.run-based CLI help test with click.testing.CliRunner
(matches the pattern already used in test_cli.py).
Speedup: CliRunner doesn't spawn a subprocess, so the test runs in
~0.5s vs ~0.2s+ for subprocess (and the subprocess version was
susceptible to fork overhead, environment leakage between tests,
and PATH/CWD surprises on different hosts).
Same assertions:
* result.exit_code == 0 (matches subprocess.returncode == 0)
* '--ingest-dir' in result.output (matches stdout check)
Test passes in isolation and as part of the full SP37 chain
(36 tests in 1.29s — same speedup as the single test).
Followup #1 from the SP37 final-state tracker. The pre-existing
('POST', '/api/resubmit'): WRITE_ROLES entry only matched the exact
path /api/resubmit (no children — the prefix-match logic requires
the trailing slash for sub-paths). No route is registered at that
path; the actual resubmit lives at /api/inbox/rejected/resubmit
(which has its own entry).
Removing the dead entry is fail-closed: any future request to
/api/resubmit (or its descendants) returns None from allowed_roles,
which means deny. Without this cleanup, the matrix silently granted
WRITE_ROLES to a non-existent route.
Tests (test_auth_permissions_matrix.py, 5 cases):
* Exact path /api/resubmit not in PERMISSIONS
* allowed_roles('POST', '/api/resubmit') is None
* /api/resubmit/, /api/resubmit/anything all deny
Pin the invariant so a future contributor can't re-add a dead prefix.
TDD: tests written first, watched fail (5 fail with the dead entry),
removed the line, tests pass (5 pass). Full SP37 chain (36 tests)
passes with the change.
Code review of f1dc06e flagged 3 consistency outliers vs every other
gated router in api_routers/:
- Drop prefix='/api' from APIRouter. Every sibling (clearhouse,
parse, claims, batches, inbox, acks, ta1_acks, claim_acks,
remittances, providers, reconciliation, activity, eligibility,
dashboard, admin, config, payers, acks) declares the full path
in the decorator instead. Submission was the only one with
prefix='/api' + '/submit-batch'.
- Rename handler post_submit_batch -> submit_batch. Siblings use
verb_noun (submit_to_clearhouse, export_batch_837, etc.). The
'post_' prefix was unique to this file.
- Move 'from cyclone.store import store as cycl_store' from inside
the handler to the module top. Sibling routers all import store
at module top — the lazy-import rationale in the inline comment
was wrong; pulling store at module import time is fine (the rest
of the package already does it).
Tests: tests/test_api_submit_batch.py + tests/test_submission.py +
tests/test_batch_txn_set_cn.py + tests/test_batch_envelope_index_sp37.py
= 26 passed in 1.37s (no behavior change, just naming).
Thin HTTP wrapper around `cyclone.submission.submit_file` — the
HTTP twin of the `cyclone submit-batch` CLI (SP37 Task 5). Same
walker pattern, same `._*` AppleDouble skip, same `limit`
semantics, same per-file outcomes; both surfaces now agree on DB +
SFTP state byte-for-byte.
Body: `{ingest_dir, validate?, actor?, limit?}`. Response:
`{submitted, skipped, failed, results:[{file, outcome, batch_id,
error}]}`. Status codes:
- 200 on completed runs (per-file failures live in the body)
- 401 unauthenticated (matrix_gate)
- 404 no clearhouse seeded
- 409 clearhouse SFTP block in stub mode
- 422 ingest_dir missing on disk or Pydantic body error
Does NOT inject `sftp_client_factory` — submit_file uses its
default paramiko factory so SKIPPED is reachable in production.
Tests monkey-patch submit_file itself.
Permissions entry: `POST /api/submit-batch` → WRITE_ROLES (admin +
user), matching `POST /api/resubmit` posture.
Tests (11): happy path, AppleDouble skip, limit, empty ingest_dir
422, mixed-outcome 200, unexpected-exception-as-failed,
Pydantic-missing-422, on-disk-missing-422, no-clearhouse-404,
stub-mode-409, auth-gate-401.
Walks batch-*-claims/*.x12 under --ingest-dir, calls submit_file per
file, prints submitted/skipped/failed counts. Exits 0 even on
per-file failures (details in stdout); exits 2 on config-level
errors (no clearhouse, stub mode, missing dir).
This is the canonical outbound path; resubmit-rejected-claims
remains for one-off cases where no DB row is wanted.
SftpClient wrapper lacks a stat() method, so the previous default
factory produced a client whose stat() raised AttributeError — making
the SKIPPED outcome unreachable in production and silently
misclassifying all uploads as SFTP_FAILED. The helper now opens a
paramiko SFTP session directly (same pattern as
resubmit-rejected-claims in cli.py:620-650), so stat() and write_file()
both work end-to-end.
Also: tighten typing (SftpBlock instead of Any), add PAYER_MISMATCH
test, drop duplicate _db fixture, rename misleading test, add class
docstrings to SubmitResult/SubmitOutcome.
Owns the parse → DB write → SFTP upload sequence in one place. CLI
and HTTP thin-call it. DB-first invariant: if add_record raises, no
SFTP call is made. Idempotency: add_record dedupes via s.get(Claim,
claim_id); SFTP layer dedupes via stat().st_size match.
- test_acks.py:test_migration_latest_idempotent_on_fresh_db was still
asserting user_version == 19; migration 0020 (Task 1) bumped it to 20.
Without this fix, the full pytest suite fails the moment SP37 ships.
- claim_acks.py module docstring + store/__init__.py facade docstring
still advertised the index as single-key {envelope.control_number:
batch.id}. SP37 Task 3 made it dual-key (ISA13 + ST02); the docs now
match the implementation.
Each 837p batch row contributes up to two entries to the join-key
index (envelope.control_number + transaction_set_control_number). One
idx.get(set_control_number) call resolves either, so 999 acks whose
AK201 echoes the source 837's ST02 now hit Pass 1 where they
previously fell through to Pass 2 (and to the orphan log).
Backward compat preserved: rows with transaction_set_control_number
NULL (pre-SP37 batches) still resolve by ISA13.
The parser now captures the source 837's ST02 (transaction set control
number) on the Envelope model as 'transaction_set_control_number'.
add_record reads it off the envelope and stores it on the Batch row,
mirroring how 'envelope.control_number' (ISA13) was already handled.
Unlocks the SP37 join-key update so 999 set_control_number (AK201)
values resolve back to the right batch via Pass 1.
Migration 0020 adds the column additively (nullable, no default) and
backfills from raw_result_json.envelope.transaction_set_control_number
for any existing batch rows that already carry it. Required for the
SP37 join-key update so 999 acks can resolve by ST02 (the source 837's
transaction set control number) instead of just ISA13.
Task 15.
api_routers/claims.py (new, 530 lines):
- GET /api/claims (paginated list + NDJSON)
- GET /api/claims/stream (NDJSON live-tail on claim_written)
- GET /api/claims/{claim_id} (drawer detail + SP28 ack_links)
- GET /api/claims/{claim_id}/serialize-837 (regenerate X12 837P)
- GET /api/claims/{claim_id}/line-reconciliation (837 vs 835 side-by-side)
- 3 single-router helpers stay in-file per D4:
_compact_ack_links_for_claim, _claim_line_dict, _svc_to_dict
- All inline imports inside handlers preserved verbatim
(select, LineReconciliation/ServiceLinePayment/CasAdjustment,
json as _json, Decimal)
Slicing: 1 contiguous cut (drop 1-indexed 1274-1688 = the empty
section divider + the entire claims block). After the cut, the
next thing is the app-level auth_router import at api.py:1274+.
api.py: 1720 -> 1305 LOC (-415; 5 routes + 3 helpers extracted).
api_routers/__init__.py: registry extended (alphabetical) with
claims. 17 routers in registry.
Added 1 backward-compat shim for test_api_stream_live.py:
from cyclone.api_routers.claims import claims_stream
The test does 'from cyclone.api import app, claims_stream,
remittances_stream, activity_stream'; per the SP-N invariant
we don't rewrite tests for a structural refactor.
Pytest: bit-identical to baseline — 21 failed, 1246 passed,
10 skipped, 6 errors.
Live-tested via curl on the running container:
GET /api/claims -> 401
GET /api/claims/stream -> 401
GET /api/claims/<id> -> 401
GET /api/claims/<id>/serialize-837 -> 401
GET /api/claims/<id>/line-reconciliation-> 401
POST /api/claims -> 405
pr-reviewer: skipped (Tasks 13-16 batch — folded into Task 17).
Task 14.
api_routers/batches.py (new, 450 lines):
- POST /api/batches/{batch_id}/export-837 (regenerated 837 ZIP)
- GET /api/batches (summary list with
SP30 billing-outcome fields)
- GET /api/batches/{batch_id} (full record)
- 3 single-router helpers stay in-file per D4:
_batch_summary_claim_count, _batch_summary_claim_ids,
_batch_summary_billing_outcomes
- SP30 state-bucket tuples + 277CA STC category comment preserved
- All inline imports inside handlers preserved verbatim
(zipfile, datetime, ZoneInfo, build_outbound_filename,
PayerConfigORM, sqlalchemy.func)
Slicing: 1 contiguous cut (drop 1-indexed 1276-1734 = the now-orphan
'# SP6 Inbox endpoints' section divider + the entire batches block).
After the cut, the next route is /api/claims at api.py:1278 with
the standard 3-blank separator.
Import fix: BatchRecord lives in cyclone.store, not cyclone.db.
Initial draft imported it from cyclone.db which crashed the
import; corrected to 'from cyclone.store import BatchRecord, store'.
api.py: 2179 -> 1720 LOC (-459; 3 routes + 3 helpers extracted).
api_routers/__init__.py: registry extended (alphabetical) with
batches. 16 routers in registry.
Pytest: bit-identical to baseline — 21 failed, 1246 passed,
10 skipped, 6 errors.
Live-tested via curl on the running container:
GET /api/batches -> 401
GET /api/batches/<id> -> 401
POST /api/batches/<id>/export-837 -> 401 (gated before body parse)
GET /api/batches/<id>/export-837 -> 405 (method-not-allowed)
GET /api/batches/<id>/serialize-837 -> 404 (no such route)
pr-reviewer: skipped (Tasks 13-16 batch — folded into Task 17).
Task 13.
api_routers/inbox.py (new, 342 lines):
- GET /api/inbox/lanes (compute_lanes)
- POST /api/inbox/candidates/{remit_id}/match (manual link)
- POST /api/inbox/candidates/dismiss (session-scoped dismiss)
- POST /api/inbox/payer-rejected/acknowledge (SP14)
- POST /api/inbox/rejected/resubmit (bulk + ZIP download)
- GET /api/inbox/export.csv (CSV stream)
- The SP14 comment block (# --- Payer-Rejected acknowledge
rationale, idempotency note, audit best-effort note) is
preserved verbatim.
Slicing note: the 6 inbox routes are non-contiguous in api.py —
5 of them are in api.py:1280-1524 and the 6th (export.csv) is
in api.py:1734-1776, with /api/batches/{batch_id}/export-837
sandwiched in between. Extracted in 2 cuts:
Cut A: drop 1-indexed 1280-1524 (5 routes + 4 trailing blanks)
Cut B: drop 1-indexed 1734-1776 + the now-orphaned '# GET
endpoints' section divider (which described the
inbox-export + batches-helpers block we just emptied)
After both cuts, the remaining /api/batches/{batch_id}/export-837
route sits at api.py:1280+ in the new file, and the
_batch_summary_* helpers follow as before. api.py: 2470 -> 2179
LOC (-291; 6 routes extracted).
api_routers/__init__.py: registry extended (alphabetical) with
inbox. 15 routers in registry.
Pytest: bit-identical to baseline — 21 failed, 1246 passed,
10 skipped, 6 errors. (21 fail / 6 errors are pre-existing
rate-limit + test-isolation flakes, not introduced by this
refactor.)
Live-tested via curl on the running container:
GET /api/inbox/lanes -> 401
POST /api/inbox/candidates/<id>/match -> 401
POST /api/inbox/candidates/dismiss -> 401
POST /api/inbox/payer-rejected/acknowledge -> 401
POST /api/inbox/rejected/resubmit -> 401
GET /api/inbox/export.csv -> 401
GET /api/inbox/export.csv?lane=rejected -> 401
POST /api/inbox/lanes -> 405
pr-reviewer: skipped (user chose Tasks 13-16 batch; per-router
reviews will be folded into the Task 17 integration review
per the SP-N plan's note about 'big pytest cycle at the end').
Tasks 11 + 12 combined per user direction (one commit for the
non-streaming singleton block).
api_routers/clearhouse.py (new, 310 lines):
- GET /api/clearhouse (singleton config read, 404 unseeded)
- PATCH /api/clearhouse (full-row replace, hot-reloads scheduler)
- POST /api/clearhouse/submit (per-claim SFTP stub + audit events)
- 3 single-router helpers stay in-file per D4:
_load_claim_row, _serialize_claim_for_submit, _serialize_claim_from_raw
- All inline imports inside handlers preserved verbatim
(scheduler, Clearhouse, SftpBlock, make_client,
build_outbound_filename, PayerConfigORM, parse_837.parse).
api_routers/providers.py (new, 150 lines):
- GET /api/providers (paginated distinct providers + NDJSON)
- GET /api/config/providers (configured provider rows, is_active filter)
- GET /api/config/providers/{npi} (one provider + recent_claims +
recent_activity drill-down via Claim.id outer-join Remittance.claim_id)
- Three URL prefixes, one router per spec.
api_routers/_shared.py:
- Early promotion: _actor_user_id moved here from api.py.
The clearhouse router needs it; leaving it in api.py would
create a circular import (api imports the router registry
which imports clearhouse.py, which would import api for
_actor_user_id). Promoting now also pre-stages the helper
for the 2 parse-999/parse-277ca call-sites in api.py that
Task 16 (parse router) will extract. The function body is
verbatim — only the location moved. Docstring documents
the early-promotion rationale.
api.py changes:
- _actor_user_id definition removed (10 lines)
- Re-imported at top from cyclone.api_routers._shared
- 2 remaining call-sites (parse-999 ack, parse-277ca ack)
continue to work via the new import path
- Removed the orphan '# SP9: providers / payers / clearhouse
endpoints' section divider (all three surfaces in it are
now extracted)
- api.py: 2858 -> 2468 LOC (-390; 6 routes extracted)
api_routers/__init__.py: registry extended (alphabetical) with
clearhouse + providers. 14 routers in registry.
Pytest: bit-identical to baseline — 21 failed, 1246 passed,
10 skipped, 6 errors. (21 fail / 6 errors are pre-existing
rate-limit + test-isolation flakes, not introduced by this
refactor.)
Live-tested via curl on the running container:
GET /api/clearhouse -> 401 (matrix_gate fires)
POST /api/clearhouse/submit -> 401 (gated before body parse)
GET /api/providers -> 401
GET /api/config/providers -> 401
GET /api/config/providers/<npi> -> 401
POST /api/providers -> 405 (method-not-allowed)
POST /api/config/providers -> 405 (method-not-allowed)
pr-reviewer: PASS
Tasks 9 + 10 combined per user direction (one commit for the streaming-NDJSON batch).
api_routers/remittances.py (new, 169 lines):
- GET /api/remittances (paginated list + NDJSON variant)
- GET /api/remittances/summary (server-aggregated KPI tiles)
- GET /api/remittances/stream (NDJSON live-tail on remittance_written)
- GET /api/remittances/{remittance_id} (single remittance + labeled CAS)
api_routers/activity.py (new, 102 lines):
- GET /api/activity (paginated event list + NDJSON variant)
- GET /api/activity/stream (NDJSON live-tail on activity_recorded)
Both routers use router-level matrix_gate (single source of auth).
api.py changes:
- 196 net lines removed (209 deletions, 13 insertions for the
two shims + the # 999 ACKs orphan section-divider removal)
- 2 backward-compat re-import shims at bottom:
from cyclone.api_routers.remittances import remittances_stream
from cyclone.api_routers.activity import activity_stream
rationale: tests/test_api_stream_live.py imports these by name
from cyclone.api; per SP-N invariant we don't rewrite tests for
a structural refactor. (Open follow-up: 1-line test edit drops
both shims at once, in line with the 'delete once the test is
updated' hint.)
- Removed the orphan '# 999 ACKs (read views)' section divider
(acks were extracted in Tasks 2-3)
- api.py: 3041 -> 2844 LOC (-197)
api_routers/__init__.py: registry extended (alphabetical) with
remittances + activity. 12 routers in registry.
Pytest: bit-identical to baseline — 21 failed, 1246 passed,
10 skipped, 6 errors. (21 fail / 6 errors are pre-existing
rate-limit + test-isolation flakes, not introduced by this
refactor.)
Live-tested via curl on the running container:
GET /api/remittances -> 401 (matrix_gate fires)
GET /api/remittances/summary -> 401
GET /api/remittances/stream -> 401
GET /api/remittances/<id> -> 401
POST /api/remittances -> 405 (method-not-allowed)
GET /api/activity -> 401
GET /api/activity/stream -> 401
POST /api/activity -> 405
pr-reviewer: PASS
Move GET /api/reconciliation/unmatched, GET /api/batch-diff, POST /api/reconciliation/match, and POST /api/reconciliation/unmatch from api.py to api_routers/reconciliation.py. Read views + manual match/unmatch write paths via store.manual_match / store.manual_unmatch. The Side-by-side batch-diff divider is preserved between routes 1 and 2; the lazy imports of cyclone.batch_diff.diff_batches_to_wire and cyclone.store.NotMatchedError inside their handlers are preserved verbatim. Orphan imports AlreadyMatchedError and InvalidStateError removed from api.py.
Move GET /api/config/payers and GET /api/config/payers/{payer_id}/configs from api.py to api_routers/config.py. Both endpoints are read-only configuration surfaces used by the UI's 'Edit payers' page. Behaviour-preserving.
Move GET /api/payers/{payer_id}/summary from api.py to api_routers/payers.py, plus the in-process memo (constants _SUMMARY_TTL_S, _summary_cache and helper _clear_summary_cache). Behaviour-preserving — endpoint URL, params, response shape, 404-on-missing behavior, and the 60s cache TTL are unchanged.
Add a one-line backward-compat shim at the bottom of api.py that re-imports _clear_summary_cache from the new home. This keeps test_payer_summary.py working (its fixture calls api_mod._clear_summary_cache() between requests to wipe the cache) without modifying the test — SP36 explicitly forbids test edits for a structural refactor.
Move POST /api/eligibility/request and POST /api/eligibility/parse-271 from api.py to api_routers/eligibility.py, plus the _validate_eligibility_request single-router helper. Behaviour-preserving — endpoint URLs, params, status codes, response shapes, and 400/422/500 error envelopes are unchanged. The auth gate moves from per-route to router-level (semantically equivalent).
Move GET /api/dashboard/kpis from api.py to api_routers/dashboard.py. Single-route router; gate via matrix_gate at the router level. Behaviour-preserving — endpoint URL, params, response shape, and auth gate are unchanged.
Moves the entire /api/admin/* namespace (audit-log ×2, db/rotate-key ×1,
backup ×10, scheduler ×6, reload-config ×1) out of api.py and into the
existing api_routers/admin.py, which already owned /api/admin/validate-provider.
- api.py: 4294 → 3547 LOC (Δ -747). Removed the orphaned # --- separator
left behind by the cut, the orphaned 'return {ok,loaded,errors}'
tail of reload-config, and the now-unused cyclone.clearhouse.InboundFile
top-level import.
- api_routers/admin.py: 60 → 851 LOC. Added the imports the moved
routes need (json, logging, threading, time, AuditEvent, verify_chain,
InboundFile) and stripped the redundant top-level imports of
symbols that the route bodies reach via the inline _db_crypto /
_secrets / _backup_svc_mod / _backup_sched_mod / _scheduler_mod
/ _audit aliases (those aliases stay — tests rely on being able to
monkeypatch cyclone.api_routers.admin._X.method). Hoisted the inline
'import time' from inside pull_inbound to module scope. Dropped
the redundant 'import threading as _threading' alias — top-level
'import threading' already covers it.
- tests/test_api_rotate_key.py: 4 monkeypatch targets migrated from
cyclone.api._db_crypto / cyclone.api._secrets to
cyclone.api_routers.admin._db_crypto / cyclone.api_routers.admin._secrets
to match the new home of the rotate-key endpoint. Lock acquire/release
also migrated.
The non-admin /api/config/* and /api/payers/{id}/summary routes that
bracketed the moved admin blocks stay in api.py — they're extracted
in Tasks 5 & 6.
Behaviour-preserving: pytest = 20 failed / 1253 passed / 10 skipped
= baseline match. Admin-only subset (audit-log + backup + scheduler +
rotate-key + reload-config) = 34 passed.
Moves GET /api/277ca-acks and GET /api/277ca-acks/{ack_id}
out of api.py (formerly lines 1278-1318) into the existing
api_routers/acks.py, which already handles 999 ACK list/detail/
stream. 277CA ACKs share the same shape-of-life contract (persisted
by a parse endpoint, listed newest-first, detail returns raw_json)
so the consolidation lands in the router that already owns the
adjacent surface — no new router file.
While here:
- collapsed the inline batch-fetch in list_277ca_acks_endpoint to
the existing _find_linked_claim_ids_for_acks(ack_ids, kind='277ca')
helper that the 999 list endpoint already uses (and ta1_acks.py
uses too). Eliminates the copy-pasted SQL; same N+1-avoidance
one-SELECT contract.
- pruned ClaimAck / to_ui_two77ca_ack imports from api.py.
Behaviour-preserving: pytest post-cut = 20 failed / 1253 passed /
10 skipped = baseline match. pytest -k '277ca or ack' = 235 passed,
1 skipped.
Moves the per-router auth gate (Depends(matrix_gate)) from the
include_router() call sites in api.py onto each router's own
APIRouter(dependencies=...) declaration. Each router now owns
its own auth dependency.
api.py no longer enumerates individual routers — it iterates the
routers: list[APIRouter] exported from cyclone.api_routers. This
is the registry that 13 future router extractions will append to.
- new: backend/src/cyclone/api_routers/__init__.py (registry)
- new: backend/src/cyclone/api_routers/_shared.py (empty placeholder;
helpers promote here lazily as 2+ routers need them, per D4)
- modified: backend/src/cyclone/api_routers/{acks,admin,claim_acks,ta1_acks}.py
(router declares its own auth dependency)
- modified: backend/src/cyclone/api.py (5-line include_router loop
replaces 5 explicit include_router calls; auth-router includes at
lines 4334-4338 untouched)
Behaviour-preserving: pytest post-cut = 20 failed / 1253 passed /
10 skipped = baseline match (the 20 are pre-existing live-DB
pollution unrelated to SP36). Health stays public (no matrix_gate).
The 999, 277CA, and TA1 parsers already enforce envelope correctness at
the parser level (parse_999.py line 290 raises 'No AK9 segment found';
parse_277ca.py line 298 raises 'Expected ST*277 or ST*277CA'; parse_ta1.py
line 111 raises 'Expected TA1, got <other>'). These tests lock the HTTP
surface contract: a wrong-kind file POSTed to those endpoints must come
back as 400, never as 200 or 500.
Tests added:
- test_api_999.py: rejects_837_input, rejects_835_input
- test_api_277ca.py: rejects_835_input, rejects_837_input
- test_api_ta1.py: rejects_837_input, rejects_835_input
If a future PR relaxes any of those parser-level guards, the
corresponding regression lock fires immediately.
Mirror of the parse-837 SP35 guards. Same defense-in-depth shape:
tokenize first, reject anything whose ST01 doesn't start with '835'
(400 'Mismatched file kind'), and after parse refuse to persist a
batch with zero CLP segments (400 'No claims parsed').
Reuses the _transaction_set_id_from_segments helper added by the
parse-837 commit.
New tests in tests/test_api_835.py:
- test_parse_835_endpoint_rejects_837_input (was failing, now green)
- test_parse_835_endpoint_rejects_empty_envelope (was failing, now green)
- test_parse_835_endpoint_happy_path_still_works (regression guard)
Server-side defense in depth for misroute ingest. Before SP35, posting
an 835 file (or any other X12 with a parseable ISA envelope) to
/api/parse-837 silently produced a BatchRecord with claims=[] and a
bogus row on the History tab. The 837 parser only required an ISA
envelope; it didn't check the ST transaction-set id.
Two new guards run before persistence:
1. Envelope check: tokenize first, read ST01, reject anything that
doesn't start with '837'. 400 with error='Mismatched file kind',
expected='837p', detected_st=<actual>. Catches an 835/999/270/etc
routed to the wrong endpoint.
2. Empty-claims check: even with the right envelope, if the parser
produces zero CLM segments, return 400 'No claims parsed' and do
NOT persist.
New tests in tests/test_api.py:
- test_parse_837_endpoint_rejects_835_input (was failing, now green)
- test_parse_837_endpoint_rejects_empty_envelope (was failing, now green)
- test_parse_837_endpoint_happy_path_still_works (regression guard)
Helper _transaction_set_id_from_segments reused by the 835 mirror.
Finishes the in-tree was_skipped draft: size-matched remote files no
longer emit clearhouse.submitted events or increment uploaded, and the
periodic reconnect now triggers only after real uploads.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- payer-mismatch files are now skipped instead of uploaded
- SFTP session persists across files (was reopened and leaked per file)
- --reconnect-every now counts successful uploads, after increment
- drop unused SftpClient instantiation; read each file once
- apply_999_rejections docstring: R/E/X, not R/E
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Adds `cyclone resubmit-rejected-claims` to push corrected single-claim
837 files to the Gainwell ToHPE SFTP dir. Idempotent (stat-then-skip by
byte size). One persistent paramiko session per batch with
reconnect-every=50 to dodge MOVEit's silent per-session file cap
(~200 puts/session, no exception).
- Validates each file via `parse_837` before upload and rejects any
whose payer_id is not `CO_TXIX` (catches a bad byte-fix early).
- Refreshes test fixtures (`minimal_837p.txt`, `co_medicaid_837p.txt`,
`co_medicaid_837p_with_renderer.txt`) and the corresponding test
assertions (`test_payer.py`, `test_payer_summary.py`,
`test_co_medicaid_fixture.py`, `test_parse_837.py`) from the old
`SKCO0`/`COHCPF` payer IDs to `CO_TXIX`, matching
PayerConfig.co_medicaid() and the HCPF 837P Companion Guide.
- Adds `ingest/` to .gitignore — local scratch / production-data
staging only.
Add the 'cyclone backfill-999-rejections' subcommand to replay the
cascade fix in apply_999_rejections for any 999 acks already in the
DB. Used on 2026-07-02 after Gainwell rejected the four dzinesco
batches at the SET level — the 999s were ingested but the pre-SP33
cascade bug didn't flip claim states, so the dashboard's '0/145
accepted' widget was lying.
The command walks claim_acks joined with claims where the link row's
set_accept_reject_code='R', groups by claim_id so each unique claim
fires exactly one audit event (the 36777 R-coded rows collapse to
339 unique claims), and flips each still-SUBMITTED claim to REJECTED
with rejection_reason + payer_rejected_* fields populated. Claims
already in REJECTED are skipped.
Also moves 'if __name__ == "__main__": main()' to the bottom of the
file. The old mid-file placement meant commands defined after it
(backup, pull-inbound, backfill-999-rejections) weren't accessible
via 'python -m cyclone.cli <sub>' — only the 'cyclone' console script
worked. Latent bug since SP17; surfaced when SP33 added another
post-block command.
The 999 handler's rejection pass was looking up claims by patient
control number, but Gainwell rejects at the SET level (ST envelope)
when the whole batch fails the NM109=CO_TXIX rule. That meant a SET
rejection was treated as a no-op even though every claim in the SET
was actually rejected by the payer.
Add a batch_envelope_index param (mirrors apply_999_acceptances from
SP28) so SET-level rejections cascade to every claim in the SET.
Falls back to the legacy PCN lookup when the index has no entry.
Also tightens test_payer.py: PayerConfig.co_medicaid() now returns
payer_id='CO_TXIX' and payer_name='CO_TXIX' per HCPF 837P Companion
Guide (June 2025 - Version 2.5).
- backend: GET /api/batches now returns acceptedCount/rejectedCount/
pendingCount/billedTotal/topRejectionReason/hasProblem per item
(one GROUP BY query + one ordered scan, no N+1)
- backend: 2 tests pin the 837p full-bucket case + the 835 zero case
- frontend: BatchSummary extended with 6 optional fields
(backwards compat preserved)
- frontend: new RecentBatchesWidget renders one row per batch with
status icon + billed total + accepted count + top rejection reason
- frontend: 837p rows show $ total + 'N/M accepted'; 835 rows show
payment count (Remittance has no batch-level total_charge)
- frontend: full-width row between KPI tiles and Activity on the
Dashboard; click navigates to /batches?batch=ID
- frontend: 3 component tests cover empty/clean/problem/835 branches
- backend: _ack_summary_for_claims helper attaches claim_acks payload
to inbox rejected-lane rows (3 tests)
- frontend: InboxRow renders newest 3 AK2 chips + '+N more' overflow
under the rejection reason, with '999 not linked' marker when an
ack couldn't be linked to a claim
- frontend: per-row Resubmit button downloads a single corrected 837
via api.serializeClaim837 (no bulk modal, no zip — one click, one
.x12 file)
- frontend: 2 new tests for the chip rendering and the per-row
download flow
Nora
Cyclone SP37
sp37 bot