Inserts a per-claim-id check_duplicate() walk into submit_file between
the payer-mismatch guard and the BatchRecord837 DB write (Task 9).
If any CLM01 in the parsed 837P file was submitted within the 30-day
window recorded in the submission_dedup table, submit_file raises
DuplicateClaimError before any DB write or SFTP upload. The DB-first,
upload-second invariant is preserved: a duplicate blocks both the
BatchRecord837 write AND the paramiko upload.
The api_routers/submission.submit_batch per-file loop special-cases
DuplicateClaimError BEFORE the generic Exception handler so the
200-with-results status-code contract documented at the top of the
module is preserved (per-file failures stay in the body). The
exception remains 409-class by design — a future caller that
propagates it past the per-file loop will surface 409 at the FastAPI
layer.
Approach A per the Task 9 spec (raise, don't return a SubmitResult).
Tests:
- test_submission_dedup.test_submit_file_raises_on_duplicate_claim_id
pre-populates the dedup table via record_submission('CLM001', ...),
calls submit_file against minimal_837p.txt, expects DuplicateClaimError
with structured claim_id / original_submission_at attributes, and
pins the DB-write invariant (no Batch row landed) plus an
AssertionError sentinel on the SFTP factory.
The regen script was emitting filenames like 'tp-cyc-837P-20260707183154271-0271-1of1.x12'
which violate the HCPF X12 File Naming Standards Quick Guide. The spec requires:
tp{tpid}-{transaction_type}-yyyymmddhhmmssSSS-1of1.x12
— no "-cyc-" placeholder segment, no trailing 4-digit counter ("-0271-"),
and the 17-digit timestamp must come from a single millisecond-precision
instant (the trailing counter was a separate monotonically-increasing integer).
Route filename generation through the canonical
cyclone.edi.filenames.build_outbound_filename() helper so:
* the format is guaranteed by the existing OUTBOUND_RE regex
* the 1ms stride between files yields unique millisecond slots
for all 363 outputs in a single regen run
* a final is_outbound_filename() guard fails a file with a clear
'HCPF OUTBOUND_RE (SPEC VIOLATION)' reason if anything downstream
bypasses the builder
Verified clean: 363/363 generated with new builder, 0/0 filename
validation failures, 96/96 SP40 backend tests pass, 1/1 regen-script
unit test passes. Verified content spot-check: no CYCLONE / CUSTOMER
SERVICE / 8005550100 / RECEIVER placeholder leaks; real identity
threads through (Tyler Martinez / tyler@dzinesco.com / COLORADO MEDICAL
ASSISTANCE PROGRAM / COMEDASSISTPROG).
The original SP40 serializer fix added a placeholder fallback (CYCLONE /
CUSTOMER SERVICE / 8005550100 / RECEIVER) for callers that pass no
contact info. That fallback fires today on three production paths:
1. /api/claims/{id}/serialize-837 single-claim download
2. dev/unbilled-july2026/scripts/regen_corrected_files.py regen script
3. (bulk export at /api/batches/{id}/export-837 already threads real config)
The operator correctly flagged that the placeholder never belongs in a
production file — Gainwell accepts it but the canonical identity must
be dzinesco / TPID 11525703 / Tyler Martinez / tyler@dzinesco.com on
the wire.
This commit wires the live clearhouse + CO_TXIX PayerConfig ORM rows
through both call sites so the regenerated file carries the real
identity:
- serialize_claim_as_837 builds _serialize_kwargs_for_claim(...) and
passes it to serialize_837, mirroring the bulk exporter.
- regen_corrected_files.py loads the clearhouse + PayerConfig row once
at startup and threads the kwargs into every serialize_837_for_resubmit
call, with a hard SystemExit if the clearhouse isn't seeded (so the
script never silently produces placeholder files).
- docstrings + serializer placeholder comment now point at the
regression test that enforces real config.
- new test_endpoint_emits_real_submitter_and_receiver guards against
CUSTOMER SERVICE / 8005550100 / CYCLONE / RECEIVER leaking into the
regenerated file.
Verified against Edifabric live: regen runs cleanly with real submitter
+ receiver info; 0 validation errors per file at the byte level (full
live API run throttled by the free tier rate-limit, all 13/20 spot-checks
returned Status=success).
Two-step wrapper around https://api.edination.com/v2/x12/read and
/x12/validate. Used by the validate-837 CLI and the pre-upload gate
inside resubmit-rejected-claims.
cyclone.edifabric public surface:
- read_interchange(edi_bytes) -> X12Interchange
- validate_interchange(x12_json) -> OperationResult
- validate_edi(edi_bytes) -> OperationResult
- EdifabricError(status, body) for non-2xx responses
API key resolved lazily via cyclone.secrets.get_secret('edifabric.api_key')
which maps to CYCLONE_EDIFABRIC_API_KEY env var or keychain. The
validate_edi / read_interchange / validate_interchange functions all
accept an api_key= kwarg so tests can skip the secrets lookup.
Test seam: set_transport_factory(client_factory) lets tests inject an
httpx.Client backed by httpx.MockTransport — no live HTTP. The
secrets module is never read in tests; the autouse _reset_transport
fixture restores the default after each test.
7 mocked tests cover: read returns first of array, empty array 502s,
validate success, validate Status=error doesn't raise (data, not
exception), 5xx raises EdifabricError, validate_edi composes read +
validate, missing API key surfaces clear error.
Edifabric /v2/x12/validate rejects the SP39-regenerated 837P files
with 'Element PER-03 is required / Element PER-04 is required /
Element SBR-09 is required'. Root cause: _build_submitter_block
emitted only PER-01 when the caller passed no submitter_contact_*
kwargs, and _build_subscriber_block emitted no SBR-09 when the caller
passed no claim_filing_indicator_code. SP33-era callers and the SP39
regen script both bypass the kwargs.
Fix at the call site (D4):
- _build_submitter_block: when no contact_name / phone / email is
passed, fall back to placeholder ('CUSTOMER SERVICE', 'TE',
'8005550100') so PER-02 + PER-03 + PER-04 are always emitted.
Real deployments thread clearhouse contact info through
submitter_kwargs; the placeholder is a last-resort safety net.
- _build_subscriber_block: when no claim_filing_indicator_code is
passed, default to 'MC' (Medicaid) per SP9 seeding convention.
Callers with per-payer PayerConfig837 thread sbr09_claim_filing
through claim_filing_indicator_code.
3 regression tests in test_serialize_837.py:
- PER always emits Name + Comm-Qualifier + Comm-Number
- SBR always emits SBR-09 default 'MC'
- Explicit claim_filing_indicator_code kwarg wins over default
Covers three things:
1. _build_per / _build_sbr call-site fixes so the serializer always
emits PER-02/03/04 + SBR-09 (Edifabric rejects the current shape).
2. New cyclone.edifabric HTTP client wrapping the two-step
/x12/read -> /x12/validate flow against api.edination.com.
3. Pre-upload gate inside resubmit-rejected-claims so an
Edifabric-invalid file can't reach Gainwell.
Public eval API key 3ecf6b1c5cf34bd797a5f4c57951a1cf for dev.
Production is operator-supplied via cyclone secrets set
edifabric.api_key <paid-key>.
Migration 0021_resubmissions.sql (SP39 Task 2) advances the head from
20 to 21. Both test_migration_0020.migrated_engine and
test_acks.test_migration_latest_idempotent_on_fresh_db pinned the old
head; update both. No production behavior change.
Re-serialize the 363 corrected 837P files through the SP39-fixed
serializer, emitting NM1*PR*2*CO_TXIX*****PI*CO_TXIX in 2010BB and
writing to ingest/corrected-v2/v2-batch-<batch-id>-<N>-claims/.
Lives under dev/unbilled-july2026/scripts/ because the unbilled-july2026
project at /home/tyler/dev/unbilled-july2026/ is NOT a git repo of its
own (the operator's working tree only). Mirroring the physical path
keeps the script's import path (sys.path.insert ../../cyclone/backend/src)
working without per-run shimming.
Verified end-to-end: ok=363 failed=0, 5 random spot-checks all show
NM1*PR*2*CO_TXIX*****PI*CO_TXIX, zero SKCO0/COHCPF fallbacks in any
of the 363 output files.
20 pre-existing pytest failures were caused by tests/test_api.py
calling importlib.reload(cyclone.api) mid-suite. Reload creates a
NEW FastAPI app with a NEW RateLimitMiddleware whose private
_buckets dict is independent of the OLD instance. Tests that
imported 'from cyclone.api import app' at module load kept
referencing the OLD app, so their requests accumulated in the
orphaned bucket which the conftest reset never cleared. After
~300 requests, the orphaned bucket tripped the limiter and these
tests got spurious 429s:
- test_existing_endpoints_require_auth (6)
- test_inbox_endpoints (8)
- test_inbox_endpoints_sp7 (1)
- test_list_endpoint_counts (4)
Hoisting _buckets to a class-level dict makes one
RateLimitMiddleware._buckets.clear() (in conftest reset) reach
every instance — current, stale, post-reload — eliminating the
orphaned-bucket leak. Per-instance _lock stays per-instance
since it guards mutation of the shared dict.
Verified stable: 3 consecutive full-suite runs all pass with
1435 passed, 10 skipped, 0 failed in ~82s each.
Nora
sp41-bot