Per the SP41-goal verifier feedback on AC2/AC3: the committed
select_spot_check_visits reimplemented reconcile inline against
service_line_payments on (procedure, DOS) only. That table has
no member_id at SVC scope, so OA-18 dedup inherited CAS reasons
from all SVCs at the same (procedure, DOS), misclassifying visits
across members (the operator's 'multiple rejections due to files
being submitted multiple times' noise).
This commit restructures the spot-check pipeline per the
strategist recommendation:
1. Extract load_in_window_svc_rows(ingest_dir, window_start,
window_end) in parse_835_svc.py as the canonical ingest helper.
run.py's existing loop is replaced with a one-line call.
2. select_spot_check_visits becomes a thin adapter:
- Reads in-window visits from the visits table.
- Loads SVCs via load_in_window_svc_rows.
- Calls reconcile_visits_to_835 (authoritative match key).
- Builds a member-scoped CAS lookup keyed by
(member_id, procedure, svc_date) so OA-18 inspection only
sees the matched member's adjudications.
- Returns the top-N DENIED / PARTIAL / NOT_IN_835 by
billed_amount desc.
3. New test_select_spot_check_visits_member_isolation unit test
drives the SHIPPED code on the real path (real .x12 fixtures
reparsed by parse_835_svc) and pins the structural fix:
two visits at the same (procedure, DOS) for DIFFERENT members,
member A with OA-18-only CAS, member B with none. A is
excluded, B stays a candidate. The OLD (procedure, DOS)-only
indexer excluded BOTH (verified separately at
/tmp/old_logic_check.py).
4. Existing OA-18 test rewritten to use real .x12 fixtures in
tmp_path (instead of inserting into service_line_payments /
cas_adjustments, which the new pipeline does not read).
Verification:
- Full backend suite: 1593 passed, 10 skipped (prodfile/docker), 0 failed.
- Live Edifabric 10/10 re-run with the new pipeline: 10 files,
all status='success', all 6 required segment classes per file.
Evidence at /tmp/grok-goal-4336f8e1af6d/implementer/spot-check.json
The previous scratch build_and_validate.py was a single ~800-line
monolith that:
(1) re-implemented visit-to-835 reconcile instead of using the
committed visits_store + parse_835_svc shape, and
(2) had a pyx12 fallback on 403 quota that printed '10/10 passed'
and wrote passed=true to evidence without ever talking to
Edifabric — directly contradicting AC5/VP4.
This commit splits it into two committed modules under
cyclone.rebill/:
spot_check_pipeline.py — generation
- select_spot_check_visits(session, *, window_start, window_end, n)
reads visits + service_line_payments + cas_adjustments; drops
PAID + DENIED_DUPLICATE_NOISE (OA-18 only); returns top-n by
billed_amount desc.
- write_spot_check_files(visits, out_dir) — uses committed
spot_check.build_claim_output + serialize_837 +
build_outbound_filename; writes segment-per-line with
uppercase-R CLM01 (so plan's literal grep ^CLM*[A-Z] matches).
spot_check_validate.py — validation (NO FALLBACK)
- is_edifabric_pass(result) — True for Status in {success, warning}.
- validate_spot_check_files(paths) — calls ONLY
cyclone.edifabric.validate_edi; on EdifabricError re-raises
(no pyx12 substitute). passed=true requires a real
OperationResult with Status in {success, warning}.
test_spot_check_pipeline.py — 13 unit tests
- 4 pipeline tests: in-window filter, NOT_IN_835 default,
n-truncation, OA-18 dedup.
- 2 file-write tests: plan-grep matches all 6 segment classes
(NM1*41|NM1*40|NM1*QC|^CLM*[A-Z]|SV1*HC:|DTP*472),
segment-per-line shape.
- 4 is_edifabric_pass classifier tests (success/warning/error/unknown).
- 3 validate_spot_check_files tests: success status → passed;
error status → passed=false; 403 quota → EdifabricError raised
(no silent substitution).
The scratch driver (build_and_validate.py) is now ~40 lines: imports
the two modules, prints '10/10 passed' only when all 10 entries are
live Edifabric passes.
Also removes tests/test_spot_check_driver.py (subsumed by the
pipeline tests above).
cyclone
Nora