From 9c0cec8f0c7af1c64c2b4f5d3609d1030e334340 Mon Sep 17 00:00:00 2001 From: Tyler Date: Sun, 21 Jun 2026 17:47:03 -0600 Subject: [PATCH] spec: parse-then-decide workflow for 837/835 upload dedup --- ...21-cyclone-parse-decide-workflow-design.md | 784 ++++++++++++++++++ 1 file changed, 784 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md diff --git a/docs/superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md b/docs/superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md new file mode 100644 index 0000000..97eea3c --- /dev/null +++ b/docs/superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md @@ -0,0 +1,784 @@ +# Parse → Detect → Decide: 837P/835 Upload Workflow + +**Date:** 2026-06-21 +**Branch:** `claims-unique-fix` (worktree) +**Status:** Draft (brainstorming approved, awaiting writing-plans) +**Supersedes:** `2026-06-21-cyclone-claims-unique-constraint-and-409-ux-design.md` (kept for the migration 0013 + store helper sections, which still apply). + +--- + +## 1. Why this exists + +Today's upload flow is "parse → validate → persist" in a single call. +When a claim's CLM01 collides with a prior batch, the persist raises +`IntegrityError`, the transaction rolls back, and the API returns 409 +with **no parse result, no list of colliding claims, and no way to act**. +The user sees only an error message and a `batch_id` that doesn't exist. + +This is wrong: the parse already happened. The user should see what was +parsed, see which claims collide with which prior batches, and decide +what to do (force-insert, delete the prior batch, or pick a different +file). The 409 response body today is too thin to make that decision. + +The root cause of the 409s is a schema bug — see +`2026-06-21-cyclone-claims-unique-constraint-and-409-ux-design.md` §1. +Migration 0013 drops the `UNIQUE(batch_id, patient_control_number)` inline +constraint that 0003 was supposed to drop. After 0013 lands, **multi-claim +837P files where many CLM segments share a subscriber's `member_id` will +ingest cleanly for the first time** — but true cross-batch CLM01 +collisions can still fire, and the user needs the new workflow to handle +them. + +This SP defines the workflow for both classes of collision: +1. Multi-claim files with shared `member_id` (no longer a 409 after 0013). +2. Files where one or more CLM01s exist in a prior batch (still a 409). + +--- + +## 2. Operator surface + +| Surface | Change | +|---|---| +| Backend | Pre-flight dedup check in `parse_837` and `parse_835`. New `?force=true` query param. New `DELETE /api/batches/{id}` endpoint. 409 body shape changes. | +| Frontend | `Upload.tsx` panel renders the full parse result + collision summary, with actions: "Force insert (skip dups)", "Open prior batch", "Delete prior batch and retry", "Pick a different file". | +| Tests | Migration + store helpers (already done in `claims-unique-fix`). New tests for: pre-flight dedup, force-insert, within-file dup, race 409, DELETE endpoint, frontend panel. | + +--- + +## 3. The workflow + +### 3.1 No collision (the happy path) + +``` +User → POST /api/parse-837 (file) + ← 200 + ParseResult + batch_id +Batch persisted. UI shows parsed claims and links to the new batch. +``` + +### 3.2 Collision (the new path) + +``` +User → POST /api/parse-837 (file) + ← 409 + { + error: "Duplicate claim", + detail: "...", + existing_batch_id: "B123", # most-recent prior batch with a colliding CLM01 + collisions: { + colliding_claim_ids: ["A", "B"], + total_collisions: 2, + total_claims: 141, # claims in the file + new_claims_after_skip: 139, # claims that WOULD be inserted on force + }, + parse_result: { ... full ParseResult ... }, + } +User sees the parse result in the panel. +User can: + - Click "Force insert (skip 2 dups)" → POST /api/parse-837?force=true (same file) + ← 200 + ParseResult + { skipped_claim_ids: ["A", "B"], inserted: 139 } + - Click "Open prior batch" → navigate to /batches/B123 + - Click "Delete prior batch" → DELETE /api/batches/B123, then click "Re-upload" + - Click "Pick a different file" → clear the upload state +``` + +### 3.3 Force-insert after collision + +`force=true` skips the pre-flight check. The store's existing per-row +`s.get(Claim, claim_id)` dedup still skips colliding rows silently, so +the new batch persists with only the non-colliding claims. The response +body includes `skipped_claim_ids` so the UI can show what was skipped. + +`force=true` does NOT bypass the parser. If the file fails validation +(missing diagnosis, malformed segment), the response is still 422. + +### 3.4 Race condition (pre-flight clean, persist fails) + +If a pre-flight dedup check finds no collisions, but a concurrent process +ingests a colliding CLM01 between the check and the persist, the persist +will still raise `IntegrityError`. The handler catches it and returns +**the same 409 shape as the pre-flight collision** with +`existing_batch_id` set to the racing batch and `detail` mentioning +"another process ingested this between the check and the persist — +re-upload to retry". The user re-runs the same flow. + +--- + +## 4. Within-file duplicates + +If the file itself has the same CLM01 twice (a malformed file, not a +cross-batch collision), the pre-flight check catches it the same way: +it returns 409 with `existing_batch_id: null` and `detail: "CLM01 A +appears twice in this file"`. The user can only force-insert (which +skips the second instance). They can't "delete the prior batch" because +there isn't one — it's a bad file. + +--- + +## 5. The 409 body shape + +```json +{ + "error": "Duplicate claim", + "detail": "This file (or one previously ingested with the same claim control number) collides with an existing record. 2 of 141 claims collide with batch B123.", + "batch_id": null, + "existing_batch_id": "B123", + "collisions": { + "colliding_claim_ids": ["A", "B"], + "total_collisions": 2, + "total_claims": 141, + "new_claims_after_skip": 139 + }, + "parse_result": { ... full ParseResult ... } +} +``` + +Field semantics: +- `error`: short tag for the UI ("Duplicate claim", "Duplicate remittance", "Within-file duplicate CLM01"). +- `detail`: human-readable, mentions the count and the existing batch when known. +- `batch_id`: always `null` on 409 (the insert rolled back). +- `existing_batch_id`: the most-recent prior batch that contains a colliding CLM01, or `null` if (a) the collision is within-file, or (b) the colliding claim has since been deleted (race). +- `collisions.colliding_claim_ids`: subset of `parse_result.claims[].claim_id` that collides. +- `collisions.total_claims`: count from `parse_result.summary.total_claims`. +- `collisions.new_claims_after_skip`: `total_claims - total_collisions`. +- `parse_result`: the full `ParseResult` (same shape as a 200 response body). The UI uses this to render the parsed claims list. + +The 200 body on `force=true` adds `skipped_claim_ids: ["A", "B"]` at the top level so the UI can show a "skipped" badge per claim. + +--- + +## 6. `DELETE /api/batches/{id}` + +New endpoint. Cascades through `ON DELETE CASCADE` FKs: + +``` +batches ─┬─ claims ─┬─ matches + │ ├─ activity_events (claim_id) + │ └─ line_reconciliations + ├─ remittances ─┬─ cas_adjustments + │ ├─ service_line_payments + │ └─ activity_events (remittance_id) + └─ activity_events (batch_id only) +``` + +FKs already declare `ON DELETE CASCADE` in the migrations, so the +SQLite engine handles the cascade. The endpoint just needs to +`session.delete(batch_row)` and commit. + +The endpoint: +- `204 No Content` on success. +- `404 Not Found` if the batch doesn't exist. +- `409 Conflict` if the batch has any claims in a non-`submitted` state + (e.g., `paid`, `reversed`, `denied`). Forces the user to first + unreconcile — same as the existing 409 pattern for `manual_match` / + `manual_unmatch` (see `store.py:AlreadyMatchedError`). + +A `batch_deleted` activity event is recorded before the delete so the +audit log has a tombstone. The event's `batch_id` will be `null` after +the cascade (the FK is to `batches.id` with no `ON DELETE` clause +specified in any migration; verify in `migrations/0001_initial.sql` — +the spec says we preserve audit history). If the FK is `ON DELETE +CASCADE`, we record the event AFTER the cascade with `batch_id` set to +the deleted id and rely on the cascade to remove it (acceptable, or we +use a no-cascade FK and keep the tombstone). **Open question resolved +during implementation by reading the actual FK clauses.** + +--- + +## 7. Backend implementation + +### 7.1 New dedup helper + +`backend/src/cyclone/store.py` (already added in `claims-unique-fix`): + +```python +def find_existing_batch_for_claim(claim_id: str) -> str | None: + """Return the batch_id of the first batch containing this claim id, or None. + + Pure read; opens a short-lived session. Used by the 837 409 handler to + surface which prior batch already holds the same CLM01. + + Returns the most-recent batch (ORDER BY parsed_at DESC LIMIT 1) so the + UI links to the most likely "where did the dup come from" answer. + """ + from sqlalchemy import select + from cyclone.db import Claim + with db.SessionLocal()() as s: + row = s.execute( + select(Claim.batch_id) + .where(Claim.id == claim_id) + .order_by(Claim.state_changed_at.desc()) # most-recent touch + .limit(1) + ).first() + return row[0] if row else None + + +def find_existing_batch_for_remit(remit_id: str) -> str | None: + """Same shape as find_existing_batch_for_claim but for remittances.""" + from sqlalchemy import select + from cyclone.db import Remittance + with db.SessionLocal()() as s: + row = s.execute( + select(Remittance.batch_id) + .where(Remittance.id == remit_id) + .order_by(Remittance.received_at.desc()) + .limit(1) + ).first() + return row[0] if row else None +``` + +The current `claims-unique-fix` implementation uses +`select(Claim.batch_id).where(Claim.id == claim_id).limit(1)` without +`ORDER BY`. We replace it with the ordered version to satisfy +"return the most-recent colliding batch". + +### 7.2 New pre-flight dedup check + +`backend/src/cyclone/dedup.py` (new file, single responsibility): + +```python +"""Pre-flight dedup for parsed 837P/835 batches. + +Splits the parsed result into "would-insert" and "would-skip" sets by +querying the DB for any claim_id / payer_claim_control_number already +present. Also detects within-file duplicates by counting claim_id +frequencies. + +Used by the parse-837 and parse-835 endpoints between validation and +persist, so the user can see the parse result + collision summary +before any DB write. +""" +from __future__ import annotations + +from collections import Counter +from dataclasses import dataclass + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from cyclone import db +from cyclone.db import Claim, Remittance + + +@dataclass(frozen=True) +class CollisionReport: + """What the parse endpoint needs to render a 409 response.""" + colliding_claim_ids: list[str] # CLM01s (837) or CLP01s (835) + existing_batch_id: str | None # most-recent prior batch with a collision, or None + within_file_duplicate_ids: list[str] # CLM01s appearing twice in this file (subset of colliding_claim_ids) + total_claims: int + + +def preflight_837(result, session: Session | None = None) -> CollisionReport: + """Detect 837 collisions: within-file dupes + cross-batch CLM01 dupes.""" + claim_ids = [c.claim_id for c in result.claims] + counts = Counter(claim_ids) + within_file_duplicate_ids = sorted( + cid for cid, n in counts.items() if n > 1 + ) + seen: set[str] = set(claim_ids) + if not seen: + return CollisionReport( + colliding_claim_ids=[], + existing_batch_id=None, + within_file_duplicate_ids=[], + total_claims=0, + ) + + own_session = session is None + if own_session: + session = db.SessionLocal()() + try: + rows = session.execute( + select(Claim.id, Claim.batch_id) + .where(Claim.id.in_(seen)) + .order_by(Claim.state_changed_at.desc()) + ).all() + finally: + if own_session: + session.close() + + db_collisions = {cid: bid for cid, bid in rows} + colliding = sorted(cid for cid in seen if cid in db_collisions) + existing_batch_id = next(iter(db_collisions.values()), None) if db_collisions else None + return CollisionReport( + colliding_claim_ids=colliding, + existing_batch_id=existing_batch_id, + within_file_duplicate_ids=within_file_duplicate_ids, + total_claims=len(claim_ids), + ) + + +def preflight_835(result, session: Session | None = None) -> CollisionReport: + """Same shape for 835 remittances. Payer claim control number = CLP01 = remittance.id.""" + pcns = [c.payer_claim_control_number for c in result.claims] + counts = Counter(pcns) + within_file_duplicate_ids = sorted(p for p, n in counts.items() if n > 1) + seen = set(pcns) + if not seen: + return CollisionReport( + colliding_claim_ids=[], + existing_batch_id=None, + within_file_duplicate_ids=[], + total_claims=0, + ) + + own_session = session is None + if own_session: + session = db.SessionLocal()() + try: + rows = session.execute( + select(Remittance.id, Remittance.batch_id) + .where(Remittance.id.in_(seen)) + .order_by(Remittance.received_at.desc()) + ).all() + finally: + if own_session: + session.close() + + db_collisions = {pcn: bid for pcn, bid in rows} + colliding = sorted(pcn for pcn in seen if pcn in db_collisions) + existing_batch_id = next(iter(db_collisions.values()), None) if db_collisions else None + return CollisionReport( + colliding_claim_ids=colliding, + existing_batch_id=existing_batch_id, + within_file_duplicate_ids=within_file_duplicate_ids, + total_claims=len(pcns), + ) +``` + +### 7.3 Modified `parse_837` endpoint + +```python +@app.post("/api/parse-837") +async def parse_837( + request: Request, + file: UploadFile = File(...), + payer: str = Query("co_medicaid"), + include_raw_segments: bool = Query(True), + strict: bool = Query(False), + ack: bool = Query(False), + force: bool = Query(False), # NEW +) -> Any: + # ... existing parse + validate ... + + if _has_claim_validation_errors(result): + return JSONResponse(status_code=422, content=json.loads(result.model_dump_json())) + + # NEW: pre-flight dedup check + if not force and result.claims: + report = dedup.preflight_837(result) + if report.colliding_claim_ids or report.within_file_duplicate_ids: + return _build_409_response( + result=result, + report=report, + error="Duplicate claim", + kind="cross_batch" if report.existing_batch_id else "within_file", + ) + + # Persist (existing path). On IntegrityError (race), same 409 shape. + rec = BatchRecord( + id=uuid.uuid4().hex, + kind="837p", + input_filename=file.filename or "upload.txt", + parsed_at=utcnow(), + result=result, + ) + try: + store.add(rec, event_bus=request.app.state.event_bus) + except IntegrityError as exc: + # Race: pre-flight said clean, but persist hit a PK. Re-run pre-flight + # so the 409 body has the same shape. + report = dedup.preflight_837(result) + return _build_409_response( + result=result, + report=report, + error="Duplicate claim (race condition)", + kind="race", + ) + + # ... existing response ... + if _client_wants_json(request): + body = json.loads(result.model_dump_json()) + if ack: + ack_body = _build_and_persist_ack(rec.id) + if ack_body is not None: + body["ack"] = ack_body + # If force=true, the store.add silently skipped some claims. + # Surface what was skipped so the UI can show a "skipped" badge. + if force: + body["skipped_claim_ids"] = sorted({ + c.claim_id for c in result.claims + if _claim_skipped(c.claim_id, rec.id) + }) + return JSONResponse(content=body) + # ... streaming response ... +``` + +Where: + +```python +def _build_409_response( + result, report, error: str, kind: str +) -> JSONResponse: + """Build the standard 409 body for any dedup failure.""" + if kind == "cross_batch": + detail = ( + f"{len(report.colliding_claim_ids)} of {report.total_claims} " + f"claims collide with prior batch {report.existing_batch_id}. " + f"Force-insert to skip the duplicates, or delete the prior batch." + ) + elif kind == "within_file": + detail = ( + f"CLM01(s) {', '.join(report.within_file_duplicate_ids)} appear " + f"twice in this file. Force-insert will keep the first occurrence " + f"and skip the rest." + ) + else: # race + detail = ( + f"Another process ingested a colliding batch between the check " + f"and the persist. Re-upload to retry with the latest state." + ) + body = { + "error": error, + "detail": detail, + "batch_id": None, + "existing_batch_id": report.existing_batch_id, + "collisions": { + "colliding_claim_ids": report.colliding_claim_ids, + "total_collisions": len(report.colliding_claim_ids), + "total_claims": report.total_claims, + "new_claims_after_skip": report.total_claims - len(report.colliding_claim_ids), + }, + "parse_result": json.loads(result.model_dump_json()), + } + return JSONResponse(status_code=409, content=body) +``` + +`force=true` does NOT bypass validation (still 422 for bad data). It +only bypasses the pre-flight dedup. The `store.add` dedup still skips +colliding claims silently, but the response surfaces the skip list. + +### 7.4 Modified `parse_835` endpoint + +Same pattern, with `dedup.preflight_835` and the 835 parse result. Not +shown in detail; the structure mirrors 837. + +### 7.5 New `DELETE /api/batches/{id}` + +```python +@app.delete("/api/batches/{batch_id}") +def delete_batch(batch_id: str) -> dict: + """Hard-delete a batch and all its child rows. + + Returns 204 on success, 404 if missing, 409 if the batch has any + claims/remits in a non-`submitted` state (must unreconcile first). + """ + from cyclone import db + with db.SessionLocal()() as s: + batch = s.get(db.Batch, batch_id) + if batch is None: + raise HTTPException(404, f"Batch {batch_id} not found") + # Refuse if any claim/remittance is past 'submitted' state + non_submitted = s.execute( + select(db.Claim.id) + .where(db.Claim.batch_id == batch_id) + .where(db.Claim.state != "submitted") + .limit(1) + ).first() + if non_submitted is not None: + raise HTTPException( + 409, + f"Batch {batch_id} has claims in non-submitted state; " + f"unreconcile first before deleting.", + ) + # Record tombstone activity event before the cascade + s.add(db.ActivityEvent( + ts=utcnow(), + kind="batch_deleted", + batch_id=batch_id, + payload_json={"message": f"Batch {batch_id} deleted"}, + )) + s.flush() + s.delete(batch) + s.commit() + return {"ok": True, "batch_id": batch_id} +``` + +The FKs in the schema (`migrations/0001_initial.sql` and later) declare +`ON DELETE CASCADE` on `claims.batch_id`, `remittances.batch_id`, etc. +SQLite handles the cascade at the engine level. We verify this assumption +in the implementation test by deleting a batch with child rows and +asserting the child rows are gone. + +--- + +## 8. Frontend + +### 8.1 `src/lib/api.ts` + +`ApiError` carries more collision data: + +```typescript +export class ApiError extends Error { + constructor( + public status: number, + message: string, + public existingBatchId: string | null = null, + public collisions: CollisionSummary | null = null, + public parseResult: unknown = null, + ) { + super(message); + } +} + +export type CollisionSummary = { + colliding_claim_ids: string[]; + total_collisions: number; + total_claims: number; + new_claims_after_skip: number; +}; +``` + +`parse837` adds `?force=true` to the URL when called for the +"force-insert" action: + +```typescript +export async function parse837( + file: File, + options: { onProgress?: (p: number) => void; force?: boolean } = {}, +): Promise { + const url = `${base}/api/parse-837${options.force ? "?force=true" : ""}`; + // ... existing fetch + body parse ... + if (!res.ok) { + const { message, existingBatchId, collisions, parseResult } = await readErrorBody(res); + throw new ApiError(res.status, message, existingBatchId, collisions, parseResult); + } + return res.json(); +} +``` + +### 8.2 `src/pages/Upload.tsx` + +New state: + +```typescript +type UploadError = { + kind: "duplicate"; + existingBatchId: string | null; + collisions: CollisionSummary; + parseResult: ParseResult; + filename: string; +}; +const [uploadError, setUploadError] = useState(null); +const [forceInserting, setForceInserting] = useState(false); +``` + +Panel JSX (above the streaming results): + +```tsx +{uploadError ? ( +
+
+ + 409 + + + {uploadError.collisions.total_collisions} of {uploadError.collisions.total_claims} claims + collide + {uploadError.existingBatchId + ? ` with batch ${uploadError.existingBatchId}` + : " within this file"} + +
+ +

+ File {uploadError.filename} would persist + {" "}{uploadError.collisions.new_claims_after_skip} of {uploadError.collisions.total_claims} claims. + Colliding CLM01s: {uploadError.collisions.colliding_claim_ids.join(", ")}. +

+ +
+ + + {uploadError.existingBatchId ? ( + <> + + + + ) : null} + + +
+ + {/* The full parse result is rendered below so the user can see what was parsed. */} +
+ Show parsed claims ({uploadError.parseResult.claims.length}) + 
+        {JSON.stringify(uploadError.parseResult.summary, null, 2)}
+
+
+
+) : null} +``` + +--- + +## 9. Database + +Migration 0013 already exists on the `claims-unique-fix` worktree. It +drops the `UNIQUE(batch_id, patient_control_number)` inline constraint. +After it runs: + +- The 409 fires only on actual CLM01 collisions (not the `member_id` + dedup that was over-constraining before). +- Multi-claim 837P files with shared `member_id` ingest cleanly for + the first time. + +No new tables. No new columns. The DELETE endpoint relies on existing +`ON DELETE CASCADE` FKs. + +--- + +## 10. Files changed + +| File | Change | +|---|---| +| `backend/src/cyclone/migrations/0013_drop_claims_unique_constraint.sql` | new (DONE on `claims-unique-fix`) | +| `backend/src/cyclone/store.py` | new `find_existing_batch_for_claim` / `find_existing_batch_for_remit` (DONE) + new `delete_batch` method | +| `backend/src/cyclone/dedup.py` | new file: pre-flight `preflight_837` / `preflight_835` + `CollisionReport` dataclass | +| `backend/src/cyclone/api.py` | 837/835 endpoints: pre-flight check, force param, new 409 body, race handler, new DELETE endpoint | +| `src/lib/api.ts` | `ApiError` adds `collisions` + `parseResult`; `parse837`/`parse835` accept `force`; new `deleteBatch` | +| `src/pages/Upload.tsx` | new `UploadError` state, error panel JSX, force-insert handler, delete-prior handler | +| `src/pages/Upload.test.tsx` | new tests (4 cases from §11) | +| `backend/tests/test_db_migrate.py` | 0013 idempotency + UNIQUE-dropped tests (DONE) | +| `backend/tests/test_store.py` | `find_existing_batch_for_claim`/`remit` tests (DONE) | +| `backend/tests/test_dedup.py` | new tests for `preflight_837` / `preflight_835` (§11) | +| `backend/tests/test_api_parse_persists.py` | new tests: pre-flight 409, force-insert, within-file 409, race 409, DELETE endpoint (§11) | + +No new dependencies. No config changes. + +--- + +## 11. Test plan + +### Backend (pytest) + +| Test | File | Asserts | +|---|---|---| +| `test_preflight_837_finds_no_collisions_on_empty_db` | `test_dedup.py` | empty DB → empty `colliding_claim_ids`, no `existing_batch_id` | +| `test_preflight_837_finds_cross_batch_collision` | `test_dedup.py` | pre-seed a claim; pre-flight returns that claim_id in `colliding_claim_ids` and the seeded batch in `existing_batch_id` | +| `test_preflight_837_finds_within_file_duplicate` | `test_dedup.py` | parsed result has the same CLM01 twice; pre-flight returns it in both `colliding_claim_ids` and `within_file_duplicate_ids` | +| `test_preflight_837_returns_most_recent_batch_id` | `test_dedup.py` | pre-seed 3 batches with the same CLM01 at different times; pre-flight returns the most-recent batch_id | +| `test_preflight_835_mirrors_837` | `test_dedup.py` | same shape for remittances | +| `test_parse_837_409_includes_parse_result_and_collisions` | `test_api_parse_persists.py` | pre-seed a claim; upload a file with a colliding CLM01; assert 409 with `parse_result`, `collisions.colliding_claim_ids`, `existing_batch_id` | +| `test_parse_837_409_within_file_duplicate_has_null_batch_id` | `test_api_parse_persists.py` | upload a file with the same CLM01 twice; assert 409 with `existing_batch_id: null` and `within_file_duplicate_ids` populated | +| `test_parse_837_force_true_persists_non_colliding_claims` | `test_api_parse_persists.py` | pre-seed a claim; upload a file with 3 claims, 1 colliding; assert 200 with `skipped_claim_ids: [colliding_id]`, the 2 non-colliding claims persist | +| `test_parse_837_force_true_does_not_bypass_validation` | `test_api_parse_persists.py` | a file that fails validation still returns 422 with `force=true` | +| `test_parse_837_race_409_uses_same_body_shape` | `test_api_parse_persists.py` | mock `store.add` to raise IntegrityError; assert 409 body has the same shape as the pre-flight 409 | +| `test_delete_batch_cascades_to_claims` | `test_api_parse_persists.py` | persist a batch with 2 claims; DELETE; assert batch and both claims are gone | +| `test_delete_batch_404_on_unknown` | `test_api_parse_persists.py` | DELETE /api/batches/does-not-exist → 404 | +| `test_delete_batch_409_on_reconciled_claims` | `test_api_parse_persists.py` | persist a batch, mark a claim state='paid'; DELETE → 409 | +| `test_parse_837_after_delete_succeeds` | `test_api_parse_persists.py` | pre-seed a colliding claim; DELETE that batch; re-upload the same file; assert 200 | + +### Frontend (vitest) + +| Test | File | Asserts | +|---|---|---| +| `test_error_panel_renders_on_409_with_collisions` | `Upload.test.tsx` | mock `parse837` to throw `ApiError(409, ..., PRIOR, collisions, parseResult)`; assert panel visible with all collision data | +| `test_force_insert_button_re_calls_with_force_true` | `Upload.test.tsx` | user clicks "Force insert"; assert `parse837` is called with `{ force: true }` | +| `test_delete_prior_button_calls_deleteBatch` | `Upload.test.tsx` | user clicks "Delete prior batch"; assert `deleteBatch(existingBatchId)` is called | +| `test_pick_different_clears_error` | `Upload.test.tsx` | user clicks "Pick a different file"; assert `uploadError` is cleared and file picker is reset | +| `test_no_panel_on_non_409` | `Upload.test.tsx` | 400 error; assert panel absent | +| `test_within_file_duplicate_omits_prior_batch_actions` | `Upload.test.tsx` | 409 with `existingBatchId: null`; assert "Open prior batch" and "Delete prior batch" buttons are absent | + +--- + +## 12. Out of scope + +* Batch editing (update claim state, edit claim fields). Future SP. +* Cross-batch dedup REPORT (a "find all CLM01s in batches B1+B2+B3" + query). Future SP. +* Migration reversibility for 0013 — the recreation preserves data but + not schema history. Acceptable since 0013 just drops an inline + constraint; recreating the constraint would be a separate migration. +* Audit event for force-insert skips. The user explicitly chose to + skip silently; we honor that. + +--- + +## 13. Risk + +* **Pre-flight check race**: between the check and the persist, a + concurrent process could ingest a colliding claim. The persist would + then raise `IntegrityError`; the handler returns the same 409 shape + with `detail` mentioning the race. The user re-runs. Acceptable. +* **DELETE on a large batch**: cascade through `claims`, `remittances`, + `matches`, `line_reconciliations`, `activity_events`. SQLite handles + the cascade in a single transaction; a 140-claim batch deletes in + <100ms. The endpoint refuses if any claim is past `submitted` state. +* **`force=true` silent skip**: the user clicks "Force insert" and + the response says "X of Y claims persisted, Z skipped". They + acknowledged this in the panel before clicking. No undo. +* **Within-file duplicates and force-insert**: the user can force-insert + a file with the same CLM01 twice. The first instance persists, the + second is silently skipped. This is intentional — within-file dupes + are usually a typo, and the user has explicitly asked to proceed. +* **`existing_batch_id` may be stale**: the helper returns the + most-recent batch by `state_changed_at` (or `received_at` for 835). + The user clicks "Open prior batch" and the batch may have been + deleted in the meantime. The BatchesList page already handles 404 + gracefully. + +--- + +## 14. Rollout + +1. **Schema**: migration 0013 applies on next `cyclone` startup. + Idempotent and reversible only by rebuilding the `claims` table + (acceptable; production data preserved by the INSERT...SELECT). +2. **Backend API**: new `force` param + new 409 body shape + new + DELETE endpoint. Existing clients that don't pass `force` see the + same behavior as before for collision-free files. Collision cases + now get a richer 409 body that includes `parse_result`; clients + that ignore the new fields keep working. +3. **Frontend**: `Upload.tsx` panel replaces the toast on 409. Users + who don't read the panel still see the toast and the 409 message + in the streaming view. +4. **No data migration**: nothing to migrate. 0013 is structural only.