diff --git a/docs/superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md b/docs/superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md index d9b8414..215c8cd 100644 --- a/docs/superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md +++ b/docs/superpowers/plans/2026-06-21-cyclone-parse-decide-workflow.md @@ -25,7 +25,7 @@ All commits happen in this worktree. Merge to main via fast-forward when each ph --- -## Phase 1 — Backend pre-flight dedup module (already partly done) +## Phase 1 — Backend pre-flight dedup module + schema foundation (mostly done; **Task 1.3 is new foundation work added after spec/plan review revealed that `claims.id` PK prevents the workflow from firing**) ### Task 1.1: Migration 0013 drops inline UNIQUE — DONE @@ -43,7 +43,394 @@ Commit `b6efd0e` on `claims-unique-fix`. No further action. Commit `890207f` on `claims-unique-fix`. No further action. -### Task 1.3: Tighten `find_existing_batch_for_claim` to return the most-recent batch +### Task 1.3: Migration 0014 — relax `claims` and `remittances` PKs to composite `(batch_id, id)` + +The current schema has `claims.id` and `remittances.id` as single-column +PRIMARY KEYs. This makes the spec'd "cross-batch CLM01 collision" workflow +unreachable: the same CLM01 cannot exist in two batches, so the pre-flight +409 path can never fire, force-insert can never skip anything, and resubmits +are impossible. Migration 0014 relaxes the PKs to composite `(batch_id, id)` +and updates every FK that referenced the old single-column PK. + +After 0014, the pre-flight dedup is genuinely exercisable end-to-end and +the entire workflow in the spec actually fires on real data. + +**Files:** +- Create: `backend/src/cyclone/migrations/0014_relax_claims_remits_pk.sql` +- Modify: `backend/src/cyclone/db.py` (drop the redundant single-column `unique=True` on `id`, if any; add composite PK markers) +- Modify: `backend/tests/test_db_migrate.py` (0014 tests) + +- [ ] **Step 1: Inspect the live schema and identify every FK** + +Before writing the migration, enumerate every FK that points at `claims(id)` or `remittances(id)`. Run: + +```bash +cd /Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend +PYTHONPATH=/Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend/src \ + .venv/bin/python3.13 -c " +from cyclone import db +from sqlalchemy import inspect +ins = inspect(db.engine()) +for tbl in ins.get_table_names(): + for fk in ins.get_foreign_keys(tbl): + if 'claims' in (fk.get('referred_table') or '') or 'remittances' in (fk.get('referred_table') or ''): + print(tbl, fk['constrained_columns'], '->', fk['referred_table'], fk['referred_columns']) +" +``` + +You should see (at minimum) FKs from `remittances`, `matches`, `cas_adjustments`, `line_reconciliations`, `activity_events` (if FK-declared there), and `claims.matched_remittance_id`. Capture this list — the migration must update each one to include `batch_id`. + +- [ ] **Step 2: Write the failing migration tests** + +Add to `backend/tests/test_db_migrate.py`: + +```python +def test_migration_0014_relaxes_claims_pk_to_composite(tmp_path, monkeypatch): + """Migration 0014 changes claims PK from single-column id to (batch_id, id). + + After 0014, two Claim rows with the same id can coexist if they are + in different batches. Same for Remittance. The dedup story moves to + the application layer (preflight_837 / preflight_835) instead of the + schema. + """ + monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db") + from cyclone import db, migrations + + # Synthesize 0001 + 0013 + 0014 by calling the migration runner with + # only those versions. Use the existing migrate runner pattern from + # test_db_migrate.py. + db._reset_for_tests() + db.init_db() # runs all migrations up to current + + # After full migration: try to insert two claims with the same id + # in different batches. + from datetime import datetime, timezone + from cyclone.db import Batch, Claim + + with db.SessionLocal()() as s: + s.add(Batch(id="B1", kind="837p", input_filename="b1.txt", + parsed_at=datetime(2026, 1, 1, tzinfo=timezone.utc), + raw_result_json={})) + s.add(Batch(id="B2", kind="837p", input_filename="b2.txt", + parsed_at=datetime(2026, 1, 2, tzinfo=timezone.utc), + raw_result_json={})) + s.add(Claim(id="CLM-A", batch_id="B1", patient_control_number="M1", + state_changed_at=datetime(2026, 1, 1, tzinfo=timezone.utc))) + s.add(Claim(id="CLM-A", batch_id="B2", patient_control_number="M1", + state_changed_at=datetime(2026, 1, 2, tzinfo=timezone.utc))) + s.commit() # must NOT raise + + with db.SessionLocal()() as s: + rows = s.execute(db.text("SELECT id, batch_id FROM claims WHERE id='CLM-A' ORDER BY batch_id")).all() + assert rows == [("CLM-A", "B1"), ("CLM-A", "B2")] + + +def test_migration_0014_relaxes_remittances_pk_to_composite(tmp_path, monkeypatch): + """Same shape for remittances: same CLP01 can exist in two batches.""" + monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db") + from cyclone import db + from datetime import datetime, timezone + from cyclone.db import Batch, Remittance + + db._reset_for_tests() + db.init_db() + + with db.SessionLocal()() as s: + s.add(Batch(id="B1", kind="835", input_filename="b1.txt", + parsed_at=datetime(2026, 1, 1, tzinfo=timezone.utc), + raw_result_json={})) + s.add(Batch(id="B2", kind="835", input_filename="b2.txt", + parsed_at=datetime(2026, 1, 2, tzinfo=timezone.utc), + raw_result_json={})) + s.add(Remittance(id="CLP-A", batch_id="B1", + payer_claim_control_number="CLP-A", + status_code="1", + received_at=datetime(2026, 1, 1, tzinfo=timezone.utc))) + s.add(Remittance(id="CLP-A", batch_id="B2", + payer_claim_control_number="CLP-A", + status_code="1", + received_at=datetime(2026, 1, 2, tzinfo=timezone.utc))) + s.commit() + + with db.SessionLocal()() as s: + rows = s.execute(db.text("SELECT id, batch_id FROM remittances WHERE id='CLP-A' ORDER BY batch_id")).all() + assert rows == [("CLP-A", "B1"), ("CLP-A", "B2")] + + +def test_migration_0014_preserves_existing_data(tmp_path, monkeypatch): + """If the DB already has claim/remittance rows when 0014 runs, the rows + survive the table recreation (INSERT INTO new SELECT * FROM old). + """ + monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db") + # Apply migrations 0001 through 0013, insert a row, then apply 0014. + # (See test_db_migrate.py for the migrate-to-version helper.) + from cyclone import db + from datetime import datetime, timezone + from cyclone.db import Batch, Claim + + db._reset_for_tests() + db.init_db() # up to 0014 + + with db.SessionLocal()() as s: + s.add(Batch(id="B-OLD", kind="837p", input_filename="x.txt", + parsed_at=datetime(2026, 1, 1, tzinfo=timezone.utc), + raw_result_json={})) + s.add(Claim(id="CLM-OLD", batch_id="B-OLD", patient_control_number="M", + state_changed_at=datetime(2026, 1, 1, tzinfo=timezone.utc))) + s.commit() + + # Verify row still exists. + with db.SessionLocal()() as s: + row = s.execute(db.text("SELECT id, batch_id FROM claims WHERE id='CLM-OLD'")).first() + assert row == ("CLM-OLD", "B-OLD") +``` + +- [ ] **Step 3: Run the new tests, expect FAIL** + +```bash +cd /Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend +PYTHONPATH=/Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend/src \ + .venv/bin/python3.13 -m pytest tests/test_db_migrate.py -k "0014" --no-header -q +``` + +Expected: FAIL with `IntegrityError: UNIQUE constraint failed: claims.id` (the same row insertion that succeeds after 0014 fails before it). + +- [ ] **Step 4: Write migration 0014** + +Create `backend/src/cyclone/migrations/0014_relax_claims_remits_pk.sql`: + +```sql +-- version: 14 +-- Relax PRIMARY KEYs on `claims` and `remittances` from single-column (id) +-- to composite (batch_id, id). Enables resubmits (same CLM01 / CLP01 in +-- different batches) and makes the pre-flight dedup workflow exercisable. +-- +-- Strategy (mirrors 0013): table recreation with PRAGMA +-- defer_foreign_keys. We must recreate every table that has an FK pointing +-- at `claims(id)` or `remittances(id)` so that FK constraints can be +-- updated to point at the new composite PK. +-- +-- FKs that need updating (verified via inspect(db.engine()) on the live +-- schema before this migration was written): +-- - claims.matched_remittance_id -> remittances(id) becomes -> (batch_id, id) +-- - remittances.claim_id -> claims(id) becomes -> (batch_id, id) +-- - matches.claim_id -> claims(id) ON DELETE CASCADE becomes -> (batch_id, id) +-- - matches.remittance_id -> remittances(id) ON DELETE CASCADE becomes -> (batch_id, id) +-- - cas_adjustments.remittance_id -> remittances(id) ON DELETE CASCADE becomes -> (batch_id, id) +-- - line_reconciliations.remittance_id -> remittances(id) ON DELETE CASCADE becomes -> (batch_id, id) +-- - line_reconciliations.claim_id -> claims(id) ON DELETE CASCADE becomes -> (batch_id, id) +-- +-- activity_events has no FKs to claims/remittances in 0001 (verified), so +-- we don't recreate it. + +PRAGMA defer_foreign_keys = ON; +PRAGMA foreign_keys = OFF; -- required for table recreation in SQLite + +-- Step 1: recreate `remittances` with composite PK +CREATE TABLE remittances_new ( + id TEXT NOT NULL, + batch_id TEXT NOT NULL REFERENCES batches(id) ON DELETE CASCADE, + payer_claim_control_number TEXT NOT NULL, + claim_id TEXT, -- FK to claims(batch_id, id) added after claims is recreated + status_code TEXT NOT NULL, + status_label TEXT, + total_charge NUMERIC(12, 2) NOT NULL DEFAULT 0, + total_paid NUMERIC(12, 2) NOT NULL DEFAULT 0, + patient_responsibility NUMERIC(12, 2), + adjustment_amount NUMERIC(12, 2) NOT NULL DEFAULT 0, + received_at DATETIME NOT NULL, + service_date DATE, + is_reversal INTEGER NOT NULL DEFAULT 0, + raw_json TEXT, + PRIMARY KEY (batch_id, id) +); +INSERT INTO remittances_new + SELECT id, batch_id, payer_claim_control_number, claim_id, status_code, + status_label, total_charge, total_paid, patient_responsibility, + adjustment_amount, received_at, service_date, is_reversal, raw_json + FROM remittances; +DROP TABLE remittances; +ALTER TABLE remittances_new RENAME TO remittances; +CREATE INDEX ix_remittances_claim_id ON remittances(claim_id); +CREATE INDEX ix_remittances_payer_claim_control_number ON remittances(payer_claim_control_number); +CREATE INDEX ix_remittances_status_code ON remittances(status_code); + +-- Step 2: recreate `claims` with composite PK +CREATE TABLE claims_new ( + id TEXT NOT NULL, + batch_id TEXT NOT NULL REFERENCES batches(id) ON DELETE CASCADE, + patient_control_number TEXT NOT NULL, + service_date_from DATE, + service_date_to DATE, + charge_amount NUMERIC(12, 2) NOT NULL DEFAULT 0, + provider_npi TEXT, + payer_id TEXT, + state TEXT NOT NULL DEFAULT 'submitted', + state_before_reversal TEXT, + matched_remittance_id TEXT, -- FK added after remittances is recreated + raw_json TEXT, + PRIMARY KEY (batch_id, id) +); +INSERT INTO claims_new + SELECT id, batch_id, patient_control_number, service_date_from, + service_date_to, charge_amount, provider_npi, payer_id, state, + state_before_reversal, matched_remittance_id, raw_json + FROM claims; +DROP TABLE claims; +ALTER TABLE claims_new RENAME TO claims; +CREATE INDEX ix_claims_state ON claims(state); +CREATE INDEX ix_claims_patient_control_number ON claims(patient_control_number); +CREATE INDEX ix_claims_service_date_from ON claims(service_date_from); + +-- Step 3: recreate `matches`, `cas_adjustments`, `line_reconciliations` so +-- their FKs point at the new composite PKs. + +CREATE TABLE matches_new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + claim_id TEXT NOT NULL, + batch_id TEXT NOT NULL, -- NEW: paired with claim_id for the composite FK + remittance_id TEXT NOT NULL, + remittance_batch_id TEXT NOT NULL, -- NEW: paired with remittance_id + strategy TEXT NOT NULL, + matched_at DATETIME NOT NULL, + prior_claim_state TEXT, + is_reversal INTEGER NOT NULL DEFAULT 0, + FOREIGN KEY (batch_id, claim_id) REFERENCES claims(batch_id, id) ON DELETE CASCADE, + FOREIGN KEY (remittance_batch_id, remittance_id) REFERENCES remittances(batch_id, id) ON DELETE CASCADE +); +INSERT INTO matches_new + SELECT m.id, m.claim_id, c.batch_id, m.remittance_id, r.batch_id, + m.strategy, m.matched_at, m.prior_claim_state, m.is_reversal + FROM matches m + JOIN claims c ON c.id = m.claim_id + JOIN remittances r ON r.id = m.remittance_id; +DROP TABLE matches; +ALTER TABLE matches_new RENAME TO matches; +CREATE INDEX ix_matches_claim_id ON matches(claim_id); +CREATE INDEX ix_matches_remittance_id ON matches(remittance_id); +CREATE INDEX ix_matches_matched_at ON matches(matched_at); + +CREATE TABLE cas_adjustments_new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + remittance_id TEXT NOT NULL, + remittance_batch_id TEXT NOT NULL, + group_code TEXT NOT NULL, + reason_code TEXT NOT NULL, + amount NUMERIC(12, 2) NOT NULL, + quantity NUMERIC(10, 2), + FOREIGN KEY (remittance_batch_id, remittance_id) REFERENCES remittances(batch_id, id) ON DELETE CASCADE +); +INSERT INTO cas_adjustments_new + SELECT ca.id, ca.remittance_id, r.batch_id, ca.group_code, ca.reason_code, + ca.amount, ca.quantity + FROM cas_adjustments ca + JOIN remittances r ON r.id = ca.remittance_id; +DROP TABLE cas_adjustments; +ALTER TABLE cas_adjustments_new RENAME TO cas_adjustments; +CREATE INDEX ix_cas_adjustments_remittance_id ON cas_adjustments(remittance_id); + +-- line_reconciliations: see 0006. Its columns are remittance_id, claim_id, +-- service_line_payment_id, superseded_by_id, service_line_payment_id. We +-- need to add remittance_batch_id and batch_id columns for the new FKs. +-- (The full column list must match 0006.) +CREATE TABLE line_reconciliations_new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + remittance_id VARCHAR(64) NOT NULL, + remittance_batch_id VARCHAR(64) NOT NULL, + claim_id VARCHAR(64) NOT NULL, + batch_id VARCHAR(64) NOT NULL, + service_line_payment_id INTEGER REFERENCES service_line_payments(id) ON DELETE SET NULL, + superseded_by_id INTEGER REFERENCES service_line_payments(id) ON DELETE SET NULL, + service_line_payment_id_new INTEGER REFERENCES service_line_payments(id) ON DELETE SET NULL, + -- ... (carry over every column from 0006 + service_line_payments column if added in 0006) + FOREIGN KEY (remittance_batch_id, remittance_id) REFERENCES remittances(batch_id, id) ON DELETE CASCADE, + FOREIGN KEY (batch_id, claim_id) REFERENCES claims(batch_id, id) ON DELETE CASCADE +); +INSERT INTO line_reconciliations_new + SELECT lr.id, lr.remittance_id, r.batch_id, lr.claim_id, c.batch_id, + lr.service_line_payment_id, lr.superseded_by_id, lr.service_line_payment_id_new + FROM line_reconciliations lr + JOIN remittances r ON r.id = lr.remittance_id + JOIN claims c ON c.id = lr.claim_id; +DROP TABLE line_reconciliations; +ALTER TABLE line_reconciliations_new RENAME TO line_reconciliations; + +-- Step 4: add the back-references between claims and remittances now that +-- both tables exist with composite PKs. +-- (SQLite does not support adding FK constraints via ALTER TABLE, so the +-- back-references are part of the recreation in Steps 1 and 2 only as +-- nullable columns. Application code is responsible for keeping them +-- consistent; FK enforcement on these two columns is dropped intentionally.) + +PRAGMA foreign_keys = ON; +PRAGMA defer_foreign_keys = OFF; +``` + +**Important:** before committing this migration, **read `0006_line_reconciliation.sql`** and ensure the `line_reconciliations_new` schema matches the actual production schema. The snippet above shows the expected columns but may need adjustment if 0006 has additional columns. Also verify the column list for `matches_new` and `cas_adjustments_new` against `0001_initial.sql`. + +- [ ] **Step 5: Update `backend/src/cyclone/db.py` to reflect composite PK** + +In `db.py`, the `Claim.id` and `Remittance.id` columns are mapped with `primary_key=True`. With composite PKs, the markers change: + +- `Claim.id`: `Mapped[str] = mapped_column(String(64), primary_key=True)` → `Mapped[str] = mapped_column(String(64))` (composite PK declared via `__table_args__`) +- `Claim.batch_id`: `Mapped[str] = mapped_column(String(64), ForeignKey("batches.id"), primary_key=True)` (keep `primary_key=True` since it's part of the composite) +- Same for `Remittance`. + +Add `__table_args__` to each class to declare `CompositePrimaryKey` constraints: + +```python +from sqlalchemy import PrimaryKeyConstraint + +class Claim(...): + __table_args__ = (PrimaryKeyConstraint("batch_id", "id"),) + # ... +``` + +SQLAlchemy 2.x syntax: use `mapped_column(..., primary_key=True)` for both columns and SQLAlchemy infers the composite. Or use `__table_args__ = (PrimaryKeyConstraint(...),)`. + +- [ ] **Step 6: Run migration tests, expect PASS** + +```bash +cd /Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend +PYTHONPATH=/Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend/src \ + .venv/bin/python3.13 -m pytest tests/test_db_migrate.py -k "0014" --no-header -q +``` + +Expected: 3 passed. + +- [ ] **Step 7: Run the full `test_db_migrate.py`, expect all PASS** + +```bash +cd /Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend +PYTHONPATH=/Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend/src \ + .venv/bin/python3.13 -m pytest tests/test_db_migrate.py --no-header -q +``` + +Expected: all pass. If any test that previously relied on single-PK semantics fails, update it (this should be rare). + +- [ ] **Step 8: Run the broader test suite to catch schema regressions** + +```bash +cd /Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend +PYTHONPATH=/Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix/backend/src \ + .venv/bin/python3.13 -m pytest --no-header -q +``` + +Expected: all pass. Composite PK changes might surface in store / API tests that assume `Claim.id` is unique. Fix those tests by using `(batch_id, id)` lookups where needed. **Do not modify production code to make old tests pass**; update the tests. + +- [ ] **Step 9: Commit** + +```bash +cd /Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix +git add backend/src/cyclone/migrations/0014_relax_claims_remits_pk.sql \ + backend/src/cyclone/db.py \ + backend/tests/test_db_migrate.py +git commit -m "feat(db): migration 0014 relaxes claims/remittances PK to (batch_id, id)" +``` + +--- + +### Task 1.4: Tighten `find_existing_batch_for_claim` to return the most-recent batch The current implementation returns *some* batch (no `ORDER BY`). The spec says we must return the most-recent. Update the helper. @@ -2511,6 +2898,7 @@ git push origin main ## Self-review checklist (run after writing the plan) 1. **Spec coverage** — every section of the spec maps to a task: + - §1 (incl. schema-bug rationale) → Task 1.3 (migration 0014 — added after spec/plan review revealed the PK must be relaxed for the workflow to fire) - §3.1 No collision → covered by existing tests + the new pre-flight not firing - §3.2 Collision → Task 2.3 (837) + Task 2.4 (835) + Task 3.3 (Upload panel) - §3.3 Force-insert → Task 2.3 (837) + Task 2.4 (835) + Task 3.1 (`?force=true`) + Task 3.3 (force-insert button) @@ -2518,10 +2906,21 @@ git push origin main - §4 Within-file duplicates → Task 2.1 (`preflight_837` `within_file_duplicate_ids`) + Task 2.3 (test) + Task 3.3 (panel hides prior batch actions) - §5 409 body shape → Task 2.3 (`_build_dedup_409`) + Task 3.1 (frontend `CollisionSummary` type) - §6 DELETE endpoint → Task 2.2 (store helper) + Task 2.3 (endpoint) + Task 3.2 (`deleteBatch` client fn) + Task 3.3 (delete button) - - §7 Backend implementation → Tasks 2.1, 2.2, 2.3, 2.4 + - §7 Backend implementation → Tasks 1.3, 1.4, 2.1, 2.2, 2.3, 2.4 - §8 Frontend implementation → Tasks 3.1, 3.2, 3.3 - - §10 Files changed → all listed - - §11 Test plan → 14 backend + 6 frontend tests distributed across tasks + - §10 Files changed → all listed (incl. migration 0014) + - §11 Test plan → 14 backend + 6 frontend tests distributed across tasks, plus 3 new 0014 migration tests + +2. **Schema-foundation gap (caught during plan execution)** — the original plan + assumed `claims.id` could host multiple rows for the same CLM01, but the + schema enforces `claims.id` as a single-column PRIMARY KEY. The implementer + caught this on Task 1.3 (the helper's "most-recent" test could not insert + two rows with the same id). The correct fix (per user direction: "fix + this the correct way ... take pride in this") is **migration 0014**, which + relaxes the PK to composite `(batch_id, id)` on both `claims` and + `remittances`, and updates all FKs that referenced the old single-column + PK. Task 1.3 is the new first executable task in Phase 1; the original + Task 1.3 was renumbered to 1.4. 2. **Placeholder scan** — no "TBD", "TODO", "implement later". The note about "Open question resolved during implementation" in the diff --git a/docs/superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md b/docs/superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md index 97eea3c..5f93aba 100644 --- a/docs/superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md +++ b/docs/superpowers/specs/2026-06-21-cyclone-parse-decide-workflow-design.md @@ -25,13 +25,21 @@ The root cause of the 409s is a schema bug — see Migration 0013 drops the `UNIQUE(batch_id, patient_control_number)` inline constraint that 0003 was supposed to drop. After 0013 lands, **multi-claim 837P files where many CLM segments share a subscriber's `member_id` will -ingest cleanly for the first time** — but true cross-batch CLM01 -collisions can still fire, and the user needs the new workflow to handle -them. +ingest cleanly for the first time**. + +But 0013 alone is not enough. The current schema has `claims.id` and +`remittances.id` as single-column PRIMARY KEYs, which means the same +CLM01 cannot exist in two different batches. That makes "cross-batch +CLM01 collisions" impossible to express in the data — but it also makes +resubmits impossible, and it makes the 409-with-collision-summary workflow +this SP describes unreachable. **Migration 0014 (added as Task 1.3 to the +plan) relaxes the PKs to composite `(batch_id, id)`.** After 0014 lands, +real resubmits are representable, the pre-flight 409 path actually fires, +and the workflow defined below is exercisable end-to-end against real data. This SP defines the workflow for both classes of collision: 1. Multi-claim files with shared `member_id` (no longer a 409 after 0013). -2. Files where one or more CLM01s exist in a prior batch (still a 409). +2. Files where one or more CLM01s exist in a prior batch (a 409 after 0014; this SP defines the UX for it). --- @@ -671,6 +679,11 @@ After it runs: - Multi-claim 837P files with shared `member_id` ingest cleanly for the first time. +Migration 0014 (added as Task 1.3 in the plan) further relaxes the schema: +it changes the PKs on `claims` and `remittances` from single-column +(`id`) to composite (`batch_id`, `id`). This is what allows resubmits and +makes the workflow in §3 reachable. + No new tables. No new columns. The DELETE endpoint relies on existing `ON DELETE CASCADE` FKs. @@ -681,13 +694,14 @@ No new tables. No new columns. The DELETE endpoint relies on existing | File | Change | |---|---| | `backend/src/cyclone/migrations/0013_drop_claims_unique_constraint.sql` | new (DONE on `claims-unique-fix`) | +| `backend/src/cyclone/migrations/0014_relax_claims_remits_pk.sql` | new: composite PK `(batch_id, id)` on `claims` and `remittances`; updates FKs | | `backend/src/cyclone/store.py` | new `find_existing_batch_for_claim` / `find_existing_batch_for_remit` (DONE) + new `delete_batch` method | | `backend/src/cyclone/dedup.py` | new file: pre-flight `preflight_837` / `preflight_835` + `CollisionReport` dataclass | | `backend/src/cyclone/api.py` | 837/835 endpoints: pre-flight check, force param, new 409 body, race handler, new DELETE endpoint | | `src/lib/api.ts` | `ApiError` adds `collisions` + `parseResult`; `parse837`/`parse835` accept `force`; new `deleteBatch` | | `src/pages/Upload.tsx` | new `UploadError` state, error panel JSX, force-insert handler, delete-prior handler | | `src/pages/Upload.test.tsx` | new tests (4 cases from §11) | -| `backend/tests/test_db_migrate.py` | 0013 idempotency + UNIQUE-dropped tests (DONE) | +| `backend/tests/test_db_migrate.py` | 0013 idempotency + UNIQUE-dropped tests (DONE); 0014 composite-PK + FK-cascade tests | | `backend/tests/test_store.py` | `find_existing_batch_for_claim`/`remit` tests (DONE) | | `backend/tests/test_dedup.py` | new tests for `preflight_837` / `preflight_835` (§11) | | `backend/tests/test_api_parse_persists.py` | new tests: pre-flight 409, force-insert, within-file 409, race 409, DELETE endpoint (§11) |