cyclone/docs/superpowers/specs/2026-07-07-cyclone-2010bb-nm109-fix-design.md

Sub-project 39 — 2010BB NM109 byte defect fix: Design Spec

Date: 2026-07-07 Status: Draft, awaiting user sign-off Branch: sp39-2010bb-nm109-fix Aesthetic direction: No new UI; one serializer normalization helper + regression tests + regen script.

1. Scope

On 2026-07-07, an audit of Cyclone's outbound 837P files surfaced a byte-level defect in loop 2010BB (payer): the NM1*PR*2*<name>*****PI*<id> segment was being emitted with NM109 = "SKCO0" (and NM103 = "COHCPF") for CO Medicaid claims, where Gainwell's MOVEit Transfer SFTP requires NM109 = "CO_TXIX" or "CO_BHA". The payer rejected four batches at the SET level with "2010BB NM109 must equal CO_TXIX or CO_BHA". SP33 already corrected the canonical PayerConfig.co_medicaid() payer id going forward, but the serializer's _build_payer_block still emits whatever claim.payer.id it is handed — so any pre-SP33 ClaimOutput rows in the DB whose raw_json captured payer.id = "SKCO0" will round-trip the defect on re-serialize.

A 363-file pre-correction set already exists at ingest/corrected/batch-*/ (regenerated earlier with an upstream raw['payer']['id'] workaround in regen_837s.py), plus four original outbound 837s at ingest/tp11525703-837P-20260701*.txt that retain the defect. The SP-N lands a defense-in-depth fix inside the serializer itself so the workaround can be removed and so any future caller that hands the serializer a stale SKCO0 (or empty) payer id still emits a byte that Gainwell accepts.

In scope:

• A new private helper _normalize_payer_id(payer) in backend/src/cyclone/parsers/serialize_837.py that substitutes "CO_TXIX" for payer.id whenever the value is empty, "SKCO0", or "CO_BHA". The substitution also aligns payer.name to "CO_TXIX" when the original payer.name was the legacy "COHCPF", "CO_BHA", or empty, so the regenerated NM103 stays consistent with the substituted NM109. Foreign payer IDs (anything not in the substitution set) are emitted verbatim — the helper only normalizes CO Medicaid-shape values.
• A WARNING log emitted by _build_payer_block whenever the helper substitutes a value, carrying the substituted-from and substituted-to strings (one log per affected call; the serializer is invoked once per claim so the volume is bounded by batch size).
• Hardening _build_payer_block to call the helper before delegating to _build_nm1.
• A regression test in backend/tests/test_serialize_837.py asserting: SKCO0 → CO_TXIX in NM109 + NM103; empty payer id → CO_TXIX in NM109; CO_BHA → CO_TXIX in NM109 + NM103; a foreign-payer id (e.g. "OTHER_PAYER") preserved unchanged.
• An idempotent regen script unbilled-july2026/scripts/regen_corrected_files.py (alongside the analysis scripts in the sibling project folder) that walks the 363 corrected ingest/corrected/batch-*/ files, re-parses each one through parse_837.parse_837_text, and re-serializes through serialize_837_for_resubmit. Output goes to ingest/corrected-v2/<batch-id>-<N>-claims/ with a single global counter for unique timestamps. Each emitted file is re-validated with parse_837_text + a content check that asserts PI*CO_TXIX is present and neither PI*SKCO0 nor *COHCPF**** appear.
• A new resubmissions table (claim_id, batch_id, resubmitted_at, source_corrected_path, interchange_control_number, group_control_number) plus a matching SQLAlchemy migration under backend/migrations/0013_*.py. The table is populated by the existing cyclone resubmit-rejected-claims CLI when the operator pushes a corrected file (one row per claim per resubmission; idempotent on (claim_id, interchange_control_number)).
• A new CLI cyclone resubmissions status [--batch-id=<id>] that joins resubmissions → claims → claim_acks (via the existing SP28/31 ack-claim auto-link) → remittances (via the existing CLP→claim auto-link) and prints a per-claim row: claim_id | patient | resubmitted_at | 999_status | 277ca_status | payment_status. Statuses are derived as: pending_999 if no ack row exists yet, 999_accepted if a 999 with accept code exists, 999_rejected if a 999 with reject code exists, 277ca_accepted / 277ca_rejected once 277CA arrives, paid once a remittance CLP links, denied_again if a 277CA reject exists post-resubmission.
• A note in docs/RUNBOOK.md under "Manual SFTP mode" pointing the operator at the new corrected-v2/ tree + the new cyclone resubmissions status CLI after this SP merges, and an inline TODO comment in submission/core.py:EXPECTED_PAYER_ID block referencing the future follow-up that will trace where ClaimOutput.payer.id gets set to "SKCO0" upstream.

Out of scope:

• Tracing and patching the upstream setter that populates ClaimOutput.payer.id = "SKCO0" (a separate future ticket; this SP only defense-in-depths the serializer). The _normalize_payer_id helper masks the upstream bug for any path that goes through serialize_837 / serialize_837_for_resubmit, but does not touch the claims table.
• Re-ingesting the four rejected ingest/tp11525703-837P-*.txt files. They are historical artifacts of the rejected submission and remain in ingest/ for audit only; the operator does not resubmit them. The 363 corrected files in ingest/corrected/batch-*/ are the resubmission payload, not these four.
• Auto-resubmitting ingest/corrected-v2/ via cyclone resubmit-rejected-claims. That CLI is the operator's workflow (per RUNBOOK §"Manual SFTP mode") and remains operator-invoked only. This SP only regenerates the corrected files; the operator chooses when to push them.
• Any change to PayerConfig.co_medicaid() (SP33 already canonicalized it) or to the submission/core.py validation gate at EXPECTED_PAYER_ID. The gate continues to enforce CO_TXIX at submit time and is not relaxed by this SP.
• A frontend surface for resubmission status. The CLI report is the only operator-facing view; a future UI increment (likely a new Inbox lane or Dashboard KPI) can render the same joined data once the workflow is proven.
• Touching the existing 999/277CA/remit auto-link code paths. The resubmission status CLI consumes the same joined tables the auto-link already populates; this SP does not modify the auto-link logic itself.

2. Decisions (locked during brainstorming)

D1: Substitute both legacy values — SKCO0 and CO_BHA — plus empty — to CO_TXIX.

The operator's policy is "every claim is CO_TXIX". The companion guide technically accepts CO_BHA for behavioral-health claims, but Cyclone is not configured to distinguish BHA claims at submission time and the operator's existing manual workflow treats CO_TXIX as the canonical value for all CO Medicaid submits. CO_BHA therefore gets normalized the same way SKCO0 does. Foreign payer IDs (any value not in {empty, "SKCO0", "CO_BHA"}) are emitted verbatim — the helper only normalizes CO Medicaid-shape values and must not corrupt a non-CO submit.

D2: The helper also fixes payer.name to match.

When the helper substitutes payer.id to CO_TXIX, it also substitutes payer.name from "COHCPF", "CO_BHA", or empty to "CO_TXIX". This keeps NM103 consistent with NM109 and matches the byte the operator's manual-mode workflow already uses in the corrected files.

D3: WARNING log per substitution, not per batch.

_build_payer_block is invoked once per claim. One log line per substitution is bounded by batch size (≤ 145 lines for the Jun 24 batches) and is the minimum signal the operator needs to detect "this batch still has upstream contamination". Per-batch aggregation would hide multi-payer batches (if any are ever supported).

D4: Regression tests live in test_serialize_837.py, not a new file.

The existing file already covers NM1 segment shape; appending three tests for the helper is the minimal-surface change. Matching the cyclone-tests convention (sibling test file per module).

D5: Regen script lives in the unbilled-july2026 sibling project.

The 363 files + the four ingest/tp11525703-837P-*.txt originals are part of the unbilled-july2026 recovery work, not the Cyclone core codebase. The sibling project folder already houses the analysis scripts; the regen script follows the same convention. The script reads ingest/corrected/batch-*/ (relative to Cyclone's repo root, since ingest/ is shared with the production SFTP workflow) and writes to ingest/corrected-v2/.

D6: Re-run into ingest/corrected-v2/, not clobber the original batch-*/ tree.

The 363 corrected files are the postmortem anchor for SP33-era SKCO0 contamination. Preserving them under batch-*/ lets a future contributor diff pre-SP39 and post-SP39 outputs and confirms the helper is a byte-faithful transformation on every input that was already correct. The corrected-v2/ tree is the new submission payload.

D7: File a follow-up note (in RUNBOOK), don't patch the upstream ClaimOutput.payer.id setter in this SP.

The operator picked hard-fix the serializer (Approach 1) over trace to root cause (Approach 3). The follow-up ticket belongs in a future SP; this SP captures the need via a TODO comment in submission/core.py and a RUNBOOK entry pointing at the new corrected-v2/ tree.

D8: resubmissions is a write-once audit table, not a state-machine.

The table records "this claim was pushed to SFTP at this time from this local file" — one row per claim per push. The CLI derives status (pending_999 / 999_accepted / paid / etc.) at read-time by joining against claim_acks + remittances

• the existing auto-link tables. We do not denormalize status onto the resubmissions row, because that would create a write-coordination problem between the SFTP push and the inbound 999/277CA ingestion.

D9: Post-submission tracking joins reuse the existing auto-link data, no new matching logic.

The 999 ack auto-link (SP28/31) already maps an inbound CLP segment back to a claim_id via CLP01 + charge + service date. Because the corrected-v2 regen preserves the original claim_id from raw_json.claim.claim_id, the 999 ack for a resubmitted file auto-links to the original claim row — and the new status CLI just joins resubmissions.claim_id against claim_acks.claim_id to surface the result. No new matching code is needed; this SP adds the read-side join and the CLI surface, not a new matcher.

D10: The resubmit-rejected-claims CLI inserts resubmissions rows; we do not add a new push CLI.

The existing cyclone resubmit-rejected-claims (per backend/src/cyclone/cli.py and the SP33 followup work) is the operator's only entry point for pushing corrected files via SFTP. We instrument it to insert one resubmissions row per claim per push (matching the same idempotency key as the SFTP upload). No new CLI command for the push side.

3. Open questions

None. The operator confirmed the design via the brainstorming Q&A on 2026-07-07: hard-fix the serializer with empty/SKCO0/CO_BHA substitution to CO_TXIX (no CO_BHA preservation), regen into corrected-v2/, add a post-submission tracking table + status CLI (joined against existing auto-link data, no new matcher), file a follow-up note for upstream tracing, no UI, no auto-resubmit.

4. Test impact

Per cyclone-tests (autouse conftest at backend/tests/conftest.py):

• backend/tests/test_serialize_837.py — append three tests targeting _build_payer_block via the public serialize_837_for_resubmit entry point. Each test constructs a minimal ClaimOutput whose payer.id is set to the value under test, calls serialize_837_for_resubmit, and asserts the emitted NM1*PR segment. Tests:
  • test_2010bb_normalizes_skco0_to_co_txix — asserts SKCO0 → CO_TXIX in NM109 and COHCPF → CO_TXIX in NM103.
  • test_2010bb_normalizes_empty_payer_id_to_co_txix — asserts empty payer.id → CO_TXIX and empty payer.name → CO_TXIX.
  • test_2010bb_normalizes_co_bha_to_co_txix — asserts CO_BHA (with name CO_BHA) → CO_TXIX in both NM109 and NM103. Plus a fourth assertion that a foreign payer id (e.g. "OTHER_PAYER" with name "OTHER PAYER NAME") is emitted verbatim — the helper must not corrupt non-CO submits.
• backend/tests/test_resubmissions_cli.py — new, tests the cyclone resubmissions status CLI via click.testing.CliRunner. Seeds the resubmissions table + claim_acks (via the existing fixture pattern) and asserts:
  • pending_999 when no ack exists yet.
  • 999_accepted after a 999 ack with accept code is linked.
  • paid after a remittance CLP links.
  • Exit code 0 with empty resubmissions table prints "no resubmissions recorded" and exits 0.
• backend/tests/test_resubmissions_table.py — new, tests the SQLAlchemy model + idempotency of (claim_id, interchange_control_number). Asserts schema columns, asserts a second insert with the same key is a no-op (raises IntegrityError or ON CONFLICT DO NOTHING, matching the convention in the rest of db.py).
• unbilled-july2026/scripts/test_regen_corrected_files.py — new, sibling test in the unbilled-july2026 project. Uses a fixture of three fake-corrected files (one already correct, one with SKCO0 from a degenerate raw_json, one with CO_BHA) and asserts the regen script emits all three with the expected normalization.
• Migration test impact: append to backend/tests/test_db_migrate.py (or equivalent) a check that migration 0013 creates the resubmissions table with the documented columns + idempotency unique constraint.
• No frontend test impact.

5. Files expected to change

• backend/src/cyclone/parsers/serialize_837.py — add _normalize_payer_id(payer), call it from _build_payer_block, emit WARNING log on substitution.
• backend/src/cyclone/db.py — add Resubmission SQLAlchemy model.
• backend/migrations/0013_resubmissions.py — new migration creating the resubmissions table.
• backend/src/cyclone/cli.py — instrument resubmit-rejected-claims to insert one Resubmission row per claim per push (idempotent on (claim_id, interchange_control_number)); add the cyclone resubmissions status subcommand.
• backend/src/cyclone/store/__init__.py — add CycloneStore.record_resubmission(...) and CycloneStore.find_resubmission_status(...) helpers; re-export through the facade.
• backend/tests/test_serialize_837.py — append the three regression tests.
• backend/tests/test_resubmissions_cli.py — new CLI tests.
• backend/tests/test_resubmissions_table.py — new model + idempotency tests.
• backend/tests/test_db_migrate.py — append migration test for 0013.
• unbilled-july2026/scripts/regen_corrected_files.py — new sibling-project regen script.
• unbilled-july2026/scripts/test_regen_corrected_files.py — new sibling-project test.
• docs/RUNBOOK.md — append "After SP39 lands" entry under "Manual SFTP mode" pointing the operator at corrected-v2/, the new cyclone resubmissions status CLI, and the workflow to push via cyclone resubmit-rejected-claims.
• backend/src/cyclone/submission/core.py — append TODO comment above EXPECTED_PAYER_ID referencing the future upstream- trace ticket.
• docs/superpowers/plans/2026-07-07-cyclone-2010bb-nm109-fix.md — the implementation plan, written after this spec is signed off.

6. Auth boundary

The auth boundary is HTTP (login required, bcrypt + HttpOnly session cookie); file-system threats remain the local-only threat model (SQLCipher at rest, macOS Keychain). The serializer helper is invoked server-side by both serialize_837 (tested via cyclone submit-batch / POST /api/submit-batch, both behind HTTP auth) and serialize_837_for_resubmit (called by the operator's regen workflow and by /api/inbox/rejected/resubmit, also behind HTTP auth). The sibling-project regen script is operator-invoked only and runs against a local file tree; it does not touch the HTTP surface or the DB. No change to the threat model.