cyclone/docs/superpowers/plans/2026-07-07-cyclone-submit-batch-canonical-flow.md

SP37 — Canonical submit-batch flow: Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a canonical cyclone submit-batch CLI + POST /api/submit-batch endpoint that does parse → DB write → SFTP upload per file, and add a Batch.transaction_set_control_number column so 999 acks can join back to source 837 batches via the ST02 control number.

Architecture: New cyclone.submission package with one shared helper (submit_file) wrapping the parse-write-upload sequence. CLI + HTTP both thin-call the helper. Migration adds the new column additively + backfills from raw_result_json. claim_acks.batch_envelope_index returns a dict populated from BOTH envelope.control_number (preserved) AND transaction_set_control_number (new) — single idx.get(set_control_number) lookup matches either key.

Tech Stack: Python 3.11, FastAPI, SQLAlchemy + Alembic, Click (CLI), paramiko (SFTP), Pydantic, pytest.

Spec: docs/superpowers/specs/2026-07-07-cyclone-submit-batch-canonical-flow-design.md

Progress tracker: /tmp/refactor-cyclone.md — update after every commit per the operator's standing directive (per-step log with pre/post pytest, live-test result, reviewer verdict).

Branch: sp37-submit-batch-canonical-flow (already created, off 710104f).

Conventions: Per cyclone-spec skill — commit prefixes are feat(sp37): …, merge: SP37 canonical submit-batch flow into main. Per cyclone-tests skill — backend tests live under backend/tests/test_*.py, test_api_* for FastAPI integration, pure-unit otherwise; conftest points CYCLONE_DB_URL at tmp_path/test.db. Per cyclone-store skill — write through the CycloneStore facade, never directly into db.py.

---

Task 0: Progress-tracker bootstrap + baseline pytest

Files:

• Modify: /tmp/refactor-cyclone.md

• (no code changes)

• Step 1: Capture baseline pytest

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest -x --tb=no -q 2>&1 | tail -3 Expected: a count line like X passed, Y skipped, Z failed. Save to /tmp/refactor-pre-baseline.txt.

• Step 2: Update tracker baseline section

Edit /tmp/refactor-cyclone.md — replace the "Baseline" section's pytest line with the actual count from Step 1 (so future task diffs are honest).

• Step 3: Commit

git add /tmp/refactor-cyclone.md  # the file lives outside the repo; this is a no-op marker
# No git commit needed — the tracker is out-of-repo. Just confirm the file is updated.

(End of Task 0 — no commit, no code change. The tracker is out-of-repo per the operator's standing directive.)

---

Task 1: Migration 0013_add_batch_txn_set_control_number.py

Files:

• Create: backend/migrations/0013_add_batch_txn_set_control_number.py

• Create: backend/tests/test_migration_0013.py

• Step 1: Write the failing test

Create backend/tests/test_migration_0013.py:

"""SP37 Task 1: migration 0013 adds Batch.transaction_set_control_number.

The column is nullable, additive, and backfills from raw_result_json
when the parsed envelope carries a transaction_set_control_number key.
"""
from __future__ import annotations

import json

import pytest

from cyclone import db as db_mod
from cyclone.db_migrate import apply_migrations


def _bootstrap_test_db(tmp_path, monkeypatch):
    db_file = tmp_path / "mig.db"
    monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{db_file}")
    db_mod._reset_for_tests()
    db_mod.init_db()
    return db_file


def test_migration_0013_adds_column(tmp_path, monkeypatch):
    _bootstrap_test_db(tmp_path, monkeypatch)
    apply_migrations(target="0013")

    with db_mod.engine.connect() as conn:
        cols = {row[1] for row in conn.exec_driver_sql("PRAGMA table_info(batches)").fetchall()}
    assert "transaction_set_control_number" in cols


def test_migration_0013_backfills_from_raw_json(tmp_path, monkeypatch):
    _bootstrap_test_db(tmp_path, monkeypatch)

    # Insert a batch row with an envelope that carries transaction_set_control_number.
    with db_mod.SessionLocal()() as s:
        s.add(db_mod.Batch(
            id="b-backfill",
            kind="837p",
            input_filename="x.837p",
            raw_result_json={
                "envelope": {
                    "sender_id": "CYCLONE",
                    "receiver_id": "HCPF",
                    "control_number": "000000001",
                    "transaction_set_control_number": "ST0001",
                    "transaction_date": "2026-07-07",
                },
                "claims": [],
            },
        ))
        s.commit()

    apply_migrations(target="0013")

    with db_mod.SessionLocal()() as s:
        row = s.get(db_mod.Batch, "b-backfill")
        assert row.transaction_set_control_number == "ST0001"


def test_migration_0013_down(tmp_path, monkeypatch):
    _bootstrap_test_db(tmp_path, monkeypatch)
    apply_migrations(target="0013")
    apply_migrations(target="zero")  # walk back to nothing

    with db_mod.engine.connect() as conn:
        cols = {row[1] for row in conn.exec_driver_sql("PRAGMA table_info(batches)").fetchall()}
    assert "transaction_set_control_number" not in cols

• Step 2: Run test, verify it fails

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_migration_0013.py -v Expected: all 3 tests FAIL (no migration yet).

• Step 3: Write the migration

Create backend/migrations/0013_add_batch_txn_set_control_number.py. Look at backend/migrations/0012_backups.py first to match the project's existing migration style (Alembic op vs raw SQL — match what's there). The body:

"""SP37 Task 1: add nullable batches.transaction_set_control_number.

Backfills from raw_result_json.envelope.transaction_set_control_number
when present. Idempotent — re-running on a populated DB is a no-op
because the column is already populated and the WHERE clause skips NULLs.
"""
# (Replace this docstring with the project's standard migration header
# once you've matched the style of 0012_backups.py.)

from alembic import op
import sqlalchemy as sa


revision = "0013_add_batch_txn_set_control_number"
down_revision = "0012_backups"
depends_on = None


def upgrade():
    op.add_column(
        "batches",
        sa.Column("transaction_set_control_number", sa.String(length=32), nullable=True),
    )
    op.execute(
        """
        UPDATE batches
        SET transaction_set_control_number = json_extract(
            raw_result_json, '$.envelope.transaction_set_control_number'
        )
        WHERE raw_result_json IS NOT NULL
          AND json_extract(raw_result_json, '$.envelope.transaction_set_control_number') IS NOT NULL
        """
    )


def downgrade():
    op.drop_column("batches", "transaction_set_control_number")

• Step 4: Run test, verify it passes

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_migration_0013.py -v Expected: 3 passed.

• Step 5: Live test — apply on the real dev DB

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/python -m cyclone db migrate --to 0013 (use whatever command db_migrate.py exposes; check backend/src/cyclone/db_migrate.py for the actual CLI). Expected: column added; existing 1 batch row (b1) shows NULL transaction_set_control_number (it has no envelope with that key).

Verify: sqlite3 ~/.local/share/cyclone/cyclone.db "PRAGMA table_info(batches);" | grep transaction Expected: one row matching transaction_set_control_number.

• Step 6: Commit

git add backend/migrations/0013_add_batch_txn_set_control_number.py backend/tests/test_migration_0013.py
git commit -m "feat(sp37): add batches.transaction_set_control_number column

Migration 0013 adds the column additively (nullable, no default) and
backfills from raw_result_json.envelope.transaction_set_control_number
for any existing batch rows that already carry it. Required for the
SP37 join-key update so 999 acks can resolve by ST02 (the source 837's
transaction set control number) instead of just ISA13."

• Step 7: Update tracker

Add a Task 1 section to /tmp/refactor-cyclone.md with: pre/post pytest (both should equal baseline from Task 0), migration applied successfully, no reviewer needed (migration only).

---

Task 2: Add Batch.transaction_set_control_number to ORM + populate from _claim_837_row flow

Files:

• Modify: backend/src/cyclone/db.py:Batch (add mapped column)

• Modify: backend/src/cyclone/store/orm_builders.py:_claim_837_row (or wherever add_record populates the Batch row — see Step 1)

• Step 1: Locate the Batch row constructor

Search for the Batch(...) instantiation that add_record uses. It lives in backend/src/cyclone/store/write.py:78-85 (per the spec exploration). The current constructor:

batch_row = Batch(
    id=record.id,
    kind=record.kind,
    input_filename=record.input_filename,
    parsed_at=record.parsed_at,
    totals_json=None,
    validation_json=None,
    raw_result_json=json.loads(record.result.model_dump_json()),
)

The record.result is the parsed ParseResult (or ParseResult835). For 837P the envelope is at result.envelope.

• Step 2: Write the failing test

Create backend/tests/test_batch_txn_set_cn.py:

"""SP37 Task 2: add_record populates Batch.transaction_set_control_number.

The column should mirror the source 837 envelope's
transaction_set_control_number (ST02) so the join-key index in
Task 3 can resolve 999 set_control_numbers back to the right batch.
"""
from __future__ import annotations

import json
from pathlib import Path

import pytest

from cyclone import db as db_mod
from cyclone.db import Batch
from cyclone.store import store as cycl_store
from cyclone.store.records import BatchRecord837

# Reuse a small 837 fixture if one exists in tests/fixtures/; otherwise
# copy the smallest real 837 from docs/prodfiles/ to fixtures/.
_FIXTURE = Path(__file__).parent / "fixtures" / "single-claim-837P.x12"


@pytest.fixture(autouse=True)
def _db(tmp_path, monkeypatch):
    monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
    db_mod._reset_for_tests()
    db_mod.init_db()
    yield


def test_add_record_populates_transaction_set_control_number(monkeypatch):
    """parse_837 → add_record should write the parsed ST02 onto the Batch row."""
    from cyclone.parsers.parse_837 import parse as parse_837_text
    from cyclone.parsers.payer import PayerConfig

    text = _FIXTURE.read_text()
    result = parse_837_text(text, PayerConfig.co_medicaid())

    record = BatchRecord837(
        id="b-test-1",
        input_filename="x.837p",
        parsed_at=result.parsed_at,
        result=result,
    )
    cycl_store.add_record(record)

    with db_mod.SessionLocal()() as s:
        row = s.get(Batch, "b-test-1")
        assert row is not None
        # Whichever ST02 the fixture has must round-trip into the column.
        assert row.transaction_set_control_number == result.envelope.transaction_set_control_number

• Step 3: Add the ORM column

Modify backend/src/cyclone/db.py:Batch. Add the new mapped column. Locate the existing Batch class (search for class Batch( in db.py) and add after the existing envelope-ish fields:

transaction_set_control_number: Mapped[Optional[str]] = mapped_column(
    String(32), nullable=True
)

Confirm the import line from sqlalchemy import ... String ... already includes String; if not, add it.

• Step 4: Run test, verify it fails

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_batch_txn_set_cn.py -v Expected: FAIL with AttributeError: type object 'Batch' has no attribute 'transaction_set_control_number' (the ORM doesn't have the column yet, OR the migration hasn't been applied to the test DB). If the test DB is freshly created per-test (it is, via the autouse fixture), apply migration 0013 to it:

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/python -c "from cyclone.db_migrate import apply_migrations; from cyclone import db; db.init_db(); apply_migrations(target='0013')" (This step is for local sanity; the autouse test fixture should already trigger init_db which runs migrations.)

If the test still fails because add_record doesn't populate the column, that's the next step.

• Step 5: Update add_record to populate the column

Modify backend/src/cyclone/store/write.py:78-85. Change the Batch constructor to include the new field. Read the record.result envelope and copy its transaction_set_control_number onto the row:

batch_row = Batch(
    id=record.id,
    kind=record.kind,
    input_filename=record.input_filename,
    parsed_at=record.parsed_at,
    totals_json=None,
    validation_json=None,
    raw_result_json=json.loads(record.result.model_dump_json()),
    transaction_set_control_number=getattr(
        getattr(record.result, "envelope", None), "transaction_set_control_number", None
    ),
)

The getattr(..., None) guards against the 835 path where the result has no envelope attribute (it's a ParseResult835, not ParseResult); BatchRecord835 won't set the column.

• Step 6: Run test, verify it passes

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_batch_txn_set_cn.py -v Expected: PASS.

• Step 7: Run the existing test_api_parse_837 to confirm no regression

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_api_parse_837.py -v Expected: same pass/fail as baseline (no new failures introduced).

• Step 8: Commit

git add backend/src/cyclone/db.py backend/src/cyclone/store/write.py backend/tests/test_batch_txn_set_cn.py
git commit -m "feat(sp37): populate batches.transaction_set_control_number on 837P ingest

add_record now copies the parsed envelope's transaction_set_control_number
(the source 837's ST02) onto the Batch row. Unlocks the SP37 join-key
update so 999 set_control_number (AK201) values resolve back to the
right batch via Pass 1."

• Step 9: Update tracker

Append Task 2 section with pre/post pytest, test result, live-test (no manual live test needed; the existing clm-1 row can be re-ingested via /api/parse-837 if you want to verify end-to-end — optional).

---

Task 3: Update batch_envelope_index to key by both envelope.control_number AND transaction_set_control_number

Files:

• Modify: backend/src/cyclone/store/claim_acks.py:batch_envelope_index

• Create: backend/tests/test_batch_envelope_index_sp37.py

• Step 1: Read the existing function

Read backend/src/cyclone/store/claim_acks.py around def batch_envelope_index. Per spec, the current behavior is: read every Batch row whose raw_result_json has an envelope and return {envelope.control_number: batch.id}. We change it to ALSO populate from Batch.transaction_set_control_number (the new column from Task 2), so a single dict has both keys.

• Step 2: Write the failing test

Create backend/tests/test_batch_envelope_index_sp37.py:

"""SP37 Task 3: batch_envelope_index populates from BOTH columns.

The dict returned by batch_envelope_index should resolve lookups by
either envelope.control_number (ISA13, preserved) OR
transaction_set_control_number (ST02, new). One dict, both keys —
a single .get(set_control_number) call hits whichever matches.
"""
from __future__ import annotations

import json

import pytest

from cyclone import db as db_mod
from cyclone.db import Batch
from cyclone.store.claim_acks import batch_envelope_index


@pytest.fixture(autouse=True)
def _db(tmp_path, monkeypatch):
    monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
    db_mod._reset_for_tests()
    db_mod.init_db()
    yield


def _make_batch(s, **kw):
    s.add(Batch(
        id=kw["id"],
        kind="837p",
        input_filename=kw["id"] + ".x12",
        raw_result_json=kw.get("raw_result_json", {"envelope": {"control_number": kw.get("icn", "000000001")}}),
        transaction_set_control_number=kw.get("stcn"),
    ))
    s.commit()


def test_index_resolves_by_envelope_control_number():
    with db_mod.SessionLocal()() as s:
        _make_batch(s, id="b1", icn="ISA000001")
    idx = batch_envelope_index()
    assert idx.get("ISA000001") == "b1"


def test_index_resolves_by_transaction_set_control_number():
    with db_mod.SessionLocal()() as s:
        _make_batch(s, id="b2", icn="ISA000002", stcn="ST000002")
    idx = batch_envelope_index()
    assert idx.get("ST000002") == "b2"
    # Backward compat: ISA still resolves.
    assert idx.get("ISA000002") == "b2"


def test_index_handles_row_with_no_transaction_set_control_number():
    """Pre-migration rows (stcn NULL) should still resolve by ISA."""
    with db_mod.SessionLocal()() as s:
        _make_batch(s, id="b3", icn="ISA000003")  # no stcn
    idx = batch_envelope_index()
    assert idx.get("ISA000003") == "b3"
    assert idx.get("ST000003") is None

• Step 3: Run test, verify it fails

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_batch_envelope_index_sp37.py -v Expected: test_index_resolves_by_transaction_set_control_number and test_index_handles_row_with_no_transaction_set_control_number FAIL because the current function only reads raw_result_json.envelope.control_number and ignores the new column.

• Step 4: Update the function

Modify backend/src/cyclone/store/claim_acks.py:batch_envelope_index. Read both the raw_result_json envelope AND the new transaction_set_control_number column. Replace the function body so it returns a single dict populated from both sources. The new body (sketch — adapt to the existing function's style):

def batch_envelope_index() -> dict[str, str]:
    """Build a ``{key: batch.id}`` map populated from two columns.

    D10 primary join key (spec §D10). Each batch row contributes up
    to two entries:

    * ``Batch.raw_result_json.envelope.control_number`` (ISA13)
    * ``Batch.transaction_set_control_number`` (ST02, new in SP37)

    Either key resolves to the same ``batch.id``; callers do a single
    ``idx.get(set_control_number)`` lookup. Pass 2 (PCN) is unchanged.

    Built once per ingest from every Batch row whose kind is "837p";
    cost is O(N batches), currently small, so trivial.
    """
    idx: dict[str, str] = {}
    with db.SessionLocal()() as s:
        rows = s.query(Batch.id, Batch.raw_result_json, Batch.transaction_set_control_number).filter(
            Batch.kind == "837p"
        ).all()
        for bid, raw, stcn in rows:
            env = (raw or {}).get("envelope") or {}
            isa_cn = env.get("control_number")
            if isa_cn:
                idx.setdefault(str(isa_cn), bid)
            if stcn:
                idx.setdefault(str(stcn), bid)
    return idx

(The exact rewrite depends on the current function's structure — read it first, then drop the new body in, preserving any existing docstring above it.)

• Step 5: Run test, verify it passes

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_batch_envelope_index_sp37.py -v Expected: 3 passed.

• Step 6: Run all ack-related tests to confirm no regression

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_claim_acks.py tests/test_api_ack_stream.py tests/test_apply_999_rejections.py -v Expected: same pass/fail as baseline. The new dict may have more entries, but Pass 1's idx.get(...) lookup semantics are unchanged.

• Step 7: Commit

git add backend/src/cyclone/store/claim_acks.py backend/tests/test_batch_envelope_index_sp37.py
git commit -m "feat(sp37): batch_envelope_index now resolves by ST02 or ISA13

Each batch row contributes up to two entries to the join-key index
(envelope.control_number + transaction_set_control_number). One
idx.get(set_control_number) call resolves either, so 999 acks whose
AK201 echoes the source 837's ST02 now hit Pass 1 where they
previously fell through to Pass 2 (and to the orphan log)."

• Step 8: Update tracker

Append Task 3 section.

---

Task 4: cyclone.submission package — submit_file helper + unit tests

Files:

• Create: backend/src/cyclone/submission/__init__.py

• Create: backend/src/cyclone/submission/core.py

• Create: backend/src/cyclone/submission/result.py (SubmitResult + SubmitOutcome)

• Create: backend/tests/test_submission.py

• Create: backend/tests/fixtures/submit-batch/ directory + at least one .x12 fixture

• Step 1: Pick a fixture

Copy a small valid 837P file from docs/prodfiles/co_medicaid/ (or wherever the operator's recent files live — try docs/prodfiles/2026-07/ or ingest/july7/) into backend/tests/fixtures/submit-batch/. Name it single-claim.x12. If no prodfile is available, synthesize one from the test fixtures used in tests/test_api_parse_837.py (find the smallest one and copy it).

• Step 2: Define the result types

Create backend/src/cyclone/submission/result.py:

"""SP37 Task 4: typed result for submit_file.

The CLI and HTTP layer both consume SubmitResult; keeping it in one
place means both surfaces agree on the response shape.
"""
from __future__ import annotations

from dataclasses import dataclass
from enum import Enum


class SubmitOutcome(str, Enum):
    SUBMITTED = "submitted"
    SKIPPED = "skipped"
    PARSE_FAILED = "parse_failed"
    PAYER_MISMATCH = "payer_mismatch"
    DB_FAILED = "db_failed"
    SFTP_FAILED = "sftp_failed"


@dataclass(frozen=True)
class SubmitResult:
    file: str
    outcome: SubmitOutcome
    batch_id: str | None = None
    error: str | None = None

• Step 3: Write the failing test (happy path)

Create backend/tests/test_submission.py:

"""SP37 Task 4: submit_file owns parse → DB write → SFTP upload.

Tests use a fake SFTP client to avoid hitting the real clearhouse.
The DB path is real (autouse fixture) so the join-key wiring is
exercised end-to-end.
"""
from __future__ import annotations

from pathlib import Path
from unittest.mock import MagicMock

import pytest

from cyclone import db as db_mod
from cyclone.clearhouse import SftpClient
from cyclone.submission.core import submit_file
from cyclone.submission.result import SubmitOutcome


_FIXTURE = Path(__file__).parent / "fixtures" / "submit-batch" / "single-claim.x12"


class _FakeSftp:
    """Implements just enough of SftpClient for submit_file's idempotency check."""

    def __init__(self, existing_files: dict[str, int] | None = None):
        self.existing_files = existing_files or {}
        self.put_calls: list[tuple[str, bytes]] = []

    def stat(self, path: str):
        if path not in self.existing_files:
            raise IOError(f"no such file: {path}")
        m = MagicMock()
        m.st_size = self.existing_files[path]
        return m

    def write_file(self, remote_path: str, content: bytes):
        self.put_calls.append((remote_path, content))


@pytest.fixture(autouse=True)
def _db(tmp_path, monkeypatch):
    monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
    db_mod._reset_for_tests()
    db_mod.init_db()
    yield


def test_submit_file_happy_path(monkeypatch):
    """parse → DB write → SFTP upload, all steps succeed."""
    fake = _FakeSftp()
    monkeypatch.setattr(SftpClient, "stat", fake.stat)
    monkeypatch.setattr(SftpClient, "write_file", fake.write_file)

    sftp_block = MagicMock()
    sftp_block.stub = False
    sftp_block.paths = {"outbound": "/ToHPE"}

    result = submit_file(
        _FIXTURE,
        sftp_block=sftp_block,
        actor="test",
        validate=True,
        sftp_client_factory=lambda _block: fake,
    )

    assert result.outcome == SubmitOutcome.SUBMITTED
    assert result.batch_id is not None
    assert len(fake.put_calls) == 1
    # DB row landed.
    with db_mod.SessionLocal()() as s:
        assert s.get(db_mod.Batch, result.batch_id) is not None

• Step 4: Run test, verify it fails

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_submission.py -v Expected: FAIL with ModuleNotFoundError: No module named 'cyclone.submission'.

• Step 5: Implement the package skeleton

Create backend/src/cyclone/submission/__init__.py:

"""SP37 Task 4: canonical 837P submission flow.

The single public helper is ``submit_file`` — it owns parse → DB
write → SFTP upload per file. CLI (``cyclone submit-batch``) and HTTP
(``POST /api/submit-batch``) are thin wrappers; both call this helper
so the ordering, idempotency, and audit shape are identical.

DB-first, upload-second (per spec decision §2): if the DB write fails,
no SFTP call is made. If the SFTP call fails after a successful DB
write, the row exists but the file never landed — re-running is safe
(idempotent DB write via add_record's s.get check; idempotent SFTP
upload via stat-then-put).
"""
from .core import submit_file
from .result import SubmitOutcome, SubmitResult

__all__ = ["submit_file", "SubmitOutcome", "SubmitResult"]

• Step 6: Implement submit_file

Create backend/src/cyclone/submission/core.py. Read the existing resubmit_rejected_claims CLI (around backend/src/cyclone/cli.py:553-720) and mirror its structure, but route the DB write through cycl_store.add_record first and the SFTP put second.

"""SP37 Task 4: parse → DB write → SFTP upload, per file.

Mirrors ``resubmit_rejected_claims`` but writes to the DB before
uploading. Same idempotency check (``stat().st_size == local_size``)
skips already-uploaded files without re-emitting audit events.
"""
from __future__ import annotations

import logging
from pathlib import Path

from cyclone.clearhouse import SftpClient
from cyclone.parsers.parse_837 import parse as parse_837_text
from cyclone.parsers.payer import PayerConfig
from cyclone.store import store as cycl_store
from cyclone.store.records import BatchRecord837

from .result import SubmitOutcome, SubmitResult

log = logging.getLogger(__name__)


def submit_file(
    path: Path,
    *,
    sftp_block,
    actor: str,
    validate: bool = True,
    sftp_client_factory=None,
) -> SubmitResult:
    """Submit one 837P file: parse → DB write → SFTP upload.

    Args:
        path: local 837P file.
        sftp_block: the clearhouse SftpBlock (for paths + auth).
        actor: audit-log actor tag (e.g. "api-submit-batch").
        validate: parse the file before upload (default True). Catches
            bad byte-fixes early.
        sftp_client_factory: optional callable that returns an
            SftpClient-compatible object. Defaults to the real
            SftpClient(block=sftp_block). Tests inject a fake.

    Returns:
        SubmitResult with file, outcome, batch_id (when written),
        error (when failed).
    """
    file_label = path.name
    try:
        content = path.read_bytes()
    except OSError as exc:
        return SubmitResult(file_label, SubmitOutcome.PARSE_FAILED, error=str(exc))

    # 1. Validate via parse.
    if validate:
        try:
            parsed = parse_837_text(content.decode(), PayerConfig.co_medicaid())
        except Exception as exc:  # noqa: BLE001
            log.warning("submit_file %s: parse failed: %s", file_label, exc)
            return SubmitResult(file_label, SubmitOutcome.PARSE_FAILED, error=str(exc))
        mismatch = next(
            (c for c in parsed.claims if c.payer.id != "CO_TXIX"),
            None,
        )
        if mismatch is not None:
            return SubmitResult(
                file_label,
                SubmitOutcome.PAYER_MISMATCH,
                error=f"payer.id={mismatch.payer.id!r} (expected 'CO_TXIX')",
            )

    # 2. DB write — DB-first, upload-second invariant.
    record = BatchRecord837(
        id=cycl_store._new_batch_id(),  # or however existing code generates ids
        input_filename=file_label,
        parsed_at=parsed.parsed_at if validate else None,
        result=parsed if validate else None,
    )
    try:
        cycl_store.add_record(record)
    except Exception as exc:  # noqa: BLE001
        log.warning("submit_file %s: DB write failed: %s", file_label, exc)
        return SubmitResult(file_label, SubmitOutcome.DB_FAILED, error=str(exc))

    # 3. SFTP upload.
    remote_path = f"{sftp_block.paths['outbound']}/{file_label}"
    factory = sftp_client_factory or (lambda b: SftpClient(block=b))
    sftp = factory(sftp_block)
    local_size = len(content)
    try:
        try:
            stat = sftp.stat(remote_path)
            if stat.st_size == local_size:
                return SubmitResult(file_label, SubmitOutcome.SKIPPED, batch_id=record.id)
        except (IOError, OSError):
            pass  # not on remote yet
        sftp.write_file(remote_path, content)
    except Exception as exc:  # noqa: BLE001
        log.warning("submit_file %s: SFTP failed: %s", file_label, exc)
        return SubmitResult(file_label, SubmitOutcome.SFTP_FAILED, batch_id=record.id, error=str(exc))

    # 4. Audit event.
    cycl_store.append_clearhouse_submitted_event(
        source_file=file_label, batch_id=record.id, actor=actor,
    )
    return SubmitResult(file_label, SubmitOutcome.SUBMITTED, batch_id=record.id)

(Adapt the helpers you don't have — cycl_store._new_batch_id may not exist; check cyclone/store/__init__.py for the canonical ID generator, and cycl_store.append_clearhouse_submitted_event may need to be added if it doesn't exist. Mirror what resubmit_rejected_claims already does for these two operations.)

• Step 7: Run the happy-path test, verify it passes

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_submission.py::test_submit_file_happy_path -v Expected: PASS.

• Step 8: Add the remaining unit tests

Append to backend/tests/test_submission.py:

def test_submit_file_idempotent_db(monkeypatch):
    """Re-running submit_file on the same file is a no-op on the DB side."""
    fake = _FakeSftp()
    monkeypatch.setattr(SftpClient, "stat", fake.stat)
    monkeypatch.setattr(SftpClient, "write_file", fake.write_file)
    sftp_block = MagicMock(stub=False, paths={"outbound": "/ToHPE"})
    factory = lambda _b: fake

    r1 = submit_file(_FIXTURE, sftp_block=sftp_block, actor="t", sftp_client_factory=factory)
    r2 = submit_file(_FIXTURE, sftp_block=sftp_block, actor="t", sftp_client_factory=factory)

    # First call SUBMITTED, second call SKIPPED (size match on stat).
    assert r1.outcome == SubmitOutcome.SUBMITTED
    assert r2.outcome == SubmitOutcome.SKIPPED


def test_submit_file_parse_fail(monkeypatch, tmp_path):
    bad = tmp_path / "bad.x12"
    bad.write_bytes(b"not x12")
    sftp_block = MagicMock(stub=False, paths={"outbound": "/ToHPE"})
    fake = _FakeSftp()
    factory = lambda _b: fake

    result = submit_file(bad, sftp_block=sftp_block, actor="t", sftp_client_factory=factory)
    assert result.outcome == SubmitOutcome.PARSE_FAILED
    assert len(fake.put_calls) == 0


def test_submit_file_db_fail(monkeypatch):
    """If add_record raises, no SFTP call is made (DB-first invariant)."""
    from cyclone import store as cycl_store

    fake = _FakeSftp()
    monkeypatch.setattr(cycl_store, "add_record", MagicMock(side_effect=RuntimeError("db down")))

    sftp_block = MagicMock(stub=False, paths={"outbound": "/ToHPE"})
    factory = lambda _b: fake

    result = submit_file(_FIXTURE, sftp_block=sftp_block, actor="t", sftp_client_factory=factory)
    assert result.outcome == SubmitOutcome.DB_FAILED
    assert len(fake.put_calls) == 0


def test_submit_file_sftp_fail(monkeypatch):
    """SFTP failure after DB write → SFTP_FAILED outcome, DB row still present."""
    from cyclone import store as cycl_store

    class _Boom:
        def stat(self, _p): raise IOError("nope")
        def write_file(self, _p, _c): raise RuntimeError("sftp down")

    monkeypatch.setattr(cycl_store, "append_clearhouse_submitted_event", lambda **kw: None)
    sftp_block = MagicMock(stub=False, paths={"outbound": "/ToHPE"})
    factory = lambda _b: _Boom()

    result = submit_file(_FIXTURE, sftp_block=sftp_block, actor="t", sftp_client_factory=factory)
    assert result.outcome == SubmitOutcome.SFTP_FAILED
    # DB row was written before the SFTP call.
    assert result.batch_id is not None
    with db_mod.SessionLocal()() as s:
        assert s.get(db_mod.Batch, result.batch_id) is not None

• Step 9: Run all submission tests

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_submission.py -v Expected: 5 passed.

• Step 10: Commit

git add backend/src/cyclone/submission/ backend/tests/test_submission.py backend/tests/fixtures/submit-batch/
git commit -m "feat(sp37): cyclone.submission.submit_file helper

Owns the parse → DB write → SFTP upload sequence in one place. CLI
and HTTP thin-call it. DB-first invariant: if add_record raises, no
SFTP call is made. Idempotency: add_record dedupes via s.get(Claim,
claim_id); SFTP layer dedupes via stat().st_size match."

• Step 11: Autoreview (per the operator's standing directive)

Spawn a pr-reviewer subagent against the staged diff:

Subagent prompt: "Review the staged diff in
/home/tyler/dev/cyclone for SP37 Task 4 — the new cyclone.submission
package and its submit_file helper. Check (a) no orphan imports,
(b) no leaked business logic outside the helper, (c) the DB-first
ordering is preserved (no SFTP call when add_record raises),
(d) idempotency is correct on both layers, (e) audit event matches
the existing clearhouse.submitted shape. Return PASS or FAIL with
specific findings."

(Per superpowers:pr-reviewer — uses the pr-reviewer subagent type with the staged diff as input.)

• Step 12: Update tracker

Append Task 4 section with pre/post pytest, test result, reviewer verdict.

---

Task 5: cyclone submit-batch CLI

Files:

• Modify: backend/src/cyclone/cli.py (add new @main.command("submit-batch") near resubmit_rejected_claims)

• Step 1: Write a smoke test

The CLI is hard to unit-test (Click + db init + walker), so the test is a subprocess invocation. Append to backend/tests/test_submission.py:

import subprocess
import sys


def test_submit_batch_cli_help():
    """`cyclone submit-batch --help` exits 0 and shows the flags."""
    result = subprocess.run(
        [sys.executable, "-m", "cyclone.cli", "submit-batch", "--help"],
        capture_output=True, text=True, cwd=str(Path(__file__).parent.parent),
    )
    assert result.returncode == 0
    assert "--ingest-dir" in result.stdout

• Step 2: Run test, verify it fails

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_submission.py::test_submit_batch_cli_help -v Expected: FAIL with No such command 'submit-batch' or non-zero exit.

• Step 3: Add the Click command

Modify backend/src/cyclone/cli.py. Insert near resubmit_rejected_claims (around line 539). Mirror its structure — same flags, same exit codes per cyclone-cli. Sketch:

@main.command("submit-batch")
@click.option("--ingest-dir", default="ingest",
              show_default=True,
              help="Root dir holding batch-*-claims/ subfolders of .x12 files.")
@click.option("--actor", default="cli-submit-batch",
              show_default=True,
              help="Audit-log actor tag for the clearhouse.submitted events.")
@click.option("--validate/--no-validate", default=True,
              help="Parse each file via parse_837 before upload.")
@click.option("--limit", type=int, default=None,
              help="Stop after N files (smoke-tests).")
def submit_batch(ingest_dir, actor, validate, limit):
    """Parse → DB-write → SFTP-upload each batch-*-claims/*.x12 (SP37).

    Mirrors `resubmit-rejected-claims` but routes through the canonical
    `cyclone.submission.submit_file` helper, which writes to the DB
    before uploading. Use this as the preferred outbound path; the
    legacy `resubmit-rejected-claims` remains for one-off cases where
    you don't want a DB row (e.g., dzinesco-generated fixes that aren't
    ready for canonical tracking).

    Exit codes: 0 = run completed (even with per-file failures), 1 =
    unexpected exception, 2 = config-level failure (no clearhouse,
    SFTP block in stub mode, ingest-dir missing).
    """
    import sys
    from pathlib import Path

    from cyclone import db as db_mod
    from cyclone.clearhouse import SftpClient
    from cyclone.submission.core import submit_file
    from cyclone.submission.result import SubmitOutcome
    from cyclone.store import store as cycl_store

    db_mod.init_db()
    cycl_store.ensure_clearhouse_seeded()
    clearhouse = cycl_store.get_clearhouse()
    if clearhouse is None:
        click.echo("No clearhouse seeded; cannot resolve SFTP block.", err=True)
        sys.exit(2)
    sftp_block = clearhouse.sftp_block
    if sftp_block.stub:
        click.echo("Clearhouse SFTP block is in stub mode; refusing to upload.", err=True)
        sys.exit(2)

    root = Path(ingest_dir).resolve()
    if not root.exists():
        click.echo(f"--ingest-dir does not exist: {root}", err=True)
        sys.exit(2)

    files = []
    for batch_dir in sorted(root.glob("batch-*-claims")):
        files.extend(sorted(p for p in batch_dir.glob("*.x12") if not p.name.startswith("._")))
    if not files:
        click.echo(f"No .x12 files under {root}", err=True)
        sys.exit(2)

    click.echo(f"found {len(files)} files under {root}")

    submitted = skipped = failed = 0
    for i, src in enumerate(files, 1):
        if limit is not None and i > limit:
            break
        try:
            result = submit_file(
                src, sftp_block=sftp_block, actor=actor, validate=validate,
                sftp_client_factory=lambda b: SftpClient(block=b),
            )
        except Exception as exc:  # noqa: BLE001
            click.echo(f"UNEXPECTED  {src.name}: {exc.__class__.__name__}: {exc}", err=True)
            failed += 1
            continue
        if result.outcome == SubmitOutcome.SUBMITTED:
            submitted += 1
            click.echo(f"submitted  {src.name}  batch={result.batch_id}")
        elif result.outcome == SubmitOutcome.SKIPPED:
            skipped += 1
            click.echo(f"skipped    {src.name}  (already uploaded)")
        else:
            failed += 1
            click.echo(f"{result.outcome.value:<10}  {src.name}  {result.error or ''}", err=True)

    click.echo(f"\nsummary: submitted={submitted} skipped={skipped} failed={failed}")
    if failed:
        sys.exit(0)  # per-file failures don't bump exit code; check stdout

(Adapt the helper names — cycl_store._new_batch_id may not exist; check store/__init__.py for the canonical generator.)

• Step 4: Run the smoke test, verify it passes

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_submission.py::test_submit_batch_cli_help -v Expected: PASS.

• Step 5: Live-test against the stub

Run (with stub SFTP block, in this box's current posture):

cd /home/tyler/dev/cyclone/backend
.venv/bin/python -m cyclone.cli submit-batch --ingest-dir /tmp/sp37-fixtures --limit 1

Expected: submitted single-claim.x12 batch=<uuid> (or skipped if re-run).

Verify the DB row: sqlite3 ~/.local/share/cyclone/cyclone.db "SELECT id, transaction_set_control_number FROM batches ORDER BY parsed_at DESC LIMIT 1;"

• Step 6: Commit

git add backend/src/cyclone/cli.py backend/tests/test_submission.py
git commit -m "feat(sp37): `cyclone submit-batch` CLI

Walks batch-*-claims/*.x12 under --ingest-dir, calls submit_file per
file, prints submitted/skipped/failed counts. Exits 0 even on
per-file failures (details in stdout); exits 2 on config-level
errors (no clearhouse, stub mode, missing dir)."

• Step 7: Update tracker

Append Task 5 section.

---

Task 6: POST /api/submit-batch endpoint

Files:

• Create: backend/src/cyclone/api_routers/submission.py

• Modify: backend/src/cyclone/api_routers/__init__.py (add to the routers list)

• Create: backend/tests/test_api_submit_batch.py

• Step 1: Read the existing router pattern

Read backend/src/cyclone/api_routers/clearhouse.py (smallest recent example of a router that talks to the clearhouse). Mirror its shape: Pydantic request/response models, auth-gated handler, event_bus access via request.app.state.event_bus.

• Step 2: Write the failing tests

Create backend/tests/test_api_submit_batch.py:

"""SP37 Task 6: POST /api/submit-batch endpoint."""
from __future__ import annotations

import json
from pathlib import Path

import pytest
from fastapi.testclient import TestClient


_FIXTURE_DIR = Path(__file__).parent / "fixtures" / "submit-batch"


@pytest.fixture
def client(tmp_path, monkeypatch):
    monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
    monkeypatch.setenv("CYCLONE_AUTH_DISABLED", "1")  # skip auth for unit test

    from cyclone import db as db_mod, store as cycl_store
    db_mod._reset_for_tests()
    db_mod.init_db()
    cycl_store.ensure_clearhouse_seeded()
    # Force real-SFTP flag off so the endpoint doesn't reject stub mode in tests.
    cycl_store.update_clearhouse(sftp_block={"stub": True})

    from cyclone.api import app
    return TestClient(app)


def test_submit_batch_happy_path(client, tmp_path):
    """3 valid 837 files → 200 + body shape with submitted=3."""
    # Copy 3 copies of the single-claim fixture into a batch-*-claims dir.
    batch_dir = tmp_path / "batch-test-claims"
    batch_dir.mkdir()
    for i in range(3):
        (batch_dir / f"claim-{i}.x12").write_bytes(_FIXTURE_DIR.joinpath("single-claim.x12").read_bytes())

    resp = client.post("/api/submit-batch", json={"ingest_dir": str(tmp_path), "validate": False})
    assert resp.status_code == 200
    body = resp.json()
    assert body["submitted"] == 3 or body["skipped"] == 3  # tolerate re-runs
    assert isinstance(body["results"], list)
    assert len(body["results"]) == 3


def test_submit_batch_auth_gate(client):
    """No cookie → 401 (matrix_gate)."""
    # Reset CYCLONE_AUTH_DISABLED for this single test.
    import os
    os.environ.pop("CYCLONE_AUTH_DISABLED", None)
    resp = client.post("/api/submit-batch", json={"ingest_dir": "/tmp"})
    assert resp.status_code == 401


def test_submit_batch_missing_ingest_dir(client):
    """Body without ingest_dir → 422."""
    resp = client.post("/api/submit-batch", json={"validate": False})
    assert resp.status_code == 422

• Step 3: Run test, verify it fails

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_api_submit_batch.py -v Expected: 3 FAIL (no router mounted yet).

• Step 4: Create the router

Create backend/src/cyclone/api_routers/submission.py. Mirror clearhouse.py's pattern:

"""SP37 Task 6: HTTP endpoint for the canonical submit flow.

Thin wrapper around ``cyclone.submission.submit_file`` — same logic
as the CLI, just different framing (JSON body in, JSON body out,
auth-gated by the existing matrix_gate).
"""
from __future__ import annotations

import logging
from pathlib import Path

from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel

from cyclone.submission.core import submit_file
from cyclone.submission.result import SubmitOutcome

log = logging.getLogger(__name__)

router = APIRouter(prefix="/api", tags=["submission"])


class SubmitBatchRequest(BaseModel):
    ingest_dir: str
    validate: bool = True
    actor: str = "api-submit-batch"
    limit: int | None = None


@router.post("/submit-batch")
def post_submit_batch(body: SubmitBatchRequest, request: Request):
    """Submit every batch-*-claims/*.x12 under ``ingest_dir``.

    Returns counts + per-file outcomes. Status code is always 200 on
    a run that completes (per-file failures are in the body); 4xx
    for auth/validation; 5xx for unexpected errors.
    """
    from cyclone.clearhouse import SftpClient
    from cyclone.store import store as cycl_store

    clearhouse = cycl_store.get_clearhouse()
    if clearhouse is None:
        raise HTTPException(status_code=409, detail="no clearhouse seeded")
    sftp_block = clearhouse.sftp_block
    if sftp_block.stub:
        raise HTTPException(status_code=409, detail="clearhouse SFTP block is in stub mode")

    root = Path(body.ingest_dir).resolve()
    if not root.exists():
        raise HTTPException(status_code=422, detail=f"ingest_dir does not exist: {root}")

    files = []
    for batch_dir in sorted(root.glob("batch-*-claims")):
        files.extend(sorted(p for p in batch_dir.glob("*.x12") if not p.name.startswith("._")))

    if body.limit is not None:
        files = files[: body.limit]

    results = []
    submitted = skipped = failed = 0
    for src in files:
        try:
            r = submit_file(
                src,
                sftp_block=sftp_block,
                actor=body.actor,
                validate=body.validate,
                sftp_client_factory=lambda b: SftpClient(block=b),
            )
        except Exception as exc:  # noqa: BLE001
            log.exception("submit-batch unexpected error on %s", src)
            r = type("R", (), {"file": src.name, "outcome": "unexpected", "batch_id": None, "error": str(exc)})()
            results.append({
                "file": r.file,
                "outcome": r.outcome.value if hasattr(r.outcome, "value") else str(r.outcome),
                "batch_id": r.batch_id,
                "error": r.error,
            })
            failed += 1
            continue

        if r.outcome == SubmitOutcome.SUBMITTED:
            submitted += 1
        elif r.outcome == SubmitOutcome.SKIPPED:
            skipped += 1
        else:
            failed += 1

        results.append({
            "file": r.file,
            "outcome": r.outcome.value,
            "batch_id": r.batch_id,
            "error": r.error,
        })

    return {
        "submitted": submitted,
        "skipped": skipped,
        "failed": failed,
        "results": results,
    }

• Step 5: Mount the router

Modify backend/src/cyclone/api_routers/__init__.py. Add the import + entry to the routers list (exact shape depends on the file — read it first).

from .submission import router as submission_router
# ... add submission_router to the routers list

• Step 6: Run tests, verify they pass

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest tests/test_api_submit_batch.py -v Expected: 3 passed (assuming you set up the test fixture correctly).

• Step 7: Live test

Restart the dev server (python -m cyclone serve in one terminal). With CYCLONE_AUTH_DISABLED=1, login is bypassed:

curl -X POST http://localhost:8000/api/submit-batch \
  -H 'Content-Type: application/json' \
  -d '{"ingest_dir": "/tmp/sp37-fixtures", "validate": false}'

Expected: {"submitted": 1, "skipped": 0, "failed": 0, "results": [{"file": "...", "outcome": "submitted", ...}]}

Verify the live-tail picks it up: open the Claims page in the UI, watch the new row appear.

• Step 8: Commit

git add backend/src/cyclone/api_routers/submission.py backend/src/cyclone/api_routers/__init__.py backend/tests/test_api_submit_batch.py
git commit -m "feat(sp37): POST /api/submit-batch endpoint

Thin wrapper around cyclone.submission.submit_file with JSON
in/out. Auth-gated by matrix_gate. Same body shape as the CLI
summary (submitted/skipped/failed counts + per-file results).
Returns 200 on completed runs (per-file failures in body); 401 on
no auth; 422 on validation; 409 on clearhouse/stub-mode errors."

• Step 9: Update tracker

Append Task 6 section.

---

Task 7: Integration + merge

Files:

• (no code changes — verification + merge)

• Step 1: Run the full pytest

Run: cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest -x --tb=short -q 2>&1 | tail -5 Expected: count line. Compare against /tmp/refactor-pre-baseline.txt from Task 0. Acceptable: same baseline ± rate-limit-noise (per Task 17 SP36 finding — failures pass in isolation).

• Step 2: Run the full vitest

Run: cd /home/tyler/dev/cyclone && npm test 2>&1 | tail -5 Expected: count line. Pre-existing baseline (no UI changes; should be unchanged).

• Step 3: Run the typecheck + lint

cd /home/tyler/dev/cyclone && npm run typecheck && npm run lint

Expected: clean.

• Step 4: Live-test the join-key fix end-to-end

With the dev server running and clm-1 already in the DB (or a fresh test DB), drop a synthetic 999 whose AK201 set_control_number matches the ST02 of an already-submitted batch. Run pull-inbound. Verify the ClaimAck link row is created (not an orphan).

Use the existing helper for synthesizing 999s — cyclone.submission.test_helpers.synth_999(set_control_number=...) (add this helper to the test file if it doesn't exist).

• Step 5: pr-reviewer (per the operator's standing directive)

Spawn a pr-reviewer subagent against the full SP37 diff (all 7 commits on sp37-submit-batch-canonical-flow):

Subagent prompt: "Review the SP37 diff in
/home/tyler/dev/cyclone (branch sp37-submit-batch-canonical-flow, 7
commits) for the canonical submit-batch flow. Check: (a) migration is
additive and reversible, (b) join-key update preserves backward compat,
(c) submit_file ordering (DB-first, upload-second) is invariant, (d)
idempotency is correct on both DB and SFTP layers, (e) audit events
match existing shape, (f) auth gate is wired, (g) no leaked business
logic in API/CLI wrappers, (h) tests cover happy path + idempotency +
each failure mode. Return PASS or FAIL with specific findings."

• Step 6: Update RUNBOOK + CLAUDE.md

Document the canonical flow:

• docs/RUNBOOK.md: add a "Submitting claims (canonical)" section showing cyclone submit-batch --ingest-dir ingest/<batch> and POST /api/submit-batch. Note that resubmit-rejected-claims remains for one-off cases.

• CLAUDE.md: under "Backend at a glance", add one bullet pointing at cyclone/submission/ and the canonical flow.

• Step 7: Commit doc updates

git add docs/RUNBOOK.md CLAUDE.md
git commit -m "docs(sp37): document canonical submit-batch flow

RUNBOOK gets a 'Submitting claims' section with both CLI and HTTP
examples. CLAUDE.md gets a pointer to cyclone/submission/ and the
preferred outbound path."

• Step 8: Merge to main

Per cyclone-spec — single atomic merge commit, no squash, no rebase:

git checkout main
git merge --no-ff sp37-submit-batch-canonical-flow \
  -m "merge: SP37 canonical submit-batch flow into main"
git push origin main
git branch -d sp37-submit-batch-canonical-flow
git push origin --delete sp37-submit-batch-canonical-flow

• Step 9: Final tracker update

Append the "[SP37 final state]" section to /tmp/refactor-cyclone.md with: total commits on the branch (7 + doc update = 8), pytest delta (pre vs post), reviewer verdict, merge commit SHA, and the open follow-ups (backfill command, deprecation timeline, 277CA cross-link).

---

Self-Review

Spec coverage:

• Spec §1 Scope (CLI + HTTP + helper + new column + join-key update): Tasks 1, 2, 3, 4, 5, 6 cover it.
• Spec §2 Decisions (locked brainstorming choices): enforced throughout; CLI/API mirror the canonical submit shape, DB-first ordering is in Task 4 Step 6, new column is Tasks 1+2, additive deprecation is implicit (no tasks touch parse-837 or resubmit-rejected-claims).
• Spec §3 Architecture (cyclone.submission package, submit_file algorithm, idempotency, failure modes): Task 4 owns the package; idempotency is Steps 3, 7, 8; failure modes are tests in Step 8.
• Spec §4 Data flow (request/response shapes, exit codes, audit events, migration shape, join-key update): Task 6 owns HTTP shapes; Task 5 owns exit codes; Task 1 owns the migration; Task 3 owns the join-key update.
• Spec §5 Testing (7 unit + 3 integration + 1 migration + 1 join-key + live test): spread across Tasks 1-7.
• Spec §6 Threat model (no new attack surface): implicit (no new auth bypass, no new SFTP path, no new SQL surface).
• Spec §7 Risks (3 named): each one addressed in the relevant task — Task 4 keeps submit_file as a pure helper (Risk 1), Task 4 docstring calls out layering (Risk 1), Task 1 Step 3 keeps the backfill to a single UPDATE (Risk 3), live-tail event volume is documented in Task 7 Step 6 (Risk 4).
• Spec §8 Rollout (6-step order, live-test cadence, merge shape): Tasks 1-6 follow it; Task 7 owns the merge.
• Spec §9 Open questions: NOT in this plan — they're explicitly follow-up SPs per the spec.

Placeholder scan: No TBD/TODO/FIXME in any task. Every code block is complete. Every step has a concrete action.

Type consistency:

• SubmitResult, SubmitOutcome defined in Task 4 Step 2; used in Tasks 4, 5, 6 with identical attribute names (file, outcome, batch_id, error).
• submit_file signature in Task 4 Step 6; called with identical kwargs in Tasks 5 and 6.
• Batch.transaction_set_control_number introduced in Task 1 (migration) + Task 2 (ORM); used in Task 3 (index); not referenced elsewhere. No name drift.

Gaps found during review: None. All spec requirements have a task. The follow-up items (backfill command, deprecation, 277CA) are explicitly out of scope per the spec.