cyclone/docs/superpowers/plans/2026-07-07-cyclone-orphan-ack-housekeeping.md

Orphan-ack housekeeping Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Surface the 804 historical orphan acks to operators via a RUNBOOK entry + CLI, and provide a one-shot idempotent synthetic-batch seeder so future acks for those ST02s can resolve against the batch_envelope_index instead of remaining forever-orphans.

Architecture: Two thin CLI subcommands under cyclone ack-orphans {status,reconcile} backed by a single store helper find_ack_orphan_st02_summary(). The helper returns a structured list[dict]; the CLI formats it. Reconcile uses the same SQLAlchemy session the rest of the store uses, so it inherits the DB-first invariant (no direct sqlite3 calls, no parallel writers).

Tech Stack: Python 3.11, FastAPI, SQLAlchemy, Click (CLI), pytest, the existing backend/tests/conftest.py autouse fixtures.

Spec: docs/superpowers/specs/2026-07-07-cyclone-orphan-ack-housekeeping-design.md

---

File structure

backend/
  src/cyclone/
    cli.py                              # + ack-orphans status + reconcile subcommands
    store/
      __init__.py                       # + find_ack_orphan_st02_summary() facade method
      claim_acks.py                     # extract _iter_orphan_st02s() helper
  tests/
    test_ack_orphan_summary.py          # NEW: store-helper tests
    test_ack_orphans_cli.py             # NEW: CLI tests via CliRunner
docs/
  RUNBOOK.md                            # + "Known historical drift" section

---

Task 1: Extract _iter_orphan_st02s() helper from find_ack_orphans

The current find_ack_orphans(kind) walks the acks table directly and does its own LEFT JOIN against claim_acks. Both the new store helper (find_ack_orphan_st02_summary) and the existing find_ack_orphans need to enumerate the orphan ST02s. Extract the walk into a private helper in claim_acks.py that returns a flat list of (st02, ack_count) tuples — the only join key needed for the summary, and the seed for find_ack_orphans's per-kind filter.

• Read backend/src/cyclone/store/claim_acks.py:250-330 to confirm the current shape of find_ack_orphans.
• Add _iter_orphan_st02s() -> Iterator[tuple[str, int]] to claim_acks.py. One SELECT against acks joining claim_acks on (claim_acks.ack_kind = '999' AND claim_acks.ack_id = acks.id), yielding (set_control_number, ack_count) for each group where no claim_acks row exists. For 277ca / ta1 the same shape but with the matching ack table and claim_acks.ack_kind filter.
• Refactor find_ack_orphans(kind) to consume the helper. No behavior change; same return shape, same idempotent semantics. All existing tests must still pass.

RED→GREEN test gate: cd backend && .venv/bin/pytest tests/test_api_claim_acks.py tests/test_apply_claim_ack_links.py -v must remain green.

Task 2: Add find_ack_orphan_st02_summary() to the store facade

• Add find_ack_orphan_st02_summary(self) -> list[dict] to CycloneStore in backend/src/cyclone/store/__init__.py. Returns [{"st02": str, "ack_count": int, "has_batch": bool, "batch_id": str | None}, ...].
• has_batch is True if a row exists in batches with transaction_set_control_number = st02. batch_id is that row's id (or None).
• Sort the result by ack_count DESC so the heaviest orphans surface first in the CLI output.

RED test first: write backend/tests/test_ack_orphan_summary.py::test_summary_returns_per_st02_counts with a seeded DB (two orphan ST02s, one with a batch, one without) and assert the shape. Watch it fail (no facade method yet), then implement to GREEN.

Task 3: Test the synthetic-batch seeder

The reconcile CLI inserts synthetic batches rows for any orphan ST02 that doesn't already have one. This is a write; the store helper needs a paired write-side method.

• Add reconcile_orphan_st02s(self, dry_run: bool = False) -> dict to CycloneStore. Returns {"created": int, "skipped": int, "synthetic_batch_ids": list[str]}.
• For each ST02 in the summary where has_batch is False, insert a batches row with: kind = '837p', input_filename = '<synthetic:orphan-reconcile>', totals_json = '{"orphan_reconcile": true, "ack_count": N}', validation_json = '{"orphan_reconcile": true, "note": "sp38 synthetic batch row; source 837 was never ingested into this DB snapshot"}', id = uuid4().hex. The remaining columns take their defaults.
• When dry_run=True, return the plan without writing.
• Idempotency: re-running after a previous reconcile returns created=0, skipped=N. Test this explicitly.

RED test: test_reconcile_is_idempotent — run reconcile twice, assert the row count is unchanged after the second call.

RED test: test_reconcile_uses_synthetic_input_filename_sentinel — assert every created row has input_filename = '<synthetic:orphan-reconcile>'.

RED test: test_reconcile_dry_run_does_not_write — pass dry_run=True, assert created == N in the plan but batches row count is unchanged.

Task 4: Add cyclone ack-orphans status CLI subcommand

• Open backend/src/cyclone/cli.py, locate the existing submit-batch group.
• Add an ack-orphans group with a status subcommand.
• status calls cycl_store.find_ack_orphan_st02_summary(), prints a table: ST02 | ACK COUNT | HAS BATCH. Total line at the bottom.
• Exit 0 on success, exit 1 on DB error (matching the cyclone-cli convention: 1 = unexpected exception, including SQLAlchemy DB errors).

RED test: tests/test_ack_orphans_cli.py::test_status_prints_table using CliRunner.invoke(["ack-orphans", "status"]). Assert exit code 0, assert the table contains the seeded ST02s and the total line.

Task 5: Add cyclone ack-orphans reconcile CLI subcommand

• Same ack-orphans group, reconcile subcommand.
• Accepts --dry-run flag (default False).
• Calls cycl_store.reconcile_orphan_st02s(dry_run=dry_run), prints Created: N synthetic batch rows. Skipped: M (already had a batch row)..
• Exit 0 on success, exit 1 on DB error (same convention as status).

RED test: tests/test_ack_orphans_cli.py::test_reconcile_creates_synthetic_batches — seed two orphan ST02s (one with batch, one without), invoke reconcile, assert created=1, assert a row with input_filename='<synthetic:orphan-reconcile>' exists.

RED test: tests/test_ack_orphans_cli.py::test_reconcile_dry_run_flag — invoke reconcile --dry-run, assert created=1 in plan but no DB row inserted.

RED test: tests/test_ack_orphans_cli.py::test_reconcile_is_idempotent — invoke reconcile twice, assert the second invocation reports created=0.

Task 6: Add RUNBOOK.md entry

• Open docs/RUNBOOK.md, find the existing "Operator triage" section.
• Add a ## Known historical drift subsection under it.
• Content (3-4 paragraphs):
  • What the 804 orphans are (real production 999s whose source 837s pre-date the current DB snapshot).
  • Why they cannot be auto-linked (the source 837s were not preserved).
  • How to triage them via the Inbox > AckOrphansLane.
  • Optional: cyclone ack-orphans status to see the ST02 breakdown, cyclone ack-orphans reconcile to seed synthetic batch rows for them.

Task 7: Live verification + autoreview + commit

• Run the full backend test suite: cd backend && .venv/bin/pytest. Confirm pre-existing failure count (20) is unchanged.
• Live-run cyclone ack-orphans status against the current ~/.local/share/cyclone/cyclone.db. Confirm output matches the SQLite investigation: 5 distinct ST02s, 805 total orphan rows.
• Run autoreview (subagent) on the diff.
• Commit with the SP-N commit prefixes. Per the SP38 spec, the work is: store-helper extraction (Tasks 1-3), CLI (Tasks 4-5), docs (Task 6). Three commits, all prefixed feat(sp38): …. Plus a merge: SP38 orphan-ack housekeeping into main commit at the end of the review cycle.

---

Anti-patterns

• Don't add a UI for the CLI commands. Spec is explicit: no UI change.
• Don't auto-run reconcile on boot. Operators invoke it manually after they accept the drift.
• Don't try to backfill claim rows. Source data is gone; no way to fabricate claim rows that would be meaningful.
• Don't put the synthetic sentinel in a constant somewhere. The string <synthetic:orphan-reconcile> appears in the spec, the plan, the store helper, the CLI output, and the seeded batches table. Keep it inline so a future grep finds it.

Related skills to load while implementing

• cyclone-spec — already loaded (this plan is its output).
• cyclone-cli — for the click.testing.CliRunner pattern and the exit-code convention.
• cyclone-store — for the facade re-export pattern in store/__init__.py.
• cyclone-tests — for the tmp_path/test.db conftest pattern and the BACKFILL_SQL-loads-from-file pattern from SP37 followup #1.