docs(plan): SP38 orphan-ack housekeeping plan
Implements docs/superpowers/specs/2026-07-07-cyclone-orphan-ack-housekeeping-design.md.
Seven tasks, all TDD-shaped (RED test first, then implementation):
Task 1: Extract _iter_orphan_st02s() helper from find_ack_orphans
(refactor; no behavior change; existing tests must stay green)
Task 2: Add find_ack_orphan_st02_summary() to CycloneStore facade
(RED: test_summary_returns_per_st02_counts)
Task 3: Add reconcile_orphan_st02s() to CycloneStore
(RED: test_reconcile_is_idempotent + sentinel + dry-run)
Task 4: Add 'cyclone ack-orphans status' CLI subcommand
(RED: test_status_prints_table via CliRunner)
Task 5: Add 'cyclone ack-orphans reconcile' CLI subcommand
(RED: test_reconcile_creates_synthetic_batches + idempotent)
Task 6: Add RUNBOOK.md 'Known historical drift' section
Task 7: Full pytest + live CLI verification + autoreview + commits
Per cyclone-spec, commits are feat(sp38): ... for implementation
work and the final merge commit is 'merge: SP38 orphan-ack
housekeeping into main'. No schema migration; no UI change; no
auto-runnable reconcile.
This commit is contained in:
Nora
@@ -0,0 +1,212 @@
|
||||
# Orphan-ack housekeeping Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||
> superpowers:subagent-driven-development (recommended) or
|
||||
> superpowers:executing-plans to implement this plan task-by-task. Steps
|
||||
> use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Surface the 804 historical orphan acks to operators via a
|
||||
RUNBOOK entry + CLI, and provide a one-shot idempotent synthetic-batch
|
||||
seeder so future acks for those ST02s can resolve against the
|
||||
`batch_envelope_index` instead of remaining forever-orphans.
|
||||
|
||||
**Architecture:** Two thin CLI subcommands under `cyclone ack-orphans
|
||||
{status,reconcile}` backed by a single store helper
|
||||
`find_ack_orphan_st02_summary()`. The helper returns a structured
|
||||
list[dict]; the CLI formats it. Reconcile uses the same SQLAlchemy
|
||||
session the rest of the store uses, so it inherits the DB-first
|
||||
invariant (no direct sqlite3 calls, no parallel writers).
|
||||
|
||||
**Tech Stack:** Python 3.11, FastAPI, SQLAlchemy, Click (CLI),
|
||||
pytest, the existing `backend/tests/conftest.py` autouse fixtures.
|
||||
|
||||
**Spec:** [`docs/superpowers/specs/2026-07-07-cyclone-orphan-ack-housekeeping-design.md`](../specs/2026-07-07-cyclone-orphan-ack-housekeeping-design.md)
|
||||
|
||||
---
|
||||
|
||||
## File structure
|
||||
|
||||
```
|
||||
backend/
|
||||
src/cyclone/
|
||||
cli.py # + ack-orphans status + reconcile subcommands
|
||||
store/
|
||||
__init__.py # + find_ack_orphan_st02_summary() facade method
|
||||
claim_acks.py # extract _iter_orphan_st02s() helper
|
||||
tests/
|
||||
test_ack_orphan_summary.py # NEW: store-helper tests
|
||||
test_ack_orphans_cli.py # NEW: CLI tests via CliRunner
|
||||
docs/
|
||||
RUNBOOK.md # + "Known historical drift" section
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Extract `_iter_orphan_st02s()` helper from `find_ack_orphans`
|
||||
|
||||
The current `find_ack_orphans(kind)` walks the `acks` table directly
|
||||
and does its own LEFT JOIN against `claim_acks`. Both the new store
|
||||
helper (`find_ack_orphan_st02_summary`) and the existing
|
||||
`find_ack_orphans` need to enumerate the orphan ST02s. Extract the
|
||||
walk into a private helper in `claim_acks.py` that returns a flat
|
||||
list of `(st02, ack_count)` tuples — the only join key needed for
|
||||
the summary, and the seed for `find_ack_orphans`'s per-kind filter.
|
||||
|
||||
- [ ] Read `backend/src/cyclone/store/claim_acks.py:250-330` to confirm
|
||||
the current shape of `find_ack_orphans`.
|
||||
- [ ] Add `_iter_orphan_st02s() -> Iterator[tuple[str, int]]` to
|
||||
`claim_acks.py`. One SELECT against `acks` joining
|
||||
`claim_acks` on `(claim_acks.ack_kind = '999' AND
|
||||
claim_acks.ack_id = acks.id)`, yielding `(set_control_number,
|
||||
ack_count)` for each group where no claim_acks row exists.
|
||||
For 277ca / ta1 the same shape but with the matching ack table
|
||||
and `claim_acks.ack_kind` filter.
|
||||
- [ ] Refactor `find_ack_orphans(kind)` to consume the helper.
|
||||
No behavior change; same return shape, same idempotent
|
||||
semantics. All existing tests must still pass.
|
||||
|
||||
**RED→GREEN test gate:** `cd backend && .venv/bin/pytest
|
||||
tests/test_api_claim_acks.py tests/test_apply_claim_ack_links.py -v`
|
||||
must remain green.
|
||||
|
||||
## Task 2: Add `find_ack_orphan_st02_summary()` to the store facade
|
||||
|
||||
- [ ] Add `find_ack_orphan_st02_summary(self) -> list[dict]` to
|
||||
`CycloneStore` in `backend/src/cyclone/store/__init__.py`.
|
||||
Returns `[{"st02": str, "ack_count": int,
|
||||
"has_batch": bool, "batch_id": str | None}, ...]`.
|
||||
- [ ] `has_batch` is `True` if a row exists in `batches` with
|
||||
`transaction_set_control_number = st02`. `batch_id` is that
|
||||
row's `id` (or `None`).
|
||||
- [ ] Sort the result by `ack_count DESC` so the heaviest orphans
|
||||
surface first in the CLI output.
|
||||
|
||||
**RED test first:** write
|
||||
`backend/tests/test_ack_orphan_summary.py::test_summary_returns_per_st02_counts`
|
||||
with a seeded DB (two orphan ST02s, one with a batch, one without)
|
||||
and assert the shape. Watch it fail (no facade method yet), then
|
||||
implement to GREEN.
|
||||
|
||||
## Task 3: Test the synthetic-batch seeder
|
||||
|
||||
The reconcile CLI inserts synthetic `batches` rows for any orphan
|
||||
ST02 that doesn't already have one. This is a write; the store
|
||||
helper needs a paired write-side method.
|
||||
|
||||
- [ ] Add `reconcile_orphan_st02s(self, dry_run: bool = False) ->
|
||||
dict` to `CycloneStore`. Returns `{"created": int, "skipped":
|
||||
int, "synthetic_batch_ids": list[str]}`.
|
||||
- [ ] For each ST02 in the summary where `has_batch is False`, insert
|
||||
a `batches` row with: `kind = '837p'`, `input_filename =
|
||||
'<synthetic:orphan-reconcile>'`, `totals_json =
|
||||
'{"orphan_reconcile": true, "ack_count": N}'`,
|
||||
`validation_json = '{"orphan_reconcile": true, "note":
|
||||
"sp38 synthetic batch row; source 837 was never ingested into
|
||||
this DB snapshot"}'`, `id = uuid4().hex`. The remaining
|
||||
columns take their defaults.
|
||||
- [ ] When `dry_run=True`, return the plan without writing.
|
||||
- [ ] Idempotency: re-running after a previous reconcile returns
|
||||
`created=0, skipped=N`. Test this explicitly.
|
||||
|
||||
**RED test:** `test_reconcile_is_idempotent` — run reconcile twice,
|
||||
assert the row count is unchanged after the second call.
|
||||
|
||||
**RED test:** `test_reconcile_uses_synthetic_input_filename_sentinel`
|
||||
— assert every created row has `input_filename =
|
||||
'<synthetic:orphan-reconcile>'`.
|
||||
|
||||
**RED test:** `test_reconcile_dry_run_does_not_write` — pass
|
||||
`dry_run=True`, assert `created == N` in the plan but
|
||||
`batches` row count is unchanged.
|
||||
|
||||
## Task 4: Add `cyclone ack-orphans status` CLI subcommand
|
||||
|
||||
- [ ] Open `backend/src/cyclone/cli.py`, locate the existing
|
||||
`submit-batch` group.
|
||||
- [ ] Add an `ack-orphans` group with a `status` subcommand.
|
||||
- [ ] `status` calls `cycl_store.find_ack_orphan_st02_summary()`,
|
||||
prints a table: `ST02 | ACK COUNT | HAS BATCH`. Total line at
|
||||
the bottom.
|
||||
- [ ] Exit 0 on success, exit 2 on DB error (matching `cyclone-cli`
|
||||
convention).
|
||||
|
||||
**RED test:** `tests/test_ack_orphans_cli.py::test_status_prints_table`
|
||||
using `CliRunner.invoke(["ack-orphans", "status"])`. Assert exit
|
||||
code 0, assert the table contains the seeded ST02s and the total
|
||||
line.
|
||||
|
||||
## Task 5: Add `cyclone ack-orphans reconcile` CLI subcommand
|
||||
|
||||
- [ ] Same `ack-orphans` group, `reconcile` subcommand.
|
||||
- [ ] Accepts `--dry-run` flag (default False).
|
||||
- [ ] Calls `cycl_store.reconcile_orphan_st02s(dry_run=dry_run)`,
|
||||
prints `Created: N synthetic batch rows. Skipped: M (already
|
||||
had a batch row).`.
|
||||
- [ ] Exit 0 on success, exit 2 on DB error.
|
||||
|
||||
**RED test:** `tests/test_ack_orphans_cli.py::test_reconcile_creates_synthetic_batches`
|
||||
— seed two orphan ST02s (one with batch, one without), invoke
|
||||
`reconcile`, assert `created=1`, assert a row with
|
||||
`input_filename='<synthetic:orphan-reconcile>'` exists.
|
||||
|
||||
**RED test:** `tests/test_ack_orphans_cli.py::test_reconcile_dry_run_flag`
|
||||
— invoke `reconcile --dry-run`, assert `created=1` in plan but
|
||||
no DB row inserted.
|
||||
|
||||
**RED test:** `tests/test_ack_orphans_cli.py::test_reconcile_is_idempotent`
|
||||
— invoke `reconcile` twice, assert the second invocation reports
|
||||
`created=0`.
|
||||
|
||||
## Task 6: Add RUNBOOK.md entry
|
||||
|
||||
- [ ] Open `docs/RUNBOOK.md`, find the existing "Operator triage"
|
||||
section.
|
||||
- [ ] Add a `## Known historical drift` subsection under it.
|
||||
- [ ] Content (3-4 paragraphs):
|
||||
- What the 804 orphans are (real production 999s whose source
|
||||
837s pre-date the current DB snapshot).
|
||||
- Why they cannot be auto-linked (the source 837s were not
|
||||
preserved).
|
||||
- How to triage them via the Inbox > AckOrphansLane.
|
||||
- Optional: `cyclone ack-orphans status` to see the ST02
|
||||
breakdown, `cyclone ack-orphans reconcile` to seed synthetic
|
||||
batch rows for them.
|
||||
|
||||
## Task 7: Live verification + autoreview + commit
|
||||
|
||||
- [ ] Run the full backend test suite: `cd backend && .venv/bin/pytest`.
|
||||
Confirm pre-existing failure count (20) is unchanged.
|
||||
- [ ] Live-run `cyclone ack-orphans status` against the current
|
||||
`~/.local/share/cyclone/cyclone.db`. Confirm output matches the
|
||||
SQLite investigation: 5 distinct ST02s, 805 total orphan rows.
|
||||
- [ ] Run autoreview (subagent) on the diff.
|
||||
- [ ] Commit with the SP-N commit prefixes. Per the SP38 spec, the
|
||||
work is: store-helper extraction (Tasks 1-3), CLI (Tasks 4-5),
|
||||
docs (Task 6). Three commits, all prefixed
|
||||
`feat(sp38): …`. Plus a `merge: SP38 orphan-ack housekeeping
|
||||
into main` commit at the end of the review cycle.
|
||||
|
||||
---
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
- **Don't add a UI for the CLI commands.** Spec is explicit: no UI
|
||||
change.
|
||||
- **Don't auto-run reconcile on boot.** Operators invoke it
|
||||
manually after they accept the drift.
|
||||
- **Don't try to backfill claim rows.** Source data is gone; no
|
||||
way to fabricate claim rows that would be meaningful.
|
||||
- **Don't put the synthetic sentinel in a constant somewhere.** The
|
||||
string `<synthetic:orphan-reconcile>` appears in the spec, the
|
||||
plan, the store helper, the CLI output, and the seeded batches
|
||||
table. Keep it inline so a future grep finds it.
|
||||
|
||||
## Related skills to load while implementing
|
||||
|
||||
- `cyclone-spec` — already loaded (this plan is its output).
|
||||
- `cyclone-cli` — for the `click.testing.CliRunner` pattern and the
|
||||
exit-code convention.
|
||||
- `cyclone-store` — for the facade re-export pattern in
|
||||
`store/__init__.py`.
|
||||
- `cyclone-tests` — for the `tmp_path/test.db` conftest pattern and
|
||||
the BACKFILL_SQL-loads-from-file pattern from SP37 followup #1.
|
||||
Reference in New Issue
Block a user