docs(sp38): RUNBOOK entry for orphan-ack housekeeping

Implements task 6 of
docs/superpowers/plans/2026-07-07-cyclone-orphan-ack-housekeeping.md.

Adds 'Known historical drift — the 804 orphan 999s' section under
the existing operator-triage content. Covers:

  - What the orphans are (real production 999s whose source 837s
    predate the current DB snapshot; valid audit history that
    cannot be auto-linked)
  - Why they cannot be auto-linked (source 837s were transmitted
    to HPE and never came back; Cyclone is downstream and does
    not retain copies of outbound 837s)
  - Triage path via the Inbox AckOrphansLane (already working
    as of SP37-followup 893a662)
  - Optional synthetic-batch seeding via 'cyclone ack-orphans
    reconcile' (one-shot, idempotent, --dry-run available)
  - 'cyclone ack-orphans status' for the per-ST02 breakdown

Explicitly calls out what the housekeeping is NOT:
  - Not a backfill (no claims rows are synthesized)
  - Not auto-runnable (operator-invoked only)
  - Not a deletion (orphans are valid audit history)

References the SP38 spec and plan for the design rationale.
This commit is contained in:
Nora
2026-07-07 12:53:51 -06:00
parent 9ef749c783
commit 76923a79f5
+74
View File
@@ -291,3 +291,77 @@ curl -X PATCH http://127.0.0.1:8000/api/clearhouse \
```
To revert, set `stub: true` and `auth: {"method": "keychain", "secret_ref": "sftp.gainwell.password"}`.
## Known historical drift — the 804 orphan 999s
The `acks` table may hold several hundred 999 acks whose source 837
batch is not present in the `batches` table — the Inbox "Ack orphans"
lane surfaces them at `GET /api/inbox/ack-orphans`. These are real
production 999s (e.g. sender_id `COMEDASSISTPROG`) whose source 837s
were submitted to HPE clearinghouse before the current
`~/.local/share/cyclone/cyclone.db` snapshot was created. The source
837s themselves were never re-ingested into the current DB, so the
`claims` table has no rows that could link against them. SP37's
canonical submit-batch flow captures ST02 going forward, so the
orphan count is stable — not a forward-looking bug, just historical
drift.
**You cannot auto-link these orphans.** The source 837s are not in
`ingest/`, `backend/var/sftp/staging/`, or any local path — they
were transmitted to HPE and never came back. Cyclone is downstream
of the clearinghouse and does not retain copies of outbound 837s
after SFTP ACK. Do not attempt to re-ingest from SFTP inbound —
those files are the 999 acks themselves, not the source 837s.
### Triage path
1. **Inspect the Inbox > AckOrphansLane** in the UI. Each row is a
999 ack with no resolvable claim. Sort by `parsed_at DESC` to
see the most recent first; older orphans are less likely to be
actionable.
2. **Decide per row.** If the operator can identify the source 837
outside Cyclone (e.g. a manual record of what was submitted that
day), manually create a `claims` row + a `claim_acks` link via
`POST /api/parse-837` followed by `POST /api/inbox/candidates/{remit_id}/match`
or `POST /api/acks/.../match-claim`. If not, leave the orphan
alone — it stays as valid audit history.
### Optional: seed synthetic batch rows (one-shot)
`cyclone ack-orphans reconcile` inserts a synthetic `batches` row
for every orphan ST02 that doesn't already have one. The synthetic
row is marked with `input_filename = '<synthetic:orphan-reconcile>'`
so it's trivially distinguishable in queries. Future 999 acks for
the same ST02s will resolve against the batch envelope index (so
the operator can see "this 999 is for a known orphan source") but
will not link to claims (because no claim rows exist).
```bash
# Preview the reconcile without writing rows.
cyclone ack-orphans reconcile --dry-run
# Insert the synthetic batch rows.
cyclone ack-orphans reconcile
```
Re-running `cyclone ack-orphans reconcile` after a successful pass
is a no-op (idempotent). To inspect the per-ST02 breakdown at any
time:
```bash
cyclone ack-orphans status
```
The status command prints a table with ST02, ACK COUNT, and HAS
BATCH columns, ranked by ack count so the heaviest backlog
surfaces first.
### What this is NOT
- **Not a backfill.** No `claims` rows are synthesized — the source
data is gone.
- **Not auto-runnable.** The reconcile CLI is operator-invoked only;
it does not run on boot, in the SFTP polling scheduler, or via
cron.
- **Not a deletion.** The orphan 999s are valid audit history and
must remain queryable.