docs(sp38): RUNBOOK entry for orphan-ack housekeeping
Implements task 6 of
docs/superpowers/plans/2026-07-07-cyclone-orphan-ack-housekeeping.md.
Adds 'Known historical drift — the 804 orphan 999s' section under
the existing operator-triage content. Covers:
- What the orphans are (real production 999s whose source 837s
predate the current DB snapshot; valid audit history that
cannot be auto-linked)
- Why they cannot be auto-linked (source 837s were transmitted
to HPE and never came back; Cyclone is downstream and does
not retain copies of outbound 837s)
- Triage path via the Inbox AckOrphansLane (already working
as of SP37-followup 893a662)
- Optional synthetic-batch seeding via 'cyclone ack-orphans
reconcile' (one-shot, idempotent, --dry-run available)
- 'cyclone ack-orphans status' for the per-ST02 breakdown
Explicitly calls out what the housekeeping is NOT:
- Not a backfill (no claims rows are synthesized)
- Not auto-runnable (operator-invoked only)
- Not a deletion (orphans are valid audit history)
References the SP38 spec and plan for the design rationale.
This commit is contained in:
@@ -291,3 +291,77 @@ curl -X PATCH http://127.0.0.1:8000/api/clearhouse \
|
||||
```
|
||||
|
||||
To revert, set `stub: true` and `auth: {"method": "keychain", "secret_ref": "sftp.gainwell.password"}`.
|
||||
|
||||
## Known historical drift — the 804 orphan 999s
|
||||
|
||||
The `acks` table may hold several hundred 999 acks whose source 837
|
||||
batch is not present in the `batches` table — the Inbox "Ack orphans"
|
||||
lane surfaces them at `GET /api/inbox/ack-orphans`. These are real
|
||||
production 999s (e.g. sender_id `COMEDASSISTPROG`) whose source 837s
|
||||
were submitted to HPE clearinghouse before the current
|
||||
`~/.local/share/cyclone/cyclone.db` snapshot was created. The source
|
||||
837s themselves were never re-ingested into the current DB, so the
|
||||
`claims` table has no rows that could link against them. SP37's
|
||||
canonical submit-batch flow captures ST02 going forward, so the
|
||||
orphan count is stable — not a forward-looking bug, just historical
|
||||
drift.
|
||||
|
||||
**You cannot auto-link these orphans.** The source 837s are not in
|
||||
`ingest/`, `backend/var/sftp/staging/`, or any local path — they
|
||||
were transmitted to HPE and never came back. Cyclone is downstream
|
||||
of the clearinghouse and does not retain copies of outbound 837s
|
||||
after SFTP ACK. Do not attempt to re-ingest from SFTP inbound —
|
||||
those files are the 999 acks themselves, not the source 837s.
|
||||
|
||||
### Triage path
|
||||
|
||||
1. **Inspect the Inbox > AckOrphansLane** in the UI. Each row is a
|
||||
999 ack with no resolvable claim. Sort by `parsed_at DESC` to
|
||||
see the most recent first; older orphans are less likely to be
|
||||
actionable.
|
||||
2. **Decide per row.** If the operator can identify the source 837
|
||||
outside Cyclone (e.g. a manual record of what was submitted that
|
||||
day), manually create a `claims` row + a `claim_acks` link via
|
||||
`POST /api/parse-837` followed by `POST /api/inbox/candidates/{remit_id}/match`
|
||||
or `POST /api/acks/.../match-claim`. If not, leave the orphan
|
||||
alone — it stays as valid audit history.
|
||||
|
||||
### Optional: seed synthetic batch rows (one-shot)
|
||||
|
||||
`cyclone ack-orphans reconcile` inserts a synthetic `batches` row
|
||||
for every orphan ST02 that doesn't already have one. The synthetic
|
||||
row is marked with `input_filename = '<synthetic:orphan-reconcile>'`
|
||||
so it's trivially distinguishable in queries. Future 999 acks for
|
||||
the same ST02s will resolve against the batch envelope index (so
|
||||
the operator can see "this 999 is for a known orphan source") but
|
||||
will not link to claims (because no claim rows exist).
|
||||
|
||||
```bash
|
||||
# Preview the reconcile without writing rows.
|
||||
cyclone ack-orphans reconcile --dry-run
|
||||
|
||||
# Insert the synthetic batch rows.
|
||||
cyclone ack-orphans reconcile
|
||||
```
|
||||
|
||||
Re-running `cyclone ack-orphans reconcile` after a successful pass
|
||||
is a no-op (idempotent). To inspect the per-ST02 breakdown at any
|
||||
time:
|
||||
|
||||
```bash
|
||||
cyclone ack-orphans status
|
||||
```
|
||||
|
||||
The status command prints a table with ST02, ACK COUNT, and HAS
|
||||
BATCH columns, ranked by ack count so the heaviest backlog
|
||||
surfaces first.
|
||||
|
||||
### What this is NOT
|
||||
|
||||
- **Not a backfill.** No `claims` rows are synthesized — the source
|
||||
data is gone.
|
||||
- **Not auto-runnable.** The reconcile CLI is operator-invoked only;
|
||||
it does not run on boot, in the SFTP polling scheduler, or via
|
||||
cron.
|
||||
- **Not a deletion.** The orphan 999s are valid audit history and
|
||||
must remain queryable.
|
||||
|
||||
Reference in New Issue
Block a user