docs(sp38): RUNBOOK entry for orphan-ack housekeeping
Implements task 6 of
docs/superpowers/plans/2026-07-07-cyclone-orphan-ack-housekeeping.md.
Adds 'Known historical drift — the 804 orphan 999s' section under
the existing operator-triage content. Covers:
- What the orphans are (real production 999s whose source 837s
predate the current DB snapshot; valid audit history that
cannot be auto-linked)
- Why they cannot be auto-linked (source 837s were transmitted
to HPE and never came back; Cyclone is downstream and does
not retain copies of outbound 837s)
- Triage path via the Inbox AckOrphansLane (already working
as of SP37-followup 893a662)
- Optional synthetic-batch seeding via 'cyclone ack-orphans
reconcile' (one-shot, idempotent, --dry-run available)
- 'cyclone ack-orphans status' for the per-ST02 breakdown
Explicitly calls out what the housekeeping is NOT:
- Not a backfill (no claims rows are synthesized)
- Not auto-runnable (operator-invoked only)
- Not a deletion (orphans are valid audit history)
References the SP38 spec and plan for the design rationale.
This commit is contained in:
@@ -291,3 +291,77 @@ curl -X PATCH http://127.0.0.1:8000/api/clearhouse \
|
|||||||
```
|
```
|
||||||
|
|
||||||
To revert, set `stub: true` and `auth: {"method": "keychain", "secret_ref": "sftp.gainwell.password"}`.
|
To revert, set `stub: true` and `auth: {"method": "keychain", "secret_ref": "sftp.gainwell.password"}`.
|
||||||
|
|
||||||
|
## Known historical drift — the 804 orphan 999s
|
||||||
|
|
||||||
|
The `acks` table may hold several hundred 999 acks whose source 837
|
||||||
|
batch is not present in the `batches` table — the Inbox "Ack orphans"
|
||||||
|
lane surfaces them at `GET /api/inbox/ack-orphans`. These are real
|
||||||
|
production 999s (e.g. sender_id `COMEDASSISTPROG`) whose source 837s
|
||||||
|
were submitted to HPE clearinghouse before the current
|
||||||
|
`~/.local/share/cyclone/cyclone.db` snapshot was created. The source
|
||||||
|
837s themselves were never re-ingested into the current DB, so the
|
||||||
|
`claims` table has no rows that could link against them. SP37's
|
||||||
|
canonical submit-batch flow captures ST02 going forward, so the
|
||||||
|
orphan count is stable — not a forward-looking bug, just historical
|
||||||
|
drift.
|
||||||
|
|
||||||
|
**You cannot auto-link these orphans.** The source 837s are not in
|
||||||
|
`ingest/`, `backend/var/sftp/staging/`, or any local path — they
|
||||||
|
were transmitted to HPE and never came back. Cyclone is downstream
|
||||||
|
of the clearinghouse and does not retain copies of outbound 837s
|
||||||
|
after SFTP ACK. Do not attempt to re-ingest from SFTP inbound —
|
||||||
|
those files are the 999 acks themselves, not the source 837s.
|
||||||
|
|
||||||
|
### Triage path
|
||||||
|
|
||||||
|
1. **Inspect the Inbox > AckOrphansLane** in the UI. Each row is a
|
||||||
|
999 ack with no resolvable claim. Sort by `parsed_at DESC` to
|
||||||
|
see the most recent first; older orphans are less likely to be
|
||||||
|
actionable.
|
||||||
|
2. **Decide per row.** If the operator can identify the source 837
|
||||||
|
outside Cyclone (e.g. a manual record of what was submitted that
|
||||||
|
day), manually create a `claims` row + a `claim_acks` link via
|
||||||
|
`POST /api/parse-837` followed by `POST /api/inbox/candidates/{remit_id}/match`
|
||||||
|
or `POST /api/acks/.../match-claim`. If not, leave the orphan
|
||||||
|
alone — it stays as valid audit history.
|
||||||
|
|
||||||
|
### Optional: seed synthetic batch rows (one-shot)
|
||||||
|
|
||||||
|
`cyclone ack-orphans reconcile` inserts a synthetic `batches` row
|
||||||
|
for every orphan ST02 that doesn't already have one. The synthetic
|
||||||
|
row is marked with `input_filename = '<synthetic:orphan-reconcile>'`
|
||||||
|
so it's trivially distinguishable in queries. Future 999 acks for
|
||||||
|
the same ST02s will resolve against the batch envelope index (so
|
||||||
|
the operator can see "this 999 is for a known orphan source") but
|
||||||
|
will not link to claims (because no claim rows exist).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Preview the reconcile without writing rows.
|
||||||
|
cyclone ack-orphans reconcile --dry-run
|
||||||
|
|
||||||
|
# Insert the synthetic batch rows.
|
||||||
|
cyclone ack-orphans reconcile
|
||||||
|
```
|
||||||
|
|
||||||
|
Re-running `cyclone ack-orphans reconcile` after a successful pass
|
||||||
|
is a no-op (idempotent). To inspect the per-ST02 breakdown at any
|
||||||
|
time:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cyclone ack-orphans status
|
||||||
|
```
|
||||||
|
|
||||||
|
The status command prints a table with ST02, ACK COUNT, and HAS
|
||||||
|
BATCH columns, ranked by ack count so the heaviest backlog
|
||||||
|
surfaces first.
|
||||||
|
|
||||||
|
### What this is NOT
|
||||||
|
|
||||||
|
- **Not a backfill.** No `claims` rows are synthesized — the source
|
||||||
|
data is gone.
|
||||||
|
- **Not auto-runnable.** The reconcile CLI is operator-invoked only;
|
||||||
|
it does not run on boot, in the SFTP polling scheduler, or via
|
||||||
|
cron.
|
||||||
|
- **Not a deletion.** The orphan 999s are valid audit history and
|
||||||
|
must remain queryable.
|
||||||
|
|||||||
Reference in New Issue
Block a user