Three pr-reviewer followups from the 2026-07-07 review of commit ad14b56:
1. BUG: find_ack_orphans refactor routed 277ca/ta1 through
_ack_control_number which only knew 999 — restored per-kind source
(999 reads raw_json.envelope.control_number, 277ca/ta1 read the ORM
control_number column). Added regression test pinning all three
kinds.
2. DOCSTRING DRIFT: reconcile_orphan_st02s said 'remaining columns
take their defaults' but explicitly passes parsed_at and
transaction_set_control_number — added both to the explicit list
and clarified the rest take schema defaults.
3. WASTED SORT: _iter_orphan_999_st02s yielded sorted() but the
caller re-sorts by (-ack_count, st02) — yielding unsorted now.
Plus three cleanups the reviewer flagged:
* Hoisted 'json' / 'uuid' / 'datetime' imports to module top of
store/__init__.py (replaced in-method imports).
* Added two missing tests: sentinel grep-discoverability via
LIKE '<synthetic:%>' + 999-walk tolerance for non-dict raw_json
(None / list shapes — bytes is unreachable through the ORM).
* Aligned spec/plan exit codes to the cyclone-cli convention
(exit 1 on DB error, not 2 — matches the existing CLI
sys.exit(1) and the cyclone-cli skill documentation).
36/36 SP38 tests pass.
Implements task 6 of
docs/superpowers/plans/2026-07-07-cyclone-orphan-ack-housekeeping.md.
Adds 'Known historical drift — the 804 orphan 999s' section under
the existing operator-triage content. Covers:
- What the orphans are (real production 999s whose source 837s
predate the current DB snapshot; valid audit history that
cannot be auto-linked)
- Why they cannot be auto-linked (source 837s were transmitted
to HPE and never came back; Cyclone is downstream and does
not retain copies of outbound 837s)
- Triage path via the Inbox AckOrphansLane (already working
as of SP37-followup 893a662)
- Optional synthetic-batch seeding via 'cyclone ack-orphans
reconcile' (one-shot, idempotent, --dry-run available)
- 'cyclone ack-orphans status' for the per-ST02 breakdown
Explicitly calls out what the housekeeping is NOT:
- Not a backfill (no claims rows are synthesized)
- Not auto-runnable (operator-invoked only)
- Not a deletion (orphans are valid audit history)
References the SP38 spec and plan for the design rationale.
Adds docs/superpowers/specs/2026-07-07-cyclone-orphan-ack-housekeeping-design.md.
The current cyclone.db holds 805 ack rows, 804 of which are unresolved
orphans — 999s whose source 837 batches pre-date the current DB
snapshot and were never re-ingested. Investigation on 2026-07-07
confirmed the source 837s are not recoverable (Clearinghouse does not
echo them back; only the 999s were preserved in ingest/ + SFTP
staging). SP37's canonical submit-batch flow already captures ST02
going forward, so the orphan count stays flat — not a forward-looking
bug, just historical drift.
This SP captures that situation and adds housekeeping around it:
- RUNBOOK.md entry under 'Known historical drift' explaining the
root cause and the operator's triage path (Inbox AckOrphansLane,
which now works as of SP37-followup 893a662).
- 'cyclone ack-orphans status' CLI: distinct orphan ST02s + ack
count per ST02 + total.
- 'cyclone ack-orphans reconcile' CLI: one-shot, idempotent
synthetic-batch seeder. Marks synthetic rows with
input_filename '<synthetic:orphan-reconcile>' so they're
distinguishable in queries and the codebase can grep the
sentinel.
No UI change, no auto-runnable reconcile, no schema migration, no
attempt to backfill claim rows for the orphan ST02s (the source
data is gone). Per the canonical SP-N spec template, header is
'Draft, awaiting user sign-off' until the plan is signed off.
Branch: sp38-orphan-ack-housekeeping (off main, ahead by 893a662).
Next step: plan at docs/superpowers/plans/2026-07-07-cyclone-orphan-ack-housekeeping.md.
RUNBOOK gets a 'Submitting claims (canonical)' section with both CLI
and HTTP examples, the shared cyclone.submission.submit_file helper,
and a 'submit-batch vs resubmit-rejected-claims' decision rule.
CLAUDE.md gets a pointer to cyclone/submission/ in the 'Backend at a
glance' subpackage list.
8-task implementation plan covering migration, ORM update, join-key
wiring, the cyclone.submission helper, CLI, HTTP endpoint, and merge.
Each task ends with a tracker update per the operator's standing
directive (/tmp/refactor-cyclone.md).
Adds a parse → DB write → SFTP upload pipeline that closes the gap
where 837P submissions leave no DB row, making every 999 ack an
orphan. Locks the four brainstorming decisions (canonical submit flow,
DB-first ordering, new Batch.transaction_set_control_number column,
additive deprecation posture) and the architecture for one new CLI +
one new HTTP endpoint sharing a cyclone.submission helper.
docs/reference/837p.md line 48: the CO Medicaid trading-partner line
was still saying 'SKCO0 (sender) <-> COHCPF (receiver)'. The HCPF
837P Companion Guide (June 2025 - Version 2.5) requires
NM1*PR NM109 = CO_TXIX; SKCO0 causes Gainwell to reject the SET
('2010BB NM109 must equal CO_TXIX or CO_BHA').
docs/reference/co-medicaid.md line 118 + the Trading Partner table at
the top: same fix, with a pointer to the SP33 spec for the
root-cause story.
No code changes. Pure doc fix to match what the source now emits
(PayerConfig.co_medicaid().payer_id = 'CO_TXIX').
The original SP28 spec assumed `Claim.patient_control_number == 999.set_control_number`.
Empirically that's wrong: Gainwell's 999 echoes the source 837's ST02, not its
CLM01, and TOC's billing software fills them differently. Measured against
prod on 2026-07-02: the PCN join matches 0 / 1,398 acks.
Fix: two-pass join. Primary is `Batch.envelope.control_number (== 837 ST02)
→ claims via batch_id`; fallback is `Claim.patient_control_number` (for
senders that fill CLM01 == ST02). Coverage after fix: 727 / 1,398 (52%);
the remaining 671 are real orphans (ST02=0001 placeholder, no matching 837
batch in our DB).
Spec adds D10 + a critical-correction paragraph at the top of §1. Plan
gains the `lookup_claims_for_ack_set_response` pure helper, the
`batch_envelope_index()` store method, and three new tests covering the
two-pass join + the false-positive guard + the one-ack-to-many case.
Fresh plan that supersedes the 2026-06-21 line-number-anchored plan.
Reflects the resumed state: store.py at 2,995 LOC, 14 target modules,
1,176/1/10 test baseline, 3 new symbols to extract (dashboard_kpis +
_claim_state_str → kpis.py; check_matched_pair_drift → claim_detail.py),
3 private-helper re-exports (added _persist_835_remit + _remittance_835_row
discovered during the audit).
Per-task code copies dropped in favor of function-name references +
read-store.py guidance — the line numbers in the original plan are
inherently stale against a 2,995-LOC file under active development.
The original 2026-06-21 plan is preserved on disk as historical record
of the design conversation.
Update the 2026-06-21 approved spec with the post-SP27 reality:
store.py grew 2,412 → 2,995 LOC. Three new top-level symbols
(dashboard_kpis, _claim_state_str, check_matched_pair_drift) get
slotted into the new kpis.py and claim_detail.py. Three private
helpers (the original _claim_status_from_validation + the newly-
discovered _persist_835_remit and _remittance_835_row) are re-
exported from the facade to preserve the 4 test files that
import them. Branch bumped to sp21-store-split per SP-N convention.
All previously-locked Decisions are preserved verbatim.
The dzinesco SP9 seed had `paths.inbound` and `paths.outbound` mapped to
the wrong Gainwell MFT directories:
- paths.outbound was FromHPE/ (HPE sends files FROM here TO us — inbound)
- paths.inbound was ToHPE/ (we send files TO here — outbound)
So `/api/clearhouse/submit` was writing 837P claims to FromHPE (where
HPE puts acks/835s) and the SP16 scheduler was polling ToHPE (where our
claims go). 999 / TA1 / 835 files in the real FromHPE inbox were
unreachable.
Semantics per operator (2026-06-24):
- FromHPE = HPE/Gainwell → us = 999, TA1, 835 (inbound)
- ToHPE = us → HPE/Gainwell = 837P claims (outbound)
**Runtime code (the actual fix):**
- backend/src/cyclone/store.py — SP9 seed paths flipped
- backend/src/cyclone/edi/filenames.py — docstring corrected
**Test fixtures + assertions (would otherwise fail on the new seed):**
- backend/tests/test_clearhouse_api.py
- backend/tests/test_providers_seed.py
- backend/tests/test_sftp_stub.py — incl. inbound dir paths
- backend/tests/test_sftp_paramiko.py
- backend/tests/test_store_update_clearhouse.py
- backend/tests/test_api_clearhouse_patch.py
- backend/tests/test_scheduler.py
- backend/tests/test_api_scheduler.py
**Docs (text-only — keeps the codebase self-consistent):**
- README.md
- docs/reference/co-medicaid.md
- docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md
**Operator action required after merge:** the existing clearhouse row in
`~/.local/share/cyclone/cyclone.db` was seeded with the old wrong
paths. Easiest recovery:
1. `rm ~/.local/share/cyclone/cyclone.db` and let `ensure_clearhouse_seeded` re-run on next boot, OR
2. PATCH /api/clearhouse with the new `paths` block (the SP25
reconfigure hook picks it up live, including by the running scheduler).
**Verification:**
- 82 tests in the affected files pass (test_clearhouse_api, test_providers_seed,
test_sftp_stub, test_sftp_paramiko, test_store_update_clearhouse,
test_api_clearhouse_patch, test_scheduler, test_api_scheduler, test_filenames).
- Full backend suite: 1029 pass + 36 pre-existing order-dependent flakes
unrelated to this change (verified by running the same tests in isolation).
Bring the SP25 SFTP Polling Enablement (PATCH /api/clearhouse,
scheduler hot-reload, env-var-first secret lookup) and SP26
SFTP Password File Companion (Docker secret _FILE tier) onto
the v1.0.0 release branch so the production stack can actually
flip the Gainwell MFT poll from stub → real.
Conflict resolved in docker-compose.yml: kept both the SP26
CYCLONE_SFTP_PASSWORD_FILE env var and the new 'Always bind
to 0.0.0.0' comment.
Reachability is controlled by the host firewall / compose port
publishing, not the bind address. Backend, compose, and docs all
now default to 0.0.0.0 so the API is reachable from the frontend
container, the host, and the LAN without per-env overrides.
Files:
- backend/src/cyclone/__main__.py: default host = 0.0.0.0
- docker-compose.yml: refresh the comment to match the new posture
- CLAUDE.md / README.md / docs/ARCHITECTURE.md / docs/REQUIREMENTS.md:
reframe the bind note accordingly
Nora
cyclone
tyler