Commit Graph

112 Commits

Author SHA1 Message Date
Nora 4c466ca054 docs(plan): SP42 doc-pass implementation plan 2026-07-08 23:01:55 -06:00
Nora 90a2055c20 docs(spec): SP42 doc-pass design 2026-07-08 23:01:55 -06:00
Nora daf55d9a38 docs(spec): SP24 design + plan for reissue-claims CLI + IG-correctness regression test
Locks in:
- cyclone.reissue subpackage (parse_inputs / emit_outputs /
  zip_outputs / ig_correctness_check — pure functions, no Click).
- @main.command('reissue-claims') thin wrapper in cli.py.
- PATIENT_LOOP_DEFAULT_INCLUDED public constant in serialize_837.py
  (replaces inspect.signature on the private _build_subscriber_block).
- IG-correctness regression test in test_serialize_837.py pinning the
  constant to False.
- deprecation shim at scripts/reissue_claims.py.
- RUNBOOK entry under 'Operator workflows → Reissue claims'.

Workflow coverage: the dzinesco July-8 batches (4 files, 358 claims)
that triggered the 999 '2000C HL must be absent when 2000B SBR02=18'
rejection on 2026-07-08.
2026-07-08 22:23:53 -06:00
Nora f21201a22e docs(spec): SP40 Edifabric validation gate + 837P byte-defect sweep
Covers three things:
1. _build_per / _build_sbr call-site fixes so the serializer always
   emits PER-02/03/04 + SBR-09 (Edifabric rejects the current shape).
2. New cyclone.edifabric HTTP client wrapping the two-step
   /x12/read -> /x12/validate flow against api.edination.com.
3. Pre-upload gate inside resubmit-rejected-claims so an
   Edifabric-invalid file can't reach Gainwell.

Public eval API key 3ecf6b1c5cf34bd797a5f4c57951a1cf for dev.
Production is operator-supplied via cyclone secrets set
edifabric.api_key <paid-key>.
2026-07-07 18:02:58 -06:00
Nora 8516b90601 docs(spec): SP39 2010BB NM109 byte defect fix + resubmission tracking design 2026-07-07 17:18:00 -06:00
Nora 01e5ee781e docs: 2026-07-07 dashboard mess postmortem (SP25 follow-up) 2026-07-07 14:35:10 -06:00
Nora 0ccc396e5e feat(sp25): read-side defensive stub for synthetic-batch rows 2026-07-07 14:29:56 -06:00
Nora c8f5af3ba5 docs(plan): implementation plan for SP25 orphan data recovery (Step 1) 2026-07-07 14:18:22 -06:00
Nora fa034b1cd7 docs(spec): design for SP25 orphan data recovery (Step 1) 2026-07-07 14:18:22 -06:00
Nora 07ea7ca1d6 fix(sp38): restore per-kind control_number in find_ack_orphans + review cleanups
Three pr-reviewer followups from the 2026-07-07 review of commit ad14b56:

1. BUG: find_ack_orphans refactor routed 277ca/ta1 through
   _ack_control_number which only knew 999 — restored per-kind source
   (999 reads raw_json.envelope.control_number, 277ca/ta1 read the ORM
   control_number column). Added regression test pinning all three
   kinds.

2. DOCSTRING DRIFT: reconcile_orphan_st02s said 'remaining columns
   take their defaults' but explicitly passes parsed_at and
   transaction_set_control_number — added both to the explicit list
   and clarified the rest take schema defaults.

3. WASTED SORT: _iter_orphan_999_st02s yielded sorted() but the
   caller re-sorts by (-ack_count, st02) — yielding unsorted now.

Plus three cleanups the reviewer flagged:

* Hoisted 'json' / 'uuid' / 'datetime' imports to module top of
  store/__init__.py (replaced in-method imports).
* Added two missing tests: sentinel grep-discoverability via
  LIKE '<synthetic:%>' + 999-walk tolerance for non-dict raw_json
  (None / list shapes — bytes is unreachable through the ORM).
* Aligned spec/plan exit codes to the cyclone-cli convention
  (exit 1 on DB error, not 2 — matches the existing CLI
  sys.exit(1) and the cyclone-cli skill documentation).

36/36 SP38 tests pass.
2026-07-07 13:11:29 -06:00
Nora 8890627014 docs(plan): SP38 orphan-ack housekeeping plan
Implements docs/superpowers/specs/2026-07-07-cyclone-orphan-ack-housekeeping-design.md.

Seven tasks, all TDD-shaped (RED test first, then implementation):

  Task 1: Extract _iter_orphan_st02s() helper from find_ack_orphans
          (refactor; no behavior change; existing tests must stay green)
  Task 2: Add find_ack_orphan_st02_summary() to CycloneStore facade
          (RED: test_summary_returns_per_st02_counts)
  Task 3: Add reconcile_orphan_st02s() to CycloneStore
          (RED: test_reconcile_is_idempotent + sentinel + dry-run)
  Task 4: Add 'cyclone ack-orphans status' CLI subcommand
          (RED: test_status_prints_table via CliRunner)
  Task 5: Add 'cyclone ack-orphans reconcile' CLI subcommand
          (RED: test_reconcile_creates_synthetic_batches + idempotent)
  Task 6: Add RUNBOOK.md 'Known historical drift' section
  Task 7: Full pytest + live CLI verification + autoreview + commits

Per cyclone-spec, commits are feat(sp38): ... for implementation
work and the final merge commit is 'merge: SP38 orphan-ack
housekeeping into main'. No schema migration; no UI change; no
auto-runnable reconcile.
2026-07-07 12:38:07 -06:00
Nora d776a4a7ec docs(spec): design for SP38 orphan-ack housekeeping
Adds docs/superpowers/specs/2026-07-07-cyclone-orphan-ack-housekeeping-design.md.

The current cyclone.db holds 805 ack rows, 804 of which are unresolved
orphans — 999s whose source 837 batches pre-date the current DB
snapshot and were never re-ingested. Investigation on 2026-07-07
confirmed the source 837s are not recoverable (Clearinghouse does not
echo them back; only the 999s were preserved in ingest/ + SFTP
staging). SP37's canonical submit-batch flow already captures ST02
going forward, so the orphan count stays flat — not a forward-looking
bug, just historical drift.

This SP captures that situation and adds housekeeping around it:

  - RUNBOOK.md entry under 'Known historical drift' explaining the
    root cause and the operator's triage path (Inbox AckOrphansLane,
    which now works as of SP37-followup 893a662).
  - 'cyclone ack-orphans status' CLI: distinct orphan ST02s + ack
    count per ST02 + total.
  - 'cyclone ack-orphans reconcile' CLI: one-shot, idempotent
    synthetic-batch seeder. Marks synthetic rows with
    input_filename '<synthetic:orphan-reconcile>' so they're
    distinguishable in queries and the codebase can grep the
    sentinel.

No UI change, no auto-runnable reconcile, no schema migration, no
attempt to backfill claim rows for the orphan ST02s (the source
data is gone). Per the canonical SP-N spec template, header is
'Draft, awaiting user sign-off' until the plan is signed off.

Branch: sp38-orphan-ack-housekeeping (off main, ahead by 893a662).
Next step: plan at docs/superpowers/plans/2026-07-07-cyclone-orphan-ack-housekeeping.md.
2026-07-07 12:37:24 -06:00
Nora 698f8660bf docs(plan): SP37 canonical submit-batch flow
8-task implementation plan covering migration, ORM update, join-key
wiring, the cyclone.submission helper, CLI, HTTP endpoint, and merge.
Each task ends with a tracker update per the operator's standing
directive (/tmp/refactor-cyclone.md).
2026-07-07 10:00:28 -06:00
Nora 178898a8e1 docs(spec): SP37 canonical submit-batch flow
Adds a parse → DB write → SFTP upload pipeline that closes the gap
where 837P submissions leave no DB row, making every 999 ack an
orphan. Locks the four brainstorming decisions (canonical submit flow,
DB-first ordering, new Batch.transaction_set_control_number column,
additive deprecation posture) and the architecture for one new CLI +
one new HTTP endpoint sharing a cyclone.submission helper.
2026-07-07 09:56:12 -06:00
cyclone 4b56416414 docs(sp36): mark spec + plan as merged into main (f005494) 2026-07-07 08:50:07 -06:00
Nora a52a85c7a2 docs(spec+plan): SP36 api-routers-split — split 4,341-LOC api.py into per-resource routers under api_routers/, 19-task plan with per-step live-test + autoreview + commit cycle 2026-07-06 13:20:12 -06:00
Nora e4f3d25f3a docs(plan): SP35 parse-input-guards — server guards + UI auto-detect + 999/277CA/TA1 regression locks, TDD-first 2026-07-06 09:45:50 -06:00
Nora 750f560ee0 docs(spec): SP35 parse-input-guards — defense in depth against misroute silent-corruption 2026-07-06 09:45:50 -06:00
Nora dae7749464 docs(plan): SP33 Co TXIX payer fix implementation 2026-07-02 20:22:17 -06:00
Nora 2e7ad471e0 docs(spec): SP33 CO TXIX payer fix design 2026-07-02 20:20:01 -06:00
Nora 15c85300f6 docs(spec): SP32 align D7 with --file/--input-dir backfill design 2026-07-02 17:18:51 -06:00
Nora 055b4ae30e docs(spec): SP32 align test list with single-NPI-per-Remit design 2026-07-02 17:04:47 -06:00
Nora 4363b4fe41 docs(plan): SP32 rendering & service-provider NPI extraction 2026-07-02 16:52:32 -06:00
Nora d44948cb90 docs(spec): SP32 update D10/testing for single-NPI per Remit 2026-07-02 16:51:18 -06:00
Nora 9e2dde6ddb docs(spec): SP32 simplify D4/D6 — single-NPI per Remit, no comma-aggregation 2026-07-02 16:51:12 -06:00
Nora 2d40fbbcbb docs(spec): SP32 rendering & service-provider NPI extraction 2026-07-02 16:43:14 -06:00
Nora 6624a0bafd docs(plan): SP31 835 strict content-match auto-link 2026-07-02 14:56:43 -06:00
Nora a12104fb0f docs(spec): SP31 835 strict content-match auto-link 2026-07-02 14:53:48 -06:00
Nora bde3060e9e docs(spec,plan): SP30 Dashboard Recent batches widget 2026-07-02 14:08:30 -06:00
Nora 9cc13e7940 docs(spec,plan): SP29 Inbox 999-rejected drill 2026-07-02 13:06:21 -06:00
Nora 37caf6af77 docs(plan): SP28 follow-up notes — Batch.kind fix + one-time backfill 2026-07-02 12:31:37 -06:00
Nora 9d11ffcd8a docs(plan): SP28 deviations log — 4 backend + 5 frontend post-impl notes
Backend: helpers return dataclasses instead of inserting directly (preserves
publish-from-store contract); handlers commit work session before calling
cycl_store.add_claim_ack (SQLite single-writer); batch_envelope_index
accepts dict OR callable; link_manual returns ClaimAckLinkRow.

Frontend: TA1 row shape in Inbox lane; cross-page nav via useNavigate;
TA1 column header 'Batches' (not 'Claims'); ack-orphans lane position
between payer_rejected and candidates; fmt.usd instead of fmt.money
in MatchedClaimPanel.

Confirmed test baselines: backend 1210/10/6-err/1-fail = pre-SP28;
frontend 580 pass / 5 fail = pre-SP28. No new failures introduced.
46 files changed, +6,467/-22.
2026-07-02 12:05:45 -06:00
Nora ec87f98f02 docs(spec): SP28 D10 — two-pass join via Batch.envelope.control_number
The original SP28 spec assumed `Claim.patient_control_number == 999.set_control_number`.
Empirically that's wrong: Gainwell's 999 echoes the source 837's ST02, not its
CLM01, and TOC's billing software fills them differently. Measured against
prod on 2026-07-02: the PCN join matches 0 / 1,398 acks.

Fix: two-pass join. Primary is `Batch.envelope.control_number (== 837 ST02)
→ claims via batch_id`; fallback is `Claim.patient_control_number` (for
senders that fill CLM01 == ST02). Coverage after fix: 727 / 1,398 (52%);
the remaining 671 are real orphans (ST02=0001 placeholder, no matching 837
batch in our DB).

Spec adds D10 + a critical-correction paragraph at the top of §1. Plan
gains the `lookup_claims_for_ack_set_response` pure helper, the
`batch_envelope_index()` store method, and three new tests covering the
two-pass join + the false-positive guard + the one-ack-to-many case.
2026-07-02 11:16:53 -06:00
Nora 999a762c86 docs(plan): SP28 ack-claim auto-link implementation plan 2026-07-02 10:32:17 -06:00
Nora 53a8c77cb8 docs(spec): SP28 ack-claim auto-link design 2026-07-02 10:30:52 -06:00
cyclone d8c03fde3f docs(plan): plan for SP25 ack live-tail 2026-07-02 08:34:54 -06:00
cyclone 014e02ad42 docs(spec): design for SP25 ack live-tail 2026-07-02 08:30:39 -06:00
Nora 9644db8c51 docs(plan): resume SP21 store split — 14 modules, function-name refs (no line numbers)
Fresh plan that supersedes the 2026-06-21 line-number-anchored plan.
Reflects the resumed state: store.py at 2,995 LOC, 14 target modules,
1,176/1/10 test baseline, 3 new symbols to extract (dashboard_kpis +
_claim_state_str → kpis.py; check_matched_pair_drift → claim_detail.py),
3 private-helper re-exports (added _persist_835_remit + _remittance_835_row
discovered during the audit).

Per-task code copies dropped in favor of function-name references +
read-store.py guidance — the line numbers in the original plan are
inherently stale against a 2,995-LOC file under active development.

The original 2026-06-21 plan is preserved on disk as historical record
of the design conversation.
2026-06-29 14:47:54 -06:00
Nora fbe7b2358d docs(spec): resume SP21 store split — 14 modules, 3 SP27 symbols, 3 private-helper re-exports
Update the 2026-06-21 approved spec with the post-SP27 reality:
store.py grew 2,412 → 2,995 LOC. Three new top-level symbols
(dashboard_kpis, _claim_state_str, check_matched_pair_drift) get
slotted into the new kpis.py and claim_detail.py. Three private
helpers (the original _claim_status_from_validation + the newly-
discovered _persist_835_remit and _remittance_835_row) are re-
exported from the facade to preserve the 4 test files that
import them. Branch bumped to sp21-store-split per SP-N convention.

All previously-locked Decisions are preserved verbatim.
2026-06-29 14:44:21 -06:00
Nora 9d8f83d111 docs(plan): implementation plan for SP27 remittances architecture refactor
17 tasks: 1 preflight + 4 handler extracts (999/TA1/277CA/835) +
helpers dedup + INBOUND_RE loosen + SFTP timeouts + status surface +
atomic 835/reconcile + match invariants + chain endpoint +
claim.rejected_after_remit + frontend chain UI + final verify + merge.

Each task ends with live-test + autoreview + commit. Spec:
docs/superpowers/specs/2026-06-29-cyclone-remittances-architecture-refactor-design.md
2026-06-29 09:59:53 -06:00
Nora 454c3598b1 docs(spec): design for SP27 remittances architecture refactor
Tier 1: split scheduler.py into handlers/ subpackage, dedup helpers,
loosen INBOUND_RE, add SFTP operation timeouts, surface SFTP errors
in Scheduler.status().

Tier 2: unify 835 ingest + reconciliation into one critical section,
add GET /api/claims/{id}/chain, guard matched_remittance_id ↔
Remittance.claim_id invariant, emit claim.rejected_after_remit audit
when a 277CA rejection hits a matched claim.

Status: Draft, awaiting user sign-off.
2026-06-29 09:54:33 -06:00
Nora dd7da18279 fix(sftp): swap inbound/outbound paths — FromHPE is inbound, ToHPE is outbound
The dzinesco SP9 seed had `paths.inbound` and `paths.outbound` mapped to
the wrong Gainwell MFT directories:

  - paths.outbound  was  FromHPE/   (HPE sends files FROM here TO us — inbound)
  - paths.inbound   was  ToHPE/     (we send files TO here — outbound)

So `/api/clearhouse/submit` was writing 837P claims to FromHPE (where
HPE puts acks/835s) and the SP16 scheduler was polling ToHPE (where our
claims go). 999 / TA1 / 835 files in the real FromHPE inbox were
unreachable.

Semantics per operator (2026-06-24):
  - FromHPE = HPE/Gainwell → us  = 999, TA1, 835  (inbound)
  - ToHPE   = us → HPE/Gainwell = 837P claims    (outbound)

**Runtime code (the actual fix):**
  - backend/src/cyclone/store.py            — SP9 seed paths flipped
  - backend/src/cyclone/edi/filenames.py    — docstring corrected

**Test fixtures + assertions (would otherwise fail on the new seed):**
  - backend/tests/test_clearhouse_api.py
  - backend/tests/test_providers_seed.py
  - backend/tests/test_sftp_stub.py        — incl. inbound dir paths
  - backend/tests/test_sftp_paramiko.py
  - backend/tests/test_store_update_clearhouse.py
  - backend/tests/test_api_clearhouse_patch.py
  - backend/tests/test_scheduler.py
  - backend/tests/test_api_scheduler.py

**Docs (text-only — keeps the codebase self-consistent):**
  - README.md
  - docs/reference/co-medicaid.md
  - docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md

**Operator action required after merge:** the existing clearhouse row in
`~/.local/share/cyclone/cyclone.db` was seeded with the old wrong
paths. Easiest recovery:
  1. `rm ~/.local/share/cyclone/cyclone.db` and let `ensure_clearhouse_seeded` re-run on next boot, OR
  2. PATCH /api/clearhouse with the new `paths` block (the SP25
     reconfigure hook picks it up live, including by the running scheduler).

**Verification:**
  - 82 tests in the affected files pass (test_clearhouse_api, test_providers_seed,
    test_sftp_stub, test_sftp_paramiko, test_store_update_clearhouse,
    test_api_clearhouse_patch, test_scheduler, test_api_scheduler, test_filenames).
  - Full backend suite: 1029 pass + 36 pre-existing order-dependent flakes
    unrelated to this change (verified by running the same tests in isolation).
2026-06-24 22:05:55 -06:00
Nora 3075955826 docs(plan): SP26 SFTP Password File Companion — implementation plan 2026-06-24 16:03:51 -06:00
Nora 5b4c1513b6 docs(spec): SP26 SFTP Password File Companion — initial design 2026-06-24 15:52:19 -06:00
Nora 9fda46b891 docs(plan): SP25 SFTP Polling Enablement — implementation plan 2026-06-24 15:28:54 -06:00
Nora 7d2000ec31 docs(spec): SP25 SFTP Polling Enablement — initial design 2026-06-24 15:25:20 -06:00
Nora 5334646992 docs(plan): SP23 — record live verification results
Appends a 'Live verification' section to the plan with the six bugs
the live bring-up surfaced + their fixes, and the end-to-end smoke
results (login, /api/auth/me, /api/parse-837 with a real 837P,
full test suite 1026 passed).
2026-06-23 17:55:54 -06:00
Nora 3ba5ca0849 feat(sp23): live-verification fixes from end-to-end bring-up
After the 8-commit SP23 implementation landed, kicking the tires on
`docker compose build && docker compose up -d` (the gated DOCKER_TESTS=1
live test) surfaced five real bugs that don't show up in unit tests:

1. Backend wheel was built from the stub `__init__.py`, not the real
   source. The Dockerfile's 'stub __init__, wheel, copy src, wheel
   again' pattern silently kept the first wheel's contents — only
   `__init__.py` got re-stubbed. The installed package had an empty
   `__init__.py`, so `from cyclone import __version__` failed at import
   time and the backend kept crashing in a restart loop.
   Fix: single `COPY src/` + single `pip wheel`. Comment explains
   why the stub trick is gone for good.

2. Backend binds to 127.0.0.1 (intentional — local-only by design,
   see CLAUDE.md). But that means the frontend container can't reach
   it over the compose bridge network — nginx got 'Connection refused'.
   Fix: `CYCLONE_HOST` env var, defaults to 127.0.0.1 (preserves local
   posture for non-Docker runs), set to 0.0.0.0 by the docker-compose
   backend service. Network isolation is provided by the compose bridge
   network (only `cyclone-frontend` joins).

3. Healthcheck probed `/api/healthz` (404 — the route is `/api/health`).
   Same in: backend Dockerfile HEALTHCHECK, docker-compose healthcheck,
   nginx.conf doesn't have one (frontend proxies through), RUNBOOK.md,
   scripts/post-deploy.sh, scripts/smoke.sh.
   Fix: `/api/healthz` → `/api/health` everywhere SP23 owns.

4. The auth matrix in `cyclone.auth.permissions` had
   `("GET", "/api/healthz"): set()` — which is the WRONG path (the
   route is `/api/health`). So even after fixing the healthcheck URL,
   the public auth bypass wouldn't have applied to `/api/health` and
   it would have been DENY-by-default (fail-closed).
   Fix: matrix entry updated to `/api/health`.

5. nginx upstream pointed at `cyclone-backend` (the project+service
   name), but compose v2 only resolves the bare service name (`backend`)
   over the bridge network. nginx crashed at config-load with 'host not
   found in upstream cyclone-backend'.
   Fix: `cyclone-backend:8000` → `backend:8000` in nginx.conf + spec
   + plan.

6. Frontend HEALTHCHECK used `http://localhost:8080/`. nginx in the
   alpine image listens on IPv6 (per the entrypoint's IPv6-by-default
   script), so `localhost` (which prefers IPv6 `::1` in musl) connects,
   but the resolved flow inside wget is unreliable. `127.0.0.1` works.
   Fix: HEALTHCHECK uses `http://127.0.0.1:8080/`.

Also moves `frontend/Dockerfile` → `Dockerfile.frontend` and
`frontend/nginx.conf` → `nginx.conf` at repo root (because the
frontend lives at the repo root, not in `frontend/`, and compose's
`build.context: .` needs them at the same root as compose.yml).
The frontend's pre-existing `.dockerignore` was empty/unused, so it's
dropped — the root `.dockerignore` covers it.

Adds `docker-compose.override.yml` for local bring-up testing on a
host without sudo. Production uses `/etc/cyclone/secrets/` directly.

Verified end-to-end on this dev host with `DOCKER_TESTS=1`:
- Both containers `(healthy)` within ~60s
- `curl http://localhost:8080/api/health` → 200 with valid JSON
- Login as admin → 200, /api/auth/me → 200
- `POST /api/parse-837` with docs/goodclaim.x12 → 200, batch created
- Full backend test suite: 1014 passed, 9 skipped (prodfiles gitignored)
2026-06-23 17:53:16 -06:00
Cyclone 364e5d7497 docs(plan): SP23 Ubuntu Docker Deployment implementation plan
12 tasks from worktree bootstrap through single-atomic-merge into main. Backend Dockerfile (multi-stage python:3.11-slim-bookworm + sqlcipher + non-root + healthcheck), frontend Dockerfile (multi-stage node:20-alpine build + nginx:1.27-alpine runtime), docker-compose.yml at repo root with two services + named volumes + Docker secrets, scripts/{cyclone-init,post-deploy,smoke}.sh, RUNBOOK.md, tests/test_docker.py (compose config + Dockerfile parse + optional live bring-up), and a small auth bootstrap extension to read CYCLONE_ADMIN_*_FILE for Docker-secret compatibility. Closes the SP23 spec; see spec section 1.1 for what auth is reused from main vs. what SP23 actually adds.
2026-06-23 17:21:34 -06:00
Cyclone 35c561de20 docs(spec): mark SP23 approved and reconcile with auth on main
Auth landed in main on 2026-06-23 (migrations 0013/0014, cyclone.auth package, bcrypt + matrix_gate, Login.tsx + AuthProvider + RoleGate, CYCLONE_AUTH_DISABLED escape hatch). The original draft was written 2026-06-22 and proposed argon2id, admin/operator/viewer roles, a fresh migration 0012, and an inline require_role dependency. All of that is already in main. Add a 1.1 Delta vs. main section and switch role names from operator to user to match cyclone.auth.permissions.Role.
2026-06-23 17:20:04 -06:00