Update the 2026-06-21 approved spec with the post-SP27 reality:
store.py grew 2,412 → 2,995 LOC. Three new top-level symbols
(dashboard_kpis, _claim_state_str, check_matched_pair_drift) get
slotted into the new kpis.py and claim_detail.py. Three private
helpers (the original _claim_status_from_validation + the newly-
discovered _persist_835_remit and _remittance_835_row) are re-
exported from the facade to preserve the 4 test files that
import them. Branch bumped to sp21-store-split per SP-N convention.
All previously-locked Decisions are preserved verbatim.
The dzinesco SP9 seed had `paths.inbound` and `paths.outbound` mapped to
the wrong Gainwell MFT directories:
- paths.outbound was FromHPE/ (HPE sends files FROM here TO us — inbound)
- paths.inbound was ToHPE/ (we send files TO here — outbound)
So `/api/clearhouse/submit` was writing 837P claims to FromHPE (where
HPE puts acks/835s) and the SP16 scheduler was polling ToHPE (where our
claims go). 999 / TA1 / 835 files in the real FromHPE inbox were
unreachable.
Semantics per operator (2026-06-24):
- FromHPE = HPE/Gainwell → us = 999, TA1, 835 (inbound)
- ToHPE = us → HPE/Gainwell = 837P claims (outbound)
**Runtime code (the actual fix):**
- backend/src/cyclone/store.py — SP9 seed paths flipped
- backend/src/cyclone/edi/filenames.py — docstring corrected
**Test fixtures + assertions (would otherwise fail on the new seed):**
- backend/tests/test_clearhouse_api.py
- backend/tests/test_providers_seed.py
- backend/tests/test_sftp_stub.py — incl. inbound dir paths
- backend/tests/test_sftp_paramiko.py
- backend/tests/test_store_update_clearhouse.py
- backend/tests/test_api_clearhouse_patch.py
- backend/tests/test_scheduler.py
- backend/tests/test_api_scheduler.py
**Docs (text-only — keeps the codebase self-consistent):**
- README.md
- docs/reference/co-medicaid.md
- docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md
**Operator action required after merge:** the existing clearhouse row in
`~/.local/share/cyclone/cyclone.db` was seeded with the old wrong
paths. Easiest recovery:
1. `rm ~/.local/share/cyclone/cyclone.db` and let `ensure_clearhouse_seeded` re-run on next boot, OR
2. PATCH /api/clearhouse with the new `paths` block (the SP25
reconfigure hook picks it up live, including by the running scheduler).
**Verification:**
- 82 tests in the affected files pass (test_clearhouse_api, test_providers_seed,
test_sftp_stub, test_sftp_paramiko, test_store_update_clearhouse,
test_api_clearhouse_patch, test_scheduler, test_api_scheduler, test_filenames).
- Full backend suite: 1029 pass + 36 pre-existing order-dependent flakes
unrelated to this change (verified by running the same tests in isolation).
Appends a 'Live verification' section to the plan with the six bugs
the live bring-up surfaced + their fixes, and the end-to-end smoke
results (login, /api/auth/me, /api/parse-837 with a real 837P,
full test suite 1026 passed).
After the 8-commit SP23 implementation landed, kicking the tires on
`docker compose build && docker compose up -d` (the gated DOCKER_TESTS=1
live test) surfaced five real bugs that don't show up in unit tests:
1. Backend wheel was built from the stub `__init__.py`, not the real
source. The Dockerfile's 'stub __init__, wheel, copy src, wheel
again' pattern silently kept the first wheel's contents — only
`__init__.py` got re-stubbed. The installed package had an empty
`__init__.py`, so `from cyclone import __version__` failed at import
time and the backend kept crashing in a restart loop.
Fix: single `COPY src/` + single `pip wheel`. Comment explains
why the stub trick is gone for good.
2. Backend binds to 127.0.0.1 (intentional — local-only by design,
see CLAUDE.md). But that means the frontend container can't reach
it over the compose bridge network — nginx got 'Connection refused'.
Fix: `CYCLONE_HOST` env var, defaults to 127.0.0.1 (preserves local
posture for non-Docker runs), set to 0.0.0.0 by the docker-compose
backend service. Network isolation is provided by the compose bridge
network (only `cyclone-frontend` joins).
3. Healthcheck probed `/api/healthz` (404 — the route is `/api/health`).
Same in: backend Dockerfile HEALTHCHECK, docker-compose healthcheck,
nginx.conf doesn't have one (frontend proxies through), RUNBOOK.md,
scripts/post-deploy.sh, scripts/smoke.sh.
Fix: `/api/healthz` → `/api/health` everywhere SP23 owns.
4. The auth matrix in `cyclone.auth.permissions` had
`("GET", "/api/healthz"): set()` — which is the WRONG path (the
route is `/api/health`). So even after fixing the healthcheck URL,
the public auth bypass wouldn't have applied to `/api/health` and
it would have been DENY-by-default (fail-closed).
Fix: matrix entry updated to `/api/health`.
5. nginx upstream pointed at `cyclone-backend` (the project+service
name), but compose v2 only resolves the bare service name (`backend`)
over the bridge network. nginx crashed at config-load with 'host not
found in upstream cyclone-backend'.
Fix: `cyclone-backend:8000` → `backend:8000` in nginx.conf + spec
+ plan.
6. Frontend HEALTHCHECK used `http://localhost:8080/`. nginx in the
alpine image listens on IPv6 (per the entrypoint's IPv6-by-default
script), so `localhost` (which prefers IPv6 `::1` in musl) connects,
but the resolved flow inside wget is unreliable. `127.0.0.1` works.
Fix: HEALTHCHECK uses `http://127.0.0.1:8080/`.
Also moves `frontend/Dockerfile` → `Dockerfile.frontend` and
`frontend/nginx.conf` → `nginx.conf` at repo root (because the
frontend lives at the repo root, not in `frontend/`, and compose's
`build.context: .` needs them at the same root as compose.yml).
The frontend's pre-existing `.dockerignore` was empty/unused, so it's
dropped — the root `.dockerignore` covers it.
Adds `docker-compose.override.yml` for local bring-up testing on a
host without sudo. Production uses `/etc/cyclone/secrets/` directly.
Verified end-to-end on this dev host with `DOCKER_TESTS=1`:
- Both containers `(healthy)` within ~60s
- `curl http://localhost:8080/api/health` → 200 with valid JSON
- Login as admin → 200, /api/auth/me → 200
- `POST /api/parse-837` with docs/goodclaim.x12 → 200, batch created
- Full backend test suite: 1014 passed, 9 skipped (prodfiles gitignored)
12 tasks from worktree bootstrap through single-atomic-merge into main. Backend Dockerfile (multi-stage python:3.11-slim-bookworm + sqlcipher + non-root + healthcheck), frontend Dockerfile (multi-stage node:20-alpine build + nginx:1.27-alpine runtime), docker-compose.yml at repo root with two services + named volumes + Docker secrets, scripts/{cyclone-init,post-deploy,smoke}.sh, RUNBOOK.md, tests/test_docker.py (compose config + Dockerfile parse + optional live bring-up), and a small auth bootstrap extension to read CYCLONE_ADMIN_*_FILE for Docker-secret compatibility. Closes the SP23 spec; see spec section 1.1 for what auth is reused from main vs. what SP23 actually adds.
Auth landed in main on 2026-06-23 (migrations 0013/0014, cyclone.auth package, bcrypt + matrix_gate, Login.tsx + AuthProvider + RoleGate, CYCLONE_AUTH_DISABLED escape hatch). The original draft was written 2026-06-22 and proposed argon2id, admin/operator/viewer roles, a fresh migration 0012, and an inline require_role dependency. All of that is already in main. Add a 1.1 Delta vs. main section and switch role names from operator to user to match cyclone.auth.permissions.Role.
Replaces every stale 'no auth' / 'no authentication' / 'no second party to
authenticate' claim in the three top-level docs with the v1 posture: the
auth boundary is HTTP (bcrypt + HttpOnly session cookie; first admin
bootstrapped from CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD env
vars); the file-system posture (SQLCipher at rest, macOS Keychain) is
unchanged; the threat model is still a stolen/imaged drive.
Closes requirements §11 R-1 (was Open; now Closed by SP24 — auth shipped
via the origin/main merge on 2026-06-23, SP24 reconciled the docs).
Also:
- backend/src/cyclone/__main__.py — emit WARNING when AUTH_DISABLED is
True at boot so a misconfigured production deploy fails loudly
- §6.1 migrations table in ARCHITECTURE.md — list 0013 (auth_users_and_
sessions) and 0014 (audit_log_user_id) with the renumbering note
- §4.2 module map — add the cyclone.auth.* package
- Three skills addenda: cyclone-tests (AUTH_DISABLED conftest bypass is
mandatory context), cyclone-api-router (every router needs
Depends(matrix_gate)), cyclone-spec (spec template threat-model is now
auth-aware)
This also tracks CLAUDE.md in git for the first time (was previously
untracked; the SP24 doc updates are in scope for the increment).
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
Plan: docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md
Plan covers the docs-only reconciliation between the auth work that landed
in main on 2026-06-23 and the three top-level docs (CLAUDE.md, docs/
REQUIREMENTS.md, docs/ARCHITECTURE.md). Includes a 6-line __main__.py edit
for the AUTH_DISABLED startup WARNING and one-paragraph addenda to the
cyclone-tests / cyclone-api-router / cyclone-spec skills.
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
The auth-merge that landed on 2026-06-23 (14 commits, a25504b..39ae988)
adds real auth: User/Session models, bcrypt, matrix_gate, AUTH_DISABLED,
a login router, RBAC on every existing endpoint, and a RequireAuth
guard on the React shell. But CLAUDE.md, docs/REQUIREMENTS.md, and
docs/ARCHITECTURE.md (all added/updated on the css-reduction branch)
still describe the pre-auth "no auth, local-only, single operator,
single host" posture. The doc/code divergence is real and would
confuse anyone reading the docs.
This is the spec for SP24 — the alignment. It is intentionally a
docs/skills-only SP: no schema, no role, no gate changes. The plan
section enumerates the doc lines to update; the implementation step
sweeps the markdown for residual "no auth" claims.
Draft, awaiting user sign-off per the SP-N spec flow.
The implementer caught that claims.id is a single-column PK, which
prevents the same CLM01 from existing in multiple batches. That
makes the spec's pre-flight 409 workflow unreachable and resubmits
impossible. Migration 0014 relaxes the PKs to composite (batch_id,
id) on both claims and remittances, and updates all FKs that
referenced the old single-column PK. Task 1.3 in the plan is now
the migration; the previous Task 1.3 (helper tightening) is
renumbered to Task 1.4 and will run after 0014 lands.
The implementation shipped with 10 inline bug fixes that weren't reflected
in the original plan document. This commit updates the plan so it
matches the as-built code and adds a summary table near the top so a
future reader of the plan knows what was adjusted and why.
- Bug 1a: AckTimeoutError.__init__ signature widened from int to float
plus adds self.phase attribute for report remediation hints.
- Bug 1b: wait_for passes timeout_s through without int() round-trip.
- Bug 2: Markdown table assertion updated to match the actual format.
- Bug 3: Added ## Remediation section to write_report_md for soft/hard fails.
- Bug 4: 835 expected-by text now says 'typically the following Monday'.
- Bug 5: Removed bogus HealthSnapshot import in test_pipeline.py.
- Bug 6: All phase tests wrapped in 'async with CyclonePipeline(...) as p:'.
- Bug 7: Playwright + UploadPage imports moved to module level in pipeline.py.
- Bug 8: Added _phase_records_to_results() helper for PhaseRecord→PhaseResult.
- Bug 9: Resume test uses _make_stub factory so stubs append to completed_phases.
- Bug 10: CLI uses context_settings={'allow_interspersed_args': True}.
Update _get_typed in the plan to:
- raise immediately on 4xx (terminal)
- retry on 5xx and RequestError (transient)
- use BACKOFF_S = (1.0, 2.0) instead of (1.0, 2.0, 4.0)
- use RuntimeError instead of assert for the context-manager guard
Also add test_4xx_is_terminal_not_retried to the plan's test list.
The verbatim plan returned an un-entered CycloneClient from the fixture,
causing all 3 tests to fail with 'use async with CycloneClient(...)'.
Convert fixture to async generator that enters the context. Verified by
running pytest tests/test_api_client.py -v → 3 passed.
The verbatim plan's test asserted isinstance(log, structlog.stdlib.BoundLogger),
but cache_logger_on_first_use=True makes get_logger return a lazy proxy on
first call. Test now uses duck-typing (hasattr('bind') + callable). Also
dropped the misleading caplog assertion and unused logging/structlog imports.
- conftest.py example now uses SAMPLE_837P constant so sample_837p_bytes
reads the file (not the directory).
- Note that Python 3.13 satisfies requires-python >= 3.11 if 3.11 isn't
installed.
Adds pure local validators for the 10-digit NPI Luhn checksum (CMS-
published algorithm with the '80840' NPPES prefix) and 9-digit EIN
format (rejects reserved prefixes 00/07/80-89). No NPPES round-trip,
no IRS e-file lookup — catches the 99% typo case at parse time.
Surface:
- cyclone.npi.is_valid_npi / is_valid_tax_id / normalize_tax_id
- CLI: 'cyclone validate-npi <npi>' and 'cyclone validate-tax-id <ein>'
- API: GET /api/admin/validate-provider?npi=&tax_id=
- Parser validator: new R021_npi_checksum rule (warning, not error,
to keep test fixtures with placeholder NPIs ingestible)
- minimal_837p.txt fixture NPI updated from '1234567890' to the
Luhn-valid '1993999998' so strict-mode CLI parses still pass
Tests:
- test_npi.py — 27 cases (Luhn math, valid/invalid NPIs, EIN cases,
normalize_tax_id edge cases)
- test_api_validate_provider.py — 4 cases (both valid, both invalid,
omitted NPI, omitted tax_id)
- test_cli_validate.py — 8 cases (valid/invalid for both subcommands,
exit codes, malformed inputs)
- test_validator.py — 4 new R021 cases (valid Luhn silent, bad Luhn
warning, skipped when format bad, skipped when NPI missing)
Total: 923 tests pass.
13 tasks implementing the spec: pre-flight baseline, atomic-move of
store.py into store/__init__.py, 10 module extractions (exceptions,
records, orm_builders, ui, write, batches, claim_detail, acks, backups,
inbox, providers), and final verification + single atomic commit.
Working tree stays dirty through Tasks 1-11; final commit happens at
end of Task 12 per the spec's single-atomic-commit requirement.
Each task ends with a focused test-suite verification (not full baseline
— that's Task 12 Step 1's job) so regressions are caught early.
Step 4 of the file-split refactor series. Targets backend/src/cyclone/store.py
(2,412 lines) which sits on the hot path of parse-999, parse-277ca,
reconciliation, and inbox match/unmatch.
Decisions locked:
- Public surface: cyclone/store/ subpackage + thin facade (__init__.py
re-exports every currently-importable name).
- Class shape: module functions for bodies, CycloneStore keeps its current
method signatures as 1-line delegations.
- Helper distribution: dedicated utility modules (records, orm_builders,
ui, exceptions).
13 target modules total; largest is write.py at ~210 lines. Zero public API
changes, zero test changes. Single atomic commit migration; rollback is a
single git revert.
Three pure-ASGI middlewares close completeness-review gaps §3.1.4
(no body/rate limits) and §3.1.25 (no security headers):
- BodySizeLimitMiddleware — rejects oversized uploads (50 MB
default, CYCLONE_MAX_BODY_BYTES override). 413 on over-cap
Content-Length and on chunked reads that cross the cap.
- RateLimitMiddleware — sliding-window per-IP limiter (300/min
default, CYCLONE_RATE_LIMIT_PER_MIN override). 429 over the
window. /api/health is exempt.
- SecurityHeadersMiddleware — stamps X-Content-Type-Options,
X-Frame-Options, Referrer-Policy, Permissions-Policy, and a
strict Content-Security-Policy on every response.
Every 413/429 also writes a tamper-evident api.request_rejected
event into the SP11 audit chain so an operator can correlate
rejections with the SP18 JSON logs.
GET /api/health is rewritten to return a subsystem snapshot:
DB connectivity (SELECT 1), MFT scheduler state, backup scheduler
state, live pubsub subscriber counts, last batch id + timestamp.
Returns status='degraded' if any subsystem is unhappy; per-subsystem
errors surfaced in the respective dict.
Cyclone.pubsub.EventBus.stats() — new method for live subscriber
counts.
13 new tests (test_security.py) + 1 updated (test_api.py health
endpoint). All 883 backend tests pass.
Cyclone previously emitted stdlib default-formatted log lines that
operators couldn't parse with anything beyond grep. SP18 replaces
that with:
- JsonFormatter: newline-delimited JSON, ISO-8601 ms timestamps,
structured "extra" dict, exception tracebacks serialized as a
single string.
- CycloneDevFormatter: tabular format for tail -f in dev.
- PiiScrubber: logging.Filter that redacts NPIs, SSNs, DOBs,
patient names — both inline in the message ("npi 1881068062")
and via PHI-keyed extras ({"dob": "1980-04-12"}).
- setup_logging(): idempotent entry point used by the API lifespan
and CLI main; respects CYCLONE_LOG_LEVEL/FILE/JSON/NO_PII_SCRUB.
- CLI --log-format=json|dev + --log-file=… flags.
- Migrate 9 highest-value log sites in scheduler/backup_scheduler/
backup_service to use extra={...} (input_filename, claims, parser,
backup_id, db_fingerprint, etc.).
34 new tests (test_logging_formatter + test_logging_scrubber +
test_logging_setup). All 867 backend tests pass.
Nora
Cyclone
Tyler