Add a second tab to the Upload page that surfaces the persisted batch
archive and lets the user re-download any 837P batch as a ZIP without
re-parsing the original file.
- Backend: /api/batches now carries per-row claimIds (837P only).
835 batches return an empty list, which the UI uses as the signal
to hide the Re-export button on those rows. Avoids an extra
round-trip to /api/batches/{id} per row.
- Frontend: BatchSummary.claimIds added to the list-endpoint type.
- Upload page: page body wrapped in Tabs.Root with a History trigger
that mirrors ?tab= in the URL for deep-link round-trip. The
History tab renders UploadHistory → HistoryTable → HistoryRow with
a one-click Re-export ZIP button per 837P row. The button calls
POST /api/batches/{id}/export-837 with the row's claim ids and
downloads the ZIP via downloadBlob. Falls back to the in-memory
parsedBatches store when the backend returns no rows so the tab
stays useful in sample-data mode.
- Backend tests: claimIds present on 837P rows, empty on 835 rows.
- Frontend tests: 13 tests covering tab switching, URL deep-link,
loading/error/empty states, the 837P-vs-835 button visibility
split, the Re-export happy path, and the failure toast.
The dev-server allow-list (VITE_DEV_ORIGINS = localhost:5173 +
127.0.0.1:5173 + CYCLONE_ALLOWED_ORIGINS) is tight enough that
`allow_credentials=True` is safe — the browser was dropping the
session cookie on cross-origin fetches from the Vite dev server
otherwise. Tight allow-list + credentials is the standard setup.
Two test-only fixes after running test_compose_up_brings_up_healthy_stack
on a non-CI host for the first time:
1. The test only used -f docker-compose.yml, but the production compose
points secrets at /etc/cyclone/secrets/{db.key,admin_username,
admin_pw} — which requires sudo to create. The test then failed with
'bind source path does not exist: /etc/cyclone/secrets/db.key' even
though the stack itself was correctly configured.
Fix: if docker-compose.override.yml exists at the repo root, the
test uses `-f compose.yml -f override.yml` so secrets come from
/tmp/cyclone-test-secrets/. Production CI skips the override.
2. The stack publishes host port 8080. If another local service (e.g.
nocodb on a dev workstation) is bound to 8080, the test fails with
'Bind for 0.0.0.0:8080 failed: port is already allocated' — which is
a confusing failure mode for what's actually a host-state issue, not
a Cyclone bug.
Fix: probe 127.0.0.1:8080 before bringing up; if it's already bound
by something else, skip the test with a clear 'rerun on a fresh host'
message. CI workers don't have this conflict.
Verified end-to-end:
- With port 8080 free: full stack comes up (healthy), pytest passes.
- With port 8080 bound: pytest skips cleanly with the message above.
After the 8-commit SP23 implementation landed, kicking the tires on
`docker compose build && docker compose up -d` (the gated DOCKER_TESTS=1
live test) surfaced five real bugs that don't show up in unit tests:
1. Backend wheel was built from the stub `__init__.py`, not the real
source. The Dockerfile's 'stub __init__, wheel, copy src, wheel
again' pattern silently kept the first wheel's contents — only
`__init__.py` got re-stubbed. The installed package had an empty
`__init__.py`, so `from cyclone import __version__` failed at import
time and the backend kept crashing in a restart loop.
Fix: single `COPY src/` + single `pip wheel`. Comment explains
why the stub trick is gone for good.
2. Backend binds to 127.0.0.1 (intentional — local-only by design,
see CLAUDE.md). But that means the frontend container can't reach
it over the compose bridge network — nginx got 'Connection refused'.
Fix: `CYCLONE_HOST` env var, defaults to 127.0.0.1 (preserves local
posture for non-Docker runs), set to 0.0.0.0 by the docker-compose
backend service. Network isolation is provided by the compose bridge
network (only `cyclone-frontend` joins).
3. Healthcheck probed `/api/healthz` (404 — the route is `/api/health`).
Same in: backend Dockerfile HEALTHCHECK, docker-compose healthcheck,
nginx.conf doesn't have one (frontend proxies through), RUNBOOK.md,
scripts/post-deploy.sh, scripts/smoke.sh.
Fix: `/api/healthz` → `/api/health` everywhere SP23 owns.
4. The auth matrix in `cyclone.auth.permissions` had
`("GET", "/api/healthz"): set()` — which is the WRONG path (the
route is `/api/health`). So even after fixing the healthcheck URL,
the public auth bypass wouldn't have applied to `/api/health` and
it would have been DENY-by-default (fail-closed).
Fix: matrix entry updated to `/api/health`.
5. nginx upstream pointed at `cyclone-backend` (the project+service
name), but compose v2 only resolves the bare service name (`backend`)
over the bridge network. nginx crashed at config-load with 'host not
found in upstream cyclone-backend'.
Fix: `cyclone-backend:8000` → `backend:8000` in nginx.conf + spec
+ plan.
6. Frontend HEALTHCHECK used `http://localhost:8080/`. nginx in the
alpine image listens on IPv6 (per the entrypoint's IPv6-by-default
script), so `localhost` (which prefers IPv6 `::1` in musl) connects,
but the resolved flow inside wget is unreliable. `127.0.0.1` works.
Fix: HEALTHCHECK uses `http://127.0.0.1:8080/`.
Also moves `frontend/Dockerfile` → `Dockerfile.frontend` and
`frontend/nginx.conf` → `nginx.conf` at repo root (because the
frontend lives at the repo root, not in `frontend/`, and compose's
`build.context: .` needs them at the same root as compose.yml).
The frontend's pre-existing `.dockerignore` was empty/unused, so it's
dropped — the root `.dockerignore` covers it.
Adds `docker-compose.override.yml` for local bring-up testing on a
host without sudo. Production uses `/etc/cyclone/secrets/` directly.
Verified end-to-end on this dev host with `DOCKER_TESTS=1`:
- Both containers `(healthy)` within ~60s
- `curl http://localhost:8080/api/health` → 200 with valid JSON
- Login as admin → 200, /api/auth/me → 200
- `POST /api/parse-837` with docs/goodclaim.x12 → 200, batch created
- Full backend test suite: 1014 passed, 9 skipped (prodfiles gitignored)
Eight tests: compose file exists, docker compose config validates, services declare required volumes/restart/healthcheck/ports/depends_on, secrets and volumes tables include the names the operator expects, backend wires CYCLONE_BACKUP_AUTOSTART=1, both Dockerfiles pass . The live bring-up test (compose up + wait for healthy) is gated on DOCKER_TESTS=1 so it skips on bare CI. PyYAML was already a hard dep.
Add _read_secret() helper that prefers *_FILE env vars (the standard Docker-secret pattern) over bare env vars. Strips trailing whitespace from file contents so printf/echo newlines don't break bcrypt verify. Existing CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD still work for non-Docker deployments. Add tests/test_auth_bootstrap_file.py covering: file-path read, file-overrides-bare, bare-fallback, whitespace-stripping, missing-file raises.
Align CLAUDE.md + REQUIREMENTS.md + ARCHITECTURE.md with the auth work
that landed on main on 2026-06-23; emit an AUTH_DISABLED WARNING at boot
when AUTH_DISABLED is true so misconfigured production deploys fail loudly.
Closes requirements R-1 (was Open; auth shipped via the origin/main merge,
SP24 reconciles the docs).
* CLAUDE.md — track in git for the first time
* docs/REQUIREMENTS.md + docs/ARCHITECTURE.md — replace every stale
'no auth' / 'no authentication' / 'no second party to authenticate'
claim with the v1 posture (bcrypt + HttpOnly session cookie; first
admin bootstrapped from CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD)
* backend/src/cyclone/__main__.py — boot-time WARNING when AUTH_DISABLED
* ARCHITECTURE §4.2 module map — add cyclone.auth.* package
* ARCHITECTURE §6.1 migrations table — list 0013 + 0014 with renumbering note
* .superpowers/skills/{cyclone-tests,cyclone-api-router,cyclone-spec} —
add AUTH_DISABLED / matrix_gate / auth-aware threat-model guidance
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
Plan: docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md
- test_migration_latest_idempotent_on_fresh_db now uses the pytest
tmp_path fixture instead of a literal /tmp path that did not exist,
so sqlite can create the test DB.
- test_drop_claims_unique_constraint_migration now inserts the NOT
NULL batches.kind / input_filename / parsed_at columns when seeding
the parent row for the two same-PCN claims.
Adds migration 0015_drop_claims_unique_constraint.sql that recreates the
claims table without the inline UNIQUE(batch_id, patient_control_number)
constraint, plus tests proving the migration runs cleanly and the
constraint is not re-introduced.
Replaces every stale 'no auth' / 'no authentication' / 'no second party to
authenticate' claim in the three top-level docs with the v1 posture: the
auth boundary is HTTP (bcrypt + HttpOnly session cookie; first admin
bootstrapped from CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD env
vars); the file-system posture (SQLCipher at rest, macOS Keychain) is
unchanged; the threat model is still a stolen/imaged drive.
Closes requirements §11 R-1 (was Open; now Closed by SP24 — auth shipped
via the origin/main merge on 2026-06-23, SP24 reconciled the docs).
Also:
- backend/src/cyclone/__main__.py — emit WARNING when AUTH_DISABLED is
True at boot so a misconfigured production deploy fails loudly
- §6.1 migrations table in ARCHITECTURE.md — list 0013 (auth_users_and_
sessions) and 0014 (audit_log_user_id) with the renumbering note
- §4.2 module map — add the cyclone.auth.* package
- Three skills addenda: cyclone-tests (AUTH_DISABLED conftest bypass is
mandatory context), cyclone-api-router (every router needs
Depends(matrix_gate)), cyclone-spec (spec template threat-model is now
auth-aware)
This also tracks CLAUDE.md in git for the first time (was previously
untracked; the SP24 doc updates are in scope for the increment).
Spec: docs/superpowers/specs/2026-06-23-cyclone-auth-posture-alignment-design.md
Plan: docs/superpowers/plans/2026-06-23-cyclone-auth-posture-alignment.md
Two bugs surfaced as 403/500 storms in the live UI:
1. PERMISSIONS matrix only had '/api/reconcile' as a prefix but the
client hits '/api/reconciliation/*' (dashboard reconciliation tab,
manual match endpoint). Every call returned 403 even for admin.
Add ('GET', '/api/reconciliation') and ('POST', '/api/reconciliation')
alongside the existing /api/reconcile entries.
2. The seed CLI built billing_provider.address as a flat string but
to_ui_claim_detail._address_to_ui expects {line1,line2,city,state,zip},
so opening any seeded claim in the detail drawer threw
'str object has no attribute get' → 500. Build the dict from the
provider fields instead.
Before this change, every claim — matched or not — came back from
`/api/claims` with `receivedAmount: 0.0`. `to_ui_claim_from_orm`
hardcoded the value, so the Dashboard's 'Received' KPI and the
claim detail drawer's paid amount always read $0 regardless of
whether the claim had been paired with a paying remittance.
The fix threads the real value through the read path:
- `to_ui_claim_from_orm` accepts a new `received_total: float`
kwarg (default 0.0 for callers that don't have a remittance in
scope).
- `iter_claims` bulk-loads `Remittance.total_paid` for every
matched claim id in the result set (single SQL roundtrip via
`IN` filter — no N+1) and stamps the sum onto each claim dict.
- `manual_match` passes `float(remit.total_paid)` from the remit
it's already holding.
- `manual_unmatch` reads `paired_remit.total_paid` before clearing
the FK so the response still reports what was paid pre-unpair.
- `add` (write path) and `list_unmatched` (filters matched-remit
is NULL) pass `0.0` explicitly for readability.
The dev seed CLI now also generates matched Remittance rows for
every PAID/PARTIAL claim — `matched_remittance_id` is set on the
claim, and the remittance's `total_paid` is derived from the
billed amount using the same ratios the frontend's old sample
fixtures used (60–100% for PAID, 20–50% for PARTIAL). That gives
the Dashboard a non-zero 'Received' KPI on a fresh dev DB.
New tests:
- tests/test_iter_claims_received.py — 4 cases covering matched,
unmatched, orphan FK, and bulk-load paths. Catches regressions
if anyone re-introduces the hardcoded 0.0 or breaks the bulk
query.
Now that Dashboard reads live API hooks instead of the in-memory
sample store, a fresh dev DB renders as $0 KPIs and 'No activity
yet.' for a long time. `python -m cyclone seed` inserts a
deterministic batch of 96 claims + 28 activity events so the
Dashboard / Claims / Activity Log pages have something to render
without going through the Inbox / EDI parser pipeline.
python -m cyclone seed # insert (no-op if seeded)
python -m cyclone seed --reset # wipe + re-insert
python -m cyclone seed --count 10
python -m cyclone seed --status # counts only
The seed mirrors the frontend `sampleData.ts` fixtures (3 TX
providers, 7 payers, 8 CPTs, 96 claims spread across the last 200
days) so dev dashboards look identical to the previous in-memory
fixture mode. `raw_json` and `payload_json` are populated in the
shape that `store.to_ui_claim_from_orm` / `recent_activity` parse
back into the UI wire format — wire parity, not shortcuts.
`--reset` cleans by id prefix (`SEED-` / `CLM-S`) rather than
relying on SQLite's `ON DELETE CASCADE`, which the dev session
doesn't enforce (no `PRAGMA foreign_keys = ON` at the SQLAlchemy
session layer). Safer than silent orphans.
Verified end-to-end: 986 backend tests pass, dashboard renders
$75,833 / 96 claims / 11.5% denial rate / 10 activity events /
top-3 providers ranked by claim count.
Two bootability gaps caught by running the live stack:
1. cyclone.auth.bootstrap.run() called SessionLocal() before db.init_db()
had run on a fresh DB, so 'python -m cyclone serve' / 'users create'
/ '--help' all crashed with 'db.init_db() has not been called'.
Add init_db() inside bootstrap (idempotent — no-op once the schema
is current) so first-boot works without manual prep.
2. PERMISSIONS matrix didn't register the inbox endpoints that were
added in SP10 (export.csv) and SP14 (candidates/dismiss,
rejected/resubmit, payer-rejected/acknowledge). The matrix's
longest-prefix match meant /api/inbox/candidates/dismiss matched
the '/api/inbox' prefix but only because that single entry
existed; adding the dedicated write-prefix entries makes the
intent explicit and keeps viewer role out of every inbox write
even if the broader /api/inbox prefix is later broadened.
Verified live:
- admin can login, GET /api/inbox/lanes (200), GET
/api/inbox/export.csv (200), POST /api/inbox/candidates/dismiss
(200).
- viewer can login, GET /api/inbox/lanes (200), GET
/api/inbox/export.csv (200), but POST /api/inbox/candidates/dismiss
and POST /api/inbox/rejected/resubmit both return 403.
- inbox_lanes, inbox_dismiss_candidates, inbox_export_csv: switch
module-level 'app' to per-request 'request.app' so per-endpoint
state stays consistent with the test's TestClient target across
importlib.reload() (test_api.py::test_cors_extra_origins_via_env
reloads the api module to mutate CORS allow-lists; pre-reload
endpoints then mutate the wrong app instance).
- _http_exc_handler: when HTTPException.detail is a dict, wrap under
'detail' so the standard envelope stays stable and callers can
branch on body['detail']['error'].
- conftest._auto_init_db: re-resolve cyclone.api.app each fixture
invocation (instead of caching at module load) so the reload pattern
doesn't leave event_bus set on a stale app.
- test_acks.test_migration_latest_idempotent_on_fresh_db: bump
user_version assertion from 12 to 14 after auth migration renumber
(0013 users+sessions, 0014 audit_log.user_id).
Also installs sqlcipher3 + paramiko into the backend venv so the
capability tests can run; both modules were optional deps that
test_db_crypto and test_sftp_paramiko assume are present.
Backend test results: 1008 passed, 9 skipped (gitignored prodfile
fixtures), 0 failed.
The auth cherry-picks brought in 0010_auth_users_and_sessions.sql
(version: 10) and 0011_audit_log_user_id.sql (version: 11), which
collide with main's 0010_payer_rejected_acknowledged.sql (version: 10)
and 0011_processed_inbound_files.sql (version: 11). The migration
runner is filename-sorted but uses the -- version: N header to gate
'application vs skip', so the second file at each version gets
silently skipped — claims.payer_rejected_acknowledged_at never
got ALTERed, breaking every test that touches the claims table.
Renumber to 0013 and 0014 (and bump the -- version headers to
match) so they apply after 0012_backups.sql and main's existing
SP14 / SP15 migrations.
- New test_existing_endpoints_require_auth.py: spot-check that existing
/api/* endpoints now require auth (gated via Depends(matrix_gate)) when
AUTH_DISABLED is False. Health remains public.
- conftest.py: flip AUTH_DISABLED=True for the suite so the legacy
pre-auth tests keep passing without login. Auth tests flip it back
off via their own autouse fixture (now patched to use monkeypatch
for cleanup).
Verified: 53 auth tests pass; 222 pre-existing non-auth failures are
unchanged.
Replaces the skipped 63ae0d2 commit: adds matrix_gate to deps.py and
applies it as a dependency on every authenticated FastAPI route, including
extracted routers (acks, ta1_acks, admin) and inline endpoints in api.py.
The PERMISSIONS matrix in auth/permissions.py controls which roles can
hit which (method, path) combinations; matrix_gate is fail-closed by
default (any endpoint not in the matrix returns 403).
Follow-up to 7c1be58. pytest-randomly was added to a PEP 735
[dependency-groups] dev block by 'uv add --dev', leaving the
project with two 'dev' groups (the legacy [project.optional-
dependencies] dev and the new [dependency-groups] dev). To get
the full dev env someone had to run 'uv sync --extra dev --group
dev', which is easy to miss.
This commit moves pytest-randomly>=4.1 into the
[project.optional-dependencies] dev list (where pytest,
pytest-cov, pytest-asyncio, and httpx already live) and drops
the [dependency-groups] block. 'uv sync --extra dev' now
installs the full dev toolchain. uv.lock updated accordingly:
the [package.dev-dependencies] and [package.metadata.requires-
dev] sections are gone, and pytest-randomly is part of the
standard dev extra.
Two follow-ups to the v0.2.0 release:
1. Add cryptography>=49.0,<50 to the [project.dependencies] list.
cyclone.backup / cyclone.backup_service import it at module top
level, so it has to be a hard dep — not an extra — or the test
suite fails to collect on a fresh 'uv sync'. A clean install
(without this commit) gets 28 collection errors with no tracked
file changes. This was latent because the original 99M venv
had cryptography installed out-of-band and masked the missing
dep declaration.
2. Add pytest-randomly>=4.1.0 to the dev dep group.
Used to characterize the test suite's order-dependence: under
alphabetic order the suite shows 7 failures; under seed=12345
it shows 34. That variance is evidence of test-order state
interactions, not venv issues. Keeping pytest-randomly in
the dev deps so this can be re-verified on demand.
Note: pytest-randomly was added with 'uv add --dev', which uses
the PEP 735 [dependency-groups] syntax. pyproject.toml now has
two 'dev' groups — the legacy [project.optional-dependencies] dev
= [pytest, pytest-cov, pytest-asyncio, httpx] and the new
[dependency-groups] dev = [pytest-randomly]. To get the full dev
env, run: uv sync --extra dev --group dev. Worth consolidating
in a follow-up.
Backend:
- New POST /api/batches/{id}/export-837: regenerate X12 837 files
for a list of claim_ids into a ZIP using HCPF file naming standards,
with a unique interchange/group control number per export. Wire
the clearhouse Loop 1000A (NM1*41 + PER) and per-payer receiver
(NM1*40) blocks so the serializer no longer falls back to
CYCLONE / RECEIVER placeholders.
- /api/parse-837 and /api/parse-835 now surface the server-side
batch_id in both JSON and NDJSON response shapes so the frontend
can hit batch-scoped endpoints without an extra listBatches
round-trip.
- Filename helpers and the 837 serializer updated to match the new
HCPF envelope; tests cover batch export, parse batch_id, and the
serializer's control-number uniqueness guarantee.
Frontend:
- New shared components: ClaimCard, ClaimCard837, DominantKpiCard,
EditorialNote, ExportBar, TickerTape, and a charts/ set
(BarChart, HBarChart, SegmentedBar, AgingBars).
- New useBatchExport hook driving ExportBar's download flow against
the new endpoint.
- ClaimDrawer, Lane, and Layout migrated from raw CSS-variable
colors to Tailwind theme tokens (bg-card, text-foreground,
border/60, etc.) for consistency with the rest of the instrument
chrome; the active tab indicator gains a subtle accent glow.
- Upload, Inbox, Batches, BatchDiff, Reconciliation, and Acks pages
reworked to compose the new shared components and consume the new
batch-scoped API surface (notably ExportBar wired into Batches).
Tooling / Docs:
- Add audit-uiux.mjs and a docs/goodclaim.x12 sample fixture.
- Update ClaimDrawer testids and add coverage for the new
components and the useBatchExport hook.
Rolls up into the v0.2.0 release tag.
SP21 Task 2.5 — the Dashboard's 'Recent activity' card now routes clicks
to the matching entity drawer / page by event kind:
- claim_* → /claims?claim=<id> (drawer in Phase 5)
- provider_added → /providers?provider=<npi> (ProviderDrawer)
- remit_received → toast 'coming in a later phase' (RemitDrawer in Phase 4)
- anything else → toast (manual_match, unknown kinds)
Implementation:
- New src/lib/event-routing.ts with the eventKindToUrl() helper,
plus a unit test covering all 6 + default branches.
- src/components/ActivityFeed.tsx gains an optional onItemClick
prop; when set, each row gets role='button', tabIndex=0, the
drillable hover affordance (chevron + tint), and an Enter/Space
keybinding. e.stopPropagation() is called before the handler so a
parent row click can't double-fire (same fix as Task 2.4).
- src/pages/Dashboard.tsx wires the handler on the 'Recent activity'
card via eventKindToUrl + sonner toast for unhandled kinds.
- Backend: CycloneStore.recent_activity() now exposes claimId and
remittanceId on each row (read from ActivityEvent.claim_id /
remittance_id) so the routing helper has the entity ids it needs.
- The frontend Activity interface gains optional claimId /
remittanceId fields; the in-memory sample data and the
addClaim store action populate them so the dashboard works in
both API-configured and sample-data modes.
Three independent improvements that fix real browser-facing bugs:
1. CORS: allow 127.0.0.1:5173 in addition to localhost:5173. Both
resolve to the same Vite dev server but CORS treats them as distinct
origins, so tabs opened via the IP form silently break.
2. CORS: support CYCLONE_ALLOWED_ORIGINS env var (comma-separated) for
LAN / staging hosts. The middleware reads it at module import.
3. Catch-all exception handler: returns JSON 500 with CORS headers
instead of a bare Uvicorn text/plain response. Without this, any
unhandled exception is misreported by browsers as a CORS error
because the body can't be read without the allow-origin header.
4. IntegrityError → 409: when (batch_id, patient_control_number) is
UNIQUE-constrained and a duplicate collides, return 409 with the
batch id instead of letting the exception 500. Same problem as (3)
for the most common ingest failure mode.
Tests added:
- test_cors_headers_present_for_loopback_ip
- test_cors_extra_origins_via_env (uses importlib.reload because the
allow-list is built at module import)
Adds pure local validators for the 10-digit NPI Luhn checksum (CMS-
published algorithm with the '80840' NPPES prefix) and 9-digit EIN
format (rejects reserved prefixes 00/07/80-89). No NPPES round-trip,
no IRS e-file lookup — catches the 99% typo case at parse time.
Surface:
- cyclone.npi.is_valid_npi / is_valid_tax_id / normalize_tax_id
- CLI: 'cyclone validate-npi <npi>' and 'cyclone validate-tax-id <ein>'
- API: GET /api/admin/validate-provider?npi=&tax_id=
- Parser validator: new R021_npi_checksum rule (warning, not error,
to keep test fixtures with placeholder NPIs ingestible)
- minimal_837p.txt fixture NPI updated from '1234567890' to the
Luhn-valid '1993999998' so strict-mode CLI parses still pass
Tests:
- test_npi.py — 27 cases (Luhn math, valid/invalid NPIs, EIN cases,
normalize_tax_id edge cases)
- test_api_validate_provider.py — 4 cases (both valid, both invalid,
omitted NPI, omitted tax_id)
- test_cli_validate.py — 8 cases (valid/invalid for both subcommands,
exit codes, malformed inputs)
- test_validator.py — 4 new R021 cases (valid Luhn silent, bad Luhn
warning, skipped when format bad, skipped when NPI missing)
Total: 923 tests pass.
Three pure-ASGI middlewares close completeness-review gaps §3.1.4
(no body/rate limits) and §3.1.25 (no security headers):
- BodySizeLimitMiddleware — rejects oversized uploads (50 MB
default, CYCLONE_MAX_BODY_BYTES override). 413 on over-cap
Content-Length and on chunked reads that cross the cap.
- RateLimitMiddleware — sliding-window per-IP limiter (300/min
default, CYCLONE_RATE_LIMIT_PER_MIN override). 429 over the
window. /api/health is exempt.
- SecurityHeadersMiddleware — stamps X-Content-Type-Options,
X-Frame-Options, Referrer-Policy, Permissions-Policy, and a
strict Content-Security-Policy on every response.
Every 413/429 also writes a tamper-evident api.request_rejected
event into the SP11 audit chain so an operator can correlate
rejections with the SP18 JSON logs.
GET /api/health is rewritten to return a subsystem snapshot:
DB connectivity (SELECT 1), MFT scheduler state, backup scheduler
state, live pubsub subscriber counts, last batch id + timestamp.
Returns status='degraded' if any subsystem is unhappy; per-subsystem
errors surfaced in the respective dict.
Cyclone.pubsub.EventBus.stats() — new method for live subscriber
counts.
13 new tests (test_security.py) + 1 updated (test_api.py health
endpoint). All 883 backend tests pass.
Cyclone previously emitted stdlib default-formatted log lines that
operators couldn't parse with anything beyond grep. SP18 replaces
that with:
- JsonFormatter: newline-delimited JSON, ISO-8601 ms timestamps,
structured "extra" dict, exception tracebacks serialized as a
single string.
- CycloneDevFormatter: tabular format for tail -f in dev.
- PiiScrubber: logging.Filter that redacts NPIs, SSNs, DOBs,
patient names — both inline in the message ("npi 1881068062")
and via PHI-keyed extras ({"dob": "1980-04-12"}).
- setup_logging(): idempotent entry point used by the API lifespan
and CLI main; respects CYCLONE_LOG_LEVEL/FILE/JSON/NO_PII_SCRUB.
- CLI --log-format=json|dev + --log-file=… flags.
- Migrate 9 highest-value log sites in scheduler/backup_scheduler/
backup_service to use extra={...} (input_filename, claims, parser,
backup_id, db_fingerprint, etc.).
34 new tests (test_logging_formatter + test_logging_scrubber +
test_logging_setup). All 867 backend tests pass.
Adds automated encrypted backups of the live SQLite file. Closes the
'no backup automation' gap called out in the completeness review
(docs/reviews/2026-06-20-cyclone-completeness-review.md §3.1 #3) and
gives the SP16 MFT scheduler a recovery path when the MFT pipeline
loses days of inbound 999/277CA work in a single crash.
Architecture
------------
- AES-256-GCM with PBKDF2-HMAC-SHA256 (200,000 iters, 16-byte salt)
- Online backups via SQLite's .backup() API — no app downtime
- Salt + passphrase persisted to macOS Keychain (separate accounts
backup.passphrase + backup.salt) so the key is reproducible across
processes
- Two-step restore (initiate → confirm) with a one-shot 64-char hex
token; the second call disposes + rebuilds the engine only if the
token matches within a 5-minute TTL
- Tamper-evident audit chain (SP11) — db.backup_created,
db.backup_failed, db.backup_pruned, db.backup_restored,
db.backup_passphrase_set
- BackupService + BackupScheduler + module-level singletons
- 8 admin endpoints + 6 CLI subcommands
- Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART=true; default
interval 24h, default retention 30 days
- Fallback posture: if no separate passphrase is set and SQLCipher
is enabled, the key is derived from the SQLCipher DB key with a
fixed salt + WARNING log (degraded but never plaintext)
New modules
-----------
- cyclone.backup — PBKDF2, AES-GCM, sidecar format
- cyclone.backup_service — create_now / list / verify / restore / prune / status
- cyclone.backup_scheduler — async tick loop with audit hooks
New surface
-----------
- 8 admin endpoints under /api/admin/backup/*
- 6 CLI subcommands under cyclone backup (init-passphrase, create,
list, verify, restore, prune, status)
- Migration 0012_backups.sql + DbBackup ORM
- store.add_backup_pending()
Tests
-----
- 14 unit tests in test_backup_crypto.py (key derivation, encrypt/
decrypt round-trip, tampered ciphertext, wrong passphrase, sidecar
round-trip, filename format)
- 19 tests in test_backup_service.py (create/list/verify/restore/
prune/status, error handling, fallback key, module singleton)
- 14 API tests in test_api_backup.py (all 8 endpoints + scheduler
endpoints, two-step restore, error responses)
- 10 tests in test_backup_scheduler.py (tick / start / stop /
audit / coalescing / module singleton)
- 5 CLI tests in test_cli_backup.py (create / list / verify /
restore confirm prompt / prune confirm prompt /
init-passphrase minimum-length check)
Total new tests: 62. All pass. Full backend suite: 833 passed,
9 skipped (gitignored prodfiles), 1 warning.
Design doc: docs/superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md
README: new 'Encrypted Backups (SP17)' section, SP17 entry in
Roadmap, retention default documented in §Project layout.
Adds an asyncio-based background scheduler that polls the Gainwell
MFT inbound path, downloads new files, and routes them through the
appropriate parser (999 / 835 / 277CA / TA1). Idempotent (re-ticks
and restarts skip already-processed files via the new
processed_inbound_files table). Crash-safe (per-file try/except so
one bad file doesn't stop the loop).
Lifespan auto-configures from the seeded dzinesco clearhouse's SFTP
block; auto-start is opt-in via CYCLONE_SCHEDULER_AUTOSTART.
Five admin endpoints added:
GET /api/admin/scheduler/status
POST /api/admin/scheduler/start
POST /api/admin/scheduler/stop
POST /api/admin/scheduler/tick
GET /api/admin/scheduler/processed-files?status=&limit=
20 new tests (15 unit + 5 API).
Self-review nits from the router-split commit:
- All four new files lacked a trailing newline (PEP 8 / POSIX).
- acks.py was lazily importing ParseResult999 / serialize_999 inside
get_ack_endpoint. Hoist to module-level — there's no import cycle
(acks.py does not import from cyclone.api), so the imports are safe
to do once.
No behavior change. Targeted tests (test_acks + test_health + test_api_gets)
still pass 41/41.
Step 2 (first half) of the architecture satisfaction loop. api.py
shrank from 2595 to 2452 lines (-143) by extracting three read-only
resource groups into cyclone.api_routers:
- health.py: GET /api/health (1 endpoint)
- acks.py: GET /api/acks, GET /api/acks/{ack_id} (2 endpoints)
- ta1_acks.py: GET /api/ta1-acks, GET /api/ta1-acks/{ack_id} (2 endpoints)
Each router owns its endpoint bodies + the small UI-shape helper that
goes with them (_ack_to_ui, _ta1_to_ui, _serialize_ta1_from_row). The
helpers stay in the router file rather than being shared because each
is only used by its own endpoints.
api.py now ends the app-wiring section with three include_router()
calls. The new package is named cyclone.api_routers (not
cyclone.api.routers) to avoid the Python package-vs-same-named-module
ambiguity that would shadow the existing cyclone.api module.
Verifies: 41 targeted tests (test_acks, test_health, test_api_gets)
pass, full pytest is 8 failed / 735 passed / 16 skipped — identical
to clean main baseline. Live curl against the running server:
GET /api/health -> 200, GET /api/acks -> 200, GET /api/ta1-acks -> 200.
See /tmp/refactor-cyclone.md for the full plan.
Nora
Cyclone
Tyler