Commit Graph

23 Commits

Author SHA1 Message Date
Nora b0956d039c docs(sp42): README.md — refresh auth/bind claim, project layout, streaming endpoints, sub-project count 2026-07-08 23:03:45 -06:00
Nora dd7da18279 fix(sftp): swap inbound/outbound paths — FromHPE is inbound, ToHPE is outbound
The dzinesco SP9 seed had `paths.inbound` and `paths.outbound` mapped to
the wrong Gainwell MFT directories:

  - paths.outbound  was  FromHPE/   (HPE sends files FROM here TO us — inbound)
  - paths.inbound   was  ToHPE/     (we send files TO here — outbound)

So `/api/clearhouse/submit` was writing 837P claims to FromHPE (where
HPE puts acks/835s) and the SP16 scheduler was polling ToHPE (where our
claims go). 999 / TA1 / 835 files in the real FromHPE inbox were
unreachable.

Semantics per operator (2026-06-24):
  - FromHPE = HPE/Gainwell → us  = 999, TA1, 835  (inbound)
  - ToHPE   = us → HPE/Gainwell = 837P claims    (outbound)

**Runtime code (the actual fix):**
  - backend/src/cyclone/store.py            — SP9 seed paths flipped
  - backend/src/cyclone/edi/filenames.py    — docstring corrected

**Test fixtures + assertions (would otherwise fail on the new seed):**
  - backend/tests/test_clearhouse_api.py
  - backend/tests/test_providers_seed.py
  - backend/tests/test_sftp_stub.py        — incl. inbound dir paths
  - backend/tests/test_sftp_paramiko.py
  - backend/tests/test_store_update_clearhouse.py
  - backend/tests/test_api_clearhouse_patch.py
  - backend/tests/test_scheduler.py
  - backend/tests/test_api_scheduler.py

**Docs (text-only — keeps the codebase self-consistent):**
  - README.md
  - docs/reference/co-medicaid.md
  - docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md

**Operator action required after merge:** the existing clearhouse row in
`~/.local/share/cyclone/cyclone.db` was seeded with the old wrong
paths. Easiest recovery:
  1. `rm ~/.local/share/cyclone/cyclone.db` and let `ensure_clearhouse_seeded` re-run on next boot, OR
  2. PATCH /api/clearhouse with the new `paths` block (the SP25
     reconfigure hook picks it up live, including by the running scheduler).

**Verification:**
  - 82 tests in the affected files pass (test_clearhouse_api, test_providers_seed,
    test_sftp_stub, test_sftp_paramiko, test_store_update_clearhouse,
    test_api_clearhouse_patch, test_scheduler, test_api_scheduler, test_filenames).
  - Full backend suite: 1029 pass + 36 pre-existing order-dependent flakes
    unrelated to this change (verified by running the same tests in isolation).
2026-06-24 22:05:55 -06:00
tyler edca04868d feat: always bind to 0.0.0.0
Reachability is controlled by the host firewall / compose port
publishing, not the bind address. Backend, compose, and docs all
now default to 0.0.0.0 so the API is reachable from the frontend
container, the host, and the LAN without per-env overrides.

Files:
- backend/src/cyclone/__main__.py: default host = 0.0.0.0
- docker-compose.yml: refresh the comment to match the new posture
- CLAUDE.md / README.md / docs/ARCHITECTURE.md / docs/REQUIREMENTS.md:
  reframe the bind note accordingly
2026-06-24 17:40:18 -06:00
Tyler 1148a03a43 docs(architecture): add top-level technical design doc
Closes round 4 of doc-prep. ARCHITECTURE.md is the day-1 read for
engineers joining Cyclone: process & network topology, tech stack
(backend + frontend + external), backend package layout with
one-line module responsibility map, frontend architecture (route
map, state management, live tail), data model (migrations 0001-0012,
ERD, 7-state claim lifecycle, audit chain), data flow (manual upload,
SFTP polling, auto-match, outbound 837P, ack ingestion), API surface,
cross-cutting concerns (logging, audit, PHI/PII, error model),
operational concerns (startup order, schedulers, health probe,
shutdown), deployment, out-of-scope, and references.

759 lines, 13 top-level sections. Cross-linked from REQUIREMENTS.md
sister-docs block and from README.md Roadmap section.
2026-06-23 14:49:16 -06:00
Tyler cad104a38c docs(requirements): add top-level requirements + DoD
38 functional requirements + 18 non-functional requirements across 13 sections.
Includes traceability matrix mapping FRs to NFRs and to existing
docs/superpowers/specs/* artifacts. Closes round 1 of doc-prep
preparation for Cyclone implementation.
2026-06-23 14:49:12 -06:00
Nora a933672411 docs: auth section + env vars 2026-06-22 15:38:38 -06:00
Tyler 888c99d848 docs: cross-link to cyclone-pipeline sibling project
Add a 'Pipeline automation agent' section under '## Dev' that
points operators to the sibling project at
/Users/openclaw/dev/cyclone-pipeline/. Add a 'Sub-project 22
(shipped)' entry to the roadmap so the new project is discoverable
from the cyclone README's release history.
2026-06-21 16:12:10 -06:00
Tyler 3d9f224eac docs(readme): add Skills section linking the 8-skill catalog 2026-06-21 12:46:52 -06:00
Tyler 1942a22629 feat(sp20): NPI Luhn checksum + Tax ID (EIN) format validation
Adds pure local validators for the 10-digit NPI Luhn checksum (CMS-
published algorithm with the '80840' NPPES prefix) and 9-digit EIN
format (rejects reserved prefixes 00/07/80-89). No NPPES round-trip,
no IRS e-file lookup — catches the 99% typo case at parse time.

Surface:
- cyclone.npi.is_valid_npi / is_valid_tax_id / normalize_tax_id
- CLI: 'cyclone validate-npi <npi>' and 'cyclone validate-tax-id <ein>'
- API: GET /api/admin/validate-provider?npi=&tax_id=
- Parser validator: new R021_npi_checksum rule (warning, not error,
  to keep test fixtures with placeholder NPIs ingestible)
- minimal_837p.txt fixture NPI updated from '1234567890' to the
  Luhn-valid '1993999998' so strict-mode CLI parses still pass

Tests:
- test_npi.py — 27 cases (Luhn math, valid/invalid NPIs, EIN cases,
  normalize_tax_id edge cases)
- test_api_validate_provider.py — 4 cases (both valid, both invalid,
  omitted NPI, omitted tax_id)
- test_cli_validate.py — 8 cases (valid/invalid for both subcommands,
  exit codes, malformed inputs)
- test_validator.py — 4 new R021 cases (valid Luhn silent, bad Luhn
  warning, skipped when format bad, skipped when NPI missing)

Total: 923 tests pass.
2026-06-21 10:46:10 -06:00
Tyler c835996bd6 feat(sp19): security hardening + rich health probe
Three pure-ASGI middlewares close completeness-review gaps §3.1.4
(no body/rate limits) and §3.1.25 (no security headers):

- BodySizeLimitMiddleware — rejects oversized uploads (50 MB
  default, CYCLONE_MAX_BODY_BYTES override). 413 on over-cap
  Content-Length and on chunked reads that cross the cap.
- RateLimitMiddleware — sliding-window per-IP limiter (300/min
  default, CYCLONE_RATE_LIMIT_PER_MIN override). 429 over the
  window. /api/health is exempt.
- SecurityHeadersMiddleware — stamps X-Content-Type-Options,
  X-Frame-Options, Referrer-Policy, Permissions-Policy, and a
  strict Content-Security-Policy on every response.

Every 413/429 also writes a tamper-evident api.request_rejected
event into the SP11 audit chain so an operator can correlate
rejections with the SP18 JSON logs.

GET /api/health is rewritten to return a subsystem snapshot:
DB connectivity (SELECT 1), MFT scheduler state, backup scheduler
state, live pubsub subscriber counts, last batch id + timestamp.
Returns status='degraded' if any subsystem is unhappy; per-subsystem
errors surfaced in the respective dict.

Cyclone.pubsub.EventBus.stats() — new method for live subscriber
counts.

13 new tests (test_security.py) + 1 updated (test_api.py health
endpoint). All 883 backend tests pass.
2026-06-21 10:21:01 -06:00
Tyler 47e0f80786 feat(sp18): structured JSON logging + PII scrubber
Cyclone previously emitted stdlib default-formatted log lines that
operators couldn't parse with anything beyond grep. SP18 replaces
that with:

- JsonFormatter: newline-delimited JSON, ISO-8601 ms timestamps,
  structured "extra" dict, exception tracebacks serialized as a
  single string.
- CycloneDevFormatter: tabular format for tail -f in dev.
- PiiScrubber: logging.Filter that redacts NPIs, SSNs, DOBs,
  patient names — both inline in the message ("npi 1881068062")
  and via PHI-keyed extras ({"dob": "1980-04-12"}).
- setup_logging(): idempotent entry point used by the API lifespan
  and CLI main; respects CYCLONE_LOG_LEVEL/FILE/JSON/NO_PII_SCRUB.
- CLI --log-format=json|dev + --log-file=… flags.
- Migrate 9 highest-value log sites in scheduler/backup_scheduler/
  backup_service to use extra={...} (input_filename, claims, parser,
  backup_id, db_fingerprint, etc.).

34 new tests (test_logging_formatter + test_logging_scrubber +
test_logging_setup). All 867 backend tests pass.
2026-06-21 09:59:34 -06:00
Tyler f003c1f73a feat(sp17): encrypted DB backup automation
Adds automated encrypted backups of the live SQLite file. Closes the
'no backup automation' gap called out in the completeness review
(docs/reviews/2026-06-20-cyclone-completeness-review.md §3.1 #3) and
gives the SP16 MFT scheduler a recovery path when the MFT pipeline
loses days of inbound 999/277CA work in a single crash.

Architecture
------------
- AES-256-GCM with PBKDF2-HMAC-SHA256 (200,000 iters, 16-byte salt)
- Online backups via SQLite's .backup() API — no app downtime
- Salt + passphrase persisted to macOS Keychain (separate accounts
  backup.passphrase + backup.salt) so the key is reproducible across
  processes
- Two-step restore (initiate → confirm) with a one-shot 64-char hex
  token; the second call disposes + rebuilds the engine only if the
  token matches within a 5-minute TTL
- Tamper-evident audit chain (SP11) — db.backup_created,
  db.backup_failed, db.backup_pruned, db.backup_restored,
  db.backup_passphrase_set
- BackupService + BackupScheduler + module-level singletons
- 8 admin endpoints + 6 CLI subcommands
- Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART=true; default
  interval 24h, default retention 30 days
- Fallback posture: if no separate passphrase is set and SQLCipher
  is enabled, the key is derived from the SQLCipher DB key with a
  fixed salt + WARNING log (degraded but never plaintext)

New modules
-----------
- cyclone.backup          — PBKDF2, AES-GCM, sidecar format
- cyclone.backup_service  — create_now / list / verify / restore / prune / status
- cyclone.backup_scheduler — async tick loop with audit hooks

New surface
-----------
- 8 admin endpoints under /api/admin/backup/*
- 6 CLI subcommands under cyclone backup (init-passphrase, create,
  list, verify, restore, prune, status)
- Migration 0012_backups.sql + DbBackup ORM
- store.add_backup_pending()

Tests
-----
- 14 unit tests in test_backup_crypto.py (key derivation, encrypt/
  decrypt round-trip, tampered ciphertext, wrong passphrase, sidecar
  round-trip, filename format)
- 19 tests in test_backup_service.py (create/list/verify/restore/
  prune/status, error handling, fallback key, module singleton)
- 14 API tests in test_api_backup.py (all 8 endpoints + scheduler
  endpoints, two-step restore, error responses)
- 10 tests in test_backup_scheduler.py (tick / start / stop /
  audit / coalescing / module singleton)
- 5 CLI tests in test_cli_backup.py (create / list / verify /
  restore confirm prompt / prune confirm prompt /
  init-passphrase minimum-length check)

Total new tests: 62. All pass. Full backend suite: 833 passed,
9 skipped (gitignored prodfiles), 1 warning.

Design doc: docs/superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md
README: new 'Encrypted Backups (SP17)' section, SP17 entry in
Roadmap, retention default documented in §Project layout.
2026-06-21 09:43:51 -06:00
Tyler d4f6fdd49c docs: sync READMEs with SP14 (Payer-Rejected lane) + endpoint inventory 2026-06-21 00:36:33 -06:00
Tyler 02841d7e6e docs(sp8): Outbound 837 Serializer section + roadmap update
- Mark SP8 as shipped (Sub-projects 2 through 8)
- Drop the 'Next up: outbound 837P serializer' trailer
- New 'Outbound 837 Serializer' section above 'Per-Line Adjustment
  Audit': design rationale (Approach A full rebuild vs spec §3.1
  hybrid), two-surface UX (single-claim drawer download + multi-claim
  inbox ZIP bundle), pointers to the serializer module, the two API
  endpoints, the frontend helpers, and the UI entry points.
- New 'SP8 endpoints' subsection in the endpoint inventory.
2026-06-20 20:55:39 -06:00
Tyler 27c1680830 docs: refresh README roadmap (SP4/6/7 shipped, add Next up) + commit SP6 plan
- Mark SP4 as fully shipped (batch diff, search, CSV, a11y all landed)
- Add SP6 (Inbox) and SP7 (Per-line reconciliation) sections
- Add SP6 + SP7 endpoint inventories
- Note next up: outbound 837P serializer

The workflow-automation plan was authored but never committed; the
features it specified have shipped, so commit it for the historical
record alongside this README refresh.
2026-06-20 19:58:26 -06:00
Tyler ceeb8b664d docs(sp7): README — Per-Line Adjustment Audit section 2026-06-20 19:51:32 -06:00
Tyler 1db33ef2fa docs(sp6): README — Inbox section 2026-06-20 18:46:24 -06:00
Tyler beff7b2128 docs(readme): document SP5 live-tail behavior, endpoints, status pill 2026-06-20 17:18:36 -06:00
Tyler 972cc99200 docs: mark SP4 partial shipped (claim drawer); tick all plan checkboxes 2026-06-20 12:37:31 -06:00
Tyler 446a02d0a2 docs: mark SP3 shipped; tick all plan checkboxes 2026-06-20 08:25:15 -06:00
Tyler 6ffa5e8c72 docs: document CYCLONE_DB_URL, backup recipe, sub-project 2 completion 2026-06-20 00:10:27 -06:00
Tyler 87cd51e1fc fix(frontend): replay row-flash on every refetch via dataUpdatedAt key
The previous static key={c.id} let React reuse the same <TableRow>
element across refetches, so animate-row-flash only played on
initial mount and on the first render of a brand-new claim_id.
Updated rows whose claim_id was unchanged never re-flashed.

Re-keying on `${c.id}-${dataUpdatedAt}` re-mounts the row on
every refetch (initial load, filter change, parse invalidation),
so the 1.2s accent-tint flash replays each time.

Also: README's 'Frontend unit tests (when added)' is stale — npm
test now exists and runs 3 passing tests.

hooks: add dataUpdatedAt: 0 to the !isConfigured fallback return
so the pages type-check.
2026-06-19 20:00:05 -06:00
Tyler fbd03cd321 docs: rewrite root README for sub-project 1 2026-06-19 19:49:48 -06:00