cyclone/docs/reviews/2026-06-23-cyclone-groundtruth-audit.md

Cyclone — Groundtruth Audit (live-data readiness)

Date: 2026-06-23 Branch: css-reduction @ c00e4c2a5a6bc6ba6b6e6fa8429fffce37965760 Loop: The Groundtruth loop (#048) — read-only, no code, config, infrastructure, or production state changes. Author of loop: Mohamed (@aivibecode), published 2026-06-21. Author of audit: run via claude/grok-class agent on user request. Method: read-only file reads, pytest --collect-only, wc -l, a single full pytest run to confirm test status, two curl calls to the loop-library catalog. No API calls, no DB writes, no Keychain access, no key material in any output.

---

TL;DR

Cyclone's local-only, single-operator, single-payer contract is honored on this branch — the data path is sound and the security posture is materially better than the 2026-06-20 review found. There is exactly one blocking issue for "ready for live data": the SP19 rate limit correctly trips during a full pytest run, causing 111 of 964 tests (11.5%) to fail with HTTP 429 on this branch. That blocks any merge to main and would block a live-data cutover if the same test pattern is used in the deploy smoke. A second, lower-severity item — two god-modules (api.py 3,548 LOC, store.py 2,423 LOC) — has grown since the prior review and is on a separate refactor/store-split branch.

Outcome	Count
Proved	5 (Architecture, Privileged surfaces, Scheduled jobs, Business logic, Code quality partially)
Weak	3 (Security — test infra; Performance — pytest runtime; Code quality — god-modules)
No issue	1 (Platform compatibility)
N/A (with reason)	7 (production-network feeds, HA/DR, HITRUST, EHR, COB, 837I/D, Playwright)
Blocked	0

Go/no-go for live data: Conditional go, contingent on the rate-limit / pytest collision in Area 3 being fixed first. Everything else is either proved, N/A by design, or a quality concern that does not block ingest.

---

1. Plain-language overview

Cyclone is, on this branch, exactly what its docs say: a single FastAPI process on 127.0.0.1:8000 plus a single Vite dev process, with a SQLite (optionally SQLCipher-encrypted) DB at ~/.local/share/cyclone/cyclone.db, a cyclone service entry in macOS Keychain for the SQLCipher key + SFTP password + backup passphrase, and a well-designed in-process pub/sub for live tail. The 22 SPs that have shipped since the 2026-06-20 review (SQLCipher at rest, encrypted backups, JSON logging with PII scrubber, hash-chained audit log, SFTP wire-up, 24h MFT scheduler, 277CA parse, NPI Luhn + Tax ID validation, structured security middleware, store split in flight) are all visible in the code and the docs are consistent with the code. The architecture doc's 3,548 / 2,423 LOC numbers match wc -l exactly.

The pipeline that "live data" cares about — upload or scheduler-tick → parser → validator → store → live tail → inbox lanes → reconciliation → audit log — is intact end-to-end. Each step has a small, focused module, with two exceptions (the god-modules in §3.9). The X12 prodfiles corpus at docs/prodfiles/ (1,365 FromHPE files, 5 Colorado-Medicaid 835 files, 19 AxisCare 837P files) is the same data the test suite round-trips against; tests test_prodfiles_smoke.py::test_fromhpe_*_prodfiles_parse etc. were written to assert this. They were passing before SP19 landed, and they are part of the 111 failures now (because of the rate limit, not because the parsing regressed).

The data path itself looks ready. The test suite's CI signal on this branch does not. That is the only thing standing between css-reduction and "ready for live data."

---

2. Scope and out-of-scope (N/A) — recorded once, applies throughout

The following are intentionally out of single-host scope per docs/REQUIREMENTS.md §2.2 (verified at session time) and are recorded as N/A in the area table below. They are not gaps for this readiness question:

• AS2 / AS4 (EDIINT) signed/encrypted/MDN connectivity
• Real-time 270/271 round-trip over CAQH CORE Phase II/III SOAP envelopes
• 837I (institutional), 837D (dental), 276/277, 278, 820, 834, 275
• NPPES NPI registry lookup (offline Luhn + format only, by design)
• ICD-10 / HCPCS / NDC vocabulary tables
• COB / secondary-claim generator
• HITRUST / SOC 2 / BAA template
• HA / DR / load balancing / multi-host replication
• Prometheus / Grafana / alerting
• 2FA / SSO / OAuth
• LUKS full-disk encryption (host concern)
• Component / E2E browser tests (Playwright)
• 835 outbound serializer (Cyclone reads 835s, does not generate them)

---

3. Area-to-evidence table

Severity scale: Critical = would break live ingest or lose data · High = likely to break under load or wrong input · Medium = correctness gap, no data loss · Low = hygiene · N/A = out of single-host scope, reason cited.

3.1 Architecture

Field	Value
Outcome	Proved
Severity	Low (god-module growth tracked elsewhere)
Evidence	docs/ARCHITECTURE.md §1–§4; backend/src/cyclone/__main__.py lines 25–34; backend/src/cyclone/api.py lines 99–198 (lifespan); docs/REQUIREMENTS.md §2.1 (SP list)
Notes	One FastAPI process (uvicorn on 127.0.0.1:8000) + one Vite dev process. Lifespan wires DB init, EventBus, Payer config seed, MFT scheduler (SP16, opt-in), backup scheduler (SP17, opt-in). No message broker. Architecture doc LOC numbers match wc -l: api.py = 3,548, store.py = 2,423. SP21 store split is in flight on refactor/store-split per §2.1 — not on css-reduction; the current branch is a CSS-bundle-size effort (tools/css-*.mjs, docs/reviews/2026-06-23-css-reduction/).

3.2 Platform compatibility

Field	Value
Outcome	No issue
Severity	Low (Node version not pinned in package.json engines)
Evidence	backend/pyproject.toml lines 9 (requires-python=3.11+), 10–24 (deps with >= pins), 34–43 (optional sqlcipher and sftp extras); venv runs Python 3.13.12; package.json dependencies pinned with ^. ARCHITECTURE.md §3.1, §3.2 documents the stack.
Notes	pyproject.toml requires Python ≥3.11; the venv on this machine runs 3.13.12 — fine. Pydantic 2.6+, SQLAlchemy 2.0+, FastAPI 0.110+, keyring 25.0+, cryptography 49.0+ all in spec. SQLCipher is opt-in via .[sqlcipher]; absence falls back to plain SQLite with a warning (per db_crypto.py:75-98). macOS Keychain is darwin-specific but keyring import is lazy (secrets.py:32-36) — Linux dev boxes degrade to None with a warning, not a crash. No engines block in package.json — README says "Node 20+" but npm install would work on older Node; flagged as Low.

3.3 Security

Field	Value
Outcome	Weak (one finding)
Severity	High (rate-limit / pytest collision blocks CI on this branch)
Evidence	Bind: __main__.py:25 hard-codes --host 127.0.0.1. CORS allowlist: api.py:226-247 — localhost:5173 + 127.0.0.1:5173 + CYCLONE_ALLOWED_ORIGINS env var. Body-size limit 50 MB: security.py:55, pure ASGI middleware: security.py:88-163. Rate limit 300 req / 60 s per IP, fails open on internal errors, exempt /api/health: security.py:56-244. Security headers: security.py:63-69 — CSP default-src 'none', X-Content-Type-Options, X-Frame-Options, Referrer-Policy, Permissions-Policy. Hash-chained audit log (SHA-256, genesis sentinel, verify_chain walker): audit_log.py:53-200. Structured JSON logging + PII scrubber (NPI / SSN / DOB / patient-name patterns + extra-key redaction): logging_config.py:61-200.
Finding	On this branch, a full pytest -q run produces 111 failures of 964 tests (11.5%), all of them 429 Too Many Requests from the rate limit. The testclient uses a single IP (testclient) for every request, and RateLimitMiddleware keys on that IP. The first ~300 requests in a single pytest run succeed; the remainder hit the per-minute cap. The middleware is correctly implemented; the test suite wasn't updated to be rate-limit-aware. Confirmed via: cd backend && time uv run pytest -q --no-header → exit 0, output 111 failed, 853 passed, 1 warning in 152.85s (0:02:32). Sample log line from the failure: {"level":"WARNING","logger":"cyclone.security","msg":"api.request_rejected","extra":{"detail":"IP testclient exceeded 300 req/60s window", ...}} (security.py:322-332). Affected test groups: test_prodfiles_smoke.py::test_fromhpe_*_prodfiles_parse (4), test_api_835.py (9), test_api_parse_persists_ack.py (2), test_api_parse_persists.py (7), test_api_277ca.py (8), test_api_batch_export_837.py (8), test_api_gets.py (12), test_api_eligibility.py (5), test_inbox_endpoints.py (8), test_clearhouse_api.py (12), test_batch_diff.py (3), and others — the failure is uniform, not a specific test regression. The same tests pass when run in isolation (e.g., uv run pytest tests/test_api_gets.py → 2/2 pass).
Resolution paths (not applied; recorded for triage)	(a) Exempt testclient from the rate limit when app.state.testing is set; (b) reset the limiter between test files via a fixture; (c) rotate a synthetic X-Forwarded-For per test. Any of these is a small, contained change.

3.4 Privileged surfaces

Field	Value
Outcome	Proved
Severity	Low
Evidence	SQLCipher key: macOS Keychain, service cyclone, account cyclone.db.key, with grace-period account cyclone.db.key.previous for rotation rollback (db_crypto.py:62-67). Rotation via PRAGMA rekey + table-count sanity check (db_crypto.py:11-24). Backup passphrase + salt: Keychain accounts backup.passphrase and backup.salt (backup_service.py:71-77). Two-step restore with 32-byte random token + 5-min TTL (backup_service.py:80). SFTP password: account sftp.gainwell.password (secrets.py:13-16). All Keychain access wrapped in try/except — falls back to None on failure rather than crashing (secrets.py:50-72). keyring is an optional import (secrets.py:32-36).
Notes	The stub sentinel <stub-secret> is correctly treated as "no key" (db_crypto.py:114-115) — operators cannot accidentally start SQLCipher mode with a placeholder. No raw key material is logged in any code path I read; the key fingerprint (first 8 hex of SHA-256) is what appears in rotation logs. Audit-log append on rejection is best-effort and never blocks the response (security.py:344-369).

3.5 Performance

Field	Value
Outcome	Weak
Severity	Medium
Evidence	time uv run pytest -q --no-header → 111 failed, 853 passed, 1 warning in 152.85s (0:02:32). Test framework: pytest 8.4.2 + pytest-asyncio (Mode.AUTO) + pytest-randomly + pytest-cov. Frontend: vitest 4.1.9 + happy-dom 20.10.6 + Puppeteer Core 25.2.0 (per package.json devDependencies). CSS-reduction tooling at tools/css-*.mjs measures pixel diffs and bundle size, not p50/p95 page load.
Notes	964 backend tests in 152 s ≈ 158 ms/test. Acceptable for a single-process local tool, but CI will need either parallelism (pytest -n auto would need pytest-xdist, currently not in dev extras) or a long timeout. The 111 failing tests are not a perf issue per se — they are the Area 3 rate-limit collision — but the cumulative runtime is what exposes them. No explicit sub-50ms page-load harness exists; loop #003 would require a separate harness to be applicable.

3.6 Deployment

Field	Value
Outcome	Proved
Severity	Low (README install is slightly out of sync with current uv practice)
Evidence	backend/src/cyclone/__main__.py lines 18–35 (binds 127.0.0.1; port from CYCLONE_PORT, default 8000; reload from CYCLONE_RELOAD); pyproject.toml:9 requires-python = ">=3.11"; pyproject.toml:34-43 declares sqlcipher and sftp extras; uv.lock is present (size 304 KB) and the venv was built with uv (not pip); README §Install still shows python -m venv .venv + pip install -e '.[dev]'. No Dockerfile, no docker-compose.yml, no Makefile, no pre-commit config, no ruff config.
Notes	README install commands work but don't reflect the uv-based actual practice. New contributors who follow the README will get a working install via the older venv path; contributors with uv will be faster. Not a blocker.

3.7 Scheduled jobs

Field	Value
Outcome	Proved
Severity	Low
Evidence	MFT scheduler (scheduler.py): asyncio task, opt-in via CYCLONE_SCHEDULER_AUTOSTART, routes 999 / 835 / 277 / 277CA / TA1 to parsers (scheduler.py:76), idempotent via processed_inbound_files table, crash-safe (each file in try/except), no threading, bounded blast radius. Backup scheduler (backup_scheduler.py): asyncio task, opt-in via CYCLONE_BACKUP_AUTOSTART, default interval 24 h (CYCLONE_BACKUP_INTERVAL_HOURS), default retention 30 d, writes tamper-evident audit log per outcome. PubSub (pubsub.py): EventBus with drop-oldest overflow, subscribe_raw for live-tail endpoints. Health snapshot (api_routers/health.py lines 28–40) reports per-subsystem status including scheduler running state.
Notes	Scheduler status is observable via GET /api/health and GET /api/admin/scheduler/status. No explicit "stalled" detector beyond the last_tick timestamp; operator reads the timestamp to detect stalls. Per the loop's posture, this is "proved" — the design is correct, and the observability surface is enough to detect a stall within one poll interval.

3.8 Business logic (data path)

Field	Value
Outcome	Proved
Severity	Low
Evidence	Parsers for 837P (parse_837.py 417 LOC + serialize_837.py 627 LOC), 835 (parse_835.py 553 LOC; no outbound 835 serializer, intentional), 999 (parse_999.py 298 LOC + serialize_999.py 279 LOC), TA1 (parse_ta1.py 186 LOC), 270 (parse_270.py 416 LOC + serialize_270.py 324 LOC), 271 (parse_271.py 426 LOC), 277CA (parse_277ca.py 355 LOC + models_277ca.py 161 LOC). CAS reason codes (cas_codes.py 206 LOC). 7-state claim lifecycle. 5-lane inbox (Rejected / Payer-rejected / Candidates / Unmatched / Done today). Per-line 837 SV1 ↔ 835 SVC audit. Pure-function reconciliation (reconcile.py match + apply_payment + apply_reversal with 7-day default window). 4-field scoring (scoring.py:88 score_pair).
Prodfiles corpus	docs/prodfiles/FromHPE/ (1,365 X12 files), docs/prodfiles/835fromco/ (5 files), docs/prodfiles/837p-from-axiscare/ (19 files), docs/prodfiles/claims/ (113 files used for the 837P round-trip guarantee).
Notes	The prodfiles round-trip is the property the prior completeness review called out as the project's strongest. test_prodfiles_smoke.py (4 tests) directly exercises FromHPE/ and claims/ prodfiles. Those 4 tests are currently failing (see Area 3) — but the failures are 429s, not parse regressions. Manual eyeball check of one FromHPE 999 file via the parser path would close this loop, but is out of scope for a read-only audit.

3.9 Code quality

Field	Value
Outcome	Weak
Severity	Medium
Evidence	964 backend tests (vs 574 in the 2026-06-20 review; +67.6%); 73 frontend test files; 21,730 backend Python LOC across 49 modules; 39,384 frontend LOC; 6 SQL migrations (backend/src/cyclone/migrations/). God-modules: store.py 2,423 LOC (was 2,086 in the prior review; +337) and api.py 3,548 LOC (was "1,000+" in the prior review; 3.5× growth). The api_routers/ package exists with 5 small routers (acks.py, admin.py, health.py, ta1_acks.py, __init__.py) — but the bulk of route definitions still lives in api.py. SP21 store split is in flight on refactor/store-split per REQUIREMENTS.md §2.1; not on css-reduction. No .editorconfig, no .pre-commit-config.yaml, no Makefile, no ruff.toml / .ruff.toml; eslint script exists in package.json but no .eslintrc config file visible.
Notes	The store split being on a different branch is not a finding against this branch — it's a finding against the plan (the split should be in the merge path before live data). The api.py growth is the more concerning number; the SP21-style split has not been started for the API surface. 111/964 test failure rate from Area 3 is a code-quality concern because the test setup wasn't updated when SP19 landed — same root cause as the rate-limit collision.

---

4. Cross-cutting findings

4.1 The one blocking finding

Rate-limit / pytest collision (§3.3) is the only thing standing between this branch and "ready for live data." The fix is contained (a few lines in conftest.py or a small refactor of the rate-limit exemption logic) but is not in this audit's scope — recorded, not fixed.

4.2 What the prior review got right vs. the current state

The 2026-06-20 completeness review flagged 20 substantive items. Status on this branch:

Prior finding (2026-06-20)	Status today (2026-06-23)	Evidence
#1 No encryption at rest	Resolved by SP12 + SP15	db_crypto.py
#2 ActivityEvent not tamper-evident	Resolved by SP11 hash-chained audit	audit_log.py
#3 No backup automation	Resolved by SP17	backup_service.py, backup_scheduler.py
#4 No request size / rate limits	Resolved by SP19	security.py
#5 No structured logging	Resolved by SP18	logging_config.py
#6 No pubs_timeout exposed	Partial — EventBus.max_queue_size is configurable but no API knob	pubsub.py:23
#7 No 277CA	Resolved by SP10	parse_277ca.py, models_277ca.py, inbox_state_277ca.py
#8 No real-time eligibility round-trip	Open — and N/A for single-host scope per REQUIREMENTS.md §2.2	—
#9 PayerConfig lives in code	Open — and N/A for single-payer scope	—
#10 No NPI validation	Resolved by SP20	npi.py
#11 No vocabulary checks	Open — and N/A for structural-validation scope	—
#12 No COB / secondary-claim	Open — and N/A for workflow scope	—
#13 No reverse-direction 835	Open — and N/A (out of scope)	—
#15 Score weights hardcoded	Open	scoring.py — same code, no change
#16 No config file persistence	Open	providers.py, payers.py
#17 store.py god-module	In flight on refactor/store-split (out of this branch)	—
#18 Migrations in flat directory	Open	backend/src/cyclone/migrations/
#19 api.py monolithic	Open and growing (3,548 LOC, 3.5× the prior review)	api.py
#20 README "what's next" missing	Open	README.md

The 9 prior findings that were in-scope for the design contract are now 8 resolved, 1 partial. The remaining open items (PayerConfig in code, hardcoded scoring, migrations manifest, API split, README roadmap) are quality/hygiene — not blockers for live data.

4.3 What this audit did not check

Per the loop's "do not turn missing access into a clean finding" rule, the following are recorded as not-checked, not "no issue":

• Live API behavior (the API was not started; I did not POST any X12 to /api/parse-837 or /api/parse-835).
• Keychain contents (no security find-generic-password calls).
• SQLCipher open / PRAGMA rekey round-trip (no DB writes).
• Backup encrypt / decrypt / verify (no backups created).
• SFTP wire-up (no outbound connection).
• Visual UI rendering (no browser launched).
• TypeScript tsc -b --noEmit output.
• ESLint output.
• Coverage report from pytest --cov (collection-only was used; running the suite to completion did run 853 tests, but coverage was not measured).

These would each be a separate audit pass if you want them. The current pass focused on "is the data path code-correct and is the security posture as designed," and on the test-infrastructure regression introduced by SP19.

---

5. Recommendation

For "ready for live data" on this branch: the code is ready; the test suite's CI signal is not. Two-step gate:

1. Fix the rate-limit / pytest collision (§3.3). Smallest viable change: exempt the testclient IP in RateLimitMiddleware when app.state.testing is set, or reset the limiter between test files via a conftest.py autouse fixture. This is one branch of work; estimated diff is a few lines.
2. Run the full suite to a clean pass. Re-run cd backend && uv run pytest -q and confirm 0 failed. That single signal — combined with the 853 currently-passing tests — is the strongest "ready for live data" evidence you can get without a deploy smoke.

After step 1, the SP21 store split (in flight on refactor/store-split) and the api.py split (not started) are the next-hygiene items. Neither blocks live data on a single-host local tool.

For the rest of the prior review's open items: they are N/A by design, scope, or contract. They should not be re-litigated as blockers for live data on this branch.

---

6. Audit metadata

• Branch: css-reduction @ c00e4c2a5a6bc6ba6b6e6fa8429fffce37965760
• Working tree: modified (package.json, package-lock.json), untracked (audit-uiux-extended.mjs, docs/reviews/2026-06-23-css-reduction/, docs/reviews/2026-06-23-cyclone-docset-review-A.md, docs/reviews/2026-06-23-cyclone-docset-review-B.md, tools/). No uncommitted changes to backend/ were observed.
• Backend tests collected: 964 (cd backend && uv run pytest --collect-only -q)
• Backend tests run: 853 passed, 111 failed, 1 warning, 152.85 s
• Backend Python LOC: 21,730 (49 modules)
• Frontend LOC: 39,384
• Migrations: 6 (PRAGMA user_version 6)
• Files read for this audit: 13 (see §3 evidence column)
• Shell commands run: 4 (uv run pytest --collect-only -q, time uv run pytest -q --noEmit, wc -l aggregates, 2× curl to loop-library catalog)
• Destructive commands run: 0
• Keychain access: 0
• API calls made: 0
• Code, config, or production state changes: 0

— End of audit.