# Cyclone Production-Readiness (local-only) — Design **Date:** 2026-06-19 **Status:** Approved (pending user review of this doc) **Scope:** First sub-project of the four-part Cyclone roadmap. Local-only deployment (no auth, `127.0.0.1`-bound). Adds a backend in-memory batch store, GET endpoints, react-query wiring on the frontend, fresh reference notes, and a new root README. Out of scope: DB persistence, reconciliation, additional X12 transaction types, additional 837P/835 validation rules. --- ## 1. Overview The Cyclone EDI suite already has working 837P and 835 parsers, a FastAPI surface for parsing uploads, and a React frontend with an Upload page. What it does not have is a way to *browse* the parsed data — once a file is uploaded, the user sees it in the Upload page, but a refresh or page navigation loses the result, and the existing Dashboard / Claims / Remittances / Providers / Activity pages still read static sample data. This sub-project closes that loop. We add a process-local in-memory store on the backend, expose GET endpoints that match the resource shape the existing UI already expects, wire react-query into the frontend so pages fetch live data and auto-refresh after a parse, and replace the lost reference notes with a clean new docs tree. After this, a user can: 1. Start the backend (`python -m cyclone serve`). 2. Start the Vite dev server (`npm run dev`). 3. Open `http://localhost:5173/upload`, drop in a `.txt` file, watch the claims stream in. 4. Navigate to `/claims`, `/remittances`, `/providers`, `/activity` and see the data they just parsed. 5. The whole stack is bound to `127.0.0.1`; no auth, no internet exposure. ## 2. Goals 1. **Persist parsed data for the session.** A new `InMemoryStore` on the backend keeps every successful parse and serves it to GET requests. Lost on backend restart — that is acceptable for a local-only tool and avoids a DB in this sub-project. 2. **Expose GET endpoints that match the UI's resource model.** `GET /api/claims`, `/api/remittances`, `/api/providers`, `/api/activity` plus `/api/batches` and `/api/batches/{id}`. The list endpoints accept filter / sort / pagination query params and support both JSON and NDJSON streaming. 3. **Wire the existing 4 UI pages to the live API** via `@tanstack/react-query` v5, with loading skeletons, error states, and automatic invalidation after a successful parse. Fall back to the in-memory sample store when `VITE_API_BASE_URL` is empty (the existing `data` adapter pattern stays). 4. **Replace the lost reference notes** with 4 fresh, short notes under `docs/reference/` (~50–80 lines each) covering 837P, 835, X12 naming, and CO Medicaid specifics. Also rewrite the root `README.md`. 5. **Tighten the local-only deploy posture.** `python -m cyclone serve` binds `127.0.0.1:8000` (not `0.0.0.0`); CORS allowlist stays `http://localhost:5173`. No auth, no API key. The README documents the exact command. ## 3. Non-goals (this sub-project) - **Database persistence.** SQLite/Postgres is sub-project 2. - **837P ↔ 835 reconciliation.** Sub-project 2. - **More 837P/835 validation rules** (REF*G1 enforcement, BHT06, CAS deep-parsing). Sub-project 3. - **999 ACK, 270/271, or other X12 transaction types.** Sub-project 3. - **Auth of any kind.** Explicitly out — local-only, single-user, `127.0.0.1`-bound. - **Structured logging, health-check enhancements, 12-factor env config, Dockerfile.** Deferred. - **Dev tooling polish** (pre-commit, Makefile, CONTRIBUTING.md, .editorconfig). Deferred. ## 4. Stack **Backend additions:** - New module `cyclone.store` (`InMemoryStore`, `BatchRecord`, mappers). - FastAPI (already in use). New routes added to `cyclone.api`. - `threading.Lock` around the store's batch list (single-process, but FastAPI may run request handlers in a threadpool). - `pytest` + `fastapi.testclient.TestClient` for new tests. **Frontend additions:** - `@tanstack/react-query` v5 added to `package.json` dependencies. - `QueryClientProvider` in `src/main.tsx`. - `useQuery` / `useMutation` hooks under `src/hooks/`. - `Skeleton` UI primitive added under `src/components/ui/` (re-uses `cn` helper). - No build-tool changes; the existing Vite + TS + Tailwind stack stays. **Docs:** - Plain Markdown under `docs/reference/`. No doc generator, no linter, no CI check. ## 5. Architecture ``` ┌──────────────────────┐ POST /api/parse-{837,835} ┌─────────────────────────┐ │ Vite/React UI │ ────────────────────────────▶ │ FastAPI backend │ │ │ GET /api/{batches,claims, │ (uvicorn 127.0.0.1) │ │ ┌────────────────┐ │ remittances,providers, │ ┌────────────────────┐ │ │ │ react-query v5 │ │ activity,batches/{id}} │ │ InMemoryStore │ │ │ │ QueryClient │ │ ◀──────────────────────────── │ │ - batches: list │ │ │ │ useQuery / │ │ (JSON or NDJSON stream) │ │ - lock │ │ │ │ useMutation │ │ │ │ - mapper funcs │ │ │ └────────────────┘ │ │ └────────────────────┘ │ │ ┌────────────────┐ │ │ ┌────────────────────┐ │ │ │ zustand store │ │ │ │ parse-837/835 │ │ │ │ (sample data) │ │ │ │ (already shipped) │ │ │ └────────────────┘ │ │ └────────────────────┘ │ └──────────────────────┘ └─────────────────────────┘ ▲ │ CORS allow │ http://localhost:5173 │ only ``` **Data flow on parse:** 1. User drops a file in `/upload`, clicks Parse. 2. Frontend `useMutation` POSTs to `/api/parse-837` (or 835) with `onProgress` (NDJSON) or `Accept: application/json` for small files. 3. Backend's existing parse route appends the resulting `ParseResult` to `InMemoryStore` on success. Failures are *not* stored. 4. `useMutation.onSuccess` calls `queryClient.invalidateQueries(['batches'], ['claims'], ['remittances'], ['activity'])` (and 837/835-specific keys). 5. Any open page reactively re-fetches via `useQuery` and re-renders with the new data. **Data flow on browse:** 1. Page mount → `useQuery` fires the GET with the page's filter / sort / pagination params. 2. First render: skeleton. Success: data. Error: error block with retry. 3. Background: `refetchOnWindowFocus: true` (default in v5) so navigating back updates the view. ## 6. Backend changes ### 6.1 New module `backend/src/cyclone/store.py` ```python class BatchRecord(BaseModel): model_config = ConfigDict(extra="ignore") id: str # uuid4 hex kind: Literal["837p", "835"] input_filename: str parsed_at: datetime # tz-aware UTC result: ParseResult | ParseResult835 # union discriminated by .kind class InMemoryStore: def __init__(self) -> None: self._batches: list[BatchRecord] = [] self._lock = threading.Lock() def add(self, record: BatchRecord) -> None: ... def list(self, *, limit: int = 100) -> list[BatchRecord]: ... def get(self, batch_id: str) -> BatchRecord | None: ... def iter_claims(self, *, batch_id: str | None = None, ...) -> Iterable[Claim]: ... def iter_remittances(self, *, batch_id: str | None = None, ...) -> Iterable[Remittance]: ... def distinct_providers(self) -> list[Provider]: ... def recent_activity(self, *, limit: int = 200) -> list[Activity]: ... store = InMemoryStore() # module-level singleton ``` **Mappers** (`to_ui_claim`, `to_ui_remittance`, `to_ui_provider`, `to_activity_event`) live in the same file. They translate the rich backend models (Pydantic, raw segments) to the simpler UI types the GET endpoints return. **Filter / sort API:** each `iter_*` method takes the relevant filter kwargs and applies them in Python. There is no index — the in-memory list is small (single-session, dozens of batches at most). Sorting uses `sorted(..., key=..., reverse=order == "desc")`. **Threading:** every public method acquires `self._lock` (RLock, since `list` may call `get` internally) and releases it. ### 6.2 New routes in `backend/src/cyclone/api.py` | Method | Path | Behavior | |---|---|---| | GET | `/api/batches` | `store.list(limit=100)`, mapped to a `BatchSummary`-shaped response. Newest first. | | GET | `/api/batches/{id}` | `store.get(id)`. 404 if missing. Returns the full `ParseResult` / `ParseResult835`. | | GET | `/api/claims` | `store.iter_claims(...)` mapped to UI `Claim[]`. Supports `batch_id`, `status`, `payer`, `provider_npi`, `date_from`, `date_to`, `sort`, `order`, `limit` (≤ 1000), `offset`. | | GET | `/api/remittances` | `store.iter_remittances(...)` mapped to UI `Remittance[]`. Supports `batch_id`, `payer`, `claim_id`, `date_from`, `date_to`, `sort`, `order`, `limit`, `offset`. | | GET | `/api/providers` | `store.distinct_providers()`. Supports `npi`, `state`, `limit`, `offset`. | | GET | `/api/activity` | `store.recent_activity(limit=200)`. Supports `kind`, `since`, `limit` (≤ 500). | | GET | `/api/health` | (unchanged) | **Streaming:** every list endpoint above accepts `Accept: application/x-ndjson`. When set, the response is a `StreamingResponse` with one JSON object per line: `{type: "item", data: {...}}` for each result, then a final `{type: "summary", data: {total: N, returned: M, has_more: bool}}`. Default JSON response wraps the same data in a `{items: [...], total: N, returned: M, has_more: bool}` envelope so the frontend can paginate uniformly. **Status mapping (837P):** each parsed `ClaimOutput` has a `validation.passed` boolean. The UI's `Claim.status` is one of `draft|submitted|accepted|denied|paid|pending`. The mapper does: - `validation.passed` and `frequency_code == "1"` → `submitted` - `validation.passed` and any `validation.warnings` → `pending` - `!validation.passed` and no error mentions `R050_diagnosis_present` → `denied` - otherwise → `draft` (rare; means validation broke somewhere) **Status mapping (835):** 835 `CLP02` status codes map to `Remittance.status` (`received|posted|reconciled`): - 1, 2, 19, 20 (paid, primary/secondary) → `received` - 4 (denied) → `received` with a `denialReason` populated from the CAS segments - 21, 22 (reversal of previous payment) → `reconciled` (out-of-scope: full reversal handling; the data is in `service_payments[*].adjustments`) - all other valid codes → `received` (default) - any code outside `cfg.allowed_status_codes` → still stored, mapped to `received`, and surfaced as a validation warning ### 6.3 Existing parse routes — wire to the store - `POST /api/parse-837` — on success, build a `BatchRecord(kind="837p", result=result)` and call `store.add(...)` *before* returning the response. On `CycloneParseError` or any unhandled exception, do *not* add. - `POST /api/parse-835` — same pattern with `kind="835"`. - Both routes also stamp `parsed_at` and generate `id = uuid.uuid4().hex`. ### 6.4 Uvicorn invocation `cyclone/__main__.py`'s `serve` branch: ```python sys.argv = [sys.argv[0], "cyclone.api:app", "--host", "127.0.0.1", "--port", "8000"] ``` - `--host 127.0.0.1` (not `0.0.0.0`). - `--port 8000` (configurable via `CYCLONE_PORT` env, default `8000`). - `--reload` only when `CYCLONE_RELOAD=1` (dev convenience). CORS in `api.py` stays as-is: `allow_origins=["http://localhost:5173"]`. ## 7. Frontend changes ### 7.1 Dependencies Add to `package.json`: ```json "dependencies": { ... "@tanstack/react-query": "^5.59.0" } ``` ### 7.2 `src/main.tsx` Wrap the existing `` in `` where `queryClient` is a module-level `new QueryClient({ defaultOptions: { queries: { staleTime: 30_000, refetchOnWindowFocus: true } } })`. ### 7.3 `src/lib/api.ts` — add GET methods Mirror the existing `request` helper. New methods (all return `Promise`): ```ts api.listBatches(): Promise api.getBatch(id: string): Promise api.listClaims(params: ListClaimsParams): Promise> api.listRemittances(params: ListRemittancesParams): Promise> api.listProviders(params?: ListProvidersParams): Promise> api.listActivity(params?: ListActivityParams): Promise> ``` NDJSON streaming is *not* required for the initial frontend wiring (the existing Upload page already streams from the POST); GET is fine as JSON. The streaming option on GET is for future large-batch scenarios and is a backend-side capability only. ### 7.4 `src/hooks/` — new query / mutation hooks ```ts // src/hooks/useBatches.ts export function useBatches() { return useQuery({ queryKey: ['batches'], queryFn: () => api.listBatches() }); } // src/hooks/useClaims.ts export function useClaims(params: ListClaimsParams) { return useQuery({ queryKey: ['claims', params], queryFn: () => api.listClaims(params) }); } // ... useRemittances, useProviders, useActivity similarly ... // src/hooks/useParse.ts export function useParse(kind: '837p' | '835') { const queryClient = useQueryClient(); return useMutation({ mutationFn: ({ file, onProgress }) => kind === '837p' ? api.parse837(file, { onProgress }) : api.parse835(file, { onProgress }), onSuccess: () => { queryClient.invalidateQueries({ queryKey: ['batches'] }); queryClient.invalidateQueries({ queryKey: ['claims'] }); queryClient.invalidateQueries({ queryKey: ['remittances'] }); queryClient.invalidateQueries({ queryKey: ['providers'] }); queryClient.invalidateQueries({ queryKey: ['activity'] }); }, }); } ``` ### 7.5 Page refactors | Page | Current | New | |---|---|---| | `src/pages/Claims.tsx` | `useAppStore((s) => s.claims)` | `useClaims({ status, search })`; falls back to store data when `!api.isConfigured` (handled in the hook) | | `src/pages/Remittances.tsx` | store-backed | `useRemittances({})` | | `src/pages/Providers.tsx` | store-backed | `useProviders({})` | | `src/pages/ActivityLog.tsx` | store-backed | `useActivity({ since: now - 7d })`, `refetchInterval: 30_000` | | `src/pages/Upload.tsx` | uses `useMutation`-less `addParsedBatch` | replace mutation with `useParse(kind)`; keep `addParsedBatch` for the "recent batches" list, but the live-data pages will refresh from the server | Each page gains: `` rows during `isLoading`, an error block with "Retry" button on `isError`, and a small footer showing `{returned} of {total}` when `has_more` is true. **Fallback when `VITE_API_BASE_URL` is empty:** the existing `data` adapter in `src/lib/api.ts` continues to provide a path that reads from the in-memory zustand store. Each new hook checks `api.isConfigured` and either runs the react-query path (default) or returns a synchronous result from the store via `useSyncExternalStore`. Page components don't need to know which path is active. The `data` adapter is removed in a follow-up sub-project once no caller depends on it. ### 7.6 UI primitive: `src/components/ui/skeleton.tsx` ~10 lines, shadcn-style: `
`. No new dep. ## 8. Docs ### 8.1 `docs/reference/837p.md` (~80 lines) Sections: - File extension: `.txt`. Encoding: ASCII or UTF-8. Delimiters: `*` element, `:` component, `~` segment, `^` repetition. - Envelope: `ISA/GS/ST` headers, `SE/GE/IEA` footers. - Loops: 2000A (billing provider), 2000B (subscriber), 2300 (claim), 2400 (service line). - Segments Cyclone parses: NM1, N3/N4, REF, CLM, HI, LX, SV1, DTP, BHT. - Segments Cyclone skips (preserved in `raw_segments` but not modeled): too many to enumerate; see the parser source for the full list. - CO Medicaid specifics: `REF*G1` prior-auth, no patient loop, `CLM05` shape `12:B:1` etc. ### 8.2 `docs/reference/835.md` (~80 lines) - Envelope: same as 837P. - Header: `BPR` (financial), `TRN` (trace), `DTM` (dates). - Loops: 1000A (payer), 1000B (payee), 2100 (CLP claim payment), 2110 (SVC service payment). - Critical balancing rules: `BPR02 == sum(CLP04)`, `CLP04 == sum(SVC03)`. - CO Medicaid specifics: `BPR10` 81-1725341 (TXIX) or 84-0644739 (BHA), `N104` 7912900843, `N1*PR` name `CO_TXIX` or `CO_BHA`. ### 8.3 `docs/reference/x12naming.md` (~50 lines) - Segments: 2–3 letter codes (`CLM`, `NM1`, `BPR`). - Elements: `*`-separated, 1-indexed. - Composite elements: `:`-separated sub-elements, indexed `01`, `02`, ... - Loops: 4-digit IDs, hierarchical (2000 contains 2300 contains 2400). - Common qualifiers we care about: `B` (POS qualifier), `ABK` (ICD-10 principal diagnosis), `MC` (Medicaid claim filing). ### 8.4 `docs/reference/co-medicaid.md` (~80 lines) - Trading partner ID (TPID) — sender / receiver roles in 837P and 835. - Payer IDs: - 837P: `NM1*PR N104 = "SKCO0"` (COHCPF). - 835: `BPR10 = "81-1725341"` (TXIX) or `"84-0644739"` (BHA); `N1*PR N104 = "7912900843"`. - Allowed frequency codes (837P `CLM05-3`): `{1, 7, 8}`. - Allowed status codes (835 `CLP02`): `{1, 2, 3, 4, 19, 20, 21, 22, 23, 25}`. - POS codes: full CMS set (89 codes; canonical list in `cyclone.parsers.payer.CMS_PLACE_OF_SERVICE_CODES`). ### 8.5 `README.md` (root, rewrite) Sections: - **What is Cyclone?** — 1 paragraph. EDI claim parser + browser UI. - **Install** — `git clone`, `cd backend && python -m venv .venv && .venv/bin/pip install -e '.[dev]'`, `cd .. && npm install`. - **Dev** — two terminals: `cd backend && .venv/bin/python -m cyclone serve` and `npm run dev`. Open `http://localhost:5173`. - **Test** — `cd backend && .venv/bin/pytest` and `npm run build`. - **Project layout** — `backend/` (Python API + parsers), `src/` (React UI), `docs/reference/` (companion notes), `docs/prodfiles/` (sample EDI files). - **Roadmap** — short list of sub-projects 2–4 with one-line descriptions and a note that they're not in this build. - **License** — no license file yet; add a `LICENSE` file when one is decided. The project is internal-use until then. ## 9. Error handling - **Backend store errors:** none expected (in-memory, no I/O). If a `BatchRecord.result` doesn't conform (e.g. we tried to store a result from a code path that returns `None`), the model validator fails loudly at insertion time. - **GET route errors:** `404` for missing `batch_id`; `400` for invalid query params (FastAPI's default validation handles this); `500` only for unexpected bugs. - **NDJSON streaming:** on a serialization error mid-stream, the response is already partially written; we log the error and end the stream. The client sees a truncated response and the final summary line is missing. The frontend falls back to "show what we have" and shows a toast. - **Frontend:** `useQuery` exposes `isError` + `error`. Each page renders an error block with a retry button that calls `refetch()`. Network errors show a sonner toast on top of the inline error. - **Parse failure:** the parse route already returns 400 / 422 with a clear message. The Upload page shows a destructive toast. The store is unchanged (failed parses are not persisted). ## 10. Testing ### 10.1 Backend (`backend/tests/`) - `test_store.py` (~8 tests) — add/list/get; concurrent adds under a thread; mapper functions; filter / sort / paginate; distinct providers; activity derivation. - `test_api_gets.py` (~12 tests) — one happy-path + one filter test per new GET endpoint (6 endpoints × 2 = 12); plus one 404 test for `/api/batches/{id}`. - `test_api_streaming.py` (1 test) — `Accept: application/x-ndjson` on `/api/claims` returns parseable lines. - `test_api_parse_persists.py` (2 tests) — parse-837 followed by `GET /api/claims` returns the claim; failed parse does *not* create a batch. Total new backend tests: **~23**. ### 10.2 Frontend (`src/`) - `src/lib/api.test.ts` — extend with 3 tests: `listClaims` builds the right URL with params, `listBatches` hits `/api/batches`, `getBatch` returns `ParseResult835` when `kind === "835"`. - No DOM / component tests in this sub-project. The page refactors are small and the smoke test below is the integration check. ### 10.3 End-to-end smoke test (manual, documented in the plan) ``` # Terminal 1 cd backend && .venv/bin/python -m cyclone serve # Terminal 2 npm run dev # In a third terminal cd backend && .venv/bin/python -m cyclone.cli parse-837 tests/fixtures/co_medicaid_837p.txt --output-dir /tmp/co-837 # (this populates a real production-like batch; we don't need the JSON files for the test, just the upload flow) # In a browser: open http://localhost:5173/upload, drop tests/fixtures/co_medicaid_837p.txt # Verify: claims stream in, summary toast appears, navigate to /claims and see them ``` ## 11. Migration / rollout - No DB, so no migration. - The frontend's existing `data` adapter stays during the transition. We delete it in a follow-up once no one calls it. - The new `InMemoryStore` is empty on first run. Existing parse-837/parse-835 callers see no behavior change (they still get the parse result; we just *also* add to the store now). - A uvicorn restart wipes the store. That is by design and called out in the README. ## 12. Out of scope (explicitly) - DB persistence, Alembic migrations, transactional safety. - 837P ↔ 835 reconciliation (matching payouts to original claims). - 999 ACK, 270/271, or other X12 transaction types. - More 837P validation rules beyond what already ships. - 835 CAS deep-parsing with reason-code explanations. - Auth (any kind: API key, JWT, session). - Rate limiting, request size limits beyond FastAPI defaults. - Structured logging, log levels via env, JSON logs. - Health-check enhancements (uptime, parser state, last-batch timestamp). - 12-factor env config beyond `CYCLONE_PORT` and `CYCLONE_RELOAD`. - Docker / docker-compose / any deploy artifact. - Pre-commit hooks, Makefile, .editorconfig, CONTRIBUTING.md, lint config. - Component tests for the React pages. - E2E browser tests (Playwright/Cypress). ## 13. Acceptance checklist - [ ] `pytest` passes with the new store + GET tests; total ≥ 121 + 23 = 144. - [ ] `npm run build` still passes. - [ ] `python -m cyclone serve` binds to `127.0.0.1:8000` and refuses connections from non-loopback. - [ ] Upload page streams a 2-claim CO fixture; navigating to `/claims` shows the same 2 claims. - [ ] Filtering by `status=submitted` on `/api/claims` returns the right subset; UI shows the same. - [ ] `GET /api/claims` with `Accept: application/x-ndjson` returns valid NDJSON. - [ ] Restarting the backend wipes the in-memory store; the UI shows "no claims yet" on first refresh. - [ ] All 4 reference notes exist under `docs/reference/` and the root `README.md` is rewritten. - [ ] The frontend still works without a backend (`VITE_API_BASE_URL` empty) — pages fall back to the zustand sample store via the existing `data` adapter.