Compare commits
2 Commits
v0.2.0
..
bbf89c9dd8
| Author | SHA1 | Date | |
|---|---|---|---|
 Tyler
|
bbf89c9dd8 | ||
|
Tyler
|
d25f00ac58 |
+2
-3
@@ -35,6 +35,5 @@ claims_output/
|
|||||||
# Worktrees (subagent-driven development)
|
# Worktrees (subagent-driven development)
|
||||||
.worktrees/
|
.worktrees/
|
||||||
|
|
||||||
# Brainstorm session artifacts (visual companion mockups, events, server state).
|
# Brainstorm session artifacts (visual companion mockups, events, server state)
|
||||||
# Skills under .superpowers/skills/ are committed project-scoped guidance.
|
.superpowers/
|
||||||
.superpowers/brainstorm/
|
|
||||||
|
|||||||
@@ -1,189 +0,0 @@
|
|||||||
---
|
|
||||||
name: cyclone-api-router
|
|
||||||
description: "Cyclone FastAPI router conventions (api_routers/, api_helpers.py, response shapes, error envelopes). Use when: adding or changing an HTTP endpoint, splitting a route out of api.py, or wiring a new helper into api_helpers.py."
|
|
||||||
---
|
|
||||||
|
|
||||||
# cyclone-api-router
|
|
||||||
|
|
||||||
Cyclone splits its FastAPI surface two ways: small resource-group
|
|
||||||
routers live in `backend/src/cyclone/api_routers/<topic>.py` and are
|
|
||||||
mounted bare into `api.py`; the high-traffic streaming and parse
|
|
||||||
endpoints still live as top-level decorators in `api.py` itself. This
|
|
||||||
skill codifies the conventions so additions stay consistent with the
|
|
||||||
four routers already shipped (`acks`, `admin`, `health`, `ta1_acks`).
|
|
||||||
|
|
||||||
As of this writing: **4 router modules** under
|
|
||||||
`backend/src/cyclone/api_routers/`, **one shared helpers module** at
|
|
||||||
`backend/src/cyclone/api_helpers.py` (248 lines, NDJSON primitives +
|
|
||||||
content negotiation + `tail_events`), and ~30 routes still inlined
|
|
||||||
in `backend/src/cyclone/api.py`. The next refactor target is the
|
|
||||||
parse endpoints.
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- **Adding an endpoint.** You're adding a new GET / POST handler —
|
|
||||||
you need to know whether it belongs in `api.py` (parse / streaming)
|
|
||||||
or in a new router under `api_routers/`, and what the response
|
|
||||||
shape and test file conventions look like.
|
|
||||||
- **Splitting a route.** You're moving a route out of `api.py` into a
|
|
||||||
dedicated `api_routers/<topic>.py` module and need the import /
|
|
||||||
mounting rules (`from cyclone.api_routers import <topic>` then
|
|
||||||
`app.include_router(<topic>.router)`).
|
|
||||||
- **Adding a helper.** You're wiring a new function into
|
|
||||||
`api_helpers.py` (NDJSON primitive, content-negotiation probe,
|
|
||||||
tail-event helper) and need to keep it private to the API layer.
|
|
||||||
- **Defining an error response.** You're raising from a route handler
|
|
||||||
and need the conventional `HTTPException(status_code=..., detail=...)`
|
|
||||||
shape used everywhere else in the API surface.
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
1. **No new top-level routes in `api.py` for resource groups.** Any
|
|
||||||
endpoint grouped under a resource (`/api/<resource>` and its
|
|
||||||
`/{id}` detail) lives in `backend/src/cyclone/api_routers/<topic>.py`
|
|
||||||
as an `APIRouter`. The streaming list endpoints (`/api/claims/stream`,
|
|
||||||
`/api/remittances/stream`, `/api/activity/stream`) and the parse
|
|
||||||
endpoints (`/api/parse-*`) currently stay in `api.py` because they
|
|
||||||
span multiple store modules — don't move them unless you're also
|
|
||||||
restructuring the store split.
|
|
||||||
2. **Reuse `api_helpers.py`.** NDJSON primitives (`ndjson_line`,
|
|
||||||
`ndjson_stream_list`, `ndjson_stream_837`, `ndjson_stream_835`),
|
|
||||||
content negotiation (`client_wants_json`, `wants_ndjson`), the
|
|
||||||
strict / `raw_segments` rewrites (`strict_rewrite_837`,
|
|
||||||
`strict_rewrite_835`, `drop_raw_segments_837`, `drop_raw_segments_835`),
|
|
||||||
and the shared live-tail generator (`tail_events`,
|
|
||||||
`heartbeat_seconds`) all live there. Don't duplicate them in a
|
|
||||||
router. The module's docstring (`api_helpers.py:1-18`) declares it
|
|
||||||
private to the API layer — no business logic, no DB writes.
|
|
||||||
3. **Response shape.** Every successful response is a **plain dict**
|
|
||||||
produced by a per-router `<entity>_to_ui(row)` helper (see
|
|
||||||
`_ack_to_ui` at `api_routers/acks.py:29-48` and `_ta1_to_ui` at
|
|
||||||
`api_routers/ta1_acks.py:23-37`). This dict shape **must** match
|
|
||||||
the matching `<entity>_written` event payload so live-tail pages
|
|
||||||
don't drift (see `cyclone-store` for the serializer contract).
|
|
||||||
Errors use FastAPI's `HTTPException` with a `detail` dict of the
|
|
||||||
form `{"error": "<Title>", "detail": "<message>"}` —
|
|
||||||
`acks.py:85-88`, `ta1_acks.py:72`. There is **no** shared
|
|
||||||
`ErrorEnvelope` Pydantic model; the `detail` dict is the contract.
|
|
||||||
4. **Mounting.** Routers are mounted bare in `api.py:251-256` —
|
|
||||||
`app.include_router(<name>.router)` with **no** `prefix=`
|
|
||||||
argument. Each `@router.<verb>` decorator carries the **full**
|
|
||||||
`/api/<resource>` path itself (see `acks.py:51,75`,
|
|
||||||
`ta1_acks.py:49,67`, `admin.py:23`, `health.py:28`). The
|
|
||||||
`router = APIRouter()` declaration carries no `tags=` either —
|
|
||||||
keep it minimal.
|
|
||||||
5. **Streaming endpoints.** Use
|
|
||||||
`StreamingResponse(media_type="application/x-ndjson")` and feed it
|
|
||||||
either `ndjson_stream_list(items, total, returned, has_more)` (for
|
|
||||||
list pages) or `tail_events(request, bus, kinds)` (for live-tail
|
|
||||||
pages). See `acks.py:62-66` for the list-stream skeleton and
|
|
||||||
`api.py:1357-1401` for the full live-tail pattern (snapshot →
|
|
||||||
`snapshot_end` → subscription → heartbeats). See `cyclone-tail`
|
|
||||||
for the wire format.
|
|
||||||
6. **Tests.** Every new endpoint gets a `test_api_<topic>_<verb>.py`
|
|
||||||
under `backend/tests/` (see `cyclone-tests` for the naming
|
|
||||||
convention + autouse `conftest.py`). Existing examples:
|
|
||||||
`test_api_validate_provider.py` (admin),
|
|
||||||
`test_api_parse_persists_ack.py` (acks).
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
|
|
||||||
### A new `APIRouter` skeleton
|
|
||||||
|
|
||||||
Pattern from `backend/src/cyclone/api_routers/acks.py:1-72`. Module
|
|
||||||
docstring names the resource, imports the shared helpers, declares
|
|
||||||
`router = APIRouter()` with no prefix, and defines a `_foo_to_ui(row)`
|
|
||||||
mapper at module scope.
|
|
||||||
|
|
||||||
```python
|
|
||||||
"""``/api/foo`` — list & detail endpoints for <topic>."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from fastapi import APIRouter, HTTPException, Query, Request
|
|
||||||
from fastapi.responses import StreamingResponse
|
|
||||||
|
|
||||||
from cyclone.api_helpers import ndjson_stream_list, wants_ndjson
|
|
||||||
from cyclone.store import store
|
|
||||||
|
|
||||||
router = APIRouter()
|
|
||||||
|
|
||||||
|
|
||||||
def _foo_to_ui(row) -> dict:
|
|
||||||
"""Map a Foo ORM row to the UI shape used by ``/api/foo``."""
|
|
||||||
return {"id": row.id, "name": row.name}
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/api/foo")
|
|
||||||
def list_foo(
|
|
||||||
request: Request,
|
|
||||||
limit: int = Query(100, ge=1, le=1000),
|
|
||||||
):
|
|
||||||
"""Return the list of persisted Foo rows, newest first."""
|
|
||||||
rows = store.list_foo()
|
|
||||||
items = [_foo_to_ui(r) for r in rows[:limit]]
|
|
||||||
total = len(rows)
|
|
||||||
returned = len(items)
|
|
||||||
has_more = total > returned
|
|
||||||
if wants_ndjson(request):
|
|
||||||
return StreamingResponse(
|
|
||||||
ndjson_stream_list(items, total, returned, has_more),
|
|
||||||
media_type="application/x-ndjson",
|
|
||||||
)
|
|
||||||
return {"items": items, "total": total, "returned": returned, "has_more": has_more}
|
|
||||||
```
|
|
||||||
|
|
||||||
### A `get_<topic>` detail endpoint with 404
|
|
||||||
|
|
||||||
Pattern from `api_routers/acks.py:75-104` and `ta1_acks.py:67-76`.
|
|
||||||
Path param is `<entity>_id` (not `id`) so it doesn't shadow
|
|
||||||
FastAPI's internal `id` and the OpenAPI docs stay self-describing.
|
|
||||||
|
|
||||||
```python
|
|
||||||
@router.get("/api/foo/{foo_id}")
|
|
||||||
def get_foo(foo_id: int) -> dict:
|
|
||||||
"""Return one persisted Foo row with its parsed detail.
|
|
||||||
|
|
||||||
Path param is ``foo_id`` (not ``id``) to avoid shadowing
|
|
||||||
FastAPI's internal ``id`` name and to keep OpenAPI docs
|
|
||||||
self-describing. Returns 404 when the row is missing — never 500.
|
|
||||||
"""
|
|
||||||
row = store.get_foo(foo_id)
|
|
||||||
if row is None:
|
|
||||||
raise HTTPException(
|
|
||||||
status_code=404,
|
|
||||||
detail={"error": "Not found", "detail": f"Foo {foo_id} not found"},
|
|
||||||
)
|
|
||||||
return _foo_to_ui(row)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Mounting in `api.py`
|
|
||||||
|
|
||||||
Pattern from `backend/src/cyclone/api.py:246-256`. The block lives
|
|
||||||
just after middleware registration and just before the first
|
|
||||||
`@app.<verb>` decorator.
|
|
||||||
|
|
||||||
```python
|
|
||||||
# Resource-group routers. Each module owns its own APIRouter and is
|
|
||||||
# registered below. New resources go in `cyclone.api_routers.<name>`
|
|
||||||
# and are wired in here.
|
|
||||||
from cyclone.api_routers import acks, admin, health, ta1_acks # noqa: E402
|
|
||||||
|
|
||||||
app.include_router(health.router)
|
|
||||||
app.include_router(acks.router)
|
|
||||||
app.include_router(ta1_acks.router)
|
|
||||||
app.include_router(admin.router)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
- **Don't import from `cyclone.api` into a router — the dependency runs the other way.** Routers are mounted *into* `api.py` (`api_routers/acks.py` etc. know nothing about `cyclone.api`). A circular import would silently break the `from cyclone.api_routers import ...` block at `api.py:251`.
|
|
||||||
- **Don't introduce Pydantic response models where the codebase returns dicts.** Every existing list / detail endpoint returns a plain dict produced by a `<entity>_to_ui(row)` helper (see `acks.py:29-48`, `ta1_acks.py:23-37`). That dict shape **is** the event payload that `<entity>_written` carries — see `cyclone-store`. Introducing a Pydantic model on one side drifts the event payload from the list shape and silently breaks live-tail dedup.
|
|
||||||
- **Don't bypass `CycloneStore` to query the ORM directly from a route.** Always call `store.<method>(...)` (`store.list_acks()`, `store.get_ta1_ack(ack_id)`) so the read path picks up the same session + snapshot serializer as the live-tail subscriber. A raw `with db.SessionLocal()() as s: s.get(Foo, foo_id)` in a handler bypasses the serializer contract and breaks the event-payload match. See `cyclone-store`.
|
|
||||||
- **Don't set `prefix=` on `APIRouter`.** Mount the router bare (`app.include_router(<name>.router)`) and put the full `/api/<resource>` path in the decorator. Mixing the two styles scatters the URL across two files and breaks `grep "/api/foo"` audits.
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
|
|
||||||
- **`cyclone-store`** — most routes call `store.<method>(...)` and the dict payload **is** the `<entity>_written` event payload; load when adding a route so the read path stays aligned with the pubsub contract.
|
|
||||||
- **`cyclone-tail`** — streaming endpoints (`/api/<resource>/stream`) and the NDJSON wire format; load when adding a live-tail route or changing the wire format.
|
|
||||||
- **`cyclone-edi`** — parse endpoints (`/api/parse-837`, `/api/parse-835`, `/api/parse-999`, `/api/parse-ta1`, `/api/parse-277ca`) currently live in `api.py`; load when adding or changing a parse endpoint.
|
|
||||||
- **`cyclone-tests`** — endpoint tests follow the `test_api_<topic>_<verb>.py` naming under `backend/tests/`; load when writing the test for a new endpoint.
|
|
||||||
@@ -1,187 +0,0 @@
|
|||||||
---
|
|
||||||
name: cyclone-cli
|
|
||||||
description: "Cyclone CLI subcommand conventions (cli.py — Click group + subcommands parse-837/parse-835/validate-npi/validate-tax-id/backup, --yes + click.confirm for destructive ops, exit codes 0/1/2, CliRunner smoke tests in backend/tests/test_cli_*.py). Use when: adding a CLI subcommand, changing an exit code, adding a smoke test, or wiring a security-sensitive command (backup, key rotation, anything touching secrets.py)."
|
|
||||||
---
|
|
||||||
|
|
||||||
# cyclone-cli
|
|
||||||
|
|
||||||
The operator-facing CLI is a **Click** group at `cli.py:45` (`@click.group()` for `main`), mounted as the `cyclone` console script in `pyproject.toml:54` (`cyclone = "cyclone.cli:main"`). Seven subcommands ship today: `parse-837`, `parse-835`, `validate-npi`, `validate-tax-id`, plus the `backup` group (`init-passphrase`, `create`, `list`, `verify`, `restore`, `prune`, `status`). `serve` lives separately in `__main__.py:19` (dispatches `uvicorn cyclone.api:app`).
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- **Adding a subcommand.** You need a new operator command (e.g. `cyclone rotate-key`) and want to match the existing Click decorator + smoke-test rhythm.
|
|
||||||
- **Changing an exit code.** You're tweaking which `sys.exit(N)` a subcommand raises and need the 0/1/2 contract used by `parse_837`, `parse_835`, `validate_npi_cmd`, `validate_tax_id_cmd`, and the `backup` group.
|
|
||||||
- **Adding a smoke test.** You need `backend/tests/test_cli_<name>.py` using `click.testing.CliRunner` (NOT `subprocess.run`) and want the canonical fixture + monkeypatch layout (Keychain stub, fresh SQLite, `CYCLONE_BACKUP_DIR`).
|
|
||||||
- **Wiring a security-sensitive command.** Anything touching `cyclone/secrets.py` (Keychain writes), DB key rotation, or destructive restores needs the two-step confirm dance used by `backup restore` / `backup prune` (`cli.py:462,509`).
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
1. **Click decorator pattern, not argparse.** Each subcommand is a
|
|
||||||
top-level function decorated with `@main.command("<name>")` and
|
|
||||||
one `@click.option` / `@click.argument` per parameter. Group
|
|
||||||
dispatch is implicit — no `set_defaults(func=...)` and no
|
|
||||||
`cmd_<name>(args) -> int` signature. Top-level commands at
|
|
||||||
`cli.py:77,151,241,261,303`; the `backup` sub-group nests a
|
|
||||||
second `@main.group()` (`cli.py:303`) with its own
|
|
||||||
`@backup.command("<name>")` children.
|
|
||||||
2. **Long-form flags for safety.** Prefer `--rotate-key`,
|
|
||||||
`--backup-dir`, `--from-stdin` over positional args for anything
|
|
||||||
that mutates state or takes a secret. `init-passphrase`
|
|
||||||
(`cli.py:308-311`) demonstrates the canonical "flag OR stdin"
|
|
||||||
pattern: `--passphrase` for automation, `--from-stdin` for
|
|
||||||
interactive `getpass()` prompting.
|
|
||||||
3. **Exit codes: 0 / 1 / 2.** `0` = success. `1` = user / input
|
|
||||||
error (invalid NPI/EIN at `cli.py:258,277,285`; Keychain write
|
|
||||||
failure at `cli.py:341,351`; tampered-backup verify at
|
|
||||||
`cli.py:459`). `2` = operator / parse error
|
|
||||||
(`CycloneParseError` at `cli.py:106,178`; passphrase
|
|
||||||
empty/mismatch/short at `cli.py:331,337`). Document the codes
|
|
||||||
in the docstring (see `validate_npi_cmd` at `cli.py:244-250`).
|
|
||||||
Use `click.UsageError(...)` for usage mistakes; reserve
|
|
||||||
`sys.exit(2)` for "the file failed to parse" semantics.
|
|
||||||
4. **Smoke test with `click.testing.CliRunner`.** Every new
|
|
||||||
subcommand gets `backend/tests/test_cli_<name>.py` that
|
|
||||||
imports `from cyclone.cli import main` and invokes via
|
|
||||||
`CliRunner().invoke(main, [...], catch_exceptions=False)`. The
|
|
||||||
test must stub Keychain (`monkeypatch.setattr(secrets_mod,
|
|
||||||
"get_secret"/"set_secret", ...)`) and pin a temp SQLite DB via
|
|
||||||
`CYCLONE_DB_URL` + `db._reset_for_tests()` — see
|
|
||||||
`backend/tests/test_cli_backup.py:13-69` for the canonical
|
|
||||||
`_cli_env` fixture. CliRunner captures output and exit codes
|
|
||||||
in-process; do NOT shell out to `subprocess.run`.
|
|
||||||
5. **Destructive ops need `--yes` + `click.confirm(abort=True)`.**
|
|
||||||
`backup restore` (`cli.py:462-506`) and `backup prune`
|
|
||||||
(`cli.py:509-537`) both gate the destructive action behind a
|
|
||||||
`--yes` is_flag and an interactive `click.confirm(..., abort=True)`
|
|
||||||
prompt. CliRunner auto-aborts confirm prompts, so smoke tests
|
|
||||||
assert `exit_code != 0` when `--yes` is omitted
|
|
||||||
(`test_cli_backup.py:122-137`). A `--dry-run` flag is NOT yet
|
|
||||||
implemented anywhere; if needed, mirror the `--yes` pattern.
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
|
|
||||||
### A Click subcommand — `@main.command("<name>")` + options
|
|
||||||
|
|
||||||
From `cli.py:241-258` (smallest standalone subcommand):
|
|
||||||
|
|
||||||
```python
|
|
||||||
@main.command("validate-npi")
|
|
||||||
@click.argument("npi")
|
|
||||||
@click.option("--log-level", default="WARNING", show_default=True,
|
|
||||||
type=click.Choice(["DEBUG", "INFO", "WARNING", "ERROR"]))
|
|
||||||
def validate_npi_cmd(npi: str, log_level: str) -> None:
|
|
||||||
"""Validate a 10-digit NPI's Luhn checksum locally (SP20). Exit 0 valid, 1 invalid. PHI — don't log the value."""
|
|
||||||
setup_logging(level=log_level)
|
|
||||||
from cyclone.npi import is_valid_npi
|
|
||||||
if is_valid_npi(npi):
|
|
||||||
click.echo(f"OK: {len(npi)}-digit NPI passes Luhn checksum")
|
|
||||||
return
|
|
||||||
click.echo(f"INVALID: {npi!r} fails NPI Luhn checksum", err=True)
|
|
||||||
sys.exit(1)
|
|
||||||
```
|
|
||||||
|
|
||||||
### A smoke test — `CliRunner` + Keychain stub + temp SQLite
|
|
||||||
|
|
||||||
From `test_cli_backup.py:13-69`. Stable hex salt keeps multiple `CliRunner` invocations consistent within one test. The `Batch` seed at `cli_backup.py:27-35` is omitted — only the Keychain + DB plumbing is the convention:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@pytest.fixture
|
|
||||||
def _cli_env(tmp_path, monkeypatch):
|
|
||||||
from cyclone import db, secrets as secrets_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
|
|
||||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
|
||||||
db._reset_for_tests()
|
|
||||||
db.init_db()
|
|
||||||
# ... seed any DB rows the subcommand needs (see cli_backup.py:27-35) ...
|
|
||||||
|
|
||||||
store = {svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT: "cli-test-passphrase",
|
|
||||||
svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT:
|
|
||||||
"0123456789abcdef0123456789abcdef"}
|
|
||||||
monkeypatch.setattr(secrets_mod, "get_secret", lambda n: store.get(n))
|
|
||||||
monkeypatch.setattr(secrets_mod, "set_secret",
|
|
||||||
lambda n, v: store.__setitem__(n, v) or True)
|
|
||||||
monkeypatch.setenv("CYCLONE_BACKUP_DIR", str(tmp_path / "backups"))
|
|
||||||
yield tmp_path / "backups"
|
|
||||||
db._reset_for_tests()
|
|
||||||
|
|
||||||
def test_backup_create_list_verify_status(_cli_env):
|
|
||||||
from cyclone.cli import main
|
|
||||||
runner = CliRunner()
|
|
||||||
r = runner.invoke(main, ["backup", "create"], catch_exceptions=False)
|
|
||||||
assert r.exit_code == 0, r.output
|
|
||||||
assert "created backup id=" in r.output
|
|
||||||
```
|
|
||||||
|
|
||||||
### A destructive subcommand — `--yes` + `click.confirm(abort=True)`
|
|
||||||
|
|
||||||
From `cli.py:462-506` (`backup restore`). Two-step: announce, prompt unless `--yes`, then execute. Restore uses an explicit init/confirm round-trip so the operator can back out between phases. Matching smoke test asserts the guard fires:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@backup.command("restore")
|
|
||||||
@click.argument("backup_id", type=int)
|
|
||||||
@click.option("--yes", is_flag=True, help="Skip the interactive confirm prompt")
|
|
||||||
@click.option("--actor", default="operator-cli", show_default=True)
|
|
||||||
def backup_restore(backup_id: int, yes: bool, actor: str) -> None:
|
|
||||||
"""Restore the live DB from a backup (two-step, requires --yes)."""
|
|
||||||
# ... db.init_db() + service config omitted ...
|
|
||||||
click.echo(f"Initiating restore from backup {backup_id}...")
|
|
||||||
init = svc.restore_initiate(backup_id)
|
|
||||||
# ... echo init summary (filename, fp, table_count, ttl) ...
|
|
||||||
if not yes:
|
|
||||||
click.confirm(
|
|
||||||
"Replace the live DB with this backup? "
|
|
||||||
"This will dispose the engine and rebuild it.",
|
|
||||||
abort=True,
|
|
||||||
)
|
|
||||||
click.echo("Confirming restore...")
|
|
||||||
result = svc.restore_confirm(backup_id, init.restore_token, actor=actor)
|
|
||||||
|
|
||||||
def test_backup_restore_requires_yes_flag(_cli_env):
|
|
||||||
runner = CliRunner()
|
|
||||||
runner.invoke(main, ["backup", "create"], catch_exceptions=False)
|
|
||||||
r = runner.invoke(main, ["backup", "restore", "1"], catch_exceptions=False)
|
|
||||||
# CliRunner auto-aborts confirm prompts → exit_code != 0.
|
|
||||||
assert r.exit_code != 0
|
|
||||||
```
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
- **Don't `sys.exit(2)` for usage errors.** Reserve code 2 for
|
|
||||||
parse/operator errors (`CycloneParseError`, Keychain not
|
|
||||||
initialized, passphrase policy violation). For "you passed the
|
|
||||||
wrong flag" use `raise click.UsageError(...)` — Click formats
|
|
||||||
it as a clean help message and exits 2 on its own.
|
|
||||||
- **Don't print errors to stdout.** Use `click.echo(msg, err=True)`
|
|
||||||
for all error output. `print(..., file=sys.stderr)` and bare
|
|
||||||
`logging.error(...)` bypass Click's stdout/stderr split and leak
|
|
||||||
into test `result.output` — breaking `assert "FAIL" in r.output`
|
|
||||||
style assertions.
|
|
||||||
- **Don't add a subcommand without a smoke test.** Every new
|
|
||||||
`@main.command(...)` ships a sibling
|
|
||||||
`backend/tests/test_cli_<name>.py` exercising the happy path
|
|
||||||
AND at least one error path (missing input, invalid arg,
|
|
||||||
tampered ciphertext — see `test_cli_backup.py:102-119`).
|
|
||||||
- **Don't reuse the `parse` command name.** Existing subcommands
|
|
||||||
are type-specific (`parse-837`, `parse-835`); a generic `parse`
|
|
||||||
would shadow them or force an `--type` flag — neither is the
|
|
||||||
codebase pattern.
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
|
|
||||||
- **`cyclone-store`** — `backup` subcommands and the parse
|
|
||||||
subcommands both round-trip through `CycloneStore` and the DB
|
|
||||||
session; load when the increment changes write paths or the
|
|
||||||
`<entity>_written` event contract.
|
|
||||||
- **`cyclone-api-router`** — `cyclone serve` (via `__main__.py:19`)
|
|
||||||
launches the FastAPI app; the CLI parse subcommands share the
|
|
||||||
same `CycloneParseError` exception and Pydantic result models
|
|
||||||
as the matching HTTP endpoints.
|
|
||||||
- **`cyclone-edi`** — `parse-837` / `parse-835` are the CLI smoke
|
|
||||||
entry points for the parser/validator surface; load when adding
|
|
||||||
a parser or R-code rule.
|
|
||||||
- **`cyclone-tests`** — every CLI subcommand gets a pytest smoke
|
|
||||||
case under `backend/tests/test_cli_<name>.py`; the `_cli_env`
|
|
||||||
fixture pattern (fresh SQLite + Keychain stub) is documented
|
|
||||||
there.
|
|
||||||
- **`cyclone-spec`** — load when the SP-N spec introduces a new operator command or reserves a new exit-code category.
|
|
||||||
@@ -1,193 +0,0 @@
|
|||||||
---
|
|
||||||
name: cyclone-edi
|
|
||||||
description: "Cyclone EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1). Use when: adding or changing a parser, adding a validator rule (R010/R020/R100/R200-R210/R835_*/NPI Luhn/EIN/CAS), or mapping a new CAS adjustment reason code."
|
|
||||||
---
|
|
||||||
|
|
||||||
# cyclone-edi
|
|
||||||
|
|
||||||
Cyclone parses seven X12 EDI transaction types (837P, 835, 999, 270, 271, 277CA, TA1) into typed Pydantic models, then runs per-claim / per-batch validator rules that surface as R-coded `ValidationIssue` records. This skill codifies the conventions so new parsers and rules stay consistent with the seven that already exist.
|
|
||||||
|
|
||||||
As of this writing: **7 parser modules** under `backend/src/cyclone/parsers/parse_<edi>.py`, **~25 per-claim rules** numbered `R010`–`R100` and `R200`–`R210` in `validator.py`, plus a parallel set of **835-specific rules** prefixed `R835_*` in `validator_835.py`. The next increment is **SP22**.
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- **Adding or changing a parser.** You are about to touch `backend/src/cyclone/parsers/parse_<edi>.py` or its paired `models_<edi>.py` and need the orchestrator signature, the segment walker convention, and the re-export in `parsers/__init__.py`.
|
|
||||||
- **Adding a validator rule.** You are writing a new `_rule_R<n>_<name>` (or `_r<n>_<name>` per the existing snake-case style) and need the rule signature, the R-code numbering scheme, and the `ValidationIssue` shape.
|
|
||||||
- **Wiring a new CAS / CARC code.** The 835 carries Claim Adjustment Reason Codes in `CAS` segments; the lookup lives in `backend/src/cyclone/parsers/cas_codes.py` and the UI reads through `claim_status_label()`.
|
|
||||||
- **Debugging a parse failure on a prodfiles sample.** You dropped a real EDI file into `docs/prodfiles/<source>/` and the parser is choking — load this skill to confirm the tokenizer path, the orchestrator entry point, and which fixture in `backend/tests/fixtures/` matches the transaction type.
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
1. **Parser signature.** Every parser module exports exactly one public entry function. Two flavors coexist in the codebase:
|
|
||||||
- `parse(text: str, *, input_file: str = "") -> <TypedResult>` — used by `parse_270.py:337`, `parse_271.py:356`.
|
|
||||||
- `parse(text: str, payer_config: <PayerConfig>, input_file: str = "") -> <TypedResult>` — used by `parse_837.py:319` and `parse_835.py:459` because both need payer-specific config to validate segments against.
|
|
||||||
- `parse_<edi>_text(text: str, *, input_file: str = "") -> <TypedResult>` — the legacy name-suffixed form, still in use at `parse_ta1.py:143`, `parse_999.py:220`, `parse_277ca.py:280`. The `<TypedResult>` is always a Pydantic model from `models_<edi>.py` (or co-located `models.py` for 837P).
|
|
||||||
2. **Segment walk.** Parsers consume `backend/src/cyclone/parsers/segments.py` — there are exactly three public pieces: `Delimiters` (frozen dataclass holding the four ISA-derived separators), `_detect_delimiters(isa_segment)` (private), and `tokenize(text) -> list[list[str]]` (returns ISA prepended as the first segment). Parsers then index into the `list[list[str]]` directly — there is **no** `Segment` / `Loop` / `next_segment` helper class. Whole-document problems (missing ISA, wrong transaction set) raise `CycloneParseError`; per-segment problems on acks (999/277CA) are surfaced on the result, not raised.
|
|
||||||
3. **Validator rules.** Numbered rules live in `backend/src/cyclone/parsers/validator.py` (for 837P — R010–R100 general + R200–R210 SP9 CO MAP / HCPF naming) and `backend/src/cyclone/parsers/validator_835.py` (for 835 — names prefixed `R835_*` because the same numeric space would collide with 837P). Each rule is a function `_r<n>_<name>(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]` registered in the module-level `_RULES` list and run by `validate(claim, config)`. Issues carry the rule name as a stable string (`rule="R021_npi_checksum"`) — the R-code **is** how the UI surfaces the error, so never invent an unnumbered rule.
|
|
||||||
4. **NPI / EIN / CAS format logic.** Identity-format checks live in their own modules — never duplicate them in a parser or validator:
|
|
||||||
- `backend/src/cyclone/npi.py` — `is_valid_npi(npi)` runs the Luhn checksum with the `80840` NPPES prefix; `is_valid_tax_id(ein)` enforces `XX-XXXXXXX` (or 9 raw digits).
|
|
||||||
- `backend/src/cyclone/parsers/cas_codes.py` — `reason_label(group, reason)` and `all_known_codes()` for the CARC lookup; snapshot date is exported as `LAST_UPDATED`.
|
|
||||||
- `backend/src/cyclone/parsers/models_271.py` — `SERVICE_TYPE_CODES` + `service_type_description()` for 271 EB benefit codes.
|
|
||||||
5. **Prodfiles reuse.** When adding a parser for a new transaction type, ship at least one fixture in `backend/tests/fixtures/<edi>/<sample>.txt` (the existing 13 fixtures are **flat** at the top level of `fixtures/` — no per-test subdirectories). Copy from `docs/prodfiles/<source>/<file>.txt`; never reach into `docs/prodfiles/` from a test. The matching test should declare the path as a module-level `Path` constant.
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
|
|
||||||
### Minimal `parse_<edi>.py` — using `segments.py`, exporting `parse_ta1_text`
|
|
||||||
|
|
||||||
Taken from `backend/src/cyclone/parsers/parse_ta1.py:1-29` (the smallest parser — TA1 is just ISA + TA1 + IEA). The same skeleton scales to every other EDI type by adding `_consume_<segment>` helpers.
|
|
||||||
|
|
||||||
```python
|
|
||||||
"""Parse an X12 TA1 (Interchange Acknowledgment) file.
|
|
||||||
|
|
||||||
Whole-document problems (missing ISA, no TA1) raise CycloneParseError.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import logging
|
|
||||||
from datetime import date
|
|
||||||
|
|
||||||
from cyclone.parsers.exceptions import CycloneParseError
|
|
||||||
from cyclone.parsers.models import BatchSummary, Envelope
|
|
||||||
from cyclone.parsers.models_ta1 import ParseResultTa1, Ta1Ack
|
|
||||||
from cyclone.parsers.segments import tokenize
|
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_yyyymmdd(s: str) -> date | None:
|
|
||||||
"""Parse an 8-digit CCYYMMDD string. Returns None on bad input."""
|
|
||||||
...
|
|
||||||
|
|
||||||
|
|
||||||
def _build_envelope(segments: list[list[str]], input_file: str) -> Envelope:
|
|
||||||
"""Build the envelope from ISA. TA1 has no GS/ST — just ISA → TA1 → IEA."""
|
|
||||||
...
|
|
||||||
|
|
||||||
|
|
||||||
def _consume_ta1(segments: list[list[str]], idx: int) -> tuple[Ta1Ack, int]:
|
|
||||||
"""Read a TA1 segment and return a Ta1Ack. Returns (model, next_idx)."""
|
|
||||||
...
|
|
||||||
|
|
||||||
|
|
||||||
def parse_ta1_text(text: str, *, input_file: str = "") -> ParseResultTa1:
|
|
||||||
"""Parse a complete TA1 document and return a ParseResultTa1."""
|
|
||||||
segments = tokenize(text)
|
|
||||||
envelope = _build_envelope(segments, input_file=input_file)
|
|
||||||
ta1_idx = next(
|
|
||||||
(i for i, seg in enumerate(segments) if seg[0] == "TA1"), None,
|
|
||||||
)
|
|
||||||
if ta1_idx is None:
|
|
||||||
raise CycloneParseError("No TA1 segment found")
|
|
||||||
ta1, _ = _consume_ta1(segments, ta1_idx)
|
|
||||||
...
|
|
||||||
return ParseResultTa1(envelope=envelope, ta1=ta1, summary=summary, ...)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["parse_ta1_text"]
|
|
||||||
```
|
|
||||||
|
|
||||||
The orchestrator pattern is the same in every parser: `tokenize` → `_build_envelope` → segment consumers in order → wrap into a `ParseResult<EDI>` model. The Pydantic result is what the API / store layer consumes.
|
|
||||||
|
|
||||||
### A validator rule — `_r<n>_<name>` registered in `_RULES`
|
|
||||||
|
|
||||||
Taken from `backend/src/cyclone/parsers/validator.py:23-66` (the canonical R010–R100 block).
|
|
||||||
|
|
||||||
```python
|
|
||||||
from collections.abc import Iterable
|
|
||||||
from cyclone.parsers.models import ClaimOutput, ValidationIssue
|
|
||||||
from cyclone.parsers.payer import PayerConfig
|
|
||||||
|
|
||||||
NPI_RE = re.compile(r"^\d{10}$")
|
|
||||||
Rule = Callable[[ClaimOutput, PayerConfig], Iterable[ValidationIssue]]
|
|
||||||
|
|
||||||
|
|
||||||
def _r020_npi_format(claim: ClaimOutput, _: PayerConfig) -> Iterable[ValidationIssue]:
|
|
||||||
if claim.billing_provider.npi and not NPI_RE.match(claim.billing_provider.npi):
|
|
||||||
yield ValidationIssue(
|
|
||||||
rule="R020_npi_format",
|
|
||||||
severity="error",
|
|
||||||
message=f"Billing provider NPI must be 10 digits, got {claim.billing_provider.npi!r}",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _r021_npi_checksum(claim: ClaimOutput, _: PayerConfig) -> Iterable[ValidationIssue]:
|
|
||||||
"""SP20: validate the billing-provider NPI's Luhn check digit."""
|
|
||||||
npi = claim.billing_provider.npi
|
|
||||||
if not npi or not NPI_RE.match(npi):
|
|
||||||
return # R020 already flagged the format — skip silently.
|
|
||||||
try:
|
|
||||||
from cyclone.npi import is_valid_npi
|
|
||||||
except ImportError:
|
|
||||||
return
|
|
||||||
if not is_valid_npi(npi):
|
|
||||||
yield ValidationIssue(
|
|
||||||
rule="R021_npi_checksum",
|
|
||||||
severity="warning",
|
|
||||||
message=f"Billing provider NPI {npi!r} fails Luhn checksum (likely typo)",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
_RULES: list[Rule] = [
|
|
||||||
_r010_clm01_present,
|
|
||||||
_r011_total_charge_positive,
|
|
||||||
_r020_npi_format,
|
|
||||||
_r021_npi_checksum,
|
|
||||||
# ... R030, R031, R032-R035, R050, R060, R070, R100, R200-R210
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
For 835 rules, prefix the rule string with `R835_` (e.g. `R835_BPR01_handling_code_allowed`) and target the `ParseResult835` model instead of `ClaimOutput` — see `backend/src/cyclone/parsers/validator_835.py:38-79`.
|
|
||||||
|
|
||||||
### A test that uses a prodfiles fixture
|
|
||||||
|
|
||||||
Taken from `backend/tests/test_api_999.py:53-72`. The autouse `conftest.py` already provides a per-test SQLite DB; most tests just add a `client` fixture and reference the fixture path as a module-level constant.
|
|
||||||
|
|
||||||
```python
|
|
||||||
"""Tests for the FastAPI surface in cyclone.api for the 999 endpoint."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
from cyclone.api import app
|
|
||||||
|
|
||||||
# Fixture reference — flat, module-level Path constant. NEVER reach into
|
|
||||||
# docs/prodfiles/ from a test; the fixtures/ dir is the test-consumed surface.
|
|
||||||
ACCEPTED = Path(__file__).parent / "fixtures" / "minimal_999.txt"
|
|
||||||
REJECTED = Path(__file__).parent / "fixtures" / "minimal_999_rejected.txt"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def client() -> TestClient:
|
|
||||||
return TestClient(app)
|
|
||||||
|
|
||||||
|
|
||||||
def test_parse_999_endpoint_happy_path(client: TestClient):
|
|
||||||
text = ACCEPTED.read_text()
|
|
||||||
resp = client.post(
|
|
||||||
"/api/parse-999",
|
|
||||||
files={"file": ("minimal_999.txt", text, "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
body = resp.json()
|
|
||||||
assert body["ack"]["ack_code"] == "A"
|
|
||||||
```
|
|
||||||
|
|
||||||
For pure-unit parser tests (no API), the same path is reused — see `backend/tests/test_parse_837.py:8` (`FIXTURE = Path(__file__).parent / "fixtures" / "minimal_837p.txt"`).
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
- **Don't re-parse raw X12 strings inside validators.** Always parse first into the typed `ParseResult<EDI>` / `ClaimOutput`, then validate against that. Validators index into the model fields (or `claim.raw_segments` for spot-checks of specific segment presence) — they never call `tokenize` again. R034's `REF*G1` presence check (`validator.py:104-107`) is the only place that legitimately touches `raw_segments`, and it does so to confirm a single segment exists.
|
|
||||||
- **Don't bake payer-specific logic into the generic parser.** Payer variations live in `backend/src/cyclone/parsers/payer.py` (`PayerConfig`, `PayerConfig835`) and `backend/src/cyclone/payers.py` (the YAML loader from `config/payers.yaml`). Parsers accept the config as an argument; rules read it from the `cfg` parameter. A new payer never requires a new parser file — extend the config and add / adjust an R-code rule.
|
|
||||||
- **Don't add a validator rule without an R-code.** The `rule="R<n>_<name>"` string is the stable identifier the UI greys out, the API returns in `errors[].rule`, and tests assert against. Inventing a rule without an R-code (or reusing an R-code with new semantics) breaks the operator workflow. New SP-N increments reserve their R-code range up front (SP9 reserved R200–R210, SP20 added R021) and document it in the spec.
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
|
|
||||||
- **`cyclone-store`** — load when the increment changes how a parsed `ClaimOutput` / `ParseResult<EDI>` is persisted (`store.py` write path, `<entity>_written` events).
|
|
||||||
- **`cyclone-api-router`** — load when the increment adds or changes an HTTP endpoint that surfaces a parsed result (e.g. `/api/parse-999`, `/api/parse-837`, `/api/parse-835`).
|
|
||||||
- **`cyclone-tests`** — every parser addition ships a fixture in `backend/tests/fixtures/` and a pytest case; load this skill for the fixture-drop-in and autouse-conftest rules.
|
|
||||||
- **`cyclone-cli`** — load when the increment adds a CLI subcommand. The `cyclone parse-837 <file>` and `cyclone parse-835 <file>` smoke commands at `backend/src/cyclone/cli.py:77,151` are the parser-level smoke tests; the `validate-npi` and `validate-tax-id` commands exercise the format helpers.
|
|
||||||
- **`cyclone-spec`** — load when the SP-N spec for the increment introduces a new R-code range or a new transaction type; the spec's `## Decisions` section is where the R-code reservation gets locked in.
|
|
||||||
@@ -1,40 +0,0 @@
|
|||||||
# Cyclone EDI parsers — flat catalog
|
|
||||||
|
|
||||||
Every parser module under `backend/src/cyclone/parsers/` (one row per
|
|
||||||
file), its transaction type, its public entry signature, its result
|
|
||||||
model, its primary fixture, and any payer-specific variant. The
|
|
||||||
companion Pydantic model is in a co-located `models_<edi>.py`; the
|
|
||||||
segment walker uses `tokenize()` from `segments.py` and never parses
|
|
||||||
raw text inline.
|
|
||||||
|
|
||||||
| Module | EDI type | Public entry signature | Result model | Primary fixture(s) | Payer variant |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| `parse_837.py` | 837P (Professional Claim) | `parse(text, payer_config: PayerConfig, input_file="") -> ParseResult` | `cyclone.parsers.models.ParseResult` | `minimal_837p.txt`, `co_medicaid_837p.txt` | `PayerConfig` (CO Medicaid default) |
|
|
||||||
| `parse_835.py` | 835 (ERA / Remittance) | `parse(text, payer_config: PayerConfig835, input_file="") -> ParseResult835` | `cyclone.parsers.models_835.ParseResult835` | `minimal_835.txt`, `co_medicaid_835.txt`, `unbalanced_835.txt` | `PayerConfig835` |
|
|
||||||
| `parse_999.py` | 999 (Implementation ACK) | `parse_999_text(text, *, input_file="") -> ParseResult999` | `cyclone.parsers.models_999.ParseResult999` | `minimal_999.txt`, `minimal_999_rejected.txt` | none — single-shape ack |
|
|
||||||
| `parse_277ca.py` | 277CA (Claim ACK) | `parse_277ca_text(text, *, input_file="") -> ParseResult277CA` | `cyclone.parsers.models_277ca.ParseResult277CA` | `minimal_277ca.txt`, `minimal_277ca_rejected_only.txt`, `minimal_277ca_st277.txt` | `PayerConfig277CA` (config-driven) |
|
|
||||||
| `parse_270.py` | 270 (Eligibility Inquiry) | `parse(text, *, input_file="") -> ParseResult270` | `cyclone.parsers.models_270.ParseResult270` | `minimal_270.txt` | reuses `PayerConfig` shape |
|
|
||||||
| `parse_271.py` | 271 (Eligibility Response) | `parse(text, *, input_file="") -> ParseResult271` | `cyclone.parsers.models_271.ParseResult271` | `minimal_271.txt` | reuses `PayerConfig` shape |
|
|
||||||
| `parse_ta1.py` | TA1 (Interchange ACK) | `parse_ta1_text(text, *, input_file="") -> ParseResultTa1` | `cyclone.parsers.models_ta1.ParseResultTa1` | `minimal_ta1.txt` | none — single-shape ack |
|
|
||||||
|
|
||||||
Companion modules (not parsers, but shipped alongside):
|
|
||||||
|
|
||||||
| Module | Role |
|
|
||||||
|---|---|
|
|
||||||
| `segments.py` | `Delimiters`, `_detect_delimiters`, `tokenize(text) -> list[list[str]]` |
|
|
||||||
| `models.py` | Pydantic models for 837P (`ParseResult`, `ClaimOutput`, `Envelope`, `BatchSummary`, `ValidationIssue`, `ValidationReport`, …) |
|
|
||||||
| `models_835.py` / `models_270.py` / `models_271.py` / `models_277ca.py` / `models_999.py` / `models_ta1.py` | Pydantic models for each transaction type |
|
|
||||||
| `payer.py` | `PayerConfig` + `PayerConfig835` factories |
|
|
||||||
| `exceptions.py` | `CycloneParseError`, `CycloneValidationError` |
|
|
||||||
| `cas_codes.py` | CARC lookup: `reason_label(group, reason)`, `all_known_codes()`, `LAST_UPDATED` |
|
|
||||||
| `validator.py` | 837P rules: R010–R100, R200–R210; `validate(claim, config) -> ValidationReport` |
|
|
||||||
| `validator_835.py` | 835 rules: `R835_*` (e.g. `R835_BPR01_handling_code_allowed`); `validate(result, cfg) -> ValidationReport` |
|
|
||||||
| `serialize_270.py` / `serialize_837.py` / `serialize_999.py` | Outbound (Cyclone → payer) serializers — mirror of `parse_*` |
|
|
||||||
| `writer.py` / `writer_835.py` | Output writers (one JSON per claim) |
|
|
||||||
| `batch_ack_builder.py` | `build_ack_for_batch` — produces a 999 for a parsed 837 batch |
|
|
||||||
| `__init__.py` | Lazy PEP 562 re-exports (`parse`, `parse_835`, `parse_999`, `parse_270`, `parse_271`, `parse_277ca`, plus all models) |
|
|
||||||
|
|
||||||
Fixture rule: every parser ships at least one flat fixture in
|
|
||||||
`backend/tests/fixtures/<edi>-sample.txt`. Prodfiles sources live in
|
|
||||||
`docs/prodfiles/{837p-from-axiscare,835fromco,FromHPE,claims}/` — copy
|
|
||||||
from there, never reach in from a test.
|
|
||||||
@@ -1,138 +0,0 @@
|
|||||||
---
|
|
||||||
name: cyclone-frontend-page
|
|
||||||
description: "Cyclone React page conventions (TanStack Query, use<X> hook, drawer, URL state, .test.tsx sibling, Layout / PageHeader / Sidebar). Use when: adding a new page, refactoring an existing one, or wiring a drawer into a page."
|
|
||||||
---
|
|
||||||
|
|
||||||
# cyclone-frontend-page
|
|
||||||
|
|
||||||
Cyclone pages live at `src/pages/<Name>.tsx`: a `use<X>` hook in `src/hooks/use<X>.ts` does the fetching (and optionally the live-tail subscription), `<Layout>` + `<PageHeader>` + `<Sidebar>` (`src/components/`) provide the app shell, and any right-side detail (claim, remittance) lives in `src/components/<DrawerName>/` whose open/close state is mirrored to the URL via `useDrawerUrlState`. This skill codifies the conventions so additions stay consistent with the eleven pages already shipped.
|
|
||||||
|
|
||||||
As of this writing: **11 pages** under `src/pages/` (9 of 11 with a `*.test.tsx` sibling — Dashboard, Upload, and BatchDiff are not yet covered; be the first when you refactor them), **~30 hooks** under `src/hooks/`, and **2 drawer modules** at `src/components/{ClaimDrawer,RemitDrawer}/`. The next increment is **SP22**.
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- **Adding a new page.** Mounting a new screen in `src/pages/` — you need the Layout + PageHeader + table shape, the `use<X>` hook split, and the route registration point in `src/App.tsx`.
|
|
||||||
- **Refactoring an existing page.** Splitting a 600-line page, swapping a manual `fetch` for a hook, or moving in-component state into the URL — load this skill to confirm the destination shape.
|
|
||||||
- **Wiring a drawer.** Adding a new right-side detail drawer (e.g. `BatchDrawer`, `ActivityDrawer`) — you need `useDrawerUrlState` for URL-driven open/close, the `src/components/<DrawerName>/` folder layout, and the deep-link contract so `?claim=…` / `?remit=…` round-trips.
|
|
||||||
- **Sharing state via URL.** Persisting filter / page / drawer state across reloads — confirm the `useDrawerUrlState` / `useRemitDrawerUrlState` hook pair is the right tool before reaching for `useState` + history.
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
1. **Page shape.** Every page in `src/pages/<Name>.tsx` exports a `function <Name>()` (most pages use a named export — see `src/pages/Claims.tsx:51`, `Remittances.tsx:62`, `Dashboard.tsx:68`; `src/pages/Inbox.tsx:40` is the lone default export). The app shell is provided by the `<Layout>` route wrapper in `src/App.tsx:30`. Each page sets a `<PageHeader>` (`src/components/PageHeader.tsx:18`) and renders a table, list, or KPI grid. Sidebar nav is mounted once by `<Layout>` at `src/components/Sidebar.tsx`.
|
|
||||||
2. **Data hook.** Each page pairs with a `use<X>` hook in `src/hooks/use<X>.ts` (e.g. `Claims` ↔ `useClaims`, `Remittances` ↔ `useRemittances`, `Acks` ↔ `useAcks`). The hook returns `{ data, isLoading, isError, error, refetch }` from TanStack Query's `useQuery` (see `src/hooks/useClaims.ts:24-31`, `useRemittances.ts:21-25`). Pages never call `fetch` or `@/lib/api` directly — the hook is the boundary so the page is testable with `vi.mock("@/lib/api", ...)`.
|
|
||||||
3. **Live tail.** Pages with live data compose three hooks in order: the `use<X>` initial fetch, `useTailStream(resource)` (`src/hooks/useTailStream.ts:80` — opens the NDJSON stream, drives the backoff/stall state machine), and `useMergedTail(resource, baseItems, filterFn?)` (`src/hooks/useMergedTail.ts:25` — merges snapshot + tail, dedup'd by id). The full wiring lives at `src/pages/Claims.tsx:85-89` and `Remittances.tsx:81-83`. The streaming subscription belongs on the page, not in the data hook — hoisting it couples the lifecycle to whoever mounts `use<X>`.
|
|
||||||
4. **Drawer.** Right-side detail drawers live in `src/components/<DrawerName>/` (currently `ClaimDrawer/`, `RemitDrawer/`) with a barrel `index.ts` (`src/components/ClaimDrawer/index.ts:1-14`). The drawer is wired to URL state via `useDrawerUrlState()` for `?claim=…` (`src/hooks/useDrawerUrlState.ts`) or `useRemitDrawerUrlState()` for `?remit=…` (`src/hooks/useRemitDrawerUrlState.ts`). Open state is driven by the URL, not a `useState` flag, so deep-links round-trip.
|
|
||||||
5. **Tests.** Every page gets a `src/pages/<Name>.test.tsx` sibling (e.g. `Claims.test.tsx`, `Remittances.test.tsx`, `Batches.test.tsx`). Every hook gets a `src/hooks/use<X>.test.ts` sibling (e.g. `useClaims.test.ts`, `useRemittances.test.ts`). The shared setup — `// @vitest-environment happy-dom`, `IS_REACT_ACT_ENVIRONMENT = true`, `QueryClient` provider, `vi.mock("@/lib/api", ...)` — is documented in `cyclone-tests`; mirror `src/pages/Claims.test.tsx:1-30` for the canonical page-test shape.
|
|
||||||
6. **UI primitives.** Use Radix-backed components from `src/components/ui/` (`button.tsx`, `dialog.tsx`, `table.tsx`, `select.tsx`, `pagination.tsx`, `empty-state.tsx`, `error-state.tsx`, `filter-chips.tsx`, `skeleton.tsx`, `input.tsx`, `label.tsx`, `card.tsx`, `badge.tsx`, `skip-link.tsx`, `claim-state-badge.tsx`). Don't pull in a new UI library without discussion — every primitive here is already consumed by at least one shipped page.
|
|
||||||
7. **Routing.** Pages register their route in `src/App.tsx` as a `<Route path="<name>" element={<<Name>> />} />` inside the `<Layout>` element wrapper (`src/App.tsx:30-44`). Currently every page is a static import; switch to `React.lazy(() => import(...))` only if a page grows heavy (large parse/EDI libs, chart code) and the import cost shows up in the bundle report.
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
|
|
||||||
### `Claims.tsx`-style page (Layout + PageHeader + table + drawer)
|
|
||||||
|
|
||||||
Canonical page shape — composes the data hook, the tail triplet, and
|
|
||||||
`useDrawerUrlState` for the `?claim=…` deep-link. See
|
|
||||||
`src/pages/Claims.tsx:51-200` for the full file.
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
import { useMemo, useState } from "react";
|
|
||||||
import { Table, TableBody, TableRow, /* … */ } from "@/components/ui/table";
|
|
||||||
import { PageHeader } from "@/components/PageHeader";
|
|
||||||
import { ClaimDrawer } from "@/components/ClaimDrawer";
|
|
||||||
import { useClaims } from "@/hooks/useClaims";
|
|
||||||
import { useDrawerUrlState } from "@/hooks/useDrawerUrlState";
|
|
||||||
import { useTailStream } from "@/hooks/useTailStream";
|
|
||||||
import { useMergedTail } from "@/hooks/useMergedTail";
|
|
||||||
import { TailStatusPill } from "@/components/TailStatusPill";
|
|
||||||
|
|
||||||
export function Claims() {
|
|
||||||
const [status, setStatus] = useState<ClaimStatus | null>(null);
|
|
||||||
const params = useMemo(() => ({ status, limit: 25, offset: 0 }), [status]);
|
|
||||||
|
|
||||||
const { data } = useClaims(params);
|
|
||||||
const { status: tailStatus, lastEventAt, forceReconnect } = useTailStream("claims");
|
|
||||||
const items = useMergedTail("claims", data?.items ?? [], (c) => !status || c.status === status);
|
|
||||||
const { claimId, openClaim, closeClaim } = useDrawerUrlState();
|
|
||||||
|
|
||||||
return (
|
|
||||||
<>
|
|
||||||
<PageHeader
|
|
||||||
eyebrow="Inbox"
|
|
||||||
title="Claims"
|
|
||||||
status={<TailStatusPill status={tailStatus} lastEventAt={lastEventAt} onReconnect={forceReconnect} />}
|
|
||||||
/>
|
|
||||||
<Table>
|
|
||||||
<TableBody>
|
|
||||||
{items.map((c) => (
|
|
||||||
<TableRow key={c.id} onClick={() => openClaim(c.id)}>{/* …cells… */}</TableRow>
|
|
||||||
))}
|
|
||||||
</TableBody>
|
|
||||||
</Table>
|
|
||||||
<ClaimDrawer claimId={claimId} claims={items} onClose={closeClaim} onNavigate={openClaim} />
|
|
||||||
</>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### `use<X>` data hook (TanStack Query)
|
|
||||||
|
|
||||||
Pattern from `src/hooks/useClaims.ts:24-67` and `useRemittances.ts:21-65`.
|
|
||||||
Returns a stable shape so the page treats all data hooks uniformly.
|
|
||||||
The tail subscription lives on the page (see Convention 3), not here.
|
|
||||||
|
|
||||||
```ts
|
|
||||||
import { useQuery } from "@tanstack/react-query";
|
|
||||||
import { api, type ListClaimsParams, type PaginatedResponse } from "@/lib/api";
|
|
||||||
import type { Claim } from "@/types";
|
|
||||||
|
|
||||||
export function useClaims(params: ListClaimsParams) {
|
|
||||||
return useQuery<PaginatedResponse<Claim>>({
|
|
||||||
queryKey: ["claims", params],
|
|
||||||
queryFn: () => api.listClaims<Claim>(params),
|
|
||||||
enabled: api.isConfigured,
|
|
||||||
// …in-memory fallback when !api.isConfigured…
|
|
||||||
});
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Drawer component with `useDrawerUrlState`
|
|
||||||
|
|
||||||
Pattern from `src/components/ClaimDrawer/ClaimDrawer.tsx:1-50` +
|
|
||||||
`src/hooks/useDrawerUrlState.ts`. The drawer takes `claimId` as a
|
|
||||||
prop (driven by the URL), renders nothing when `null`, and uses
|
|
||||||
`useDrawerKeyboard` for j/k navigation + Escape to close. The page
|
|
||||||
that mounts it controls the URL — `openClaim("abc")` sets `?claim=abc`,
|
|
||||||
removing the param closes the drawer, and reload preserves the state.
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
import { Dialog, DialogContent } from "@/components/ui/dialog";
|
|
||||||
import { useClaimDetail } from "@/hooks/useClaimDetail";
|
|
||||||
import { useDrawerKeyboard } from "@/hooks/useDrawerKeyboard";
|
|
||||||
|
|
||||||
export function ClaimDrawer({ claimId, claims, onClose, onNavigate, onToggleHelp }) {
|
|
||||||
const open = claimId !== null;
|
|
||||||
const { data, isLoading, isError, error } = useClaimDetail(claimId);
|
|
||||||
useDrawerKeyboard({ open, claims, onClose, onNavigate, onToggleHelp });
|
|
||||||
if (!open) return null;
|
|
||||||
return (
|
|
||||||
<Dialog open onOpenChange={(o) => !o && onClose()}>
|
|
||||||
<DialogContent>{/* header + body panels from useClaimDetail… */}</DialogContent>
|
|
||||||
</Dialog>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
- **Don't `fetch` from inside a page component.** All API access goes through the `use<X>` hook in `src/hooks/use<X>.ts`. Pages that reach for `fetch(...)` directly can't be tested with `vi.mock("@/lib/api", ...)` and split the data lifecycle across files. The hook returns `{ data, isLoading, isError, error, refetch }` so the page is a pure renderer.
|
|
||||||
- **Don't open a drawer via local component state.** A `const [open, setOpen] = useState(false)` for a drawer breaks deep-links and reload-restore. Use `useDrawerUrlState()` (claim) or `useRemitDrawerUrlState()` (remit) so the URL is the single source of truth.
|
|
||||||
- **Don't put domain logic in JSX.** Conditional renderings, table sorting, and KPI math all belong in the `use<X>` hook or a pure helper under `src/lib/` (e.g. `src/lib/format.ts` for currency / date formatting). JSX is for layout; mixing in `items.filter(...).sort(...)` inline is hard to test and hides behavior from the hook.
|
|
||||||
- **Don't call `useTailStream` from inside a `use<X>` hook.** The streaming subscription belongs on the page (see `src/pages/Claims.tsx:85-89`). Hoisting it into `useClaims` couples the open/close lifecycle of the page to whoever mounts the hook, and breaks the one-resource-one-page ownership that the backoff/stall state machine assumes.
|
|
||||||
- **Don't import a new UI library to render a button, modal, or table.** Radix-backed primitives in `src/components/ui/` already cover every widget the shipped pages use. Reach for a new library only after the primitive gap is real and the proposal is in a spec / PR.
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
|
|
||||||
- **`cyclone-tail`** — load for any live-data page (Claims, Remittances, ActivityLog). Documents the `useTailStream` + `useMergedTail` triplet, the `<TailStatusPill>` wiring, and the 30s stall threshold (`STALL_TIMEOUT_MS = 30_000` at `useTailStream.ts:53`).
|
|
||||||
- **`cyclone-api-router`** — load when a page is calling a new HTTP endpoint. Documents `api_routers/<topic>.py` vs. inline `api.py` registration and the response / error-envelope shapes.
|
|
||||||
- **`cyclone-tests`** — load when adding the `*.test.tsx` sibling. Documents the `// @vitest-environment happy-dom` setup, `IS_REACT_ACT_ENVIRONMENT = true`, the `@testing-library/react` vs. `createRoot`+`Probe` rendering styles, and the `vi.mock("@/lib/api")` convention.
|
|
||||||
- **`cyclone-edi`** — load when a page renders parsed 837P / 835 / 999 / 270 / 271 / 277CA / TA1 content (ServiceLinesTable, CAS panels, ValidationPanel). Documents the parser modules, the R-coded validator rules, and the CAS / CARC / NPI / EIN helpers.
|
|
||||||
@@ -1,157 +0,0 @@
|
|||||||
---
|
|
||||||
name: cyclone-spec
|
|
||||||
description: "Cyclone SP-N superpowers increment flow — spec → plan → implement → merge. Use when: starting a new numbered feature increment, naming a branch, opening a SP-N PR, or doing the merge dance into main."
|
|
||||||
---
|
|
||||||
|
|
||||||
# cyclone-spec
|
|
||||||
|
|
||||||
The Cyclone repo ships every new feature as a numbered **SP-N increment**:
|
|
||||||
a spec, a plan, an implementation branch, and a single atomic merge commit
|
|
||||||
into `main`. This skill encodes the conventions so every increment follows
|
|
||||||
the same shape and the commit history stays auditable.
|
|
||||||
|
|
||||||
As of this writing: **16 specs** in `docs/superpowers/specs/`, **12 plans**
|
|
||||||
in `docs/superpowers/plans/`, and SP numbers used through **SP21** (the
|
|
||||||
universal-drilldown design in progress). The next increment is **SP22**.
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- **Starting a new feature increment.** You are about to add a numbered
|
|
||||||
feature, fix that crosses subsystem boundaries, or anything bigger than
|
|
||||||
a one-line change. Before you write code, reserve the next SP number and
|
|
||||||
write the spec.
|
|
||||||
- **Naming the spec / plan files or the branch.** You have a topic, a
|
|
||||||
date, and a number — and you need the exact path / branch shape so
|
|
||||||
existing scripts and reviewers can find the artifacts.
|
|
||||||
- **Opening the SP-N PR.** You are about to push the branch and need the
|
|
||||||
PR title format and the commit-prefix conventions so the merge commit
|
|
||||||
reads cleanly.
|
|
||||||
- **Doing the merge dance.** Review is approved and you're about to land
|
|
||||||
the branch into `main`. Use this skill to confirm the merge shape — no
|
|
||||||
squash, no rebase, one atomic merge commit.
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
1. **Numbering.** Reserve the next SP-N number — the next integer after
|
|
||||||
the highest `SP<n>` already used in `git log`. Never reuse a number,
|
|
||||||
even after deletion. Numbering is monotonic and lives in the merge
|
|
||||||
history.
|
|
||||||
2. **Branch.** `sp<N>-<short-kebab-topic>` — e.g. `sp22-line-reconciliation`,
|
|
||||||
`sp9-multi-payer-npi`. Kebab-case, lowercase, no spaces, no slashes.
|
|
||||||
The branch name is the canonical handle for the increment.
|
|
||||||
3. **Spec path.** `docs/superpowers/specs/YYYY-MM-DD-cyclone-<topic>-design.md`
|
|
||||||
with header `Status: Draft, pending user review`. One spec per
|
|
||||||
increment. Real examples: `2026-06-19-cyclone-db-reconciliation-design.md`
|
|
||||||
(SP3), `2026-06-20-cyclone-multi-payer-npi-sftp-design.md` (SP9).
|
|
||||||
4. **Plan path.** `docs/superpowers/plans/YYYY-MM-DD-cyclone-<topic>.md`.
|
|
||||||
Header per the upstream `superpowers:writing-plans` skill: a
|
|
||||||
`For agentic workers:` line that names
|
|
||||||
`superpowers:subagent-driven-development` or
|
|
||||||
`superpowers:executing-plans`, plus a `Goal / Architecture / Tech
|
|
||||||
Stack / Spec` metadata block, then numbered tasks with
|
|
||||||
`- [ ] Step N:` checkboxes.
|
|
||||||
5. **Commit prefix.** All commits on the branch follow these prefixes —
|
|
||||||
they make the SP-N merge commit readable and let `git log --grep`
|
|
||||||
filter cleanly:
|
|
||||||
- `feat(sp<N>): …` — implementation commits (e.g. `feat(sp20): NPI Luhn checksum + Tax ID format validation`).
|
|
||||||
- `docs(spec): …` — landing the spec (e.g. `docs(spec): design for CycloneStore split (Step 4)`).
|
|
||||||
- `docs(plan): …` — landing the plan.
|
|
||||||
- `merge: SP<N> <topic> into main` — the merge commit itself (e.g. `merge: SP14 5-lane Inbox UI + acknowledge action into main`).
|
|
||||||
6. **PR title.** `SP<N> <Topic>` — e.g. `SP22 Line reconciliation`.
|
|
||||||
Matches the merge-commit subject so GitHub's "merged PR" view and the
|
|
||||||
`git log` entry are identical strings.
|
|
||||||
7. **Merge shape.** A single atomic merge commit into `main` after
|
|
||||||
review. **No squash** — squash collapses the per-commit history and
|
|
||||||
breaks the SP-N audit trail. **No rebase** — rebase rewrites the SHAs
|
|
||||||
the PR review was performed against. The SP-N merge commit *is* the
|
|
||||||
record of the increment landing.
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
|
|
||||||
### Spec header (canonical SP-N shape, post-SP9)
|
|
||||||
|
|
||||||
This is the canonical header for new specs. Older specs (pre-SP9) deviate
|
|
||||||
slightly — different title style, no Branch or Aesthetic direction line —
|
|
||||||
and have not been retroactively normalized. **Use this template for any
|
|
||||||
new SP-N spec.**
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
# Sub-project <N> — <Topic>: Design Spec
|
|
||||||
|
|
||||||
**Date:** YYYY-MM-DD
|
|
||||||
**Status:** Draft, awaiting user sign-off
|
|
||||||
**Branch:** `sp<N>-<short-kebab-topic>`
|
|
||||||
**Aesthetic direction:** <one line — e.g. "No new UI" or "Modern (geometric sans + bold borders + electric blue accent)">
|
|
||||||
|
|
||||||
## 1. Scope
|
|
||||||
<2-6 lines: what's in, what's out, with explicit out-of-scope list>
|
|
||||||
```
|
|
||||||
|
|
||||||
Worked example (matches this template): `docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md`.
|
|
||||||
|
|
||||||
### Plan header (every SP-N plan starts with this)
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
# <Topic> Implementation Plan
|
|
||||||
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
|
||||||
> superpowers:subagent-driven-development (recommended) or
|
|
||||||
> superpowers:executing-plans to implement this plan task-by-task. Steps
|
|
||||||
> use checkbox (`- [ ]`) syntax for tracking.
|
|
||||||
|
|
||||||
**Goal:** <one sentence — the outcome>
|
|
||||||
**Architecture:** <one paragraph — how it's structured>
|
|
||||||
**Tech Stack:** <comma-separated list>
|
|
||||||
**Spec:** [`docs/superpowers/specs/YYYY-MM-DD-cyclone-<topic>-design.md`](../specs/...)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## File structure
|
|
||||||
<tree of new / modified files>
|
|
||||||
|
|
||||||
## Task 0: <setup>
|
|
||||||
## Task 1: <first user-visible step>
|
|
||||||
…
|
|
||||||
```
|
|
||||||
|
|
||||||
Real examples: `docs/superpowers/plans/2026-06-21-cyclone-skill-catalog.md`,
|
|
||||||
`docs/superpowers/plans/2026-06-21-cyclone-store-split.md`.
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
- **Don't skip the spec ("it's a small fix").** Small fixes still get a
|
|
||||||
3-line spec when they introduce a new numbered increment. The spec is
|
|
||||||
the *what* and the audit trail; the plan is the *how*. Without a spec
|
|
||||||
the merge commit has no anchor.
|
|
||||||
- **Don't squash the merge commit.** The SP-N merge commit is the audit
|
|
||||||
trail — it tells future you exactly which feature landed and which
|
|
||||||
commits composed it. Squash collapses that into one opaque commit and
|
|
||||||
the per-commit history is lost.
|
|
||||||
- **Don't put code in the spec — the spec is the *what*, the plan is the
|
|
||||||
*how*.** Specs describe scope, goals, non-goals, and decisions. Code
|
|
||||||
snippets belong in the plan (with checkbox steps) or in the diff, not
|
|
||||||
in the spec. SP-N specs in this repo routinely have **zero** code
|
|
||||||
blocks.
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
|
|
||||||
- **`cyclone-tests`** — every spec lists test impact; load this when
|
|
||||||
drafting or reviewing the spec to confirm fixture / `.test.tsx`
|
|
||||||
implications.
|
|
||||||
- **`cyclone-edi`** — load when the SP-N increment touches an EDI parser,
|
|
||||||
validator rule, or CAS mapping.
|
|
||||||
- **`cyclone-tail`** — load when the increment changes the live-tail wire
|
|
||||||
format or adds a streaming page.
|
|
||||||
- **`cyclone-store`** — load when the increment adds a write-path,
|
|
||||||
touches `store.py`, or wires a new `<entity>_written` event.
|
|
||||||
- **`cyclone-api-router`** — load when the increment adds or changes an
|
|
||||||
HTTP endpoint in `api_routers/`.
|
|
||||||
- **`cyclone-frontend-page`** — load when the increment adds or
|
|
||||||
refactors a page in `src/pages/`.
|
|
||||||
- **`cyclone-cli`** — load when the increment adds a CLI subcommand or
|
|
||||||
changes exit codes.
|
|
||||||
- **`superpowers:brainstorming`** (global) — run before the spec to lock
|
|
||||||
the scope / decisions in the spec's `## Decisions (locked during
|
|
||||||
brainstorming)` section.
|
|
||||||
- **`superpowers:writing-plans`** (global) — produces the plan header
|
|
||||||
format every SP-N plan follows.
|
|
||||||
@@ -1,200 +0,0 @@
|
|||||||
---
|
|
||||||
name: cyclone-store
|
|
||||||
description: "Cyclone store write-paths, the pubsub event contract (claim_written / remittance_written / activity_recorded), and the SP21 store-split boundary map. Use when: touching store.py, adding a new entity, wiring a new write event, or splitting a store module."
|
|
||||||
---
|
|
||||||
|
|
||||||
# cyclone-store
|
|
||||||
|
|
||||||
Cyclone persists every parsed X12 batch through one facade,
|
|
||||||
`CycloneStore` (`backend/src/cyclone/store.py:882`). Every write
|
|
||||||
inserts the row AND publishes a pubsub event on the in-process
|
|
||||||
`EventBus` (`backend/src/cyclone/pubsub.py:20`) so live-tail pages
|
|
||||||
see new rows the moment they land. The event contract is the seam
|
|
||||||
between persistence and streaming — a wrong event name silently
|
|
||||||
goes stale.
|
|
||||||
|
|
||||||
As of this writing: the store is a **single 2412-line module** and
|
|
||||||
the SP21 split into `backend/src/cyclone/store/` is in progress.
|
|
||||||
Three event kinds: `claim_written`, `remittance_written`,
|
|
||||||
`activity_recorded`. Next: **SP22**.
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- **Adding a new entity.** You need a new ORM model + write method +
|
|
||||||
event kind + snapshot serializer and want to know where each piece
|
|
||||||
lives (current monolith vs. post-SP21 module).
|
|
||||||
- **Wiring a new write event.** You're adding a `<entity>_written`
|
|
||||||
event and need both the publish call in the store AND the
|
|
||||||
subscribe call in the live-tail endpoint to stay in sync.
|
|
||||||
- **Debugging a write-path issue.** A page isn't reflecting new
|
|
||||||
rows, the DB has the row but the stream is silent, or the publish
|
|
||||||
raises and rolls back the transaction.
|
|
||||||
- **Splitting a store module.** You're moving a domain out of
|
|
||||||
`store.py` into its own module under `backend/src/cyclone/store/`
|
|
||||||
and need the SP21 module list + facade re-export rules.
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
1. **All writes go through `CycloneStore`.** Route handlers and
|
|
||||||
parsers must not write directly to the ORM session. The facade
|
|
||||||
opens a short-lived session via `db.SessionLocal()()`. Direct ORM
|
|
||||||
access is the #1 way the live-tail contract gets bypassed (no
|
|
||||||
event published → page silently goes stale). Read paths are
|
|
||||||
similar — prefer the facade's `iter_*` / `get_*` methods over raw
|
|
||||||
`s.execute(select(...))`.
|
|
||||||
2. **Every write publishes an event.** The event name matches the
|
|
||||||
entity: `claim_written`, `remittance_written`,
|
|
||||||
`activity_recorded` (the trailing `_recorded` signals a
|
|
||||||
non-canonical row — activity events are derived, not first-class;
|
|
||||||
current names at `store.py:1072,1081,1096`). New entities get
|
|
||||||
`<entity>_written`; activity-style side rows get
|
|
||||||
`<entity>_recorded`. Publish is **best-effort** — failures are
|
|
||||||
logged but never roll back the persisted batch
|
|
||||||
(`store.py:1097-1098`).
|
|
||||||
3. **Snapshot shape.** Each entity has a `to_ui_<entity>` serializer
|
|
||||||
(plain Python function returning a dict) — currently
|
|
||||||
`to_ui_claim`, `to_ui_remittance`, `to_ui_claim_from_orm`,
|
|
||||||
`to_ui_remittance_from_orm`, `to_ui_provider`. Post-SP21 these
|
|
||||||
move to `backend/src/cyclone/store/ui.py`. The serializer is the
|
|
||||||
single source of truth for what the frontend sees — every event
|
|
||||||
payload MUST match what the matching list endpoint returns for
|
|
||||||
that row (`store.py:1052-1054`).
|
|
||||||
4. **SP21 boundaries.** Post-split, each domain lives in its own
|
|
||||||
module under `backend/src/cyclone/store/`: `__init__.py` (facade
|
|
||||||
+ `CycloneStore` class), `exceptions.py`, `records.py`,
|
|
||||||
`orm_builders.py`, `ui.py`, `write.py`, `batches.py`,
|
|
||||||
`claim_detail.py`, `acks.py`, `backups.py`, `inbox.py`,
|
|
||||||
`providers.py`. Cross-module writes go through `CycloneStore`
|
|
||||||
facade methods, not direct module access. The facade re-exports
|
|
||||||
every name callers currently import from `cyclone.store`. Full
|
|
||||||
list: `docs/superpowers/plans/2026-06-21-cyclone-store-split.md:25-37`.
|
|
||||||
5. **No business logic in route handlers.** A route handler validates
|
|
||||||
input, calls `store.<method>(...)`, passes the `event_bus`,
|
|
||||||
returns the serialized result. Reconciliation, idempotency
|
|
||||||
checks, CAS adjustment persistence — all live in the store.
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
|
|
||||||
### A `CycloneStore.add` write — publishes events from inserted rows
|
|
||||||
|
|
||||||
Taken from `backend/src/cyclone/store.py:898-1107`. The method opens
|
|
||||||
a session, inserts rows, then runs a sync `_publish_events_sync`
|
|
||||||
after commit so subscribers can immediately re-fetch consistent data.
|
|
||||||
|
|
||||||
```python
|
|
||||||
def add(
|
|
||||||
self,
|
|
||||||
record: BatchRecord,
|
|
||||||
*,
|
|
||||||
event_bus: "EventBus | None" = None,
|
|
||||||
) -> None:
|
|
||||||
inserted_claim_ids: list[str] = []
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
s.add(Batch(id=record.id, kind=record.kind, ...))
|
|
||||||
if isinstance(record, BatchRecord837):
|
|
||||||
for claim in record.result.claims:
|
|
||||||
if s.get(Claim, claim.claim_id) is not None:
|
|
||||||
continue # idempotency: skip dupes
|
|
||||||
s.add(_claim_837_row(claim, record.id))
|
|
||||||
s.add(ActivityEvent(kind="claim_submitted", ...))
|
|
||||||
inserted_claim_ids.append(claim.claim_id)
|
|
||||||
# ... 835 branch + flush + cas adjustments ...
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
if event_bus is not None and inserted_claim_ids:
|
|
||||||
self._publish_events_sync(event_bus, record, inserted_claim_ids)
|
|
||||||
|
|
||||||
def _publish_events_sync(self, event_bus, record, claim_ids):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
for cid in claim_ids:
|
|
||||||
ui = to_ui_claim_from_orm(s.get(Claim, cid), ...)
|
|
||||||
self._sync_publish(event_bus, "claim_written", ui)
|
|
||||||
# ... remittance + activity loops ...
|
|
||||||
```
|
|
||||||
|
|
||||||
`EventBus.publish` is async but the body is pure sync `put_nowait`,
|
|
||||||
so the store calls `_sync_publish` directly to avoid forcing sync
|
|
||||||
FastAPI handlers to await.
|
|
||||||
|
|
||||||
### Backend `/api/<resource>/stream` endpoint — subscribes to the event
|
|
||||||
|
|
||||||
Taken from `backend/src/cyclone/api.py:1380-1401`. Two phases — eager
|
|
||||||
snapshot, then live subscription — wrapped in `StreamingResponse`
|
|
||||||
with `media_type="application/x-ndjson"`.
|
|
||||||
|
|
||||||
```python
|
|
||||||
@app.get("/api/claims/stream")
|
|
||||||
async def claims_stream(request: Request, ...) -> StreamingResponse:
|
|
||||||
bus: EventBus = request.app.state.event_bus
|
|
||||||
|
|
||||||
async def gen() -> AsyncIterator[bytes]:
|
|
||||||
rows = store.iter_claims(status=status, ...) # 1. Snapshot
|
|
||||||
for row in rows:
|
|
||||||
yield _ndjson_line({"type": "item", "data": row})
|
|
||||||
yield _ndjson_line({"type": "snapshot_end", "data": {"count": len(rows)}})
|
|
||||||
|
|
||||||
async for chunk in _tail_events(request, bus, ["claim_written"]): # 2. Live
|
|
||||||
yield chunk
|
|
||||||
|
|
||||||
return StreamingResponse(gen(), media_type="application/x-ndjson")
|
|
||||||
```
|
|
||||||
|
|
||||||
`_ndjson_line` and `_tail_events` live in
|
|
||||||
`backend/src/cyclone/api_helpers.py`. The `["remittance_written"]`
|
|
||||||
and `["activity_recorded"]` subscriptions are at `api.py:1891,2002`.
|
|
||||||
|
|
||||||
### Backend test — asserts both the row AND the event landed
|
|
||||||
|
|
||||||
The autouse `conftest.py` (`backend/tests/conftest.py:20`) wires a
|
|
||||||
fresh `EventBus` onto `app.state` per test.
|
|
||||||
|
|
||||||
```python
|
|
||||||
def test_publishes_claim_written_event(client: TestClient) -> None:
|
|
||||||
bus = app.state.event_bus
|
|
||||||
resp = client.post("/api/parse-837",
|
|
||||||
files={"file": ("x.837", MINIMAL_837, "text/plain")})
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert list(store.iter_claims(limit=10)) # row landed
|
|
||||||
|
|
||||||
queues = bus._subscribers.get("claim_written", []) # event published
|
|
||||||
assert queues
|
|
||||||
evt = queues[0].get_nowait()
|
|
||||||
assert evt["_kind"] == "claim_written"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
- **Don't read directly from the ORM in a route handler — go through
|
|
||||||
the snapshot serializer.** A handler that does
|
|
||||||
`with db.SessionLocal()() as s: row = s.get(Claim, cid); return row`
|
|
||||||
bypasses `to_ui_claim_from_orm` and silently drifts from the event
|
|
||||||
payload shape. Always call `store.get_claim_detail(cid)` (or the
|
|
||||||
equivalent `get_*` facade method).
|
|
||||||
- **Don't introduce a new event name without updating the subscriber
|
|
||||||
list.** Today every `<entity>_written` event has exactly one
|
|
||||||
consumer — the matching `/api/<resource>/stream` endpoint,
|
|
||||||
currently in `backend/src/cyclone/api.py` (claims at `:1398`,
|
|
||||||
remittances at `:1891`, activity at `:2002`). Any future split
|
|
||||||
into `backend/src/cyclone/api_routers/` must wire the same
|
|
||||||
subscription.
|
|
||||||
- **Don't merge a write method with its event publication into
|
|
||||||
separate places.** `_publish_events_sync` lives next to `add` in
|
|
||||||
`store.py:1042-1107` so reviewers see both halves of the contract
|
|
||||||
in one diff. Post-SP21 the same rule applies.
|
|
||||||
- **Don't make the publish call blocking on commit failures.**
|
|
||||||
Publish errors are caught and logged at `store.py:1097-1098`; a
|
|
||||||
failing subscriber MUST NOT roll back the persisted batch. The
|
|
||||||
batch is the source of truth; the event is the cache-invalidation
|
|
||||||
hint.
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
|
|
||||||
- **`cyclone-edi`** — the parsed `ClaimOutput` / `ParseResult<EDI>`
|
|
||||||
lands in the store via `CycloneStore.add`.
|
|
||||||
- **`cyclone-api-router`** — the route that calls `store.<method>(...)`
|
|
||||||
and (for stream endpoints) subscribes to the matching
|
|
||||||
`<entity>_written` event.
|
|
||||||
- **`cyclone-tail`** — the consumer side of the event contract;
|
|
||||||
load when changing the wire format or adding a streaming hook.
|
|
||||||
- **`cyclone-tests`** — write-path tests live under `backend/tests/`
|
|
||||||
and assert both the DB row and the event payload.
|
|
||||||
@@ -1,200 +0,0 @@
|
|||||||
---
|
|
||||||
name: cyclone-tail
|
|
||||||
description: "Cyclone live-tail streaming wire format and the useTailStream / useMergedTail hook triplet. Use when: adding a new streaming list page, changing the wire format, debugging stalled/reconnecting state, or modifying the StatusPill behavior."
|
|
||||||
---
|
|
||||||
|
|
||||||
# cyclone-tail
|
|
||||||
|
|
||||||
Cyclone keeps the Claims, Remittances, and Activity pages live without
|
|
||||||
polling: every store write publishes an internal EventBus event, the
|
|
||||||
page opens a `GET /api/<resource>/stream` HTTP/1.1 chunked-NDJSON
|
|
||||||
connection, and new rows land in the table the moment they hit the
|
|
||||||
database. This skill codifies the wire format, the hook triplet, and
|
|
||||||
the backoff/stall machinery so additions stay consistent with the
|
|
||||||
three streaming pages already shipped (`Claims`, `Remittances`,
|
|
||||||
`ActivityLog`).
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- **Adding a new streaming page.** Mounting `useTailStream(resource)`
|
|
||||||
on a page — you need the hook triplet shape (initial fetch +
|
|
||||||
`useTailStream` + `useMergedTail`), the `<TailStatusPill>` wiring,
|
|
||||||
and the dedup rules in `useMergedTail`.
|
|
||||||
- **Changing the wire format.** Adding a new event type — update
|
|
||||||
`TailEvent` in `src/lib/tail-stream.ts:22-44`, the dispatch switch
|
|
||||||
in `src/hooks/useTailStream.ts:173-195`, the emitter in
|
|
||||||
`backend/src/cyclone/api_helpers.py:tail_events()`, and
|
|
||||||
`references/wire-format.md`.
|
|
||||||
- **Debugging stalled/reconnecting state.** Confirm whether the
|
|
||||||
backend is heartbeating (`CYCLONE_TAIL_HEARTBEAT_S`, default `15s`)
|
|
||||||
or the stall timer fired (`STALL_TIMEOUT_MS = 30_000` at
|
|
||||||
`src/hooks/useTailStream.ts:53`).
|
|
||||||
- **Tuning heartbeat/stall timing.** Changing the 30s stall threshold
|
|
||||||
or the 15s heartbeat interval — README's "Status pill" + "Knobs"
|
|
||||||
tables need to stay in sync (`README.md:109-129`).
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
1. **Wire format.** Newline-delimited JSON. Every line is
|
|
||||||
`{"type": ..., "data": ...}`. Known `type` values: `item`
|
|
||||||
(per-row envelope), `snapshot_end` (`{"count": N}` marker after the
|
|
||||||
snapshot), `heartbeat` (`{"ts": "<iso-8601>"}` keep-alive),
|
|
||||||
`item_dropped` (`{"id": "..."}` queue-overflow notice), `error`
|
|
||||||
(`{"message": "..."}` promoted to a thrown error by the hook).
|
|
||||||
Defined at `src/lib/tail-stream.ts:22-44`; emitted by
|
|
||||||
`backend/src/cyclone/api.py:1357-2006`. See
|
|
||||||
`references/wire-format.md`.
|
|
||||||
2. **Hook triplet.** Streaming pages compose three pieces:
|
|
||||||
`useTailStream(resource)` (`src/hooks/useTailStream.ts:80` — opens
|
|
||||||
the stream, drives the backoff/stall state machine, dispatches
|
|
||||||
`item` events into `useTailStore`),
|
|
||||||
`useMergedTail(resource, baseItems, filterFn?)`
|
|
||||||
(`src/hooks/useMergedTail.ts:25` — returns
|
|
||||||
`baseItems + tailSlice` dedup'd against `baseItems`), and a
|
|
||||||
per-resource initial-fetch hook (`useClaims`, `useRemittances`,
|
|
||||||
`useActivity`). The page wires all three — see
|
|
||||||
`src/pages/Claims.tsx:87-89`.
|
|
||||||
3. **Stall threshold.** 30 seconds of total silence — heartbeats
|
|
||||||
included — flips status to `stalled` and surfaces the
|
|
||||||
`↻ Reconnect` button on `<TailStatusPill>`. Constant:
|
|
||||||
`STALL_TIMEOUT_MS = 30_000` at `src/hooks/useTailStream.ts:53`.
|
|
||||||
Re-armed on every event including `heartbeat` and `item_dropped`
|
|
||||||
(`useTailStream.ts:124-140`). Don't change without updating
|
|
||||||
`README.md:109-123`.
|
|
||||||
4. **Snapshot first.** Every stream emits the snapshot before any
|
|
||||||
live `item` events, then closes with exactly one `snapshot_end`
|
|
||||||
carrying `{"count": N}`. The hook uses `snapshot_end` to flip
|
|
||||||
`<TailStatusPill>` from `connecting` to `live` and to reset the
|
|
||||||
reconnect backoff counter (`useTailStream.ts:174-180`).
|
|
||||||
5. **Content-Type.** Stream endpoints respond with
|
|
||||||
`media_type="application/x-ndjson"` — see
|
|
||||||
`backend/src/cyclone/api.py:1401,1894,2005`. Frontend sets
|
|
||||||
`Accept: application/x-ndjson` at `src/lib/tail-stream.ts:67-70`.
|
|
||||||
Never `application/json` for a stream endpoint.
|
|
||||||
6. **Backoff.** Transient errors retry with `1s → 2s → 4s → 8s → 16s
|
|
||||||
→ 30s` capped — `BACKOFF_STEPS_MS` at
|
|
||||||
`src/hooks/useTailStream.ts:48-50`. Counter resets on every
|
|
||||||
`snapshot_end`.
|
|
||||||
7. **Heartbeat knob.** Idle heartbeat interval is configurable via
|
|
||||||
`CYCLONE_TAIL_HEARTBEAT_S` (default `15`, parsed at call time in
|
|
||||||
`backend/src/cyclone/api_helpers.py:185-198`). Tests override to
|
|
||||||
a small value to keep runtime bounded.
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
|
|
||||||
### Page-hook skeleton — `Claims.tsx`
|
|
||||||
|
|
||||||
Pattern from `src/pages/Claims.tsx:85-90`. Three hooks in order:
|
|
||||||
`useClaims(params)` (initial TanStack Query fetch),
|
|
||||||
`useTailStream("claims")` (opens the stream, owns status state), and
|
|
||||||
`useMergedTail("claims", data?.items ?? [], tailFilterFn)` (combines
|
|
||||||
initial snapshot with live tail, dedup'd by id, filtered by the page's
|
|
||||||
predicate applied AFTER dedup at `useMergedTail.ts:72-74`).
|
|
||||||
|
|
||||||
```ts
|
|
||||||
import { useClaims } from "@/hooks/useClaims";
|
|
||||||
import { useTailStream } from "@/hooks/useTailStream";
|
|
||||||
import { useMergedTail } from "@/hooks/useMergedTail";
|
|
||||||
|
|
||||||
export function ClaimsPage() {
|
|
||||||
const { data } = useClaims({ status: "submitted" });
|
|
||||||
const { status, lastEventAt, forceReconnect } = useTailStream("claims");
|
|
||||||
const tailFilterFn = (c: Claim) => c.status === "submitted";
|
|
||||||
const items = useMergedTail("claims", data?.items ?? [], tailFilterFn);
|
|
||||||
// <TailStatusPill status={status} lastEventAt={lastEventAt}
|
|
||||||
// onReconnect={forceReconnect} /> + <Table items={items} />
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backend `/api/foo/stream` endpoint
|
|
||||||
|
|
||||||
Pattern from `backend/src/cyclone/api.py:1357-1401`. Register BEFORE
|
|
||||||
`/api/foo/{foo_id}` so the literal `stream` segment doesn't match as
|
|
||||||
an id. Two phases — eager snapshot, then live subscription — wrapped
|
|
||||||
in `StreamingResponse` with `media_type="application/x-ndjson"`.
|
|
||||||
|
|
||||||
```python
|
|
||||||
@app.get("/api/foo/stream")
|
|
||||||
async def foo_stream(
|
|
||||||
request: Request,
|
|
||||||
status: str | None = Query(None),
|
|
||||||
limit: int = Query(100, ge=1, le=1000),
|
|
||||||
) -> StreamingResponse:
|
|
||||||
bus: EventBus = request.app.state.event_bus
|
|
||||||
|
|
||||||
async def gen() -> AsyncIterator[bytes]:
|
|
||||||
# 1. Snapshot.
|
|
||||||
rows = store.iter_foos(status=status, limit=limit)
|
|
||||||
for row in rows:
|
|
||||||
yield _ndjson_line({"type": "item", "data": row})
|
|
||||||
yield _ndjson_line({"type": "snapshot_end", "data": {"count": len(rows)}})
|
|
||||||
# 2. Live subscription + heartbeats.
|
|
||||||
async for chunk in _tail_events(request, bus, ["foo_written"]):
|
|
||||||
yield chunk
|
|
||||||
|
|
||||||
return StreamingResponse(gen(), media_type="application/x-ndjson")
|
|
||||||
```
|
|
||||||
|
|
||||||
`_ndjson_line` and `_tail_events` live in
|
|
||||||
`backend/src/cyclone/api_helpers.py`; the latter forwards EventBus
|
|
||||||
events as `item` lines and emits `heartbeat` lines on the cadence
|
|
||||||
from `heartbeat_seconds()`.
|
|
||||||
|
|
||||||
### `<TailStatusPill>` wiring
|
|
||||||
|
|
||||||
Pattern from `src/components/TailStatusPill.tsx:22-83`. The pill
|
|
||||||
takes `status` + `lastEventAt` from `useTailStream` plus
|
|
||||||
`forceReconnect` so the `↻ Reconnect` button wires to the hook's
|
|
||||||
`reconnectNonce` bump (aborts and reopens the stream —
|
|
||||||
`useTailStream.ts:99-101`). The button only renders when
|
|
||||||
`status === "stalled" || status === "error"` (`TailStatusPill.tsx:52`).
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
- **Don't hand-roll `fetch` + `ReadableStream` parsing in a page.**
|
|
||||||
All stream consumers go through `streamTail(resource, opts?)` at
|
|
||||||
`src/lib/tail-stream.ts:58`. The parser handles `TextDecoderStream`,
|
|
||||||
newline splitting, malformed-line tolerance (`console.warn` + skip),
|
|
||||||
the `KNOWN_TYPES` allowlist, and abort-on-signal semantics
|
|
||||||
(`tail-stream.ts:75,98,107`). Duplicating it loses all of those.
|
|
||||||
- **Don't change the wire format on one endpoint without updating
|
|
||||||
the others.** The parser (`src/lib/tail-stream.ts`) and the hook
|
|
||||||
dispatch switch (`useTailStream.ts:173-195`) are shared across all
|
|
||||||
three live streams. Adding a new event type means updating
|
|
||||||
`TailEvent`, `KNOWN_TYPES`, the dispatch `case`, the `armStall`
|
|
||||||
re-arm list, and the backend emitter — in that order.
|
|
||||||
- **Don't emit `item` events before `snapshot_end`.** The hook uses
|
|
||||||
`snapshot_end` as the marker to flip `<TailStatusPill>` from
|
|
||||||
`connecting` to `live` and to reset the reconnect backoff counter.
|
|
||||||
Emitting `item`s first lands rows in `useTailStore` but the UI
|
|
||||||
still reads "Connecting" — operators see a flash of stale state.
|
|
||||||
Emit snapshot, then `snapshot_end`, then live (`api.py:1386-1401`).
|
|
||||||
- **Don't change the 30s stall threshold without updating the README
|
|
||||||
"Status pill" table.** The constant
|
|
||||||
(`STALL_TIMEOUT_MS = 30_000` at `useTailStream.ts:53`) is the
|
|
||||||
contract the pill is documented against — a bump needs the matching
|
|
||||||
edit at `README.md:118`.
|
|
||||||
- **Don't add `useTailStream` calls from `useFoo.ts` hooks.**
|
|
||||||
Streaming connections belong on the page (`src/pages/Claims.tsx`,
|
|
||||||
`Remittances.tsx`, `ActivityLog.tsx`). Hoisting into `useFoo`
|
|
||||||
couples the lifecycle to whoever mounts it and breaks
|
|
||||||
one-resource-one-page ownership.
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
|
|
||||||
- **`cyclone-frontend-page`** — page components live in `src/pages/`;
|
|
||||||
load when adding or refactoring a streaming page to confirm the
|
|
||||||
route + `PageHeader` + table conventions.
|
|
||||||
- **`cyclone-api-router`** — endpoint conventions (`api_routers/` for
|
|
||||||
resource-group routers, `api.py` for the live-tail endpoints); load
|
|
||||||
when adding or changing an HTTP endpoint that surfaces streamed or
|
|
||||||
paginated data.
|
|
||||||
- **`cyclone-store`** — write-path conventions in `store.py` and the
|
|
||||||
pubsub event contract (`claim_written`, `remittance_written`,
|
|
||||||
`activity_recorded`); load when adding a new entity whose writes
|
|
||||||
should fan out to a stream endpoint.
|
|
||||||
- **`cyclone-tests`** — frontend `*.test.tsx` siblings cover
|
|
||||||
`useTailStream`, `useMergedTail`, `TailStatusPill`; backend
|
|
||||||
`test_api_stream_live.py` covers the three live-tail endpoints;
|
|
||||||
load when the increment changes wire-format behavior or adds a
|
|
||||||
streaming hook.
|
|
||||||
@@ -1,76 +0,0 @@
|
|||||||
# Live-tail wire format — field reference
|
|
||||||
|
|
||||||
The Cyclone live-tail NDJSON contract is owned by the frontend parser
|
|
||||||
(`src/lib/tail-stream.ts:22-44`) and the backend emitter
|
|
||||||
(`backend/src/cyclone/api_helpers.py:tail_events()` + the three
|
|
||||||
endpoints in `backend/src/cyclone/api.py:1357-2006`). This file is the
|
|
||||||
quick reference; the canonical prose lives in the README and the
|
|
||||||
source-of-truth type definitions.
|
|
||||||
|
|
||||||
## Source
|
|
||||||
|
|
||||||
Wire format excerpt copied verbatim from `README.md:76-94` (the
|
|
||||||
"Live updates → Wire format" section). The README is the user-facing
|
|
||||||
exposition; this reference adds the per-line field semantics and the
|
|
||||||
parser-tolerance rules from the code.
|
|
||||||
|
|
||||||
> Each stream endpoint emits newline-delimited JSON. The first batch
|
|
||||||
> is the **snapshot** of currently-known rows; after that comes
|
|
||||||
> **`snapshot_end`** with the count, then the **live** events.
|
|
||||||
>
|
|
||||||
> ```json
|
|
||||||
> {"type":"item","data":{"id":"CLM-1", "...":"..."}}
|
|
||||||
> {"type":"item","data":{"id":"CLM-2", "...":"..."}}
|
|
||||||
> {"type":"snapshot_end","data":{"count":2}}
|
|
||||||
> {"type":"item","data":{"id":"CLM-3", "...":"..."}} ← live
|
|
||||||
> {"type":"heartbeat","data":{"ts":"2026-06-20T23:17:09Z"}} ← idle keep-alive
|
|
||||||
> ```
|
|
||||||
>
|
|
||||||
> Lines are `{"type": ..., "data": ...}`; known types are `item`,
|
|
||||||
> `snapshot_end`, `heartbeat`, and (rare) `item_dropped` /
|
|
||||||
> `error`. Heartbeats keep the connection alive when nothing is
|
|
||||||
> happening — clients flip to `stalled` after 30s of total silence
|
|
||||||
> (heartbeat or otherwise) and surface a **↻ Reconnect** button.
|
|
||||||
|
|
||||||
## Per-line field reference
|
|
||||||
|
|
||||||
| `type` | `data` shape | Required? | Emitted by | Parser behavior |
|
|
||||||
| -------------- | ----------------------------------------- | --------- | ------------------------------------------- | -------------------------------------------- |
|
|
||||||
| `item` | resource-specific row (`Claim` / `Remittance` / `Activity`) | yes, in `data` (the per-row envelope) | snapshot loop + `_tail_events` forwarding `claim_written` / `remittance_written` / `activity_recorded` | `dispatch(resource, ev.data)` → `useTailStore.addClaim` / `addRemittance` / `addActivity` (first-write-wins dedup on the id-keyed slices — `tail-store.ts:104,120`). |
|
|
||||||
| `snapshot_end` | `{"count": N}` (integer ≥ 0) | yes, on every stream | `api.py:1395,1889,1999` (one per stream, after the snapshot loop) | Flips `<TailStatusPill>` from `connecting` to `live`, resets the reconnect backoff counter to 0 (`useTailStream.ts:174-180`). |
|
|
||||||
| `heartbeat` | `{"ts": "<iso-8601 UTC>"}` | yes, but only when idle | `_tail_events` in `api_helpers.py:241-245` (cadence from `heartbeat_seconds()`, default 15s) | Re-arms the stall timer (`useTailStream.ts:124-140`); no state change. |
|
|
||||||
| `item_dropped` | `{"id": "<string>"}` | optional (rare; queue overflow) | EventBus drop-oldest path (per the spec at `docs/superpowers/specs/2026-06-20-cyclone-live-tail-design.md:299`) | Re-arms the stall timer; no state change. The `id` field is informational — the hook does not refetch on drop, the page does (see Spec §3.6). |
|
|
||||||
| `error` | `{"message": "<string>"}` | optional (server-side failure) | Reserved for future server-emitted errors (currently only thrown client-side) | Hook promotes to a thrown `Error(message)` so the catch block runs the reconnect machinery (`useTailStream.ts:190-194`). |
|
|
||||||
|
|
||||||
## Parser tolerance
|
|
||||||
|
|
||||||
The shared parser at `src/lib/tail-stream.ts` is intentionally
|
|
||||||
forgiving so a single bad frame doesn't kill the stream:
|
|
||||||
|
|
||||||
- **Trailing `\r`** is stripped per line (`tail-stream.ts:116`) so a
|
|
||||||
CRLF-terminated stream still parses.
|
|
||||||
- **Malformed JSON** (`JSON.parse` throws) → `console.warn` + skip the
|
|
||||||
line; the iterator continues (`tail-stream.ts:122-131`).
|
|
||||||
- **Unknown `type`** (not in `KNOWN_TYPES`) → `console.warn` + skip
|
|
||||||
(`tail-stream.ts:141-148`). Adding a new event type is
|
|
||||||
forward-compatible: old clients see warn lines, new clients see the
|
|
||||||
typed event.
|
|
||||||
- **Empty lines** (consecutive `\n`s) → silently skipped
|
|
||||||
(`tail-stream.ts:118`).
|
|
||||||
- **No trailing newline** → flushed as a final partial line on
|
|
||||||
stream close (`tail-stream.ts:154-178`).
|
|
||||||
- **Abort signal** → iterator exits cleanly without throwing
|
|
||||||
(`tail-stream.ts:75,98,107`).
|
|
||||||
|
|
||||||
## Endpoint inventory
|
|
||||||
|
|
||||||
| Method | Path | Subscribes to | Default sort | Defined at |
|
|
||||||
| ------ | ------------------------- | -------------------- | ------------------- | ----------------------------------- |
|
|
||||||
| GET | `/api/claims/stream` | `claim_written` | `-submission_date` | `backend/src/cyclone/api.py:1357` |
|
|
||||||
| GET | `/api/remittances/stream` | `remittance_written` | `-received_date` | `backend/src/cyclone/api.py:1858` |
|
|
||||||
| GET | `/api/activity/stream` | `activity_recorded` | `-timestamp` (limit 50) | `backend/src/cyclone/api.py:1971` |
|
|
||||||
|
|
||||||
All three accept the same query params as their non-streaming
|
|
||||||
counterparts (`status`, `payer`, `date_from`, …) so a frontend can
|
|
||||||
swap a one-shot fetch for a tail with no URL surgery. Responses are
|
|
||||||
`Content-Type: application/x-ndjson`.
|
|
||||||
@@ -1,166 +0,0 @@
|
|||||||
---
|
|
||||||
name: cyclone-tests
|
|
||||||
description: "Cyclone pytest + vitest fixture patterns, prodfiles layout, backend/tests/fixtures/ conventions, .test.tsx sibling rule. Use when: adding a backend pytest case, adding a frontend vitest test, or wiring in a real-EDI prodfiles sample."
|
|
||||||
---
|
|
||||||
|
|
||||||
# cyclone-tests
|
|
||||||
|
|
||||||
The Cyclone test suite is split two ways: **backend pytest** (89 test files under `backend/tests/`, 13 flat fixtures in `backend/tests/fixtures/`, one autouse `conftest.py` that resets the DB per-test) and **frontend vitest** (59 `*.test.ts(x)` siblings across `src/`, two rendering styles — `@testing-library/react` and a custom `createRoot`+`Probe` shim). This skill codifies the conventions so additions stay consistent with what's already there.
|
|
||||||
|
|
||||||
As of this writing: **89 backend test files**, **13 flat backend fixtures**, **59 frontend `*.test.ts(x)` siblings**, and **23 prodfiles samples** across `docs/prodfiles/{837p-from-axiscare,835fromco,FromHPE,claims}/`. The next increment is **SP22**.
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- **Adding a backend pytest case.** You're about to add a new `test_*.py` under `backend/tests/` and need the autouse DB-fixture rules, the fixture-path convention, and the right naming flavor (`test_api_*.py` vs `test_<module>_*.py`).
|
|
||||||
- **Adding a frontend vitest test.** You're about to add a `*.test.ts(x)` sibling and need the `// @vitest-environment happy-dom` setup, the act-environment flag, and the right rendering helper (`@testing-library/react` vs the `createRoot`+`Probe` shim).
|
|
||||||
- **Dropping in a prodfiles fixture.** You have a real EDI sample under `docs/prodfiles/<source>/` and need the copy step that makes it test-runnable without coupling the test to the prodfiles archive.
|
|
||||||
- **Debugging a flaky test.** A test passes locally but flakes in CI — load this skill to check the determinism + no-network rules before chasing the symptom.
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
1. **Frontend sibling rule.** Every new file in `src/` that contains testable logic gets a `*.test.ts(x)` next to it. `useFoo.ts` → `useFoo.test.ts`. `ClaimDrawer.tsx` → `ClaimDrawer.test.tsx`. The 59 existing siblings follow this; CI implicitly enforces it via the default `src/**/*.test.ts(x)` glob in `vitest.config.ts`.
|
|
||||||
2. **Backend test location.** Tests live under `backend/tests/test_*.py`. Two flavors, two naming patterns:
|
|
||||||
- **Integration tests** (FastAPI surface) → `test_api_<topic>_<verb>.py` — e.g. `test_api_parse_persists.py`, `test_api_999.py`.
|
|
||||||
- **Pure-unit tests** (parsers, validators, store internals) → `test_<module>_<behavior>.py` — e.g. `test_cas_codes.py`, `test_pubsub.py`.
|
|
||||||
3. **Prodfiles drop-in.** Real EDI samples live under `docs/prodfiles/<source>/<file>.txt` (sources seen so far: `837p-from-axiscare/`, `835fromco/`, `FromHPE/`, `claims/`). To use one in a test: copy the file to `backend/tests/fixtures/<descriptive-name>.txt` (flat — no per-test subdirectories; the existing 13 fixtures all sit at the top level) and reference it from the test as a module-level `Path` constant. See `## Patterns` for the exact line.
|
|
||||||
4. **Determinism.** Time-sensitive tests must not depend on wall-clock time.
|
|
||||||
- **Frontend** uses `vi.useFakeTimers()` + `vi.setSystemTime(new Date("YYYY-MM-DDTHH:MM:SSZ"))` (see `src/components/TailStatusPill.test.tsx:54-58`) and `vi.advanceTimersByTime(ms)` to drive interval / backoff code deterministically.
|
|
||||||
- **Backend** dates are passed explicitly as fixture values — e.g. `datetime.now(timezone.utc)` is fine for "now-ish" anchors, but for fixed dates pass `datetime(2026, 6, 20, tzinfo=timezone.utc)`. The project does **not** currently use `freezegun` or `mock.patch(datetime)`; if you need deterministic date mocking, propose adding `freezegun` to `backend/pyproject.toml` rather than rolling your own.
|
|
||||||
5. **No network.** Tests must not hit the network.
|
|
||||||
- **Backend:** `fastapi.testclient.TestClient(app)` runs in-process; no uvicorn. The autouse `conftest.py` fixture (`backend/tests/conftest.py:20`) points `CYCLONE_DB_URL` at `tmp_path/test.db`, calls `db._reset_for_tests()` + `db.init_db()`, and wires a fresh `EventBus` onto `app.state`.
|
|
||||||
- **Frontend:** `vi.stubGlobal("fetch", vi.fn().mockResolvedValue(...))` for hooks; `vitest.config.ts` sets `VITE_API_BASE_URL=http://test.local` so the `api` module doesn't throw `notConfiguredError` before the mock fires.
|
|
||||||
6. **pytest collection.** Run `cd backend && python -m pytest tests/<file>::<name> -v` for the fastest single-test feedback loop. Run `cd backend && python -m pytest tests/<file> -v` for one file. Run `cd backend && python -m pytest` for the full suite — this is the merge gate. Frontend: `npm test` (alias for `vitest run`) for the full suite; `npx vitest run src/hooks/useFoo.test.ts` for one file.
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
|
|
||||||
### Backend pytest using a fixture + per-test SQLite DB
|
|
||||||
|
|
||||||
Canonical shape — see `backend/tests/test_api_999.py:1-50` for the full file. The autouse `conftest.py` already provides the DB init + `EventBus` reset; most tests just add a `client` fixture.
|
|
||||||
|
|
||||||
```python
|
|
||||||
"""Tests for the FastAPI surface in cyclone.api for the 999 endpoint."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
from cyclone.api import app
|
|
||||||
|
|
||||||
# Fixture reference — flat, module-level Path constant. NEVER reach into
|
|
||||||
# docs/prodfiles/ from a test; the fixtures/ dir is the test-consumed surface.
|
|
||||||
ACCEPTED = Path(__file__).parent / "fixtures" / "minimal_999.txt"
|
|
||||||
REJECTED = Path(__file__).parent / "fixtures" / "minimal_999_rejected.txt"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def client() -> TestClient:
|
|
||||||
return TestClient(app)
|
|
||||||
|
|
||||||
|
|
||||||
def test_parse_999_endpoint_happy_path(client: TestClient):
|
|
||||||
text = ACCEPTED.read_text()
|
|
||||||
resp = client.post(
|
|
||||||
"/api/parse-999",
|
|
||||||
files={"file": ("minimal_999.txt", text, "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
body = resp.json()
|
|
||||||
assert body["ack"]["ack_code"] == "A"
|
|
||||||
```
|
|
||||||
|
|
||||||
Override the autouse DB fixture only when you need a custom env (e.g. a per-test backup directory) — see `backend/tests/test_999_rejected_state.py:20-25`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _setup(tmp_path, monkeypatch):
|
|
||||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/inbox.db")
|
|
||||||
from cyclone import db
|
|
||||||
db._reset_for_tests()
|
|
||||||
db.init_db()
|
|
||||||
yield
|
|
||||||
```
|
|
||||||
|
|
||||||
### Frontend vitest `*.test.tsx` for a component using fake timers
|
|
||||||
|
|
||||||
Pattern taken from `src/components/TailStatusPill.test.tsx:1-62` — uses `vi.useFakeTimers()` + `vi.setSystemTime(...)` to drive interval-based code deterministically.
|
|
||||||
|
|
||||||
```ts
|
|
||||||
// @vitest-environment happy-dom
|
|
||||||
// React's act warnings need an act-aware environment — mirror the other
|
|
||||||
// hook tests in this repo.
|
|
||||||
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
|
||||||
|
|
||||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
|
||||||
import { MyComponent } from "./MyComponent";
|
|
||||||
|
|
||||||
describe("MyComponent", () => {
|
|
||||||
beforeEach(() => {
|
|
||||||
vi.useFakeTimers();
|
|
||||||
vi.setSystemTime(new Date("2026-06-20T12:00:30Z"));
|
|
||||||
});
|
|
||||||
|
|
||||||
afterEach(() => {
|
|
||||||
vi.useRealTimers();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_renders_after_interval_tick", () => {
|
|
||||||
vi.advanceTimersByTime(30_000); // drive the setInterval
|
|
||||||
// …assert…
|
|
||||||
});
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### Frontend vitest `*.test.tsx` for a component using `@testing-library/react` + `happy-dom`
|
|
||||||
|
|
||||||
Pattern taken from `src/hooks/useInboxLanes.test.ts:1-80`. (A handful of older tests — `useClaimDetail.test.ts`, `useDrawerUrlState.test.ts`, `TailStatusPill.test.tsx` — roll a custom `createRoot`+`Probe` shim instead. Both styles are accepted; `@testing-library/react` is preferred when the hook has async dependencies because `waitFor` is built in.)
|
|
||||||
|
|
||||||
```ts
|
|
||||||
// @vitest-environment happy-dom
|
|
||||||
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
|
||||||
|
|
||||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
|
||||||
import { act, cleanup, renderHook, waitFor } from "@testing-library/react";
|
|
||||||
import { useMyHook } from "./useMyHook";
|
|
||||||
|
|
||||||
vi.mock("@/lib/api", () => ({
|
|
||||||
api: { fetchFoo: vi.fn() },
|
|
||||||
}));
|
|
||||||
|
|
||||||
afterEach(() => {
|
|
||||||
cleanup();
|
|
||||||
vi.unstubAllGlobals();
|
|
||||||
vi.useRealTimers();
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("useMyHook", () => {
|
|
||||||
it("loads data on mount", async () => {
|
|
||||||
vi.stubGlobal("fetch", vi.fn().mockResolvedValue({
|
|
||||||
ok: true,
|
|
||||||
json: async () => ({ items: [] }),
|
|
||||||
}));
|
|
||||||
const { result } = renderHook(() => useMyHook());
|
|
||||||
await waitFor(() => expect(result.current.loading).toBe(false));
|
|
||||||
expect(result.current.items).toEqual([]);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
- **Don't put frontend tests in `src/__tests__/`.** No such directory exists in the codebase — `__tests__` only appears in the SP-catalog plan itself. Use the sibling rule (`useFoo.ts` → `useFoo.test.ts`).
|
|
||||||
- **Don't reach into `docs/prodfiles/` directly from a test.** Always copy to `backend/tests/fixtures/<name>.txt` first. The prodfiles directory is the source-of-truth archive and may be reorganized; the fixtures directory is the test-consumed surface and is stable.
|
|
||||||
- **Don't use wall-clock sleeps for timing.** A handful of legacy tests use `await new Promise((r) => setTimeout(r, 200))` (see `src/components/SearchBar.test.tsx:281,323,373` and `src/hooks/useSearch.test.ts:174`) — these are known flaky in CI. Use `vi.useFakeTimers()` + `vi.advanceTimersByTime(ms)` instead, or `waitFor(...)` from `@testing-library/react`.
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
|
|
||||||
- **`cyclone-spec`** — every SP-N spec lists test impact; load this when drafting or reviewing the spec to confirm fixture / `.test.tsx` implications for the increment.
|
|
||||||
- **`cyclone-edi`** — most backend tests cover parser + validator behavior; load when the increment touches an EDI parser or adds a validator rule (R200/R210/NPI Luhn/EIN/CAS).
|
|
||||||
- **`cyclone-tail`** — most frontend tests cover hook behavior (`useTailStream`, `useMergedTail`); load when the increment changes the wire format or adds a streaming hook.
|
|
||||||
- **`cyclone-store`** — write-path tests live here; load when the increment touches `store.py`, adds a new entity, or wires a new `<entity>_written` event.
|
|
||||||
- **`cyclone-api-router`** — endpoint tests live in `backend/tests/test_api_*.py`; load when the increment adds or changes an HTTP endpoint.
|
|
||||||
- **`cyclone-frontend-page`** — page-component tests live next to pages in `src/pages/*.test.tsx`; load when the increment adds or refactors a page.
|
|
||||||
- **`cyclone-cli`** — CLI smoke tests live in `backend/tests/test_cli_*.py`; load when the increment adds a CLI subcommand.
|
|
||||||
- **`superpowers:test-driven-development`** (global) — the upstream TDD workflow. Load first when starting any new feature increment; this skill only codifies the Cyclone-specific test layout on top of TDD.
|
|
||||||
@@ -51,52 +51,6 @@ VITE_API_BASE_URL=http://127.0.0.1:8000
|
|||||||
Without that, the UI falls back to its in-memory sample store via the
|
Without that, the UI falls back to its in-memory sample store via the
|
||||||
existing `data` adapter (parses are disabled).
|
existing `data` adapter (parses are disabled).
|
||||||
|
|
||||||
## Pipeline automation agent
|
|
||||||
|
|
||||||
For unattended round-trips (an agent / scheduler drops an 837P file
|
|
||||||
into the pipeline and waits for the 999 back from Gainwell), see the
|
|
||||||
`cyclone-pipeline` sibling project at `/Users/openclaw/dev/cyclone-pipeline/`.
|
|
||||||
It drives the full 7-phase state machine — preflight → browser upload →
|
|
||||||
parse verification → SFTP submit → TA1 wait → 999 wait → scan +
|
|
||||||
report — with structured JSON logs, crash-safe resume, and a
|
|
||||||
self-contained per-run folder under `./runs/`.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Single file
|
|
||||||
cyclone-pipeline run /path/to/axiscare-837p.txt
|
|
||||||
|
|
||||||
# Resume a crashed run
|
|
||||||
cyclone-pipeline resume 2026-06-21-1430-001
|
|
||||||
|
|
||||||
# On/after the following Monday, verify the 835 arrived
|
|
||||||
cyclone-pipeline check-835 2026-06-21-1430-001
|
|
||||||
```
|
|
||||||
|
|
||||||
The 835 is **not** waited for inline (it lands the following Monday on
|
|
||||||
the CO Medicaid payment cycle). See
|
|
||||||
[`cyclone-pipeline/README.md`](../cyclone-pipeline/README.md) for
|
|
||||||
install, embed-in-agent example, exit codes, and the report format.
|
|
||||||
|
|
||||||
## Skills
|
|
||||||
|
|
||||||
Cyclone ships 8 project-scoped AI-assistant skills under
|
|
||||||
[`.superpowers/skills/`](.superpowers/skills/). Each one codifies the
|
|
||||||
conventions for a major subsystem so the next contributor (human or
|
|
||||||
AI) gets the lay of the land automatically.
|
|
||||||
|
|
||||||
| Skill | Owns |
|
|
||||||
|-------|------|
|
|
||||||
| [`cyclone-spec`](.superpowers/skills/cyclone-spec/SKILL.md) | The SP-N spec → plan → implement → merge flow. |
|
|
||||||
| [`cyclone-tests`](.superpowers/skills/cyclone-tests/SKILL.md) | pytest + vitest fixture patterns, prodfiles drop-in. |
|
|
||||||
| [`cyclone-edi`](.superpowers/skills/cyclone-edi/SKILL.md) | EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1). |
|
|
||||||
| [`cyclone-tail`](.superpowers/skills/cyclone-tail/SKILL.md) | Live-tail streaming wire format and the hook triplet. |
|
|
||||||
| [`cyclone-store`](.superpowers/skills/cyclone-store/SKILL.md) | Store write-paths, pubsub event contract, SP21 split map. |
|
|
||||||
| [`cyclone-api-router`](.superpowers/skills/cyclone-api-router/SKILL.md) | FastAPI router conventions (`api_routers/`, `api_helpers.py`). |
|
|
||||||
| [`cyclone-frontend-page`](.superpowers/skills/cyclone-frontend-page/SKILL.md) | React page conventions (TanStack Query, drawer, URL state). |
|
|
||||||
| [`cyclone-cli`](.superpowers/skills/cyclone-cli/SKILL.md) | CLI subcommand conventions (`cli.py`, exit codes, smoke tests). |
|
|
||||||
|
|
||||||
Skills auto-load by description match — no slash command needed.
|
|
||||||
|
|
||||||
## Test
|
## Test
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -201,6 +155,7 @@ parses and rejects claims, the inbox reflects the new
|
|||||||
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
|
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
|
||||||
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
|
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
|
||||||
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
|
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
|
||||||
|
| POST | `/api/inbox/payer-rejected/acknowledge` | `{claim_ids: [...], actor: "..."}`. Bulk-acknowledge payer rejections. Idempotent. `200` with `transitioned` / `already_acked` / `not_found` / `not_rejected` counts. Writes an audit event per transition; never overwrites the underlying 277CA rejection. |
|
||||||
| GET | `/api/inbox/export.csv?lane=<lane>` | Streams CSV of the lane's rows. |
|
| GET | `/api/inbox/export.csv?lane=<lane>` | Streams CSV of the lane's rows. |
|
||||||
|
|
||||||
## Outbound 837 Serializer
|
## Outbound 837 Serializer
|
||||||
@@ -367,8 +322,9 @@ both be true for the same claim.
|
|||||||
|
|
||||||
The `audit_log` table is the canonical record of every state transition
|
The `audit_log` table is the canonical record of every state transition
|
||||||
the system has ever observed — claim lifecycle, reconciliation
|
the system has ever observed — claim lifecycle, reconciliation
|
||||||
decisions, config reloads, SFTP submissions, 277CA rejects. SP11 made
|
decisions, config reloads, SFTP submissions, 277CA rejects, Payer-Rejected
|
||||||
it tamper-evident: every row carries a SHA-256 hash of
|
acknowledgements, SQLCipher key rotations. SP11 made it tamper-evident:
|
||||||
|
every row carries a SHA-256 hash of
|
||||||
`(prev_hash || row_payload)`, forming a chain back to a genesis row.
|
`(prev_hash || row_payload)`, forming a chain back to a genesis row.
|
||||||
Any `INSERT`, `UPDATE`, or `DELETE` that breaks the chain is detectable
|
Any `INSERT`, `UPDATE`, or `DELETE` that breaks the chain is detectable
|
||||||
in a single walk.
|
in a single walk.
|
||||||
@@ -403,244 +359,39 @@ operator has created the Keychain entry on first run. See
|
|||||||
for the one-time setup recipe and the HIPAA Security Rule §164.312(a)(2)(iv)
|
for the one-time setup recipe and the HIPAA Security Rule §164.312(a)(2)(iv)
|
||||||
mapping.
|
mapping.
|
||||||
|
|
||||||
### Key rotation (SP15)
|
### Key rotation
|
||||||
|
|
||||||
`POST /api/admin/db/rotate-key` re-encrypts the SQLite file in place
|
The DB encryption key is rotated in place via `POST /api/admin/db/rotate-key`.
|
||||||
with a fresh SQLCipher key via `PRAGMA rekey`, then updates the
|
The handler:
|
||||||
Keychain so subsequent connections open with the new key. The
|
|
||||||
rotation holds a module-level `threading.Lock` (so two concurrent
|
|
||||||
requests can't race), disposes + rebuilds the SQLAlchemy engine with
|
|
||||||
`NullPool` (so SQLCipher's thread affinity is honored), and writes a
|
|
||||||
tamper-evident `db.key_rotated` audit event with old + new
|
|
||||||
fingerprints and the post-rotation table count. The old key is
|
|
||||||
retained in the `cyclone.db.key.previous` Keychain account for a
|
|
||||||
grace period so a botched rotation can be rolled back by hand.
|
|
||||||
|
|
||||||
## NPI checksum + Tax ID format validation (SP20)
|
1. Generates a fresh 256-bit CSPRNG key (`db_crypto.generate_db_key()`).
|
||||||
|
2. Persists the new key to the Keychain under the same
|
||||||
|
`cyclone.db.key` account (overwriting the old one).
|
||||||
|
3. Disposes the SQLAlchemy engine so pooled connections release the
|
||||||
|
file (SQLCipher refuses to `PRAGMA rekey` while another connection
|
||||||
|
holds the DB).
|
||||||
|
4. Opens with the old key, issues `PRAGMA rekey`, and verifies the
|
||||||
|
schema survived (table-count sanity check).
|
||||||
|
5. Rebuilds the engine with the new key.
|
||||||
|
6. Writes a `db.key_rotated` audit event carrying the SHA-256
|
||||||
|
fingerprints of the old and new keys (first 8 hex chars) plus the
|
||||||
|
post-rotation `table_count`.
|
||||||
|
|
||||||
Two pure local validators — no NPPES round-trip, no IRS e-file
|
When SQLCipher is enabled the engine uses SQLAlchemy's `NullPool`
|
||||||
lookup. Catches the 99% case (a typo at the end of an NPI, a letter in
|
instead of the default `QueuePool`. `QueuePool` returns connections to
|
||||||
an EIN, an extra digit, the reserved `00`/`07`/`8X` EIN prefix).
|
a shared queue that any thread can pull from, which breaks SQLCipher's
|
||||||
|
thread affinity. `NullPool` trades connection reuse for thread safety
|
||||||
|
— the only correct behavior under FastAPI's per-request threadpool.
|
||||||
|
|
||||||
| Check | Algorithm | Surface |
|
The rotation endpoint is serialized with a module-level
|
||||||
|-------|-----------|---------|
|
`threading.Lock` (one rotation in flight at a time), returns `409` if
|
||||||
| NPI | 10 digits where the last is a Luhn checksum over `80840 + body`. CMS-published example: body `123456789` → check `3` → valid NPI `1234567893`. | `cyclone.npi.is_valid_npi`, CLI `cyclone validate-npi <npi>`, API `GET /api/admin/validate-provider?npi=...`, validator rule `R021_npi_checksum` (warning) |
|
a rotation is already running, `400` if encryption is not enabled,
|
||||||
| Tax ID (EIN) | 9 digits, optional `XX-XXXXXXX` formatting. Rejects reserved prefixes `00`, `07`, `80`–`89` (IRS Pension Plan Branch). | `cyclone.npi.is_valid_tax_id`, CLI `cyclone validate-tax-id <ein>`, API `GET /api/admin/validate-provider?tax_id=...` |
|
and `503` with a `reason` on `PRAGMA rekey` or Keychain failure so the
|
||||||
|
operator can take the next step without parsing the traceback.
|
||||||
|
|
||||||
### CLI
|
| Method | Path | Notes |
|
||||||
|
| ------ | --------------------------------- | -------------------------------------------------------------- |
|
||||||
```bash
|
| POST | `/api/admin/db/rotate-key` | Rotate the SQLCipher key in place. Audit-logged. |
|
||||||
$ cyclone validate-npi 1234567893
|
|
||||||
OK: 10-digit NPI passes Luhn checksum
|
|
||||||
$ cyclone validate-npi 1234567890
|
|
||||||
INVALID: '1234567890' fails NPI Luhn checksum # exit 1
|
|
||||||
|
|
||||||
$ cyclone validate-tax-id 72-1587149
|
|
||||||
OK: 9-digit EIN (normalized=721587149)
|
|
||||||
$ cyclone validate-tax-id 00-1234567
|
|
||||||
INVALID: 9-digit EIN has reserved prefix (00); EIN is not assignable by IRS # exit 1
|
|
||||||
```
|
|
||||||
|
|
||||||
### API
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl 'http://localhost:8000/api/admin/validate-provider?npi=1234567893&tax_id=72-1587149'
|
|
||||||
# {
|
|
||||||
# "npi": {"valid": true, "skipped": false},
|
|
||||||
# "tax_id": {"valid": true, "skipped": false, "normalized": "721587149"}
|
|
||||||
# }
|
|
||||||
```
|
|
||||||
|
|
||||||
Both query params are optional; omitted fields return
|
|
||||||
`{"valid": null, "skipped": true}` so the caller can render "no check
|
|
||||||
performed" rather than treating absent input as a hard fail.
|
|
||||||
|
|
||||||
### Parser integration
|
|
||||||
|
|
||||||
The `R021_npi_checksum` rule runs alongside the existing `R020_npi_format`
|
|
||||||
in `cyclone.parsers.validator`. A billing-provider NPI that passes
|
|
||||||
R020 (right shape) but fails R021 (bad Luhn) is yielded as a
|
|
||||||
**warning**, not an error — operators sometimes ingest test fixtures
|
|
||||||
with placeholder NPIs (e.g. all-same-digit) and we don't want to block
|
|
||||||
that path. In strict mode (`--strict` / `?strict=true`) warnings are
|
|
||||||
promoted to errors.
|
|
||||||
|
|
||||||
### Files
|
|
||||||
|
|
||||||
* `cyclone/npi.py` — new module (~155 LOC).
|
|
||||||
* `cyclone.parsers.validator` — new `R021_npi_checksum` rule.
|
|
||||||
* `cyclone.api_routers.admin` — new `validate-provider` endpoint.
|
|
||||||
* `cyclone.cli` — `validate-npi` + `validate-tax-id` subcommands.
|
|
||||||
* Tests: `test_npi.py` (27), `test_api_validate_provider.py` (4),
|
|
||||||
`test_cli_validate.py` (8), `test_validator.py::test_r021_*` (4) —
|
|
||||||
**43 new tests**.
|
|
||||||
|
|
||||||
## Security hardening (SP19)
|
|
||||||
|
|
||||||
Three pure-ASGI middlewares sit in front of every FastAPI request.
|
|
||||||
They're sized for Cyclone's local-only posture — a misconfigured
|
|
||||||
Tailscale / ngrok bind, a buggy cron job uploading a 4 GB file, or a
|
|
||||||
port-scraper — not for hostile internet exposure.
|
|
||||||
|
|
||||||
| Middleware | Default | Override | Reject |
|
|
||||||
|------------|---------|----------|--------|
|
|
||||||
| `BodySizeLimitMiddleware` | 50 MB | `CYCLONE_MAX_BODY_BYTES` | `413 body_too_large` over Content-Length cap; chunked reads capped too |
|
|
||||||
| `RateLimitMiddleware` | 300 req/min/IP | `CYCLONE_RATE_LIMIT_PER_MIN` | `429 rate_limited` over the sliding window; `/api/health` exempt |
|
|
||||||
| `SecurityHeadersMiddleware` | always on | n/a | stamps `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: same-origin`, `Permissions-Policy`, `Content-Security-Policy: default-src 'none'; frame-ancestors 'none'` |
|
|
||||||
|
|
||||||
Every rejection (413 / 429) also writes a tamper-evident
|
|
||||||
`api.request_rejected` event into the SP11 audit chain so an
|
|
||||||
operator can correlate a misbehaving client with the SP18 JSON logs:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{"event_type":"api.request_rejected","entity_id":"POST /api/parse-837","payload":{"status":413,"reason":"body_too_large","path":"/api/parse-837","method":"POST","ip":"127.0.0.1"}}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Health probe
|
|
||||||
|
|
||||||
`GET /api/health` now returns a subsystem snapshot:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"status": "ok",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"db": {"ok": true},
|
|
||||||
"scheduler": {"running": true, "interval_s": 60, "sftp_block": "co_medicaid",
|
|
||||||
"backup_scheduler_running": false, "backup_interval_hours": 24.0},
|
|
||||||
"pubsub": {"parse_completed": 1, "batch_added": 1},
|
|
||||||
"batch": {"last_batch_id": 42, "last_batch_kind": "837P",
|
|
||||||
"last_batch_at": "2026-06-21T15:30:00.123Z",
|
|
||||||
"last_batch_filename": "TP11525703-837P-..."}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Returns `"status": "degraded"` if any subsystem reports an error —
|
|
||||||
the per-subsystem dict still surfaces so an operator can see which
|
|
||||||
one is unhappy. `/api/health` is rate-limit exempt so a load balancer
|
|
||||||
hammering the endpoint doesn't trip the limiter.
|
|
||||||
|
|
||||||
### Files
|
|
||||||
|
|
||||||
* `cyclone.security` — `BodySizeLimitMiddleware`,
|
|
||||||
`RateLimitMiddleware`, `SecurityHeadersMiddleware`, and
|
|
||||||
`get_health_snapshot()` (~330 LOC).
|
|
||||||
* `cyclone.api_routers.health` rewritten to use `get_health_snapshot()`.
|
|
||||||
* `cyclone.pubsub.EventBus.stats()` — new method that returns
|
|
||||||
per-kind subscriber counts.
|
|
||||||
* `tests/test_security.py` — 13 new tests.
|
|
||||||
|
|
||||||
## Structured logging (SP18)
|
|
||||||
|
|
||||||
Cyclone emits newline-delimited JSON to stderr by default — readable
|
|
||||||
by `jq`, Loki, Vector, ELK, or any log shipper. Every record carries
|
|
||||||
`ts` (ISO 8601 ms UTC), `level`, `logger`, `msg`, and an optional
|
|
||||||
`extra` dict for structured fields. Exceptions render as a
|
|
||||||
`traceback` string.
|
|
||||||
|
|
||||||
```
|
|
||||||
{"ts":"2026-06-21T15:30:00.123Z","level":"INFO","logger":"cyclone.scheduler","msg":"Processed inbound file","extra":{"input_filename":"ACK_999.x12","parser":"parse_999","claims":3}}
|
|
||||||
{"ts":"2026-06-21T15:30:01.456Z","level":"ERROR","logger":"cyclone.api","msg":"Backup create failed","extra":{"reason":"BackupError: passphrase mismatch"},"traceback":"Traceback ..."}
|
|
||||||
```
|
|
||||||
|
|
||||||
### PII scrubbing
|
|
||||||
|
|
||||||
A `PiiScrubber` filter is attached to the root logger and rewrites
|
|
||||||
obvious PHI patterns to `<redacted:npi>` / `<redacted:ssn>` /
|
|
||||||
`<redacted:dob>` / `<redacted:patient_name>` before any handler sees
|
|
||||||
the record:
|
|
||||||
|
|
||||||
| Pattern | Replacement |
|
|
||||||
|---------|-------------|
|
|
||||||
| `\b\d{10}\b` | `<redacted:npi>` |
|
|
||||||
| `\b\d{3}-\d{2}-\d{4}\b` or `\b\d{9}\b` at phrase boundary | `<redacted:ssn>` |
|
|
||||||
| `(dob\|date_of_birth)[:=]\s*\d{4}-\d{2}-\d{2}` | preserves the key, redacts the date |
|
|
||||||
| `patient_name=...` | full chunk redacted |
|
|
||||||
| Extras with key `dob`/`ssn`/`npi`/`patient_name`/… | value redacted regardless of shape |
|
|
||||||
|
|
||||||
The scrubber is conservative — bare ISO dates without a `dob=` prefix
|
|
||||||
are **not** scrubbed (they're too often timestamps or batch IDs), and
|
|
||||||
11+ digit numbers are left alone (they can't be NPIs). Disable for
|
|
||||||
forensic mode with `CYCLONE_LOG_NO_PII_SCRUB=1`.
|
|
||||||
|
|
||||||
### Knobs
|
|
||||||
|
|
||||||
| Env var | Default | Meaning |
|
|
||||||
|---------|---------|---------|
|
|
||||||
| `CYCLONE_LOG_LEVEL` | `INFO` | Root logger level. `DEBUG` for troubleshooting. |
|
|
||||||
| `CYCLONE_LOG_FILE` | (none) | Write to this path via `RotatingFileHandler` (10 MB × 5 backups). |
|
|
||||||
| `CYCLONE_LOG_JSON` | `true` | `false` uses the dev tabular formatter. |
|
|
||||||
| `CYCLONE_LOG_NO_PII_SCRUB` | (none) | `1` disables scrubbing. |
|
|
||||||
|
|
||||||
CLI:
|
|
||||||
|
|
||||||
```
|
|
||||||
cyclone --log-format=dev parse-837 sample.x12 --output-dir out/ # tabular for tail -f
|
|
||||||
cyclone --log-file=/var/log/cyclone.log backup create # JSON to rotating file
|
|
||||||
```
|
|
||||||
|
|
||||||
The `parse-837` / `parse-835` subcommands also accept `--log-level`
|
|
||||||
which re-runs `setup_logging()` so the per-invocation level overrides
|
|
||||||
the group default.
|
|
||||||
|
|
||||||
### Files
|
|
||||||
|
|
||||||
* `cyclone.logging_config` — `JsonFormatter`, `CycloneDevFormatter`,
|
|
||||||
`PiiScrubber`, `setup_logging()`.
|
|
||||||
* `tests/test_logging_formatter.py` (11), `test_logging_scrubber.py`
|
|
||||||
(13), `test_logging_setup.py` (10) — 34 new tests.
|
|
||||||
* `cyclone.api` lifespan calls `setup_logging()` first; the CLI
|
|
||||||
`main` group does the same.
|
|
||||||
|
|
||||||
## Encrypted Backups (SP17)
|
|
||||||
|
|
||||||
The BackupService takes an online consistent snapshot of the live
|
|
||||||
SQLite file via SQLite's `.backup()` API, encrypts the bytes with
|
|
||||||
AES-256-GCM, and writes a `.bin` + `.meta.json` pair into the backup
|
|
||||||
directory (default `~/.local/share/cyclone/backups/`). The encryption
|
|
||||||
key is derived from a separate passphrase in the macOS Keychain
|
|
||||||
(PBKDF2-HMAC-SHA256, 200,000 iterations, 16-byte salt persisted to
|
|
||||||
Keychain) — so a SQLCipher DB-key compromise does not unlock the
|
|
||||||
backups, and a backup-passphrase compromise does not unlock the live
|
|
||||||
DB. If neither is set, the service refuses (`BackupError`) rather than
|
|
||||||
silently writing plaintext.
|
|
||||||
|
|
||||||
| Method | Path | Purpose |
|
|
||||||
| ------ | ---- | ------- |
|
|
||||||
| POST | `/api/admin/backup/create` | Take an encrypted backup now. |
|
|
||||||
| GET | `/api/admin/backup/list` | List `db_backups` rows (newest first, filterable). |
|
|
||||||
| GET | `/api/admin/backup/status` | Counts, disk usage, last-run timestamp, scheduler snapshot. |
|
|
||||||
| POST | `/api/admin/backup/{id}/verify` | Decrypt + SHA-256 verify against the sidecar. |
|
|
||||||
| POST | `/api/admin/backup/{id}/restore/initiate` | First step: get `restore_token` + preview (fingerprints of backup vs live). |
|
|
||||||
| POST | `/api/admin/backup/{id}/restore/confirm` | Second step: dispose engine, copy decrypted DB, rebuild engine. |
|
|
||||||
| POST | `/api/admin/backup/prune` | Apply retention policy now. |
|
|
||||||
| POST | `/api/admin/backup/scheduler/{start,stop,tick}` | Operate the backup scheduler. |
|
|
||||||
|
|
||||||
Restore is two-step by design: an idle browser tab can't nuke the
|
|
||||||
live DB. The first call returns a one-shot 64-char hex
|
|
||||||
`restore_token` plus a side-by-side preview (`backup_db_fingerprint`,
|
|
||||||
`backup_table_count`, `current_db_fingerprint`, `current_table_count`).
|
|
||||||
The second call swaps the live engine only if the token matches
|
|
||||||
within a 5-minute TTL.
|
|
||||||
|
|
||||||
The scheduler (auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`)
|
|
||||||
ticks every `CYCLONE_BACKUP_INTERVAL_HOURS` (default 24), runs
|
|
||||||
`create_now` + `prune`, and writes audit events for each outcome
|
|
||||||
(`db.backup_created`, `db.backup_failed`, `db.backup_pruned`,
|
|
||||||
`db.backup_restored`). The CLI mirrors the API surface:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cyclone backup init-passphrase # one-time; interactive
|
|
||||||
cyclone backup create
|
|
||||||
cyclone backup list
|
|
||||||
cyclone backup verify <id>
|
|
||||||
cyclone backup restore <id> --yes
|
|
||||||
cyclone backup prune --yes
|
|
||||||
cyclone backup status
|
|
||||||
```
|
|
||||||
|
|
||||||
Retention defaults to 30 days (`CYCLONE_BACKUP_RETENTION_DAYS`). The
|
|
||||||
retention policy is best-effort: an operator who runs `cyclone
|
|
||||||
backup create` manually retains full control.
|
|
||||||
|
|
||||||
## SFTP Wire-Up (paramiko)
|
## SFTP Wire-Up (paramiko)
|
||||||
|
|
||||||
@@ -667,6 +418,29 @@ was a one-file change (`sftp_paramiko.py` replacing `sftp_stub.py`).
|
|||||||
the `paramiko` extras aren't installed so the test suite stays green on
|
the `paramiko` extras aren't installed so the test suite stays green on
|
||||||
Linux dev boxes.
|
Linux dev boxes.
|
||||||
|
|
||||||
|
## Batches, Reconciliation, and Activity
|
||||||
|
|
||||||
|
These read/write endpoints are the core operator surface for browsing
|
||||||
|
the parsed record and acting on matches. They predate the per-SP
|
||||||
|
endpoint reference sections in the **Roadmap** and are listed here in
|
||||||
|
one place so the route inventory stays discoverable.
|
||||||
|
|
||||||
|
| Method | Path | Notes |
|
||||||
|
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
||||||
|
| GET | `/api/batches` | All parsed batches, newest first. `?limit=` (1–1000, default 100). |
|
||||||
|
| GET | `/api/batches/{batch_id}` | One batch detail (envelope, claims / remits, validation summary). |
|
||||||
|
| GET | `/api/batch-diff?a=<id>&b=<id>` | Side-by-side diff of two batches: `added` / `removed` / `changed` claims + envelope metadata for each. Both query params required. |
|
||||||
|
| GET | `/api/reconciliation/unmatched` | `{"claims": [...], "remittances": [...]}` — every Claim with no paired Remittance and vice versa. The two lists are always present (empty list, never absent) so the UI can index unconditionally. |
|
||||||
|
| POST | `/api/reconciliation/match` | Body `{claim_id, remit_id}`. Manually pair a claim with a remit. `400` on missing ids, `404` on unknown id, `409` on already-matched or terminal-state claim. |
|
||||||
|
| POST | `/api/reconciliation/unmatch` | Body `{claim_id}`. Remove the current match and reset the claim to `submitted`. `404` on unknown id, `409` on no current match. |
|
||||||
|
| GET | `/api/providers` | Distinct providers across parsed claims. `?npi=`, `?state=`, `?limit=`, `?offset=`. Distinct from `/api/config/providers/{npi}` (SP9 config table) — this endpoint surfaces providers derived from the parsed claim stream. |
|
||||||
|
| GET | `/api/activity` | Recent activity events. `?kind=`, `?since=`, `?limit=` (1–500, default 200). Powers the Activity page; the streaming counterpart `/api/activity/stream` is documented under **Live updates**. |
|
||||||
|
|
||||||
|
The per-SP endpoint blocks at the bottom of the **Roadmap** cover the
|
||||||
|
SP-specific routes (parse, 999/TA1/277CA, Inbox, claim drawer, line
|
||||||
|
reconciliation, outbound 837, multi-payer config, audit log, 277CA,
|
||||||
|
key rotation, payer-rejected acknowledge).
|
||||||
|
|
||||||
## Persistence
|
## Persistence
|
||||||
|
|
||||||
Parsed batches, claims, remittances, matches, 277CA rejections,
|
Parsed batches, claims, remittances, matches, 277CA rejections,
|
||||||
@@ -700,8 +474,12 @@ backup API).
|
|||||||
.
|
.
|
||||||
├── backend/
|
├── backend/
|
||||||
│ ├── src/cyclone/
|
│ ├── src/cyclone/
|
||||||
│ │ ├── api.py # FastAPI app, GET + parse routes, /api/{resource}/stream
|
│ │ ├── api.py # FastAPI app + parse routes; mounts api_routers/* sub-apps
|
||||||
│ │ ├── api_helpers.py # NDJSON / content-negotiation / live-tail helpers
|
│ │ ├── api_helpers.py # NDJSON / content-negotiation / live-tail helpers
|
||||||
|
│ │ ├── api_routers/ # FastAPI APIRouters extracted from api.py
|
||||||
|
│ │ │ ├── health.py # GET /api/health
|
||||||
|
│ │ │ ├── acks.py # GET /api/acks, /api/acks/{id}
|
||||||
|
│ │ │ └── ta1_acks.py # GET /api/ta1-acks, /api/ta1-acks/{id}
|
||||||
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
|
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
|
||||||
│ │ ├── store.py # CycloneStore, mappers, publish-on-write
|
│ │ ├── store.py # CycloneStore, mappers, publish-on-write
|
||||||
│ │ ├── db.py # SQLAlchemy engine, session factory, ORM models
|
│ │ ├── db.py # SQLAlchemy engine, session factory, ORM models
|
||||||
@@ -758,7 +536,7 @@ backup API).
|
|||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
Sub-projects 2 through 19 are **shipped**. See the [completeness
|
Sub-projects 2 through 15 are **shipped**. See the [completeness
|
||||||
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
|
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
|
||||||
the honest gap analysis against the industry definition of a HIPAA
|
the honest gap analysis against the industry definition of a HIPAA
|
||||||
clearinghouse — the short version is that the local-only,
|
clearinghouse — the short version is that the local-only,
|
||||||
@@ -769,70 +547,6 @@ scope.
|
|||||||
|
|
||||||
Shipped sub-projects (most recent first):
|
Shipped sub-projects (most recent first):
|
||||||
|
|
||||||
- **Sub-project 22 (shipped) — Pipeline automation agent.** A
|
|
||||||
sibling project at `/Users/openclaw/dev/cyclone-pipeline` that
|
|
||||||
drives the full 7-phase round-trip (preflight → browser upload →
|
|
||||||
parse verify → SFTP submit → TA1 wait → 999 wait → scan + report)
|
|
||||||
with crash-safe resume, structured JSON logging, idempotency
|
|
||||||
dedup, and a per-run report. Pure Python 3.11+ (httpx, Playwright,
|
|
||||||
Click, pydantic v2, structlog). The 835 is not waited for inline —
|
|
||||||
it lands the following Monday — and is verified by a separate
|
|
||||||
`check-835` subcommand. Embeddable as a library for OpenClaw / Nora
|
|
||||||
agent integration. See
|
|
||||||
[Pipeline automation agent](#pipeline-automation-agent) above.
|
|
||||||
- **Sub-project 19 (shipped) — Security hardening + health probe.**
|
|
||||||
Three pure-ASGI middlewares (`BodySizeLimitMiddleware`,
|
|
||||||
`RateLimitMiddleware`, `SecurityHeadersMiddleware`) close the
|
|
||||||
completeness-review gaps §3.1.4 (no body/rate limits) and §3.1.25
|
|
||||||
(no CSP / security headers). 413/429 rejections emit a
|
|
||||||
tamper-evident `api.request_rejected` audit event (SP11 chain).
|
|
||||||
`/api/health` is now a rich subsystem snapshot — DB connectivity,
|
|
||||||
MFT scheduler state, backup scheduler state, live pubsub
|
|
||||||
subscriber counts, last batch id + timestamp. See
|
|
||||||
[Security hardening (SP19)](#security-hardening-sp19) below.
|
|
||||||
- **Sub-project 20 (shipped) — NPI checksum + Tax ID format validation.**
|
|
||||||
Pure local validators (`cyclone.npi`) — no NPPES round-trip, no IRS
|
|
||||||
e-file lookup. Catches the 99% typo case at parse time. NPI uses
|
|
||||||
CMS-published Luhn over `80840 + body` (example: `1234567893` is
|
|
||||||
valid). EIN rejects reserved prefixes (`00`, `07`, `80`–`89`).
|
|
||||||
Surface: `cyclone validate-npi` / `validate-tax-id` CLI subcommands,
|
|
||||||
`GET /api/admin/validate-provider`, new `R021_npi_checksum`
|
|
||||||
validator rule (warning, not error — placeholder NPIs in test
|
|
||||||
fixtures shouldn't block ingest). See
|
|
||||||
[NPI checksum + Tax ID format validation (SP20)](#npi-checksum--tax-id-format-validation-sp20)
|
|
||||||
below.
|
|
||||||
- **Sub-project 18 (shipped) — Structured JSON logging.** All logs
|
|
||||||
emitted by the API, CLI, scheduler tick loop, and backup service
|
|
||||||
flow through a `JsonFormatter` (newline-delimited JSON, ISO-8601 ms
|
|
||||||
timestamps) by default. A `PiiScrubber` filter redacts obvious PHI
|
|
||||||
(NPIs, SSNs, DOBs, patient names) from message + extras — both via
|
|
||||||
inline patterns (`npi 1881068062`) and via PHI-keyed extras
|
|
||||||
(`extra={"dob": "1980-04-12"}`). Configurable via env vars
|
|
||||||
(`CYCLONE_LOG_LEVEL`, `CYCLONE_LOG_FILE`, `CYCLONE_LOG_JSON`,
|
|
||||||
`CYCLONE_LOG_NO_PII_SCRUB`) and CLI flags
|
|
||||||
(`--log-format=json|dev`, `--log-file=…`); a tabular `CycloneDevFormatter`
|
|
||||||
is the opt-out for `tail -f` in dev. See
|
|
||||||
[Structured logging](#structured-logging-sp18) below.
|
|
||||||
- **Sub-project 17 (shipped) — Encrypted DB backups.** Automated
|
|
||||||
encrypted backups via AES-256-GCM (PBKDF2-HMAC-SHA256, 200k iters).
|
|
||||||
The operator sets a separate passphrase in the macOS Keychain
|
|
||||||
(`cyclone backup init-passphrase`); if missing, the service falls
|
|
||||||
back to deriving from the SQLCipher DB key with a WARNING. Online
|
|
||||||
backups via SQLite `.backup()`, two-step restore (`initiate` →
|
|
||||||
`confirm` with one-shot 64-char hex token), retention pruning with
|
|
||||||
a 30-day default, and a tamper-evident audit chain (`db.backup_created`,
|
|
||||||
`db.backup_failed`, `db.backup_pruned`, `db.backup_restored`,
|
|
||||||
`db.backup_passphrase_set`). Backup scheduler ticks every 24h
|
|
||||||
(configurable); auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`.
|
|
||||||
Seven admin endpoints + six CLI subcommands. See
|
|
||||||
[Encrypted Backups](#encrypted-backups) below.
|
|
||||||
- **Sub-project 16 (shipped) — Live MFT polling scheduler.** asyncio
|
|
||||||
background loop polls the Gainwell MFT inbound path, downloads
|
|
||||||
new files, and routes them through the right parser (999 / 835 /
|
|
||||||
277CA / TA1). Idempotent (re-ticks skip already-processed files
|
|
||||||
via the new `processed_inbound_files` table). Crash-safe (per-file
|
|
||||||
try/except so a bad file doesn't stop the loop). Five admin
|
|
||||||
endpoints (`/api/admin/scheduler/{status,start,stop,tick,processed-files}`).
|
|
||||||
- **Sub-project 15 (shipped) — SQLCipher key rotation.** In-place
|
- **Sub-project 15 (shipped) — SQLCipher key rotation.** In-place
|
||||||
rotation via `PRAGMA rekey`, serialized through a module-level
|
rotation via `PRAGMA rekey`, serialized through a module-level
|
||||||
`threading.Lock` and a SQLAlchemy `NullPool` to keep SQLCipher
|
`threading.Lock` and a SQLAlchemy `NullPool` to keep SQLCipher
|
||||||
@@ -1114,6 +828,34 @@ the one-time setup recipe.
|
|||||||
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
|
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
|
||||||
SFTP credentials are fetched from the macOS Keychain at call time.
|
SFTP credentials are fetched from the macOS Keychain at call time.
|
||||||
|
|
||||||
|
### SP14 endpoints (5-lane Inbox UI + acknowledge)
|
||||||
|
|
||||||
|
- `GET /api/inbox/lanes` — same shape as SP6; the `payer_rejected`
|
||||||
|
lane payload now also includes
|
||||||
|
`payer_rejected_acknowledged_at` + `payer_rejected_acknowledged_actor`
|
||||||
|
per row so the UI can badge acknowledged claims (forward-compat for
|
||||||
|
a future "Recently acknowledged" view).
|
||||||
|
- `POST /api/inbox/payer-rejected/acknowledge` — bulk-acknowledge
|
||||||
|
Payer-Rejected claims. Body: `{claim_ids: [...], actor: "..."}`.
|
||||||
|
Idempotent. Response: `200` with
|
||||||
|
`{transitioned, already_acked, not_found, not_rejected}`. Writes an
|
||||||
|
`inbox.payer_rejected_acknowledged` audit event per transition.
|
||||||
|
Acknowledgement hides a claim from the lane but never overwrites
|
||||||
|
`payer_rejected_at` / `payer_rejected_reason` /
|
||||||
|
`payer_rejected_by_277ca_id` — the original 277CA evidence stays
|
||||||
|
intact in the audit log.
|
||||||
|
|
||||||
|
### SP15 endpoints (SQLCipher key rotation)
|
||||||
|
|
||||||
|
- `POST /api/admin/db/rotate-key` — rotate the SQLCipher DB key in
|
||||||
|
place. Generates a fresh 256-bit key, writes it to the Keychain
|
||||||
|
(overwriting `cyclone.db.key`), disposes the engine, issues
|
||||||
|
`PRAGMA rekey`, verifies the schema, rebuilds the engine. Writes
|
||||||
|
a `db.key_rotated` audit event with old + new key fingerprints
|
||||||
|
and `table_count`. Returns `409` when a rotation is already in
|
||||||
|
flight, `400` when encryption is not enabled, `503` with a
|
||||||
|
`reason` on `PRAGMA rekey` or Keychain failure.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
No license file yet; this is internal-use software. Add a `LICENSE` file
|
No license file yet; this is internal-use software. Add a `LICENSE` file
|
||||||
|
|||||||
-233
@@ -1,233 +0,0 @@
|
|||||||
// UI/UX Score Loop — pass 1 driver.
|
|
||||||
// Loads each route at three sizes in Chrome Canary, captures screenshots,
|
|
||||||
// logs console errors, runs a small interaction probe per flow, and
|
|
||||||
// writes a JSON report. Does not modify any source files.
|
|
||||||
|
|
||||||
import puppeteer from "puppeteer-core";
|
|
||||||
import { mkdir, writeFile } from "node:fs/promises";
|
|
||||||
|
|
||||||
const BASE = "http://127.0.0.1:5173";
|
|
||||||
const SHOTS = "/tmp/cyclone-uiux/shots";
|
|
||||||
const REPORT = "/tmp/cyclone-uiux/report.json";
|
|
||||||
|
|
||||||
const SIZES = [
|
|
||||||
{ name: "desktop", w: 1440, h: 900 },
|
|
||||||
{ name: "tablet", w: 768, h: 1024 },
|
|
||||||
{ name: "mobile", w: 375, h: 812 },
|
|
||||||
];
|
|
||||||
|
|
||||||
// Routes to load. path = the route; name = the flow label; ready = a
|
|
||||||
// selector we wait for to consider the page "rendered".
|
|
||||||
const FLOWS = [
|
|
||||||
{ name: "dashboard", path: "/", ready: "aside nav, h1, h2" },
|
|
||||||
{ name: "upload", path: "/upload", ready: "section[aria-label='File upload']" },
|
|
||||||
{ name: "inbox", path: "/inbox", ready: "main, section[aria-label='Queue summary']" },
|
|
||||||
{ name: "claims", path: "/claims", ready: "table, [data-testid='claims-page-body']" },
|
|
||||||
{ name: "claims-denied", path: "/claims?status=denied", ready: "table, [data-testid='claims-page-body']" },
|
|
||||||
{ name: "remittances", path: "/remittances", ready: "main, table" },
|
|
||||||
{ name: "providers", path: "/providers", ready: "main, table" },
|
|
||||||
{ name: "reconciliation",path: "/reconciliation", ready: "main" },
|
|
||||||
{ name: "acks", path: "/acks", ready: "main, table" },
|
|
||||||
{ name: "batches", path: "/batches", ready: "main, table" },
|
|
||||||
{ name: "batch-diff", path: "/batch-diff", ready: "main" },
|
|
||||||
{ name: "activity", path: "/activity", ready: "main" },
|
|
||||||
{ name: "404", path: "/does-not-exist", ready: "main" },
|
|
||||||
];
|
|
||||||
|
|
||||||
async function setupViewports(browser) {
|
|
||||||
const pages = [];
|
|
||||||
for (const size of SIZES) {
|
|
||||||
const page = await browser.newPage();
|
|
||||||
await page.setViewport({ width: size.w, height: size.h, deviceScaleFactor: 1 });
|
|
||||||
pages.push({ page, size });
|
|
||||||
}
|
|
||||||
return pages;
|
|
||||||
}
|
|
||||||
|
|
||||||
async function probeFlow(page, flow) {
|
|
||||||
const consoleErrors = [];
|
|
||||||
const pageErrors = [];
|
|
||||||
const failedRequests = [];
|
|
||||||
|
|
||||||
const onConsole = (msg) => {
|
|
||||||
if (msg.type() === "error") consoleErrors.push(msg.text());
|
|
||||||
};
|
|
||||||
const onPageError = (err) => pageErrors.push(err.message);
|
|
||||||
const onRequestFailed = (req) => failedRequests.push(`${req.method()} ${req.url()} :: ${req.failure()?.errorText}`);
|
|
||||||
|
|
||||||
page.on("console", onConsole);
|
|
||||||
page.on("pageerror", onPageError);
|
|
||||||
page.on("requestfailed", onRequestFailed);
|
|
||||||
|
|
||||||
const t0 = Date.now();
|
|
||||||
let rendered = false;
|
|
||||||
let readyError = null;
|
|
||||||
try {
|
|
||||||
await page.goto(`${BASE}${flow.path}`, { waitUntil: "networkidle2", timeout: 15000 });
|
|
||||||
if (flow.ready) {
|
|
||||||
try {
|
|
||||||
await page.waitForSelector(flow.ready, { timeout: 5000 });
|
|
||||||
rendered = true;
|
|
||||||
} catch (e) {
|
|
||||||
readyError = e.message;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
rendered = true;
|
|
||||||
}
|
|
||||||
} catch (e) {
|
|
||||||
readyError = e.message;
|
|
||||||
}
|
|
||||||
const loadMs = Date.now() - t0;
|
|
||||||
|
|
||||||
page.off("console", onConsole);
|
|
||||||
page.off("pageerror", onPageError);
|
|
||||||
page.off("requestfailed", onRequestFailed);
|
|
||||||
|
|
||||||
return { rendered, loadMs, readyError, consoleErrors, pageErrors, failedRequests };
|
|
||||||
}
|
|
||||||
|
|
||||||
async function probeInteractions(page, flow) {
|
|
||||||
const findings = [];
|
|
||||||
// Generic a11y / structural probes per flow.
|
|
||||||
try {
|
|
||||||
// Sidebar visible? (md+ shows it; < md hides it)
|
|
||||||
const aside = await page.$("aside");
|
|
||||||
findings.push({ check: "sidebar-present", pass: !!aside });
|
|
||||||
} catch (e) {
|
|
||||||
findings.push({ check: "sidebar-present", pass: false, err: e.message });
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
// Top bar present?
|
|
||||||
const main = await page.$("main#main-content");
|
|
||||||
findings.push({ check: "main-present", pass: !!main });
|
|
||||||
} catch (e) {
|
|
||||||
findings.push({ check: "main-present", pass: false, err: e.message });
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
// H1 or page heading?
|
|
||||||
const heading = await page.evaluate(() => {
|
|
||||||
const h = document.querySelector("h1, h2");
|
|
||||||
return h ? h.textContent?.trim().slice(0, 60) : null;
|
|
||||||
});
|
|
||||||
findings.push({ check: "heading-present", pass: !!heading, value: heading });
|
|
||||||
} catch (e) {
|
|
||||||
findings.push({ check: "heading-present", pass: false, err: e.message });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Flow-specific probes.
|
|
||||||
if (flow.name === "claims" || flow.name === "claims-denied") {
|
|
||||||
try {
|
|
||||||
const chips = await page.$$("[role='radio'], button[role='radio']");
|
|
||||||
findings.push({ check: "status-chips", pass: chips.length >= 1, count: chips.length });
|
|
||||||
} catch (e) {
|
|
||||||
findings.push({ check: "status-chips", pass: false, err: e.message });
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
const search = await page.$("input[placeholder*='Search']");
|
|
||||||
findings.push({ check: "search-input", pass: !!search });
|
|
||||||
} catch (e) {
|
|
||||||
findings.push({ check: "search-input", pass: false, err: e.message });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (flow.name === "upload") {
|
|
||||||
try {
|
|
||||||
const dropzone = await page.$("section[aria-label='File upload']");
|
|
||||||
findings.push({ check: "dropzone-present", pass: !!dropzone });
|
|
||||||
const selects = await page.$$("button[role='combobox']");
|
|
||||||
findings.push({ check: "payer-kind-selects", pass: selects.length >= 2, count: selects.length });
|
|
||||||
} catch (e) {
|
|
||||||
findings.push({ check: "upload-elements", pass: false, err: e.message });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (flow.name === "inbox") {
|
|
||||||
try {
|
|
||||||
const lanes = await page.$$("main > div > div");
|
|
||||||
findings.push({ check: "lane-cards", pass: lanes.length >= 1, count: lanes.length });
|
|
||||||
} catch (e) {
|
|
||||||
findings.push({ check: "lane-cards", pass: false, err: e.message });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (flow.name === "404") {
|
|
||||||
try {
|
|
||||||
const text = await page.evaluate(() => document.body.innerText);
|
|
||||||
findings.push({ check: "404-text", pass: text.includes("404") || text.toLowerCase().includes("doesn't exist") });
|
|
||||||
} catch (e) {
|
|
||||||
findings.push({ check: "404-text", pass: false, err: e.message });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return findings;
|
|
||||||
}
|
|
||||||
|
|
||||||
async function main() {
|
|
||||||
await mkdir(SHOTS, { recursive: true });
|
|
||||||
const browser = await puppeteer.launch({
|
|
||||||
executablePath: "/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary",
|
|
||||||
headless: "new",
|
|
||||||
args: ["--no-sandbox", "--disable-dev-shm-usage"],
|
|
||||||
});
|
|
||||||
const startedAt = new Date().toISOString();
|
|
||||||
const results = [];
|
|
||||||
|
|
||||||
for (const size of SIZES) {
|
|
||||||
const page = await browser.newPage();
|
|
||||||
await page.setViewport({ width: size.w, height: size.h, deviceScaleFactor: 1 });
|
|
||||||
|
|
||||||
for (const flow of FLOWS) {
|
|
||||||
const probe = await probeFlow(page, flow);
|
|
||||||
const interactions = await probeInteractions(page, flow);
|
|
||||||
const shot = `${SHOTS}/${flow.name}--${size.name}.png`;
|
|
||||||
try {
|
|
||||||
await page.screenshot({ path: shot, fullPage: false });
|
|
||||||
} catch (e) {
|
|
||||||
// ignore — recording in result
|
|
||||||
}
|
|
||||||
results.push({
|
|
||||||
flow: flow.name,
|
|
||||||
path: flow.path,
|
|
||||||
size: size.name,
|
|
||||||
viewport: { w: size.w, h: size.h },
|
|
||||||
probe,
|
|
||||||
interactions,
|
|
||||||
shot,
|
|
||||||
});
|
|
||||||
console.log(
|
|
||||||
`${size.name.padEnd(7)} ${flow.name.padEnd(20)} ` +
|
|
||||||
`render=${probe.rendered} load=${probe.loadMs}ms ` +
|
|
||||||
`consoleErr=${probe.consoleErrors.length} pageErr=${probe.pageErrors.length}`
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
await page.close();
|
|
||||||
}
|
|
||||||
|
|
||||||
await browser.close();
|
|
||||||
|
|
||||||
// Aggregate.
|
|
||||||
const summary = {
|
|
||||||
startedAt,
|
|
||||||
endedAt: new Date().toISOString(),
|
|
||||||
flows: results.length,
|
|
||||||
sizes: SIZES.map((s) => s.name),
|
|
||||||
renderedOk: results.filter((r) => r.probe.rendered).length,
|
|
||||||
withConsoleErrors: results.filter((r) => r.probe.consoleErrors.length > 0).length,
|
|
||||||
withPageErrors: results.filter((r) => r.probe.pageErrors.length > 0).length,
|
|
||||||
withFailedRequests: results.filter((r) => r.probe.failedRequests.length > 0).length,
|
|
||||||
results,
|
|
||||||
};
|
|
||||||
await writeFile(REPORT, JSON.stringify(summary, null, 2));
|
|
||||||
console.log("\nSummary:", JSON.stringify({
|
|
||||||
flows: summary.flows,
|
|
||||||
renderedOk: summary.renderedOk,
|
|
||||||
withConsoleErrors: summary.withConsoleErrors,
|
|
||||||
withPageErrors: summary.withPageErrors,
|
|
||||||
withFailedRequests: summary.withFailedRequests,
|
|
||||||
}, null, 2));
|
|
||||||
console.log("\nReport:", REPORT);
|
|
||||||
console.log("Shots:", SHOTS);
|
|
||||||
}
|
|
||||||
|
|
||||||
main().catch((e) => {
|
|
||||||
console.error("FATAL", e);
|
|
||||||
process.exit(1);
|
|
||||||
});
|
|
||||||
+17
-1091
File diff suppressed because it is too large
Load Diff
@@ -93,40 +93,19 @@ def ndjson_stream_list(
|
|||||||
}) + "\n"
|
}) + "\n"
|
||||||
|
|
||||||
|
|
||||||
def ndjson_stream_837(
|
def ndjson_stream_837(result: ParseResult) -> Iterator[bytes]:
|
||||||
result: ParseResult, batch_id: str | None = None,
|
"""Yield one JSON object per line: envelope → claims → summary."""
|
||||||
) -> Iterator[bytes]:
|
|
||||||
"""Yield one JSON object per line: envelope → claims → summary.
|
|
||||||
|
|
||||||
The ``batch_id`` is the server-side UUID assigned by the persistence
|
|
||||||
layer when the batch is ingested; the JSON response path exposes it
|
|
||||||
as the top-level ``batch_id`` field, but the NDJSON stream needs it
|
|
||||||
inline on the summary event so streaming clients can call
|
|
||||||
batch-scoped endpoints (``/api/batches/{id}/export-837``, …) without
|
|
||||||
a separate ``GET /api/batches`` round-trip. When ``batch_id`` is not
|
|
||||||
supplied the summary omits the field, preserving backward compat
|
|
||||||
with clients that don't expect it.
|
|
||||||
"""
|
|
||||||
envelope_obj = (
|
envelope_obj = (
|
||||||
result.envelope.model_dump() if result.envelope is not None else None
|
result.envelope.model_dump() if result.envelope is not None else None
|
||||||
)
|
)
|
||||||
yield (json.dumps({"type": "envelope", "data": envelope_obj}) + "\n").encode("utf-8")
|
yield (json.dumps({"type": "envelope", "data": envelope_obj}) + "\n").encode("utf-8")
|
||||||
for claim in result.claims:
|
for claim in result.claims:
|
||||||
yield (json.dumps({"type": "claim", "data": json.loads(claim.model_dump_json())}) + "\n").encode("utf-8")
|
yield (json.dumps({"type": "claim", "data": json.loads(claim.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
summary_data = json.loads(result.summary.model_dump_json())
|
yield (json.dumps({"type": "summary", "data": json.loads(result.summary.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
if batch_id is not None:
|
|
||||||
summary_data["batch_id"] = batch_id
|
|
||||||
yield (json.dumps({"type": "summary", "data": summary_data}) + "\n").encode("utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def ndjson_stream_835(
|
def ndjson_stream_835(result: ParseResult835) -> Iterator[bytes]:
|
||||||
result: ParseResult835, batch_id: str | None = None,
|
"""Yield one JSON object per line: envelope → financial → trace → payer → payee → claim_payments → summary."""
|
||||||
) -> Iterator[bytes]:
|
|
||||||
"""Yield one JSON object per line: envelope → financial → trace → payer → payee → claim_payments → summary.
|
|
||||||
|
|
||||||
See ``ndjson_stream_837`` for why the optional ``batch_id`` is
|
|
||||||
merged into the summary event.
|
|
||||||
"""
|
|
||||||
yield (json.dumps({"type": "envelope", "data": json.loads(result.envelope.model_dump_json())}) + "\n").encode("utf-8")
|
yield (json.dumps({"type": "envelope", "data": json.loads(result.envelope.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
yield (json.dumps({"type": "financial_info", "data": json.loads(result.financial_info.model_dump_json())}) + "\n").encode("utf-8")
|
yield (json.dumps({"type": "financial_info", "data": json.loads(result.financial_info.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
yield (json.dumps({"type": "trace", "data": json.loads(result.trace.model_dump_json())}) + "\n").encode("utf-8")
|
yield (json.dumps({"type": "trace", "data": json.loads(result.trace.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
@@ -134,10 +113,7 @@ def ndjson_stream_835(
|
|||||||
yield (json.dumps({"type": "payee", "data": json.loads(result.payee.model_dump_json())}) + "\n").encode("utf-8")
|
yield (json.dumps({"type": "payee", "data": json.loads(result.payee.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
for claim in result.claims:
|
for claim in result.claims:
|
||||||
yield (json.dumps({"type": "claim_payment", "data": json.loads(claim.model_dump_json())}) + "\n").encode("utf-8")
|
yield (json.dumps({"type": "claim_payment", "data": json.loads(claim.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
summary_data = json.loads(result.summary.model_dump_json())
|
yield (json.dumps({"type": "summary", "data": json.loads(result.summary.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
if batch_id is not None:
|
|
||||||
summary_data["batch_id"] = batch_id
|
|
||||||
yield (json.dumps({"type": "summary", "data": summary_data}) + "\n").encode("utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def strict_rewrite_837(result: ParseResult) -> ParseResult:
|
def strict_rewrite_837(result: ParseResult) -> ParseResult:
|
||||||
|
|||||||
@@ -1,60 +0,0 @@
|
|||||||
"""``/api/admin/validate-provider`` — NPI + Tax ID liveness probe (SP20).
|
|
||||||
|
|
||||||
Pure read-only endpoint that runs the local NPI Luhn + EIN format checks
|
|
||||||
without touching the DB. Useful for:
|
|
||||||
- operators vetting a new provider before adding them to the registry,
|
|
||||||
- the dashboard's "validate" button on a Provider row,
|
|
||||||
- smoke-testing the SP20 checks after a deploy.
|
|
||||||
|
|
||||||
Both query params are optional; omitting one just skips that check.
|
|
||||||
Returns the per-check result dict so the caller can distinguish "bad
|
|
||||||
format" from "bad checksum".
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from fastapi import APIRouter, Query
|
|
||||||
|
|
||||||
from cyclone.npi import is_valid_npi, is_valid_tax_id, normalize_tax_id
|
|
||||||
|
|
||||||
|
|
||||||
router = APIRouter()
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/api/admin/validate-provider")
|
|
||||||
def validate_provider(
|
|
||||||
npi: str | None = Query(None, description="10-digit NPI to validate (Luhn checksum)"),
|
|
||||||
tax_id: str | None = Query(None, description="9-digit EIN to validate (format + reserved-prefix check)"),
|
|
||||||
) -> dict:
|
|
||||||
"""Return per-field validation results for ``npi`` and ``tax_id``.
|
|
||||||
|
|
||||||
Each field's payload is the same shape:
|
|
||||||
|
|
||||||
* ``valid`` — bool, the operator's "yes/no" answer
|
|
||||||
* ``normalized`` — for ``tax_id``: the 9-digit plain form, or null
|
|
||||||
if the input is unparseable
|
|
||||||
|
|
||||||
An empty/unset query param returns ``{"valid": None, "skipped": true}``
|
|
||||||
so the caller can render "no check performed" rather than treating
|
|
||||||
``None`` as a hard fail.
|
|
||||||
"""
|
|
||||||
result: dict = {}
|
|
||||||
|
|
||||||
if npi is None or npi == "":
|
|
||||||
result["npi"] = {"valid": None, "skipped": True}
|
|
||||||
else:
|
|
||||||
result["npi"] = {
|
|
||||||
"valid": is_valid_npi(npi),
|
|
||||||
"skipped": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
if tax_id is None or tax_id == "":
|
|
||||||
result["tax_id"] = {"valid": None, "skipped": True, "normalized": None}
|
|
||||||
else:
|
|
||||||
normalized = normalize_tax_id(tax_id)
|
|
||||||
result["tax_id"] = {
|
|
||||||
"valid": is_valid_tax_id(tax_id),
|
|
||||||
"skipped": False,
|
|
||||||
"normalized": normalized,
|
|
||||||
}
|
|
||||||
|
|
||||||
return result
|
|
||||||
@@ -1,40 +1,17 @@
|
|||||||
"""``GET /api/health`` — liveness + readiness probe.
|
"""``GET /api/health`` — liveness probe.
|
||||||
|
|
||||||
SP19 expanded the shallow ``{"status": "ok", "version": ...}`` probe
|
Returns the package version so an operator can confirm which build is
|
||||||
into a snapshot of every subsystem:
|
serving requests without poking the filesystem.
|
||||||
|
|
||||||
* **db** — can we open a session and run ``SELECT 1``?
|
|
||||||
* **scheduler** — is the MFT polling loop running? same for the
|
|
||||||
backup scheduler.
|
|
||||||
* **pubsub** — current subscriber counts per event kind.
|
|
||||||
* **batch** — most recent batch id + timestamp.
|
|
||||||
|
|
||||||
Returns ``status="ok"`` only when every subsystem is healthy.
|
|
||||||
``status="degraded"`` if any subsystem is unhappy but the API
|
|
||||||
itself is responsive. Per-subsystem errors are surfaced in the
|
|
||||||
respective dict so an operator doesn't have to guess.
|
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from fastapi import APIRouter, Request
|
from fastapi import APIRouter
|
||||||
|
|
||||||
from cyclone import __version__
|
from cyclone import __version__
|
||||||
from cyclone.security import get_health_snapshot
|
|
||||||
|
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
@router.get("/api/health")
|
@router.get("/api/health")
|
||||||
def health(request: Request) -> dict:
|
def health() -> dict[str, str]:
|
||||||
snap = get_health_snapshot()
|
return {"status": "ok", "version": __version__}
|
||||||
# Fill in live pubsub subscriber counts using the per-request app
|
|
||||||
# state (the snapshot builder doesn't have request context).
|
|
||||||
bus = getattr(request.app.state, "event_bus", None)
|
|
||||||
if bus is not None and hasattr(bus, "stats"):
|
|
||||||
snap.pubsub = bus.stats()
|
|
||||||
elif bus is not None:
|
|
||||||
snap.pubsub = {"note": "EventBus.stats() not available"}
|
|
||||||
else:
|
|
||||||
snap.pubsub = {"note": "EventBus not attached (running outside lifespan?)"}
|
|
||||||
return snap.to_dict()
|
|
||||||
|
|||||||
@@ -1,280 +0,0 @@
|
|||||||
"""SP17 — Encrypted backup primitives.
|
|
||||||
|
|
||||||
This module provides the low-level building blocks the rest of the
|
|
||||||
backup stack uses:
|
|
||||||
|
|
||||||
* ``derive_key`` — PBKDF2-HMAC-SHA256 key derivation (200,000
|
|
||||||
iterations, 32-byte output). The salt is per-backup, not global,
|
|
||||||
so identical passphrases produce different keys per backup.
|
|
||||||
|
|
||||||
* ``encrypt`` / ``decrypt`` — AES-256-GCM authenticated encryption.
|
|
||||||
Output layout: ``salt (16) | nonce (12) | ciphertext | tag (16)``.
|
|
||||||
The GCM tag is appended to the ciphertext by the cryptography
|
|
||||||
library; we don't prepend it.
|
|
||||||
|
|
||||||
* ``fingerprint`` — SHA-256 of a byte string, returned in the
|
|
||||||
``sha256:<hex>`` format we use across the codebase for DB keys and
|
|
||||||
audit events.
|
|
||||||
|
|
||||||
* ``BackupError`` / ``BackupDecryptError`` — typed exceptions so
|
|
||||||
callers can distinguish "wrong passphrase" from "I/O failed".
|
|
||||||
|
|
||||||
The crypto choices are deliberate:
|
|
||||||
|
|
||||||
* **AES-256-GCM** is the modern AEAD standard; the tag authenticates
|
|
||||||
both the ciphertext and the AAD (we pass an empty AAD; the
|
|
||||||
format itself is self-describing).
|
|
||||||
|
|
||||||
* **PBKDF2-HMAC-SHA256 @ 200k iters** is OWASP's 2023+ minimum for
|
|
||||||
PBKDF2-SHA256. Argon2id would be better but adds a C dependency;
|
|
||||||
PBKDF2 is stdlib via the ``cryptography`` package.
|
|
||||||
|
|
||||||
* **Random salt per backup** prevents rainbow-table attacks across
|
|
||||||
the operator's backup set.
|
|
||||||
|
|
||||||
* **Random 96-bit nonce per encryption** is what AES-GCM requires;
|
|
||||||
we use ``os.urandom`` which is a CSPRNG on every platform we run
|
|
||||||
on (macOS, Linux).
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import os
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from datetime import datetime
|
|
||||||
from typing import Optional
|
|
||||||
|
|
||||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
|
||||||
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
|
|
||||||
from cryptography.hazmat.primitives import hashes
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Constants
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# PBKDF2 iterations. OWASP 2023 minimum for PBKDF2-HMAC-SHA256 is
|
|
||||||
# 600,000; we use 200,000 as a balance between security and operator
|
|
||||||
# pain on the first backup creation (each backup does one KDF; on
|
|
||||||
# modern hardware 200k iters takes ~100ms). Bump this constant if you
|
|
||||||
# rotate the format version.
|
|
||||||
KDF_ITERATIONS = 200_000
|
|
||||||
|
|
||||||
# Salt + nonce sizes are AES-GCM / PBKDF2 standards, not negotiable.
|
|
||||||
SALT_LEN = 16
|
|
||||||
NONCE_LEN = 12
|
|
||||||
|
|
||||||
# Output key length for AES-256 = 32 bytes.
|
|
||||||
KEY_LEN = 32
|
|
||||||
|
|
||||||
# Format version. Bump when the on-disk layout changes (e.g. switch
|
|
||||||
# to Argon2id). Decryption reads this off the sidecar's
|
|
||||||
# encryption.kdf_iterations + cipher fields, not the version, so
|
|
||||||
# old backups remain decryptable until manually migrated.
|
|
||||||
FORMAT_VERSION = "v1"
|
|
||||||
|
|
||||||
# Fallback salt for the SQLCipher-key-derived backup key. Used only
|
|
||||||
# when the operator hasn't set a separate backup passphrase in the
|
|
||||||
# Keychain. This is a *constant* on purpose: the SQLCipher key is
|
|
||||||
# already random, so a fixed salt doesn't reduce entropy (the salt's
|
|
||||||
# job is to prevent rainbow tables, which require a *guessable*
|
|
||||||
# password; SQLCipher's key is unguessable).
|
|
||||||
FALLBACK_SALT = b"cyclone-db-backup-fallback-v1"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Exceptions
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class BackupError(Exception):
|
|
||||||
"""Generic backup failure. See BackupDecryptError for crypto errors."""
|
|
||||||
|
|
||||||
|
|
||||||
class BackupDecryptError(BackupError):
|
|
||||||
"""Decryption failed — wrong passphrase, tampered ciphertext, or
|
|
||||||
truncated file. Caller should NOT retry with the same key."""
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Key derivation + encryption
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def derive_key(passphrase: str, salt: bytes) -> bytes:
|
|
||||||
"""Derive a 32-byte AES key from a passphrase + salt.
|
|
||||||
|
|
||||||
Uses PBKDF2-HMAC-SHA256 with :data:`KDF_ITERATIONS` rounds. The
|
|
||||||
passphrase is encoded as UTF-8 bytes; the salt is used verbatim.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
passphrase: The operator's passphrase (any string).
|
|
||||||
salt: Per-backup random bytes of length :data:`SALT_LEN`.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
32 bytes suitable for AES-256-GCM.
|
|
||||||
"""
|
|
||||||
kdf = PBKDF2HMAC(
|
|
||||||
algorithm=hashes.SHA256(),
|
|
||||||
length=KEY_LEN,
|
|
||||||
salt=salt,
|
|
||||||
iterations=KDF_ITERATIONS,
|
|
||||||
)
|
|
||||||
return kdf.derive(passphrase.encode("utf-8"))
|
|
||||||
|
|
||||||
|
|
||||||
def encrypt(plaintext: bytes, key: bytes) -> bytes:
|
|
||||||
"""AES-256-GCM encrypt with a fresh random 12-byte nonce.
|
|
||||||
|
|
||||||
Returns ``nonce (12) || ciphertext || tag (16)``. The
|
|
||||||
``cryptography`` library appends the tag automatically.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
plaintext: The bytes to encrypt (e.g. the SQLite .backup blob).
|
|
||||||
key: 32-byte AES key from :func:`derive_key`.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
The combined nonce+ciphertext+tag blob.
|
|
||||||
"""
|
|
||||||
if len(key) != KEY_LEN:
|
|
||||||
raise BackupError(f"key must be {KEY_LEN} bytes; got {len(key)}")
|
|
||||||
nonce = os.urandom(NONCE_LEN)
|
|
||||||
aesgcm = AESGCM(key)
|
|
||||||
ciphertext = aesgcm.encrypt(nonce, plaintext, associated_data=None)
|
|
||||||
return nonce + ciphertext
|
|
||||||
|
|
||||||
|
|
||||||
def decrypt(blob: bytes, key: bytes) -> bytes:
|
|
||||||
"""AES-256-GCM decrypt. Raises :class:`BackupDecryptError` on auth failure.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
blob: The ``nonce||ciphertext||tag`` bytes from :func:`encrypt`.
|
|
||||||
key: The same 32-byte key used to encrypt.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
The original plaintext.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
BackupDecryptError: If the blob is too short, the tag fails to
|
|
||||||
verify (wrong key or tampered ciphertext), or the input is
|
|
||||||
otherwise malformed.
|
|
||||||
"""
|
|
||||||
if len(key) != KEY_LEN:
|
|
||||||
raise BackupError(f"key must be {KEY_LEN} bytes; got {len(key)}")
|
|
||||||
if len(blob) < NONCE_LEN + 16:
|
|
||||||
# 12 (nonce) + 16 (tag) = minimum; no room for ciphertext.
|
|
||||||
raise BackupDecryptError(
|
|
||||||
f"blob too short ({len(blob)} bytes); expected >= {NONCE_LEN + 16}",
|
|
||||||
)
|
|
||||||
nonce = blob[:NONCE_LEN]
|
|
||||||
ciphertext = blob[NONCE_LEN:]
|
|
||||||
aesgcm = AESGCM(key)
|
|
||||||
try:
|
|
||||||
return aesgcm.decrypt(nonce, ciphertext, associated_data=None)
|
|
||||||
except Exception as exc: # cryptography raises InvalidTag
|
|
||||||
raise BackupDecryptError(f"decryption failed: {exc}") from exc
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Fingerprint
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def fingerprint(data: bytes) -> str:
|
|
||||||
"""SHA-256 of ``data`` as ``"sha256:<64-hex-chars>"``."""
|
|
||||||
return "sha256:" + hashlib.sha256(data).hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def fingerprint_file(path: "os.PathLike[str] | str") -> str:
|
|
||||||
"""SHA-256 of a file's bytes, streamed. Memory-bounded for big DBs."""
|
|
||||||
h = hashlib.sha256()
|
|
||||||
with open(path, "rb") as f:
|
|
||||||
for chunk in iter(lambda: f.read(1024 * 1024), b""):
|
|
||||||
h.update(chunk)
|
|
||||||
return "sha256:" + h.hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Sidecar dataclass
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class Sidecar:
|
|
||||||
"""Plaintext metadata written next to each backup file.
|
|
||||||
|
|
||||||
Not required for decryption — it's a manifest an operator
|
|
||||||
consults to decide whether to restore. Kept intentionally
|
|
||||||
tiny so it survives most format rotations.
|
|
||||||
"""
|
|
||||||
|
|
||||||
format_version: str
|
|
||||||
created_at: str # ISO 8601 UTC
|
|
||||||
db_fingerprint: str # "sha256:..."
|
|
||||||
table_count: int
|
|
||||||
size_bytes: int
|
|
||||||
kdf: str # "PBKDF2-HMAC-SHA256"
|
|
||||||
kdf_iterations: int
|
|
||||||
cipher: str # "AES-256-GCM"
|
|
||||||
key_fingerprint: str # "sha256:..." of the derived key
|
|
||||||
|
|
||||||
def to_json(self) -> str:
|
|
||||||
import json
|
|
||||||
return json.dumps(
|
|
||||||
{
|
|
||||||
"format_version": self.format_version,
|
|
||||||
"created_at": self.created_at,
|
|
||||||
"db_fingerprint": self.db_fingerprint,
|
|
||||||
"table_count": self.table_count,
|
|
||||||
"size_bytes": self.size_bytes,
|
|
||||||
"encryption": {
|
|
||||||
"kdf": self.kdf,
|
|
||||||
"kdf_iterations": self.kdf_iterations,
|
|
||||||
"cipher": self.cipher,
|
|
||||||
"key_fingerprint": self.key_fingerprint,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
indent=2,
|
|
||||||
sort_keys=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_json(cls, text: str) -> "Sidecar":
|
|
||||||
import json
|
|
||||||
d = json.loads(text)
|
|
||||||
enc = d.get("encryption") or {}
|
|
||||||
return cls(
|
|
||||||
format_version=d["format_version"],
|
|
||||||
created_at=d["created_at"],
|
|
||||||
db_fingerprint=d["db_fingerprint"],
|
|
||||||
table_count=int(d["table_count"]),
|
|
||||||
size_bytes=int(d["size_bytes"]),
|
|
||||||
kdf=enc.get("kdf", "PBKDF2-HMAC-SHA256"),
|
|
||||||
kdf_iterations=int(enc.get("kdf_iterations", KDF_ITERATIONS)),
|
|
||||||
cipher=enc.get("cipher", "AES-256-GCM"),
|
|
||||||
key_fingerprint=enc.get("key_fingerprint", ""),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Filename helpers
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def backup_filename(timestamp: Optional[datetime] = None) -> str:
|
|
||||||
"""``cyclone-backup-YYYYMMDDTHHMMSSZ-<rand>.bin`` for a UTC timestamp.
|
|
||||||
|
|
||||||
The random suffix is a 4-byte hex string so two backups in the
|
|
||||||
same second don't collide on the ``db_backups`` unique index.
|
|
||||||
"""
|
|
||||||
import secrets as _secrets
|
|
||||||
from datetime import timezone as _tz
|
|
||||||
ts = timestamp or datetime.now(_tz.utc)
|
|
||||||
suffix = _secrets.token_hex(4)
|
|
||||||
return f"cyclone-backup-{ts.strftime('%Y%m%dT%H%M%SZ')}-{suffix}.bin"
|
|
||||||
|
|
||||||
|
|
||||||
def sidecar_filename(bin_filename: str) -> str:
|
|
||||||
"""``<bin_filename>.meta.json``."""
|
|
||||||
return bin_filename + ".meta.json"
|
|
||||||
@@ -1,368 +0,0 @@
|
|||||||
"""SP17 — Backup scheduler.
|
|
||||||
|
|
||||||
Wraps :class:`cyclone.backup_service.BackupService` in an
|
|
||||||
asyncio task, mirroring the MFT scheduler pattern (SP16). A backup
|
|
||||||
tick:
|
|
||||||
|
|
||||||
1. Calls :meth:`BackupService.create_now` to take + encrypt a backup.
|
|
||||||
2. Calls :meth:`BackupService.prune` to apply the retention policy.
|
|
||||||
3. Writes a tamper-evident ``audit_log`` row (SP11) for each outcome.
|
|
||||||
|
|
||||||
The scheduler is OFF by default. Operators opt in via
|
|
||||||
``CYCLONE_BACKUP_AUTOSTART=true``. The poll interval is
|
|
||||||
``CYCLONE_BACKUP_INTERVAL_HOURS`` (default 24).
|
|
||||||
|
|
||||||
Like the MFT scheduler, this is single-asyncio-task — no
|
|
||||||
threading, no APScheduler. All access (start/stop/tick/status)
|
|
||||||
must happen on the same event loop; the FastAPI app satisfies
|
|
||||||
that trivially because endpoints run on the loop.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import traceback
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Optional
|
|
||||||
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone.audit_log import AuditEvent, append_event
|
|
||||||
from cyclone.backup_service import BackupService
|
|
||||||
|
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class BackupTickResult:
|
|
||||||
"""Outcome of a single backup tick (one cycle of create + prune + audit)."""
|
|
||||||
|
|
||||||
started_at: datetime
|
|
||||||
finished_at: Optional[datetime] = None
|
|
||||||
created: Optional[svc_mod.BackupRecord] = None
|
|
||||||
pruned_paths: list[str] = field(default_factory=list)
|
|
||||||
error: Optional[str] = None
|
|
||||||
|
|
||||||
@property
|
|
||||||
def ok(self) -> bool:
|
|
||||||
return self.error is None and self.created is not None
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"started_at": self.started_at.isoformat(),
|
|
||||||
"finished_at": (
|
|
||||||
self.finished_at.isoformat() if self.finished_at else None
|
|
||||||
),
|
|
||||||
"ok": self.ok,
|
|
||||||
"created": (
|
|
||||||
{
|
|
||||||
"id": self.created.id,
|
|
||||||
"filename": self.created.filename,
|
|
||||||
"size_bytes": self.created.size_bytes,
|
|
||||||
"db_fingerprint": self.created.db_fingerprint,
|
|
||||||
"table_count": self.created.table_count,
|
|
||||||
}
|
|
||||||
if self.created else None
|
|
||||||
),
|
|
||||||
"pruned_paths": list(self.pruned_paths),
|
|
||||||
"error": self.error,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class BackupSchedulerStatus:
|
|
||||||
running: bool
|
|
||||||
interval_hours: float
|
|
||||||
backup_dir: str
|
|
||||||
retention_days: int
|
|
||||||
last_tick: Optional[BackupTickResult] = None
|
|
||||||
tick_count: int = 0
|
|
||||||
total_created: int = 0
|
|
||||||
total_errors: int = 0
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"running": self.running,
|
|
||||||
"interval_hours": self.interval_hours,
|
|
||||||
"backup_dir": self.backup_dir,
|
|
||||||
"retention_days": self.retention_days,
|
|
||||||
"tick_count": self.tick_count,
|
|
||||||
"total_created": self.total_created,
|
|
||||||
"total_errors": self.total_errors,
|
|
||||||
"last_tick": self.last_tick.as_dict() if self.last_tick else None,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class BackupScheduler:
|
|
||||||
"""Asyncio loop that ticks the BackupService on an interval.
|
|
||||||
|
|
||||||
Lifecycle mirrors :class:`cyclone.scheduler.Scheduler`:
|
|
||||||
|
|
||||||
sched = BackupScheduler(backup_service)
|
|
||||||
await sched.start() # begin ticking
|
|
||||||
await sched.stop() # finish current tick, exit
|
|
||||||
status = sched.status() # snapshot
|
|
||||||
|
|
||||||
Threading: NOT thread-safe. All access must happen on the
|
|
||||||
same event loop. FastAPI endpoints satisfy this automatically.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
service: BackupService,
|
|
||||||
*,
|
|
||||||
interval_hours: float = 24.0,
|
|
||||||
) -> None:
|
|
||||||
self._service = service
|
|
||||||
self._interval_hours = max(0.1, float(interval_hours))
|
|
||||||
self._task: Optional[asyncio.Task[None]] = None
|
|
||||||
self._stop_event = asyncio.Event()
|
|
||||||
self._tick_in_progress = False
|
|
||||||
self._last_tick: Optional[BackupTickResult] = None
|
|
||||||
self._tick_count = 0
|
|
||||||
self._total_created = 0
|
|
||||||
self._total_errors = 0
|
|
||||||
|
|
||||||
@property
|
|
||||||
def service(self) -> BackupService:
|
|
||||||
return self._service
|
|
||||||
|
|
||||||
# ---- Public API -------------------------------------------------------
|
|
||||||
|
|
||||||
async def start(self) -> None:
|
|
||||||
if self._task is not None and not self._task.done():
|
|
||||||
log.info("BackupScheduler already running; start() is a no-op")
|
|
||||||
return
|
|
||||||
self._stop_event.clear()
|
|
||||||
self._task = asyncio.create_task(self._run(), name="backup-scheduler")
|
|
||||||
log.info(
|
|
||||||
"BackupScheduler started",
|
|
||||||
extra={
|
|
||||||
"interval_hours": self._interval_hours,
|
|
||||||
"backup_dir": str(self._service.backup_dir),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
async def stop(self) -> None:
|
|
||||||
if self._task is None or self._task.done():
|
|
||||||
return
|
|
||||||
self._stop_event.set()
|
|
||||||
try:
|
|
||||||
await asyncio.wait_for(self._task, timeout=60)
|
|
||||||
except asyncio.TimeoutError:
|
|
||||||
log.warning("BackupScheduler did not stop within 60s; cancelling")
|
|
||||||
self._task.cancel()
|
|
||||||
try:
|
|
||||||
await self._task
|
|
||||||
except (asyncio.CancelledError, Exception): # noqa: BLE001
|
|
||||||
pass
|
|
||||||
self._task = None
|
|
||||||
log.info("BackupScheduler stopped")
|
|
||||||
|
|
||||||
def status(self) -> BackupSchedulerStatus:
|
|
||||||
return BackupSchedulerStatus(
|
|
||||||
running=self.is_running(),
|
|
||||||
interval_hours=self._interval_hours,
|
|
||||||
backup_dir=str(self._service.backup_dir),
|
|
||||||
retention_days=self._service._retention_days,
|
|
||||||
last_tick=self._last_tick,
|
|
||||||
tick_count=self._tick_count,
|
|
||||||
total_created=self._total_created,
|
|
||||||
total_errors=self._total_errors,
|
|
||||||
)
|
|
||||||
|
|
||||||
def is_running(self) -> bool:
|
|
||||||
return self._task is not None and not self._task.done()
|
|
||||||
|
|
||||||
async def tick(self) -> BackupTickResult:
|
|
||||||
"""Run a single backup tick (create + prune + audit).
|
|
||||||
|
|
||||||
Concurrent ticks are coalesced: if a tick is already in
|
|
||||||
progress, the second caller waits for it. This protects
|
|
||||||
against a slow backup holding up multiple operator-driven
|
|
||||||
``POST /api/admin/backup/tick`` calls.
|
|
||||||
"""
|
|
||||||
while self._tick_in_progress:
|
|
||||||
await asyncio.sleep(0.05)
|
|
||||||
self._tick_in_progress = True
|
|
||||||
try:
|
|
||||||
result = await self._tick_impl()
|
|
||||||
self._last_tick = result
|
|
||||||
self._tick_count += 1
|
|
||||||
if result.created is not None:
|
|
||||||
self._total_created += 1
|
|
||||||
if result.error is not None:
|
|
||||||
self._total_errors += 1
|
|
||||||
return result
|
|
||||||
finally:
|
|
||||||
self._tick_in_progress = False
|
|
||||||
|
|
||||||
# ---- Internals --------------------------------------------------------
|
|
||||||
|
|
||||||
async def _run(self) -> None:
|
|
||||||
# Stagger the first tick (same rationale as the MFT scheduler).
|
|
||||||
await asyncio.sleep(5)
|
|
||||||
while not self._stop_event.is_set():
|
|
||||||
try:
|
|
||||||
await self.tick()
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
# tick() catches its own exceptions and returns them
|
|
||||||
# in the result. This is the safety net for
|
|
||||||
# programmer errors in the loop body.
|
|
||||||
log.exception("BackupScheduler tick raised", extra={"error": str(exc)})
|
|
||||||
try:
|
|
||||||
await asyncio.wait_for(
|
|
||||||
self._stop_event.wait(),
|
|
||||||
timeout=self._interval_hours * 3600,
|
|
||||||
)
|
|
||||||
except asyncio.TimeoutError:
|
|
||||||
pass # interval elapsed
|
|
||||||
|
|
||||||
async def _tick_impl(self) -> BackupTickResult:
|
|
||||||
started = datetime.now(timezone.utc)
|
|
||||||
result = BackupTickResult(started_at=started)
|
|
||||||
try:
|
|
||||||
create_result = await asyncio.to_thread(self._service.create_now)
|
|
||||||
result.created = create_result.backup
|
|
||||||
# Audit event for the created backup.
|
|
||||||
await asyncio.to_thread(
|
|
||||||
_audit_backup_created,
|
|
||||||
create_result.backup.id,
|
|
||||||
create_result.backup.db_fingerprint,
|
|
||||||
create_result.backup.table_count,
|
|
||||||
"backup-scheduler",
|
|
||||||
)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.exception("Backup create failed during tick")
|
|
||||||
result.error = f"create: {type(exc).__name__}: {exc}"
|
|
||||||
await asyncio.to_thread(
|
|
||||||
_audit_backup_failed,
|
|
||||||
f"create: {type(exc).__name__}: {exc}",
|
|
||||||
traceback.format_exc()[-500:],
|
|
||||||
"backup-scheduler",
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
pruned = await asyncio.to_thread(self._service.prune)
|
|
||||||
result.pruned_paths = pruned
|
|
||||||
if pruned:
|
|
||||||
await asyncio.to_thread(
|
|
||||||
_audit_backup_pruned, pruned, "backup-scheduler",
|
|
||||||
)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.exception("Backup prune failed during tick")
|
|
||||||
# Don't clobber the create error if there was one.
|
|
||||||
if result.error is None:
|
|
||||||
result.error = f"prune: {type(exc).__name__}: {exc}"
|
|
||||||
await asyncio.to_thread(
|
|
||||||
_audit_backup_failed,
|
|
||||||
f"prune: {type(exc).__name__}: {exc}",
|
|
||||||
traceback.format_exc()[-500:],
|
|
||||||
"backup-scheduler",
|
|
||||||
)
|
|
||||||
|
|
||||||
result.finished_at = datetime.now(timezone.utc)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Audit helpers (run in a thread so the asyncio loop doesn't block)
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _audit_backup_created(
|
|
||||||
backup_id: int, db_fingerprint: str, table_count: int, actor: str,
|
|
||||||
) -> None:
|
|
||||||
from cyclone import db
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
try:
|
|
||||||
append_event(s, AuditEvent(
|
|
||||||
event_type="db.backup_created",
|
|
||||||
entity_type="database",
|
|
||||||
entity_id="cyclone.db",
|
|
||||||
actor=actor,
|
|
||||||
payload={
|
|
||||||
"backup_id": backup_id,
|
|
||||||
"db_fingerprint": db_fingerprint,
|
|
||||||
"table_count": table_count,
|
|
||||||
},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
log.exception("Failed to write db.backup_created audit event")
|
|
||||||
|
|
||||||
|
|
||||||
def _audit_backup_failed(
|
|
||||||
reason: str, traceback_tail: str, actor: str,
|
|
||||||
) -> None:
|
|
||||||
from cyclone import db
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
try:
|
|
||||||
append_event(s, AuditEvent(
|
|
||||||
event_type="db.backup_failed",
|
|
||||||
entity_type="database",
|
|
||||||
entity_id="cyclone.db",
|
|
||||||
actor=actor,
|
|
||||||
payload={"reason": reason, "traceback_tail": traceback_tail},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
log.exception("Failed to write db.backup_failed audit event")
|
|
||||||
|
|
||||||
|
|
||||||
def _audit_backup_pruned(deleted_paths: list[str], actor: str) -> None:
|
|
||||||
from cyclone import db
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
try:
|
|
||||||
append_event(s, AuditEvent(
|
|
||||||
event_type="db.backup_pruned",
|
|
||||||
entity_type="database",
|
|
||||||
entity_id="cyclone.db",
|
|
||||||
actor=actor,
|
|
||||||
payload={"deleted_paths": deleted_paths},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
log.exception("Failed to write db.backup_pruned audit event")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Module-level singleton
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
_scheduler: Optional[BackupScheduler] = None
|
|
||||||
|
|
||||||
|
|
||||||
def configure_backup_scheduler(
|
|
||||||
service: BackupService,
|
|
||||||
*,
|
|
||||||
interval_hours: float = 24.0,
|
|
||||||
) -> BackupScheduler:
|
|
||||||
"""Create (or return existing) the module-level BackupScheduler."""
|
|
||||||
global _scheduler
|
|
||||||
if _scheduler is not None:
|
|
||||||
return _scheduler
|
|
||||||
hours = float(
|
|
||||||
os.environ.get("CYCLONE_BACKUP_INTERVAL_HOURS", interval_hours),
|
|
||||||
)
|
|
||||||
_scheduler = BackupScheduler(service, interval_hours=hours)
|
|
||||||
return _scheduler
|
|
||||||
|
|
||||||
|
|
||||||
def get_backup_scheduler() -> BackupScheduler:
|
|
||||||
"""Return the configured BackupScheduler. Raises if not set up."""
|
|
||||||
if _scheduler is None:
|
|
||||||
raise RuntimeError(
|
|
||||||
"backup scheduler not configured; call configure_backup_scheduler() first",
|
|
||||||
)
|
|
||||||
return _scheduler
|
|
||||||
|
|
||||||
|
|
||||||
def reset_backup_scheduler_for_tests() -> None:
|
|
||||||
"""Clear the module-level singleton. Test-only."""
|
|
||||||
global _scheduler
|
|
||||||
_scheduler = None
|
|
||||||
@@ -1,851 +0,0 @@
|
|||||||
"""SP17 — High-level backup coordinator.
|
|
||||||
|
|
||||||
Owns the lifecycle of every backup the operator (or the scheduler)
|
|
||||||
takes:
|
|
||||||
|
|
||||||
* ``create_now`` — runs SQLite's online ``.backup()`` against the
|
|
||||||
live engine, encrypts the bytes with :func:`cyclone.backup.encrypt`,
|
|
||||||
writes a ``.bin`` + ``.meta.json`` pair into the backup directory,
|
|
||||||
and persists a row in ``db_backups``.
|
|
||||||
|
|
||||||
* ``list_backups`` — directory listing joined with ``db_backups`` rows.
|
|
||||||
|
|
||||||
* ``verify`` — decrypts + recomputes SHA-256, compares to the sidecar.
|
|
||||||
|
|
||||||
* ``restore_initiate`` / ``restore_confirm`` — two-step restore so an
|
|
||||||
idle browser tab can't nuke the live DB. The first call returns a
|
|
||||||
``restore_token`` (a random 32-byte hex string) plus a preview
|
|
||||||
(``db_fingerprint``, ``table_count``). The second call swaps the
|
|
||||||
engine only if the token matches.
|
|
||||||
|
|
||||||
* ``prune`` — deletes backups older than ``retention_days``.
|
|
||||||
|
|
||||||
This module is intentionally engine-aware: ``create_now`` reaches
|
|
||||||
into the live SQLAlchemy engine to get a raw SQLite connection and
|
|
||||||
call ``.backup()`` (the only way to take an online consistent
|
|
||||||
snapshot). ``restore`` reaches into :func:`cyclone.db.dispose_engine`
|
|
||||||
+ :func:`cyclone.db.reinit_engine` to swap to the restored file.
|
|
||||||
|
|
||||||
The encryption key is loaded once at construction time:
|
|
||||||
|
|
||||||
* If a backup passphrase is set in the Keychain (``backup.passphrase``
|
|
||||||
account under service ``cyclone``), use it directly with the salt
|
|
||||||
stored in the companion ``backup.salt`` account. The salt must be
|
|
||||||
persisted — a fresh random salt per process would defeat the key.
|
|
||||||
* Otherwise fall back to deriving from the SQLCipher DB key + a fixed
|
|
||||||
salt. Logged at WARNING because this is a degraded-but-still-safe
|
|
||||||
posture.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import secrets as _secrets
|
|
||||||
import shutil
|
|
||||||
import sqlite3
|
|
||||||
import threading
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Optional, Union
|
|
||||||
|
|
||||||
from sqlalchemy.exc import SQLAlchemyError
|
|
||||||
|
|
||||||
from cyclone import backup as backup_mod
|
|
||||||
from cyclone.backup import BackupError
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
|
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
# Status values for db_backups.status (mirrored in the ORM).
|
|
||||||
STATUS_PENDING = "pending"
|
|
||||||
STATUS_OK = "ok"
|
|
||||||
STATUS_ERROR = "error"
|
|
||||||
STATUS_PRUNED = "pruned"
|
|
||||||
|
|
||||||
# Where the operator's backup passphrase lives in the Keychain.
|
|
||||||
KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT = "backup.passphrase"
|
|
||||||
|
|
||||||
# Companion account for the salt. Stored as hex. Same value across
|
|
||||||
# processes so the derived key is reproducible — a fresh random salt
|
|
||||||
# per BackupService would defeat the key.
|
|
||||||
KEYCHAIN_BACKUP_SALT_ACCOUNT = "backup.salt"
|
|
||||||
|
|
||||||
# Restore token TTL (seconds). The two-step confirm must complete
|
|
||||||
# within this window or the operator re-runs initiate.
|
|
||||||
RESTORE_TOKEN_TTL_SECONDS = 300
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Result dataclasses
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class BackupRecord:
|
|
||||||
"""Public view of a backup row joined with filesystem state."""
|
|
||||||
|
|
||||||
id: int
|
|
||||||
filename: str
|
|
||||||
backup_dir: str
|
|
||||||
size_bytes: int
|
|
||||||
db_fingerprint: str
|
|
||||||
table_count: int
|
|
||||||
created_at: datetime
|
|
||||||
completed_at: Optional[datetime]
|
|
||||||
status: str
|
|
||||||
error_message: Optional[str]
|
|
||||||
key_fingerprint: str
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class CreateResult:
|
|
||||||
"""Outcome of ``create_now``."""
|
|
||||||
|
|
||||||
backup: BackupRecord
|
|
||||||
sidecar: backup_mod.Sidecar
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class VerifyResult:
|
|
||||||
"""Outcome of ``verify``."""
|
|
||||||
|
|
||||||
backup_id: int
|
|
||||||
filename: str
|
|
||||||
ok: bool
|
|
||||||
expected_fingerprint: str
|
|
||||||
actual_fingerprint: str
|
|
||||||
table_count: int
|
|
||||||
reason: Optional[str] = None
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class RestoreInitiateResult:
|
|
||||||
"""Returned by the first call of the two-step restore."""
|
|
||||||
|
|
||||||
backup_id: int
|
|
||||||
filename: str
|
|
||||||
restore_token: str
|
|
||||||
expires_at: datetime
|
|
||||||
db_fingerprint: str
|
|
||||||
table_count: int
|
|
||||||
current_db_fingerprint: str
|
|
||||||
current_table_count: int
|
|
||||||
size_bytes: int
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class RestoreConfirmResult:
|
|
||||||
"""Returned by the second call of the two-step restore."""
|
|
||||||
|
|
||||||
backup_id: int
|
|
||||||
filename: str
|
|
||||||
restored_from_fingerprint: str
|
|
||||||
restored_at: datetime
|
|
||||||
new_db_fingerprint: str
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# BackupService
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class BackupService:
|
|
||||||
"""Coordinator for encrypted DB backups.
|
|
||||||
|
|
||||||
Construct once at app startup; share across requests. Not
|
|
||||||
thread-safe for *creation* (the SQLite ``.backup()`` call uses
|
|
||||||
the live engine and is best serialized through the scheduler),
|
|
||||||
but ``list_backups`` / ``prune`` / ``status`` are safe to call
|
|
||||||
concurrently.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
backup_dir: Union[str, Path],
|
|
||||||
*,
|
|
||||||
passphrase: Optional[str] = None,
|
|
||||||
salt: Optional[bytes] = None,
|
|
||||||
retention_days: int = 30,
|
|
||||||
db_url: Optional[str] = None,
|
|
||||||
) -> None:
|
|
||||||
self._backup_dir = Path(backup_dir)
|
|
||||||
self._retention_days = max(1, int(retention_days))
|
|
||||||
self._db_url = db_url
|
|
||||||
# The derived key + its salt. If ``passphrase`` is None we
|
|
||||||
# fall back to deriving from the SQLCipher DB key (with a
|
|
||||||
# WARNING log).
|
|
||||||
#
|
|
||||||
# Salt is per-BackupService-instance and MUST be stable across
|
|
||||||
# processes — otherwise the same passphrase would derive
|
|
||||||
# different keys in different invocations and decrypt would
|
|
||||||
# always fail. Two options:
|
|
||||||
#
|
|
||||||
# 1. Caller passes an explicit ``salt`` (the Keychain flow
|
|
||||||
# reads the persisted salt from backup.salt account).
|
|
||||||
# 2. We accept a None salt here; ``_ensure_key`` then either
|
|
||||||
# uses the persisted salt (if available) or generates one
|
|
||||||
# and persists it on first use.
|
|
||||||
#
|
|
||||||
# Tests typically pass an explicit random salt; production
|
|
||||||
# should always pass the persisted one.
|
|
||||||
self._passphrase = passphrase
|
|
||||||
self._salt = salt
|
|
||||||
self._key: Optional[bytes] = None
|
|
||||||
self._used_fallback = False
|
|
||||||
# Pending restore tokens: token -> (backup_id, expires_at).
|
|
||||||
# A simple in-memory dict is sufficient — the token only
|
|
||||||
# needs to survive between the two API calls in one process.
|
|
||||||
self._pending_restores: dict[str, tuple[int, datetime]] = {}
|
|
||||||
self._lock = threading.Lock()
|
|
||||||
|
|
||||||
# ---- Public API -------------------------------------------------------
|
|
||||||
|
|
||||||
@property
|
|
||||||
def backup_dir(self) -> Path:
|
|
||||||
return self._backup_dir
|
|
||||||
|
|
||||||
@property
|
|
||||||
def key_fingerprint(self) -> str:
|
|
||||||
"""SHA-256 of the current derived key, or "" if not yet derived."""
|
|
||||||
if self._key is None:
|
|
||||||
return ""
|
|
||||||
return backup_mod.fingerprint(self._key)
|
|
||||||
|
|
||||||
def create_now(self) -> CreateResult:
|
|
||||||
"""Take an encrypted backup of the live DB right now.
|
|
||||||
|
|
||||||
Crash-safe: any failure marks the ``db_backups`` row as
|
|
||||||
``error``, removes any partial files from the backup dir, and
|
|
||||||
re-raises the exception.
|
|
||||||
"""
|
|
||||||
# 1. Make sure the backup dir exists.
|
|
||||||
self._backup_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
# 2. Allocate a filename + insert a pending row.
|
|
||||||
from cyclone.db import DbBackup # late import — circular otherwise
|
|
||||||
from cyclone.store import store as cycl_store
|
|
||||||
|
|
||||||
filename = backup_mod.backup_filename()
|
|
||||||
created_at = datetime.now(timezone.utc)
|
|
||||||
|
|
||||||
row = cycl_store.add_backup_pending(
|
|
||||||
filename=filename,
|
|
||||||
backup_dir=str(self._backup_dir),
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
# 3. Run SQLite's online .backup() to a temp file.
|
|
||||||
# We use a private path *inside* the backup dir so the
|
|
||||||
# operator can see what crashed if it does.
|
|
||||||
staging_db = self._backup_dir / f".{filename}.staging.db"
|
|
||||||
self._sqlite_backup_to(staging_db)
|
|
||||||
|
|
||||||
# 4. Encrypt.
|
|
||||||
plaintext = staging_db.read_bytes()
|
|
||||||
db_fp = backup_mod.fingerprint(plaintext)
|
|
||||||
key = self._ensure_key()
|
|
||||||
blob = backup_mod.encrypt(plaintext, key)
|
|
||||||
|
|
||||||
# 5. Move encrypted blob into place + write sidecar.
|
|
||||||
target = self._backup_dir / filename
|
|
||||||
target.write_bytes(blob)
|
|
||||||
staging_db.unlink()
|
|
||||||
|
|
||||||
table_count = self._count_tables_in_blob(plaintext)
|
|
||||||
sidecar = backup_mod.Sidecar(
|
|
||||||
format_version=backup_mod.FORMAT_VERSION,
|
|
||||||
created_at=created_at.isoformat(),
|
|
||||||
db_fingerprint=db_fp,
|
|
||||||
table_count=table_count,
|
|
||||||
size_bytes=len(blob),
|
|
||||||
kdf="PBKDF2-HMAC-SHA256",
|
|
||||||
kdf_iterations=backup_mod.KDF_ITERATIONS,
|
|
||||||
cipher="AES-256-GCM",
|
|
||||||
key_fingerprint=backup_mod.fingerprint(key),
|
|
||||||
)
|
|
||||||
sidecar_path = self._backup_dir / backup_mod.sidecar_filename(filename)
|
|
||||||
sidecar_path.write_text(sidecar.to_json())
|
|
||||||
|
|
||||||
# 6. Mark the row as ok.
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.get(DbBackup, row.id)
|
|
||||||
row.status = STATUS_OK
|
|
||||||
row.size_bytes = len(blob)
|
|
||||||
row.db_fingerprint = db_fp
|
|
||||||
row.table_count = table_count
|
|
||||||
row.completed_at = datetime.now(timezone.utc)
|
|
||||||
s.commit()
|
|
||||||
s.refresh(row)
|
|
||||||
|
|
||||||
record = self._row_to_record(row)
|
|
||||||
log.info(
|
|
||||||
"Backup created",
|
|
||||||
extra={
|
|
||||||
"backup_id": record.id,
|
|
||||||
"backup_filename": record.filename,
|
|
||||||
"size_bytes": record.size_bytes,
|
|
||||||
"db_fingerprint": record.db_fingerprint,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
return CreateResult(backup=record, sidecar=sidecar)
|
|
||||||
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.exception("Backup create failed")
|
|
||||||
try:
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.get(DbBackup, row.id)
|
|
||||||
row.status = STATUS_ERROR
|
|
||||||
row.error_message = f"{type(exc).__name__}: {exc}"[:500]
|
|
||||||
row.completed_at = datetime.now(timezone.utc)
|
|
||||||
s.commit()
|
|
||||||
# Best-effort cleanup of any partial files.
|
|
||||||
for p in [
|
|
||||||
self._backup_dir / filename,
|
|
||||||
self._backup_dir / f".{filename}.staging.db",
|
|
||||||
self._backup_dir / backup_mod.sidecar_filename(filename),
|
|
||||||
]:
|
|
||||||
if p.exists():
|
|
||||||
try:
|
|
||||||
p.unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
log.exception("Failed to mark backup row as error")
|
|
||||||
raise
|
|
||||||
|
|
||||||
def list_backups(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
limit: int = 100,
|
|
||||||
status: Optional[str] = None,
|
|
||||||
) -> list[BackupRecord]:
|
|
||||||
"""List ``db_backups`` rows newest first.
|
|
||||||
|
|
||||||
Joins the filesystem state (presence of ``.bin`` and
|
|
||||||
``.meta.json``) implicitly via :attr:`BackupRecord.status`:
|
|
||||||
a row marked ``pruned`` had its files deleted by the
|
|
||||||
retention policy.
|
|
||||||
"""
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
q = s.query(DbBackup)
|
|
||||||
if status is not None:
|
|
||||||
q = q.filter(DbBackup.status == status)
|
|
||||||
rows = q.order_by(DbBackup.id.desc()).limit(limit).all()
|
|
||||||
return [self._row_to_record(r) for r in rows]
|
|
||||||
|
|
||||||
def verify(self, backup_id: int) -> VerifyResult:
|
|
||||||
"""Decrypt + checksum-verify a backup against its sidecar.
|
|
||||||
|
|
||||||
Does NOT trust the sidecar's ``db_fingerprint`` field alone;
|
|
||||||
recomputes the SHA-256 from the decrypted blob and compares.
|
|
||||||
"""
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.get(DbBackup, backup_id)
|
|
||||||
if row is None:
|
|
||||||
raise BackupError(f"backup {backup_id} not found")
|
|
||||||
record = self._row_to_record(row)
|
|
||||||
sidecar = self._read_sidecar(record.filename)
|
|
||||||
if sidecar is None:
|
|
||||||
return VerifyResult(
|
|
||||||
backup_id=record.id, filename=record.filename,
|
|
||||||
ok=False, expected_fingerprint="", actual_fingerprint="",
|
|
||||||
table_count=0, reason="sidecar missing",
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
blob = (self._backup_dir / record.filename).read_bytes()
|
|
||||||
except FileNotFoundError:
|
|
||||||
return VerifyResult(
|
|
||||||
backup_id=record.id, filename=record.filename,
|
|
||||||
ok=False,
|
|
||||||
expected_fingerprint=sidecar.db_fingerprint,
|
|
||||||
actual_fingerprint="",
|
|
||||||
table_count=sidecar.table_count,
|
|
||||||
reason="backup file missing",
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
plaintext = backup_mod.decrypt(blob, self._ensure_key())
|
|
||||||
except backup_mod.BackupDecryptError as exc:
|
|
||||||
return VerifyResult(
|
|
||||||
backup_id=record.id, filename=record.filename,
|
|
||||||
ok=False,
|
|
||||||
expected_fingerprint=sidecar.db_fingerprint,
|
|
||||||
actual_fingerprint="",
|
|
||||||
table_count=sidecar.table_count,
|
|
||||||
reason=str(exc),
|
|
||||||
)
|
|
||||||
|
|
||||||
actual_fp = backup_mod.fingerprint(plaintext)
|
|
||||||
return VerifyResult(
|
|
||||||
backup_id=record.id,
|
|
||||||
filename=record.filename,
|
|
||||||
ok=(actual_fp == sidecar.db_fingerprint),
|
|
||||||
expected_fingerprint=sidecar.db_fingerprint,
|
|
||||||
actual_fingerprint=actual_fp,
|
|
||||||
table_count=sidecar.table_count,
|
|
||||||
reason=None if actual_fp == sidecar.db_fingerprint else "fingerprint mismatch",
|
|
||||||
)
|
|
||||||
|
|
||||||
def restore_initiate(self, backup_id: int) -> RestoreInitiateResult:
|
|
||||||
"""First half of the two-step restore.
|
|
||||||
|
|
||||||
Decrypts the backup into a temp file and reads its current
|
|
||||||
``db_fingerprint`` + ``table_count``. Returns a one-shot
|
|
||||||
``restore_token`` the operator must echo back to
|
|
||||||
:meth:`restore_confirm` within 5 minutes.
|
|
||||||
"""
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.get(DbBackup, backup_id)
|
|
||||||
if row is None:
|
|
||||||
raise BackupError(f"backup {backup_id} not found")
|
|
||||||
if row.status != STATUS_OK:
|
|
||||||
raise BackupError(
|
|
||||||
f"backup {backup_id} status is {row.status!r}; only 'ok' backups can be restored",
|
|
||||||
)
|
|
||||||
record = self._row_to_record(row)
|
|
||||||
|
|
||||||
# Decrypt into a staging file so the confirm step is fast.
|
|
||||||
staging = self._backup_dir / f".restore-{record.filename}.staging.db"
|
|
||||||
try:
|
|
||||||
blob = (self._backup_dir / record.filename).read_bytes()
|
|
||||||
except FileNotFoundError as exc:
|
|
||||||
raise BackupError(f"backup file missing: {record.filename}") from exc
|
|
||||||
try:
|
|
||||||
plaintext = backup_mod.decrypt(blob, self._ensure_key())
|
|
||||||
except backup_mod.BackupDecryptError as exc:
|
|
||||||
raise BackupError(f"decrypt failed: {exc}") from exc
|
|
||||||
staging.write_bytes(plaintext)
|
|
||||||
|
|
||||||
# Snapshot the live DB's fingerprint for the operator's "are
|
|
||||||
# you sure you want to do this?" preview.
|
|
||||||
live_fp, live_count = self._live_fingerprint_and_count()
|
|
||||||
|
|
||||||
token = _secrets.token_hex(32)
|
|
||||||
expires_at = datetime.now(timezone.utc) + timedelta(
|
|
||||||
seconds=RESTORE_TOKEN_TTL_SECONDS,
|
|
||||||
)
|
|
||||||
with self._lock:
|
|
||||||
self._pending_restores[token] = (record.id, expires_at)
|
|
||||||
|
|
||||||
log.info(
|
|
||||||
"Restore initiated",
|
|
||||||
extra={
|
|
||||||
"backup_id": record.id,
|
|
||||||
"token_prefix": token[:8],
|
|
||||||
"expires_at": expires_at.isoformat(),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
return RestoreInitiateResult(
|
|
||||||
backup_id=record.id,
|
|
||||||
filename=record.filename,
|
|
||||||
restore_token=token,
|
|
||||||
expires_at=expires_at,
|
|
||||||
db_fingerprint=backup_mod.fingerprint(plaintext),
|
|
||||||
table_count=self._count_tables_in_blob(plaintext),
|
|
||||||
current_db_fingerprint=live_fp,
|
|
||||||
current_table_count=live_count,
|
|
||||||
size_bytes=len(plaintext),
|
|
||||||
)
|
|
||||||
|
|
||||||
def restore_confirm(
|
|
||||||
self,
|
|
||||||
backup_id: int,
|
|
||||||
restore_token: str,
|
|
||||||
*,
|
|
||||||
actor: str = "operator",
|
|
||||||
) -> RestoreConfirmResult:
|
|
||||||
"""Second half of the two-step restore.
|
|
||||||
|
|
||||||
Validates the token, copies the decrypted staging file over
|
|
||||||
the live DB path, disposes + reopens the engine. Raises
|
|
||||||
``BackupError`` on any mismatch.
|
|
||||||
"""
|
|
||||||
now = datetime.now(timezone.utc)
|
|
||||||
with self._lock:
|
|
||||||
entry = self._pending_restores.pop(restore_token, None)
|
|
||||||
if entry is None:
|
|
||||||
raise BackupError("restore_token not found (already consumed or never issued)")
|
|
||||||
token_backup_id, expires_at = entry
|
|
||||||
if token_backup_id != backup_id:
|
|
||||||
raise BackupError(
|
|
||||||
f"restore_token was for backup {token_backup_id}, not {backup_id}",
|
|
||||||
)
|
|
||||||
if now > expires_at:
|
|
||||||
raise BackupError(
|
|
||||||
f"restore_token expired at {expires_at.isoformat()}; re-run initiate",
|
|
||||||
)
|
|
||||||
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.get(DbBackup, backup_id)
|
|
||||||
if row is None:
|
|
||||||
raise BackupError(f"backup {backup_id} disappeared mid-restore")
|
|
||||||
record = self._row_to_record(row)
|
|
||||||
|
|
||||||
staging = self._backup_dir / f".restore-{record.filename}.staging.db"
|
|
||||||
if not staging.exists():
|
|
||||||
raise BackupError(
|
|
||||||
f"staging restore file missing: {staging.name}; re-run initiate",
|
|
||||||
)
|
|
||||||
|
|
||||||
target_db_path = self._live_db_path()
|
|
||||||
if target_db_path is None:
|
|
||||||
raise BackupError(
|
|
||||||
"cannot determine live DB file path (non-sqlite URL?)",
|
|
||||||
)
|
|
||||||
|
|
||||||
# Pre-restore fingerprint for the audit event.
|
|
||||||
restored_from_fp = backup_mod.fingerprint(staging.read_bytes())
|
|
||||||
|
|
||||||
# The swap: dispose engine → copy file → reinit engine.
|
|
||||||
# Anything between dispose and reinit raises (queries that
|
|
||||||
# are in-flight get a "database is locked" or
|
|
||||||
# "no such table" error); we accept that because the
|
|
||||||
# operator already confirmed.
|
|
||||||
db.dispose_engine()
|
|
||||||
try:
|
|
||||||
# Atomic copy via temp + rename so a crash mid-copy
|
|
||||||
# doesn't leave a half-written DB file.
|
|
||||||
tmp_target = target_db_path.with_suffix(
|
|
||||||
target_db_path.suffix + f".restoring-{_secrets.token_hex(4)}",
|
|
||||||
)
|
|
||||||
shutil.copyfile(staging, tmp_target)
|
|
||||||
os.replace(tmp_target, target_db_path)
|
|
||||||
finally:
|
|
||||||
staging.unlink(missing_ok=True)
|
|
||||||
db.reinit_engine()
|
|
||||||
|
|
||||||
# Post-restore fingerprint from the now-live engine.
|
|
||||||
new_fp, _ = self._live_fingerprint_and_count()
|
|
||||||
|
|
||||||
log.warning(
|
|
||||||
"Restore complete: backup_id=%d actor=%s from=%s to=%s",
|
|
||||||
backup_id, actor, restored_from_fp, new_fp,
|
|
||||||
)
|
|
||||||
return RestoreConfirmResult(
|
|
||||||
backup_id=record.id,
|
|
||||||
filename=record.filename,
|
|
||||||
restored_from_fingerprint=restored_from_fp,
|
|
||||||
restored_at=datetime.now(timezone.utc),
|
|
||||||
new_db_fingerprint=new_fp,
|
|
||||||
)
|
|
||||||
|
|
||||||
def prune(self, *, now: Optional[datetime] = None) -> list[str]:
|
|
||||||
"""Delete backups older than ``retention_days``. Returns deleted paths.
|
|
||||||
|
|
||||||
Marks the ``db_backups`` rows ``pruned`` so the operator can
|
|
||||||
still see what was deleted (and when).
|
|
||||||
"""
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
|
|
||||||
cutoff = (now or datetime.now(timezone.utc)) - timedelta(
|
|
||||||
days=self._retention_days,
|
|
||||||
)
|
|
||||||
deleted: list[str] = []
|
|
||||||
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
q = s.query(DbBackup).filter(
|
|
||||||
DbBackup.status == STATUS_OK,
|
|
||||||
DbBackup.created_at < cutoff,
|
|
||||||
)
|
|
||||||
for row in q.all():
|
|
||||||
# Delete the file pair; ignore if already gone.
|
|
||||||
bin_path = Path(row.backup_dir) / row.filename
|
|
||||||
meta_path = Path(row.backup_dir) / backup_mod.sidecar_filename(row.filename)
|
|
||||||
for p in (bin_path, meta_path):
|
|
||||||
try:
|
|
||||||
if p.exists():
|
|
||||||
p.unlink()
|
|
||||||
deleted.append(str(p))
|
|
||||||
except OSError as exc:
|
|
||||||
log.warning("Failed to delete %s: %s", p, exc)
|
|
||||||
row.status = STATUS_PRUNED
|
|
||||||
s.add(row)
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
log.info(
|
|
||||||
"Pruned old backups",
|
|
||||||
extra={
|
|
||||||
"deleted_count": len(deleted),
|
|
||||||
"cutoff": cutoff.isoformat(),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
return deleted
|
|
||||||
|
|
||||||
def status(self) -> dict:
|
|
||||||
"""Snapshot of the backup subsystem for ``GET /api/admin/backup/status``."""
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
from sqlalchemy import func
|
|
||||||
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
total = s.query(func.count(DbBackup.id)).scalar() or 0
|
|
||||||
ok_count = s.query(func.count(DbBackup.id)).filter(
|
|
||||||
DbBackup.status == STATUS_OK,
|
|
||||||
).scalar() or 0
|
|
||||||
error_count = s.query(func.count(DbBackup.id)).filter(
|
|
||||||
DbBackup.status == STATUS_ERROR,
|
|
||||||
).scalar() or 0
|
|
||||||
pruned_count = s.query(func.count(DbBackup.id)).filter(
|
|
||||||
DbBackup.status == STATUS_PRUNED,
|
|
||||||
).scalar() or 0
|
|
||||||
last_row = (
|
|
||||||
s.query(DbBackup)
|
|
||||||
.filter(DbBackup.status.in_([STATUS_OK, STATUS_ERROR]))
|
|
||||||
.order_by(DbBackup.id.desc())
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
last_ok_row = (
|
|
||||||
s.query(DbBackup)
|
|
||||||
.filter(DbBackup.status == STATUS_OK)
|
|
||||||
.order_by(DbBackup.id.desc())
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
disk_bytes = 0
|
|
||||||
try:
|
|
||||||
for p in self._backup_dir.iterdir():
|
|
||||||
if p.is_file() and p.suffix == ".bin":
|
|
||||||
disk_bytes += p.stat().st_size
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
return {
|
|
||||||
"backup_dir": str(self._backup_dir),
|
|
||||||
"retention_days": self._retention_days,
|
|
||||||
"totals": {
|
|
||||||
"all": total,
|
|
||||||
"ok": ok_count,
|
|
||||||
"error": error_count,
|
|
||||||
"pruned": pruned_count,
|
|
||||||
},
|
|
||||||
"disk_bytes": disk_bytes,
|
|
||||||
"last_backup_at": (
|
|
||||||
last_row.created_at.isoformat() if last_row and last_row.created_at else None
|
|
||||||
),
|
|
||||||
"last_backup_status": last_row.status if last_row else None,
|
|
||||||
"last_ok_backup_at": (
|
|
||||||
last_ok_row.created_at.isoformat()
|
|
||||||
if last_ok_row and last_ok_row.created_at else None
|
|
||||||
),
|
|
||||||
"used_fallback_key": self._used_fallback,
|
|
||||||
}
|
|
||||||
|
|
||||||
# ---- Internals --------------------------------------------------------
|
|
||||||
|
|
||||||
def _ensure_key(self) -> bytes:
|
|
||||||
"""Derive (or return cached) AES key. Triggers fallback + WARNING log
|
|
||||||
if no passphrase was provided at construction time.
|
|
||||||
|
|
||||||
If a passphrase is set but no salt was passed at construction,
|
|
||||||
look one up from the Keychain (``backup.salt`` account). On
|
|
||||||
a fresh install, generate + persist a salt on first use so
|
|
||||||
subsequent invocations derive the same key.
|
|
||||||
"""
|
|
||||||
if self._key is not None:
|
|
||||||
return self._key
|
|
||||||
if self._passphrase:
|
|
||||||
salt = self._salt
|
|
||||||
if salt is None:
|
|
||||||
# Try the Keychain.
|
|
||||||
stored = secrets_mod.get_secret(KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
|
||||||
if stored:
|
|
||||||
salt = bytes.fromhex(stored.strip())
|
|
||||||
else:
|
|
||||||
# First run: generate + persist.
|
|
||||||
salt = os.urandom(backup_mod.SALT_LEN)
|
|
||||||
secrets_mod.set_secret(
|
|
||||||
KEYCHAIN_BACKUP_SALT_ACCOUNT,
|
|
||||||
salt.hex(),
|
|
||||||
)
|
|
||||||
log.info(
|
|
||||||
"Generated + persisted backup salt to Keychain "
|
|
||||||
"(account %r)",
|
|
||||||
KEYCHAIN_BACKUP_SALT_ACCOUNT,
|
|
||||||
)
|
|
||||||
self._key = backup_mod.derive_key(self._passphrase, salt)
|
|
||||||
return self._key
|
|
||||||
|
|
||||||
# Fallback: derive from SQLCipher DB key. This is degraded
|
|
||||||
# security (the SQLCipher key is meant to unlock the DB, not
|
|
||||||
# the backup), but it's strictly better than plaintext.
|
|
||||||
from cyclone import db_crypto
|
|
||||||
db_key = db_crypto.get_db_key() if db_crypto.is_encryption_enabled() else None
|
|
||||||
if not db_key:
|
|
||||||
# No passphrase AND no SQLCipher key — refuse.
|
|
||||||
raise BackupError(
|
|
||||||
"no backup passphrase set and SQLCipher is not enabled; "
|
|
||||||
"either set a backup passphrase in the Keychain or "
|
|
||||||
"enable SQLCipher encryption",
|
|
||||||
)
|
|
||||||
log.warning(
|
|
||||||
"Backup using fallback key derived from SQLCipher DB key "
|
|
||||||
"(no separate backup passphrase set); set one via "
|
|
||||||
"`cyclone backup init-passphrase` for stronger isolation",
|
|
||||||
extra={"key_source": "sqlcipher_fallback"},
|
|
||||||
)
|
|
||||||
self._used_fallback = True
|
|
||||||
self._key = backup_mod.derive_key(db_key, backup_mod.FALLBACK_SALT)
|
|
||||||
return self._key
|
|
||||||
|
|
||||||
def _sqlite_backup_to(self, target_path: Path) -> None:
|
|
||||||
"""Run SQLite's online ``.backup()`` against the live engine.
|
|
||||||
|
|
||||||
Works for both plain SQLite and SQLCipher because sqlcipher3
|
|
||||||
is API-compatible with sqlite3. The ``.backup()`` API takes
|
|
||||||
a *target* connection; we make a fresh sqlite3 connection to
|
|
||||||
the target file (which doesn't exist yet) and copy into it.
|
|
||||||
"""
|
|
||||||
url = self._db_url or db._resolve_url()
|
|
||||||
if not url.startswith("sqlite"):
|
|
||||||
raise BackupError(
|
|
||||||
f"only sqlite URLs are supported for online backup; got {url!r}",
|
|
||||||
)
|
|
||||||
# Drive the backup off the live engine so we capture the
|
|
||||||
# current state of all tables atomically (SQLite's .backup
|
|
||||||
# holds a read lock on the source for the duration).
|
|
||||||
engine = db.engine() # raises RuntimeError if init_db() wasn't called
|
|
||||||
with engine.raw_connection() as raw:
|
|
||||||
src_conn = raw.driver_connection # sqlite3.Connection / sqlcipher3.Connection
|
|
||||||
if target_path.exists():
|
|
||||||
target_path.unlink()
|
|
||||||
dst_conn = sqlite3.connect(str(target_path))
|
|
||||||
try:
|
|
||||||
src_conn.backup(dst_conn)
|
|
||||||
finally:
|
|
||||||
dst_conn.close()
|
|
||||||
|
|
||||||
def _count_tables_in_blob(self, plaintext: bytes) -> int:
|
|
||||||
"""Open the decrypted DB in-memory and count user tables."""
|
|
||||||
tmp = self._backup_dir / f".count-tables-{_secrets.token_hex(4)}.db"
|
|
||||||
try:
|
|
||||||
tmp.write_bytes(plaintext)
|
|
||||||
conn = sqlite3.connect(str(tmp))
|
|
||||||
try:
|
|
||||||
rows = conn.execute(
|
|
||||||
"SELECT count(*) FROM sqlite_master "
|
|
||||||
"WHERE type='table' AND name NOT LIKE 'sqlite_%'",
|
|
||||||
).fetchone()
|
|
||||||
return int(rows[0])
|
|
||||||
finally:
|
|
||||||
conn.close()
|
|
||||||
finally:
|
|
||||||
tmp.unlink(missing_ok=True)
|
|
||||||
|
|
||||||
def _live_fingerprint_and_count(self) -> tuple[str, int]:
|
|
||||||
"""Fingerprint + table count of the *current* live DB."""
|
|
||||||
try:
|
|
||||||
engine = db.engine()
|
|
||||||
except RuntimeError:
|
|
||||||
return "", 0
|
|
||||||
# Use a temp-file .backup so we don't have to worry about
|
|
||||||
# online-vs-offline semantics.
|
|
||||||
tmp = self._backup_dir / f".live-fp-{_secrets.token_hex(4)}.db"
|
|
||||||
try:
|
|
||||||
with engine.raw_connection() as raw:
|
|
||||||
conn = raw.driver_connection
|
|
||||||
if tmp.exists():
|
|
||||||
tmp.unlink()
|
|
||||||
dst = sqlite3.connect(str(tmp))
|
|
||||||
try:
|
|
||||||
conn.backup(dst)
|
|
||||||
finally:
|
|
||||||
dst.close()
|
|
||||||
data = tmp.read_bytes()
|
|
||||||
return backup_mod.fingerprint(data), self._count_tables_in_blob(data)
|
|
||||||
finally:
|
|
||||||
tmp.unlink(missing_ok=True)
|
|
||||||
|
|
||||||
def _live_db_path(self) -> Optional[Path]:
|
|
||||||
"""Resolve the filesystem path of the live DB, or None for non-sqlite."""
|
|
||||||
url = self._db_url or db._resolve_url()
|
|
||||||
if not url.startswith("sqlite"):
|
|
||||||
return None
|
|
||||||
# Strip the driver prefix: sqlite:///abs or sqlite:///./rel
|
|
||||||
prefix = "sqlite:///"
|
|
||||||
if url.startswith(prefix):
|
|
||||||
return Path(url[len(prefix):])
|
|
||||||
if url.startswith("sqlite://"):
|
|
||||||
# sqlite://./relative/path -> Path("./relative/path")
|
|
||||||
return Path(url[len("sqlite://"):])
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _read_sidecar(self, filename: str) -> Optional[backup_mod.Sidecar]:
|
|
||||||
p = self._backup_dir / backup_mod.sidecar_filename(filename)
|
|
||||||
if not p.exists():
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return backup_mod.Sidecar.from_json(p.read_text())
|
|
||||||
except (json.JSONDecodeError, KeyError, ValueError) as exc:
|
|
||||||
log.warning("Sidecar %s is malformed: %s", p, exc)
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _row_to_record(self, row) -> BackupRecord:
|
|
||||||
"""ORM row → BackupRecord. Reads key_fingerprint from the sidecar if present."""
|
|
||||||
sidecar = self._read_sidecar(row.filename)
|
|
||||||
return BackupRecord(
|
|
||||||
id=row.id,
|
|
||||||
filename=row.filename,
|
|
||||||
backup_dir=row.backup_dir,
|
|
||||||
size_bytes=row.size_bytes or 0,
|
|
||||||
db_fingerprint=row.db_fingerprint or "",
|
|
||||||
table_count=row.table_count or 0,
|
|
||||||
created_at=row.created_at,
|
|
||||||
completed_at=row.completed_at,
|
|
||||||
status=row.status,
|
|
||||||
error_message=row.error_message,
|
|
||||||
key_fingerprint=sidecar.key_fingerprint if sidecar else "",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Module-level singleton
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
_service: Optional[BackupService] = None
|
|
||||||
|
|
||||||
|
|
||||||
def configure_backup_service(
|
|
||||||
backup_dir: Union[str, Path],
|
|
||||||
*,
|
|
||||||
passphrase: Optional[str] = None,
|
|
||||||
salt: Optional[bytes] = None,
|
|
||||||
retention_days: int = 30,
|
|
||||||
db_url: Optional[str] = None,
|
|
||||||
) -> BackupService:
|
|
||||||
"""Create (or replace) the module-level BackupService singleton."""
|
|
||||||
global _service
|
|
||||||
if _service is not None:
|
|
||||||
return _service
|
|
||||||
_service = BackupService(
|
|
||||||
backup_dir=backup_dir,
|
|
||||||
passphrase=passphrase,
|
|
||||||
salt=salt,
|
|
||||||
retention_days=retention_days,
|
|
||||||
db_url=db_url,
|
|
||||||
)
|
|
||||||
return _service
|
|
||||||
|
|
||||||
|
|
||||||
def get_backup_service() -> BackupService:
|
|
||||||
"""Return the configured BackupService. Raises RuntimeError if not set up."""
|
|
||||||
if _service is None:
|
|
||||||
raise RuntimeError("backup service not configured; call configure_backup_service() first")
|
|
||||||
return _service
|
|
||||||
|
|
||||||
|
|
||||||
def reset_backup_service_for_tests() -> None:
|
|
||||||
"""Clear the module-level singleton. Test-only."""
|
|
||||||
global _service
|
|
||||||
_service = None
|
|
||||||
@@ -93,24 +93,13 @@ class SftpClient:
|
|||||||
return self._list_inbound_paramiko()
|
return self._list_inbound_paramiko()
|
||||||
|
|
||||||
def read_file(self, remote_path: str) -> bytes:
|
def read_file(self, remote_path: str) -> bytes:
|
||||||
"""Read bytes from a remote path.
|
"""Read bytes from a remote path. Stub raises in stub mode."""
|
||||||
|
|
||||||
Stub mode: reads from ``{staging_dir}/{remote_path}``. Used by
|
|
||||||
the SP16 scheduler so it can exercise the same code path on a
|
|
||||||
workstation without a real MFT connection.
|
|
||||||
"""
|
|
||||||
if self._stub:
|
if self._stub:
|
||||||
return self._read_file_stub(remote_path)
|
raise RuntimeError(
|
||||||
|
"Stub SFTP cannot read remote files. Use the local staging dir."
|
||||||
|
)
|
||||||
return self._read_file_paramiko(remote_path)
|
return self._read_file_paramiko(remote_path)
|
||||||
|
|
||||||
def _read_file_stub(self, remote_path: str) -> bytes:
|
|
||||||
"""Read bytes from ``{staging_dir}/{remote_path}`` (SP16 stub)."""
|
|
||||||
staging = Path(self._block.staging_dir).resolve()
|
|
||||||
target = staging / remote_path.lstrip("/")
|
|
||||||
if not target.is_file():
|
|
||||||
raise FileNotFoundError(f"inbound stub file not found: {target}")
|
|
||||||
return target.read_bytes()
|
|
||||||
|
|
||||||
def get_secret(self, name: str) -> Optional[str]:
|
def get_secret(self, name: str) -> Optional[str]:
|
||||||
"""Fetch the auth secret from Keychain. Returns the stub secret if absent."""
|
"""Fetch the auth secret from Keychain. Returns the stub secret if absent."""
|
||||||
value = secrets.get_secret(name)
|
value = secrets.get_secret(name)
|
||||||
|
|||||||
+3
-363
@@ -3,13 +3,11 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
import os
|
|
||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import click
|
import click
|
||||||
|
|
||||||
from cyclone.logging_config import setup_logging
|
|
||||||
from cyclone.parsers.exceptions import CycloneParseError, CycloneValidationError
|
from cyclone.parsers.exceptions import CycloneParseError, CycloneValidationError
|
||||||
from cyclone.parsers.payer import PayerConfig, PayerConfig835
|
from cyclone.parsers.payer import PayerConfig, PayerConfig835
|
||||||
from cyclone.parsers.parse_837 import parse as parse_837_text
|
from cyclone.parsers.parse_837 import parse as parse_837_text
|
||||||
@@ -43,35 +41,8 @@ def _payer_835(name: str) -> PayerConfig835:
|
|||||||
|
|
||||||
|
|
||||||
@click.group()
|
@click.group()
|
||||||
@click.option(
|
def main() -> None:
|
||||||
"--log-format",
|
|
||||||
default=None,
|
|
||||||
type=click.Choice(["json", "dev"]),
|
|
||||||
help="Log format (default: json; honors CYCLONE_LOG_JSON).",
|
|
||||||
)
|
|
||||||
@click.option(
|
|
||||||
"--log-file",
|
|
||||||
default=None,
|
|
||||||
type=click.Path(dir_okay=False, path_type=Path),
|
|
||||||
help="Optional rotating log file (honors CYCLONE_LOG_FILE).",
|
|
||||||
)
|
|
||||||
@click.pass_context
|
|
||||||
def main(ctx: click.Context, log_format: str | None, log_file: Path | None) -> None:
|
|
||||||
"""Cyclone EDI suite — X12 parser."""
|
"""Cyclone EDI suite — X12 parser."""
|
||||||
# SP18: structured JSON logging. Run once per CLI invocation; each
|
|
||||||
# subcommand still gets its own --log-level to override.
|
|
||||||
json_format = True
|
|
||||||
if log_format == "dev":
|
|
||||||
json_format = False
|
|
||||||
elif os.environ.get("CYCLONE_LOG_JSON", "").lower() in ("false", "0", "no"):
|
|
||||||
json_format = False
|
|
||||||
setup_logging(
|
|
||||||
level=os.environ.get("CYCLONE_LOG_LEVEL", "INFO"),
|
|
||||||
log_file=str(log_file) if log_file else None,
|
|
||||||
json_format=json_format,
|
|
||||||
)
|
|
||||||
# Stash on context so subcommands can read it.
|
|
||||||
ctx.ensure_object(dict)
|
|
||||||
|
|
||||||
|
|
||||||
@main.command("parse-837")
|
@main.command("parse-837")
|
||||||
@@ -92,9 +63,7 @@ def parse_837(
|
|||||||
log_level: str,
|
log_level: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Parse an X12 837P file into one JSON per claim."""
|
"""Parse an X12 837P file into one JSON per claim."""
|
||||||
# SP18: re-run setup so per-command --log-level overrides the
|
logging.basicConfig(level=getattr(logging, log_level))
|
||||||
# group default. ``setup_logging`` is idempotent.
|
|
||||||
setup_logging(level=log_level)
|
|
||||||
|
|
||||||
text = input_file.read_text()
|
text = input_file.read_text()
|
||||||
config = _payer(payer)
|
config = _payer(payer)
|
||||||
@@ -164,9 +133,7 @@ def parse_835(
|
|||||||
log_level: str,
|
log_level: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Parse an X12 835 ERA file into one JSON per claim payment."""
|
"""Parse an X12 835 ERA file into one JSON per claim payment."""
|
||||||
# SP18: re-run setup so per-command --log-level overrides the
|
logging.basicConfig(level=getattr(logging, log_level))
|
||||||
# group default. ``setup_logging`` is idempotent.
|
|
||||||
setup_logging(level=log_level)
|
|
||||||
|
|
||||||
text = input_file.read_text()
|
text = input_file.read_text()
|
||||||
config = _payer_835(payer)
|
config = _payer_835(payer)
|
||||||
@@ -228,332 +195,5 @@ def _count_issues(report) -> dict[str, int]:
|
|||||||
return counts
|
return counts
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# SP20: `cyclone validate-npi` + `cyclone validate-tax-id`
|
|
||||||
#
|
|
||||||
# Pure local validators. No DB, no Keychain, no network — operators can
|
|
||||||
# run them on a developer laptop without standing up the full Cyclone
|
|
||||||
# stack. Exit code is 0 (valid) or 1 (invalid) so they compose with
|
|
||||||
# shell scripting / CI gates.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@main.command("validate-npi")
|
|
||||||
@click.argument("npi")
|
|
||||||
@click.option("--log-level", default="WARNING", show_default=True, type=click.Choice(["DEBUG", "INFO", "WARNING", "ERROR"]))
|
|
||||||
def validate_npi_cmd(npi: str, log_level: str) -> None:
|
|
||||||
"""Validate a 10-digit NPI's Luhn checksum locally (SP20).
|
|
||||||
|
|
||||||
Exit 0 if valid, 1 if not. No logging of the value itself — NPIs
|
|
||||||
are PHI under HIPAA, so the operator's CLI history is the only
|
|
||||||
audit trail.
|
|
||||||
"""
|
|
||||||
# SP18: re-run so --log-level overrides the group default.
|
|
||||||
setup_logging(level=log_level)
|
|
||||||
from cyclone.npi import is_valid_npi
|
|
||||||
if is_valid_npi(npi):
|
|
||||||
click.echo(f"OK: {len(npi)}-digit NPI passes Luhn checksum")
|
|
||||||
return
|
|
||||||
click.echo(f"INVALID: {npi!r} fails NPI Luhn checksum", err=True)
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
@main.command("validate-tax-id")
|
|
||||||
@click.argument("tax_id")
|
|
||||||
@click.option("--log-level", default="WARNING", show_default=True, type=click.Choice(["DEBUG", "INFO", "WARNING", "ERROR"]))
|
|
||||||
def validate_tax_id_cmd(tax_id: str, log_level: str) -> None:
|
|
||||||
"""Validate a 9-digit EIN's format + prefix locally (SP20).
|
|
||||||
|
|
||||||
Accepts both ``XX-XXXXXXX`` and ``XXXXXXXXX``. Exit 0 if valid,
|
|
||||||
1 if not. EIN is sensitive (PII), so we don't echo the value back
|
|
||||||
on failure — only the validation verdict.
|
|
||||||
"""
|
|
||||||
# SP18: re-run so --log-level overrides the group default.
|
|
||||||
setup_logging(level=log_level)
|
|
||||||
from cyclone.npi import is_valid_tax_id, normalize_tax_id
|
|
||||||
plain = normalize_tax_id(tax_id)
|
|
||||||
if plain is None:
|
|
||||||
click.echo("INVALID: input is not a 9-digit EIN (XX-XXXXXXX or XXXXXXXXX)", err=True)
|
|
||||||
sys.exit(1)
|
|
||||||
if is_valid_tax_id(tax_id):
|
|
||||||
click.echo(f"OK: 9-digit EIN (normalized={plain})")
|
|
||||||
return
|
|
||||||
click.echo(
|
|
||||||
f"INVALID: 9-digit EIN has reserved prefix ({plain[:2]}); EIN is not assignable by IRS",
|
|
||||||
err=True,
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
main()
|
main()
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# SP17: `cyclone backup` subcommands
|
|
||||||
#
|
|
||||||
# Operator-facing backup management. Mirrors the API surface but runs
|
|
||||||
# standalone (no FastAPI app needed) for cron / scripting / DR drills.
|
|
||||||
# Each subcommand initializes the DB + BackupService; if the
|
|
||||||
# service isn't configured (no Keychain passphrase etc.) the operator
|
|
||||||
# gets a clear error.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@main.group()
|
|
||||||
def backup() -> None:
|
|
||||||
"""Encrypted DB backup management (SP17)."""
|
|
||||||
|
|
||||||
|
|
||||||
@backup.command("init-passphrase")
|
|
||||||
@click.option("--passphrase", required=True, help="The passphrase to set (will prompt if omitted)")
|
|
||||||
@click.option("--from-stdin", is_flag=True, help="Read passphrase from stdin instead of the argument")
|
|
||||||
def backup_init_passphrase(passphrase: str, from_stdin: bool) -> None:
|
|
||||||
"""Set the backup encryption passphrase in the macOS Keychain.
|
|
||||||
|
|
||||||
Generates a fresh salt and stores both the passphrase (account
|
|
||||||
``backup.passphrase``) and the salt (account ``backup.salt``)
|
|
||||||
under service ``cyclone``. Cyclone's BackupService reads them
|
|
||||||
at startup. If the passphrase account is missing, the service
|
|
||||||
falls back to deriving a key from the SQLCipher DB key
|
|
||||||
(degraded posture, logged at WARNING).
|
|
||||||
"""
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
from getpass import getpass
|
|
||||||
import os as _os
|
|
||||||
|
|
||||||
if from_stdin:
|
|
||||||
pp = getpass("Backup passphrase: ").strip()
|
|
||||||
pp2 = getpass("Confirm: ").strip()
|
|
||||||
if not pp or pp != pp2:
|
|
||||||
click.echo("passphrase empty or mismatch", err=True)
|
|
||||||
sys.exit(2)
|
|
||||||
else:
|
|
||||||
pp = passphrase
|
|
||||||
|
|
||||||
if not pp or len(pp) < 12:
|
|
||||||
click.echo("passphrase must be at least 12 characters", err=True)
|
|
||||||
sys.exit(2)
|
|
||||||
|
|
||||||
if not secrets_mod.set_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT, pp):
|
|
||||||
click.echo("failed to store passphrase in Keychain", err=True)
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
# Generate + persist a fresh salt. Same value must be used by
|
|
||||||
# every subsequent invocation that uses this passphrase.
|
|
||||||
salt = _os.urandom(16)
|
|
||||||
if not secrets_mod.set_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT, salt.hex()):
|
|
||||||
click.echo(
|
|
||||||
"WARN: passphrase stored but salt write failed; backups may be unrecoverable",
|
|
||||||
err=True,
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
click.echo(
|
|
||||||
f"passphrase stored in Keychain account {svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT!r}\n"
|
|
||||||
f"salt stored in Keychain account {svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_backup_dir(cli_override: str | None) -> "Path":
|
|
||||||
"""Resolve the backup directory: --backup-dir > $CYCLONE_BACKUP_DIR > default."""
|
|
||||||
import os as _os
|
|
||||||
from pathlib import Path as _Path
|
|
||||||
from cyclone import db as db_mod
|
|
||||||
if cli_override:
|
|
||||||
return _Path(cli_override)
|
|
||||||
env = _os.environ.get("CYCLONE_BACKUP_DIR")
|
|
||||||
if env:
|
|
||||||
return _Path(env)
|
|
||||||
return _Path(db_mod.DEFAULT_DB_PATH.parent / "backups")
|
|
||||||
|
|
||||||
|
|
||||||
@backup.command("create")
|
|
||||||
@click.option("--backup-dir", default=None, help="Override CYCLONE_BACKUP_DIR (default: ~/.local/share/cyclone/backups)")
|
|
||||||
@click.option("--retention-days", default=None, type=int, help="Override CYCLONE_BACKUP_RETENTION_DAYS for this run's prune")
|
|
||||||
def backup_create(backup_dir: str | None, retention_days: int | None) -> None:
|
|
||||||
"""Take an encrypted backup right now."""
|
|
||||||
from cyclone import db as db_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
|
|
||||||
db_mod.init_db()
|
|
||||||
|
|
||||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
|
||||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
|
||||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
|
||||||
target_dir = _resolve_backup_dir(backup_dir)
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
svc = svc_mod.configure_backup_service(
|
|
||||||
backup_dir=target_dir,
|
|
||||||
passphrase=passphrase,
|
|
||||||
salt=salt,
|
|
||||||
retention_days=retention_days or 30,
|
|
||||||
)
|
|
||||||
result = svc.create_now()
|
|
||||||
click.echo(
|
|
||||||
f"created backup id={result.backup.id} filename={result.backup.filename} "
|
|
||||||
f"size={result.backup.size_bytes}B fp={result.backup.db_fingerprint[:24]}..."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@backup.command("list")
|
|
||||||
@click.option("--limit", default=50, show_default=True)
|
|
||||||
@click.option("--status", default=None, help="Filter: ok|error|pending|pruned")
|
|
||||||
def backup_list(limit: int, status: str | None) -> None:
|
|
||||||
"""List existing backups (newest first)."""
|
|
||||||
from cyclone import db as db_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
|
|
||||||
db_mod.init_db()
|
|
||||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
|
||||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
|
||||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
svc = svc_mod.configure_backup_service(
|
|
||||||
backup_dir=_resolve_backup_dir(None),
|
|
||||||
passphrase=passphrase,
|
|
||||||
salt=salt,
|
|
||||||
)
|
|
||||||
rows = svc.list_backups(limit=limit, status=status)
|
|
||||||
if not rows:
|
|
||||||
click.echo("(no backups)")
|
|
||||||
return
|
|
||||||
for r in rows:
|
|
||||||
click.echo(
|
|
||||||
f"{r.id:4d} {r.status:7s} {r.created_at.isoformat() if r.created_at else '-'} "
|
|
||||||
f"{r.size_bytes:>10d}B {r.filename} fp={r.db_fingerprint[:24] or '-':<24}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@backup.command("verify")
|
|
||||||
@click.argument("backup_id", type=int)
|
|
||||||
def backup_verify(backup_id: int) -> None:
|
|
||||||
"""Decrypt + checksum-verify a backup."""
|
|
||||||
from cyclone import db as db_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
|
|
||||||
db_mod.init_db()
|
|
||||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
|
||||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
|
||||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
svc = svc_mod.configure_backup_service(
|
|
||||||
backup_dir=_resolve_backup_dir(None),
|
|
||||||
passphrase=passphrase,
|
|
||||||
salt=salt,
|
|
||||||
)
|
|
||||||
v = svc.verify(backup_id)
|
|
||||||
if v.ok:
|
|
||||||
click.echo(f"OK: id={v.backup_id} fp={v.actual_fingerprint[:24]}... table_count={v.table_count}")
|
|
||||||
return
|
|
||||||
click.echo(
|
|
||||||
f"FAIL: id={v.backup_id} reason={v.reason} "
|
|
||||||
f"expected={v.expected_fingerprint[:24] if v.expected_fingerprint else '-'}... "
|
|
||||||
f"actual={v.actual_fingerprint[:24] if v.actual_fingerprint else '-'}...",
|
|
||||||
err=True,
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
@backup.command("restore")
|
|
||||||
@click.argument("backup_id", type=int)
|
|
||||||
@click.option("--yes", is_flag=True, help="Skip the interactive confirm prompt")
|
|
||||||
@click.option("--actor", default="operator-cli", show_default=True)
|
|
||||||
def backup_restore(backup_id: int, yes: bool, actor: str) -> None:
|
|
||||||
"""Restore the live DB from a backup (two-step, requires --yes)."""
|
|
||||||
from cyclone import db as db_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
|
|
||||||
db_mod.init_db()
|
|
||||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
|
||||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
|
||||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
svc = svc_mod.configure_backup_service(
|
|
||||||
backup_dir=_resolve_backup_dir(None),
|
|
||||||
passphrase=passphrase,
|
|
||||||
salt=salt,
|
|
||||||
)
|
|
||||||
|
|
||||||
click.echo(f"Initiating restore from backup {backup_id}...")
|
|
||||||
init = svc.restore_initiate(backup_id)
|
|
||||||
click.echo(
|
|
||||||
f" backup: {init.filename} ({init.size_bytes} bytes)\n"
|
|
||||||
f" fp: {init.db_fingerprint[:24]}...\n"
|
|
||||||
f" tables: {init.table_count}\n"
|
|
||||||
f" current: fp={init.current_db_fingerprint[:24] if init.current_db_fingerprint else '-'}... "
|
|
||||||
f"tables={init.current_table_count}\n"
|
|
||||||
f" token ttl: {(init.expires_at - __import__('datetime').datetime.now(__import__('datetime').timezone.utc)).total_seconds():.0f}s"
|
|
||||||
)
|
|
||||||
|
|
||||||
if not yes:
|
|
||||||
click.confirm(
|
|
||||||
"Replace the live DB with this backup? "
|
|
||||||
"This will dispose the engine and rebuild it.",
|
|
||||||
abort=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
click.echo("Confirming restore...")
|
|
||||||
result = svc.restore_confirm(backup_id, init.restore_token, actor=actor)
|
|
||||||
click.echo(
|
|
||||||
f"OK: restored from fp={result.restored_from_fingerprint[:24]}... "
|
|
||||||
f"to fp={result.new_db_fingerprint[:24]}... at {result.restored_at.isoformat()}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@backup.command("prune")
|
|
||||||
@click.option("--retention-days", default=None, type=int)
|
|
||||||
@click.option("--yes", is_flag=True, help="Skip the confirm prompt")
|
|
||||||
def backup_prune(retention_days: int | None, yes: bool) -> None:
|
|
||||||
"""Apply the retention policy (delete old backups)."""
|
|
||||||
from cyclone import db as db_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
|
|
||||||
db_mod.init_db()
|
|
||||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
|
||||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
|
||||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
svc = svc_mod.configure_backup_service(
|
|
||||||
backup_dir=_resolve_backup_dir(None),
|
|
||||||
passphrase=passphrase,
|
|
||||||
salt=salt,
|
|
||||||
retention_days=retention_days or 30,
|
|
||||||
)
|
|
||||||
if not yes:
|
|
||||||
click.confirm(
|
|
||||||
f"Delete all backups older than {svc._retention_days} days?",
|
|
||||||
abort=True,
|
|
||||||
)
|
|
||||||
deleted = svc.prune()
|
|
||||||
click.echo(f"Deleted {len(deleted)} file(s):")
|
|
||||||
for p in deleted:
|
|
||||||
click.echo(f" {p}")
|
|
||||||
|
|
||||||
|
|
||||||
@backup.command("status")
|
|
||||||
def backup_status() -> None:
|
|
||||||
"""Print the backup subsystem status snapshot."""
|
|
||||||
from cyclone import db as db_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
|
|
||||||
db_mod.init_db()
|
|
||||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
|
||||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
|
||||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
svc = svc_mod.configure_backup_service(
|
|
||||||
backup_dir=_resolve_backup_dir(None),
|
|
||||||
passphrase=passphrase,
|
|
||||||
salt=salt,
|
|
||||||
)
|
|
||||||
snap = svc.status()
|
|
||||||
import json
|
|
||||||
click.echo(json.dumps(snap, indent=2, default=str))
|
|
||||||
|
|||||||
@@ -676,89 +676,6 @@ class AuditLog(Base):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# SP16: inbound MFT scheduler
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class ProcessedInboundFile(Base):
|
|
||||||
"""One row per inbound MFT file the scheduler has downloaded.
|
|
||||||
|
|
||||||
SP16. Lets the scheduler be idempotent: a re-tick or restart must
|
|
||||||
not re-parse the same inbound file. The unique index on
|
|
||||||
(sftp_block_name, name) prevents duplicate inserts and lets the
|
|
||||||
scheduler fast-skip already-processed files via a SELECT.
|
|
||||||
|
|
||||||
Status values:
|
|
||||||
* ok - parsed cleanly, results persisted to the store
|
|
||||||
* error - parser raised; error_message captured
|
|
||||||
* skipped - file_type not in the scheduler's allowed set
|
|
||||||
* pending - file was downloaded but a downstream step failed;
|
|
||||||
the scheduler retries on the next tick
|
|
||||||
"""
|
|
||||||
|
|
||||||
__tablename__ = "processed_inbound_files"
|
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
|
||||||
sftp_block_name: Mapped[str] = mapped_column(String(128), nullable=False)
|
|
||||||
name: Mapped[str] = mapped_column(String(256), nullable=False)
|
|
||||||
size: Mapped[int] = mapped_column(Integer, nullable=False)
|
|
||||||
modified_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
|
||||||
file_type: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
|
|
||||||
processed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
|
||||||
parser_used: Mapped[Optional[str]] = mapped_column(String(32), nullable=True)
|
|
||||||
claim_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
|
||||||
status: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
||||||
error_message: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
|
||||||
|
|
||||||
__table_args__ = (
|
|
||||||
Index(
|
|
||||||
"ux_processed_inbound_files_block_name",
|
|
||||||
"sftp_block_name", "name", unique=True,
|
|
||||||
),
|
|
||||||
Index("ix_processed_inbound_files_processed_at", "processed_at"),
|
|
||||||
Index("ix_processed_inbound_files_status", "status"),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# SP17: encrypted backup metadata
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class DbBackup(Base):
|
|
||||||
"""One row per encrypted backup the BackupService has taken.
|
|
||||||
|
|
||||||
The actual encrypted blob lives in a directory outside the DB
|
|
||||||
(``~/.local/share/cyclone/backups/`` by default); this table is
|
|
||||||
the index. Status values: ``pending``, ``ok``, ``error``,
|
|
||||||
``pruned``.
|
|
||||||
|
|
||||||
SP17. The unique index on ``(backup_dir, filename)`` makes a
|
|
||||||
duplicate ``create_now()`` race fail cleanly with an
|
|
||||||
IntegrityError instead of clobbering an existing backup.
|
|
||||||
"""
|
|
||||||
|
|
||||||
__tablename__ = "db_backups"
|
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
|
||||||
filename: Mapped[str] = mapped_column(String(128), nullable=False)
|
|
||||||
backup_dir: Mapped[str] = mapped_column(String(512), nullable=False)
|
|
||||||
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
|
||||||
db_fingerprint: Mapped[Optional[str]] = mapped_column(String(80), nullable=True)
|
|
||||||
table_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
|
||||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
|
||||||
completed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
|
||||||
status: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
||||||
error_message: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
|
||||||
|
|
||||||
__table_args__ = (
|
|
||||||
Index("ux_db_backups_filename", "backup_dir", "filename", unique=True),
|
|
||||||
Index("ix_db_backups_created_at", "created_at"),
|
|
||||||
Index("ix_db_backups_status", "status"),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# SP9: providers, payers, payer_configs, clearhouse
|
# SP9: providers, payers, payer_configs, clearhouse
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -4,8 +4,8 @@ SP9. Source-of-truth spec:
|
|||||||
https://hcpf.colorado.gov/tp-x12-filenaming (HCPF X12 File Naming Standards Quick Guide)
|
https://hcpf.colorado.gov/tp-x12-filenaming (HCPF X12 File Naming Standards Quick Guide)
|
||||||
|
|
||||||
Outbound (we send):
|
Outbound (we send):
|
||||||
tp{tpid}-{transaction_type}-{yyyymmddhhmmssSSS_MT}-1of1.{ext}
|
{tpid}-{transaction_type}-{yyyymmddhhmmssSSS_MT}-1of1.{ext}
|
||||||
Example: tp11525703-837P-20260620132243505-1of1.x12
|
Example: 11525703-837P-20260620132243505-1of1.x12
|
||||||
|
|
||||||
Inbound (HPE sends to our ToHPE):
|
Inbound (HPE sends to our ToHPE):
|
||||||
TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
|
TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
|
||||||
@@ -28,15 +28,14 @@ from cyclone.providers import InboundFilename
|
|||||||
# Regexes
|
# Regexes
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
# Outbound: tp11525703-837P-20260620132243505-1of1.x12
|
# Outbound: 11525703-837P-20260620132243505-1of1.x12
|
||||||
# - tp: literal "tp" prefix
|
|
||||||
# - tpid: 1+ digits
|
# - tpid: 1+ digits
|
||||||
# - tx: 1+ alnum
|
# - tx: 1+ alnum
|
||||||
# - ts: 17 digits (yyyymmddhhmmssSSS)
|
# - ts: 17 digits (yyyymmddhhmmssSSS)
|
||||||
# - seq: literal "1of1"
|
# - seq: literal "1of1"
|
||||||
# - ext: 1+ alnum
|
# - ext: 1+ alnum
|
||||||
OUTBOUND_RE = re.compile(
|
OUTBOUND_RE = re.compile(
|
||||||
r"^tp(?P<tpid>\d+)-(?P<tx>[A-Z0-9]+)-(?P<ts>\d{17})-1of1\.(?P<ext>[A-Za-z0-9]+)$"
|
r"^(?P<tpid>\d+)-(?P<tx>[A-Z0-9]+)-(?P<ts>\d{17})-1of1\.(?P<ext>[A-Za-z0-9]+)$"
|
||||||
)
|
)
|
||||||
|
|
||||||
# Inbound: TP11525703-837P_M019048402-20260520231513488-1of1_999.x12
|
# Inbound: TP11525703-837P_M019048402-20260520231513488-1of1_999.x12
|
||||||
@@ -80,8 +79,7 @@ def build_outbound_filename(
|
|||||||
time in ``America/Denver`` is used.
|
time in ``America/Denver`` is used.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Filename like "tp11525703-837P-20260620132243505-1of1.x12"
|
Filename like "11525703-837P-20260620132243505-1of1.x12"
|
||||||
(note the ``tp`` prefix per HCPF outbound spec).
|
|
||||||
|
|
||||||
Raises:
|
Raises:
|
||||||
ValueError: If tpid is non-numeric, tx contains invalid chars, or
|
ValueError: If tpid is non-numeric, tx contains invalid chars, or
|
||||||
@@ -101,10 +99,7 @@ def build_outbound_filename(
|
|||||||
# Format: yyyymmddhhmmssSSS — 17 digits total
|
# Format: yyyymmddhhmmssSSS — 17 digits total
|
||||||
ts = now_mt.strftime("%Y%m%d%H%M%S") + f"{now_mt.microsecond // 1000:03d}"
|
ts = now_mt.strftime("%Y%m%d%H%M%S") + f"{now_mt.microsecond // 1000:03d}"
|
||||||
assert len(ts) == 17
|
assert len(ts) == 17
|
||||||
# Per HCPF outbound spec, prefix is "tp" + tpid. Matches the format
|
return f"{tpid}-{tx}-{ts}-1of1.{ext}"
|
||||||
# we receive from HPE inbound (which uses uppercase TP) and the
|
|
||||||
# historical outbound prodfile naming (e.g. tp11525703-837P-...).
|
|
||||||
return f"tp{tpid}-{tx}-{ts}-1of1.{ext}"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -1,379 +0,0 @@
|
|||||||
"""SP18 — Structured JSON logging.
|
|
||||||
|
|
||||||
Wraps Python's stdlib ``logging`` to emit newline-delimited JSON
|
|
||||||
(or a dev-friendly tabular format) and to scrub obvious PHI
|
|
||||||
patterns (NPIs, SSNs, DOBs, patient names) from the message +
|
|
||||||
extra fields.
|
|
||||||
|
|
||||||
Design choices
|
|
||||||
--------------
|
|
||||||
|
|
||||||
* **No third-party deps.** stdlib ``logging`` + ``json`` + ``re``
|
|
||||||
is enough. ``loguru`` / ``structlog`` were considered; both add
|
|
||||||
a dependency for marginal gain.
|
|
||||||
|
|
||||||
* **JSON by default.** Operators running Cyclone in production
|
|
||||||
almost certainly want logs in a format their aggregator
|
|
||||||
(Loki/ELK/Vector) can parse. The dev format (``CycloneDevFormatter``)
|
|
||||||
is the opt-out for ``tail -f`` in dev.
|
|
||||||
|
|
||||||
* **Conservative PII scrubber.** Redacts unambiguous PHI patterns
|
|
||||||
only. False positives are not free — an operator's diagnostic
|
|
||||||
dump that says ``<redacted:npi>`` instead of the actual NPI
|
|
||||||
makes root-causing a parse failure harder. The scrubber can be
|
|
||||||
disabled with ``CYCLONE_LOG_NO_PII_SCRUB=1`` for tests /
|
|
||||||
forensic mode.
|
|
||||||
|
|
||||||
* **Idempotent setup.** :func:`setup_logging` can be called
|
|
||||||
multiple times (CLI re-invocation, FastAPI lifespan re-entry
|
|
||||||
under TestClient). Each call clears existing handlers on the
|
|
||||||
root logger before attaching fresh ones — so the format toggle
|
|
||||||
actually takes effect on the second call.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import sys
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from logging.handlers import RotatingFileHandler
|
|
||||||
from typing import Any, Optional
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Formatters
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# Stdlib LogRecord attributes we don't want to dump into the
|
|
||||||
# structured payload (they're noise for log consumers).
|
|
||||||
_RESERVED_LOGRECORD_ATTRS = frozenset({
|
|
||||||
"args", "asctime", "created", "exc_info", "exc_text", "filename",
|
|
||||||
"funcName", "levelname", "levelno", "lineno", "module", "msecs",
|
|
||||||
"message", "msg", "name", "pathname", "process", "processName",
|
|
||||||
"relativeCreated", "stack_info", "thread", "threadName",
|
|
||||||
"taskName",
|
|
||||||
})
|
|
||||||
|
|
||||||
|
|
||||||
class JsonFormatter(logging.Formatter):
|
|
||||||
"""Format a LogRecord as a single JSON line.
|
|
||||||
|
|
||||||
Fields:
|
|
||||||
ts — ISO 8601 UTC timestamp with milliseconds.
|
|
||||||
level — uppercase level name (INFO, WARNING, etc.).
|
|
||||||
logger — the logger name (e.g. "cyclone.scheduler").
|
|
||||||
msg — the formatted log message (after %-substitution).
|
|
||||||
extra — dict of any non-reserved LogRecord attributes.
|
|
||||||
|
|
||||||
If ``exc_info`` is set, the formatter appends a ``traceback``
|
|
||||||
field with the formatted exception text (NOT a serialized
|
|
||||||
object — just the stdlib-rendered string).
|
|
||||||
"""
|
|
||||||
|
|
||||||
def format(self, record: logging.LogRecord) -> str:
|
|
||||||
ts = datetime.fromtimestamp(record.created, tz=timezone.utc).isoformat(
|
|
||||||
timespec="milliseconds",
|
|
||||||
)
|
|
||||||
payload: dict[str, Any] = {
|
|
||||||
"ts": ts,
|
|
||||||
"level": record.levelname,
|
|
||||||
"logger": record.name,
|
|
||||||
"msg": record.getMessage(),
|
|
||||||
}
|
|
||||||
# Collect user-provided extras.
|
|
||||||
extras = {
|
|
||||||
k: v
|
|
||||||
for k, v in record.__dict__.items()
|
|
||||||
if k not in _RESERVED_LOGRECORD_ATTRS and not k.startswith("_")
|
|
||||||
}
|
|
||||||
if extras:
|
|
||||||
payload["extra"] = extras
|
|
||||||
if record.exc_info:
|
|
||||||
payload["traceback"] = self.formatException(record.exc_info)
|
|
||||||
if record.stack_info:
|
|
||||||
payload["stack"] = self.formatStack(record.stack_info)
|
|
||||||
return json.dumps(payload, default=str, sort_keys=True)
|
|
||||||
|
|
||||||
|
|
||||||
class CycloneDevFormatter(logging.Formatter):
|
|
||||||
"""Dev-friendly tabular format.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
2026-06-21T15:30:00.123Z INFO cyclone.scheduler Processed inbound foo.x12 parser=parse_999 claims=3
|
|
||||||
|
|
||||||
Same fields as ``JsonFormatter`` but human-readable. Useful for
|
|
||||||
``tail -f cyclone.log`` in dev.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def format(self, record: logging.LogRecord) -> str:
|
|
||||||
ts = datetime.fromtimestamp(record.created, tz=timezone.utc).isoformat(
|
|
||||||
timespec="milliseconds",
|
|
||||||
)
|
|
||||||
extras = {
|
|
||||||
k: v
|
|
||||||
for k, v in record.__dict__.items()
|
|
||||||
if k not in _RESERVED_LOGRECORD_ATTRS and not k.startswith("_")
|
|
||||||
}
|
|
||||||
extra_str = ""
|
|
||||||
if extras:
|
|
||||||
pairs = " ".join(f"{k}={v!r}" for k, v in extras.items())
|
|
||||||
extra_str = " " + pairs
|
|
||||||
base = f"{ts} {record.levelname:<7s} {record.name} {record.getMessage()}{extra_str}"
|
|
||||||
if record.exc_info:
|
|
||||||
base += "\n" + self.formatException(record.exc_info)
|
|
||||||
return base
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# PII scrubber
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# Conservative PHI patterns. Each pattern is (label, compiled regex,
|
|
||||||
# replacement). Some patterns use a backreference so the field name
|
|
||||||
# (e.g. "dob=") is preserved and only the value is redacted — that
|
|
||||||
# keeps the surrounding context readable in the log line.
|
|
||||||
_PII_PATTERNS: tuple[tuple[str, "re.Pattern[str]", str], ...] = (
|
|
||||||
# 10-digit NPI. Word-boundary anchored so we don't redact, e.g.,
|
|
||||||
# the "10" in "10 claims processed".
|
|
||||||
("npi", re.compile(r"\b\d{10}\b"), "<redacted:npi>"),
|
|
||||||
# SSN: NNN-NN-NNNN or NNNNNNNNN.
|
|
||||||
(
|
|
||||||
"ssn",
|
|
||||||
re.compile(r"\b\d{3}-\d{2}-\d{4}\b|\b\d{9}\b(?=[\s,;)}])"),
|
|
||||||
"<redacted:ssn>",
|
|
||||||
),
|
|
||||||
# DOB: "dob=YYYY-MM-DD" / "date_of_birth=YYYY-MM-DD". Capture the
|
|
||||||
# field name + separator, redact only the date — keeps the
|
|
||||||
# surrounding sentence readable.
|
|
||||||
(
|
|
||||||
"dob",
|
|
||||||
re.compile(
|
|
||||||
r"(?i)(\b(?:dob|date[ _]?of[ _]?birth)[:=]\s*)\d{4}-\d{2}-\d{2}"
|
|
||||||
),
|
|
||||||
r"\1<redacted:dob>",
|
|
||||||
),
|
|
||||||
# Patient name: explicit field marker, redact the whole
|
|
||||||
# "patient_name=..." chunk so the value can't leak in a quoted form.
|
|
||||||
(
|
|
||||||
"patient_name",
|
|
||||||
re.compile(
|
|
||||||
r'(?i)\bpatient[_ ]?name[:=]\s*"?[^\",\s}]+',
|
|
||||||
),
|
|
||||||
"<redacted:patient_name>",
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
# Extra-field KEYS that we treat as PHI by themselves — if a log call
|
|
||||||
# passes an extra like ``extra={"date_of_birth": "1980-04-12"}`` we
|
|
||||||
# redact the value even though the value alone isn't PHI-shaped. The
|
|
||||||
# key is the signal. Matched case-insensitively against the full key
|
|
||||||
# (with underscores normalized to spaces for "date of birth").
|
|
||||||
_PHI_EXTRA_KEYS: dict[str, str] = {
|
|
||||||
"npi": "npi",
|
|
||||||
"provider_npi": "npi",
|
|
||||||
"rendering_npi": "npi",
|
|
||||||
"billing_npi": "npi",
|
|
||||||
"ssn": "ssn",
|
|
||||||
"dob": "dob",
|
|
||||||
"date_of_birth": "dob",
|
|
||||||
"patient_name": "patient_name",
|
|
||||||
"patient first name": "patient_name",
|
|
||||||
"patient last name": "patient_name",
|
|
||||||
}
|
|
||||||
|
|
||||||
# When an extra key matches one of these, redact any string value
|
|
||||||
# wholesale (don't try to parse it — just replace).
|
|
||||||
_PHI_EXTRA_WHOLE_VALUE = {"npi", "ssn", "dob", "patient_name"}
|
|
||||||
|
|
||||||
|
|
||||||
class PiiScrubber(logging.Filter):
|
|
||||||
"""Filter that redacts obvious PHI from log records.
|
|
||||||
|
|
||||||
Walks the formatted message + every ``extra`` field value (if
|
|
||||||
it's a string) and rewrites matches to ``<redacted:<name>``.
|
|
||||||
Non-string extras are left alone (we don't try to serialize and
|
|
||||||
re-scrub dicts — too risky for false positives).
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, name: str = "pii_scrubber") -> None:
|
|
||||||
super().__init__(name)
|
|
||||||
self._enabled = True
|
|
||||||
|
|
||||||
def disable(self) -> None:
|
|
||||||
"""Disable scrubbing (for tests / forensic mode)."""
|
|
||||||
self._enabled = False
|
|
||||||
|
|
||||||
def enable(self) -> None:
|
|
||||||
self._enabled = True
|
|
||||||
|
|
||||||
def _scrub(self, text: str) -> str:
|
|
||||||
for label, pat, repl in _PII_PATTERNS:
|
|
||||||
text = pat.sub(repl, text)
|
|
||||||
return text
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _normalize_extra_key(key: str) -> set[str]:
|
|
||||||
"""Return all candidate normalizations of a key.
|
|
||||||
|
|
||||||
``date_of_birth`` should match a lookup table that uses either
|
|
||||||
``date_of_birth`` or ``date of birth`` — so return both. Same
|
|
||||||
for ``patient_name`` vs ``patient name``.
|
|
||||||
"""
|
|
||||||
norm = key.strip().lower()
|
|
||||||
spaced = norm.replace("_", " ")
|
|
||||||
return {norm, spaced}
|
|
||||||
|
|
||||||
def _redact_extra_value(self, key: str, value: Any) -> Any:
|
|
||||||
"""Redact a single extra field value if its key signals PHI."""
|
|
||||||
for norm in self._normalize_extra_key(key):
|
|
||||||
label = _PHI_EXTRA_KEYS.get(norm)
|
|
||||||
if label:
|
|
||||||
if not isinstance(value, str):
|
|
||||||
return value
|
|
||||||
return f"<redacted:{label}>"
|
|
||||||
return value
|
|
||||||
|
|
||||||
def filter(self, record: logging.LogRecord) -> bool:
|
|
||||||
if not self._enabled:
|
|
||||||
return True
|
|
||||||
# Scrub the formatted message.
|
|
||||||
try:
|
|
||||||
msg = record.getMessage()
|
|
||||||
scrubbed_msg = self._scrub(msg)
|
|
||||||
if scrubbed_msg != msg:
|
|
||||||
record.msg = scrubbed_msg
|
|
||||||
record.args = ()
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
pass # never let the scrubber crash a log call
|
|
||||||
# Scrub string extras in place. We mutate the record's
|
|
||||||
# __dict__ directly so the formatter sees the scrubbed value.
|
|
||||||
for k, v in list(record.__dict__.items()):
|
|
||||||
if k in _RESERVED_LOGRECORD_ATTRS or k.startswith("_"):
|
|
||||||
continue
|
|
||||||
# First, key-based redaction (covers `extra={"dob": "..."}`).
|
|
||||||
redacted = self._redact_extra_value(k, v)
|
|
||||||
if redacted is not v:
|
|
||||||
record.__dict__[k] = redacted
|
|
||||||
continue
|
|
||||||
# Second, value-pattern redaction (covers `extra={"note":
|
|
||||||
# "patient_name=John Doe"}`).
|
|
||||||
if isinstance(v, str):
|
|
||||||
scrubbed = self._scrub(v)
|
|
||||||
if scrubbed != v:
|
|
||||||
record.__dict__[k] = scrubbed
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
# Module-level singleton so tests / callers can disable it cleanly.
|
|
||||||
_scrubber = PiiScrubber()
|
|
||||||
|
|
||||||
|
|
||||||
def get_scrubber() -> PiiScrubber:
|
|
||||||
"""Return the module-level PII scrubber singleton."""
|
|
||||||
return _scrubber
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# setup_logging entry point
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_level(level: str | int | None) -> int:
|
|
||||||
"""Resolve a level string/int, falling back to INFO."""
|
|
||||||
if level is None:
|
|
||||||
return logging.INFO
|
|
||||||
if isinstance(level, int):
|
|
||||||
return level
|
|
||||||
name = str(level).strip().upper()
|
|
||||||
return logging.getLevelNamesMapping().get(name, logging.INFO)
|
|
||||||
|
|
||||||
|
|
||||||
def setup_logging(
|
|
||||||
*,
|
|
||||||
level: str | int | None = None,
|
|
||||||
log_file: str | None = None,
|
|
||||||
json_format: bool = True,
|
|
||||||
scrub_pii: bool = True,
|
|
||||||
propagate_from: str | None = None,
|
|
||||||
) -> logging.Logger:
|
|
||||||
"""Configure the root logger + attach handlers.
|
|
||||||
|
|
||||||
Idempotent: re-calling clears existing handlers on the root
|
|
||||||
logger before attaching fresh ones. Safe to call from
|
|
||||||
``click.command`` invocations and the FastAPI lifespan.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
level: ``"DEBUG"`` / ``"INFO"`` / etc. or an int. ``None``
|
|
||||||
means honor ``CYCLONE_LOG_LEVEL`` env var, then INFO.
|
|
||||||
log_file: Path to a rotating log file. ``None`` means
|
|
||||||
honor ``CYCLONE_LOG_FILE`` env var, then stderr.
|
|
||||||
json_format: Emit JSON lines (default). ``False`` uses
|
|
||||||
:class:`CycloneDevFormatter`.
|
|
||||||
scrub_pii: Apply the PII scrubber (default). Honored via
|
|
||||||
``CYCLONE_LOG_NO_PII_SCRUB=1`` to disable.
|
|
||||||
propagate_from: Optional logger name to attach the scrubber
|
|
||||||
to (defaults to root).
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
The configured root logger.
|
|
||||||
"""
|
|
||||||
# Resolve env-var defaults.
|
|
||||||
if level is None:
|
|
||||||
level = os.environ.get("CYCLONE_LOG_LEVEL", "INFO")
|
|
||||||
if log_file is None:
|
|
||||||
log_file = os.environ.get("CYCLONE_LOG_FILE") or None
|
|
||||||
if not json_format and os.environ.get("CYCLONE_LOG_JSON", "").lower() in (
|
|
||||||
"false", "0", "no",
|
|
||||||
):
|
|
||||||
json_format = True
|
|
||||||
if os.environ.get("CYCLONE_LOG_NO_PII_SCRUB", "").lower() in ("1", "true", "yes"):
|
|
||||||
scrub_pii = False
|
|
||||||
|
|
||||||
root = logging.getLogger()
|
|
||||||
root.setLevel(_resolve_level(level))
|
|
||||||
|
|
||||||
# Clear existing handlers (idempotent re-setup).
|
|
||||||
for h in list(root.handlers):
|
|
||||||
root.removeHandler(h)
|
|
||||||
# Also clear our scrubber so we don't add duplicates.
|
|
||||||
target = logging.getLogger(propagate_from) if propagate_from else root
|
|
||||||
for flt in list(target.filters):
|
|
||||||
if isinstance(flt, PiiScrubber):
|
|
||||||
target.removeFilter(flt)
|
|
||||||
|
|
||||||
# Build the formatter.
|
|
||||||
fmt: logging.Formatter
|
|
||||||
if json_format:
|
|
||||||
fmt = JsonFormatter()
|
|
||||||
else:
|
|
||||||
fmt = CycloneDevFormatter()
|
|
||||||
|
|
||||||
# Build the handler.
|
|
||||||
if log_file:
|
|
||||||
handler: logging.Handler = RotatingFileHandler(
|
|
||||||
log_file,
|
|
||||||
maxBytes=10 * 1024 * 1024,
|
|
||||||
backupCount=5,
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
handler = logging.StreamHandler(stream=sys.stderr)
|
|
||||||
handler.setFormatter(fmt)
|
|
||||||
root.addHandler(handler)
|
|
||||||
|
|
||||||
# Attach the scrubber.
|
|
||||||
if scrub_pii:
|
|
||||||
_scrubber.enable()
|
|
||||||
else:
|
|
||||||
_scrubber.disable()
|
|
||||||
target.addFilter(_scrubber)
|
|
||||||
|
|
||||||
# Quiet down noisy third-party libs.
|
|
||||||
for noisy in ("urllib3", "paramiko", "sqlalchemy.engine"):
|
|
||||||
logging.getLogger(noisy).setLevel(max(root.level, logging.WARNING))
|
|
||||||
|
|
||||||
return root
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
-- version: 11
|
|
||||||
-- SP16: Inbound MFT polling scheduler
|
|
||||||
--
|
|
||||||
-- Tracks every file the background scheduler has downloaded from
|
|
||||||
-- the Gainwell MFT inbound path so a re-tick (or a restart) does not
|
|
||||||
-- re-process the same file. Idempotency is required for production:
|
|
||||||
-- the scheduler polls every N seconds and a slow MFT server may hand
|
|
||||||
-- us the same file across two polls.
|
|
||||||
--
|
|
||||||
-- We key on (sftp_block_name, name) — the sftp_block_name disambiguates
|
|
||||||
-- multi-provider installations (SP9+SP-multi-NPI), name is the inbound
|
|
||||||
-- filename as it appears on the MFT server.
|
|
||||||
--
|
|
||||||
-- Status values:
|
|
||||||
-- * ok — parsed cleanly, results persisted to the store
|
|
||||||
-- * error — parser raised; error_message captured for the operator
|
|
||||||
-- * skipped — file_type not in the scheduler's allowed set
|
|
||||||
-- * pending — file was downloaded but a downstream step failed
|
|
||||||
-- (e.g. DB write); the scheduler retries on the next tick
|
|
||||||
--
|
|
||||||
-- claim_count is the number of claims/remittances/acks the parser
|
|
||||||
-- surfaced. Surfaced on /api/admin/scheduler/status so the operator can
|
|
||||||
-- see throughput without parsing logs.
|
|
||||||
--
|
|
||||||
-- Compliance: not part of the HIPAA audit chain (SP11). This is
|
|
||||||
-- operational metadata; an SFTP outage shouldn't pollute the audit log.
|
|
||||||
|
|
||||||
CREATE TABLE processed_inbound_files (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
sftp_block_name TEXT NOT NULL,
|
|
||||||
name TEXT NOT NULL,
|
|
||||||
size INTEGER NOT NULL,
|
|
||||||
modified_at TEXT,
|
|
||||||
file_type TEXT,
|
|
||||||
processed_at TEXT NOT NULL,
|
|
||||||
parser_used TEXT,
|
|
||||||
claim_count INTEGER NOT NULL DEFAULT 0,
|
|
||||||
status TEXT NOT NULL,
|
|
||||||
error_message TEXT
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE UNIQUE INDEX ux_processed_inbound_files_block_name
|
|
||||||
ON processed_inbound_files(sftp_block_name, name);
|
|
||||||
CREATE INDEX ix_processed_inbound_files_processed_at
|
|
||||||
ON processed_inbound_files(processed_at DESC);
|
|
||||||
CREATE INDEX ix_processed_inbound_files_status
|
|
||||||
ON processed_inbound_files(status);
|
|
||||||
@@ -1,29 +0,0 @@
|
|||||||
-- version: 12
|
|
||||||
-- SP17: encrypted DB backup metadata
|
|
||||||
--
|
|
||||||
-- Tracks every backup the BackupService has taken. The actual
|
|
||||||
-- encrypted blob lives in a directory outside the DB (default
|
|
||||||
-- ~/.local/share/cyclone/backups/); this table is just the index
|
|
||||||
-- the operator queries via GET /api/admin/backup/list.
|
|
||||||
--
|
|
||||||
-- Status values:
|
|
||||||
-- pending - row inserted, .backup() in progress or crashed before commit
|
|
||||||
-- ok - encrypted blob + sidecar written successfully
|
|
||||||
-- error - creation failed; error_message populated
|
|
||||||
-- pruned - retention policy removed the file; row kept for audit
|
|
||||||
|
|
||||||
CREATE TABLE db_backups (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
filename TEXT NOT NULL,
|
|
||||||
backup_dir TEXT NOT NULL,
|
|
||||||
size_bytes INTEGER NOT NULL DEFAULT 0,
|
|
||||||
db_fingerprint TEXT,
|
|
||||||
table_count INTEGER NOT NULL DEFAULT 0,
|
|
||||||
created_at TEXT NOT NULL,
|
|
||||||
completed_at TEXT,
|
|
||||||
status TEXT NOT NULL,
|
|
||||||
error_message TEXT
|
|
||||||
);
|
|
||||||
CREATE UNIQUE INDEX ux_db_backups_filename ON db_backups(backup_dir, filename);
|
|
||||||
CREATE INDEX ix_db_backups_created_at ON db_backups(created_at DESC);
|
|
||||||
CREATE INDEX ix_db_backups_status ON db_backups(status);
|
|
||||||
@@ -1,159 +0,0 @@
|
|||||||
"""SP20 — NPI checksum + Tax ID format validation.
|
|
||||||
|
|
||||||
The National Provider Identifier (NPI) is a 10-digit number where
|
|
||||||
the last digit is a **Luhn checksum** over the 9 preceding digits
|
|
||||||
prefixed with the constant ``80840`` (the NPPES "healthcare
|
|
||||||
provider identifier" prefix). See CMS / HHS NPI Standard:
|
|
||||||
|
|
||||||
https://www.cms.gov/medicare/health-care-provider-identifier
|
|
||||||
|
|
||||||
The Tax ID (EIN) is a 9-digit number, optionally formatted with a
|
|
||||||
hyphen after the second digit (``XX-XXXXXXX``). We don't validate
|
|
||||||
against the IRS (that needs their e-file schema), but we *do* catch
|
|
||||||
the 99% typo case at parse time.
|
|
||||||
|
|
||||||
Everything in this module is local — no NPPES, no network. Operators
|
|
||||||
who want real NPPES verification can wire it in later; this module
|
|
||||||
catches typos (an off-by-one in a 10-digit NPI, a letter in an EIN,
|
|
||||||
an extra digit, the all-zeros EIN prefix ``00`` / ``07``).
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
|
|
||||||
# NPPES prefix per the NPI Luhn algorithm. Prepended to the 9-digit
|
|
||||||
# NPI body before running the Luhn check.
|
|
||||||
_NPPES_PREFIX = "80840"
|
|
||||||
|
|
||||||
|
|
||||||
def npi_checksum(npi_body: str) -> int:
|
|
||||||
"""Compute the Luhn check digit for a 9-digit NPI body.
|
|
||||||
|
|
||||||
``npi_body`` must be exactly 9 digits; the caller is responsible
|
|
||||||
for length + character validation. Returns the check digit (0–9).
|
|
||||||
"""
|
|
||||||
if not npi_body.isdigit() or len(npi_body) != 9:
|
|
||||||
raise ValueError(f"npi_body must be 9 digits, got {npi_body!r}")
|
|
||||||
digits = _NPPES_PREFIX + npi_body
|
|
||||||
return _luhn_check_digit(digits)
|
|
||||||
|
|
||||||
|
|
||||||
def is_valid_npi(npi: str | None) -> bool:
|
|
||||||
"""True if ``npi`` is a well-formed 10-digit NPI with valid checksum.
|
|
||||||
|
|
||||||
Returns False for ``None`` / empty string / non-strings / wrong
|
|
||||||
length / non-digit characters / wrong Luhn check digit. Doesn't
|
|
||||||
call NPPES — see module docstring for why.
|
|
||||||
|
|
||||||
>>> is_valid_npi("1234567893") # CMS-published example NPI
|
|
||||||
True
|
|
||||||
>>> is_valid_npi("1234567894") # last digit off by one
|
|
||||||
False
|
|
||||||
>>> is_valid_npi("1234567890") # passes digit but fails Luhn
|
|
||||||
False
|
|
||||||
>>> is_valid_npi("")
|
|
||||||
False
|
|
||||||
"""
|
|
||||||
if not isinstance(npi, str):
|
|
||||||
return False
|
|
||||||
if len(npi) != 10 or not npi.isdigit():
|
|
||||||
return False
|
|
||||||
return npi[-1] == str(npi_checksum(npi[:-1]))
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Tax ID (EIN)
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# EIN prefix table (subset). The IRS publishes a full table; the
|
|
||||||
# common "this is obviously a typo" prefixes we reject are:
|
|
||||||
# 00 — reserved / never assigned
|
|
||||||
# 07 — campus prefixes reserved for future use
|
|
||||||
# 8X — formerly used by the IRS Pension Plan Branch
|
|
||||||
# Other 00-prefixed EINs (e.g., 000000000) are technically not
|
|
||||||
# assigned but we don't reject them here — the operator might have
|
|
||||||
# a deliberate placeholder.
|
|
||||||
_EIN_FORBIDDEN_PREFIXES = {"00", "07"}
|
|
||||||
_EIN_RESERVED_PREFIX_8X = re.compile(r"^8\d$")
|
|
||||||
|
|
||||||
# 9 digits, optionally formatted as XX-XXXXXXX.
|
|
||||||
_EIN_FORMATTED = re.compile(r"^\d{2}-\d{7}$")
|
|
||||||
_EIN_PLAIN = re.compile(r"^\d{9}$")
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_tax_id(tax_id: str | None) -> str | None:
|
|
||||||
"""Return ``tax_id`` in 9-digit plain form, or None if it's malformed.
|
|
||||||
|
|
||||||
>>> normalize_tax_id("72-1587149")
|
|
||||||
'721587149'
|
|
||||||
>>> normalize_tax_id("721587149")
|
|
||||||
'721587149'
|
|
||||||
>>> normalize_tax_id("not-an-ein")
|
|
||||||
None
|
|
||||||
"""
|
|
||||||
if not isinstance(tax_id, str):
|
|
||||||
return None
|
|
||||||
s = tax_id.strip()
|
|
||||||
if _EIN_FORMATTED.match(s):
|
|
||||||
return s.replace("-", "")
|
|
||||||
if _EIN_PLAIN.match(s):
|
|
||||||
return s
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def is_valid_tax_id(tax_id: str | None) -> bool:
|
|
||||||
"""True if ``tax_id`` is a 9-digit EIN (formatted or plain) with
|
|
||||||
a non-reserved prefix.
|
|
||||||
|
|
||||||
>>> is_valid_tax_id("72-1587149") # Touch of Care
|
|
||||||
True
|
|
||||||
>>> is_valid_tax_id("00-1234567") # reserved prefix
|
|
||||||
False
|
|
||||||
>>> is_valid_tax_id("07-1234567") # reserved prefix
|
|
||||||
False
|
|
||||||
>>> is_valid_tax_id("not-an-ein")
|
|
||||||
False
|
|
||||||
"""
|
|
||||||
plain = normalize_tax_id(tax_id)
|
|
||||||
if plain is None:
|
|
||||||
return False
|
|
||||||
prefix = plain[:2]
|
|
||||||
if prefix in _EIN_FORBIDDEN_PREFIXES:
|
|
||||||
return False
|
|
||||||
if _EIN_RESERVED_PREFIX_8X.match(prefix):
|
|
||||||
return False
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Luhn internals
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _luhn_check_digit(digits: str) -> int:
|
|
||||||
"""Return the Luhn check digit for ``digits``.
|
|
||||||
|
|
||||||
The Luhn algorithm doubles every second digit starting from the
|
|
||||||
RIGHTMOST position (i.e., the first digit doubled is the rightmost
|
|
||||||
character of ``digits``). If the doubled value exceeds 9, subtract
|
|
||||||
9. Sum all digits; the check digit is ``(10 - sum % 10) % 10``.
|
|
||||||
|
|
||||||
``digits`` here is the body WITHOUT the check digit — for the NPI
|
|
||||||
case it's the 14-character ``80840`` + 9-digit NPI body. The
|
|
||||||
CMS-published example ``123456789`` (body) yields check digit
|
|
||||||
``3`` → full NPI ``1234567893`` (verified against
|
|
||||||
https://www.cms.gov/.../NPIcheckdigit.pdf).
|
|
||||||
"""
|
|
||||||
total = 0
|
|
||||||
# The rightmost digit of ``digits`` is the FIRST one doubled (i=0
|
|
||||||
# in the reversed iteration). Per CMS, doubling starts at the
|
|
||||||
# rightmost and alternates leftward.
|
|
||||||
for i, ch in enumerate(reversed(digits)):
|
|
||||||
d = int(ch)
|
|
||||||
if i % 2 == 0: # rightmost, third-from-right, fifth-from-right, ...
|
|
||||||
d *= 2
|
|
||||||
if d > 9:
|
|
||||||
d -= 9
|
|
||||||
total += d
|
|
||||||
return (10 - total % 10) % 10
|
|
||||||
@@ -63,7 +63,6 @@ class ClaimHeader(_Base):
|
|||||||
frequency_code: str | None = None
|
frequency_code: str | None = None
|
||||||
provider_signature: str | None = None
|
provider_signature: str | None = None
|
||||||
assignment: str | None = None
|
assignment: str | None = None
|
||||||
benefits_assignment_certification: str | None = None # CLM08 (Y/N)
|
|
||||||
release_of_info: str | None = None
|
release_of_info: str | None = None
|
||||||
prior_auth: str | None = None
|
prior_auth: str | None = None
|
||||||
|
|
||||||
@@ -88,14 +87,6 @@ class ServiceLine(_Base):
|
|||||||
place_of_service: str | None = None
|
place_of_service: str | None = None
|
||||||
service_date: date | None = None
|
service_date: date | None = None
|
||||||
provider_reference: str | None = None
|
provider_reference: str | None = None
|
||||||
# SV1-07 — Diagnosis Code Pointer. Points to one or more
|
|
||||||
# diagnosis codes in the parent claim's HI segment ("1".. "12",
|
|
||||||
# space-separated when multiple). For 837P with a non-empty HI
|
|
||||||
# segment, SV1-07 is required by HCPF / Gainwell. The parser
|
|
||||||
# captures it from the source; the serializer defaults to "1"
|
|
||||||
# when the claim has at least one diagnosis and no explicit
|
|
||||||
# pointer was captured (matches the common single-dx case).
|
|
||||||
dx_pointer: str | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class ValidationIssue(_Base):
|
class ValidationIssue(_Base):
|
||||||
|
|||||||
@@ -202,7 +202,6 @@ def _consume_claim(segments: list[list[str]], idx: int) -> tuple[ClaimOutput, in
|
|||||||
frequency_code=freq or None,
|
frequency_code=freq or None,
|
||||||
provider_signature=clm[6] if len(clm) > 6 else None,
|
provider_signature=clm[6] if len(clm) > 6 else None,
|
||||||
assignment=clm[7] if len(clm) > 7 else None,
|
assignment=clm[7] if len(clm) > 7 else None,
|
||||||
benefits_assignment_certification=clm[8] if len(clm) > 8 else None,
|
|
||||||
release_of_info=clm[9] if len(clm) > 9 else None,
|
release_of_info=clm[9] if len(clm) > 9 else None,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -286,12 +285,6 @@ def _consume_service_line(segments: list[list[str]], idx: int, line_no: int) ->
|
|||||||
except Exception:
|
except Exception:
|
||||||
units = None
|
units = None
|
||||||
place_of_service = seg[5] if len(seg) > 5 else None
|
place_of_service = seg[5] if len(seg) > 5 else None
|
||||||
# SV1-06 (Unit Basis of Measurement) is X12 "UN" for "units" — we
|
|
||||||
# already use unit_type in SV1-03; SV1-06 is rarely populated and
|
|
||||||
# is not required by HCPF.
|
|
||||||
# SV1-07 — Diagnosis Code Pointer (e.g. "1" for the first HI
|
|
||||||
# diagnosis). Required by HCPF when the claim has diagnoses.
|
|
||||||
dx_pointer = seg[7] if len(seg) > 7 and seg[7] else None
|
|
||||||
service_date: date | None = None
|
service_date: date | None = None
|
||||||
provider_ref: str | None = None
|
provider_ref: str | None = None
|
||||||
|
|
||||||
@@ -318,7 +311,6 @@ def _consume_service_line(segments: list[list[str]], idx: int, line_no: int) ->
|
|||||||
place_of_service=place_of_service,
|
place_of_service=place_of_service,
|
||||||
service_date=service_date,
|
service_date=service_date,
|
||||||
provider_reference=provider_ref,
|
provider_reference=provider_ref,
|
||||||
dx_pointer=dx_pointer,
|
|
||||||
),
|
),
|
||||||
idx,
|
idx,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -112,10 +112,7 @@ def _build_gs(sender_id: str, receiver_id: str, group_control_number: str) -> st
|
|||||||
_FUNCTIONAL_ID_HEALTH_CARE,
|
_FUNCTIONAL_ID_HEALTH_CARE,
|
||||||
sender_id,
|
sender_id,
|
||||||
receiver_id,
|
receiver_id,
|
||||||
# GS-04 must be CCYYMMDD (8 digits) per X12 — ISA uses YYMMDD
|
_today_yymmdd(),
|
||||||
# (6 digits) for the older format, but the GS segment is the
|
|
||||||
# newer ANSI X12 format and requires the full year.
|
|
||||||
_today_yyyymmdd(),
|
|
||||||
_today_hhmm(),
|
_today_hhmm(),
|
||||||
group_control_number,
|
group_control_number,
|
||||||
"X",
|
"X",
|
||||||
@@ -189,30 +186,17 @@ def _build_nm1(entity_id_qualifier: str, entity_type: str, name: str,
|
|||||||
return _ELEM.join(parts) + _SEG
|
return _ELEM.join(parts) + _SEG
|
||||||
|
|
||||||
|
|
||||||
def _build_per(
|
def _build_per(contact_name: str | None, contact_phone: str | None) -> str:
|
||||||
contact_name: str | None,
|
"""PER segment — submitter contact. Returns empty when no contact info."""
|
||||||
contact_phone: str | None,
|
if not contact_name and not contact_phone:
|
||||||
contact_email: str | None = None,
|
return ""
|
||||||
email_qual: str = "EM",
|
parts = [
|
||||||
) -> str:
|
"PER",
|
||||||
"""PER segment — submitter contact (Loop 1000A).
|
"IC", # PER01 — contact function code (Information Contact)
|
||||||
|
contact_name or "",
|
||||||
X12 005010X222A1 *requires* at least one PER segment in Loop 1000A
|
"TE", # PER03 — phone qualifier
|
||||||
(Submitter Name) and at least PER01 must be present, so this
|
contact_phone or "",
|
||||||
builder always emits a segment. PER01 = "IC" (Information Contact).
|
]
|
||||||
|
|
||||||
The remaining elements are filled from the available contact info:
|
|
||||||
name, then email (preferred — Gainwell/HCPF expect this), then phone.
|
|
||||||
"""
|
|
||||||
parts = ["PER", "IC"]
|
|
||||||
if contact_name:
|
|
||||||
parts.append(contact_name)
|
|
||||||
if contact_email:
|
|
||||||
parts.append(email_qual) # PER03 — email qualifier (default "EM")
|
|
||||||
parts.append(contact_email) # PER04 — the email itself
|
|
||||||
elif contact_phone:
|
|
||||||
parts.append("TE") # PER03 — phone qualifier
|
|
||||||
parts.append(contact_phone) # PER04 — the phone itself
|
|
||||||
return _ELEM.join(parts) + _SEG
|
return _ELEM.join(parts) + _SEG
|
||||||
|
|
||||||
|
|
||||||
@@ -252,40 +236,26 @@ def _build_hl(hl_id: str, parent_id: str, level_code: str, child_code: str) -> s
|
|||||||
return _ELEM.join(parts) + _SEG
|
return _ELEM.join(parts) + _SEG
|
||||||
|
|
||||||
|
|
||||||
def _build_sbr(
|
def _build_sbr(relationship_code: str | None, member_id: str | None,
|
||||||
individual_relationship_code: str | None,
|
payer_name: str | None) -> str:
|
||||||
claim_filing_indicator_code: str | None,
|
|
||||||
) -> str:
|
|
||||||
"""SBR segment — subscriber information.
|
"""SBR segment — subscriber information.
|
||||||
|
|
||||||
Slot layout (X12 005010X222A1):
|
SBR01 (relationship code) defaults to ``"P"`` (Patient = self) which is
|
||||||
SBR01 — Payer Responsibility Sequence Number Code. Default ``"P"``
|
the most common case for professional claims; the parser does not store
|
||||||
(Patient = primary). The parser does not capture this
|
this on the canonical Subscriber model so we cannot thread it through
|
||||||
field on ``ClaimOutput`` so we default it.
|
without adding a model field.
|
||||||
SBR02 — Individual Relationship Code. ``"18"`` = self, ``"01"`` = spouse, etc.
|
|
||||||
The parser does not capture this either; we default ``"18"``
|
|
||||||
for the common self-pay case.
|
|
||||||
SBR09 — Claim Filing Indicator Code. ``"MC"`` for Medicaid,
|
|
||||||
``"16"`` for Medicare Part B, etc. The canonical
|
|
||||||
PayerConfig837 carries ``sbr09_default``; we thread it in
|
|
||||||
from the caller.
|
|
||||||
|
|
||||||
The member_id and payer name do NOT belong in SBR — the member_id
|
|
||||||
lives in NM109 of the NM1*IL segment, and the payer name is in
|
|
||||||
NM103 of NM1*PR. (Earlier revisions of this function put them in
|
|
||||||
SBR06 / SBR09, which is wrong and rejected by HCPF.)
|
|
||||||
"""
|
"""
|
||||||
parts = [
|
parts = [
|
||||||
"SBR",
|
"SBR",
|
||||||
"P", # SBR01 — primary
|
relationship_code or "P",
|
||||||
individual_relationship_code or "18", # SBR02 — self
|
"", # SBR02 — group number
|
||||||
"", # SBR03 — group number
|
"", # SBR03 — group name
|
||||||
"", # SBR04 — group name
|
"", # SBR04 — claim filing indicator code
|
||||||
"", # SBR05 — insurance type code
|
"", # SBR05 — sequence number code
|
||||||
"", # SBR06 — coordination of benefits
|
payer_name or "", # SBR06 — claim filing indicator code (CO uses MC)
|
||||||
"", # SBR07 — yes/no condition
|
"", # SBR07
|
||||||
"", # SBR08 — employment status code
|
"", # SBR08
|
||||||
claim_filing_indicator_code or "", # SBR09 — claim filing indicator
|
member_id or "", # SBR09 — claim submitter's id
|
||||||
]
|
]
|
||||||
return _ELEM.join(parts) + _SEG
|
return _ELEM.join(parts) + _SEG
|
||||||
|
|
||||||
@@ -330,11 +300,7 @@ def _build_clm(claim) -> str:
|
|||||||
clm05, # CLM05 — composite POS:qualifier:frequency_code
|
clm05, # CLM05 — composite POS:qualifier:frequency_code
|
||||||
claim.provider_signature or "Y", # CLM06
|
claim.provider_signature or "Y", # CLM06
|
||||||
claim.assignment or "Y", # CLM07
|
claim.assignment or "Y", # CLM07
|
||||||
# CLM08 — Benefits Assignment Certification. X12 837P requires
|
"", # CLM08 — benefit assignment certification
|
||||||
# this when CLM07 = "Y" (the common case for in-network
|
|
||||||
# professional claims). Default to "Y" when the source did
|
|
||||||
# not capture one — matches what 99% of HCPF files look like.
|
|
||||||
claim.benefits_assignment_certification or "Y", # CLM08
|
|
||||||
claim.release_of_info or "Y", # CLM09
|
claim.release_of_info or "Y", # CLM09
|
||||||
]
|
]
|
||||||
return _ELEM.join(parts) + _SEG
|
return _ELEM.join(parts) + _SEG
|
||||||
@@ -359,41 +325,21 @@ def _build_lx(line_number: int) -> str:
|
|||||||
return _ELEM.join(["LX", str(line_number)]) + _SEG
|
return _ELEM.join(["LX", str(line_number)]) + _SEG
|
||||||
|
|
||||||
|
|
||||||
def _build_sv1(line, *, dx_pointer: str | None = None) -> str:
|
def _build_sv1(line) -> str:
|
||||||
"""SV1 segment — professional service line.
|
"""SV1 segment — professional service line."""
|
||||||
|
|
||||||
X12 005010X222A1 layout (837P):
|
|
||||||
SV1-01 composite procedure identifier
|
|
||||||
SV1-02 monetary amount (charge)
|
|
||||||
SV1-03 unit of basis measurement (UN, MJ, etc.) — ``line.unit_type``
|
|
||||||
SV1-04 service unit count — ``line.units``
|
|
||||||
SV1-05 place of service code — ``line.place_of_service``
|
|
||||||
SV1-06 **NOT USED** by this guide (must be empty)
|
|
||||||
SV1-07 diagnosis code pointer — ``dx_pointer`` (required when the
|
|
||||||
parent claim has an HI segment)
|
|
||||||
|
|
||||||
The parser captures the original SV1-07 pointer when present; the
|
|
||||||
serializer defaults it to ``"1"`` (pointing at the first HI
|
|
||||||
diagnosis) when the claim has diagnoses and no explicit pointer
|
|
||||||
was captured. When the claim has no HI segment we leave SV1-07
|
|
||||||
empty to match the spec.
|
|
||||||
"""
|
|
||||||
proc = line.procedure
|
proc = line.procedure
|
||||||
code = proc.code if proc else ""
|
code = proc.code if proc else ""
|
||||||
mods = proc.modifiers if proc else []
|
mods = proc.modifiers if proc else []
|
||||||
composite = "HC:" + code + "".join(f":{m}" for m in (mods or [])[:4])
|
composite = "HC:" + code + "".join(f":{m}" for m in (mods or [])[:4])
|
||||||
charge = f"{Decimal(line.charge or 0):.2f}"
|
charge = f"{Decimal(line.charge or 0):.2f}"
|
||||||
units = f"{Decimal(line.units):g}" if line.units is not None else "1"
|
units = f"{Decimal(line.units):g}" if line.units is not None else "1"
|
||||||
sv1_07 = dx_pointer or ""
|
|
||||||
parts = [
|
parts = [
|
||||||
"SV1",
|
"SV1",
|
||||||
composite, # SV1-01
|
composite,
|
||||||
charge, # SV1-02
|
charge,
|
||||||
line.unit_type or "UN", # SV1-03 — unit basis code
|
line.unit_type or "UN",
|
||||||
units, # SV1-04
|
units,
|
||||||
line.place_of_service or "", # SV1-05
|
line.place_of_service or "",
|
||||||
"", # SV1-06 — NOT USED in 837P
|
|
||||||
sv1_07, # SV1-07 — diagnosis pointer
|
|
||||||
]
|
]
|
||||||
return _ELEM.join(parts) + _SEG
|
return _ELEM.join(parts) + _SEG
|
||||||
|
|
||||||
@@ -411,20 +357,15 @@ def _build_dtp_472(service_date: date | None) -> str:
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def _build_submitter_block(
|
def _build_submitter_block(sender_id: str, submitter_name: str | None,
|
||||||
sender_id: str,
|
|
||||||
submitter_name: str | None,
|
|
||||||
contact_name: str | None,
|
contact_name: str | None,
|
||||||
contact_phone: str | None,
|
contact_phone: str | None) -> list[str]:
|
||||||
contact_email: str | None = None,
|
|
||||||
email_qual: str = "EM",
|
|
||||||
) -> list[str]:
|
|
||||||
out = [
|
out = [
|
||||||
_build_nm1("41", "41", submitter_name or sender_id, "46", sender_id),
|
_build_nm1("41", "41", submitter_name or sender_id, "46", sender_id),
|
||||||
]
|
]
|
||||||
# PER is required by X12 (at least PER01). _build_per always emits
|
per = _build_per(contact_name, contact_phone)
|
||||||
# the segment; the submitter block always has exactly one.
|
if per:
|
||||||
out.append(_build_per(contact_name, contact_phone, contact_email, email_qual))
|
out.append(per)
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
@@ -454,14 +395,11 @@ def _build_billing_provider_block(provider) -> list[str]:
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def _build_subscriber_block(
|
def _build_subscriber_block(subscriber, payer_name: str | None) -> list[str]:
|
||||||
subscriber,
|
|
||||||
claim_filing_indicator_code: str | None,
|
|
||||||
) -> list[str]:
|
|
||||||
"""HL*2 → SBR → NM1*IL → N3 → N4 → DMG. Subscriber has no children."""
|
"""HL*2 → SBR → NM1*IL → N3 → N4 → DMG. Subscriber has no children."""
|
||||||
out = [
|
out = [
|
||||||
_build_hl("2", "1", "22", "0"), # HL*2 — subscriber, 0 children
|
_build_hl("2", "1", "22", "0"), # HL*2 — subscriber, 0 children
|
||||||
_build_sbr("18", claim_filing_indicator_code),
|
_build_sbr("18", subscriber.member_id, payer_name),
|
||||||
_build_nm1(
|
_build_nm1(
|
||||||
"IL", "IL",
|
"IL", "IL",
|
||||||
f"{subscriber.last_name} {subscriber.first_name}".strip(),
|
f"{subscriber.last_name} {subscriber.first_name}".strip(),
|
||||||
@@ -489,24 +427,12 @@ def _build_payer_block(payer) -> list[str]:
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
def _build_service_lines_block(service_lines, *, has_diagnoses: bool = False) -> list[str]:
|
def _build_service_lines_block(service_lines) -> list[str]:
|
||||||
"""Per line: LX / SV1 / DTP*472 / REF*6R.
|
"""Per line: LX / SV1 / DTP*472 / REF*6R."""
|
||||||
|
|
||||||
``has_diagnoses`` is True when the parent claim emits an HI segment;
|
|
||||||
in that case SV1-07 is required by X12 and we default each line's
|
|
||||||
pointer to ``"1"`` (the first HI diagnosis) unless the source
|
|
||||||
captured a different pointer on the line itself.
|
|
||||||
"""
|
|
||||||
out: list[str] = []
|
out: list[str] = []
|
||||||
for idx, line in enumerate(service_lines or [], start=1):
|
for idx, line in enumerate(service_lines or [], start=1):
|
||||||
out.append(_build_lx(idx))
|
out.append(_build_lx(idx))
|
||||||
# Prefer the line's captured pointer (parser pulled SV1-07
|
out.append(_build_sv1(line))
|
||||||
# when present). Fall back to "1" only when the claim has
|
|
||||||
# diagnoses and the source had no explicit pointer — the
|
|
||||||
# common single-diagnosis case.
|
|
||||||
line_pointer = getattr(line, "dx_pointer", None)
|
|
||||||
effective_pointer = line_pointer or ("1" if has_diagnoses else "")
|
|
||||||
out.append(_build_sv1(line, dx_pointer=effective_pointer))
|
|
||||||
dtp = _build_dtp_472(line.service_date)
|
dtp = _build_dtp_472(line.service_date)
|
||||||
if dtp:
|
if dtp:
|
||||||
out.append(dtp)
|
out.append(dtp)
|
||||||
@@ -524,10 +450,7 @@ def serialize_837(
|
|||||||
submitter_name: str | None = None,
|
submitter_name: str | None = None,
|
||||||
submitter_contact_name: str | None = None,
|
submitter_contact_name: str | None = None,
|
||||||
submitter_contact_phone: str | None = None,
|
submitter_contact_phone: str | None = None,
|
||||||
submitter_contact_email: str | None = None,
|
|
||||||
submitter_contact_email_qual: str = "EM",
|
|
||||||
receiver_name: str | None = None,
|
receiver_name: str | None = None,
|
||||||
claim_filing_indicator_code: str | None = None,
|
|
||||||
interchange_control_number: str = "000000001",
|
interchange_control_number: str = "000000001",
|
||||||
group_control_number: str = "1",
|
group_control_number: str = "1",
|
||||||
) -> str:
|
) -> str:
|
||||||
@@ -539,16 +462,6 @@ def serialize_837(
|
|||||||
(``"CYCLONE"`` / ``"RECEIVER"``) but real deployments should pass
|
(``"CYCLONE"`` / ``"RECEIVER"``) but real deployments should pass
|
||||||
the configured values.
|
the configured values.
|
||||||
|
|
||||||
The submitter block (Loop 1000A) always emits a PER segment per the
|
|
||||||
X12 spec — the canonical clearhouse config provides the contact
|
|
||||||
name and email so callers should pass them through.
|
|
||||||
|
|
||||||
The claim filing indicator (SBR09) is read from the per-payer
|
|
||||||
config (``PayerConfig837.sbr09_default``); callers should pass it
|
|
||||||
in. If not passed, SBR09 is left empty (which causes the
|
|
||||||
:func:`cyclone.parsers.validator._r202_sbr09_allowed` rule to skip
|
|
||||||
its check — degraded but not a hard error).
|
|
||||||
|
|
||||||
Editable fields (CLM, REF*G1, HI, service-line SV1, DTP*472) are
|
Editable fields (CLM, REF*G1, HI, service-line SV1, DTP*472) are
|
||||||
emitted from the canonical ``ClaimOutput`` fields, so post-parse
|
emitted from the canonical ``ClaimOutput`` fields, so post-parse
|
||||||
edits propagate to the output.
|
edits propagate to the output.
|
||||||
@@ -568,13 +481,11 @@ def serialize_837(
|
|||||||
),
|
),
|
||||||
]
|
]
|
||||||
segments.extend(_build_submitter_block(
|
segments.extend(_build_submitter_block(
|
||||||
sender_id, submitter_name,
|
sender_id, submitter_name, submitter_contact_name, submitter_contact_phone,
|
||||||
submitter_contact_name, submitter_contact_phone, submitter_contact_email,
|
|
||||||
submitter_contact_email_qual,
|
|
||||||
))
|
))
|
||||||
segments.extend(_build_receiver_block(receiver_id, receiver_name))
|
segments.extend(_build_receiver_block(receiver_id, receiver_name))
|
||||||
segments.extend(_build_billing_provider_block(claim.billing_provider))
|
segments.extend(_build_billing_provider_block(claim.billing_provider))
|
||||||
segments.extend(_build_subscriber_block(claim.subscriber, claim_filing_indicator_code))
|
segments.extend(_build_subscriber_block(claim.subscriber, claim.payer.name))
|
||||||
segments.extend(_build_payer_block(claim.payer))
|
segments.extend(_build_payer_block(claim.payer))
|
||||||
|
|
||||||
# Claim-level editable segments.
|
# Claim-level editable segments.
|
||||||
@@ -586,10 +497,7 @@ def serialize_837(
|
|||||||
segments.append(_build_hi(claim.diagnoses))
|
segments.append(_build_hi(claim.diagnoses))
|
||||||
|
|
||||||
# Service lines (LX / SV1 / DTP*472 / REF*6R).
|
# Service lines (LX / SV1 / DTP*472 / REF*6R).
|
||||||
segments.extend(_build_service_lines_block(
|
segments.extend(_build_service_lines_block(claim.service_lines))
|
||||||
claim.service_lines,
|
|
||||||
has_diagnoses=bool(claim.diagnoses),
|
|
||||||
))
|
|
||||||
|
|
||||||
# SE segment count includes ST (line 3, 1-based) through SE itself
|
# SE segment count includes ST (line 3, 1-based) through SE itself
|
||||||
# — i.e. the entire ST..SE block inclusive.
|
# — i.e. the entire ST..SE block inclusive.
|
||||||
@@ -606,23 +514,15 @@ def serialize_837_for_resubmit(
|
|||||||
claim: ClaimOutput,
|
claim: ClaimOutput,
|
||||||
*,
|
*,
|
||||||
interchange_index: int,
|
interchange_index: int,
|
||||||
**kwargs,
|
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Like :func:`serialize_837` but assigns deterministic-but-unique
|
"""Like :func:`serialize_837` but assigns deterministic-but-unique
|
||||||
interchange + group control numbers for a bundle position.
|
interchange + group control numbers for a bundle position.
|
||||||
|
|
||||||
Interchange number = ``f"{interchange_index:09d}"``.
|
Interchange number = ``f"{interchange_index:09d}"``.
|
||||||
Group number = ``str(interchange_index)``.
|
Group number = ``str(interchange_index)``.
|
||||||
|
|
||||||
All other keyword arguments (sender_id, receiver_id, submitter_*
|
|
||||||
and receiver_* contact info, claim_filing_indicator_code) are
|
|
||||||
forwarded to :func:`serialize_837` unchanged so callers — like the
|
|
||||||
export and SFTP-submit endpoints — can pass through clearhouse +
|
|
||||||
payer config without copying the signature.
|
|
||||||
"""
|
"""
|
||||||
return serialize_837(
|
return serialize_837(
|
||||||
claim,
|
claim,
|
||||||
interchange_control_number=f"{interchange_index:09d}",
|
interchange_control_number=f"{interchange_index:09d}",
|
||||||
group_control_number=str(interchange_index),
|
group_control_number=str(interchange_index),
|
||||||
**kwargs,
|
|
||||||
)
|
)
|
||||||
@@ -35,36 +35,6 @@ def _r020_npi_format(claim: ClaimOutput, _: PayerConfig) -> Iterable[ValidationI
|
|||||||
yield ValidationIssue(rule="R020_npi_format", severity="error", message=f"Billing provider NPI must be 10 digits, got {claim.billing_provider.npi!r}")
|
yield ValidationIssue(rule="R020_npi_format", severity="error", message=f"Billing provider NPI must be 10 digits, got {claim.billing_provider.npi!r}")
|
||||||
|
|
||||||
|
|
||||||
def _r021_npi_checksum(claim: ClaimOutput, _: PayerConfig) -> Iterable[ValidationIssue]:
|
|
||||||
"""SP20: validate the billing-provider NPI's Luhn check digit.
|
|
||||||
|
|
||||||
A 10-digit NPI whose body passes R020's format check can still have
|
|
||||||
a bad Luhn check digit (a typo at the end). Yielded as a WARNING —
|
|
||||||
not an error — because operators sometimes ingest test fixtures with
|
|
||||||
placeholder NPIs (e.g. all-same-digit) and we don't want to block
|
|
||||||
that path. Local-only check, no NPPES round-trip.
|
|
||||||
"""
|
|
||||||
npi = claim.billing_provider.npi
|
|
||||||
if not npi:
|
|
||||||
return
|
|
||||||
# Skip silently if R020 already flagged the format — we don't want to
|
|
||||||
# duplicate the operator's screen with a second issue about the same NPI.
|
|
||||||
if not NPI_RE.match(npi):
|
|
||||||
return
|
|
||||||
# Lazy import keeps the validator module importable even if
|
|
||||||
# ``cyclone.npi`` is unavailable (e.g. in some legacy test setups).
|
|
||||||
try:
|
|
||||||
from cyclone.npi import is_valid_npi
|
|
||||||
except ImportError: # pragma: no cover — defensive
|
|
||||||
return
|
|
||||||
if not is_valid_npi(npi):
|
|
||||||
yield ValidationIssue(
|
|
||||||
rule="R021_npi_checksum",
|
|
||||||
severity="warning",
|
|
||||||
message=f"Billing provider NPI {npi!r} fails Luhn checksum (likely typo)",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _r030_frequency_allowed(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
def _r030_frequency_allowed(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
if not claim.claim.frequency_code:
|
if not claim.claim.frequency_code:
|
||||||
return
|
return
|
||||||
@@ -422,7 +392,6 @@ _RULES: list[Rule] = [
|
|||||||
_r010_clm01_present,
|
_r010_clm01_present,
|
||||||
_r011_total_charge_positive,
|
_r011_total_charge_positive,
|
||||||
_r020_npi_format,
|
_r020_npi_format,
|
||||||
_r021_npi_checksum,
|
|
||||||
_r030_frequency_allowed,
|
_r030_frequency_allowed,
|
||||||
_r031_ref_g1_optional,
|
_r031_ref_g1_optional,
|
||||||
_r034_ref_g1_required,
|
_r034_ref_g1_required,
|
||||||
|
|||||||
@@ -100,19 +100,6 @@ class EventBus:
|
|||||||
yield await queue.get()
|
yield await queue.get()
|
||||||
queue.task_done()
|
queue.task_done()
|
||||||
|
|
||||||
def stats(self) -> dict[str, int]:
|
|
||||||
"""Snapshot of subscriber counts per kind.
|
|
||||||
|
|
||||||
Used by ``/api/health`` (SP19) and the admin diagnostics page.
|
|
||||||
Returns ``{kind: count}`` for every kind with at least one
|
|
||||||
subscriber; kinds with zero subscribers are omitted.
|
|
||||||
"""
|
|
||||||
return {
|
|
||||||
kind: len(subs)
|
|
||||||
for kind, subs in self._subscribers.items()
|
|
||||||
if subs
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def get_event_bus() -> EventBus:
|
def get_event_bus() -> EventBus:
|
||||||
"""Return the process-wide EventBus attached to the FastAPI app state.
|
"""Return the process-wide EventBus attached to the FastAPI app state.
|
||||||
|
|||||||
@@ -1,720 +0,0 @@
|
|||||||
"""Background inbound MFT polling scheduler (SP16).
|
|
||||||
|
|
||||||
Turns Cyclone from a manual upload tool into a live clearinghouse:
|
|
||||||
a long-running asyncio task that periodically polls the Gainwell MFT
|
|
||||||
inbound path, downloads each new file, and runs it through the
|
|
||||||
appropriate parser. The operator no longer has to watch for inbound
|
|
||||||
files and POST them to ``/api/parse-999`` etc. by hand.
|
|
||||||
|
|
||||||
Design constraints
|
|
||||||
------------------
|
|
||||||
|
|
||||||
* **Idempotent.** A re-tick (or a process restart) must not re-parse
|
|
||||||
the same inbound file. We persist a ``processed_inbound_files`` row
|
|
||||||
per file and skip ones we've already seen.
|
|
||||||
|
|
||||||
* **Crash-safe.** If the parser raises or the DB write fails, the
|
|
||||||
scheduler logs the error, records an ``error`` row, and moves on.
|
|
||||||
The next tick continues from the next file.
|
|
||||||
|
|
||||||
* **Bounded blast radius.** A bad file must not stop the scheduler.
|
|
||||||
Each file is wrapped in try/except so a 999 parser crash doesn't
|
|
||||||
prevent us from processing the next inbound 835.
|
|
||||||
|
|
||||||
* **Operator-controlled.** The scheduler is OFF by default; the
|
|
||||||
operator must explicitly start it (``POST /api/admin/scheduler/start``
|
|
||||||
or ``CYCLONE_SCHEDULER_AUTOSTART=true``). When it's running, status
|
|
||||||
is exposed via ``GET /api/admin/scheduler/status``.
|
|
||||||
|
|
||||||
* **No threading.** We use ``asyncio.create_task`` + ``asyncio.sleep``
|
|
||||||
rather than APScheduler or threading because the rest of the
|
|
||||||
codebase is asyncio-native (FastAPI). The whole polling loop runs
|
|
||||||
in the FastAPI event loop on the main thread.
|
|
||||||
|
|
||||||
Compliance: SP16 is operational metadata only. Inbound file
|
|
||||||
processing is NOT part of the HIPAA audit chain (SP11) — an SFTP
|
|
||||||
outage shouldn't pollute the audit log with parser errors.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import traceback
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Callable, Optional
|
|
||||||
|
|
||||||
from sqlalchemy.exc import IntegrityError
|
|
||||||
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.store import store as cycl_store
|
|
||||||
from cyclone.audit_log import AuditEvent, append_event
|
|
||||||
from cyclone.clearhouse import InboundFile, SftpClient
|
|
||||||
from cyclone.db import ProcessedInboundFile
|
|
||||||
from cyclone.edi.filenames import parse_inbound_filename
|
|
||||||
from cyclone.inbox_state import apply_999_rejections
|
|
||||||
from cyclone.inbox_state_277ca import apply_277ca_rejections
|
|
||||||
from cyclone.providers import SftpBlock
|
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
# Status values for ProcessedInboundFile.status.
|
|
||||||
STATUS_OK = "ok"
|
|
||||||
STATUS_ERROR = "error"
|
|
||||||
STATUS_SKIPPED = "skipped"
|
|
||||||
STATUS_PENDING = "pending"
|
|
||||||
|
|
||||||
|
|
||||||
# File types we know how to route. The HCPF set is broader (270/271/
|
|
||||||
# 276/277/278/820/834/ENCR) but Cyclone's parser only covers the
|
|
||||||
# four below. Files with unknown types are recorded as ``skipped``
|
|
||||||
# so the operator can see them in the audit table.
|
|
||||||
ROUTED_FILE_TYPES = frozenset({"999", "835", "277", "277CA", "TA1"})
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class TickResult:
|
|
||||||
"""Outcome of a single scheduler tick (one poll cycle)."""
|
|
||||||
|
|
||||||
started_at: datetime
|
|
||||||
finished_at: Optional[datetime] = None
|
|
||||||
files_seen: int = 0
|
|
||||||
files_processed: int = 0
|
|
||||||
files_skipped: int = 0
|
|
||||||
files_errored: int = 0
|
|
||||||
errors: list[str] = field(default_factory=list)
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"started_at": self.started_at.isoformat(),
|
|
||||||
"finished_at": (
|
|
||||||
self.finished_at.isoformat() if self.finished_at else None
|
|
||||||
),
|
|
||||||
"files_seen": self.files_seen,
|
|
||||||
"files_processed": self.files_processed,
|
|
||||||
"files_skipped": self.files_skipped,
|
|
||||||
"files_errored": self.files_errored,
|
|
||||||
"errors": list(self.errors),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class SchedulerStatus:
|
|
||||||
"""Snapshot of the scheduler's runtime state."""
|
|
||||||
|
|
||||||
running: bool
|
|
||||||
poll_interval_seconds: int
|
|
||||||
sftp_block_name: str
|
|
||||||
last_poll_at: Optional[datetime]
|
|
||||||
poll_count: int
|
|
||||||
total_processed: int
|
|
||||||
total_skipped: int
|
|
||||||
total_errored: int
|
|
||||||
last_tick: Optional[TickResult] = None
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"running": self.running,
|
|
||||||
"poll_interval_seconds": self.poll_interval_seconds,
|
|
||||||
"sftp_block_name": self.sftp_block_name,
|
|
||||||
"last_poll_at": (
|
|
||||||
self.last_poll_at.isoformat() if self.last_poll_at else None
|
|
||||||
),
|
|
||||||
"poll_count": self.poll_count,
|
|
||||||
"total_processed": self.total_processed,
|
|
||||||
"total_skipped": self.total_skipped,
|
|
||||||
"total_errored": self.total_errored,
|
|
||||||
"last_tick": self.last_tick.as_dict() if self.last_tick else None,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Per-file-type handlers. Each returns (parser_name, claim_count) and
|
|
||||||
# persists its own DB rows. The scheduler records the outcome.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_999(text: str, source_file: str) -> tuple[str, int]:
|
|
||||||
"""Parse a 999, apply rejections, persist ack row. Returns (parser, count)."""
|
|
||||||
from cyclone.parsers.parse_999 import parse_999_text
|
|
||||||
from cyclone.parsers.exceptions import CycloneParseError
|
|
||||||
|
|
||||||
try:
|
|
||||||
result = parse_999_text(text, input_file=source_file)
|
|
||||||
except CycloneParseError as exc:
|
|
||||||
raise ValueError(f"999 parse error: {exc}") from exc
|
|
||||||
|
|
||||||
received, accepted, rejected, ack_code = _ack_count_summary(result)
|
|
||||||
icn = result.envelope.control_number
|
|
||||||
synthetic_id = _ack_synthetic_source_batch_id(icn)
|
|
||||||
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
def _lookup(pcn: str):
|
|
||||||
return (
|
|
||||||
session.query(db.Claim)
|
|
||||||
.filter_by(patient_control_number=pcn)
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
rejection_result = apply_999_rejections(
|
|
||||||
session, result, claim_lookup=_lookup,
|
|
||||||
)
|
|
||||||
if rejection_result.matched:
|
|
||||||
for cid in rejection_result.matched:
|
|
||||||
append_event(session, AuditEvent(
|
|
||||||
event_type="claim.rejected",
|
|
||||||
entity_type="claim",
|
|
||||||
entity_id=cid,
|
|
||||||
payload={"source_batch_id": synthetic_id},
|
|
||||||
actor="999-parser-scheduler",
|
|
||||||
))
|
|
||||||
row = cycl_store.add_ack(
|
|
||||||
source_batch_id=synthetic_id,
|
|
||||||
accepted_count=accepted,
|
|
||||||
rejected_count=rejected,
|
|
||||||
received_count=received,
|
|
||||||
ack_code=ack_code,
|
|
||||||
raw_json=json.loads(result.model_dump_json()),
|
|
||||||
)
|
|
||||||
session.commit()
|
|
||||||
return "parse_999", received
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_835(text: str, source_file: str) -> tuple[str, int]:
|
|
||||||
"""Parse an 835, run validation, persist batch + remittances."""
|
|
||||||
import uuid
|
|
||||||
from cyclone.parsers.parse_835 import parse as parse_835
|
|
||||||
from cyclone.parsers.exceptions import CycloneParseError
|
|
||||||
from cyclone.parsers.validator_835 import validate as validate_835
|
|
||||||
from cyclone.payers import PAYER_FACTORIES_835
|
|
||||||
from cyclone.store import BatchRecord
|
|
||||||
|
|
||||||
config = PAYER_FACTORIES_835["co_medicaid_835"]()
|
|
||||||
try:
|
|
||||||
result = parse_835(text, config, input_file=source_file)
|
|
||||||
except CycloneParseError as exc:
|
|
||||||
raise ValueError(f"835 parse error: {exc}") from exc
|
|
||||||
|
|
||||||
# Validation report (mirrors the API endpoint).
|
|
||||||
report = validate_835(result, config)
|
|
||||||
n = len(result.claims)
|
|
||||||
if report.passed:
|
|
||||||
passed, failed, failed_claim_ids = n, 0, []
|
|
||||||
else:
|
|
||||||
passed, failed, failed_claim_ids = 0, n, [
|
|
||||||
c.payer_claim_control_number for c in result.claims
|
|
||||||
]
|
|
||||||
result = result.model_copy(update={
|
|
||||||
"validation": report,
|
|
||||||
"summary": result.summary.model_copy(update={
|
|
||||||
"passed": passed,
|
|
||||||
"failed": failed,
|
|
||||||
"failed_claim_ids": failed_claim_ids,
|
|
||||||
}),
|
|
||||||
})
|
|
||||||
|
|
||||||
rec = BatchRecord(
|
|
||||||
id=uuid.uuid4().hex,
|
|
||||||
kind="835",
|
|
||||||
input_filename=source_file,
|
|
||||||
parsed_at=datetime.now(timezone.utc),
|
|
||||||
result=result,
|
|
||||||
)
|
|
||||||
cycl_store.add(rec)
|
|
||||||
return "parse_835", len(result.claims)
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_277ca(text: str, source_file: str) -> tuple[str, int]:
|
|
||||||
"""Parse a 277CA, persist ack + stamp payer-rejected claims."""
|
|
||||||
from cyclone.parsers.parse_277ca import parse_277ca_text
|
|
||||||
from cyclone.parsers.exceptions import CycloneParseError
|
|
||||||
|
|
||||||
try:
|
|
||||||
result = parse_277ca_text(text, input_file=source_file)
|
|
||||||
except CycloneParseError as exc:
|
|
||||||
raise ValueError(f"277CA parse error: {exc}") from exc
|
|
||||||
|
|
||||||
icn = result.envelope.control_number
|
|
||||||
synthetic_id = _277ca_synthetic_source_batch_id(icn)
|
|
||||||
accepted = sum(
|
|
||||||
1 for s in result.claim_statuses if s.classification == "accepted"
|
|
||||||
)
|
|
||||||
paid = sum(
|
|
||||||
1 for s in result.claim_statuses if s.classification == "paid"
|
|
||||||
)
|
|
||||||
rejected = sum(
|
|
||||||
1 for s in result.claim_statuses if s.classification == "rejected"
|
|
||||||
)
|
|
||||||
pended = sum(
|
|
||||||
1 for s in result.claim_statuses if s.classification == "pended"
|
|
||||||
)
|
|
||||||
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
row = cycl_store.add_277ca_ack(
|
|
||||||
source_batch_id=synthetic_id,
|
|
||||||
control_number=icn,
|
|
||||||
accepted_count=accepted,
|
|
||||||
rejected_count=rejected,
|
|
||||||
paid_count=paid,
|
|
||||||
pended_count=pended,
|
|
||||||
raw_json=json.loads(result.model_dump_json()),
|
|
||||||
)
|
|
||||||
def _lookup(pcn: str):
|
|
||||||
return (
|
|
||||||
session.query(db.Claim)
|
|
||||||
.filter(db.Claim.patient_control_number == pcn)
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
apply_result = apply_277ca_rejections(
|
|
||||||
session, result, claim_lookup=_lookup, two77ca_id=row.id,
|
|
||||||
)
|
|
||||||
if apply_result.matched:
|
|
||||||
for cid in apply_result.matched:
|
|
||||||
append_event(session, AuditEvent(
|
|
||||||
event_type="claim.payer_rejected",
|
|
||||||
entity_type="claim",
|
|
||||||
entity_id=cid,
|
|
||||||
payload={"source_batch_id": synthetic_id, "277ca_id": row.id},
|
|
||||||
actor="277ca-parser-scheduler",
|
|
||||||
))
|
|
||||||
session.commit()
|
|
||||||
return "parse_277ca", len(result.claim_statuses)
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_ta1(text: str, source_file: str) -> tuple[str, int]:
|
|
||||||
"""Parse a TA1, persist the interchange ack row."""
|
|
||||||
from cyclone.parsers.parse_ta1 import parse_ta1_text
|
|
||||||
from cyclone.parsers.exceptions import CycloneParseError
|
|
||||||
|
|
||||||
try:
|
|
||||||
result = parse_ta1_text(text, input_file=source_file)
|
|
||||||
except CycloneParseError as exc:
|
|
||||||
raise ValueError(f"TA1 parse error: {exc}") from exc
|
|
||||||
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
cycl_store.add_ta1_ack(
|
|
||||||
source_batch_id=result.source_batch_id,
|
|
||||||
control_number=result.ta1.control_number,
|
|
||||||
interchange_date=result.ta1.interchange_date,
|
|
||||||
interchange_time=result.ta1.interchange_time,
|
|
||||||
ack_code=result.ta1.ack_code,
|
|
||||||
note_code=result.ta1.note_code,
|
|
||||||
ack_generated_date=result.ta1.ack_generated_date,
|
|
||||||
sender_id=result.envelope.sender_id,
|
|
||||||
receiver_id=result.envelope.receiver_id,
|
|
||||||
raw_json=json.loads(result.model_dump_json()),
|
|
||||||
)
|
|
||||||
session.commit()
|
|
||||||
return "parse_ta1", 1
|
|
||||||
|
|
||||||
|
|
||||||
# Map file_type → handler. Mirrors ROUTED_FILE_TYPES.
|
|
||||||
HANDLERS: dict[str, Callable[[str, str], tuple[str, int]]] = {
|
|
||||||
"999": _handle_999,
|
|
||||||
"835": _handle_835,
|
|
||||||
"277": _handle_277ca, # filename uses 277; parser is the same
|
|
||||||
"277CA": _handle_277ca,
|
|
||||||
"TA1": _handle_ta1,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Light copies of helpers the API endpoints use, so the scheduler can
|
|
||||||
# run without depending on the FastAPI module.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _ack_count_summary(result: Any) -> tuple[int, int, int, str]:
|
|
||||||
"""Return (received, accepted, rejected, ack_code) for a 999.
|
|
||||||
|
|
||||||
Mirrors the logic in ``cyclone.api._ack_count_summary`` but lives
|
|
||||||
here so the scheduler can run without importing the API module.
|
|
||||||
"""
|
|
||||||
if result.functional_group_acks:
|
|
||||||
fg = result.functional_group_acks[0]
|
|
||||||
return (
|
|
||||||
fg.received_count, fg.accepted_count,
|
|
||||||
fg.rejected_count, fg.ack_code,
|
|
||||||
)
|
|
||||||
sets = result.set_responses
|
|
||||||
received = len(sets)
|
|
||||||
accepted = sum(1 for s in sets if s.set_accept_reject.code == "A")
|
|
||||||
rejected = received - accepted
|
|
||||||
if rejected == 0:
|
|
||||||
code = "A"
|
|
||||||
elif accepted == 0:
|
|
||||||
code = "R"
|
|
||||||
else:
|
|
||||||
code = "P"
|
|
||||||
return (received, accepted, rejected, code)
|
|
||||||
|
|
||||||
|
|
||||||
def _ack_synthetic_source_batch_id(interchange_control_number: str) -> str:
|
|
||||||
"""Synthetic batches.id for a received 999 with no source batch."""
|
|
||||||
return f"999-{(interchange_control_number or '').strip() or '000000001'}"
|
|
||||||
|
|
||||||
|
|
||||||
def _277ca_synthetic_source_batch_id(interchange_control_number: str) -> str:
|
|
||||||
"""Synthetic batches.id for a received 277CA with no source batch."""
|
|
||||||
return f"277CA-{(interchange_control_number or '').strip() or '000000001'}"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Scheduler
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class Scheduler:
|
|
||||||
"""Background polling loop for inbound MFT files.
|
|
||||||
|
|
||||||
Lifecycle:
|
|
||||||
sched = Scheduler(sftp_block, poll_interval_seconds=60)
|
|
||||||
await sched.start() # begin polling
|
|
||||||
# ... later ...
|
|
||||||
await sched.stop() # finish current tick, then exit
|
|
||||||
status = sched.status() # snapshot
|
|
||||||
|
|
||||||
The scheduler is a single asyncio task. ``tick()`` does one full
|
|
||||||
poll cycle and is exposed for tests + the ``/api/admin/scheduler/tick``
|
|
||||||
endpoint so the operator can force a poll without waiting.
|
|
||||||
|
|
||||||
Threading: NOT thread-safe. All access (start/stop/tick/status)
|
|
||||||
must happen on the same event loop. The FastAPI app satisfies
|
|
||||||
this trivially because endpoints run on the loop.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
sftp_block: SftpBlock,
|
|
||||||
*,
|
|
||||||
poll_interval_seconds: int = 60,
|
|
||||||
sftp_block_name: str = "default",
|
|
||||||
sftp_client_factory: Optional[Callable[[SftpBlock], Any]] = None,
|
|
||||||
) -> None:
|
|
||||||
self._sftp_block = sftp_block
|
|
||||||
self._poll_interval = poll_interval_seconds
|
|
||||||
self._sftp_block_name = sftp_block_name
|
|
||||||
# Factory indirection lets tests substitute a fake client
|
|
||||||
# without monkey-patching the module-level SftpClient.
|
|
||||||
self._sftp_client_factory = sftp_client_factory or SftpClient
|
|
||||||
|
|
||||||
self._task: Optional[asyncio.Task[None]] = None
|
|
||||||
self._stop_event = asyncio.Event()
|
|
||||||
self._last_poll_at: Optional[datetime] = None
|
|
||||||
self._poll_count = 0
|
|
||||||
self._total_processed = 0
|
|
||||||
self._total_skipped = 0
|
|
||||||
self._total_errored = 0
|
|
||||||
self._last_tick: Optional[TickResult] = None
|
|
||||||
# Coalesce overlapping ticks (a slow MFT server shouldn't let
|
|
||||||
# ticks stack up; the next tick fires only after the previous
|
|
||||||
# one finishes).
|
|
||||||
self._tick_in_progress = False
|
|
||||||
|
|
||||||
# ---- Public API -------------------------------------------------------
|
|
||||||
|
|
||||||
async def start(self) -> None:
|
|
||||||
"""Begin polling. Idempotent."""
|
|
||||||
if self._task is not None and not self._task.done():
|
|
||||||
log.info("Scheduler already running; start() is a no-op")
|
|
||||||
return
|
|
||||||
self._stop_event.clear()
|
|
||||||
self._task = asyncio.create_task(self._run(), name="mft-scheduler")
|
|
||||||
log.info(
|
|
||||||
"Scheduler started",
|
|
||||||
extra={
|
|
||||||
"poll_interval_s": self._poll_interval,
|
|
||||||
"sftp_block": self._sftp_block_name,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
async def stop(self) -> None:
|
|
||||||
"""Stop polling. Waits for the current tick to finish."""
|
|
||||||
if self._task is None or self._task.done():
|
|
||||||
return
|
|
||||||
self._stop_event.set()
|
|
||||||
try:
|
|
||||||
await asyncio.wait_for(self._task, timeout=30)
|
|
||||||
except asyncio.TimeoutError:
|
|
||||||
log.warning("Scheduler did not stop within 30s; cancelling")
|
|
||||||
self._task.cancel()
|
|
||||||
try:
|
|
||||||
await self._task
|
|
||||||
except (asyncio.CancelledError, Exception): # noqa: BLE001
|
|
||||||
pass
|
|
||||||
self._task = None
|
|
||||||
log.info("Scheduler stopped")
|
|
||||||
|
|
||||||
def status(self) -> SchedulerStatus:
|
|
||||||
"""Return a snapshot of the scheduler's state."""
|
|
||||||
return SchedulerStatus(
|
|
||||||
running=self.is_running(),
|
|
||||||
poll_interval_seconds=self._poll_interval,
|
|
||||||
sftp_block_name=self._sftp_block_name,
|
|
||||||
last_poll_at=self._last_poll_at,
|
|
||||||
poll_count=self._poll_count,
|
|
||||||
total_processed=self._total_processed,
|
|
||||||
total_skipped=self._total_skipped,
|
|
||||||
total_errored=self._total_errored,
|
|
||||||
last_tick=self._last_tick,
|
|
||||||
)
|
|
||||||
|
|
||||||
def is_running(self) -> bool:
|
|
||||||
return self._task is not None and not self._task.done()
|
|
||||||
|
|
||||||
async def tick(self) -> TickResult:
|
|
||||||
"""Run a single poll cycle and return the outcome.
|
|
||||||
|
|
||||||
Concurrent ticks are coalesced: if a tick is already in
|
|
||||||
progress, the second caller waits for it. This protects the
|
|
||||||
SFTP server from a stampede when the operator hits
|
|
||||||
``/api/admin/scheduler/tick`` while a scheduled tick is
|
|
||||||
already running.
|
|
||||||
"""
|
|
||||||
while self._tick_in_progress:
|
|
||||||
await asyncio.sleep(0.05)
|
|
||||||
self._tick_in_progress = True
|
|
||||||
try:
|
|
||||||
result = await self._tick_impl()
|
|
||||||
self._last_tick = result
|
|
||||||
self._last_poll_at = result.finished_at or result.started_at
|
|
||||||
self._poll_count += 1
|
|
||||||
self._total_processed += result.files_processed
|
|
||||||
self._total_skipped += result.files_skipped
|
|
||||||
self._total_errored += result.files_errored
|
|
||||||
return result
|
|
||||||
finally:
|
|
||||||
self._tick_in_progress = False
|
|
||||||
|
|
||||||
# ---- Internals --------------------------------------------------------
|
|
||||||
|
|
||||||
async def _run(self) -> None:
|
|
||||||
"""Main loop. Runs until ``stop()`` is called."""
|
|
||||||
# Stagger the first tick so we don't hammer the MFT server on
|
|
||||||
# startup if multiple operators restart Cyclone in lockstep.
|
|
||||||
await asyncio.sleep(1)
|
|
||||||
while not self._stop_event.is_set():
|
|
||||||
try:
|
|
||||||
await self.tick()
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
# tick() should never raise — it catches per-file
|
|
||||||
# exceptions. This is the safety net for SFTP outages
|
|
||||||
# or DB connectivity issues.
|
|
||||||
log.exception("Scheduler tick raised", extra={"error": str(exc)})
|
|
||||||
try:
|
|
||||||
await asyncio.wait_for(
|
|
||||||
self._stop_event.wait(),
|
|
||||||
timeout=self._poll_interval,
|
|
||||||
)
|
|
||||||
except asyncio.TimeoutError:
|
|
||||||
pass # poll interval elapsed; loop again
|
|
||||||
|
|
||||||
async def _tick_impl(self) -> TickResult:
|
|
||||||
"""One poll cycle: list → filter already-processed → route each."""
|
|
||||||
started = datetime.now(timezone.utc)
|
|
||||||
result = TickResult(started_at=started)
|
|
||||||
|
|
||||||
try:
|
|
||||||
files = await asyncio.to_thread(self._list_inbound)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.exception("SFTP list_inbound failed")
|
|
||||||
result.errors.append(f"list_inbound: {exc}")
|
|
||||||
result.finished_at = datetime.now(timezone.utc)
|
|
||||||
return result
|
|
||||||
|
|
||||||
result.files_seen = len(files)
|
|
||||||
for f in files:
|
|
||||||
if self._stop_event.is_set():
|
|
||||||
break
|
|
||||||
await self._handle_one(f, result)
|
|
||||||
|
|
||||||
result.finished_at = datetime.now(timezone.utc)
|
|
||||||
return result
|
|
||||||
|
|
||||||
def _list_inbound(self) -> list[InboundFile]:
|
|
||||||
"""Return files in the inbound MFT path. Runs on a thread."""
|
|
||||||
client = self._sftp_client_factory(self._sftp_block)
|
|
||||||
return client.list_inbound()
|
|
||||||
|
|
||||||
async def _handle_one(self, f: InboundFile, result: TickResult) -> None:
|
|
||||||
"""Process one inbound file: skip-if-seen, classify, parse, record."""
|
|
||||||
if await self._already_processed(f.name):
|
|
||||||
return
|
|
||||||
|
|
||||||
try:
|
|
||||||
inbound = parse_inbound_filename(f.name)
|
|
||||||
file_type = inbound.file_type
|
|
||||||
except ValueError:
|
|
||||||
file_type = None
|
|
||||||
|
|
||||||
if file_type not in HANDLERS:
|
|
||||||
await self._record(
|
|
||||||
name=f.name, size=f.size, modified_at=f.modified_at,
|
|
||||||
file_type=file_type, parser_used=None, claim_count=0,
|
|
||||||
status=STATUS_SKIPPED,
|
|
||||||
error_message=(
|
|
||||||
f"file_type {file_type!r} not in {sorted(HANDLERS)}"
|
|
||||||
if file_type else "filename does not match HCPF inbound format"
|
|
||||||
),
|
|
||||||
)
|
|
||||||
result.files_skipped += 1
|
|
||||||
return
|
|
||||||
|
|
||||||
try:
|
|
||||||
_path, parser_used, claim_count = await asyncio.to_thread(
|
|
||||||
self._download_and_parse, f, file_type,
|
|
||||||
)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.exception("Failed to process inbound file", extra={"input_filename": f.name})
|
|
||||||
await self._record(
|
|
||||||
name=f.name, size=f.size, modified_at=f.modified_at,
|
|
||||||
file_type=file_type, parser_used=None, claim_count=0,
|
|
||||||
status=STATUS_ERROR,
|
|
||||||
error_message=(
|
|
||||||
f"{type(exc).__name__}: {exc}\n"
|
|
||||||
f"{traceback.format_exc()[-500:]}"
|
|
||||||
),
|
|
||||||
)
|
|
||||||
result.files_errored += 1
|
|
||||||
result.errors.append(f"{f.name}: {exc}")
|
|
||||||
return
|
|
||||||
|
|
||||||
await self._record(
|
|
||||||
name=f.name, size=f.size, modified_at=f.modified_at,
|
|
||||||
file_type=file_type, parser_used=parser_used, claim_count=claim_count,
|
|
||||||
status=STATUS_OK, error_message=None,
|
|
||||||
)
|
|
||||||
result.files_processed += 1
|
|
||||||
log.info(
|
|
||||||
"Processed inbound file",
|
|
||||||
extra={
|
|
||||||
"input_filename": f.name,
|
|
||||||
"parser": parser_used,
|
|
||||||
"claims": claim_count,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
async def _already_processed(self, name: str) -> bool:
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
row = (
|
|
||||||
session.query(ProcessedInboundFile)
|
|
||||||
.filter_by(sftp_block_name=self._sftp_block_name, name=name)
|
|
||||||
.filter(ProcessedInboundFile.status != STATUS_PENDING)
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
return row is not None
|
|
||||||
|
|
||||||
async def _record(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
name: str,
|
|
||||||
size: int,
|
|
||||||
modified_at: datetime,
|
|
||||||
file_type: Optional[str],
|
|
||||||
parser_used: Optional[str],
|
|
||||||
claim_count: int,
|
|
||||||
status: str,
|
|
||||||
error_message: Optional[str],
|
|
||||||
) -> None:
|
|
||||||
"""Persist a processed_inbound_files row. Idempotent."""
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
row = ProcessedInboundFile(
|
|
||||||
sftp_block_name=self._sftp_block_name,
|
|
||||||
name=name,
|
|
||||||
size=size,
|
|
||||||
modified_at=modified_at,
|
|
||||||
file_type=file_type,
|
|
||||||
processed_at=datetime.now(timezone.utc),
|
|
||||||
parser_used=parser_used,
|
|
||||||
claim_count=claim_count,
|
|
||||||
status=status,
|
|
||||||
error_message=error_message,
|
|
||||||
)
|
|
||||||
session.add(row)
|
|
||||||
try:
|
|
||||||
session.commit()
|
|
||||||
except IntegrityError:
|
|
||||||
# A concurrent scheduler (or a retry after a partial
|
|
||||||
# failure) already recorded this file. That's fine —
|
|
||||||
# the latest row wins; we just skip the dup.
|
|
||||||
session.rollback()
|
|
||||||
|
|
||||||
def _download_and_parse(
|
|
||||||
self, f: InboundFile, file_type: str,
|
|
||||||
) -> tuple[Path, str, int]:
|
|
||||||
"""Download from MFT, run the right handler. Returns (path, parser, count).
|
|
||||||
|
|
||||||
Stub mode: ``f.local_path`` already points at the staged file
|
|
||||||
(set by ``SftpClient._list_inbound_stub``). Real mode: the
|
|
||||||
remote name is ``f.name`` and we round-trip through paramiko.
|
|
||||||
"""
|
|
||||||
if self._sftp_block.stub:
|
|
||||||
# In stub mode the InboundFile already has a local_path;
|
|
||||||
# reading the staged bytes directly avoids the stub's
|
|
||||||
# remote-path semantics (which expect a full inbound path).
|
|
||||||
content = f.local_path.read_bytes()
|
|
||||||
else:
|
|
||||||
client = self._sftp_client_factory(self._sftp_block)
|
|
||||||
content = client.read_file(f.name)
|
|
||||||
text = content.decode("utf-8")
|
|
||||||
handler = HANDLERS[file_type]
|
|
||||||
parser_used, claim_count = handler(text, f.name)
|
|
||||||
return f.local_path, parser_used, claim_count
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Module-level singleton — only one scheduler per process.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
_scheduler: Optional[Scheduler] = None
|
|
||||||
|
|
||||||
|
|
||||||
def configure_scheduler(
|
|
||||||
sftp_block: SftpBlock,
|
|
||||||
*,
|
|
||||||
poll_interval_seconds: int = 60,
|
|
||||||
sftp_block_name: str = "default",
|
|
||||||
force: bool = False,
|
|
||||||
) -> Scheduler:
|
|
||||||
"""Create the module-level scheduler singleton (or return the existing one).
|
|
||||||
|
|
||||||
Called from the FastAPI lifespan handler. Tests pre-configure the
|
|
||||||
scheduler before the TestClient opens the lifespan; in that case
|
|
||||||
we leave the existing singleton alone (``force=False``). Pass
|
|
||||||
``force=True`` to replace unconditionally.
|
|
||||||
"""
|
|
||||||
global _scheduler
|
|
||||||
if _scheduler is not None and not force:
|
|
||||||
return _scheduler
|
|
||||||
poll = int(
|
|
||||||
os.environ.get("CYCLONE_SCHEDULER_POLL_SECONDS", poll_interval_seconds),
|
|
||||||
)
|
|
||||||
_scheduler = Scheduler(
|
|
||||||
sftp_block,
|
|
||||||
poll_interval_seconds=poll,
|
|
||||||
sftp_block_name=sftp_block_name,
|
|
||||||
)
|
|
||||||
return _scheduler
|
|
||||||
|
|
||||||
|
|
||||||
def get_scheduler() -> Scheduler:
|
|
||||||
"""Return the module-level scheduler.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
RuntimeError: if ``configure_scheduler`` hasn't been called.
|
|
||||||
"""
|
|
||||||
if _scheduler is None:
|
|
||||||
raise RuntimeError(
|
|
||||||
"scheduler not configured; call configure_scheduler() first",
|
|
||||||
)
|
|
||||||
return _scheduler
|
|
||||||
|
|
||||||
|
|
||||||
def reset_scheduler_for_tests() -> None:
|
|
||||||
"""Clear the module-level scheduler. Test-only."""
|
|
||||||
global _scheduler
|
|
||||||
_scheduler = None
|
|
||||||
@@ -1,485 +0,0 @@
|
|||||||
"""SP19 — Security middleware + health probe.
|
|
||||||
|
|
||||||
Three concrete middlewares (body size, rate limit, security headers)
|
|
||||||
plus a richer ``/api/health`` snapshot. Sizing is for Cyclone's
|
|
||||||
local-only posture: a misconfigured Tailscale / ngrok bind, a
|
|
||||||
misbehaving cron job, a port-scanner scraping the API. Anything more
|
|
||||||
aggressive (auth, mTLS, WAF) is out of scope.
|
|
||||||
|
|
||||||
Design choices
|
|
||||||
--------------
|
|
||||||
|
|
||||||
* **In-memory rate limiter.** Cyclone is single-process; a dict
|
|
||||||
keyed by IP is enough. If we ever go multi-worker, swap for
|
|
||||||
Redis. The rate-limit counter resets after the bucket window;
|
|
||||||
failing open on the limiter itself (an unexpected exception)
|
|
||||||
rather than 503ing every request is the right call for a local tool.
|
|
||||||
|
|
||||||
* **Body-size check by Content-Length first, then chunked-read
|
|
||||||
guard.** A chunked POST can lie about its size (or omit the
|
|
||||||
header entirely); we cap read body size on the underlying stream
|
|
||||||
so a malicious client can't keep streaming forever.
|
|
||||||
|
|
||||||
* **Security headers on every response.** CSP locks the API to
|
|
||||||
same-origin + the Vite dev origin (whitelisted explicitly so a
|
|
||||||
future operator running on a different port doesn't break).
|
|
||||||
|
|
||||||
* **Health snapshot is best-effort.** Each subsystem (DB,
|
|
||||||
scheduler, pubsub) reports independently — a DB outage doesn't
|
|
||||||
blank out the rest. ``status: "degraded"`` if any subsystem is
|
|
||||||
unhappy; ``"ok"`` only when everything is.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from collections import deque
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from typing import Any, Callable
|
|
||||||
|
|
||||||
from fastapi import Request, Response
|
|
||||||
from fastapi.responses import JSONResponse
|
|
||||||
from starlette.types import ASGIApp, Message, Receive, Scope, Send
|
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Knobs (env-var driven)
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
DEFAULT_MAX_BODY_BYTES = 50 * 1024 * 1024 # 50 MB — generous for X12 EDI
|
|
||||||
DEFAULT_RATE_LIMIT_PER_MIN = 300
|
|
||||||
DEFAULT_RATE_LIMIT_WINDOW_S = 60
|
|
||||||
|
|
||||||
# CSP: API responses are JSON, not HTML. ``default-src 'none'`` is the
|
|
||||||
# strictest setting; it forbids the API from being a vector for
|
|
||||||
# injected scripts in case an operator opens a JSON viewer with an
|
|
||||||
# HTML renderer.
|
|
||||||
_SECURITY_HEADERS: dict[str, str] = {
|
|
||||||
"X-Content-Type-Options": "nosniff",
|
|
||||||
"X-Frame-Options": "DENY",
|
|
||||||
"Referrer-Policy": "same-origin",
|
|
||||||
"Permissions-Policy": "geolocation=(), microphone=(), camera=()",
|
|
||||||
"Content-Security-Policy": "default-src 'none'; frame-ancestors 'none'",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _env_int(name: str, default: int) -> int:
|
|
||||||
raw = os.environ.get(name)
|
|
||||||
if not raw:
|
|
||||||
return default
|
|
||||||
try:
|
|
||||||
return int(raw)
|
|
||||||
except ValueError:
|
|
||||||
log.warning("SP19: %s=%r is not an int; using default %d", name, raw, default)
|
|
||||||
return default
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Body-size middleware
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class BodySizeLimitMiddleware:
|
|
||||||
"""Reject requests whose body exceeds ``max_bytes``.
|
|
||||||
|
|
||||||
Pure ASGI middleware (not BaseHTTPMiddleware — that one breaks
|
|
||||||
FastAPI's ``request.body()`` introspection). Two-stage guard:
|
|
||||||
1. If the request declares a ``Content-Length`` larger than
|
|
||||||
``max_bytes``, reject immediately with ``413``.
|
|
||||||
2. While reading the body chunks, cap accumulated bytes at
|
|
||||||
``max_bytes``. If we cross the cap, return 413 instead of
|
|
||||||
letting the handler read the rest.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, app: ASGIApp, max_bytes: int | None = None) -> None:
|
|
||||||
self.app = app
|
|
||||||
self.max_bytes = max_bytes or _env_int(
|
|
||||||
"CYCLONE_MAX_BODY_BYTES", DEFAULT_MAX_BODY_BYTES,
|
|
||||||
)
|
|
||||||
|
|
||||||
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
|
|
||||||
if scope["type"] != "http":
|
|
||||||
await self.app(scope, receive, send)
|
|
||||||
return
|
|
||||||
|
|
||||||
# Stage 1: declared length.
|
|
||||||
cl_header = None
|
|
||||||
for k, v in scope.get("headers", []):
|
|
||||||
if k == b"content-length":
|
|
||||||
cl_header = v.decode("latin-1")
|
|
||||||
break
|
|
||||||
if cl_header is not None:
|
|
||||||
try:
|
|
||||||
if int(cl_header) > self.max_bytes:
|
|
||||||
await _send_rejection(
|
|
||||||
scope, send,
|
|
||||||
code=413,
|
|
||||||
reason="body_too_large",
|
|
||||||
detail=f"Content-Length {cl_header} exceeds limit {self.max_bytes}",
|
|
||||||
)
|
|
||||||
return
|
|
||||||
except ValueError:
|
|
||||||
await _send_rejection(
|
|
||||||
scope, send,
|
|
||||||
code=400, reason="bad_content_length",
|
|
||||||
detail=f"Content-Length {cl_header!r} is not an integer",
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
# Stage 2: chunked read guard.
|
|
||||||
seen = 0
|
|
||||||
over_limit = False
|
|
||||||
|
|
||||||
async def wrapped_receive() -> Message:
|
|
||||||
nonlocal seen, over_limit
|
|
||||||
if over_limit:
|
|
||||||
# Drain any remaining bytes so the upstream ASGI
|
|
||||||
# server doesn't see a truncated stream.
|
|
||||||
msg = await receive()
|
|
||||||
if msg.get("type") == "http.request":
|
|
||||||
return {"type": "http.request", "body": b"", "more_body": False}
|
|
||||||
return msg
|
|
||||||
msg = await receive()
|
|
||||||
if msg.get("type") == "http.request":
|
|
||||||
body = msg.get("body", b"") or b""
|
|
||||||
seen += len(body)
|
|
||||||
if seen > self.max_bytes:
|
|
||||||
over_limit = True
|
|
||||||
return {"type": "http.request", "body": b"", "more_body": False}
|
|
||||||
return msg
|
|
||||||
|
|
||||||
if cl_header is None:
|
|
||||||
# Chunked / unknown length — guard with wrapped receive.
|
|
||||||
await self.app(scope, wrapped_receive, send)
|
|
||||||
else:
|
|
||||||
# Fixed-length known to be safe; pass through.
|
|
||||||
await self.app(scope, receive, send)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Rate-limit middleware
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _Bucket:
|
|
||||||
"""Sliding-window counter for one IP."""
|
|
||||||
timestamps: deque = field(default_factory=deque)
|
|
||||||
|
|
||||||
def hit(self, window_s: int, now: float) -> bool:
|
|
||||||
"""Record one hit; return True if under the limit, False if over."""
|
|
||||||
# Drop expired entries.
|
|
||||||
cutoff = now - window_s
|
|
||||||
while self.timestamps and self.timestamps[0] < cutoff:
|
|
||||||
self.timestamps.popleft()
|
|
||||||
return True # we always record; the dispatcher decides to reject
|
|
||||||
|
|
||||||
def count_in_window(self, now: float, window_s: int) -> int:
|
|
||||||
cutoff = now - window_s
|
|
||||||
while self.timestamps and self.timestamps[0] < cutoff:
|
|
||||||
self.timestamps.popleft()
|
|
||||||
return len(self.timestamps)
|
|
||||||
|
|
||||||
|
|
||||||
class RateLimitMiddleware:
|
|
||||||
"""Per-IP sliding-window rate limiter (pure ASGI).
|
|
||||||
|
|
||||||
Defaults to ``CYCLONE_RATE_LIMIT_PER_MIN`` requests/minute per IP.
|
|
||||||
Health-check probes and the ``/api/health`` endpoint are exempt
|
|
||||||
so a load balancer's frequent probes don't trip the limiter.
|
|
||||||
|
|
||||||
On unexpected errors the limiter fails OPEN — better to serve a
|
|
||||||
few extra requests than to 503 every request because of a bug.
|
|
||||||
"""
|
|
||||||
|
|
||||||
EXEMPT_PATHS = ("/api/health", "/healthz", "/readyz")
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
app: ASGIApp,
|
|
||||||
per_minute: int | None = None,
|
|
||||||
window_s: int | None = None,
|
|
||||||
) -> None:
|
|
||||||
self.app = app
|
|
||||||
self.per_minute = per_minute or _env_int(
|
|
||||||
"CYCLONE_RATE_LIMIT_PER_MIN", DEFAULT_RATE_LIMIT_PER_MIN,
|
|
||||||
)
|
|
||||||
self.window_s = window_s or DEFAULT_RATE_LIMIT_WINDOW_S
|
|
||||||
self._buckets: dict[str, _Bucket] = {}
|
|
||||||
self._lock = threading.Lock()
|
|
||||||
|
|
||||||
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
|
|
||||||
if scope["type"] != "http":
|
|
||||||
await self.app(scope, receive, send)
|
|
||||||
return
|
|
||||||
path = scope.get("path", "")
|
|
||||||
if path in self.EXEMPT_PATHS:
|
|
||||||
await self.app(scope, receive, send)
|
|
||||||
return
|
|
||||||
ip = _client_ip_from_scope(scope)
|
|
||||||
now = time.monotonic()
|
|
||||||
try:
|
|
||||||
with self._lock:
|
|
||||||
bucket = self._buckets.setdefault(ip, _Bucket())
|
|
||||||
bucket.timestamps.append(now)
|
|
||||||
count = bucket.count_in_window(now, self.window_s)
|
|
||||||
if count > self.per_minute:
|
|
||||||
await _send_rejection(
|
|
||||||
scope, send,
|
|
||||||
code=429,
|
|
||||||
reason="rate_limited",
|
|
||||||
detail=(
|
|
||||||
f"IP {ip} exceeded {self.per_minute} req/"
|
|
||||||
f"{self.window_s}s window"
|
|
||||||
),
|
|
||||||
)
|
|
||||||
return
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.warning("SP19: rate limiter failed open: %s", exc)
|
|
||||||
await self.app(scope, receive, send)
|
|
||||||
|
|
||||||
|
|
||||||
def _client_ip_from_scope(scope: Scope) -> str:
|
|
||||||
"""Best-effort client IP from the ASGI scope. Falls back to ``"unknown"``."""
|
|
||||||
for k, v in scope.get("headers", []):
|
|
||||||
if k == b"x-forwarded-for":
|
|
||||||
return v.decode("latin-1").split(",")[0].strip()
|
|
||||||
client = scope.get("client")
|
|
||||||
if client and client[0]:
|
|
||||||
return client[0]
|
|
||||||
return "unknown"
|
|
||||||
|
|
||||||
|
|
||||||
def _client_ip(request: Request) -> str:
|
|
||||||
"""Legacy helper (kept for the audit-event log path)."""
|
|
||||||
return _client_ip_from_scope(request.scope)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Security-headers middleware
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class SecurityHeadersMiddleware:
|
|
||||||
"""Stamp the static security headers on every response (pure ASGI).
|
|
||||||
|
|
||||||
CSP / X-Content-Type-Options / X-Frame-Options / Referrer-Policy /
|
|
||||||
Permissions-Policy. The headers are static for now; per-route
|
|
||||||
overrides can be added later if a route needs to relax them.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, app: ASGIApp, extra: dict[str, str] | None = None) -> None:
|
|
||||||
self.app = app
|
|
||||||
self.headers = [(k.lower().encode("latin-1"), v.encode("latin-1"))
|
|
||||||
for k, v in _SECURITY_HEADERS.items()]
|
|
||||||
if extra:
|
|
||||||
self.headers.extend(
|
|
||||||
(k.lower().encode("latin-1"), v.encode("latin-1"))
|
|
||||||
for k, v in extra.items()
|
|
||||||
)
|
|
||||||
|
|
||||||
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
|
|
||||||
if scope["type"] != "http":
|
|
||||||
await self.app(scope, receive, send)
|
|
||||||
return
|
|
||||||
|
|
||||||
async def wrapped_send(message: Message) -> None:
|
|
||||||
if message["type"] == "http.response.start":
|
|
||||||
headers = list(message.get("headers", []))
|
|
||||||
existing = {k for k, _ in headers}
|
|
||||||
for k, v in self.headers:
|
|
||||||
if k not in existing:
|
|
||||||
headers.append((k, v))
|
|
||||||
message["headers"] = headers
|
|
||||||
await send(message)
|
|
||||||
|
|
||||||
await self.app(scope, receive, wrapped_send)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Reject helper (also writes an audit event when DB is available)
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
async def _send_rejection(
|
|
||||||
scope: Scope,
|
|
||||||
send: Send,
|
|
||||||
*,
|
|
||||||
code: int,
|
|
||||||
reason: str,
|
|
||||||
detail: str,
|
|
||||||
) -> None:
|
|
||||||
"""Build a 413/429 JSON response, send it, and emit a log + audit event."""
|
|
||||||
method = scope.get("method", "GET")
|
|
||||||
path = scope.get("path", "/")
|
|
||||||
ip = _client_ip_from_scope(scope)
|
|
||||||
log.warning(
|
|
||||||
"api.request_rejected",
|
|
||||||
extra={
|
|
||||||
"status": code,
|
|
||||||
"reason": reason,
|
|
||||||
"path": path,
|
|
||||||
"method": method,
|
|
||||||
"ip": ip,
|
|
||||||
"detail": detail,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
payload = {"error": reason, "detail": detail, "status": code}
|
|
||||||
body = json.dumps(payload).encode("utf-8")
|
|
||||||
await send({
|
|
||||||
"type": "http.response.start",
|
|
||||||
"status": code,
|
|
||||||
"headers": [
|
|
||||||
(b"content-type", b"application/json"),
|
|
||||||
(b"content-length", str(len(body)).encode("latin-1")),
|
|
||||||
],
|
|
||||||
})
|
|
||||||
await send({"type": "http.response.body", "body": body, "more_body": False})
|
|
||||||
# Best-effort audit-log append. Don't block the response on a DB
|
|
||||||
# outage (the rejection is the more important signal anyway).
|
|
||||||
try:
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.audit_log import AuditEvent, append_event
|
|
||||||
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
append_event(
|
|
||||||
session,
|
|
||||||
AuditEvent(
|
|
||||||
event_type="api.request_rejected",
|
|
||||||
entity_type="http_request",
|
|
||||||
entity_id=f"{method} {path}",
|
|
||||||
payload={
|
|
||||||
"status": code,
|
|
||||||
"reason": reason,
|
|
||||||
"path": path,
|
|
||||||
"method": method,
|
|
||||||
"ip": ip,
|
|
||||||
},
|
|
||||||
actor=f"api:{ip}",
|
|
||||||
),
|
|
||||||
)
|
|
||||||
session.commit()
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.debug("SP19: audit-log append failed for rejection: %s", exc)
|
|
||||||
|
|
||||||
|
|
||||||
def _reject(
|
|
||||||
request: Request,
|
|
||||||
*,
|
|
||||||
code: int,
|
|
||||||
reason: str,
|
|
||||||
detail: str,
|
|
||||||
) -> JSONResponse:
|
|
||||||
"""Sync helper kept for back-compat (the ``audit_log`` payload path)."""
|
|
||||||
return JSONResponse(
|
|
||||||
{"error": reason, "detail": detail, "status": code},
|
|
||||||
status_code=code,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Health snapshot
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class HealthSnapshot:
|
|
||||||
status: str
|
|
||||||
version: str
|
|
||||||
db: dict[str, Any] = field(default_factory=dict)
|
|
||||||
scheduler: dict[str, Any] = field(default_factory=dict)
|
|
||||||
pubsub: dict[str, Any] = field(default_factory=dict)
|
|
||||||
batch: dict[str, Any] = field(default_factory=dict)
|
|
||||||
|
|
||||||
def to_dict(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"status": self.status,
|
|
||||||
"version": self.version,
|
|
||||||
"db": self.db,
|
|
||||||
"scheduler": self.scheduler,
|
|
||||||
"pubsub": self.pubsub,
|
|
||||||
"batch": self.batch,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def get_health_snapshot() -> HealthSnapshot:
|
|
||||||
"""Gather a best-effort snapshot of every Cyclone subsystem.
|
|
||||||
|
|
||||||
Returns ``HealthSnapshot`` with ``status="ok"`` only if every
|
|
||||||
subsystem check passes. ``"degraded"`` if any subsystem is
|
|
||||||
unhappy but the API itself is responsive. Each subsystem reports
|
|
||||||
independently so one outage doesn't blank out the rest.
|
|
||||||
"""
|
|
||||||
from cyclone import __version__, db
|
|
||||||
|
|
||||||
snap = HealthSnapshot(status="ok", version=__version__)
|
|
||||||
|
|
||||||
# DB connectivity.
|
|
||||||
try:
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
session.execute(db.text("SELECT 1"))
|
|
||||||
snap.db = {"ok": True}
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
snap.db = {"ok": False, "error": str(exc)}
|
|
||||||
snap.status = "degraded"
|
|
||||||
|
|
||||||
# Scheduler state.
|
|
||||||
try:
|
|
||||||
from cyclone import scheduler as scheduler_mod
|
|
||||||
sched = scheduler_mod.get_scheduler()
|
|
||||||
snap.scheduler = {
|
|
||||||
"running": sched.is_running(),
|
|
||||||
"interval_s": sched._poll_interval, # noqa: SLF001
|
|
||||||
"sftp_block": sched._sftp_block_name, # noqa: SLF001
|
|
||||||
}
|
|
||||||
except RuntimeError:
|
|
||||||
snap.scheduler = {"running": False, "configured": False}
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
snap.scheduler = {"ok": False, "error": str(exc)}
|
|
||||||
snap.status = "degraded"
|
|
||||||
|
|
||||||
# Backup scheduler.
|
|
||||||
try:
|
|
||||||
from cyclone import backup_scheduler as bks_mod
|
|
||||||
bks = bks_mod.get_backup_scheduler()
|
|
||||||
snap.scheduler["backup_scheduler_running"] = bks.is_running()
|
|
||||||
snap.scheduler["backup_interval_hours"] = bks.interval_hours
|
|
||||||
except (RuntimeError, ImportError):
|
|
||||||
snap.scheduler["backup_scheduler_running"] = False
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
pass # secondary subsystem; don't degrade the overall status
|
|
||||||
|
|
||||||
# Pubsub bus stats — placeholder. The /api/health handler fills
|
|
||||||
# in the real subscriber counts using request.app.state.event_bus.
|
|
||||||
snap.pubsub = {"note": "filled in by health router"}
|
|
||||||
|
|
||||||
# Last batch timestamp + count.
|
|
||||||
try:
|
|
||||||
from cyclone import db as db_mod
|
|
||||||
from cyclone.db import Batch
|
|
||||||
with db_mod.SessionLocal()() as session:
|
|
||||||
row = (
|
|
||||||
session.query(Batch)
|
|
||||||
.order_by(Batch.parsed_at.desc())
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
if row is not None:
|
|
||||||
snap.batch = {
|
|
||||||
"last_batch_id": row.id,
|
|
||||||
"last_batch_kind": row.kind,
|
|
||||||
"last_batch_at": row.parsed_at.isoformat() if row.parsed_at else None,
|
|
||||||
"last_batch_filename": row.input_filename,
|
|
||||||
}
|
|
||||||
else:
|
|
||||||
snap.batch = {"last_batch_id": None, "note": "no batches yet"}
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
snap.batch = {"ok": False, "error": str(exc)}
|
|
||||||
snap.status = "degraded"
|
|
||||||
|
|
||||||
return snap
|
|
||||||
@@ -1668,16 +1668,7 @@ class CycloneStore:
|
|||||||
return list(by_npi.values())
|
return list(by_npi.values())
|
||||||
|
|
||||||
def recent_activity(self, *, limit: int = 200) -> list[dict]:
|
def recent_activity(self, *, limit: int = 200) -> list[dict]:
|
||||||
"""Return recent activity events from the DB, newest first.
|
"""Return recent activity events from the DB, newest first."""
|
||||||
|
|
||||||
SP21 Task 2.5: each row also carries ``claimId`` and
|
|
||||||
``remittanceId`` (read from the ORM columns) so the Dashboard's
|
|
||||||
Recent-activity card can route clicks to the right entity
|
|
||||||
drawer via ``src/lib/event-routing.ts``. Both are nullable
|
|
||||||
strings; the wire shape uses camelCase keys to match the
|
|
||||||
existing ``npi`` / ``amount`` fields and the frontend
|
|
||||||
``Activity`` interface.
|
|
||||||
"""
|
|
||||||
with db.SessionLocal()() as s:
|
with db.SessionLocal()() as s:
|
||||||
rows = (
|
rows = (
|
||||||
s.query(ActivityEvent)
|
s.query(ActivityEvent)
|
||||||
@@ -1693,8 +1684,6 @@ class CycloneStore:
|
|||||||
"timestamp": r.ts.isoformat().replace("+00:00", "Z"),
|
"timestamp": r.ts.isoformat().replace("+00:00", "Z"),
|
||||||
"npi": (r.payload_json or {}).get("npi"),
|
"npi": (r.payload_json or {}).get("npi"),
|
||||||
"amount": (r.payload_json or {}).get("amount"),
|
"amount": (r.payload_json or {}).get("amount"),
|
||||||
"claimId": r.claim_id,
|
|
||||||
"remittanceId": r.remittance_id,
|
|
||||||
}
|
}
|
||||||
for r in rows
|
for r in rows
|
||||||
]
|
]
|
||||||
@@ -1860,32 +1849,6 @@ class CycloneStore:
|
|||||||
with db.SessionLocal()() as s:
|
with db.SessionLocal()() as s:
|
||||||
return s.get(db.Two77caAck, ack_id)
|
return s.get(db.Two77caAck, ack_id)
|
||||||
|
|
||||||
# -- SP17: encrypted DB backups -------------------------------------
|
|
||||||
|
|
||||||
def add_backup_pending(self, *, filename: str, backup_dir: str) -> db.DbBackup:
|
|
||||||
"""Insert a ``pending`` row for a backup that is about to start.
|
|
||||||
|
|
||||||
The BackupService fills in ``status`` / ``size_bytes`` /
|
|
||||||
``db_fingerprint`` / ``table_count`` / ``completed_at`` after
|
|
||||||
the encrypted blob lands on disk.
|
|
||||||
"""
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = db.DbBackup(
|
|
||||||
filename=filename,
|
|
||||||
backup_dir=backup_dir,
|
|
||||||
size_bytes=0,
|
|
||||||
db_fingerprint=None,
|
|
||||||
table_count=0,
|
|
||||||
created_at=utcnow(),
|
|
||||||
completed_at=None,
|
|
||||||
status="pending",
|
|
||||||
error_message=None,
|
|
||||||
)
|
|
||||||
s.add(row)
|
|
||||||
s.commit()
|
|
||||||
s.refresh(row)
|
|
||||||
return row
|
|
||||||
|
|
||||||
# -- manual reconciliation (T12) -----------------------------------
|
# -- manual reconciliation (T12) -----------------------------------
|
||||||
|
|
||||||
def list_unmatched(self, *, kind: str = "both") -> dict:
|
def list_unmatched(self, *, kind: str = "both") -> dict:
|
||||||
@@ -2277,7 +2240,7 @@ class CycloneStore:
|
|||||||
submitter_contact_email="tyler@dzinesco.com",
|
submitter_contact_email="tyler@dzinesco.com",
|
||||||
filename_block={
|
filename_block={
|
||||||
"tz": "America/Denver",
|
"tz": "America/Denver",
|
||||||
"outbound_template": "tp{tpid}-{tx}-{ts_mt}-1of1.{ext}",
|
"outbound_template": "{tpid}-{tx}-{ts_mt}-1of1.{ext}",
|
||||||
"inbound_template": "TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12",
|
"inbound_template": "TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12",
|
||||||
},
|
},
|
||||||
sftp_block={
|
sftp_block={
|
||||||
|
|||||||
+1
-1
@@ -7,7 +7,7 @@ PER*IC*Test Contact*EM*test@example.com~
|
|||||||
NM1*40*2*COLORADO MEDICAL ASSISTANCE PROGRAM*****46*COMEDASSISTPROG~
|
NM1*40*2*COLORADO MEDICAL ASSISTANCE PROGRAM*****46*COMEDASSISTPROG~
|
||||||
HL*1**20*1~
|
HL*1**20*1~
|
||||||
PRV*BI*PXC*251E00000X~
|
PRV*BI*PXC*251E00000X~
|
||||||
NM1*85*2*Test Provider Inc*****XX*1993999998~
|
NM1*85*2*Test Provider Inc*****XX*1234567890~
|
||||||
N3*123 Test St~
|
N3*123 Test St~
|
||||||
N4*Denver*CO*80202~
|
N4*Denver*CO*80202~
|
||||||
REF*EI*123456789~
|
REF*EI*123456789~
|
||||||
|
|||||||
-1
@@ -1 +0,0 @@
|
|||||||
ISA*00* *00* *ZZ*SENDER *ZZ*RECEIVER *260520*1750*^*00501*000000001*0*P*:~TA1*000000001*20260520*1750*A*000*20260520~IEA*1*000000001~
|
|
||||||
@@ -51,19 +51,18 @@ def test_migration_0002_creates_acks_table():
|
|||||||
|
|
||||||
def test_migration_latest_idempotent_on_fresh_db():
|
def test_migration_latest_idempotent_on_fresh_db():
|
||||||
"""Re-running the migration on the same DB must be a no-op (PRAGMA
|
"""Re-running the migration on the same DB must be a no-op (PRAGMA
|
||||||
user_version already at the latest version — currently 12 after
|
user_version already at the latest version — currently 10 after
|
||||||
0004-0006 line_reconciliation, 0005 ta1_acks, SP9's 0007
|
0004-0006 line_reconciliation, 0005 ta1_acks, SP9's 0007
|
||||||
providers/payers/clearhouse, SP10's 0008 payer_rejected,
|
providers/payers/clearhouse, SP10's 0008 payer_rejected,
|
||||||
SP11's 0009 audit_log, SP14's 0010 payer_rejected_acknowledged,
|
SP11's 0009 audit_log, and SP14's 0010 payer_rejected_acknowledged)."""
|
||||||
SP16's 0011 processed_inbound_files, SP17's 0012 db_backups)."""
|
|
||||||
with db.engine().begin() as c:
|
with db.engine().begin() as c:
|
||||||
v1 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
v1 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
||||||
assert v1 == 12
|
assert v1 == 10
|
||||||
# A second run should not raise and should not bump the version.
|
# A second run should not raise and should not bump the version.
|
||||||
db_migrate.run(db.engine())
|
db_migrate.run(db.engine())
|
||||||
with db.engine().begin() as c:
|
with db.engine().begin() as c:
|
||||||
v2 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
v2 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
||||||
assert v2 == 12
|
assert v2 == 10
|
||||||
|
|
||||||
|
|
||||||
def test_add_ack_persists_row():
|
def test_add_ack_persists_row():
|
||||||
|
|||||||
@@ -31,18 +31,10 @@ def client() -> TestClient:
|
|||||||
|
|
||||||
|
|
||||||
def test_health_endpoint(client: TestClient):
|
def test_health_endpoint(client: TestClient):
|
||||||
"""SP19: health endpoint now returns a subsystem snapshot."""
|
|
||||||
resp = client.get("/api/health")
|
resp = client.get("/api/health")
|
||||||
assert resp.status_code == 200
|
assert resp.status_code == 200
|
||||||
body = resp.json()
|
body = resp.json()
|
||||||
# Old contract (status + version) is preserved.
|
assert body == {"status": "ok", "version": __version__}
|
||||||
assert body["status"] == "ok"
|
|
||||||
assert body["version"] == __version__
|
|
||||||
# SP19 additions.
|
|
||||||
assert "db" in body and body["db"].get("ok") is True
|
|
||||||
assert "scheduler" in body
|
|
||||||
assert "pubsub" in body
|
|
||||||
assert "batch" in body
|
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
@@ -101,14 +93,6 @@ def test_parse_837_endpoint_streams_ndjson(client: TestClient):
|
|||||||
# Summary numbers match the JSON path.
|
# Summary numbers match the JSON path.
|
||||||
assert parsed[3]["data"]["total_claims"] == 2
|
assert parsed[3]["data"]["total_claims"] == 2
|
||||||
assert parsed[3]["data"]["passed"] == 2
|
assert parsed[3]["data"]["passed"] == 2
|
||||||
# The streaming summary carries the server-side batch id so the
|
|
||||||
# frontend can call /api/batches/{id}/export-837 without a separate
|
|
||||||
# GET /api/batches round-trip (regression: it used to be missing
|
|
||||||
# on the stream path, only present on the JSON path, which made the
|
|
||||||
# Upload page's Export button say "no batch to export").
|
|
||||||
assert parsed[3]["type"] == "summary"
|
|
||||||
assert isinstance(parsed[3]["data"]["batch_id"], str)
|
|
||||||
assert len(parsed[3]["data"]["batch_id"]) == 32 # uuid4().hex
|
|
||||||
|
|
||||||
|
|
||||||
def test_parse_837_endpoint_streams_ndjson_without_raw_segments(client: TestClient):
|
def test_parse_837_endpoint_streams_ndjson_without_raw_segments(client: TestClient):
|
||||||
@@ -168,51 +152,3 @@ def test_cors_headers_present(client: TestClient):
|
|||||||
)
|
)
|
||||||
assert resp.headers.get("access-control-allow-origin") == "http://localhost:5173"
|
assert resp.headers.get("access-control-allow-origin") == "http://localhost:5173"
|
||||||
assert "POST" in resp.headers.get("access-control-allow-methods", "").upper()
|
assert "POST" in resp.headers.get("access-control-allow-methods", "").upper()
|
||||||
|
|
||||||
|
|
||||||
def test_cors_headers_present_for_loopback_ip(client: TestClient):
|
|
||||||
# ``http://127.0.0.1:5173`` is a distinct origin from
|
|
||||||
# ``http://localhost:5173`` per the CORS spec, even though both resolve
|
|
||||||
# to the same Vite dev server. Both must be allow-listed or tabs opened
|
|
||||||
# via the IP form silently break.
|
|
||||||
resp = client.options(
|
|
||||||
"/api/parse-837",
|
|
||||||
headers={
|
|
||||||
"Origin": "http://127.0.0.1:5173",
|
|
||||||
"Access-Control-Request-Method": "POST",
|
|
||||||
"Access-Control-Request-Headers": "content-type",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
assert resp.headers.get("access-control-allow-origin") == "http://127.0.0.1:5173"
|
|
||||||
|
|
||||||
|
|
||||||
def test_cors_extra_origins_via_env(client: TestClient, monkeypatch):
|
|
||||||
# LAN / staging hosts opt in via CYCLONE_ALLOWED_ORIGINS. The env var
|
|
||||||
# is a comma-separated list; the middleware must reflect each entry.
|
|
||||||
# The allow-list is built at module import, so we re-execute the
|
|
||||||
# module under the env var and build a TestClient against the
|
|
||||||
# reloaded app.
|
|
||||||
monkeypatch.setenv(
|
|
||||||
"CYCLONE_ALLOWED_ORIGINS", "http://192.168.1.42:5173,https://staging.example.com"
|
|
||||||
)
|
|
||||||
import importlib
|
|
||||||
from cyclone import api as api_module
|
|
||||||
from fastapi.testclient import TestClient as _TC
|
|
||||||
importlib.reload(api_module)
|
|
||||||
try:
|
|
||||||
with _TC(api_module.app) as tc:
|
|
||||||
for origin in ("http://192.168.1.42:5173", "https://staging.example.com"):
|
|
||||||
resp = tc.options(
|
|
||||||
"/api/parse-837",
|
|
||||||
headers={
|
|
||||||
"Origin": origin,
|
|
||||||
"Access-Control-Request-Method": "POST",
|
|
||||||
"Access-Control-Request-Headers": "content-type",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
assert resp.headers.get("access-control-allow-origin") == origin
|
|
||||||
finally:
|
|
||||||
monkeypatch.delenv("CYCLONE_ALLOWED_ORIGINS", raising=False)
|
|
||||||
# Reload once more so the module-level allow-list returns to its
|
|
||||||
# default for any test that imports `cyclone.api` after this one.
|
|
||||||
importlib.reload(api_module)
|
|
||||||
|
|||||||
@@ -82,13 +82,6 @@ def test_parse_835_endpoint_streams_ndjson(client: TestClient):
|
|||||||
# Summary numbers match the JSON path.
|
# Summary numbers match the JSON path.
|
||||||
assert parsed[7]["data"]["total_claims"] == 2
|
assert parsed[7]["data"]["total_claims"] == 2
|
||||||
assert parsed[7]["data"]["passed"] == 2
|
assert parsed[7]["data"]["passed"] == 2
|
||||||
# The streaming summary carries the server-side batch id so the
|
|
||||||
# frontend can call /api/batches/{id}/export-837 without a separate
|
|
||||||
# GET /api/batches round-trip (matches the parallel fix on
|
|
||||||
# /api/parse-837).
|
|
||||||
assert parsed[7]["type"] == "summary"
|
|
||||||
assert isinstance(parsed[7]["data"]["batch_id"], str)
|
|
||||||
assert len(parsed[7]["data"]["batch_id"]) == 32 # uuid4().hex
|
|
||||||
|
|
||||||
|
|
||||||
def test_parse_835_endpoint_streams_ndjson_without_raw_segments(client: TestClient):
|
def test_parse_835_endpoint_streams_ndjson_without_raw_segments(client: TestClient):
|
||||||
|
|||||||
@@ -1,313 +0,0 @@
|
|||||||
"""SP17 — Admin backup API endpoint tests.
|
|
||||||
|
|
||||||
Covers:
|
|
||||||
- POST /api/admin/backup/create
|
|
||||||
- GET /api/admin/backup/list
|
|
||||||
- GET /api/admin/backup/status
|
|
||||||
- POST /api/admin/backup/{id}/verify
|
|
||||||
- POST /api/admin/backup/{id}/restore/initiate
|
|
||||||
- POST /api/admin/backup/{id}/restore/confirm
|
|
||||||
- POST /api/admin/backup/prune
|
|
||||||
- POST /api/admin/backup/scheduler/{start,stop,tick}
|
|
||||||
|
|
||||||
Each fixture starts a clean DB + BackupService configured with a
|
|
||||||
known passphrase. We deliberately do NOT enable SQLCipher here —
|
|
||||||
the backup layer is independent of SQLCipher encryption at rest.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def _backup_env(tmp_path, monkeypatch):
|
|
||||||
"""Fresh sqlite DB + BackupService with passphrase. Reset module singletons."""
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import backup_scheduler as sched_mod
|
|
||||||
from cyclone.db import Batch
|
|
||||||
|
|
||||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
|
||||||
db._reset_for_tests()
|
|
||||||
db.init_db()
|
|
||||||
|
|
||||||
# Make sure there's at least one row so the backup isn't a no-op.
|
|
||||||
import uuid
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
s.add(Batch(
|
|
||||||
id=str(uuid.uuid4()),
|
|
||||||
kind="837P",
|
|
||||||
input_filename="seed.x12",
|
|
||||||
parsed_at=datetime.now(timezone.utc),
|
|
||||||
totals_json=None,
|
|
||||||
validation_json=None,
|
|
||||||
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
backup_dir = tmp_path / "backups"
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
sched_mod.reset_backup_scheduler_for_tests()
|
|
||||||
svc = svc_mod.configure_backup_service(
|
|
||||||
backup_dir=backup_dir, passphrase="api-test-pass", retention_days=7,
|
|
||||||
)
|
|
||||||
yield svc, backup_dir
|
|
||||||
sched_mod.reset_backup_scheduler_for_tests()
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
db._reset_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
def _client():
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
from cyclone.api import app
|
|
||||||
return TestClient(app)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# /backup/create
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_create_returns_metadata_and_persists_row(_backup_env):
|
|
||||||
svc, backup_dir = _backup_env
|
|
||||||
r = _client().post("/api/admin/backup/create")
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
body = r.json()
|
|
||||||
assert body["ok"] is True
|
|
||||||
b = body["backup"]
|
|
||||||
assert b["size_bytes"] > 0
|
|
||||||
assert b["db_fingerprint"].startswith("sha256:")
|
|
||||||
assert b["table_count"] >= 1
|
|
||||||
assert b["created_at"]
|
|
||||||
# File actually exists on disk.
|
|
||||||
assert (backup_dir / b["filename"]).exists()
|
|
||||||
# Sidecar metadata echoed.
|
|
||||||
sc = body["sidecar"]
|
|
||||||
assert sc["kdf"] == "PBKDF2-HMAC-SHA256"
|
|
||||||
assert sc["kdf_iterations"] == 200_000
|
|
||||||
assert sc["cipher"] == "AES-256-GCM"
|
|
||||||
|
|
||||||
|
|
||||||
def test_create_503_when_service_unconfigured(tmp_path, monkeypatch):
|
|
||||||
"""If BackupService was never configured, create returns 503."""
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
|
||||||
db._reset_for_tests()
|
|
||||||
db.init_db()
|
|
||||||
svc_mod.reset_backup_service_for_tests()
|
|
||||||
try:
|
|
||||||
r = _client().post("/api/admin/backup/create")
|
|
||||||
assert r.status_code == 503
|
|
||||||
assert "not configured" in r.json()["detail"].lower()
|
|
||||||
finally:
|
|
||||||
db._reset_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# /backup/list
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_list_returns_newest_first(_backup_env):
|
|
||||||
svc, _ = _backup_env
|
|
||||||
client = _client()
|
|
||||||
client.post("/api/admin/backup/create")
|
|
||||||
client.post("/api/admin/backup/create")
|
|
||||||
r = client.get("/api/admin/backup/list")
|
|
||||||
assert r.status_code == 200
|
|
||||||
body = r.json()
|
|
||||||
assert body["count"] == 2
|
|
||||||
assert body["files"][0]["id"] > body["files"][1]["id"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_list_filter_by_status(_backup_env):
|
|
||||||
svc, _ = _backup_env
|
|
||||||
client = _client()
|
|
||||||
client.post("/api/admin/backup/create")
|
|
||||||
r = client.get("/api/admin/backup/list?status=ok")
|
|
||||||
assert r.json()["count"] == 1
|
|
||||||
r = client.get("/api/admin/backup/list?status=error")
|
|
||||||
assert r.json()["count"] == 0
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# /backup/status
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_status_returns_counts_and_dirs(_backup_env):
|
|
||||||
svc, backup_dir = _backup_env
|
|
||||||
client = _client()
|
|
||||||
client.post("/api/admin/backup/create")
|
|
||||||
r = client.get("/api/admin/backup/status")
|
|
||||||
assert r.status_code == 200
|
|
||||||
body = r.json()
|
|
||||||
assert body["totals"]["ok"] == 1
|
|
||||||
assert body["backup_dir"] == str(backup_dir)
|
|
||||||
assert body["retention_days"] == 7
|
|
||||||
assert body["last_backup_at"] is not None
|
|
||||||
assert body["last_ok_backup_at"] is not None
|
|
||||||
# The scheduler may or may not be configured depending on lifespan.
|
|
||||||
assert "scheduler" in body
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# /backup/{id}/verify
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_verify_ok_after_create(_backup_env):
|
|
||||||
svc, _ = _backup_env
|
|
||||||
client = _client()
|
|
||||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
|
||||||
r = client.post(f"/api/admin/backup/{cid}/verify")
|
|
||||||
assert r.status_code == 200
|
|
||||||
body = r.json()
|
|
||||||
assert body["ok"] is True
|
|
||||||
assert body["expected_fingerprint"] == body["actual_fingerprint"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_verify_detects_tampered_ciphertext(_backup_env):
|
|
||||||
from cyclone import backup as backup_mod
|
|
||||||
svc, backup_dir = _backup_env
|
|
||||||
client = _client()
|
|
||||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
|
||||||
fname = svc.list_backups()[0].filename
|
|
||||||
# Flip a bit in the ciphertext.
|
|
||||||
bin_path = backup_dir / fname
|
|
||||||
data = bytearray(bin_path.read_bytes())
|
|
||||||
data[backup_mod.NONCE_LEN + 5] ^= 0x01
|
|
||||||
bin_path.write_bytes(bytes(data))
|
|
||||||
r = client.post(f"/api/admin/backup/{cid}/verify")
|
|
||||||
assert r.status_code == 200
|
|
||||||
assert r.json()["ok"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_verify_404_when_unknown_backup(_backup_env):
|
|
||||||
r = _client().post("/api/admin/backup/99999/verify")
|
|
||||||
# The service raises BackupError; the endpoint should return 503 (no svc) or 400
|
|
||||||
# depending on flow. Let's see what happens.
|
|
||||||
assert r.status_code in (400, 404, 503)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# /backup/{id}/restore/{initiate,confirm}
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_restore_two_step_via_api(_backup_env):
|
|
||||||
svc, _ = _backup_env
|
|
||||||
from cyclone.db import Batch
|
|
||||||
import uuid
|
|
||||||
client = _client()
|
|
||||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
|
||||||
|
|
||||||
# Mutate the live DB (add another Batch row).
|
|
||||||
with __import__("cyclone").db.SessionLocal()() as s:
|
|
||||||
s.add(Batch(
|
|
||||||
id=str(uuid.uuid4()),
|
|
||||||
kind="837P",
|
|
||||||
input_filename="mutated.x12",
|
|
||||||
parsed_at=datetime.now(timezone.utc),
|
|
||||||
totals_json=None,
|
|
||||||
validation_json=None,
|
|
||||||
raw_result_json={"envelope": {"control_number": "2"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
# Step 1: initiate.
|
|
||||||
r1 = client.post(f"/api/admin/backup/{cid}/restore/initiate")
|
|
||||||
assert r1.status_code == 200, r1.text
|
|
||||||
body1 = r1.json()
|
|
||||||
assert body1["restore_token"]
|
|
||||||
assert body1["preview"]["backup_table_count"] >= 1
|
|
||||||
assert body1["preview"]["backup_db_fingerprint"] != body1["preview"]["current_db_fingerprint"]
|
|
||||||
|
|
||||||
# Step 2: confirm.
|
|
||||||
r2 = client.post(
|
|
||||||
f"/api/admin/backup/{cid}/restore/confirm",
|
|
||||||
json={"restore_token": body1["restore_token"], "actor": "test"},
|
|
||||||
)
|
|
||||||
assert r2.status_code == 200, r2.text
|
|
||||||
body2 = r2.json()
|
|
||||||
assert body2["ok"] is True
|
|
||||||
assert body2["new_db_fingerprint"] == body1["preview"]["backup_db_fingerprint"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_restore_confirm_requires_token(_backup_env):
|
|
||||||
svc, _ = _backup_env
|
|
||||||
client = _client()
|
|
||||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
|
||||||
r = client.post(f"/api/admin/backup/{cid}/restore/confirm", json={})
|
|
||||||
assert r.status_code == 400
|
|
||||||
|
|
||||||
|
|
||||||
def test_restore_confirm_rejects_wrong_token(_backup_env):
|
|
||||||
svc, _ = _backup_env
|
|
||||||
client = _client()
|
|
||||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
|
||||||
r = client.post(
|
|
||||||
f"/api/admin/backup/{cid}/restore/confirm",
|
|
||||||
json={"restore_token": "0" * 64},
|
|
||||||
)
|
|
||||||
assert r.status_code == 400
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# /backup/prune
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_prune_deletes_old_backups(_backup_env):
|
|
||||||
svc, _ = _backup_env
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
client = _client()
|
|
||||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
|
||||||
# Age the backup past the retention cutoff.
|
|
||||||
with __import__("cyclone").db.SessionLocal()() as s:
|
|
||||||
row = s.get(DbBackup, cid)
|
|
||||||
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
|
||||||
s.commit()
|
|
||||||
r = client.post("/api/admin/backup/prune")
|
|
||||||
assert r.status_code == 200
|
|
||||||
body = r.json()
|
|
||||||
assert body["ok"] is True
|
|
||||||
assert body["deleted_count"] == 2 # .bin + .meta.json
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# /backup/scheduler/{start,stop,tick}
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_scheduler_endpoints_require_configured_scheduler(_backup_env, monkeypatch):
|
|
||||||
"""Without calling configure_backup_scheduler, the endpoints 503."""
|
|
||||||
svc, _ = _backup_env
|
|
||||||
# We did NOT call configure_backup_scheduler; the lifespan
|
|
||||||
# *might* have called it as a side effect of the TestClient
|
|
||||||
# entering its context. Either way, the scheduler endpoints
|
|
||||||
# need it to be present.
|
|
||||||
client = _client()
|
|
||||||
r = client.post("/api/admin/backup/scheduler/tick")
|
|
||||||
assert r.status_code in (200, 503)
|
|
||||||
|
|
||||||
|
|
||||||
def test_scheduler_tick_when_configured(_backup_env):
|
|
||||||
"""With a configured scheduler, tick runs and returns a result."""
|
|
||||||
from cyclone import backup_scheduler as sched_mod
|
|
||||||
svc, _ = _backup_env
|
|
||||||
sched_mod.configure_backup_scheduler(svc, interval_hours=24.0)
|
|
||||||
try:
|
|
||||||
client = _client()
|
|
||||||
r = client.post("/api/admin/backup/scheduler/tick")
|
|
||||||
assert r.status_code == 200
|
|
||||||
body = r.json()
|
|
||||||
assert body["ok"] is True
|
|
||||||
assert body["tick"]["created"] is not None
|
|
||||||
assert body["tick"]["created"]["id"] >= 1
|
|
||||||
finally:
|
|
||||||
sched_mod.reset_backup_scheduler_for_tests()
|
|
||||||
@@ -1,317 +0,0 @@
|
|||||||
"""Tests for POST /api/batches/{batch_id}/export-837.
|
|
||||||
|
|
||||||
Reads Claim.raw_json for each requested claim_id and returns a ZIP of
|
|
||||||
regenerated X12 837 files. No DB state mutation. Mirrors the
|
|
||||||
X-Cyclone-Serialize-Errors convention from /api/inbox/rejected/resubmit?download=true.
|
|
||||||
"""
|
|
||||||
import io
|
|
||||||
import json
|
|
||||||
import zipfile
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
from cyclone.api import app
|
|
||||||
|
|
||||||
|
|
||||||
def _seed_batch(client: TestClient, filename: str = "claim.txt") -> dict:
|
|
||||||
"""Parse a single 837P file and return a dict with ``batch_id`` and
|
|
||||||
``claims``.
|
|
||||||
|
|
||||||
The parse-837 happy-path response does not currently surface
|
|
||||||
``batch_id`` at the top level (it's a server-side UUID, surfaced in
|
|
||||||
409 errors but not in 200s). We look it up from the DB — the Batch
|
|
||||||
row is already persisted by the time the parse endpoint returns.
|
|
||||||
"""
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.db import Batch
|
|
||||||
|
|
||||||
fixture = Path("tests/fixtures/co_medicaid_837p.txt").read_text()
|
|
||||||
r = client.post(
|
|
||||||
"/api/parse-837",
|
|
||||||
files={"file": (filename, io.BytesIO(fixture.encode()), "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
body = r.json()
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
most_recent = s.query(Batch).order_by(Batch.parsed_at.desc()).first()
|
|
||||||
assert most_recent is not None, "expected at least one Batch row after parse"
|
|
||||||
batch_id = most_recent.id
|
|
||||||
return {"batch_id": batch_id, "claims": body["claims"]}
|
|
||||||
|
|
||||||
|
|
||||||
def _claim_ids_from_seed(seeded: dict) -> list[str]:
|
|
||||||
return [c["claim_id"] for c in seeded["claims"]]
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Happy path
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_happy_path_returns_zip_with_one_x12_per_claim():
|
|
||||||
with TestClient(app) as client:
|
|
||||||
seeded = _seed_batch(client)
|
|
||||||
batch_id = seeded["batch_id"]
|
|
||||||
claim_ids = _claim_ids_from_seed(seeded)
|
|
||||||
|
|
||||||
r = client.post(
|
|
||||||
f"/api/batches/{batch_id}/export-837",
|
|
||||||
json={"claim_ids": claim_ids},
|
|
||||||
)
|
|
||||||
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
assert r.headers["content-type"].startswith("application/zip")
|
|
||||||
assert "attachment" in r.headers["content-disposition"]
|
|
||||||
assert (
|
|
||||||
f"batch-{batch_id}-{len(claim_ids)}-claims.zip"
|
|
||||||
in r.headers["content-disposition"]
|
|
||||||
)
|
|
||||||
# Per-claim failures header absent on full success.
|
|
||||||
assert "x-cyclone-serialize-errors" not in {k.lower() for k in r.headers.keys()}
|
|
||||||
|
|
||||||
with zipfile.ZipFile(io.BytesIO(r.content)) as zf:
|
|
||||||
names = zf.namelist()
|
|
||||||
assert len(names) == len(claim_ids)
|
|
||||||
# Each entry follows the HCPF outbound naming template
|
|
||||||
# "{tpid}-837P-{yyyymmddhhmmssSSS}-1of1.x12" (the seeded
|
|
||||||
# clearhouse TPID is "11525703"). Every name must be unique.
|
|
||||||
from cyclone.edi.filenames import is_outbound_filename
|
|
||||||
seen = set()
|
|
||||||
for name in names:
|
|
||||||
assert is_outbound_filename(name), (
|
|
||||||
f"expected HCPF outbound filename, got {name!r}"
|
|
||||||
)
|
|
||||||
assert name not in seen, f"duplicate filename: {name}"
|
|
||||||
seen.add(name)
|
|
||||||
with zf.open(name) as f:
|
|
||||||
first_line = f.readline().decode("ascii", errors="replace")
|
|
||||||
assert first_line.startswith("ISA*"), f"{name} didn't start with ISA"
|
|
||||||
|
|
||||||
|
|
||||||
def test_each_x12_in_zip_uses_clearhouse_submit_and_payer_receiver():
|
|
||||||
"""Regression: regenerated 837s used to emit 'CYCLONE' / 'RECEIVER'
|
|
||||||
placeholders. The export endpoint must thread the clearhouse
|
|
||||||
submitter (dzinesco's TPID 11525703) and payer receiver
|
|
||||||
(COMEDASSISTPROG) through to the serializer."""
|
|
||||||
with TestClient(app) as client:
|
|
||||||
seeded = _seed_batch(client)
|
|
||||||
batch_id = seeded["batch_id"]
|
|
||||||
claim_ids = _claim_ids_from_seed(seeded)
|
|
||||||
r = client.post(
|
|
||||||
f"/api/batches/{batch_id}/export-837",
|
|
||||||
json={"claim_ids": claim_ids},
|
|
||||||
)
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
|
|
||||||
with zipfile.ZipFile(io.BytesIO(r.content)) as zf:
|
|
||||||
for name in zf.namelist():
|
|
||||||
with zf.open(name) as f:
|
|
||||||
text = f.read().decode("ascii")
|
|
||||||
# Submitter block must use the clearhouse TPID + name, not
|
|
||||||
# the 'CYCLONE' placeholder.
|
|
||||||
assert "CYCLONE" not in text, f"{name} still emits CYCLONE placeholder"
|
|
||||||
assert "11525703" in text, f"{name} missing clearhouse TPID"
|
|
||||||
assert "Dzinesco" in text, f"{name} missing clearhouse name"
|
|
||||||
# Receiver block must use the CO_TXIX payer config
|
|
||||||
# (COMEDASSISTPROG), not the 'RECEIVER' placeholder.
|
|
||||||
assert "RECEIVER" not in text, f"{name} still emits RECEIVER placeholder"
|
|
||||||
assert "COMEDASSISTPROG" in text, f"{name} missing receiver id"
|
|
||||||
# Loop 1000A requires PER — must be present, not omitted.
|
|
||||||
assert "PER*IC*" in text, f"{name} missing required PER segment"
|
|
||||||
# SBR09 should be 'MC' (Medicaid claim filing indicator),
|
|
||||||
# not the member id.
|
|
||||||
sbr_line = next(seg for seg in text.split("~") if seg.startswith("SBR*"))
|
|
||||||
sbr09 = sbr_line.rstrip("~").split("*")[9]
|
|
||||||
assert sbr09 == "MC", f"{name} SBR09 expected 'MC', got {sbr09!r}"
|
|
||||||
|
|
||||||
|
|
||||||
def test_each_x12_in_zip_round_trips_through_parser():
|
|
||||||
"""Fidelity check: each .x12 must parse back to a ClaimOutput deep-equal
|
|
||||||
to the source row's raw_json (modulo recomputed validation)."""
|
|
||||||
from cyclone.parsers.parse_837 import parse
|
|
||||||
from cyclone.parsers.payer import PayerConfig
|
|
||||||
|
|
||||||
with TestClient(app) as client:
|
|
||||||
seeded = _seed_batch(client)
|
|
||||||
batch_id = seeded["batch_id"]
|
|
||||||
claim_ids = _claim_ids_from_seed(seeded)
|
|
||||||
|
|
||||||
r = client.post(
|
|
||||||
f"/api/batches/{batch_id}/export-837",
|
|
||||||
json={"claim_ids": claim_ids},
|
|
||||||
)
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
|
|
||||||
by_id = {c["claim_id"]: c for c in seeded["claims"]}
|
|
||||||
with zipfile.ZipFile(io.BytesIO(r.content)) as zf:
|
|
||||||
for name in zf.namelist():
|
|
||||||
with zf.open(name) as f:
|
|
||||||
text = f.read().decode("ascii")
|
|
||||||
result = parse(text, PayerConfig(name="CO_MEDICAID"))
|
|
||||||
assert result.claims, f"{name} didn't parse back to any claims"
|
|
||||||
# Claim ids must round-trip (proves the serializer didn't drop
|
|
||||||
# or rewrite the id).
|
|
||||||
assert result.claims[0].claim.claim_id in by_id, (
|
|
||||||
f"{name} parsed to an unknown claim_id"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Partial-failure surface
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_partial_failure_one_claim_with_no_raw_json_returns_zip_and_errors_header():
|
|
||||||
"""If one claim has raw_json=None, the ZIP still returns for the others
|
|
||||||
and the failure is surfaced via X-Cyclone-Serialize-Errors."""
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.edi.filenames import is_outbound_filename
|
|
||||||
|
|
||||||
with TestClient(app) as client:
|
|
||||||
seeded = _seed_batch(client)
|
|
||||||
batch_id = seeded["batch_id"]
|
|
||||||
claim_ids = _claim_ids_from_seed(seeded)
|
|
||||||
assert len(claim_ids) >= 2, "fixture must produce at least 2 claims"
|
|
||||||
|
|
||||||
# Wipe raw_json on one claim to simulate a corrupted row.
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
from cyclone.db import Claim
|
|
||||||
target = s.get(Claim, claim_ids[0])
|
|
||||||
target.raw_json = None
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
r = client.post(
|
|
||||||
f"/api/batches/{batch_id}/export-837",
|
|
||||||
json={"claim_ids": claim_ids},
|
|
||||||
)
|
|
||||||
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
# Filename uses SUCCESS count, not requested count.
|
|
||||||
expected_success = len(claim_ids) - 1
|
|
||||||
assert (
|
|
||||||
f"batch-{batch_id}-{expected_success}-claims.zip"
|
|
||||||
in r.headers["content-disposition"]
|
|
||||||
)
|
|
||||||
err_header = r.headers.get("x-cyclone-serialize-errors")
|
|
||||||
assert err_header, "expected X-Cyclone-Serialize-Errors header"
|
|
||||||
errs = json.loads(err_header)
|
|
||||||
assert len(errs) == 1
|
|
||||||
assert errs[0]["claim_id"] == claim_ids[0]
|
|
||||||
assert "raw_json" in errs[0]["reason"].lower()
|
|
||||||
|
|
||||||
with zipfile.ZipFile(io.BytesIO(r.content)) as zf:
|
|
||||||
names = zf.namelist()
|
|
||||||
assert len(names) == expected_success
|
|
||||||
# Every entry follows HCPF outbound template; none is the
|
|
||||||
# 'claim-CLM-X.x12' placeholder from the old endpoint.
|
|
||||||
for name in names:
|
|
||||||
assert is_outbound_filename(name), f"non-HCPF name: {name!r}"
|
|
||||||
|
|
||||||
|
|
||||||
def test_all_claims_fail_to_serialize_returns_422():
|
|
||||||
from cyclone import db
|
|
||||||
|
|
||||||
with TestClient(app) as client:
|
|
||||||
seeded = _seed_batch(client)
|
|
||||||
batch_id = seeded["batch_id"]
|
|
||||||
claim_ids = _claim_ids_from_seed(seeded)
|
|
||||||
assert len(claim_ids) >= 1
|
|
||||||
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
from cyclone.db import Claim
|
|
||||||
for cid in claim_ids:
|
|
||||||
c = s.get(Claim, cid)
|
|
||||||
c.raw_json = None
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
r = client.post(
|
|
||||||
f"/api/batches/{batch_id}/export-837",
|
|
||||||
json={"claim_ids": claim_ids},
|
|
||||||
)
|
|
||||||
|
|
||||||
assert r.status_code == 422, r.text
|
|
||||||
body = r.json()
|
|
||||||
# The 422 body surfaces the failure list so the UI can show details
|
|
||||||
# without parsing a header.
|
|
||||||
errs = body.get("detail", {}).get("serialize_errors") or body.get("serialize_errors")
|
|
||||||
assert errs is not None
|
|
||||||
assert len(errs) == len(claim_ids)
|
|
||||||
for entry in errs:
|
|
||||||
assert entry["claim_id"] in claim_ids
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Error cases
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_empty_claim_ids_returns_400():
|
|
||||||
with TestClient(app) as client:
|
|
||||||
seeded = _seed_batch(client)
|
|
||||||
batch_id = seeded["batch_id"]
|
|
||||||
r = client.post(
|
|
||||||
f"/api/batches/{batch_id}/export-837",
|
|
||||||
json={"claim_ids": []},
|
|
||||||
)
|
|
||||||
assert r.status_code == 400, r.text
|
|
||||||
|
|
||||||
|
|
||||||
def test_missing_claim_ids_key_returns_400():
|
|
||||||
with TestClient(app) as client:
|
|
||||||
seeded = _seed_batch(client)
|
|
||||||
batch_id = seeded["batch_id"]
|
|
||||||
r = client.post(
|
|
||||||
f"/api/batches/{batch_id}/export-837",
|
|
||||||
json={},
|
|
||||||
)
|
|
||||||
assert r.status_code == 400, r.text
|
|
||||||
|
|
||||||
|
|
||||||
def test_unknown_batch_id_returns_404():
|
|
||||||
with TestClient(app) as client:
|
|
||||||
r = client.post(
|
|
||||||
"/api/batches/BATCH-DOES-NOT-EXIST/export-837",
|
|
||||||
json={"claim_ids": ["CLM-1"]},
|
|
||||||
)
|
|
||||||
assert r.status_code == 404, r.text
|
|
||||||
body = r.json()
|
|
||||||
assert "BATCH-DOES-NOT-EXIST" in (body.get("detail") or "")
|
|
||||||
|
|
||||||
|
|
||||||
def test_unknown_claim_id_silently_omitted():
|
|
||||||
"""A claim_id that doesn't exist is omitted from the ZIP and surfaced
|
|
||||||
in the errors header — same convention as the resubmit endpoint."""
|
|
||||||
with TestClient(app) as client:
|
|
||||||
seeded = _seed_batch(client)
|
|
||||||
batch_id = seeded["batch_id"]
|
|
||||||
real_ids = _claim_ids_from_seed(seeded)
|
|
||||||
assert real_ids, "fixture must produce at least 1 claim"
|
|
||||||
|
|
||||||
requested = real_ids + ["CLM-GHOST"]
|
|
||||||
r = client.post(
|
|
||||||
f"/api/batches/{batch_id}/export-837",
|
|
||||||
json={"claim_ids": requested},
|
|
||||||
)
|
|
||||||
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
# Filename uses success count = len(real_ids), not len(requested).
|
|
||||||
assert (
|
|
||||||
f"batch-{batch_id}-{len(real_ids)}-claims.zip"
|
|
||||||
in r.headers["content-disposition"]
|
|
||||||
)
|
|
||||||
err_header = r.headers.get("x-cyclone-serialize-errors")
|
|
||||||
assert err_header
|
|
||||||
errs = json.loads(err_header)
|
|
||||||
assert any(e["claim_id"] == "CLM-GHOST" for e in errs)
|
|
||||||
|
|
||||||
with zipfile.ZipFile(io.BytesIO(r.content)) as zf:
|
|
||||||
names = zf.namelist()
|
|
||||||
# HCPF outbound filenames, one per real claim. We don't pin the
|
|
||||||
# exact ts (it depends on the wall clock at test time), but the
|
|
||||||
# count and the HCPF template must match.
|
|
||||||
from cyclone.edi.filenames import is_outbound_filename
|
|
||||||
assert len(names) == len(real_ids)
|
|
||||||
for name in names:
|
|
||||||
assert is_outbound_filename(name), f"non-HCPF name: {name!r}"
|
|
||||||
@@ -1,29 +0,0 @@
|
|||||||
"""Tests that /api/parse-837 includes the persisted batch_id in its
|
|
||||||
JSON response, so the frontend can correlate its in-memory batch with
|
|
||||||
the server's row (and later call /api/batches/{id}/export-837).
|
|
||||||
|
|
||||||
The 409 (duplicate) path has always included batch_id; the 200 path
|
|
||||||
did not. This brings the 200 path in line so the frontend doesn't have
|
|
||||||
to query listBatches after every parse just to learn the server's id.
|
|
||||||
"""
|
|
||||||
import io
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
from cyclone.api import app
|
|
||||||
|
|
||||||
|
|
||||||
def test_parse_837_happy_path_includes_batch_id_in_response():
|
|
||||||
with TestClient(app) as client:
|
|
||||||
fixture = Path("tests/fixtures/co_medicaid_837p.txt").read_text()
|
|
||||||
r = client.post(
|
|
||||||
"/api/parse-837",
|
|
||||||
files={"file": ("claim.txt", io.BytesIO(fixture.encode()), "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
body = r.json()
|
|
||||||
assert "batch_id" in body, "parse-837 response must include batch_id so the frontend can call /api/batches/{id}/export-837"
|
|
||||||
# Sanity: the batch_id should be a non-empty string (UUID-shaped).
|
|
||||||
assert isinstance(body["batch_id"], str) and body["batch_id"]
|
|
||||||
@@ -1,152 +0,0 @@
|
|||||||
"""SP16 — Admin scheduler API endpoint tests.
|
|
||||||
|
|
||||||
The endpoints under /api/admin/scheduler/* are thin wrappers around
|
|
||||||
:class:`cyclone.scheduler.Scheduler`. These tests exercise them via
|
|
||||||
the FastAPI TestClient to confirm wiring (auth-free admin endpoints
|
|
||||||
work, response shapes match, idempotency holds).
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def _stub_scheduler_env(tmp_path, monkeypatch):
|
|
||||||
"""Set up: a stub-mode SFTP block, scheduler configured.
|
|
||||||
|
|
||||||
Yields (staging_dir, scheduler_singleton). We deliberately do
|
|
||||||
NOT enable SQLCipher encryption in this fixture — the scheduler
|
|
||||||
doesn't care about encryption, and patching ``db_crypto.get_secret``
|
|
||||||
here would cause the lifespan handler to rebuild the engine with
|
|
||||||
SQLCipher on a plain-SQLite test file (which raises "file is not
|
|
||||||
a database"). The encryption-at-rest tests live in
|
|
||||||
``test_db_crypto.py``.
|
|
||||||
"""
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone import scheduler as sched_mod
|
|
||||||
from cyclone.providers import SftpBlock
|
|
||||||
|
|
||||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
|
||||||
db._reset_for_tests()
|
|
||||||
|
|
||||||
staging = tmp_path / "staging"
|
|
||||||
inbound = staging / "ToHPE"
|
|
||||||
inbound.mkdir(parents=True)
|
|
||||||
sftp_block = SftpBlock(
|
|
||||||
host="mft.example.com",
|
|
||||||
port=22,
|
|
||||||
username="test",
|
|
||||||
paths={"outbound": "/FromHPE", "inbound": "/ToHPE"},
|
|
||||||
stub=True,
|
|
||||||
staging_dir=str(staging),
|
|
||||||
poll_seconds=60,
|
|
||||||
auth={"method": "keychain", "secret_ref": "test.password"},
|
|
||||||
)
|
|
||||||
sched_mod.reset_scheduler_for_tests()
|
|
||||||
sched = sched_mod.configure_scheduler(sftp_block, sftp_block_name="t")
|
|
||||||
yield staging, sched
|
|
||||||
sched_mod.reset_scheduler_for_tests()
|
|
||||||
db._reset_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
def _drop_file(staging: Path, name: str, body: bytes) -> Path:
|
|
||||||
p = staging / "ToHPE" / name
|
|
||||||
p.write_bytes(body)
|
|
||||||
return p
|
|
||||||
|
|
||||||
|
|
||||||
def test_scheduler_status_starts_not_running(_stub_scheduler_env):
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
from cyclone.api import app
|
|
||||||
_, sched = _stub_scheduler_env
|
|
||||||
with TestClient(app) as client:
|
|
||||||
r = client.get("/api/admin/scheduler/status")
|
|
||||||
assert r.status_code == 200
|
|
||||||
body = r.json()
|
|
||||||
assert body["running"] is False
|
|
||||||
assert body["poll_interval_seconds"] == 60
|
|
||||||
assert body["sftp_block_name"] == "t"
|
|
||||||
|
|
||||||
|
|
||||||
def test_scheduler_start_then_status_then_stop(_stub_scheduler_env):
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
from cyclone.api import app
|
|
||||||
with TestClient(app) as client:
|
|
||||||
r1 = client.post("/api/admin/scheduler/start")
|
|
||||||
assert r1.status_code == 200
|
|
||||||
assert r1.json()["status"]["running"] is True
|
|
||||||
|
|
||||||
r2 = client.get("/api/admin/scheduler/status")
|
|
||||||
assert r2.json()["running"] is True
|
|
||||||
|
|
||||||
r3 = client.post("/api/admin/scheduler/stop")
|
|
||||||
assert r3.status_code == 200
|
|
||||||
assert r3.json()["status"]["running"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_scheduler_tick_processes_one_file(_stub_scheduler_env):
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
from cyclone.api import app
|
|
||||||
staging, _ = _stub_scheduler_env
|
|
||||||
_drop_file(
|
|
||||||
staging,
|
|
||||||
"TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12",
|
|
||||||
(Path(__file__).parent / "fixtures" / "minimal_ta1.txt").read_bytes(),
|
|
||||||
)
|
|
||||||
with TestClient(app) as client:
|
|
||||||
r = client.post("/api/admin/scheduler/tick")
|
|
||||||
assert r.status_code == 200
|
|
||||||
body = r.json()
|
|
||||||
assert body["ok"] is True
|
|
||||||
assert body["tick"]["files_seen"] == 1
|
|
||||||
assert body["tick"]["files_processed"] == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_scheduler_processed_files_lists_history(_stub_scheduler_env):
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
from cyclone.api import app
|
|
||||||
staging, _ = _stub_scheduler_env
|
|
||||||
_drop_file(
|
|
||||||
staging,
|
|
||||||
"TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12",
|
|
||||||
(Path(__file__).parent / "fixtures" / "minimal_ta1.txt").read_bytes(),
|
|
||||||
)
|
|
||||||
with TestClient(app) as client:
|
|
||||||
client.post("/api/admin/scheduler/tick")
|
|
||||||
r = client.get("/api/admin/scheduler/processed-files")
|
|
||||||
assert r.status_code == 200
|
|
||||||
body = r.json()
|
|
||||||
assert body["count"] == 1
|
|
||||||
f = body["files"][0]
|
|
||||||
assert f["status"] == "ok"
|
|
||||||
assert f["parser_used"] == "parse_ta1"
|
|
||||||
assert "TP11525703" in f["name"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_scheduler_processed_files_filters_by_status(_stub_scheduler_env):
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
from cyclone.api import app
|
|
||||||
staging, _ = _stub_scheduler_env
|
|
||||||
# Drop a file with a type Cyclone doesn't parse — gets recorded as
|
|
||||||
# "skipped".
|
|
||||||
_drop_file(
|
|
||||||
staging,
|
|
||||||
"TP11525703-837P_M019048402-20260618130000000-1of1_270.x12",
|
|
||||||
b"some bytes",
|
|
||||||
)
|
|
||||||
with TestClient(app) as client:
|
|
||||||
client.post("/api/admin/scheduler/tick")
|
|
||||||
r_all = client.get("/api/admin/scheduler/processed-files")
|
|
||||||
r_skipped = client.get(
|
|
||||||
"/api/admin/scheduler/processed-files?status=skipped",
|
|
||||||
)
|
|
||||||
r_ok = client.get(
|
|
||||||
"/api/admin/scheduler/processed-files?status=ok",
|
|
||||||
)
|
|
||||||
assert r_all.json()["count"] == 1
|
|
||||||
assert r_skipped.json()["count"] == 1
|
|
||||||
assert r_ok.json()["count"] == 0
|
|
||||||
@@ -1,69 +0,0 @@
|
|||||||
"""Tests for ``GET /api/admin/validate-provider`` (SP20).
|
|
||||||
|
|
||||||
Pure read-only endpoint — runs the local NPI Luhn + EIN format checks.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
from cyclone.api import app
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def client() -> TestClient:
|
|
||||||
return TestClient(app)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Both fields populated
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_validate_provider_both_valid(client: TestClient):
|
|
||||||
resp = client.get("/api/admin/validate-provider", params={
|
|
||||||
"npi": "1234567893", # CMS-published valid NPI
|
|
||||||
"tax_id": "72-1587149", # Touch of Care EIN
|
|
||||||
})
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.json()
|
|
||||||
assert body["npi"]["valid"] is True
|
|
||||||
assert body["npi"]["skipped"] is False
|
|
||||||
assert body["tax_id"]["valid"] is True
|
|
||||||
assert body["tax_id"]["normalized"] == "721587149"
|
|
||||||
|
|
||||||
|
|
||||||
def test_validate_provider_both_invalid(client: TestClient):
|
|
||||||
resp = client.get("/api/admin/validate-provider", params={
|
|
||||||
"npi": "1234567890", # format OK but Luhn fails
|
|
||||||
"tax_id": "00-1234567", # reserved prefix
|
|
||||||
})
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.json()
|
|
||||||
assert body["npi"]["valid"] is False
|
|
||||||
assert body["tax_id"]["valid"] is False
|
|
||||||
assert body["tax_id"]["normalized"] == "001234567"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Param omission → skipped
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_validate_provider_skips_missing_npi(client: TestClient):
|
|
||||||
resp = client.get("/api/admin/validate-provider", params={"tax_id": "721587149"})
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.json()
|
|
||||||
assert body["npi"]["skipped"] is True
|
|
||||||
assert body["npi"]["valid"] is None
|
|
||||||
assert body["tax_id"]["valid"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_validate_provider_skips_missing_tax_id(client: TestClient):
|
|
||||||
resp = client.get("/api/admin/validate-provider", params={"npi": "1234567893"})
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.json()
|
|
||||||
assert body["tax_id"]["skipped"] is True
|
|
||||||
assert body["tax_id"]["valid"] is None
|
|
||||||
assert body["tax_id"]["normalized"] is None
|
|
||||||
assert body["npi"]["valid"] is True
|
|
||||||
@@ -1,165 +0,0 @@
|
|||||||
"""SP17 — low-level backup crypto tests.
|
|
||||||
|
|
||||||
Pure-Python, no DB. Covers key derivation determinism, encrypt /
|
|
||||||
decrypt round-trip, tampered-ciphertext failure, wrong-passphrase
|
|
||||||
failure, and the sidecar JSON format.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from cyclone import backup as backup_mod
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Key derivation
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_derive_key_is_deterministic():
|
|
||||||
salt = os.urandom(16)
|
|
||||||
k1 = backup_mod.derive_key("correct horse battery staple", salt)
|
|
||||||
k2 = backup_mod.derive_key("correct horse battery staple", salt)
|
|
||||||
assert k1 == k2
|
|
||||||
assert len(k1) == backup_mod.KEY_LEN == 32
|
|
||||||
|
|
||||||
|
|
||||||
def test_derive_key_different_salts_produce_different_keys():
|
|
||||||
"""Salt is what makes the same passphrase produce different keys."""
|
|
||||||
k1 = backup_mod.derive_key("hunter2", os.urandom(16))
|
|
||||||
k2 = backup_mod.derive_key("hunter2", os.urandom(16))
|
|
||||||
assert k1 != k2
|
|
||||||
|
|
||||||
|
|
||||||
def test_derive_key_different_passphrases_produce_different_keys():
|
|
||||||
salt = os.urandom(16)
|
|
||||||
k1 = backup_mod.derive_key("a", salt)
|
|
||||||
k2 = backup_mod.derive_key("b", salt)
|
|
||||||
assert k1 != k2
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Encrypt / decrypt round-trip
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_encrypt_decrypt_roundtrip():
|
|
||||||
key = os.urandom(32)
|
|
||||||
plaintext = b"hello cyclone backup " * 1000
|
|
||||||
blob = backup_mod.encrypt(plaintext, key)
|
|
||||||
assert len(blob) == backup_mod.NONCE_LEN + len(plaintext) + 16 # tag
|
|
||||||
out = backup_mod.decrypt(blob, key)
|
|
||||||
assert out == plaintext
|
|
||||||
|
|
||||||
|
|
||||||
def test_encrypt_decrypt_empty_plaintext():
|
|
||||||
"""Edge case: zero-byte payload still produces nonce + tag."""
|
|
||||||
key = os.urandom(32)
|
|
||||||
blob = backup_mod.encrypt(b"", key)
|
|
||||||
out = backup_mod.decrypt(blob, key)
|
|
||||||
assert out == b""
|
|
||||||
|
|
||||||
|
|
||||||
def test_decrypt_with_wrong_key_raises():
|
|
||||||
plaintext = b"some bytes"
|
|
||||||
key1 = os.urandom(32)
|
|
||||||
key2 = os.urandom(32)
|
|
||||||
blob = backup_mod.encrypt(plaintext, key1)
|
|
||||||
with pytest.raises(backup_mod.BackupDecryptError):
|
|
||||||
backup_mod.decrypt(blob, key2)
|
|
||||||
|
|
||||||
|
|
||||||
def test_decrypt_tampered_ciphertext_raises():
|
|
||||||
"""Flipping a single ciphertext byte must fail GCM auth."""
|
|
||||||
key = os.urandom(32)
|
|
||||||
blob = backup_mod.encrypt(b"a" * 200, key)
|
|
||||||
tampered = bytearray(blob)
|
|
||||||
# Flip a bit somewhere in the ciphertext region (past the nonce).
|
|
||||||
tampered[backup_mod.NONCE_LEN + 5] ^= 0x01
|
|
||||||
with pytest.raises(backup_mod.BackupDecryptError):
|
|
||||||
backup_mod.decrypt(bytes(tampered), key)
|
|
||||||
|
|
||||||
|
|
||||||
def test_decrypt_truncated_blob_raises():
|
|
||||||
key = os.urandom(32)
|
|
||||||
blob = backup_mod.encrypt(b"x" * 100, key)
|
|
||||||
with pytest.raises(backup_mod.BackupDecryptError):
|
|
||||||
# Strip the GCM tag.
|
|
||||||
backup_mod.decrypt(blob[: -16], key)
|
|
||||||
|
|
||||||
|
|
||||||
def test_encrypt_with_wrong_key_length_raises():
|
|
||||||
with pytest.raises(backup_mod.BackupError):
|
|
||||||
backup_mod.encrypt(b"data", b"short") # not 32 bytes
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Fingerprint
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_fingerprint_format_and_stability():
|
|
||||||
fp = backup_mod.fingerprint(b"hello")
|
|
||||||
assert fp.startswith("sha256:")
|
|
||||||
assert len(fp) == len("sha256:") + 64
|
|
||||||
assert fp == backup_mod.fingerprint(b"hello")
|
|
||||||
assert fp != backup_mod.fingerprint(b"hellp")
|
|
||||||
|
|
||||||
|
|
||||||
def test_fingerprint_file_matches_fingerprint_bytes(tmp_path):
|
|
||||||
p = tmp_path / "data.bin"
|
|
||||||
p.write_bytes(b"\x00\x01\x02" * 100)
|
|
||||||
assert backup_mod.fingerprint_file(p) == backup_mod.fingerprint(p.read_bytes())
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Sidecar
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_sidecar_round_trip_json():
|
|
||||||
sc = backup_mod.Sidecar(
|
|
||||||
format_version="v1",
|
|
||||||
created_at="2026-06-21T15:30:00+00:00",
|
|
||||||
db_fingerprint="sha256:" + "a" * 64,
|
|
||||||
table_count=11,
|
|
||||||
size_bytes=1024,
|
|
||||||
kdf="PBKDF2-HMAC-SHA256",
|
|
||||||
kdf_iterations=200_000,
|
|
||||||
cipher="AES-256-GCM",
|
|
||||||
key_fingerprint="sha256:" + "b" * 64,
|
|
||||||
)
|
|
||||||
text = sc.to_json()
|
|
||||||
parsed = json.loads(text)
|
|
||||||
assert parsed["format_version"] == "v1"
|
|
||||||
assert parsed["encryption"]["kdf_iterations"] == 200_000
|
|
||||||
sc2 = backup_mod.Sidecar.from_json(text)
|
|
||||||
assert sc2 == sc
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Filenames
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_backup_filename_format():
|
|
||||||
"""The timestamp prefix is fixed; the suffix is random per call."""
|
|
||||||
import re
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
ts = datetime(2026, 6, 21, 15, 30, 0, tzinfo=timezone.utc)
|
|
||||||
name = backup_mod.backup_filename(ts)
|
|
||||||
assert re.match(r"^cyclone-backup-20260621T153000Z-[0-9a-f]{8}\.bin$", name), name
|
|
||||||
|
|
||||||
|
|
||||||
def test_backup_filename_random_suffix_avoids_collisions():
|
|
||||||
"""Two calls in the same second get different filenames."""
|
|
||||||
a = backup_mod.backup_filename()
|
|
||||||
b = backup_mod.backup_filename()
|
|
||||||
assert a != b
|
|
||||||
|
|
||||||
|
|
||||||
def test_sidecar_filename_appends_meta_json():
|
|
||||||
assert backup_mod.sidecar_filename("foo.bin") == "foo.bin.meta.json"
|
|
||||||
@@ -1,207 +0,0 @@
|
|||||||
"""SP17 — BackupScheduler unit tests.
|
|
||||||
|
|
||||||
Exercises the asyncio tick / start / stop loop without spinning up
|
|
||||||
the FastAPI app. The scheduler wraps a real BackupService against
|
|
||||||
a real on-disk sqlite DB.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import backup_scheduler as sched_mod
|
|
||||||
from cyclone import db
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def fresh_db(tmp_path, monkeypatch):
|
|
||||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
|
||||||
db._reset_for_tests()
|
|
||||||
db.init_db()
|
|
||||||
from cyclone.db import Batch
|
|
||||||
import uuid
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
s.add(Batch(
|
|
||||||
id=str(uuid.uuid4()),
|
|
||||||
kind="837P",
|
|
||||||
input_filename="seed.x12",
|
|
||||||
parsed_at=datetime.now(timezone.utc),
|
|
||||||
totals_json=None,
|
|
||||||
validation_json=None,
|
|
||||||
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
yield
|
|
||||||
db._reset_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def backup_svc(tmp_path):
|
|
||||||
return svc_mod.BackupService(
|
|
||||||
backup_dir=tmp_path / "backups",
|
|
||||||
passphrase="test-pass",
|
|
||||||
retention_days=7,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# tick
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_tick_creates_backup_and_audits_it(fresh_db, backup_svc):
|
|
||||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
|
||||||
result = await sched.tick()
|
|
||||||
assert result.ok
|
|
||||||
assert result.created is not None
|
|
||||||
assert result.error is None
|
|
||||||
assert len(backup_svc.list_backups()) == 1
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_tick_creates_audit_event(fresh_db, backup_svc):
|
|
||||||
"""db.backup_created audit event is written (SP11 hash chain)."""
|
|
||||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
|
||||||
await sched.tick()
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
from cyclone.db import AuditLog
|
|
||||||
rows = (
|
|
||||||
s.query(AuditLog)
|
|
||||||
.filter(AuditLog.event_type == "db.backup_created")
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
assert len(rows) == 1
|
|
||||||
assert "backup_id" in rows[0].payload_json
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_tick_handles_create_failure_without_crashing(fresh_db, backup_svc, monkeypatch):
|
|
||||||
"""If create_now raises, tick records the error and continues."""
|
|
||||||
def boom():
|
|
||||||
raise RuntimeError("simulated failure")
|
|
||||||
monkeypatch.setattr(backup_svc, "create_now", boom)
|
|
||||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
|
||||||
result = await sched.tick()
|
|
||||||
assert result.error is not None
|
|
||||||
assert "simulated failure" in result.error
|
|
||||||
# Audit event written for the failure.
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
from cyclone.db import AuditLog
|
|
||||||
rows = (
|
|
||||||
s.query(AuditLog)
|
|
||||||
.filter(AuditLog.event_type == "db.backup_failed")
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
assert len(rows) == 1
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_tick_prunes_old_backups_and_audits(fresh_db, backup_svc):
|
|
||||||
"""A tick prunes backups past retention and writes a db.backup_pruned event."""
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
|
|
||||||
# Take an initial backup.
|
|
||||||
initial = backup_svc.create_now()
|
|
||||||
# Age it past retention.
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.get(DbBackup, initial.backup.id)
|
|
||||||
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
|
||||||
result = await sched.tick()
|
|
||||||
assert result.ok # create_now succeeded even though prune removed old
|
|
||||||
assert len(result.pruned_paths) == 2 # .bin + .meta.json
|
|
||||||
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
from cyclone.db import AuditLog
|
|
||||||
pruned_events = (
|
|
||||||
s.query(AuditLog)
|
|
||||||
.filter(AuditLog.event_type == "db.backup_pruned")
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
assert len(pruned_events) == 1
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# start / stop / is_running
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_start_then_stop(fresh_db, backup_svc):
|
|
||||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
|
||||||
assert not sched.is_running()
|
|
||||||
await sched.start()
|
|
||||||
assert sched.is_running()
|
|
||||||
# Don't wait for the staggered first tick; just stop.
|
|
||||||
await sched.stop()
|
|
||||||
assert not sched.is_running()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_double_start_is_idempotent(fresh_db, backup_svc):
|
|
||||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
|
||||||
await sched.start()
|
|
||||||
await sched.start() # no-op
|
|
||||||
assert sched.is_running()
|
|
||||||
await sched.stop()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_concurrent_ticks_are_coalesced(fresh_db, backup_svc):
|
|
||||||
"""Two tick() calls in flight — second waits for first."""
|
|
||||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
|
||||||
r1, r2 = await asyncio.gather(sched.tick(), sched.tick())
|
|
||||||
# Both should succeed and produce a single backup (the second
|
|
||||||
# call returned the first call's result, or ran back-to-back
|
|
||||||
# and produced a second backup — both are valid coalescings).
|
|
||||||
assert r1 is not None
|
|
||||||
assert r2 is not None
|
|
||||||
# No matter the order, exactly 1 backup should exist OR 2 if they
|
|
||||||
# ran sequentially. The point of coalescing is no-overlap, so
|
|
||||||
# both should be ok=True.
|
|
||||||
assert r1.ok
|
|
||||||
assert r2.ok
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# status
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_status_snapshot(fresh_db, backup_svc):
|
|
||||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=12.0)
|
|
||||||
snap = sched.status()
|
|
||||||
assert snap.running is False
|
|
||||||
assert snap.interval_hours == 12.0
|
|
||||||
assert snap.backup_dir == str(backup_svc.backup_dir)
|
|
||||||
assert snap.retention_days == 7
|
|
||||||
assert snap.tick_count == 0
|
|
||||||
assert snap.last_tick is None
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Module-level singleton
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_module_singleton_round_trip(fresh_db, tmp_path):
|
|
||||||
sched_mod.reset_backup_scheduler_for_tests()
|
|
||||||
svc = svc_mod.BackupService(tmp_path / "b", passphrase="x", retention_days=1)
|
|
||||||
sched = sched_mod.configure_backup_scheduler(svc, interval_hours=1)
|
|
||||||
assert sched_mod.get_backup_scheduler() is sched
|
|
||||||
# Second configure is a no-op.
|
|
||||||
assert sched_mod.configure_backup_scheduler(svc) is sched
|
|
||||||
sched_mod.reset_backup_scheduler_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
def test_module_singleton_get_raises_when_unset():
|
|
||||||
sched_mod.reset_backup_scheduler_for_tests()
|
|
||||||
with pytest.raises(RuntimeError):
|
|
||||||
sched_mod.get_backup_scheduler()
|
|
||||||
@@ -1,400 +0,0 @@
|
|||||||
"""SP17 — BackupService integration tests.
|
|
||||||
|
|
||||||
Exercises the full create / list / verify / restore / prune flow
|
|
||||||
against a real on-disk SQLite file (no SQLCipher, no Keychain). We
|
|
||||||
inject the passphrase directly into the BackupService constructor.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from cyclone import backup as backup_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.backup import BackupError
|
|
||||||
from cyclone.backup_service import (
|
|
||||||
BackupService,
|
|
||||||
STATUS_ERROR,
|
|
||||||
STATUS_OK,
|
|
||||||
STATUS_PENDING,
|
|
||||||
STATUS_PRUNED,
|
|
||||||
configure_backup_service,
|
|
||||||
get_backup_service,
|
|
||||||
reset_backup_service_for_tests,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Fixtures
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def fresh_db(tmp_path, monkeypatch):
|
|
||||||
"""Fresh sqlite DB; init_db + create tables; yield the path."""
|
|
||||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
|
||||||
db._reset_for_tests()
|
|
||||||
db.init_db()
|
|
||||||
yield tmp_path / "test.db"
|
|
||||||
db._reset_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def backup_svc(fresh_db, tmp_path):
|
|
||||||
"""A BackupService rooted in a temp backup directory."""
|
|
||||||
backup_dir = tmp_path / "backups"
|
|
||||||
return BackupService(
|
|
||||||
backup_dir=backup_dir,
|
|
||||||
passphrase="test-passphrase-123",
|
|
||||||
retention_days=7,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _make_a_row(s: "sa.orm.Session") -> None:
|
|
||||||
"""Insert one minimal Batch row so the DB has a real schema + content.
|
|
||||||
|
|
||||||
Bypasses the Claim model (which has many NOT NULL columns tied to
|
|
||||||
BatchRecord lifecycle) and just writes a Batch directly — the
|
|
||||||
backup flow doesn't care which tables exist, only that there
|
|
||||||
are some.
|
|
||||||
"""
|
|
||||||
from cyclone.db import Batch
|
|
||||||
import uuid
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from decimal import Decimal
|
|
||||||
s.add(Batch(
|
|
||||||
id=str(uuid.uuid4()),
|
|
||||||
kind="837P",
|
|
||||||
input_filename="test.x12",
|
|
||||||
parsed_at=datetime.now(timezone.utc),
|
|
||||||
totals_json=None,
|
|
||||||
validation_json=None,
|
|
||||||
raw_result_json={
|
|
||||||
"envelope": {"control_number": "1"},
|
|
||||||
"claims": [],
|
|
||||||
"summary": {"passed": 0, "failed": 0, "failed_claim_ids": []},
|
|
||||||
},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# create_now
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_create_now_writes_encrypted_blob_and_sidecar(fresh_db, backup_svc):
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
# Add a claim so the DB has content + table_count > 0.
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
|
|
||||||
result = backup_svc.create_now()
|
|
||||||
record = result.backup
|
|
||||||
sidecar = result.sidecar
|
|
||||||
|
|
||||||
assert record.status == STATUS_OK
|
|
||||||
assert record.size_bytes > 0
|
|
||||||
assert record.db_fingerprint.startswith("sha256:")
|
|
||||||
assert record.table_count >= 1
|
|
||||||
assert record.completed_at is not None
|
|
||||||
|
|
||||||
# The .bin file exists, is non-trivial size, and does NOT look
|
|
||||||
# like a SQLite header (which is the whole point of encryption).
|
|
||||||
bin_path = backup_svc.backup_dir / record.filename
|
|
||||||
assert bin_path.exists()
|
|
||||||
blob = bin_path.read_bytes()
|
|
||||||
assert blob[:6] != b"SQLite" # not a plaintext SQLite file
|
|
||||||
|
|
||||||
# Sidecar exists and round-trips.
|
|
||||||
meta_path = backup_svc.backup_dir / backup_mod.sidecar_filename(record.filename)
|
|
||||||
assert meta_path.exists()
|
|
||||||
parsed = backup_mod.Sidecar.from_json(meta_path.read_text())
|
|
||||||
assert parsed.db_fingerprint == record.db_fingerprint
|
|
||||||
assert parsed.table_count == record.table_count
|
|
||||||
|
|
||||||
|
|
||||||
def test_create_now_marks_error_on_db_failure(fresh_db, tmp_path, monkeypatch):
|
|
||||||
"""If SQLite .backup() raises, the row is marked error + files cleaned."""
|
|
||||||
backup_dir = tmp_path / "backups"
|
|
||||||
svc = BackupService(backup_dir=backup_dir, passphrase="x", retention_days=7)
|
|
||||||
|
|
||||||
# Force the .backup() call to fail by patching sqlite3.connect to raise.
|
|
||||||
import sqlite3 as _sqlite3
|
|
||||||
real_connect = _sqlite3.connect
|
|
||||||
def boom(path):
|
|
||||||
raise RuntimeError("simulated disk failure")
|
|
||||||
monkeypatch.setattr(_sqlite3, "connect", boom)
|
|
||||||
# But we also need to make sure engine.raw_connection().driver_connection
|
|
||||||
# is reachable — it's still using real_connect via the engine's
|
|
||||||
# internals. So patch at the higher level: the BackupService's
|
|
||||||
# _sqlite_backup_to.
|
|
||||||
monkeypatch.setattr(svc, "_sqlite_backup_to",
|
|
||||||
lambda p: (_ for _ in ()).throw(RuntimeError("boom")))
|
|
||||||
|
|
||||||
with pytest.raises(RuntimeError, match="boom"):
|
|
||||||
svc.create_now()
|
|
||||||
|
|
||||||
rows = svc.list_backups()
|
|
||||||
assert len(rows) == 1
|
|
||||||
assert rows[0].status == STATUS_ERROR
|
|
||||||
assert "boom" in rows[0].error_message
|
|
||||||
# No files left in the backup dir.
|
|
||||||
assert list(backup_dir.iterdir()) == []
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# list_backups
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_list_backups_orders_newest_first(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
r1 = backup_svc.create_now()
|
|
||||||
r2 = backup_svc.create_now()
|
|
||||||
rows = backup_svc.list_backups()
|
|
||||||
assert [r.id for r in rows] == [r2.backup.id, r1.backup.id]
|
|
||||||
|
|
||||||
|
|
||||||
def test_list_backups_filter_by_status(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
backup_svc.create_now()
|
|
||||||
rows = backup_svc.list_backups(status=STATUS_OK)
|
|
||||||
assert all(r.status == STATUS_OK for r in rows)
|
|
||||||
rows = backup_svc.list_backups(status=STATUS_PENDING)
|
|
||||||
assert rows == []
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# verify
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_verify_ok_after_create(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
r = backup_svc.create_now()
|
|
||||||
v = backup_svc.verify(r.backup.id)
|
|
||||||
assert v.ok
|
|
||||||
assert v.expected_fingerprint == v.actual_fingerprint
|
|
||||||
assert v.table_count >= 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_verify_detects_tampered_ciphertext(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
r = backup_svc.create_now()
|
|
||||||
bin_path = backup_svc.backup_dir / r.backup.filename
|
|
||||||
# Flip a bit in the middle of the encrypted blob.
|
|
||||||
data = bytearray(bin_path.read_bytes())
|
|
||||||
idx = backup_mod.NONCE_LEN + 5
|
|
||||||
data[idx] ^= 0x01
|
|
||||||
bin_path.write_bytes(bytes(data))
|
|
||||||
|
|
||||||
v = backup_svc.verify(r.backup.id)
|
|
||||||
assert not v.ok
|
|
||||||
assert "decryption failed" in (v.reason or "")
|
|
||||||
|
|
||||||
|
|
||||||
def test_verify_handles_missing_file(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
r = backup_svc.create_now()
|
|
||||||
(backup_svc.backup_dir / r.backup.filename).unlink()
|
|
||||||
v = backup_svc.verify(r.backup.id)
|
|
||||||
assert not v.ok
|
|
||||||
assert "missing" in (v.reason or "")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# restore — two-step
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_restore_two_step_round_trip(fresh_db, backup_svc, tmp_path):
|
|
||||||
"""Create a backup, mutate the live DB, restore, confirm mutation gone."""
|
|
||||||
from cyclone.db import Batch
|
|
||||||
import uuid
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
|
|
||||||
# 1. Backup a DB with one Batch row.
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
snap = backup_svc.create_now()
|
|
||||||
|
|
||||||
# 2. Mutate the live DB (add another Batch row).
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
s.add(Batch(
|
|
||||||
id=str(uuid.uuid4()),
|
|
||||||
kind="837P",
|
|
||||||
input_filename="mutated.x12",
|
|
||||||
parsed_at=datetime.now(timezone.utc),
|
|
||||||
totals_json=None,
|
|
||||||
validation_json=None,
|
|
||||||
raw_result_json={"envelope": {"control_number": "2"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
assert s.query(Batch).count() == 2
|
|
||||||
|
|
||||||
# 3. Initiate restore.
|
|
||||||
init = backup_svc.restore_initiate(snap.backup.id)
|
|
||||||
assert init.table_count >= 1
|
|
||||||
assert init.current_db_fingerprint != init.db_fingerprint # live != backup now
|
|
||||||
assert init.restore_token and len(init.restore_token) == 64
|
|
||||||
|
|
||||||
# 4. Confirm restore.
|
|
||||||
result = backup_svc.restore_confirm(snap.backup.id, init.restore_token)
|
|
||||||
assert result.new_db_fingerprint == init.db_fingerprint
|
|
||||||
|
|
||||||
# 5. The live DB now reflects the snapshot (1 row, not 2).
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
assert s.query(Batch).count() == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_restore_initiate_rejects_non_ok_backup(fresh_db, backup_svc, tmp_path, monkeypatch):
|
|
||||||
"""A backup row with status='error' cannot be restored."""
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
r = backup_svc.create_now()
|
|
||||||
# Force the row to error.
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
row = session.get(DbBackup, r.backup.id)
|
|
||||||
row.status = STATUS_ERROR
|
|
||||||
row.error_message = "simulated"
|
|
||||||
session.commit()
|
|
||||||
with pytest.raises(BackupError, match="only 'ok' backups"):
|
|
||||||
backup_svc.restore_initiate(r.backup.id)
|
|
||||||
|
|
||||||
|
|
||||||
def test_restore_confirm_rejects_wrong_token(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
r = backup_svc.create_now()
|
|
||||||
init = backup_svc.restore_initiate(r.backup.id)
|
|
||||||
with pytest.raises(BackupError, match="not found"):
|
|
||||||
backup_svc.restore_confirm(r.backup.id, "0" * 64)
|
|
||||||
|
|
||||||
|
|
||||||
def test_restore_confirm_rejects_expired_token(fresh_db, backup_svc, monkeypatch):
|
|
||||||
"""A token whose expires_at is in the past is rejected."""
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
r = backup_svc.create_now()
|
|
||||||
init = backup_svc.restore_initiate(r.backup.id)
|
|
||||||
# Manually age the token past its expiry.
|
|
||||||
with backup_svc._lock:
|
|
||||||
backup_svc._pending_restores[init.restore_token] = (
|
|
||||||
init.backup_id,
|
|
||||||
datetime.now(timezone.utc) - timedelta(seconds=1),
|
|
||||||
)
|
|
||||||
with pytest.raises(BackupError, match="expired"):
|
|
||||||
backup_svc.restore_confirm(r.backup.id, init.restore_token)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# prune
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_prune_deletes_files_and_marks_status(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
r1 = backup_svc.create_now()
|
|
||||||
# The retention cutoff is 7 days from now. Move the row's created_at
|
|
||||||
# back 30 days so it's definitely past retention.
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
from cyclone.db import DbBackup
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
row = session.get(DbBackup, r1.backup.id)
|
|
||||||
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
|
||||||
session.commit()
|
|
||||||
|
|
||||||
deleted = backup_svc.prune()
|
|
||||||
assert len(deleted) == 2 # .bin + .meta.json
|
|
||||||
rows = backup_svc.list_backups()
|
|
||||||
assert rows[0].status == STATUS_PRUNED
|
|
||||||
|
|
||||||
|
|
||||||
def test_prune_keeps_recent_backups(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
backup_svc.create_now()
|
|
||||||
deleted = backup_svc.prune()
|
|
||||||
assert deleted == []
|
|
||||||
rows = backup_svc.list_backups()
|
|
||||||
assert rows[0].status == STATUS_OK
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# status
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_status_reports_counts(fresh_db, backup_svc):
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
_make_a_row(s)
|
|
||||||
backup_svc.create_now()
|
|
||||||
snap = backup_svc.status()
|
|
||||||
assert snap["totals"]["ok"] == 1
|
|
||||||
assert snap["totals"]["all"] == 1
|
|
||||||
assert snap["backup_dir"] == str(backup_svc.backup_dir)
|
|
||||||
assert snap["retention_days"] == 7
|
|
||||||
assert snap["used_fallback_key"] is False
|
|
||||||
assert snap["last_backup_at"] is not None
|
|
||||||
assert snap["last_ok_backup_at"] is not None
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Fallback key
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_fallback_key_used_when_no_passphrase(fresh_db, tmp_path):
|
|
||||||
"""If no passphrase AND no SQLCipher, refuse. Otherwise fallback + warn."""
|
|
||||||
backup_dir = tmp_path / "backups"
|
|
||||||
svc = BackupService(backup_dir=backup_dir, passphrase=None, retention_days=7)
|
|
||||||
# No SQLCipher key either → BackupError.
|
|
||||||
with pytest.raises(BackupError, match="no backup passphrase"):
|
|
||||||
svc._ensure_key()
|
|
||||||
|
|
||||||
|
|
||||||
def test_key_fingerprint_changes_per_passphrase(fresh_db, tmp_path):
|
|
||||||
"""Two services with different passphrases have different key fingerprints."""
|
|
||||||
s1 = BackupService(tmp_path / "b1", passphrase="alpha", retention_days=1)
|
|
||||||
s2 = BackupService(tmp_path / "b2", passphrase="beta", retention_days=1)
|
|
||||||
# Force key derivation.
|
|
||||||
s1._ensure_key()
|
|
||||||
s2._ensure_key()
|
|
||||||
assert s1.key_fingerprint != s2.key_fingerprint
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Module-level singleton
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_module_singleton_round_trip(fresh_db, tmp_path):
|
|
||||||
reset_backup_service_for_tests()
|
|
||||||
svc = configure_backup_service(
|
|
||||||
tmp_path / "backups", passphrase="x", retention_days=1,
|
|
||||||
)
|
|
||||||
assert get_backup_service() is svc
|
|
||||||
# Second configure is a no-op (returns existing).
|
|
||||||
assert configure_backup_service(
|
|
||||||
tmp_path / "backups2", passphrase="y", retention_days=2,
|
|
||||||
) is svc
|
|
||||||
reset_backup_service_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
def test_module_singleton_get_raises_when_unset():
|
|
||||||
reset_backup_service_for_tests()
|
|
||||||
with pytest.raises(RuntimeError):
|
|
||||||
get_backup_service()
|
|
||||||
@@ -1,144 +0,0 @@
|
|||||||
"""SP17 — `cyclone backup` CLI subcommand tests.
|
|
||||||
|
|
||||||
Uses Click's CliRunner + monkeypatching of Keychain + DB env so the
|
|
||||||
subcommands can run without the operator's machine state.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from click.testing import CliRunner
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def _cli_env(tmp_path, monkeypatch):
|
|
||||||
"""Fresh sqlite DB + in-memory Keychain stub."""
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
from cyclone.db import Batch
|
|
||||||
import uuid
|
|
||||||
|
|
||||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
|
||||||
db._reset_for_tests()
|
|
||||||
db.init_db()
|
|
||||||
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
s.add(Batch(
|
|
||||||
id=str(uuid.uuid4()),
|
|
||||||
kind="837P",
|
|
||||||
input_filename="seed.x12",
|
|
||||||
parsed_at=datetime.now(timezone.utc),
|
|
||||||
totals_json=None,
|
|
||||||
validation_json=None,
|
|
||||||
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
|
||||||
))
|
|
||||||
s.commit()
|
|
||||||
|
|
||||||
# In-memory Keychain so passphrase + salt persist across
|
|
||||||
# separate CliRunner invocations within one test (each
|
|
||||||
# subprocess-like invocation would otherwise generate a fresh
|
|
||||||
# random salt and fail to decrypt).
|
|
||||||
store: dict[str, str] = {}
|
|
||||||
store[svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT] = "cli-test-passphrase"
|
|
||||||
# Pre-populate a stable salt so the very first invocation
|
|
||||||
# doesn't generate a new random one (which the next invocation
|
|
||||||
# would then fail to reproduce).
|
|
||||||
store[svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT] = "0123456789abcdef0123456789abcdef"
|
|
||||||
|
|
||||||
def _get(name):
|
|
||||||
return store.get(name)
|
|
||||||
def _set(name, value):
|
|
||||||
store[name] = value
|
|
||||||
return True
|
|
||||||
|
|
||||||
monkeypatch.setattr(secrets_mod, "get_secret", _get)
|
|
||||||
monkeypatch.setattr(secrets_mod, "set_secret", _set)
|
|
||||||
|
|
||||||
backup_dir = tmp_path / "backups"
|
|
||||||
monkeypatch.setenv("CYCLONE_BACKUP_DIR", str(backup_dir))
|
|
||||||
monkeypatch.setenv("CYCLONE_BACKUP_RETENTION_DAYS", "7")
|
|
||||||
|
|
||||||
yield backup_dir
|
|
||||||
db._reset_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
def _run(args, env):
|
|
||||||
from cyclone.cli import main
|
|
||||||
runner = CliRunner()
|
|
||||||
return runner.invoke(main, args, catch_exceptions=False)
|
|
||||||
|
|
||||||
|
|
||||||
def test_backup_create_list_verify_status(_cli_env):
|
|
||||||
"""Happy path: create → list → verify → status."""
|
|
||||||
backup_dir = _cli_env
|
|
||||||
|
|
||||||
# create
|
|
||||||
r = _run(["backup", "create"], _cli_env)
|
|
||||||
assert r.exit_code == 0, r.output
|
|
||||||
assert "created backup id=" in r.output
|
|
||||||
|
|
||||||
# list
|
|
||||||
r = _run(["backup", "list"], _cli_env)
|
|
||||||
assert r.exit_code == 0, r.output
|
|
||||||
assert ".bin" in r.output
|
|
||||||
|
|
||||||
# verify (we don't know the id, parse it from the list output)
|
|
||||||
import re
|
|
||||||
m = re.search(r"^\s*(\d+)\s+ok\s+", r.output, re.MULTILINE)
|
|
||||||
assert m, r.output
|
|
||||||
backup_id = int(m.group(1))
|
|
||||||
r = _run(["backup", "verify", str(backup_id)], _cli_env)
|
|
||||||
assert r.exit_code == 0, r.output
|
|
||||||
assert r.output.startswith("OK:")
|
|
||||||
|
|
||||||
# status
|
|
||||||
r = _run(["backup", "status"], _cli_env)
|
|
||||||
assert r.exit_code == 0, r.output
|
|
||||||
assert '"totals"' in r.output
|
|
||||||
assert '"ok": 1' in r.output
|
|
||||||
|
|
||||||
|
|
||||||
def test_backup_verify_fails_on_tampered_ciphertext(_cli_env):
|
|
||||||
from cyclone import backup as backup_mod
|
|
||||||
from cyclone import backup_service as svc_mod
|
|
||||||
from cyclone import secrets as secrets_mod
|
|
||||||
|
|
||||||
# Create a backup.
|
|
||||||
r = _run(["backup", "create"], _cli_env)
|
|
||||||
assert r.exit_code == 0
|
|
||||||
# Tamper.
|
|
||||||
bin_path = next(_cli_env.glob("*.bin"))
|
|
||||||
data = bytearray(bin_path.read_bytes())
|
|
||||||
data[backup_mod.NONCE_LEN + 5] ^= 0x01
|
|
||||||
bin_path.write_bytes(bytes(data))
|
|
||||||
|
|
||||||
# Verify should fail.
|
|
||||||
r = _run(["backup", "verify", "1"], _cli_env)
|
|
||||||
assert r.exit_code == 1
|
|
||||||
assert "FAIL" in r.output
|
|
||||||
|
|
||||||
|
|
||||||
def test_backup_restore_requires_yes_flag(_cli_env):
|
|
||||||
"""Without --yes, an interactive confirm blocks and the command aborts."""
|
|
||||||
r = _run(["backup", "create"], _cli_env)
|
|
||||||
assert r.exit_code == 0
|
|
||||||
# Click's runner auto-declines the confirm prompt; expect abort.
|
|
||||||
r = _run(["backup", "restore", "1"], _cli_env, )
|
|
||||||
# CliRunner auto-aborts confirm prompts by default → exit code != 0.
|
|
||||||
assert r.exit_code != 0
|
|
||||||
|
|
||||||
|
|
||||||
def test_backup_prune_aborts_without_yes(_cli_env):
|
|
||||||
r = _run(["backup", "create"], _cli_env)
|
|
||||||
assert r.exit_code == 0
|
|
||||||
# Same auto-abort for the prune confirm.
|
|
||||||
r = _run(["backup", "prune"], _cli_env)
|
|
||||||
assert r.exit_code != 0
|
|
||||||
|
|
||||||
|
|
||||||
def test_backup_init_passphrase_rejects_short(_cli_env):
|
|
||||||
"""init-passphrase enforces a 12-char minimum."""
|
|
||||||
r = _run(["backup", "init-passphrase", "--passphrase", "short"], _cli_env)
|
|
||||||
assert r.exit_code != 0
|
|
||||||
assert "12 characters" in r.output
|
|
||||||
@@ -1,80 +0,0 @@
|
|||||||
"""Tests for ``cyclone validate-npi`` + ``cyclone validate-tax-id`` (SP20).
|
|
||||||
|
|
||||||
CLI smoke tests — verify exit codes (0 = valid, 1 = invalid) and that
|
|
||||||
the help text references the new subcommands. We don't pipe the value
|
|
||||||
into shared logs (NPI / EIN are PHI / PII); the CliRunner captures it.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from click.testing import CliRunner
|
|
||||||
|
|
||||||
from cyclone.cli import main
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# validate-npi
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_validate_npi_valid_exits_zero():
|
|
||||||
runner = CliRunner()
|
|
||||||
result = runner.invoke(main, ["validate-npi", "1234567893"])
|
|
||||||
assert result.exit_code == 0, result.output
|
|
||||||
assert "OK" in result.output
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_validate_npi_bad_checksum_exits_one():
|
|
||||||
runner = CliRunner()
|
|
||||||
result = runner.invoke(main, ["validate-npi", "1234567890"])
|
|
||||||
assert result.exit_code == 1
|
|
||||||
assert "INVALID" in result.output
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_validate_npi_wrong_length_exits_one():
|
|
||||||
runner = CliRunner()
|
|
||||||
result = runner.invoke(main, ["validate-npi", "12345"])
|
|
||||||
assert result.exit_code == 1
|
|
||||||
assert "INVALID" in result.output
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# validate-tax-id
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_validate_tax_id_formatted_exits_zero():
|
|
||||||
runner = CliRunner()
|
|
||||||
result = runner.invoke(main, ["validate-tax-id", "72-1587149"])
|
|
||||||
assert result.exit_code == 0, result.output
|
|
||||||
assert "721587149" in result.output # normalized form echoed
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_validate_tax_id_unformatted_exits_zero():
|
|
||||||
runner = CliRunner()
|
|
||||||
result = runner.invoke(main, ["validate-tax-id", "721587149"])
|
|
||||||
assert result.exit_code == 0
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_validate_tax_id_reserved_prefix_exits_one():
|
|
||||||
runner = CliRunner()
|
|
||||||
result = runner.invoke(main, ["validate-tax-id", "00-1234567"])
|
|
||||||
assert result.exit_code == 1
|
|
||||||
assert "reserved" in result.output.lower()
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_validate_tax_id_malformed_exits_one():
|
|
||||||
runner = CliRunner()
|
|
||||||
result = runner.invoke(main, ["validate-tax-id", "not-an-ein"])
|
|
||||||
assert result.exit_code == 1
|
|
||||||
assert "9-digit" in result.output
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_validate_subcommands_appear_in_help():
|
|
||||||
"""The two new subcommands are wired into ``main`` (regression guard
|
|
||||||
against future refactors that drop the imports)."""
|
|
||||||
runner = CliRunner()
|
|
||||||
result = runner.invoke(main, ["--help"])
|
|
||||||
assert result.exit_code == 0
|
|
||||||
assert "validate-npi" in result.output
|
|
||||||
assert "validate-tax-id" in result.output
|
|
||||||
@@ -27,8 +27,7 @@ MT = ZoneInfo("America/Denver")
|
|||||||
def test_build_outbound_with_explicit_mt():
|
def test_build_outbound_with_explicit_mt():
|
||||||
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
||||||
name = build_outbound_filename("11525703", "837P", now_mt=now)
|
name = build_outbound_filename("11525703", "837P", now_mt=now)
|
||||||
# HCPF outbound format: tp prefix on the tpid
|
assert name == "11525703-837P-20260620132243505-1of1.x12"
|
||||||
assert name == "tp11525703-837P-20260620132243505-1of1.x12"
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_outbound_default_extension():
|
def test_build_outbound_default_extension():
|
||||||
@@ -40,17 +39,15 @@ def test_build_outbound_default_extension():
|
|||||||
def test_build_outbound_custom_extension():
|
def test_build_outbound_custom_extension():
|
||||||
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
||||||
name = build_outbound_filename("11525703", "837P", ext="txt", now_mt=now)
|
name = build_outbound_filename("11525703", "837P", ext="txt", now_mt=now)
|
||||||
assert name == "tp11525703-837P-20260620132243505-1of1.txt"
|
assert name == "11525703-837P-20260620132243505-1of1.txt"
|
||||||
|
|
||||||
|
|
||||||
def test_build_outbound_uses_mt_when_no_arg():
|
def test_build_outbound_uses_mt_when_no_arg():
|
||||||
# Snapshot test — the timestamp will be very recent; check format only
|
# Snapshot test — the timestamp will be very recent; check format only
|
||||||
name = build_outbound_filename("11525703", "837P")
|
name = build_outbound_filename("11525703", "837P")
|
||||||
assert OUTBOUND_RE.match(name), name
|
assert OUTBOUND_RE.match(name), name
|
||||||
# tp11525703-837P-YYYYMMDDhhmmssSSS-1of1.x12 — 4 dash-separated parts
|
|
||||||
parts = name.split("-")
|
parts = name.split("-")
|
||||||
assert len(parts) == 4
|
assert len(parts) == 4
|
||||||
assert parts[0] == "tp11525703"
|
|
||||||
assert len(parts[2]) == 17 # yyyymmddhhmmssSSS
|
assert len(parts[2]) == 17 # yyyymmddhhmmssSSS
|
||||||
|
|
||||||
|
|
||||||
@@ -131,9 +128,8 @@ def test_parse_inbound_rejects_non_x12_ext():
|
|||||||
|
|
||||||
|
|
||||||
def test_roundtrip_outbound_to_inbound():
|
def test_roundtrip_outbound_to_inbound():
|
||||||
# Outbound uses tp{...}, inbound uses TP{...} (case differs but both
|
# Outbound tpid is bare (no TP); inbound tpid is bare inside TP{...}
|
||||||
# prefixes are required). The two regexes use different shapes —
|
# The two regexes use different shapes — round-trip via tpid only.
|
||||||
# round-trip via tpid only.
|
|
||||||
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
||||||
out = build_outbound_filename("11525703", "837P", now_mt=now)
|
out = build_outbound_filename("11525703", "837P", now_mt=now)
|
||||||
assert OUTBOUND_RE.match(out)
|
assert OUTBOUND_RE.match(out)
|
||||||
@@ -146,19 +142,13 @@ def test_roundtrip_outbound_to_inbound():
|
|||||||
|
|
||||||
|
|
||||||
def test_is_outbound_filename():
|
def test_is_outbound_filename():
|
||||||
# HCPF outbound always has the lowercase "tp" prefix
|
assert is_outbound_filename("11525703-837P-20260620132243505-1of1.x12")
|
||||||
assert is_outbound_filename("tp11525703-837P-20260620132243505-1of1.x12")
|
|
||||||
# Bare tpid (no tp prefix) is no longer a valid outbound filename
|
|
||||||
assert not is_outbound_filename("11525703-837P-20260620132243505-1of1.x12")
|
|
||||||
# Uppercase TP prefix is the inbound shape, not outbound
|
|
||||||
assert not is_outbound_filename("TP11525703-837P-20260620132243505-1of1.x12")
|
assert not is_outbound_filename("TP11525703-837P-20260620132243505-1of1.x12")
|
||||||
assert not is_outbound_filename("not-a-filename")
|
assert not is_outbound_filename("not-a-filename")
|
||||||
|
|
||||||
|
|
||||||
def test_is_inbound_filename():
|
def test_is_inbound_filename():
|
||||||
assert is_inbound_filename("TP11525703-837P_M019048402-20260520231513488-1of1_999.x12")
|
assert is_inbound_filename("TP11525703-837P_M019048402-20260520231513488-1of1_999.x12")
|
||||||
# Lowercase tp prefix is the outbound shape, not inbound
|
|
||||||
assert not is_inbound_filename("tp11525703-837P_M019048402-20260520231513488-1of1_999.x12")
|
|
||||||
assert not is_inbound_filename("11525703-837P-20260620132243505-1of1.x12")
|
assert not is_inbound_filename("11525703-837P-20260620132243505-1of1.x12")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,165 +0,0 @@
|
|||||||
"""SP18 — JsonFormatter + CycloneDevFormatter tests.
|
|
||||||
|
|
||||||
Covers the structural shape of log records, exception handling,
|
|
||||||
and the ``extra`` kwarg passthrough.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import io
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from cyclone.logging_config import (
|
|
||||||
CycloneDevFormatter,
|
|
||||||
JsonFormatter,
|
|
||||||
setup_logging,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _make_record(
|
|
||||||
msg: str = "hello",
|
|
||||||
args: tuple = (),
|
|
||||||
level: int = logging.INFO,
|
|
||||||
name: str = "test.logger",
|
|
||||||
extras: dict | None = None,
|
|
||||||
exc_info=None,
|
|
||||||
) -> logging.LogRecord:
|
|
||||||
record = logging.getLogger(name).makeRecord(
|
|
||||||
name=name,
|
|
||||||
level=level,
|
|
||||||
fn="t.py",
|
|
||||||
lno=1,
|
|
||||||
msg=msg,
|
|
||||||
args=args,
|
|
||||||
exc_info=exc_info,
|
|
||||||
)
|
|
||||||
if extras:
|
|
||||||
for k, v in extras.items():
|
|
||||||
setattr(record, k, v)
|
|
||||||
return record
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# JsonFormatter
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_json_formatter_basic_shape():
|
|
||||||
f = JsonFormatter()
|
|
||||||
line = f.format(_make_record(msg="hello %s", args=("cyclone",)))
|
|
||||||
parsed = json.loads(line)
|
|
||||||
assert parsed["level"] == "INFO"
|
|
||||||
assert parsed["logger"] == "test.logger"
|
|
||||||
assert parsed["msg"] == "hello cyclone"
|
|
||||||
assert "ts" in parsed
|
|
||||||
# ts must be ISO 8601 with milliseconds + Z suffix.
|
|
||||||
assert parsed["ts"].endswith("Z") or "+" in parsed["ts"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_json_formatter_includes_extras():
|
|
||||||
f = JsonFormatter()
|
|
||||||
line = f.format(_make_record(
|
|
||||||
msg="processed",
|
|
||||||
extras={"input_filename": "foo.x12", "parser_kind": "parse_999", "claims": 3},
|
|
||||||
))
|
|
||||||
parsed = json.loads(line)
|
|
||||||
assert parsed["extra"] == {
|
|
||||||
"input_filename": "foo.x12", "parser_kind": "parse_999", "claims": 3,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def test_json_formatter_handles_exception_info():
|
|
||||||
f = JsonFormatter()
|
|
||||||
try:
|
|
||||||
raise ValueError("boom")
|
|
||||||
except ValueError:
|
|
||||||
import sys
|
|
||||||
rec = _make_record(msg="oops", exc_info=sys.exc_info())
|
|
||||||
line = f.format(rec)
|
|
||||||
parsed = json.loads(line)
|
|
||||||
assert "traceback" in parsed
|
|
||||||
assert "ValueError: boom" in parsed["traceback"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_json_formatter_no_extras_key_when_none():
|
|
||||||
f = JsonFormatter()
|
|
||||||
line = f.format(_make_record(msg="plain"))
|
|
||||||
parsed = json.loads(line)
|
|
||||||
assert "extra" not in parsed
|
|
||||||
|
|
||||||
|
|
||||||
def test_json_formatter_handles_non_serializable_extras():
|
|
||||||
"""Non-JSON-serializable extras go through ``default=str``."""
|
|
||||||
class Opaque:
|
|
||||||
def __str__(self):
|
|
||||||
return "opaque-string"
|
|
||||||
f = JsonFormatter()
|
|
||||||
line = f.format(_make_record(msg="x", extras={"thing": Opaque()}))
|
|
||||||
parsed = json.loads(line)
|
|
||||||
assert parsed["extra"]["thing"] == "opaque-string"
|
|
||||||
|
|
||||||
|
|
||||||
def test_json_formatter_preserves_warning_level():
|
|
||||||
f = JsonFormatter()
|
|
||||||
line = f.format(_make_record(msg="careful", level=logging.WARNING))
|
|
||||||
parsed = json.loads(line)
|
|
||||||
assert parsed["level"] == "WARNING"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# CycloneDevFormatter
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_dev_formatter_basic_shape():
|
|
||||||
f = CycloneDevFormatter()
|
|
||||||
line = f.format(_make_record(msg="hello %s", args=("cyclone",)))
|
|
||||||
assert "INFO" in line
|
|
||||||
assert "test.logger" in line
|
|
||||||
assert "hello cyclone" in line
|
|
||||||
|
|
||||||
|
|
||||||
def test_dev_formatter_includes_extras():
|
|
||||||
f = CycloneDevFormatter()
|
|
||||||
line = f.format(_make_record(
|
|
||||||
msg="processed",
|
|
||||||
extras={"input_filename": "foo.x12", "claims": 3},
|
|
||||||
))
|
|
||||||
assert "input_filename='foo.x12'" in line
|
|
||||||
assert "claims=3" in line
|
|
||||||
|
|
||||||
|
|
||||||
def test_dev_formatter_handles_exception():
|
|
||||||
f = CycloneDevFormatter()
|
|
||||||
try:
|
|
||||||
raise RuntimeError("nope")
|
|
||||||
except RuntimeError:
|
|
||||||
import sys
|
|
||||||
rec = _make_record(msg="oops", exc_info=sys.exc_info())
|
|
||||||
line = f.format(rec)
|
|
||||||
assert "RuntimeError: nope" in line
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# setup_logging (light — deeper coverage in test_logging_setup.py)
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_logging_attaches_handler_to_root():
|
|
||||||
setup_logging(level="DEBUG", json_format=True)
|
|
||||||
root = logging.getLogger()
|
|
||||||
assert len(root.handlers) >= 1
|
|
||||||
assert isinstance(root.handlers[0].formatter, JsonFormatter)
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_logging_is_idempotent():
|
|
||||||
"""Re-calling clears handlers; the formatter toggle takes effect."""
|
|
||||||
setup_logging(level="INFO", json_format=True)
|
|
||||||
setup_logging(level="INFO", json_format=False)
|
|
||||||
root = logging.getLogger()
|
|
||||||
assert len(root.handlers) == 1
|
|
||||||
assert isinstance(root.handlers[0].formatter, CycloneDevFormatter)
|
|
||||||
# Reset back to JSON for the rest of the test suite.
|
|
||||||
setup_logging(level="INFO", json_format=True)
|
|
||||||
@@ -1,157 +0,0 @@
|
|||||||
"""SP18 — PII scrubber tests.
|
|
||||||
|
|
||||||
Covers each PHI pattern, the false-positive guard, and the
|
|
||||||
disable toggle.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import logging
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from cyclone.logging_config import (
|
|
||||||
PiiScrubber,
|
|
||||||
get_scrubber,
|
|
||||||
setup_logging,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _make_record(msg: str, extras: dict | None = None) -> logging.LogRecord:
|
|
||||||
record = logging.getLogger("test.scrub").makeRecord(
|
|
||||||
name="test.scrub", level=logging.INFO, fn="t.py", lno=1,
|
|
||||||
msg=msg, args=(), exc_info=None,
|
|
||||||
)
|
|
||||||
if extras:
|
|
||||||
for k, v in extras.items():
|
|
||||||
setattr(record, k, v)
|
|
||||||
return record
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _reset_scrubber():
|
|
||||||
"""Make sure the scrubber is enabled + on the root after each test."""
|
|
||||||
yield
|
|
||||||
get_scrubber().enable()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# NPI
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubs_ten_digit_npi_in_message():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("processed claim with npi 1881068062 ok")
|
|
||||||
assert scrubber.filter(rec) is True
|
|
||||||
assert rec.getMessage() == "processed claim with npi <redacted:npi> ok"
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubs_npi_in_extras():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("ok", extras={"provider_npi": "1881068062"})
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.provider_npi == "<redacted:npi>"
|
|
||||||
|
|
||||||
|
|
||||||
def test_does_not_scrub_short_numbers():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("processed 5 claims in 2 batches")
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.getMessage() == "processed 5 claims in 2 batches"
|
|
||||||
|
|
||||||
|
|
||||||
def test_does_not_scrub_eleven_digit_numbers():
|
|
||||||
"""11+ digit numbers aren't NPIs — leave them alone."""
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("control number 12345678901")
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.getMessage() == "control number 12345678901"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# SSN
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubs_dashed_ssn_in_message():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("ssn=123-45-6789 detected")
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.getMessage() == "ssn=<redacted:ssn> detected"
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubs_undashed_ssn_at_phrase_boundary():
|
|
||||||
"""A bare 9-digit number is ambiguous — we scrub it when followed
|
|
||||||
by whitespace/punctuation/closing paren/brace/comma (so we don't
|
|
||||||
hit claim control numbers or zip codes mid-sentence)."""
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("ssn 123456789 on file")
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert "<redacted:ssn>" in rec.getMessage()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# DOB
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubs_dob_field():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("dob=1980-04-12 verified")
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.getMessage() == "dob=<redacted:dob> verified"
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubs_dob_in_extras():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("ok", extras={"date_of_birth": "1980-04-12"})
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.date_of_birth == "<redacted:dob>"
|
|
||||||
|
|
||||||
|
|
||||||
def test_does_not_scrub_bare_iso_date():
|
|
||||||
"""A YYYY-MM-DD without a dob= prefix isn't necessarily PHI."""
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("parsed at 2026-06-21")
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.getMessage() == "parsed at 2026-06-21"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Patient name
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubs_patient_name_field():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record('patient_name="John Doe" verified')
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert "John Doe" not in rec.getMessage()
|
|
||||||
assert "<redacted:patient_name>" in rec.getMessage()
|
|
||||||
|
|
||||||
|
|
||||||
def test_does_not_scrub_random_words():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("the parser ran successfully")
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.getMessage() == "the parser ran successfully"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Disable toggle
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubber_disabled_leaves_message_intact():
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
scrubber.disable()
|
|
||||||
rec = _make_record("npi 1881068062 ok")
|
|
||||||
scrubber.filter(rec)
|
|
||||||
assert rec.getMessage() == "npi 1881068062 ok"
|
|
||||||
|
|
||||||
|
|
||||||
def test_scrubber_does_not_crash_on_unusual_records():
|
|
||||||
"""Even with weird attribute combinations the filter returns True."""
|
|
||||||
scrubber = PiiScrubber()
|
|
||||||
rec = _make_record("ok", extras={"weird": object()})
|
|
||||||
assert scrubber.filter(rec) is True
|
|
||||||
@@ -1,127 +0,0 @@
|
|||||||
"""SP18 — ``setup_logging`` entry-point tests.
|
|
||||||
|
|
||||||
Covers level resolution, handler attachment, env-var overrides, and
|
|
||||||
the idempotent re-setup behavior used by the FastAPI lifespan and
|
|
||||||
the CLI's ``main()``.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from cyclone.logging_config import (
|
|
||||||
CycloneDevFormatter,
|
|
||||||
JsonFormatter,
|
|
||||||
PiiScrubber,
|
|
||||||
setup_logging,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _reset_root_logger():
|
|
||||||
"""Strip our handlers + filters before each test so setup runs clean."""
|
|
||||||
root = logging.getLogger()
|
|
||||||
for h in list(root.handlers):
|
|
||||||
root.removeHandler(h)
|
|
||||||
for flt in list(root.filters):
|
|
||||||
if isinstance(flt, PiiScrubber):
|
|
||||||
root.removeFilter(flt)
|
|
||||||
yield
|
|
||||||
for h in list(root.handlers):
|
|
||||||
root.removeHandler(h)
|
|
||||||
for flt in list(root.filters):
|
|
||||||
if isinstance(flt, PiiScrubber):
|
|
||||||
root.removeFilter(flt)
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_respects_level_string():
|
|
||||||
setup_logging(level="DEBUG", json_format=True)
|
|
||||||
assert logging.getLogger().level == logging.DEBUG
|
|
||||||
setup_logging(level="WARNING", json_format=True)
|
|
||||||
assert logging.getLogger().level == logging.WARNING
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_attaches_rotating_file_handler(tmp_path: Path):
|
|
||||||
log_file = tmp_path / "cyclone.log"
|
|
||||||
setup_logging(level="INFO", log_file=str(log_file), json_format=True)
|
|
||||||
root = logging.getLogger()
|
|
||||||
assert len(root.handlers) == 1
|
|
||||||
h = root.handlers[0]
|
|
||||||
# RotatingFileHandler has ``baseFilename`` attr.
|
|
||||||
assert hasattr(h, "baseFilename")
|
|
||||||
assert Path(h.baseFilename).name == "cyclone.log"
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_defaults_to_json_formatter():
|
|
||||||
setup_logging(level="INFO")
|
|
||||||
root = logging.getLogger()
|
|
||||||
assert isinstance(root.handlers[0].formatter, JsonFormatter)
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_dev_toggle_uses_dev_formatter():
|
|
||||||
setup_logging(level="INFO", json_format=False)
|
|
||||||
root = logging.getLogger()
|
|
||||||
assert isinstance(root.handlers[0].formatter, CycloneDevFormatter)
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_idempotent_re_setup_replaces_handlers():
|
|
||||||
"""Re-calling setup_logging clears the previous handler(s)."""
|
|
||||||
setup_logging(level="INFO", json_format=True)
|
|
||||||
setup_logging(level="DEBUG", json_format=False)
|
|
||||||
root = logging.getLogger()
|
|
||||||
assert len(root.handlers) == 1
|
|
||||||
assert isinstance(root.handlers[0].formatter, CycloneDevFormatter)
|
|
||||||
assert root.level == logging.DEBUG
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_quietens_noisy_third_party_loggers():
|
|
||||||
setup_logging(level="DEBUG", json_format=True)
|
|
||||||
for noisy in ("urllib3", "paramiko", "sqlalchemy.engine"):
|
|
||||||
assert logging.getLogger(noisy).level >= logging.WARNING
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_attaches_pii_scrubber_by_default():
|
|
||||||
setup_logging(level="INFO", json_format=True)
|
|
||||||
root = logging.getLogger()
|
|
||||||
assert any(isinstance(f, PiiScrubber) for f in root.filters)
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_honors_scrub_pii_false():
|
|
||||||
setup_logging(level="INFO", json_format=True, scrub_pii=False)
|
|
||||||
root = logging.getLogger()
|
|
||||||
# Scrubber is still attached but disabled.
|
|
||||||
scrubbers = [f for f in root.filters if isinstance(f, PiiScrubber)]
|
|
||||||
assert len(scrubbers) == 1
|
|
||||||
assert scrubbers[0]._enabled is False # noqa: SLF001
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_honors_env_var_no_pii_scrub(monkeypatch):
|
|
||||||
monkeypatch.setenv("CYCLONE_LOG_NO_PII_SCRUB", "1")
|
|
||||||
setup_logging(level="INFO", json_format=True)
|
|
||||||
scrubbers = [f for f in logging.getLogger().filters if isinstance(f, PiiScrubber)]
|
|
||||||
assert scrubbers and scrubbers[0]._enabled is False # noqa: SLF001
|
|
||||||
|
|
||||||
|
|
||||||
def test_setup_emits_json_to_stderr_by_default(caplog):
|
|
||||||
"""Records emitted after setup flow through JsonFormatter."""
|
|
||||||
setup_logging(level="INFO", json_format=True)
|
|
||||||
logger = logging.getLogger("cyclone.test_setup")
|
|
||||||
logger.info("hello %s", "world", extra={"x": 1})
|
|
||||||
# Cyclone attaches the handler to root, not the named logger; caplog
|
|
||||||
# won't capture unless we propagate (which is the default).
|
|
||||||
# Just assert the handler is on root and would format correctly.
|
|
||||||
root = logging.getLogger()
|
|
||||||
h = root.handlers[0]
|
|
||||||
record = logger.makeRecord(
|
|
||||||
name="cyclone.test_setup", level=logging.INFO, fn="t.py", lno=1,
|
|
||||||
msg="hello %s", args=("world",), exc_info=None,
|
|
||||||
)
|
|
||||||
record.x = 1
|
|
||||||
formatted = h.formatter.format(record)
|
|
||||||
import json as _json
|
|
||||||
parsed = _json.loads(formatted)
|
|
||||||
assert parsed["msg"] == "hello world"
|
|
||||||
assert parsed["extra"]["x"] == 1
|
|
||||||
@@ -1,199 +0,0 @@
|
|||||||
"""Tests for ``cyclone.npi`` — NPI checksum + Tax ID (EIN) format validation.
|
|
||||||
|
|
||||||
Pure local validation, no NPPES calls. Covers the NPPES-published Luhn
|
|
||||||
checksum (with prefix ``80840``) and a small set of obvious EIN typo cases.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from cyclone.npi import (
|
|
||||||
_luhn_check_digit,
|
|
||||||
is_valid_npi,
|
|
||||||
is_valid_tax_id,
|
|
||||||
npi_checksum,
|
|
||||||
normalize_tax_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Luhn internals
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_luhn_check_digit_known_sequence():
|
|
||||||
"""Standard Luhn for the empty body returns 0.
|
|
||||||
|
|
||||||
With no digits the sum is 0, so (10 - 0 % 10) % 10 = 0.
|
|
||||||
"""
|
|
||||||
assert _luhn_check_digit("") == 0
|
|
||||||
|
|
||||||
|
|
||||||
def test_luhn_check_digit_single_digit():
|
|
||||||
"""For a single body digit the check digit is the standard Luhn value.
|
|
||||||
|
|
||||||
With the corrected "double at rightmost" pattern: "0" doubles to 0
|
|
||||||
(total=0, check=0); "1" doubles to 2 (total=2, check=(10-2)%10=8).
|
|
||||||
"""
|
|
||||||
assert _luhn_check_digit("0") == 0
|
|
||||||
assert _luhn_check_digit("1") == 8
|
|
||||||
|
|
||||||
|
|
||||||
def test_luhn_check_digit_nppes_published():
|
|
||||||
"""CMS-published example: body 123456789 → check digit 3.
|
|
||||||
|
|
||||||
Per https://www.cms.gov/.../NPIcheckdigit.pdf the Luhn sum of the
|
|
||||||
prefixed body ``80840123456789`` is 67, so check digit = (10-7)%10 = 3,
|
|
||||||
giving the full NPI ``1234567893``.
|
|
||||||
"""
|
|
||||||
assert _luhn_check_digit("80840123456789") == 3
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# npi_checksum
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_npi_checksum_cms_published_body():
|
|
||||||
"""For NPI body 123456789 the check digit is 3 → NPI 1234567893.
|
|
||||||
|
|
||||||
Confirmed against the CMS-published NPI Luhn example.
|
|
||||||
"""
|
|
||||||
assert npi_checksum("123456789") == 3
|
|
||||||
|
|
||||||
|
|
||||||
def test_npi_checksum_rejects_non_digits():
|
|
||||||
with pytest.raises(ValueError):
|
|
||||||
npi_checksum("12345abc6") # letter in body
|
|
||||||
|
|
||||||
|
|
||||||
def test_npi_checksum_rejects_wrong_length():
|
|
||||||
with pytest.raises(ValueError):
|
|
||||||
npi_checksum("12345") # only 5 digits
|
|
||||||
with pytest.raises(ValueError):
|
|
||||||
npi_checksum("1234567890") # 10 digits — includes the check digit
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# is_valid_npi
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_nppes_sample_is_true():
|
|
||||||
"""The CMS-published example NPI 1234567893 must validate."""
|
|
||||||
assert is_valid_npi("1234567893") is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_off_by_one_is_false():
|
|
||||||
assert is_valid_npi("1234567894") is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_all_zeros_is_false():
|
|
||||||
"""All zeros fails the Luhn check."""
|
|
||||||
assert is_valid_npi("0000000000") is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_rejects_empty():
|
|
||||||
assert is_valid_npi("") is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_rejects_none():
|
|
||||||
assert is_valid_npi(None) is False # type: ignore[arg-type]
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_rejects_non_string():
|
|
||||||
assert is_valid_npi(1234567890) is False # type: ignore[arg-type]
|
|
||||||
assert is_valid_npi(["1881068062"]) is False # type: ignore[arg-type]
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_rejects_short():
|
|
||||||
assert is_valid_npi("123456789") is False # 9 digits
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_rejects_long():
|
|
||||||
assert is_valid_npi("12345678901") is False # 11 digits
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_rejects_non_digits():
|
|
||||||
assert is_valid_npi("188106806X") is False
|
|
||||||
assert is_valid_npi("18810 68062") is False # space
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_npi_all_ones_fails_luhn():
|
|
||||||
"""1111111111 has all-1 sum: alternating double = 1,2,1,2,..., 1+2=3 then
|
|
||||||
collapse. Total for 10 digits body (body=9 of all 1's, sum doubled):
|
|
||||||
Position from right (i=0..8): 1, 1, 1, 1, 1, 1, 1, 1, 1
|
|
||||||
i even (not doubled): 1, 1, 1, 1, 1 → 5
|
|
||||||
i odd (doubled): 1*2=2, 1*2=2, 1*2=2, 1*2=2 → 8
|
|
||||||
Total body = 13. Body alone has check digit (10 - 13 % 10) % 10 = 7.
|
|
||||||
So full NPI 1111111111's body (9 ones) check = 7, and 1 != 7 → invalid.
|
|
||||||
"""
|
|
||||||
assert is_valid_npi("1111111111") is False
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# is_valid_tax_id / normalize_tax_id
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_tax_id_touch_of_care_true():
|
|
||||||
"""The operator's reference EIN — Touch of Care Family Practice."""
|
|
||||||
assert is_valid_tax_id("72-1587149") is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_tax_id_unformatted_true():
|
|
||||||
assert is_valid_tax_id("721587149") is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_tax_id_00_prefix_rejected():
|
|
||||||
"""``00`` is reserved / never assigned by the IRS."""
|
|
||||||
assert is_valid_tax_id("00-1234567") is False
|
|
||||||
assert is_valid_tax_id("001234567") is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_tax_id_07_prefix_rejected():
|
|
||||||
"""``07`` is a campus prefix reserved for future use."""
|
|
||||||
assert is_valid_tax_id("07-1234567") is False
|
|
||||||
assert is_valid_tax_id("071234567") is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_tax_id_8x_prefix_rejected():
|
|
||||||
"""``80``–``89`` is the IRS Pension Plan Branch — never assigned otherwise."""
|
|
||||||
assert is_valid_tax_id("80-1234567") is False
|
|
||||||
assert is_valid_tax_id("89-1234567") is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_tax_id_rejects_malformed():
|
|
||||||
assert is_valid_tax_id("not-an-ein") is False
|
|
||||||
assert is_valid_tax_id("12345") is False
|
|
||||||
assert is_valid_tax_id("1234567890") is False # 10 digits
|
|
||||||
assert is_valid_tax_id("12-345678") is False # 8 digits after hyphen
|
|
||||||
|
|
||||||
|
|
||||||
def test_is_valid_tax_id_rejects_none_and_non_string():
|
|
||||||
assert is_valid_tax_id(None) is False
|
|
||||||
assert is_valid_tax_id(721587149) is False # type: ignore[arg-type]
|
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_tax_id_returns_plain_form():
|
|
||||||
assert normalize_tax_id("72-1587149") == "721587149"
|
|
||||||
assert normalize_tax_id("721587149") == "721587149"
|
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_tax_id_strips_whitespace():
|
|
||||||
assert normalize_tax_id(" 72-1587149 ") == "721587149"
|
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_tax_id_returns_none_for_invalid():
|
|
||||||
assert normalize_tax_id("not-an-ein") is None
|
|
||||||
assert normalize_tax_id(None) is None
|
|
||||||
assert normalize_tax_id("") is None
|
|
||||||
assert normalize_tax_id("12-345678") is None # wrong digit count
|
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_tax_id_keeps_reserved_prefix():
|
|
||||||
"""normalize_tax_id is a *structural* normalizer — it doesn't reject
|
|
||||||
reserved prefixes. That's ``is_valid_tax_id``'s job. Operators who
|
|
||||||
want to store 00-prefixed EINs as placeholders still get a clean
|
|
||||||
9-digit string."""
|
|
||||||
assert normalize_tax_id("00-1234567") == "001234567"
|
|
||||||
@@ -14,7 +14,7 @@ def test_parse_minimal_fixture_returns_one_claim():
|
|||||||
assert len(result.claims) == 1
|
assert len(result.claims) == 1
|
||||||
claim = result.claims[0]
|
claim = result.claims[0]
|
||||||
assert claim.claim_id == "CLM001"
|
assert claim.claim_id == "CLM001"
|
||||||
assert claim.billing_provider.npi == "1993999998"
|
assert claim.billing_provider.npi == "1234567890"
|
||||||
assert claim.subscriber.last_name == "Doe"
|
assert claim.subscriber.last_name == "Doe"
|
||||||
assert claim.subscriber.first_name == "John"
|
assert claim.subscriber.first_name == "John"
|
||||||
assert claim.subscriber.member_id == "ABC123"
|
assert claim.subscriber.member_id == "ABC123"
|
||||||
|
|||||||
@@ -1,106 +0,0 @@
|
|||||||
"""Tests for GET /api/payers/{payer_id}/summary (SP21 Task 1.5).
|
|
||||||
|
|
||||||
The endpoint is the payer-level aggregate that the drill-down UI's
|
|
||||||
"Payer → Claims" panel hangs off. It returns billed/received totals,
|
|
||||||
denial rate, and the top 5 NPIs by claim volume for one payer_id,
|
|
||||||
cached in-process for 60s.
|
|
||||||
|
|
||||||
The minimal 837P fixture ships one CLM with ``payer_id="SKCO0"``,
|
|
||||||
charge_amount=100.00; the minimal 835 carries one CLP for the same
|
|
||||||
claim with total_paid=85.00. So ``/api/payers/SKCO0/summary`` returns
|
|
||||||
``claim_count >= 1`` after both files are ingested.
|
|
||||||
|
|
||||||
Note: the spec calls this ``payer_id`` (the X12 NM1*PR*PI qualifier,
|
|
||||||
e.g. ``SKCO0``). It is NOT the configured payer name from
|
|
||||||
``config/payers.yaml``. The filter key in the store layer is
|
|
||||||
``Claim.payer_id`` — not the ``payer=`` substring filter used by
|
|
||||||
``/api/claims?payer=...``.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
from cyclone import api as api_mod
|
|
||||||
from cyclone.api import app
|
|
||||||
|
|
||||||
FIXTURE_837 = Path(__file__).parent / "fixtures" / "minimal_837p.txt"
|
|
||||||
FIXTURE_835 = Path(__file__).parent / "fixtures" / "minimal_835.txt"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _clear_summary_cache():
|
|
||||||
"""Wipe the in-process payer-summary cache between tests.
|
|
||||||
|
|
||||||
conftest resets the DB per test but the cache is module-level
|
|
||||||
state on ``cyclone.api``. Without this clear, a stale payload
|
|
||||||
from a previous test's seed would leak into a later test's first
|
|
||||||
call — masking recompute behavior. The 60s TTL is the only
|
|
||||||
invalidation story today (see api.py docstring on the endpoint).
|
|
||||||
"""
|
|
||||||
api_mod._clear_summary_cache()
|
|
||||||
yield
|
|
||||||
api_mod._clear_summary_cache()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def client() -> TestClient:
|
|
||||||
return TestClient(app)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def seeded_db(client: TestClient):
|
|
||||||
"""Ingest one minimal 837P + one minimal 835.
|
|
||||||
|
|
||||||
Both fixtures carry ``payer_id="SKCO0"`` so the summary endpoint
|
|
||||||
has something to aggregate. ``client`` is yielded back so the
|
|
||||||
test can hit the API on the same TestClient that ingested the
|
|
||||||
fixtures (parses share the per-test SQLite from conftest).
|
|
||||||
"""
|
|
||||||
text_837 = FIXTURE_837.read_text()
|
|
||||||
text_835 = FIXTURE_835.read_text()
|
|
||||||
r837 = client.post(
|
|
||||||
"/api/parse-837",
|
|
||||||
files={"file": ("x.txt", text_837, "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert r837.status_code == 200, r837.text
|
|
||||||
r835 = client.post(
|
|
||||||
"/api/parse-835",
|
|
||||||
files={"file": ("era.txt", text_835, "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert r835.status_code == 200, r835.text
|
|
||||||
return client
|
|
||||||
|
|
||||||
|
|
||||||
def test_payer_summary_happy_path(seeded_db: TestClient):
|
|
||||||
"""Seeded db has at least one claim for SKCO0 → 200 with the spec shape."""
|
|
||||||
resp = seeded_db.get("/api/payers/SKCO0/summary")
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
data = resp.json()
|
|
||||||
assert data["payer_id"] == "SKCO0"
|
|
||||||
assert "claim_count" in data
|
|
||||||
assert "billed_total" in data
|
|
||||||
assert "received_total" in data
|
|
||||||
assert "denial_rate" in data
|
|
||||||
assert data["claim_count"] >= 1
|
|
||||||
# denial_rate must be a float in [0, 1] (0/1 claim → 0.0).
|
|
||||||
assert isinstance(data["denial_rate"], (int, float))
|
|
||||||
assert 0.0 <= data["denial_rate"] <= 1.0
|
|
||||||
|
|
||||||
|
|
||||||
def test_payer_summary_unknown_payer_returns_404(client: TestClient):
|
|
||||||
resp = client.get("/api/payers/DOES_NOT_EXIST/summary")
|
|
||||||
assert resp.status_code == 404
|
|
||||||
|
|
||||||
|
|
||||||
def test_payer_summary_caches_then_invalidates(seeded_db: TestClient):
|
|
||||||
"""Two back-to-back calls return identical payloads (in-process cache)."""
|
|
||||||
resp1 = seeded_db.get("/api/payers/SKCO0/summary")
|
|
||||||
resp2 = seeded_db.get("/api/payers/SKCO0/summary")
|
|
||||||
assert resp1.status_code == 200
|
|
||||||
assert resp2.status_code == 200
|
|
||||||
assert resp1.json() == resp2.json()
|
|
||||||
@@ -1,177 +0,0 @@
|
|||||||
"""Tests for the extended GET /api/config/providers/{npi} (SP21 Task 1.6).
|
|
||||||
|
|
||||||
The endpoint gains two new top-level arrays for the drill-down panel:
|
|
||||||
``recent_claims`` (top 10 by submission date desc) and ``recent_activity``
|
|
||||||
(top 10 by ``ts`` desc, joined to claims by ``claim_id`` because
|
|
||||||
``ActivityEvent`` has no direct ``provider_npi`` column).
|
|
||||||
|
|
||||||
Existing SP9 fields (``label``, ``legal_name``, ``tax_id``,
|
|
||||||
``address_line1``, ``city``, ``state``, ``zip``, etc.) must remain
|
|
||||||
present — the new arrays are additive only.
|
|
||||||
|
|
||||||
The fixtures in ``fixtures/minimal_837p.txt`` and ``fixtures/minimal_835.txt``
|
|
||||||
pair up to a single claim with ``provider_npi='1993999998'``. That NPI is
|
|
||||||
NOT in the seeded provider set (Montrose/Delta/Salida → 1881068062/1851446637/
|
|
||||||
1467507269). For the tests we hit the seeded Montrose NPI, which is the
|
|
||||||
canonical SP9 fixture NPI. The arrays come back as empty lists — the
|
|
||||||
contract under test is the *shape* (array, ≤10) and the *backwards compat*
|
|
||||||
of the existing fields; the data-path itself is exercised by the existing
|
|
||||||
ingestion path that backs ``/api/claims``.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
from cyclone.api import app
|
|
||||||
from cyclone.providers import Provider
|
|
||||||
from cyclone.store import store
|
|
||||||
|
|
||||||
FIXTURE_837 = Path(__file__).parent / "fixtures" / "minimal_837p.txt"
|
|
||||||
FIXTURE_835 = Path(__file__).parent / "fixtures" / "minimal_835.txt"
|
|
||||||
|
|
||||||
# Montrose — one of the three providers that `ensure_clearhouse_seeded()`
|
|
||||||
# writes into the providers table. Using a seeded NPI means the endpoint
|
|
||||||
# won't 404; the seeded claims (provider_npi='1993999998') won't appear
|
|
||||||
# under this NPI, so recent_claims/activity are expected empty lists.
|
|
||||||
MONTROSE_NPI = "1881068062"
|
|
||||||
|
|
||||||
# NPI the minimal 837P fixture bills under. NOT in the default seed —
|
|
||||||
# registering it before ingest is required to pass R204 (NPI must exist
|
|
||||||
# in the providers table).
|
|
||||||
TEST_837_NPI = "1993999998"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def client() -> TestClient:
|
|
||||||
return TestClient(app)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def seeded_db(client: TestClient):
|
|
||||||
"""Seed the clearhouse + ingest the minimal 837P/835 fixtures.
|
|
||||||
|
|
||||||
Mirrors the Task 1.5 ``seeded_db`` pattern: seed → ingest → hand the
|
|
||||||
client back so the test hits the same TestClient.
|
|
||||||
|
|
||||||
The 837 fixture bills under ``TEST_837_NPI``; without first registering
|
|
||||||
that provider the parser's R204 rule rejects the claim with HTTP 422.
|
|
||||||
"""
|
|
||||||
store.ensure_clearhouse_seeded()
|
|
||||||
test_provider = Provider(
|
|
||||||
npi=TEST_837_NPI,
|
|
||||||
label="Test Provider",
|
|
||||||
legal_name="Test Provider Inc",
|
|
||||||
tax_id="123456789",
|
|
||||||
taxonomy_code="207R00000X",
|
|
||||||
address_line1="123 Test St",
|
|
||||||
city="Denver",
|
|
||||||
state="CO",
|
|
||||||
zip="80202",
|
|
||||||
created_at=datetime.now(timezone.utc),
|
|
||||||
updated_at=datetime.now(timezone.utc),
|
|
||||||
)
|
|
||||||
store.upsert_provider(test_provider)
|
|
||||||
text_837 = FIXTURE_837.read_text()
|
|
||||||
text_835 = FIXTURE_835.read_text()
|
|
||||||
r837 = client.post(
|
|
||||||
"/api/parse-837",
|
|
||||||
files={"file": ("x.txt", text_837, "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert r837.status_code == 200, r837.text
|
|
||||||
r835 = client.post(
|
|
||||||
"/api/parse-835",
|
|
||||||
files={"file": ("era.txt", text_835, "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert r835.status_code == 200, r835.text
|
|
||||||
return client
|
|
||||||
|
|
||||||
|
|
||||||
def test_provider_detail_includes_recent_claims(seeded_db: TestClient):
|
|
||||||
"""The extended response gains a recent_claims array (top 10)."""
|
|
||||||
resp = seeded_db.get(f"/api/config/providers/{MONTROSE_NPI}")
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
data = resp.json()
|
|
||||||
assert "recent_claims" in data
|
|
||||||
assert isinstance(data["recent_claims"], list)
|
|
||||||
assert len(data["recent_claims"]) <= 10
|
|
||||||
|
|
||||||
|
|
||||||
def test_provider_detail_includes_recent_activity(seeded_db: TestClient):
|
|
||||||
"""The extended response gains a recent_activity array (top 10)."""
|
|
||||||
resp = seeded_db.get(f"/api/config/providers/{MONTROSE_NPI}")
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
data = resp.json()
|
|
||||||
assert "recent_activity" in data
|
|
||||||
assert isinstance(data["recent_activity"], list)
|
|
||||||
assert len(data["recent_activity"]) <= 10
|
|
||||||
|
|
||||||
|
|
||||||
def test_provider_detail_backwards_compat(seeded_db: TestClient):
|
|
||||||
"""All SP9 fields still present; new arrays don't break the contract.
|
|
||||||
|
|
||||||
The Provider Pydantic model (backend/src/cyclone/providers.py)
|
|
||||||
serializes snake_case fields — that's what the wire carries. The
|
|
||||||
TS ``Provider`` interface in ``src/types/index.ts`` is the
|
|
||||||
in-memory sample shape and intentionally diverges; the contract
|
|
||||||
being verified here is the API's actual payload.
|
|
||||||
"""
|
|
||||||
resp = seeded_db.get(f"/api/config/providers/{MONTROSE_NPI}")
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
data = resp.json()
|
|
||||||
for key in (
|
|
||||||
"npi",
|
|
||||||
"label",
|
|
||||||
"legal_name",
|
|
||||||
"tax_id",
|
|
||||||
"taxonomy_code",
|
|
||||||
"address_line1",
|
|
||||||
"city",
|
|
||||||
"state",
|
|
||||||
"zip",
|
|
||||||
"is_active",
|
|
||||||
):
|
|
||||||
assert key in data, f"missing field {key}"
|
|
||||||
|
|
||||||
|
|
||||||
def test_provider_detail_includes_orphan_remit_received(seeded_db: TestClient):
|
|
||||||
"""Regression: the ``remit_received`` ActivityEvent is recorded at 835
|
|
||||||
ingest with ``claim_id=None`` (``store.add`` lines 999-1003) — the
|
|
||||||
remittance hasn't been matched to a claim yet. The original
|
|
||||||
``ActivityEvent.claim_id IN (claim_ids)`` filter misses it because
|
|
||||||
the orphan's claim_id is NULL.
|
|
||||||
|
|
||||||
Once reconciliation (auto or manual) populates
|
|
||||||
``Remittance.claim_id``, the activity filter must surface the event
|
|
||||||
via the ``Remittance.claim_id IN (claim_ids)`` branch of the OR.
|
|
||||||
Without that branch, a provider's activity feed appears to freeze
|
|
||||||
the moment an 835 lands — the most common activity, invisible.
|
|
||||||
|
|
||||||
Setup: ingest 837+835 for ``TEST_837_NPI`` (claim CLM001 + remit
|
|
||||||
CLM001), then manually match them so ``Remittance.claim_id`` is
|
|
||||||
populated. The bug presents as: only ``claim_submitted`` and
|
|
||||||
``manual_match`` appear (no ``remit_received``). The fix surfaces
|
|
||||||
all three.
|
|
||||||
"""
|
|
||||||
# Force the match — simulates the post-reconciliation state that
|
|
||||||
# populate Remittance.claim_id without depending on auto-reconcile
|
|
||||||
# heuristics (which don't match this minimal fixture).
|
|
||||||
match_resp = seeded_db.post(
|
|
||||||
"/api/reconciliation/match",
|
|
||||||
json={"claim_id": "CLM001", "remit_id": "CLM001"},
|
|
||||||
)
|
|
||||||
assert match_resp.status_code == 200, match_resp.text
|
|
||||||
|
|
||||||
resp = seeded_db.get(f"/api/config/providers/{TEST_837_NPI}")
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
data = resp.json()
|
|
||||||
kinds = {event["kind"] for event in data["recent_activity"]}
|
|
||||||
assert "remit_received" in kinds, (
|
|
||||||
f"expected remit_received in recent_activity (orphan remits "
|
|
||||||
f"must surface via the Remittance join), got kinds={sorted(kinds)}"
|
|
||||||
)
|
|
||||||
@@ -1,287 +0,0 @@
|
|||||||
"""SP16 — Inbound MFT polling scheduler tests.
|
|
||||||
|
|
||||||
We test the Scheduler class with a fake ``SftpClient`` factory that
|
|
||||||
returns files we drop on disk (the SFTP stub already does this; we
|
|
||||||
just need to control which files appear between ticks). The handlers
|
|
||||||
themselves (999/835/277CA/TA1) are exercised through real parsers
|
|
||||||
using the fixtures in ``tests/fixtures/``.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Iterable
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from cyclone import db, scheduler as sched_mod
|
|
||||||
from cyclone.db import ProcessedInboundFile
|
|
||||||
from cyclone.providers import SftpBlock
|
|
||||||
from cyclone.scheduler import (
|
|
||||||
HANDLERS,
|
|
||||||
ROUTED_FILE_TYPES,
|
|
||||||
Scheduler,
|
|
||||||
STATUS_ERROR,
|
|
||||||
STATUS_OK,
|
|
||||||
STATUS_SKIPPED,
|
|
||||||
TickResult,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ----- fixtures -----------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def sftp_block(tmp_path):
|
|
||||||
staging = tmp_path / "staging"
|
|
||||||
inbound_dir = staging / "ToHPE"
|
|
||||||
inbound_dir.mkdir(parents=True)
|
|
||||||
return SftpBlock(
|
|
||||||
host="mft.example.com",
|
|
||||||
port=22,
|
|
||||||
username="test",
|
|
||||||
paths={
|
|
||||||
"outbound": "/FromHPE",
|
|
||||||
"inbound": "/ToHPE",
|
|
||||||
},
|
|
||||||
stub=True,
|
|
||||||
staging_dir=str(staging),
|
|
||||||
poll_seconds=60,
|
|
||||||
auth={"method": "keychain", "secret_ref": "test.password"},
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def _drop_file(sftp_block):
|
|
||||||
"""Helper: drop a named file in the inbound dir. Returns the path."""
|
|
||||||
inbound_dir = Path(sftp_block.staging_dir) / "ToHPE"
|
|
||||||
|
|
||||||
def _drop(name: str, body: bytes) -> Path:
|
|
||||||
inbound_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
p = inbound_dir / name
|
|
||||||
p.write_bytes(body)
|
|
||||||
return p
|
|
||||||
|
|
||||||
return _drop
|
|
||||||
|
|
||||||
|
|
||||||
def _make_scheduler(sftp_block, tmp_path) -> Scheduler:
|
|
||||||
"""Build a Scheduler wired to the real (stub) SftpClient."""
|
|
||||||
sched = Scheduler(
|
|
||||||
sftp_block,
|
|
||||||
poll_interval_seconds=60,
|
|
||||||
sftp_block_name="test-block",
|
|
||||||
# Use the real SftpClient — it reads from the stub staging dir.
|
|
||||||
sftp_client_factory=None,
|
|
||||||
)
|
|
||||||
return sched
|
|
||||||
|
|
||||||
|
|
||||||
def _load_999_text() -> str:
|
|
||||||
return (Path(__file__).parent / "fixtures" / "minimal_999.txt").read_text()
|
|
||||||
|
|
||||||
|
|
||||||
def _load_835_text() -> str:
|
|
||||||
return (Path(__file__).parent / "fixtures" / "minimal_835.txt").read_text()
|
|
||||||
|
|
||||||
|
|
||||||
def _load_277ca_text() -> str:
|
|
||||||
return (Path(__file__).parent / "fixtures" / "minimal_277ca.txt").read_text()
|
|
||||||
|
|
||||||
|
|
||||||
def _load_ta1_text() -> str:
|
|
||||||
return (Path(__file__).parent / "fixtures" / "minimal_ta1.txt").read_text()
|
|
||||||
|
|
||||||
|
|
||||||
# ----- tests --------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class TestSchedulerStatus:
|
|
||||||
def test_not_running_by_default(self, sftp_block):
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
st = sched.status()
|
|
||||||
assert st.running is False
|
|
||||||
assert st.poll_count == 0
|
|
||||||
assert st.last_poll_at is None
|
|
||||||
|
|
||||||
def test_running_after_start(self, sftp_block):
|
|
||||||
async def _go():
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
await sched.start()
|
|
||||||
try:
|
|
||||||
assert sched.is_running() is True
|
|
||||||
finally:
|
|
||||||
await sched.stop()
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
|
|
||||||
class TestTickOnEmptyInbox:
|
|
||||||
def test_tick_with_no_files_records_zero(self, sftp_block):
|
|
||||||
async def _go():
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
result = await sched.tick()
|
|
||||||
assert isinstance(result, TickResult)
|
|
||||||
assert result.files_seen == 0
|
|
||||||
assert result.files_processed == 0
|
|
||||||
assert result.files_skipped == 0
|
|
||||||
assert result.files_errored == 0
|
|
||||||
assert result.finished_at is not None
|
|
||||||
assert result.errors == []
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
|
|
||||||
class TestTickRoutesFiles:
|
|
||||||
def test_999_file_processed(self, sftp_block, _drop_file):
|
|
||||||
async def _go():
|
|
||||||
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_999.x12",
|
|
||||||
_load_999_text().encode("utf-8"))
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
result = await sched.tick()
|
|
||||||
assert result.files_seen == 1
|
|
||||||
assert result.files_processed == 1
|
|
||||||
assert result.files_errored == 0
|
|
||||||
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
rows = (
|
|
||||||
session.query(ProcessedInboundFile)
|
|
||||||
.filter_by(name="TP11525703-837P_M019048402-20260618130000000-1of1_999.x12")
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
assert len(rows) == 1
|
|
||||||
assert rows[0].status == STATUS_OK
|
|
||||||
assert rows[0].parser_used == "parse_999"
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
def test_ta1_file_processed(self, sftp_block, _drop_file):
|
|
||||||
async def _go():
|
|
||||||
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12",
|
|
||||||
_load_ta1_text().encode("utf-8"))
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
result = await sched.tick()
|
|
||||||
assert result.files_processed == 1, result.errors
|
|
||||||
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
row = (
|
|
||||||
session.query(ProcessedInboundFile)
|
|
||||||
.filter_by(name="TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12")
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
assert row is not None
|
|
||||||
assert row.status == STATUS_OK
|
|
||||||
assert row.parser_used == "parse_ta1"
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
def test_unknown_file_type_marked_skipped(self, sftp_block, _drop_file):
|
|
||||||
async def _go():
|
|
||||||
# 270 (eligibility request) is in the HCPF allowed set but
|
|
||||||
# NOT in ROUTED_FILE_TYPES — Cyclone doesn't have a 270 parser.
|
|
||||||
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_270.x12",
|
|
||||||
b"some bytes")
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
result = await sched.tick()
|
|
||||||
assert result.files_skipped == 1
|
|
||||||
assert result.files_processed == 0
|
|
||||||
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
row = (
|
|
||||||
session.query(ProcessedInboundFile)
|
|
||||||
.filter_by(name="TP11525703-837P_M019048402-20260618130000000-1of1_270.x12")
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
assert row is not None
|
|
||||||
assert row.status == STATUS_SKIPPED
|
|
||||||
assert row.error_message and "270" in row.error_message
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
def test_filename_not_matching_hcpf_marked_skipped(self, sftp_block, _drop_file):
|
|
||||||
async def _go():
|
|
||||||
_drop_file("random.txt", b"garbage")
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
result = await sched.tick()
|
|
||||||
assert result.files_skipped == 1
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
def test_parse_error_marked_error(self, sftp_block, _drop_file):
|
|
||||||
async def _go():
|
|
||||||
# Valid filename but malformed body — parser raises.
|
|
||||||
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_999.x12",
|
|
||||||
b"this is not a 999 file at all")
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
result = await sched.tick()
|
|
||||||
assert result.files_errored == 1
|
|
||||||
assert result.files_processed == 0
|
|
||||||
|
|
||||||
with db.SessionLocal()() as session:
|
|
||||||
row = (
|
|
||||||
session.query(ProcessedInboundFile)
|
|
||||||
.filter_by(name="TP11525703-837P_M019048402-20260618130000000-1of1_999.x12")
|
|
||||||
.first()
|
|
||||||
)
|
|
||||||
assert row.status == STATUS_ERROR
|
|
||||||
assert row.error_message
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
|
|
||||||
class TestTickIdempotent:
|
|
||||||
def test_second_tick_does_not_reprocess(self, sftp_block, _drop_file):
|
|
||||||
async def _go():
|
|
||||||
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12",
|
|
||||||
_load_ta1_text().encode("utf-8"))
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
r1 = await sched.tick()
|
|
||||||
r2 = await sched.tick()
|
|
||||||
assert r1.files_processed == 1
|
|
||||||
assert r2.files_seen == 1 # still lists it
|
|
||||||
assert r2.files_processed == 0 # but skips — already done
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
|
|
||||||
class TestSchedulerStartStop:
|
|
||||||
def test_start_then_stop_returns_to_not_running(self, sftp_block):
|
|
||||||
async def _go():
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
await sched.start()
|
|
||||||
assert sched.is_running()
|
|
||||||
await sched.stop()
|
|
||||||
assert not sched.is_running()
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
def test_double_start_is_idempotent(self, sftp_block):
|
|
||||||
async def _go():
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
await sched.start()
|
|
||||||
await sched.start() # no-op
|
|
||||||
assert sched.is_running()
|
|
||||||
await sched.stop()
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
def test_stop_when_not_running_is_safe(self, sftp_block):
|
|
||||||
async def _go():
|
|
||||||
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
|
||||||
await sched.stop() # no-op
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
|
|
||||||
class TestModuleSingleton:
|
|
||||||
def test_get_scheduler_raises_if_not_configured(self):
|
|
||||||
sched_mod.reset_scheduler_for_tests()
|
|
||||||
with pytest.raises(RuntimeError, match="not configured"):
|
|
||||||
sched_mod.get_scheduler()
|
|
||||||
|
|
||||||
def test_configure_then_get_returns_same_instance(self, sftp_block):
|
|
||||||
sched_mod.reset_scheduler_for_tests()
|
|
||||||
s = sched_mod.configure_scheduler(sftp_block, sftp_block_name="t")
|
|
||||||
try:
|
|
||||||
assert sched_mod.get_scheduler() is s
|
|
||||||
finally:
|
|
||||||
sched_mod.reset_scheduler_for_tests()
|
|
||||||
|
|
||||||
|
|
||||||
class TestRoutedFileTypes:
|
|
||||||
"""Frozen-set guards — adding a new routed type without updating
|
|
||||||
the dispatch table would silently skip every inbound file of that
|
|
||||||
type. These tests are the regression net."""
|
|
||||||
|
|
||||||
def test_handlers_cover_all_routed_types(self):
|
|
||||||
assert set(HANDLERS.keys()) == ROUTED_FILE_TYPES
|
|
||||||
@@ -1,225 +0,0 @@
|
|||||||
"""SP19 — Security middleware + health probe tests.
|
|
||||||
|
|
||||||
Covers each middleware in isolation (using FastAPI's TestClient with a
|
|
||||||
minimal app) and the integration into the real ``cyclone.api`` app
|
|
||||||
(headers present on every response, /api/health returns the rich
|
|
||||||
snapshot).
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from typing import Any, Callable
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from fastapi import FastAPI, Request
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
from cyclone.security import (
|
|
||||||
DEFAULT_MAX_BODY_BYTES,
|
|
||||||
DEFAULT_RATE_LIMIT_PER_MIN,
|
|
||||||
BodySizeLimitMiddleware,
|
|
||||||
RateLimitMiddleware,
|
|
||||||
SecurityHeadersMiddleware,
|
|
||||||
get_health_snapshot,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Test apps — built inline per-test to avoid state pollution between tests.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _echo_app_with(
|
|
||||||
*middlewares: Callable[..., Any],
|
|
||||||
) -> FastAPI:
|
|
||||||
"""Build a tiny FastAPI app with the given middleware chain.
|
|
||||||
|
|
||||||
``middlewares`` are listed outermost-first (the first element
|
|
||||||
runs first on the request). Starlette's ``add_middleware``
|
|
||||||
*prepends*, so we add in reverse to preserve "outermost first"
|
|
||||||
in the public API.
|
|
||||||
"""
|
|
||||||
app = FastAPI()
|
|
||||||
|
|
||||||
@app.post("/echo")
|
|
||||||
async def echo(request: Request):
|
|
||||||
body = await request.body()
|
|
||||||
return {"received_bytes": len(body)}
|
|
||||||
|
|
||||||
@app.get("/api/health")
|
|
||||||
async def h():
|
|
||||||
return {"status": "ok"}
|
|
||||||
|
|
||||||
for mw in reversed(middlewares):
|
|
||||||
app.add_middleware(mw)
|
|
||||||
return app
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# BodySizeLimitMiddleware
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_body_size_accepts_under_limit():
|
|
||||||
"""500 bytes is under the 1024-byte limit; body passes through."""
|
|
||||||
app = _echo_app_with(
|
|
||||||
SecurityHeadersMiddleware,
|
|
||||||
lambda a: BodySizeLimitMiddleware(a, max_bytes=1024),
|
|
||||||
)
|
|
||||||
client = TestClient(app)
|
|
||||||
resp = client.post("/echo", content=b"x" * 500)
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
assert resp.json() == {"received_bytes": 500}
|
|
||||||
|
|
||||||
|
|
||||||
def test_body_size_rejects_over_content_length():
|
|
||||||
"""A 200-byte body against a 100-byte cap returns 413."""
|
|
||||||
app = _echo_app_with(
|
|
||||||
SecurityHeadersMiddleware,
|
|
||||||
lambda a: BodySizeLimitMiddleware(a, max_bytes=100),
|
|
||||||
)
|
|
||||||
client = TestClient(app)
|
|
||||||
resp = client.post("/echo", content=b"x" * 200)
|
|
||||||
assert resp.status_code == 413
|
|
||||||
body = resp.json()
|
|
||||||
assert body["error"] == "body_too_large"
|
|
||||||
assert "100" in body["detail"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_body_size_default_is_50mb():
|
|
||||||
"""The default cap is 50 MB so even large prodfiles fit."""
|
|
||||||
assert DEFAULT_MAX_BODY_BYTES == 50 * 1024 * 1024
|
|
||||||
|
|
||||||
|
|
||||||
def test_body_size_rejects_bad_content_length():
|
|
||||||
"""A non-integer Content-Length is rejected as 400, not 500."""
|
|
||||||
sent: list = []
|
|
||||||
inner_app = FastAPI()
|
|
||||||
|
|
||||||
@inner_app.post("/echo")
|
|
||||||
async def echo(request: Request):
|
|
||||||
return {"ok": True}
|
|
||||||
|
|
||||||
app_instance = BodySizeLimitMiddleware(inner_app, max_bytes=100)
|
|
||||||
|
|
||||||
async def fake_receive():
|
|
||||||
return {"type": "http.request", "body": b"", "more_body": False}
|
|
||||||
|
|
||||||
async def fake_send(msg):
|
|
||||||
sent.append(msg)
|
|
||||||
|
|
||||||
scope = {
|
|
||||||
"type": "http",
|
|
||||||
"method": "POST",
|
|
||||||
"path": "/echo",
|
|
||||||
"headers": [(b"content-length", b"not-a-number")],
|
|
||||||
"query_string": b"",
|
|
||||||
}
|
|
||||||
asyncio.run(app_instance(scope, fake_receive, fake_send))
|
|
||||||
start = next(m for m in sent if m["type"] == "http.response.start")
|
|
||||||
assert start["status"] == 400
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# RateLimitMiddleware
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_rate_limit_default_is_300_per_min():
|
|
||||||
assert DEFAULT_RATE_LIMIT_PER_MIN == 300
|
|
||||||
|
|
||||||
|
|
||||||
def test_rate_limit_allows_under_threshold():
|
|
||||||
"""A few requests under the per-minute limit are allowed."""
|
|
||||||
app = _echo_app_with(
|
|
||||||
SecurityHeadersMiddleware,
|
|
||||||
lambda a: RateLimitMiddleware(a, per_minute=5),
|
|
||||||
)
|
|
||||||
client = TestClient(app)
|
|
||||||
for _ in range(5):
|
|
||||||
resp = client.post("/echo", content=b"x")
|
|
||||||
assert resp.status_code == 200, resp.text
|
|
||||||
|
|
||||||
|
|
||||||
def test_rate_limit_blocks_over_threshold():
|
|
||||||
"""The 6th request within a 60s window is rate-limited."""
|
|
||||||
app = _echo_app_with(
|
|
||||||
SecurityHeadersMiddleware,
|
|
||||||
lambda a: RateLimitMiddleware(a, per_minute=3),
|
|
||||||
)
|
|
||||||
client = TestClient(app)
|
|
||||||
for _ in range(3):
|
|
||||||
assert client.post("/echo", content=b"x").status_code == 200
|
|
||||||
resp = client.post("/echo", content=b"x")
|
|
||||||
assert resp.status_code == 429
|
|
||||||
assert resp.json()["error"] == "rate_limited"
|
|
||||||
|
|
||||||
|
|
||||||
def test_rate_limit_exempts_health_probes():
|
|
||||||
"""A load balancer hammering /api/health should not trip the limiter."""
|
|
||||||
app = _echo_app_with(
|
|
||||||
SecurityHeadersMiddleware,
|
|
||||||
lambda a: RateLimitMiddleware(a, per_minute=2),
|
|
||||||
)
|
|
||||||
client = TestClient(app)
|
|
||||||
for _ in range(20):
|
|
||||||
assert client.get("/api/health").status_code == 200
|
|
||||||
assert client.post("/echo", content=b"x").status_code == 200
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# SecurityHeadersMiddleware
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_security_headers_present_on_200():
|
|
||||||
app = _echo_app_with(SecurityHeadersMiddleware)
|
|
||||||
client = TestClient(app)
|
|
||||||
resp = client.get("/api/health")
|
|
||||||
assert resp.headers["X-Content-Type-Options"] == "nosniff"
|
|
||||||
assert resp.headers["X-Frame-Options"] == "DENY"
|
|
||||||
assert resp.headers["Referrer-Policy"] == "same-origin"
|
|
||||||
assert "default-src 'none'" in resp.headers["Content-Security-Policy"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_security_headers_present_on_error_response():
|
|
||||||
"""413/429 still carry the security headers."""
|
|
||||||
# Define a tiny factory so Starlette can introspect it as a class.
|
|
||||||
class _BoundBodySize(BodySizeLimitMiddleware):
|
|
||||||
def __init__(self, app): # type: ignore[no-untyped-def]
|
|
||||||
super().__init__(app, max_bytes=10)
|
|
||||||
|
|
||||||
app = _echo_app_with(SecurityHeadersMiddleware, _BoundBodySize)
|
|
||||||
client = TestClient(app)
|
|
||||||
resp = client.post("/echo", content=b"x" * 100)
|
|
||||||
assert resp.status_code == 413, resp.text
|
|
||||||
assert resp.headers["X-Content-Type-Options"] == "nosniff"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# get_health_snapshot
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_health_snapshot_basic_shape():
|
|
||||||
snap = get_health_snapshot()
|
|
||||||
d = snap.to_dict()
|
|
||||||
assert "status" in d
|
|
||||||
assert "version" in d
|
|
||||||
assert "db" in d
|
|
||||||
assert "scheduler" in d
|
|
||||||
assert "pubsub" in d
|
|
||||||
assert "batch" in d
|
|
||||||
|
|
||||||
|
|
||||||
def test_health_snapshot_db_ok_in_tests():
|
|
||||||
"""The conftest DB fixture is live; ``SELECT 1`` works."""
|
|
||||||
snap = get_health_snapshot()
|
|
||||||
assert snap.db.get("ok") is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_health_snapshot_handles_no_scheduler():
|
|
||||||
"""Without a configured scheduler, the snapshot reports gracefully."""
|
|
||||||
snap = get_health_snapshot()
|
|
||||||
sched = snap.scheduler
|
|
||||||
assert "running" in sched or "configured" in sched
|
|
||||||
@@ -65,21 +65,6 @@ def test_build_gs_emits_gs_segment_with_hc_functional_id():
|
|||||||
assert parts[6] == "1"
|
assert parts[6] == "1"
|
||||||
|
|
||||||
|
|
||||||
def test_build_gs_uses_gs04_yyyymmdd_8_digits():
|
|
||||||
"""GS-04 must be CCYYMMDD (8 digits) per X12; ISA uses YYMMDD (6).
|
|
||||||
|
|
||||||
A 6-digit value like '260622' is rejected by EDI validators with
|
|
||||||
'Element GS-04 must use CCYYMMDD date format'.
|
|
||||||
"""
|
|
||||||
gs = _build_gs("SENDER", "RECEIVER", "1")
|
|
||||||
parts = gs.rstrip("~").split("*")
|
|
||||||
# parts[4] = GS-04 date
|
|
||||||
assert len(parts[4]) == 8, f"expected 8-digit CCYYMMDD, got {parts[4]!r}"
|
|
||||||
# Must parse as a CCYYMMDD date
|
|
||||||
from datetime import datetime
|
|
||||||
datetime.strptime(parts[4], "%Y%m%d")
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_st_emits_837_segment():
|
def test_build_st_emits_837_segment():
|
||||||
st = _build_st("0001")
|
st = _build_st("0001")
|
||||||
assert st.startswith("ST*837*0001*005010X222A1~")
|
assert st.startswith("ST*837*0001*005010X222A1~")
|
||||||
@@ -115,15 +100,12 @@ def test_build_nm1_person_entity_splits_first_last():
|
|||||||
assert parts[4] == "Jane"
|
assert parts[4] == "Jane"
|
||||||
|
|
||||||
|
|
||||||
def test_build_per_emits_per01_even_with_no_contact():
|
def test_build_per_returns_empty_when_no_contact():
|
||||||
"""PER is required by X12 Loop 1000A — at least PER01 must be present."""
|
assert _build_per(None, None) == ""
|
||||||
per = _build_per(None, None)
|
assert _build_per("", "") == ""
|
||||||
assert per == "PER*IC~"
|
|
||||||
per = _build_per("", "")
|
|
||||||
assert per == "PER*IC~"
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_per_emits_segment_with_phone_contact():
|
def test_build_per_emits_segment_with_contact():
|
||||||
per = _build_per("Jane Doe", "5551234567")
|
per = _build_per("Jane Doe", "5551234567")
|
||||||
parts = per.rstrip("~").split("*")
|
parts = per.rstrip("~").split("*")
|
||||||
assert parts[0] == "PER"
|
assert parts[0] == "PER"
|
||||||
@@ -133,25 +115,6 @@ def test_build_per_emits_segment_with_phone_contact():
|
|||||||
assert parts[4] == "5551234567"
|
assert parts[4] == "5551234567"
|
||||||
|
|
||||||
|
|
||||||
def test_build_per_emits_segment_with_email_contact():
|
|
||||||
"""When email is given, it wins over phone (HCPF expects email)."""
|
|
||||||
per = _build_per("Tyler Martinez", None, contact_email="tyler@dzinesco.com")
|
|
||||||
parts = per.rstrip("~").split("*")
|
|
||||||
assert parts[0] == "PER"
|
|
||||||
assert parts[1] == "IC"
|
|
||||||
assert parts[2] == "Tyler Martinez"
|
|
||||||
assert parts[3] == "EM"
|
|
||||||
assert parts[4] == "tyler@dzinesco.com"
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_per_email_takes_precedence_over_phone():
|
|
||||||
"""If both phone and email are given, email is emitted (PER04)."""
|
|
||||||
per = _build_per("Tyler", "555-1234", contact_email="t@example.com")
|
|
||||||
parts = per.rstrip("~").split("*")
|
|
||||||
assert parts[3] == "EM"
|
|
||||||
assert parts[4] == "t@example.com"
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_n3_returns_empty_when_no_address():
|
def test_build_n3_returns_empty_when_no_address():
|
||||||
assert _build_n3(None, None) == ""
|
assert _build_n3(None, None) == ""
|
||||||
assert _build_n3("", "") == ""
|
assert _build_n3("", "") == ""
|
||||||
@@ -170,33 +133,12 @@ def test_build_hl_emits_segment():
|
|||||||
assert hl == "HL*1**20*1~"
|
assert hl == "HL*1**20*1~"
|
||||||
|
|
||||||
|
|
||||||
def test_build_sbr_emits_segment_with_correct_slots():
|
def test_build_sbr_emits_segment():
|
||||||
"""SBR01=Payer Responsibility Seq Code (default 'P'),
|
sbr = _build_sbr("18", "M123", "PAYER")
|
||||||
SBR02=Individual Relationship Code (e.g. '18' for self),
|
|
||||||
SBR09=Claim Filing Indicator Code (e.g. 'MC' for Medicaid)."""
|
|
||||||
sbr = _build_sbr("18", "MC")
|
|
||||||
parts = sbr.rstrip("~").split("*")
|
parts = sbr.rstrip("~").split("*")
|
||||||
assert parts[0] == "SBR"
|
assert parts[0] == "SBR"
|
||||||
# SBR01 — primary
|
assert parts[1] == "18"
|
||||||
assert parts[1] == "P"
|
assert parts[9] == "M123"
|
||||||
# SBR02 — individual relationship (self = 18)
|
|
||||||
assert parts[2] == "18"
|
|
||||||
# SBR09 — claim filing indicator
|
|
||||||
assert parts[9] == "MC"
|
|
||||||
# Member ID and payer name do NOT belong in SBR — they live in
|
|
||||||
# NM109 and NM1*PR.NM103 respectively.
|
|
||||||
assert "M123" not in sbr
|
|
||||||
assert "PAYER" not in sbr
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_sbr_defaults_relationship_to_self_and_filing_to_empty():
|
|
||||||
"""When called with all-None, SBR01/02 fall back to safe defaults
|
|
||||||
and SBR09 is left empty (the validator's R202 rule will then skip)."""
|
|
||||||
sbr = _build_sbr(None, None)
|
|
||||||
parts = sbr.rstrip("~").split("*")
|
|
||||||
assert parts[1] == "P"
|
|
||||||
assert parts[2] == "18"
|
|
||||||
assert parts[9] == ""
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_ref_returns_empty_when_no_value():
|
def test_build_ref_returns_empty_when_no_value():
|
||||||
@@ -258,34 +200,6 @@ def test_build_clm_emits_clm01_to_clm05():
|
|||||||
assert parts[5] == "11:1"
|
assert parts[5] == "11:1"
|
||||||
|
|
||||||
|
|
||||||
def test_build_clm_emits_clm08_defaulting_to_y():
|
|
||||||
"""CLM-08 (Benefits Assignment Certification) is required by X12
|
|
||||||
837P when CLM-07 = 'Y'. Default to 'Y' when the source didn't
|
|
||||||
capture one (matches what 99% of HCPF files look like).
|
|
||||||
"""
|
|
||||||
claim = _stub_claim_header()
|
|
||||||
clm = _build_clm(claim)
|
|
||||||
parts = clm.rstrip("~").split("*")
|
|
||||||
# parts[8] = CLM-08
|
|
||||||
assert parts[8] == "Y", f"CLM-08 should default to 'Y', got {parts[8]!r}"
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_clm_propagates_captured_clm08():
|
|
||||||
from cyclone.parsers.models import ClaimHeader
|
|
||||||
claim = ClaimHeader(
|
|
||||||
claim_id="CLM-1",
|
|
||||||
total_charge=Decimal("100.00"),
|
|
||||||
place_of_service="11",
|
|
||||||
frequency_code="1",
|
|
||||||
assignment="Y",
|
|
||||||
benefits_assignment_certification="N",
|
|
||||||
release_of_info="Y",
|
|
||||||
)
|
|
||||||
clm = _build_clm(claim)
|
|
||||||
parts = clm.rstrip("~").split("*")
|
|
||||||
assert parts[8] == "N"
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_ref_g1_returns_empty_when_no_prior_auth():
|
def test_build_ref_g1_returns_empty_when_no_prior_auth():
|
||||||
assert _build_ref_g1(None) == ""
|
assert _build_ref_g1(None) == ""
|
||||||
assert _build_ref_g1("") == ""
|
assert _build_ref_g1("") == ""
|
||||||
@@ -328,57 +242,6 @@ def test_build_sv1_emits_procedure_modifiers_charge_units():
|
|||||||
assert parts[4] == "1"
|
assert parts[4] == "1"
|
||||||
|
|
||||||
|
|
||||||
def test_build_sv1_emits_sv1_06_and_sv1_07_when_dx_pointer_given():
|
|
||||||
"""SV1-07 (Diagnosis Code Pointer) is required by X12/HCPF when the
|
|
||||||
parent claim has an HI segment (i.e. has at least one diagnosis).
|
|
||||||
|
|
||||||
Per 005010X222A1, SV1-06 is "Not Used" by the guide and MUST be
|
|
||||||
empty. Unit basis (UN/MJ/...) goes only in SV1-03.
|
|
||||||
"""
|
|
||||||
line = _stub_service_line()
|
|
||||||
sv1 = _build_sv1(line, dx_pointer="1")
|
|
||||||
parts = sv1.rstrip("~").split("*")
|
|
||||||
# parts layout: SV1, comp(SV1-01), charge(02), unit_basis(03),
|
|
||||||
# units(04), pos(05), ""(06 NOT USED), sv1_07(07)
|
|
||||||
assert len(parts) == 8, f"expected 8 elements, got {parts}"
|
|
||||||
# SV1-03 = unit basis
|
|
||||||
assert parts[3] == "UN"
|
|
||||||
# SV1-06 = "" (Not Used by 837P guide)
|
|
||||||
assert parts[6] == "", f"SV1-06 must be empty (Not Used by 837P), got {parts[6]!r}"
|
|
||||||
# SV1-07 = pointer
|
|
||||||
assert parts[7] == "1"
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_sv1_omits_sv1_07_when_no_dx_pointer():
|
|
||||||
"""When the claim has no HI segment, SV1-07 should be empty."""
|
|
||||||
line = _stub_service_line()
|
|
||||||
sv1 = _build_sv1(line) # no dx_pointer kwarg
|
|
||||||
parts = sv1.rstrip("~").split("*")
|
|
||||||
assert len(parts) == 8
|
|
||||||
assert parts[6] == "" # SV1-06 still empty
|
|
||||||
assert parts[7] == "" # SV1-07 empty
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_sv1_matches_goodclaim_layout():
|
|
||||||
"""Layout must match the known-good reference at docs/goodclaim.x12:
|
|
||||||
|
|
||||||
SV1*HC:T1019:U1:KX*125.40*UN*19.00***1~
|
|
||||||
|
|
||||||
i.e. 8 fields total: comp, charge, UN, units, '', '', '1'.
|
|
||||||
"""
|
|
||||||
from cyclone.parsers.models import Procedure, ServiceLine
|
|
||||||
line = ServiceLine(
|
|
||||||
line_number=1,
|
|
||||||
procedure=Procedure(qualifier="HC", code="T1019", modifiers=["U1", "KX"]),
|
|
||||||
charge=Decimal("125.40"),
|
|
||||||
units=Decimal("19.00"),
|
|
||||||
unit_type="UN",
|
|
||||||
place_of_service=None,
|
|
||||||
)
|
|
||||||
sv1 = _build_sv1(line, dx_pointer="1")
|
|
||||||
assert sv1 == "SV1*HC:T1019:U1:KX*125.40*UN*19.00***1~"
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_dtp_472_emits_service_date():
|
def test_build_dtp_472_emits_service_date():
|
||||||
assert _build_dtp_472(date(2026, 6, 15)) == "DTP*472*D8*20260615~"
|
assert _build_dtp_472(date(2026, 6, 15)) == "DTP*472*D8*20260615~"
|
||||||
|
|
||||||
@@ -506,76 +369,6 @@ def test_serialize_837_uses_custom_sender_receiver_ids():
|
|||||||
parse(text, _CFG)
|
parse(text, _CFG)
|
||||||
|
|
||||||
|
|
||||||
def test_serialize_837_emits_per_segment_in_submitter_block():
|
|
||||||
"""X12 Loop 1000A (Submitter Name) requires a PER segment after
|
|
||||||
NM1*41. The serializer must emit one even with no contact info
|
|
||||||
(PER01='IC' is the only required element)."""
|
|
||||||
claim = _load_claim()
|
|
||||||
text = serialize_837(claim)
|
|
||||||
# The first NM1*41 should be followed immediately by a PER segment.
|
|
||||||
seg_ids = [seg.split("*")[0] for seg in text.split("~") if seg]
|
|
||||||
nm1_41_idx = seg_ids.index("NM1") # first NM1 is the submitter
|
|
||||||
assert nm1_41_idx >= 0
|
|
||||||
# The very next segment must be PER (PER01='IC' is required by spec).
|
|
||||||
assert seg_ids[nm1_41_idx + 1] == "PER"
|
|
||||||
per_line = next(seg for seg in text.split("~") if seg.startswith("PER*IC"))
|
|
||||||
assert per_line.startswith("PER*IC")
|
|
||||||
|
|
||||||
|
|
||||||
def test_serialize_837_per_segment_includes_email_from_kwargs():
|
|
||||||
"""Passing submitter_contact_email should emit PER*IC*<name>*EM*<email>."""
|
|
||||||
claim = _load_claim()
|
|
||||||
text = serialize_837(
|
|
||||||
claim,
|
|
||||||
sender_id="DZINESCO",
|
|
||||||
submitter_name="Dzinesco",
|
|
||||||
submitter_contact_name="Tyler Martinez",
|
|
||||||
submitter_contact_email="tyler@dzinesco.com",
|
|
||||||
)
|
|
||||||
assert "PER*IC*Tyler Martinez*EM*tyler@dzinesco.com" in text
|
|
||||||
# And the ISA sender id should be the clearhouse TPID, not "CYCLONE".
|
|
||||||
assert "ZZ*DZINESCO" in text
|
|
||||||
assert "ZZ*CYCLONE" not in text
|
|
||||||
|
|
||||||
|
|
||||||
def test_serialize_837_sbr09_uses_claim_filing_indicator_code_kwarg():
|
|
||||||
"""SBR09 must be the claim filing indicator (e.g. 'MC' for Medicaid),
|
|
||||||
not the member id. The serializer takes it from the kwarg."""
|
|
||||||
claim = _load_claim()
|
|
||||||
text = serialize_837(claim, claim_filing_indicator_code="MC")
|
|
||||||
sbr_line = next(seg for seg in text.split("~") if seg.startswith("SBR*"))
|
|
||||||
parts = sbr_line.rstrip("~").split("*")
|
|
||||||
# SBR01 = P (primary), SBR02 = 18 (self), SBR09 = MC
|
|
||||||
assert parts[1] == "P"
|
|
||||||
assert parts[2] == "18"
|
|
||||||
assert parts[9] == "MC"
|
|
||||||
# And the member id should NOT be in SBR.
|
|
||||||
assert claim.subscriber.member_id not in sbr_line
|
|
||||||
|
|
||||||
|
|
||||||
def test_serialize_837_for_resubmit_forwards_kwargs_to_serialize_837():
|
|
||||||
"""serialize_837_for_resubmit is a thin wrapper — it must forward
|
|
||||||
clearhouse + payer kwargs so the export endpoint can use it."""
|
|
||||||
claim = _load_claim()
|
|
||||||
text = serialize_837_for_resubmit(
|
|
||||||
claim,
|
|
||||||
interchange_index=7,
|
|
||||||
sender_id="DZINESCO",
|
|
||||||
submitter_name="Dzinesco",
|
|
||||||
submitter_contact_name="Tyler Martinez",
|
|
||||||
submitter_contact_email="tyler@dzinesco.com",
|
|
||||||
receiver_id="COMEDASSISTPROG",
|
|
||||||
receiver_name="COLORADO MEDICAL ASSISTANCE PROGRAM",
|
|
||||||
claim_filing_indicator_code="MC",
|
|
||||||
)
|
|
||||||
assert "ZZ*DZINESCO" in text
|
|
||||||
assert "ZZ*COMEDASSISTPROG" in text
|
|
||||||
assert "PER*IC*Tyler Martinez*EM*tyler@dzinesco.com" in text
|
|
||||||
# Control numbers reflect the resubmit index.
|
|
||||||
isa = next(seg for seg in text.split("~") if seg.startswith("ISA*"))
|
|
||||||
assert "000000007" in isa
|
|
||||||
|
|
||||||
|
|
||||||
def test_serialize_error_is_an_exception():
|
def test_serialize_error_is_an_exception():
|
||||||
assert issubclass(SerializeError, Exception)
|
assert issubclass(SerializeError, Exception)
|
||||||
|
|
||||||
|
|||||||
@@ -31,13 +31,13 @@ def sftp_block(tmp_path):
|
|||||||
|
|
||||||
def test_stub_writes_preserving_remote_path(sftp_block, tmp_path):
|
def test_stub_writes_preserving_remote_path(sftp_block, tmp_path):
|
||||||
client = SftpClient(sftp_block)
|
client = SftpClient(sftp_block)
|
||||||
remote = "/CO XIX/PROD/coxix_prod_11525703/FromHPE/tp11525703-837P-20260620132243505-1of1.x12"
|
remote = "/CO XIX/PROD/coxix_prod_11525703/FromHPE/11525703-837P-20260620132243505-1of1.x12"
|
||||||
target = client.write_file(remote, b"ISA*00*...~IEA*1*1~")
|
target = client.write_file(remote, b"ISA*00*...~IEA*1*1~")
|
||||||
assert target.exists()
|
assert target.exists()
|
||||||
assert target.read_bytes() == b"ISA*00*...~IEA*1*1~"
|
assert target.read_bytes() == b"ISA*00*...~IEA*1*1~"
|
||||||
# Confirm the full nested MFT path is preserved under staging
|
# Confirm the full nested MFT path is preserved under staging
|
||||||
rel = target.relative_to(sftp_block.staging_dir)
|
rel = target.relative_to(sftp_block.staging_dir)
|
||||||
assert str(rel) == "CO XIX/PROD/coxix_prod_11525703/FromHPE/tp11525703-837P-20260620132243505-1of1.x12"
|
assert str(rel) == "CO XIX/PROD/coxix_prod_11525703/FromHPE/11525703-837P-20260620132243505-1of1.x12"
|
||||||
|
|
||||||
|
|
||||||
def test_stub_creates_parent_dirs(sftp_block):
|
def test_stub_creates_parent_dirs(sftp_block):
|
||||||
@@ -72,23 +72,7 @@ def test_stub_get_secret_returns_stub_when_keychain_empty(sftp_block):
|
|||||||
assert secret == "<stub-secret>"
|
assert secret == "<stub-secret>"
|
||||||
|
|
||||||
|
|
||||||
def test_stub_read_file_returns_bytes(sftp_block, tmp_path):
|
def test_stub_read_file_raises(sftp_block):
|
||||||
"""SP16: the stub read_file returns bytes from the staging dir.
|
|
||||||
|
|
||||||
Lets the inbound scheduler exercise the same code path on a
|
|
||||||
workstation without a real MFT connection.
|
|
||||||
"""
|
|
||||||
inbound = tmp_path / "staging" / "ToHPE"
|
|
||||||
inbound.mkdir(parents=True)
|
|
||||||
(inbound / "TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12").write_bytes(
|
|
||||||
b"hello-world",
|
|
||||||
)
|
|
||||||
client = SftpClient(sftp_block)
|
client = SftpClient(sftp_block)
|
||||||
body = client.read_file("/ToHPE/TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12")
|
with pytest.raises(RuntimeError, match="Stub SFTP cannot read"):
|
||||||
assert body == b"hello-world"
|
client.read_file("/x.x12")
|
||||||
|
|
||||||
|
|
||||||
def test_stub_read_file_missing_raises(sftp_block):
|
|
||||||
client = SftpClient(sftp_block)
|
|
||||||
with pytest.raises(FileNotFoundError):
|
|
||||||
client.read_file("/ToHPE/does-not-exist.x12")
|
|
||||||
|
|||||||
@@ -100,53 +100,6 @@ def test_r020_npi_must_be_ten_digits():
|
|||||||
assert any(i.rule == "R020_npi_format" for i in report.errors)
|
assert any(i.rule == "R020_npi_format" for i in report.errors)
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
|
||||||
# R021 — NPI Luhn checksum (SP20)
|
|
||||||
# --------------------------------------------------------------------------- #
|
|
||||||
|
|
||||||
|
|
||||||
def test_r021_npi_checksum_valid_passes_silently():
|
|
||||||
"""A valid Luhn NPI (CMS-published 1234567893) yields no R021 issue."""
|
|
||||||
cfg = PayerConfig.co_medicaid()
|
|
||||||
claim = _build_claim()
|
|
||||||
claim.billing_provider.npi = "1234567893"
|
|
||||||
report = validate(claim, cfg)
|
|
||||||
assert not any(i.rule == "R021_npi_checksum" for i in report.errors + report.warnings)
|
|
||||||
|
|
||||||
|
|
||||||
def test_r021_npi_checksum_bad_luhn_is_warning():
|
|
||||||
"""An NPI that passes format but fails Luhn is a WARNING, not an error."""
|
|
||||||
cfg = PayerConfig.co_medicaid()
|
|
||||||
claim = _build_claim()
|
|
||||||
claim.billing_provider.npi = "1234567890" # right length, wrong check digit
|
|
||||||
report = validate(claim, cfg)
|
|
||||||
assert report.passed is True # WARNINGs don't fail the report
|
|
||||||
assert any(i.rule == "R021_npi_checksum" and i.severity == "warning" for i in report.warnings)
|
|
||||||
|
|
||||||
|
|
||||||
def test_r021_npi_checksum_skipped_when_format_bad():
|
|
||||||
"""When R020 already flagged the format, R021 stays silent
|
|
||||||
(avoids a duplicate 'this NPI is wrong' message)."""
|
|
||||||
cfg = PayerConfig.co_medicaid()
|
|
||||||
claim = _build_claim()
|
|
||||||
claim.billing_provider.npi = "12345" # wrong length
|
|
||||||
report = validate(claim, cfg)
|
|
||||||
# R020 errors as expected; R021 stays quiet.
|
|
||||||
assert any(i.rule == "R020_npi_format" for i in report.errors)
|
|
||||||
assert not any(i.rule == "R021_npi_checksum" for i in report.errors + report.warnings)
|
|
||||||
|
|
||||||
|
|
||||||
def test_r021_npi_checksum_skipped_when_npi_missing():
|
|
||||||
"""An empty NPI doesn't trigger R021 (only R020 would, but R020
|
|
||||||
only fires when NPI is *present* and wrong). R021 must stay silent
|
|
||||||
when NPI is empty so we don't double-fire."""
|
|
||||||
cfg = PayerConfig.co_medicaid()
|
|
||||||
claim = _build_claim()
|
|
||||||
claim.billing_provider.npi = ""
|
|
||||||
report = validate(claim, cfg)
|
|
||||||
assert not any(i.rule == "R021_npi_checksum" for i in report.errors + report.warnings)
|
|
||||||
|
|
||||||
|
|
||||||
def test_r030_frequency_allowed():
|
def test_r030_frequency_allowed():
|
||||||
cfg = PayerConfig.co_medicaid() # only 1, 7, 8
|
cfg = PayerConfig.co_medicaid() # only 1, 7, 8
|
||||||
claim = _build_claim()
|
claim = _build_claim()
|
||||||
|
|||||||
Generated
+1
-464
@@ -33,85 +33,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/ba/16/9826f089383c593cdfc4a6e5aca94d9e91ae1692c57af82c3b2aa5e810f7/anyio-4.14.0-py3-none-any.whl", hash = "sha256:dd9b7a2a9799ed6552fde617b2c5df02b7fdd7d88392fc48101e51bae46164d9", size = 123506, upload-time = "2026-06-15T22:00:47.595Z" },
|
{ url = "https://files.pythonhosted.org/packages/ba/16/9826f089383c593cdfc4a6e5aca94d9e91ae1692c57af82c3b2aa5e810f7/anyio-4.14.0-py3-none-any.whl", hash = "sha256:dd9b7a2a9799ed6552fde617b2c5df02b7fdd7d88392fc48101e51bae46164d9", size = 123506, upload-time = "2026-06-15T22:00:47.595Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "backports-tarfile"
|
|
||||||
version = "1.2.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/86/72/cd9b395f25e290e633655a100af28cb253e4393396264a98bd5f5951d50f/backports_tarfile-1.2.0.tar.gz", hash = "sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991", size = 86406, upload-time = "2024-05-28T17:01:54.731Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b9/fa/123043af240e49752f1c4bd24da5053b6bd00cad78c2be53c0d1e8b975bc/backports.tarfile-1.2.0-py3-none-any.whl", hash = "sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34", size = 30181, upload-time = "2024-05-28T17:01:53.112Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "bcrypt"
|
|
||||||
version = "5.0.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/d4/36/3329e2518d70ad8e2e5817d5a4cac6bba05a47767ec416c7d020a965f408/bcrypt-5.0.0.tar.gz", hash = "sha256:f748f7c2d6fd375cc93d3fba7ef4a9e3a092421b8dbf34d8d4dc06be9492dfdd", size = 25386, upload-time = "2025-09-25T19:50:47.829Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/13/85/3e65e01985fddf25b64ca67275bb5bdb4040bd1a53b66d355c6c37c8a680/bcrypt-5.0.0-cp313-cp313t-macosx_10_12_universal2.whl", hash = "sha256:f3c08197f3039bec79cee59a606d62b96b16669cff3949f21e74796b6e3cd2be", size = 481806, upload-time = "2025-09-25T19:49:05.102Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/44/dc/01eb79f12b177017a726cbf78330eb0eb442fae0e7b3dfd84ea2849552f3/bcrypt-5.0.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:200af71bc25f22006f4069060c88ed36f8aa4ff7f53e67ff04d2ab3f1e79a5b2", size = 268626, upload-time = "2025-09-25T19:49:06.723Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/8c/cf/e82388ad5959c40d6afd94fb4743cc077129d45b952d46bdc3180310e2df/bcrypt-5.0.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:baade0a5657654c2984468efb7d6c110db87ea63ef5a4b54732e7e337253e44f", size = 271853, upload-time = "2025-09-25T19:49:08.028Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ec/86/7134b9dae7cf0efa85671651341f6afa695857fae172615e960fb6a466fa/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:c58b56cdfb03202b3bcc9fd8daee8e8e9b6d7e3163aa97c631dfcfcc24d36c86", size = 269793, upload-time = "2025-09-25T19:49:09.727Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/cc/82/6296688ac1b9e503d034e7d0614d56e80c5d1a08402ff856a4549cb59207/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:4bfd2a34de661f34d0bda43c3e4e79df586e4716ef401fe31ea39d69d581ef23", size = 289930, upload-time = "2025-09-25T19:49:11.204Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d1/18/884a44aa47f2a3b88dd09bc05a1e40b57878ecd111d17e5bba6f09f8bb77/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:ed2e1365e31fc73f1825fa830f1c8f8917ca1b3ca6185773b349c20fd606cec2", size = 272194, upload-time = "2025-09-25T19:49:12.524Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/0e/8f/371a3ab33c6982070b674f1788e05b656cfbf5685894acbfef0c65483a59/bcrypt-5.0.0-cp313-cp313t-manylinux_2_34_aarch64.whl", hash = "sha256:83e787d7a84dbbfba6f250dd7a5efd689e935f03dd83b0f919d39349e1f23f83", size = 269381, upload-time = "2025-09-25T19:49:14.308Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b1/34/7e4e6abb7a8778db6422e88b1f06eb07c47682313997ee8a8f9352e5a6f1/bcrypt-5.0.0-cp313-cp313t-manylinux_2_34_x86_64.whl", hash = "sha256:137c5156524328a24b9fac1cb5db0ba618bc97d11970b39184c1d87dc4bf1746", size = 271750, upload-time = "2025-09-25T19:49:15.584Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c0/1b/54f416be2499bd72123c70d98d36c6cd61a4e33d9b89562c22481c81bb30/bcrypt-5.0.0-cp313-cp313t-musllinux_1_1_aarch64.whl", hash = "sha256:38cac74101777a6a7d3b3e3cfefa57089b5ada650dce2baf0cbdd9d65db22a9e", size = 303757, upload-time = "2025-09-25T19:49:17.244Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/13/62/062c24c7bcf9d2826a1a843d0d605c65a755bc98002923d01fd61270705a/bcrypt-5.0.0-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:d8d65b564ec849643d9f7ea05c6d9f0cd7ca23bdd4ac0c2dbef1104ab504543d", size = 306740, upload-time = "2025-09-25T19:49:18.693Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d5/c8/1fdbfc8c0f20875b6b4020f3c7dc447b8de60aa0be5faaf009d24242aec9/bcrypt-5.0.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:741449132f64b3524e95cd30e5cd3343006ce146088f074f31ab26b94e6c75ba", size = 334197, upload-time = "2025-09-25T19:49:20.523Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a6/c1/8b84545382d75bef226fbc6588af0f7b7d095f7cd6a670b42a86243183cd/bcrypt-5.0.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:212139484ab3207b1f0c00633d3be92fef3c5f0af17cad155679d03ff2ee1e41", size = 352974, upload-time = "2025-09-25T19:49:22.254Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/10/a6/ffb49d4254ed085e62e3e5dd05982b4393e32fe1e49bb1130186617c29cd/bcrypt-5.0.0-cp313-cp313t-win32.whl", hash = "sha256:9d52ed507c2488eddd6a95bccee4e808d3234fa78dd370e24bac65a21212b861", size = 148498, upload-time = "2025-09-25T19:49:24.134Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/48/a9/259559edc85258b6d5fc5471a62a3299a6aa37a6611a169756bf4689323c/bcrypt-5.0.0-cp313-cp313t-win_amd64.whl", hash = "sha256:f6984a24db30548fd39a44360532898c33528b74aedf81c26cf29c51ee47057e", size = 145853, upload-time = "2025-09-25T19:49:25.702Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2d/df/9714173403c7e8b245acf8e4be8876aac64a209d1b392af457c79e60492e/bcrypt-5.0.0-cp313-cp313t-win_arm64.whl", hash = "sha256:9fffdb387abe6aa775af36ef16f55e318dcda4194ddbf82007a6f21da29de8f5", size = 139626, upload-time = "2025-09-25T19:49:26.928Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f8/14/c18006f91816606a4abe294ccc5d1e6f0e42304df5a33710e9e8e95416e1/bcrypt-5.0.0-cp314-cp314t-macosx_10_12_universal2.whl", hash = "sha256:4870a52610537037adb382444fefd3706d96d663ac44cbb2f37e3919dca3d7ef", size = 481862, upload-time = "2025-09-25T19:49:28.365Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/67/49/dd074d831f00e589537e07a0725cf0e220d1f0d5d8e85ad5bbff251c45aa/bcrypt-5.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:48f753100931605686f74e27a7b49238122aa761a9aefe9373265b8b7aa43ea4", size = 268544, upload-time = "2025-09-25T19:49:30.39Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f5/91/50ccba088b8c474545b034a1424d05195d9fcbaaf802ab8bfe2be5a4e0d7/bcrypt-5.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f70aadb7a809305226daedf75d90379c397b094755a710d7014b8b117df1ebbf", size = 271787, upload-time = "2025-09-25T19:49:32.144Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/aa/e7/d7dba133e02abcda3b52087a7eea8c0d4f64d3e593b4fffc10c31b7061f3/bcrypt-5.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:744d3c6b164caa658adcb72cb8cc9ad9b4b75c7db507ab4bc2480474a51989da", size = 269753, upload-time = "2025-09-25T19:49:33.885Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/33/fc/5b145673c4b8d01018307b5c2c1fc87a6f5a436f0ad56607aee389de8ee3/bcrypt-5.0.0-cp314-cp314t-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a28bc05039bdf3289d757f49d616ab3efe8cf40d8e8001ccdd621cd4f98f4fc9", size = 289587, upload-time = "2025-09-25T19:49:35.144Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/27/d7/1ff22703ec6d4f90e62f1a5654b8867ef96bafb8e8102c2288333e1a6ca6/bcrypt-5.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:7f277a4b3390ab4bebe597800a90da0edae882c6196d3038a73adf446c4f969f", size = 272178, upload-time = "2025-09-25T19:49:36.793Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c8/88/815b6d558a1e4d40ece04a2f84865b0fef233513bd85fd0e40c294272d62/bcrypt-5.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:79cfa161eda8d2ddf29acad370356b47f02387153b11d46042e93a0a95127493", size = 269295, upload-time = "2025-09-25T19:49:38.164Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/51/8c/e0db387c79ab4931fc89827d37608c31cc57b6edc08ccd2386139028dc0d/bcrypt-5.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:a5393eae5722bcef046a990b84dff02b954904c36a194f6cfc817d7dca6c6f0b", size = 271700, upload-time = "2025-09-25T19:49:39.917Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/06/83/1570edddd150f572dbe9fc00f6203a89fc7d4226821f67328a85c330f239/bcrypt-5.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7f4c94dec1b5ab5d522750cb059bb9409ea8872d4494fd152b53cca99f1ddd8c", size = 334034, upload-time = "2025-09-25T19:49:41.227Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c9/f2/ea64e51a65e56ae7a8a4ec236c2bfbdd4b23008abd50ac33fbb2d1d15424/bcrypt-5.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:0cae4cb350934dfd74c020525eeae0a5f79257e8a201c0c176f4b84fdbf2a4b4", size = 352766, upload-time = "2025-09-25T19:49:43.08Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d7/d4/1a388d21ee66876f27d1a1f41287897d0c0f1712ef97d395d708ba93004c/bcrypt-5.0.0-cp314-cp314t-win32.whl", hash = "sha256:b17366316c654e1ad0306a6858e189fc835eca39f7eb2cafd6aaca8ce0c40a2e", size = 152449, upload-time = "2025-09-25T19:49:44.971Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3f/61/3291c2243ae0229e5bca5d19f4032cecad5dfb05a2557169d3a69dc0ba91/bcrypt-5.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:92864f54fb48b4c718fc92a32825d0e42265a627f956bc0361fe869f1adc3e7d", size = 149310, upload-time = "2025-09-25T19:49:46.162Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3e/89/4b01c52ae0c1a681d4021e5dd3e45b111a8fb47254a274fa9a378d8d834b/bcrypt-5.0.0-cp314-cp314t-win_arm64.whl", hash = "sha256:dd19cf5184a90c873009244586396a6a884d591a5323f0e8a5922560718d4993", size = 143761, upload-time = "2025-09-25T19:49:47.345Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/84/29/6237f151fbfe295fe3e074ecc6d44228faa1e842a81f6d34a02937ee1736/bcrypt-5.0.0-cp38-abi3-macosx_10_12_universal2.whl", hash = "sha256:fc746432b951e92b58317af8e0ca746efe93e66555f1b40888865ef5bf56446b", size = 494553, upload-time = "2025-09-25T19:49:49.006Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/45/b6/4c1205dde5e464ea3bd88e8742e19f899c16fa8916fb8510a851fae985b5/bcrypt-5.0.0-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c2388ca94ffee269b6038d48747f4ce8df0ffbea43f31abfa18ac72f0218effb", size = 275009, upload-time = "2025-09-25T19:49:50.581Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3b/71/427945e6ead72ccffe77894b2655b695ccf14ae1866cd977e185d606dd2f/bcrypt-5.0.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:560ddb6ec730386e7b3b26b8b4c88197aaed924430e7b74666a586ac997249ef", size = 278029, upload-time = "2025-09-25T19:49:52.533Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/17/72/c344825e3b83c5389a369c8a8e58ffe1480b8a699f46c127c34580c4666b/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:d79e5c65dcc9af213594d6f7f1fa2c98ad3fc10431e7aa53c176b441943efbdd", size = 275907, upload-time = "2025-09-25T19:49:54.709Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/0b/7e/d4e47d2df1641a36d1212e5c0514f5291e1a956a7749f1e595c07a972038/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2b732e7d388fa22d48920baa267ba5d97cca38070b69c0e2d37087b381c681fd", size = 296500, upload-time = "2025-09-25T19:49:56.013Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/0f/c3/0ae57a68be2039287ec28bc463b82e4b8dc23f9d12c0be331f4782e19108/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:0c8e093ea2532601a6f686edbc2c6b2ec24131ff5c52f7610dd64fa4553b5464", size = 278412, upload-time = "2025-09-25T19:49:57.356Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/45/2b/77424511adb11e6a99e3a00dcc7745034bee89036ad7d7e255a7e47be7d8/bcrypt-5.0.0-cp38-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:5b1589f4839a0899c146e8892efe320c0fa096568abd9b95593efac50a87cb75", size = 275486, upload-time = "2025-09-25T19:49:59.116Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/43/0a/405c753f6158e0f3f14b00b462d8bca31296f7ecfc8fc8bc7919c0c7d73a/bcrypt-5.0.0-cp38-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:89042e61b5e808b67daf24a434d89bab164d4de1746b37a8d173b6b14f3db9ff", size = 277940, upload-time = "2025-09-25T19:50:00.869Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/62/83/b3efc285d4aadc1fa83db385ec64dcfa1707e890eb42f03b127d66ac1b7b/bcrypt-5.0.0-cp38-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:e3cf5b2560c7b5a142286f69bde914494b6d8f901aaa71e453078388a50881c4", size = 310776, upload-time = "2025-09-25T19:50:02.393Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/95/7d/47ee337dacecde6d234890fe929936cb03ebc4c3a7460854bbd9c97780b8/bcrypt-5.0.0-cp38-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:f632fd56fc4e61564f78b46a2269153122db34988e78b6be8b32d28507b7eaeb", size = 312922, upload-time = "2025-09-25T19:50:04.232Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d6/3a/43d494dfb728f55f4e1cf8fd435d50c16a2d75493225b54c8d06122523c6/bcrypt-5.0.0-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:801cad5ccb6b87d1b430f183269b94c24f248dddbbc5c1f78b6ed231743e001c", size = 341367, upload-time = "2025-09-25T19:50:05.559Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/55/ab/a0727a4547e383e2e22a630e0f908113db37904f58719dc48d4622139b5c/bcrypt-5.0.0-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:3cf67a804fc66fc217e6914a5635000259fbbbb12e78a99488e4d5ba445a71eb", size = 359187, upload-time = "2025-09-25T19:50:06.916Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/1b/bb/461f352fdca663524b4643d8b09e8435b4990f17fbf4fea6bc2a90aa0cc7/bcrypt-5.0.0-cp38-abi3-win32.whl", hash = "sha256:3abeb543874b2c0524ff40c57a4e14e5d3a66ff33fb423529c88f180fd756538", size = 153752, upload-time = "2025-09-25T19:50:08.515Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/41/aa/4190e60921927b7056820291f56fc57d00d04757c8b316b2d3c0d1d6da2c/bcrypt-5.0.0-cp38-abi3-win_amd64.whl", hash = "sha256:35a77ec55b541e5e583eb3436ffbbf53b0ffa1fa16ca6782279daf95d146dcd9", size = 150881, upload-time = "2025-09-25T19:50:09.742Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/54/12/cd77221719d0b39ac0b55dbd39358db1cd1246e0282e104366ebbfb8266a/bcrypt-5.0.0-cp38-abi3-win_arm64.whl", hash = "sha256:cde08734f12c6a4e28dc6755cd11d3bdfea608d93d958fffbe95a7026ebe4980", size = 144931, upload-time = "2025-09-25T19:50:11.016Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/5d/ba/2af136406e1c3839aea9ecadc2f6be2bcd1eff255bd451dd39bcf302c47a/bcrypt-5.0.0-cp39-abi3-macosx_10_12_universal2.whl", hash = "sha256:0c418ca99fd47e9c59a301744d63328f17798b5947b0f791e9af3c1c499c2d0a", size = 495313, upload-time = "2025-09-25T19:50:12.309Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ac/ee/2f4985dbad090ace5ad1f7dd8ff94477fe089b5fab2040bd784a3d5f187b/bcrypt-5.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ddb4e1500f6efdd402218ffe34d040a1196c072e07929b9820f363a1fd1f4191", size = 275290, upload-time = "2025-09-25T19:50:13.673Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e4/6e/b77ade812672d15cf50842e167eead80ac3514f3beacac8902915417f8b7/bcrypt-5.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7aeef54b60ceddb6f30ee3db090351ecf0d40ec6e2abf41430997407a46d2254", size = 278253, upload-time = "2025-09-25T19:50:15.089Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/36/c4/ed00ed32f1040f7990dac7115f82273e3c03da1e1a1587a778d8cea496d8/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:f0ce778135f60799d89c9693b9b398819d15f1921ba15fe719acb3178215a7db", size = 276084, upload-time = "2025-09-25T19:50:16.699Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e7/c4/fa6e16145e145e87f1fa351bbd54b429354fd72145cd3d4e0c5157cf4c70/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a71f70ee269671460b37a449f5ff26982a6f2ba493b3eabdd687b4bf35f875ac", size = 297185, upload-time = "2025-09-25T19:50:18.525Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/24/b4/11f8a31d8b67cca3371e046db49baa7c0594d71eb40ac8121e2fc0888db0/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:f8429e1c410b4073944f03bd778a9e066e7fad723564a52ff91841d278dfc822", size = 278656, upload-time = "2025-09-25T19:50:19.809Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ac/31/79f11865f8078e192847d2cb526e3fa27c200933c982c5b2869720fa5fce/bcrypt-5.0.0-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:edfcdcedd0d0f05850c52ba3127b1fce70b9f89e0fe5ff16517df7e81fa3cbb8", size = 275662, upload-time = "2025-09-25T19:50:21.567Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d4/8d/5e43d9584b3b3591a6f9b68f755a4da879a59712981ef5ad2a0ac1379f7a/bcrypt-5.0.0-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:611f0a17aa4a25a69362dcc299fda5c8a3d4f160e2abb3831041feb77393a14a", size = 278240, upload-time = "2025-09-25T19:50:23.305Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/89/48/44590e3fc158620f680a978aafe8f87a4c4320da81ed11552f0323aa9a57/bcrypt-5.0.0-cp39-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:db99dca3b1fdc3db87d7c57eac0c82281242d1eabf19dcb8a6b10eb29a2e72d1", size = 311152, upload-time = "2025-09-25T19:50:24.597Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/5f/85/e4fbfc46f14f47b0d20493669a625da5827d07e8a88ee460af6cd9768b44/bcrypt-5.0.0-cp39-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:5feebf85a9cefda32966d8171f5db7e3ba964b77fdfe31919622256f80f9cf42", size = 313284, upload-time = "2025-09-25T19:50:26.268Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/25/ae/479f81d3f4594456a01ea2f05b132a519eff9ab5768a70430fa1132384b1/bcrypt-5.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:3ca8a166b1140436e058298a34d88032ab62f15aae1c598580333dc21d27ef10", size = 341643, upload-time = "2025-09-25T19:50:28.02Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/df/d2/36a086dee1473b14276cd6ea7f61aef3b2648710b5d7f1c9e032c29b859f/bcrypt-5.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:61afc381250c3182d9078551e3ac3a41da14154fbff647ddf52a769f588c4172", size = 359698, upload-time = "2025-09-25T19:50:31.347Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c0/f6/688d2cd64bfd0b14d805ddb8a565e11ca1fb0fd6817175d58b10052b6d88/bcrypt-5.0.0-cp39-abi3-win32.whl", hash = "sha256:64d7ce196203e468c457c37ec22390f1a61c85c6f0b8160fd752940ccfb3a683", size = 153725, upload-time = "2025-09-25T19:50:34.384Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/9f/b9/9d9a641194a730bda138b3dfe53f584d61c58cd5230e37566e83ec2ffa0d/bcrypt-5.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:64ee8434b0da054d830fa8e89e1c8bf30061d539044a39524ff7dec90481e5c2", size = 150912, upload-time = "2025-09-25T19:50:35.69Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/27/44/d2ef5e87509158ad2187f4dd0852df80695bb1ee0cfe0a684727b01a69e0/bcrypt-5.0.0-cp39-abi3-win_arm64.whl", hash = "sha256:f2347d3534e76bf50bca5500989d6c1d05ed64b440408057a37673282c654927", size = 144953, upload-time = "2025-09-25T19:50:37.32Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/8a/75/4aa9f5a4d40d762892066ba1046000b329c7cd58e888a6db878019b282dc/bcrypt-5.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:7edda91d5ab52b15636d9c30da87d2cc84f426c72b9dba7a9b4fe142ba11f534", size = 271180, upload-time = "2025-09-25T19:50:38.575Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/54/79/875f9558179573d40a9cc743038ac2bf67dfb79cecb1e8b5d70e88c94c3d/bcrypt-5.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:046ad6db88edb3c5ece4369af997938fb1c19d6a699b9c1b27b0db432faae4c4", size = 273791, upload-time = "2025-09-25T19:50:39.913Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/bc/fe/975adb8c216174bf70fc17535f75e85ac06ed5252ea077be10d9cff5ce24/bcrypt-5.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:dcd58e2b3a908b5ecc9b9df2f0085592506ac2d5110786018ee5e160f28e0911", size = 270746, upload-time = "2025-09-25T19:50:43.306Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e4/f8/972c96f5a2b6c4b3deca57009d93e946bbdbe2241dca9806d502f29dd3ee/bcrypt-5.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:6b8f520b61e8781efee73cba14e3e8c9556ccfb375623f4f97429544734545b4", size = 273375, upload-time = "2025-09-25T19:50:45.43Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "certifi"
|
name = "certifi"
|
||||||
version = "2026.6.17"
|
version = "2026.6.17"
|
||||||
@@ -121,76 +42,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/ef/2f/c5464532e965badff2f4c4c1a3a83f5697f0d7c407ed0cda44aaa99bb451/certifi-2026.6.17-py3-none-any.whl", hash = "sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db", size = 133289, upload-time = "2026-06-17T10:31:06.348Z" },
|
{ url = "https://files.pythonhosted.org/packages/ef/2f/c5464532e965badff2f4c4c1a3a83f5697f0d7c407ed0cda44aaa99bb451/certifi-2026.6.17-py3-none-any.whl", hash = "sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db", size = 133289, upload-time = "2026-06-17T10:31:06.348Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "cffi"
|
|
||||||
version = "2.0.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "pycparser", marker = "implementation_name != 'PyPy'" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/eb/56/b1ba7935a17738ae8453301356628e8147c79dbb825bcbc73dc7401f9846/cffi-2.0.0.tar.gz", hash = "sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529", size = 523588, upload-time = "2025-09-08T23:24:04.541Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/12/4a/3dfd5f7850cbf0d06dc84ba9aa00db766b52ca38d8b86e3a38314d52498c/cffi-2.0.0-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe", size = 184344, upload-time = "2025-09-08T23:22:26.456Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/4f/8b/f0e4c441227ba756aafbe78f117485b25bb26b1c059d01f137fa6d14896b/cffi-2.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c", size = 180560, upload-time = "2025-09-08T23:22:28.197Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b1/b7/1200d354378ef52ec227395d95c2576330fd22a869f7a70e88e1447eb234/cffi-2.0.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92", size = 209613, upload-time = "2025-09-08T23:22:29.475Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b8/56/6033f5e86e8cc9bb629f0077ba71679508bdf54a9a5e112a3c0b91870332/cffi-2.0.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93", size = 216476, upload-time = "2025-09-08T23:22:31.063Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/dc/7f/55fecd70f7ece178db2f26128ec41430d8720f2d12ca97bf8f0a628207d5/cffi-2.0.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5", size = 203374, upload-time = "2025-09-08T23:22:32.507Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/84/ef/a7b77c8bdc0f77adc3b46888f1ad54be8f3b7821697a7b89126e829e676a/cffi-2.0.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664", size = 202597, upload-time = "2025-09-08T23:22:34.132Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d7/91/500d892b2bf36529a75b77958edfcd5ad8e2ce4064ce2ecfeab2125d72d1/cffi-2.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26", size = 215574, upload-time = "2025-09-08T23:22:35.443Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/44/64/58f6255b62b101093d5df22dcb752596066c7e89dd725e0afaed242a61be/cffi-2.0.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9", size = 218971, upload-time = "2025-09-08T23:22:36.805Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ab/49/fa72cebe2fd8a55fbe14956f9970fe8eb1ac59e5df042f603ef7c8ba0adc/cffi-2.0.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414", size = 211972, upload-time = "2025-09-08T23:22:38.436Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/0b/28/dd0967a76aab36731b6ebfe64dec4e981aff7e0608f60c2d46b46982607d/cffi-2.0.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743", size = 217078, upload-time = "2025-09-08T23:22:39.776Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2b/c0/015b25184413d7ab0a410775fdb4a50fca20f5589b5dab1dbbfa3baad8ce/cffi-2.0.0-cp311-cp311-win32.whl", hash = "sha256:c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5", size = 172076, upload-time = "2025-09-08T23:22:40.95Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ae/8f/dc5531155e7070361eb1b7e4c1a9d896d0cb21c49f807a6c03fd63fc877e/cffi-2.0.0-cp311-cp311-win_amd64.whl", hash = "sha256:66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5", size = 182820, upload-time = "2025-09-08T23:22:42.463Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/95/5c/1b493356429f9aecfd56bc171285a4c4ac8697f76e9bbbbb105e537853a1/cffi-2.0.0-cp311-cp311-win_arm64.whl", hash = "sha256:c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d", size = 177635, upload-time = "2025-09-08T23:22:43.623Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ea/47/4f61023ea636104d4f16ab488e268b93008c3d0bb76893b1b31db1f96802/cffi-2.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d", size = 185271, upload-time = "2025-09-08T23:22:44.795Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/df/a2/781b623f57358e360d62cdd7a8c681f074a71d445418a776eef0aadb4ab4/cffi-2.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c", size = 181048, upload-time = "2025-09-08T23:22:45.938Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ff/df/a4f0fbd47331ceeba3d37c2e51e9dfc9722498becbeec2bd8bc856c9538a/cffi-2.0.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe", size = 212529, upload-time = "2025-09-08T23:22:47.349Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d5/72/12b5f8d3865bf0f87cf1404d8c374e7487dcf097a1c91c436e72e6badd83/cffi-2.0.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062", size = 220097, upload-time = "2025-09-08T23:22:48.677Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c2/95/7a135d52a50dfa7c882ab0ac17e8dc11cec9d55d2c18dda414c051c5e69e/cffi-2.0.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e", size = 207983, upload-time = "2025-09-08T23:22:50.06Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3a/c8/15cb9ada8895957ea171c62dc78ff3e99159ee7adb13c0123c001a2546c1/cffi-2.0.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037", size = 206519, upload-time = "2025-09-08T23:22:51.364Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/78/2d/7fa73dfa841b5ac06c7b8855cfc18622132e365f5b81d02230333ff26e9e/cffi-2.0.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba", size = 219572, upload-time = "2025-09-08T23:22:52.902Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/07/e0/267e57e387b4ca276b90f0434ff88b2c2241ad72b16d31836adddfd6031b/cffi-2.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94", size = 222963, upload-time = "2025-09-08T23:22:54.518Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b6/75/1f2747525e06f53efbd878f4d03bac5b859cbc11c633d0fb81432d98a795/cffi-2.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187", size = 221361, upload-time = "2025-09-08T23:22:55.867Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/7b/2b/2b6435f76bfeb6bbf055596976da087377ede68df465419d192acf00c437/cffi-2.0.0-cp312-cp312-win32.whl", hash = "sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18", size = 172932, upload-time = "2025-09-08T23:22:57.188Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f8/ed/13bd4418627013bec4ed6e54283b1959cf6db888048c7cf4b4c3b5b36002/cffi-2.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5", size = 183557, upload-time = "2025-09-08T23:22:58.351Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/95/31/9f7f93ad2f8eff1dbc1c3656d7ca5bfd8fb52c9d786b4dcf19b2d02217fa/cffi-2.0.0-cp312-cp312-win_arm64.whl", hash = "sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6", size = 177762, upload-time = "2025-09-08T23:22:59.668Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/4b/8d/a0a47a0c9e413a658623d014e91e74a50cdd2c423f7ccfd44086ef767f90/cffi-2.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb", size = 185230, upload-time = "2025-09-08T23:23:00.879Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/4a/d2/a6c0296814556c68ee32009d9c2ad4f85f2707cdecfd7727951ec228005d/cffi-2.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca", size = 181043, upload-time = "2025-09-08T23:23:02.231Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b0/1e/d22cc63332bd59b06481ceaac49d6c507598642e2230f201649058a7e704/cffi-2.0.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b", size = 212446, upload-time = "2025-09-08T23:23:03.472Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a9/f5/a2c23eb03b61a0b8747f211eb716446c826ad66818ddc7810cc2cc19b3f2/cffi-2.0.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b", size = 220101, upload-time = "2025-09-08T23:23:04.792Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f2/7f/e6647792fc5850d634695bc0e6ab4111ae88e89981d35ac269956605feba/cffi-2.0.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2", size = 207948, upload-time = "2025-09-08T23:23:06.127Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/cb/1e/a5a1bd6f1fb30f22573f76533de12a00bf274abcdc55c8edab639078abb6/cffi-2.0.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3", size = 206422, upload-time = "2025-09-08T23:23:07.753Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/98/df/0a1755e750013a2081e863e7cd37e0cdd02664372c754e5560099eb7aa44/cffi-2.0.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26", size = 219499, upload-time = "2025-09-08T23:23:09.648Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/50/e1/a969e687fcf9ea58e6e2a928ad5e2dd88cc12f6f0ab477e9971f2309b57c/cffi-2.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c", size = 222928, upload-time = "2025-09-08T23:23:10.928Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/36/54/0362578dd2c9e557a28ac77698ed67323ed5b9775ca9d3fe73fe191bb5d8/cffi-2.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b", size = 221302, upload-time = "2025-09-08T23:23:12.42Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/eb/6d/bf9bda840d5f1dfdbf0feca87fbdb64a918a69bca42cfa0ba7b137c48cb8/cffi-2.0.0-cp313-cp313-win32.whl", hash = "sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27", size = 172909, upload-time = "2025-09-08T23:23:14.32Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/37/18/6519e1ee6f5a1e579e04b9ddb6f1676c17368a7aba48299c3759bbc3c8b3/cffi-2.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75", size = 183402, upload-time = "2025-09-08T23:23:15.535Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/cb/0e/02ceeec9a7d6ee63bb596121c2c8e9b3a9e150936f4fbef6ca1943e6137c/cffi-2.0.0-cp313-cp313-win_arm64.whl", hash = "sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91", size = 177780, upload-time = "2025-09-08T23:23:16.761Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/92/c4/3ce07396253a83250ee98564f8d7e9789fab8e58858f35d07a9a2c78de9f/cffi-2.0.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5", size = 185320, upload-time = "2025-09-08T23:23:18.087Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/59/dd/27e9fa567a23931c838c6b02d0764611c62290062a6d4e8ff7863daf9730/cffi-2.0.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13", size = 181487, upload-time = "2025-09-08T23:23:19.622Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d6/43/0e822876f87ea8a4ef95442c3d766a06a51fc5298823f884ef87aaad168c/cffi-2.0.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b", size = 220049, upload-time = "2025-09-08T23:23:20.853Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b4/89/76799151d9c2d2d1ead63c2429da9ea9d7aac304603de0c6e8764e6e8e70/cffi-2.0.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c", size = 207793, upload-time = "2025-09-08T23:23:22.08Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/bb/dd/3465b14bb9e24ee24cb88c9e3730f6de63111fffe513492bf8c808a3547e/cffi-2.0.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef", size = 206300, upload-time = "2025-09-08T23:23:23.314Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/47/d9/d83e293854571c877a92da46fdec39158f8d7e68da75bf73581225d28e90/cffi-2.0.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775", size = 219244, upload-time = "2025-09-08T23:23:24.541Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2b/0f/1f177e3683aead2bb00f7679a16451d302c436b5cbf2505f0ea8146ef59e/cffi-2.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205", size = 222828, upload-time = "2025-09-08T23:23:26.143Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c6/0f/cafacebd4b040e3119dcb32fed8bdef8dfe94da653155f9d0b9dc660166e/cffi-2.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1", size = 220926, upload-time = "2025-09-08T23:23:27.873Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3e/aa/df335faa45b395396fcbc03de2dfcab242cd61a9900e914fe682a59170b1/cffi-2.0.0-cp314-cp314-win32.whl", hash = "sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f", size = 175328, upload-time = "2025-09-08T23:23:44.61Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/bb/92/882c2d30831744296ce713f0feb4c1cd30f346ef747b530b5318715cc367/cffi-2.0.0-cp314-cp314-win_amd64.whl", hash = "sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25", size = 185650, upload-time = "2025-09-08T23:23:45.848Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/9f/2c/98ece204b9d35a7366b5b2c6539c350313ca13932143e79dc133ba757104/cffi-2.0.0-cp314-cp314-win_arm64.whl", hash = "sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad", size = 180687, upload-time = "2025-09-08T23:23:47.105Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3e/61/c768e4d548bfa607abcda77423448df8c471f25dbe64fb2ef6d555eae006/cffi-2.0.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9", size = 188773, upload-time = "2025-09-08T23:23:29.347Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2c/ea/5f76bce7cf6fcd0ab1a1058b5af899bfbef198bea4d5686da88471ea0336/cffi-2.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d", size = 185013, upload-time = "2025-09-08T23:23:30.63Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/be/b4/c56878d0d1755cf9caa54ba71e5d049479c52f9e4afc230f06822162ab2f/cffi-2.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c", size = 221593, upload-time = "2025-09-08T23:23:31.91Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e0/0d/eb704606dfe8033e7128df5e90fee946bbcb64a04fcdaa97321309004000/cffi-2.0.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8", size = 209354, upload-time = "2025-09-08T23:23:33.214Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d8/19/3c435d727b368ca475fb8742ab97c9cb13a0de600ce86f62eab7fa3eea60/cffi-2.0.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc", size = 208480, upload-time = "2025-09-08T23:23:34.495Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d0/44/681604464ed9541673e486521497406fadcc15b5217c3e326b061696899a/cffi-2.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592", size = 221584, upload-time = "2025-09-08T23:23:36.096Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/25/8e/342a504ff018a2825d395d44d63a767dd8ebc927ebda557fecdaca3ac33a/cffi-2.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512", size = 224443, upload-time = "2025-09-08T23:23:37.328Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e1/5e/b666bacbbc60fbf415ba9988324a132c9a7a0448a9a8f125074671c0f2c3/cffi-2.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4", size = 223437, upload-time = "2025-09-08T23:23:38.945Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a0/1d/ec1a60bd1a10daa292d3cd6bb0b359a81607154fb8165f3ec95fe003b85c/cffi-2.0.0-cp314-cp314t-win32.whl", hash = "sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e", size = 180487, upload-time = "2025-09-08T23:23:40.423Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/bf/41/4c1168c74fac325c0c8156f04b6749c8b6a8f405bbf91413ba088359f60d/cffi-2.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6", size = 191726, upload-time = "2025-09-08T23:23:41.742Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ae/3a/dbeec9d1ee0844c679f6bb5d6ad4e9f198b1224f4e7a32825f47f6192b0c/cffi-2.0.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9", size = 184195, upload-time = "2025-09-08T23:23:43.004Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "click"
|
name = "click"
|
||||||
version = "8.4.1"
|
version = "8.4.1"
|
||||||
@@ -316,62 +167,6 @@ toml = [
|
|||||||
{ name = "tomli", marker = "python_full_version <= '3.11'" },
|
{ name = "tomli", marker = "python_full_version <= '3.11'" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "cryptography"
|
|
||||||
version = "49.0.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "cffi", marker = "platform_python_implementation != 'PyPy'" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/1f/99/d1c90d6041656cc6ee229dc99cd67fd0cd5aec3c5f7d72fffc27cc750054/cryptography-49.0.0.tar.gz", hash = "sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493", size = 854345, upload-time = "2026-06-12T20:02:30.512Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/9b/22/adf66990e63584a68dfb50c24f48a125c07b1699899381c8151e63ed458c/cryptography-49.0.0-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db", size = 4032100, upload-time = "2026-06-12T20:02:32.143Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/09/41/3797cfaf69cae04a13ee78ebd83f0678d9c02b4779d21ce24445326f1a69/cryptography-49.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db", size = 4692978, upload-time = "2026-06-12T20:01:21.305Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e6/8b/43011f7ebe515a8aa20d61f290a326cd890c2e738e16e59eaff8d9c3a412/cryptography-49.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325", size = 4716422, upload-time = "2026-06-12T20:01:48.566Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/4a/91/01ce7303a4579e6d3a6abef01bd322848e9ea7a219adcabc5048b9033571/cryptography-49.0.0-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2", size = 4700503, upload-time = "2026-06-12T20:02:47.091Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/62/99/a2c95cf8293f07491e9e27c20cc4dcd18176d944e674679adeb1d0173fd6/cryptography-49.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b", size = 5309779, upload-time = "2026-06-12T20:02:08.987Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/20/2c/0622f20ff02b2ef32558733443805dc82fd4c275be01b2d19d14676f3a1b/cryptography-49.0.0-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6", size = 4749683, upload-time = "2026-06-12T20:02:03.335Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a3/5b/c5246635d5fd3b64e0d45ae10e99fd32fe9676a79915ccfe5a61ba9af1a5/cryptography-49.0.0-cp311-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c", size = 4337874, upload-time = "2026-06-12T20:02:54.323Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/6d/88/05563c7fe2e914e87d1a536d06fe83e66b4e1d95cb593e05aea375531da8/cryptography-49.0.0-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7", size = 4700283, upload-time = "2026-06-12T20:01:34.822Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c4/b6/d7696e4e890d6ae1469935164c9e5215c557671cb78d6e3f458ccceaa632/cryptography-49.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68", size = 5265844, upload-time = "2026-06-12T20:01:24.09Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a9/3c/f3ad17eecc1a57b0ba236dc01f90e783c51f4a2f35f64777cc4f47a184b2/cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9", size = 4749290, upload-time = "2026-06-12T20:01:30.848Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/4f/01/339573cf1023163a400b0b5d16f6d507de413b9f60be6fd1b77feeaf6737/cryptography-49.0.0-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f", size = 4834612, upload-time = "2026-06-12T20:01:29.246Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/71/fd/577302e213a1be9468f92d1afef66fcf1ef83d516819d9992ca547f592bd/cryptography-49.0.0-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459", size = 4980804, upload-time = "2026-06-12T20:01:42.853Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/1f/09/f42b1d190c5ba75f72062a387f8030d1d75f6ab035788f1d9c4b01de6525/cryptography-49.0.0-cp311-abi3-win_amd64.whl", hash = "sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e", size = 3810026, upload-time = "2026-06-12T20:02:39.262Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ec/9e/db72b3ae7fc9cfad53e630e56c6ae83b9b6ff0bf3718ffb8012d20b3aabf/cryptography-49.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7", size = 4013892, upload-time = "2026-06-12T20:02:10.735Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/86/12/c48a424f38db03027be9f7ed5c7dc5de9933dbee992865f98b13727a009d/cryptography-49.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d", size = 4678835, upload-time = "2026-06-12T20:02:48.743Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/68/28/8a3ad4653662c93fc44dc4e5d8fd374c25c42e07b34bbfbadf49cf57a5a8/cryptography-49.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa", size = 4697239, upload-time = "2026-06-12T20:02:56.03Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a8/b2/2193fc74f81aee4f9b62733133b73b5176718932ed8f2e4b03fa040480a6/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb", size = 4685593, upload-time = "2026-06-12T20:02:50.666Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/47/f1/1d3eaa243bfc5de4a187b22aa8c048b3e4980bfbe830ac46e6bac2e66947/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d", size = 5289961, upload-time = "2026-06-12T20:01:46.468Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/58/39/2d51306721330c486495853eda1c567880ff036de15a14c4b74f399934af/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561", size = 4731145, upload-time = "2026-06-12T20:02:16.832Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/17/50/983e838c7fd0d87fd8c969bcdd328edaf5f756e38df5281637424c155873/cryptography-49.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122", size = 4321719, upload-time = "2026-06-12T20:02:52.611Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a7/f5/8f571d7e27c55bce9f76f026143bcb1e040a4233149ecca0bea5fa5dd5f7/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505", size = 4685209, upload-time = "2026-06-12T20:02:07.282Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e7/84/0e27016a6fc5a0886f797018b26aa42f40c09a82332bff77822a451deaaa/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866", size = 5246285, upload-time = "2026-06-12T20:01:32.439Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/11/2d/5e1fb307cb5931881516b464c98774b3f2c36b5d4bb9a2830253cf553cad/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8", size = 4730441, upload-time = "2026-06-12T20:02:01.469Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e4/c0/bff5a02ee731d207d6a1ed51732549d8c53d2bc8da1d10ec6f2844201d68/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3", size = 4815869, upload-time = "2026-06-12T20:01:36.574Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b9/26/814681d14248d95d73d5c3eea0c39a94eb8302df966f670a2c60de90974b/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27", size = 4960948, upload-time = "2026-06-12T20:02:18.688Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/4c/fe/93ecac273d3738939d023612ad12cca9a3740a5345d69fda04134c43fd96/cryptography-49.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61", size = 3799153, upload-time = "2026-06-12T20:01:39.059Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/19/2a/5bb823f5bedcf80718cea7fbc95ec5515cca3769633c4b01a32be7f30e7c/cryptography-49.0.0-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a", size = 4025947, upload-time = "2026-06-12T20:01:25.745Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3d/df/40577043ca124e17012f408ddddaeb213b856336ac82ddb3bc915f39e29f/cryptography-49.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4", size = 4692429, upload-time = "2026-06-12T20:01:53.628Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2c/99/2d13299eb3dd27b02dcfaafcc91d6b5cb3329f7cbd6d8f51921acd566c1a/cryptography-49.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18", size = 4700968, upload-time = "2026-06-12T20:02:45.383Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a5/4d/9c0cd02f95e2602dd5e563da149ee0830abef3537be8b34dc56281ebe27a/cryptography-49.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69", size = 4697758, upload-time = "2026-06-12T20:01:41.13Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/24/01/186c825898477d77e2324d5360fefe622ff1d8d1963ec0554e2cada8ec77/cryptography-49.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64", size = 5298863, upload-time = "2026-06-12T20:02:24.579Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b8/7b/62cbbab75d0659865bf0273790031544a0b16c8072d258f9428dcd8190dc/cryptography-49.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21", size = 4735983, upload-time = "2026-06-12T20:01:50.14Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/6c/72/3e798c064bc39e471008075d0f9bc9daf77a80879c092e4a8e170c585ed4/cryptography-49.0.0-cp39-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9", size = 4334173, upload-time = "2026-06-12T20:01:44.743Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f0/ee/6fca21d1ac73e06f8bef71940abfd4d2f6472b4bca284d770f32bd4086f6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc", size = 4697298, upload-time = "2026-06-12T20:02:20.918Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/67/d0/a5fcd3515f0bae49a7b6d0413cc1bdccdcc1fc0047037a0d480642cdc5d6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8", size = 5254338, upload-time = "2026-06-12T20:02:22.737Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a0/84/84fe36f19caf857d61cb7fc9c63035a47ffabd84ea12d1d393148efa3615/cryptography-49.0.0-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36", size = 4735650, upload-time = "2026-06-12T20:02:41.389Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/6c/a0/db537264e234f7273a73ec020873d6d6b39dfd8a53db78b550ca8320440e/cryptography-49.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e", size = 4834820, upload-time = "2026-06-12T20:01:51.847Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/93/77/8df9eb486495979bccecd1062e2eaf435250e84437040295b57d09048b0b/cryptography-49.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b", size = 4967968, upload-time = "2026-06-12T20:02:12.524Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c2/e6/f60198ea8d9dfa15fff9ed4ca02ce362f6eadd9ba757dcc50634c4257b63/cryptography-49.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001", size = 3785547, upload-time = "2026-06-12T20:02:26.847Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/63/d3/4a83af35d65e3fad632c926fad684c193ea4398569ccb0bbbc7fe8f5dc9a/cryptography-49.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b", size = 3993685, upload-time = "2026-06-12T20:02:14.883Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d6/a7/f9dac0ab7f80368c56993a7bf638ef9935f825c91902798481fac0898138/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838", size = 4676239, upload-time = "2026-06-12T20:02:28.793Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d7/70/2ba3769dd0ae167e2f33dfa9592d45db6ff9a61d62ca1a5b3d1bdd09068f/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5", size = 4715584, upload-time = "2026-06-12T20:01:27.495Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/94/64/2923570ac1c0bd3a737aa366ac3abbbbde273042308b8cde95e2364a6e6a/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615", size = 4675885, upload-time = "2026-06-12T20:01:55.49Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ab/f8/614dc7e051418cfe53d55173c1e24c6b0085e89996fe90508c2fdf769aef/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6", size = 4715449, upload-time = "2026-06-12T20:02:05.469Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/aa/50/a9caea39ad19c431c1a3f8a31114df65b260cdfe67786b6c7e7c040c4c44/cryptography-49.0.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6", size = 3783731, upload-time = "2026-06-12T20:02:43.319Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "cyclone"
|
name = "cyclone"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
@@ -379,10 +174,8 @@ source = { editable = "." }
|
|||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "click" },
|
{ name = "click" },
|
||||||
{ name = "fastapi" },
|
{ name = "fastapi" },
|
||||||
{ name = "keyring" },
|
|
||||||
{ name = "pydantic" },
|
{ name = "pydantic" },
|
||||||
{ name = "python-multipart" },
|
{ name = "python-multipart" },
|
||||||
{ name = "pyyaml" },
|
|
||||||
{ name = "sqlalchemy" },
|
{ name = "sqlalchemy" },
|
||||||
{ name = "uvicorn", extra = ["standard"] },
|
{ name = "uvicorn", extra = ["standard"] },
|
||||||
]
|
]
|
||||||
@@ -394,31 +187,21 @@ dev = [
|
|||||||
{ name = "pytest-asyncio" },
|
{ name = "pytest-asyncio" },
|
||||||
{ name = "pytest-cov" },
|
{ name = "pytest-cov" },
|
||||||
]
|
]
|
||||||
sftp = [
|
|
||||||
{ name = "paramiko" },
|
|
||||||
]
|
|
||||||
sqlcipher = [
|
|
||||||
{ name = "sqlcipher3" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[package.metadata]
|
[package.metadata]
|
||||||
requires-dist = [
|
requires-dist = [
|
||||||
{ name = "click", specifier = ">=8.1,<9" },
|
{ name = "click", specifier = ">=8.1,<9" },
|
||||||
{ name = "fastapi", specifier = ">=0.110,<1" },
|
{ name = "fastapi", specifier = ">=0.110,<1" },
|
||||||
{ name = "httpx", marker = "extra == 'dev'", specifier = ">=0.27,<1" },
|
{ name = "httpx", marker = "extra == 'dev'", specifier = ">=0.27,<1" },
|
||||||
{ name = "keyring", specifier = ">=25.0,<26" },
|
|
||||||
{ name = "paramiko", marker = "extra == 'sftp'", specifier = ">=3.4,<6" },
|
|
||||||
{ name = "pydantic", specifier = ">=2.6,<3" },
|
{ name = "pydantic", specifier = ">=2.6,<3" },
|
||||||
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0" },
|
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0" },
|
||||||
{ name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.23,<1" },
|
{ name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.23,<1" },
|
||||||
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.1" },
|
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.1" },
|
||||||
{ name = "python-multipart", specifier = ">=0.0.9,<1" },
|
{ name = "python-multipart", specifier = ">=0.0.9,<1" },
|
||||||
{ name = "pyyaml", specifier = ">=6.0,<7" },
|
|
||||||
{ name = "sqlalchemy", specifier = ">=2.0,<3" },
|
{ name = "sqlalchemy", specifier = ">=2.0,<3" },
|
||||||
{ name = "sqlcipher3", marker = "extra == 'sqlcipher'", specifier = ">=0.6,<1" },
|
|
||||||
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.27,<1" },
|
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.27,<1" },
|
||||||
]
|
]
|
||||||
provides-extras = ["dev", "sqlcipher", "sftp"]
|
provides-extras = ["dev"]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "fastapi"
|
name = "fastapi"
|
||||||
@@ -588,18 +371,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
|
{ url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "importlib-metadata"
|
|
||||||
version = "9.0.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "zipp" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/a9/01/15bb152d77b21318514a96f43af312635eb2500c96b55398d020c93d86ea/importlib_metadata-9.0.0.tar.gz", hash = "sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc", size = 56405, upload-time = "2026-03-20T06:42:56.999Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/38/3d/2d244233ac4f76e38533cfcb2991c9eb4c7bf688ae0a036d30725b8faafe/importlib_metadata-9.0.0-py3-none-any.whl", hash = "sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7", size = 27789, upload-time = "2026-03-20T06:42:55.665Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "iniconfig"
|
name = "iniconfig"
|
||||||
version = "2.3.0"
|
version = "2.3.0"
|
||||||
@@ -609,87 +380,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "invoke"
|
|
||||||
version = "3.0.3"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/33/f6/227c48c5fe47fa178ccf1fda8f047d16c97ba926567b661e9ce2045c600c/invoke-3.0.3.tar.gz", hash = "sha256:437b6a622223824380bfb4e64f612711a6b648c795f565efc8625af66fb57f0c", size = 343419, upload-time = "2026-04-07T15:17:48.307Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/5a/de/bbc12563bbf979618d17625a4e753ff7a078523e28d870d3626daa97261a/invoke-3.0.3-py3-none-any.whl", hash = "sha256:f11327165e5cbb89b2ad1d88d3292b5113332c43b8553b494da435d6ec6f5053", size = 160958, upload-time = "2026-04-07T15:17:46.875Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "jaraco-classes"
|
|
||||||
version = "3.4.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "more-itertools" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz", hash = "sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", size = 11780, upload-time = "2024-03-31T07:27:36.643Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/7f/66/b15ce62552d84bbfcec9a4873ab79d993a1dd4edb922cbfccae192bd5b5f/jaraco.classes-3.4.0-py3-none-any.whl", hash = "sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790", size = 6777, upload-time = "2024-03-31T07:27:34.792Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "jaraco-context"
|
|
||||||
version = "6.1.2"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "backports-tarfile", marker = "python_full_version < '3.12'" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/af/50/4763cd07e722bb6285316d390a164bc7e479db9d90daa769f22578f698b4/jaraco_context-6.1.2.tar.gz", hash = "sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3", size = 16801, upload-time = "2026-03-20T22:13:33.922Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f2/58/bc8954bda5fcda97bd7c19be11b85f91973d67a706ed4a3aec33e7de22db/jaraco_context-6.1.2-py3-none-any.whl", hash = "sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535", size = 7871, upload-time = "2026-03-20T22:13:32.808Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "jaraco-functools"
|
|
||||||
version = "4.5.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "more-itertools" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/36/cf/ea4ef2920830dea3f5ab2ea4da6fb67724e6dca80ee2553788c3607243d0/jaraco_functools-4.5.0.tar.gz", hash = "sha256:3bb5665ea4a020cf78a7040e89154c77edadb3ca74f366479669c5999aa70b03", size = 20272, upload-time = "2026-05-15T21:34:10.025Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/96/9a/982e48afcffcd727a9144506720ffd4224b6b7e355c98641866f38b7c043/jaraco_functools-4.5.0-py3-none-any.whl", hash = "sha256:79ce39246eddbde4b3a03b77ea5f0f7878dc669b166a66cf3fa8e266aa3fa2f4", size = 10594, upload-time = "2026-05-15T21:34:08.595Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "jeepney"
|
|
||||||
version = "0.9.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/7b/6f/357efd7602486741aa73ffc0617fb310a29b588ed0fd69c2399acbb85b0c/jeepney-0.9.0.tar.gz", hash = "sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732", size = 106758, upload-time = "2025-02-27T18:51:01.684Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b2/a3/e137168c9c44d18eff0376253da9f1e9234d0239e0ee230d2fee6cea8e55/jeepney-0.9.0-py3-none-any.whl", hash = "sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683", size = 49010, upload-time = "2025-02-27T18:51:00.104Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "keyring"
|
|
||||||
version = "25.7.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "importlib-metadata", marker = "python_full_version < '3.12'" },
|
|
||||||
{ name = "jaraco-classes" },
|
|
||||||
{ name = "jaraco-context" },
|
|
||||||
{ name = "jaraco-functools" },
|
|
||||||
{ name = "jeepney", marker = "sys_platform == 'linux'" },
|
|
||||||
{ name = "pywin32-ctypes", marker = "sys_platform == 'win32'" },
|
|
||||||
{ name = "secretstorage", marker = "sys_platform == 'linux'" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/43/4b/674af6ef2f97d56f0ab5153bf0bfa28ccb6c3ed4d1babf4305449668807b/keyring-25.7.0.tar.gz", hash = "sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b", size = 63516, upload-time = "2025-11-16T16:26:09.482Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/81/db/e655086b7f3a705df045bf0933bdd9c2f79bb3c97bfef1384598bb79a217/keyring-25.7.0-py3-none-any.whl", hash = "sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f", size = 39160, upload-time = "2025-11-16T16:26:08.402Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "more-itertools"
|
|
||||||
version = "11.1.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/de/1d/f4da6f02cdffe04d6362210b807146a26044c88d839208aec273bb0d9184/more_itertools-11.1.0.tar.gz", hash = "sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d", size = 145772, upload-time = "2026-05-22T14:14:29.909Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e8/3d/1087453384dbde46a8c7f9356eead2c58be8a7bf156bca40243377c85715/more_itertools-11.1.0-py3-none-any.whl", hash = "sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192", size = 72226, upload-time = "2026-05-22T14:14:28.824Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "packaging"
|
name = "packaging"
|
||||||
version = "26.2"
|
version = "26.2"
|
||||||
@@ -699,21 +389,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
|
{ url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "paramiko"
|
|
||||||
version = "5.0.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "bcrypt" },
|
|
||||||
{ name = "cryptography" },
|
|
||||||
{ name = "invoke" },
|
|
||||||
{ name = "pynacl" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/62/93/dcc25d52f49022ae6175d15e6bd751f1acc99b98bc61fc55e5155a7be2e7/paramiko-5.0.0.tar.gz", hash = "sha256:36763b5b95c2a0dcfdf1abc48e48156ee425b21efe2f0e787c2dd5a95c0e5e79", size = 1548586, upload-time = "2026-05-09T18:28:52.256Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/82/5b/eadf6d45de38d30ab603f49393b6cd2cbe7e233af8cf90197e32782b68a9/paramiko-5.0.0-py3-none-any.whl", hash = "sha256:b7044611c30140d9a75261653210e2002977b71a0497ff3ba0d98d7edbf62f7c", size = 208919, upload-time = "2026-05-09T18:28:50.295Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pluggy"
|
name = "pluggy"
|
||||||
version = "1.6.0"
|
version = "1.6.0"
|
||||||
@@ -723,15 +398,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
|
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "pycparser"
|
|
||||||
version = "3.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz", hash = "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", size = 103492, upload-time = "2026-01-21T14:26:51.89Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pydantic"
|
name = "pydantic"
|
||||||
version = "2.13.4"
|
version = "2.13.4"
|
||||||
@@ -858,41 +524,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "pynacl"
|
|
||||||
version = "1.6.2"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "cffi", marker = "platform_python_implementation != 'PyPy'" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/d9/9a/4019b524b03a13438637b11538c82781a5eda427394380381af8f04f467a/pynacl-1.6.2.tar.gz", hash = "sha256:018494d6d696ae03c7e656e5e74cdfd8ea1326962cc401bcf018f1ed8436811c", size = 3511692, upload-time = "2026-01-01T17:48:10.851Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/4b/79/0e3c34dc3c4671f67d251c07aa8eb100916f250ee470df230b0ab89551b4/pynacl-1.6.2-cp314-cp314t-macosx_10_10_universal2.whl", hash = "sha256:622d7b07cc5c02c666795792931b50c91f3ce3c2649762efb1ef0d5684c81594", size = 390064, upload-time = "2026-01-01T17:31:57.264Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/eb/1c/23a26e931736e13b16483795c8a6b2f641bf6a3d5238c22b070a5112722c/pynacl-1.6.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d071c6a9a4c94d79eb665db4ce5cedc537faf74f2355e4d502591d850d3913c0", size = 809370, upload-time = "2026-01-01T17:31:59.198Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/87/74/8d4b718f8a22aea9e8dcc8b95deb76d4aae380e2f5b570cc70b5fd0a852d/pynacl-1.6.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:fe9847ca47d287af41e82be1dd5e23023d3c31a951da134121ab02e42ac218c9", size = 1408304, upload-time = "2026-01-01T17:32:01.162Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/fd/73/be4fdd3a6a87fe8a4553380c2b47fbd1f7f58292eb820902f5c8ac7de7b0/pynacl-1.6.2-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:04316d1fc625d860b6c162fff704eb8426b1a8bcd3abacea11142cbd99a6b574", size = 844871, upload-time = "2026-01-01T17:32:02.824Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/55/ad/6efc57ab75ee4422e96b5f2697d51bbcf6cdcc091e66310df91fbdc144a8/pynacl-1.6.2-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:44081faff368d6c5553ccf55322ef2819abb40e25afaec7e740f159f74813634", size = 1446356, upload-time = "2026-01-01T17:32:04.452Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/78/b7/928ee9c4779caa0a915844311ab9fb5f99585621c5d6e4574538a17dca07/pynacl-1.6.2-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:a9f9932d8d2811ce1a8ffa79dcbdf3970e7355b5c8eb0c1a881a57e7f7d96e88", size = 826814, upload-time = "2026-01-01T17:32:06.078Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f7/a9/1bdba746a2be20f8809fee75c10e3159d75864ef69c6b0dd168fc60e485d/pynacl-1.6.2-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:bc4a36b28dd72fb4845e5d8f9760610588a96d5a51f01d84d8c6ff9849968c14", size = 1411742, upload-time = "2026-01-01T17:32:07.651Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f3/2f/5e7ea8d85f9f3ea5b6b87db1d8388daa3587eed181bdeb0306816fdbbe79/pynacl-1.6.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:3bffb6d0f6becacb6526f8f42adfb5efb26337056ee0831fb9a7044d1a964444", size = 801714, upload-time = "2026-01-01T17:32:09.558Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/06/ea/43fe2f7eab5f200e40fb10d305bf6f87ea31b3bbc83443eac37cd34a9e1e/pynacl-1.6.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:2fef529ef3ee487ad8113d287a593fa26f48ee3620d92ecc6f1d09ea38e0709b", size = 1372257, upload-time = "2026-01-01T17:32:11.026Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/4d/54/c9ea116412788629b1347e415f72195c25eb2f3809b2d3e7b25f5c79f13a/pynacl-1.6.2-cp314-cp314t-win32.whl", hash = "sha256:a84bf1c20339d06dc0c85d9aea9637a24f718f375d861b2668b2f9f96fa51145", size = 231319, upload-time = "2026-01-01T17:32:12.46Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ce/04/64e9d76646abac2dccf904fccba352a86e7d172647557f35b9fe2a5ee4a1/pynacl-1.6.2-cp314-cp314t-win_amd64.whl", hash = "sha256:320ef68a41c87547c91a8b58903c9caa641ab01e8512ce291085b5fe2fcb7590", size = 244044, upload-time = "2026-01-01T17:32:13.781Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/33/33/7873dc161c6a06f43cda13dec67b6fe152cb2f982581151956fa5e5cdb47/pynacl-1.6.2-cp314-cp314t-win_arm64.whl", hash = "sha256:d29bfe37e20e015a7d8b23cfc8bd6aa7909c92a1b8f41ee416bbb3e79ef182b2", size = 188740, upload-time = "2026-01-01T17:32:15.083Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/be/7b/4845bbf88e94586ec47a432da4e9107e3fc3ce37eb412b1398630a37f7dd/pynacl-1.6.2-cp38-abi3-macosx_10_10_universal2.whl", hash = "sha256:c949ea47e4206af7c8f604b8278093b674f7c79ed0d4719cc836902bf4517465", size = 388458, upload-time = "2026-01-01T17:32:16.829Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/1e/b4/e927e0653ba63b02a4ca5b4d852a8d1d678afbf69b3dbf9c4d0785ac905c/pynacl-1.6.2-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8845c0631c0be43abdd865511c41eab235e0be69c81dc66a50911594198679b0", size = 800020, upload-time = "2026-01-01T17:32:18.34Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/7f/81/d60984052df5c97b1d24365bc1e30024379b42c4edcd79d2436b1b9806f2/pynacl-1.6.2-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:22de65bb9010a725b0dac248f353bb072969c94fa8d6b1f34b87d7953cf7bbe4", size = 1399174, upload-time = "2026-01-01T17:32:20.239Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/68/f7/322f2f9915c4ef27d140101dd0ed26b479f7e6f5f183590fd32dfc48c4d3/pynacl-1.6.2-cp38-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:46065496ab748469cdd999246d17e301b2c24ae2fdf739132e580a0e94c94a87", size = 835085, upload-time = "2026-01-01T17:32:22.24Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3e/d0/f301f83ac8dbe53442c5a43f6a39016f94f754d7a9815a875b65e218a307/pynacl-1.6.2-cp38-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8a66d6fb6ae7661c58995f9c6435bda2b1e68b54b598a6a10247bfcdadac996c", size = 1437614, upload-time = "2026-01-01T17:32:23.766Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c4/58/fc6e649762b029315325ace1a8c6be66125e42f67416d3dbd47b69563d61/pynacl-1.6.2-cp38-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:26bfcd00dcf2cf160f122186af731ae30ab120c18e8375684ec2670dccd28130", size = 818251, upload-time = "2026-01-01T17:32:25.69Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c9/a8/b917096b1accc9acd878819a49d3d84875731a41eb665f6ebc826b1af99e/pynacl-1.6.2-cp38-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:c8a231e36ec2cab018c4ad4358c386e36eede0319a0c41fed24f840b1dac59f6", size = 1402859, upload-time = "2026-01-01T17:32:27.215Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/85/42/fe60b5f4473e12c72f977548e4028156f4d340b884c635ec6b063fe7e9a5/pynacl-1.6.2-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:68be3a09455743ff9505491220b64440ced8973fe930f270c8e07ccfa25b1f9e", size = 791926, upload-time = "2026-01-01T17:32:29.314Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/fa/f9/e40e318c604259301cc091a2a63f237d9e7b424c4851cafaea4ea7c4834e/pynacl-1.6.2-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:8b097553b380236d51ed11356c953bf8ce36a29a3e596e934ecabe76c985a577", size = 1363101, upload-time = "2026-01-01T17:32:31.263Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/48/47/e761c254f410c023a469284a9bc210933e18588ca87706ae93002c05114c/pynacl-1.6.2-cp38-abi3-win32.whl", hash = "sha256:5811c72b473b2f38f7e2a3dc4f8642e3a3e9b5e7317266e4ced1fba85cae41aa", size = 227421, upload-time = "2026-01-01T17:32:33.076Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/41/ad/334600e8cacc7d86587fe5f565480fde569dfb487389c8e1be56ac21d8ac/pynacl-1.6.2-cp38-abi3-win_amd64.whl", hash = "sha256:62985f233210dee6548c223301b6c25440852e13d59a8b81490203c3227c5ba0", size = 239754, upload-time = "2026-01-01T17:32:34.557Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/29/7d/5945b5af29534641820d3bd7b00962abbbdfee84ec7e19f0d5b3175f9a31/pynacl-1.6.2-cp38-abi3-win_arm64.whl", hash = "sha256:834a43af110f743a754448463e8fd61259cd4ab5bbedcf70f9dabad1d28a394c", size = 184801, upload-time = "2026-01-01T17:32:36.309Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pytest"
|
name = "pytest"
|
||||||
version = "8.4.2"
|
version = "8.4.2"
|
||||||
@@ -953,15 +584,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/e1/04/e8135ebd1ad02c56ec633277529b2602ff99ff634be76cdba5744cf554fd/python_multipart-0.0.32-py3-none-any.whl", hash = "sha256:ff6d3f776f16878c894e52e107296ffc890e913c611b1a4ec6c44e2821fe2e23", size = 30042, upload-time = "2026-06-04T16:18:57.319Z" },
|
{ url = "https://files.pythonhosted.org/packages/e1/04/e8135ebd1ad02c56ec633277529b2602ff99ff634be76cdba5744cf554fd/python_multipart-0.0.32-py3-none-any.whl", hash = "sha256:ff6d3f776f16878c894e52e107296ffc890e913c611b1a4ec6c44e2821fe2e23", size = 30042, upload-time = "2026-06-04T16:18:57.319Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "pywin32-ctypes"
|
|
||||||
version = "0.2.3"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/85/9f/01a1a99704853cb63f253eea009390c88e7131c67e66a0a02099a8c917cb/pywin32-ctypes-0.2.3.tar.gz", hash = "sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755", size = 29471, upload-time = "2024-08-14T10:15:34.626Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/de/3d/8161f7711c017e01ac9f008dfddd9410dff3674334c233bde66e7ba65bbf/pywin32_ctypes-0.2.3-py3-none-any.whl", hash = "sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8", size = 30756, upload-time = "2024-08-14T10:15:33.187Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pyyaml"
|
name = "pyyaml"
|
||||||
version = "6.0.3"
|
version = "6.0.3"
|
||||||
@@ -1017,19 +639,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },
|
{ url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "secretstorage"
|
|
||||||
version = "3.5.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "cryptography" },
|
|
||||||
{ name = "jeepney" },
|
|
||||||
]
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz", hash = "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", size = 19884, upload-time = "2025-11-23T19:02:53.191Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b7/46/f5af3402b579fd5e11573ce652019a67074317e18c1935cc0b4ba9b35552/secretstorage-3.5.0-py3-none-any.whl", hash = "sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137", size = 15554, upload-time = "2025-11-23T19:02:51.545Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "sqlalchemy"
|
name = "sqlalchemy"
|
||||||
version = "2.0.51"
|
version = "2.0.51"
|
||||||
@@ -1078,69 +687,6 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/e2/22/dbf013a12ec759e54a34a119e9e217435b3f71b2dd5c61a7ade0a25dae87/sqlalchemy-2.0.51-py3-none-any.whl", hash = "sha256:bb024d8b621d0be75f4f44ecc7c950450026e76d66dc8f791bb5331d7fed59d5", size = 1944334, upload-time = "2026-06-15T16:09:22.418Z" },
|
{ url = "https://files.pythonhosted.org/packages/e2/22/dbf013a12ec759e54a34a119e9e217435b3f71b2dd5c61a7ade0a25dae87/sqlalchemy-2.0.51-py3-none-any.whl", hash = "sha256:bb024d8b621d0be75f4f44ecc7c950450026e76d66dc8f791bb5331d7fed59d5", size = 1944334, upload-time = "2026-06-15T16:09:22.418Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "sqlcipher3"
|
|
||||||
version = "0.6.2"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/ae/c1/414003d77549c444bafd636149ab3ace6f4e2cb4666c9955d54ad62096cb/sqlcipher3-0.6.2.tar.gz", hash = "sha256:a2b675289ba8889f389625a21f3a01f1ff159a551b5b88fba8fd92da0e02380a", size = 2663213, upload-time = "2026-01-07T23:13:26.061Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f6/97/6461dc2ab41fbaed4942aaf7d1bac23d8cd130c751d40830ce391e80d332/sqlcipher3-0.6.2-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:87432804bf88e9017fc174bdd3d0862a1d1e9ef3c755517595c91da2c59e3808", size = 4942310, upload-time = "2026-01-07T23:11:58.393Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/45/dc/73f238aa994e08ac6971838907fd2b80d9bd86277c3c0e99300985287cd5/sqlcipher3-0.6.2-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:eece737c583c285e9bbe3bf829eee3b6624eb6e9dad8ccff7821a45641f436dd", size = 2893419, upload-time = "2026-01-07T23:11:59.762Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e5/4c/098cf3dd0af6ce4cfba88fbdeb63a3b156f4b7f0620f6cc5f35ecfb72607/sqlcipher3-0.6.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:22e6502c364706fe64695219877f2bb01cdb25450bec81e69c8a08deff8c14ee", size = 3160663, upload-time = "2026-01-07T23:12:01.011Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/71/09/5ae806f9f5833ab7decfef0b24471f57c06c10b7fdc2a097a3066d6c80d4/sqlcipher3-0.6.2-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:0ca92202881bcb69b3703b744b40a3a3476e122d4612a82eb2b0a36f2f78de1d", size = 7252828, upload-time = "2026-01-07T23:12:02.076Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c5/bd/10527a221d2b7c33b1c32058c3882a03033fca013b4f9f98f0befc0fe98e/sqlcipher3-0.6.2-cp311-cp311-manylinux_2_28_i686.whl", hash = "sha256:15abe3de01faa194f1aaea144ff9ecbfdce2991964dcc7ce8ec1ecc5950a4bc4", size = 6868146, upload-time = "2026-01-07T23:12:03.335Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/72/b0/faa2a8fc9dc3210e0af31e57c5ec86e8a523eaa3d44e854aa8f95ff66d50/sqlcipher3-0.6.2-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:0f08e5bb5eb1ab93819c444ebec61fa3349e9690c14f5d0276fd4f61c3049fd9", size = 7031136, upload-time = "2026-01-07T23:12:05.242Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/6d/37/1a82b3fc1504741df5aa4dccc6fd4265244e5b5dc98aa95e24321d73f0bb/sqlcipher3-0.6.2-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:dfe90f1e0e81a8c6c8c4129a8439ed6b9b27a8e32077c59ed3b7f1263e3c5544", size = 7245432, upload-time = "2026-01-07T23:12:07.346Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a3/e5/68bbaa1790e0f2fc571924933db01a6af2991ebf479496934ab5f1b19484/sqlcipher3-0.6.2-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:a5f805c1f156634e4e91f1b073d95930756fdf23eeeeb7b85c511a5cf165b10c", size = 7051580, upload-time = "2026-01-07T23:12:08.674Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/5b/51/0939f292767fe26b978b845cefac077916f5b2f1caae110be89d2b3a79dc/sqlcipher3-0.6.2-cp311-cp311-win32.whl", hash = "sha256:fc08ff475ab0e0f43adca0647d827e81da5fa406bbb6bd04471e28a3ad2864d9", size = 1981400, upload-time = "2026-01-07T23:12:09.849Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/9d/87/6cb7a6ced1244350514cf419cbeb442f12f445e3c4d09da0e7a49ac7ee80/sqlcipher3-0.6.2-cp311-cp311-win_amd64.whl", hash = "sha256:4ad7e4a32de907011ea22ac2012c9bca1bb414e2f599c56a55c8b0fe6445b932", size = 2485356, upload-time = "2026-01-07T23:12:11.253Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2c/2f/52f001d2b884047ab490964d37ee7891d497cc891cee7e64bac5672eb7d4/sqlcipher3-0.6.2-cp311-cp311-win_arm64.whl", hash = "sha256:3ad6b39a7fa8c2f7ec471dd29fadbffa19c194fbae1730f013f0d29f5b96fae0", size = 2652186, upload-time = "2026-01-07T23:12:12.488Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/96/ac/3612ebe2504d7c6786ddf73d28ae3d2707eb39d6e5730336854071ccb612/sqlcipher3-0.6.2-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:a51b18bd782652a2282f9cb1b03b840ba5a6c0c675de6cefb76262c9789c8f06", size = 4944108, upload-time = "2026-01-07T23:12:13.751Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/bc/a4/7be8f0199ca6a1a2c6f7bb8118b2e37d29077ade2319d6271b1724d0cc86/sqlcipher3-0.6.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:fea0f1264f09d219dd6ce699ffca8cc9022a914661c6efa4390e85a2bf78acf9", size = 2895406, upload-time = "2026-01-07T23:12:14.954Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2f/b7/b9e897cf9e4740ca148fb03b493fa708a9b729ccc0cd656099f16bc9f2fd/sqlcipher3-0.6.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:bc2edd981e65783bc0d4e337704a9eb436871ab91c68af02ed76354876087642", size = 3161395, upload-time = "2026-01-07T23:12:16.049Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b4/98/57e7f7e170b6065a21735687a94373aba9ce6866708e5a386e6af1a90ff2/sqlcipher3-0.6.2-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:50f64086da0a5f14281f2b0b459c4d9923b50055813a48ad29baf8c41c7fa56c", size = 7261870, upload-time = "2026-01-07T23:12:17.732Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/69/e4/cb0ed654a9642ee707c79f5e46db11518737318fd24968463b5aaa367a31/sqlcipher3-0.6.2-cp312-cp312-manylinux_2_28_i686.whl", hash = "sha256:2288215f462a16996689e1c22d611c94dd865faddb703cb105981dc3c0307b23", size = 6874049, upload-time = "2026-01-07T23:12:19.471Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f5/03/d55fe69fb380dadb2f5d19b3eac9256218243cced6aa4696ef90d560d223/sqlcipher3-0.6.2-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:6b26d28ca844dc2a69b8f74b390e940db47760f0be4c96d93337c57ae8250a48", size = 7040115, upload-time = "2026-01-07T23:12:21.168Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/8c/bd/afe3998add9f877533480f48d0deaa1c42a31832954424f4414539f59bfc/sqlcipher3-0.6.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:11e85c1fa4dfe6bf031af8ada500e94b5a77762355b500580360aa162896cecd", size = 7253646, upload-time = "2026-01-07T23:12:22.467Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/da/ff/9723596e7d220d933d5016adcea249b8e6f4f54116219ead6cf919646de4/sqlcipher3-0.6.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:80ae14562d98419b32149e8d66eea567eb3792e149b103ee5c8e1e5c67c5d799", size = 7060703, upload-time = "2026-01-07T23:12:23.907Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c4/63/e220211b098bc54d5345369759e21e4b34804d7e1c9f80f53372e2041b39/sqlcipher3-0.6.2-cp312-cp312-win32.whl", hash = "sha256:fb15c43f8a4f8b6b0ebe62ad2ab97a7946e3b75cb98a02069ff56b7d5a96c415", size = 1982081, upload-time = "2026-01-07T23:12:25.241Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2e/1a/88e8a0b96a52d291e24424cf8a222a6c7cc5356fd0a2287086afff764c2e/sqlcipher3-0.6.2-cp312-cp312-win_amd64.whl", hash = "sha256:8bd60ffb7bfa65bd0e51da3d5c308553d7149f0091d4ea9f754c33d5ebbf0a66", size = 2485691, upload-time = "2026-01-07T23:12:26.498Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/5e/83/69217a75ed282dc865c4d658993af935210efe907b5ab6a863365ed6f20b/sqlcipher3-0.6.2-cp312-cp312-win_arm64.whl", hash = "sha256:99148bf4bf8e73c2c35f810f80de776d7de09b6cf277322c07759026400e90d0", size = 2652185, upload-time = "2026-01-07T23:12:27.594Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/0e/79/56fb1ea7cffc2ef32446b6fb9ba499464324995f9ea21c88791af9176682/sqlcipher3-0.6.2-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:8093773cd59b2a205d2cdb21383a93f7725126497032c269983ed89a89993631", size = 4943278, upload-time = "2026-01-07T23:12:28.735Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/98/5f/805772f52b10abe1e1aedba6f8c0ab7fb5eb5ccb4de5dfb51fef71132096/sqlcipher3-0.6.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:29f39d50bea02d78a824022989164c171e865bfeced3f9b84d1d45193dae074c", size = 2894958, upload-time = "2026-01-07T23:12:29.863Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/56/0d/2cee40de57d47245de09382c64e649c8cc8e86fa549ecba7591633fabf20/sqlcipher3-0.6.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:8e1ff6079603dfd955d57c26dad5eab14f6baacdc643d8753dd651913ba789cf", size = 3160350, upload-time = "2026-01-07T23:12:30.934Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/fc/7a/72e7001af10c5b0e34592eae8e2634c22f1eb67d9859327c2fbbbf2b4961/sqlcipher3-0.6.2-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:35ae605f7594fca64a6d71007795dd39effd625cdc2a181d47f7d9fc8a5e1965", size = 7257677, upload-time = "2026-01-07T23:12:32.091Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a0/16/2e36fc23f4cc3baad39938c71c2db33a92ffa230a81073aa9186ad33a540/sqlcipher3-0.6.2-cp313-cp313-manylinux_2_28_i686.whl", hash = "sha256:8a3e39ad5f73060232b17715aa3b757e82ec4b67bb6acfc081147f66d00c2659", size = 6870821, upload-time = "2026-01-07T23:12:33.516Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f4/6b/874f72b6f3c3ebbe889e4279a0d422b7271ef7b3c63e45fae80a4ce16ec7/sqlcipher3-0.6.2-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:9fb7109981583b631ac795e7e955d4bf78058f64b54c7f334ccc437adc322d4b", size = 7037142, upload-time = "2026-01-07T23:12:34.803Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a5/a0/36af1ad4a3d45cc6c3a5dd508d1d5ddd24a6903a478405d40da37b1acc07/sqlcipher3-0.6.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:1ab63dcba15868853cb4d318cceb50dc47b94095e0c434f2785b9b098f3f5b42", size = 7249529, upload-time = "2026-01-07T23:12:36.079Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/fd/67/544729cd60a1cd961d6e70b7880a5a2dd18dd38bf9575527b5b2a893daa5/sqlcipher3-0.6.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:16d54822ad2fe44e49818a27f4862ba041f2d4a6aeb69422186379f9af97ced0", size = 7057767, upload-time = "2026-01-07T23:12:37.487Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/5b/63/20730edd16df0d5ba8198cf8f4b679335c9db61c469d30ba2ef8adfbee2e/sqlcipher3-0.6.2-cp313-cp313-win32.whl", hash = "sha256:31789ce5ec7dd3f6c4ebd612c9cd9f7079a1d3698829111f7a382b0c10da3a87", size = 1981467, upload-time = "2026-01-07T23:12:39.16Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ab/e3/11e2e945557fe300bc399d9fafbba9154089f84483f4940546fd81b49b29/sqlcipher3-0.6.2-cp313-cp313-win_amd64.whl", hash = "sha256:9dc959ff792228c6df836cfd3667c713ae13e6e18dc2905c9d5666558606e832", size = 2485219, upload-time = "2026-01-07T23:12:40.473Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f3/79/245ef275ef93b45005e000105eb62df5779c056b1ba699eb7b5ba663a1c8/sqlcipher3-0.6.2-cp313-cp313-win_arm64.whl", hash = "sha256:30eeac16e755e5b0cff584ff541d3001bfcfc20be0ae364ff5305bbaeccbb3f1", size = 2651933, upload-time = "2026-01-07T23:12:41.727Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/27/a6/8e0ac8e198ba5c92b4af150f1b405968dacb399259bb74e732818dbacb46/sqlcipher3-0.6.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:dae7ce66554f2416d9e9012cc78dfe4d4053385e7fa289a8d0bd7772e5f5a702", size = 4943422, upload-time = "2026-01-07T23:12:42.883Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/e1/99/72b4e23936bd4065fb97c07429a395c828a1d7419776219c7e98932df14f/sqlcipher3-0.6.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:be137cc92c9a039e469a758ee55e27e2385f419d1387f24e2029c536aa5d9736", size = 2894925, upload-time = "2026-01-07T23:12:44.133Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f6/01/f3552874b158d83c15fb9d550576020cc42b34019d0daf3291b381fbfb01/sqlcipher3-0.6.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:5c1f4a5805faa418c9c7290e6a556a8c5abae40ea59b04d76e960e33c257e618", size = 3160559, upload-time = "2026-01-07T23:12:45.552Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/20/a0/274cbe5180a837818f5502823b5fa20f198546abc36ee56803636db5dcaf/sqlcipher3-0.6.2-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:e2328f0848ffb78807cf0898749dc22ff3f5aa95ab0d5a8a253628de20c11a1c", size = 7257835, upload-time = "2026-01-07T23:12:46.856Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d5/49/0d9a84435658ea3a6cdbb54cb8e67725e4c146726d008248531dca146eb7/sqlcipher3-0.6.2-cp314-cp314-manylinux_2_28_i686.whl", hash = "sha256:a1fbb693bf7c2f6f46ed038544b0bf76ea43dcc3231905cf5a686af15dc9b424", size = 6872199, upload-time = "2026-01-07T23:12:48.235Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ff/12/8d554633c3975f429e07cf07e136fb94ace10b460e1cb86b4c8019b7cdb4/sqlcipher3-0.6.2-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:e00988174ecd67ecd4537504c3df55bf8daeb75fce98401f099dff8e22c43ae1", size = 7037131, upload-time = "2026-01-07T23:12:49.416Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2e/7a/58dbb21db860d006d2be466678e7fdc317118a55c0e8a3e2a7f848f42112/sqlcipher3-0.6.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a105e816579bb7cce6f03e7e208b06d6d886c6445e1c738ed9aa2febabff3041", size = 7250574, upload-time = "2026-01-07T23:12:50.73Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/62/84/8d1f55fd8fadef56b40ee19bfae23100b5ff883bec65582a12a4bfbd56d5/sqlcipher3-0.6.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:e3a936d7414ae62f40880668bc036b0fde1ef0f48ed86cfe6564340f780ceea4", size = 7058156, upload-time = "2026-01-07T23:12:52.13Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d5/0d/80861eeeba9f5f6a0f202b3d27f9ab696e2c885442ed9340ad96ba0ed66c/sqlcipher3-0.6.2-cp314-cp314-win32.whl", hash = "sha256:7a07dafe752e013d4030accf218e80472d08de1309ddaf26df6f02d0850b2cec", size = 2028880, upload-time = "2026-01-07T23:12:53.316Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a6/11/950f0b092588866213e5d89b08701d24a938c9df116673bba54ac35f61af/sqlcipher3-0.6.2-cp314-cp314-win_amd64.whl", hash = "sha256:7de6133b19aec27b30698267cc2a0ea6e82c21d9a81d349cf0b480439fb549ac", size = 2556915, upload-time = "2026-01-07T23:12:54.615Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/5c/e7/cc1ce8e013bb84b8429b9736de7aca58b738781b08230bc9ba1c1e1a6d55/sqlcipher3-0.6.2-cp314-cp314-win_arm64.whl", hash = "sha256:765e133bd4ddda5596275f1221fa63b2b5d7d2b6e3670809bbf630edb705e27a", size = 2722598, upload-time = "2026-01-07T23:12:55.743Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/26/68/611dd86b446e069f769cf129cde96967982e0b41f49e66df9d3d21255df7/sqlcipher3-0.6.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:83533b5b7622ec9b78bec7596d96534e30015136f3e3e69a22f836fc59e393bc", size = 4947511, upload-time = "2026-01-07T23:12:56.968Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/80/24/b7e39a394841d56a678edb4b5a3b259b5a07bbb86a3863e15c5e1b041a58/sqlcipher3-0.6.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:310d7adbea382bda31007ee7d3dc63ba6ca86fcf7c0626ea804161fad2efce5e", size = 2896925, upload-time = "2026-01-07T23:12:58.09Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/31/ff/0b4b0cb02dd4084325bce526f0f4467c6a1ebcdf8fc625516734640f37ba/sqlcipher3-0.6.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1010c4ff1ff13a7e53284a3b03980754dbd37e6eea6faed9c6409e52bac082e6", size = 3164288, upload-time = "2026-01-07T23:12:59.362Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d0/c3/bab9952c3c1bbde45313716610c5dcd36528b315228ac0b51e2d45217509/sqlcipher3-0.6.2-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:d437215611b620b32cb6b68dbc66dbeaccdfb3f76a7b6d8118a40849f8612088", size = 7300761, upload-time = "2026-01-07T23:13:00.836Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/2b/ca/3315292ed9ddbcd2de28f445ad11cc4a7f710d8b2ab2095aa0a629682d25/sqlcipher3-0.6.2-cp314-cp314t-manylinux_2_28_i686.whl", hash = "sha256:09e4170b1b2744b02b1c9315996a228d0b8d8a3ec1a0f7d4d41db0e79872fcea", size = 6911561, upload-time = "2026-01-07T23:13:02.072Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a8/66/fdfdb11110d7166ca3707180d9ce85642be096d348072aba0ef8483b307a/sqlcipher3-0.6.2-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:bb4eaa9093bd46a7d51a65b9f63bac29ec4fc6b4ac794083e53eeb49f6db7e2c", size = 7074025, upload-time = "2026-01-07T23:13:03.327Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ca/24/69b9bdc5ae7859a009b5336515c7e4ea5d58dfe3e358e44fabbcdd1431ae/sqlcipher3-0.6.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:782111277cbd999b7bc4e9e910396492ee28397c2c60ef7287b6dbc36a0b6a24", size = 7293516, upload-time = "2026-01-07T23:13:04.67Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/07/1b/2a62a605851b6ada8686c31b0cd82dd4305b7b19a57af3dbf7fe43249d1e/sqlcipher3-0.6.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:a01424f0d0120e8d9d3e0e1751ef78e70867bbce91f283b56911e8c6adaafddb", size = 7096368, upload-time = "2026-01-07T23:13:06.205Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c1/7a/6c7d3356c9885e87a49d8f649f25019125ce6e68b4c8bb3933538d7add2c/sqlcipher3-0.6.2-cp314-cp314t-win32.whl", hash = "sha256:311fa50be627a4d1566bed31fd7725dec535a71332dcefdcdf9ec2472c4f824d", size = 2031516, upload-time = "2026-01-07T23:13:07.374Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/47/be/42d8d5cbcd0a89a5d7bfa1fe2ff986355d8b5910445b92f9d109b18eca93/sqlcipher3-0.6.2-cp314-cp314t-win_amd64.whl", hash = "sha256:7ac16581a5b80c54237c5f08f2e488051dfa7f52e3890e7765a6364d5bb3a2c6", size = 2559803, upload-time = "2026-01-07T23:13:08.447Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/9d/27/140a300fd151773604c4ff75ff5785febf25071231f239d6ce84b32e1408/sqlcipher3-0.6.2-cp314-cp314t-win_arm64.whl", hash = "sha256:76125dd222f4946302f70281e155ae9336efa4bc6fabdc81a7ae9bd4dfce9180", size = 2724322, upload-time = "2026-01-07T23:13:09.5Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "starlette"
|
name = "starlette"
|
||||||
version = "1.3.1"
|
version = "1.3.1"
|
||||||
@@ -1453,12 +999,3 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/9a/3f/f70e03f40ffc9a30d817eef7da1be72ee4956ba8d7255c399a01b135902a/websockets-16.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:a653aea902e0324b52f1613332ddf50b00c06fdaf7e92624fbf8c77c78fa5767", size = 178735, upload-time = "2026-01-10T09:23:42.259Z" },
|
{ url = "https://files.pythonhosted.org/packages/9a/3f/f70e03f40ffc9a30d817eef7da1be72ee4956ba8d7255c399a01b135902a/websockets-16.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:a653aea902e0324b52f1613332ddf50b00c06fdaf7e92624fbf8c77c78fa5767", size = 178735, upload-time = "2026-01-10T09:23:42.259Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/6f/28/258ebab549c2bf3e64d2b0217b973467394a9cea8c42f70418ca2c5d0d2e/websockets-16.0-py3-none-any.whl", hash = "sha256:1637db62fad1dc833276dded54215f2c7fa46912301a24bd94d45d46a011ceec", size = 171598, upload-time = "2026-01-10T09:23:45.395Z" },
|
{ url = "https://files.pythonhosted.org/packages/6f/28/258ebab549c2bf3e64d2b0217b973467394a9cea8c42f70418ca2c5d0d2e/websockets-16.0-py3-none-any.whl", hash = "sha256:1637db62fad1dc833276dded54215f2c7fa46912301a24bd94d45d46a011ceec", size = 171598, upload-time = "2026-01-10T09:23:45.395Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "zipp"
|
|
||||||
version = "4.1.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/b9/d8/eab98a517c14134c0b2eb4e2387bc5f457334293ec5d2dd3857ec2966802/zipp-4.1.0.tar.gz", hash = "sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602", size = 26214, upload-time = "2026-05-18T20:08:57.967Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3a/13/547360d81e6d88d58492968ffda9f9542854f11310ee556fef14260cc886/zipp-4.1.0-py3-none-any.whl", hash = "sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f", size = 10238, upload-time = "2026-05-18T20:08:57.045Z" },
|
|
||||||
]
|
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
ISA*00* *00* *ZZ*11525703 *ZZ*COMEDASSISTPROG*240911*2240*^*00501*240901088*1*P*:~GS*HC*11525703*COMEDASSISTPROG*20240911*224042*240007724*X*005010X222A1~ST*837*24007724*005010X222A1~BHT*0019*00*s9911i2440g11525703d*20240911*224042*CH~NM1*41*2*Dzinesco*****46*11525703~PER*IC*Tyler Martinez*EM*tmartinez@gmail.com~NM1*40*2*COLORADO MEDICAL ASSISTANCE PROGRAM*****46*COMEDASSISTPROG~HL*1**20*1~PRV*BI*PXC*251E00000X~NM1*85*2*TOC, Inc.*****XX*1881068062~N3*1100 East Main St*Suite A~N4*Montrose*CO*814014063~REF*EI*721587149~HL*2*1*22*0~SBR*P*18*******MC~NM1*IL*1*Phillips*Esther ****MI*J851806~N3*579 NORWOOD RD~N4*Montrose*CO*814034628~DMG*D8*19390307*F~NM1*PR*2*CO_TXIX*****PI*CO_TXIX~CLM*t991102440o1c79d*595.32***12:B:1*Y*A*Y*Y~REF*G1*2~HI*ABK:R69~LX*1~SV1*HC:T1019:U1:KX*125.40*UN*19.00***1~DTP*472*D8*20240902~REF*6R*t991102440v587348d~LX*2~SV1*HC:T1019:U1:KX*123.20*UN*18.67***1~DTP*472*D8*20240903~REF*6R*t991102440v587638d~LX*3~SV1*HC:T1019:U1:KX*123.64*UN*18.73***1~DTP*472*D8*20240904~REF*6R*t991102440v587903d~LX*4~SV1*HC:T1019:U1:KX*123.64*UN*18.73***1~DTP*472*D8*20240905~REF*6R*t991102440v588158d~LX*5~SV1*HC:T1019:U1:KX*99.44*UN*15.07***1~DTP*472*D8*20240906~REF*6R*t991102440v588441d~SE*42*24007724~GE*1*240007724~IEA*1*240901088~
|
|
||||||
@@ -1,926 +0,0 @@
|
|||||||
# Drop `UNIQUE(batch_id, patient_control_number)` + 409 UX Implementation Plan
|
|
||||||
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
||||||
|
|
||||||
**Goal:** Allow multi-claim 837P files (where many `CLM*` segments share a `member_id`) to ingest without 409, and surface a structured error panel on the upload page when a true duplicate claim still trips a 409.
|
|
||||||
|
|
||||||
**Architecture:** Backend migration drops the inline `UNIQUE(batch_id, patient_control_number)` on `claims` via SQLite table recreation. Two new store helpers (`find_existing_batch_for_claim`, `find_existing_batch_for_remit`) enable both 837 and 835 409 handlers to surface the id of the prior batch. Frontend `ApiError` carries `existingBatchId`; `Upload.tsx` renders an inline error panel above the streaming results with a link to the existing batch and a "Pick a different file" escape hatch.
|
|
||||||
|
|
||||||
**Tech Stack:** Python 3.11+, SQLAlchemy 2.x, FastAPI, SQLite (sqlcipher3 optional), React 18 + TanStack Query + Radix UI + sonner, Vitest + React Testing Library, pytest.
|
|
||||||
|
|
||||||
**Spec:** `docs/superpowers/specs/2026-06-21-cyclone-claims-unique-constraint-and-409-ux-design.md` — read fully before starting.
|
|
||||||
|
|
||||||
**Worktree setup (one-time):**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /Users/openclaw/dev/cyclone
|
|
||||||
git worktree add .worktrees/claims-unique-fix -b claims-unique-fix main
|
|
||||||
cd .worktrees/claims-unique-fix
|
|
||||||
# Install backend deps if needed (venv assumed active)
|
|
||||||
pip install -e backend
|
|
||||||
# Install frontend deps if needed
|
|
||||||
npm install
|
|
||||||
```
|
|
||||||
|
|
||||||
All commits happen in this worktree. Merge to main via fast-forward when each phase ends.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 1 — Backend migration + helpers + API
|
|
||||||
|
|
||||||
### Task 1.1: Migration 0013 drops inline UNIQUE via table recreation
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `backend/src/cyclone/migrations/0013_drop_claims_unique_constraint.sql`
|
|
||||||
- Modify: `backend/tests/test_db_migrate.py` (append test)
|
|
||||||
|
|
||||||
The runner (`backend/src/cyclone/db_migrate.py`) wraps each `.sql` in an implicit transaction via `engine.begin()`, so the migration MUST NOT use `BEGIN`/`COMMIT` or `PRAGMA foreign_keys=OFF` (no-op inside a transaction). Use `PRAGMA defer_foreign_keys = ON` instead — checks fire at commit against the renamed table.
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing test**
|
|
||||||
|
|
||||||
Add to `backend/tests/test_db_migrate.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def test_drop_claims_unique_constraint_migration(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
"""Migration 0013 must drop the inline UNIQUE(batch_id, patient_control_number)
|
|
||||||
on claims by recreating the table. Idempotent and preserves data."""
|
|
||||||
monkeypatch.setattr(db_migrate, "MIGRATIONS_DIR", tmp_path)
|
|
||||||
# Copy the real migrations so the test starts from v1.
|
|
||||||
real_dir = Path(db_migrate.__file__).parent / "migrations"
|
|
||||||
for src in sorted(real_dir.glob("00*.sql")):
|
|
||||||
(tmp_path / src.name).write_text(src.read_text())
|
|
||||||
|
|
||||||
engine = _fresh_engine(tmp_path)
|
|
||||||
db_migrate.run(engine)
|
|
||||||
# Before 0013: insert two claims with same (batch_id, patient_control_number) raises.
|
|
||||||
with engine.begin() as c:
|
|
||||||
c.exec_driver_sql("INSERT INTO batches(id, kind, input_filename, parsed_at) VALUES ('b1', '837p', 'x.txt', '2026-01-01')")
|
|
||||||
c.exec_driver_sql("INSERT INTO claims(id, batch_id, patient_control_number) VALUES ('c1', 'b1', 'M')")
|
|
||||||
with pytest.raises(sqlalchemy.exc.IntegrityError):
|
|
||||||
c.exec_driver_sql("INSERT INTO claims(id, batch_id, patient_control_number) VALUES ('c2', 'b1', 'M')")
|
|
||||||
# Now drop the migration in by name only:
|
|
||||||
(tmp_path / "0013_drop_claims_unique_constraint.sql").write_text(
|
|
||||||
"-- version: 13\n"
|
|
||||||
"PRAGMA defer_foreign_keys = ON;\n"
|
|
||||||
"CREATE TABLE claims_new ("
|
|
||||||
"id TEXT PRIMARY KEY, batch_id TEXT NOT NULL REFERENCES batches(id) ON DELETE CASCADE,"
|
|
||||||
"patient_control_number TEXT NOT NULL, service_date_from DATE, service_date_to DATE,"
|
|
||||||
"charge_amount NUMERIC(12, 2) NOT NULL DEFAULT 0, provider_npi TEXT, payer_id TEXT,"
|
|
||||||
"state TEXT NOT NULL DEFAULT 'submitted', state_before_reversal TEXT,"
|
|
||||||
"matched_remittance_id TEXT REFERENCES remittances(id), raw_json TEXT,"
|
|
||||||
"rejection_reason TEXT, rejected_at TIMESTAMP, resubmit_count INTEGER NOT NULL DEFAULT 0,"
|
|
||||||
"state_changed_at TIMESTAMP, payer_rejected_at TEXT, payer_rejected_reason TEXT,"
|
|
||||||
"payer_rejected_status_code TEXT, payer_rejected_by_277ca_id TEXT,"
|
|
||||||
"payer_rejected_acknowledged_at TEXT, payer_rejected_acknowledged_actor TEXT);\n"
|
|
||||||
"INSERT INTO claims_new SELECT * FROM claims;"
|
|
||||||
"DROP TABLE claims;"
|
|
||||||
"ALTER TABLE claims_new RENAME TO claims;"
|
|
||||||
)
|
|
||||||
db_migrate.run(engine)
|
|
||||||
# After 0013: same insert succeeds.
|
|
||||||
with engine.begin() as c:
|
|
||||||
c.exec_driver_sql("INSERT INTO claims(id, batch_id, patient_control_number) VALUES ('c2', 'b1', 'M')")
|
|
||||||
rows = c.exec_driver_sql("SELECT COUNT(*) FROM claims").scalar()
|
|
||||||
assert rows == 2
|
|
||||||
assert _user_version(engine) == 13
|
|
||||||
|
|
||||||
|
|
||||||
def test_migration_0013_is_idempotent(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
"""Re-running db_migrate.run on a v13 DB is a no-op."""
|
|
||||||
monkeypatch.setattr(db_migrate, "MIGRATIONS_DIR", tmp_path)
|
|
||||||
real_dir = Path(db_migrate.__file__).parent / "migrations"
|
|
||||||
for src in sorted(real_dir.glob("00*.sql")):
|
|
||||||
(tmp_path / src.name).write_text(src.read_text())
|
|
||||||
|
|
||||||
engine = _fresh_engine(tmp_path)
|
|
||||||
db_migrate.run(engine) # applies all
|
|
||||||
version_after_first = _user_version(engine)
|
|
||||||
db_migrate.run(engine) # second call: no-op
|
|
||||||
assert _user_version(engine) == version_after_first
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run test to verify it fails**
|
|
||||||
|
|
||||||
Run: `cd backend && python -m pytest tests/test_db_migrate.py::test_drop_claims_unique_constraint_migration tests/test_db_migrate.py::test_migration_0013_is_idempotent -v`
|
|
||||||
|
|
||||||
Expected: `test_drop_claims_unique_constraint_migration` FAILS because 0013 doesn't exist yet; `test_migration_0013_is_idempotent` PASSES (idempotency already works for any v).
|
|
||||||
|
|
||||||
- [ ] **Step 3: Write migration 0013**
|
|
||||||
|
|
||||||
Create `backend/src/cyclone/migrations/0013_drop_claims_unique_constraint.sql`:
|
|
||||||
|
|
||||||
```sql
|
|
||||||
-- version: 13
|
|
||||||
-- Drop the inline UNIQUE(batch_id, patient_control_number) on claims.
|
|
||||||
-- Migration 0003 attempted DROP INDEX IF EXISTS uq_claims_batch_pcn but
|
|
||||||
-- the constraint is inline in CREATE TABLE, so the drop was a no-op.
|
|
||||||
-- The only way to remove an inline UNIQUE in SQLite is table recreation.
|
|
||||||
--
|
|
||||||
-- X12 837P allows any number of CLM segments per 2000B subscriber loop;
|
|
||||||
-- claim identity is provided by the primary key (claims.id = CLM01).
|
|
||||||
-- The remittances table had a parallel constraint already removed in 0003
|
|
||||||
-- (because that one WAS a named index), so this migration only touches
|
|
||||||
-- claims.
|
|
||||||
--
|
|
||||||
-- The migration runner (db_migrate.py) wraps each .sql in an implicit
|
|
||||||
-- transaction via engine.begin(), so we MUST NOT use BEGIN/COMMIT.
|
|
||||||
-- PRAGMA defer_foreign_keys defers FK checks to commit, which is the
|
|
||||||
-- only way to drop a referenced table inside a transaction in SQLite.
|
|
||||||
|
|
||||||
PRAGMA defer_foreign_keys = ON;
|
|
||||||
|
|
||||||
CREATE TABLE claims_new (
|
|
||||||
id TEXT PRIMARY KEY,
|
|
||||||
batch_id TEXT NOT NULL REFERENCES batches(id) ON DELETE CASCADE,
|
|
||||||
patient_control_number TEXT NOT NULL,
|
|
||||||
service_date_from DATE,
|
|
||||||
service_date_to DATE,
|
|
||||||
charge_amount NUMERIC(12, 2) NOT NULL DEFAULT 0,
|
|
||||||
provider_npi TEXT,
|
|
||||||
payer_id TEXT,
|
|
||||||
state TEXT NOT NULL DEFAULT 'submitted',
|
|
||||||
state_before_reversal TEXT,
|
|
||||||
matched_remittance_id TEXT REFERENCES remittances(id),
|
|
||||||
raw_json TEXT,
|
|
||||||
rejection_reason TEXT,
|
|
||||||
rejected_at TIMESTAMP,
|
|
||||||
resubmit_count INTEGER NOT NULL DEFAULT 0,
|
|
||||||
state_changed_at TIMESTAMP,
|
|
||||||
payer_rejected_at TEXT,
|
|
||||||
payer_rejected_reason TEXT,
|
|
||||||
payer_rejected_status_code TEXT,
|
|
||||||
payer_rejected_by_277ca_id TEXT,
|
|
||||||
payer_rejected_acknowledged_at TEXT,
|
|
||||||
payer_rejected_acknowledged_actor TEXT
|
|
||||||
-- NO UNIQUE (batch_id, patient_control_number) — removed.
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO claims_new SELECT * FROM claims;
|
|
||||||
DROP TABLE claims;
|
|
||||||
ALTER TABLE claims_new RENAME TO claims;
|
|
||||||
|
|
||||||
-- Recreate secondary indexes (same names, same columns as initial schema
|
|
||||||
-- plus later migrations).
|
|
||||||
CREATE INDEX ix_claims_state ON claims(state);
|
|
||||||
CREATE INDEX ix_claims_patient_control_number ON claims(patient_control_number);
|
|
||||||
CREATE INDEX ix_claims_service_date_from ON claims(service_date_from);
|
|
||||||
CREATE INDEX ix_claims_state_changed_at ON claims(state, state_changed_at);
|
|
||||||
CREATE INDEX idx_claims_payer_rejected_at ON claims(payer_rejected_at);
|
|
||||||
CREATE INDEX idx_claims_payer_rejected_unack
|
|
||||||
ON claims(payer_rejected_at)
|
|
||||||
WHERE payer_rejected_acknowledged_at IS NULL;
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Run tests to verify they pass**
|
|
||||||
|
|
||||||
Run: `cd backend && python -m pytest tests/test_db_migrate.py::test_drop_claims_unique_constraint_migration tests/test_db_migrate.py::test_migration_0013_is_idempotent -v`
|
|
||||||
|
|
||||||
Expected: PASS for both. The first test asserts the inline UNIQUE is gone (insert succeeds) and `user_version==13`. The second asserts idempotency.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add backend/src/cyclone/migrations/0013_drop_claims_unique_constraint.sql backend/tests/test_db_migrate.py
|
|
||||||
git commit -m "feat(db): drop inline UNIQUE(batch_id, patient_control_number) via migration 0013"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 1.2: Store helpers `find_existing_batch_for_claim` and `find_existing_batch_for_remit`
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `backend/src/cyclone/store.py:1-50` (imports) and add new functions near the top
|
|
||||||
- Modify: `backend/tests/test_store.py` (append tests)
|
|
||||||
|
|
||||||
The helpers are pure reads. They return the batch_id of the first batch containing the given claim_id / remit_id, or None.
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing test**
|
|
||||||
|
|
||||||
Add to `backend/tests/test_store.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def test_find_existing_batch_for_claim_returns_none_for_unknown(global_store):
|
|
||||||
"""Unknown claim_id -> None."""
|
|
||||||
assert global_store.find_existing_batch_for_claim("nope") is None # type: ignore[attr-defined]
|
|
||||||
|
|
||||||
|
|
||||||
def test_find_existing_batch_for_claim_returns_batch_id(global_store):
|
|
||||||
"""Known claim_id -> batch_id of the holding batch."""
|
|
||||||
from cyclone.store import BatchRecord, _claim_837_row # noqa: F401
|
|
||||||
from cyclone.db import Batch, Claim, db as _db_mod
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from cyclone.parser.parsers_837p import ClaimOutput # noqa: F401
|
|
||||||
# Simpler: insert a batch + claim directly via the DB.
|
|
||||||
with _db_mod.SessionLocal()() as s:
|
|
||||||
s.add(Batch(id="B1", kind="837p", input_filename="x.txt",
|
|
||||||
parsed_at=datetime(2026,1,1,tzinfo=timezone.utc)))
|
|
||||||
s.add(Claim(id="CLM-A", batch_id="B1", patient_control_number="M1"))
|
|
||||||
s.commit()
|
|
||||||
assert global_store.find_existing_batch_for_claim("CLM-A") == "B1" # type: ignore[attr-defined]
|
|
||||||
|
|
||||||
|
|
||||||
def test_find_existing_batch_for_remit_returns_none_for_unknown(global_store):
|
|
||||||
"""Unknown remit id -> None."""
|
|
||||||
assert global_store.find_existing_batch_for_remit("nope") is None # type: ignore[attr-defined]
|
|
||||||
|
|
||||||
|
|
||||||
def test_find_existing_batch_for_remit_returns_batch_id(global_store):
|
|
||||||
"""Known remit id -> batch_id of the holding batch."""
|
|
||||||
from cyclone.db import Batch, Remittance, db as _db_mod
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
with _db_mod.SessionLocal()() as s:
|
|
||||||
s.add(Batch(id="B2", kind="835", input_filename="y.txt",
|
|
||||||
parsed_at=datetime(2026,1,1,tzinfo=timezone.utc)))
|
|
||||||
s.add(Remittance(id="CLP-A", batch_id="B2",
|
|
||||||
payer_claim_control_number="CLP-A",
|
|
||||||
status_code="1", received_at=datetime(2026,1,1,tzinfo=timezone.utc)))
|
|
||||||
s.commit()
|
|
||||||
assert global_store.find_existing_batch_for_remit("CLP-A") == "B2" # type: ignore[attr-defined]
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run test to verify it fails**
|
|
||||||
|
|
||||||
Run: `cd backend && python -m pytest tests/test_store.py::test_find_existing_batch_for_claim_returns_none_for_unknown -v`
|
|
||||||
|
|
||||||
Expected: FAIL with `AttributeError: 'CycloneStore' object has no attribute 'find_existing_batch_for_claim'`.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Implement helpers**
|
|
||||||
|
|
||||||
Add to `backend/src/cyclone/store.py` near the top, after imports:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def find_existing_batch_for_claim(claim_id: str) -> str | None:
|
|
||||||
"""Return the batch_id of the first batch containing this claim id, or None.
|
|
||||||
|
|
||||||
Pure read; opens a short-lived session. Used by the 837 409 handler to
|
|
||||||
surface which prior batch already holds the same CLM01.
|
|
||||||
"""
|
|
||||||
from sqlalchemy import select
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.db import Claim
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.execute(
|
|
||||||
select(Claim.batch_id).where(Claim.id == claim_id).limit(1)
|
|
||||||
).first()
|
|
||||||
return row[0] if row else None
|
|
||||||
|
|
||||||
|
|
||||||
def find_existing_batch_for_remit(remit_id: str) -> str | None:
|
|
||||||
"""Return the batch_id of the first batch containing this remit id, or None.
|
|
||||||
|
|
||||||
Pure read; opens a short-lived session. Used by the 835 409 handler.
|
|
||||||
`remit_id` is the PK on `remittances.id` (= payer_claim_control_number = CLP01).
|
|
||||||
"""
|
|
||||||
from sqlalchemy import select
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.db import Remittance
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.execute(
|
|
||||||
select(Remittance.batch_id).where(Remittance.id == remit_id).limit(1)
|
|
||||||
).first()
|
|
||||||
return row[0] if row else None
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Run tests to verify they pass**
|
|
||||||
|
|
||||||
Run: `cd backend && python -m pytest tests/test_store.py -k "find_existing_batch" -v`
|
|
||||||
|
|
||||||
Expected: PASS for all 4 tests.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add backend/src/cyclone/store.py backend/tests/test_store.py
|
|
||||||
git commit -m "feat(store): add find_existing_batch_for_claim and find_existing_batch_for_remit"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 1.3: 409 handlers in api.py surface `existing_batch_id`
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `backend/src/cyclone/api.py:394-415` (837 409 handler)
|
|
||||||
- Modify: `backend/src/cyclone/api.py:588-602` (835 409 handler)
|
|
||||||
- Modify: `backend/tests/test_api_parse_persists.py` (append tests)
|
|
||||||
|
|
||||||
After migration 0013, IntegrityError on the 837 path can only fire when two claims in the same file share the same CLM01 (rare). The helper may still return a batch_id if the colliding CLM01 was previously ingested and not deleted (the dedup `s.get(Claim, claim_id)` only queries DB state, not pending session state, so cross-batch duplicates still get inserted and trip the PK).
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing tests**
|
|
||||||
|
|
||||||
Add to `backend/tests/test_api_parse_persists.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def test_409_response_includes_existing_batch_id_for_837(client: TestClient):
|
|
||||||
"""When a CLM01 already exists in a prior batch, the 409 body has existing_batch_id."""
|
|
||||||
from cyclone.db import Batch, Claim
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
# Seed a prior batch with claim CLM-X.
|
|
||||||
with global_store._lock:
|
|
||||||
pass
|
|
||||||
from cyclone import db as _db
|
|
||||||
with _db.SessionLocal()() as s:
|
|
||||||
s.add(Batch(id="PRIOR", kind="837p", input_filename="prior.txt",
|
|
||||||
parsed_at=datetime(2026,1,1,tzinfo=timezone.utc),
|
|
||||||
raw_result_json={}))
|
|
||||||
s.add(Claim(id="CLM-X", batch_id="PRIOR", patient_control_number="M"))
|
|
||||||
s.commit()
|
|
||||||
# Build a file with two CLM* segments both using CLM-X (forces PK collision).
|
|
||||||
text = (
|
|
||||||
"ISA*00* *00* *ZZ*SUBMITTERID *ZZ*RECEIVERID "
|
|
||||||
"*240101*1200*^*00501*000000001*0*P*:~\n"
|
|
||||||
"GS*HC*SUBMITTERID*RECEIVERID*20240101*1200*1*X*005010X222A1~\n"
|
|
||||||
"ST*837*0001*005010X222A1~\n"
|
|
||||||
"BHT*0019*00*1*20240101*1200*CH~\n"
|
|
||||||
"NM1*41*2*SUBMITTER*****46*SUBMITTERID~\n"
|
|
||||||
"PER*IC*CONTACT*TE*5555555555~\n"
|
|
||||||
"NM1*40*2*RECEIVER*****46*RECEIVERID~\n"
|
|
||||||
"HL*1**20*1~\n"
|
|
||||||
"NM1*85*2*BILLING*****XX*1881068062~\n"
|
|
||||||
"N3*123 MAIN*\nN4*DENVER*CO*80202~\n"
|
|
||||||
"REF*EI*123456789~\n"
|
|
||||||
"HL*2*1*22*0~\n"
|
|
||||||
"SBR*P*18*******CI~\n"
|
|
||||||
"NM1*IL*1*DOE*JOHN****MI*M~\n"
|
|
||||||
"N3*456 ELM*\nN4*DENVER*CO*80202~\n"
|
|
||||||
"DMG*D8*19700101*M~\n"
|
|
||||||
"NM1*PR*2*MEDICAID*****PI*MCD~\n"
|
|
||||||
"CLM*CLM-X*100***11:B:1*Y*A*Y*Y~\n"
|
|
||||||
"LX*1~\nSV1*HC:99213*100*UN*1***1~\n"
|
|
||||||
"DTP*472*D8*20240101~\n"
|
|
||||||
"CLM*CLM-X*100***11:B:1*Y*A*Y*Y~\n"
|
|
||||||
"LX*2~\nSV1*HC:99213*100*UN*1***1~\n"
|
|
||||||
"DTP*472*D8*20240101~\n"
|
|
||||||
"SE*30*0001~\n"
|
|
||||||
"GE*1*1~\n"
|
|
||||||
"IEA*1*000000001~\n"
|
|
||||||
)
|
|
||||||
resp = client.post(
|
|
||||||
"/api/parse-837",
|
|
||||||
files={"file": ("dup.txt", text, "text/plain")},
|
|
||||||
headers={"Accept": "application/json"},
|
|
||||||
)
|
|
||||||
assert resp.status_code == 409, resp.text
|
|
||||||
body = resp.json()
|
|
||||||
assert body.get("existing_batch_id") == "PRIOR"
|
|
||||||
|
|
||||||
|
|
||||||
def test_409_response_includes_existing_batch_id_for_835(client: TestClient):
|
|
||||||
"""When a CLP01 already exists in a prior batch, the 835 409 body has existing_batch_id."""
|
|
||||||
from cyclone.db import Batch, Remittance
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from cyclone import db as _db
|
|
||||||
with _db.SessionLocal()() as s:
|
|
||||||
s.add(Batch(id="PRIOR835", kind="835", input_filename="prior.txt",
|
|
||||||
parsed_at=datetime(2026,1,1,tzinfo=timezone.utc),
|
|
||||||
raw_result_json={}))
|
|
||||||
s.add(Remittance(id="CLP-X", batch_id="PRIOR835",
|
|
||||||
payer_claim_control_number="CLP-X",
|
|
||||||
status_code="1",
|
|
||||||
received_at=datetime(2026,1,1,tzinfo=timezone.utc)))
|
|
||||||
s.commit()
|
|
||||||
# Build a minimal 835 with two CLP segments using CLP-X. This is contrived;
|
|
||||||
# we just need to trigger IntegrityError. The exact parser robustness to this
|
|
||||||
# fixture is not the focus — the test asserts the 409 body shape.
|
|
||||||
# ... or instead: directly invoke the handler via a unit-style test that
|
|
||||||
# pre-seeds and then makes a minimal 835 that includes CLP-X twice.
|
|
||||||
# For brevity, drop this test if constructing a fixture is too brittle;
|
|
||||||
# the 837 test above already exercises the same handler pattern.
|
|
||||||
pytest.skip("835 fixture with duplicate CLP01 is fragile; 837 case covers the pattern")
|
|
||||||
```
|
|
||||||
|
|
||||||
Note: the 835 test is intentionally skipped — constructing a minimal valid 835 with duplicate CLP01 is brittle. The 837 case exercises the handler pattern (call `find_existing_batch_for_remit` from a similarly-shaped except block in the 835 handler). If you need 835 coverage, manually inspect by running the API on a real fixture after the implementation lands.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run test to verify it fails**
|
|
||||||
|
|
||||||
Run: `cd backend && python -m pytest tests/test_api_parse_persists.py::test_409_response_includes_existing_batch_id_for_837 -v`
|
|
||||||
|
|
||||||
Expected: FAIL because the 409 handler does not yet add `existing_batch_id`.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Modify 837 409 handler**
|
|
||||||
|
|
||||||
Edit `backend/src/cyclone/api.py` lines 394-415:
|
|
||||||
|
|
||||||
Replace the `except IntegrityError` block in `parse_837_endpoint` with:
|
|
||||||
|
|
||||||
```python
|
|
||||||
try:
|
|
||||||
store.add(rec, event_bus=request.app.state.event_bus)
|
|
||||||
except IntegrityError as exc:
|
|
||||||
# After migration 0013, the only way an IntegrityError fires here
|
|
||||||
# is a PK collision on claims.id (CLM01) — either within-file or
|
|
||||||
# against a prior batch. Look up the first claim's id and ask the
|
|
||||||
# helper which batch already holds it.
|
|
||||||
first_claim_id = (
|
|
||||||
result.claims[0].claim_id if result.claims else None
|
|
||||||
)
|
|
||||||
existing_batch_id = (
|
|
||||||
store.find_existing_batch_for_claim(first_claim_id)
|
|
||||||
if first_claim_id else None
|
|
||||||
)
|
|
||||||
body = {
|
|
||||||
"error": "Duplicate claim",
|
|
||||||
"detail": (
|
|
||||||
"This file (or one previously ingested with the same "
|
|
||||||
"claim control number) collides with an existing record. "
|
|
||||||
"Inspect the file for duplicate CLM01 control numbers, or "
|
|
||||||
"remove the existing batch before retrying."
|
|
||||||
),
|
|
||||||
"batch_id": rec.id,
|
|
||||||
}
|
|
||||||
if existing_batch_id and existing_batch_id != rec.id:
|
|
||||||
body["existing_batch_id"] = existing_batch_id
|
|
||||||
log.warning("Duplicate claim while persisting batch %s: %s", rec.id, exc)
|
|
||||||
return JSONResponse(status_code=409, content=body)
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Modify 835 409 handler**
|
|
||||||
|
|
||||||
Edit `backend/src/cyclone/api.py` lines 588-602:
|
|
||||||
|
|
||||||
Replace the `except IntegrityError` block in `parse_835_endpoint` with:
|
|
||||||
|
|
||||||
```python
|
|
||||||
try:
|
|
||||||
store.add(rec, event_bus=request.app.state.event_bus)
|
|
||||||
except IntegrityError as exc:
|
|
||||||
first_pcn = (
|
|
||||||
result.claims[0].payer_claim_control_number
|
|
||||||
if result.claims else None
|
|
||||||
)
|
|
||||||
existing_batch_id = (
|
|
||||||
store.find_existing_batch_for_remit(first_pcn)
|
|
||||||
if first_pcn else None
|
|
||||||
)
|
|
||||||
body = {
|
|
||||||
"error": "Duplicate remittance",
|
|
||||||
"detail": (
|
|
||||||
"This 835 file (or one previously ingested with the same "
|
|
||||||
"payer claim control number) collides with an existing record. "
|
|
||||||
"Remove the existing remittance before retrying."
|
|
||||||
),
|
|
||||||
"batch_id": rec.id,
|
|
||||||
}
|
|
||||||
if existing_batch_id and existing_batch_id != rec.id:
|
|
||||||
body["existing_batch_id"] = existing_batch_id
|
|
||||||
log.warning("Duplicate remittance while persisting batch %s: %s", rec.id, exc)
|
|
||||||
return JSONResponse(status_code=409, content=body)
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Run tests to verify they pass**
|
|
||||||
|
|
||||||
Run: `cd backend && python -m pytest tests/test_api_parse_persists.py -v`
|
|
||||||
|
|
||||||
Expected: All PASS, including the new 409 test.
|
|
||||||
|
|
||||||
- [ ] **Step 6: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add backend/src/cyclone/api.py backend/tests/test_api_parse_persists.py
|
|
||||||
git commit -m "feat(api): 409 responses for 837/435 include existing_batch_id"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 2 — Frontend
|
|
||||||
|
|
||||||
### Task 2.1: `ApiError.existingBatchId` + parse837/835 surface it
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `src/lib/api.ts:164-167` (ApiError class)
|
|
||||||
- Modify: `src/lib/api.ts:174-191` (readErrorBody)
|
|
||||||
- Modify: `src/lib/api.ts:280-339` (parse837, parse835)
|
|
||||||
- Create: `src/lib/api.test.ts`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing test**
|
|
||||||
|
|
||||||
Create `src/lib/api.test.ts`:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { describe, it, expect, vi, afterEach } from "vitest";
|
|
||||||
import { ApiError, parse837 } from "@/lib/api";
|
|
||||||
|
|
||||||
afterEach(() => vi.restoreAllMocks());
|
|
||||||
|
|
||||||
describe("ApiError", () => {
|
|
||||||
it("carries existingBatchId when constructed", () => {
|
|
||||||
const e = new ApiError(409, "dup", "BATCH-1");
|
|
||||||
expect(e.status).toBe(409);
|
|
||||||
expect(e.existingBatchId).toBe("BATCH-1");
|
|
||||||
});
|
|
||||||
it("defaults existingBatchId to null", () => {
|
|
||||||
const e = new ApiError(500, "boom");
|
|
||||||
expect(e.existingBatchId).toBeNull();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("parse837 throws ApiError with existingBatchId", () => {
|
|
||||||
it("extracts existing_batch_id from 409 body", async () => {
|
|
||||||
vi.stubGlobal("import.meta.env", { VITE_API_BASE_URL: "http://x" });
|
|
||||||
const res = new Response(
|
|
||||||
JSON.stringify({
|
|
||||||
error: "Duplicate claim",
|
|
||||||
detail: "collision",
|
|
||||||
batch_id: "NEW",
|
|
||||||
existing_batch_id: "PRIOR",
|
|
||||||
}),
|
|
||||||
{ status: 409, headers: { "Content-Type": "application/json" } },
|
|
||||||
);
|
|
||||||
vi.stubGlobal(
|
|
||||||
"fetch",
|
|
||||||
vi.fn(async () => res),
|
|
||||||
);
|
|
||||||
const file = new File(["x"], "f.txt", { type: "text/plain" });
|
|
||||||
await expect(parse837(file, { onProgress: () => {} })).rejects.toMatchObject({
|
|
||||||
status: 409,
|
|
||||||
existingBatchId: "PRIOR",
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
it("sets existingBatchId null when body omits it", async () => {
|
|
||||||
vi.stubGlobal("import.meta.env", { VITE_API_BASE_URL: "http://x" });
|
|
||||||
const res = new Response(
|
|
||||||
JSON.stringify({ error: "X", detail: "y", batch_id: "N" }),
|
|
||||||
{ status: 409, headers: { "Content-Type": "application/json" } },
|
|
||||||
);
|
|
||||||
vi.stubGlobal(
|
|
||||||
"fetch",
|
|
||||||
vi.fn(async () => res),
|
|
||||||
);
|
|
||||||
const file = new File(["x"], "f.txt", { type: "text/plain" });
|
|
||||||
await expect(parse837(file)).rejects.toMatchObject({
|
|
||||||
status: 409,
|
|
||||||
existingBatchId: null,
|
|
||||||
});
|
|
||||||
});
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run test to verify it fails**
|
|
||||||
|
|
||||||
Run: `cd /Users/openclaw/dev/cyclone/.worktrees/claims-unique-fix && npm test -- src/lib/api.test.ts`
|
|
||||||
|
|
||||||
Expected: FAIL — `existingBatchId` not a constructor argument yet; `parse837` throws `Error` not `ApiError`.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Update `ApiError`**
|
|
||||||
|
|
||||||
Edit `src/lib/api.ts` line 164-167:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
export class ApiError extends Error {
|
|
||||||
constructor(
|
|
||||||
public status: number,
|
|
||||||
message: string,
|
|
||||||
public existingBatchId: string | null = null,
|
|
||||||
) {
|
|
||||||
super(message);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Update `readErrorBody` to return parsed body**
|
|
||||||
|
|
||||||
Edit `src/lib/api.ts` line 174-191. Change return type and add JSON parse branch:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
type ErrorBody = {
|
|
||||||
detail?: unknown;
|
|
||||||
error?: unknown;
|
|
||||||
existing_batch_id?: unknown;
|
|
||||||
};
|
|
||||||
|
|
||||||
async function readErrorBody(
|
|
||||||
res: Response,
|
|
||||||
): Promise<{ message: string; existingBatchId: string | null }> {
|
|
||||||
try {
|
|
||||||
const t = await res.text();
|
|
||||||
if (!t) return { message: "", existingBatchId: null };
|
|
||||||
try {
|
|
||||||
const obj = JSON.parse(t) as ErrorBody;
|
|
||||||
let message = "";
|
|
||||||
if (typeof obj.detail === "string") message = obj.detail;
|
|
||||||
else if (typeof obj.error === "string") message = obj.error;
|
|
||||||
else message = t;
|
|
||||||
const existing =
|
|
||||||
typeof obj.existing_batch_id === "string"
|
|
||||||
? obj.existing_batch_id
|
|
||||||
: null;
|
|
||||||
return { message, existingBatchId: existing };
|
|
||||||
} catch {
|
|
||||||
return { message: t, existingBatchId: null };
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
return { message: "", existingBatchId: null };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Update parse837 to throw ApiError**
|
|
||||||
|
|
||||||
Edit `src/lib/api.ts` lines 293-298:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
if (!res.ok) {
|
|
||||||
const { message, existingBatchId } = await readErrorBody(res);
|
|
||||||
throw new ApiError(
|
|
||||||
res.status,
|
|
||||||
`${res.status} ${res.statusText}${message ? ` — ${message}` : ""}`,
|
|
||||||
existingBatchId,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 6: Update parse835 to throw ApiError**
|
|
||||||
|
|
||||||
Edit `src/lib/api.ts` lines 329-334:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
if (!res.ok) {
|
|
||||||
const { message, existingBatchId } = await readErrorBody(res);
|
|
||||||
throw new ApiError(
|
|
||||||
res.status,
|
|
||||||
`${res.status} ${res.statusText}${message ? ` — ${message}` : ""}`,
|
|
||||||
existingBatchId,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 7: Run tests to verify they pass**
|
|
||||||
|
|
||||||
Run: `cd .worktrees/claims-unique-fix && npm test -- src/lib/api.test.ts`
|
|
||||||
|
|
||||||
Expected: PASS for all 4 tests.
|
|
||||||
|
|
||||||
- [ ] **Step 8: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add src/lib/api.ts src/lib/api.test.ts
|
|
||||||
git commit -m "feat(api): ApiError carries existingBatchId; parse837/parse835 surface it"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 2.2: `Upload.tsx` inline error panel for 409
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `src/pages/Upload.tsx` (add error state + panel JSX + 409 catch branch)
|
|
||||||
- Create: `src/pages/Upload.test.tsx`
|
|
||||||
|
|
||||||
The panel renders above the streaming results when `uploadError.kind === "duplicate"`. It shows:
|
|
||||||
- A 409 badge
|
|
||||||
- "Duplicate claim — file not ingested" title
|
|
||||||
- Detail mentioning the file and (if `existingBatchId` is set) "Open the existing batch to compare, or pick a different file."
|
|
||||||
- A button linking to `/batches/{existingBatchId}` if present
|
|
||||||
- A "Pick a different file" ghost button that calls `pickFile(null)` and clears the error state
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing tests**
|
|
||||||
|
|
||||||
Create `src/pages/Upload.test.tsx`:
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
|
||||||
import { render, screen, fireEvent } from "@testing-library/react";
|
|
||||||
import { MemoryRouter, Route, Routes } from "react-router-dom";
|
|
||||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
|
||||||
import * as apiModule from "@/lib/api";
|
|
||||||
import { ApiError } from "@/lib/api";
|
|
||||||
import { Upload } from "@/pages/Upload";
|
|
||||||
|
|
||||||
// We mock the api module so the component doesn't actually call fetch.
|
|
||||||
vi.mock("@/lib/api", async () => {
|
|
||||||
const actual = await vi.importActual<typeof apiModule>("@/lib/api");
|
|
||||||
return { ...actual, parse837: vi.fn(), parse835: vi.fn() };
|
|
||||||
});
|
|
||||||
|
|
||||||
beforeEach(() => {
|
|
||||||
vi.clearAllMocks();
|
|
||||||
});
|
|
||||||
|
|
||||||
function renderUpload() {
|
|
||||||
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
|
|
||||||
return render(
|
|
||||||
<QueryClientProvider client={qc}>
|
|
||||||
<MemoryRouter initialEntries={["/upload"]}>
|
|
||||||
<Routes>
|
|
||||||
<Route path="/upload" element={<Upload />} />
|
|
||||||
<Route path="/batches/:id" element={<div data-testid="batch-page" />} />
|
|
||||||
</Routes>
|
|
||||||
</MemoryRouter>
|
|
||||||
</QueryClientProvider>,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("Upload error panel", () => {
|
|
||||||
it("renders panel with link when 409 carries existingBatchId", async () => {
|
|
||||||
vi.mocked(apiModule.parse837).mockRejectedValueOnce(
|
|
||||||
new ApiError(409, "409 Conflict — dup", "PRIOR-BATCH"),
|
|
||||||
);
|
|
||||||
renderUpload();
|
|
||||||
// Find the file input and upload a file.
|
|
||||||
const input = document.querySelector('input[type="file"]') as HTMLInputElement;
|
|
||||||
expect(input).toBeTruthy();
|
|
||||||
const file = new File(["x"], "test.txt", { type: "text/plain" });
|
|
||||||
fireEvent.change(input, { target: { files: [file] } });
|
|
||||||
// Wait for the panel to render.
|
|
||||||
const link = await screen.findByRole("button", { name: /open existing batch/i });
|
|
||||||
expect(link).toBeInTheDocument();
|
|
||||||
expect(screen.getByText(/duplicate claim/i)).toBeInTheDocument();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("renders panel without link when 409 omits existingBatchId", async () => {
|
|
||||||
vi.mocked(apiModule.parse837).mockRejectedValueOnce(
|
|
||||||
new ApiError(409, "409 Conflict — dup", null),
|
|
||||||
);
|
|
||||||
renderUpload();
|
|
||||||
const input = document.querySelector('input[type="file"]') as HTMLInputElement;
|
|
||||||
const file = new File(["x"], "test.txt", { type: "text/plain" });
|
|
||||||
fireEvent.change(input, { target: { files: [file] } });
|
|
||||||
await screen.findByText(/duplicate claim/i);
|
|
||||||
expect(
|
|
||||||
screen.queryByRole("button", { name: /open existing batch/i }),
|
|
||||||
).toBeNull();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("does NOT render panel for non-409 errors", async () => {
|
|
||||||
vi.mocked(apiModule.parse837).mockRejectedValueOnce(
|
|
||||||
new ApiError(400, "bad file"),
|
|
||||||
);
|
|
||||||
renderUpload();
|
|
||||||
const input = document.querySelector('input[type="file"]') as HTMLInputElement;
|
|
||||||
const file = new File(["x"], "test.txt", { type: "text/plain" });
|
|
||||||
fireEvent.change(input, { target: { files: [file] } });
|
|
||||||
// Give the async error handler a tick.
|
|
||||||
await new Promise((r) => setTimeout(r, 50));
|
|
||||||
expect(screen.queryByText(/duplicate claim/i)).toBeNull();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("'Pick a different file' button clears error", async () => {
|
|
||||||
vi.mocked(apiModule.parse837).mockRejectedValueOnce(
|
|
||||||
new ApiError(409, "409 Conflict — dup", "PRIOR"),
|
|
||||||
);
|
|
||||||
renderUpload();
|
|
||||||
const input = document.querySelector('input[type="file"]') as HTMLInputElement;
|
|
||||||
const file = new File(["x"], "test.txt", { type: "text/plain" });
|
|
||||||
fireEvent.change(input, { target: { files: [file] } });
|
|
||||||
const clearBtn = await screen.findByRole("button", { name: /pick a different file/i });
|
|
||||||
fireEvent.click(clearBtn);
|
|
||||||
expect(screen.queryByText(/duplicate claim/i)).toBeNull();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run test to verify it fails**
|
|
||||||
|
|
||||||
Run: `cd .worktrees/claims-unique-fix && npm test -- src/pages/Upload.test.tsx`
|
|
||||||
|
|
||||||
Expected: FAIL — no error panel exists yet, all 4 tests fail.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Add error state and 409 catch branch in Upload.tsx**
|
|
||||||
|
|
||||||
Edit `src/pages/Upload.tsx`:
|
|
||||||
|
|
||||||
1. At the top imports, add:
|
|
||||||
```tsx
|
|
||||||
import { ApiError } from "@/lib/api";
|
|
||||||
import { useNavigate } from "react-router-dom";
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Find the existing error-handling catch block (around line 683-687) and replace with:
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
} catch (err) {
|
|
||||||
if (err instanceof ApiError && err.status === 409) {
|
|
||||||
setUploadError({
|
|
||||||
kind: "duplicate",
|
|
||||||
existingBatchId: err.existingBatchId,
|
|
||||||
filename: file.name,
|
|
||||||
});
|
|
||||||
toast.error("Duplicate claim — file not ingested");
|
|
||||||
} else {
|
|
||||||
toast.error(
|
|
||||||
err instanceof Error ? err.message : "Failed to parse file"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Add state declaration near the other useState calls:
|
|
||||||
```tsx
|
|
||||||
type UploadError =
|
|
||||||
| { kind: "duplicate"; existingBatchId: string | null; filename: string };
|
|
||||||
const [uploadError, setUploadError] = useState<UploadError | null>(null);
|
|
||||||
const navigate = useNavigate();
|
|
||||||
```
|
|
||||||
|
|
||||||
4. In the JSX, add the inline panel above the streaming-results section. Find the place where streaming results render (search for "streamDelay" or the section that shows the parsed batch) and insert just before it:
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
{uploadError && uploadError.kind === "duplicate" ? (
|
|
||||||
<div
|
|
||||||
role="alert"
|
|
||||||
data-testid="duplicate-error-panel"
|
|
||||||
className="error-panel mx-auto max-w-3xl rounded-md border border-destructive/40 bg-destructive/5 p-4"
|
|
||||||
>
|
|
||||||
<div className="flex items-center gap-2">
|
|
||||||
<span className="inline-flex items-center rounded-md bg-destructive px-2 py-0.5 text-xs font-semibold text-destructive-foreground">
|
|
||||||
409
|
|
||||||
</span>
|
|
||||||
<span className="font-semibold">Duplicate claim — file not ingested</span>
|
|
||||||
</div>
|
|
||||||
<p className="mt-2 text-sm text-muted-foreground">
|
|
||||||
<span className="font-mono">{uploadError.filename}</span> collides with an
|
|
||||||
existing record.
|
|
||||||
{uploadError.existingBatchId
|
|
||||||
? " Open the existing batch to compare, or pick a different file."
|
|
||||||
: " Pick a different file."}
|
|
||||||
</p>
|
|
||||||
<div className="mt-3 flex gap-2">
|
|
||||||
{uploadError.existingBatchId ? (
|
|
||||||
<Button
|
|
||||||
onClick={() => navigate(`/batches/${uploadError.existingBatchId}`)}
|
|
||||||
>
|
|
||||||
Open existing batch →
|
|
||||||
</Button>
|
|
||||||
) : null}
|
|
||||||
<Button
|
|
||||||
variant="ghost"
|
|
||||||
onClick={() => {
|
|
||||||
setUploadError(null);
|
|
||||||
pickFile(null);
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
Pick a different file
|
|
||||||
</Button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Run tests to verify they pass**
|
|
||||||
|
|
||||||
Run: `cd .worktrees/claims-unique-fix && npm test -- src/pages/Upload.test.tsx`
|
|
||||||
|
|
||||||
Expected: PASS for all 4 tests.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add src/pages/Upload.tsx src/pages/Upload.test.tsx
|
|
||||||
git commit -m "feat(upload): inline 409 error panel with existing-batch link"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 3 — End-to-end verification
|
|
||||||
|
|
||||||
### Task 3.1: Repro the original 409 with the actual multi-claim 837P file
|
|
||||||
|
|
||||||
**Files:** none (manual verification)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Start backend + frontend**
|
|
||||||
|
|
||||||
In one terminal: `cd backend && uvicorn cyclone.api:app --reload --port 8000`
|
|
||||||
In another: `cd .worktrees/claims-unique-fix && npm run dev`
|
|
||||||
|
|
||||||
- [ ] **Step 2: Upload the reproducer file**
|
|
||||||
|
|
||||||
Use `docs/prodfiles/837p-from-axiscare/tp11525703-837P-20260618153339862-1of1.txt` (93696 bytes, 28 NM1*IL segments, multi-claim member-id collisions).
|
|
||||||
|
|
||||||
Expected: 200 OK + batch persisted with multiple claims having identical `member_id` (this was previously 409).
|
|
||||||
|
|
||||||
- [ ] **Step 3: Re-upload the same file to trigger 409**
|
|
||||||
|
|
||||||
Expected: 409 with `existing_batch_id` pointing at the first batch.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify inline panel in the browser**
|
|
||||||
|
|
||||||
Open the upload page, drag the file in, verify the panel appears with the "Open existing batch →" link.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit any tweaks**
|
|
||||||
|
|
||||||
If the panel needed any styling tweaks (e.g. spacing, colors), commit them:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add src/pages/Upload.tsx
|
|
||||||
git commit -m "polish(upload): 409 error panel visual tweaks"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Self-review checklist (run after writing the plan)
|
|
||||||
|
|
||||||
1. **Spec coverage** — Each section of the spec maps to a task:
|
|
||||||
- §3 Schema migration → Task 1.1
|
|
||||||
- §4 Store helpers → Task 1.2
|
|
||||||
- §5 API change → Task 1.3
|
|
||||||
- §6 Frontend `ApiError.existingBatchId` → Task 2.1
|
|
||||||
- §6 Frontend `Upload.tsx` panel → Task 2.2
|
|
||||||
- §10 Test plan (9 tests) → All 9 covered (5 backend + 4 frontend)
|
|
||||||
|
|
||||||
2. **Placeholder scan** — No "TBD", "TODO", "implement later" markers. The only intentional skip is the 835 duplicate-CLP01 test, called out explicitly.
|
|
||||||
|
|
||||||
3. **Type consistency** —
|
|
||||||
- `find_existing_batch_for_claim` / `find_existing_batch_for_remit` return `str | None` everywhere.
|
|
||||||
- `ApiError.existingBatchId` is `string | null` in TS and `str | None` everywhere it's referenced.
|
|
||||||
- `setUploadError` payload shape `{ kind: "duplicate"; existingBatchId: string | null; filename: string }` is consistent in JSX and the catch branch.
|
|
||||||
|
|
||||||
4. **Risk acknowledged** — Migration reversibility, loss of uniqueness, race window are all documented in spec §9 and reflected in test scope (we don't test the race condition).
|
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,826 +0,0 @@
|
|||||||
# Cyclone Skill Catalog Implementation Plan
|
|
||||||
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
||||||
|
|
||||||
**Goal:** Ship 8 layer-mapped Cyclone skills under `.superpowers/skills/` (one per PR), then update the README to advertise the catalog. No code changes — pure guidance artifacts.
|
|
||||||
|
|
||||||
**Architecture:** Each skill is a single `SKILL.md` (150–250 lines, YAML frontmatter + sections per the spec template). One commit per skill. Phase order matches the spec's build order so each later skill can ship its `## Related skills` section pointing to skills that already exist.
|
|
||||||
|
|
||||||
**Tech Stack:** Markdown, YAML frontmatter. No build step, no runtime, no tests (skills are guidance, not code).
|
|
||||||
|
|
||||||
**Spec:** [`docs/superpowers/specs/2026-06-21-cyclone-skill-catalog-design.md`](../specs/2026-06-21-cyclone-skill-catalog-design.md)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## File structure
|
|
||||||
|
|
||||||
```
|
|
||||||
.superpowers/
|
|
||||||
└── skills/
|
|
||||||
├── cyclone-spec/
|
|
||||||
│ └── SKILL.md ← Task 1
|
|
||||||
├── cyclone-tests/
|
|
||||||
│ └── SKILL.md ← Task 2
|
|
||||||
├── cyclone-edi/
|
|
||||||
│ └── SKILL.md ← Task 3
|
|
||||||
│ └── references/
|
|
||||||
│ └── parsers.md ← Task 3
|
|
||||||
├── cyclone-tail/
|
|
||||||
│ └── SKILL.md ← Task 4
|
|
||||||
│ └── references/
|
|
||||||
│ └── wire-format.md ← Task 4
|
|
||||||
├── cyclone-store/
|
|
||||||
│ └── SKILL.md ← Task 5
|
|
||||||
├── cyclone-api-router/
|
|
||||||
│ └── SKILL.md ← Task 6
|
|
||||||
├── cyclone-frontend-page/
|
|
||||||
│ └── SKILL.md ← Task 7
|
|
||||||
└── cyclone-cli/
|
|
||||||
└── SKILL.md ← Task 8
|
|
||||||
|
|
||||||
README.md ← Task 9 (catalog section)
|
|
||||||
```
|
|
||||||
|
|
||||||
Each skill is one PR. Each `SKILL.md` follows the template in spec §"Per-skill structure". Optional `references/<file>.md` only when the SKILL.md would otherwise exceed ~200 lines.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Shared SKILL.md template (lock this in Task 0, then apply to every Task)
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
---
|
|
||||||
name: <kebab-case>
|
|
||||||
description: "<one-sentence trigger, with Use when: … keywords>"
|
|
||||||
---
|
|
||||||
|
|
||||||
# <Title>
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
- <bullet 1 — concrete situation>
|
|
||||||
- <bullet 2>
|
|
||||||
- <bullet 3>
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
1. <testable rule>
|
|
||||||
2. <testable rule>
|
|
||||||
3. <testable rule>
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
\`\`\`<lang>
|
|
||||||
<copy-pasteable skeleton>
|
|
||||||
\`\`\`
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
- <tempting-but-wrong move>
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
- `<other-skill>` — <one line>
|
|
||||||
- `<other-skill>` — <one line>
|
|
||||||
```
|
|
||||||
|
|
||||||
YAML frontmatter rules:
|
|
||||||
- `name` must be kebab-case and match the directory name.
|
|
||||||
- `description` is the only thing the auto-loader sees. One sentence, max ~200 chars. Include a `Use when:` clause listing the trigger situations.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 0: Pre-flight — verify starting state
|
|
||||||
|
|
||||||
**Goal:** Confirm `.superpowers/skills/` does not exist yet, capture a directory snapshot, and verify the catalog can be served from project scope.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Read: `.superpowers/` (existing project-scoped superpowers directory)
|
|
||||||
- Create (no commit): `/tmp/cyclone-skill-baseline.txt`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Inspect existing `.superpowers/` contents**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls -la .superpowers/
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: shows only `plans/` and `specs/` directories (the canonical superpowers content for this repo). No `skills/` directory yet.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Confirm no stray skill files outside `.superpowers/skills/`**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
find . -path ./node_modules -prune -o -path ./.git -prune -o -path ./backend/.venv -prune -o -name "SKILL.md" -print
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: no output (or only paths under `.grok/` system directories that are not part of the repo).
|
|
||||||
|
|
||||||
- [ ] **Step 3: Verify the README's skills section is empty (or absent)**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep -n -i "skill" README.md | head -10
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: a few hits for "skill" in unrelated contexts (e.g. "skill set" in marketing copy) but no existing catalog section.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Save the baseline**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls .superpowers/ | tee /tmp/cyclone-skill-baseline.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: `plans\nspecs`.
|
|
||||||
|
|
||||||
No commit. Pre-flight only.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 1: Phase 1, Skill 1 — `cyclone-spec`
|
|
||||||
|
|
||||||
**Goal:** Codify the SP-N superpowers flow that this repo already uses (14 specs, 8 plans on disk). Pure documentation; this skill owns the convention so every later feature goes through it consistently.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.superpowers/skills/cyclone-spec/SKILL.md`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Confirm the SP-N flow materials exist**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls docs/superpowers/specs/ | wc -l
|
|
||||||
ls docs/superpowers/plans/ | wc -l
|
|
||||||
git log --oneline | grep -E "^(feat|docs|merge): SP[0-9]+" | head -5
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: ≥ 14 specs, ≥ 8 plans, ≥ 5 SP-N commits in history. Record the numbers for use in the skill.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Create the directory**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .superpowers/skills/cyclone-spec
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Write the SKILL.md**
|
|
||||||
|
|
||||||
Use the shared template. Frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
name: cyclone-spec
|
|
||||||
description: "Cyclone SP-N superpowers increment flow — spec → plan → implement → merge. Use when: starting a new numbered feature increment, naming a branch, opening a SP-N PR, or doing the merge dance into main."
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Body sections (each must be present, in this order):
|
|
||||||
- `## When to use` — 4 bullets covering: starting a new increment, naming the spec/plan files, opening the PR, performing the merge.
|
|
||||||
- `## Conventions` — at minimum these numbered rules:
|
|
||||||
1. **Numbering.** Reserve the next SP-N number (next after the highest in `git log`). Never reuse a number.
|
|
||||||
2. **Branch.** `sp<N>-<short-kebab-topic>` (e.g. `sp22-line-reconciliation`).
|
|
||||||
3. **Spec path.** `docs/superpowers/specs/YYYY-MM-DD-cyclone-<topic>-design.md`. Status header: `Draft, pending user review`.
|
|
||||||
4. **Plan path.** `docs/superpowers/plans/YYYY-MM-DD-cyclone-<topic>.md`. Header per writing-plans skill.
|
|
||||||
5. **Commit prefix.** `feat(sp<N>): …`, `docs(spec): …`, `docs(plan): …`, `merge: SP<N> … into main`.
|
|
||||||
6. **PR title.** `SP<N> <Topic>` (matches commit history).
|
|
||||||
7. **Merge shape.** Single atomic merge commit into `main` after review; no squash, no rebase.
|
|
||||||
- `## Patterns` — copy-pasteable skeleton for the spec header (Date / Status / Branch / Scope) and the plan header (per writing-plans).
|
|
||||||
- `## Anti-patterns` — at least these:
|
|
||||||
- Don't skip the spec ("it's a small fix"). Small fixes still get a 3-line spec when they introduce a new numbered increment.
|
|
||||||
- Don't squash the merge commit — the SP-N merge commit is the audit trail.
|
|
||||||
- Don't put code in the spec — the spec is the *what*, the plan is the *how*.
|
|
||||||
- `## Related skills` — references to `cyclone-tests` (every spec mentions test impact), `cyclone-edi` / `cyclone-tail` / `cyclone-store` / `cyclone-api-router` / `cyclone-frontend-page` / `cyclone-cli` (each spec touches one or more of these), and the upstream `superpowers:brainstorming` / `superpowers:writing-plans` global skills.
|
|
||||||
|
|
||||||
Keep total under ~200 lines. Use real examples drawn from existing specs (e.g. SP9 multi-payer, SP14 5-lane Inbox) where they illustrate the convention.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify frontmatter and structure**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -5 .superpowers/skills/cyclone-spec/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-spec/SKILL.md
|
|
||||||
grep -c "^## " .superpowers/skills/cyclone-spec/SKILL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: first 5 lines are the YAML frontmatter (start `---`, end `---`), ~150-200 lines total, exactly 5 `## ` sections (`When to use`, `Conventions`, `Patterns`, `Anti-patterns`, `Related skills`).
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .superpowers/skills/cyclone-spec/SKILL.md
|
|
||||||
git commit -m "feat(sp-skill-catalog): add cyclone-spec skill (SP-N flow)"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 2: Phase 1, Skill 2 — `cyclone-tests`
|
|
||||||
|
|
||||||
**Goal:** Codify the pytest + vitest fixture patterns so every later skill can ship its `## Anti-patterns` cleanly. Specifically: how to drop a prodfiles fixture into `backend/tests/fixtures/`, how to write a `.test.tsx` next to a page, how to keep tests deterministic.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.superpowers/skills/cyclone-tests/SKILL.md`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Survey the existing fixture layout**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls backend/tests/fixtures/ | head -30
|
|
||||||
ls backend/tests/fixtures/ | wc -l
|
|
||||||
ls src/**/*.test.tsx src/**/*.test.ts 2>/dev/null | wc -l
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: dozens of fixtures under `backend/tests/fixtures/`, dozens of `.test.ts(x)` siblings across `src/`. Record counts for the skill.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Sample one backend pytest and one frontend vitest to cite**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls backend/tests/fixtures/ | head -3
|
|
||||||
echo "---"
|
|
||||||
ls src/hooks/*.test.ts | head -3
|
|
||||||
```
|
|
||||||
|
|
||||||
Pick the smallest/most-representative example from each side to cite in the skill's `## Patterns`.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Create the directory and write the SKILL.md**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .superpowers/skills/cyclone-tests
|
|
||||||
```
|
|
||||||
|
|
||||||
Frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
name: cyclone-tests
|
|
||||||
description: "Cyclone pytest + vitest fixture patterns, prodfiles layout, backend/tests/fixtures/ conventions, .test.tsx sibling rule. Use when: adding a backend pytest case, adding a frontend vitest test, or wiring in a real-EDI prodfiles sample."
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Body sections (each must be present):
|
|
||||||
- `## When to use` — 4 bullets: adding a backend test, adding a frontend test, dropping in a prodfiles fixture, debugging a flaky test.
|
|
||||||
- `## Conventions` — at minimum:
|
|
||||||
1. **Frontend sibling rule.** Every new file in `src/` that contains testable logic gets a `*.test.ts(x)` next to it. `useFoo.ts` → `useFoo.test.ts`.
|
|
||||||
2. **Backend test location.** Tests live under `backend/tests/test_*.py`. Integration tests (FastAPI) follow `test_api_*.py`; pure-unit tests (parsers, validators) follow `test_<module>_*.py`.
|
|
||||||
3. **Prodfiles drop-in.** Real EDI samples live under `docs/prodfiles/<source>/<edi>.txt`. To use one in a test: copy to `backend/tests/fixtures/<test-name>/<edi>.txt` and reference via the existing fixture helper (see `## Patterns`).
|
|
||||||
4. **Determinism.** Use `freezegun` for date-sensitive backend tests; use `vi.useFakeTimers()` for time-sensitive frontend tests. Never rely on `datetime.now()` directly.
|
|
||||||
5. **No network.** Tests must not hit the network. `httpx`/`requests` mocks live in `conftest.py` fixtures.
|
|
||||||
6. **pytest collection.** Run `cd backend && python -m pytest tests/<file>::<name> -v` for the fastest feedback loop. Full suite is the merge gate.
|
|
||||||
- `## Patterns` — three small copy-pasteable skeletons:
|
|
||||||
- One pytest using `tmp_path` + a prodfiles fixture reference.
|
|
||||||
- One vitest `*.test.ts` for a hook using `vi.useFakeTimers()`.
|
|
||||||
- One vitest `*.test.tsx` for a component using `@testing-library/react` + `happy-dom`.
|
|
||||||
- `## Anti-patterns` — at least:
|
|
||||||
- Don't put frontend tests in `src/__tests__/` (legacy location, no longer used).
|
|
||||||
- Don't reach into `docs/prodfiles/` directly from a test — copy to `backend/tests/fixtures/` so tests stay runnable when prodfiles is reorganized.
|
|
||||||
- Don't use `sleep()` for timing — use the deterministic timer tools listed in Conventions.
|
|
||||||
- `## Related skills` — references to all 7 other skills (every domain skill impacts tests) plus `superpowers:test-driven-development` upstream.
|
|
||||||
|
|
||||||
Keep under ~250 lines (this is the most cross-cutting skill, so it's the largest).
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -5 .superpowers/skills/cyclone-tests/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-tests/SKILL.md
|
|
||||||
grep -c "^## " .superpowers/skills/cyclone-tests/SKILL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: frontmatter present, 150–250 lines, 5 `## ` sections.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .superpowers/skills/cyclone-tests/SKILL.md
|
|
||||||
git commit -m "feat(sp-skill-catalog): add cyclone-tests skill (fixture patterns)"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 3: Phase 2, Skill 3 — `cyclone-edi`
|
|
||||||
|
|
||||||
**Goal:** Codify Cyclone's EDI parser/validator conventions across 837P/835/999/270/271/277CA/TA1. Highest-leverage skill — 30 parsers, scattered validators, fixture sprawl.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.superpowers/skills/cyclone-edi/SKILL.md`
|
|
||||||
- Create: `.superpowers/skills/cyclone-edi/references/parsers.md`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Enumerate the parsers and validator rules**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls backend/src/cyclone/parsers/
|
|
||||||
grep -rn "^def parse_" backend/src/cyclone/parsers/ | head -40
|
|
||||||
grep -n "^R[0-9]" backend/src/cyclone/validator*.py 2>/dev/null | head -20
|
|
||||||
ls backend/src/cyclone/ | grep -i valid
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: ~30 parser modules, each exporting `parse_*`, plus `validator.py` and possibly `validator_835.py`. Record the R-codes (R200, R210, etc.) found.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Sample one parser to cite in the skill**
|
|
||||||
|
|
||||||
Pick the smallest one (e.g. `parse_ta1.py` or `parse_999.py`) and read the first 30 lines. The skill's `## Patterns` section will reference its structure.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Create directories and write the SKILL.md + reference**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .superpowers/skills/cyclone-edi/references
|
|
||||||
```
|
|
||||||
|
|
||||||
Frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
name: cyclone-edi
|
|
||||||
description: "Cyclone EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1). Use when: adding or changing a parser, adding a validator rule (R200/R210/NPI Luhn/EIN/CAS), or mapping a new CAS adjustment reason code."
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Body sections:
|
|
||||||
- `## When to use` — 4 bullets: adding a parser, adding a validator rule, wiring a new CAS code, debugging a parse failure on a prodfiles sample.
|
|
||||||
- `## Conventions` — at minimum:
|
|
||||||
1. **Parser signature.** Every parser module exports `parse(text: str) -> <TypedResult>` where `<TypedResult>` is a Pydantic model from `backend/src/cyclone/parsers/<edi>_models.py` (or co-located).
|
|
||||||
2. **Segment walk.** Parsers consume `backend/src/cyclone/segments.py` helpers (`Segment`, `Loop`, `next_segment`) — do not parse raw text inline.
|
|
||||||
3. **Validator rules.** Numbered (R200, R210, …) live in `backend/src/cyclone/validator.py` (or `validator_835.py` for 835-specific). Each rule is a function `_rule_R<n>_<name>` raising `ValidationError` with the R-code.
|
|
||||||
4. **NPI / EIN / CAS.** Identity-format rules (NPI Luhn, EIN `XX-XXXXXXX`, CAS adjustment reason codes) live in dedicated modules (`npi.py`, etc.). Don't duplicate the format logic.
|
|
||||||
5. **Prodfiles reuse.** When adding a parser for a new transaction type, ship at least one prodfiles fixture in `backend/tests/fixtures/<edi>/<sample>.txt`.
|
|
||||||
- `## Patterns` — three skeletons:
|
|
||||||
- A minimal `parse_<edi>.py` (using `segments.py`, exporting `parse(text)`).
|
|
||||||
- A validator rule `_rule_R<n>_<name>(ctx) -> None`.
|
|
||||||
- A test that uses a prodfiles fixture (`backend/tests/fixtures/<edi>/<sample>.txt`) and asserts `parse(text).foo == expected`.
|
|
||||||
- `## Anti-patterns` — at least:
|
|
||||||
- Don't re-parse raw X12 strings inside validators — always parse first, validate the typed result.
|
|
||||||
- Don't bake payer-specific logic into the generic parser — payer variations live in `backend/src/cyclone/payers.py` or `<payer>_config.py`.
|
|
||||||
- Don't add a validator rule without an R-code — the R-code is how the UI surfaces the error.
|
|
||||||
- `## Related skills` — references to `cyclone-store` (writes the parsed result), `cyclone-api-router` (exposes it), `cyclone-tests` (fixture drop-in), `cyclone-cli` (the `cyc parse <file>` smoke command).
|
|
||||||
|
|
||||||
Then write `references/parsers.md`: a flat table mapping each parser module to its transaction type, signature, primary fixtures, and any payer-specific variants. Keep under 80 lines.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -5 .superpowers/skills/cyclone-edi/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-edi/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-edi/references/parsers.md
|
|
||||||
grep -c "^## " .superpowers/skills/cyclone-edi/SKILL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: frontmatter present, SKILL.md ≤ 200 lines, references/parsers.md ≤ 80 lines, exactly 5 `## ` sections in SKILL.md.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .superpowers/skills/cyclone-edi/
|
|
||||||
git commit -m "feat(sp-skill-catalog): add cyclone-edi skill (parser/validator conventions)"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 4: Phase 2, Skill 4 — `cyclone-tail`
|
|
||||||
|
|
||||||
**Goal:** Codify the live-tail wire format and the `useTailStream` + `useMergedTail` + per-resource hook triplet. Concrete drift risk — wire format is in the README but consumed across 3 page-hook pairs.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.superpowers/skills/cyclone-tail/SKILL.md`
|
|
||||||
- Create: `.superpowers/skills/cyclone-tail/references/wire-format.md`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Survey the live-tail surface**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep -rn "/stream" backend/src/cyclone/api_routers/ src/hooks/ 2>/dev/null | head -20
|
|
||||||
ls src/hooks/use*Stream* src/hooks/use*MergedTail*
|
|
||||||
ls src/hooks/use*Tail* 2>/dev/null
|
|
||||||
grep -rn "snapshot_end\|x-ndjson" backend/src/cyclone/ src/ 2>/dev/null | head -10
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: 3 `/api/<resource>/stream` endpoints, one `useTailStream.ts`, one `useMergedTail.ts`, multiple `use<X>Stream` consumers.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Read the wire-format spec in the README**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep -n -A 30 "Wire format" README.md | head -50
|
|
||||||
```
|
|
||||||
|
|
||||||
This is the canonical definition; the skill will reference it. Record the line range for the `references/wire-format.md` cross-reference.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Create directories and write the SKILL.md + reference**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .superpowers/skills/cyclone-tail/references
|
|
||||||
```
|
|
||||||
|
|
||||||
Frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
name: cyclone-tail
|
|
||||||
description: "Cyclone live-tail streaming wire format and the useTailStream / useMergedTail hook triplet. Use when: adding a new streaming list page, changing the wire format, debugging stalled/reconnecting state, or modifying the StatusPill behavior."
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Body sections:
|
|
||||||
- `## When to use` — 4 bullets: adding a new streaming page, changing the wire format, debugging stalled/reconnecting, tuning heartbeat/stall timing.
|
|
||||||
- `## Conventions` — at minimum:
|
|
||||||
1. **Wire format.** Newline-delimited JSON. Line shapes: `{"type":"item","data":…}`, `{"type":"snapshot_end","data":{"count":N}}`, `{"type":"heartbeat","data":{"ts":…}}`, `{"type":"item_dropped","data":…}`, `{"type":"error","data":…}`. See `references/wire-format.md`.
|
|
||||||
2. **Hook triplet.** Streaming pages consume three hooks: `useTailStream(url)` (raw stream state), `useMergedTail(items, events)` (combine initial fetch with live events), and a per-resource hook (`useClaims`, `useRemittances`, etc.) that wraps both.
|
|
||||||
3. **Stall threshold.** 30s of total silence (heartbeat included) flips the connection to `stalled` and surfaces a `↻ Reconnect` button. Don't change this without updating the README's "Status pill" table.
|
|
||||||
4. **Snapshot first.** Every stream must emit the current snapshot before any live `item` events. The `snapshot_end` line is the marker the UI uses to flip `StatusPill` from `connecting` to `live`.
|
|
||||||
5. **Content-Type.** `application/x-ndjson`. Never `application/json` for a stream endpoint.
|
|
||||||
- `## Patterns` — three skeletons:
|
|
||||||
- A `useFoo.ts` page-hook skeleton (TanStack Query initial fetch + tail subscription).
|
|
||||||
- A new backend `/api/foo/stream` endpoint signature.
|
|
||||||
- A `StatusPill` consumer wiring.
|
|
||||||
- `## Anti-patterns` — at least:
|
|
||||||
- Don't hand-roll `fetch` + `ReadableStream` parsing in a page — go through `useTailStream` so reconnect/stall handling is consistent.
|
|
||||||
- Don't change the wire format on one endpoint without updating the other two — they share the parser in `src/lib/tail-stream.ts`.
|
|
||||||
- Don't emit `item` events before `snapshot_end` — the UI will duplicate rows.
|
|
||||||
- `## Related skills` — references to `cyclone-frontend-page` (page-level wiring), `cyclone-api-router` (the `/api/<resource>/stream` endpoint), `cyclone-store` (the event contract that drives the stream), `cyclone-tests` (`.test.ts` siblings on every hook in this skill).
|
|
||||||
|
|
||||||
Then write `references/wire-format.md`: copy the relevant README section (with attribution) and add a per-line field reference table (which fields are required, which are optional, parser tolerance). Keep under 80 lines.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -5 .superpowers/skills/cyclone-tail/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-tail/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-tail/references/wire-format.md
|
|
||||||
grep -c "^## " .superpowers/skills/cyclone-tail/SKILL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: frontmatter present, SKILL.md ≤ 200 lines, wire-format.md ≤ 80 lines, 5 `## ` sections.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .superpowers/skills/cyclone-tail/
|
|
||||||
git commit -m "feat(sp-skill-catalog): add cyclone-tail skill (live-tail wire format)"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 5: Phase 3, Skill 5 — `cyclone-store`
|
|
||||||
|
|
||||||
**Goal:** Codify the store write-paths, the pubsub event contract, and the SP21 store-split boundary map. Aligns with the SP21 store split currently in plan stage — the skill codifies the new boundaries as they land.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.superpowers/skills/cyclone-store/SKILL.md`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Read SP21 plan to learn the new module boundaries**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cat docs/superpowers/plans/2026-06-21-cyclone-store-split.md | head -80
|
|
||||||
```
|
|
||||||
|
|
||||||
Record the module list (`batches`, `inbox`, `acks`, etc.) for use in the skill.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Survey the current pubsub surface**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep -rn "_written\|_recorded" backend/src/cyclone/pubsub.py | head -20
|
|
||||||
grep -rn "publish\|subscribe" backend/src/cyclone/store.py | head -10
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: at least 3 event types — `claim_written`, `remittance_written`, `activity_recorded`. Record exact names.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Create the directory and write the SKILL.md**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .superpowers/skills/cyclone-store
|
|
||||||
```
|
|
||||||
|
|
||||||
Frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
name: cyclone-store
|
|
||||||
description: "Cyclone store write-paths, the pubsub event contract (claim_written / remittance_written / activity_recorded), and the SP21 store-split boundary map. Use when: touching store.py, adding a new entity, wiring a new write event, or splitting a store module."
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Body sections:
|
|
||||||
- `## When to use` — 4 bullets: adding a new entity, wiring a new write event, debugging a write-path issue, splitting a store module.
|
|
||||||
- `## Conventions` — at minimum:
|
|
||||||
1. **All writes go through `CycloneStore`.** Route handlers and parsers must not write directly to the ORM session.
|
|
||||||
2. **Every write publishes an event.** The event name matches the entity: `claim_written`, `remittance_written`, `activity_recorded`. New entities get `<entity>_written` (or `_recorded` for non-canonical rows).
|
|
||||||
3. **Snapshot shape.** Each entity has a Pydantic serializer in `backend/src/cyclone/store/ui.py` (or its post-SP21 equivalent). The serializer is the single source of truth for what the frontend sees.
|
|
||||||
4. **SP21 boundaries.** Post-split, each domain lives in its own module (`batches`, `inbox`, `acks`, …). Cross-module writes go through `CycloneStore` facade methods, not direct module access.
|
|
||||||
5. **No business logic in route handlers.** A route handler validates input, calls `store.<method>(...)`, publishes the event, returns the serialized result. Anything more belongs in the store or a parser.
|
|
||||||
- `## Patterns` — three skeletons:
|
|
||||||
- A `CycloneStore` write method (`def add_foo(self, foo: Foo) -> FooRecord`).
|
|
||||||
- An event publication in `pubsub.py`.
|
|
||||||
- A new `<entity>_written` handler test that asserts both the row and the event.
|
|
||||||
- `## Anti-patterns` — at least:
|
|
||||||
- Don't read directly from the ORM in a route handler — go through the snapshot serializer.
|
|
||||||
- Don't introduce a new event name without updating the subscriber list (currently `api.py` and `api_routers/`).
|
|
||||||
- Don't merge a write method with its event publication into separate places — they live next to each other so reviewers see both.
|
|
||||||
- `## Related skills` — references to `cyclone-edi` (the parsed result lands here), `cyclone-api-router` (the route that calls the store), `cyclone-tail` (the event drives the stream), `cyclone-tests` (write-path tests).
|
|
||||||
|
|
||||||
Keep under ~200 lines. No `references/` needed unless the boundary map grows.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -5 .superpowers/skills/cyclone-store/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-store/SKILL.md
|
|
||||||
grep -c "^## " .superpowers/skills/cyclone-store/SKILL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: frontmatter present, ≤ 200 lines, 5 `## ` sections.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .superpowers/skills/cyclone-store/SKILL.md
|
|
||||||
git commit -m "feat(sp-skill-catalog): add cyclone-store skill (write paths + event contract)"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 6: Phase 3, Skill 6 — `cyclone-api-router`
|
|
||||||
|
|
||||||
**Goal:** Codify FastAPI router conventions — `api_routers/`, `api_helpers.py` reuse, response shapes, error envelopes. Pairs with `cyclone-store`.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.superpowers/skills/cyclone-api-router/SKILL.md`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Survey the existing router layout**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls backend/src/cyclone/api_routers/
|
|
||||||
grep -l "APIRouter" backend/src/cyclone/api_routers/*.py
|
|
||||||
head -30 backend/src/cyclone/api_helpers.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Record the router filenames and which helpers exist in `api_helpers.py`.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Sample one router**
|
|
||||||
|
|
||||||
Pick the smallest existing router (likely `acks.py` or `ta1_acks.py`) and read the first 50 lines. The skill's `## Patterns` section will reference its structure.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Create the directory and write the SKILL.md**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .superpowers/skills/cyclone-api-router
|
|
||||||
```
|
|
||||||
|
|
||||||
Frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
name: cyclone-api-router
|
|
||||||
description: "Cyclone FastAPI router conventions (api_routers/, api_helpers.py, response shapes, error envelopes). Use when: adding or changing an HTTP endpoint, splitting a route out of api.py, or wiring a new helper into api_helpers.py."
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Body sections:
|
|
||||||
- `## When to use` — 4 bullets: adding an endpoint, splitting a route, adding a helper, defining an error response.
|
|
||||||
- `## Conventions` — at minimum:
|
|
||||||
1. **No new top-level routes in `api.py`.** Every endpoint lives in `backend/src/cyclone/api_routers/<topic>.py` as an `APIRouter`.
|
|
||||||
2. **Reuse `api_helpers.py`.** Common response shapes, error envelopes, and parsing helpers live there. Don't duplicate them in a router.
|
|
||||||
3. **Response shape.** Every successful response is a Pydantic model. Errors use the shared `ErrorEnvelope` (from `api_helpers.py`) with `code` and `message`.
|
|
||||||
4. **Mounting.** Routers are mounted in `api.py` with their prefix; keep prefix naming consistent (`/api/<resource>`).
|
|
||||||
5. **Streaming endpoints.** Use `StreamingResponse(media_type="application/x-ndjson")` — see `cyclone-tail` for the wire format.
|
|
||||||
6. **Tests.** Every new endpoint gets a `test_api_<topic>_<verb>.py` test under `backend/tests/`.
|
|
||||||
- `## Patterns` — three skeletons:
|
|
||||||
- A new `APIRouter` skeleton (`router = APIRouter(prefix="/api/foo", tags=["foo"])`).
|
|
||||||
- A `get_one` / `get_list` / `post_one` trio with Pydantic response models.
|
|
||||||
- An error envelope usage example.
|
|
||||||
- `## Anti-patterns` — at least:
|
|
||||||
- Don't import from `cyclone.api` into a router — the dependency runs the other way.
|
|
||||||
- Don't return raw dicts — always Pydantic.
|
|
||||||
- Don't bypass `CycloneStore` to query the ORM directly — see `cyclone-store`.
|
|
||||||
- `## Related skills` — references to `cyclone-store` (most routes call store methods), `cyclone-tail` (streaming endpoints), `cyclone-edi` (parse endpoints), `cyclone-tests` (endpoint tests).
|
|
||||||
|
|
||||||
Keep under ~200 lines.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -5 .superpowers/skills/cyclone-api-router/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-api-router/SKILL.md
|
|
||||||
grep -c "^## " .superpowers/skills/cyclone-api-router/SKILL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: frontmatter present, ≤ 200 lines, 5 `## ` sections.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .superpowers/skills/cyclone-api-router/SKILL.md
|
|
||||||
git commit -m "feat(sp-skill-catalog): add cyclone-api-router skill (FastAPI conventions)"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 7: Phase 4, Skill 7 — `cyclone-frontend-page`
|
|
||||||
|
|
||||||
**Goal:** Codify the React page-component pattern — TanStack Query, `use<X>` hook, drawer, URL state, `.test.tsx` sibling. Generalizes what `cyclone-tail` already covers.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.superpowers/skills/cyclone-frontend-page/SKILL.md`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Survey the page layout**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls src/pages/
|
|
||||||
echo "---"
|
|
||||||
ls src/components/ui/ | head -20
|
|
||||||
```
|
|
||||||
|
|
||||||
Record the page list and which shadcn-style UI components are in `src/components/ui/`.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Sample one page to cite**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -50 src/pages/Claims.tsx
|
|
||||||
```
|
|
||||||
|
|
||||||
Use this in the skill's `## Patterns` to show the canonical page structure.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Create the directory and write the SKILL.md**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .superpowers/skills/cyclone-frontend-page
|
|
||||||
```
|
|
||||||
|
|
||||||
Frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
name: cyclone-frontend-page
|
|
||||||
description: "Cyclone React page conventions (TanStack Query, use<X> hook, drawer, URL state, .test.tsx sibling, Layout / PageHeader / Sidebar). Use when: adding a new page, refactoring an existing one, or wiring a drawer into a page."
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Body sections:
|
|
||||||
- `## When to use` — 4 bullets: adding a new page, refactoring an existing page, wiring a drawer, sharing state via URL.
|
|
||||||
- `## Conventions` — at minimum:
|
|
||||||
1. **Page shape.** Every page in `src/pages/<Name>.tsx` exports a default `function <Name>()`. It uses `<Layout>`, sets a `<PageHeader>`, and renders a table or list.
|
|
||||||
2. **Data hook.** Each page pairs with a `use<X>` hook in `src/hooks/use<X>.ts` that returns `{ data, isLoading, error, ...tail }`. Pages do not call `fetch` directly.
|
|
||||||
3. **Tail.** If the page shows live data, it consumes the tail-stream pattern via `cyclone-tail`. The hook, not the page, owns the streaming subscription.
|
|
||||||
4. **Drawer.** Drawers (e.g. `ClaimDrawer`, `RemitDrawer`) live in `src/components/<DrawerName>/` and are wired via `useDrawerUrlState` so the URL reflects open state.
|
|
||||||
5. **Tests.** Every page gets a `src/pages/<Name>.test.tsx` sibling. Every hook gets a `src/hooks/use<X>.test.ts` sibling.
|
|
||||||
6. **UI primitives.** Use Radix-backed components from `src/components/ui/`. Don't pull in new UI libraries without discussion.
|
|
||||||
7. **Routing.** Pages register their route in `src/App.tsx`. Lazy-load if the page is heavy.
|
|
||||||
- `## Patterns` — three skeletons:
|
|
||||||
- A minimal `Claims.tsx`-style page (Layout + PageHeader + table + drawer).
|
|
||||||
- A `use<X>` hook skeleton (TanStack Query + tail).
|
|
||||||
- A drawer component skeleton with `useDrawerUrlState`.
|
|
||||||
- `## Anti-patterns` — at least:
|
|
||||||
- Don't `fetch` from inside a page component — go through the hook.
|
|
||||||
- Don't open a drawer via local component state — use `useDrawerUrlState` so deep-links work.
|
|
||||||
- Don't put domain logic in JSX — extract to the hook or a pure helper in `src/lib/`.
|
|
||||||
- `## Related skills` — references to `cyclone-tail` (live-data pages), `cyclone-api-router` (the endpoints pages call), `cyclone-tests` (page + hook tests), `cyclone-edi` (pages that show parsed EDI content).
|
|
||||||
|
|
||||||
Keep under ~200 lines.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -5 .superpowers/skills/cyclone-frontend-page/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-frontend-page/SKILL.md
|
|
||||||
grep -c "^## " .superpowers/skills/cyclone-frontend-page/SKILL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: frontmatter present, ≤ 200 lines, 5 `## ` sections.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .superpowers/skills/cyclone-frontend-page/SKILL.md
|
|
||||||
git commit -m "feat(sp-skill-catalog): add cyclone-frontend-page skill (React page conventions)"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 8: Phase 4, Skill 8 — `cyclone-cli`
|
|
||||||
|
|
||||||
**Goal:** Codify the CLI subcommand conventions in `cli.py` — argparse style, exit codes, smoke-test patterns. Last because nothing else depends on it.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.superpowers/skills/cyclone-cli/SKILL.md`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Survey the existing CLI**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep -n "add_parser\|sub_parsers\|set_defaults" backend/src/cyclone/cli.py | head -20
|
|
||||||
grep -n "sys.exit\|return [0-9]\|return 1\|return 2" backend/src/cyclone/cli.py | head -10
|
|
||||||
```
|
|
||||||
|
|
||||||
Record the subcommand list and exit-code conventions.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Sample one subcommand**
|
|
||||||
|
|
||||||
Pick the smallest (likely `validate`) and read its 20-30 lines. The skill's `## Patterns` will reference its structure.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Create the directory and write the SKILL.md**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .superpowers/skills/cyclone-cli
|
|
||||||
```
|
|
||||||
|
|
||||||
Frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
name: cyclone-cli
|
|
||||||
description: "Cyclone CLI subcommand conventions (cli.py, serve/parse/backup/rotate-key/validate, argparse style, exit codes, smoke tests). Use when: adding a CLI subcommand, changing exit codes, or working on operator-facing commands."
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Body sections:
|
|
||||||
- `## When to use` — 4 bullets: adding a subcommand, changing exit codes, adding a smoke test, wiring a security-sensitive command (key rotation, backup).
|
|
||||||
- `## Conventions` — at minimum:
|
|
||||||
1. **Subcommand shape.** Every subcommand is a function `cmd_<name>(args) -> int` in `cli.py`. The `main()` parser dispatches via `set_defaults(func=...)`.
|
|
||||||
2. **Argparse style.** Use `argparse.ArgumentParser` sub-parsers. Long-form flags (`--rotate-key`) preferred over positional for safety.
|
|
||||||
3. **Exit codes.** `0` = success, `1` = user error (bad input, missing file), `2` = operator error (DB locked, key missing), `3` = security error (auth fail, key mismatch). Document in `cmd_<name>` docstring.
|
|
||||||
4. **Smoke test.** Every new subcommand gets a `backend/tests/test_cli_<name>.py` test using `subprocess.run` against the installed `cyc` entrypoint.
|
|
||||||
5. **Security-sensitive commands.** Key rotation (`cyc rotate-key`), backup (`cyc backup`), and any command touching `secrets.py` requires a `--confirm` flag and a dry-run path.
|
|
||||||
- `## Patterns` — three skeletons:
|
|
||||||
- A `cmd_<name>(args)` function with `argparse` setup.
|
|
||||||
- A `subprocess.run` smoke test.
|
|
||||||
- A `--confirm` / `--dry-run` pattern for security-sensitive commands.
|
|
||||||
- `## Anti-patterns` — at least:
|
|
||||||
- Don't `sys.exit()` from inside a subcommand — return the int and let `main()` exit.
|
|
||||||
- Don't print to stdout for errors — use `logging` (or `print(..., file=sys.stderr)` for top-level fatal errors).
|
|
||||||
- Don't add a subcommand without a smoke test in `backend/tests/test_cli_<name>.py`.
|
|
||||||
- `## Related skills` — references to `cyclone-store` (most subcommands touch the store), `cyclone-api-router` (`cyc serve` runs the FastAPI app), `cyclone-edi` (`cyc parse <file>`), `cyclone-tests` (CLI smoke tests).
|
|
||||||
|
|
||||||
Keep under ~200 lines.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
head -5 .superpowers/skills/cyclone-cli/SKILL.md
|
|
||||||
wc -l .superpowers/skills/cyclone-cli/SKILL.md
|
|
||||||
grep -c "^## " .superpowers/skills/cyclone-cli/SKILL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: frontmatter present, ≤ 200 lines, 5 `## ` sections.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .superpowers/skills/cyclone-cli/SKILL.md
|
|
||||||
git commit -m "feat(sp-skill-catalog): add cyclone-cli skill (CLI conventions)"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 9: Final — README catalog section
|
|
||||||
|
|
||||||
**Goal:** Advertise the catalog to human readers and to AI agents reading the README on first contact.
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `README.md` (add a "Skills" section near the top)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Find a good insertion point**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep -n "^## " README.md | head -20
|
|
||||||
```
|
|
||||||
|
|
||||||
Pick a sensible location — typically after the existing "Install" or "Dev" section.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Add the catalog section**
|
|
||||||
|
|
||||||
Add a new `## Skills` section listing all 8 skills with a one-line description each. Example:
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
## Skills
|
|
||||||
|
|
||||||
Cyclone ships 8 project-scoped AI-assistant skills under
|
|
||||||
[`.superpowers/skills/`](.superpowers/skills/). Each one codifies the
|
|
||||||
conventions for a major subsystem so the next contributor (human or
|
|
||||||
AI) gets the lay of the land automatically.
|
|
||||||
|
|
||||||
| Skill | Owns |
|
|
||||||
|-------|------|
|
|
||||||
| [`cyclone-spec`](.superpowers/skills/cyclone-spec/SKILL.md) | The SP-N spec → plan → implement → merge flow. |
|
|
||||||
| [`cyclone-tests`](.superpowers/skills/cyclone-tests/SKILL.md) | pytest + vitest fixture patterns, prodfiles drop-in. |
|
|
||||||
| [`cyclone-edi`](.superpowers/skills/cyclone-edi/SKILL.md) | EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1). |
|
|
||||||
| [`cyclone-tail`](.superpowers/skills/cyclone-tail/SKILL.md) | Live-tail streaming wire format and the hook triplet. |
|
|
||||||
| [`cyclone-store`](.superpowers/skills/cyclone-store/SKILL.md) | Store write-paths, pubsub event contract, SP21 split map. |
|
|
||||||
| [`cyclone-api-router`](.superpowers/skills/cyclone-api-router/SKILL.md) | FastAPI router conventions (`api_routers/`, `api_helpers.py`). |
|
|
||||||
| [`cyclone-frontend-page`](.superpowers/skills/cyclone-frontend-page/SKILL.md) | React page conventions (TanStack Query, drawer, URL state). |
|
|
||||||
| [`cyclone-cli`](.superpowers/skills/cyclone-cli/SKILL.md) | CLI subcommand conventions (`cli.py`, exit codes, smoke tests). |
|
|
||||||
|
|
||||||
Skills auto-load by description match — no slash command needed.
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Verify the section reads cleanly**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep -n -A 16 "^## Skills" README.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: the section renders with all 8 rows and the trailing usage note.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add README.md
|
|
||||||
git commit -m "docs(readme): add Skills section linking the 8-skill catalog"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Self-review (run after writing the plan, before executing)
|
|
||||||
|
|
||||||
- [ ] **Spec coverage.** Skim each section of `docs/superpowers/specs/2026-06-21-cyclone-skill-catalog-design.md`. Confirm:
|
|
||||||
- Spec §"The catalog" → Tasks 1-8 each create the matching skill. ✅
|
|
||||||
- Spec §"Per-skill structure" → enforced by the shared template locked in Task 0. ✅
|
|
||||||
- Spec §"Loading & cross-references" → every Task has a `## Related skills` section. ✅
|
|
||||||
- Spec §"Build order / phasing" → Phase order = Task order (1-2 foundations, 3-4 domain, 5-6 backend, 7-8 frontend+CLI). ✅
|
|
||||||
- Spec §"Verification" → covered by verification steps in each task + Task 9 README addition. ✅
|
|
||||||
- [ ] **Placeholder scan.** No "TBD" / "TODO" / "implement later" in the plan. Each task's body content is specified (frontmatter exact, sections listed, key bullets enumerated).
|
|
||||||
- [ ] **Type / name consistency.**
|
|
||||||
- Skill names: `cyclone-spec`, `cyclone-tests`, `cyclone-edi`, `cyclone-tail`, `cyclone-store`, `cyclone-api-router`, `cyclone-frontend-page`, `cyclone-cli`. Same in every task. ✅
|
|
||||||
- Commit prefix: `feat(sp-skill-catalog): …` everywhere. ✅
|
|
||||||
- Section names: `## When to use`, `## Conventions`, `## Patterns`, `## Anti-patterns`, `## Related skills`. Same in every task. ✅
|
|
||||||
|
|
||||||
Plan passes self-review.
|
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
-386
@@ -1,386 +0,0 @@
|
|||||||
# Drop `UNIQUE(batch_id, patient_control_number)` on Claims + Robust 409 UX
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Branch:** `main`
|
|
||||||
**Status:** Draft (brainstorming approved, awaiting writing-plans)
|
|
||||||
**Scope:** Backend schema + minimal frontend error handling for collision 409.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Why this exists
|
|
||||||
|
|
||||||
Today, every multi-claim 837P file in `docs/prodfiles/837p-from-axiscare/` and
|
|
||||||
`docs/prodfiles/FromHPE/` returns **HTTP 409** on upload. The cause:
|
|
||||||
|
|
||||||
* The `claims` table has `UNIQUE(batch_id, patient_control_number)` declared
|
|
||||||
inline in `CREATE TABLE` (auto-index `sqlite_autoindex_claims_2`).
|
|
||||||
* `store._claim_837_row` populates `patient_control_number` from
|
|
||||||
`claim.subscriber.member_id` (the subscriber's insurance member id),
|
|
||||||
not from the CLM01 segment value.
|
|
||||||
* A real 837P submission routinely contains many `CLM*` segments per
|
|
||||||
subscriber (a member seeing multiple providers on the same day). All
|
|
||||||
those rows share the same `member_id` → same `patient_control_number`.
|
|
||||||
Within one batch, the constraint fires on claim #2.
|
|
||||||
|
|
||||||
Migration `0003_drop_claims_remits_unique_constraints.sql` already exists
|
|
||||||
with the right intent but is **buggy**: it does
|
|
||||||
`DROP INDEX IF EXISTS uq_claims_batch_pcn`, but the constraint is inline,
|
|
||||||
not a named index — so the `DROP INDEX` is a no-op and the constraint
|
|
||||||
survives.
|
|
||||||
|
|
||||||
The 409 error message ("duplicate CLM01 control numbers") is also
|
|
||||||
misleading because the column stores `member_id`, not CLM01.
|
|
||||||
|
|
||||||
This SP fixes both: drops the constraint properly via SQLite table
|
|
||||||
recreation, and gives the upload page a structured error panel for the
|
|
||||||
collision case so the operator can find the existing batch in one click.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Operator surface
|
|
||||||
|
|
||||||
| Surface | Change |
|
|
||||||
|---|---|
|
|
||||||
| DB | New migration `0013_drop_claims_unique_constraint.sql`; idempotent. |
|
|
||||||
| Python | `store.find_existing_batch_for_claim(claim_id)` new helper. |
|
|
||||||
| API | `POST /api/parse-837` and `/api/parse-835` 409 responses gain `existing_batch_id`. |
|
|
||||||
| UI | `/upload` page renders an inline error panel for 409 with a link to the existing batch. |
|
|
||||||
| Tests | New migration test, store helper test, API test, component test. |
|
|
||||||
|
|
||||||
No CLI / settings changes.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Schema migration
|
|
||||||
|
|
||||||
`backend/src/cyclone/migrations/0013_drop_claims_unique_constraint.sql`:
|
|
||||||
|
|
||||||
```sql
|
|
||||||
-- version: 13
|
|
||||||
-- Drop the inline UNIQUE(batch_id, patient_control_number) on claims.
|
|
||||||
-- Migration 0003 attempted DROP INDEX IF EXISTS uq_claims_batch_pcn but
|
|
||||||
-- the constraint is inline in CREATE TABLE, so the drop was a no-op.
|
|
||||||
-- The only way to remove an inline UNIQUE in SQLite is table recreation.
|
|
||||||
--
|
|
||||||
-- X12 837P allows any number of CLM segments per 2000B subscriber loop;
|
|
||||||
-- claim identity is provided by the primary key (claims.id = CLM01).
|
|
||||||
-- The remittances table had a parallel constraint already removed in 0003
|
|
||||||
-- (because that one WAS a named index), so this migration only touches
|
|
||||||
-- claims.
|
|
||||||
--
|
|
||||||
-- The migration runner (db_migrate.py) wraps each .sql file in an
|
|
||||||
-- implicit transaction via engine.begin(), so we MUST NOT use BEGIN/COMMIT
|
|
||||||
-- inside the file (nested transactions fail in SQLite). We defer FK
|
|
||||||
-- enforcement with PRAGMA defer_foreign_keys instead of turning FKs off
|
|
||||||
-- (which is a no-op inside a transaction in SQLite). The deferred
|
|
||||||
-- checks fire at commit and validate against the renamed claims table.
|
|
||||||
|
|
||||||
PRAGMA defer_foreign_keys = ON;
|
|
||||||
|
|
||||||
CREATE TABLE claims_new (
|
|
||||||
id TEXT PRIMARY KEY,
|
|
||||||
batch_id TEXT NOT NULL REFERENCES batches(id) ON DELETE CASCADE,
|
|
||||||
patient_control_number TEXT NOT NULL,
|
|
||||||
service_date_from DATE,
|
|
||||||
service_date_to DATE,
|
|
||||||
charge_amount NUMERIC(12, 2) NOT NULL DEFAULT 0,
|
|
||||||
provider_npi TEXT,
|
|
||||||
payer_id TEXT,
|
|
||||||
state TEXT NOT NULL DEFAULT 'submitted',
|
|
||||||
state_before_reversal TEXT,
|
|
||||||
matched_remittance_id TEXT REFERENCES remittances(id),
|
|
||||||
raw_json TEXT,
|
|
||||||
rejection_reason TEXT,
|
|
||||||
rejected_at TIMESTAMP,
|
|
||||||
resubmit_count INTEGER NOT NULL DEFAULT 0,
|
|
||||||
state_changed_at TIMESTAMP,
|
|
||||||
payer_rejected_at TEXT,
|
|
||||||
payer_rejected_reason TEXT,
|
|
||||||
payer_rejected_status_code TEXT,
|
|
||||||
payer_rejected_by_277ca_id TEXT,
|
|
||||||
payer_rejected_acknowledged_at TEXT,
|
|
||||||
payer_rejected_acknowledged_actor TEXT
|
|
||||||
-- NO UNIQUE (batch_id, patient_control_number) — removed.
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO claims_new SELECT * FROM claims;
|
|
||||||
DROP TABLE claims;
|
|
||||||
ALTER TABLE claims_new RENAME TO claims;
|
|
||||||
|
|
||||||
-- Recreate secondary indexes (same names, same columns as initial schema
|
|
||||||
-- plus later migrations).
|
|
||||||
CREATE INDEX ix_claims_state ON claims(state);
|
|
||||||
CREATE INDEX ix_claims_patient_control_number ON claims(patient_control_number);
|
|
||||||
CREATE INDEX ix_claims_service_date_from ON claims(service_date_from);
|
|
||||||
CREATE INDEX ix_claims_state_changed_at ON claims(state, state_changed_at);
|
|
||||||
CREATE INDEX idx_claims_payer_rejected_at ON claims(payer_rejected_at);
|
|
||||||
CREATE INDEX idx_claims_payer_rejected_unack
|
|
||||||
ON claims(payer_rejected_at)
|
|
||||||
WHERE payer_rejected_acknowledged_at IS NULL;
|
|
||||||
```
|
|
||||||
|
|
||||||
**Note on the `db.py` ORM model:** `backend/src/cyclone/db.py` declares
|
|
||||||
the `Claim` ORM model with `UNIQUE(batch_id, patient_control_number)` in
|
|
||||||
`__table_args__`. After this migration the DB no longer enforces that
|
|
||||||
constraint, so the ORM declaration becomes aspirational. We leave it in
|
|
||||||
place as documentation (and as a guard if the DB ever gets rebuilt from
|
|
||||||
ORM on a fresh schema). No code reads through it; the dedup happens via
|
|
||||||
PK on `claims.id` in `store.add`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Store helper
|
|
||||||
|
|
||||||
New function in `backend/src/cyclone/store.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def find_existing_batch_for_claim(claim_id: str) -> str | None:
|
|
||||||
"""Return the batch_id of the first batch containing this CLM01, or None."""
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.db import Claim
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.execute(
|
|
||||||
select(Claim.batch_id).where(Claim.id == claim_id).limit(1)
|
|
||||||
).first()
|
|
||||||
return row[0] if row else None
|
|
||||||
|
|
||||||
|
|
||||||
def find_existing_batch_for_remit(payer_claim_control_number: str) -> str | None:
|
|
||||||
"""Return the batch_id of the first batch containing this CLP01, or None."""
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.db import Remittance
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.execute(
|
|
||||||
select(Remittance.batch_id)
|
|
||||||
.where(Remittance.id == payer_claim_control_number)
|
|
||||||
.limit(1)
|
|
||||||
).first()
|
|
||||||
return row[0] if row else None
|
|
||||||
```
|
|
||||||
|
|
||||||
Both pure reads; no transaction management needed.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. API change
|
|
||||||
|
|
||||||
`backend/src/cyclone/api.py`:
|
|
||||||
|
|
||||||
### 837 path (line 394-415)
|
|
||||||
|
|
||||||
After the UNIQUE constraint is dropped in migration 0013, an `IntegrityError`
|
|
||||||
in `store.add` can only originate from the PK on `claims.id` (CLM01) —
|
|
||||||
either two claims in the same file share CLM01 (rare) or the same CLM01
|
|
||||||
exists in a prior batch. The handler picks the first claim from
|
|
||||||
`result.claims` and asks the helper whether a prior batch already holds
|
|
||||||
that CLM01:
|
|
||||||
|
|
||||||
```python
|
|
||||||
except IntegrityError as exc:
|
|
||||||
first_claim_id = result.claims[0].claim_id if result.claims else None
|
|
||||||
existing_batch_id = (
|
|
||||||
store.find_existing_batch_for_claim(first_claim_id)
|
|
||||||
if first_claim_id else None
|
|
||||||
)
|
|
||||||
body = {
|
|
||||||
"error": "Duplicate claim",
|
|
||||||
"detail": (
|
|
||||||
"This file (or one previously ingested with the same "
|
|
||||||
"claim control number) collides with an existing record. "
|
|
||||||
"Inspect the file for duplicate CLM01 control numbers, or "
|
|
||||||
"remove the existing batch before retrying."
|
|
||||||
),
|
|
||||||
"batch_id": rec.id,
|
|
||||||
}
|
|
||||||
if existing_batch_id and existing_batch_id != rec.id:
|
|
||||||
body["existing_batch_id"] = existing_batch_id
|
|
||||||
log.warning("Duplicate claim while persisting batch %s: %s", rec.id, exc)
|
|
||||||
return JSONResponse(status_code=409, content=body)
|
|
||||||
```
|
|
||||||
|
|
||||||
The `existing_batch_id != rec.id` guard avoids surfacing the just-failed
|
|
||||||
batch as a "previous" batch (it never persisted).
|
|
||||||
|
|
||||||
### 835 path (line 588-602)
|
|
||||||
|
|
||||||
Same pattern with `find_existing_batch_for_remit` and the first remit's
|
|
||||||
`payer_claim_control_number` (`result.claims[0].payer_claim_control_number`):
|
|
||||||
|
|
||||||
```python
|
|
||||||
except IntegrityError as exc:
|
|
||||||
first_pcn = result.claims[0].payer_claim_control_number if result.claims else None
|
|
||||||
existing_batch_id = (
|
|
||||||
store.find_existing_batch_for_remit(first_pcn)
|
|
||||||
if first_pcn else None
|
|
||||||
)
|
|
||||||
body = {
|
|
||||||
"error": "Duplicate remittance",
|
|
||||||
"detail": (
|
|
||||||
"This 835 file (or one previously ingested with the same "
|
|
||||||
"payer claim control number) collides with an existing record. "
|
|
||||||
"Remove the existing remittance before retrying."
|
|
||||||
),
|
|
||||||
"batch_id": rec.id,
|
|
||||||
}
|
|
||||||
if existing_batch_id and existing_batch_id != rec.id:
|
|
||||||
body["existing_batch_id"] = existing_batch_id
|
|
||||||
log.warning("Duplicate remittance while persisting batch %s: %s", rec.id, exc)
|
|
||||||
return JSONResponse(status_code=409, content=body)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. Frontend change
|
|
||||||
|
|
||||||
### `src/lib/api.ts`
|
|
||||||
|
|
||||||
`ApiError` carries an optional `existingBatchId`:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
export class ApiError extends Error {
|
|
||||||
constructor(
|
|
||||||
public status: number,
|
|
||||||
message: string,
|
|
||||||
public existingBatchId: string | null = null,
|
|
||||||
) {
|
|
||||||
super(message);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
`readErrorBody()` parses JSON and, when status ≥ 400, attempts to extract
|
|
||||||
`existing_batch_id` and pass it through. `parse837`/`parse835` throw
|
|
||||||
`new ApiError(res.status, detail, existingBatchId)`.
|
|
||||||
|
|
||||||
### `src/pages/Upload.tsx`
|
|
||||||
|
|
||||||
New local state:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
type UploadError = {
|
|
||||||
kind: "duplicate";
|
|
||||||
existingBatchId: string | null;
|
|
||||||
filename: string;
|
|
||||||
};
|
|
||||||
const [uploadError, setUploadError] = useState<UploadError | null>(null);
|
|
||||||
```
|
|
||||||
|
|
||||||
On mutation error:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
} catch (err) {
|
|
||||||
if (err instanceof ApiError && err.status === 409) {
|
|
||||||
setUploadError({
|
|
||||||
kind: "duplicate",
|
|
||||||
existingBatchId: err.existingBatchId,
|
|
||||||
filename: file.name,
|
|
||||||
});
|
|
||||||
toast.error("Duplicate claim — file not ingested");
|
|
||||||
} else {
|
|
||||||
toast.error(err instanceof Error ? err.message : "Failed to parse file");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Inline error panel JSX (renders above streaming results when
|
|
||||||
`uploadError` is set):
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
{uploadError ? (
|
|
||||||
<div className="error-panel">
|
|
||||||
<Badge variant="destructive">409</Badge>
|
|
||||||
<div className="error-title">Duplicate claim — file not ingested</div>
|
|
||||||
<div className="error-detail">
|
|
||||||
{filename} collides with an existing record.
|
|
||||||
{existingBatchId
|
|
||||||
? " Open the existing batch to compare, or pick a different file."
|
|
||||||
: " Pick a different file."}
|
|
||||||
</div>
|
|
||||||
<div className="error-actions">
|
|
||||||
{existingBatchId ? (
|
|
||||||
<Button onClick={() => navigate(`/batches/${existingBatchId}`)}>
|
|
||||||
Open existing batch →
|
|
||||||
</Button>
|
|
||||||
) : null}
|
|
||||||
<Button variant="ghost" onClick={() => { pickFile(null); }}>
|
|
||||||
Pick a different file
|
|
||||||
</Button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
```
|
|
||||||
|
|
||||||
Styling matches the existing paper-toned surface (slate/parchment) per
|
|
||||||
the hybrid dark/paper treatment used elsewhere in the app.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Files changed
|
|
||||||
|
|
||||||
* `backend/src/cyclone/migrations/0013_drop_claims_unique_constraint.sql` — new
|
|
||||||
* `backend/src/cyclone/store.py` — `find_existing_batch_for_claim`,
|
|
||||||
`find_existing_batch_for_remit`
|
|
||||||
* `backend/src/cyclone/api.py` — both 409 handlers add
|
|
||||||
`existing_batch_id` lookup
|
|
||||||
* `src/lib/api.ts` — `ApiError.existingBatchId`
|
|
||||||
* `src/pages/Upload.tsx` — error state + inline panel
|
|
||||||
* `src/pages/Upload.test.tsx` — new test file
|
|
||||||
* `backend/tests/test_api_parse_persists.py` — new test cases
|
|
||||||
* `backend/tests/test_db_migrate.py` — new migration test
|
|
||||||
* `backend/tests/test_store.py` — new helper tests
|
|
||||||
|
|
||||||
No new dependencies. No config changes.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. Out of scope
|
|
||||||
|
|
||||||
* A "Replace existing batch" destructive action — requires
|
|
||||||
`DELETE /api/batches/{id}` and reconciliation cascade handling; deferred.
|
|
||||||
* General 4xx error UI for other statuses (empty file, parse error,
|
|
||||||
validation errors). Those already surface in toasts and the streaming
|
|
||||||
view; a future SP can generalize the inline panel pattern.
|
|
||||||
* Renaming `patient_control_number` to `subscriber_member_id` to match
|
|
||||||
what it actually stores. Out of scope for this fix; tracked separately
|
|
||||||
if it becomes a source of confusion.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9. Risk
|
|
||||||
|
|
||||||
* **Migration reversibility**: SQLite has no `DROP CONSTRAINT`; the
|
|
||||||
recreation is destructive to schema (but not to data — `INSERT INTO
|
|
||||||
claims_new SELECT * FROM claims` preserves every row). If the
|
|
||||||
migration fails mid-way, the implicit transaction (`engine.begin()`
|
|
||||||
in `db_migrate.py`) rolls back. `PRAGMA defer_foreign_keys = ON` is
|
|
||||||
required because SQLite otherwise can't drop a table that other
|
|
||||||
tables reference (`remittances.claim_id`, `matches.claim_id`,
|
|
||||||
`line_reconciliations.claim_id`, `activity_events.claim_id`). The
|
|
||||||
deferred checks fire at commit and validate against the renamed
|
|
||||||
`claims` table.
|
|
||||||
* **Loss of uniqueness**: after the migration, two claims in one batch
|
|
||||||
*can* share a `patient_control_number`. This is the intended behavior.
|
|
||||||
Claim identity is still unique via `claims.id` (CLM01, PK) and the
|
|
||||||
existing dedup check in `store.add` (`s.get(Claim, claim.claim_id)`).
|
|
||||||
* **`existing_batch_id` race**: the helper runs after the failed
|
|
||||||
`store.add` transaction has rolled back. Between rollback and helper
|
|
||||||
call, another writer could delete the colliding claim. Result: 409
|
|
||||||
fires with no `existing_batch_id`; UI shows panel without link.
|
|
||||||
Acceptable — the panel still explains the collision and offers
|
|
||||||
"Pick a different file."
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 10. Test plan
|
|
||||||
|
|
||||||
| Test | Asserts |
|
|
||||||
|---|---|
|
|
||||||
| `test_db_migrate.py::test_drop_claims_unique_constraint` | After running migrations from v12, `user_version=13`, no `*claims*unique*` index exists, two rows with same `(batch_id, patient_control_number)` insert cleanly. |
|
|
||||||
| `test_db_migrate.py::test_migration_idempotent` | Running migrations on a v13 DB is a no-op. |
|
|
||||||
| `test_store.py::test_find_existing_batch_for_claim` | Helper returns None for unknown, returns batch_id for known, deterministic on duplicates. |
|
|
||||||
| `test_api_parse_persists.py::test_409_response_includes_existing_batch_id` | Upload duplicate; assert body has `existing_batch_id` pointing to the right batch. |
|
|
||||||
| `test_api_parse_persists.py::test_multi_claim_batch_with_duplicate_member_id_succeeds` | Upload a file with duplicate `member_id` claims; assert 200, all claims persisted. |
|
|
||||||
| `Upload.test.tsx::test_error_panel_renders_on_409_with_link` | Mock `useParse` to throw `ApiError(409, ..., existingBatchId)`; assert panel visible, link present. |
|
|
||||||
| `Upload.test.tsx::test_error_panel_renders_on_409_without_link` | Mock 409 with `existingBatchId: null`; assert panel visible, no link. |
|
|
||||||
| `Upload.test.tsx::test_no_panel_on_non_409` | Mock 400; assert panel absent, only toast. |
|
|
||||||
| `Upload.test.tsx::test_pick_different_clears_error` | Click button; assert `pickFile(null)` called and `errorState` cleared. |
|
|
||||||
@@ -1,228 +0,0 @@
|
|||||||
# SP17 — Automated Encrypted DB Backups
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Branch:** `sp17-encrypted-backups`
|
|
||||||
**Status:** Shipped
|
|
||||||
**Scope:** Backend only. No frontend changes (operator-only admin function).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Why this exists
|
|
||||||
|
|
||||||
The Cyclone SQLite database at `~/.local/share/cyclone/cyclone.db` is
|
|
||||||
the only authoritative store of every claim, remittance, audit event,
|
|
||||||
and reconciliation decision Cyclone has ever made. The README documents
|
|
||||||
`sqldiff .backup /path/to/backup.db` as a *manual* recipe. That has two
|
|
||||||
problems:
|
|
||||||
|
|
||||||
1. **Manual is unreliable.** The operator forgets. A disk failure on
|
|
||||||
the operator's laptop is unrecoverable. The 6-year HIPAA retention
|
|
||||||
expectation is not met by a recipe.
|
|
||||||
2. **No encryption.** The backup file inherits SQLCipher's encryption
|
|
||||||
if the live DB is encrypted, but only because the `.backup` API
|
|
||||||
copies the raw pages verbatim. If the operator ever exports a
|
|
||||||
backup for off-site storage (the obvious DR move) the file is
|
|
||||||
already encrypted — but there's no second layer, no key rotation,
|
|
||||||
no test of decryption, and no restore-drill automation.
|
|
||||||
|
|
||||||
SP17 fixes both: an automated tick creates an encrypted backup on a
|
|
||||||
schedule (default 24h), applies a retention policy (default 30 days),
|
|
||||||
and exposes API + CLI surface for create / list / restore / verify.
|
|
||||||
Restoration is two-step (initiate → confirm) to prevent an idle
|
|
||||||
browser tab from nuking a live DB.
|
|
||||||
|
|
||||||
The encryption layer is independent of SQLCipher: AES-256-GCM with a
|
|
||||||
passphrase-derived key (PBKDF2-HMAC-SHA256, 200k iterations). The
|
|
||||||
passphrase lives in the macOS Keychain alongside the SQLCipher key.
|
|
||||||
If the passphrase is missing, the backup layer falls back to deriving
|
|
||||||
a key from the SQLCipher key (less ideal but never silently broken).
|
|
||||||
|
|
||||||
## 2. File format
|
|
||||||
|
|
||||||
Each backup is a single file `<dir>/cyclone-backup-YYYYMMDDTHHMMSSZ.bin`
|
|
||||||
plus a sidecar `<...>.meta.json`:
|
|
||||||
|
|
||||||
```
|
|
||||||
+----------------+------------------+---------+--------+
|
|
||||||
| salt (16 bytes) | nonce (12 bytes) | cipher | tag |
|
|
||||||
+----------------+------------------+---------+--------+
|
|
||||||
AES-256-GCM over the SQLite .backup bytes
|
|
||||||
```
|
|
||||||
|
|
||||||
The sidecar is plaintext JSON with the *metadata* an operator needs to
|
|
||||||
decide whether to restore:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"created_at": "2026-06-21T15:30:00Z",
|
|
||||||
"db_fingerprint": "sha256:7a1c...",
|
|
||||||
"table_count": 11,
|
|
||||||
"size_bytes": 245760,
|
|
||||||
"encryption": {
|
|
||||||
"kdf": "PBKDF2-HMAC-SHA256",
|
|
||||||
"kdf_iterations": 200000,
|
|
||||||
"cipher": "AES-256-GCM",
|
|
||||||
"key_fingerprint": "sha256:5e7c..."
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The sidecar is *not* required to decrypt; it's a manifest. A real
|
|
||||||
DR drill is: pull the `.bin` from cold storage, decrypt with the
|
|
||||||
passphrase, restore.
|
|
||||||
|
|
||||||
## 3. Components
|
|
||||||
|
|
||||||
### 3.1 `cyclone.backup` — low-level crypto + file I/O
|
|
||||||
|
|
||||||
Pure functions, no DB dependency:
|
|
||||||
|
|
||||||
- `derive_key(passphrase: str, salt: bytes) -> bytes` — PBKDF2-HMAC-SHA256, 200k iters, 32-byte output.
|
|
||||||
- `encrypt(plaintext: bytes, key: bytes) -> bytes` — returns salt||nonce||ciphertext||tag.
|
|
||||||
- `decrypt(blob: bytes, key: bytes) -> bytes` — raises `BackupDecryptError` on auth failure.
|
|
||||||
- `BackupFile` dataclass — `(path, size, created_at, table_count, db_fingerprint)`.
|
|
||||||
|
|
||||||
### 3.2 `cyclone.backup_service` — high-level coordinator
|
|
||||||
|
|
||||||
- `BackupService(backup_dir, passphrase, retention_days=30, db_url=None)`.
|
|
||||||
- `create_now() -> BackupRecord` — runs SQLite `.backup()` to a temp file, encrypts, moves into `backup_dir`, writes sidecar, persists a row in `db_backups`. Crash-safe: on any failure the temp file is removed and the DB row is marked `error` with the reason.
|
|
||||||
- `list_backups() -> list[BackupRecord]` — directory listing, joined with `db_backups` rows for status.
|
|
||||||
- `restore(backup_id, *, confirm: bool) -> RestoreResult` — copies encrypted backup aside, decrypts into a temp file, asks SQLite to load it via a fresh engine, then disposes the live engine and reopens. Two-step: first call returns `{restore_token, preview_table_count}`, second call with the token performs the swap.
|
|
||||||
- `verify(backup_id) -> VerifyResult` — decrypts, recomputes SHA-256, compares to sidecar's `db_fingerprint`.
|
|
||||||
- `prune() -> list[str]` — delete `.bin`/`.meta.json` pairs and `db_backups` rows older than `retention_days`. Returns the deleted paths.
|
|
||||||
|
|
||||||
### 3.3 Migration `0012_backups.sql`
|
|
||||||
|
|
||||||
```sql
|
|
||||||
-- version: 12
|
|
||||||
CREATE TABLE db_backups (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
filename TEXT NOT NULL,
|
|
||||||
backup_dir TEXT NOT NULL,
|
|
||||||
size_bytes INTEGER NOT NULL DEFAULT 0,
|
|
||||||
db_fingerprint TEXT,
|
|
||||||
table_count INTEGER NOT NULL DEFAULT 0,
|
|
||||||
created_at TEXT NOT NULL,
|
|
||||||
status TEXT NOT NULL, -- 'pending' | 'ok' | 'error' | 'pruned'
|
|
||||||
error_message TEXT,
|
|
||||||
completed_at TEXT
|
|
||||||
);
|
|
||||||
CREATE UNIQUE INDEX ux_db_backups_filename ON db_backups(backup_dir, filename);
|
|
||||||
CREATE INDEX ix_db_backups_created_at ON db_backups(created_at DESC);
|
|
||||||
CREATE INDEX ix_db_backups_status ON db_backups(status);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3.4 Scheduler integration (SP16 extension)
|
|
||||||
|
|
||||||
`BackupService` is configured in the lifespan alongside the MFT
|
|
||||||
scheduler. A separate `BackupScheduler` class wraps `BackupService`
|
|
||||||
and ticks on its own interval. Auto-start opt-in via
|
|
||||||
`CYCLONE_BACKUP_AUTOSTART=true`.
|
|
||||||
|
|
||||||
### 3.5 API endpoints
|
|
||||||
|
|
||||||
| Method | Path | Purpose |
|
|
||||||
|--------|------|---------|
|
|
||||||
| POST | `/api/admin/backup/create` | Create a backup now |
|
|
||||||
| GET | `/api/admin/backup/list` | List backups (newest first) |
|
|
||||||
| GET | `/api/admin/backup/status` | Last backup time, count, schedule |
|
|
||||||
| POST | `/api/admin/backup/{id}/verify` | Decrypt + checksum verify |
|
|
||||||
| POST | `/api/admin/backup/{id}/restore/initiate` | First call: get restore_token + preview |
|
|
||||||
| POST | `/api/admin/backup/{id}/restore/confirm` | Second call: actually swap |
|
|
||||||
| POST | `/api/admin/backup/prune` | Apply retention policy now |
|
|
||||||
|
|
||||||
### 3.6 CLI
|
|
||||||
|
|
||||||
```
|
|
||||||
cyclone backup create
|
|
||||||
cyclone backup list
|
|
||||||
cyclone backup verify <id|filename>
|
|
||||||
cyclone backup restore <id|filename> --yes
|
|
||||||
cyclone backup prune
|
|
||||||
cyclone backup init-passphrase # interactively set the Keychain passphrase
|
|
||||||
```
|
|
||||||
|
|
||||||
## 4. Audit events (SP11)
|
|
||||||
|
|
||||||
Every backup lifecycle event writes a tamper-evident `audit_log` row:
|
|
||||||
|
|
||||||
- `db.backup_created` — payload includes `backup_id`, `db_fingerprint`, `table_count`, `actor`.
|
|
||||||
- `db.backup_failed` — payload includes `reason`, `traceback_tail`.
|
|
||||||
- `db.backup_restored` — payload includes `backup_id`, `restored_at`, `actor`.
|
|
||||||
- `db.backup_pruned` — payload includes `deleted_paths: list[str]`, `actor`.
|
|
||||||
- `db.backup_passphrase_set` — payload includes `key_fingerprint`, `actor`.
|
|
||||||
|
|
||||||
## 5. Failure modes
|
|
||||||
|
|
||||||
| Failure | Behavior |
|
|
||||||
|---------|----------|
|
|
||||||
| SQLCipher key missing | create_now() refuses with `BackupError("encryption not enabled")` |
|
|
||||||
| Passphrase missing | Falls back to deriving key from SQLCipher key + a fixed salt (`cyclone-db-backup-fallback-v1`). Logged at WARNING. |
|
|
||||||
| Disk full | Temp file removed, row marked error, audit event written. |
|
|
||||||
| Decrypt fails (wrong passphrase) | `BackupDecryptError` raised, row marked error. |
|
|
||||||
| Restore initiated while app has live traffic | Two-step confirm gates the actual swap; the engine is rebuilt in `dispose_engine` + `reinit_engine`. Brief downtime (~50ms) acknowledged to operator. |
|
|
||||||
| Clock skew on sidecar.created_at | We use the filesystem mtime as ground truth, not the OS-reported time. |
|
|
||||||
|
|
||||||
## 6. Out of scope
|
|
||||||
|
|
||||||
- Off-site upload (S3, B2, etc.) — operator's `rsync` to their offsite is the v1 answer.
|
|
||||||
- Compression — `.backup` is already a copy of pages, not much win.
|
|
||||||
- Incremental backups — full `.backup` is the right atomicity unit.
|
|
||||||
- Backup encryption with HSM / KMS — local Keychain is the operator model.
|
|
||||||
- Backup-of-backups — that's a DR runbook item, not a v1 feature.
|
|
||||||
|
|
||||||
## 7. Tests
|
|
||||||
|
|
||||||
| Suite | Count | Covers |
|
|
||||||
|-------|-------|--------|
|
|
||||||
| `test_backup_crypto.py` | 8 | key derivation, encrypt/decrypt round-trip, tampered ciphertext, wrong passphrase |
|
|
||||||
| `test_backup_service.py` | 12 | create/list/verify/restore/prune, sidecar I/O, retention policy |
|
|
||||||
| `test_api_backup.py` | 9 | all 7 endpoints, error responses, two-step restore |
|
|
||||||
| `test_cli_backup.py` | 5 | all 5 subcommands |
|
|
||||||
|
|
||||||
Total: 34 new tests. All pass.
|
|
||||||
|
|
||||||
## 8. Operator runbook (post-SP17)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# One-time: set the backup passphrase (separate from the SQLCipher key)
|
|
||||||
cyclone backup init-passphrase
|
|
||||||
# Enter + confirm a strong passphrase; stored in macOS Keychain under
|
|
||||||
# service "cyclone", account "backup.passphrase".
|
|
||||||
|
|
||||||
# Manual backup
|
|
||||||
cyclone backup create
|
|
||||||
# → cyclone-backup-20260621T153000Z.bin + .meta.json in $CYCLONE_BACKUP_DIR
|
|
||||||
# (default: ~/.local/share/cyclone/backups/)
|
|
||||||
|
|
||||||
# List
|
|
||||||
cyclone backup list
|
|
||||||
|
|
||||||
# Verify a backup
|
|
||||||
cyclone backup verify 42
|
|
||||||
# → {"ok": true, "db_fingerprint": "sha256:...", "table_count": 11}
|
|
||||||
|
|
||||||
# Restore
|
|
||||||
cyclone backup restore 42 --yes
|
|
||||||
# → prompts for confirmation; rebuilds the engine against the restored DB
|
|
||||||
|
|
||||||
# Prune (also runs nightly on the scheduler tick)
|
|
||||||
cyclone backup prune
|
|
||||||
# → deletes backups older than CYCLONE_BACKUP_RETENTION_DAYS (default 30)
|
|
||||||
```
|
|
||||||
|
|
||||||
Auto-start the scheduler at app launch:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export CYCLONE_BACKUP_AUTOSTART=true
|
|
||||||
export CYCLONE_BACKUP_INTERVAL_HOURS=24 # default
|
|
||||||
export CYCLONE_BACKUP_RETENTION_DAYS=30 # default
|
|
||||||
export CYCLONE_BACKUP_DIR=~/.local/share/cyclone/backups # default
|
|
||||||
```
|
|
||||||
|
|
||||||
## 9. Why this ships after SP16
|
|
||||||
|
|
||||||
SP16 (MFT polling) was the last big operational gap before backups
|
|
||||||
became urgent: with the scheduler running, an operator can lose days
|
|
||||||
of inbound 999/277CA work in one crash if there's no recent backup.
|
|
||||||
SP17 closes that loop.
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
# SP20 — NPI Checksum + Tax ID Format Validation
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Branch:** `sp20-npi-validation`
|
|
||||||
**Status:** Shipped
|
|
||||||
**Scope:** Backend only. No frontend changes.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Why this exists
|
|
||||||
|
|
||||||
Cyclone's completeness review (`docs/reviews/2026-06-20-cyclone-completeness-review.md`
|
|
||||||
§3.1.10) flags this gap:
|
|
||||||
|
|
||||||
> **No NPI validation.** NPIs are captured (`claim.party.npi`) and
|
|
||||||
> used in matching, but never validated against the NPPES registry.
|
|
||||||
> Bad NPIs (typos, deactivated) silently propagate.
|
|
||||||
|
|
||||||
A real NPI is 10 digits where the last digit is a **Luhn checksum**
|
|
||||||
over the 9 preceding digits prefixed with the constant `80840`
|
|
||||||
(NPPES's "healthcare provider identifier" prefix).
|
|
||||||
|
|
||||||
**Tax ID** format: 9 digits, optionally formatted `XX-XXXXXXX`. We
|
|
||||||
don't validate against the IRS (that requires their published
|
|
||||||
e-file schema), but we *do* catch obvious typos — non-numeric chars,
|
|
||||||
wrong length, EIN prefix `00`/`07`/`8X` (reserved / never assigned).
|
|
||||||
|
|
||||||
This SP adds both checks as pure local validators — no network, no
|
|
||||||
NPPES call. Operators who want real NPPES verification can wire it
|
|
||||||
in later; this SP catches the 99% case (a typo) at parse time.
|
|
||||||
|
|
||||||
## 2. Operator surface
|
|
||||||
|
|
||||||
| Surface | Usage |
|
|
||||||
|---------|-------|
|
|
||||||
| Python | `from cyclone.npi import is_valid_npi, is_valid_tax_id` |
|
|
||||||
| CLI | `cyclone validate-npi 1881068062` |
|
|
||||||
| CLI | `cyclone validate-tax-id 72-1587149` |
|
|
||||||
| API | `GET /api/admin/validate-provider?npi=1881068062&tax_id=72-1587149` |
|
|
||||||
|
|
||||||
The parser's claim-level validator (cyclone.parsers.validator) gains
|
|
||||||
a new R-rule that flags bad NPIs as a `validation.warning` (not an
|
|
||||||
error — the operator might be intentionally ingesting test files with
|
|
||||||
placeholder NPIs).
|
|
||||||
|
|
||||||
## 3. Files
|
|
||||||
|
|
||||||
* `cyclone/npi.py` — new module (~120 LOC). `is_valid_npi()`,
|
|
||||||
`is_valid_tax_id()`, `npi_checksum()`, plus the NPPES constant.
|
|
||||||
* `cyclone.parsers.validator` — new rule that flags bad NPIs in the
|
|
||||||
billing / rendering / referring / service-facility loops.
|
|
||||||
* `cyclone.api` — new admin endpoint.
|
|
||||||
* `cyclone.cli` — `validate-npi` + `validate-tax-id` subcommands.
|
|
||||||
* Tests: `test_npi.py` (12), `test_api_validate_provider.py` (4),
|
|
||||||
`test_cli_validate.py` (4) — 20 new tests.
|
|
||||||
|
|
||||||
## 4. Threat model
|
|
||||||
|
|
||||||
NPIs and tax IDs are sensitive (PHI under HIPAA). The validators run
|
|
||||||
locally; nothing leaves the process. The CLI's `validate-npi`
|
|
||||||
subcommand doesn't log the value (operators shouldn't paste real
|
|
||||||
NPIs into shared logs).
|
|
||||||
@@ -1,798 +0,0 @@
|
|||||||
# Parse → Detect → Decide: 837P/835 Upload Workflow
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Branch:** `claims-unique-fix` (worktree)
|
|
||||||
**Status:** Draft (brainstorming approved, awaiting writing-plans)
|
|
||||||
**Supersedes:** `2026-06-21-cyclone-claims-unique-constraint-and-409-ux-design.md` (kept for the migration 0013 + store helper sections, which still apply).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Why this exists
|
|
||||||
|
|
||||||
Today's upload flow is "parse → validate → persist" in a single call.
|
|
||||||
When a claim's CLM01 collides with a prior batch, the persist raises
|
|
||||||
`IntegrityError`, the transaction rolls back, and the API returns 409
|
|
||||||
with **no parse result, no list of colliding claims, and no way to act**.
|
|
||||||
The user sees only an error message and a `batch_id` that doesn't exist.
|
|
||||||
|
|
||||||
This is wrong: the parse already happened. The user should see what was
|
|
||||||
parsed, see which claims collide with which prior batches, and decide
|
|
||||||
what to do (force-insert, delete the prior batch, or pick a different
|
|
||||||
file). The 409 response body today is too thin to make that decision.
|
|
||||||
|
|
||||||
The root cause of the 409s is a schema bug — see
|
|
||||||
`2026-06-21-cyclone-claims-unique-constraint-and-409-ux-design.md` §1.
|
|
||||||
Migration 0013 drops the `UNIQUE(batch_id, patient_control_number)` inline
|
|
||||||
constraint that 0003 was supposed to drop. After 0013 lands, **multi-claim
|
|
||||||
837P files where many CLM segments share a subscriber's `member_id` will
|
|
||||||
ingest cleanly for the first time**.
|
|
||||||
|
|
||||||
But 0013 alone is not enough. The current schema has `claims.id` and
|
|
||||||
`remittances.id` as single-column PRIMARY KEYs, which means the same
|
|
||||||
CLM01 cannot exist in two different batches. That makes "cross-batch
|
|
||||||
CLM01 collisions" impossible to express in the data — but it also makes
|
|
||||||
resubmits impossible, and it makes the 409-with-collision-summary workflow
|
|
||||||
this SP describes unreachable. **Migration 0014 (added as Task 1.3 to the
|
|
||||||
plan) relaxes the PKs to composite `(batch_id, id)`.** After 0014 lands,
|
|
||||||
real resubmits are representable, the pre-flight 409 path actually fires,
|
|
||||||
and the workflow defined below is exercisable end-to-end against real data.
|
|
||||||
|
|
||||||
This SP defines the workflow for both classes of collision:
|
|
||||||
1. Multi-claim files with shared `member_id` (no longer a 409 after 0013).
|
|
||||||
2. Files where one or more CLM01s exist in a prior batch (a 409 after 0014; this SP defines the UX for it).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Operator surface
|
|
||||||
|
|
||||||
| Surface | Change |
|
|
||||||
|---|---|
|
|
||||||
| Backend | Pre-flight dedup check in `parse_837` and `parse_835`. New `?force=true` query param. New `DELETE /api/batches/{id}` endpoint. 409 body shape changes. |
|
|
||||||
| Frontend | `Upload.tsx` panel renders the full parse result + collision summary, with actions: "Force insert (skip dups)", "Open prior batch", "Delete prior batch and retry", "Pick a different file". |
|
|
||||||
| Tests | Migration + store helpers (already done in `claims-unique-fix`). New tests for: pre-flight dedup, force-insert, within-file dup, race 409, DELETE endpoint, frontend panel. |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. The workflow
|
|
||||||
|
|
||||||
### 3.1 No collision (the happy path)
|
|
||||||
|
|
||||||
```
|
|
||||||
User → POST /api/parse-837 (file)
|
|
||||||
← 200 + ParseResult + batch_id
|
|
||||||
Batch persisted. UI shows parsed claims and links to the new batch.
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3.2 Collision (the new path)
|
|
||||||
|
|
||||||
```
|
|
||||||
User → POST /api/parse-837 (file)
|
|
||||||
← 409 + {
|
|
||||||
error: "Duplicate claim",
|
|
||||||
detail: "...",
|
|
||||||
existing_batch_id: "B123", # most-recent prior batch with a colliding CLM01
|
|
||||||
collisions: {
|
|
||||||
colliding_claim_ids: ["A", "B"],
|
|
||||||
total_collisions: 2,
|
|
||||||
total_claims: 141, # claims in the file
|
|
||||||
new_claims_after_skip: 139, # claims that WOULD be inserted on force
|
|
||||||
},
|
|
||||||
parse_result: { ... full ParseResult ... },
|
|
||||||
}
|
|
||||||
User sees the parse result in the panel.
|
|
||||||
User can:
|
|
||||||
- Click "Force insert (skip 2 dups)" → POST /api/parse-837?force=true (same file)
|
|
||||||
← 200 + ParseResult + { skipped_claim_ids: ["A", "B"], inserted: 139 }
|
|
||||||
- Click "Open prior batch" → navigate to /batches/B123
|
|
||||||
- Click "Delete prior batch" → DELETE /api/batches/B123, then click "Re-upload"
|
|
||||||
- Click "Pick a different file" → clear the upload state
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3.3 Force-insert after collision
|
|
||||||
|
|
||||||
`force=true` skips the pre-flight check. The store's existing per-row
|
|
||||||
`s.get(Claim, claim_id)` dedup still skips colliding rows silently, so
|
|
||||||
the new batch persists with only the non-colliding claims. The response
|
|
||||||
body includes `skipped_claim_ids` so the UI can show what was skipped.
|
|
||||||
|
|
||||||
`force=true` does NOT bypass the parser. If the file fails validation
|
|
||||||
(missing diagnosis, malformed segment), the response is still 422.
|
|
||||||
|
|
||||||
### 3.4 Race condition (pre-flight clean, persist fails)
|
|
||||||
|
|
||||||
If a pre-flight dedup check finds no collisions, but a concurrent process
|
|
||||||
ingests a colliding CLM01 between the check and the persist, the persist
|
|
||||||
will still raise `IntegrityError`. The handler catches it and returns
|
|
||||||
**the same 409 shape as the pre-flight collision** with
|
|
||||||
`existing_batch_id` set to the racing batch and `detail` mentioning
|
|
||||||
"another process ingested this between the check and the persist —
|
|
||||||
re-upload to retry". The user re-runs the same flow.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Within-file duplicates
|
|
||||||
|
|
||||||
If the file itself has the same CLM01 twice (a malformed file, not a
|
|
||||||
cross-batch collision), the pre-flight check catches it the same way:
|
|
||||||
it returns 409 with `existing_batch_id: null` and `detail: "CLM01 A
|
|
||||||
appears twice in this file"`. The user can only force-insert (which
|
|
||||||
skips the second instance). They can't "delete the prior batch" because
|
|
||||||
there isn't one — it's a bad file.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. The 409 body shape
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"error": "Duplicate claim",
|
|
||||||
"detail": "This file (or one previously ingested with the same claim control number) collides with an existing record. 2 of 141 claims collide with batch B123.",
|
|
||||||
"batch_id": null,
|
|
||||||
"existing_batch_id": "B123",
|
|
||||||
"collisions": {
|
|
||||||
"colliding_claim_ids": ["A", "B"],
|
|
||||||
"total_collisions": 2,
|
|
||||||
"total_claims": 141,
|
|
||||||
"new_claims_after_skip": 139
|
|
||||||
},
|
|
||||||
"parse_result": { ... full ParseResult ... }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Field semantics:
|
|
||||||
- `error`: short tag for the UI ("Duplicate claim", "Duplicate remittance", "Within-file duplicate CLM01").
|
|
||||||
- `detail`: human-readable, mentions the count and the existing batch when known.
|
|
||||||
- `batch_id`: always `null` on 409 (the insert rolled back).
|
|
||||||
- `existing_batch_id`: the most-recent prior batch that contains a colliding CLM01, or `null` if (a) the collision is within-file, or (b) the colliding claim has since been deleted (race).
|
|
||||||
- `collisions.colliding_claim_ids`: subset of `parse_result.claims[].claim_id` that collides.
|
|
||||||
- `collisions.total_claims`: count from `parse_result.summary.total_claims`.
|
|
||||||
- `collisions.new_claims_after_skip`: `total_claims - total_collisions`.
|
|
||||||
- `parse_result`: the full `ParseResult` (same shape as a 200 response body). The UI uses this to render the parsed claims list.
|
|
||||||
|
|
||||||
The 200 body on `force=true` adds `skipped_claim_ids: ["A", "B"]` at the top level so the UI can show a "skipped" badge per claim.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. `DELETE /api/batches/{id}`
|
|
||||||
|
|
||||||
New endpoint. Cascades through `ON DELETE CASCADE` FKs:
|
|
||||||
|
|
||||||
```
|
|
||||||
batches ─┬─ claims ─┬─ matches
|
|
||||||
│ ├─ activity_events (claim_id)
|
|
||||||
│ └─ line_reconciliations
|
|
||||||
├─ remittances ─┬─ cas_adjustments
|
|
||||||
│ ├─ service_line_payments
|
|
||||||
│ └─ activity_events (remittance_id)
|
|
||||||
└─ activity_events (batch_id only)
|
|
||||||
```
|
|
||||||
|
|
||||||
FKs already declare `ON DELETE CASCADE` in the migrations, so the
|
|
||||||
SQLite engine handles the cascade. The endpoint just needs to
|
|
||||||
`session.delete(batch_row)` and commit.
|
|
||||||
|
|
||||||
The endpoint:
|
|
||||||
- `204 No Content` on success.
|
|
||||||
- `404 Not Found` if the batch doesn't exist.
|
|
||||||
- `409 Conflict` if the batch has any claims in a non-`submitted` state
|
|
||||||
(e.g., `paid`, `reversed`, `denied`). Forces the user to first
|
|
||||||
unreconcile — same as the existing 409 pattern for `manual_match` /
|
|
||||||
`manual_unmatch` (see `store.py:AlreadyMatchedError`).
|
|
||||||
|
|
||||||
A `batch_deleted` activity event is recorded before the delete so the
|
|
||||||
audit log has a tombstone. The event's `batch_id` will be `null` after
|
|
||||||
the cascade (the FK is to `batches.id` with no `ON DELETE` clause
|
|
||||||
specified in any migration; verify in `migrations/0001_initial.sql` —
|
|
||||||
the spec says we preserve audit history). If the FK is `ON DELETE
|
|
||||||
CASCADE`, we record the event AFTER the cascade with `batch_id` set to
|
|
||||||
the deleted id and rely on the cascade to remove it (acceptable, or we
|
|
||||||
use a no-cascade FK and keep the tombstone). **Open question resolved
|
|
||||||
during implementation by reading the actual FK clauses.**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Backend implementation
|
|
||||||
|
|
||||||
### 7.1 New dedup helper
|
|
||||||
|
|
||||||
`backend/src/cyclone/store.py` (already added in `claims-unique-fix`):
|
|
||||||
|
|
||||||
```python
|
|
||||||
def find_existing_batch_for_claim(claim_id: str) -> str | None:
|
|
||||||
"""Return the batch_id of the first batch containing this claim id, or None.
|
|
||||||
|
|
||||||
Pure read; opens a short-lived session. Used by the 837 409 handler to
|
|
||||||
surface which prior batch already holds the same CLM01.
|
|
||||||
|
|
||||||
Returns the most-recent batch (ORDER BY parsed_at DESC LIMIT 1) so the
|
|
||||||
UI links to the most likely "where did the dup come from" answer.
|
|
||||||
"""
|
|
||||||
from sqlalchemy import select
|
|
||||||
from cyclone.db import Claim
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.execute(
|
|
||||||
select(Claim.batch_id)
|
|
||||||
.where(Claim.id == claim_id)
|
|
||||||
.order_by(Claim.state_changed_at.desc()) # most-recent touch
|
|
||||||
.limit(1)
|
|
||||||
).first()
|
|
||||||
return row[0] if row else None
|
|
||||||
|
|
||||||
|
|
||||||
def find_existing_batch_for_remit(remit_id: str) -> str | None:
|
|
||||||
"""Same shape as find_existing_batch_for_claim but for remittances."""
|
|
||||||
from sqlalchemy import select
|
|
||||||
from cyclone.db import Remittance
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
row = s.execute(
|
|
||||||
select(Remittance.batch_id)
|
|
||||||
.where(Remittance.id == remit_id)
|
|
||||||
.order_by(Remittance.received_at.desc())
|
|
||||||
.limit(1)
|
|
||||||
).first()
|
|
||||||
return row[0] if row else None
|
|
||||||
```
|
|
||||||
|
|
||||||
The current `claims-unique-fix` implementation uses
|
|
||||||
`select(Claim.batch_id).where(Claim.id == claim_id).limit(1)` without
|
|
||||||
`ORDER BY`. We replace it with the ordered version to satisfy
|
|
||||||
"return the most-recent colliding batch".
|
|
||||||
|
|
||||||
### 7.2 New pre-flight dedup check
|
|
||||||
|
|
||||||
`backend/src/cyclone/dedup.py` (new file, single responsibility):
|
|
||||||
|
|
||||||
```python
|
|
||||||
"""Pre-flight dedup for parsed 837P/835 batches.
|
|
||||||
|
|
||||||
Splits the parsed result into "would-insert" and "would-skip" sets by
|
|
||||||
querying the DB for any claim_id / payer_claim_control_number already
|
|
||||||
present. Also detects within-file duplicates by counting claim_id
|
|
||||||
frequencies.
|
|
||||||
|
|
||||||
Used by the parse-837 and parse-835 endpoints between validation and
|
|
||||||
persist, so the user can see the parse result + collision summary
|
|
||||||
before any DB write.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from collections import Counter
|
|
||||||
from dataclasses import dataclass
|
|
||||||
|
|
||||||
from sqlalchemy import select
|
|
||||||
from sqlalchemy.orm import Session
|
|
||||||
|
|
||||||
from cyclone import db
|
|
||||||
from cyclone.db import Claim, Remittance
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class CollisionReport:
|
|
||||||
"""What the parse endpoint needs to render a 409 response."""
|
|
||||||
colliding_claim_ids: list[str] # CLM01s (837) or CLP01s (835)
|
|
||||||
existing_batch_id: str | None # most-recent prior batch with a collision, or None
|
|
||||||
within_file_duplicate_ids: list[str] # CLM01s appearing twice in this file (subset of colliding_claim_ids)
|
|
||||||
total_claims: int
|
|
||||||
|
|
||||||
|
|
||||||
def preflight_837(result, session: Session | None = None) -> CollisionReport:
|
|
||||||
"""Detect 837 collisions: within-file dupes + cross-batch CLM01 dupes."""
|
|
||||||
claim_ids = [c.claim_id for c in result.claims]
|
|
||||||
counts = Counter(claim_ids)
|
|
||||||
within_file_duplicate_ids = sorted(
|
|
||||||
cid for cid, n in counts.items() if n > 1
|
|
||||||
)
|
|
||||||
seen: set[str] = set(claim_ids)
|
|
||||||
if not seen:
|
|
||||||
return CollisionReport(
|
|
||||||
colliding_claim_ids=[],
|
|
||||||
existing_batch_id=None,
|
|
||||||
within_file_duplicate_ids=[],
|
|
||||||
total_claims=0,
|
|
||||||
)
|
|
||||||
|
|
||||||
own_session = session is None
|
|
||||||
if own_session:
|
|
||||||
session = db.SessionLocal()()
|
|
||||||
try:
|
|
||||||
rows = session.execute(
|
|
||||||
select(Claim.id, Claim.batch_id)
|
|
||||||
.where(Claim.id.in_(seen))
|
|
||||||
.order_by(Claim.state_changed_at.desc())
|
|
||||||
).all()
|
|
||||||
finally:
|
|
||||||
if own_session:
|
|
||||||
session.close()
|
|
||||||
|
|
||||||
db_collisions = {cid: bid for cid, bid in rows}
|
|
||||||
colliding = sorted(cid for cid in seen if cid in db_collisions)
|
|
||||||
existing_batch_id = next(iter(db_collisions.values()), None) if db_collisions else None
|
|
||||||
return CollisionReport(
|
|
||||||
colliding_claim_ids=colliding,
|
|
||||||
existing_batch_id=existing_batch_id,
|
|
||||||
within_file_duplicate_ids=within_file_duplicate_ids,
|
|
||||||
total_claims=len(claim_ids),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def preflight_835(result, session: Session | None = None) -> CollisionReport:
|
|
||||||
"""Same shape for 835 remittances. Payer claim control number = CLP01 = remittance.id."""
|
|
||||||
pcns = [c.payer_claim_control_number for c in result.claims]
|
|
||||||
counts = Counter(pcns)
|
|
||||||
within_file_duplicate_ids = sorted(p for p, n in counts.items() if n > 1)
|
|
||||||
seen = set(pcns)
|
|
||||||
if not seen:
|
|
||||||
return CollisionReport(
|
|
||||||
colliding_claim_ids=[],
|
|
||||||
existing_batch_id=None,
|
|
||||||
within_file_duplicate_ids=[],
|
|
||||||
total_claims=0,
|
|
||||||
)
|
|
||||||
|
|
||||||
own_session = session is None
|
|
||||||
if own_session:
|
|
||||||
session = db.SessionLocal()()
|
|
||||||
try:
|
|
||||||
rows = session.execute(
|
|
||||||
select(Remittance.id, Remittance.batch_id)
|
|
||||||
.where(Remittance.id.in_(seen))
|
|
||||||
.order_by(Remittance.received_at.desc())
|
|
||||||
).all()
|
|
||||||
finally:
|
|
||||||
if own_session:
|
|
||||||
session.close()
|
|
||||||
|
|
||||||
db_collisions = {pcn: bid for pcn, bid in rows}
|
|
||||||
colliding = sorted(pcn for pcn in seen if pcn in db_collisions)
|
|
||||||
existing_batch_id = next(iter(db_collisions.values()), None) if db_collisions else None
|
|
||||||
return CollisionReport(
|
|
||||||
colliding_claim_ids=colliding,
|
|
||||||
existing_batch_id=existing_batch_id,
|
|
||||||
within_file_duplicate_ids=within_file_duplicate_ids,
|
|
||||||
total_claims=len(pcns),
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
### 7.3 Modified `parse_837` endpoint
|
|
||||||
|
|
||||||
```python
|
|
||||||
@app.post("/api/parse-837")
|
|
||||||
async def parse_837(
|
|
||||||
request: Request,
|
|
||||||
file: UploadFile = File(...),
|
|
||||||
payer: str = Query("co_medicaid"),
|
|
||||||
include_raw_segments: bool = Query(True),
|
|
||||||
strict: bool = Query(False),
|
|
||||||
ack: bool = Query(False),
|
|
||||||
force: bool = Query(False), # NEW
|
|
||||||
) -> Any:
|
|
||||||
# ... existing parse + validate ...
|
|
||||||
|
|
||||||
if _has_claim_validation_errors(result):
|
|
||||||
return JSONResponse(status_code=422, content=json.loads(result.model_dump_json()))
|
|
||||||
|
|
||||||
# NEW: pre-flight dedup check
|
|
||||||
if not force and result.claims:
|
|
||||||
report = dedup.preflight_837(result)
|
|
||||||
if report.colliding_claim_ids or report.within_file_duplicate_ids:
|
|
||||||
return _build_409_response(
|
|
||||||
result=result,
|
|
||||||
report=report,
|
|
||||||
error="Duplicate claim",
|
|
||||||
kind="cross_batch" if report.existing_batch_id else "within_file",
|
|
||||||
)
|
|
||||||
|
|
||||||
# Persist (existing path). On IntegrityError (race), same 409 shape.
|
|
||||||
rec = BatchRecord(
|
|
||||||
id=uuid.uuid4().hex,
|
|
||||||
kind="837p",
|
|
||||||
input_filename=file.filename or "upload.txt",
|
|
||||||
parsed_at=utcnow(),
|
|
||||||
result=result,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
store.add(rec, event_bus=request.app.state.event_bus)
|
|
||||||
except IntegrityError as exc:
|
|
||||||
# Race: pre-flight said clean, but persist hit a PK. Re-run pre-flight
|
|
||||||
# so the 409 body has the same shape.
|
|
||||||
report = dedup.preflight_837(result)
|
|
||||||
return _build_409_response(
|
|
||||||
result=result,
|
|
||||||
report=report,
|
|
||||||
error="Duplicate claim (race condition)",
|
|
||||||
kind="race",
|
|
||||||
)
|
|
||||||
|
|
||||||
# ... existing response ...
|
|
||||||
if _client_wants_json(request):
|
|
||||||
body = json.loads(result.model_dump_json())
|
|
||||||
if ack:
|
|
||||||
ack_body = _build_and_persist_ack(rec.id)
|
|
||||||
if ack_body is not None:
|
|
||||||
body["ack"] = ack_body
|
|
||||||
# If force=true, the store.add silently skipped some claims.
|
|
||||||
# Surface what was skipped so the UI can show a "skipped" badge.
|
|
||||||
if force:
|
|
||||||
body["skipped_claim_ids"] = sorted({
|
|
||||||
c.claim_id for c in result.claims
|
|
||||||
if _claim_skipped(c.claim_id, rec.id)
|
|
||||||
})
|
|
||||||
return JSONResponse(content=body)
|
|
||||||
# ... streaming response ...
|
|
||||||
```
|
|
||||||
|
|
||||||
Where:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def _build_409_response(
|
|
||||||
result, report, error: str, kind: str
|
|
||||||
) -> JSONResponse:
|
|
||||||
"""Build the standard 409 body for any dedup failure."""
|
|
||||||
if kind == "cross_batch":
|
|
||||||
detail = (
|
|
||||||
f"{len(report.colliding_claim_ids)} of {report.total_claims} "
|
|
||||||
f"claims collide with prior batch {report.existing_batch_id}. "
|
|
||||||
f"Force-insert to skip the duplicates, or delete the prior batch."
|
|
||||||
)
|
|
||||||
elif kind == "within_file":
|
|
||||||
detail = (
|
|
||||||
f"CLM01(s) {', '.join(report.within_file_duplicate_ids)} appear "
|
|
||||||
f"twice in this file. Force-insert will keep the first occurrence "
|
|
||||||
f"and skip the rest."
|
|
||||||
)
|
|
||||||
else: # race
|
|
||||||
detail = (
|
|
||||||
f"Another process ingested a colliding batch between the check "
|
|
||||||
f"and the persist. Re-upload to retry with the latest state."
|
|
||||||
)
|
|
||||||
body = {
|
|
||||||
"error": error,
|
|
||||||
"detail": detail,
|
|
||||||
"batch_id": None,
|
|
||||||
"existing_batch_id": report.existing_batch_id,
|
|
||||||
"collisions": {
|
|
||||||
"colliding_claim_ids": report.colliding_claim_ids,
|
|
||||||
"total_collisions": len(report.colliding_claim_ids),
|
|
||||||
"total_claims": report.total_claims,
|
|
||||||
"new_claims_after_skip": report.total_claims - len(report.colliding_claim_ids),
|
|
||||||
},
|
|
||||||
"parse_result": json.loads(result.model_dump_json()),
|
|
||||||
}
|
|
||||||
return JSONResponse(status_code=409, content=body)
|
|
||||||
```
|
|
||||||
|
|
||||||
`force=true` does NOT bypass validation (still 422 for bad data). It
|
|
||||||
only bypasses the pre-flight dedup. The `store.add` dedup still skips
|
|
||||||
colliding claims silently, but the response surfaces the skip list.
|
|
||||||
|
|
||||||
### 7.4 Modified `parse_835` endpoint
|
|
||||||
|
|
||||||
Same pattern, with `dedup.preflight_835` and the 835 parse result. Not
|
|
||||||
shown in detail; the structure mirrors 837.
|
|
||||||
|
|
||||||
### 7.5 New `DELETE /api/batches/{id}`
|
|
||||||
|
|
||||||
```python
|
|
||||||
@app.delete("/api/batches/{batch_id}")
|
|
||||||
def delete_batch(batch_id: str) -> dict:
|
|
||||||
"""Hard-delete a batch and all its child rows.
|
|
||||||
|
|
||||||
Returns 204 on success, 404 if missing, 409 if the batch has any
|
|
||||||
claims/remits in a non-`submitted` state (must unreconcile first).
|
|
||||||
"""
|
|
||||||
from cyclone import db
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
batch = s.get(db.Batch, batch_id)
|
|
||||||
if batch is None:
|
|
||||||
raise HTTPException(404, f"Batch {batch_id} not found")
|
|
||||||
# Refuse if any claim/remittance is past 'submitted' state
|
|
||||||
non_submitted = s.execute(
|
|
||||||
select(db.Claim.id)
|
|
||||||
.where(db.Claim.batch_id == batch_id)
|
|
||||||
.where(db.Claim.state != "submitted")
|
|
||||||
.limit(1)
|
|
||||||
).first()
|
|
||||||
if non_submitted is not None:
|
|
||||||
raise HTTPException(
|
|
||||||
409,
|
|
||||||
f"Batch {batch_id} has claims in non-submitted state; "
|
|
||||||
f"unreconcile first before deleting.",
|
|
||||||
)
|
|
||||||
# Record tombstone activity event before the cascade
|
|
||||||
s.add(db.ActivityEvent(
|
|
||||||
ts=utcnow(),
|
|
||||||
kind="batch_deleted",
|
|
||||||
batch_id=batch_id,
|
|
||||||
payload_json={"message": f"Batch {batch_id} deleted"},
|
|
||||||
))
|
|
||||||
s.flush()
|
|
||||||
s.delete(batch)
|
|
||||||
s.commit()
|
|
||||||
return {"ok": True, "batch_id": batch_id}
|
|
||||||
```
|
|
||||||
|
|
||||||
The FKs in the schema (`migrations/0001_initial.sql` and later) declare
|
|
||||||
`ON DELETE CASCADE` on `claims.batch_id`, `remittances.batch_id`, etc.
|
|
||||||
SQLite handles the cascade at the engine level. We verify this assumption
|
|
||||||
in the implementation test by deleting a batch with child rows and
|
|
||||||
asserting the child rows are gone.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. Frontend
|
|
||||||
|
|
||||||
### 8.1 `src/lib/api.ts`
|
|
||||||
|
|
||||||
`ApiError` carries more collision data:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
export class ApiError extends Error {
|
|
||||||
constructor(
|
|
||||||
public status: number,
|
|
||||||
message: string,
|
|
||||||
public existingBatchId: string | null = null,
|
|
||||||
public collisions: CollisionSummary | null = null,
|
|
||||||
public parseResult: unknown = null,
|
|
||||||
) {
|
|
||||||
super(message);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export type CollisionSummary = {
|
|
||||||
colliding_claim_ids: string[];
|
|
||||||
total_collisions: number;
|
|
||||||
total_claims: number;
|
|
||||||
new_claims_after_skip: number;
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
`parse837` adds `?force=true` to the URL when called for the
|
|
||||||
"force-insert" action:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
export async function parse837(
|
|
||||||
file: File,
|
|
||||||
options: { onProgress?: (p: number) => void; force?: boolean } = {},
|
|
||||||
): Promise<ParseResult> {
|
|
||||||
const url = `${base}/api/parse-837${options.force ? "?force=true" : ""}`;
|
|
||||||
// ... existing fetch + body parse ...
|
|
||||||
if (!res.ok) {
|
|
||||||
const { message, existingBatchId, collisions, parseResult } = await readErrorBody(res);
|
|
||||||
throw new ApiError(res.status, message, existingBatchId, collisions, parseResult);
|
|
||||||
}
|
|
||||||
return res.json();
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 8.2 `src/pages/Upload.tsx`
|
|
||||||
|
|
||||||
New state:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
type UploadError = {
|
|
||||||
kind: "duplicate";
|
|
||||||
existingBatchId: string | null;
|
|
||||||
collisions: CollisionSummary;
|
|
||||||
parseResult: ParseResult;
|
|
||||||
filename: string;
|
|
||||||
};
|
|
||||||
const [uploadError, setUploadError] = useState<UploadError | null>(null);
|
|
||||||
const [forceInserting, setForceInserting] = useState(false);
|
|
||||||
```
|
|
||||||
|
|
||||||
Panel JSX (above the streaming results):
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
{uploadError ? (
|
|
||||||
<div
|
|
||||||
role="alert"
|
|
||||||
className="rounded-md border border-destructive/40 bg-destructive/5 p-4 mx-auto max-w-3xl"
|
|
||||||
>
|
|
||||||
<div className="flex items-center gap-2">
|
|
||||||
<span className="inline-flex items-center rounded-md bg-destructive px-2 py-0.5 text-xs font-semibold text-destructive-foreground">
|
|
||||||
409
|
|
||||||
</span>
|
|
||||||
<span className="font-semibold">
|
|
||||||
{uploadError.collisions.total_collisions} of {uploadError.collisions.total_claims} claims
|
|
||||||
collide
|
|
||||||
{uploadError.existingBatchId
|
|
||||||
? ` with batch ${uploadError.existingBatchId}`
|
|
||||||
: " within this file"}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<p className="mt-2 text-sm text-muted-foreground">
|
|
||||||
File <span className="font-mono">{uploadError.filename}</span> would persist
|
|
||||||
{" "}{uploadError.collisions.new_claims_after_skip} of {uploadError.collisions.total_claims} claims.
|
|
||||||
Colliding CLM01s: {uploadError.collisions.colliding_claim_ids.join(", ")}.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<div className="mt-3 flex flex-wrap gap-2">
|
|
||||||
<Button
|
|
||||||
disabled={forceInserting}
|
|
||||||
onClick={async () => {
|
|
||||||
setForceInserting(true);
|
|
||||||
try {
|
|
||||||
// re-call with force=true; the response will be 200 + skipped_claim_ids
|
|
||||||
const result = await parse837(file, { onProgress: () => {}, force: true });
|
|
||||||
setParseResult(result);
|
|
||||||
setUploadError(null);
|
|
||||||
toast.success(
|
|
||||||
`Force-inserted: ${result.summary.total_claims - (result.skipped_claim_ids?.length ?? 0)} of ${result.summary.total_claims} claims (skipped ${result.skipped_claim_ids?.length ?? 0} dups)`,
|
|
||||||
);
|
|
||||||
} catch (err) {
|
|
||||||
toast.error(err instanceof Error ? err.message : "Force-insert failed");
|
|
||||||
} finally {
|
|
||||||
setForceInserting(false);
|
|
||||||
}
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
Force insert (skip {uploadError.collisions.total_collisions} dups)
|
|
||||||
</Button>
|
|
||||||
|
|
||||||
{uploadError.existingBatchId ? (
|
|
||||||
<>
|
|
||||||
<Button variant="outline" onClick={() => navigate(`/batches/${uploadError.existingBatchId}`)}>
|
|
||||||
Open prior batch
|
|
||||||
</Button>
|
|
||||||
<Button
|
|
||||||
variant="outline"
|
|
||||||
onClick={async () => {
|
|
||||||
if (!confirm(`Delete batch ${uploadError.existingBatchId}? This cannot be undone.`)) return;
|
|
||||||
await deleteBatch(uploadError.existingBatchId);
|
|
||||||
toast.success(`Deleted ${uploadError.existingBatchId}`);
|
|
||||||
setUploadError(null);
|
|
||||||
pickFile(null);
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
Delete prior batch
|
|
||||||
</Button>
|
|
||||||
</>
|
|
||||||
) : null}
|
|
||||||
|
|
||||||
<Button variant="ghost" onClick={() => { setUploadError(null); pickFile(null); }}>
|
|
||||||
Pick a different file
|
|
||||||
</Button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{/* The full parse result is rendered below so the user can see what was parsed. */}
|
|
||||||
<details className="mt-3 text-sm">
|
|
||||||
<summary>Show parsed claims ({uploadError.parseResult.claims.length})</summary>
|
|
||||||
<pre className="mt-2 max-h-64 overflow-auto rounded bg-muted p-2 text-xs">
|
|
||||||
{JSON.stringify(uploadError.parseResult.summary, null, 2)}
|
|
||||||
</pre>
|
|
||||||
</details>
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9. Database
|
|
||||||
|
|
||||||
Migration 0013 already exists on the `claims-unique-fix` worktree. It
|
|
||||||
drops the `UNIQUE(batch_id, patient_control_number)` inline constraint.
|
|
||||||
After it runs:
|
|
||||||
|
|
||||||
- The 409 fires only on actual CLM01 collisions (not the `member_id`
|
|
||||||
dedup that was over-constraining before).
|
|
||||||
- Multi-claim 837P files with shared `member_id` ingest cleanly for
|
|
||||||
the first time.
|
|
||||||
|
|
||||||
Migration 0014 (added as Task 1.3 in the plan) further relaxes the schema:
|
|
||||||
it changes the PKs on `claims` and `remittances` from single-column
|
|
||||||
(`id`) to composite (`batch_id`, `id`). This is what allows resubmits and
|
|
||||||
makes the workflow in §3 reachable.
|
|
||||||
|
|
||||||
No new tables. No new columns. The DELETE endpoint relies on existing
|
|
||||||
`ON DELETE CASCADE` FKs.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 10. Files changed
|
|
||||||
|
|
||||||
| File | Change |
|
|
||||||
|---|---|
|
|
||||||
| `backend/src/cyclone/migrations/0013_drop_claims_unique_constraint.sql` | new (DONE on `claims-unique-fix`) |
|
|
||||||
| `backend/src/cyclone/migrations/0014_relax_claims_remits_pk.sql` | new: composite PK `(batch_id, id)` on `claims` and `remittances`; updates FKs |
|
|
||||||
| `backend/src/cyclone/store.py` | new `find_existing_batch_for_claim` / `find_existing_batch_for_remit` (DONE) + new `delete_batch` method |
|
|
||||||
| `backend/src/cyclone/dedup.py` | new file: pre-flight `preflight_837` / `preflight_835` + `CollisionReport` dataclass |
|
|
||||||
| `backend/src/cyclone/api.py` | 837/835 endpoints: pre-flight check, force param, new 409 body, race handler, new DELETE endpoint |
|
|
||||||
| `src/lib/api.ts` | `ApiError` adds `collisions` + `parseResult`; `parse837`/`parse835` accept `force`; new `deleteBatch` |
|
|
||||||
| `src/pages/Upload.tsx` | new `UploadError` state, error panel JSX, force-insert handler, delete-prior handler |
|
|
||||||
| `src/pages/Upload.test.tsx` | new tests (4 cases from §11) |
|
|
||||||
| `backend/tests/test_db_migrate.py` | 0013 idempotency + UNIQUE-dropped tests (DONE); 0014 composite-PK + FK-cascade tests |
|
|
||||||
| `backend/tests/test_store.py` | `find_existing_batch_for_claim`/`remit` tests (DONE) |
|
|
||||||
| `backend/tests/test_dedup.py` | new tests for `preflight_837` / `preflight_835` (§11) |
|
|
||||||
| `backend/tests/test_api_parse_persists.py` | new tests: pre-flight 409, force-insert, within-file 409, race 409, DELETE endpoint (§11) |
|
|
||||||
|
|
||||||
No new dependencies. No config changes.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 11. Test plan
|
|
||||||
|
|
||||||
### Backend (pytest)
|
|
||||||
|
|
||||||
| Test | File | Asserts |
|
|
||||||
|---|---|---|
|
|
||||||
| `test_preflight_837_finds_no_collisions_on_empty_db` | `test_dedup.py` | empty DB → empty `colliding_claim_ids`, no `existing_batch_id` |
|
|
||||||
| `test_preflight_837_finds_cross_batch_collision` | `test_dedup.py` | pre-seed a claim; pre-flight returns that claim_id in `colliding_claim_ids` and the seeded batch in `existing_batch_id` |
|
|
||||||
| `test_preflight_837_finds_within_file_duplicate` | `test_dedup.py` | parsed result has the same CLM01 twice; pre-flight returns it in both `colliding_claim_ids` and `within_file_duplicate_ids` |
|
|
||||||
| `test_preflight_837_returns_most_recent_batch_id` | `test_dedup.py` | pre-seed 3 batches with the same CLM01 at different times; pre-flight returns the most-recent batch_id |
|
|
||||||
| `test_preflight_835_mirrors_837` | `test_dedup.py` | same shape for remittances |
|
|
||||||
| `test_parse_837_409_includes_parse_result_and_collisions` | `test_api_parse_persists.py` | pre-seed a claim; upload a file with a colliding CLM01; assert 409 with `parse_result`, `collisions.colliding_claim_ids`, `existing_batch_id` |
|
|
||||||
| `test_parse_837_409_within_file_duplicate_has_null_batch_id` | `test_api_parse_persists.py` | upload a file with the same CLM01 twice; assert 409 with `existing_batch_id: null` and `within_file_duplicate_ids` populated |
|
|
||||||
| `test_parse_837_force_true_persists_non_colliding_claims` | `test_api_parse_persists.py` | pre-seed a claim; upload a file with 3 claims, 1 colliding; assert 200 with `skipped_claim_ids: [colliding_id]`, the 2 non-colliding claims persist |
|
|
||||||
| `test_parse_837_force_true_does_not_bypass_validation` | `test_api_parse_persists.py` | a file that fails validation still returns 422 with `force=true` |
|
|
||||||
| `test_parse_837_race_409_uses_same_body_shape` | `test_api_parse_persists.py` | mock `store.add` to raise IntegrityError; assert 409 body has the same shape as the pre-flight 409 |
|
|
||||||
| `test_delete_batch_cascades_to_claims` | `test_api_parse_persists.py` | persist a batch with 2 claims; DELETE; assert batch and both claims are gone |
|
|
||||||
| `test_delete_batch_404_on_unknown` | `test_api_parse_persists.py` | DELETE /api/batches/does-not-exist → 404 |
|
|
||||||
| `test_delete_batch_409_on_reconciled_claims` | `test_api_parse_persists.py` | persist a batch, mark a claim state='paid'; DELETE → 409 |
|
|
||||||
| `test_parse_837_after_delete_succeeds` | `test_api_parse_persists.py` | pre-seed a colliding claim; DELETE that batch; re-upload the same file; assert 200 |
|
|
||||||
|
|
||||||
### Frontend (vitest)
|
|
||||||
|
|
||||||
| Test | File | Asserts |
|
|
||||||
|---|---|---|
|
|
||||||
| `test_error_panel_renders_on_409_with_collisions` | `Upload.test.tsx` | mock `parse837` to throw `ApiError(409, ..., PRIOR, collisions, parseResult)`; assert panel visible with all collision data |
|
|
||||||
| `test_force_insert_button_re_calls_with_force_true` | `Upload.test.tsx` | user clicks "Force insert"; assert `parse837` is called with `{ force: true }` |
|
|
||||||
| `test_delete_prior_button_calls_deleteBatch` | `Upload.test.tsx` | user clicks "Delete prior batch"; assert `deleteBatch(existingBatchId)` is called |
|
|
||||||
| `test_pick_different_clears_error` | `Upload.test.tsx` | user clicks "Pick a different file"; assert `uploadError` is cleared and file picker is reset |
|
|
||||||
| `test_no_panel_on_non_409` | `Upload.test.tsx` | 400 error; assert panel absent |
|
|
||||||
| `test_within_file_duplicate_omits_prior_batch_actions` | `Upload.test.tsx` | 409 with `existingBatchId: null`; assert "Open prior batch" and "Delete prior batch" buttons are absent |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 12. Out of scope
|
|
||||||
|
|
||||||
* Batch editing (update claim state, edit claim fields). Future SP.
|
|
||||||
* Cross-batch dedup REPORT (a "find all CLM01s in batches B1+B2+B3"
|
|
||||||
query). Future SP.
|
|
||||||
* Migration reversibility for 0013 — the recreation preserves data but
|
|
||||||
not schema history. Acceptable since 0013 just drops an inline
|
|
||||||
constraint; recreating the constraint would be a separate migration.
|
|
||||||
* Audit event for force-insert skips. The user explicitly chose to
|
|
||||||
skip silently; we honor that.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 13. Risk
|
|
||||||
|
|
||||||
* **Pre-flight check race**: between the check and the persist, a
|
|
||||||
concurrent process could ingest a colliding claim. The persist would
|
|
||||||
then raise `IntegrityError`; the handler returns the same 409 shape
|
|
||||||
with `detail` mentioning the race. The user re-runs. Acceptable.
|
|
||||||
* **DELETE on a large batch**: cascade through `claims`, `remittances`,
|
|
||||||
`matches`, `line_reconciliations`, `activity_events`. SQLite handles
|
|
||||||
the cascade in a single transaction; a 140-claim batch deletes in
|
|
||||||
<100ms. The endpoint refuses if any claim is past `submitted` state.
|
|
||||||
* **`force=true` silent skip**: the user clicks "Force insert" and
|
|
||||||
the response says "X of Y claims persisted, Z skipped". They
|
|
||||||
acknowledged this in the panel before clicking. No undo.
|
|
||||||
* **Within-file duplicates and force-insert**: the user can force-insert
|
|
||||||
a file with the same CLM01 twice. The first instance persists, the
|
|
||||||
second is silently skipped. This is intentional — within-file dupes
|
|
||||||
are usually a typo, and the user has explicitly asked to proceed.
|
|
||||||
* **`existing_batch_id` may be stale**: the helper returns the
|
|
||||||
most-recent batch by `state_changed_at` (or `received_at` for 835).
|
|
||||||
The user clicks "Open prior batch" and the batch may have been
|
|
||||||
deleted in the meantime. The BatchesList page already handles 404
|
|
||||||
gracefully.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 14. Rollout
|
|
||||||
|
|
||||||
1. **Schema**: migration 0013 applies on next `cyclone` startup.
|
|
||||||
Idempotent and reversible only by rebuilding the `claims` table
|
|
||||||
(acceptable; production data preserved by the INSERT...SELECT).
|
|
||||||
2. **Backend API**: new `force` param + new 409 body shape + new
|
|
||||||
DELETE endpoint. Existing clients that don't pass `force` see the
|
|
||||||
same behavior as before for collision-free files. Collision cases
|
|
||||||
now get a richer 409 body that includes `parse_result`; clients
|
|
||||||
that ignore the new fields keep working.
|
|
||||||
3. **Frontend**: `Upload.tsx` panel replaces the toast on 409. Users
|
|
||||||
who don't read the panel still see the toast and the 409 message
|
|
||||||
in the streaming view.
|
|
||||||
4. **No data migration**: nothing to migrate. 0013 is structural only.
|
|
||||||
@@ -1,502 +0,0 @@
|
|||||||
# Sub-project 22 — Cyclone Pipeline Agent: Design Spec
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Status:** Draft (awaiting user review)
|
|
||||||
**Branch:** `sp22-pipeline-agent` (not yet created; will branch from `main`)
|
|
||||||
**Aesthetic direction:** n/a (backend automation; no UI surface)
|
|
||||||
|
|
||||||
## 1. Scope
|
|
||||||
|
|
||||||
A production-grade agent (sibling Python package `cyclone-pipeline`) that
|
|
||||||
automates the full submission lifecycle of an EDI 837P file through Cyclone
|
|
||||||
to the Gainwell MFT, and watches for the inbound TA1 + 999 ACKs that
|
|
||||||
confirm acceptance. The agent exists so a single human operator can drop a
|
|
||||||
file in and walk away; the agent handles the browser upload, SFTP
|
|
||||||
submission, ACK polling, and report generation end to end, and writes a
|
|
||||||
self-contained run directory that OpenClaw / Nora / any other automation
|
|
||||||
can read.
|
|
||||||
|
|
||||||
**In scope:**
|
|
||||||
|
|
||||||
- Sibling Python package at `/Users/openclaw/dev/cyclone-pipeline/` (no
|
|
||||||
shared imports with the cyclone app; talks to cyclone exclusively over
|
|
||||||
HTTP).
|
|
||||||
- CLI surface: `run <file>`, `run-batch <dir>`, `status <run-id>`,
|
|
||||||
`resume <run-id>`, `check-835 <run-id>`.
|
|
||||||
- Library surface: `CyclonePipeline` class with `run()`, `run_batch()`,
|
|
||||||
`resume()`, `check_835()` for embedding in OpenClaw / Nora.
|
|
||||||
- 7-phase state machine: pre-flight → upload (browser) → verify parse →
|
|
||||||
SFTP submit → wait TA1 → wait 999 → scan HTML + report. The "scan
|
|
||||||
HTML" and "final report" steps are combined as phase 7 (the HTML
|
|
||||||
scan populates the report; the report write is phase 7's
|
|
||||||
postcondition).
|
|
||||||
- Per-run output folder with `report.{md,json}`, `run.log`,
|
|
||||||
`run.state.json`, and `screenshots/` with stable filenames.
|
|
||||||
- Crash-safe resume: `run.state.json` is written after each phase's
|
|
||||||
postcondition; `resume` skips completed phases.
|
|
||||||
- Headless + visible browser modes; default visible so screenshots are
|
|
||||||
useful for debugging.
|
|
||||||
- Defensive reliability: typed exceptions, 3× retry with backoff on
|
|
||||||
idempotent operations, timeouts per phase, screenshots on failure,
|
|
||||||
Keychain-failure guidance.
|
|
||||||
- Test suite: `pytest` + `pytest-asyncio` + `respx` (httpx mock) +
|
|
||||||
`pytest-playwright` for the small browser surface. Non-browser tests
|
|
||||||
run on every PR; Playwright tests gated to `main`.
|
|
||||||
|
|
||||||
**Out of scope (deferred):**
|
|
||||||
|
|
||||||
- **835 waiting within a single run.** CO Medicaid 835 remittances land
|
|
||||||
the following Monday, on the payment cycle. A run that blocks for up
|
|
||||||
to 7 days is impractical. The agent defers 835 detection to a separate
|
|
||||||
`check-835 <run-id>` subcommand that the operator runs on/after
|
|
||||||
Monday. The report always carries the `check-835` recipe.
|
|
||||||
- **277CA polling.** CO Medicaid does not use 277CA claim-level ACKs;
|
|
||||||
the agent never polls for them. (The `GET /api/277ca-acks` endpoint
|
|
||||||
exists in the API surface but is not exercised by this agent.)
|
|
||||||
- **Watch-folder / daemon mode.** The agent is on-demand. A cron job that
|
|
||||||
invokes `cyclone-pipeline run-batch /inbox/*.txt --strict` on a 5-min
|
|
||||||
schedule achieves the same effect with no daemon in the loop.
|
|
||||||
- **Multi-payer support.** The agent targets CO Medicaid (the only
|
|
||||||
configured payer in `config/payers.yaml`). A new payer means new
|
|
||||||
payer-specific regex / file-naming / SFTP block — out of scope for v1.
|
|
||||||
- **Patient peek or other drill-down features.** This is a backend
|
|
||||||
automation; no UI changes.
|
|
||||||
- **Editing the audit log or any existing cyclone data.** The agent
|
|
||||||
only reads + triggers; it never writes to cyclone's DB except through
|
|
||||||
the published APIs.
|
|
||||||
|
|
||||||
## 2. Locked decisions
|
|
||||||
|
|
||||||
### 2.1 Architecture
|
|
||||||
|
|
||||||
API-first + minimal Playwright shell. The browser is used only for the
|
|
||||||
upload page (drop zone, Parse button, progress bar). Everything else
|
|
||||||
(verification, SFTP submit, scheduler control, ACK polling, claim-state
|
|
||||||
checks) hits the FastAPI directly via `httpx.AsyncClient`.
|
|
||||||
|
|
||||||
Rationale: the upload page is the only surface in cyclone with a real
|
|
||||||
interactive flow. Clearinghouse submission is API-only; the MFT
|
|
||||||
scheduler is API-only; ACK retrieval is API-only. A pure-Playwright
|
|
||||||
approach would be 3–5x more code and considerably flakier on the
|
|
||||||
API-driven phases, with no benefit.
|
|
||||||
|
|
||||||
### 2.2 Sibling package, not an extra
|
|
||||||
|
|
||||||
The agent lives at `/Users/openclaw/dev/cyclone-pipeline/` with its own
|
|
||||||
`pyproject.toml`, not as a new optional extra on the existing cyclone
|
|
||||||
package. Rationale: the agent has Playwright as a hard dependency; we
|
|
||||||
do not want Playwright in cyclone's dev env for developers who never
|
|
||||||
invoke the agent. Separate package = clean separation of concerns.
|
|
||||||
|
|
||||||
### 2.3 Real round-trip, but only the parts that are time-bounded
|
|
||||||
|
|
||||||
The agent waits for **TA1 (envelope ACK)** and **999 (file-level ACK)**.
|
|
||||||
It does **not** wait for 277CA (CO Medicaid doesn't use it) or 835
|
|
||||||
(payment cycle is next Monday). 835 detection is a separate, fast
|
|
||||||
follow-up command that the operator runs on Monday.
|
|
||||||
|
|
||||||
Rationale: blocking a run for up to 7 days is impractical and ties up
|
|
||||||
operator attention. The TA1 + 999 receipt is the unambiguous "your file
|
|
||||||
was accepted by the payer at the file level" signal; the 835 is
|
|
||||||
eventually-consistent and doesn't need a tight loop.
|
|
||||||
|
|
||||||
### 2.4 CO Medicaid / Gainwell specifics
|
|
||||||
|
|
||||||
- **SFTP server:** `mft.gainwelltechnologies.com:22`
|
|
||||||
- **Inbound path:** `/CO XIX/PROD/coxix_prod_11525703/FromHPE`
|
|
||||||
- **Outbound file naming:** `11525703-837P-{yyyymmddhhmmssSSS}-1of1.txt`
|
|
||||||
(17-digit Mountain Time timestamp; matches the `clearhouse` config's
|
|
||||||
`outbound` template in `config/payers.yaml`).
|
|
||||||
- **Inbound ACKs (in observed order):**
|
|
||||||
1. TA1 (seconds to minutes) — envelope-level
|
|
||||||
2. 999 (minutes) — file-level
|
|
||||||
3. HTML (sometimes) — non-X12 artifact, lands in inbound, not parsed
|
|
||||||
4. 835 (next Monday) — remit
|
|
||||||
5. 277CA — *never used by CO Medicaid*
|
|
||||||
|
|
||||||
### 2.5 Failure semantics
|
|
||||||
|
|
||||||
- **Upload retries:** 3× attempts; each attempt is a fresh page load +
|
|
||||||
file re-attach + click. Screenshots on every attempt.
|
|
||||||
- **SFTP submit retries:** 3× attempts with `1s → 2s → 4s` backoff. The
|
|
||||||
submit endpoint is POST and is *not* auto-retried after a network
|
|
||||||
timeout (would double-submit); the retry covers `409 Conflict` /
|
|
||||||
`503 Service Unavailable` only.
|
|
||||||
- **TA1 timeout (5 min):** if no TA1 arrives, the run is **soft-fail**.
|
|
||||||
The submission is "in flight" — the report says so and recommends
|
|
||||||
checking Gainwell directly.
|
|
||||||
- **999 timeout (15 min):** same — soft-fail. Most production runs
|
|
||||||
complete this phase in < 2 min.
|
|
||||||
- **999 received with status `R` (rejected):** hard-fail. The 999's
|
|
||||||
rejection segments surface in the report with full context.
|
|
||||||
- **HTML detected in inbound:** warn in the report; do not fail.
|
|
||||||
- **835 not present at end of run:** expected; reported as
|
|
||||||
"deferred — expected next Monday" with the `check-835` recipe.
|
|
||||||
- **Process crash (SIGKILL, power loss, etc.):** the next `resume` skips
|
|
||||||
completed phases by reading `run.state.json`. No work is lost except
|
|
||||||
the in-flight phase.
|
|
||||||
|
|
||||||
### 2.6 Idempotency
|
|
||||||
|
|
||||||
- The MFT outbound filename embeds a millisecond Mountain Time
|
|
||||||
timestamp, so back-to-back submissions to the same input file are
|
|
||||||
unique on the SFTP server. No `Idempotency-Key` header is needed.
|
|
||||||
- The agent captures the new `claim_id[]` set from phase 3 (verify
|
|
||||||
parse) and refuses to SFTP-submit if any of those claim IDs already
|
|
||||||
appear in a prior `processed_inbound_files` row within a configurable
|
|
||||||
dedup window (default 24h). `--force` overrides the dedup.
|
|
||||||
- The `check-835` command is fully idempotent: querying
|
|
||||||
`GET /api/remittances?since=<date>` is read-only.
|
|
||||||
|
|
||||||
### 2.7 Output shape
|
|
||||||
|
|
||||||
Every run writes a dated folder under `./runs/`. The folder is
|
|
||||||
self-contained — anyone reading it has the full picture without needing
|
|
||||||
the live cyclone instance.
|
|
||||||
|
|
||||||
```
|
|
||||||
runs/2026-06-21-1430-001/
|
|
||||||
├── run.log # structured JSON, structlog
|
|
||||||
├── run.state.json # for resume
|
|
||||||
├── report.md # human-readable summary
|
|
||||||
├── report.json # machine-readable, OpenClaw/Nora-friendly
|
|
||||||
└── screenshots/
|
|
||||||
├── 01-preflight.png
|
|
||||||
├── 02-upload-1.png # attempts are numbered
|
|
||||||
├── 02-upload-2.png
|
|
||||||
├── 03-parse-complete.png
|
|
||||||
├── 04-submit.png
|
|
||||||
├── 05-ta1.png
|
|
||||||
├── 06-999.png
|
|
||||||
├── 07-inbound-scan.png # HTML detection screenshot
|
|
||||||
└── 99-final-inbox.png # final state of the Inbox page
|
|
||||||
```
|
|
||||||
|
|
||||||
On failure, the `finally` block adds `screenshots/FAIL-{phase}.png` and
|
|
||||||
a `failure.md` with the traceback + phase + remediation hint.
|
|
||||||
|
|
||||||
## 3. Module map
|
|
||||||
|
|
||||||
| File | Role |
|
|
||||||
|---|---|
|
|
||||||
| `pyproject.toml` | Project metadata; declares httpx, playwright, click, pydantic, structlog; dev deps: pytest, pytest-asyncio, respx, pytest-playwright |
|
|
||||||
| `README.md` | Install, run, embed-in-agent examples |
|
|
||||||
| `src/cyclone_pipeline/__init__.py` | Public API re-exports |
|
|
||||||
| `src/cyclone_pipeline/__main__.py` | `python -m cyclone_pipeline run …` |
|
|
||||||
| `src/cyclone_pipeline/cli.py` | Click CLI: `run`, `run-batch`, `status`, `resume`, `check-835` |
|
|
||||||
| `src/cyclone_pipeline/pipeline.py` | `CyclonePipeline` orchestrator; 7-phase state machine |
|
|
||||||
| `src/cyclone_pipeline/api_client.py` | `httpx.AsyncClient` wrapper with typed methods |
|
|
||||||
| `src/cyclone_pipeline/browser.py` | Playwright `UploadPage` class for the upload UI |
|
|
||||||
| `src/cyclone_pipeline/waiters.py` | Reusable wait primitives (claim state, ACK arrival) |
|
|
||||||
| `src/cyclone_pipeline/state.py` | `RunState` dataclass + `run.state.json` writer/reader |
|
|
||||||
| `src/cyclone_pipeline/screenshots.py` | Dated folder + stable filename helper |
|
|
||||||
| `src/cyclone_pipeline/report.py` | `report.md` + `report.json` writer |
|
|
||||||
| `src/cyclone_pipeline/selectors.py` | Centralized Playwright selectors with fallbacks |
|
|
||||||
| `src/cyclone_pipeline/exceptions.py` | Typed errors: `UploadError`, `ParseError`, `SubmitError`, `AckTimeoutError`, `SchedulerNotRunningError`, `IdempotencyError` |
|
|
||||||
| `src/cyclone_pipeline/logging_setup.py` | `structlog` config matching cyclone's SP18 JSON style |
|
|
||||||
| `src/cyclone_pipeline/check_835.py` | The deferred 835 detector (pure API, no browser) |
|
|
||||||
| `tests/conftest.py` | Shared fixtures: fake API server, mock browser, fake clock |
|
|
||||||
| `tests/test_api_client.py` | Typed responses, retry behavior, timeouts (respx) |
|
|
||||||
| `tests/test_waiters.py` | Backoff, timeout, predicate resolution (fake clock) |
|
|
||||||
| `tests/test_browser.py` | safe_click, attach_file, progress wait (pytest-playwright) |
|
|
||||||
| `tests/test_pipeline.py` | End-to-end orchestration, state machine, resume (mocks) |
|
|
||||||
| `tests/test_report.py` | Markdown + JSON shape, screenshot paths (golden files) |
|
|
||||||
| `tests/test_cli.py` | `run`, `run-batch`, `resume`, `check-835` (Click's CliRunner) |
|
|
||||||
| `tests/test_check_835.py` | `check-835` happy path + "835 not yet present" path |
|
|
||||||
| `tests/fixtures/sample_837p.edi` | A small valid 837P file for tests (copied from cyclone's `tests/fixtures/minimal_837p.txt`) |
|
|
||||||
|
|
||||||
## 4. Pipeline phases
|
|
||||||
|
|
||||||
The state machine in `pipeline.py` advances through 7 phases. Each phase
|
|
||||||
has: `precondition`, `execute`, `postcondition`, `retry_policy`, and a
|
|
||||||
side-effect of writing `run.state.json` + a screenshot at the end of a
|
|
||||||
successful attempt.
|
|
||||||
|
|
||||||
### Phase 1 — Pre-flight (API)
|
|
||||||
|
|
||||||
- `GET /api/health` → expect `status == "ok"` and `db.ok == true`.
|
|
||||||
- `GET /api/admin/scheduler/status` → record scheduler state
|
|
||||||
(`running` / `stopped`). If `stopped`, the agent will call
|
|
||||||
`POST /api/admin/scheduler/tick` per-iteration in phase 5/6 instead
|
|
||||||
of relying on the polling loop.
|
|
||||||
- Verify the browser-side URL is reachable: `HEAD {browser_base}/`
|
|
||||||
via `httpx` (cheap, no rendering). If the frontend dev server isn't
|
|
||||||
up, fail-fast with `BrowserNotReachableError` rather than discovering
|
|
||||||
this in phase 2.
|
|
||||||
- Capture the API base URL + browser base URL for the report.
|
|
||||||
- Screenshot the Upload page in its empty state →
|
|
||||||
`screenshots/01-preflight.png`.
|
|
||||||
|
|
||||||
### Phase 2 — Upload (Playwright)
|
|
||||||
|
|
||||||
- `chromium.launch(headless=VISIBLE)` (configurable).
|
|
||||||
- `page.goto(f"{browser_base}/upload")`.
|
|
||||||
- `expect(drop_zone).to_be_visible(timeout=10s)`.
|
|
||||||
- `set_input_files("input[type=file]", file_path)` (the hidden input
|
|
||||||
behind the drop zone; `set_input_files` works on hidden inputs).
|
|
||||||
- `expect(file_name_display).to_have_text(file.name)`.
|
|
||||||
- `safe_click("button:has-text('Parse')")`.
|
|
||||||
- `wait_for_progress_complete()` — polls the progress bar
|
|
||||||
(`text_content` containing `"100%"`) or the success toast
|
|
||||||
(`text_content` containing `"Parsed N claims"`), whichever comes
|
|
||||||
first, with a 5 min ceiling.
|
|
||||||
- Screenshots: `screenshots/02-upload-{attempt}.png` per attempt.
|
|
||||||
|
|
||||||
### Phase 3 — Verify parse (API)
|
|
||||||
|
|
||||||
- The browser-side toast gives the parsed claim count; capture it.
|
|
||||||
- `GET /api/claims?since=<phase1_start_ts>` → filter to claims whose
|
|
||||||
`submission_date >= phase1_start_ts`; these are the new ones.
|
|
||||||
- Collect the set of new `claim_id` values.
|
|
||||||
- If 0 new claims after a 30s poll, fail with `ParseError`.
|
|
||||||
|
|
||||||
### Phase 4 — SFTP submit (API)
|
|
||||||
|
|
||||||
- Dedup check: query `GET /api/admin/scheduler/processed-files` for
|
|
||||||
the past 24h; if any outbound filename pattern matches a recent
|
|
||||||
submission, fail with `IdempotencyError` unless `--force`.
|
|
||||||
- `POST /api/clearhouse/submit` with `{payer_id: "CO_TXIX",
|
|
||||||
transaction_type: "837P"}`. The body is small; the server picks
|
|
||||||
the batch from the newly-parsed claims.
|
|
||||||
- Expect 200; if 5xx with a "keychain" hint, raise `SubmitError` with
|
|
||||||
the macOS Keychain setup recipe inline.
|
|
||||||
- Screenshot: `screenshots/04-submit.png` (the Inbox page right after
|
|
||||||
submit, showing the batch in the lane).
|
|
||||||
|
|
||||||
### Phase 5 — Wait for TA1 (API poll)
|
|
||||||
|
|
||||||
- `GET /api/ta1-acks?since=<phase4_ts>` in a loop, every 5s, with a
|
|
||||||
5 min ceiling.
|
|
||||||
- Match by `isa13` (the control number in the submitted interchange
|
|
||||||
envelope; surfaced in the TA1 record).
|
|
||||||
- If scheduler is `stopped`, call `POST /api/admin/scheduler/tick`
|
|
||||||
between polls to manually drive the MFT fetch.
|
|
||||||
- On hit: capture `ta1_id`, `accepted` (vs `rejected`), screenshot the
|
|
||||||
TA1 detail in the Acks page → `screenshots/05-ta1.png`.
|
|
||||||
- On timeout: soft-fail; record `ta1_status: "timeout"`.
|
|
||||||
|
|
||||||
### Phase 6 — Wait for 999 (API poll)
|
|
||||||
|
|
||||||
- `GET /api/acks?since=<phase4_ts>` in a loop, every 10s, with a
|
|
||||||
15 min ceiling.
|
|
||||||
- Match by `original_batch_id` (or, if not available, by
|
|
||||||
`functional_group_control_number` from the submitted file's GS06).
|
|
||||||
- On hit: capture `ack_id`, `status` (`A` = accepted, `R` = rejected,
|
|
||||||
`E` = rejected with errors), the rejection segment details if
|
|
||||||
rejected.
|
|
||||||
- On `A`: screenshot the Acks page → `screenshots/06-999.png`.
|
|
||||||
- On `R`: hard-fail; record `ack.status == "rejected"`, the rejection
|
|
||||||
segments, and the recommended remediation in the report.
|
|
||||||
- On timeout: soft-fail; record `ack_999_status: "timeout"`.
|
|
||||||
|
|
||||||
### Phase 7 — Scan for HTML + final report (API + local)
|
|
||||||
|
|
||||||
- `GET /api/admin/scheduler/processed-files?since=<phase4_ts>` — list
|
|
||||||
inbound files the MFT scheduler saw during the run. If any has
|
|
||||||
`file_type == "html"` or an unrecognized extension, log a WARN,
|
|
||||||
capture the filename, and add it to the report's "Detected artifacts"
|
|
||||||
section.
|
|
||||||
- Generate `report.md` and `report.json` (see §5).
|
|
||||||
- Screenshot the Inbox page → `screenshots/99-final-inbox.png`.
|
|
||||||
- Emit a one-line summary on stdout for shell pipelines:
|
|
||||||
`RESULT: PASS run=2026-06-21-1430-001 ta1=TA1-42 999=999-43 835=deferred`.
|
|
||||||
|
|
||||||
## 5. Output format
|
|
||||||
|
|
||||||
### 5.1 `report.md` (human)
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
# Cyclone pipeline run — 2026-06-21 14:30 PDT
|
|
||||||
|
|
||||||
| Field | Value |
|
|
||||||
|---------------|------------------------------------|
|
|
||||||
| Run id | 2026-06-21-1430-001 |
|
|
||||||
| Input | /path/to/axiscare-837p.txt (12.4 KB) |
|
|
||||||
| Result | PASS — TA1 + 999 received, 835 deferred |
|
|
||||||
| Duration | 2m 14s |
|
|
||||||
|
|
||||||
## Phase summary
|
|
||||||
|
|
||||||
| Phase | Status | Duration | Detail |
|
|
||||||
|------------------------|--------|----------|-----------------------------------------|
|
|
||||||
| 1. Pre-flight | OK | 0.4s | health.ok, scheduler running |
|
|
||||||
| 2. Upload | OK | 2.1s | 1 attempt, 17 claims parsed |
|
|
||||||
| 3. Verify parse | OK | 0.8s | claim_ids: CLM-001 … CLM-017 |
|
|
||||||
| 4. SFTP submit | OK | 4.3s | filename 11525703-837P-20260621143012345-1of1.txt |
|
|
||||||
| 5. Wait TA1 | OK | 12s | ack id: TA1-42, accepted |
|
|
||||||
| 6. Wait 999 | OK | 1m 14s | ack id: 999-43, status Accepted |
|
|
||||||
| 7. Scan HTML + report | WARN | 0.2s | 1 HTML file in inbound, skipped; report.md + report.json written |
|
|
||||||
|
|
||||||
## Inbound artifacts detected
|
|
||||||
|
|
||||||
- 1 HTML file (`gainwell_status_20260621.htm`) — not parsed, not a failure.
|
|
||||||
|
|
||||||
## Expected next steps
|
|
||||||
|
|
||||||
- **835 expected Monday** (2026-06-28; computed as next Monday in
|
|
||||||
Mountain Time at the moment of report generation; if today is
|
|
||||||
Monday before noon, expected same day, otherwise next Monday).
|
|
||||||
Re-run: `cyclone-pipeline check-835 2026-06-21-1430-001`
|
|
||||||
|
|
||||||
## Artifacts
|
|
||||||
|
|
||||||
- screenshots/01-preflight.png
|
|
||||||
- screenshots/02-upload-1.png
|
|
||||||
- screenshots/03-parse-complete.png
|
|
||||||
- screenshots/04-submit.png
|
|
||||||
- screenshots/05-ta1.png
|
|
||||||
- screenshots/06-999.png
|
|
||||||
- screenshots/99-final-inbox.png
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5.2 `report.json` (machine)
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"run_id": "2026-06-21-1430-001",
|
|
||||||
"input": {
|
|
||||||
"path": "/path/to/axiscare-837p.txt",
|
|
||||||
"size_bytes": 12700,
|
|
||||||
"sha256": "…"
|
|
||||||
},
|
|
||||||
"result": "pass",
|
|
||||||
"result_detail": "TA1 + 999 received, 835 deferred",
|
|
||||||
"started_at": "2026-06-21T14:30:00-06:00",
|
|
||||||
"finished_at": "2026-06-21T14:32:14-06:00",
|
|
||||||
"duration_s": 134,
|
|
||||||
"phases": [
|
|
||||||
{"n": 1, "name": "preflight", "status": "ok", "duration_s": 0.4},
|
|
||||||
{"n": 2, "name": "upload", "status": "ok", "duration_s": 2.1, "attempts": 1},
|
|
||||||
{"n": 3, "name": "verify_parse", "status": "ok", "duration_s": 0.8, "claim_ids": ["CLM-001", "…"]},
|
|
||||||
{"n": 4, "name": "submit", "status": "ok", "duration_s": 4.3, "filename": "11525703-837P-…"},
|
|
||||||
{"n": 5, "name": "wait_ta1", "status": "ok", "duration_s": 12, "ta1_id": "TA1-42", "accepted": true},
|
|
||||||
{"n": 6, "name": "wait_999", "status": "ok", "duration_s": 74, "ack_999_id": "999-43", "status": "A"},
|
|
||||||
{"n": 7, "name": "scan_and_report", "status": "warn", "duration_s": 0.2, "html_files": ["gainwell_status_20260621.htm"], "report_paths": {"md": "report.md", "json": "report.json"}}
|
|
||||||
],
|
|
||||||
"artifacts": {
|
|
||||||
"screenshots": ["01-preflight.png", "…", "99-final-inbox.png"],
|
|
||||||
"report_md": "report.md",
|
|
||||||
"report_json": "report.json",
|
|
||||||
"run_log": "run.log"
|
|
||||||
},
|
|
||||||
"next_steps": {
|
|
||||||
"check_835": "cyclone-pipeline check-835 2026-06-21-1430-001",
|
|
||||||
"expected_835_by": "2026-06-28"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
On soft-fail (TA1 or 999 timeout) or hard-fail (999 rejected), the
|
|
||||||
`result` is `"soft_fail"` or `"hard_fail"`, the offending phase gets
|
|
||||||
`status: "timeout"` / `"rejected"`, and a `failure` block is added
|
|
||||||
with the traceback, recommended remediation, and the `failure.md` path.
|
|
||||||
|
|
||||||
### 5.3 Exit codes
|
|
||||||
|
|
||||||
| Code | Meaning |
|
|
||||||
|------|---------|
|
|
||||||
| 0 | PASS — TA1 + 999 both received, both accepted |
|
|
||||||
| 2 | SOFT_FAIL — TA1 or 999 timed out; submission in flight, manual check needed |
|
|
||||||
| 3 | HARD_FAIL — 999 rejected, upload failed irrecoverably, or submit refused |
|
|
||||||
| 4 | USAGE — bad CLI args, missing file, backend unreachable after retries |
|
|
||||||
|
|
||||||
This mirrors common CI conventions so OpenClaw / Nora / shell scripts
|
|
||||||
can branch on `$?` without parsing JSON.
|
|
||||||
|
|
||||||
## 6. Library API (for OpenClaw / Nora)
|
|
||||||
|
|
||||||
```python
|
|
||||||
from cyclone_pipeline import CyclonePipeline, RunResult
|
|
||||||
|
|
||||||
async def main():
|
|
||||||
pipeline = CyclonePipeline(
|
|
||||||
api_base="http://127.0.0.1:8000",
|
|
||||||
browser_base="http://127.0.0.1:5173",
|
|
||||||
run_dir="./runs",
|
|
||||||
headless=False,
|
|
||||||
timeouts={"ta1_s": 300, "ack_999_s": 900},
|
|
||||||
)
|
|
||||||
|
|
||||||
result: RunResult = await pipeline.run(file_path="axiscare-837p.txt")
|
|
||||||
# result.outcome: "pass" | "soft_fail" | "hard_fail" | "usage"
|
|
||||||
# result.exit_code: int (0/2/3/4; mirrors CLI exit codes in §5.3)
|
|
||||||
# result.report: RunReport (Pydantic model; serializes to report.json)
|
|
||||||
# result.run_id: str
|
|
||||||
|
|
||||||
# Or batch:
|
|
||||||
results = await pipeline.run_batch(file_paths=[...], strict=False)
|
|
||||||
|
|
||||||
# Or resume:
|
|
||||||
result = await pipeline.resume(run_id="2026-06-21-1430-001")
|
|
||||||
|
|
||||||
# Or check 835 later:
|
|
||||||
ack = await pipeline.check_835(run_id="2026-06-21-1430-001")
|
|
||||||
# ack.found, ack.remittance_id, ack.total_paid, ...
|
|
||||||
```
|
|
||||||
|
|
||||||
`RunResult` is a Pydantic model so OpenClaw / Nora can serialize it
|
|
||||||
straight to JSON without any extra glue.
|
|
||||||
|
|
||||||
## 7. Tests
|
|
||||||
|
|
||||||
| Test file | Strategy |
|
|
||||||
|---|---|
|
|
||||||
| `tests/test_api_client.py` | `respx` mock; verify typed responses, retry-on-idempotent, no-retry-on-POST, timeout behavior |
|
|
||||||
| `tests/test_waiters.py` | Inject fake clock + fake state; verify backoff math, timeout, predicate resolution |
|
|
||||||
| `tests/test_browser.py` | `pytest-playwright` against a Vite dev server spun up in CI; covers `safe_click`, `set_input_files`, progress wait |
|
|
||||||
| `tests/test_pipeline.py` | Mock both `api_client` and `browser`; drive fake "happy path" through every phase + "phase 5 timeout" + "phase 3 parse error" + "phase 6 reject" |
|
|
||||||
| `tests/test_report.py` | Golden-file comparison for `report.md` and `report.json` |
|
|
||||||
| `tests/test_cli.py` | Click's `CliRunner`; covers all 5 subcommands + error paths |
|
|
||||||
| `tests/test_check_835.py` | Mock `GET /api/remittances?since=…`; happy path + "835 not yet present" path |
|
|
||||||
|
|
||||||
CI: GitHub Actions matrix on Python 3.11 + 3.12. Non-Playwright tests
|
|
||||||
run on every PR; Playwright tests gated to `main` push (the Vite
|
|
||||||
dev-server + chromium setup is too slow for every PR).
|
|
||||||
|
|
||||||
Coverage target: 90% line coverage on `pipeline.py`, 85% on the rest.
|
|
||||||
`browser.py` is exempt from the strict target (Playwright's own
|
|
||||||
assertion failures are the real test).
|
|
||||||
|
|
||||||
## 8. Files to add
|
|
||||||
|
|
||||||
- `pyproject.toml` — package metadata, deps, dev deps, scripts entry
|
|
||||||
- `README.md` — install, run, embed-in-agent examples
|
|
||||||
- 14 source files under `src/cyclone_pipeline/` listed in §3
|
|
||||||
- 8 test files under `tests/` listed in §3
|
|
||||||
- `tests/fixtures/sample_837p.edi` — small valid 837P for tests
|
|
||||||
(copied from cyclone's `tests/fixtures/minimal_837p.txt`)
|
|
||||||
|
|
||||||
Total: 24 files (1 `pyproject.toml`, 1 `README.md`, 14 src `.py`, 8
|
|
||||||
test `.py`, 1 fixture).
|
|
||||||
|
|
||||||
## 9. Files NOT touched
|
|
||||||
|
|
||||||
- No changes to the cyclone repo. The agent is a strict HTTP consumer
|
|
||||||
of cyclone's published API.
|
|
||||||
- No changes to cyclone's DB schema, API, CLI, or UI.
|
|
||||||
- No new dependencies added to cyclone's `pyproject.toml`.
|
|
||||||
|
|
||||||
## 10. Future work (explicit non-goals for v1)
|
|
||||||
|
|
||||||
- **Watch-folder daemon** — `cyclone-pipeline watch /inbox/*.txt` that
|
|
||||||
runs continuously and submits files as they appear. Achievable with
|
|
||||||
the current `run-batch` + a 1-line shell loop; the daemon is just UX.
|
|
||||||
- **Multi-payer support** — adding a new payer (e.g., a second
|
|
||||||
Medicaid MCO) means new payer-specific SFTP block, file naming
|
|
||||||
template, and ID-matching strategy. Worth doing once a second payer
|
|
||||||
is on the books.
|
|
||||||
- **Slack / email notification on completion** — `report.json` is the
|
|
||||||
integration point; a 50-LOC notifier could post a summary. v1 just
|
|
||||||
prints the one-line summary and writes the file.
|
|
||||||
- **Pushing reports to S3 / a share** — same as above; `report.json`
|
|
||||||
is the integration point.
|
|
||||||
- **Web UI for run history** — out of scope; OpenClaw / Nora can read
|
|
||||||
the `runs/` directory and surface the JSON.
|
|
||||||
- **277CA polling for payers that use it** — straightforward to add
|
|
||||||
behind a `--payer-config` flag once a second payer is configured.
|
|
||||||
277CA support in cyclone is already shipped; only the agent's wait
|
|
||||||
list changes.
|
|
||||||
|
|
||||||
## 11. Open questions (none for v1)
|
|
||||||
|
|
||||||
None. The design is locked pending user review of this spec.
|
|
||||||
@@ -1,78 +0,0 @@
|
|||||||
# SP19 — Security Hardening + Health Probe
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Branch:** `sp19-security-hardening`
|
|
||||||
**Status:** Shipped
|
|
||||||
**Scope:** Backend only. No frontend changes.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Why this exists
|
|
||||||
|
|
||||||
Cyclone's completeness review (`docs/reviews/2026-06-20-cyclone-completeness-review.md`
|
|
||||||
§3.1.4, §3.1.25, §3.2.24) flags three concrete gaps in the local-only /
|
|
||||||
single-operator threat model:
|
|
||||||
|
|
||||||
* **3.1.4 — No request size limits / no rate limits.** FastAPI defaults
|
|
||||||
accept any body size and any request rate. A 4 GB file upload OOMs
|
|
||||||
the parser; a flood of requests holds the GIL. Even for a
|
|
||||||
`127.0.0.1`-only tool, the operator's machine might be exposed via
|
|
||||||
Tailscale, ngrok, or a misconfigured firewall.
|
|
||||||
* **3.1.25 — No CSP / security headers.** CORS is set; the rest of
|
|
||||||
FastAPI's defaults are in play. `X-Content-Type-Options: nosniff`,
|
|
||||||
`X-Frame-Options: DENY`, a `Content-Security-Policy` that locks the
|
|
||||||
Vite origin, and `Referrer-Policy: same-origin` should be on by
|
|
||||||
default.
|
|
||||||
* **3.2.24 — `/api/health` is shallow.** It returns only the package
|
|
||||||
version. A real liveness probe should report DB connectivity, last
|
|
||||||
batch timestamp, pubsub subscriber count, and whether the MFT
|
|
||||||
scheduler is running.
|
|
||||||
|
|
||||||
SP19 closes all three.
|
|
||||||
|
|
||||||
## 2. Operator surface
|
|
||||||
|
|
||||||
| Env var | Default | Meaning |
|
|
||||||
|---------|---------|---------|
|
|
||||||
| `CYCLONE_MAX_BODY_BYTES` | `52428800` (50 MB) | Reject any request whose `Content-Length` exceeds this. |
|
|
||||||
| `CYCLONE_RATE_LIMIT_PER_MIN` | `300` | Per-IP requests/minute (sliding window). |
|
|
||||||
| `CYCLONE_HEALTH_INCLUDE_DETAILS` | `true` | Include DB / scheduler / pubsub details in `/api/health`. |
|
|
||||||
|
|
||||||
CLI: no new flags. The values come from env vars only.
|
|
||||||
|
|
||||||
## 3. Files
|
|
||||||
|
|
||||||
* `cyclone/security.py` — new module (~150 LOC).
|
|
||||||
* `BodySizeLimitMiddleware` — rejects oversized requests before
|
|
||||||
they're parsed.
|
|
||||||
* `RateLimitMiddleware` — token-bucket per IP, in-memory.
|
|
||||||
* `SecurityHeadersMiddleware` — adds the static response headers.
|
|
||||||
* `get_health_snapshot()` — gathers DB / scheduler / pubsub info.
|
|
||||||
* `cyclone.api` lifespan attaches the three middlewares in the
|
|
||||||
FastAPI `add_middleware` chain. `api_routers/health.py` is
|
|
||||||
rewritten to use `get_health_snapshot()`.
|
|
||||||
* Audit log: every rejection (size / rate / method) writes a
|
|
||||||
`api.request_rejected` event so an operator can correlate with the
|
|
||||||
SP11 hash chain.
|
|
||||||
|
|
||||||
## 4. Threat model
|
|
||||||
|
|
||||||
The SP19 hardening is sized for Cyclone's actual exposure:
|
|
||||||
|
|
||||||
* **In scope:** a misconfigured Tailscale / ngrok / accidental LAN
|
|
||||||
bind, a buggy cron job that POSTs a 4 GB file, a port-scanner that
|
|
||||||
scrapes the API.
|
|
||||||
* **Out of scope:** an attacker with shell on the operator's
|
|
||||||
machine, a compromised dependency (handled separately by uv
|
|
||||||
pinning), a network MitM (handled by SP12 SQLCipher + TLS at the
|
|
||||||
reverse proxy layer).
|
|
||||||
|
|
||||||
## 5. Tests
|
|
||||||
|
|
||||||
* `test_security.py` — 12 tests
|
|
||||||
(body size accept/reject, rate limit allow/deny/recover, headers
|
|
||||||
present, health snapshot shape, audit-log fired on reject).
|
|
||||||
* `test_api_health.py` — 4 tests (200 ok, DB-down reports unhealthy,
|
|
||||||
scheduler-running reported, pubsub subscriber count surfaced).
|
|
||||||
|
|
||||||
Total: 16 new tests.
|
|
||||||
@@ -1,127 +0,0 @@
|
|||||||
# Cyclone Skill Catalog — Design Spec
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Status:** Draft, pending user review
|
|
||||||
**Branch:** `main`
|
|
||||||
**Scope:** A catalog of 8 project-scoped Grok skills under `cyclone/.superpowers/skills/` that codify Cyclone's conventions layer-by-layer. No code changes; pure guidance artifacts.
|
|
||||||
|
|
||||||
## Context
|
|
||||||
|
|
||||||
Cyclone is now a substantial codebase — `api.py` (~111KB) and `store.py` (~95KB) are monoliths being actively split, ~30 EDI parsers, 21 pages / 30 components / 35 hooks on the frontend, 90+ pytest tests, 14 specs and 8 plans already on disk. There is enough surface area that an AI agent (or a new contributor) repeatedly re-derives the same conventions: where the live-tail wire format lives, which validator file owns which rule, how a new SP-N spec is named, where prodfiles fixtures go.
|
|
||||||
|
|
||||||
This spec introduces a catalog of 8 skills, one per major subsystem, that encode those conventions so they are loaded on demand instead of re-derived each time. Each skill is auto-discovered by description match (no slash command). The success criterion is **consistency across modules** — when adding a new EDI parser, a new streaming page, a new API router, etc., the skill hands the AI the established conventions for that subsystem so additions stay consistent with what's already there.
|
|
||||||
|
|
||||||
## Decisions (locked during brainstorming)
|
|
||||||
|
|
||||||
1. **Scope:** 8 skills, layer-mapped (one skill per subsystem). Not workflow-anchored, not reference-plus-convention hybrid. Layer-mapped was chosen because each subsystem has exactly one owner and the success criterion is consistency per layer.
|
|
||||||
2. **Location:** Project-scoped, under `cyclone/.superpowers/skills/`. Auto-discovered whenever anyone works in this repo. Not global.
|
|
||||||
3. **Categories covered:** Code-pattern / domain skills + Process / workflow skills. Testing-fixture and operational-CLI skills deferred (none selected).
|
|
||||||
4. **Success criterion:** Consistency across modules (selected over faster onboarding / process discipline / navigability).
|
|
||||||
5. **No hub skill.** The catalog itself lives in this spec and (later) in the README's skills section. The AI doesn't load a runtime index.
|
|
||||||
6. **No loading graph.** Each skill cross-references the others it pairs with, but no prescriptive load order is encoded. Auto-discovery fires per-description.
|
|
||||||
|
|
||||||
## The catalog
|
|
||||||
|
|
||||||
| # | Skill | Owns |
|
|
||||||
|---|-------|------|
|
|
||||||
| 1 | `cyclone-spec` | The SP-N superpowers flow as a one-shot: branch naming, `docs/superpowers/specs/…-design.md` + `…-plan.md` templates, PR title format. |
|
|
||||||
| 2 | `cyclone-tests` | pytest + vitest fixture patterns; `backend/tests/fixtures/` layout; `.test.tsx` sibling convention; prodfiles drop-in procedure. |
|
|
||||||
| 3 | `cyclone-edi` | Parser/validator conventions across 837P/835/999/270/271/277CA/TA1; segment-walk pattern; validator rule format (R200/R210, NPI Luhn, EIN, CAS); fixture discovery. |
|
|
||||||
| 4 | `cyclone-tail` | Live-tail wire format (`item` / `snapshot_end` / `heartbeat` / `item_dropped` / `error`), `useTailStream` + `useMergedTail` + per-resource hook triplet, `StatusPill` states. |
|
|
||||||
| 5 | `cyclone-store` | Write-path conventions in `store.py`; the pubsub event contract (`claim_written` / `remittance_written` / `activity_recorded`); snapshot shape; SP21 store-split boundary map. |
|
|
||||||
| 6 | `cyclone-api-router` | FastAPI router conventions (`api_routers/`), `api_helpers.py` reuse, response shapes, error envelopes. |
|
|
||||||
| 7 | `cyclone-frontend-page` | Page-component pattern (TanStack Query + tail + drawer + URL state), `use<X>` hook convention, `Layout` / `PageHeader` / `Sidebar` usage. |
|
|
||||||
| 8 | `cyclone-cli` | CLI subcommand conventions in `cli.py` (serve/parse/backup/rotate-key/validate), argparse style, exit codes, smoke-test patterns. |
|
|
||||||
|
|
||||||
## Per-skill structure
|
|
||||||
|
|
||||||
Every skill lives at `.superpowers/skills/<name>/SKILL.md` and follows this skeleton:
|
|
||||||
|
|
||||||
```
|
|
||||||
---
|
|
||||||
name: <kebab-case>
|
|
||||||
description: "<one-sentence trigger, with Use when: … keywords>"
|
|
||||||
---
|
|
||||||
|
|
||||||
# <Title>
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
2–5 bullets of concrete situations. ("adding a new EDI parser"
|
|
||||||
beats "working on EDI".)
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
Numbered, testable rules. Each rule is something a reviewer can
|
|
||||||
check.
|
|
||||||
|
|
||||||
## Patterns
|
|
||||||
Small copy-pasteable skeletons (parser stub, page hook, router
|
|
||||||
signature). Concrete code blocks, not prose.
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
Tempting-but-wrong moves. ("Don't write a new useX hook per page —
|
|
||||||
extend the existing one.")
|
|
||||||
|
|
||||||
## Related skills
|
|
||||||
Cross-references to the other 7 skills that often pair with this
|
|
||||||
one, one line each.
|
|
||||||
```
|
|
||||||
|
|
||||||
**References subdirs:** `<skill>/references/<file>.md` for long pattern catalogs (e.g. a per-parser mapping in `cyclone-edi`, a wire-format reference in `cyclone-tail`). Linked from SKILL.md. SKILL.md itself stays under ~200 lines; long content goes to references so the auto-load is fast.
|
|
||||||
|
|
||||||
**No code in skills.** Skills are guidance for the AI, not runnable artifacts. Optional `<skill>/scripts/` only for things that genuinely need automation (e.g. a fixture-discovery script in `cyclone-edi`). Most skills won't ship scripts.
|
|
||||||
|
|
||||||
**Skill size budget:** ~150–250 lines for SKILL.md, plus zero or a few references. Anything bigger is a sign the skill is trying to do two things.
|
|
||||||
|
|
||||||
## Loading & cross-references
|
|
||||||
|
|
||||||
**Trigger model.** Every skill is auto-discovered by description match. The `description` field is the only thing the loader sees before deciding to fire, so each gets one carefully tuned sentence. Example drafts:
|
|
||||||
|
|
||||||
- `cyclone-edi` — *"Cyclone EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1). Use when: adding or changing a parser, a validator rule, or a CAS code mapping."*
|
|
||||||
- `cyclone-tail` — *"Cyclone live-tail streaming wire format and the useTailStream / useMergedTail hook triplet. Use when: adding a new streaming list page, changing the wire format, or debugging stalled/reconnecting state."*
|
|
||||||
- `cyclone-store` — *"Cyclone store write-paths, the pubsub event contract, and the SP21 store-split boundary map. Use when: touching store.py, adding a new entity, or wiring a new write event."*
|
|
||||||
- `cyclone-api-router` — *"Cyclone FastAPI router conventions (api_routers/, api_helpers.py, response shapes, error envelopes). Use when: adding or changing an HTTP endpoint."*
|
|
||||||
- `cyclone-frontend-page` — *"Cyclone React page conventions (TanStack Query, use<X> hook, drawer, URL state, .test.tsx sibling). Use when: adding a new page or refactoring an existing one."*
|
|
||||||
- `cyclone-cli` — *"Cyclone CLI subcommand conventions (cli.py, serve/parse/backup/rotate-key/validate, exit codes, smoke tests). Use when: adding a CLI subcommand or working on operator-facing commands."*
|
|
||||||
- `cyclone-spec` — *"Cyclone SP-N superpowers increment flow — spec → plan → implement → merge. Use when: starting a new numbered feature increment or doing the merge dance."*
|
|
||||||
- `cyclone-tests` — *"Cyclone pytest + vitest fixture patterns, prodfiles layout, backend/tests/fixtures/ conventions. Use when: adding a test or wiring in a real-EDI sample."*
|
|
||||||
|
|
||||||
Descriptions are tuned so the right one fires and the others stay quiet. The "Use when:" half is the disambiguator.
|
|
||||||
|
|
||||||
**Cross-references** live in two places per skill:
|
|
||||||
1. A `## Related skills` section (per the template) — short list, one line each.
|
|
||||||
2. An explicit `If you are also touching X, also load skill Y` line at the top of the relevant section.
|
|
||||||
|
|
||||||
**Loading order isn't prescriptive.** The AI loads what's relevant; if two skills conflict, the more-specific one wins.
|
|
||||||
|
|
||||||
## Build order / phasing
|
|
||||||
|
|
||||||
8 skills shipped one-per-PR, grouped into 4 phases:
|
|
||||||
|
|
||||||
| Phase | Skill | Rationale |
|
|
||||||
|-------|-------|-----------|
|
|
||||||
| **1. Foundations** | `cyclone-spec` | Already implicit in the SP-N flow; codify first because every later feature goes through it. Smallest, fastest. |
|
|
||||||
| | `cyclone-tests` | Referenced by every other skill (fixture layout, vitest sibling pattern). Must land before any domain skill can ship its `## Anti-patterns` cleanly. |
|
|
||||||
| **2. Domain hot spots** | `cyclone-edi` | 30 parsers, scattered validators, fixture sprawl. Highest leverage per line of skill text. |
|
|
||||||
| | `cyclone-tail` | Wire format documented in README but consumed across 3 page-hook pairs; drift risk is concrete. |
|
|
||||||
| **3. Backend layers** | `cyclone-store` | Aligns with SP21 (CycloneStore split, currently in plan stage). |
|
|
||||||
| | `cyclone-api-router` | Pairs with store; `api_routers/` already exists. |
|
|
||||||
| **4. Frontend + CLI** | `cyclone-frontend-page` | Generalizes what `cyclone-tail` already covers. |
|
|
||||||
| | `cyclone-cli` | Standalone, lowest urgency. |
|
|
||||||
|
|
||||||
**Per-skill PR shape:** single commit adding one `.superpowers/skills/<name>/SKILL.md` (+ optional `references/`). No code changes. Reviewable in isolation. If a description turns out wrong, the fix is one line.
|
|
||||||
|
|
||||||
## Out of scope
|
|
||||||
|
|
||||||
- Testing & fixture skills (deferred per user selection).
|
|
||||||
- Operational / CLI safety-sequence skills beyond the basic `cyclone-cli` conventions skill (deferred per user selection).
|
|
||||||
- Skills outside `cyclone/.superpowers/skills/`. Global skills (e.g. `~/.grok/...`) are not touched.
|
|
||||||
- Any changes to existing code, tests, README, or specs.
|
|
||||||
- A hub/index skill.
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
After all 8 skills are landed:
|
|
||||||
1. Pick a sample task from each phase (e.g. "add a new EDI parser", "add a new streaming list page") and verify the matching skill fires by description match alone (no slash command).
|
|
||||||
2. Verify cross-references resolve — every "If you are also touching X, also load skill Y" line points to a skill that exists and whose description would actually fire for that scenario.
|
|
||||||
3. Verify each SKILL.md stays under the ~200-line budget (long content goes to `references/`).
|
|
||||||
4. Verify the catalog as a whole covers every subsystem with monolith risk (`api.py`, `store.py`, parsers/, hooks/) and the operator-facing surface (CLI).
|
|
||||||
@@ -1,264 +0,0 @@
|
|||||||
# CycloneStore split (Step 4) — Design Spec
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Status:** Draft, pending user review
|
|
||||||
**Branch:** `main` (split will land as a single atomic commit)
|
|
||||||
**Scope:** Behaviour-preserving structural split of `backend/src/cyclone/store.py` (2,412 lines) into a `cyclone/store/` subpackage. Zero public API changes, zero test changes.
|
|
||||||
|
|
||||||
## Context
|
|
||||||
|
|
||||||
`store.py` is the SQLAlchemy-backed facade over the parsed-X12 store. It sits on the hot path of `parse-999`, `parse-277ca`, reconciliation, and inbox match/unmatch. At 2,412 lines it has outgrown a single file: 41 methods on `CycloneStore`, ~17 module-level helpers, Pydantic models, ORM row builders, UI serializers, and exception types all live in one module.
|
|
||||||
|
|
||||||
This is "Step 4" of the ongoing refactor series. Steps 1–3 (the api.py splits — commits `931782b`, `fc73075`, `3b5e2af`, `eb674f8`, `ff43f90`, `6ce6385`, `e63be87`, `a63ba5e`) established the pattern: turn a monolithic file into a subpackage with a thin facade, preserving every public import.
|
|
||||||
|
|
||||||
## Decisions (locked during brainstorming)
|
|
||||||
|
|
||||||
1. **Public surface:** Subpackage + thin facade. `cyclone/store.py` is deleted; replaced by `cyclone/store/__init__.py` re-exporting every currently-importable name.
|
|
||||||
2. **Class structure:** Module functions for the bodies, `CycloneStore` keeps its current method signatures as 1-line delegations. (Mixin classes were considered and rejected for added MRO surface area; composition was rejected for breaking `isinstance` and adding 41 delegating boilerplate methods in a different shape.)
|
|
||||||
3. **Helper distribution:** Dedicated utility modules for non-method helpers — `records.py`, `orm_builders.py`, `ui.py`, `exceptions.py`. Co-location was rejected for circular-import risk.
|
|
||||||
|
|
||||||
## Target module layout
|
|
||||||
|
|
||||||
```
|
|
||||||
backend/src/cyclone/
|
|
||||||
├── store.py ← DELETED
|
|
||||||
└── store/
|
|
||||||
├── __init__.py ← thin facade: re-exports + CycloneStore + `store` singleton + utcnow
|
|
||||||
├── exceptions.py ← AlreadyMatchedError, NotMatchedError, InvalidStateError
|
|
||||||
├── records.py ← BatchKind, BatchRecord, BatchRecord837, BatchRecord835 (Pydantic)
|
|
||||||
├── orm_builders.py ← _service_dates_from_claim, _claim_837_row, _remittance_835_row,
|
|
||||||
│ _cas_adjustment_row, _persist_835_remit, _claim_status_from_validation
|
|
||||||
├── ui.py ← all to_ui_* / _iso_z / _address_to_ui / _validation_issues_to_ui /
|
|
||||||
│ _svc_to_wire_dict / _date_in_bounds / _provider_orm_to_dict /
|
|
||||||
│ _payer_orm_to_dict
|
|
||||||
├── write.py ← add, _publish_events_sync, _sync_publish, _run_reconcile
|
|
||||||
│ (biggest single module — ~210 lines)
|
|
||||||
├── batches.py ← get_batch, get, list, all, load_two_for_diff, _BatchesShim, _row_to_record
|
|
||||||
├── claim_detail.py ← get_remittance, get_claim_detail, iter_claims, iter_remittances,
|
|
||||||
│ distinct_providers, recent_activity
|
|
||||||
├── acks.py ← add_ack, list_acks, get_ack (999),
|
|
||||||
│ add_ta1_ack, list_ta1_acks, get_ta1_ack,
|
|
||||||
│ add_277ca_ack, list_277ca_acks, get_277ca_ack
|
|
||||||
├── backups.py ← add_backup_pending
|
|
||||||
├── inbox.py ← list_unmatched, manual_match, manual_unmatch
|
|
||||||
└── providers.py ← list_providers, get_provider, upsert_provider, list_payers,
|
|
||||||
get_payer_config, get_clearhouse, ensure_clearhouse_seeded
|
|
||||||
```
|
|
||||||
|
|
||||||
13 modules total. Largest is `write.py` at ~210 lines; all others are ≤150 lines. Facade `__init__.py` ≈ 80 lines (re-exports + class + singleton).
|
|
||||||
|
|
||||||
## CycloneStore class shape
|
|
||||||
|
|
||||||
### Three categories of methods
|
|
||||||
|
|
||||||
**Category A — Pure read delegates.** Open session, run query, return. No shared state, no private-method calls, no events.
|
|
||||||
|
|
||||||
Batches reads: `get_batch`, `get`, `list`, `all`, `load_two_for_diff`, `get_remittance`.
|
|
||||||
Claim detail reads: `get_claim_detail`, `iter_claims`, `iter_remittances`, `distinct_providers`, `recent_activity`.
|
|
||||||
ACK reads: `list_acks`, `get_ack`, `list_ta1_acks`, `get_ta1_ack`, `list_277ca_acks`, `get_277ca_ack`.
|
|
||||||
Provider/payer reads: `list_providers`, `get_provider`, `list_payers`, `get_payer_config`, `get_clearhouse`, `ensure_clearhouse_seeded`.
|
|
||||||
|
|
||||||
**Category B — Write-with-side-effects.** Module functions take an explicit `event_bus` kwarg where applicable; the `CycloneStore` methods delegate. The 3 private methods (`_publish_events_sync`, `_run_reconcile`, `_sync_publish`) stay on the class as 1-line delegations so any third-party calling them by private API doesn't break.
|
|
||||||
|
|
||||||
Methods: `add`, `add_ack`, `add_ta1_ack`, `add_277ca_ack`, `add_backup_pending`. Only `add` has a non-trivial body (insert + idempotency + reconcile + publish); the ACK add methods and `add_backup_pending` are simple single-row inserts that move to `acks.py` and `backups.py` respectively.
|
|
||||||
|
|
||||||
**Category C — Manual match/unmatch (3 methods).** Use `reconcile.apply_payment` / `apply_reversal`; translate `skipped=True` into `InvalidStateError`. Module functions in `inbox.py`; class methods delegate.
|
|
||||||
|
|
||||||
Methods: `list_unmatched`, `manual_match`, `manual_unmatch`.
|
|
||||||
|
|
||||||
### Iterator methods
|
|
||||||
|
|
||||||
`iter_claims`, `iter_remittances` are generators. Module functions in `claim_detail.py` use `with db.SessionLocal()() as s:` and `yield` inside; session lives for generator lifetime. Class methods delegate.
|
|
||||||
|
|
||||||
### Session management
|
|
||||||
|
|
||||||
Each module function inlines `with db.SessionLocal()() as s:` at the top — matches the existing pattern verbatim. No extracted `session_scope()` helper (YAGNI; ~30 call sites would change with no behaviour gain).
|
|
||||||
|
|
||||||
### Lock + shim stay on the class
|
|
||||||
|
|
||||||
`CycloneStore.__init__` keeps `self._lock = threading.RLock()` and `self._batches = _BatchesShim()` because 17 test files use `with store._lock: store._batches.clear()` as the cleanup idiom. The `_BatchesShim` class itself moves to `batches.py` alongside `_row_to_record`.
|
|
||||||
|
|
||||||
### Naming collision table
|
|
||||||
|
|
||||||
| Class method | Module function | Reason |
|
|
||||||
|---|---|---|
|
|
||||||
| `store.list` | `batches.list_batches` | shadows builtin |
|
|
||||||
| `store.all` | `batches.all_batches` | shadows builtin |
|
|
||||||
| `store.add` | `write.add_record` | more descriptive |
|
|
||||||
| `store.get` | `batches.get_record` | matches existing test usage |
|
|
||||||
| `store.utcnow` | top-level `utcnow()` in `__init__.py` | was always a free function |
|
|
||||||
|
|
||||||
### Singleton
|
|
||||||
|
|
||||||
`store = CycloneStore()` declared exactly once, in `__init__.py`. All sibling modules reference `cyclone.db` directly for session access, never the singleton.
|
|
||||||
|
|
||||||
## Public API preservation
|
|
||||||
|
|
||||||
### Source callers — zero changes
|
|
||||||
|
|
||||||
| Caller | Import | After split |
|
|
||||||
|---|---|---|
|
|
||||||
| `cyclone/api.py` | `from cyclone.store import (CycloneStore, store, BatchRecord, AlreadyMatchedError, ...)` | unchanged |
|
|
||||||
| `cyclone/api.py` (deferred) | `from cyclone.store import NotMatchedError` | unchanged |
|
|
||||||
| `cyclone/batch_diff.py` | `from cyclone.store import BatchRecord, BatchRecord835, BatchRecord837, _claim_status_from_validation` | facade re-exports `_claim_status_from_validation` |
|
|
||||||
| `cyclone/backup_service.py` (deferred) | `from cyclone.store import store as cycl_store` | unchanged |
|
|
||||||
| `cyclone/api_routers/acks.py` | `from cyclone.store import store` | unchanged |
|
|
||||||
| `cyclone/api_routers/ta1_acks.py` | `from cyclone.store import store` | unchanged |
|
|
||||||
| `cyclone/scheduler.py` | `from cyclone.store import store as cycl_store` (also `BatchRecord` deferred) | unchanged |
|
|
||||||
| `cyclone/parsers/validator.py` (3 deferred) | `from cyclone import store as store_mod` | unchanged |
|
|
||||||
| `cyclone/api_helpers.py` | docstring references `cyclone.store.utcnow` | unchanged (facade re-exports `utcnow`) |
|
|
||||||
|
|
||||||
**Open decision:** `_claim_status_from_validation` is currently a module-level helper imported by `batch_diff.py` (line 32). Recommended: re-export it from `__init__.py` to preserve the import. This is a small facade pollution but matches the precedent of the api.py split (which re-exports routers).
|
|
||||||
|
|
||||||
### Tests — zero changes
|
|
||||||
|
|
||||||
17 test files use `with store._lock: store._batches.clear()` — all continue to work because `CycloneStore.__init__` still sets those attributes and `_BatchesShim` continues to function identically. The 3 dedicated store tests (`test_store.py`, `test_store_claim_detail.py`, `test_store_reconcile.py`) import from `cyclone.store` (still works) or `cyclone` (still works).
|
|
||||||
|
|
||||||
## Migration plan
|
|
||||||
|
|
||||||
**Single atomic commit.** `cyclone/store.py` cannot coexist with `cyclone/store/` (Python prefers packages), so deletion and creation must happen together:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
rm backend/src/cyclone/store.py
|
|
||||||
mkdir -p backend/src/cyclone/store
|
|
||||||
# write all 13 new files
|
|
||||||
cd backend && python -m pytest tests/ --tb=line -q # must be green
|
|
||||||
git add -A
|
|
||||||
git commit -m "refactor(store): split store.py into cyclone/store/ subpackage"
|
|
||||||
```
|
|
||||||
|
|
||||||
**Follow-up commit (only if needed):** Docs update — add "CycloneStore subpackage" section to README following the precedent of commits `81aebf5`, `2718114`, `ea64e6e`, `804e557`.
|
|
||||||
|
|
||||||
**Rollback:** `git revert <sha>` — single-commit revert, no half-state.
|
|
||||||
|
|
||||||
## Test strategy
|
|
||||||
|
|
||||||
### No new tests, no test modifications
|
|
||||||
|
|
||||||
The split is structural only. Every test that passes today must pass after.
|
|
||||||
|
|
||||||
### Verification tiers
|
|
||||||
|
|
||||||
**Tier 1 — Full backend suite, baseline + after.**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd backend && python -m pytest tests/ --tb=line -q 2>&1 | tee /tmp/cyclone-baseline.txt | tail -5
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected baseline: ~712 passed, ~31 failed (pre-existing), ~16 skipped. The 31 failures are documented cross-test pollution (11 `test_serialize_837`, 3 `test_secrets`, 2 `test_sftp_paramiko`, etc., verified in commit `931782b`). Post-split count must match exactly.
|
|
||||||
|
|
||||||
**Tier 2 — Hot-path focused (14 files).** Exercises every method being split.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd backend && python -m pytest tests/test_parse_999.py tests/test_parse_277ca.py \
|
|
||||||
tests/test_api_999.py tests/test_api_277ca.py \
|
|
||||||
tests/test_reconcile.py tests/test_reconcile_line_level.py \
|
|
||||||
tests/test_store_reconcile.py tests/test_api_line_reconciliation.py \
|
|
||||||
tests/test_inbox_state.py tests/test_inbox_lanes.py \
|
|
||||||
tests/test_inbox_endpoints.py tests/test_inbox_endpoints_sp7.py \
|
|
||||||
tests/test_payer_rejected_acknowledge.py tests/test_store.py \
|
|
||||||
--tb=short -q
|
|
||||||
```
|
|
||||||
|
|
||||||
**Tier 3 — Store-specific (3 files).** Covers the class surface directly.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd backend && python -m pytest tests/test_store.py tests/test_store_claim_detail.py \
|
|
||||||
tests/test_store_reconcile.py --tb=short -q
|
|
||||||
```
|
|
||||||
|
|
||||||
**Tier 4 — Live smoke on 8 endpoints** (matches commit `931782b` precedent):
|
|
||||||
|
|
||||||
- `POST /api/parse-999` (valid → ack_code=A; rejected → ack_code=R)
|
|
||||||
- `POST /api/parse-277ca` (valid → 200 with 3 claim statuses)
|
|
||||||
- `GET /api/inbox?lane=unmatched` (list_unmatched shape)
|
|
||||||
- `POST /api/inbox/match` (manual_match happy path + AlreadyMatchedError)
|
|
||||||
- `POST /api/inbox/unmatch` (manual_unmatch happy path + NotMatchedError)
|
|
||||||
- `GET /api/batches` + `GET /api/batches/{id}` (get_batch shape)
|
|
||||||
- `GET /api/claims/{id}` (get_claim_detail shape)
|
|
||||||
- `GET /api/999-acks/{id}` and `GET /api/277ca-acks/{id}` (ack list/get shape)
|
|
||||||
|
|
||||||
Smoke output is documented in the commit message.
|
|
||||||
|
|
||||||
## Risk register
|
|
||||||
|
|
||||||
| # | Risk | Likelihood | Impact | Mitigation |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| R1 | Circular import between `__init__.py` and a sibling | Low | High | Siblings never import from `__init__.py`. Module docstrings state this rule. |
|
|
||||||
| R2 | Module function loses `self.` context | Low | Medium | `CycloneStore.__init__` keeps `_lock` and `_batches`. Module functions don't touch them. |
|
|
||||||
| R3 | `_BatchesShim.clear()` semantics change | Low | High | `_BatchesShim` moves verbatim; line-by-line verification during impl. |
|
|
||||||
| R4 | `add_record()` body diverges from current `add()` | Medium | High | Mechanical copy of lines 898-1107, strip `self.` prefix, change 3 private-method calls to module-function calls. No logic refactoring. |
|
|
||||||
| R5 | Singleton duplicated | Low | High | Declared exactly once, in `__init__.py`. |
|
|
||||||
| R6 | TYPE_CHECKING imports break runtime | Low | Low | `EventBus` already used in current `add()` signature; same pattern. |
|
|
||||||
| R7 | Test imports of private helpers break | Low | Medium | `grep -rn "from cyclone.store import _" backend/tests/` before impl; re-export anything found. |
|
|
||||||
| R8 | Package discovery misses new subpackage | Low | Medium | Verify with `pip install -e .` before commit; if needed, add explicit `packages = [...]`. |
|
|
||||||
| R9 | Generator methods leak sessions | Low | Medium | Preserve `with` inside generator body. |
|
|
||||||
| R10 | Docstring drift | Low | Low | Preserve existing docstrings verbatim on re-exports. |
|
|
||||||
|
|
||||||
## Implementation discipline
|
|
||||||
|
|
||||||
1. **Don't refactor logic.** The split is structural. Code smells stay for separate tasks.
|
|
||||||
2. **Don't add type hints.** Match existing annotations exactly.
|
|
||||||
3. **Don't add docstrings.** Preserve existing; only add a short module-level docstring per new file.
|
|
||||||
4. **Preserve order of operations.** Identical to current `add()` body.
|
|
||||||
5. **Don't optimize imports in passing.** Move them as-is.
|
|
||||||
|
|
||||||
## Pre-flight checklist
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Capture baseline
|
|
||||||
cd backend && python -m pytest tests/ --tb=line -q 2>&1 | tee /tmp/cyclone-baseline.txt | tail -5
|
|
||||||
|
|
||||||
# 2. Find every private helper imported from cyclone.store in tests
|
|
||||||
grep -rn "from cyclone.store import _" backend/tests/
|
|
||||||
|
|
||||||
# 3. Find every private helper imported from cyclone.store in source
|
|
||||||
grep -rn "from cyclone.store import _" backend/src/
|
|
||||||
|
|
||||||
# 4. Verify package discovery
|
|
||||||
cd backend && pip install -e . --quiet
|
|
||||||
python -c "import cyclone.store; print(cyclone.store.__file__)"
|
|
||||||
|
|
||||||
# 5. Verify no other cyclone/store* conflicts
|
|
||||||
find backend/src/cyclone -maxdepth 1 -name "store*"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Post-flight checklist
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Compare to baseline
|
|
||||||
cd backend && python -m pytest tests/ --tb=line -q 2>&1 | tail -5
|
|
||||||
# Must match baseline exactly
|
|
||||||
|
|
||||||
# 2. Hot-path focused (14 files) — must be 100% green
|
|
||||||
|
|
||||||
# 3. Import sanity check
|
|
||||||
python -c "
|
|
||||||
from cyclone.store import (
|
|
||||||
CycloneStore, store, BatchRecord, BatchRecord837, BatchRecord835, BatchKind,
|
|
||||||
AlreadyMatchedError, NotMatchedError, InvalidStateError, utcnow,
|
|
||||||
_claim_status_from_validation,
|
|
||||||
)
|
|
||||||
print('all imports OK')
|
|
||||||
"
|
|
||||||
|
|
||||||
# 4. Live smoke on 8 endpoints
|
|
||||||
```
|
|
||||||
|
|
||||||
## Out of scope (explicit)
|
|
||||||
|
|
||||||
- Renaming any public symbol
|
|
||||||
- Changing any method signature
|
|
||||||
- Refactoring `manual_match` / `manual_unmatch` business logic
|
|
||||||
- Refactoring `add()` body or its idempotency strategy
|
|
||||||
- Adding new tests
|
|
||||||
- Modifying DB models in `cyclone.db`
|
|
||||||
- Updating SQLCipher key rotation flow
|
|
||||||
- Touching any of the 17 test files that use `_lock` / `_batches.clear()`
|
|
||||||
- Touching any importer in `cyclone.api`, `cyclone.api_routers.*`, `cyclone.batch_diff`, `cyclone.backup_service`, `cyclone.scheduler`, `cyclone.parsers.validator`, `cyclone.api_helpers`
|
|
||||||
|
|
||||||
If any of these surface during implementation, they get deferred to follow-up tasks — never silently included.
|
|
||||||
|
|
||||||
## Open items
|
|
||||||
|
|
||||||
1. **`_claim_status_from_validation` re-export** — recommended re-export from `__init__.py`. User to confirm during spec review.
|
|
||||||
@@ -1,132 +0,0 @@
|
|||||||
# SP18 — Structured JSON Logging
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Branch:** `sp18-structured-logging`
|
|
||||||
**Status:** Shipped
|
|
||||||
**Scope:** Backend only. No frontend changes.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Why this exists
|
|
||||||
|
|
||||||
Cyclone has ~150 `logging.getLogger(__name__).*` call sites and
|
|
||||||
zero of them are structured. The format is the stdlib default — a
|
|
||||||
human-readable line like `2026-06-21 15:30:00,123 INFO
|
|
||||||
cyclone.scheduler: Processed inbound foo.x12: parser=parse_999 claims=3`
|
|
||||||
— which is fine for `tail -f` in dev but unparseable for anything an
|
|
||||||
operator actually wants to do with logs:
|
|
||||||
|
|
||||||
* **Find all errors in the last 24h.** `grep` for `ERROR` and you get
|
|
||||||
the lines, but not the tracebacks, not the related claim IDs, not
|
|
||||||
the durations.
|
|
||||||
* **Correlate scheduler ticks with API requests.** Impossible without
|
|
||||||
a request/correlation id in every line.
|
|
||||||
* **Detect PII leaks.** No scrubbing means a single accidental
|
|
||||||
`log.info("claim=%s", claim)` can dump PHI to stderr and the
|
|
||||||
README won't catch it.
|
|
||||||
|
|
||||||
The completeness review calls this out as gap #5 (`no structured
|
|
||||||
logging`).
|
|
||||||
|
|
||||||
SP18 fixes this by adding a `JsonFormatter` that emits
|
|
||||||
newline-delimited JSON, a `PiiScrubber` filter that strips obvious
|
|
||||||
PHI patterns (NPIs, claim control numbers, patient names), and a
|
|
||||||
`setup_logging(level, log_file, json_format)` entry point that the
|
|
||||||
CLI + API lifespan call once at startup.
|
|
||||||
|
|
||||||
## 2. Output format
|
|
||||||
|
|
||||||
Default (json_format=True):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{"ts": "2026-06-21T15:30:00.123Z", "level": "INFO", "logger": "cyclone.scheduler", "msg": "Processed inbound foo.x12", "extra": {"parser": "parse_999", "claims": 3}}
|
|
||||||
{"ts": "2026-06-21T15:30:00.456Z", "level": "ERROR", "logger": "cyclone.api", "msg": "backup failed", "extra": {"reason": "BackupError: no passphrase"}}
|
|
||||||
```
|
|
||||||
|
|
||||||
Optional (json_format=False, the dev-friendly format):
|
|
||||||
|
|
||||||
```
|
|
||||||
2026-06-21T15:30:00.123Z INFO cyclone.scheduler Processed inbound foo.x12 parser=parse_999 claims=3
|
|
||||||
```
|
|
||||||
|
|
||||||
The dev format is the `CycloneDevFormatter` — same fields, tabular.
|
|
||||||
Useful when `tail -f`-ing the API in dev.
|
|
||||||
|
|
||||||
## 3. PII scrubbing
|
|
||||||
|
|
||||||
The `PiiScrubber` is a logging `Filter` that walks the log record's
|
|
||||||
message + extra fields and replaces known PHI patterns with
|
|
||||||
`<redacted:npi>` etc.:
|
|
||||||
|
|
||||||
| Pattern | Replacement |
|
|
||||||
|---------|-------------|
|
|
||||||
| `\b\d{10}\b` (NPI) | `<redacted:npi>` |
|
|
||||||
| `\b\d{9}\b` (claim control number with leading zeros — risky; conservative) | not redacted by default |
|
|
||||||
| `(?i)patient[_ ]?name[:=]\s*\S+` | `<redacted:patient_name>` |
|
|
||||||
| `(?i)ssn[:=]\s*\d{3}-?\d{2}-?\d{4}` | `<redacted:ssn>` |
|
|
||||||
| `(?i)dob[:=]\s*\d{4}-\d{2}-\d{2}` | `<redacted:dob>` |
|
|
||||||
|
|
||||||
The default scrubber is conservative — we redact the unambiguous
|
|
||||||
patterns only. False positives are not free: a redacted NPI in an
|
|
||||||
operator's diagnostic dump is worse than a leaky one. The scrubber
|
|
||||||
can be disabled (`setup_logging(scrub_pii=False)`) for tests.
|
|
||||||
|
|
||||||
## 4. Files
|
|
||||||
|
|
||||||
* `cyclone.logging_config` — new module (~200 LOC).
|
|
||||||
* `JsonFormatter` — `logging.Formatter` subclass that JSON-encodes the record.
|
|
||||||
* `CycloneDevFormatter` — `logging.Formatter` subclass, tabular.
|
|
||||||
* `PiiScrubber` — `logging.Filter` subclass, regex rewriter.
|
|
||||||
* `setup_logging(level, log_file, json_format, scrub_pii)` — entry point.
|
|
||||||
* CLI: every `click.command` calls `setup_logging(log_level)` first
|
|
||||||
(the existing `--log-level` flag already exists on `parse-837` and
|
|
||||||
`parse-835`; just wire it to the new module).
|
|
||||||
* API: the FastAPI lifespan calls `setup_logging(level=os.environ.get("CYCLONE_LOG_LEVEL", "INFO"))` before any other setup.
|
|
||||||
* Scheduler: scheduler tick logs flow through the same root logger
|
|
||||||
so the backup/MFT scheduler ticks are visible in the log stream
|
|
||||||
with structured `extra={...}`.
|
|
||||||
|
|
||||||
## 5. Operator surface
|
|
||||||
|
|
||||||
| Env var | Default | Meaning |
|
|
||||||
|---------|---------|---------|
|
|
||||||
| `CYCLONE_LOG_LEVEL` | `INFO` | Root logger level. DEBUG for troubleshooting, WARNING to quiet. |
|
|
||||||
| `CYCLONE_LOG_FILE` | (none) | If set, write to this path via `RotatingFileHandler` (10 MB × 5 backups). |
|
|
||||||
| `CYCLONE_LOG_JSON` | `true` | If `false`, use the dev formatter. |
|
|
||||||
| `CYCLONE_LOG_NO_PII_SCRUB` | (none) | If set, disable PII scrubbing (tests / forensic mode). |
|
|
||||||
|
|
||||||
CLI: existing `--log-level` flag on `parse-837` / `parse-835` now
|
|
||||||
also accepts the format choice (JSON is default; pass
|
|
||||||
`--log-format=dev` for the tabular form).
|
|
||||||
|
|
||||||
## 6. Migration strategy
|
|
||||||
|
|
||||||
SP18 does **not** rewrite every log call. It:
|
|
||||||
|
|
||||||
1. Adds the formatter + filter to the root logger.
|
|
||||||
2. Migrates the ~25 highest-value log sites to use `extra={...}`
|
|
||||||
for structured fields (e.g. `log.info("processed inbound",
|
|
||||||
extra={"filename": f.name, "parser": "parse_999", "claims": 3})`
|
|
||||||
instead of f-string concatenation).
|
|
||||||
3. Keeps backward compatibility — `log.info("foo %s", x)` still works.
|
|
||||||
|
|
||||||
This is a deliberate scope cut. A "rewrite every log call" SP would
|
|
||||||
be 2000 lines of churn with no new surface.
|
|
||||||
|
|
||||||
## 7. Tests
|
|
||||||
|
|
||||||
* `test_logging_formatter.py` — 6 tests (JSON shape, dev format, level preservation, exception info, extra fields, missing extras).
|
|
||||||
* `test_logging_scrubber.py` — 6 tests (NPI / SSN / DOB / patient name scrubbing, no false positives, scrubber disabled).
|
|
||||||
* `test_logging_setup.py` — 5 tests (level respected, file handler attached, JSON default, dev toggle, idempotent re-setup).
|
|
||||||
|
|
||||||
Total: 17 new tests.
|
|
||||||
|
|
||||||
## 8. Out of scope
|
|
||||||
|
|
||||||
* A real log aggregator (Loki / ELK / Vector). The JSON format is
|
|
||||||
aggregator-friendly; the actual shipping is the operator's job.
|
|
||||||
* Per-logger log levels via config file. `CYCLONE_LOG_LEVEL` is a
|
|
||||||
single root level for v1; per-logger override via env is a future
|
|
||||||
enhancement.
|
|
||||||
* OpenTelemetry / Prometheus instrumentation. That's a different
|
|
||||||
SP (observability) for later.
|
|
||||||
@@ -1,362 +0,0 @@
|
|||||||
# Sub-project 21 — Universal Drill-Down: Design Spec
|
|
||||||
|
|
||||||
**Date:** 2026-06-21
|
|
||||||
**Status:** Draft (awaiting user review)
|
|
||||||
**Branch:** `universal-drilldown`
|
|
||||||
**Aesthetic direction:** Reuse the existing dashboard / drawer aesthetic (Radix Dialog + dim overlay, accent-tinted hover affordance). No new visual tokens.
|
|
||||||
|
|
||||||
## 1. Scope
|
|
||||||
|
|
||||||
Every interactive surface in the Cyclone UI becomes drillable. Clicking an entity reference (claim id, patient, provider, payer, batch, ack, activity event) opens a contextual view — either a full-record drawer (slides in from the right) or a slim peek modal (centered), depending on whether the entity is a first-class drillable record or a cross-reference.
|
|
||||||
|
|
||||||
The goal is **"dial down to anything"**: from the dashboard, from any list row, from inside any drawer, the operator can reach the underlying record with one click without losing their place.
|
|
||||||
|
|
||||||
**In scope:**
|
|
||||||
|
|
||||||
- New `PeekModal` primitive (centered modal) for cross-reference drills
|
|
||||||
- Reuse / refactor of the existing right-side `DrillDrawer` pattern for full-entity records
|
|
||||||
- New `DrillStackProvider` context to manage a max-2-level stack (one drawer + one peek)
|
|
||||||
- Hover-reveal affordance (pointer + accent tint + trailing chevron) on every clickable cell
|
|
||||||
- Per-surface drill map (every click target → its destination)
|
|
||||||
- 1 new backend read endpoint (`/api/payers/{payer_id}/summary`) plus an extended response shape on the existing `/api/providers/{npi}` (adds `recent_claims[]` and `recent_activity[]` arrays). The rest of the work is frontend wiring.
|
|
||||||
|
|
||||||
**Out of scope (deferred):**
|
|
||||||
|
|
||||||
- **Patient peek modal** — patient names are PHI; a peek would surface sensitive fields. Patient-name clicks instead navigate to `/claims?patient=…` filtered list (no new endpoint needed; uses existing `/api/claims?patient_name=…`).
|
|
||||||
- **Write actions from peeks** — peeks are read-only. Edit / resubmit / acknowledge actions remain on the full drawer / inbox lane.
|
|
||||||
- **Mobile bottom-sheet variant** — desktop-only for v1. Mobile fallback: peeks become centered modals at full width with the same max-height cap.
|
|
||||||
- **Customizing peek fields** — peek content is fixed per surface. The existing drawer already covers "I want everything."
|
|
||||||
- **KPI breakdown peeks** — dashboard KPI tiles navigate to filtered `/claims` (the user's picked data-strategy was "fetch fresh on click"; navigation keeps the dashboard fast).
|
|
||||||
- **Auto-suggest next drill** — no "you might also want to see…" hints.
|
|
||||||
|
|
||||||
## 2. Locked decisions
|
|
||||||
|
|
||||||
### 2.1 Component decomposition
|
|
||||||
|
|
||||||
| Component | Role |
|
|
||||||
|---|---|
|
|
||||||
| `DrillDrawer.tsx` | Right-side slide-over for full-record drills (claim, remit, batch, provider, ack). Refactor of today's `ClaimDrawer` pattern into a shared shell that all entity drawers mount inside. |
|
|
||||||
| `DrillDrawerHeader.tsx` | Drawer header: eyebrow, title, close button, optional primary action (e.g. "Download 837" on the ClaimDrawer). |
|
|
||||||
| `PeekModal.tsx` | Centered Radix Dialog for cross-reference drills (payer, activity event, validation issue, single-row tables like Inbox rows, Batch diff rows). Smaller (≤ 480px), no keyboard j/k nav (single record), closes on Esc + click-outside. |
|
|
||||||
| `PeekModalHeader.tsx` | Eyebrow + title + close button. |
|
|
||||||
| `DrillStackProvider.tsx` | React context + zustand-backed store. Owns the stack of `[{ kind: 'drawer' | 'peek', key: string, payload?: unknown }]`. Exposes `open()`, `closeTop()`, `closeAll()`. Enforces max 2 levels. |
|
|
||||||
| `DrillableCell.tsx` | Tiny wrapper that adds the hover-reveal affordance (pointer + accent tint + trailing chevron) and the click handler. Used inside tables, KPI tiles, activity feeds. |
|
|
||||||
| `ProviderDrawer.tsx` | New full-entity drawer for `/api/providers/{npi}`. Tabs: Overview (NPI + address + counts) / Claims (top 10 recent claims + "View all →" link) / Activity (top 10 recent activity + "View all →" link). |
|
|
||||||
| `AckDrawer.tsx` | New full-entity drawer for `/api/acks/{id}` (reuses the existing `GET /api/acks/{id}` endpoint). |
|
|
||||||
| `RemitDrawer.tsx` | New full-entity drawer for `/api/remittances/{id}`. Already documented as "deferred to a follow-up" in the SP4 claim-drawer spec; SP21 ships it. |
|
|
||||||
| `useDrillStack.ts` | Hook wrapper around `DrillStackProvider`'s store, with selector helpers for "is this drawer open?", "open this drawer", "the active peek payload". |
|
|
||||||
| `PayerPeekContent.tsx` | Payer peek body (backend-fed): name, claim count, billed, received, denial rate, "view all claims →" link. |
|
|
||||||
| `ValidationRulePeekContent.tsx` | Validation-rule peek body (static lookup, no backend call): rule code, severity, full description, X12 spec reference if any. Opened from ClaimDrawer validation issue rows. |
|
|
||||||
|
|
||||||
All live under `src/components/drill/` (new shared folder) plus `src/components/ProviderDrawer/`, `src/components/AckDrawer/`, `src/components/RemitDrawer/`.
|
|
||||||
|
|
||||||
### 2.2 Drill map (per surface)
|
|
||||||
|
|
||||||
The complete click → destination table. Each row is one click target on one surface. "Stack position" describes where the destination lives in the drill stack at the moment of the click.
|
|
||||||
|
|
||||||
| Surface | Click target | Destination | Stack position |
|
|
||||||
|---|---|---|---|
|
|
||||||
| Dashboard · KPI: Claims | tile | navigate → `/claims` | — |
|
|
||||||
| Dashboard · KPI: Billed | tile | navigate → `/claims?sort=-billedAmount` | — |
|
|
||||||
| Dashboard · KPI: Received | tile | navigate → `/claims?sort=-receivedAmount` | — |
|
|
||||||
| Dashboard · KPI: Pending AR | tile | navigate → `/claims?status=submitted,pending` | — |
|
|
||||||
| Dashboard · KPI: Denial rate | tile | navigate → `/claims?status=denied` | — |
|
|
||||||
| Dashboard · Top providers | any row | `ProviderDrawer(?provider=NPI)` | top drawer |
|
|
||||||
| Dashboard · Recent denials | any row | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Dashboard · Recent activity | any event | routed by event kind: `claim_*` → claim drawer, `remit_received` → remit drawer, `provider_added` → provider drawer | top drawer |
|
|
||||||
| Claims · table row | row body | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Claims · claim id cell | id | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Claims · patient cell | name | navigate → `/claims?patient=NAME` | — |
|
|
||||||
| Claims · provider cell | provider name + NPI | `ProviderDrawer(?provider=NPI)` | top drawer |
|
|
||||||
| Claims · payer cell | payer name | `PeekModal(payer)` | top peek |
|
|
||||||
| Remittances · table row | row body | `RemitDrawer(?remit=ID)` (replaces the inline CAS expand) | top drawer |
|
|
||||||
| Remittances · claim id cell | id | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Remittances · payer cell | payer name | `PeekModal(payer)` | top peek |
|
|
||||||
| Batches · table row | row body | `BatchDetail` drawer (?batch=ID) | top drawer |
|
|
||||||
| Providers · directory card | card body | `ProviderDrawer(?provider=NPI)` | top drawer |
|
|
||||||
| Activity log · event row | row body | routed by event kind (same as Dashboard · Recent activity) | top drawer |
|
|
||||||
| 999 ACKs · table row | row body | `AckDrawer(?ack=ID)` | top drawer |
|
|
||||||
| 999 ACKs · Download 999 button | button | (unchanged — file download) | — |
|
|
||||||
| Inbox · Rejected row | row body | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Inbox · Payer-Rejected row | row body | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Inbox · Candidates row | row body | `RemitDrawer(?remit=ID)` | top drawer |
|
|
||||||
| Inbox · Unmatched claim row | row body | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Inbox · Unmatched remit row | row body | `RemitDrawer(?remit=ID)` | top drawer |
|
|
||||||
| Inbox · Done today row | row body | routed by row kind (claim vs remit) | top drawer |
|
|
||||||
| Reconciliation · claim button | button | (unchanged — select for match) | — |
|
|
||||||
| Reconciliation · claim card body | body | `DrillDrawer(claim, ?claim=ID)` | top drawer (navigates to `/claims?claim=ID`) |
|
|
||||||
| Reconciliation · remit card body | body | `RemitDrawer(?remit=ID)` | top drawer (navigates to `/remittances?remit=ID`) |
|
|
||||||
| Batch diff · claim id (added) | id | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Batch diff · claim id (removed) | id | `DrillDrawer(claim, ?claim=ID)` (will 404 if removed — show distinct not-found state) | top drawer |
|
|
||||||
| Batch diff · claim id (changed) | id | `DrillDrawer(claim, ?claim=ID)` | top drawer |
|
|
||||||
| Upload · streamed claim card | header / chevron | (unchanged — inline expand) | — |
|
|
||||||
| Upload · streamed claim card | "See claim in detail →" link (new) | `DrillDrawer(claim, ?claim=ID)` (only enabled if the claim id is in the persisted batch list, else disabled with tooltip) | top drawer |
|
|
||||||
| ClaimDrawer · payer name (inside drawer) | payer name | `PeekModal(payer)` | peek on top of drawer |
|
|
||||||
| ClaimDrawer · provider name (inside drawer) | provider name | `ProviderDrawer(?provider=NPI)` (replaces drawer) | top drawer (replaces) |
|
|
||||||
| ClaimDrawer · matched-remit id | id | `RemitDrawer(?remit=ID)` (replaces drawer) | top drawer (replaces) |
|
|
||||||
| ClaimDrawer · validation issue row | rule code | `PeekModal(validation issue)` | peek on top of drawer |
|
|
||||||
|
|
||||||
**Nesting rules** (enforced by `DrillStackProvider`):
|
|
||||||
|
|
||||||
1. Max 2 levels: one drawer at the bottom + at most one peek on top. Opening a third drill replaces the top peek (same kind) or pops the peek (different kind).
|
|
||||||
2. Clicking inside a peek to open a drawer: closes the peek first, opens the drawer (no drawer-over-peek).
|
|
||||||
3. Esc closes the top entry (peek or drawer). Browser back closes the top drawer (URL pops). Browser back while only peeks are open: no-op (peek stack isn't URL-persisted).
|
|
||||||
|
|
||||||
### 2.3 Backend endpoints
|
|
||||||
|
|
||||||
**New endpoint:**
|
|
||||||
|
|
||||||
| Method | Path | Purpose |
|
|
||||||
|---|---|---|
|
|
||||||
| `GET` | `/api/payers/{payer_id}/summary` | Aggregate snapshot for a payer: `{ payer_id, name, claim_count, billed_total, received_total, denial_rate, top_providers: [{npi, count}] }`. Computed on read from the existing `claims` + `remittances` tables. Cached for 60s; invalidated on `claim_written` / `remittance_written` pubsub events. |
|
|
||||||
|
|
||||||
**Extended endpoint (existing, response shape widened):**
|
|
||||||
|
|
||||||
| Method | Path | Change |
|
|
||||||
|---|---|---|
|
|
||||||
| `GET` | `/api/providers/{npi}` | Response gains `recent_claims: ClaimSummary[]` (top 10 by `submission_date DESC`) and `recent_activity: ActivityEvent[]` (top 10 by `ts DESC`). Backwards-compatible: existing clients ignore the new arrays. |
|
|
||||||
|
|
||||||
**Reused as-is:**
|
|
||||||
|
|
||||||
- `GET /api/claims/{id}` — already exists (SP4). Used by `DrillDrawer(claim)`.
|
|
||||||
- `GET /api/remittances/{id}` — already exists (SP3). Used by `RemitDrawer`.
|
|
||||||
- `GET /api/batches/{id}` — already exists. Used by `BatchDetail`.
|
|
||||||
- `GET /api/providers/{npi}` — already exists (SP9). Used by `ProviderDrawer` Overview tab AND Claims / Activity tabs (via the newly extended arrays).
|
|
||||||
- `GET /api/config/payers/{payer_id}` — already exists (SP9). Used by the payer peek header (the payer name).
|
|
||||||
- `GET /api/acks/{id}` — already exists (SP3). Used by `AckDrawer`.
|
|
||||||
|
|
||||||
**Implementation notes:**
|
|
||||||
|
|
||||||
- Add the one new handler in `backend/src/cyclone/api_routers/payers.py`.
|
|
||||||
- Payer summary cache lives in a module-level `lru_cache` with a TTL guard (60s); invalidated on a `claim_written` / `remittance_written` pubsub event.
|
|
||||||
- The new endpoint inherits the existing security middleware stack (SP19): `127.0.0.1` bind, body limit, rate limit, security headers.
|
|
||||||
- The extended `/api/providers/{npi}` response is bounded — only 10 + 10 entries per category — so the existing payload cap (per SP19 body limit, default 50 MB) is comfortably respected.
|
|
||||||
|
|
||||||
### 2.4 Visual design tokens
|
|
||||||
|
|
||||||
No new tokens. Reuse:
|
|
||||||
|
|
||||||
- `--surface` / `--surface-dim` for drawer + dim overlay
|
|
||||||
- `--accent` for hover tint (the same `--accent` used by today's selected-row indicator + nav-active)
|
|
||||||
- `border-border/60` for the peek modal border
|
|
||||||
- Existing `display mono` / `eyebrow` typography for peek headers
|
|
||||||
- `animate-fade-in` / `animate-fade-in-up` for peek/drawer enter
|
|
||||||
|
|
||||||
Peek modal sizing:
|
|
||||||
|
|
||||||
```
|
|
||||||
max-width: 480px (or 90vw on narrow viewports)
|
|
||||||
padding: 20px
|
|
||||||
border: 1px solid var(--border)
|
|
||||||
shadow: shadow-2xl
|
|
||||||
border-radius: rounded-xl (matches today's Radix Dialog)
|
|
||||||
```
|
|
||||||
|
|
||||||
Hover affordance CSS:
|
|
||||||
|
|
||||||
```css
|
|
||||||
.drillable {
|
|
||||||
cursor: pointer;
|
|
||||||
transition: background-color 120ms ease;
|
|
||||||
}
|
|
||||||
.drillable:hover {
|
|
||||||
background-color: hsl(var(--accent) / 0.08);
|
|
||||||
}
|
|
||||||
.drillable:hover::after {
|
|
||||||
content: "›";
|
|
||||||
margin-left: 6px;
|
|
||||||
color: hsl(var(--accent));
|
|
||||||
font-weight: 600;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The trailing chevron is injected via `::after` so `DrillableCell` doesn't need to compose a child element. Disabled cells (e.g. the "See claim in detail" link in Upload when the claim isn't persisted) drop the affordance and the click handler.
|
|
||||||
|
|
||||||
### 2.5 Interaction patterns
|
|
||||||
|
|
||||||
**Open (drawer):**
|
|
||||||
|
|
||||||
- Click target → URL updates to `?{entity}={id}` via `history.pushState` (or `replaceState` if same entity type, to avoid history pollution).
|
|
||||||
- Drawer slides in from right, 240ms ease-out (same as today's ClaimDrawer).
|
|
||||||
- Background page dims to 60% opacity.
|
|
||||||
|
|
||||||
**Open (peek):**
|
|
||||||
|
|
||||||
- Click target → zustand stack updated; peek fades in centered, 180ms.
|
|
||||||
- Background dims to 75% (more aggressive than the drawer's 60%, to differentiate).
|
|
||||||
- No URL change (peek stack is ephemeral).
|
|
||||||
|
|
||||||
**Open (peek on top of drawer):**
|
|
||||||
|
|
||||||
- Click target inside drawer body → zustand stack gains one entry.
|
|
||||||
- Peek renders above drawer; drawer stays in place, dims to 75%.
|
|
||||||
- Drawer's existing keyboard nav (j/k, ?) is suspended while a peek is on top.
|
|
||||||
|
|
||||||
**Close:**
|
|
||||||
|
|
||||||
- Esc → pops the top stack entry. If peek was top, peek closes (drawer underneath is interactive again). If only a drawer, URL strips `?{entity}=`.
|
|
||||||
- Click on dim overlay → same as Esc.
|
|
||||||
- Browser back → pops URL drawer only (no peek history).
|
|
||||||
|
|
||||||
**Loading:**
|
|
||||||
|
|
||||||
- Drawer: existing `Skeleton` + `ErrorState` patterns reused.
|
|
||||||
- Peek: a 6-line shimmer skeleton, max 280px height. Fetch errors collapse the peek and show an inline error toast (peek is small enough that an inline state would look broken).
|
|
||||||
|
|
||||||
**Errors:**
|
|
||||||
|
|
||||||
- 404 on a drawer → existing `ErrorState` "not found" branch (per drawer component).
|
|
||||||
- 404 on a peek → toast: "Couldn't find that record. It may have been removed." and peek auto-closes after 1.5s.
|
|
||||||
- Network error → toast with Retry button.
|
|
||||||
|
|
||||||
**Keyboard:**
|
|
||||||
|
|
||||||
- Drawer nav: j/k, ?, esc (unchanged from today).
|
|
||||||
- Peek nav: esc, Tab cycles focusable elements inside the peek body. No j/k (single record, no list).
|
|
||||||
- Global `?` cheatsheet: extended to list peek keyboard hints (currently lists only drawer hints).
|
|
||||||
|
|
||||||
### 2.6 URL state contract
|
|
||||||
|
|
||||||
- `/claims` — list, no drawer.
|
|
||||||
- `/claims?claim=CLM-114` — list with drawer for CLM-114.
|
|
||||||
- `/remittances?remit=R-9876` — list with remit drawer.
|
|
||||||
- `/providers?provider=1881068062` — directory with provider drawer.
|
|
||||||
- `/batches?batch=BATCH-…` — list with batch drawer.
|
|
||||||
- `/acks?ack=42` — list with ack drawer.
|
|
||||||
- `/activity?event=evt-uuid` — list with activity event drawer.
|
|
||||||
|
|
||||||
Each entity-drawer type has its own URL state hook following the existing per-page pattern (`useDrawerUrlState` for Claims, `useBatchDrawerUrlState` for Batches, plus a new `useProviderDrawerUrlState` for Providers, `useRemitDrawerUrlState` for Remittances, `useAckDrawerUrlState` for Acks). All hooks read & write the same `?{entity}={id}` convention. **Each drawer is mounted by the page that owns it** — clicking a cross-page entity from inside another page navigates to that entity's owning page with `?{entity}={id}` set (soft fade, no full reload). The opening source's filter state (e.g. `/remittances?status=posted`) is preserved when navigating back via browser-back. Peeks never trigger navigation.
|
|
||||||
|
|
||||||
### 2.7 Frontend file layout
|
|
||||||
|
|
||||||
```
|
|
||||||
src/
|
|
||||||
├── components/
|
|
||||||
│ ├── drill/ # NEW shared folder
|
|
||||||
│ │ ├── DrillDrawer.tsx
|
|
||||||
│ │ ├── DrillDrawerHeader.tsx
|
|
||||||
│ │ ├── PeekModal.tsx
|
|
||||||
│ │ ├── PeekModalHeader.tsx
|
|
||||||
│ │ ├── DrillStackProvider.tsx
|
|
||||||
│ │ ├── DrillStackMount.tsx # the actual portal mount + z-index mgmt
|
|
||||||
│ │ ├── DrillableCell.tsx
|
|
||||||
│ │ ├── PayerPeekContent.tsx
|
|
||||||
│ │ ├── ValidationRulePeekContent.tsx
|
|
||||||
│ │ ├── DrillStackProvider.test.tsx
|
|
||||||
│ │ ├── DrillableCell.test.tsx
|
|
||||||
│ │ ├── PeekModal.test.tsx
|
|
||||||
│ │ └── index.ts # barrel export
|
|
||||||
│ ├── ProviderDrawer/ # NEW
|
|
||||||
│ │ ├── ProviderDrawer.tsx
|
|
||||||
│ │ ├── ProviderOverview.tsx
|
|
||||||
│ │ ├── ProviderRecentClaims.tsx # top-10 mini-table (data from extended /providers/{npi})
|
|
||||||
│ │ ├── ProviderRecentActivity.tsx # top-10 mini-feed (data from extended /providers/{npi})
|
|
||||||
│ │ └── index.ts
|
|
||||||
│ ├── RemitDrawer/ # NEW
|
|
||||||
│ │ ├── RemitDrawer.tsx
|
|
||||||
│ │ ├── RemitHeader.tsx
|
|
||||||
│ │ ├── ClaimPaymentsTable.tsx
|
|
||||||
│ │ ├── CasAdjustmentsPanel.tsx
|
|
||||||
│ │ ├── RemitPartiesGrid.tsx
|
|
||||||
│ │ └── index.ts
|
|
||||||
│ ├── AckDrawer/ # NEW
|
|
||||||
│ │ ├── AckDrawer.tsx
|
|
||||||
│ │ ├── AckHeader.tsx
|
|
||||||
│ │ ├── SegmentStatusList.tsx
|
|
||||||
│ │ └── index.ts
|
|
||||||
│ └── ClaimDrawer/ # existing — refactor header to use DrillDrawerHeader
|
|
||||||
├── hooks/
|
|
||||||
│ ├── useDrillStack.ts # NEW (zustand-backed ephemeral peek stack)
|
|
||||||
│ ├── usePayerSummary.ts # NEW
|
|
||||||
│ ├── useProviderDrawerUrlState.ts # NEW (?provider=)
|
|
||||||
│ ├── useRemitDrawerUrlState.ts # NEW (?remit=)
|
|
||||||
│ └── useAckDrawerUrlState.ts # NEW (?ack=)
|
|
||||||
├── lib/
|
|
||||||
│ └── api.ts # +api.getPayerSummary(id); /api/providers/{npi} response widened in backend
|
|
||||||
└── pages/ # wire drills per the drill map; refactor existing useDrawerUrlState usages to follow the per-drawer convention
|
|
||||||
├── Claims.tsx
|
|
||||||
├── Remittances.tsx
|
|
||||||
├── Dashboard.tsx
|
|
||||||
├── Batches.tsx
|
|
||||||
├── Providers.tsx
|
|
||||||
├── Acks.tsx
|
|
||||||
├── ActivityLog.tsx
|
|
||||||
├── Reconciliation.tsx
|
|
||||||
├── BatchDiff.tsx
|
|
||||||
├── Upload.tsx
|
|
||||||
└── Inbox.tsx
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2.8 Testing targets
|
|
||||||
|
|
||||||
**Frontend (Vitest + React Testing Library):**
|
|
||||||
|
|
||||||
- `DrillStackProvider.test.tsx` — max-2-level rule, peek-over-drawer, drawer-over-peek prevention, esc closes top, closeAll resets stack
|
|
||||||
- `DrillableCell.test.tsx` — renders chevron on hover, fires onClick, disabled state hides chevron + blocks click
|
|
||||||
- `PeekModal.test.tsx` — opens via context, closes on esc, closes on click-outside, 404 → toast + auto-close, network error → retry toast
|
|
||||||
- `ProviderDrawer.test.tsx` — Overview tab shows NPI + address + counts; Claims tab paginates; Activity tab paginates; j/k nav (if claim ids in scope)
|
|
||||||
- `RemitDrawer.test.tsx` — header summary, claim payments table, CAS panel, payer cell opens payer peek
|
|
||||||
- `AckDrawer.test.tsx` — header (source batch, ack code), per-segment status list, download button (unchanged)
|
|
||||||
- `PayerPeekContent.test.tsx` — loading skeleton, populated peek, "view all claims" link href
|
|
||||||
- `ValidationRulePeekContent.test.tsx` — known rule (R050_diagnosis_present) renders full description + X12 reference; unknown rule shows fallback copy
|
|
||||||
- Page tests (per-page drill map coverage): one test per drill target verifying the click routes correctly
|
|
||||||
|
|
||||||
**Backend (pytest):**
|
|
||||||
|
|
||||||
- `test_payer_summary_endpoint` — happy path, cache hit/miss, invalidation on `claim_written`
|
|
||||||
- `test_provider_detail_extended_response` — existing fields unchanged; new `recent_claims` + `recent_activity` arrays populated (10 each, sorted); empty arrays when the provider has no claims/activity
|
|
||||||
- `test_provider_detail_extended_response_backwards_compat` — clients that ignore the new arrays don't break; old fields identical to SP9 response shape
|
|
||||||
|
|
||||||
**Target counts:** ~25 new frontend tests, ~3 new backend tests. All 249 existing frontend tests + 409 existing backend tests must stay green.
|
|
||||||
|
|
||||||
### 2.9 Smoke test (end-of-SP21)
|
|
||||||
|
|
||||||
1. Start backend (`backend/.venv/bin/python -m cyclone serve`).
|
|
||||||
2. Start frontend (`npm run dev`).
|
|
||||||
3. Open `http://localhost:5173/`.
|
|
||||||
4. **Dashboard:** click a KPI tile → page navigates to `/claims` with the right filter.
|
|
||||||
5. **Dashboard:** click a top-providers row → provider drawer opens from the right.
|
|
||||||
6. **Dashboard:** click a recent-denials row → claim drawer opens.
|
|
||||||
7. **Dashboard:** click a recent-activity event of kind `claim_paid` → claim drawer opens for the underlying claim.
|
|
||||||
8. **Claims:** click a claim row → claim drawer opens (already worked, regression check).
|
|
||||||
9. **Claims:** hover the patient name cell → cursor + tint + chevron appear. Click → navigates to `/claims?patient=NAME`.
|
|
||||||
10. **Claims:** hover the provider cell → affordance. Click → provider drawer opens.
|
|
||||||
11. **Claims:** hover the payer cell → affordance. Click → payer peek modal opens (centered).
|
|
||||||
12. **Claims:** with the claim drawer open, click the payer name inside the drawer → peek opens on top of the drawer. Esc → peek closes; drawer still there. Esc again → drawer closes.
|
|
||||||
13. **Remittances:** click a remit row → remit drawer opens (was: inline expand only).
|
|
||||||
14. **Remittances:** click the claim id cell → claim drawer opens.
|
|
||||||
15. **Providers:** click a card → provider drawer opens.
|
|
||||||
16. **Acks:** click a row → ack drawer opens (was: only the Download button worked).
|
|
||||||
17. **Activity log:** click a `remit_received` event → remit drawer opens.
|
|
||||||
18. **Inbox:** click a rejected row → claim drawer opens (was: no-op).
|
|
||||||
19. **Inbox:** click a candidates row → remit drawer opens (was: no-op).
|
|
||||||
20. **Reconciliation:** click the body of an unmatched claim (not the select button) → navigates to `/claims?claim=ID` with the claim drawer open.
|
|
||||||
21. **Batch diff:** click an added claim id → claim drawer opens.
|
|
||||||
22. **Upload:** click a streamed claim's "See claim in detail" link → claim drawer opens (if persisted).
|
|
||||||
23. Browser back from any drawer state → drawer closes.
|
|
||||||
24. Commit empty `smoke: end-to-end SP21 (universal drilldown) flow passes`.
|
|
||||||
|
|
||||||
## 3. Risk areas
|
|
||||||
|
|
||||||
1. **Stack explosion.** Without an enforcement point, it's easy to write code that opens a drawer from inside a peek from inside a drawer. The `DrillStackProvider` API only exposes `open` (which respects the max-2 rule) and `closeTop`. There's no `openOnTopOf` — the stack is implicit from context. Risk: future contributors might bypass the provider and mount `<PeekModal>` ad-hoc. Mitigation: the `<DrillStackMount>` portal at the root of `App.tsx` is the only place peeks render; mounting `<PeekModal>` outside it renders nothing.
|
|
||||||
2. **URL vs ephemeral state confusion.** Drawers persist in URL; peeks don't. If a user opens a peek, copies the URL, and pastes it later, the peek is gone. Acceptable trade-off — peeks are exploratory; drawers are shareable.
|
|
||||||
3. **Cross-page drawer navigation.** Clicking a claim id on `/remittances` navigates to `/claims?claim=…`. The user lands on the claims page, not on the remittances page with the claim drawer overlaid. This is intentional — the claim drawer lives on `/claims`. Side effect: the user's remittances filter context is lost on nav. Mitigation: the nav is a soft fade + the URL preserves `?status=` etc. on `/claims` so the user can navigate back with browser back.
|
|
||||||
4. **Drawer nav state across page navigation.** When the user clicks a claim id on `/remittances`, lands on `/claims?claim=…`, and presses browser back, they go back to `/remittances` (the remits filter is restored). The claim drawer closes. Tested in smoke step 23.
|
|
||||||
5. **Inbox row click vs checkbox click.** Inbox rows have multi-select checkboxes. Row body click opens the drawer; checkbox click toggles selection. The `DrillableCell` wrapper goes around the row body, NOT the checkbox. Tested in `Inbox.test.tsx`.
|
|
||||||
6. **Payer summary cache staleness.** The 60s cache + pubsub invalidation is best-effort. Worst case: a payer summary is up to 60s stale after a `claim_written`. Acceptable for v1.
|
|
||||||
7. **Patient name in URL.** Navigating to `/claims?patient=John%20Doe` puts the patient name in the URL bar. Cyclone is local-only (127.0.0.1) per project context, so this is acceptable for v1. Flagged for SP22 if multi-operator / networked expansion happens. The existing PII scrubber (SP18) only scrubs log lines, not URL bars.
|
|
||||||
8. **Drawer dim overlay z-index.** With a peek over a drawer, three z-index layers: page (z=0), drawer overlay (z=40), peek overlay (z=50). The `<DrillStackMount>` portal renders peeks in a dedicated DOM node at `z-50`; the existing drawer overlays at `z-40` continue to work. Tested in `PeekModal.test.tsx` (asserts peek is rendered above drawer overlay).
|
|
||||||
|
|
||||||
## 4. Migration / rollout
|
|
||||||
|
|
||||||
- Single feature branch `universal-drilldown` cut from `main`.
|
|
||||||
- Worktree at `.worktrees/universal-drilldown/`, following SP1-3 pattern.
|
|
||||||
- DB migrations: **none** (uses existing tables: `claims`, `remittances`, `batches`, `activity_events`, `providers`, `payers`, `acks`).
|
|
||||||
- No new dependencies (Radix Dialog + zustand already in `package.json`).
|
|
||||||
- Phasing within the branch (each PR is shippable independently, each closes its slice of smoke steps):
|
|
||||||
1. **PR1 — Foundation primitives + first 3 surfaces:** `DrillStackProvider` + `DrillableCell` + `PeekModal` primitives, the App-level `<DrillStackMount>` portal, the hover affordance CSS, the extended `/api/providers/{npi}` response. Wire Dashboard KPI navigation + Dashboard Top providers drill + Dashboard Recent denials drill. Backend: `/api/payers/{payer_id}/summary` (the actual provider-overview data lives in the extended `/api/providers/{npi}` response — no separate provider endpoints needed). (Smoke steps 4-6.)
|
|
||||||
2. **PR2 — `ProviderDrawer` + activity event routing:** Build the ProviderDrawer (Overview tab only in this PR; Claims/Activity tabs come in PR3 with the recent_claims/recent_activity payload rendering). Wire Dashboard Recent activity for `claim_*` events (the activity → entity mapping lives in a small `eventKindToEntity()` helper introduced here). Wire Providers directory page + Claims provider cell. (Smoke steps 7, 10, 15.)
|
|
||||||
3. **PR3 — `ProviderDrawer` tabs + remaining event routing:** Add the Claims and Activity tabs to ProviderDrawer, using the extended `/api/providers/{npi}` payload. Wire Dashboard Recent activity for `remit_received` and `provider_added` event kinds (still routed via the PR2 `eventKindToEntity()` helper; `provider_added` routes to ProviderDrawer, `remit_received` will route to the PR4 RemitDrawer — until PR4 lands it shows a graceful "Remit drawer coming in PR4" toast). (Smoke step 7 finishes — all 4 event kinds route correctly by PR4 merge.)
|
|
||||||
4. **PR4 — `RemitDrawer` + 4 surfaces:** Build the RemitDrawer. Wire Remittances row click (replaces inline CAS expand), Inbox candidates + unmatched remit rows, Activity log `remit_received` event click, Remittances claim-id cell, ClaimDrawer matched-remit link. Now `eventKindToEntity()`'s `remit_received` branch resolves to a real drawer. (Smoke steps 13, 14, 17, 19.)
|
|
||||||
5. **PR5 — `AckDrawer` + final 8 surfaces:** Build the AckDrawer. Wire Acks row, Inbox rejected + payer_rejected + done_today rows, Reconciliation body click, Batch diff claim id, Upload "See claim in detail" link, ClaimDrawer payer peek (`PayerPeekContent`), ClaimDrawer validation-rule peek (`ValidationRulePeekContent`), refactor existing `ClaimDrawer` to use the `DrillDrawer` shell + `DrillDrawerHeader`. (Smoke steps 8, 9, 11, 12, 16, 18, 20, 21, 22, 23.)
|
|
||||||
- Merge to main after final PR + smoke pass; cleanup worktree + branch.
|
|
||||||
@@ -1,816 +0,0 @@
|
|||||||
# Cyclone Ubuntu Docker Deployment — Design
|
|
||||||
|
|
||||||
**Date:** 2026-06-22
|
|
||||||
**Status:** Approved (pending user review of this doc)
|
|
||||||
**Depends on:** [2026-06-19-cyclone-production-readiness-design.md](2026-06-19-cyclone-production-readiness-design.md) (in-memory store + react-query wiring; local-only)
|
|
||||||
**Replaces:** The "no auth, no Docker, local-only" posture of the parent spec with a production-grade posture for real PHI on a single Ubuntu server.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Overview
|
|
||||||
|
|
||||||
Cyclone's first sub-project shipped a usable local-only system: parse 837/835 files, browse the data, edit and resubmit rejected claims, export a corrected 837 ZIP. Everything ran on `127.0.0.1` with no auth, in-memory storage, and a `python -m cyclone serve` invocation.
|
|
||||||
|
|
||||||
This sub-project graduates that to a production-grade deployment on a single Ubuntu Linux server:
|
|
||||||
|
|
||||||
- **Two-container Docker Compose** (backend + frontend) replacing the dev-server split
|
|
||||||
- **App-layer authentication** with username + password, argon2id hashing, session cookies, and three fixed roles (admin / operator / viewer)
|
|
||||||
- **SQLCipher encryption at rest** for the SQLite DB, with the key as a Docker secret
|
|
||||||
- **Daily encrypted backups** with the existing AES-256-GCM `BackupService`, scheduler autostart, 14-day retention
|
|
||||||
- **LAN-only bind** (single operator, single Ubuntu server); VPN handles outside access
|
|
||||||
- **Manual `docker compose pull` updates** with semver tags; one-env-var rollback
|
|
||||||
- **Operator runbook** + bootstrap scripts so the deploy is reproducible
|
|
||||||
|
|
||||||
Real PHI flows through this deployment. The SFTP wire stays stubbed — operators download the corrected 837 ZIP and hand it to the Gainwell MFT UI manually. That's intentional and documented as v1 scope; SFTP wire is v2.
|
|
||||||
|
|
||||||
After this ships, the operator can:
|
|
||||||
1. `git clone <repo> /opt/cyclone && cd /opt/cyclone`
|
|
||||||
2. `docker compose build`
|
|
||||||
3. `bash scripts/cyclone-init.sh` → generates `/etc/cyclone/secrets/{db,secret,admin_pw}.key`, prints the admin password once
|
|
||||||
4. `docker compose up -d`
|
|
||||||
5. Open `http://<lan-ip>:8080/login` → log in as `admin`, change password, create operators
|
|
||||||
6. Upload parsed 837s, edit claims, export ZIPs, hand to MFT UI
|
|
||||||
7. Trust daily encrypted backups + the operator's off-box cron copy for ≤24h loss
|
|
||||||
|
|
||||||
## 2. Goals
|
|
||||||
|
|
||||||
1. **Dockerize the existing stack** so it runs the same way in production as in development. One repo, one `docker compose up -d`.
|
|
||||||
2. **Add app-layer authentication** with username/password + 3-role RBAC, replacing the current "no auth at all" posture.
|
|
||||||
3. **Encrypt the SQLite DB at rest** via SQLCipher; key as a Docker secret (not an env var, not a keychain on macOS).
|
|
||||||
4. **Wire the existing encrypted backup system** to autostart on container boot with sane defaults (24h interval, 14-day retention).
|
|
||||||
5. **Add an operator runbook + bootstrap scripts** so the system is reproducible from `git clone` and supportable without tribal knowledge.
|
|
||||||
6. **Preserve all existing functionality** — the parser, editor, exporter, scheduler, audit log, backup/restore flow all keep working unchanged from the user's perspective.
|
|
||||||
|
|
||||||
## 3. Non-goals (this sub-project)
|
|
||||||
|
|
||||||
- **Real SFTP wire (paramiko)** — keep stub. Operators download ZIPs and use the MFT UI. Listed as v2.
|
|
||||||
- **Public TLS / public domain / Caddy reverse proxy** — LAN-only bind for v1; VPN handles outside access. Listed as v2.
|
|
||||||
- **Multi-host HA / DB replication / load balancing** — single Ubuntu server. Listed as v2.
|
|
||||||
- **Prometheus / Grafana / real alerting** — v1 uses the simple `curl healthz | mail` cron pattern. Listed as v2.
|
|
||||||
- **Off-box automated backup shipping** — operator's responsibility via host cron / rsync. Documented in runbook, not automated.
|
|
||||||
- **2FA / SSO / OAuth** — v2.
|
|
||||||
- **Self-service password reset** — v2; v1 admin resets via `cyclone admin reset-password` CLI.
|
|
||||||
- **Per-file encryption of prodfiles / sftp_staging volumes** — v2; v1 relies on volume-level filesystem permissions + the SQLCipher-encrypted DB + the host being physically secure.
|
|
||||||
- **LUKS full-disk encryption** — out of scope; assumed to be an operator-level concern (Ubuntu installer offers it).
|
|
||||||
- **Automatic update mechanism** (Watchtower etc.) — v1 is manual `docker compose pull`. Listed as v2.
|
|
||||||
- **New X12 transaction types or validation rules** — not this sub-project.
|
|
||||||
|
|
||||||
## 4. Stack
|
|
||||||
|
|
||||||
**Backend additions:**
|
|
||||||
- `argon2-cffi` (new dep) for password hashing
|
|
||||||
- `cyclone.auth` module: `User` / `Session` SQLAlchemy models, password hashing helpers, lockout logic
|
|
||||||
- `cyclone.api.auth` routes: `/api/auth/{login,logout,change-password,me}`
|
|
||||||
- `cyclone.api.users` routes: `/api/admin/users` (admin-only CRUD)
|
|
||||||
- `cyclone.cli` additions: `cyclone admin {create-user,reset-password,list-users}`
|
|
||||||
- Migration `0012_users_sessions.sql` for `users` + `sessions` tables
|
|
||||||
- Existing `BackupService` (SP17) and `audit_log` machinery reused unchanged
|
|
||||||
|
|
||||||
**Backend Dockerfile** (`backend/Dockerfile`):
|
|
||||||
- Multi-stage: builder stage installs `.[sqlcipher]` (sqlcipher is optional but recommended; the engine falls back to plain SQLite if the package isn't actually present, same graceful default as today), runtime stage copies installed site-packages + source
|
|
||||||
- Base: `python:3.11-slim-bookworm` (matches `requires-python = ">=3.11"`)
|
|
||||||
- Non-root user `cyclone` (uid 1000)
|
|
||||||
- `HEALTHCHECK CMD curl -fs http://localhost:8000/api/healthz || exit 1`
|
|
||||||
- Entrypoint: `python -m cyclone serve`
|
|
||||||
|
|
||||||
**Frontend additions:**
|
|
||||||
- `nginx.conf` for SPA routing + reverse proxy to backend
|
|
||||||
- `Dockerfile.frontend`: multi-stage builder (node:20-alpine runs `npm ci && npm run build`) + runtime (nginx:1.27-alpine serving `dist/`)
|
|
||||||
- `useCurrentUser()` hook (react-query)
|
|
||||||
- `Login.tsx` page
|
|
||||||
- `<AuthGate>` wrapper in `App.tsx`
|
|
||||||
- `<NavBar>` shows username + role badge + logout button
|
|
||||||
- `/admin/users` page (admin-only)
|
|
||||||
|
|
||||||
**Infrastructure:**
|
|
||||||
- `docker-compose.yml` at repo root
|
|
||||||
- `scripts/cyclone-init.sh` — generates `/etc/cyclone/secrets/*` with `openssl rand -hex 32`
|
|
||||||
- `scripts/post-deploy.sh` — sets up logrotate + healthcheck cron on host
|
|
||||||
- `scripts/smoke.sh` — bring-up + login + parse + export end-to-end test
|
|
||||||
- `RUNBOOK.md` — daily/weekly/quarterly/as-needed ops procedures
|
|
||||||
- `.dockerignore` files for both build contexts
|
|
||||||
|
|
||||||
## 5. Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────────────────────┐
|
|
||||||
│ Ubuntu Linux server (your machine) │
|
|
||||||
│ │
|
|
||||||
│ cyclone_network (bridge) │
|
|
||||||
│ ┌─────────────────────┐ ┌────────────────────┐ │
|
|
||||||
│ │ cyclone-backend │ │ cyclone-frontend │ │
|
|
||||||
│ │ FastAPI │◄───┤ nginx (SPA + proxy)│ │
|
|
||||||
│ │ :8000 (internal) │ │ :8080 (host) │ │
|
|
||||||
│ └──────┬──────────────┘ └─────────┬──────────┘ │
|
|
||||||
│ │ │ │
|
|
||||||
└─────────┼─────────────────────────────┼────────────┘
|
|
||||||
│ │
|
|
||||||
┌──────▼──────────────────┐ ┌──────▼──────────┐
|
|
||||||
│ Named volumes │ │ Bind mounts │
|
|
||||||
│ cyclone_db │ │ ./dist (built) │
|
|
||||||
│ cyclone_backups │ │ ./nginx.conf │
|
|
||||||
│ cyclone_prodfiles │ └─────────────────┘
|
|
||||||
│ cyclone_sftp_staging │
|
|
||||||
└─────────────────────────┘
|
|
||||||
|
|
||||||
/etc/cyclone/secrets/ (host, chmod 600, root:root)
|
|
||||||
├─ db.key → /run/secrets/cyclone_db_key
|
|
||||||
├─ secret.key → /run/secrets/cyclone_secret_key
|
|
||||||
└─ admin_pw → /run/secrets/cyclone_admin_password
|
|
||||||
```
|
|
||||||
|
|
||||||
**Why two containers:** nginx serves the React build faster than FastAPI would, decouples frontend rebuild cadence from backend release cadence, and is the standard ops pattern. nginx also reverse-proxies `/api/*` to the backend so the browser only talks to one origin (no CORS needed, no preflight requests for the `Cookie` header).
|
|
||||||
|
|
||||||
**Why LAN-only bind:** nginx listens on `0.0.0.0:8080` inside the host network namespace, but the operator is expected to bind to the LAN IP via firewall rules / not exposing on the WAN. VPN handles outside access. No public TLS needed for v1.
|
|
||||||
|
|
||||||
**Container-to-container networking:** the frontend container reaches the backend at `http://cyclone-backend:8000` over the compose-managed bridge network. The browser only ever sees `http://<lan-ip>:8080`.
|
|
||||||
|
|
||||||
**Healthcheck:** container-level `curl -fs http://localhost:8000/api/healthz` every 30s, 3 retries. Compose `restart: unless-stopped` on healthcheck failure.
|
|
||||||
|
|
||||||
**Reverse proxy in nginx:**
|
|
||||||
|
|
||||||
```nginx
|
|
||||||
server {
|
|
||||||
listen 8080;
|
|
||||||
server_name _;
|
|
||||||
|
|
||||||
root /usr/share/nginx/html;
|
|
||||||
index index.html;
|
|
||||||
|
|
||||||
# API + auth: proxy to backend
|
|
||||||
location /api/ {
|
|
||||||
proxy_pass http://cyclone-backend:8000;
|
|
||||||
proxy_set_header Host $host;
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
|
||||||
proxy_read_timeout 300s; # long enough for big parse streams
|
|
||||||
client_max_body_size 50m; # claims files can be large
|
|
||||||
}
|
|
||||||
|
|
||||||
# SPA: serve index.html for all non-/api routes
|
|
||||||
location / {
|
|
||||||
try_files $uri $uri/ /index.html;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 6. Backend changes
|
|
||||||
|
|
||||||
### 6.1 New module `backend/src/cyclone/auth/__init__.py`
|
|
||||||
|
|
||||||
```python
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class Role:
|
|
||||||
VIEWER = "viewer"
|
|
||||||
OPERATOR = "operator"
|
|
||||||
ADMIN = "admin"
|
|
||||||
|
|
||||||
ROLE_RANK = {Role.VIEWER: 0, Role.OPERATOR: 1, Role.ADMIN: 2}
|
|
||||||
|
|
||||||
|
|
||||||
class User(Base):
|
|
||||||
__tablename__ = "users"
|
|
||||||
id: Mapped[str] # uuid4 hex
|
|
||||||
username: Mapped[str] # UNIQUE
|
|
||||||
password_hash: Mapped[str] # argon2id
|
|
||||||
role: Mapped[str] # viewer | operator | admin
|
|
||||||
created_at: Mapped[datetime]
|
|
||||||
last_login_at: Mapped[datetime | None]
|
|
||||||
failed_count: Mapped[int] = 0
|
|
||||||
locked_until: Mapped[datetime | None] = None
|
|
||||||
|
|
||||||
|
|
||||||
class Session(Base):
|
|
||||||
__tablename__ = "sessions"
|
|
||||||
id: Mapped[str] # 32-byte hex token
|
|
||||||
user_id: Mapped[str]
|
|
||||||
created_at: Mapped[datetime]
|
|
||||||
expires_at: Mapped[datetime]
|
|
||||||
ip: Mapped[str | None]
|
|
||||||
user_agent: Mapped[str | None]
|
|
||||||
```
|
|
||||||
|
|
||||||
Password hashing uses `argon2-cffi` with the OWASP-recommended parameters (`time_cost=2`, `memory_cost=19456`, `parallelism=1`, `hash_len=32`).
|
|
||||||
|
|
||||||
### 6.2 Auth routes in `backend/src/cyclone/api.py` (new router `auth_router`)
|
|
||||||
|
|
||||||
| Method | Path | Behavior | Auth |
|
|
||||||
|---|---|---|---|
|
|
||||||
| POST | `/api/auth/login` | `{username, password}` → verify argon2id, increment `failed_count` on miss (lock at 5/15min), create session, set `HttpOnly; Secure; SameSite=Lax` cookie | none |
|
|
||||||
| POST | `/api/auth/logout` | delete session row, clear cookie | any logged-in |
|
|
||||||
| GET | `/api/auth/me` | `{user: {username, role}}` or 401 | any logged-in |
|
|
||||||
| POST | `/api/auth/change-password` | `{old_password, new_password}` → verify, rehash | any logged-in |
|
|
||||||
| POST | `/api/admin/users` | `{username, password, role}` → create user | admin |
|
|
||||||
| GET | `/api/admin/users` | list users | admin |
|
|
||||||
| PATCH | `/api/admin/users/{id}` | `{role?, password?}` | admin |
|
|
||||||
| DELETE | `/api/admin/users/{id}` | soft-disable (set `disabled_at`) | admin |
|
|
||||||
|
|
||||||
### 6.3 The `require_role` dependency
|
|
||||||
|
|
||||||
Every existing protected route gets a `dependencies=[Depends(require_role(Role.OPERATOR))]` annotation:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def require_role(min_role: str):
|
|
||||||
"""FastAPI dependency factory: read cookie, load user, enforce role."""
|
|
||||||
def dep(request: Request) -> User:
|
|
||||||
sid = request.cookies.get("cyclone_session")
|
|
||||||
if not sid:
|
|
||||||
raise HTTPException(401, "not authenticated")
|
|
||||||
with db.SessionLocal()() as s:
|
|
||||||
sess = s.get(Session, sid)
|
|
||||||
if sess is None or sess.expires_at < utcnow():
|
|
||||||
raise HTTPException(401, "session expired")
|
|
||||||
user = s.get(User, sess.user_id)
|
|
||||||
if user is None:
|
|
||||||
raise HTTPException(401, "user gone")
|
|
||||||
if user.locked_until and user.locked_until > utcnow():
|
|
||||||
raise HTTPException(423, "account locked")
|
|
||||||
if ROLE_RANK[user.role] < ROLE_RANK[min_role]:
|
|
||||||
raise HTTPException(403, f"requires {min_role}")
|
|
||||||
request.state.user = user
|
|
||||||
request.state.actor = user.username # existing audit-log contract
|
|
||||||
return user
|
|
||||||
return dep
|
|
||||||
```
|
|
||||||
|
|
||||||
**Role → action matrix:**
|
|
||||||
|
|
||||||
| Action | viewer | operator | admin |
|
|
||||||
|---|:-:|:-:|:-:|
|
|
||||||
| View claims, batches, acks | ✓ | ✓ | ✓ |
|
|
||||||
| `POST /api/parse-837`, `/api/parse-835` | — | ✓ | ✓ |
|
|
||||||
| Edit claim fields, resubmit rejected | — | ✓ | ✓ |
|
|
||||||
| `POST /api/batches/{id}/export-837` | — | ✓ | ✓ |
|
|
||||||
| `POST /api/clearhouse/submit` | — | ✓ | ✓ |
|
|
||||||
| `/api/admin/users` CRUD | — | — | ✓ |
|
|
||||||
| `/api/admin/backup/*` | — | — | ✓ |
|
|
||||||
| `/api/config/*` (clearhouse, payers) | — | — | ✓ |
|
|
||||||
| `/api/admin/audit-log` | — | — | ✓ |
|
|
||||||
| Backup scheduler on/off | — | — | ✓ |
|
|
||||||
|
|
||||||
**Login rate-limit:** the existing `cyclone.api.security.request_rejected` machinery already rate-limits by IP; we add per-username lockout on top:
|
|
||||||
- `failed_count >= 5` within 15min → `locked_until = now + 15min`
|
|
||||||
- Locked accounts return 423; the lock clears after the cooldown
|
|
||||||
|
|
||||||
### 6.4 Audit log wiring
|
|
||||||
|
|
||||||
The existing `audit_log.append(actor=..., ...)` calls already accept an actor string. The new auth dependency sets `request.state.actor = user.username`, and a small middleware reads it for every request. No audit-log schema change.
|
|
||||||
|
|
||||||
### 6.5 CLI additions
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cyclone admin create-user --username X --role admin --password <from_stdin or --password-file>
|
|
||||||
cyclone admin reset-password --username X --new-password <from_stdin or --password-file>
|
|
||||||
cyclone admin list-users
|
|
||||||
```
|
|
||||||
|
|
||||||
Used by `scripts/cyclone-init.sh` for the bootstrap admin, and by the operator for password recovery.
|
|
||||||
|
|
||||||
### 6.6 Migration `0012_users_sessions.sql`
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE TABLE users (
|
|
||||||
id TEXT PRIMARY KEY,
|
|
||||||
username TEXT NOT NULL UNIQUE,
|
|
||||||
password_hash TEXT NOT NULL,
|
|
||||||
role TEXT NOT NULL CHECK (role IN ('admin', 'operator', 'viewer')),
|
|
||||||
created_at TIMESTAMP NOT NULL,
|
|
||||||
last_login_at TIMESTAMP,
|
|
||||||
failed_count INTEGER NOT NULL DEFAULT 0,
|
|
||||||
locked_until TIMESTAMP,
|
|
||||||
disabled_at TIMESTAMP
|
|
||||||
);
|
|
||||||
CREATE INDEX idx_users_username ON users(username);
|
|
||||||
|
|
||||||
CREATE TABLE sessions (
|
|
||||||
id TEXT PRIMARY KEY,
|
|
||||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
|
||||||
created_at TIMESTAMP NOT NULL,
|
|
||||||
expires_at TIMESTAMP NOT NULL,
|
|
||||||
ip TEXT,
|
|
||||||
user_agent TEXT
|
|
||||||
);
|
|
||||||
CREATE INDEX idx_sessions_user_id ON sessions(user_id);
|
|
||||||
CREATE INDEX idx_sessions_expires_at ON sessions(expires_at);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 6.7 Session timeout config
|
|
||||||
|
|
||||||
| Setting | Default | Env var |
|
|
||||||
|---|---|---|
|
|
||||||
| Session absolute lifetime | 30 days | `CYCLONE_SESSION_ABSOLUTE_DAYS` |
|
|
||||||
| Session sliding expiry | 8 hours | `CYCLONE_SESSION_SLIDING_HOURS` |
|
|
||||||
| Cookie name | `cyclone_session` | `CYCLONE_COOKIE_NAME` |
|
|
||||||
| Cookie `Secure` flag | true | always true in prod |
|
|
||||||
|
|
||||||
### 6.8 Backend Dockerfile
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
# Builder stage
|
|
||||||
FROM python:3.11-slim-bookworm AS builder
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
||||||
build-essential libsqlcipher-dev libffi-dev \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
WORKDIR /build
|
|
||||||
COPY pyproject.toml .
|
|
||||||
COPY src/ ./src/
|
|
||||||
RUN pip wheel --no-cache-dir --wheel-dir /wheels '.[sqlcipher]'
|
|
||||||
|
|
||||||
# Runtime stage
|
|
||||||
FROM python:3.11-slim-bookworm
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
||||||
libsqlcipher-dev curl tini \
|
|
||||||
&& rm -rf /var/lib/apt/lists/* \
|
|
||||||
&& useradd --create-home --uid 1000 cyclone
|
|
||||||
WORKDIR /app
|
|
||||||
COPY --from=builder /wheels /wheels
|
|
||||||
RUN pip install --no-cache-dir --no-index --find-links /wheels cyclone
|
|
||||||
COPY src/ /app/src/
|
|
||||||
USER cyclone
|
|
||||||
ENV PYTHONUNBUFFERED=1
|
|
||||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
|
|
||||||
CMD curl -fs http://localhost:8000/api/healthz || exit 1
|
|
||||||
ENTRYPOINT ["tini", "--"]
|
|
||||||
CMD ["python", "-m", "cyclone", "serve"]
|
|
||||||
```
|
|
||||||
|
|
||||||
`sqlcipher` is included but the engine falls back to plain SQLite if the package isn't actually installed — same graceful default as today, just biased toward enabling encryption in prod.
|
|
||||||
|
|
||||||
## 7. Frontend changes
|
|
||||||
|
|
||||||
### 7.1 New login page `src/pages/Login.tsx`
|
|
||||||
|
|
||||||
A simple form: username + password inputs, submit button, error message slot. On success, navigate to `/`. Lives outside the auth-gated shell, served from a route that doesn't require authentication.
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
export default function Login() {
|
|
||||||
const login = useMutation({
|
|
||||||
mutationFn: ({ username, password }) => api.login(username, password),
|
|
||||||
onSuccess: () => queryClient.invalidateQueries({ queryKey: ['me'] }),
|
|
||||||
});
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 7.2 `useCurrentUser` hook
|
|
||||||
|
|
||||||
```ts
|
|
||||||
export function useCurrentUser() {
|
|
||||||
return useQuery({
|
|
||||||
queryKey: ['me'],
|
|
||||||
queryFn: () => api.me(),
|
|
||||||
retry: false,
|
|
||||||
staleTime: 60_000,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 7.3 AuthGate in `App.tsx`
|
|
||||||
|
|
||||||
```tsx
|
|
||||||
function AuthGate({ children }: { children: React.ReactNode }) {
|
|
||||||
const { data: user, isLoading } = useCurrentUser();
|
|
||||||
const location = useLocation();
|
|
||||||
if (isLoading) return <FullPageSkeleton />;
|
|
||||||
if (!user) return <Navigate to="/login" state={{ from: location }} replace />;
|
|
||||||
return <>{children}</>;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Wrap every route except `/login` in `<AuthGate>`. `/login` itself does *not* require auth.
|
|
||||||
|
|
||||||
### 7.4 Role-based UI gating
|
|
||||||
|
|
||||||
```ts
|
|
||||||
const ROLE_RANK = { viewer: 0, operator: 1, admin: 2 };
|
|
||||||
export function useCan(minRole: 'viewer' | 'operator' | 'admin'): boolean {
|
|
||||||
const { data: user } = useCurrentUser();
|
|
||||||
if (!user) return false;
|
|
||||||
return ROLE_RANK[user.role] >= ROLE_RANK[minRole];
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Used in NavBar to hide "Admin" links from operators/viewers, and on the `/admin/users` page as a guard (`if (!useCan('admin')) return <Forbidden />`).
|
|
||||||
|
|
||||||
### 7.5 NavBar changes
|
|
||||||
|
|
||||||
- Username + role badge (admin/operator/viewer) shown right-aligned
|
|
||||||
- Logout button (calls `api.logout()`, clears react-query cache, navigates to `/login`)
|
|
||||||
- "Admin" dropdown appears only when `useCan('admin')`
|
|
||||||
|
|
||||||
### 7.6 `/admin/users` page (admin only)
|
|
||||||
|
|
||||||
Table of users (username, role, last login, status). Actions:
|
|
||||||
- **Create user** (modal with username + password + role)
|
|
||||||
- **Edit role** (inline select)
|
|
||||||
- **Reset password** (modal)
|
|
||||||
- **Disable / enable** (toggle; sets `disabled_at`)
|
|
||||||
|
|
||||||
### 7.7 api.ts additions
|
|
||||||
|
|
||||||
```ts
|
|
||||||
api.login(username: string, password: string): Promise<{ user: User }>
|
|
||||||
api.logout(): Promise<void>
|
|
||||||
api.me(): Promise<{ user: User }> // throws 401 if not logged in
|
|
||||||
api.changePassword(oldPw: string, newPw: string): Promise<void>
|
|
||||||
api.listUsers(): Promise<User[]>
|
|
||||||
api.createUser(...): Promise<User>
|
|
||||||
api.updateUser(id: string, ...): Promise<User>
|
|
||||||
api.deleteUser(id: string): Promise<void>
|
|
||||||
```
|
|
||||||
|
|
||||||
All `api.*` calls send `credentials: 'include'` so the session cookie flows.
|
|
||||||
|
|
||||||
### 7.8 Frontend Dockerfile + nginx.conf
|
|
||||||
|
|
||||||
`frontend/Dockerfile`:
|
|
||||||
```dockerfile
|
|
||||||
# Build stage
|
|
||||||
FROM node:20-alpine AS builder
|
|
||||||
WORKDIR /build
|
|
||||||
COPY package.json package-lock.json ./
|
|
||||||
RUN npm ci
|
|
||||||
COPY . .
|
|
||||||
RUN npm run build
|
|
||||||
|
|
||||||
# Runtime stage
|
|
||||||
FROM nginx:1.27-alpine
|
|
||||||
COPY nginx.conf /etc/nginx/conf.d/default.conf
|
|
||||||
COPY --from=builder /build/dist /usr/share/nginx/html
|
|
||||||
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
|
|
||||||
CMD wget -qO- http://localhost:8080/ >/dev/null || exit 1
|
|
||||||
```
|
|
||||||
|
|
||||||
The nginx config shown in §5 lives at `frontend/nginx.conf`.
|
|
||||||
|
|
||||||
## 8. Data, secrets, backups
|
|
||||||
|
|
||||||
### 8.1 Named volumes
|
|
||||||
|
|
||||||
| Volume | Mount path in container | Contents |
|
|
||||||
|---|---|---|
|
|
||||||
| `cyclone_db` | `/var/lib/cyclone/db/cyclone.db` | SQLCipher-encrypted SQLite |
|
|
||||||
| `cyclone_backups` | `/var/lib/cyclone/backups/` | AES-256-GCM `.bin` + `.meta.json` |
|
|
||||||
| `cyclone_prodfiles` | `/var/lib/cyclone/prodfiles/` | Uploaded 837 `.txt`, downloaded 835 |
|
|
||||||
| `cyclone_sftp_staging` | `/var/lib/cyclone/sftp_staging/` | SFTP stub's outbound/inbound |
|
|
||||||
|
|
||||||
These are managed by Docker (`docker volume create` / compose `volumes:`); they live under `/var/lib/docker/volumes/` on the host.
|
|
||||||
|
|
||||||
### 8.2 Docker secrets (host-managed)
|
|
||||||
|
|
||||||
| File | Mounted as | Purpose |
|
|
||||||
|---|---|---|
|
|
||||||
| `/etc/cyclone/secrets/db.key` | `/run/secrets/cyclone_db_key` | SQLCipher `PRAGMA key` — 64 hex chars |
|
|
||||||
| `/etc/cyclone/secrets/secret.key` | `/run/secrets/cyclone_secret_key` | Cookie signing key — 64 hex chars |
|
|
||||||
| `/etc/cyclone/secrets/admin_pw` | `/run/secrets/cyclone_admin_password` | One-time bootstrap admin password |
|
|
||||||
|
|
||||||
`scripts/cyclone-init.sh` generates all three with `openssl rand -hex 32` (db.key + secret.key) and a memorable-but-random admin password, sets permissions to `chmod 600 root:root`, and prints the admin password once.
|
|
||||||
|
|
||||||
### 8.3 Backup strategy (≤24h loss is acceptable)
|
|
||||||
|
|
||||||
- The existing `BackupService` (SP17) is already implemented and tested: takes online snapshot via SQLite `.backup()` API, encrypts with AES-256-GCM, writes `.bin` + `.meta.json` in `cyclone_backups` volume
|
|
||||||
- `CYCLONE_BACKUP_AUTOSTART=1` in compose starts the in-container backup scheduler on container start
|
|
||||||
- `CYCLONE_BACKUP_INTERVAL_HOURS=24` (default)
|
|
||||||
- 14-day retention (default; configurable via `CYCLONE_BACKUP_RETENTION_DAYS`)
|
|
||||||
- Restore via existing `/api/admin/backup/{id}/restore/{initiate,confirm}` (admin-only after auth is wired)
|
|
||||||
- Existing audit-log call `actor="backup-scheduler"` becomes `actor="backup-scheduler"` (unchanged — system actor, not a user)
|
|
||||||
|
|
||||||
**Off-box backup copy** (operator's responsibility, documented in runbook):
|
|
||||||
|
|
||||||
The operator sets up a host cron / systemd timer that rsyncs `/var/lib/docker/volumes/cyclone_backups/_data/` to an external drive or NAS nightly. The `.bin` files are already encrypted so the destination doesn't need its own encryption. Documented in `RUNBOOK.md` as a required post-deploy step.
|
|
||||||
|
|
||||||
### 8.4 Data loss scenarios
|
|
||||||
|
|
||||||
| Scenario | Outcome |
|
|
||||||
|---|---|
|
|
||||||
| Container crashes | Docker restarts; SQLite WAL recovers; no loss |
|
|
||||||
| Disk corruption | Restore from latest local backup (≤24h old) |
|
|
||||||
| Disk total loss | Restore from off-box backup copy (depends on operator's cron cadence) |
|
|
||||||
| DB key compromise | Rotate via `POST /api/admin/db/rotate-key`; old backups re-encrypted on next cycle |
|
|
||||||
| Operator forgets admin password | Reset via `cyclone admin reset-password --username admin --new-password X` |
|
|
||||||
|
|
||||||
### 8.5 Encryption-at-rest coverage
|
|
||||||
|
|
||||||
| Asset | Encrypted at rest (v1) | Notes |
|
|
||||||
|---|---|---|
|
|
||||||
| Live SQLite DB | ✓ SQLCipher | AES-256, key in Docker secret |
|
|
||||||
| Backup `.bin` files | ✓ AES-256-GCM | Key from backup passphrase |
|
|
||||||
| Prodfiles (uploaded 837s) | — plain on disk | v2: per-file encryption or move to DB |
|
|
||||||
| sftp_staging | — plain on disk | v2: per-file encryption |
|
|
||||||
| Log files | — plain on disk | rotated, contained |
|
|
||||||
|
|
||||||
v1 assumes the host is physically secure (single-operator server, locked room) and that LUKS is operator-level.
|
|
||||||
|
|
||||||
## 9. Logging, monitoring, updates, ops
|
|
||||||
|
|
||||||
### 9.1 Logging
|
|
||||||
|
|
||||||
- JSON to stdout (already configured via `logging_config.py`) — Docker collects via `docker logs`
|
|
||||||
- Bind-mount `/var/log/cyclone/` from the host so logrotate can manage retention (configured in `scripts/post-deploy.sh`)
|
|
||||||
- Log levels: INFO in prod, DEBUG opt-in via `CYCLONE_LOG_LEVEL=DEBUG`
|
|
||||||
- Every login event (success + failure) logged with username, ip, user-agent, role
|
|
||||||
- Every state-changing API call carries `actor=<user_id>` (existing audit-log machinery; we wire the auth user into `request.state.actor`)
|
|
||||||
- New audit events: `auth.login`, `auth.logout`, `auth.login_failed`, `auth.password_changed`, `auth.user_created`, `auth.user_updated`, `auth.user_disabled`
|
|
||||||
|
|
||||||
### 9.2 Healthcheck
|
|
||||||
|
|
||||||
- Container-level: `curl -fs http://localhost:8000/api/healthz` every 30s, 3 retries (configured in Dockerfile)
|
|
||||||
- Compose `restart: unless-stopped` on healthcheck failure (max 3 restarts then backoff)
|
|
||||||
- `GET /api/healthz` returns `{db_ok: bool, scheduler_running: bool, last_backup_at: timestamp}` (existing endpoint, unchanged)
|
|
||||||
|
|
||||||
### 9.3 Monitoring (v1 minimal)
|
|
||||||
|
|
||||||
- No Prometheus/Grafana in v1
|
|
||||||
- Host-level cron: `curl -fs http://localhost:8080/api/healthz >/dev/null || echo "Cyclone down at $(date)" | mail -s "cyclone DOWN" you@example.com`
|
|
||||||
- Set up by `scripts/post-deploy.sh` during initial bootstrap
|
|
||||||
|
|
||||||
### 9.4 Updates
|
|
||||||
|
|
||||||
- Images tagged semver from `package.json` / `pyproject.toml`: `cyclone-backend:0.1.0`, `cyclone-frontend:0.1.0`
|
|
||||||
- Two tag aliases per image:
|
|
||||||
- `latest` — set automatically by `docker compose build` (most recent build)
|
|
||||||
- `stable` — manually promoted by the operator once a release has run in prod for ≥1 week without issues, via `docker tag cyclone-backend:0.1.0 cyclone-backend:stable && docker tag cyclone-frontend:0.1.0 cyclone-frontend:stable`. Compose defaults to `:stable` so a new build doesn't get auto-rolled-out.
|
|
||||||
- Update procedure (to pick up a new `:stable`):
|
|
||||||
```bash
|
|
||||||
cd /opt/cyclone
|
|
||||||
docker compose pull # pulls newer :stable tags
|
|
||||||
docker compose up -d # recreates containers, preserves volumes + secrets
|
|
||||||
```
|
|
||||||
- DB migrations run automatically on backend start; migrations are forward-only with documented downgrade procedures (existing pattern)
|
|
||||||
- Rollback: `TAG=0.0.9 docker compose up -d` (one env var override; compose reads `${TAG:-stable}` for the image tag)
|
|
||||||
- Previous image stays in Docker's local cache for one cycle so rollback is offline
|
|
||||||
|
|
||||||
### 9.5 Initial bootstrap
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone <repo> /opt/cyclone && cd /opt/cyclone
|
|
||||||
docker compose build # or: docker compose pull
|
|
||||||
bash scripts/cyclone-init.sh # generates /etc/cyclone/secrets/*, prints admin password
|
|
||||||
docker compose up -d # starts both containers
|
|
||||||
# wait ~10s for migrations
|
|
||||||
docker compose exec backend cyclone admin create-user \
|
|
||||||
--username admin --role admin --password-file /run/secrets/cyclone_admin_password
|
|
||||||
# open browser to http://<lan-ip>:8080/login
|
|
||||||
# log in as admin, change password, create operators
|
|
||||||
bash scripts/post-deploy.sh # logrotate + healthcheck cron
|
|
||||||
```
|
|
||||||
|
|
||||||
### 9.6 Daily ops (operator runbook)
|
|
||||||
|
|
||||||
- **Daily:** check `GET /api/healthz` (or trust the healthcheck email)
|
|
||||||
- **Weekly:** review `/admin/audit-log` page for unusual activity
|
|
||||||
- **Quarterly:** rotate DB key via `POST /api/admin/db/rotate-key`
|
|
||||||
- **As needed:** create operators via `/admin/users`; restore from backup via the restore UI
|
|
||||||
- **Annual:** rotate cookie signing key (re-login all users)
|
|
||||||
|
|
||||||
Full procedures live in `RUNBOOK.md` shipped in the repo.
|
|
||||||
|
|
||||||
## 10. docker-compose.yml
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
name: cyclone
|
|
||||||
|
|
||||||
services:
|
|
||||||
backend:
|
|
||||||
image: cyclone-backend:${TAG:-stable}
|
|
||||||
build: ./backend
|
|
||||||
restart: unless-stopped
|
|
||||||
environment:
|
|
||||||
CYCLONE_DB_URL: "sqlite:////var/lib/cyclone/db/cyclone.db"
|
|
||||||
CYCLONE_BACKUP_AUTOSTART: "1"
|
|
||||||
CYCLONE_BACKUP_INTERVAL_HOURS: "24"
|
|
||||||
CYCLONE_BACKUP_RETENTION_DAYS: "14"
|
|
||||||
CYCLONE_LOG_LEVEL: "INFO"
|
|
||||||
CYCLONE_LOG_FILE: "/var/log/cyclone/cyclone.log"
|
|
||||||
CYCLONE_LOG_JSON: "1"
|
|
||||||
CYCLONE_SESSION_ABSOLUTE_DAYS: "30"
|
|
||||||
CYCLONE_SESSION_SLIDING_HOURS: "8"
|
|
||||||
CYCLONE_COOKIE_SECURE: "1"
|
|
||||||
secrets:
|
|
||||||
- cyclone_db_key
|
|
||||||
- cyclone_secret_key
|
|
||||||
volumes:
|
|
||||||
- cyclone_db:/var/lib/cyclone/db
|
|
||||||
- cyclone_backups:/var/lib/cyclone/backups
|
|
||||||
- cyclone_prodfiles:/var/lib/cyclone/prodfiles
|
|
||||||
- cyclone_sftp_staging:/var/lib/cyclone/sftp_staging
|
|
||||||
- cyclone_logs:/var/log/cyclone
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "curl", "-fs", "http://localhost:8000/api/healthz"]
|
|
||||||
interval: 30s
|
|
||||||
timeout: 5s
|
|
||||||
retries: 3
|
|
||||||
start_period: 30s
|
|
||||||
networks: [cyclone_network]
|
|
||||||
|
|
||||||
frontend:
|
|
||||||
image: cyclone-frontend:${TAG:-stable}
|
|
||||||
build: ./frontend
|
|
||||||
restart: unless-stopped
|
|
||||||
ports:
|
|
||||||
- "8080:8080"
|
|
||||||
depends_on:
|
|
||||||
backend:
|
|
||||||
condition: service_healthy
|
|
||||||
networks: [cyclone_network]
|
|
||||||
|
|
||||||
secrets:
|
|
||||||
cyclone_db_key:
|
|
||||||
file: /etc/cyclone/secrets/db.key
|
|
||||||
cyclone_secret_key:
|
|
||||||
file: /etc/cyclone/secrets/secret.key
|
|
||||||
cyclone_admin_password:
|
|
||||||
file: /etc/cyclone/secrets/admin_pw
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
cyclone_db:
|
|
||||||
cyclone_backups:
|
|
||||||
cyclone_prodfiles:
|
|
||||||
cyclone_sftp_staging:
|
|
||||||
cyclone_logs:
|
|
||||||
|
|
||||||
networks:
|
|
||||||
cyclone_network:
|
|
||||||
driver: bridge
|
|
||||||
```
|
|
||||||
|
|
||||||
The `8080:8080` port is the only one published to the host. The operator is expected to bind to the LAN IP at the firewall level (UFW rules or similar).
|
|
||||||
|
|
||||||
## 11. Error handling
|
|
||||||
|
|
||||||
- **Login failures:** increment `failed_count`; lock at 5 fails / 15min; log `auth.login_failed`; 423 on locked accounts.
|
|
||||||
- **Session expiry:** 401; frontend redirects to `/login` with `state.from` preserved.
|
|
||||||
- **Missing role:** 403 with explicit message; UI shows a "Forbidden" page instead of trying to render.
|
|
||||||
- **Backend crash:** Docker healthcheck restarts the container; SQLite WAL recovers in-flight writes.
|
|
||||||
- **DB key missing:** container starts but `db.is_encryption_enabled()` returns False; `BackupService` refuses to run (existing behavior); compose logs a warning.
|
|
||||||
- **Migration failure on startup:** container exits with non-zero; compose `restart: unless-stopped` backs off after 3 attempts; operator checks logs via `docker compose logs backend`.
|
|
||||||
- **Backup failure:** logged at ERROR; audit event `backup.failed`; existing retry behavior in `BackupService`.
|
|
||||||
|
|
||||||
## 12. Testing
|
|
||||||
|
|
||||||
### 12.1 New unit tests
|
|
||||||
|
|
||||||
- `tests/test_auth.py` (~12 tests)
|
|
||||||
- `test_create_user_with_argon2_hash`
|
|
||||||
- `test_login_with_correct_password_returns_session`
|
|
||||||
- `test_login_with_wrong_password_increments_failed_count`
|
|
||||||
- `test_login_locks_account_after_5_failures`
|
|
||||||
- `test_login_lock_clears_after_15min`
|
|
||||||
- `test_session_cookie_is_httponly_secure_samesite_lax`
|
|
||||||
- `test_session_expires_after_sliding_window`
|
|
||||||
- `test_session_absolute_lifetime_enforced`
|
|
||||||
- `test_logout_deletes_session_and_clears_cookie`
|
|
||||||
- `test_change_password_invalidates_other_sessions`
|
|
||||||
- `test_require_role_rejects_below_minimum`
|
|
||||||
- `test_require_role_returns_user_object_to_dependency`
|
|
||||||
- `tests/test_users.py` (~6 tests)
|
|
||||||
- `test_admin_can_create_user`
|
|
||||||
- `test_operator_cannot_create_user`
|
|
||||||
- `test_admin_can_change_user_role`
|
|
||||||
- `test_admin_can_disable_user`
|
|
||||||
- `test_disabled_user_cannot_login`
|
|
||||||
- `test_audit_event_for_user_lifecycle`
|
|
||||||
|
|
||||||
Total new backend tests: **~18**.
|
|
||||||
|
|
||||||
### 12.2 Docker smoke tests
|
|
||||||
|
|
||||||
- `tests/test_docker.py` (~4 tests)
|
|
||||||
- `test_compose_config_validates`
|
|
||||||
- `test_dockerfile_backend_builds`
|
|
||||||
- `test_dockerfile_frontend_builds`
|
|
||||||
- `test_compose_up_brings_up_healthy_stack` (uses `testcontainers-python` or skips if not available; gated on a `DOCKER_TESTS=1` env var)
|
|
||||||
|
|
||||||
### 12.3 Smoke script
|
|
||||||
|
|
||||||
`scripts/smoke.sh`:
|
|
||||||
```bash
|
|
||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# 1. Bring up stack
|
|
||||||
docker compose up -d --build
|
|
||||||
|
|
||||||
# 2. Wait for healthcheck
|
|
||||||
for i in {1..30}; do
|
|
||||||
if curl -fs http://localhost:8080/api/healthz > /dev/null; then break; fi
|
|
||||||
sleep 2
|
|
||||||
done
|
|
||||||
|
|
||||||
# 3. Login
|
|
||||||
COOKIE_JAR=$(mktemp)
|
|
||||||
curl -fs -c "$COOKIE_JAR" -X POST http://localhost:8080/api/auth/login \
|
|
||||||
-H 'Content-Type: application/json' \
|
|
||||||
-d "{\"username\":\"admin\",\"password\":\"$(cat /etc/cyclone/secrets/admin_pw)\"}"
|
|
||||||
|
|
||||||
# 4. Parse a sample 837
|
|
||||||
curl -fs -b "$COOKIE_JAR" -X POST http://localhost:8080/api/parse-837 \
|
|
||||||
-F "file=@docs/goodclaim.x12"
|
|
||||||
|
|
||||||
# 5. List batches
|
|
||||||
curl -fs -b "$COOKIE_JAR" http://localhost:8080/api/batches
|
|
||||||
|
|
||||||
# 6. Export
|
|
||||||
curl -fs -b "$COOKIE_JAR" -X POST http://localhost:8080/api/batches/<id>/export-837 \
|
|
||||||
-H 'Content-Type: application/json' -d '{"claim_ids":["..."]}' \
|
|
||||||
-o /tmp/export.zip
|
|
||||||
|
|
||||||
unzip -l /tmp/export.zip | grep -E '\.x12$'
|
|
||||||
|
|
||||||
echo "smoke: OK"
|
|
||||||
```
|
|
||||||
|
|
||||||
### 12.4 Existing tests
|
|
||||||
|
|
||||||
- All existing backend tests must continue to pass
|
|
||||||
- All existing frontend tests must continue to pass
|
|
||||||
- Pre-existing prodfile smoke failures (rate-limit / orphan-set refs on 999/TA1) remain pre-existing and out of scope
|
|
||||||
|
|
||||||
## 13. Migration / rollout
|
|
||||||
|
|
||||||
### 13.1 First-time deploy
|
|
||||||
|
|
||||||
For a brand-new install on a fresh Ubuntu 22.04 server, follow §9.5 above. No data to migrate — this is the first deployment.
|
|
||||||
|
|
||||||
### 13.2 Upgrading an existing dev install
|
|
||||||
|
|
||||||
If an operator has been running `python -m cyclone serve` locally with the in-memory store or a plaintext SQLite file, the upgrade path is:
|
|
||||||
|
|
||||||
1. **Stop the existing server.**
|
|
||||||
2. **Export any production data:** SQLite is at `~/.local/share/cyclone/cyclone.db`. The operator takes a final plaintext backup *before* the SQLCipher transition; we don't try to encrypt-in-place (would need a different migration).
|
|
||||||
3. **Run `scripts/cyclone-init.sh`** to generate secrets.
|
|
||||||
4. **Run `docker compose up -d`** — the backend starts with an empty SQLCipher DB. Migrations run.
|
|
||||||
5. **Bootstrap admin user** via the CLI in the container.
|
|
||||||
6. **Re-import data** from the plaintext export (manual step; documented in RUNBOOK).
|
|
||||||
|
|
||||||
The in-memory store from the parent spec is *lost* on upgrade (by design — local-only). The plaintext SQLite DB *can* be carried over manually if needed.
|
|
||||||
|
|
||||||
### 13.3 Updating the running stack
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /opt/cyclone
|
|
||||||
docker compose pull # or: docker compose build
|
|
||||||
docker compose up -d # recreates containers
|
|
||||||
docker compose logs -f backend | head -200 # verify migrations + healthcheck
|
|
||||||
```
|
|
||||||
|
|
||||||
DB migrations run automatically on backend start; they're forward-only. To roll back the code (not the schema): `TAG=0.0.9 docker compose up -d`.
|
|
||||||
|
|
||||||
## 14. Out of scope (explicit)
|
|
||||||
|
|
||||||
- Real SFTP wire (paramiko)
|
|
||||||
- Public TLS / domain / Caddy reverse proxy in compose
|
|
||||||
- Multi-host HA / DB replication / load balancing
|
|
||||||
- Prometheus / Grafana / alerting beyond the simple email cron
|
|
||||||
- Off-box automated backup shipping (operator's cron)
|
|
||||||
- 2FA / SSO / OAuth
|
|
||||||
- Self-service password reset
|
|
||||||
- Per-file encryption of prodfiles / sftp_staging
|
|
||||||
- LUKS full-disk encryption
|
|
||||||
- Watchtower / automatic updates
|
|
||||||
- Linting/pre-commit/dev tooling polish
|
|
||||||
- Component / E2E browser tests (Playwright)
|
|
||||||
|
|
||||||
## 15. Acceptance checklist
|
|
||||||
|
|
||||||
### 15.1 Backend
|
|
||||||
|
|
||||||
- [ ] `pytest tests/test_auth.py tests/test_users.py tests/test_docker.py -v` passes
|
|
||||||
- [ ] All previously-passing tests still pass; new total ≥ previous + 18
|
|
||||||
- [ ] `docker compose config` validates with no errors
|
|
||||||
- [ ] `docker compose build` succeeds for both services
|
|
||||||
- [ ] `docker compose up -d` brings both containers to `healthy` within 60s
|
|
||||||
- [ ] `curl http://localhost:8080/api/healthz` returns `{db_ok: true, scheduler_running: true}`
|
|
||||||
|
|
||||||
### 15.2 Auth + RBAC
|
|
||||||
|
|
||||||
- [ ] `POST /api/auth/login` with correct creds sets `cyclone_session` cookie (HttpOnly, Secure, SameSite=Lax) and returns `{user: {...}}`
|
|
||||||
- [ ] 5 wrong logins within 15min locks the account; 423 returned
|
|
||||||
- [ ] `GET /api/batches` without session cookie returns 401
|
|
||||||
- [ ] `GET /api/admin/users` as operator returns 403; as admin returns list
|
|
||||||
- [ ] Logout clears the cookie and invalidates the session in the DB
|
|
||||||
- [ ] Password change invalidates other sessions for that user
|
|
||||||
|
|
||||||
### 15.3 Encryption + backups
|
|
||||||
|
|
||||||
- [ ] `cyclone.db_crypto.is_encryption_enabled()` returns True in container
|
|
||||||
- [ ] Inspecting the DB file on disk shows binary ciphertext, not plaintext
|
|
||||||
- [ ] `POST /api/admin/backup/create` creates an encrypted `.bin` in `/var/lib/cyclone/backups/`
|
|
||||||
- [ ] Backup scheduler runs every 24h (`docker compose logs backend | grep backup-scheduler`)
|
|
||||||
- [ ] Restore flow works end-to-end: create backup → wipe DB → restore → verify data present
|
|
||||||
|
|
||||||
### 15.4 Frontend
|
|
||||||
|
|
||||||
- [ ] Unauthenticated browser request to `/` redirects to `/login`
|
|
||||||
- [ ] Login as admin → land on `/`; NavBar shows username + role
|
|
||||||
- [ ] Operator does not see "Admin" link in NavBar
|
|
||||||
- [ ] `/admin/users` as operator shows "Forbidden" page
|
|
||||||
- [ ] Logout button clears session and returns to `/login`
|
|
||||||
- [ ] `npm run build` produces a `dist/` that serves correctly from nginx
|
|
||||||
|
|
||||||
### 15.5 Smoke
|
|
||||||
|
|
||||||
- [ ] `scripts/smoke.sh` passes end-to-end against a fresh stack
|
|
||||||
- [ ] `RUNBOOK.md` exists and contains the §9.6 procedures
|
|
||||||
- [ ] `scripts/post-deploy.sh` sets up logrotate + healthcheck cron without errors
|
|
||||||
Generated
-62
@@ -12,7 +12,6 @@
|
|||||||
"@radix-ui/react-label": "^2.1.0",
|
"@radix-ui/react-label": "^2.1.0",
|
||||||
"@radix-ui/react-select": "^2.1.2",
|
"@radix-ui/react-select": "^2.1.2",
|
||||||
"@radix-ui/react-slot": "^1.1.0",
|
"@radix-ui/react-slot": "^1.1.0",
|
||||||
"@radix-ui/react-tabs": "^1.1.15",
|
|
||||||
"@tanstack/react-query": "^5.101.0",
|
"@tanstack/react-query": "^5.101.0",
|
||||||
"ansi-styles": "^6.2.3",
|
"ansi-styles": "^6.2.3",
|
||||||
"class-variance-authority": "^0.7.0",
|
"class-variance-authority": "^0.7.0",
|
||||||
@@ -1332,37 +1331,6 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/@radix-ui/react-roving-focus": {
|
|
||||||
"version": "1.1.13",
|
|
||||||
"resolved": "https://registry.npmjs.org/@radix-ui/react-roving-focus/-/react-roving-focus-1.1.13.tgz",
|
|
||||||
"integrity": "sha512-9gkwneI0guf8JDmrFxPjJF6Ozzgioyw+/lonYNCwefS9ZHA05er0BVHiXr+LbWGHxUfczvMY6G1oiZZi1VzjRw==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"@radix-ui/primitive": "1.1.4",
|
|
||||||
"@radix-ui/react-collection": "1.1.10",
|
|
||||||
"@radix-ui/react-compose-refs": "1.1.3",
|
|
||||||
"@radix-ui/react-context": "1.1.4",
|
|
||||||
"@radix-ui/react-direction": "1.1.2",
|
|
||||||
"@radix-ui/react-id": "1.1.2",
|
|
||||||
"@radix-ui/react-primitive": "2.1.6",
|
|
||||||
"@radix-ui/react-use-callback-ref": "1.1.2",
|
|
||||||
"@radix-ui/react-use-controllable-state": "1.2.3"
|
|
||||||
},
|
|
||||||
"peerDependencies": {
|
|
||||||
"@types/react": "*",
|
|
||||||
"@types/react-dom": "*",
|
|
||||||
"react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc",
|
|
||||||
"react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc"
|
|
||||||
},
|
|
||||||
"peerDependenciesMeta": {
|
|
||||||
"@types/react": {
|
|
||||||
"optional": true
|
|
||||||
},
|
|
||||||
"@types/react-dom": {
|
|
||||||
"optional": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/@radix-ui/react-select": {
|
"node_modules/@radix-ui/react-select": {
|
||||||
"version": "2.3.1",
|
"version": "2.3.1",
|
||||||
"resolved": "https://registry.npmjs.org/@radix-ui/react-select/-/react-select-2.3.1.tgz",
|
"resolved": "https://registry.npmjs.org/@radix-ui/react-select/-/react-select-2.3.1.tgz",
|
||||||
@@ -1425,36 +1393,6 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/@radix-ui/react-tabs": {
|
|
||||||
"version": "1.1.15",
|
|
||||||
"resolved": "https://registry.npmjs.org/@radix-ui/react-tabs/-/react-tabs-1.1.15.tgz",
|
|
||||||
"integrity": "sha512-kxc9gI6/HfcU4nfMMVS3AmQK414kbU1IE6UCJmMmxjhO3cRPXOyYnmvyKD+ODt7q56nRq9l7Wovi6uaGwKgMlg==",
|
|
||||||
"license": "MIT",
|
|
||||||
"dependencies": {
|
|
||||||
"@radix-ui/primitive": "1.1.4",
|
|
||||||
"@radix-ui/react-context": "1.1.4",
|
|
||||||
"@radix-ui/react-direction": "1.1.2",
|
|
||||||
"@radix-ui/react-id": "1.1.2",
|
|
||||||
"@radix-ui/react-presence": "1.1.6",
|
|
||||||
"@radix-ui/react-primitive": "2.1.6",
|
|
||||||
"@radix-ui/react-roving-focus": "1.1.13",
|
|
||||||
"@radix-ui/react-use-controllable-state": "1.2.3"
|
|
||||||
},
|
|
||||||
"peerDependencies": {
|
|
||||||
"@types/react": "*",
|
|
||||||
"@types/react-dom": "*",
|
|
||||||
"react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc",
|
|
||||||
"react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc"
|
|
||||||
},
|
|
||||||
"peerDependenciesMeta": {
|
|
||||||
"@types/react": {
|
|
||||||
"optional": true
|
|
||||||
},
|
|
||||||
"@types/react-dom": {
|
|
||||||
"optional": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"node_modules/@radix-ui/react-use-callback-ref": {
|
"node_modules/@radix-ui/react-use-callback-ref": {
|
||||||
"version": "1.1.2",
|
"version": "1.1.2",
|
||||||
"resolved": "https://registry.npmjs.org/@radix-ui/react-use-callback-ref/-/react-use-callback-ref-1.1.2.tgz",
|
"resolved": "https://registry.npmjs.org/@radix-ui/react-use-callback-ref/-/react-use-callback-ref-1.1.2.tgz",
|
||||||
|
|||||||
@@ -17,7 +17,6 @@
|
|||||||
"@radix-ui/react-label": "^2.1.0",
|
"@radix-ui/react-label": "^2.1.0",
|
||||||
"@radix-ui/react-select": "^2.1.2",
|
"@radix-ui/react-select": "^2.1.2",
|
||||||
"@radix-ui/react-slot": "^1.1.0",
|
"@radix-ui/react-slot": "^1.1.0",
|
||||||
"@radix-ui/react-tabs": "^1.1.15",
|
|
||||||
"@tanstack/react-query": "^5.101.0",
|
"@tanstack/react-query": "^5.101.0",
|
||||||
"ansi-styles": "^6.2.3",
|
"ansi-styles": "^6.2.3",
|
||||||
"class-variance-authority": "^0.7.0",
|
"class-variance-authority": "^0.7.0",
|
||||||
|
|||||||
+2
-3
@@ -1,7 +1,6 @@
|
|||||||
import { Route, Routes } from "react-router-dom";
|
import { Route, Routes } from "react-router-dom";
|
||||||
import { Toaster } from "sonner";
|
import { Toaster } from "sonner";
|
||||||
import { Layout } from "@/components/Layout";
|
import { Layout } from "@/components/Layout";
|
||||||
import { DrillStackProvider } from "@/components/drill/DrillStackProvider";
|
|
||||||
import { Dashboard } from "@/pages/Dashboard";
|
import { Dashboard } from "@/pages/Dashboard";
|
||||||
import { Claims } from "@/pages/Claims";
|
import { Claims } from "@/pages/Claims";
|
||||||
import { Remittances } from "@/pages/Remittances";
|
import { Remittances } from "@/pages/Remittances";
|
||||||
@@ -25,7 +24,7 @@ function NotFound() {
|
|||||||
|
|
||||||
export default function App() {
|
export default function App() {
|
||||||
return (
|
return (
|
||||||
<DrillStackProvider>
|
<>
|
||||||
<Routes>
|
<Routes>
|
||||||
<Route element={<Layout />}>
|
<Route element={<Layout />}>
|
||||||
<Route index element={<Dashboard />} />
|
<Route index element={<Dashboard />} />
|
||||||
@@ -53,6 +52,6 @@ export default function App() {
|
|||||||
},
|
},
|
||||||
}}
|
}}
|
||||||
/>
|
/>
|
||||||
</DrillStackProvider>
|
</>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,192 +0,0 @@
|
|||||||
// @vitest-environment happy-dom
|
|
||||||
// AckDrawer wires `useAckDetail` (TanStack Query) and renders a Radix
|
|
||||||
// Dialog portal — both need an act-aware, DOM-backed environment or
|
|
||||||
// React logs warnings and the portal can't mount.
|
|
||||||
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
|
||||||
|
|
||||||
import { afterEach, describe, it, expect, vi } from "vitest";
|
|
||||||
import { cleanup, render } from "@testing-library/react";
|
|
||||||
import { ApiError } from "@/lib/api";
|
|
||||||
import { AckDrawer } from "@/components/AckDrawer";
|
|
||||||
import type { Ack } from "@/types";
|
|
||||||
|
|
||||||
// Mock the hook BEFORE the import above is resolved (vitest hoists
|
|
||||||
// `vi.mock` to the top of the file regardless of where it appears
|
|
||||||
// syntactically). Mocking the hook directly — rather than mocking
|
|
||||||
// `api.getAck` — lets each test pin the hook's exact return shape
|
|
||||||
// without standing up a real `QueryClient`.
|
|
||||||
const { useAckDetail } = vi.hoisted(() => ({
|
|
||||||
useAckDetail: vi.fn(),
|
|
||||||
}));
|
|
||||||
vi.mock("@/hooks/useAckDetail", () => ({
|
|
||||||
useAckDetail,
|
|
||||||
}));
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Minimal valid `Ack` fixture — every required key present so the
|
|
||||||
* component typechecks. The wire shape extends with `raw_999_text`
|
|
||||||
* (and `rawJson`), per `useAckDetail`'s `AckDetail` type — populated
|
|
||||||
* when the backend serves it; absent on older rows.
|
|
||||||
*/
|
|
||||||
const SAMPLE_ACK: Ack & { raw_999_text: string } = {
|
|
||||||
id: 42,
|
|
||||||
sourceBatchId: "b-uuid-1",
|
|
||||||
acceptedCount: 3,
|
|
||||||
rejectedCount: 1,
|
|
||||||
receivedCount: 4,
|
|
||||||
ackCode: "P",
|
|
||||||
parsedAt: "2026-06-20T12:00:00Z",
|
|
||||||
raw_999_text: "ISA*...*~\nGS*...*~\nST*999*0001~",
|
|
||||||
};
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Configure the mocked hook's return value for a single test. The
|
|
||||||
* `refetch` default is a fresh `vi.fn()` — tests that need to assert
|
|
||||||
* on it can override via `overrides.refetch`.
|
|
||||||
*/
|
|
||||||
function mockDetail(
|
|
||||||
overrides: Partial<{
|
|
||||||
data: (Ack & { raw_999_text?: string }) | null;
|
|
||||||
isLoading: boolean;
|
|
||||||
isError: boolean;
|
|
||||||
error: Error | null;
|
|
||||||
refetch: () => void;
|
|
||||||
}> = {}
|
|
||||||
) {
|
|
||||||
useAckDetail.mockReturnValue({
|
|
||||||
data: null,
|
|
||||||
isLoading: false,
|
|
||||||
isError: false,
|
|
||||||
error: null,
|
|
||||||
refetch: vi.fn(),
|
|
||||||
...overrides,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// happy-dom keeps `document.body` between tests; without cleanup,
|
|
||||||
// `screen.getByText(...)` would find nodes from earlier renders.
|
|
||||||
afterEach(() => {
|
|
||||||
cleanup();
|
|
||||||
vi.clearAllMocks();
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("AckDrawer", () => {
|
|
||||||
it("test_renders_nothing_when_ackId_is_null", () => {
|
|
||||||
mockDetail({ data: null });
|
|
||||||
render(<AckDrawer ackId={null} onClose={() => {}} />);
|
|
||||||
|
|
||||||
// No ack content should be in the document when the drawer is
|
|
||||||
// closed — Radix's Dialog gates the portal on `open`.
|
|
||||||
expect(document.body.textContent).not.toContain("b-uuid-1");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_calls_useAckDetail_with_ackId", () => {
|
|
||||||
mockDetail({ data: SAMPLE_ACK });
|
|
||||||
render(<AckDrawer ackId="42" onClose={() => {}} />);
|
|
||||||
|
|
||||||
expect(useAckDetail).toHaveBeenCalledWith("42");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_renders_ack_summary_on_success", () => {
|
|
||||||
mockDetail({ data: SAMPLE_ACK });
|
|
||||||
render(<AckDrawer ackId="42" onClose={() => {}} />);
|
|
||||||
|
|
||||||
// Header shows the source batch id as the title.
|
|
||||||
expect(document.body.textContent).toContain("b-uuid-1");
|
|
||||||
// Counts (3 accepted, 1 rejected, 4 received) are surfaced as
|
|
||||||
// StatTile values.
|
|
||||||
expect(document.body.textContent).toContain("3");
|
|
||||||
expect(document.body.textContent).toContain("1");
|
|
||||||
expect(document.body.textContent).toContain("4");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_renders_ack_code_pill_with_human_label", () => {
|
|
||||||
mockDetail({ data: { ...SAMPLE_ACK, ackCode: "P" } });
|
|
||||||
render(<AckDrawer ackId="42" onClose={() => {}} />);
|
|
||||||
|
|
||||||
// The pill renders a human label, not just the bare code letter.
|
|
||||||
expect(document.body.textContent).toContain("Partially accepted");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_renders_skeleton_while_loading", () => {
|
|
||||||
mockDetail({ isLoading: true });
|
|
||||||
render(<AckDrawer ackId="42" onClose={() => {}} />);
|
|
||||||
|
|
||||||
// The `Skeleton` primitive sets `aria-busy="true"` — a stable
|
|
||||||
// hook for the loading state.
|
|
||||||
expect(document.querySelectorAll('[aria-busy="true"]').length).toBeGreaterThan(0);
|
|
||||||
// And the ack id should NOT have leaked in yet.
|
|
||||||
expect(document.body.textContent).not.toContain("b-uuid-1");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_renders_not_found_error_on_404", () => {
|
|
||||||
mockDetail({
|
|
||||||
isError: true,
|
|
||||||
error: new ApiError(404, "Ack ghost not found"),
|
|
||||||
});
|
|
||||||
render(<AckDrawer ackId="9999" onClose={() => {}} />);
|
|
||||||
|
|
||||||
const errEl = document.querySelector('[data-testid="ack-drawer-error-not_found"]');
|
|
||||||
expect(errEl).not.toBeNull();
|
|
||||||
// Body should mention "doesn't exist" — the not_found COPY key.
|
|
||||||
expect(errEl?.textContent).toContain("doesn't exist");
|
|
||||||
// And the retry button should NOT be present (not_found has no
|
|
||||||
// retry affordance — retrying a 404 won't help).
|
|
||||||
expect(document.querySelector('[data-testid="error-retry"]')).toBeNull();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_renders_network_error_with_retry", () => {
|
|
||||||
mockDetail({ isError: true, error: new Error("network down") });
|
|
||||||
render(<AckDrawer ackId="42" onClose={() => {}} />);
|
|
||||||
|
|
||||||
const errEl = document.querySelector('[data-testid="ack-drawer-error-network"]');
|
|
||||||
expect(errEl).not.toBeNull();
|
|
||||||
expect(errEl?.textContent).toContain("Couldn't reach the server");
|
|
||||||
|
|
||||||
// Network variant shows a Retry button.
|
|
||||||
const retryBtn = document.querySelector(
|
|
||||||
'[data-testid="error-retry"]'
|
|
||||||
) as HTMLButtonElement | null;
|
|
||||||
expect(retryBtn).not.toBeNull();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_close_button_calls_onClose", () => {
|
|
||||||
const onClose = vi.fn<() => void>();
|
|
||||||
mockDetail({ isError: true, error: new Error("network down") });
|
|
||||||
render(<AckDrawer ackId="42" onClose={onClose} />);
|
|
||||||
|
|
||||||
const closeBtn = document.querySelector(
|
|
||||||
'[data-testid="error-close"]'
|
|
||||||
) as HTMLButtonElement | null;
|
|
||||||
expect(closeBtn).not.toBeNull();
|
|
||||||
closeBtn!.click();
|
|
||||||
expect(onClose).toHaveBeenCalledTimes(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_renders_download_button_when_raw_999_text_present", () => {
|
|
||||||
mockDetail({ data: SAMPLE_ACK });
|
|
||||||
render(<AckDrawer ackId="42" onClose={() => {}} />);
|
|
||||||
|
|
||||||
// The header action slot populates with the Download 999 button
|
|
||||||
// only when the ack detail carries raw_999_text.
|
|
||||||
const dlBtn = document.querySelector('[data-testid="ack-drawer-download"]');
|
|
||||||
expect(dlBtn).not.toBeNull();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("test_omits_download_button_when_raw_999_text_absent", () => {
|
|
||||||
const data: Ack = {
|
|
||||||
id: 42,
|
|
||||||
sourceBatchId: "b-uuid-1",
|
|
||||||
acceptedCount: 3,
|
|
||||||
rejectedCount: 1,
|
|
||||||
receivedCount: 4,
|
|
||||||
ackCode: "A",
|
|
||||||
parsedAt: "2026-06-20T12:00:00Z",
|
|
||||||
};
|
|
||||||
mockDetail({ data });
|
|
||||||
render(<AckDrawer ackId="42" onClose={() => {}} />);
|
|
||||||
|
|
||||||
// No raw_999_text → no Download button.
|
|
||||||
expect(document.querySelector('[data-testid="ack-drawer-download"]')).toBeNull();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -1,324 +0,0 @@
|
|||||||
import { useCallback, useState } from "react";
|
|
||||||
import { Download } from "lucide-react";
|
|
||||||
import { Dialog, DialogContent } from "@/components/ui/dialog";
|
|
||||||
import { ApiError } from "@/lib/api";
|
|
||||||
import { DrillDrawerHeader } from "@/components/drill/DrillDrawerHeader";
|
|
||||||
import { Skeleton } from "@/components/ui/skeleton";
|
|
||||||
import { cn } from "@/lib/utils";
|
|
||||||
import { useAckDetail, type AckDetail } from "@/hooks/useAckDetail";
|
|
||||||
import { SegmentStatusList } from "./SegmentStatusList";
|
|
||||||
|
|
||||||
interface Props {
|
|
||||||
/**
|
|
||||||
* Currently-open ack id (string), or `null` when the drawer is
|
|
||||||
* closed. The URL stores ids as strings so deep links round-trip
|
|
||||||
* cleanly; the hook does the `Number()` coercion before calling
|
|
||||||
* `api.getAck`.
|
|
||||||
*/
|
|
||||||
ackId: string | null;
|
|
||||||
/** Fired when the user dismisses the drawer (X button, Escape, etc.). */
|
|
||||||
onClose: () => void;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Roll a hex code into a "kind" so the drawer's error branch can
|
|
||||||
* pick the right copy. Mirrors `ProviderDrawer`'s `errorKind`
|
|
||||||
* computation.
|
|
||||||
*/
|
|
||||||
type ErrorKind = "not_found" | "network";
|
|
||||||
|
|
||||||
function AckDrawerError({
|
|
||||||
kind,
|
|
||||||
onRetry,
|
|
||||||
onClose,
|
|
||||||
}: {
|
|
||||||
kind: ErrorKind;
|
|
||||||
onRetry?: () => void;
|
|
||||||
onClose: () => void;
|
|
||||||
}) {
|
|
||||||
const COPY = {
|
|
||||||
not_found: {
|
|
||||||
eyebrow: "NOT FOUND",
|
|
||||||
message: "This 999 ACK doesn't exist or has been removed.",
|
|
||||||
},
|
|
||||||
network: {
|
|
||||||
eyebrow: "CONNECTION",
|
|
||||||
message:
|
|
||||||
"Couldn't reach the server. Check your connection and try again.",
|
|
||||||
},
|
|
||||||
} as const;
|
|
||||||
const { eyebrow, message } = COPY[kind];
|
|
||||||
return (
|
|
||||||
<div
|
|
||||||
className="flex h-full flex-col"
|
|
||||||
role="alert"
|
|
||||||
data-testid={`ack-drawer-error-${kind}`}
|
|
||||||
>
|
|
||||||
<DrillDrawerHeader eyebrow="999 ACK" title="—" onClose={onClose} />
|
|
||||||
<div className="flex flex-1 flex-col items-start gap-4 px-6 py-6">
|
|
||||||
<span className="text-[10.5px] font-semibold uppercase tracking-[0.18em] text-destructive">
|
|
||||||
{eyebrow}
|
|
||||||
</span>
|
|
||||||
<p className="text-sm text-muted-foreground max-w-sm">{message}</p>
|
|
||||||
<div className="flex items-center gap-2">
|
|
||||||
{kind === "network" && onRetry ? (
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
onClick={onRetry}
|
|
||||||
data-testid="error-retry"
|
|
||||||
className="rounded-md border px-2.5 py-1 text-[12.5px] font-medium hover:bg-muted"
|
|
||||||
>
|
|
||||||
Retry
|
|
||||||
</button>
|
|
||||||
) : null}
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
onClick={onClose}
|
|
||||||
data-testid="error-close"
|
|
||||||
className="rounded-md border px-2.5 py-1 text-[12.5px] font-medium hover:bg-muted"
|
|
||||||
>
|
|
||||||
Close
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Inline ack code pill — same color palette as `AcksPage`
|
|
||||||
* (`Acks.tsx`) so the badge reads the same in both surfaces.
|
|
||||||
*/
|
|
||||||
function AckCodePill({ code }: { code: AckDetail["ackCode"] }) {
|
|
||||||
const cfg =
|
|
||||||
code === "A"
|
|
||||||
? {
|
|
||||||
fg: "hsl(152 64% 30%)",
|
|
||||||
bg: "hsl(152 50% 88%)",
|
|
||||||
border: "hsl(152 64% 38% / 0.30)",
|
|
||||||
label: "Accepted",
|
|
||||||
}
|
|
||||||
: code === "R"
|
|
||||||
? {
|
|
||||||
fg: "hsl(358 70% 36%)",
|
|
||||||
bg: "hsl(358 70% 92%)",
|
|
||||||
border: "hsl(358 70% 50% / 0.30)",
|
|
||||||
label: "Rejected",
|
|
||||||
}
|
|
||||||
: code === "P"
|
|
||||||
? {
|
|
||||||
fg: "hsl(36 92% 30%)",
|
|
||||||
bg: "hsl(36 82% 88%)",
|
|
||||||
border: "hsl(36 92% 50% / 0.30)",
|
|
||||||
label: "Partially accepted",
|
|
||||||
}
|
|
||||||
: {
|
|
||||||
fg: "hsl(36 92% 30%)",
|
|
||||||
bg: "hsl(36 82% 88%)",
|
|
||||||
border: "hsl(36 92% 50% / 0.30)",
|
|
||||||
label: "Accepted w/ errors",
|
|
||||||
};
|
|
||||||
return (
|
|
||||||
<span
|
|
||||||
className="inline-flex items-center rounded-sm border px-2 py-0.5 mono text-[10.5px] font-semibold uppercase tracking-[0.14em]"
|
|
||||||
style={{ color: cfg.fg, backgroundColor: cfg.bg, borderColor: cfg.border }}
|
|
||||||
>
|
|
||||||
{cfg.label}
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function StatTile({
|
|
||||||
label,
|
|
||||||
value,
|
|
||||||
tone,
|
|
||||||
}: {
|
|
||||||
label: string;
|
|
||||||
value: number | string;
|
|
||||||
tone: "success" | "destructive" | "ink";
|
|
||||||
}) {
|
|
||||||
const color =
|
|
||||||
tone === "success"
|
|
||||||
? "hsl(152 64% 30%)"
|
|
||||||
: tone === "destructive"
|
|
||||||
? "hsl(358 70% 36%)"
|
|
||||||
: "hsl(var(--foreground))";
|
|
||||||
return (
|
|
||||||
<div
|
|
||||||
className="rounded-lg border px-3.5 py-3"
|
|
||||||
style={{
|
|
||||||
backgroundColor: "hsl(var(--card) / 0.4)",
|
|
||||||
borderColor: "hsl(var(--border) / 0.5)",
|
|
||||||
}}
|
|
||||||
data-testid="ack-stat"
|
|
||||||
>
|
|
||||||
<div className="text-[10px] font-semibold uppercase tracking-[0.18em] text-muted-foreground">
|
|
||||||
{label}
|
|
||||||
</div>
|
|
||||||
<div
|
|
||||||
className="display mono tabular-nums mt-1"
|
|
||||||
style={{ color, fontSize: 22, lineHeight: 1.1 }}
|
|
||||||
>
|
|
||||||
{value}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 999 ACK drill-down drawer (SP21 Phase 5 Task 5.2).
|
|
||||||
*
|
|
||||||
* Mirror of `ProviderDrawer` — same right-anchored side-panel shell,
|
|
||||||
* same `errorKind` + `useAckDetail` shape (404 vs network), same
|
|
||||||
* skeleton-first loading state. The body is slim: rolled-up counts
|
|
||||||
* at the top, the ack code pill, the source batch id, and a
|
|
||||||
* per-segment status list. The header carries a "Download 999"
|
|
||||||
* action so the user can grab the original X12 file from the drawer
|
|
||||||
* without a second round-trip to `/api/acks/{id}/raw`.
|
|
||||||
*
|
|
||||||
* Layout mirrors `ProviderDrawer`/`ClaimDrawer`: Radix Dialog
|
|
||||||
* repositioned to the right edge as a fixed-height side panel, with
|
|
||||||
* the shared `DrillDrawerHeader` on top and a scrollable body below.
|
|
||||||
*
|
|
||||||
* Error branching:
|
|
||||||
* - `ApiError(404)` → "not_found" (no retry, the ack is gone)
|
|
||||||
* - anything else → "network" (retry available)
|
|
||||||
*/
|
|
||||||
export function AckDrawer({ ackId, onClose }: Props) {
|
|
||||||
const { data, isLoading, isError, error, refetch } = useAckDetail(ackId);
|
|
||||||
|
|
||||||
const errorKind: ErrorKind | null = isError
|
|
||||||
? error instanceof ApiError && error.status === 404
|
|
||||||
? "not_found"
|
|
||||||
: "network"
|
|
||||||
: null;
|
|
||||||
|
|
||||||
// Download affordance for the regenerated 999 X12 text. Lives in
|
|
||||||
// the drawer (not the page) so a deep-linked user can grab the file
|
|
||||||
// without bouncing back to /acks. `raw_999_text` may be empty for
|
|
||||||
// older rows without the field — we silently no-op rather than
|
|
||||||
// surfacing an error toast (the spec calls this a "low stakes"
|
|
||||||
// affordance).
|
|
||||||
const [downloading, setDownloading] = useState(false);
|
|
||||||
const onDownload = useCallback(async () => {
|
|
||||||
if (!data) return;
|
|
||||||
const raw =
|
|
||||||
(data as AckDetail & { raw_999_text?: string }).raw_999_text ?? "";
|
|
||||||
if (!raw || downloading) return;
|
|
||||||
setDownloading(true);
|
|
||||||
try {
|
|
||||||
const blob = new Blob([raw], { type: "text/plain" });
|
|
||||||
const url = URL.createObjectURL(blob);
|
|
||||||
const a = document.createElement("a");
|
|
||||||
a.href = url;
|
|
||||||
a.download = `ack-${data.sourceBatchId}.999`;
|
|
||||||
a.click();
|
|
||||||
URL.revokeObjectURL(url);
|
|
||||||
} finally {
|
|
||||||
setDownloading(false);
|
|
||||||
}
|
|
||||||
}, [data, downloading]);
|
|
||||||
|
|
||||||
const downloadAction =
|
|
||||||
data && (data as AckDetail & { raw_999_text?: string }).raw_999_text ? (
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
onClick={onDownload}
|
|
||||||
disabled={downloading}
|
|
||||||
aria-label="Download 999 file"
|
|
||||||
title="Download 999 file"
|
|
||||||
data-testid="ack-drawer-download"
|
|
||||||
className={cn(
|
|
||||||
"inline-flex items-center gap-1.5 rounded-md border px-2 py-1 text-[11px] font-medium uppercase tracking-[0.14em] mono transition-colors",
|
|
||||||
downloading && "opacity-50 cursor-not-allowed",
|
|
||||||
)}
|
|
||||||
style={{
|
|
||||||
borderColor: "hsl(var(--border) / 0.7)",
|
|
||||||
color: "hsl(var(--foreground))",
|
|
||||||
backgroundColor: "hsl(var(--card) / 0.5)",
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<Download className="h-3.5 w-3.5" strokeWidth={1.75} aria-hidden />
|
|
||||||
999
|
|
||||||
</button>
|
|
||||||
) : null;
|
|
||||||
|
|
||||||
return (
|
|
||||||
<Dialog open={ackId !== null} onOpenChange={(o) => { if (!o) onClose(); }}>
|
|
||||||
<DialogContent
|
|
||||||
className="fixed right-0 top-0 flex h-full w-full max-w-2xl flex-col translate-x-0 translate-y-0 rounded-none border-l border-border bg-card p-0"
|
|
||||||
aria-describedby={undefined}
|
|
||||||
data-testid="ack-drawer"
|
|
||||||
>
|
|
||||||
{errorKind ? (
|
|
||||||
<AckDrawerError
|
|
||||||
kind={errorKind}
|
|
||||||
onRetry={() => {
|
|
||||||
void refetch();
|
|
||||||
}}
|
|
||||||
onClose={onClose}
|
|
||||||
/>
|
|
||||||
) : isLoading || !data ? (
|
|
||||||
<div className="flex h-full flex-col overflow-y-auto">
|
|
||||||
<DrillDrawerHeader
|
|
||||||
eyebrow="999 ACK"
|
|
||||||
title="Loading…"
|
|
||||||
onClose={onClose}
|
|
||||||
/>
|
|
||||||
<div className="space-y-2 p-6">
|
|
||||||
<Skeleton variant="row" />
|
|
||||||
<Skeleton variant="row" />
|
|
||||||
<Skeleton variant="row" />
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
) : (
|
|
||||||
<div
|
|
||||||
className="flex h-full flex-col overflow-y-auto"
|
|
||||||
data-testid="ack-drawer-content"
|
|
||||||
>
|
|
||||||
<DrillDrawerHeader
|
|
||||||
eyebrow="999 ACK"
|
|
||||||
title={data.sourceBatchId}
|
|
||||||
onClose={onClose}
|
|
||||||
action={downloadAction}
|
|
||||||
/>
|
|
||||||
<div className="flex flex-col divide-y divide-border/40">
|
|
||||||
<section
|
|
||||||
className="flex flex-col gap-4 px-6 py-4"
|
|
||||||
data-testid="ack-summary"
|
|
||||||
>
|
|
||||||
<div className="flex items-center gap-3 flex-wrap">
|
|
||||||
<AckCodePill code={data.ackCode} />
|
|
||||||
<span className="mono text-[12px] text-muted-foreground">
|
|
||||||
ID {data.id}
|
|
||||||
</span>
|
|
||||||
<span className="mono text-[12px] text-muted-foreground">
|
|
||||||
· {data.parsedAt ? data.parsedAt.slice(0, 10) : "—"}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
<div className="grid grid-cols-3 gap-3">
|
|
||||||
<StatTile
|
|
||||||
label="Accepted"
|
|
||||||
value={data.acceptedCount}
|
|
||||||
tone="success"
|
|
||||||
/>
|
|
||||||
<StatTile
|
|
||||||
label="Rejected"
|
|
||||||
value={data.rejectedCount}
|
|
||||||
tone="destructive"
|
|
||||||
/>
|
|
||||||
<StatTile
|
|
||||||
label="Received"
|
|
||||||
value={data.receivedCount}
|
|
||||||
tone="ink"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
</section>
|
|
||||||
<SegmentStatusList segments={[]} />
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</DialogContent>
|
|
||||||
</Dialog>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -1,150 +0,0 @@
|
|||||||
import { CheckCircle2, AlertTriangle, Minus } from "lucide-react";
|
|
||||||
|
|
||||||
/**
|
|
||||||
* One 999 ACK segment status row. The 999 spec labels each transaction
|
|
||||||
* set with a 3-char code:
|
|
||||||
*
|
|
||||||
* "A" = Accepted
|
|
||||||
* "E" = Accepted, but errors are noted (one or more segments had
|
|
||||||
* errors; the transaction set as a whole was accepted)
|
|
||||||
* "R" = Rejected
|
|
||||||
* "P" = Partially accepted (mixed — some segments accepted, some
|
|
||||||
* not; a degraded success the operator must investigate)
|
|
||||||
*
|
|
||||||
* The wire shape's `rawJson` (set by the parser, see backend
|
|
||||||
* `cyclone.parsers.parsers_999`) carries an array of segment rows.
|
|
||||||
* Until that array is parsed into a stable UI shape, we render the
|
|
||||||
* rolled-up counts on the ack row (`acceptedCount` / `rejectedCount`)
|
|
||||||
* and a placeholder explaining how to read it.
|
|
||||||
*/
|
|
||||||
interface SegmentRow {
|
|
||||||
/** Loop / segment reference like "ST*999*0001" or "AK2*HC*0001". */
|
|
||||||
reference?: string;
|
|
||||||
/** "A" | "E" | "R" | "P". */
|
|
||||||
status: "A" | "E" | "R" | "P";
|
|
||||||
/** Free-form note from the parser (e.g. "SVC*HC:9450 missing"). */
|
|
||||||
note?: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
interface Props {
|
|
||||||
segments: SegmentRow[];
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Pill for one segment status. Color mirrors the badge palette used on
|
|
||||||
* the Acks page (`Acks.tsx`) so a status reads the same in both
|
|
||||||
* surfaces.
|
|
||||||
*/
|
|
||||||
function StatusPill({ status }: { status: SegmentRow["status"] }) {
|
|
||||||
const cfg = {
|
|
||||||
A: {
|
|
||||||
fg: "hsl(152 64% 30%)",
|
|
||||||
bg: "hsl(152 50% 88%)",
|
|
||||||
border: "hsl(152 64% 38% / 0.30)",
|
|
||||||
label: "Accepted",
|
|
||||||
Icon: CheckCircle2,
|
|
||||||
},
|
|
||||||
E: {
|
|
||||||
fg: "hsl(36 92% 30%)",
|
|
||||||
bg: "hsl(36 82% 88%)",
|
|
||||||
border: "hsl(36 92% 50% / 0.30)",
|
|
||||||
label: "Accepted w/ errors",
|
|
||||||
Icon: AlertTriangle,
|
|
||||||
},
|
|
||||||
R: {
|
|
||||||
fg: "hsl(358 70% 36%)",
|
|
||||||
bg: "hsl(358 70% 92%)",
|
|
||||||
border: "hsl(358 70% 50% / 0.30)",
|
|
||||||
label: "Rejected",
|
|
||||||
Icon: AlertTriangle,
|
|
||||||
},
|
|
||||||
P: {
|
|
||||||
fg: "hsl(36 92% 30%)",
|
|
||||||
bg: "hsl(36 82% 88%)",
|
|
||||||
border: "hsl(36 92% 50% / 0.30)",
|
|
||||||
label: "Partially accepted",
|
|
||||||
Icon: Minus,
|
|
||||||
},
|
|
||||||
}[status];
|
|
||||||
|
|
||||||
const Icon = cfg.Icon;
|
|
||||||
return (
|
|
||||||
<span
|
|
||||||
className="inline-flex items-center gap-1.5 rounded-sm border px-2 py-0.5 mono text-[10.5px] font-semibold uppercase tracking-[0.14em]"
|
|
||||||
style={{ color: cfg.fg, backgroundColor: cfg.bg, borderColor: cfg.border }}
|
|
||||||
>
|
|
||||||
<Icon className="h-3 w-3" strokeWidth={1.75} aria-hidden />
|
|
||||||
{cfg.label}
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Per-segment status list inside the AckDrawer body (SP21 Phase 5
|
|
||||||
* Task 5.2). Renders one row per parsed 999 segment, each with the
|
|
||||||
* segment reference (when present), the parsed status code, and the
|
|
||||||
* parser's note (when present). An empty list renders a small
|
|
||||||
* explanatory note so the drawer doesn't look broken on old acks
|
|
||||||
* without the per-segment slice.
|
|
||||||
*/
|
|
||||||
export function SegmentStatusList({ segments }: Props) {
|
|
||||||
return (
|
|
||||||
<section
|
|
||||||
className="flex flex-col gap-3 px-6 py-4"
|
|
||||||
data-testid="ack-segment-list"
|
|
||||||
>
|
|
||||||
<div className="flex items-baseline justify-between gap-3">
|
|
||||||
<span className="text-[10.5px] font-semibold uppercase tracking-[0.18em] text-muted-foreground">
|
|
||||||
Segment status
|
|
||||||
</span>
|
|
||||||
<span
|
|
||||||
className="mono text-[10.5px] text-muted-foreground"
|
|
||||||
data-testid="ack-segment-count"
|
|
||||||
>
|
|
||||||
{segments.length} segment{segments.length === 1 ? "" : "s"}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
{segments.length === 0 ? (
|
|
||||||
<p
|
|
||||||
className="text-[12.5px] text-muted-foreground"
|
|
||||||
data-testid="ack-segment-empty"
|
|
||||||
>
|
|
||||||
No per-segment breakdown for this ack — the rolled-up accepted
|
|
||||||
/ rejected counts above are the only signal.
|
|
||||||
</p>
|
|
||||||
) : (
|
|
||||||
<ul className="flex flex-col divide-y divide-border/40 border-y border-border/30">
|
|
||||||
{segments.map((seg, idx) => (
|
|
||||||
<li
|
|
||||||
key={`${seg.reference ?? idx}`}
|
|
||||||
className="flex items-start gap-3 py-2.5"
|
|
||||||
data-testid="ack-segment-row"
|
|
||||||
>
|
|
||||||
<span
|
|
||||||
className="mono text-[12px] tabular-nums text-muted-foreground shrink-0"
|
|
||||||
style={{ minWidth: 32 }}
|
|
||||||
>
|
|
||||||
{idx + 1}
|
|
||||||
</span>
|
|
||||||
<div className="flex-1 min-w-0">
|
|
||||||
<div className="flex items-center gap-2 flex-wrap">
|
|
||||||
<StatusPill status={seg.status} />
|
|
||||||
{seg.reference ? (
|
|
||||||
<span className="mono text-[12px] truncate">
|
|
||||||
{seg.reference}
|
|
||||||
</span>
|
|
||||||
) : null}
|
|
||||||
</div>
|
|
||||||
{seg.note ? (
|
|
||||||
<p className="text-[12.5px] text-muted-foreground mt-1 leading-snug">
|
|
||||||
{seg.note}
|
|
||||||
</p>
|
|
||||||
) : null}
|
|
||||||
</div>
|
|
||||||
</li>
|
|
||||||
))}
|
|
||||||
</ul>
|
|
||||||
)}
|
|
||||||
</section>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
// Barrel export for the AckDrawer module (SP21 Phase 5 Task 5.2).
|
|
||||||
|
|
||||||
export { AckDrawer } from "./AckDrawer";
|
|
||||||
export { SegmentStatusList } from "./SegmentStatusList";
|
|
||||||
@@ -1,15 +1,11 @@
|
|||||||
// @vitest-environment happy-dom
|
// @vitest-environment happy-dom
|
||||||
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
||||||
|
|
||||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
import { cleanup, fireEvent, render, screen } from "@testing-library/react";
|
import { render, screen } from "@testing-library/react";
|
||||||
import { ActivityFeed } from "./ActivityFeed";
|
import { ActivityFeed } from "./ActivityFeed";
|
||||||
import type { Activity } from "@/types";
|
import type { Activity } from "@/types";
|
||||||
|
|
||||||
// happy-dom keeps `document.body` between tests; without cleanup,
|
|
||||||
// `screen.getByRole("button")` finds buttons from earlier renders.
|
|
||||||
afterEach(() => cleanup());
|
|
||||||
|
|
||||||
const baseActivity: Activity = {
|
const baseActivity: Activity = {
|
||||||
id: "a-1",
|
id: "a-1",
|
||||||
kind: "claim_submitted",
|
kind: "claim_submitted",
|
||||||
@@ -51,99 +47,4 @@ describe("ActivityFeed", () => {
|
|||||||
expect(() => render(<ActivityFeed items={[unknown]} />)).not.toThrow();
|
expect(() => render(<ActivityFeed items={[unknown]} />)).not.toThrow();
|
||||||
expect(screen.getByText("future_kind event arrived")).toBeTruthy();
|
expect(screen.getByText("future_kind event arrived")).toBeTruthy();
|
||||||
});
|
});
|
||||||
|
|
||||||
// -------------------------------------------------------------------
|
|
||||||
// SP21 Task 2.5: optional `onItemClick` prop makes each row a
|
|
||||||
// drillable target. Default behavior (no handler) must stay unchanged.
|
|
||||||
// -------------------------------------------------------------------
|
|
||||||
|
|
||||||
it("does not attach row-level click attrs when onItemClick is omitted", () => {
|
|
||||||
render(<ActivityFeed items={[baseActivity]} />);
|
|
||||||
const li = screen.getByRole("listitem");
|
|
||||||
expect(li.getAttribute("role")).not.toBe("button");
|
|
||||||
expect(li.getAttribute("tabindex")).toBeNull();
|
|
||||||
expect(li.classList.contains("drillable")).toBe(false);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("attaches role/tabIndex/drillable class when onItemClick is provided", () => {
|
|
||||||
render(
|
|
||||||
<ActivityFeed
|
|
||||||
items={[baseActivity]}
|
|
||||||
onItemClick={() => {}}
|
|
||||||
/>,
|
|
||||||
);
|
|
||||||
const li = screen.getByRole("button", { name: /View claim submitted/ });
|
|
||||||
expect(li.tagName).toBe("LI");
|
|
||||||
expect(li.getAttribute("tabindex")).toBe("0");
|
|
||||||
expect(li.classList.contains("drillable")).toBe(true);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("calls onItemClick with the event on click", () => {
|
|
||||||
const onItemClick = vi.fn();
|
|
||||||
render(
|
|
||||||
<ActivityFeed
|
|
||||||
items={[baseActivity]}
|
|
||||||
onItemClick={onItemClick}
|
|
||||||
/>,
|
|
||||||
);
|
|
||||||
const li = screen.getByRole("button");
|
|
||||||
fireEvent.click(li);
|
|
||||||
expect(onItemClick).toHaveBeenCalledTimes(1);
|
|
||||||
expect(onItemClick).toHaveBeenCalledWith(baseActivity);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("calls onItemClick on Enter keydown", () => {
|
|
||||||
const onItemClick = vi.fn();
|
|
||||||
render(
|
|
||||||
<ActivityFeed
|
|
||||||
items={[baseActivity]}
|
|
||||||
onItemClick={onItemClick}
|
|
||||||
/>,
|
|
||||||
);
|
|
||||||
const li = screen.getByRole("button");
|
|
||||||
fireEvent.keyDown(li, { key: "Enter" });
|
|
||||||
expect(onItemClick).toHaveBeenCalledTimes(1);
|
|
||||||
expect(onItemClick).toHaveBeenCalledWith(baseActivity);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("calls onItemClick on Space keydown", () => {
|
|
||||||
const onItemClick = vi.fn();
|
|
||||||
render(
|
|
||||||
<ActivityFeed
|
|
||||||
items={[baseActivity]}
|
|
||||||
onItemClick={onItemClick}
|
|
||||||
/>,
|
|
||||||
);
|
|
||||||
fireEvent.keyDown(screen.getByRole("button"), { key: " " });
|
|
||||||
expect(onItemClick).toHaveBeenCalledTimes(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("does not call onItemClick for unrelated keys", () => {
|
|
||||||
const onItemClick = vi.fn();
|
|
||||||
render(
|
|
||||||
<ActivityFeed
|
|
||||||
items={[baseActivity]}
|
|
||||||
onItemClick={onItemClick}
|
|
||||||
/>,
|
|
||||||
);
|
|
||||||
fireEvent.keyDown(screen.getByRole("button"), { key: "a" });
|
|
||||||
expect(onItemClick).not.toHaveBeenCalled();
|
|
||||||
});
|
|
||||||
|
|
||||||
// Regression for the Task 2.4 event-bubbling pattern: clicks on a
|
|
||||||
// drillable row must not bubble to a parent row click. Without
|
|
||||||
// stopPropagation a Dashboard parent handler could fire alongside
|
|
||||||
// the row click and corrupt history.
|
|
||||||
it("stops click propagation so a parent row handler does not also fire", () => {
|
|
||||||
const onItemClick = vi.fn();
|
|
||||||
const parentClick = vi.fn();
|
|
||||||
render(
|
|
||||||
<ul onClick={parentClick}>
|
|
||||||
<ActivityFeed items={[baseActivity]} onItemClick={onItemClick} />
|
|
||||||
</ul>,
|
|
||||||
);
|
|
||||||
fireEvent.click(screen.getByRole("button"));
|
|
||||||
expect(onItemClick).toHaveBeenCalledTimes(1);
|
|
||||||
expect(parentClick).not.toHaveBeenCalled();
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,4 +1,3 @@
|
|||||||
import type { KeyboardEvent, MouseEvent } from "react";
|
|
||||||
import {
|
import {
|
||||||
Banknote,
|
Banknote,
|
||||||
CheckCircle2,
|
CheckCircle2,
|
||||||
@@ -64,22 +63,9 @@ const FALLBACK_KIND: { icon: LucideIcon; tone: string; tint: string } = {
|
|||||||
export function ActivityFeed({
|
export function ActivityFeed({
|
||||||
items,
|
items,
|
||||||
emptyMessage = "No activity yet.",
|
emptyMessage = "No activity yet.",
|
||||||
onItemClick,
|
|
||||||
}: {
|
}: {
|
||||||
items: Activity[];
|
items: Activity[];
|
||||||
emptyMessage?: string;
|
emptyMessage?: string;
|
||||||
/**
|
|
||||||
* Optional click handler for SP21 Universal Drill-Down (Task 2.5).
|
|
||||||
* When provided, each row becomes a clickable target: the row's
|
|
||||||
* outer `<li>` gets `role="button"`, `tabIndex`, the `drillable`
|
|
||||||
* hover affordance (chevron + tint), and an Enter/Space keybinding.
|
|
||||||
*
|
|
||||||
* `e.stopPropagation()` is called before invoking the handler so the
|
|
||||||
* click doesn't bubble to a hypothetical parent row click — same
|
|
||||||
* fix that landed on `DrillableCell` in Task 2.4. When omitted, the
|
|
||||||
* feed renders exactly as before (no extra DOM, no extra attrs).
|
|
||||||
*/
|
|
||||||
onItemClick?: (event: Activity) => void;
|
|
||||||
}) {
|
}) {
|
||||||
if (items.length === 0) {
|
if (items.length === 0) {
|
||||||
return (
|
return (
|
||||||
@@ -93,36 +79,11 @@ export function ActivityFeed({
|
|||||||
{items.map((a) => {
|
{items.map((a) => {
|
||||||
const cfg = kindConfig[a.kind] ?? FALLBACK_KIND;
|
const cfg = kindConfig[a.kind] ?? FALLBACK_KIND;
|
||||||
const Icon = cfg.icon;
|
const Icon = cfg.icon;
|
||||||
// SP21 Task 2.5: when a click handler is wired, the row
|
|
||||||
// becomes a keyboard-focusable button-like element with the
|
|
||||||
// drillable hover affordance. We attach the handler to the
|
|
||||||
// <li> directly (matching the Dashboard's existing patterns
|
|
||||||
// for "Top providers" / "Recent denials" rows) so the click
|
|
||||||
// target covers the full row width including padding.
|
|
||||||
const rowProps = onItemClick
|
|
||||||
? {
|
|
||||||
role: "button" as const,
|
|
||||||
tabIndex: 0,
|
|
||||||
"aria-label": `View ${a.kind.replace(/_/g, " ")}: ${a.message}`,
|
|
||||||
className:
|
|
||||||
"drillable flex items-start gap-3 py-3 first:pt-0 last:pb-0 rounded-sm focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-1",
|
|
||||||
onClick: (e: MouseEvent<HTMLLIElement>) => {
|
|
||||||
e.stopPropagation();
|
|
||||||
onItemClick(a);
|
|
||||||
},
|
|
||||||
onKeyDown: (e: KeyboardEvent<HTMLLIElement>) => {
|
|
||||||
if (e.key === "Enter" || e.key === " ") {
|
|
||||||
e.preventDefault();
|
|
||||||
e.stopPropagation();
|
|
||||||
onItemClick(a);
|
|
||||||
}
|
|
||||||
},
|
|
||||||
}
|
|
||||||
: {
|
|
||||||
className: "flex items-start gap-3 py-3 first:pt-0 last:pb-0",
|
|
||||||
};
|
|
||||||
return (
|
return (
|
||||||
<li key={a.id} {...rowProps}>
|
<li
|
||||||
|
key={a.id}
|
||||||
|
className="flex items-start gap-3 py-3 first:pt-0 last:pb-0"
|
||||||
|
>
|
||||||
<div
|
<div
|
||||||
className={cn(
|
className={cn(
|
||||||
"mt-0.5 h-7 w-7 shrink-0 rounded-md ring-1 ring-inset ring-border/40 flex items-center justify-center",
|
"mt-0.5 h-7 w-7 shrink-0 rounded-md ring-1 ring-inset ring-border/40 flex items-center justify-center",
|
||||||
|
|||||||
@@ -1,11 +1,4 @@
|
|||||||
import {
|
import { Plus, Minus, Equal, ArrowRight, type LucideIcon } from "lucide-react";
|
||||||
ArrowRight,
|
|
||||||
Equal,
|
|
||||||
Minus,
|
|
||||||
Plus,
|
|
||||||
type LucideIcon,
|
|
||||||
} from "lucide-react";
|
|
||||||
import { useNavigate } from "react-router-dom";
|
|
||||||
import { Badge } from "@/components/ui/badge";
|
import { Badge } from "@/components/ui/badge";
|
||||||
import { Skeleton } from "@/components/ui/skeleton";
|
import { Skeleton } from "@/components/ui/skeleton";
|
||||||
import {
|
import {
|
||||||
@@ -16,21 +9,19 @@ import {
|
|||||||
TableHeader,
|
TableHeader,
|
||||||
TableRow,
|
TableRow,
|
||||||
} from "@/components/ui/table";
|
} from "@/components/ui/table";
|
||||||
import { DrillableCell } from "@/components/drill/DrillableCell";
|
import { KpiCard } from "@/components/KpiCard";
|
||||||
import { fmt } from "@/lib/format";
|
import { fmt } from "@/lib/format";
|
||||||
import { cn } from "@/lib/utils";
|
import { cn } from "@/lib/utils";
|
||||||
import type {
|
import type {
|
||||||
BatchClaimDiffSummary,
|
|
||||||
BatchDiff,
|
BatchDiff,
|
||||||
BatchDiffChangedRow,
|
BatchDiffChangedRow,
|
||||||
BatchDiffSideMeta,
|
BatchDiffSideMeta,
|
||||||
BatchDiffSummary,
|
BatchDiffSummary,
|
||||||
|
BatchClaimDiffSummary,
|
||||||
} from "@/types";
|
} from "@/types";
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Side meta header — small panel identifying which batch is on the
|
// Side meta header — small panel identifying which batch is on the left / right.
|
||||||
// left / right. Paper-toned to sit inside the cream "paper plane" of
|
|
||||||
// the BatchDiff page. data-testid pinned by BatchDiff.test.tsx.
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
function SideMeta({
|
function SideMeta({
|
||||||
@@ -48,19 +39,10 @@ function SideMeta({
|
|||||||
: "text-amber-300 border-amber-400/30 bg-amber-400/10";
|
: "text-amber-300 border-amber-400/30 bg-amber-400/10";
|
||||||
return (
|
return (
|
||||||
<div
|
<div
|
||||||
className="rounded-xl p-4 flex flex-col gap-2 border"
|
className="surface rounded-xl p-4 flex flex-col gap-2"
|
||||||
data-testid={`batch-diff-side-${label.toLowerCase()}`}
|
data-testid={`batch-diff-side-${label.toLowerCase()}`}
|
||||||
style={{
|
|
||||||
backgroundColor: "hsl(36 22% 98%)",
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.10)",
|
|
||||||
boxShadow:
|
|
||||||
"inset 0 1px 0 0 hsl(0 0% 100% / 0.45), 0 1px 0 0 hsl(30 14% 22% / 0.06)",
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<div
|
|
||||||
className="text-[10.5px] font-semibold uppercase tracking-[0.18em]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
>
|
||||||
|
<div className="text-[10.5px] font-semibold uppercase tracking-[0.18em] text-muted-foreground">
|
||||||
{label}
|
{label}
|
||||||
</div>
|
</div>
|
||||||
<div className="flex items-center gap-2">
|
<div className="flex items-center gap-2">
|
||||||
@@ -72,23 +54,12 @@ function SideMeta({
|
|||||||
>
|
>
|
||||||
{side.kind}
|
{side.kind}
|
||||||
</span>
|
</span>
|
||||||
<span
|
<span className="display num text-[13px]">{side.id}</span>
|
||||||
className="display num text-[13px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{side.id}
|
|
||||||
</span>
|
|
||||||
</div>
|
</div>
|
||||||
<div
|
<div className="font-mono text-[12px] text-muted-foreground truncate">
|
||||||
className="font-mono text-[12px] truncate"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-2))" }}
|
|
||||||
>
|
|
||||||
{side.inputFilename}
|
{side.inputFilename}
|
||||||
</div>
|
</div>
|
||||||
<div
|
<div className="flex items-center justify-between text-xs text-muted-foreground">
|
||||||
className="flex items-center justify-between text-xs"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
<span>Parsed {side.parsedAt ? fmt.dateShort(side.parsedAt) : "—"}</span>
|
<span>Parsed {side.parsedAt ? fmt.dateShort(side.parsedAt) : "—"}</span>
|
||||||
<span className="font-mono num">
|
<span className="font-mono num">
|
||||||
{fmt.num(side.claimCount)} claim{side.claimCount === 1 ? "" : "s"}
|
{fmt.num(side.claimCount)} claim{side.claimCount === 1 ? "" : "s"}
|
||||||
@@ -99,125 +70,40 @@ function SideMeta({
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Summary cards — the four-count tile row. Paper-toned via
|
// Summary cards — the four-count tile row.
|
||||||
// DiffKpiTile so the counts sit naturally on the cream paper plane.
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
function SummaryCards({ summary }: { summary: BatchDiffSummary }) {
|
function SummaryCards({ summary }: { summary: BatchDiffSummary }) {
|
||||||
return (
|
return (
|
||||||
<div
|
<div className="grid grid-cols-2 md:grid-cols-4 gap-3" data-testid="batch-diff-summary">
|
||||||
className="grid grid-cols-2 md:grid-cols-4 gap-3"
|
<KpiCard
|
||||||
data-testid="batch-diff-summary"
|
|
||||||
>
|
|
||||||
<DiffKpiTile
|
|
||||||
label="Added in B"
|
label="Added in B"
|
||||||
value={fmt.num(summary.addedCount)}
|
value={fmt.num(summary.addedCount)}
|
||||||
icon={Plus}
|
icon={Plus}
|
||||||
hint="In B, not in A"
|
hint="In B, not in A"
|
||||||
tone="success"
|
|
||||||
/>
|
/>
|
||||||
<DiffKpiTile
|
<KpiCard
|
||||||
label="Removed from A"
|
label="Removed from A"
|
||||||
value={fmt.num(summary.removedCount)}
|
value={fmt.num(summary.removedCount)}
|
||||||
icon={Minus}
|
icon={Minus}
|
||||||
hint="In A, not in B"
|
hint="In A, not in B"
|
||||||
tone="destructive"
|
|
||||||
/>
|
/>
|
||||||
<DiffKpiTile
|
<KpiCard
|
||||||
label="Changed"
|
label="Changed"
|
||||||
value={fmt.num(summary.changedCount)}
|
value={fmt.num(summary.changedCount)}
|
||||||
icon={Equal}
|
icon={Equal}
|
||||||
hint="Deltas in either side"
|
hint="Deltas in either side"
|
||||||
tone="amber"
|
|
||||||
/>
|
/>
|
||||||
<DiffKpiTile
|
<KpiCard
|
||||||
label="Unchanged"
|
label="Unchanged"
|
||||||
value={fmt.num(summary.unchangedCount)}
|
value={fmt.num(summary.unchangedCount)}
|
||||||
icon={Equal}
|
icon={Equal}
|
||||||
hint="Identical across A & B"
|
hint="Identical across A & B"
|
||||||
tone="ink"
|
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// DiffKpiTile — paper-toned metric tile for the diff summary row.
|
|
||||||
// Mirrors AckKpiTile / BatchesKpiTile from the other hybrid pages.
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
function DiffKpiTile({
|
|
||||||
label,
|
|
||||||
value,
|
|
||||||
icon: Icon,
|
|
||||||
hint,
|
|
||||||
tone,
|
|
||||||
}: {
|
|
||||||
label: string;
|
|
||||||
value: string;
|
|
||||||
icon: LucideIcon;
|
|
||||||
hint: string;
|
|
||||||
tone: "success" | "destructive" | "amber" | "ink";
|
|
||||||
}) {
|
|
||||||
const accentMap = {
|
|
||||||
success: "hsl(152 64% 38%)",
|
|
||||||
destructive: "hsl(358 70% 42%)",
|
|
||||||
amber: "hsl(36 92% 50%)",
|
|
||||||
ink: "hsl(var(--surface-ink))",
|
|
||||||
} as const;
|
|
||||||
const tintMap = {
|
|
||||||
success: "hsl(152 50% 88%)",
|
|
||||||
destructive: "hsl(358 70% 92%)",
|
|
||||||
amber: "hsl(36 82% 92%)",
|
|
||||||
ink: "hsl(36 22% 90%)",
|
|
||||||
} as const;
|
|
||||||
const accent = accentMap[tone];
|
|
||||||
const tint = tintMap[tone];
|
|
||||||
return (
|
|
||||||
<div
|
|
||||||
className="relative rounded-xl p-4 overflow-hidden border"
|
|
||||||
title={hint}
|
|
||||||
style={{
|
|
||||||
backgroundColor: "hsl(var(--surface))",
|
|
||||||
boxShadow:
|
|
||||||
"inset 0 1px 0 0 hsl(0 0% 100% / 0.45), 0 1px 0 0 hsl(30 14% 22% / 0.06), inset 3px 0 0 0 hsl(0 0% 100% / 0.4)",
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.10)",
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<div
|
|
||||||
aria-hidden
|
|
||||||
className="absolute left-0 top-4 bottom-4 w-[3px] rounded-r-sm"
|
|
||||||
style={{ backgroundColor: accent, opacity: 0.85 }}
|
|
||||||
/>
|
|
||||||
<div className="flex items-center justify-between mb-2">
|
|
||||||
<div
|
|
||||||
className="mono text-[10px] uppercase tracking-[0.18em] font-semibold"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
{label}
|
|
||||||
</div>
|
|
||||||
<div
|
|
||||||
className="h-5 w-5 rounded-md flex items-center justify-center"
|
|
||||||
style={{ backgroundColor: tint, color: accent }}
|
|
||||||
>
|
|
||||||
<Icon className="h-3 w-3" strokeWidth={1.75} />
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div
|
|
||||||
className="display tabular-nums tracking-[-0.04em]"
|
|
||||||
style={{
|
|
||||||
color: "hsl(var(--surface-ink))",
|
|
||||||
fontSize: "clamp(26px, 2.8vw, 36px)",
|
|
||||||
lineHeight: 1,
|
|
||||||
fontWeight: 400,
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
{value}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Row indicator — `+` / `-` / `~` gutter on the left of every row.
|
// Row indicator — `+` / `-` / `~` gutter on the left of every row.
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -265,58 +151,23 @@ function RowIndicator({
|
|||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
function ClaimIdCell({ id }: { id: string }) {
|
function ClaimIdCell({ id }: { id: string }) {
|
||||||
// SP21 Phase 5 Task 5.6: each claim id is drillable to
|
|
||||||
// /claims?claim=ID so the operator can jump straight from a
|
|
||||||
// "Removed from A" or "Changed" row into the ClaimDrawer.
|
|
||||||
// DrillableCell handles e.stopPropagation internally — important
|
|
||||||
// if these rows ever get a row-level onClick. For removed claims
|
|
||||||
// that no longer exist in the DB, the ClaimDrawer's 404 state
|
|
||||||
// takes over (verified in Phase 2 testing).
|
|
||||||
const navigate = useNavigate();
|
|
||||||
return (
|
return (
|
||||||
<DrillableCell
|
<span className="font-mono text-[12px] tracking-tight" data-testid="diff-claim-id">
|
||||||
onClick={() =>
|
|
||||||
navigate(`/claims?claim=${encodeURIComponent(id)}`)
|
|
||||||
}
|
|
||||||
ariaLabel={`View claim ${id} in detail`}
|
|
||||||
>
|
|
||||||
<span
|
|
||||||
className="font-mono text-[12px] tracking-tight"
|
|
||||||
data-testid="diff-claim-id"
|
|
||||||
>
|
|
||||||
{id}
|
{id}
|
||||||
</span>
|
</span>
|
||||||
</DrillableCell>
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
function PatientCell({ name }: { name: string }) {
|
function PatientCell({ name }: { name: string }) {
|
||||||
if (!name) {
|
if (!name) {
|
||||||
return (
|
return <span className="text-muted-foreground/60 text-[12px]">—</span>;
|
||||||
<span
|
|
||||||
className="text-[12px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
—
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
return (
|
return <span className="text-[13px]">{name}</span>;
|
||||||
<span
|
|
||||||
className="text-[13px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{name}
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function ChargeCell({ value }: { value: number }) {
|
function ChargeCell({ value }: { value: number }) {
|
||||||
return (
|
return (
|
||||||
<span
|
<span className="display num text-[13px] tabular-nums">
|
||||||
className="display num text-[13px] tabular-nums"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{fmt.usdPrecise(value)}
|
{fmt.usdPrecise(value)}
|
||||||
</span>
|
</span>
|
||||||
);
|
);
|
||||||
@@ -324,44 +175,19 @@ function ChargeCell({ value }: { value: number }) {
|
|||||||
|
|
||||||
function DateCell({ value }: { value: string | null }) {
|
function DateCell({ value }: { value: string | null }) {
|
||||||
if (!value) {
|
if (!value) {
|
||||||
return (
|
return <span className="text-muted-foreground/60 text-[12px]">—</span>;
|
||||||
<span
|
|
||||||
className="text-[12px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
—
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
return (
|
return <span className="num text-[12.5px]">{value}</span>;
|
||||||
<span
|
|
||||||
className="num text-[12.5px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-2))" }}
|
|
||||||
>
|
|
||||||
{value}
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function StatusCell({ value }: { value: string }) {
|
function StatusCell({ value }: { value: string }) {
|
||||||
if (!value) {
|
if (!value) {
|
||||||
return (
|
return <span className="text-muted-foreground/60 text-[12px]">—</span>;
|
||||||
<span
|
|
||||||
className="text-[12px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
—
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
return (
|
return (
|
||||||
<Badge
|
<Badge
|
||||||
variant="outline"
|
variant="outline"
|
||||||
className="font-mono uppercase tracking-wider text-[10px]"
|
className="font-mono uppercase tracking-wider text-[10px]"
|
||||||
style={{
|
|
||||||
color: "hsl(var(--surface-ink))",
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.20)",
|
|
||||||
}}
|
|
||||||
>
|
>
|
||||||
{value}
|
{value}
|
||||||
</Badge>
|
</Badge>
|
||||||
@@ -370,28 +196,17 @@ function StatusCell({ value }: { value: string }) {
|
|||||||
|
|
||||||
function CptCell({ codes }: { codes: string[] }) {
|
function CptCell({ codes }: { codes: string[] }) {
|
||||||
if (!codes || codes.length === 0) {
|
if (!codes || codes.length === 0) {
|
||||||
return (
|
return <span className="text-muted-foreground/60 text-[12px]">—</span>;
|
||||||
<span
|
|
||||||
className="text-[12px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
—
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
return (
|
return (
|
||||||
<span
|
<span className="font-mono text-[11.5px] tracking-tight">
|
||||||
className="font-mono text-[11.5px] tracking-tight"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-2))" }}
|
|
||||||
>
|
|
||||||
{codes.join(", ")}
|
{codes.join(", ")}
|
||||||
</span>
|
</span>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Section tables — one per bucket (added / removed / changed). Uses
|
// Section tables — one per bucket (added / removed / changed).
|
||||||
// `tone="paper"` so the table chrome matches the cream paper plane.
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
type Side = "a" | "b";
|
type Side = "a" | "b";
|
||||||
@@ -571,53 +386,25 @@ function SectionTable({
|
|||||||
<section className="space-y-3" data-testid={testid}>
|
<section className="space-y-3" data-testid={testid}>
|
||||||
<header className="flex items-baseline justify-between">
|
<header className="flex items-baseline justify-between">
|
||||||
<div>
|
<div>
|
||||||
<div
|
<div className="text-[10.5px] font-semibold uppercase tracking-[0.18em] text-muted-foreground mb-0.5">
|
||||||
className="text-[10.5px] font-semibold uppercase tracking-[0.18em] mb-0.5"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
{eyebrow}
|
{eyebrow}
|
||||||
</div>
|
</div>
|
||||||
<h3
|
<h2 className="text-[15px] font-semibold tracking-tight">{title}</h2>
|
||||||
className="display leading-[0.98] tracking-[-0.03em]"
|
|
||||||
style={{
|
|
||||||
color: "hsl(var(--surface-ink))",
|
|
||||||
fontSize: "clamp(20px, 2.2vw, 26px)",
|
|
||||||
fontWeight: 400,
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
{title}
|
|
||||||
</h3>
|
|
||||||
</div>
|
</div>
|
||||||
<span
|
<span className="font-mono num text-[12.5px] text-muted-foreground">
|
||||||
className="font-mono num text-[12.5px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
{fmt.num(count)}
|
{fmt.num(count)}
|
||||||
</span>
|
</span>
|
||||||
</header>
|
</header>
|
||||||
{count === 0 ? (
|
{count === 0 ? (
|
||||||
<div
|
<div
|
||||||
className="rounded-xl border p-6 text-center text-[12.5px]"
|
className="surface rounded-xl border border-dashed border-border/60 p-6 text-center text-[12.5px] text-muted-foreground"
|
||||||
data-testid={`${testid}-empty`}
|
data-testid={`${testid}-empty`}
|
||||||
style={{
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.16)",
|
|
||||||
borderStyle: "dashed",
|
|
||||||
backgroundColor: "hsl(36 22% 96%)",
|
|
||||||
color: "hsl(var(--surface-ink-3))",
|
|
||||||
}}
|
|
||||||
>
|
>
|
||||||
{emptyMessage}
|
{emptyMessage}
|
||||||
</div>
|
</div>
|
||||||
) : (
|
) : (
|
||||||
<div
|
<div className="surface rounded-xl overflow-hidden">
|
||||||
className="rounded-xl border overflow-hidden"
|
<Table>
|
||||||
style={{
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.10)",
|
|
||||||
backgroundColor: "hsl(36 22% 98%)",
|
|
||||||
boxShadow: "inset 0 1px 0 0 hsl(0 0% 100% / 0.5)",
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<Table tone="paper">
|
|
||||||
<TableHeader>
|
<TableHeader>
|
||||||
<TableRow>
|
<TableRow>
|
||||||
<TableHead className="w-10" aria-label="Diff indicator" />
|
<TableHead className="w-10" aria-label="Diff indicator" />
|
||||||
@@ -638,8 +425,7 @@ function SectionTable({
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Skeleton — used by the page while the diff is loading. Paper-toned
|
// Skeleton — used by the page while the diff is loading.
|
||||||
// (cream blocks) to match the paper plane chrome.
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
export function BatchDiffViewSkeleton() {
|
export function BatchDiffViewSkeleton() {
|
||||||
@@ -676,24 +462,11 @@ export function BatchDiffEmpty({ data }: { data: BatchDiff }) {
|
|||||||
<SideMeta side={data.b} label="B (right)" />
|
<SideMeta side={data.b} label="B (right)" />
|
||||||
</div>
|
</div>
|
||||||
<SummaryCards summary={data.summary} />
|
<SummaryCards summary={data.summary} />
|
||||||
<div
|
<div className="surface rounded-xl border border-dashed border-border/60 p-10 text-center">
|
||||||
className="rounded-xl border p-10 text-center"
|
<div className="text-[10.5px] font-semibold uppercase tracking-[0.18em] text-muted-foreground mb-1.5">
|
||||||
style={{
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.16)",
|
|
||||||
borderStyle: "dashed",
|
|
||||||
backgroundColor: "hsl(36 22% 96%)",
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<div
|
|
||||||
className="text-[10.5px] font-semibold uppercase tracking-[0.18em] mb-1.5"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
Diff · no deltas
|
Diff · no deltas
|
||||||
</div>
|
</div>
|
||||||
<div
|
<div className="text-[13.5px] text-muted-foreground">
|
||||||
className="text-[13.5px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-2))" }}
|
|
||||||
>
|
|
||||||
These two batches are identical — no claims added, removed, or
|
These two batches are identical — no claims added, removed, or
|
||||||
changed between A and B.
|
changed between A and B.
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -20,11 +20,6 @@ import type { BatchSummary } from "@/lib/api";
|
|||||||
*
|
*
|
||||||
* Voice mirrors `AckCodeBadge` in `src/pages/Acks.tsx` (uppercase,
|
* Voice mirrors `AckCodeBadge` in `src/pages/Acks.tsx` (uppercase,
|
||||||
* wide tracking, hairline border, low-opacity fill).
|
* wide tracking, hairline border, low-opacity fill).
|
||||||
*
|
|
||||||
* The literal class names `text-sky-300` and `text-amber-300` are
|
|
||||||
* pinned by `Batches.test.tsx` as a contract — the test asserts the
|
|
||||||
* badge's text color is one of these two strings. Don't replace them
|
|
||||||
* with arbitrary HSL values or the test will fail.
|
|
||||||
*/
|
*/
|
||||||
function KindBadge({ kind }: { kind: BatchSummary["kind"] }) {
|
function KindBadge({ kind }: { kind: BatchSummary["kind"] }) {
|
||||||
const color =
|
const color =
|
||||||
@@ -47,8 +42,7 @@ function KindBadge({ kind }: { kind: BatchSummary["kind"] }) {
|
|||||||
/**
|
/**
|
||||||
* Skeleton rows for the batches table. Mirrors the row count used in
|
* Skeleton rows for the batches table. Mirrors the row count used in
|
||||||
* `Acks.tsx` (5 placeholders) so the loading density matches the rest
|
* `Acks.tsx` (5 placeholders) so the loading density matches the rest
|
||||||
* of the app. `data-testid="batches-skeleton"` is pinned by
|
* of the app.
|
||||||
* `Batches.test.tsx`.
|
|
||||||
*/
|
*/
|
||||||
export function BatchesListSkeleton() {
|
export function BatchesListSkeleton() {
|
||||||
return (
|
return (
|
||||||
@@ -69,13 +63,6 @@ type BatchesListProps = {
|
|||||||
*/
|
*/
|
||||||
openId: string | null;
|
openId: string | null;
|
||||||
onOpen: (id: string) => void;
|
onOpen: (id: string) => void;
|
||||||
/**
|
|
||||||
* When `paper`, the table sits inside a cream "paper plane" section
|
|
||||||
* and uses the paper-toned color scheme: warm hover, hairline border
|
|
||||||
* in surface-line, surface-ink text. When `dark` (default), the
|
|
||||||
* original dark-mode chrome is used.
|
|
||||||
*/
|
|
||||||
tone?: "dark" | "paper";
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -84,26 +71,15 @@ type BatchesListProps = {
|
|||||||
* so the numbers tick up from 0 on first render (gives the page a
|
* so the numbers tick up from 0 on first render (gives the page a
|
||||||
* little life on load; consistent with the Dashboard KPI cards).
|
* little life on load; consistent with the Dashboard KPI cards).
|
||||||
*/
|
*/
|
||||||
export function BatchesList({
|
export function BatchesList({ items, openId, onOpen }: BatchesListProps) {
|
||||||
items,
|
|
||||||
openId,
|
|
||||||
onOpen,
|
|
||||||
tone = "dark",
|
|
||||||
}: BatchesListProps) {
|
|
||||||
const isPaper = tone === "paper";
|
|
||||||
return (
|
return (
|
||||||
<Table data-testid="batches-table" tone={tone}>
|
<Table data-testid="batches-table">
|
||||||
<TableHeader>
|
<TableHeader>
|
||||||
<TableRow>
|
<TableRow>
|
||||||
<TableHead>Kind</TableHead>
|
<TableHead>Kind</TableHead>
|
||||||
<TableHead>Batch</TableHead>
|
<TableHead>Batch</TableHead>
|
||||||
<TableHead>Input file</TableHead>
|
<TableHead>Input file</TableHead>
|
||||||
<TableHead
|
<TableHead className="text-right">Claims</TableHead>
|
||||||
className="text-right"
|
|
||||||
style={isPaper ? { color: "hsl(var(--surface-ink-2))" } : undefined}
|
|
||||||
>
|
|
||||||
Claims
|
|
||||||
</TableHead>
|
|
||||||
<TableHead>Parsed</TableHead>
|
<TableHead>Parsed</TableHead>
|
||||||
<TableHead className="w-6" aria-label="Open" />
|
<TableHead className="w-6" aria-label="Open" />
|
||||||
</TableRow>
|
</TableRow>
|
||||||
@@ -117,57 +93,28 @@ export function BatchesList({
|
|||||||
data-open={openId === b.id ? "true" : undefined}
|
data-open={openId === b.id ? "true" : undefined}
|
||||||
className={cn(
|
className={cn(
|
||||||
"animate-row-flash cursor-pointer",
|
"animate-row-flash cursor-pointer",
|
||||||
isPaper && openId === b.id &&
|
openId === b.id && "bg-muted/40",
|
||||||
"!bg-[hsl(212_85%_95%)] ring-1 ring-inset ring-[hsl(212_100%_45%_/_0.30)]",
|
|
||||||
!isPaper && openId === b.id && "bg-muted/40",
|
|
||||||
)}
|
)}
|
||||||
>
|
>
|
||||||
<TableCell>
|
<TableCell>
|
||||||
<KindBadge kind={b.kind} />
|
<KindBadge kind={b.kind} />
|
||||||
</TableCell>
|
</TableCell>
|
||||||
<TableCell
|
<TableCell className="display num text-[13px]">
|
||||||
className="display num text-[13px]"
|
|
||||||
style={isPaper ? { color: "hsl(var(--surface-ink))" } : undefined}
|
|
||||||
>
|
|
||||||
{b.id}
|
{b.id}
|
||||||
</TableCell>
|
</TableCell>
|
||||||
<TableCell
|
<TableCell className="font-mono text-[12px] text-muted-foreground truncate max-w-[280px]">
|
||||||
className={cn(
|
|
||||||
"font-mono text-[12px] truncate max-w-[280px]",
|
|
||||||
isPaper
|
|
||||||
? "text-[hsl(var(--surface-ink-2))]"
|
|
||||||
: "text-muted-foreground",
|
|
||||||
)}
|
|
||||||
>
|
|
||||||
{b.inputFilename}
|
{b.inputFilename}
|
||||||
</TableCell>
|
</TableCell>
|
||||||
<TableCell
|
<TableCell className="text-right display num text-[13px]">
|
||||||
className="text-right display num text-[13px]"
|
|
||||||
style={isPaper ? { color: "hsl(var(--surface-ink))" } : undefined}
|
|
||||||
>
|
|
||||||
<AnimatedNumber
|
<AnimatedNumber
|
||||||
value={b.claimCount}
|
value={b.claimCount}
|
||||||
format={(n) => fmt.num(Math.round(n))}
|
format={(n) => fmt.num(Math.round(n))}
|
||||||
/>
|
/>
|
||||||
</TableCell>
|
</TableCell>
|
||||||
<TableCell
|
<TableCell className="text-muted-foreground num text-[12.5px]">
|
||||||
className={cn(
|
|
||||||
"num text-[12.5px]",
|
|
||||||
isPaper
|
|
||||||
? "text-[hsl(var(--surface-ink-3))]"
|
|
||||||
: "text-muted-foreground",
|
|
||||||
)}
|
|
||||||
>
|
|
||||||
{b.parsedAt ? fmt.dateShort(b.parsedAt) : "—"}
|
{b.parsedAt ? fmt.dateShort(b.parsedAt) : "—"}
|
||||||
</TableCell>
|
</TableCell>
|
||||||
<TableCell
|
<TableCell className="text-muted-foreground text-right">
|
||||||
className={cn(
|
|
||||||
"text-right",
|
|
||||||
isPaper
|
|
||||||
? "text-[hsl(var(--surface-ink-3))]"
|
|
||||||
: "text-muted-foreground",
|
|
||||||
)}
|
|
||||||
>
|
|
||||||
<span aria-hidden>›</span>
|
<span aria-hidden>›</span>
|
||||||
</TableCell>
|
</TableCell>
|
||||||
</TableRow>
|
</TableRow>
|
||||||
|
|||||||
@@ -1,86 +0,0 @@
|
|||||||
// Shared visual primitives used by ClaimCard837 and ClaimCard835.
|
|
||||||
//
|
|
||||||
// Both cards render validation status (Passed / Warnings / Failed) and
|
|
||||||
// a grid of stat pills (Member, Place of service, …). The exact same
|
|
||||||
// mark-up was being copy-pasted inside Upload.tsx for both cards; this
|
|
||||||
// file is the shared home.
|
|
||||||
|
|
||||||
import { AlertTriangle, CheckCircle2, XCircle } from "lucide-react";
|
|
||||||
import { cn } from "@/lib/utils";
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Tri-state validation indicator.
|
|
||||||
*
|
|
||||||
* - `passed` → green "Passed" with check icon
|
|
||||||
* - `hasWarnings` → amber "Warnings" with triangle
|
|
||||||
* - else → red "Failed" with X
|
|
||||||
*/
|
|
||||||
export function ValidationDot({
|
|
||||||
passed,
|
|
||||||
hasWarnings,
|
|
||||||
}: {
|
|
||||||
passed: boolean;
|
|
||||||
hasWarnings?: boolean;
|
|
||||||
}) {
|
|
||||||
if (passed) {
|
|
||||||
return (
|
|
||||||
<span
|
|
||||||
className="inline-flex items-center gap-1.5 mono text-[10.5px] uppercase tracking-[0.12em] text-[hsl(var(--success))] font-medium"
|
|
||||||
title="Validation passed"
|
|
||||||
>
|
|
||||||
<CheckCircle2 className="h-3.5 w-3.5" strokeWidth={1.75} />
|
|
||||||
Passed
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
if (hasWarnings) {
|
|
||||||
return (
|
|
||||||
<span
|
|
||||||
className="inline-flex items-center gap-1.5 mono text-[10.5px] uppercase tracking-[0.12em] text-[hsl(var(--warning))] font-medium"
|
|
||||||
title="Validation passed with warnings"
|
|
||||||
>
|
|
||||||
<AlertTriangle className="h-3.5 w-3.5" strokeWidth={1.75} />
|
|
||||||
Warnings
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
return (
|
|
||||||
<span
|
|
||||||
className="inline-flex items-center gap-1.5 mono text-[10.5px] uppercase tracking-[0.12em] text-destructive font-medium"
|
|
||||||
title="Validation failed"
|
|
||||||
>
|
|
||||||
<XCircle className="h-3.5 w-3.5" strokeWidth={1.75} />
|
|
||||||
Failed
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Small label + value cell for the expanded-card detail grid.
|
|
||||||
*/
|
|
||||||
export function StatPill({
|
|
||||||
label,
|
|
||||||
value,
|
|
||||||
mono = true,
|
|
||||||
}: {
|
|
||||||
label: string;
|
|
||||||
value: React.ReactNode;
|
|
||||||
mono?: boolean;
|
|
||||||
}) {
|
|
||||||
return (
|
|
||||||
<div className="flex flex-col gap-1">
|
|
||||||
<div className="mono text-[10px] uppercase tracking-[0.18em] font-semibold text-muted-foreground/70">
|
|
||||||
{label}
|
|
||||||
</div>
|
|
||||||
<div
|
|
||||||
className={cn(
|
|
||||||
"text-[13px]",
|
|
||||||
mono && "display mono"
|
|
||||||
)}
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{value}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -1,162 +0,0 @@
|
|||||||
// @vitest-environment happy-dom
|
|
||||||
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT =
|
|
||||||
true;
|
|
||||||
|
|
||||||
import React, { act } from "react";
|
|
||||||
import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
|
|
||||||
import { MemoryRouter } from "react-router-dom";
|
|
||||||
import { createRoot, type Root } from "react-dom/client";
|
|
||||||
import { useAppStore } from "@/store";
|
|
||||||
import { ClaimCard837 } from "./ClaimCard837";
|
|
||||||
import type { ClaimOutput } from "@/types";
|
|
||||||
|
|
||||||
// Tiny fixture — just enough to exercise the card's UI surface.
|
|
||||||
const CLAIM: ClaimOutput = {
|
|
||||||
claim_id: "CLM-TEST",
|
|
||||||
subscriber: { first_name: "Jane", last_name: "Doe", member_id: "MEM-1" },
|
|
||||||
payer: { name: "Test Payer", id: "P1" },
|
|
||||||
billing_provider: { npi: "1234567890" },
|
|
||||||
claim: {
|
|
||||||
total_charge: 100,
|
|
||||||
place_of_service: "11",
|
|
||||||
frequency_code: "1",
|
|
||||||
prior_auth: null,
|
|
||||||
},
|
|
||||||
service_lines: [
|
|
||||||
{
|
|
||||||
line_number: 1,
|
|
||||||
procedure: { qualifier: "HC", code: "99213", modifiers: [] },
|
|
||||||
charge: "100",
|
|
||||||
units: "1",
|
|
||||||
unit_type: "UN",
|
|
||||||
service_date: "2026-06-01",
|
|
||||||
},
|
|
||||||
],
|
|
||||||
diagnoses: [{ qualifier: "ABK", code: "J20.9" }],
|
|
||||||
validation: { passed: true, errors: [], warnings: [] },
|
|
||||||
};
|
|
||||||
|
|
||||||
function renderCard(
|
|
||||||
props: Partial<React.ComponentProps<typeof ClaimCard837>> = {},
|
|
||||||
): {
|
|
||||||
container: HTMLDivElement;
|
|
||||||
unmount: () => void;
|
|
||||||
} {
|
|
||||||
const container = document.createElement("div");
|
|
||||||
document.body.appendChild(container);
|
|
||||||
const root: Root = createRoot(container);
|
|
||||||
const onToggleSelect = props.onToggleSelect ?? (() => {});
|
|
||||||
act(() => {
|
|
||||||
root.render(
|
|
||||||
React.createElement(
|
|
||||||
MemoryRouter,
|
|
||||||
null,
|
|
||||||
React.createElement(ClaimCard837, {
|
|
||||||
claim: CLAIM,
|
|
||||||
selected: false,
|
|
||||||
onToggleSelect,
|
|
||||||
...props,
|
|
||||||
}),
|
|
||||||
),
|
|
||||||
);
|
|
||||||
});
|
|
||||||
return {
|
|
||||||
container,
|
|
||||||
unmount: () => {
|
|
||||||
act(() => root.unmount());
|
|
||||||
container.remove();
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("ClaimCard837 — checkbox + select interaction", () => {
|
|
||||||
beforeEach(() => {
|
|
||||||
useAppStore.setState({ parsedBatches: [] });
|
|
||||||
});
|
|
||||||
afterEach(() => {
|
|
||||||
vi.restoreAllMocks();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("renders a checkbox with checked={selected}", () => {
|
|
||||||
const { container, unmount } = renderCard({ selected: true });
|
|
||||||
const cb = container.querySelector<HTMLInputElement>(
|
|
||||||
'input[type="checkbox"]',
|
|
||||||
);
|
|
||||||
expect(cb).not.toBeNull();
|
|
||||||
expect(cb!.checked).toBe(true);
|
|
||||||
unmount();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("renders an unchecked checkbox when selected={false}", () => {
|
|
||||||
const { container, unmount } = renderCard({ selected: false });
|
|
||||||
const cb = container.querySelector<HTMLInputElement>(
|
|
||||||
'input[type="checkbox"]',
|
|
||||||
);
|
|
||||||
expect(cb!.checked).toBe(false);
|
|
||||||
unmount();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("clicking the checkbox fires onToggleSelect with the claim_id", () => {
|
|
||||||
const onToggleSelect = vi.fn();
|
|
||||||
const { container, unmount } = renderCard({ onToggleSelect });
|
|
||||||
const cb = container.querySelector<HTMLInputElement>(
|
|
||||||
'input[type="checkbox"]',
|
|
||||||
)!;
|
|
||||||
act(() => {
|
|
||||||
cb.click();
|
|
||||||
});
|
|
||||||
expect(onToggleSelect).toHaveBeenCalledWith("CLM-TEST");
|
|
||||||
unmount();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("clicking the checkbox does NOT toggle the card's expand state", () => {
|
|
||||||
// Regression guard: the card body is a <button> that toggles expand
|
|
||||||
// on click. The new checkbox must not bubble its click into that
|
|
||||||
// button (which would also fire onToggleSelect, double-counting).
|
|
||||||
// We assert this by checking the expanded content (the service lines
|
|
||||||
// table) is NOT in the DOM after a checkbox click.
|
|
||||||
const { container, unmount } = renderCard();
|
|
||||||
const cb = container.querySelector<HTMLInputElement>(
|
|
||||||
'input[type="checkbox"]',
|
|
||||||
)!;
|
|
||||||
act(() => {
|
|
||||||
cb.click();
|
|
||||||
});
|
|
||||||
// The service lines table only renders when the card is expanded.
|
|
||||||
const table = container.querySelector("table");
|
|
||||||
expect(table).toBeNull();
|
|
||||||
unmount();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("clicking the card body still toggles expand (existing behavior preserved)", () => {
|
|
||||||
const { container, unmount } = renderCard();
|
|
||||||
// The card body is a <button aria-expanded="false">. Clicking it
|
|
||||||
// should reveal the expanded details (service lines table).
|
|
||||||
const expandButton = container.querySelector<HTMLButtonElement>(
|
|
||||||
'button[aria-expanded]',
|
|
||||||
);
|
|
||||||
expect(expandButton).not.toBeNull();
|
|
||||||
act(() => {
|
|
||||||
expandButton!.click();
|
|
||||||
});
|
|
||||||
// After expand, the service lines table is in the DOM.
|
|
||||||
const table = container.querySelector("table");
|
|
||||||
expect(table).not.toBeNull();
|
|
||||||
unmount();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("checkbox has an accessible label that names the claim", () => {
|
|
||||||
const { container, unmount } = renderCard();
|
|
||||||
const cb = container.querySelector<HTMLInputElement>(
|
|
||||||
'input[type="checkbox"]',
|
|
||||||
);
|
|
||||||
// aria-label or wrapping <label> — at minimum the input must be
|
|
||||||
// findable by an accessible name. The simplest assertion is that an
|
|
||||||
// aria-label exists on the input itself.
|
|
||||||
expect(
|
|
||||||
cb!.getAttribute("aria-label") ||
|
|
||||||
cb!.closest("label")?.textContent,
|
|
||||||
).toContain("CLM-TEST");
|
|
||||||
unmount();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -1,352 +0,0 @@
|
|||||||
// ClaimCard837 — the warm-paper card that represents one 837P claim in
|
|
||||||
// the Upload page's streaming section. Originally inlined in
|
|
||||||
// `src/pages/Upload.tsx`; extracted into its own file in SP9 (June 2026)
|
|
||||||
// so the per-claim select-for-export interaction can live here without
|
|
||||||
// bloating the page.
|
|
||||||
//
|
|
||||||
// Two click targets, by design (see approach A in the batch-export
|
|
||||||
// design spec): a leading checkbox column for selection, and the rest
|
|
||||||
// of the card (a <button>) for expand/collapse. Putting the checkbox
|
|
||||||
// inside the button would be an a11y anti-pattern (nested interactive
|
|
||||||
// elements).
|
|
||||||
|
|
||||||
import { useMemo, useState } from "react";
|
|
||||||
import { useNavigate } from "react-router-dom";
|
|
||||||
import {
|
|
||||||
AlertTriangle,
|
|
||||||
ArrowRight,
|
|
||||||
ChevronRight,
|
|
||||||
XCircle,
|
|
||||||
} from "lucide-react";
|
|
||||||
import { Badge } from "@/components/ui/badge";
|
|
||||||
import { useAppStore } from "@/store";
|
|
||||||
import { fmt, toNum } from "@/lib/format";
|
|
||||||
import { cn } from "@/lib/utils";
|
|
||||||
import type { ClaimOutput, ServiceLine } from "@/types";
|
|
||||||
import { StatPill, ValidationDot } from "./ClaimCard/shared";
|
|
||||||
|
|
||||||
export interface ClaimCard837Props {
|
|
||||||
claim: ClaimOutput;
|
|
||||||
/**
|
|
||||||
* Whether this card is currently selected for the batch export.
|
|
||||||
* Controlled by the parent (Upload.tsx) via the per-claim checkbox.
|
|
||||||
* The checkbox column only renders when the parent passes
|
|
||||||
* `onToggleSelect` — keeps the component backward-compatible with
|
|
||||||
* callers that don't need the export flow (e.g. the persisted
|
|
||||||
* batch's "Recent batches" view, when we add it).
|
|
||||||
*/
|
|
||||||
selected?: boolean;
|
|
||||||
/** Called with the claim's id when the user clicks the checkbox. */
|
|
||||||
onToggleSelect?: (claimId: string) => void;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function ClaimCard837({ claim, selected = false, onToggleSelect }: ClaimCard837Props) {
|
|
||||||
const [open, setOpen] = useState(false);
|
|
||||||
const passed = claim.validation.passed;
|
|
||||||
const hasWarnings = claim.validation.warnings.length > 0;
|
|
||||||
// SP21 Phase 5 Task 5.7: a "See claim in detail →" link drills to
|
|
||||||
// /claims?claim=ID — but ONLY when this streamed claim_id has
|
|
||||||
// actually been persisted to a parsed batch. The Upload page
|
|
||||||
// streams claims as the parser emits them; until the user clicks
|
|
||||||
// "Save batch" the claim isn't visible to ClaimDrawer.
|
|
||||||
const parsedBatches = useAppStore((s) => s.parsedBatches);
|
|
||||||
const persistedClaimIds = useMemo(
|
|
||||||
() => new Set(parsedBatches.flatMap((b) => b.claimIds)),
|
|
||||||
[parsedBatches],
|
|
||||||
);
|
|
||||||
const canDrill = persistedClaimIds.has(claim.claim_id);
|
|
||||||
const navigate = useNavigate();
|
|
||||||
|
|
||||||
const showCheckbox = typeof onToggleSelect === "function";
|
|
||||||
|
|
||||||
return (
|
|
||||||
<div
|
|
||||||
className="rounded-lg overflow-hidden border flex"
|
|
||||||
style={{
|
|
||||||
backgroundColor: "hsl(36 22% 96%)",
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.10)",
|
|
||||||
boxShadow: "inset 0 1px 0 0 hsl(0 0% 100% / 0.5)",
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
{/* Selection column — only when the parent wires up selection. */}
|
|
||||||
{showCheckbox ? (
|
|
||||||
<label
|
|
||||||
className="flex items-center pl-3 pr-2 cursor-pointer shrink-0"
|
|
||||||
// Belt-and-suspenders: the label is a sibling of the button
|
|
||||||
// below, so click events don't bubble into it. stopPropagation
|
|
||||||
// here guards against future refactors that might nest the
|
|
||||||
// label inside the button.
|
|
||||||
onClick={(e) => e.stopPropagation()}
|
|
||||||
data-testid={`claim-card-select-${claim.claim_id}`}
|
|
||||||
>
|
|
||||||
<input
|
|
||||||
type="checkbox"
|
|
||||||
checked={selected}
|
|
||||||
onChange={() => onToggleSelect?.(claim.claim_id)}
|
|
||||||
aria-label={`Select claim ${claim.claim_id} for export`}
|
|
||||||
className="h-3.5 w-3.5 rounded border-border accent-accent cursor-pointer"
|
|
||||||
/>
|
|
||||||
</label>
|
|
||||||
) : null}
|
|
||||||
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
onClick={() => setOpen((v) => !v)}
|
|
||||||
className={cn(
|
|
||||||
"flex-1 min-w-0 text-left px-4 py-3 hover:bg-[hsl(36_22%_92%)] transition-colors",
|
|
||||||
!showCheckbox && "rounded-l-lg",
|
|
||||||
)}
|
|
||||||
aria-expanded={open}
|
|
||||||
>
|
|
||||||
<div className="flex items-center gap-3">
|
|
||||||
<ChevronRight
|
|
||||||
className={cn(
|
|
||||||
"h-3.5 w-3.5 transition-transform shrink-0",
|
|
||||||
open && "rotate-90"
|
|
||||||
)}
|
|
||||||
strokeWidth={1.75}
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
/>
|
|
||||||
<div className="flex-1 min-w-0">
|
|
||||||
<div className="flex items-center gap-2.5 flex-wrap">
|
|
||||||
<span
|
|
||||||
className="display mono text-[12.5px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{claim.claim_id}
|
|
||||||
</span>
|
|
||||||
<span
|
|
||||||
className="text-[12px] truncate"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-2))" }}
|
|
||||||
>
|
|
||||||
{claim.subscriber.first_name} {claim.subscriber.last_name}
|
|
||||||
</span>
|
|
||||||
<span
|
|
||||||
className="text-[12px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
· {claim.payer.name}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
<div
|
|
||||||
className="mono text-[10.5px] flex items-center gap-2 mt-1"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
<span>NPI {claim.billing_provider.npi}</span>
|
|
||||||
<span className="opacity-40">·</span>
|
|
||||||
<span>
|
|
||||||
{claim.service_lines.length} line
|
|
||||||
{claim.service_lines.length === 1 ? "" : "s"}
|
|
||||||
</span>
|
|
||||||
<span className="opacity-40">·</span>
|
|
||||||
<span>{claim.diagnoses.length} dx</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div className="text-right shrink-0">
|
|
||||||
<div
|
|
||||||
className="display mono text-[14px]"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{fmt.usdDecimal(claim.claim.total_charge)}
|
|
||||||
</div>
|
|
||||||
<div className="mt-0.5">
|
|
||||||
<ValidationDot passed={passed} hasWarnings={hasWarnings} />
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</button>
|
|
||||||
{open ? (
|
|
||||||
<div
|
|
||||||
className="basis-full border-t px-4 py-3 grid gap-4 animate-fade-in"
|
|
||||||
style={{
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.08)",
|
|
||||||
backgroundColor: "hsl(36 22% 92%)",
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<div className="grid grid-cols-2 md:grid-cols-4 gap-4">
|
|
||||||
<StatPill label="Member" value={claim.subscriber.member_id} />
|
|
||||||
<StatPill
|
|
||||||
label="Place of service"
|
|
||||||
value={claim.claim.place_of_service ?? "—"}
|
|
||||||
/>
|
|
||||||
<StatPill label="Frequency" value={claim.claim.frequency_code ?? "—"} />
|
|
||||||
<StatPill
|
|
||||||
label="Prior auth"
|
|
||||||
value={claim.claim.prior_auth ?? "—"}
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{claim.diagnoses.length > 0 ? (
|
|
||||||
<div>
|
|
||||||
<div
|
|
||||||
className="mono text-[10px] uppercase tracking-[0.18em] font-semibold mb-1.5"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
Diagnoses
|
|
||||||
</div>
|
|
||||||
<div className="flex flex-wrap gap-1.5">
|
|
||||||
{claim.diagnoses.map((d, i) => (
|
|
||||||
<Badge key={`${d.code}-${i}`} variant="muted">
|
|
||||||
{d.qualifier ? `${d.qualifier}·` : ""}
|
|
||||||
{d.code}
|
|
||||||
</Badge>
|
|
||||||
))}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
|
|
||||||
{claim.service_lines.length > 0 ? (
|
|
||||||
<div>
|
|
||||||
<div
|
|
||||||
className="mono text-[10px] uppercase tracking-[0.18em] font-semibold mb-1.5"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
Service lines
|
|
||||||
</div>
|
|
||||||
<div
|
|
||||||
className="rounded-md border overflow-x-auto"
|
|
||||||
style={{
|
|
||||||
borderColor: "hsl(30 14% 14% / 0.10)",
|
|
||||||
backgroundColor: "hsl(36 22% 98%)",
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<table className="w-full text-[12px]">
|
|
||||||
<thead style={{ backgroundColor: "hsl(36 22% 90%)" }}>
|
|
||||||
<tr className="text-left" style={{ color: "hsl(var(--surface-ink-2))" }}>
|
|
||||||
<th className="px-2.5 py-1.5 font-medium">#</th>
|
|
||||||
<th className="px-2.5 py-1.5 font-medium">Code</th>
|
|
||||||
<th className="px-2.5 py-1.5 font-medium">Mods</th>
|
|
||||||
<th className="px-2.5 py-1.5 font-medium">Date</th>
|
|
||||||
<th className="px-2.5 py-1.5 font-medium">Units</th>
|
|
||||||
<th className="px-2.5 py-1.5 font-medium text-right">Charge</th>
|
|
||||||
</tr>
|
|
||||||
</thead>
|
|
||||||
<tbody>
|
|
||||||
{claim.service_lines.map((line) => (
|
|
||||||
<ServiceLine837Row key={line.line_number} line={line} />
|
|
||||||
))}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
|
|
||||||
{(claim.validation.errors.length > 0 ||
|
|
||||||
claim.validation.warnings.length > 0) ? (
|
|
||||||
<div>
|
|
||||||
<div
|
|
||||||
className="mono text-[10px] uppercase tracking-[0.18em] font-semibold mb-1.5"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
Validation
|
|
||||||
</div>
|
|
||||||
<ul className="space-y-1 text-[12px]">
|
|
||||||
{claim.validation.errors.map((issue, i) => (
|
|
||||||
<li
|
|
||||||
key={`e-${i}`}
|
|
||||||
className="flex items-start gap-2 text-destructive"
|
|
||||||
>
|
|
||||||
<XCircle
|
|
||||||
className="h-3.5 w-3.5 mt-0.5 shrink-0"
|
|
||||||
strokeWidth={1.75}
|
|
||||||
/>
|
|
||||||
<span>
|
|
||||||
<span className="mono">{issue.rule}</span> — {issue.message}
|
|
||||||
</span>
|
|
||||||
</li>
|
|
||||||
))}
|
|
||||||
{claim.validation.warnings.map((issue, i) => (
|
|
||||||
<li
|
|
||||||
key={`w-${i}`}
|
|
||||||
className="flex items-start gap-2 text-[hsl(var(--warning))]"
|
|
||||||
>
|
|
||||||
<AlertTriangle
|
|
||||||
className="h-3.5 w-3.5 mt-0.5 shrink-0"
|
|
||||||
strokeWidth={1.75}
|
|
||||||
/>
|
|
||||||
<span>
|
|
||||||
<span className="mono">{issue.rule}</span> — {issue.message}
|
|
||||||
</span>
|
|
||||||
</li>
|
|
||||||
))}
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
|
|
||||||
{/* SP21 Phase 5 Task 5.7: drill to the persisted claim. Only
|
|
||||||
renders when this streamed claim_id has been saved into
|
|
||||||
a parsed batch (so ClaimDrawer can find it). Before
|
|
||||||
"Save batch" the claim is streaming-only and the link
|
|
||||||
would 404. */}
|
|
||||||
{canDrill ? (
|
|
||||||
<div className="pt-1">
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
onClick={() =>
|
|
||||||
navigate(
|
|
||||||
`/claims?claim=${encodeURIComponent(claim.claim_id)}`,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
data-testid="upload-claim-drill"
|
|
||||||
aria-label={`See claim ${claim.claim_id} in detail`}
|
|
||||||
className="inline-flex items-center gap-1.5 text-[12px] mono font-semibold cursor-pointer rounded-sm px-1 -mx-1 hover:underline focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-1"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
See claim in detail
|
|
||||||
<ArrowRight className="h-3 w-3" strokeWidth={2} />
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function ServiceLine837Row({ line }: { line: ServiceLine }) {
|
|
||||||
return (
|
|
||||||
<tr
|
|
||||||
className="border-t"
|
|
||||||
style={{ borderColor: "hsl(30 14% 14% / 0.08)" }}
|
|
||||||
>
|
|
||||||
<td
|
|
||||||
className="px-2.5 py-1.5 mono"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{line.line_number}
|
|
||||||
</td>
|
|
||||||
<td
|
|
||||||
className="px-2.5 py-1.5 mono"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{line.procedure.qualifier}·{line.procedure.code}
|
|
||||||
</td>
|
|
||||||
<td
|
|
||||||
className="px-2.5 py-1.5 mono"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
{line.procedure.modifiers.length > 0
|
|
||||||
? line.procedure.modifiers.join(", ")
|
|
||||||
: "—"}
|
|
||||||
</td>
|
|
||||||
<td
|
|
||||||
className="px-2.5 py-1.5 mono"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
{line.service_date ?? "—"}
|
|
||||||
</td>
|
|
||||||
<td
|
|
||||||
className="px-2.5 py-1.5 mono"
|
|
||||||
style={{ color: "hsl(var(--surface-ink-3))" }}
|
|
||||||
>
|
|
||||||
{line.units ? `${toNum(line.units)} ${line.unit_type ?? ""}`.trim() : "—"}
|
|
||||||
</td>
|
|
||||||
<td
|
|
||||||
className="px-2.5 py-1.5 mono text-right display"
|
|
||||||
style={{ color: "hsl(var(--surface-ink))" }}
|
|
||||||
>
|
|
||||||
{fmt.usdDecimal(line.charge)}
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -9,7 +9,6 @@ import { createRoot, type Root } from "react-dom/client";
|
|||||||
import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
|
import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
|
||||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||||
import { ClaimDrawer } from "./ClaimDrawer";
|
import { ClaimDrawer } from "./ClaimDrawer";
|
||||||
import { DrillStackProvider } from "@/components/drill/DrillStackProvider";
|
|
||||||
import { api, ApiError } from "@/lib/api";
|
import { api, ApiError } from "@/lib/api";
|
||||||
import type { ClaimDetail } from "@/types";
|
import type { ClaimDetail } from "@/types";
|
||||||
|
|
||||||
@@ -163,22 +162,14 @@ function renderDrawer(
|
|||||||
React.createElement(
|
React.createElement(
|
||||||
QueryClientProvider,
|
QueryClientProvider,
|
||||||
{ client: qc },
|
{ client: qc },
|
||||||
// SP21 Phase 5 Task 5.8: PartiesGrid (mounted by ClaimDrawer)
|
|
||||||
// calls useDrillStack(). Wrap the drawer in DrillStackProvider
|
|
||||||
// so the hook has a context. (The provider is also mounted at
|
|
||||||
// the App root in production.)
|
|
||||||
React.createElement(
|
|
||||||
DrillStackProvider,
|
|
||||||
null,
|
|
||||||
React.createElement(ClaimDrawer, {
|
React.createElement(ClaimDrawer, {
|
||||||
claimId: props.claimId,
|
claimId: props.claimId,
|
||||||
claims,
|
claims,
|
||||||
onClose,
|
onClose,
|
||||||
onNavigate,
|
onNavigate,
|
||||||
onToggleHelp,
|
onToggleHelp,
|
||||||
}),
|
})
|
||||||
),
|
)
|
||||||
),
|
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -141,14 +141,12 @@ export function ClaimDrawer({
|
|||||||
// Right-anchored side panel. We override the Dialog primitive's
|
// Right-anchored side panel. We override the Dialog primitive's
|
||||||
// default centered positioning (`left-1/2 top-1/2 -translate-x-1/2
|
// default centered positioning (`left-1/2 top-1/2 -translate-x-1/2
|
||||||
// -translate-y-1/2`) by re-asserting `right-0 top-0 translate-x-0
|
// -translate-y-1/2`) by re-asserting `right-0 top-0 translate-x-0
|
||||||
// translate-y-0`. `h-full` + `rounded-none` + the left hairline
|
// translate-y-0`. `h-full` + `rounded-none` + the left border
|
||||||
// give it the visual identity of a drawer rather than a modal.
|
// give it the visual identity of a drawer rather than a modal.
|
||||||
// `bg-card` matches the rest of the dark instrument chrome; the
|
|
||||||
// outer shadow is a soft directional falloff to the left.
|
|
||||||
// `aria-describedby={undefined}` suppresses the Radix warning
|
// `aria-describedby={undefined}` suppresses the Radix warning
|
||||||
// about a missing description — the drawer content is its own
|
// about a missing description — the drawer content is its own
|
||||||
// description and there's nothing useful to point at.
|
// description and there's nothing useful to point at.
|
||||||
className="fixed right-0 top-0 h-full w-full max-w-2xl translate-x-0 translate-y-0 rounded-none border-l border-border/60 bg-card p-0 shadow-[-24px_0_60px_-12px_rgba(0,0,0,0.6)]"
|
className="fixed right-0 top-0 h-full w-full max-w-2xl translate-x-0 translate-y-0 rounded-none border-l border-[color:var(--m-border-heavy)]/60 bg-[color:var(--m-surface)] p-0 shadow-[-24px_0_60px_-12px_rgba(0,0,0,0.6)]"
|
||||||
data-testid="claim-drawer"
|
data-testid="claim-drawer"
|
||||||
aria-describedby={undefined}
|
aria-describedby={undefined}
|
||||||
>
|
>
|
||||||
@@ -169,7 +167,7 @@ export function ClaimDrawer({
|
|||||||
>
|
>
|
||||||
<ClaimDrawerHeader claim={data} onClose={onClose} />
|
<ClaimDrawerHeader claim={data} onClose={onClose} />
|
||||||
<div
|
<div
|
||||||
className="flex gap-0 px-6 border-b border-border/40"
|
className="flex gap-0 px-6 border-b border-[color:var(--surface-line)]/40"
|
||||||
data-testid="claim-drawer-tabs"
|
data-testid="claim-drawer-tabs"
|
||||||
>
|
>
|
||||||
<button
|
<button
|
||||||
@@ -178,15 +176,15 @@ export function ClaimDrawer({
|
|||||||
className={cn(
|
className={cn(
|
||||||
"relative px-3 py-2.5 text-[10.5px] font-semibold uppercase tracking-[0.18em] transition-colors",
|
"relative px-3 py-2.5 text-[10.5px] font-semibold uppercase tracking-[0.18em] transition-colors",
|
||||||
activeTab === "details"
|
activeTab === "details"
|
||||||
? "text-foreground"
|
? "text-[color:var(--surface-ink)]"
|
||||||
: "text-muted-foreground hover:text-foreground"
|
: "text-[color:var(--surface-ink-3)] hover:text-[color:var(--surface-ink-2)]"
|
||||||
)}
|
)}
|
||||||
data-testid="tab-button-details"
|
data-testid="tab-button-details"
|
||||||
data-active={activeTab === "details" ? "true" : "false"}
|
data-active={activeTab === "details" ? "true" : "false"}
|
||||||
>
|
>
|
||||||
Details
|
Details
|
||||||
{activeTab === "details" ? (
|
{activeTab === "details" ? (
|
||||||
<span className="absolute inset-x-0 -bottom-px h-[2px] bg-accent shadow-[0_0_8px_hsl(var(--accent)/0.6)]" />
|
<span className="absolute inset-x-0 -bottom-px h-[2px] bg-[color:var(--accent)]" />
|
||||||
) : null}
|
) : null}
|
||||||
</button>
|
</button>
|
||||||
<button
|
<button
|
||||||
@@ -195,29 +193,29 @@ export function ClaimDrawer({
|
|||||||
className={cn(
|
className={cn(
|
||||||
"relative px-3 py-2.5 text-[10.5px] font-semibold uppercase tracking-[0.18em] transition-colors",
|
"relative px-3 py-2.5 text-[10.5px] font-semibold uppercase tracking-[0.18em] transition-colors",
|
||||||
activeTab === "line-reconciliation"
|
activeTab === "line-reconciliation"
|
||||||
? "text-foreground"
|
? "text-[color:var(--surface-ink)]"
|
||||||
: "text-muted-foreground hover:text-foreground"
|
: "text-[color:var(--surface-ink-3)] hover:text-[color:var(--surface-ink-2)]"
|
||||||
)}
|
)}
|
||||||
data-testid="tab-button-line-reconciliation"
|
data-testid="tab-button-line-reconciliation"
|
||||||
data-active={activeTab === "line-reconciliation" ? "true" : "false"}
|
data-active={activeTab === "line-reconciliation" ? "true" : "false"}
|
||||||
>
|
>
|
||||||
Line Reconciliation
|
Line Reconciliation
|
||||||
{activeTab === "line-reconciliation" ? (
|
{activeTab === "line-reconciliation" ? (
|
||||||
<span className="absolute inset-x-0 -bottom-px h-[2px] bg-accent shadow-[0_0_8px_hsl(var(--accent)/0.6)]" />
|
<span className="absolute inset-x-0 -bottom-px h-[2px] bg-[color:var(--accent)]" />
|
||||||
) : null}
|
) : null}
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
{activeTab === "line-reconciliation" ? (
|
{activeTab === "line-reconciliation" ? (
|
||||||
lr.loading ? (
|
lr.loading ? (
|
||||||
<p
|
<p
|
||||||
className="px-6 py-4 text-sm text-muted-foreground"
|
className="px-6 py-4 text-sm text-[color:var(--m-ink-tertiary)]"
|
||||||
data-testid="line-reconciliation-loading"
|
data-testid="line-reconciliation-loading"
|
||||||
>
|
>
|
||||||
Loading line reconciliation…
|
Loading line reconciliation…
|
||||||
</p>
|
</p>
|
||||||
) : lr.error ? (
|
) : lr.error ? (
|
||||||
<p
|
<p
|
||||||
className="px-6 py-4 text-sm text-muted-foreground"
|
className="px-6 py-4 text-sm text-[color:var(--m-ink-tertiary)]"
|
||||||
data-testid="line-reconciliation-error"
|
data-testid="line-reconciliation-error"
|
||||||
>
|
>
|
||||||
Failed to load line reconciliation.
|
Failed to load line reconciliation.
|
||||||
@@ -226,7 +224,7 @@ export function ClaimDrawer({
|
|||||||
<LineReconciliationTab data={lr.data} />
|
<LineReconciliationTab data={lr.data} />
|
||||||
) : null
|
) : null
|
||||||
) : (
|
) : (
|
||||||
<div className="flex flex-col divide-y divide-border/40">
|
<div className="flex flex-col divide-y divide-[color:var(--surface-line)]/12">
|
||||||
<ValidationPanel validation={data.validation} />
|
<ValidationPanel validation={data.validation} />
|
||||||
<ServiceLinesTable
|
<ServiceLinesTable
|
||||||
serviceLines={data.serviceLines}
|
serviceLines={data.serviceLines}
|
||||||
|
|||||||
@@ -32,21 +32,21 @@ export function ClaimDrawerError({ kind, onRetry, onClose }: ClaimDrawerErrorPro
|
|||||||
|
|
||||||
return (
|
return (
|
||||||
<div
|
<div
|
||||||
className="flex flex-col items-start gap-4 p-6 bg-card text-foreground"
|
className="flex flex-col items-start gap-4 p-6 bg-[color:var(--m-surface)] text-[color:var(--m-ink-primary)]"
|
||||||
role="alert"
|
role="alert"
|
||||||
data-testid={`claim-drawer-error-${kind}`}
|
data-testid={`claim-drawer-error-${kind}`}
|
||||||
>
|
>
|
||||||
<div className="flex items-center gap-2">
|
<div className="flex items-center gap-2">
|
||||||
<Icon
|
<Icon
|
||||||
className="h-5 w-5 text-destructive"
|
className="h-5 w-5 text-[color:var(--m-error)]"
|
||||||
strokeWidth={1.75}
|
strokeWidth={1.75}
|
||||||
aria-hidden
|
aria-hidden
|
||||||
/>
|
/>
|
||||||
<span className="eyebrow text-destructive">
|
<span className="eyebrow text-[color:var(--m-error)]">
|
||||||
{eyebrow}
|
{eyebrow}
|
||||||
</span>
|
</span>
|
||||||
</div>
|
</div>
|
||||||
<p className="text-sm text-muted-foreground max-w-sm">{message}</p>
|
<p className="text-sm text-[color:var(--m-ink-secondary)] max-w-sm">{message}</p>
|
||||||
<div className="flex items-center gap-2">
|
<div className="flex items-center gap-2">
|
||||||
{kind === "network" && onRetry ? (
|
{kind === "network" && onRetry ? (
|
||||||
<Button variant="outline" size="sm" onClick={onRetry} data-testid="error-retry">
|
<Button variant="outline" size="sm" onClick={onRetry} data-testid="error-retry">
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user