Compare commits
87 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 65d98cf674 | |||
| d15c04d983 | |||
| 980627b675 | |||
| 1db6b8841c | |||
| 0678e25de7 | |||
| 5b3b8619e6 | |||
| b6607b2009 | |||
| c0b7924aad | |||
| ebd2834cc4 | |||
| 888c99d848 | |||
| 388ea6e49b | |||
| 736bf4d333 | |||
| 7db5e448cd | |||
| 5d6b5894f3 | |||
| 88da3a6246 | |||
| 50dc0b2fb3 | |||
| e6ae364dad | |||
| 50a454db59 | |||
| 9129543f5d | |||
| fb7a5c6d2e | |||
| 7dd6a5d025 | |||
| be93dbff72 | |||
| feb15bf48d | |||
| ff0985d244 | |||
| 94990932f9 | |||
| 9133baa070 | |||
| d0a18bd796 | |||
| 646d00adde | |||
| f6f821e082 | |||
| 3e00fb3f63 | |||
| 2faf7bfd48 | |||
| 73be586110 | |||
| d7f37f845a | |||
| df10b55a34 | |||
| 6300280142 | |||
| fff000ed2e | |||
| 134eb4f404 | |||
| 022b229d81 | |||
| dc70bdacf6 | |||
| 3d9f224eac | |||
| 4f16fac937 | |||
| 1f2259d413 | |||
| 191162964e | |||
| 5e7e9e2e68 | |||
| 4c591ee655 | |||
| 963e608b10 | |||
| 4bdca6168c | |||
| b0f0bd9d73 | |||
| 870632604c | |||
| 7e35bd9dee | |||
| 3fa61bb3f6 | |||
| 2e07d7eaf7 | |||
| b329c9eb6c | |||
| 2d7d73a9db | |||
| 02b06bf4d9 | |||
| b6ca171209 | |||
| e4945c4274 | |||
| 5b0e945f1b | |||
| 1942a22629 | |||
| b9260d9969 | |||
| 5eb490851d | |||
| f26bbe061a | |||
| c835996bd6 | |||
| c7fd7ecc0e | |||
| 47e0f80786 | |||
| daf3206d55 | |||
| f003c1f73a | |||
| 7119b7a2b1 | |||
| 40f184c858 | |||
| 1267a341e3 | |||
| a63ba5e88c | |||
| d4f6fdd49c | |||
| ab00909715 | |||
| 47902fd6b2 | |||
| 6233df1270 | |||
| 8a65baab3d | |||
| 5c9365ec33 | |||
| fdfbde35c6 | |||
| a11e051f82 | |||
| 1225013fb0 | |||
| d54c44f04a | |||
| 84d2f39760 | |||
| 62bb09f183 | |||
| 9acdcb8dbd | |||
| 2c0afbe9c5 | |||
| ae2d48102e | |||
| c62826daea |
+3
-2
@@ -35,5 +35,6 @@ claims_output/
|
|||||||
# Worktrees (subagent-driven development)
|
# Worktrees (subagent-driven development)
|
||||||
.worktrees/
|
.worktrees/
|
||||||
|
|
||||||
# Brainstorm session artifacts (visual companion mockups, events, server state)
|
# Brainstorm session artifacts (visual companion mockups, events, server state).
|
||||||
.superpowers/
|
# Skills under .superpowers/skills/ are committed project-scoped guidance.
|
||||||
|
.superpowers/brainstorm/
|
||||||
|
|||||||
@@ -0,0 +1,189 @@
|
|||||||
|
---
|
||||||
|
name: cyclone-api-router
|
||||||
|
description: "Cyclone FastAPI router conventions (api_routers/, api_helpers.py, response shapes, error envelopes). Use when: adding or changing an HTTP endpoint, splitting a route out of api.py, or wiring a new helper into api_helpers.py."
|
||||||
|
---
|
||||||
|
|
||||||
|
# cyclone-api-router
|
||||||
|
|
||||||
|
Cyclone splits its FastAPI surface two ways: small resource-group
|
||||||
|
routers live in `backend/src/cyclone/api_routers/<topic>.py` and are
|
||||||
|
mounted bare into `api.py`; the high-traffic streaming and parse
|
||||||
|
endpoints still live as top-level decorators in `api.py` itself. This
|
||||||
|
skill codifies the conventions so additions stay consistent with the
|
||||||
|
four routers already shipped (`acks`, `admin`, `health`, `ta1_acks`).
|
||||||
|
|
||||||
|
As of this writing: **4 router modules** under
|
||||||
|
`backend/src/cyclone/api_routers/`, **one shared helpers module** at
|
||||||
|
`backend/src/cyclone/api_helpers.py` (248 lines, NDJSON primitives +
|
||||||
|
content negotiation + `tail_events`), and ~30 routes still inlined
|
||||||
|
in `backend/src/cyclone/api.py`. The next refactor target is the
|
||||||
|
parse endpoints.
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- **Adding an endpoint.** You're adding a new GET / POST handler —
|
||||||
|
you need to know whether it belongs in `api.py` (parse / streaming)
|
||||||
|
or in a new router under `api_routers/`, and what the response
|
||||||
|
shape and test file conventions look like.
|
||||||
|
- **Splitting a route.** You're moving a route out of `api.py` into a
|
||||||
|
dedicated `api_routers/<topic>.py` module and need the import /
|
||||||
|
mounting rules (`from cyclone.api_routers import <topic>` then
|
||||||
|
`app.include_router(<topic>.router)`).
|
||||||
|
- **Adding a helper.** You're wiring a new function into
|
||||||
|
`api_helpers.py` (NDJSON primitive, content-negotiation probe,
|
||||||
|
tail-event helper) and need to keep it private to the API layer.
|
||||||
|
- **Defining an error response.** You're raising from a route handler
|
||||||
|
and need the conventional `HTTPException(status_code=..., detail=...)`
|
||||||
|
shape used everywhere else in the API surface.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
1. **No new top-level routes in `api.py` for resource groups.** Any
|
||||||
|
endpoint grouped under a resource (`/api/<resource>` and its
|
||||||
|
`/{id}` detail) lives in `backend/src/cyclone/api_routers/<topic>.py`
|
||||||
|
as an `APIRouter`. The streaming list endpoints (`/api/claims/stream`,
|
||||||
|
`/api/remittances/stream`, `/api/activity/stream`) and the parse
|
||||||
|
endpoints (`/api/parse-*`) currently stay in `api.py` because they
|
||||||
|
span multiple store modules — don't move them unless you're also
|
||||||
|
restructuring the store split.
|
||||||
|
2. **Reuse `api_helpers.py`.** NDJSON primitives (`ndjson_line`,
|
||||||
|
`ndjson_stream_list`, `ndjson_stream_837`, `ndjson_stream_835`),
|
||||||
|
content negotiation (`client_wants_json`, `wants_ndjson`), the
|
||||||
|
strict / `raw_segments` rewrites (`strict_rewrite_837`,
|
||||||
|
`strict_rewrite_835`, `drop_raw_segments_837`, `drop_raw_segments_835`),
|
||||||
|
and the shared live-tail generator (`tail_events`,
|
||||||
|
`heartbeat_seconds`) all live there. Don't duplicate them in a
|
||||||
|
router. The module's docstring (`api_helpers.py:1-18`) declares it
|
||||||
|
private to the API layer — no business logic, no DB writes.
|
||||||
|
3. **Response shape.** Every successful response is a **plain dict**
|
||||||
|
produced by a per-router `<entity>_to_ui(row)` helper (see
|
||||||
|
`_ack_to_ui` at `api_routers/acks.py:29-48` and `_ta1_to_ui` at
|
||||||
|
`api_routers/ta1_acks.py:23-37`). This dict shape **must** match
|
||||||
|
the matching `<entity>_written` event payload so live-tail pages
|
||||||
|
don't drift (see `cyclone-store` for the serializer contract).
|
||||||
|
Errors use FastAPI's `HTTPException` with a `detail` dict of the
|
||||||
|
form `{"error": "<Title>", "detail": "<message>"}` —
|
||||||
|
`acks.py:85-88`, `ta1_acks.py:72`. There is **no** shared
|
||||||
|
`ErrorEnvelope` Pydantic model; the `detail` dict is the contract.
|
||||||
|
4. **Mounting.** Routers are mounted bare in `api.py:251-256` —
|
||||||
|
`app.include_router(<name>.router)` with **no** `prefix=`
|
||||||
|
argument. Each `@router.<verb>` decorator carries the **full**
|
||||||
|
`/api/<resource>` path itself (see `acks.py:51,75`,
|
||||||
|
`ta1_acks.py:49,67`, `admin.py:23`, `health.py:28`). The
|
||||||
|
`router = APIRouter()` declaration carries no `tags=` either —
|
||||||
|
keep it minimal.
|
||||||
|
5. **Streaming endpoints.** Use
|
||||||
|
`StreamingResponse(media_type="application/x-ndjson")` and feed it
|
||||||
|
either `ndjson_stream_list(items, total, returned, has_more)` (for
|
||||||
|
list pages) or `tail_events(request, bus, kinds)` (for live-tail
|
||||||
|
pages). See `acks.py:62-66` for the list-stream skeleton and
|
||||||
|
`api.py:1357-1401` for the full live-tail pattern (snapshot →
|
||||||
|
`snapshot_end` → subscription → heartbeats). See `cyclone-tail`
|
||||||
|
for the wire format.
|
||||||
|
6. **Tests.** Every new endpoint gets a `test_api_<topic>_<verb>.py`
|
||||||
|
under `backend/tests/` (see `cyclone-tests` for the naming
|
||||||
|
convention + autouse `conftest.py`). Existing examples:
|
||||||
|
`test_api_validate_provider.py` (admin),
|
||||||
|
`test_api_parse_persists_ack.py` (acks).
|
||||||
|
|
||||||
|
## Patterns
|
||||||
|
|
||||||
|
### A new `APIRouter` skeleton
|
||||||
|
|
||||||
|
Pattern from `backend/src/cyclone/api_routers/acks.py:1-72`. Module
|
||||||
|
docstring names the resource, imports the shared helpers, declares
|
||||||
|
`router = APIRouter()` with no prefix, and defines a `_foo_to_ui(row)`
|
||||||
|
mapper at module scope.
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""``/api/foo`` — list & detail endpoints for <topic>."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Query, Request
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
|
||||||
|
from cyclone.api_helpers import ndjson_stream_list, wants_ndjson
|
||||||
|
from cyclone.store import store
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
def _foo_to_ui(row) -> dict:
|
||||||
|
"""Map a Foo ORM row to the UI shape used by ``/api/foo``."""
|
||||||
|
return {"id": row.id, "name": row.name}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/foo")
|
||||||
|
def list_foo(
|
||||||
|
request: Request,
|
||||||
|
limit: int = Query(100, ge=1, le=1000),
|
||||||
|
):
|
||||||
|
"""Return the list of persisted Foo rows, newest first."""
|
||||||
|
rows = store.list_foo()
|
||||||
|
items = [_foo_to_ui(r) for r in rows[:limit]]
|
||||||
|
total = len(rows)
|
||||||
|
returned = len(items)
|
||||||
|
has_more = total > returned
|
||||||
|
if wants_ndjson(request):
|
||||||
|
return StreamingResponse(
|
||||||
|
ndjson_stream_list(items, total, returned, has_more),
|
||||||
|
media_type="application/x-ndjson",
|
||||||
|
)
|
||||||
|
return {"items": items, "total": total, "returned": returned, "has_more": has_more}
|
||||||
|
```
|
||||||
|
|
||||||
|
### A `get_<topic>` detail endpoint with 404
|
||||||
|
|
||||||
|
Pattern from `api_routers/acks.py:75-104` and `ta1_acks.py:67-76`.
|
||||||
|
Path param is `<entity>_id` (not `id`) so it doesn't shadow
|
||||||
|
FastAPI's internal `id` and the OpenAPI docs stay self-describing.
|
||||||
|
|
||||||
|
```python
|
||||||
|
@router.get("/api/foo/{foo_id}")
|
||||||
|
def get_foo(foo_id: int) -> dict:
|
||||||
|
"""Return one persisted Foo row with its parsed detail.
|
||||||
|
|
||||||
|
Path param is ``foo_id`` (not ``id``) to avoid shadowing
|
||||||
|
FastAPI's internal ``id`` name and to keep OpenAPI docs
|
||||||
|
self-describing. Returns 404 when the row is missing — never 500.
|
||||||
|
"""
|
||||||
|
row = store.get_foo(foo_id)
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=404,
|
||||||
|
detail={"error": "Not found", "detail": f"Foo {foo_id} not found"},
|
||||||
|
)
|
||||||
|
return _foo_to_ui(row)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Mounting in `api.py`
|
||||||
|
|
||||||
|
Pattern from `backend/src/cyclone/api.py:246-256`. The block lives
|
||||||
|
just after middleware registration and just before the first
|
||||||
|
`@app.<verb>` decorator.
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Resource-group routers. Each module owns its own APIRouter and is
|
||||||
|
# registered below. New resources go in `cyclone.api_routers.<name>`
|
||||||
|
# and are wired in here.
|
||||||
|
from cyclone.api_routers import acks, admin, health, ta1_acks # noqa: E402
|
||||||
|
|
||||||
|
app.include_router(health.router)
|
||||||
|
app.include_router(acks.router)
|
||||||
|
app.include_router(ta1_acks.router)
|
||||||
|
app.include_router(admin.router)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Don't import from `cyclone.api` into a router — the dependency runs the other way.** Routers are mounted *into* `api.py` (`api_routers/acks.py` etc. know nothing about `cyclone.api`). A circular import would silently break the `from cyclone.api_routers import ...` block at `api.py:251`.
|
||||||
|
- **Don't introduce Pydantic response models where the codebase returns dicts.** Every existing list / detail endpoint returns a plain dict produced by a `<entity>_to_ui(row)` helper (see `acks.py:29-48`, `ta1_acks.py:23-37`). That dict shape **is** the event payload that `<entity>_written` carries — see `cyclone-store`. Introducing a Pydantic model on one side drifts the event payload from the list shape and silently breaks live-tail dedup.
|
||||||
|
- **Don't bypass `CycloneStore` to query the ORM directly from a route.** Always call `store.<method>(...)` (`store.list_acks()`, `store.get_ta1_ack(ack_id)`) so the read path picks up the same session + snapshot serializer as the live-tail subscriber. A raw `with db.SessionLocal()() as s: s.get(Foo, foo_id)` in a handler bypasses the serializer contract and breaks the event-payload match. See `cyclone-store`.
|
||||||
|
- **Don't set `prefix=` on `APIRouter`.** Mount the router bare (`app.include_router(<name>.router)`) and put the full `/api/<resource>` path in the decorator. Mixing the two styles scatters the URL across two files and breaks `grep "/api/foo"` audits.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- **`cyclone-store`** — most routes call `store.<method>(...)` and the dict payload **is** the `<entity>_written` event payload; load when adding a route so the read path stays aligned with the pubsub contract.
|
||||||
|
- **`cyclone-tail`** — streaming endpoints (`/api/<resource>/stream`) and the NDJSON wire format; load when adding a live-tail route or changing the wire format.
|
||||||
|
- **`cyclone-edi`** — parse endpoints (`/api/parse-837`, `/api/parse-835`, `/api/parse-999`, `/api/parse-ta1`, `/api/parse-277ca`) currently live in `api.py`; load when adding or changing a parse endpoint.
|
||||||
|
- **`cyclone-tests`** — endpoint tests follow the `test_api_<topic>_<verb>.py` naming under `backend/tests/`; load when writing the test for a new endpoint.
|
||||||
@@ -0,0 +1,187 @@
|
|||||||
|
---
|
||||||
|
name: cyclone-cli
|
||||||
|
description: "Cyclone CLI subcommand conventions (cli.py — Click group + subcommands parse-837/parse-835/validate-npi/validate-tax-id/backup, --yes + click.confirm for destructive ops, exit codes 0/1/2, CliRunner smoke tests in backend/tests/test_cli_*.py). Use when: adding a CLI subcommand, changing an exit code, adding a smoke test, or wiring a security-sensitive command (backup, key rotation, anything touching secrets.py)."
|
||||||
|
---
|
||||||
|
|
||||||
|
# cyclone-cli
|
||||||
|
|
||||||
|
The operator-facing CLI is a **Click** group at `cli.py:45` (`@click.group()` for `main`), mounted as the `cyclone` console script in `pyproject.toml:54` (`cyclone = "cyclone.cli:main"`). Seven subcommands ship today: `parse-837`, `parse-835`, `validate-npi`, `validate-tax-id`, plus the `backup` group (`init-passphrase`, `create`, `list`, `verify`, `restore`, `prune`, `status`). `serve` lives separately in `__main__.py:19` (dispatches `uvicorn cyclone.api:app`).
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- **Adding a subcommand.** You need a new operator command (e.g. `cyclone rotate-key`) and want to match the existing Click decorator + smoke-test rhythm.
|
||||||
|
- **Changing an exit code.** You're tweaking which `sys.exit(N)` a subcommand raises and need the 0/1/2 contract used by `parse_837`, `parse_835`, `validate_npi_cmd`, `validate_tax_id_cmd`, and the `backup` group.
|
||||||
|
- **Adding a smoke test.** You need `backend/tests/test_cli_<name>.py` using `click.testing.CliRunner` (NOT `subprocess.run`) and want the canonical fixture + monkeypatch layout (Keychain stub, fresh SQLite, `CYCLONE_BACKUP_DIR`).
|
||||||
|
- **Wiring a security-sensitive command.** Anything touching `cyclone/secrets.py` (Keychain writes), DB key rotation, or destructive restores needs the two-step confirm dance used by `backup restore` / `backup prune` (`cli.py:462,509`).
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
1. **Click decorator pattern, not argparse.** Each subcommand is a
|
||||||
|
top-level function decorated with `@main.command("<name>")` and
|
||||||
|
one `@click.option` / `@click.argument` per parameter. Group
|
||||||
|
dispatch is implicit — no `set_defaults(func=...)` and no
|
||||||
|
`cmd_<name>(args) -> int` signature. Top-level commands at
|
||||||
|
`cli.py:77,151,241,261,303`; the `backup` sub-group nests a
|
||||||
|
second `@main.group()` (`cli.py:303`) with its own
|
||||||
|
`@backup.command("<name>")` children.
|
||||||
|
2. **Long-form flags for safety.** Prefer `--rotate-key`,
|
||||||
|
`--backup-dir`, `--from-stdin` over positional args for anything
|
||||||
|
that mutates state or takes a secret. `init-passphrase`
|
||||||
|
(`cli.py:308-311`) demonstrates the canonical "flag OR stdin"
|
||||||
|
pattern: `--passphrase` for automation, `--from-stdin` for
|
||||||
|
interactive `getpass()` prompting.
|
||||||
|
3. **Exit codes: 0 / 1 / 2.** `0` = success. `1` = user / input
|
||||||
|
error (invalid NPI/EIN at `cli.py:258,277,285`; Keychain write
|
||||||
|
failure at `cli.py:341,351`; tampered-backup verify at
|
||||||
|
`cli.py:459`). `2` = operator / parse error
|
||||||
|
(`CycloneParseError` at `cli.py:106,178`; passphrase
|
||||||
|
empty/mismatch/short at `cli.py:331,337`). Document the codes
|
||||||
|
in the docstring (see `validate_npi_cmd` at `cli.py:244-250`).
|
||||||
|
Use `click.UsageError(...)` for usage mistakes; reserve
|
||||||
|
`sys.exit(2)` for "the file failed to parse" semantics.
|
||||||
|
4. **Smoke test with `click.testing.CliRunner`.** Every new
|
||||||
|
subcommand gets `backend/tests/test_cli_<name>.py` that
|
||||||
|
imports `from cyclone.cli import main` and invokes via
|
||||||
|
`CliRunner().invoke(main, [...], catch_exceptions=False)`. The
|
||||||
|
test must stub Keychain (`monkeypatch.setattr(secrets_mod,
|
||||||
|
"get_secret"/"set_secret", ...)`) and pin a temp SQLite DB via
|
||||||
|
`CYCLONE_DB_URL` + `db._reset_for_tests()` — see
|
||||||
|
`backend/tests/test_cli_backup.py:13-69` for the canonical
|
||||||
|
`_cli_env` fixture. CliRunner captures output and exit codes
|
||||||
|
in-process; do NOT shell out to `subprocess.run`.
|
||||||
|
5. **Destructive ops need `--yes` + `click.confirm(abort=True)`.**
|
||||||
|
`backup restore` (`cli.py:462-506`) and `backup prune`
|
||||||
|
(`cli.py:509-537`) both gate the destructive action behind a
|
||||||
|
`--yes` is_flag and an interactive `click.confirm(..., abort=True)`
|
||||||
|
prompt. CliRunner auto-aborts confirm prompts, so smoke tests
|
||||||
|
assert `exit_code != 0` when `--yes` is omitted
|
||||||
|
(`test_cli_backup.py:122-137`). A `--dry-run` flag is NOT yet
|
||||||
|
implemented anywhere; if needed, mirror the `--yes` pattern.
|
||||||
|
|
||||||
|
## Patterns
|
||||||
|
|
||||||
|
### A Click subcommand — `@main.command("<name>")` + options
|
||||||
|
|
||||||
|
From `cli.py:241-258` (smallest standalone subcommand):
|
||||||
|
|
||||||
|
```python
|
||||||
|
@main.command("validate-npi")
|
||||||
|
@click.argument("npi")
|
||||||
|
@click.option("--log-level", default="WARNING", show_default=True,
|
||||||
|
type=click.Choice(["DEBUG", "INFO", "WARNING", "ERROR"]))
|
||||||
|
def validate_npi_cmd(npi: str, log_level: str) -> None:
|
||||||
|
"""Validate a 10-digit NPI's Luhn checksum locally (SP20). Exit 0 valid, 1 invalid. PHI — don't log the value."""
|
||||||
|
setup_logging(level=log_level)
|
||||||
|
from cyclone.npi import is_valid_npi
|
||||||
|
if is_valid_npi(npi):
|
||||||
|
click.echo(f"OK: {len(npi)}-digit NPI passes Luhn checksum")
|
||||||
|
return
|
||||||
|
click.echo(f"INVALID: {npi!r} fails NPI Luhn checksum", err=True)
|
||||||
|
sys.exit(1)
|
||||||
|
```
|
||||||
|
|
||||||
|
### A smoke test — `CliRunner` + Keychain stub + temp SQLite
|
||||||
|
|
||||||
|
From `test_cli_backup.py:13-69`. Stable hex salt keeps multiple `CliRunner` invocations consistent within one test. The `Batch` seed at `cli_backup.py:27-35` is omitted — only the Keychain + DB plumbing is the convention:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@pytest.fixture
|
||||||
|
def _cli_env(tmp_path, monkeypatch):
|
||||||
|
from cyclone import db, secrets as secrets_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
# ... seed any DB rows the subcommand needs (see cli_backup.py:27-35) ...
|
||||||
|
|
||||||
|
store = {svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT: "cli-test-passphrase",
|
||||||
|
svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT:
|
||||||
|
"0123456789abcdef0123456789abcdef"}
|
||||||
|
monkeypatch.setattr(secrets_mod, "get_secret", lambda n: store.get(n))
|
||||||
|
monkeypatch.setattr(secrets_mod, "set_secret",
|
||||||
|
lambda n, v: store.__setitem__(n, v) or True)
|
||||||
|
monkeypatch.setenv("CYCLONE_BACKUP_DIR", str(tmp_path / "backups"))
|
||||||
|
yield tmp_path / "backups"
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
def test_backup_create_list_verify_status(_cli_env):
|
||||||
|
from cyclone.cli import main
|
||||||
|
runner = CliRunner()
|
||||||
|
r = runner.invoke(main, ["backup", "create"], catch_exceptions=False)
|
||||||
|
assert r.exit_code == 0, r.output
|
||||||
|
assert "created backup id=" in r.output
|
||||||
|
```
|
||||||
|
|
||||||
|
### A destructive subcommand — `--yes` + `click.confirm(abort=True)`
|
||||||
|
|
||||||
|
From `cli.py:462-506` (`backup restore`). Two-step: announce, prompt unless `--yes`, then execute. Restore uses an explicit init/confirm round-trip so the operator can back out between phases. Matching smoke test asserts the guard fires:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@backup.command("restore")
|
||||||
|
@click.argument("backup_id", type=int)
|
||||||
|
@click.option("--yes", is_flag=True, help="Skip the interactive confirm prompt")
|
||||||
|
@click.option("--actor", default="operator-cli", show_default=True)
|
||||||
|
def backup_restore(backup_id: int, yes: bool, actor: str) -> None:
|
||||||
|
"""Restore the live DB from a backup (two-step, requires --yes)."""
|
||||||
|
# ... db.init_db() + service config omitted ...
|
||||||
|
click.echo(f"Initiating restore from backup {backup_id}...")
|
||||||
|
init = svc.restore_initiate(backup_id)
|
||||||
|
# ... echo init summary (filename, fp, table_count, ttl) ...
|
||||||
|
if not yes:
|
||||||
|
click.confirm(
|
||||||
|
"Replace the live DB with this backup? "
|
||||||
|
"This will dispose the engine and rebuild it.",
|
||||||
|
abort=True,
|
||||||
|
)
|
||||||
|
click.echo("Confirming restore...")
|
||||||
|
result = svc.restore_confirm(backup_id, init.restore_token, actor=actor)
|
||||||
|
|
||||||
|
def test_backup_restore_requires_yes_flag(_cli_env):
|
||||||
|
runner = CliRunner()
|
||||||
|
runner.invoke(main, ["backup", "create"], catch_exceptions=False)
|
||||||
|
r = runner.invoke(main, ["backup", "restore", "1"], catch_exceptions=False)
|
||||||
|
# CliRunner auto-aborts confirm prompts → exit_code != 0.
|
||||||
|
assert r.exit_code != 0
|
||||||
|
```
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Don't `sys.exit(2)` for usage errors.** Reserve code 2 for
|
||||||
|
parse/operator errors (`CycloneParseError`, Keychain not
|
||||||
|
initialized, passphrase policy violation). For "you passed the
|
||||||
|
wrong flag" use `raise click.UsageError(...)` — Click formats
|
||||||
|
it as a clean help message and exits 2 on its own.
|
||||||
|
- **Don't print errors to stdout.** Use `click.echo(msg, err=True)`
|
||||||
|
for all error output. `print(..., file=sys.stderr)` and bare
|
||||||
|
`logging.error(...)` bypass Click's stdout/stderr split and leak
|
||||||
|
into test `result.output` — breaking `assert "FAIL" in r.output`
|
||||||
|
style assertions.
|
||||||
|
- **Don't add a subcommand without a smoke test.** Every new
|
||||||
|
`@main.command(...)` ships a sibling
|
||||||
|
`backend/tests/test_cli_<name>.py` exercising the happy path
|
||||||
|
AND at least one error path (missing input, invalid arg,
|
||||||
|
tampered ciphertext — see `test_cli_backup.py:102-119`).
|
||||||
|
- **Don't reuse the `parse` command name.** Existing subcommands
|
||||||
|
are type-specific (`parse-837`, `parse-835`); a generic `parse`
|
||||||
|
would shadow them or force an `--type` flag — neither is the
|
||||||
|
codebase pattern.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- **`cyclone-store`** — `backup` subcommands and the parse
|
||||||
|
subcommands both round-trip through `CycloneStore` and the DB
|
||||||
|
session; load when the increment changes write paths or the
|
||||||
|
`<entity>_written` event contract.
|
||||||
|
- **`cyclone-api-router`** — `cyclone serve` (via `__main__.py:19`)
|
||||||
|
launches the FastAPI app; the CLI parse subcommands share the
|
||||||
|
same `CycloneParseError` exception and Pydantic result models
|
||||||
|
as the matching HTTP endpoints.
|
||||||
|
- **`cyclone-edi`** — `parse-837` / `parse-835` are the CLI smoke
|
||||||
|
entry points for the parser/validator surface; load when adding
|
||||||
|
a parser or R-code rule.
|
||||||
|
- **`cyclone-tests`** — every CLI subcommand gets a pytest smoke
|
||||||
|
case under `backend/tests/test_cli_<name>.py`; the `_cli_env`
|
||||||
|
fixture pattern (fresh SQLite + Keychain stub) is documented
|
||||||
|
there.
|
||||||
|
- **`cyclone-spec`** — load when the SP-N spec introduces a new operator command or reserves a new exit-code category.
|
||||||
@@ -0,0 +1,193 @@
|
|||||||
|
---
|
||||||
|
name: cyclone-edi
|
||||||
|
description: "Cyclone EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1). Use when: adding or changing a parser, adding a validator rule (R010/R020/R100/R200-R210/R835_*/NPI Luhn/EIN/CAS), or mapping a new CAS adjustment reason code."
|
||||||
|
---
|
||||||
|
|
||||||
|
# cyclone-edi
|
||||||
|
|
||||||
|
Cyclone parses seven X12 EDI transaction types (837P, 835, 999, 270, 271, 277CA, TA1) into typed Pydantic models, then runs per-claim / per-batch validator rules that surface as R-coded `ValidationIssue` records. This skill codifies the conventions so new parsers and rules stay consistent with the seven that already exist.
|
||||||
|
|
||||||
|
As of this writing: **7 parser modules** under `backend/src/cyclone/parsers/parse_<edi>.py`, **~25 per-claim rules** numbered `R010`–`R100` and `R200`–`R210` in `validator.py`, plus a parallel set of **835-specific rules** prefixed `R835_*` in `validator_835.py`. The next increment is **SP22**.
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- **Adding or changing a parser.** You are about to touch `backend/src/cyclone/parsers/parse_<edi>.py` or its paired `models_<edi>.py` and need the orchestrator signature, the segment walker convention, and the re-export in `parsers/__init__.py`.
|
||||||
|
- **Adding a validator rule.** You are writing a new `_rule_R<n>_<name>` (or `_r<n>_<name>` per the existing snake-case style) and need the rule signature, the R-code numbering scheme, and the `ValidationIssue` shape.
|
||||||
|
- **Wiring a new CAS / CARC code.** The 835 carries Claim Adjustment Reason Codes in `CAS` segments; the lookup lives in `backend/src/cyclone/parsers/cas_codes.py` and the UI reads through `claim_status_label()`.
|
||||||
|
- **Debugging a parse failure on a prodfiles sample.** You dropped a real EDI file into `docs/prodfiles/<source>/` and the parser is choking — load this skill to confirm the tokenizer path, the orchestrator entry point, and which fixture in `backend/tests/fixtures/` matches the transaction type.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
1. **Parser signature.** Every parser module exports exactly one public entry function. Two flavors coexist in the codebase:
|
||||||
|
- `parse(text: str, *, input_file: str = "") -> <TypedResult>` — used by `parse_270.py:337`, `parse_271.py:356`.
|
||||||
|
- `parse(text: str, payer_config: <PayerConfig>, input_file: str = "") -> <TypedResult>` — used by `parse_837.py:319` and `parse_835.py:459` because both need payer-specific config to validate segments against.
|
||||||
|
- `parse_<edi>_text(text: str, *, input_file: str = "") -> <TypedResult>` — the legacy name-suffixed form, still in use at `parse_ta1.py:143`, `parse_999.py:220`, `parse_277ca.py:280`. The `<TypedResult>` is always a Pydantic model from `models_<edi>.py` (or co-located `models.py` for 837P).
|
||||||
|
2. **Segment walk.** Parsers consume `backend/src/cyclone/parsers/segments.py` — there are exactly three public pieces: `Delimiters` (frozen dataclass holding the four ISA-derived separators), `_detect_delimiters(isa_segment)` (private), and `tokenize(text) -> list[list[str]]` (returns ISA prepended as the first segment). Parsers then index into the `list[list[str]]` directly — there is **no** `Segment` / `Loop` / `next_segment` helper class. Whole-document problems (missing ISA, wrong transaction set) raise `CycloneParseError`; per-segment problems on acks (999/277CA) are surfaced on the result, not raised.
|
||||||
|
3. **Validator rules.** Numbered rules live in `backend/src/cyclone/parsers/validator.py` (for 837P — R010–R100 general + R200–R210 SP9 CO MAP / HCPF naming) and `backend/src/cyclone/parsers/validator_835.py` (for 835 — names prefixed `R835_*` because the same numeric space would collide with 837P). Each rule is a function `_r<n>_<name>(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]` registered in the module-level `_RULES` list and run by `validate(claim, config)`. Issues carry the rule name as a stable string (`rule="R021_npi_checksum"`) — the R-code **is** how the UI surfaces the error, so never invent an unnumbered rule.
|
||||||
|
4. **NPI / EIN / CAS format logic.** Identity-format checks live in their own modules — never duplicate them in a parser or validator:
|
||||||
|
- `backend/src/cyclone/npi.py` — `is_valid_npi(npi)` runs the Luhn checksum with the `80840` NPPES prefix; `is_valid_tax_id(ein)` enforces `XX-XXXXXXX` (or 9 raw digits).
|
||||||
|
- `backend/src/cyclone/parsers/cas_codes.py` — `reason_label(group, reason)` and `all_known_codes()` for the CARC lookup; snapshot date is exported as `LAST_UPDATED`.
|
||||||
|
- `backend/src/cyclone/parsers/models_271.py` — `SERVICE_TYPE_CODES` + `service_type_description()` for 271 EB benefit codes.
|
||||||
|
5. **Prodfiles reuse.** When adding a parser for a new transaction type, ship at least one fixture in `backend/tests/fixtures/<edi>/<sample>.txt` (the existing 13 fixtures are **flat** at the top level of `fixtures/` — no per-test subdirectories). Copy from `docs/prodfiles/<source>/<file>.txt`; never reach into `docs/prodfiles/` from a test. The matching test should declare the path as a module-level `Path` constant.
|
||||||
|
|
||||||
|
## Patterns
|
||||||
|
|
||||||
|
### Minimal `parse_<edi>.py` — using `segments.py`, exporting `parse_ta1_text`
|
||||||
|
|
||||||
|
Taken from `backend/src/cyclone/parsers/parse_ta1.py:1-29` (the smallest parser — TA1 is just ISA + TA1 + IEA). The same skeleton scales to every other EDI type by adding `_consume_<segment>` helpers.
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""Parse an X12 TA1 (Interchange Acknowledgment) file.
|
||||||
|
|
||||||
|
Whole-document problems (missing ISA, no TA1) raise CycloneParseError.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from datetime import date
|
||||||
|
|
||||||
|
from cyclone.parsers.exceptions import CycloneParseError
|
||||||
|
from cyclone.parsers.models import BatchSummary, Envelope
|
||||||
|
from cyclone.parsers.models_ta1 import ParseResultTa1, Ta1Ack
|
||||||
|
from cyclone.parsers.segments import tokenize
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_yyyymmdd(s: str) -> date | None:
|
||||||
|
"""Parse an 8-digit CCYYMMDD string. Returns None on bad input."""
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
|
def _build_envelope(segments: list[list[str]], input_file: str) -> Envelope:
|
||||||
|
"""Build the envelope from ISA. TA1 has no GS/ST — just ISA → TA1 → IEA."""
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
|
def _consume_ta1(segments: list[list[str]], idx: int) -> tuple[Ta1Ack, int]:
|
||||||
|
"""Read a TA1 segment and return a Ta1Ack. Returns (model, next_idx)."""
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
|
def parse_ta1_text(text: str, *, input_file: str = "") -> ParseResultTa1:
|
||||||
|
"""Parse a complete TA1 document and return a ParseResultTa1."""
|
||||||
|
segments = tokenize(text)
|
||||||
|
envelope = _build_envelope(segments, input_file=input_file)
|
||||||
|
ta1_idx = next(
|
||||||
|
(i for i, seg in enumerate(segments) if seg[0] == "TA1"), None,
|
||||||
|
)
|
||||||
|
if ta1_idx is None:
|
||||||
|
raise CycloneParseError("No TA1 segment found")
|
||||||
|
ta1, _ = _consume_ta1(segments, ta1_idx)
|
||||||
|
...
|
||||||
|
return ParseResultTa1(envelope=envelope, ta1=ta1, summary=summary, ...)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["parse_ta1_text"]
|
||||||
|
```
|
||||||
|
|
||||||
|
The orchestrator pattern is the same in every parser: `tokenize` → `_build_envelope` → segment consumers in order → wrap into a `ParseResult<EDI>` model. The Pydantic result is what the API / store layer consumes.
|
||||||
|
|
||||||
|
### A validator rule — `_r<n>_<name>` registered in `_RULES`
|
||||||
|
|
||||||
|
Taken from `backend/src/cyclone/parsers/validator.py:23-66` (the canonical R010–R100 block).
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from cyclone.parsers.models import ClaimOutput, ValidationIssue
|
||||||
|
from cyclone.parsers.payer import PayerConfig
|
||||||
|
|
||||||
|
NPI_RE = re.compile(r"^\d{10}$")
|
||||||
|
Rule = Callable[[ClaimOutput, PayerConfig], Iterable[ValidationIssue]]
|
||||||
|
|
||||||
|
|
||||||
|
def _r020_npi_format(claim: ClaimOutput, _: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
if claim.billing_provider.npi and not NPI_RE.match(claim.billing_provider.npi):
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R020_npi_format",
|
||||||
|
severity="error",
|
||||||
|
message=f"Billing provider NPI must be 10 digits, got {claim.billing_provider.npi!r}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _r021_npi_checksum(claim: ClaimOutput, _: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
"""SP20: validate the billing-provider NPI's Luhn check digit."""
|
||||||
|
npi = claim.billing_provider.npi
|
||||||
|
if not npi or not NPI_RE.match(npi):
|
||||||
|
return # R020 already flagged the format — skip silently.
|
||||||
|
try:
|
||||||
|
from cyclone.npi import is_valid_npi
|
||||||
|
except ImportError:
|
||||||
|
return
|
||||||
|
if not is_valid_npi(npi):
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R021_npi_checksum",
|
||||||
|
severity="warning",
|
||||||
|
message=f"Billing provider NPI {npi!r} fails Luhn checksum (likely typo)",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_RULES: list[Rule] = [
|
||||||
|
_r010_clm01_present,
|
||||||
|
_r011_total_charge_positive,
|
||||||
|
_r020_npi_format,
|
||||||
|
_r021_npi_checksum,
|
||||||
|
# ... R030, R031, R032-R035, R050, R060, R070, R100, R200-R210
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
For 835 rules, prefix the rule string with `R835_` (e.g. `R835_BPR01_handling_code_allowed`) and target the `ParseResult835` model instead of `ClaimOutput` — see `backend/src/cyclone/parsers/validator_835.py:38-79`.
|
||||||
|
|
||||||
|
### A test that uses a prodfiles fixture
|
||||||
|
|
||||||
|
Taken from `backend/tests/test_api_999.py:53-72`. The autouse `conftest.py` already provides a per-test SQLite DB; most tests just add a `client` fixture and reference the fixture path as a module-level constant.
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""Tests for the FastAPI surface in cyclone.api for the 999 endpoint."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone.api import app
|
||||||
|
|
||||||
|
# Fixture reference — flat, module-level Path constant. NEVER reach into
|
||||||
|
# docs/prodfiles/ from a test; the fixtures/ dir is the test-consumed surface.
|
||||||
|
ACCEPTED = Path(__file__).parent / "fixtures" / "minimal_999.txt"
|
||||||
|
REJECTED = Path(__file__).parent / "fixtures" / "minimal_999_rejected.txt"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client() -> TestClient:
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_999_endpoint_happy_path(client: TestClient):
|
||||||
|
text = ACCEPTED.read_text()
|
||||||
|
resp = client.post(
|
||||||
|
"/api/parse-999",
|
||||||
|
files={"file": ("minimal_999.txt", text, "text/plain")},
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
body = resp.json()
|
||||||
|
assert body["ack"]["ack_code"] == "A"
|
||||||
|
```
|
||||||
|
|
||||||
|
For pure-unit parser tests (no API), the same path is reused — see `backend/tests/test_parse_837.py:8` (`FIXTURE = Path(__file__).parent / "fixtures" / "minimal_837p.txt"`).
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Don't re-parse raw X12 strings inside validators.** Always parse first into the typed `ParseResult<EDI>` / `ClaimOutput`, then validate against that. Validators index into the model fields (or `claim.raw_segments` for spot-checks of specific segment presence) — they never call `tokenize` again. R034's `REF*G1` presence check (`validator.py:104-107`) is the only place that legitimately touches `raw_segments`, and it does so to confirm a single segment exists.
|
||||||
|
- **Don't bake payer-specific logic into the generic parser.** Payer variations live in `backend/src/cyclone/parsers/payer.py` (`PayerConfig`, `PayerConfig835`) and `backend/src/cyclone/payers.py` (the YAML loader from `config/payers.yaml`). Parsers accept the config as an argument; rules read it from the `cfg` parameter. A new payer never requires a new parser file — extend the config and add / adjust an R-code rule.
|
||||||
|
- **Don't add a validator rule without an R-code.** The `rule="R<n>_<name>"` string is the stable identifier the UI greys out, the API returns in `errors[].rule`, and tests assert against. Inventing a rule without an R-code (or reusing an R-code with new semantics) breaks the operator workflow. New SP-N increments reserve their R-code range up front (SP9 reserved R200–R210, SP20 added R021) and document it in the spec.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- **`cyclone-store`** — load when the increment changes how a parsed `ClaimOutput` / `ParseResult<EDI>` is persisted (`store.py` write path, `<entity>_written` events).
|
||||||
|
- **`cyclone-api-router`** — load when the increment adds or changes an HTTP endpoint that surfaces a parsed result (e.g. `/api/parse-999`, `/api/parse-837`, `/api/parse-835`).
|
||||||
|
- **`cyclone-tests`** — every parser addition ships a fixture in `backend/tests/fixtures/` and a pytest case; load this skill for the fixture-drop-in and autouse-conftest rules.
|
||||||
|
- **`cyclone-cli`** — load when the increment adds a CLI subcommand. The `cyclone parse-837 <file>` and `cyclone parse-835 <file>` smoke commands at `backend/src/cyclone/cli.py:77,151` are the parser-level smoke tests; the `validate-npi` and `validate-tax-id` commands exercise the format helpers.
|
||||||
|
- **`cyclone-spec`** — load when the SP-N spec for the increment introduces a new R-code range or a new transaction type; the spec's `## Decisions` section is where the R-code reservation gets locked in.
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
# Cyclone EDI parsers — flat catalog
|
||||||
|
|
||||||
|
Every parser module under `backend/src/cyclone/parsers/` (one row per
|
||||||
|
file), its transaction type, its public entry signature, its result
|
||||||
|
model, its primary fixture, and any payer-specific variant. The
|
||||||
|
companion Pydantic model is in a co-located `models_<edi>.py`; the
|
||||||
|
segment walker uses `tokenize()` from `segments.py` and never parses
|
||||||
|
raw text inline.
|
||||||
|
|
||||||
|
| Module | EDI type | Public entry signature | Result model | Primary fixture(s) | Payer variant |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| `parse_837.py` | 837P (Professional Claim) | `parse(text, payer_config: PayerConfig, input_file="") -> ParseResult` | `cyclone.parsers.models.ParseResult` | `minimal_837p.txt`, `co_medicaid_837p.txt` | `PayerConfig` (CO Medicaid default) |
|
||||||
|
| `parse_835.py` | 835 (ERA / Remittance) | `parse(text, payer_config: PayerConfig835, input_file="") -> ParseResult835` | `cyclone.parsers.models_835.ParseResult835` | `minimal_835.txt`, `co_medicaid_835.txt`, `unbalanced_835.txt` | `PayerConfig835` |
|
||||||
|
| `parse_999.py` | 999 (Implementation ACK) | `parse_999_text(text, *, input_file="") -> ParseResult999` | `cyclone.parsers.models_999.ParseResult999` | `minimal_999.txt`, `minimal_999_rejected.txt` | none — single-shape ack |
|
||||||
|
| `parse_277ca.py` | 277CA (Claim ACK) | `parse_277ca_text(text, *, input_file="") -> ParseResult277CA` | `cyclone.parsers.models_277ca.ParseResult277CA` | `minimal_277ca.txt`, `minimal_277ca_rejected_only.txt`, `minimal_277ca_st277.txt` | `PayerConfig277CA` (config-driven) |
|
||||||
|
| `parse_270.py` | 270 (Eligibility Inquiry) | `parse(text, *, input_file="") -> ParseResult270` | `cyclone.parsers.models_270.ParseResult270` | `minimal_270.txt` | reuses `PayerConfig` shape |
|
||||||
|
| `parse_271.py` | 271 (Eligibility Response) | `parse(text, *, input_file="") -> ParseResult271` | `cyclone.parsers.models_271.ParseResult271` | `minimal_271.txt` | reuses `PayerConfig` shape |
|
||||||
|
| `parse_ta1.py` | TA1 (Interchange ACK) | `parse_ta1_text(text, *, input_file="") -> ParseResultTa1` | `cyclone.parsers.models_ta1.ParseResultTa1` | `minimal_ta1.txt` | none — single-shape ack |
|
||||||
|
|
||||||
|
Companion modules (not parsers, but shipped alongside):
|
||||||
|
|
||||||
|
| Module | Role |
|
||||||
|
|---|---|
|
||||||
|
| `segments.py` | `Delimiters`, `_detect_delimiters`, `tokenize(text) -> list[list[str]]` |
|
||||||
|
| `models.py` | Pydantic models for 837P (`ParseResult`, `ClaimOutput`, `Envelope`, `BatchSummary`, `ValidationIssue`, `ValidationReport`, …) |
|
||||||
|
| `models_835.py` / `models_270.py` / `models_271.py` / `models_277ca.py` / `models_999.py` / `models_ta1.py` | Pydantic models for each transaction type |
|
||||||
|
| `payer.py` | `PayerConfig` + `PayerConfig835` factories |
|
||||||
|
| `exceptions.py` | `CycloneParseError`, `CycloneValidationError` |
|
||||||
|
| `cas_codes.py` | CARC lookup: `reason_label(group, reason)`, `all_known_codes()`, `LAST_UPDATED` |
|
||||||
|
| `validator.py` | 837P rules: R010–R100, R200–R210; `validate(claim, config) -> ValidationReport` |
|
||||||
|
| `validator_835.py` | 835 rules: `R835_*` (e.g. `R835_BPR01_handling_code_allowed`); `validate(result, cfg) -> ValidationReport` |
|
||||||
|
| `serialize_270.py` / `serialize_837.py` / `serialize_999.py` | Outbound (Cyclone → payer) serializers — mirror of `parse_*` |
|
||||||
|
| `writer.py` / `writer_835.py` | Output writers (one JSON per claim) |
|
||||||
|
| `batch_ack_builder.py` | `build_ack_for_batch` — produces a 999 for a parsed 837 batch |
|
||||||
|
| `__init__.py` | Lazy PEP 562 re-exports (`parse`, `parse_835`, `parse_999`, `parse_270`, `parse_271`, `parse_277ca`, plus all models) |
|
||||||
|
|
||||||
|
Fixture rule: every parser ships at least one flat fixture in
|
||||||
|
`backend/tests/fixtures/<edi>-sample.txt`. Prodfiles sources live in
|
||||||
|
`docs/prodfiles/{837p-from-axiscare,835fromco,FromHPE,claims}/` — copy
|
||||||
|
from there, never reach in from a test.
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
---
|
||||||
|
name: cyclone-frontend-page
|
||||||
|
description: "Cyclone React page conventions (TanStack Query, use<X> hook, drawer, URL state, .test.tsx sibling, Layout / PageHeader / Sidebar). Use when: adding a new page, refactoring an existing one, or wiring a drawer into a page."
|
||||||
|
---
|
||||||
|
|
||||||
|
# cyclone-frontend-page
|
||||||
|
|
||||||
|
Cyclone pages live at `src/pages/<Name>.tsx`: a `use<X>` hook in `src/hooks/use<X>.ts` does the fetching (and optionally the live-tail subscription), `<Layout>` + `<PageHeader>` + `<Sidebar>` (`src/components/`) provide the app shell, and any right-side detail (claim, remittance) lives in `src/components/<DrawerName>/` whose open/close state is mirrored to the URL via `useDrawerUrlState`. This skill codifies the conventions so additions stay consistent with the eleven pages already shipped.
|
||||||
|
|
||||||
|
As of this writing: **11 pages** under `src/pages/` (9 of 11 with a `*.test.tsx` sibling — Dashboard, Upload, and BatchDiff are not yet covered; be the first when you refactor them), **~30 hooks** under `src/hooks/`, and **2 drawer modules** at `src/components/{ClaimDrawer,RemitDrawer}/`. The next increment is **SP22**.
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- **Adding a new page.** Mounting a new screen in `src/pages/` — you need the Layout + PageHeader + table shape, the `use<X>` hook split, and the route registration point in `src/App.tsx`.
|
||||||
|
- **Refactoring an existing page.** Splitting a 600-line page, swapping a manual `fetch` for a hook, or moving in-component state into the URL — load this skill to confirm the destination shape.
|
||||||
|
- **Wiring a drawer.** Adding a new right-side detail drawer (e.g. `BatchDrawer`, `ActivityDrawer`) — you need `useDrawerUrlState` for URL-driven open/close, the `src/components/<DrawerName>/` folder layout, and the deep-link contract so `?claim=…` / `?remit=…` round-trips.
|
||||||
|
- **Sharing state via URL.** Persisting filter / page / drawer state across reloads — confirm the `useDrawerUrlState` / `useRemitDrawerUrlState` hook pair is the right tool before reaching for `useState` + history.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
1. **Page shape.** Every page in `src/pages/<Name>.tsx` exports a `function <Name>()` (most pages use a named export — see `src/pages/Claims.tsx:51`, `Remittances.tsx:62`, `Dashboard.tsx:68`; `src/pages/Inbox.tsx:40` is the lone default export). The app shell is provided by the `<Layout>` route wrapper in `src/App.tsx:30`. Each page sets a `<PageHeader>` (`src/components/PageHeader.tsx:18`) and renders a table, list, or KPI grid. Sidebar nav is mounted once by `<Layout>` at `src/components/Sidebar.tsx`.
|
||||||
|
2. **Data hook.** Each page pairs with a `use<X>` hook in `src/hooks/use<X>.ts` (e.g. `Claims` ↔ `useClaims`, `Remittances` ↔ `useRemittances`, `Acks` ↔ `useAcks`). The hook returns `{ data, isLoading, isError, error, refetch }` from TanStack Query's `useQuery` (see `src/hooks/useClaims.ts:24-31`, `useRemittances.ts:21-25`). Pages never call `fetch` or `@/lib/api` directly — the hook is the boundary so the page is testable with `vi.mock("@/lib/api", ...)`.
|
||||||
|
3. **Live tail.** Pages with live data compose three hooks in order: the `use<X>` initial fetch, `useTailStream(resource)` (`src/hooks/useTailStream.ts:80` — opens the NDJSON stream, drives the backoff/stall state machine), and `useMergedTail(resource, baseItems, filterFn?)` (`src/hooks/useMergedTail.ts:25` — merges snapshot + tail, dedup'd by id). The full wiring lives at `src/pages/Claims.tsx:85-89` and `Remittances.tsx:81-83`. The streaming subscription belongs on the page, not in the data hook — hoisting it couples the lifecycle to whoever mounts `use<X>`.
|
||||||
|
4. **Drawer.** Right-side detail drawers live in `src/components/<DrawerName>/` (currently `ClaimDrawer/`, `RemitDrawer/`) with a barrel `index.ts` (`src/components/ClaimDrawer/index.ts:1-14`). The drawer is wired to URL state via `useDrawerUrlState()` for `?claim=…` (`src/hooks/useDrawerUrlState.ts`) or `useRemitDrawerUrlState()` for `?remit=…` (`src/hooks/useRemitDrawerUrlState.ts`). Open state is driven by the URL, not a `useState` flag, so deep-links round-trip.
|
||||||
|
5. **Tests.** Every page gets a `src/pages/<Name>.test.tsx` sibling (e.g. `Claims.test.tsx`, `Remittances.test.tsx`, `Batches.test.tsx`). Every hook gets a `src/hooks/use<X>.test.ts` sibling (e.g. `useClaims.test.ts`, `useRemittances.test.ts`). The shared setup — `// @vitest-environment happy-dom`, `IS_REACT_ACT_ENVIRONMENT = true`, `QueryClient` provider, `vi.mock("@/lib/api", ...)` — is documented in `cyclone-tests`; mirror `src/pages/Claims.test.tsx:1-30` for the canonical page-test shape.
|
||||||
|
6. **UI primitives.** Use Radix-backed components from `src/components/ui/` (`button.tsx`, `dialog.tsx`, `table.tsx`, `select.tsx`, `pagination.tsx`, `empty-state.tsx`, `error-state.tsx`, `filter-chips.tsx`, `skeleton.tsx`, `input.tsx`, `label.tsx`, `card.tsx`, `badge.tsx`, `skip-link.tsx`, `claim-state-badge.tsx`). Don't pull in a new UI library without discussion — every primitive here is already consumed by at least one shipped page.
|
||||||
|
7. **Routing.** Pages register their route in `src/App.tsx` as a `<Route path="<name>" element={<<Name>> />} />` inside the `<Layout>` element wrapper (`src/App.tsx:30-44`). Currently every page is a static import; switch to `React.lazy(() => import(...))` only if a page grows heavy (large parse/EDI libs, chart code) and the import cost shows up in the bundle report.
|
||||||
|
|
||||||
|
## Patterns
|
||||||
|
|
||||||
|
### `Claims.tsx`-style page (Layout + PageHeader + table + drawer)
|
||||||
|
|
||||||
|
Canonical page shape — composes the data hook, the tail triplet, and
|
||||||
|
`useDrawerUrlState` for the `?claim=…` deep-link. See
|
||||||
|
`src/pages/Claims.tsx:51-200` for the full file.
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
import { useMemo, useState } from "react";
|
||||||
|
import { Table, TableBody, TableRow, /* … */ } from "@/components/ui/table";
|
||||||
|
import { PageHeader } from "@/components/PageHeader";
|
||||||
|
import { ClaimDrawer } from "@/components/ClaimDrawer";
|
||||||
|
import { useClaims } from "@/hooks/useClaims";
|
||||||
|
import { useDrawerUrlState } from "@/hooks/useDrawerUrlState";
|
||||||
|
import { useTailStream } from "@/hooks/useTailStream";
|
||||||
|
import { useMergedTail } from "@/hooks/useMergedTail";
|
||||||
|
import { TailStatusPill } from "@/components/TailStatusPill";
|
||||||
|
|
||||||
|
export function Claims() {
|
||||||
|
const [status, setStatus] = useState<ClaimStatus | null>(null);
|
||||||
|
const params = useMemo(() => ({ status, limit: 25, offset: 0 }), [status]);
|
||||||
|
|
||||||
|
const { data } = useClaims(params);
|
||||||
|
const { status: tailStatus, lastEventAt, forceReconnect } = useTailStream("claims");
|
||||||
|
const items = useMergedTail("claims", data?.items ?? [], (c) => !status || c.status === status);
|
||||||
|
const { claimId, openClaim, closeClaim } = useDrawerUrlState();
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<PageHeader
|
||||||
|
eyebrow="Inbox"
|
||||||
|
title="Claims"
|
||||||
|
status={<TailStatusPill status={tailStatus} lastEventAt={lastEventAt} onReconnect={forceReconnect} />}
|
||||||
|
/>
|
||||||
|
<Table>
|
||||||
|
<TableBody>
|
||||||
|
{items.map((c) => (
|
||||||
|
<TableRow key={c.id} onClick={() => openClaim(c.id)}>{/* …cells… */}</TableRow>
|
||||||
|
))}
|
||||||
|
</TableBody>
|
||||||
|
</Table>
|
||||||
|
<ClaimDrawer claimId={claimId} claims={items} onClose={closeClaim} onNavigate={openClaim} />
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `use<X>` data hook (TanStack Query)
|
||||||
|
|
||||||
|
Pattern from `src/hooks/useClaims.ts:24-67` and `useRemittances.ts:21-65`.
|
||||||
|
Returns a stable shape so the page treats all data hooks uniformly.
|
||||||
|
The tail subscription lives on the page (see Convention 3), not here.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { useQuery } from "@tanstack/react-query";
|
||||||
|
import { api, type ListClaimsParams, type PaginatedResponse } from "@/lib/api";
|
||||||
|
import type { Claim } from "@/types";
|
||||||
|
|
||||||
|
export function useClaims(params: ListClaimsParams) {
|
||||||
|
return useQuery<PaginatedResponse<Claim>>({
|
||||||
|
queryKey: ["claims", params],
|
||||||
|
queryFn: () => api.listClaims<Claim>(params),
|
||||||
|
enabled: api.isConfigured,
|
||||||
|
// …in-memory fallback when !api.isConfigured…
|
||||||
|
});
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Drawer component with `useDrawerUrlState`
|
||||||
|
|
||||||
|
Pattern from `src/components/ClaimDrawer/ClaimDrawer.tsx:1-50` +
|
||||||
|
`src/hooks/useDrawerUrlState.ts`. The drawer takes `claimId` as a
|
||||||
|
prop (driven by the URL), renders nothing when `null`, and uses
|
||||||
|
`useDrawerKeyboard` for j/k navigation + Escape to close. The page
|
||||||
|
that mounts it controls the URL — `openClaim("abc")` sets `?claim=abc`,
|
||||||
|
removing the param closes the drawer, and reload preserves the state.
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
import { Dialog, DialogContent } from "@/components/ui/dialog";
|
||||||
|
import { useClaimDetail } from "@/hooks/useClaimDetail";
|
||||||
|
import { useDrawerKeyboard } from "@/hooks/useDrawerKeyboard";
|
||||||
|
|
||||||
|
export function ClaimDrawer({ claimId, claims, onClose, onNavigate, onToggleHelp }) {
|
||||||
|
const open = claimId !== null;
|
||||||
|
const { data, isLoading, isError, error } = useClaimDetail(claimId);
|
||||||
|
useDrawerKeyboard({ open, claims, onClose, onNavigate, onToggleHelp });
|
||||||
|
if (!open) return null;
|
||||||
|
return (
|
||||||
|
<Dialog open onOpenChange={(o) => !o && onClose()}>
|
||||||
|
<DialogContent>{/* header + body panels from useClaimDetail… */}</DialogContent>
|
||||||
|
</Dialog>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Don't `fetch` from inside a page component.** All API access goes through the `use<X>` hook in `src/hooks/use<X>.ts`. Pages that reach for `fetch(...)` directly can't be tested with `vi.mock("@/lib/api", ...)` and split the data lifecycle across files. The hook returns `{ data, isLoading, isError, error, refetch }` so the page is a pure renderer.
|
||||||
|
- **Don't open a drawer via local component state.** A `const [open, setOpen] = useState(false)` for a drawer breaks deep-links and reload-restore. Use `useDrawerUrlState()` (claim) or `useRemitDrawerUrlState()` (remit) so the URL is the single source of truth.
|
||||||
|
- **Don't put domain logic in JSX.** Conditional renderings, table sorting, and KPI math all belong in the `use<X>` hook or a pure helper under `src/lib/` (e.g. `src/lib/format.ts` for currency / date formatting). JSX is for layout; mixing in `items.filter(...).sort(...)` inline is hard to test and hides behavior from the hook.
|
||||||
|
- **Don't call `useTailStream` from inside a `use<X>` hook.** The streaming subscription belongs on the page (see `src/pages/Claims.tsx:85-89`). Hoisting it into `useClaims` couples the open/close lifecycle of the page to whoever mounts the hook, and breaks the one-resource-one-page ownership that the backoff/stall state machine assumes.
|
||||||
|
- **Don't import a new UI library to render a button, modal, or table.** Radix-backed primitives in `src/components/ui/` already cover every widget the shipped pages use. Reach for a new library only after the primitive gap is real and the proposal is in a spec / PR.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- **`cyclone-tail`** — load for any live-data page (Claims, Remittances, ActivityLog). Documents the `useTailStream` + `useMergedTail` triplet, the `<TailStatusPill>` wiring, and the 30s stall threshold (`STALL_TIMEOUT_MS = 30_000` at `useTailStream.ts:53`).
|
||||||
|
- **`cyclone-api-router`** — load when a page is calling a new HTTP endpoint. Documents `api_routers/<topic>.py` vs. inline `api.py` registration and the response / error-envelope shapes.
|
||||||
|
- **`cyclone-tests`** — load when adding the `*.test.tsx` sibling. Documents the `// @vitest-environment happy-dom` setup, `IS_REACT_ACT_ENVIRONMENT = true`, the `@testing-library/react` vs. `createRoot`+`Probe` rendering styles, and the `vi.mock("@/lib/api")` convention.
|
||||||
|
- **`cyclone-edi`** — load when a page renders parsed 837P / 835 / 999 / 270 / 271 / 277CA / TA1 content (ServiceLinesTable, CAS panels, ValidationPanel). Documents the parser modules, the R-coded validator rules, and the CAS / CARC / NPI / EIN helpers.
|
||||||
@@ -0,0 +1,157 @@
|
|||||||
|
---
|
||||||
|
name: cyclone-spec
|
||||||
|
description: "Cyclone SP-N superpowers increment flow — spec → plan → implement → merge. Use when: starting a new numbered feature increment, naming a branch, opening a SP-N PR, or doing the merge dance into main."
|
||||||
|
---
|
||||||
|
|
||||||
|
# cyclone-spec
|
||||||
|
|
||||||
|
The Cyclone repo ships every new feature as a numbered **SP-N increment**:
|
||||||
|
a spec, a plan, an implementation branch, and a single atomic merge commit
|
||||||
|
into `main`. This skill encodes the conventions so every increment follows
|
||||||
|
the same shape and the commit history stays auditable.
|
||||||
|
|
||||||
|
As of this writing: **16 specs** in `docs/superpowers/specs/`, **12 plans**
|
||||||
|
in `docs/superpowers/plans/`, and SP numbers used through **SP21** (the
|
||||||
|
universal-drilldown design in progress). The next increment is **SP22**.
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- **Starting a new feature increment.** You are about to add a numbered
|
||||||
|
feature, fix that crosses subsystem boundaries, or anything bigger than
|
||||||
|
a one-line change. Before you write code, reserve the next SP number and
|
||||||
|
write the spec.
|
||||||
|
- **Naming the spec / plan files or the branch.** You have a topic, a
|
||||||
|
date, and a number — and you need the exact path / branch shape so
|
||||||
|
existing scripts and reviewers can find the artifacts.
|
||||||
|
- **Opening the SP-N PR.** You are about to push the branch and need the
|
||||||
|
PR title format and the commit-prefix conventions so the merge commit
|
||||||
|
reads cleanly.
|
||||||
|
- **Doing the merge dance.** Review is approved and you're about to land
|
||||||
|
the branch into `main`. Use this skill to confirm the merge shape — no
|
||||||
|
squash, no rebase, one atomic merge commit.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
1. **Numbering.** Reserve the next SP-N number — the next integer after
|
||||||
|
the highest `SP<n>` already used in `git log`. Never reuse a number,
|
||||||
|
even after deletion. Numbering is monotonic and lives in the merge
|
||||||
|
history.
|
||||||
|
2. **Branch.** `sp<N>-<short-kebab-topic>` — e.g. `sp22-line-reconciliation`,
|
||||||
|
`sp9-multi-payer-npi`. Kebab-case, lowercase, no spaces, no slashes.
|
||||||
|
The branch name is the canonical handle for the increment.
|
||||||
|
3. **Spec path.** `docs/superpowers/specs/YYYY-MM-DD-cyclone-<topic>-design.md`
|
||||||
|
with header `Status: Draft, pending user review`. One spec per
|
||||||
|
increment. Real examples: `2026-06-19-cyclone-db-reconciliation-design.md`
|
||||||
|
(SP3), `2026-06-20-cyclone-multi-payer-npi-sftp-design.md` (SP9).
|
||||||
|
4. **Plan path.** `docs/superpowers/plans/YYYY-MM-DD-cyclone-<topic>.md`.
|
||||||
|
Header per the upstream `superpowers:writing-plans` skill: a
|
||||||
|
`For agentic workers:` line that names
|
||||||
|
`superpowers:subagent-driven-development` or
|
||||||
|
`superpowers:executing-plans`, plus a `Goal / Architecture / Tech
|
||||||
|
Stack / Spec` metadata block, then numbered tasks with
|
||||||
|
`- [ ] Step N:` checkboxes.
|
||||||
|
5. **Commit prefix.** All commits on the branch follow these prefixes —
|
||||||
|
they make the SP-N merge commit readable and let `git log --grep`
|
||||||
|
filter cleanly:
|
||||||
|
- `feat(sp<N>): …` — implementation commits (e.g. `feat(sp20): NPI Luhn checksum + Tax ID format validation`).
|
||||||
|
- `docs(spec): …` — landing the spec (e.g. `docs(spec): design for CycloneStore split (Step 4)`).
|
||||||
|
- `docs(plan): …` — landing the plan.
|
||||||
|
- `merge: SP<N> <topic> into main` — the merge commit itself (e.g. `merge: SP14 5-lane Inbox UI + acknowledge action into main`).
|
||||||
|
6. **PR title.** `SP<N> <Topic>` — e.g. `SP22 Line reconciliation`.
|
||||||
|
Matches the merge-commit subject so GitHub's "merged PR" view and the
|
||||||
|
`git log` entry are identical strings.
|
||||||
|
7. **Merge shape.** A single atomic merge commit into `main` after
|
||||||
|
review. **No squash** — squash collapses the per-commit history and
|
||||||
|
breaks the SP-N audit trail. **No rebase** — rebase rewrites the SHAs
|
||||||
|
the PR review was performed against. The SP-N merge commit *is* the
|
||||||
|
record of the increment landing.
|
||||||
|
|
||||||
|
## Patterns
|
||||||
|
|
||||||
|
### Spec header (canonical SP-N shape, post-SP9)
|
||||||
|
|
||||||
|
This is the canonical header for new specs. Older specs (pre-SP9) deviate
|
||||||
|
slightly — different title style, no Branch or Aesthetic direction line —
|
||||||
|
and have not been retroactively normalized. **Use this template for any
|
||||||
|
new SP-N spec.**
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# Sub-project <N> — <Topic>: Design Spec
|
||||||
|
|
||||||
|
**Date:** YYYY-MM-DD
|
||||||
|
**Status:** Draft, awaiting user sign-off
|
||||||
|
**Branch:** `sp<N>-<short-kebab-topic>`
|
||||||
|
**Aesthetic direction:** <one line — e.g. "No new UI" or "Modern (geometric sans + bold borders + electric blue accent)">
|
||||||
|
|
||||||
|
## 1. Scope
|
||||||
|
<2-6 lines: what's in, what's out, with explicit out-of-scope list>
|
||||||
|
```
|
||||||
|
|
||||||
|
Worked example (matches this template): `docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md`.
|
||||||
|
|
||||||
|
### Plan header (every SP-N plan starts with this)
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# <Topic> Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||||
|
> superpowers:subagent-driven-development (recommended) or
|
||||||
|
> superpowers:executing-plans to implement this plan task-by-task. Steps
|
||||||
|
> use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** <one sentence — the outcome>
|
||||||
|
**Architecture:** <one paragraph — how it's structured>
|
||||||
|
**Tech Stack:** <comma-separated list>
|
||||||
|
**Spec:** [`docs/superpowers/specs/YYYY-MM-DD-cyclone-<topic>-design.md`](../specs/...)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File structure
|
||||||
|
<tree of new / modified files>
|
||||||
|
|
||||||
|
## Task 0: <setup>
|
||||||
|
## Task 1: <first user-visible step>
|
||||||
|
…
|
||||||
|
```
|
||||||
|
|
||||||
|
Real examples: `docs/superpowers/plans/2026-06-21-cyclone-skill-catalog.md`,
|
||||||
|
`docs/superpowers/plans/2026-06-21-cyclone-store-split.md`.
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Don't skip the spec ("it's a small fix").** Small fixes still get a
|
||||||
|
3-line spec when they introduce a new numbered increment. The spec is
|
||||||
|
the *what* and the audit trail; the plan is the *how*. Without a spec
|
||||||
|
the merge commit has no anchor.
|
||||||
|
- **Don't squash the merge commit.** The SP-N merge commit is the audit
|
||||||
|
trail — it tells future you exactly which feature landed and which
|
||||||
|
commits composed it. Squash collapses that into one opaque commit and
|
||||||
|
the per-commit history is lost.
|
||||||
|
- **Don't put code in the spec — the spec is the *what*, the plan is the
|
||||||
|
*how*.** Specs describe scope, goals, non-goals, and decisions. Code
|
||||||
|
snippets belong in the plan (with checkbox steps) or in the diff, not
|
||||||
|
in the spec. SP-N specs in this repo routinely have **zero** code
|
||||||
|
blocks.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- **`cyclone-tests`** — every spec lists test impact; load this when
|
||||||
|
drafting or reviewing the spec to confirm fixture / `.test.tsx`
|
||||||
|
implications.
|
||||||
|
- **`cyclone-edi`** — load when the SP-N increment touches an EDI parser,
|
||||||
|
validator rule, or CAS mapping.
|
||||||
|
- **`cyclone-tail`** — load when the increment changes the live-tail wire
|
||||||
|
format or adds a streaming page.
|
||||||
|
- **`cyclone-store`** — load when the increment adds a write-path,
|
||||||
|
touches `store.py`, or wires a new `<entity>_written` event.
|
||||||
|
- **`cyclone-api-router`** — load when the increment adds or changes an
|
||||||
|
HTTP endpoint in `api_routers/`.
|
||||||
|
- **`cyclone-frontend-page`** — load when the increment adds or
|
||||||
|
refactors a page in `src/pages/`.
|
||||||
|
- **`cyclone-cli`** — load when the increment adds a CLI subcommand or
|
||||||
|
changes exit codes.
|
||||||
|
- **`superpowers:brainstorming`** (global) — run before the spec to lock
|
||||||
|
the scope / decisions in the spec's `## Decisions (locked during
|
||||||
|
brainstorming)` section.
|
||||||
|
- **`superpowers:writing-plans`** (global) — produces the plan header
|
||||||
|
format every SP-N plan follows.
|
||||||
@@ -0,0 +1,200 @@
|
|||||||
|
---
|
||||||
|
name: cyclone-store
|
||||||
|
description: "Cyclone store write-paths, the pubsub event contract (claim_written / remittance_written / activity_recorded), and the SP21 store-split boundary map. Use when: touching store.py, adding a new entity, wiring a new write event, or splitting a store module."
|
||||||
|
---
|
||||||
|
|
||||||
|
# cyclone-store
|
||||||
|
|
||||||
|
Cyclone persists every parsed X12 batch through one facade,
|
||||||
|
`CycloneStore` (`backend/src/cyclone/store.py:882`). Every write
|
||||||
|
inserts the row AND publishes a pubsub event on the in-process
|
||||||
|
`EventBus` (`backend/src/cyclone/pubsub.py:20`) so live-tail pages
|
||||||
|
see new rows the moment they land. The event contract is the seam
|
||||||
|
between persistence and streaming — a wrong event name silently
|
||||||
|
goes stale.
|
||||||
|
|
||||||
|
As of this writing: the store is a **single 2412-line module** and
|
||||||
|
the SP21 split into `backend/src/cyclone/store/` is in progress.
|
||||||
|
Three event kinds: `claim_written`, `remittance_written`,
|
||||||
|
`activity_recorded`. Next: **SP22**.
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- **Adding a new entity.** You need a new ORM model + write method +
|
||||||
|
event kind + snapshot serializer and want to know where each piece
|
||||||
|
lives (current monolith vs. post-SP21 module).
|
||||||
|
- **Wiring a new write event.** You're adding a `<entity>_written`
|
||||||
|
event and need both the publish call in the store AND the
|
||||||
|
subscribe call in the live-tail endpoint to stay in sync.
|
||||||
|
- **Debugging a write-path issue.** A page isn't reflecting new
|
||||||
|
rows, the DB has the row but the stream is silent, or the publish
|
||||||
|
raises and rolls back the transaction.
|
||||||
|
- **Splitting a store module.** You're moving a domain out of
|
||||||
|
`store.py` into its own module under `backend/src/cyclone/store/`
|
||||||
|
and need the SP21 module list + facade re-export rules.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
1. **All writes go through `CycloneStore`.** Route handlers and
|
||||||
|
parsers must not write directly to the ORM session. The facade
|
||||||
|
opens a short-lived session via `db.SessionLocal()()`. Direct ORM
|
||||||
|
access is the #1 way the live-tail contract gets bypassed (no
|
||||||
|
event published → page silently goes stale). Read paths are
|
||||||
|
similar — prefer the facade's `iter_*` / `get_*` methods over raw
|
||||||
|
`s.execute(select(...))`.
|
||||||
|
2. **Every write publishes an event.** The event name matches the
|
||||||
|
entity: `claim_written`, `remittance_written`,
|
||||||
|
`activity_recorded` (the trailing `_recorded` signals a
|
||||||
|
non-canonical row — activity events are derived, not first-class;
|
||||||
|
current names at `store.py:1072,1081,1096`). New entities get
|
||||||
|
`<entity>_written`; activity-style side rows get
|
||||||
|
`<entity>_recorded`. Publish is **best-effort** — failures are
|
||||||
|
logged but never roll back the persisted batch
|
||||||
|
(`store.py:1097-1098`).
|
||||||
|
3. **Snapshot shape.** Each entity has a `to_ui_<entity>` serializer
|
||||||
|
(plain Python function returning a dict) — currently
|
||||||
|
`to_ui_claim`, `to_ui_remittance`, `to_ui_claim_from_orm`,
|
||||||
|
`to_ui_remittance_from_orm`, `to_ui_provider`. Post-SP21 these
|
||||||
|
move to `backend/src/cyclone/store/ui.py`. The serializer is the
|
||||||
|
single source of truth for what the frontend sees — every event
|
||||||
|
payload MUST match what the matching list endpoint returns for
|
||||||
|
that row (`store.py:1052-1054`).
|
||||||
|
4. **SP21 boundaries.** Post-split, each domain lives in its own
|
||||||
|
module under `backend/src/cyclone/store/`: `__init__.py` (facade
|
||||||
|
+ `CycloneStore` class), `exceptions.py`, `records.py`,
|
||||||
|
`orm_builders.py`, `ui.py`, `write.py`, `batches.py`,
|
||||||
|
`claim_detail.py`, `acks.py`, `backups.py`, `inbox.py`,
|
||||||
|
`providers.py`. Cross-module writes go through `CycloneStore`
|
||||||
|
facade methods, not direct module access. The facade re-exports
|
||||||
|
every name callers currently import from `cyclone.store`. Full
|
||||||
|
list: `docs/superpowers/plans/2026-06-21-cyclone-store-split.md:25-37`.
|
||||||
|
5. **No business logic in route handlers.** A route handler validates
|
||||||
|
input, calls `store.<method>(...)`, passes the `event_bus`,
|
||||||
|
returns the serialized result. Reconciliation, idempotency
|
||||||
|
checks, CAS adjustment persistence — all live in the store.
|
||||||
|
|
||||||
|
## Patterns
|
||||||
|
|
||||||
|
### A `CycloneStore.add` write — publishes events from inserted rows
|
||||||
|
|
||||||
|
Taken from `backend/src/cyclone/store.py:898-1107`. The method opens
|
||||||
|
a session, inserts rows, then runs a sync `_publish_events_sync`
|
||||||
|
after commit so subscribers can immediately re-fetch consistent data.
|
||||||
|
|
||||||
|
```python
|
||||||
|
def add(
|
||||||
|
self,
|
||||||
|
record: BatchRecord,
|
||||||
|
*,
|
||||||
|
event_bus: "EventBus | None" = None,
|
||||||
|
) -> None:
|
||||||
|
inserted_claim_ids: list[str] = []
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
s.add(Batch(id=record.id, kind=record.kind, ...))
|
||||||
|
if isinstance(record, BatchRecord837):
|
||||||
|
for claim in record.result.claims:
|
||||||
|
if s.get(Claim, claim.claim_id) is not None:
|
||||||
|
continue # idempotency: skip dupes
|
||||||
|
s.add(_claim_837_row(claim, record.id))
|
||||||
|
s.add(ActivityEvent(kind="claim_submitted", ...))
|
||||||
|
inserted_claim_ids.append(claim.claim_id)
|
||||||
|
# ... 835 branch + flush + cas adjustments ...
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
if event_bus is not None and inserted_claim_ids:
|
||||||
|
self._publish_events_sync(event_bus, record, inserted_claim_ids)
|
||||||
|
|
||||||
|
def _publish_events_sync(self, event_bus, record, claim_ids):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
for cid in claim_ids:
|
||||||
|
ui = to_ui_claim_from_orm(s.get(Claim, cid), ...)
|
||||||
|
self._sync_publish(event_bus, "claim_written", ui)
|
||||||
|
# ... remittance + activity loops ...
|
||||||
|
```
|
||||||
|
|
||||||
|
`EventBus.publish` is async but the body is pure sync `put_nowait`,
|
||||||
|
so the store calls `_sync_publish` directly to avoid forcing sync
|
||||||
|
FastAPI handlers to await.
|
||||||
|
|
||||||
|
### Backend `/api/<resource>/stream` endpoint — subscribes to the event
|
||||||
|
|
||||||
|
Taken from `backend/src/cyclone/api.py:1380-1401`. Two phases — eager
|
||||||
|
snapshot, then live subscription — wrapped in `StreamingResponse`
|
||||||
|
with `media_type="application/x-ndjson"`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
@app.get("/api/claims/stream")
|
||||||
|
async def claims_stream(request: Request, ...) -> StreamingResponse:
|
||||||
|
bus: EventBus = request.app.state.event_bus
|
||||||
|
|
||||||
|
async def gen() -> AsyncIterator[bytes]:
|
||||||
|
rows = store.iter_claims(status=status, ...) # 1. Snapshot
|
||||||
|
for row in rows:
|
||||||
|
yield _ndjson_line({"type": "item", "data": row})
|
||||||
|
yield _ndjson_line({"type": "snapshot_end", "data": {"count": len(rows)}})
|
||||||
|
|
||||||
|
async for chunk in _tail_events(request, bus, ["claim_written"]): # 2. Live
|
||||||
|
yield chunk
|
||||||
|
|
||||||
|
return StreamingResponse(gen(), media_type="application/x-ndjson")
|
||||||
|
```
|
||||||
|
|
||||||
|
`_ndjson_line` and `_tail_events` live in
|
||||||
|
`backend/src/cyclone/api_helpers.py`. The `["remittance_written"]`
|
||||||
|
and `["activity_recorded"]` subscriptions are at `api.py:1891,2002`.
|
||||||
|
|
||||||
|
### Backend test — asserts both the row AND the event landed
|
||||||
|
|
||||||
|
The autouse `conftest.py` (`backend/tests/conftest.py:20`) wires a
|
||||||
|
fresh `EventBus` onto `app.state` per test.
|
||||||
|
|
||||||
|
```python
|
||||||
|
def test_publishes_claim_written_event(client: TestClient) -> None:
|
||||||
|
bus = app.state.event_bus
|
||||||
|
resp = client.post("/api/parse-837",
|
||||||
|
files={"file": ("x.837", MINIMAL_837, "text/plain")})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert list(store.iter_claims(limit=10)) # row landed
|
||||||
|
|
||||||
|
queues = bus._subscribers.get("claim_written", []) # event published
|
||||||
|
assert queues
|
||||||
|
evt = queues[0].get_nowait()
|
||||||
|
assert evt["_kind"] == "claim_written"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Don't read directly from the ORM in a route handler — go through
|
||||||
|
the snapshot serializer.** A handler that does
|
||||||
|
`with db.SessionLocal()() as s: row = s.get(Claim, cid); return row`
|
||||||
|
bypasses `to_ui_claim_from_orm` and silently drifts from the event
|
||||||
|
payload shape. Always call `store.get_claim_detail(cid)` (or the
|
||||||
|
equivalent `get_*` facade method).
|
||||||
|
- **Don't introduce a new event name without updating the subscriber
|
||||||
|
list.** Today every `<entity>_written` event has exactly one
|
||||||
|
consumer — the matching `/api/<resource>/stream` endpoint,
|
||||||
|
currently in `backend/src/cyclone/api.py` (claims at `:1398`,
|
||||||
|
remittances at `:1891`, activity at `:2002`). Any future split
|
||||||
|
into `backend/src/cyclone/api_routers/` must wire the same
|
||||||
|
subscription.
|
||||||
|
- **Don't merge a write method with its event publication into
|
||||||
|
separate places.** `_publish_events_sync` lives next to `add` in
|
||||||
|
`store.py:1042-1107` so reviewers see both halves of the contract
|
||||||
|
in one diff. Post-SP21 the same rule applies.
|
||||||
|
- **Don't make the publish call blocking on commit failures.**
|
||||||
|
Publish errors are caught and logged at `store.py:1097-1098`; a
|
||||||
|
failing subscriber MUST NOT roll back the persisted batch. The
|
||||||
|
batch is the source of truth; the event is the cache-invalidation
|
||||||
|
hint.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- **`cyclone-edi`** — the parsed `ClaimOutput` / `ParseResult<EDI>`
|
||||||
|
lands in the store via `CycloneStore.add`.
|
||||||
|
- **`cyclone-api-router`** — the route that calls `store.<method>(...)`
|
||||||
|
and (for stream endpoints) subscribes to the matching
|
||||||
|
`<entity>_written` event.
|
||||||
|
- **`cyclone-tail`** — the consumer side of the event contract;
|
||||||
|
load when changing the wire format or adding a streaming hook.
|
||||||
|
- **`cyclone-tests`** — write-path tests live under `backend/tests/`
|
||||||
|
and assert both the DB row and the event payload.
|
||||||
@@ -0,0 +1,200 @@
|
|||||||
|
---
|
||||||
|
name: cyclone-tail
|
||||||
|
description: "Cyclone live-tail streaming wire format and the useTailStream / useMergedTail hook triplet. Use when: adding a new streaming list page, changing the wire format, debugging stalled/reconnecting state, or modifying the StatusPill behavior."
|
||||||
|
---
|
||||||
|
|
||||||
|
# cyclone-tail
|
||||||
|
|
||||||
|
Cyclone keeps the Claims, Remittances, and Activity pages live without
|
||||||
|
polling: every store write publishes an internal EventBus event, the
|
||||||
|
page opens a `GET /api/<resource>/stream` HTTP/1.1 chunked-NDJSON
|
||||||
|
connection, and new rows land in the table the moment they hit the
|
||||||
|
database. This skill codifies the wire format, the hook triplet, and
|
||||||
|
the backoff/stall machinery so additions stay consistent with the
|
||||||
|
three streaming pages already shipped (`Claims`, `Remittances`,
|
||||||
|
`ActivityLog`).
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- **Adding a new streaming page.** Mounting `useTailStream(resource)`
|
||||||
|
on a page — you need the hook triplet shape (initial fetch +
|
||||||
|
`useTailStream` + `useMergedTail`), the `<TailStatusPill>` wiring,
|
||||||
|
and the dedup rules in `useMergedTail`.
|
||||||
|
- **Changing the wire format.** Adding a new event type — update
|
||||||
|
`TailEvent` in `src/lib/tail-stream.ts:22-44`, the dispatch switch
|
||||||
|
in `src/hooks/useTailStream.ts:173-195`, the emitter in
|
||||||
|
`backend/src/cyclone/api_helpers.py:tail_events()`, and
|
||||||
|
`references/wire-format.md`.
|
||||||
|
- **Debugging stalled/reconnecting state.** Confirm whether the
|
||||||
|
backend is heartbeating (`CYCLONE_TAIL_HEARTBEAT_S`, default `15s`)
|
||||||
|
or the stall timer fired (`STALL_TIMEOUT_MS = 30_000` at
|
||||||
|
`src/hooks/useTailStream.ts:53`).
|
||||||
|
- **Tuning heartbeat/stall timing.** Changing the 30s stall threshold
|
||||||
|
or the 15s heartbeat interval — README's "Status pill" + "Knobs"
|
||||||
|
tables need to stay in sync (`README.md:109-129`).
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
1. **Wire format.** Newline-delimited JSON. Every line is
|
||||||
|
`{"type": ..., "data": ...}`. Known `type` values: `item`
|
||||||
|
(per-row envelope), `snapshot_end` (`{"count": N}` marker after the
|
||||||
|
snapshot), `heartbeat` (`{"ts": "<iso-8601>"}` keep-alive),
|
||||||
|
`item_dropped` (`{"id": "..."}` queue-overflow notice), `error`
|
||||||
|
(`{"message": "..."}` promoted to a thrown error by the hook).
|
||||||
|
Defined at `src/lib/tail-stream.ts:22-44`; emitted by
|
||||||
|
`backend/src/cyclone/api.py:1357-2006`. See
|
||||||
|
`references/wire-format.md`.
|
||||||
|
2. **Hook triplet.** Streaming pages compose three pieces:
|
||||||
|
`useTailStream(resource)` (`src/hooks/useTailStream.ts:80` — opens
|
||||||
|
the stream, drives the backoff/stall state machine, dispatches
|
||||||
|
`item` events into `useTailStore`),
|
||||||
|
`useMergedTail(resource, baseItems, filterFn?)`
|
||||||
|
(`src/hooks/useMergedTail.ts:25` — returns
|
||||||
|
`baseItems + tailSlice` dedup'd against `baseItems`), and a
|
||||||
|
per-resource initial-fetch hook (`useClaims`, `useRemittances`,
|
||||||
|
`useActivity`). The page wires all three — see
|
||||||
|
`src/pages/Claims.tsx:87-89`.
|
||||||
|
3. **Stall threshold.** 30 seconds of total silence — heartbeats
|
||||||
|
included — flips status to `stalled` and surfaces the
|
||||||
|
`↻ Reconnect` button on `<TailStatusPill>`. Constant:
|
||||||
|
`STALL_TIMEOUT_MS = 30_000` at `src/hooks/useTailStream.ts:53`.
|
||||||
|
Re-armed on every event including `heartbeat` and `item_dropped`
|
||||||
|
(`useTailStream.ts:124-140`). Don't change without updating
|
||||||
|
`README.md:109-123`.
|
||||||
|
4. **Snapshot first.** Every stream emits the snapshot before any
|
||||||
|
live `item` events, then closes with exactly one `snapshot_end`
|
||||||
|
carrying `{"count": N}`. The hook uses `snapshot_end` to flip
|
||||||
|
`<TailStatusPill>` from `connecting` to `live` and to reset the
|
||||||
|
reconnect backoff counter (`useTailStream.ts:174-180`).
|
||||||
|
5. **Content-Type.** Stream endpoints respond with
|
||||||
|
`media_type="application/x-ndjson"` — see
|
||||||
|
`backend/src/cyclone/api.py:1401,1894,2005`. Frontend sets
|
||||||
|
`Accept: application/x-ndjson` at `src/lib/tail-stream.ts:67-70`.
|
||||||
|
Never `application/json` for a stream endpoint.
|
||||||
|
6. **Backoff.** Transient errors retry with `1s → 2s → 4s → 8s → 16s
|
||||||
|
→ 30s` capped — `BACKOFF_STEPS_MS` at
|
||||||
|
`src/hooks/useTailStream.ts:48-50`. Counter resets on every
|
||||||
|
`snapshot_end`.
|
||||||
|
7. **Heartbeat knob.** Idle heartbeat interval is configurable via
|
||||||
|
`CYCLONE_TAIL_HEARTBEAT_S` (default `15`, parsed at call time in
|
||||||
|
`backend/src/cyclone/api_helpers.py:185-198`). Tests override to
|
||||||
|
a small value to keep runtime bounded.
|
||||||
|
|
||||||
|
## Patterns
|
||||||
|
|
||||||
|
### Page-hook skeleton — `Claims.tsx`
|
||||||
|
|
||||||
|
Pattern from `src/pages/Claims.tsx:85-90`. Three hooks in order:
|
||||||
|
`useClaims(params)` (initial TanStack Query fetch),
|
||||||
|
`useTailStream("claims")` (opens the stream, owns status state), and
|
||||||
|
`useMergedTail("claims", data?.items ?? [], tailFilterFn)` (combines
|
||||||
|
initial snapshot with live tail, dedup'd by id, filtered by the page's
|
||||||
|
predicate applied AFTER dedup at `useMergedTail.ts:72-74`).
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { useClaims } from "@/hooks/useClaims";
|
||||||
|
import { useTailStream } from "@/hooks/useTailStream";
|
||||||
|
import { useMergedTail } from "@/hooks/useMergedTail";
|
||||||
|
|
||||||
|
export function ClaimsPage() {
|
||||||
|
const { data } = useClaims({ status: "submitted" });
|
||||||
|
const { status, lastEventAt, forceReconnect } = useTailStream("claims");
|
||||||
|
const tailFilterFn = (c: Claim) => c.status === "submitted";
|
||||||
|
const items = useMergedTail("claims", data?.items ?? [], tailFilterFn);
|
||||||
|
// <TailStatusPill status={status} lastEventAt={lastEventAt}
|
||||||
|
// onReconnect={forceReconnect} /> + <Table items={items} />
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Backend `/api/foo/stream` endpoint
|
||||||
|
|
||||||
|
Pattern from `backend/src/cyclone/api.py:1357-1401`. Register BEFORE
|
||||||
|
`/api/foo/{foo_id}` so the literal `stream` segment doesn't match as
|
||||||
|
an id. Two phases — eager snapshot, then live subscription — wrapped
|
||||||
|
in `StreamingResponse` with `media_type="application/x-ndjson"`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
@app.get("/api/foo/stream")
|
||||||
|
async def foo_stream(
|
||||||
|
request: Request,
|
||||||
|
status: str | None = Query(None),
|
||||||
|
limit: int = Query(100, ge=1, le=1000),
|
||||||
|
) -> StreamingResponse:
|
||||||
|
bus: EventBus = request.app.state.event_bus
|
||||||
|
|
||||||
|
async def gen() -> AsyncIterator[bytes]:
|
||||||
|
# 1. Snapshot.
|
||||||
|
rows = store.iter_foos(status=status, limit=limit)
|
||||||
|
for row in rows:
|
||||||
|
yield _ndjson_line({"type": "item", "data": row})
|
||||||
|
yield _ndjson_line({"type": "snapshot_end", "data": {"count": len(rows)}})
|
||||||
|
# 2. Live subscription + heartbeats.
|
||||||
|
async for chunk in _tail_events(request, bus, ["foo_written"]):
|
||||||
|
yield chunk
|
||||||
|
|
||||||
|
return StreamingResponse(gen(), media_type="application/x-ndjson")
|
||||||
|
```
|
||||||
|
|
||||||
|
`_ndjson_line` and `_tail_events` live in
|
||||||
|
`backend/src/cyclone/api_helpers.py`; the latter forwards EventBus
|
||||||
|
events as `item` lines and emits `heartbeat` lines on the cadence
|
||||||
|
from `heartbeat_seconds()`.
|
||||||
|
|
||||||
|
### `<TailStatusPill>` wiring
|
||||||
|
|
||||||
|
Pattern from `src/components/TailStatusPill.tsx:22-83`. The pill
|
||||||
|
takes `status` + `lastEventAt` from `useTailStream` plus
|
||||||
|
`forceReconnect` so the `↻ Reconnect` button wires to the hook's
|
||||||
|
`reconnectNonce` bump (aborts and reopens the stream —
|
||||||
|
`useTailStream.ts:99-101`). The button only renders when
|
||||||
|
`status === "stalled" || status === "error"` (`TailStatusPill.tsx:52`).
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Don't hand-roll `fetch` + `ReadableStream` parsing in a page.**
|
||||||
|
All stream consumers go through `streamTail(resource, opts?)` at
|
||||||
|
`src/lib/tail-stream.ts:58`. The parser handles `TextDecoderStream`,
|
||||||
|
newline splitting, malformed-line tolerance (`console.warn` + skip),
|
||||||
|
the `KNOWN_TYPES` allowlist, and abort-on-signal semantics
|
||||||
|
(`tail-stream.ts:75,98,107`). Duplicating it loses all of those.
|
||||||
|
- **Don't change the wire format on one endpoint without updating
|
||||||
|
the others.** The parser (`src/lib/tail-stream.ts`) and the hook
|
||||||
|
dispatch switch (`useTailStream.ts:173-195`) are shared across all
|
||||||
|
three live streams. Adding a new event type means updating
|
||||||
|
`TailEvent`, `KNOWN_TYPES`, the dispatch `case`, the `armStall`
|
||||||
|
re-arm list, and the backend emitter — in that order.
|
||||||
|
- **Don't emit `item` events before `snapshot_end`.** The hook uses
|
||||||
|
`snapshot_end` as the marker to flip `<TailStatusPill>` from
|
||||||
|
`connecting` to `live` and to reset the reconnect backoff counter.
|
||||||
|
Emitting `item`s first lands rows in `useTailStore` but the UI
|
||||||
|
still reads "Connecting" — operators see a flash of stale state.
|
||||||
|
Emit snapshot, then `snapshot_end`, then live (`api.py:1386-1401`).
|
||||||
|
- **Don't change the 30s stall threshold without updating the README
|
||||||
|
"Status pill" table.** The constant
|
||||||
|
(`STALL_TIMEOUT_MS = 30_000` at `useTailStream.ts:53`) is the
|
||||||
|
contract the pill is documented against — a bump needs the matching
|
||||||
|
edit at `README.md:118`.
|
||||||
|
- **Don't add `useTailStream` calls from `useFoo.ts` hooks.**
|
||||||
|
Streaming connections belong on the page (`src/pages/Claims.tsx`,
|
||||||
|
`Remittances.tsx`, `ActivityLog.tsx`). Hoisting into `useFoo`
|
||||||
|
couples the lifecycle to whoever mounts it and breaks
|
||||||
|
one-resource-one-page ownership.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- **`cyclone-frontend-page`** — page components live in `src/pages/`;
|
||||||
|
load when adding or refactoring a streaming page to confirm the
|
||||||
|
route + `PageHeader` + table conventions.
|
||||||
|
- **`cyclone-api-router`** — endpoint conventions (`api_routers/` for
|
||||||
|
resource-group routers, `api.py` for the live-tail endpoints); load
|
||||||
|
when adding or changing an HTTP endpoint that surfaces streamed or
|
||||||
|
paginated data.
|
||||||
|
- **`cyclone-store`** — write-path conventions in `store.py` and the
|
||||||
|
pubsub event contract (`claim_written`, `remittance_written`,
|
||||||
|
`activity_recorded`); load when adding a new entity whose writes
|
||||||
|
should fan out to a stream endpoint.
|
||||||
|
- **`cyclone-tests`** — frontend `*.test.tsx` siblings cover
|
||||||
|
`useTailStream`, `useMergedTail`, `TailStatusPill`; backend
|
||||||
|
`test_api_stream_live.py` covers the three live-tail endpoints;
|
||||||
|
load when the increment changes wire-format behavior or adds a
|
||||||
|
streaming hook.
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
# Live-tail wire format — field reference
|
||||||
|
|
||||||
|
The Cyclone live-tail NDJSON contract is owned by the frontend parser
|
||||||
|
(`src/lib/tail-stream.ts:22-44`) and the backend emitter
|
||||||
|
(`backend/src/cyclone/api_helpers.py:tail_events()` + the three
|
||||||
|
endpoints in `backend/src/cyclone/api.py:1357-2006`). This file is the
|
||||||
|
quick reference; the canonical prose lives in the README and the
|
||||||
|
source-of-truth type definitions.
|
||||||
|
|
||||||
|
## Source
|
||||||
|
|
||||||
|
Wire format excerpt copied verbatim from `README.md:76-94` (the
|
||||||
|
"Live updates → Wire format" section). The README is the user-facing
|
||||||
|
exposition; this reference adds the per-line field semantics and the
|
||||||
|
parser-tolerance rules from the code.
|
||||||
|
|
||||||
|
> Each stream endpoint emits newline-delimited JSON. The first batch
|
||||||
|
> is the **snapshot** of currently-known rows; after that comes
|
||||||
|
> **`snapshot_end`** with the count, then the **live** events.
|
||||||
|
>
|
||||||
|
> ```json
|
||||||
|
> {"type":"item","data":{"id":"CLM-1", "...":"..."}}
|
||||||
|
> {"type":"item","data":{"id":"CLM-2", "...":"..."}}
|
||||||
|
> {"type":"snapshot_end","data":{"count":2}}
|
||||||
|
> {"type":"item","data":{"id":"CLM-3", "...":"..."}} ← live
|
||||||
|
> {"type":"heartbeat","data":{"ts":"2026-06-20T23:17:09Z"}} ← idle keep-alive
|
||||||
|
> ```
|
||||||
|
>
|
||||||
|
> Lines are `{"type": ..., "data": ...}`; known types are `item`,
|
||||||
|
> `snapshot_end`, `heartbeat`, and (rare) `item_dropped` /
|
||||||
|
> `error`. Heartbeats keep the connection alive when nothing is
|
||||||
|
> happening — clients flip to `stalled` after 30s of total silence
|
||||||
|
> (heartbeat or otherwise) and surface a **↻ Reconnect** button.
|
||||||
|
|
||||||
|
## Per-line field reference
|
||||||
|
|
||||||
|
| `type` | `data` shape | Required? | Emitted by | Parser behavior |
|
||||||
|
| -------------- | ----------------------------------------- | --------- | ------------------------------------------- | -------------------------------------------- |
|
||||||
|
| `item` | resource-specific row (`Claim` / `Remittance` / `Activity`) | yes, in `data` (the per-row envelope) | snapshot loop + `_tail_events` forwarding `claim_written` / `remittance_written` / `activity_recorded` | `dispatch(resource, ev.data)` → `useTailStore.addClaim` / `addRemittance` / `addActivity` (first-write-wins dedup on the id-keyed slices — `tail-store.ts:104,120`). |
|
||||||
|
| `snapshot_end` | `{"count": N}` (integer ≥ 0) | yes, on every stream | `api.py:1395,1889,1999` (one per stream, after the snapshot loop) | Flips `<TailStatusPill>` from `connecting` to `live`, resets the reconnect backoff counter to 0 (`useTailStream.ts:174-180`). |
|
||||||
|
| `heartbeat` | `{"ts": "<iso-8601 UTC>"}` | yes, but only when idle | `_tail_events` in `api_helpers.py:241-245` (cadence from `heartbeat_seconds()`, default 15s) | Re-arms the stall timer (`useTailStream.ts:124-140`); no state change. |
|
||||||
|
| `item_dropped` | `{"id": "<string>"}` | optional (rare; queue overflow) | EventBus drop-oldest path (per the spec at `docs/superpowers/specs/2026-06-20-cyclone-live-tail-design.md:299`) | Re-arms the stall timer; no state change. The `id` field is informational — the hook does not refetch on drop, the page does (see Spec §3.6). |
|
||||||
|
| `error` | `{"message": "<string>"}` | optional (server-side failure) | Reserved for future server-emitted errors (currently only thrown client-side) | Hook promotes to a thrown `Error(message)` so the catch block runs the reconnect machinery (`useTailStream.ts:190-194`). |
|
||||||
|
|
||||||
|
## Parser tolerance
|
||||||
|
|
||||||
|
The shared parser at `src/lib/tail-stream.ts` is intentionally
|
||||||
|
forgiving so a single bad frame doesn't kill the stream:
|
||||||
|
|
||||||
|
- **Trailing `\r`** is stripped per line (`tail-stream.ts:116`) so a
|
||||||
|
CRLF-terminated stream still parses.
|
||||||
|
- **Malformed JSON** (`JSON.parse` throws) → `console.warn` + skip the
|
||||||
|
line; the iterator continues (`tail-stream.ts:122-131`).
|
||||||
|
- **Unknown `type`** (not in `KNOWN_TYPES`) → `console.warn` + skip
|
||||||
|
(`tail-stream.ts:141-148`). Adding a new event type is
|
||||||
|
forward-compatible: old clients see warn lines, new clients see the
|
||||||
|
typed event.
|
||||||
|
- **Empty lines** (consecutive `\n`s) → silently skipped
|
||||||
|
(`tail-stream.ts:118`).
|
||||||
|
- **No trailing newline** → flushed as a final partial line on
|
||||||
|
stream close (`tail-stream.ts:154-178`).
|
||||||
|
- **Abort signal** → iterator exits cleanly without throwing
|
||||||
|
(`tail-stream.ts:75,98,107`).
|
||||||
|
|
||||||
|
## Endpoint inventory
|
||||||
|
|
||||||
|
| Method | Path | Subscribes to | Default sort | Defined at |
|
||||||
|
| ------ | ------------------------- | -------------------- | ------------------- | ----------------------------------- |
|
||||||
|
| GET | `/api/claims/stream` | `claim_written` | `-submission_date` | `backend/src/cyclone/api.py:1357` |
|
||||||
|
| GET | `/api/remittances/stream` | `remittance_written` | `-received_date` | `backend/src/cyclone/api.py:1858` |
|
||||||
|
| GET | `/api/activity/stream` | `activity_recorded` | `-timestamp` (limit 50) | `backend/src/cyclone/api.py:1971` |
|
||||||
|
|
||||||
|
All three accept the same query params as their non-streaming
|
||||||
|
counterparts (`status`, `payer`, `date_from`, …) so a frontend can
|
||||||
|
swap a one-shot fetch for a tail with no URL surgery. Responses are
|
||||||
|
`Content-Type: application/x-ndjson`.
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
---
|
||||||
|
name: cyclone-tests
|
||||||
|
description: "Cyclone pytest + vitest fixture patterns, prodfiles layout, backend/tests/fixtures/ conventions, .test.tsx sibling rule. Use when: adding a backend pytest case, adding a frontend vitest test, or wiring in a real-EDI prodfiles sample."
|
||||||
|
---
|
||||||
|
|
||||||
|
# cyclone-tests
|
||||||
|
|
||||||
|
The Cyclone test suite is split two ways: **backend pytest** (89 test files under `backend/tests/`, 13 flat fixtures in `backend/tests/fixtures/`, one autouse `conftest.py` that resets the DB per-test) and **frontend vitest** (59 `*.test.ts(x)` siblings across `src/`, two rendering styles — `@testing-library/react` and a custom `createRoot`+`Probe` shim). This skill codifies the conventions so additions stay consistent with what's already there.
|
||||||
|
|
||||||
|
As of this writing: **89 backend test files**, **13 flat backend fixtures**, **59 frontend `*.test.ts(x)` siblings**, and **23 prodfiles samples** across `docs/prodfiles/{837p-from-axiscare,835fromco,FromHPE,claims}/`. The next increment is **SP22**.
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- **Adding a backend pytest case.** You're about to add a new `test_*.py` under `backend/tests/` and need the autouse DB-fixture rules, the fixture-path convention, and the right naming flavor (`test_api_*.py` vs `test_<module>_*.py`).
|
||||||
|
- **Adding a frontend vitest test.** You're about to add a `*.test.ts(x)` sibling and need the `// @vitest-environment happy-dom` setup, the act-environment flag, and the right rendering helper (`@testing-library/react` vs the `createRoot`+`Probe` shim).
|
||||||
|
- **Dropping in a prodfiles fixture.** You have a real EDI sample under `docs/prodfiles/<source>/` and need the copy step that makes it test-runnable without coupling the test to the prodfiles archive.
|
||||||
|
- **Debugging a flaky test.** A test passes locally but flakes in CI — load this skill to check the determinism + no-network rules before chasing the symptom.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
1. **Frontend sibling rule.** Every new file in `src/` that contains testable logic gets a `*.test.ts(x)` next to it. `useFoo.ts` → `useFoo.test.ts`. `ClaimDrawer.tsx` → `ClaimDrawer.test.tsx`. The 59 existing siblings follow this; CI implicitly enforces it via the default `src/**/*.test.ts(x)` glob in `vitest.config.ts`.
|
||||||
|
2. **Backend test location.** Tests live under `backend/tests/test_*.py`. Two flavors, two naming patterns:
|
||||||
|
- **Integration tests** (FastAPI surface) → `test_api_<topic>_<verb>.py` — e.g. `test_api_parse_persists.py`, `test_api_999.py`.
|
||||||
|
- **Pure-unit tests** (parsers, validators, store internals) → `test_<module>_<behavior>.py` — e.g. `test_cas_codes.py`, `test_pubsub.py`.
|
||||||
|
3. **Prodfiles drop-in.** Real EDI samples live under `docs/prodfiles/<source>/<file>.txt` (sources seen so far: `837p-from-axiscare/`, `835fromco/`, `FromHPE/`, `claims/`). To use one in a test: copy the file to `backend/tests/fixtures/<descriptive-name>.txt` (flat — no per-test subdirectories; the existing 13 fixtures all sit at the top level) and reference it from the test as a module-level `Path` constant. See `## Patterns` for the exact line.
|
||||||
|
4. **Determinism.** Time-sensitive tests must not depend on wall-clock time.
|
||||||
|
- **Frontend** uses `vi.useFakeTimers()` + `vi.setSystemTime(new Date("YYYY-MM-DDTHH:MM:SSZ"))` (see `src/components/TailStatusPill.test.tsx:54-58`) and `vi.advanceTimersByTime(ms)` to drive interval / backoff code deterministically.
|
||||||
|
- **Backend** dates are passed explicitly as fixture values — e.g. `datetime.now(timezone.utc)` is fine for "now-ish" anchors, but for fixed dates pass `datetime(2026, 6, 20, tzinfo=timezone.utc)`. The project does **not** currently use `freezegun` or `mock.patch(datetime)`; if you need deterministic date mocking, propose adding `freezegun` to `backend/pyproject.toml` rather than rolling your own.
|
||||||
|
5. **No network.** Tests must not hit the network.
|
||||||
|
- **Backend:** `fastapi.testclient.TestClient(app)` runs in-process; no uvicorn. The autouse `conftest.py` fixture (`backend/tests/conftest.py:20`) points `CYCLONE_DB_URL` at `tmp_path/test.db`, calls `db._reset_for_tests()` + `db.init_db()`, and wires a fresh `EventBus` onto `app.state`.
|
||||||
|
- **Frontend:** `vi.stubGlobal("fetch", vi.fn().mockResolvedValue(...))` for hooks; `vitest.config.ts` sets `VITE_API_BASE_URL=http://test.local` so the `api` module doesn't throw `notConfiguredError` before the mock fires.
|
||||||
|
6. **pytest collection.** Run `cd backend && python -m pytest tests/<file>::<name> -v` for the fastest single-test feedback loop. Run `cd backend && python -m pytest tests/<file> -v` for one file. Run `cd backend && python -m pytest` for the full suite — this is the merge gate. Frontend: `npm test` (alias for `vitest run`) for the full suite; `npx vitest run src/hooks/useFoo.test.ts` for one file.
|
||||||
|
|
||||||
|
## Patterns
|
||||||
|
|
||||||
|
### Backend pytest using a fixture + per-test SQLite DB
|
||||||
|
|
||||||
|
Canonical shape — see `backend/tests/test_api_999.py:1-50` for the full file. The autouse `conftest.py` already provides the DB init + `EventBus` reset; most tests just add a `client` fixture.
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""Tests for the FastAPI surface in cyclone.api for the 999 endpoint."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone.api import app
|
||||||
|
|
||||||
|
# Fixture reference — flat, module-level Path constant. NEVER reach into
|
||||||
|
# docs/prodfiles/ from a test; the fixtures/ dir is the test-consumed surface.
|
||||||
|
ACCEPTED = Path(__file__).parent / "fixtures" / "minimal_999.txt"
|
||||||
|
REJECTED = Path(__file__).parent / "fixtures" / "minimal_999_rejected.txt"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client() -> TestClient:
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_999_endpoint_happy_path(client: TestClient):
|
||||||
|
text = ACCEPTED.read_text()
|
||||||
|
resp = client.post(
|
||||||
|
"/api/parse-999",
|
||||||
|
files={"file": ("minimal_999.txt", text, "text/plain")},
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
body = resp.json()
|
||||||
|
assert body["ack"]["ack_code"] == "A"
|
||||||
|
```
|
||||||
|
|
||||||
|
Override the autouse DB fixture only when you need a custom env (e.g. a per-test backup directory) — see `backend/tests/test_999_rejected_state.py:20-25`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _setup(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/inbox.db")
|
||||||
|
from cyclone import db
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
yield
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontend vitest `*.test.tsx` for a component using fake timers
|
||||||
|
|
||||||
|
Pattern taken from `src/components/TailStatusPill.test.tsx:1-62` — uses `vi.useFakeTimers()` + `vi.setSystemTime(...)` to drive interval-based code deterministically.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// @vitest-environment happy-dom
|
||||||
|
// React's act warnings need an act-aware environment — mirror the other
|
||||||
|
// hook tests in this repo.
|
||||||
|
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
||||||
|
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { MyComponent } from "./MyComponent";
|
||||||
|
|
||||||
|
describe("MyComponent", () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
vi.setSystemTime(new Date("2026-06-20T12:00:30Z"));
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
vi.useRealTimers();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("test_renders_after_interval_tick", () => {
|
||||||
|
vi.advanceTimersByTime(30_000); // drive the setInterval
|
||||||
|
// …assert…
|
||||||
|
});
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontend vitest `*.test.tsx` for a component using `@testing-library/react` + `happy-dom`
|
||||||
|
|
||||||
|
Pattern taken from `src/hooks/useInboxLanes.test.ts:1-80`. (A handful of older tests — `useClaimDetail.test.ts`, `useDrawerUrlState.test.ts`, `TailStatusPill.test.tsx` — roll a custom `createRoot`+`Probe` shim instead. Both styles are accepted; `@testing-library/react` is preferred when the hook has async dependencies because `waitFor` is built in.)
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// @vitest-environment happy-dom
|
||||||
|
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
||||||
|
|
||||||
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { act, cleanup, renderHook, waitFor } from "@testing-library/react";
|
||||||
|
import { useMyHook } from "./useMyHook";
|
||||||
|
|
||||||
|
vi.mock("@/lib/api", () => ({
|
||||||
|
api: { fetchFoo: vi.fn() },
|
||||||
|
}));
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
cleanup();
|
||||||
|
vi.unstubAllGlobals();
|
||||||
|
vi.useRealTimers();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("useMyHook", () => {
|
||||||
|
it("loads data on mount", async () => {
|
||||||
|
vi.stubGlobal("fetch", vi.fn().mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: async () => ({ items: [] }),
|
||||||
|
}));
|
||||||
|
const { result } = renderHook(() => useMyHook());
|
||||||
|
await waitFor(() => expect(result.current.loading).toBe(false));
|
||||||
|
expect(result.current.items).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Don't put frontend tests in `src/__tests__/`.** No such directory exists in the codebase — `__tests__` only appears in the SP-catalog plan itself. Use the sibling rule (`useFoo.ts` → `useFoo.test.ts`).
|
||||||
|
- **Don't reach into `docs/prodfiles/` directly from a test.** Always copy to `backend/tests/fixtures/<name>.txt` first. The prodfiles directory is the source-of-truth archive and may be reorganized; the fixtures directory is the test-consumed surface and is stable.
|
||||||
|
- **Don't use wall-clock sleeps for timing.** A handful of legacy tests use `await new Promise((r) => setTimeout(r, 200))` (see `src/components/SearchBar.test.tsx:281,323,373` and `src/hooks/useSearch.test.ts:174`) — these are known flaky in CI. Use `vi.useFakeTimers()` + `vi.advanceTimersByTime(ms)` instead, or `waitFor(...)` from `@testing-library/react`.
|
||||||
|
|
||||||
|
## Related skills
|
||||||
|
|
||||||
|
- **`cyclone-spec`** — every SP-N spec lists test impact; load this when drafting or reviewing the spec to confirm fixture / `.test.tsx` implications for the increment.
|
||||||
|
- **`cyclone-edi`** — most backend tests cover parser + validator behavior; load when the increment touches an EDI parser or adds a validator rule (R200/R210/NPI Luhn/EIN/CAS).
|
||||||
|
- **`cyclone-tail`** — most frontend tests cover hook behavior (`useTailStream`, `useMergedTail`); load when the increment changes the wire format or adds a streaming hook.
|
||||||
|
- **`cyclone-store`** — write-path tests live here; load when the increment touches `store.py`, adds a new entity, or wires a new `<entity>_written` event.
|
||||||
|
- **`cyclone-api-router`** — endpoint tests live in `backend/tests/test_api_*.py`; load when the increment adds or changes an HTTP endpoint.
|
||||||
|
- **`cyclone-frontend-page`** — page-component tests live next to pages in `src/pages/*.test.tsx`; load when the increment adds or refactors a page.
|
||||||
|
- **`cyclone-cli`** — CLI smoke tests live in `backend/tests/test_cli_*.py`; load when the increment adds a CLI subcommand.
|
||||||
|
- **`superpowers:test-driven-development`** (global) — the upstream TDD workflow. Load first when starting any new feature increment; this skill only codifies the Cyclone-specific test layout on top of TDD.
|
||||||
@@ -51,6 +51,52 @@ VITE_API_BASE_URL=http://127.0.0.1:8000
|
|||||||
Without that, the UI falls back to its in-memory sample store via the
|
Without that, the UI falls back to its in-memory sample store via the
|
||||||
existing `data` adapter (parses are disabled).
|
existing `data` adapter (parses are disabled).
|
||||||
|
|
||||||
|
## Pipeline automation agent
|
||||||
|
|
||||||
|
For unattended round-trips (an agent / scheduler drops an 837P file
|
||||||
|
into the pipeline and waits for the 999 back from Gainwell), see the
|
||||||
|
`cyclone-pipeline` sibling project at `/Users/openclaw/dev/cyclone-pipeline/`.
|
||||||
|
It drives the full 7-phase state machine — preflight → browser upload →
|
||||||
|
parse verification → SFTP submit → TA1 wait → 999 wait → scan +
|
||||||
|
report — with structured JSON logs, crash-safe resume, and a
|
||||||
|
self-contained per-run folder under `./runs/`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Single file
|
||||||
|
cyclone-pipeline run /path/to/axiscare-837p.txt
|
||||||
|
|
||||||
|
# Resume a crashed run
|
||||||
|
cyclone-pipeline resume 2026-06-21-1430-001
|
||||||
|
|
||||||
|
# On/after the following Monday, verify the 835 arrived
|
||||||
|
cyclone-pipeline check-835 2026-06-21-1430-001
|
||||||
|
```
|
||||||
|
|
||||||
|
The 835 is **not** waited for inline (it lands the following Monday on
|
||||||
|
the CO Medicaid payment cycle). See
|
||||||
|
[`cyclone-pipeline/README.md`](../cyclone-pipeline/README.md) for
|
||||||
|
install, embed-in-agent example, exit codes, and the report format.
|
||||||
|
|
||||||
|
## Skills
|
||||||
|
|
||||||
|
Cyclone ships 8 project-scoped AI-assistant skills under
|
||||||
|
[`.superpowers/skills/`](.superpowers/skills/). Each one codifies the
|
||||||
|
conventions for a major subsystem so the next contributor (human or
|
||||||
|
AI) gets the lay of the land automatically.
|
||||||
|
|
||||||
|
| Skill | Owns |
|
||||||
|
|-------|------|
|
||||||
|
| [`cyclone-spec`](.superpowers/skills/cyclone-spec/SKILL.md) | The SP-N spec → plan → implement → merge flow. |
|
||||||
|
| [`cyclone-tests`](.superpowers/skills/cyclone-tests/SKILL.md) | pytest + vitest fixture patterns, prodfiles drop-in. |
|
||||||
|
| [`cyclone-edi`](.superpowers/skills/cyclone-edi/SKILL.md) | EDI parser/validator conventions (837P/835/999/270/271/277CA/TA1). |
|
||||||
|
| [`cyclone-tail`](.superpowers/skills/cyclone-tail/SKILL.md) | Live-tail streaming wire format and the hook triplet. |
|
||||||
|
| [`cyclone-store`](.superpowers/skills/cyclone-store/SKILL.md) | Store write-paths, pubsub event contract, SP21 split map. |
|
||||||
|
| [`cyclone-api-router`](.superpowers/skills/cyclone-api-router/SKILL.md) | FastAPI router conventions (`api_routers/`, `api_helpers.py`). |
|
||||||
|
| [`cyclone-frontend-page`](.superpowers/skills/cyclone-frontend-page/SKILL.md) | React page conventions (TanStack Query, drawer, URL state). |
|
||||||
|
| [`cyclone-cli`](.superpowers/skills/cyclone-cli/SKILL.md) | CLI subcommand conventions (`cli.py`, exit codes, smoke tests). |
|
||||||
|
|
||||||
|
Skills auto-load by description match — no slash command needed.
|
||||||
|
|
||||||
## Test
|
## Test
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -130,10 +176,13 @@ backoff schedule on error is `1s → 2s → 4s → 8s → 16s → 30s` capped.
|
|||||||
|
|
||||||
## Inbox
|
## Inbox
|
||||||
|
|
||||||
`/inbox` is the working surface. Four lanes, dark by default (Ticker Tape
|
`/inbox` is the working surface. Five lanes, dark by default (Ticker Tape
|
||||||
aesthetic):
|
aesthetic):
|
||||||
|
|
||||||
- **Rejected** — claims whose 999 set-level response was R or E. Re-submit in bulk.
|
- **Rejected** — claims whose 999 set-level response was R or E. Re-submit in bulk.
|
||||||
|
- **Payer-rejected** — claims whose 277CA STC category is A4, A6, or A7 (the payer
|
||||||
|
accepted the file but denied the claim). Stamped at 277CA ingest time and never
|
||||||
|
overwritten by a looser later 277CA.
|
||||||
- **Candidates** — remits whose CLP-claim-id didn't match exactly; each one shows its top scored claim. One-click manual match or dismiss.
|
- **Candidates** — remits whose CLP-claim-id didn't match exactly; each one shows its top scored claim. One-click manual match or dismiss.
|
||||||
- **Unmatched** — claims still waiting for a remit, and remits with no candidates above the threshold.
|
- **Unmatched** — claims still waiting for a remit, and remits with no candidates above the threshold.
|
||||||
- **Done today** — terminal state transitions in the last 24 hours.
|
- **Done today** — terminal state transitions in the last 24 hours.
|
||||||
@@ -148,7 +197,7 @@ parses and rejects claims, the inbox reflects the new
|
|||||||
|
|
||||||
| Method | Path | Notes |
|
| Method | Path | Notes |
|
||||||
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
||||||
| GET | `/api/inbox/lanes` | All four lanes in one call. |
|
| GET | `/api/inbox/lanes` | All five lanes in one call. |
|
||||||
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
|
| POST | `/api/inbox/candidates/{remit_id}/match` | Manual match. `409` if the claim state moved out from under us. |
|
||||||
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
|
| POST | `/api/inbox/candidates/dismiss` | `{pairs: [{claim_id, remit_id}]}`. Session-scoped. |
|
||||||
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
|
| POST | `/api/inbox/rejected/resubmit` | `{claim_ids: [...]}`. `200` with `conflicts` for non-rejected. |
|
||||||
@@ -255,11 +304,378 @@ drawer surfaces a per-line "no 837 line matched" note.
|
|||||||
Tiers: **strong** (≥75, full opacity, Match enabled), **weak**
|
Tiers: **strong** (≥75, full opacity, Match enabled), **weak**
|
||||||
(50–74, dimmed), **hidden** (<50, not surfaced).
|
(50–74, dimmed), **hidden** (<50, not surfaced).
|
||||||
|
|
||||||
|
## Multi-Payer, Multi-NPI & Clearhouse
|
||||||
|
|
||||||
|
The payer and provider identity that used to live as a single hard-coded
|
||||||
|
`PayerConfig` dict in the backend is now data, not code. Three new tables
|
||||||
|
plus a YAML file drive the entire configuration:
|
||||||
|
|
||||||
|
- **`providers` table** — one row per billing-provider NPI (Montrose
|
||||||
|
`1881068062`, Delta `1851446637`, Salida `1467507269`). All three share
|
||||||
|
the same `TOC, Inc.` legal name, tax ID `721587149`, and taxonomy
|
||||||
|
`251E00000X`. Outbound 837 files pick the right `BillingProvider` by
|
||||||
|
NPI; `claim.party.npi` is now a foreign key into `providers`.
|
||||||
|
- **`payers` table** — one row per payer (`CO_TXIX`, …) with its
|
||||||
|
receiver identity (NM1*40 / ISA08 / GS03).
|
||||||
|
- **`payer_configs` join table** — one row per `(payer_id, transaction_type)`
|
||||||
|
pair. 837P and 835 can carry different `BHT06`, SBR defaults, and
|
||||||
|
allowed status codes per payer.
|
||||||
|
- **`clearhouse` single-row config** — dzinesco's identity: TPID
|
||||||
|
`11525703`, submitter name, MT-clock file-naming block, SFTP block.
|
||||||
|
- **`config/payers.yaml`** — the on-disk source for everything above,
|
||||||
|
schema-validated at boot against a Pydantic model. A typo or missing
|
||||||
|
field fails the boot with a precise error. The original in-code
|
||||||
|
`PAYER_FACTORIES` dict is kept as a fallback for ad-hoc testing.
|
||||||
|
|
||||||
|
### Config + clearhouse endpoints
|
||||||
|
|
||||||
|
| Method | Path | Notes |
|
||||||
|
| ------ | --------------------------------------------- | ---------------------------------------------------------------- |
|
||||||
|
| GET | `/api/clearhouse` | The `clearhouse` singleton (name, TPID, file/SFTP blocks). |
|
||||||
|
| POST | `/api/clearhouse/submit` | Push a batch of generated 837 files via SFTP (see SFTP section). |
|
||||||
|
| GET | `/api/config/providers` | All providers. |
|
||||||
|
| GET | `/api/config/providers/{npi}` | One provider. |
|
||||||
|
| GET | `/api/config/payers` | All payers. |
|
||||||
|
| GET | `/api/config/payers/{payer_id}/configs` | All `(payer_id, transaction_type)` configs for one payer. |
|
||||||
|
| POST | `/api/admin/reload-config` | Re-read `config/payers.yaml` and refresh the in-process cache. |
|
||||||
|
|
||||||
|
## 277CA Claim Acknowledgment
|
||||||
|
|
||||||
|
A 277CA (`005010X214`) is the per-claim acknowledgment CMS and
|
||||||
|
Colorado Medicaid rely on: the file was syntactically valid *and* each
|
||||||
|
named claim was accepted, pended, or rejected by the payer at the claim
|
||||||
|
level. It is distinct from a 999 (file-level) and a TA1 (envelope-level).
|
||||||
|
|
||||||
|
Cyclone ingests 277CA files the same way it ingests 999 / 835 — drop the
|
||||||
|
file on the Upload page or `POST /api/parse-277ca` — and stamps every
|
||||||
|
claim whose `STC` category is `A4`, `A6`, or `A7` with a non-null
|
||||||
|
`payer_rejected_at` + `payer_rejected_reason` + originating 277CA row id.
|
||||||
|
|
||||||
|
| Method | Path | Notes |
|
||||||
|
| ------ | -------------------------- | ---------------------------------------------------------------- |
|
||||||
|
| POST | `/api/parse-277ca` | Upload a 277CA, persist the parsed status rows. |
|
||||||
|
| GET | `/api/277ca-acks` | List 277CA acks (filterable by date / payer). |
|
||||||
|
| GET | `/api/277ca-acks/{id}` | One 277CA ack with its per-claim status rows + regenerated text. |
|
||||||
|
|
||||||
|
The `payer_rejected` stamp is **monotonic**: a later 277CA with a looser
|
||||||
|
status set cannot clear a previous rejection. The Payer-Rejected inbox
|
||||||
|
lane surfaces every claim with a non-null `payer_rejected_at` — it is
|
||||||
|
distinct from the 999 `Rejected` lane (envelope reject) and they can
|
||||||
|
both be true for the same claim.
|
||||||
|
|
||||||
|
## Tamper-Evident Audit Log
|
||||||
|
|
||||||
|
The `audit_log` table is the canonical record of every state transition
|
||||||
|
the system has ever observed — claim lifecycle, reconciliation
|
||||||
|
decisions, config reloads, SFTP submissions, 277CA rejects. SP11 made
|
||||||
|
it tamper-evident: every row carries a SHA-256 hash of
|
||||||
|
`(prev_hash || row_payload)`, forming a chain back to a genesis row.
|
||||||
|
Any `INSERT`, `UPDATE`, or `DELETE` that breaks the chain is detectable
|
||||||
|
in a single walk.
|
||||||
|
|
||||||
|
| Method | Path | Notes |
|
||||||
|
| ------ | --------------------------------- | -------------------------------------------------------------- |
|
||||||
|
| GET | `/api/admin/audit-log` | Paginated audit log (filterable by event type / actor / date). |
|
||||||
|
| GET | `/api/admin/audit-log/verify` | Walk the chain; return the first broken link, or `{ok: true}`. |
|
||||||
|
|
||||||
|
`verify_chain` is the integrity check that backs the audit promise —
|
||||||
|
it is intentionally cheap (one indexed walk) and intentionally
|
||||||
|
side-effect-free so a scheduler can run it on a cron and alert on any
|
||||||
|
non-`{ok: true}` result. Chain verification is **not** access-gated
|
||||||
|
beyond the same `127.0.0.1` bind the rest of the API uses; for a
|
||||||
|
hostile multi-operator deployment, wrap the route in your reverse proxy.
|
||||||
|
|
||||||
|
## Encryption at Rest
|
||||||
|
|
||||||
|
When the macOS Keychain carries an entry at service `cyclone`, account
|
||||||
|
`cyclone.db.key`, and the optional `sqlcipher3` Python package is
|
||||||
|
installed, the SQLite file at `~/.local/share/cyclone/cyclone.db` is
|
||||||
|
opened with SQLCipher (AES-256). The key is read from the Keychain
|
||||||
|
once at process start, applied via a SQLAlchemy `connect` event so
|
||||||
|
every connection — including migrations and tests — gets the same
|
||||||
|
`PRAGMA key`. The key is never written to disk or to a Python global.
|
||||||
|
|
||||||
|
When the Keychain entry is missing **or** `sqlcipher3` is not
|
||||||
|
installed, the DB falls back to plain SQLite. The intent is a graceful
|
||||||
|
default for developers and CI; the production posture is that every
|
||||||
|
operator has created the Keychain entry on first run. See
|
||||||
|
[docs/reference/co-medicaid.md §Keychain setup](docs/reference/co-medicaid.md)
|
||||||
|
for the one-time setup recipe and the HIPAA Security Rule §164.312(a)(2)(iv)
|
||||||
|
mapping.
|
||||||
|
|
||||||
|
### Key rotation (SP15)
|
||||||
|
|
||||||
|
`POST /api/admin/db/rotate-key` re-encrypts the SQLite file in place
|
||||||
|
with a fresh SQLCipher key via `PRAGMA rekey`, then updates the
|
||||||
|
Keychain so subsequent connections open with the new key. The
|
||||||
|
rotation holds a module-level `threading.Lock` (so two concurrent
|
||||||
|
requests can't race), disposes + rebuilds the SQLAlchemy engine with
|
||||||
|
`NullPool` (so SQLCipher's thread affinity is honored), and writes a
|
||||||
|
tamper-evident `db.key_rotated` audit event with old + new
|
||||||
|
fingerprints and the post-rotation table count. The old key is
|
||||||
|
retained in the `cyclone.db.key.previous` Keychain account for a
|
||||||
|
grace period so a botched rotation can be rolled back by hand.
|
||||||
|
|
||||||
|
## NPI checksum + Tax ID format validation (SP20)
|
||||||
|
|
||||||
|
Two pure local validators — no NPPES round-trip, no IRS e-file
|
||||||
|
lookup. Catches the 99% case (a typo at the end of an NPI, a letter in
|
||||||
|
an EIN, an extra digit, the reserved `00`/`07`/`8X` EIN prefix).
|
||||||
|
|
||||||
|
| Check | Algorithm | Surface |
|
||||||
|
|-------|-----------|---------|
|
||||||
|
| NPI | 10 digits where the last is a Luhn checksum over `80840 + body`. CMS-published example: body `123456789` → check `3` → valid NPI `1234567893`. | `cyclone.npi.is_valid_npi`, CLI `cyclone validate-npi <npi>`, API `GET /api/admin/validate-provider?npi=...`, validator rule `R021_npi_checksum` (warning) |
|
||||||
|
| Tax ID (EIN) | 9 digits, optional `XX-XXXXXXX` formatting. Rejects reserved prefixes `00`, `07`, `80`–`89` (IRS Pension Plan Branch). | `cyclone.npi.is_valid_tax_id`, CLI `cyclone validate-tax-id <ein>`, API `GET /api/admin/validate-provider?tax_id=...` |
|
||||||
|
|
||||||
|
### CLI
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ cyclone validate-npi 1234567893
|
||||||
|
OK: 10-digit NPI passes Luhn checksum
|
||||||
|
$ cyclone validate-npi 1234567890
|
||||||
|
INVALID: '1234567890' fails NPI Luhn checksum # exit 1
|
||||||
|
|
||||||
|
$ cyclone validate-tax-id 72-1587149
|
||||||
|
OK: 9-digit EIN (normalized=721587149)
|
||||||
|
$ cyclone validate-tax-id 00-1234567
|
||||||
|
INVALID: 9-digit EIN has reserved prefix (00); EIN is not assignable by IRS # exit 1
|
||||||
|
```
|
||||||
|
|
||||||
|
### API
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'http://localhost:8000/api/admin/validate-provider?npi=1234567893&tax_id=72-1587149'
|
||||||
|
# {
|
||||||
|
# "npi": {"valid": true, "skipped": false},
|
||||||
|
# "tax_id": {"valid": true, "skipped": false, "normalized": "721587149"}
|
||||||
|
# }
|
||||||
|
```
|
||||||
|
|
||||||
|
Both query params are optional; omitted fields return
|
||||||
|
`{"valid": null, "skipped": true}` so the caller can render "no check
|
||||||
|
performed" rather than treating absent input as a hard fail.
|
||||||
|
|
||||||
|
### Parser integration
|
||||||
|
|
||||||
|
The `R021_npi_checksum` rule runs alongside the existing `R020_npi_format`
|
||||||
|
in `cyclone.parsers.validator`. A billing-provider NPI that passes
|
||||||
|
R020 (right shape) but fails R021 (bad Luhn) is yielded as a
|
||||||
|
**warning**, not an error — operators sometimes ingest test fixtures
|
||||||
|
with placeholder NPIs (e.g. all-same-digit) and we don't want to block
|
||||||
|
that path. In strict mode (`--strict` / `?strict=true`) warnings are
|
||||||
|
promoted to errors.
|
||||||
|
|
||||||
|
### Files
|
||||||
|
|
||||||
|
* `cyclone/npi.py` — new module (~155 LOC).
|
||||||
|
* `cyclone.parsers.validator` — new `R021_npi_checksum` rule.
|
||||||
|
* `cyclone.api_routers.admin` — new `validate-provider` endpoint.
|
||||||
|
* `cyclone.cli` — `validate-npi` + `validate-tax-id` subcommands.
|
||||||
|
* Tests: `test_npi.py` (27), `test_api_validate_provider.py` (4),
|
||||||
|
`test_cli_validate.py` (8), `test_validator.py::test_r021_*` (4) —
|
||||||
|
**43 new tests**.
|
||||||
|
|
||||||
|
## Security hardening (SP19)
|
||||||
|
|
||||||
|
Three pure-ASGI middlewares sit in front of every FastAPI request.
|
||||||
|
They're sized for Cyclone's local-only posture — a misconfigured
|
||||||
|
Tailscale / ngrok bind, a buggy cron job uploading a 4 GB file, or a
|
||||||
|
port-scraper — not for hostile internet exposure.
|
||||||
|
|
||||||
|
| Middleware | Default | Override | Reject |
|
||||||
|
|------------|---------|----------|--------|
|
||||||
|
| `BodySizeLimitMiddleware` | 50 MB | `CYCLONE_MAX_BODY_BYTES` | `413 body_too_large` over Content-Length cap; chunked reads capped too |
|
||||||
|
| `RateLimitMiddleware` | 300 req/min/IP | `CYCLONE_RATE_LIMIT_PER_MIN` | `429 rate_limited` over the sliding window; `/api/health` exempt |
|
||||||
|
| `SecurityHeadersMiddleware` | always on | n/a | stamps `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: same-origin`, `Permissions-Policy`, `Content-Security-Policy: default-src 'none'; frame-ancestors 'none'` |
|
||||||
|
|
||||||
|
Every rejection (413 / 429) also writes a tamper-evident
|
||||||
|
`api.request_rejected` event into the SP11 audit chain so an
|
||||||
|
operator can correlate a misbehaving client with the SP18 JSON logs:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"event_type":"api.request_rejected","entity_id":"POST /api/parse-837","payload":{"status":413,"reason":"body_too_large","path":"/api/parse-837","method":"POST","ip":"127.0.0.1"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Health probe
|
||||||
|
|
||||||
|
`GET /api/health` now returns a subsystem snapshot:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"db": {"ok": true},
|
||||||
|
"scheduler": {"running": true, "interval_s": 60, "sftp_block": "co_medicaid",
|
||||||
|
"backup_scheduler_running": false, "backup_interval_hours": 24.0},
|
||||||
|
"pubsub": {"parse_completed": 1, "batch_added": 1},
|
||||||
|
"batch": {"last_batch_id": 42, "last_batch_kind": "837P",
|
||||||
|
"last_batch_at": "2026-06-21T15:30:00.123Z",
|
||||||
|
"last_batch_filename": "TP11525703-837P-..."}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Returns `"status": "degraded"` if any subsystem reports an error —
|
||||||
|
the per-subsystem dict still surfaces so an operator can see which
|
||||||
|
one is unhappy. `/api/health` is rate-limit exempt so a load balancer
|
||||||
|
hammering the endpoint doesn't trip the limiter.
|
||||||
|
|
||||||
|
### Files
|
||||||
|
|
||||||
|
* `cyclone.security` — `BodySizeLimitMiddleware`,
|
||||||
|
`RateLimitMiddleware`, `SecurityHeadersMiddleware`, and
|
||||||
|
`get_health_snapshot()` (~330 LOC).
|
||||||
|
* `cyclone.api_routers.health` rewritten to use `get_health_snapshot()`.
|
||||||
|
* `cyclone.pubsub.EventBus.stats()` — new method that returns
|
||||||
|
per-kind subscriber counts.
|
||||||
|
* `tests/test_security.py` — 13 new tests.
|
||||||
|
|
||||||
|
## Structured logging (SP18)
|
||||||
|
|
||||||
|
Cyclone emits newline-delimited JSON to stderr by default — readable
|
||||||
|
by `jq`, Loki, Vector, ELK, or any log shipper. Every record carries
|
||||||
|
`ts` (ISO 8601 ms UTC), `level`, `logger`, `msg`, and an optional
|
||||||
|
`extra` dict for structured fields. Exceptions render as a
|
||||||
|
`traceback` string.
|
||||||
|
|
||||||
|
```
|
||||||
|
{"ts":"2026-06-21T15:30:00.123Z","level":"INFO","logger":"cyclone.scheduler","msg":"Processed inbound file","extra":{"input_filename":"ACK_999.x12","parser":"parse_999","claims":3}}
|
||||||
|
{"ts":"2026-06-21T15:30:01.456Z","level":"ERROR","logger":"cyclone.api","msg":"Backup create failed","extra":{"reason":"BackupError: passphrase mismatch"},"traceback":"Traceback ..."}
|
||||||
|
```
|
||||||
|
|
||||||
|
### PII scrubbing
|
||||||
|
|
||||||
|
A `PiiScrubber` filter is attached to the root logger and rewrites
|
||||||
|
obvious PHI patterns to `<redacted:npi>` / `<redacted:ssn>` /
|
||||||
|
`<redacted:dob>` / `<redacted:patient_name>` before any handler sees
|
||||||
|
the record:
|
||||||
|
|
||||||
|
| Pattern | Replacement |
|
||||||
|
|---------|-------------|
|
||||||
|
| `\b\d{10}\b` | `<redacted:npi>` |
|
||||||
|
| `\b\d{3}-\d{2}-\d{4}\b` or `\b\d{9}\b` at phrase boundary | `<redacted:ssn>` |
|
||||||
|
| `(dob\|date_of_birth)[:=]\s*\d{4}-\d{2}-\d{2}` | preserves the key, redacts the date |
|
||||||
|
| `patient_name=...` | full chunk redacted |
|
||||||
|
| Extras with key `dob`/`ssn`/`npi`/`patient_name`/… | value redacted regardless of shape |
|
||||||
|
|
||||||
|
The scrubber is conservative — bare ISO dates without a `dob=` prefix
|
||||||
|
are **not** scrubbed (they're too often timestamps or batch IDs), and
|
||||||
|
11+ digit numbers are left alone (they can't be NPIs). Disable for
|
||||||
|
forensic mode with `CYCLONE_LOG_NO_PII_SCRUB=1`.
|
||||||
|
|
||||||
|
### Knobs
|
||||||
|
|
||||||
|
| Env var | Default | Meaning |
|
||||||
|
|---------|---------|---------|
|
||||||
|
| `CYCLONE_LOG_LEVEL` | `INFO` | Root logger level. `DEBUG` for troubleshooting. |
|
||||||
|
| `CYCLONE_LOG_FILE` | (none) | Write to this path via `RotatingFileHandler` (10 MB × 5 backups). |
|
||||||
|
| `CYCLONE_LOG_JSON` | `true` | `false` uses the dev tabular formatter. |
|
||||||
|
| `CYCLONE_LOG_NO_PII_SCRUB` | (none) | `1` disables scrubbing. |
|
||||||
|
|
||||||
|
CLI:
|
||||||
|
|
||||||
|
```
|
||||||
|
cyclone --log-format=dev parse-837 sample.x12 --output-dir out/ # tabular for tail -f
|
||||||
|
cyclone --log-file=/var/log/cyclone.log backup create # JSON to rotating file
|
||||||
|
```
|
||||||
|
|
||||||
|
The `parse-837` / `parse-835` subcommands also accept `--log-level`
|
||||||
|
which re-runs `setup_logging()` so the per-invocation level overrides
|
||||||
|
the group default.
|
||||||
|
|
||||||
|
### Files
|
||||||
|
|
||||||
|
* `cyclone.logging_config` — `JsonFormatter`, `CycloneDevFormatter`,
|
||||||
|
`PiiScrubber`, `setup_logging()`.
|
||||||
|
* `tests/test_logging_formatter.py` (11), `test_logging_scrubber.py`
|
||||||
|
(13), `test_logging_setup.py` (10) — 34 new tests.
|
||||||
|
* `cyclone.api` lifespan calls `setup_logging()` first; the CLI
|
||||||
|
`main` group does the same.
|
||||||
|
|
||||||
|
## Encrypted Backups (SP17)
|
||||||
|
|
||||||
|
The BackupService takes an online consistent snapshot of the live
|
||||||
|
SQLite file via SQLite's `.backup()` API, encrypts the bytes with
|
||||||
|
AES-256-GCM, and writes a `.bin` + `.meta.json` pair into the backup
|
||||||
|
directory (default `~/.local/share/cyclone/backups/`). The encryption
|
||||||
|
key is derived from a separate passphrase in the macOS Keychain
|
||||||
|
(PBKDF2-HMAC-SHA256, 200,000 iterations, 16-byte salt persisted to
|
||||||
|
Keychain) — so a SQLCipher DB-key compromise does not unlock the
|
||||||
|
backups, and a backup-passphrase compromise does not unlock the live
|
||||||
|
DB. If neither is set, the service refuses (`BackupError`) rather than
|
||||||
|
silently writing plaintext.
|
||||||
|
|
||||||
|
| Method | Path | Purpose |
|
||||||
|
| ------ | ---- | ------- |
|
||||||
|
| POST | `/api/admin/backup/create` | Take an encrypted backup now. |
|
||||||
|
| GET | `/api/admin/backup/list` | List `db_backups` rows (newest first, filterable). |
|
||||||
|
| GET | `/api/admin/backup/status` | Counts, disk usage, last-run timestamp, scheduler snapshot. |
|
||||||
|
| POST | `/api/admin/backup/{id}/verify` | Decrypt + SHA-256 verify against the sidecar. |
|
||||||
|
| POST | `/api/admin/backup/{id}/restore/initiate` | First step: get `restore_token` + preview (fingerprints of backup vs live). |
|
||||||
|
| POST | `/api/admin/backup/{id}/restore/confirm` | Second step: dispose engine, copy decrypted DB, rebuild engine. |
|
||||||
|
| POST | `/api/admin/backup/prune` | Apply retention policy now. |
|
||||||
|
| POST | `/api/admin/backup/scheduler/{start,stop,tick}` | Operate the backup scheduler. |
|
||||||
|
|
||||||
|
Restore is two-step by design: an idle browser tab can't nuke the
|
||||||
|
live DB. The first call returns a one-shot 64-char hex
|
||||||
|
`restore_token` plus a side-by-side preview (`backup_db_fingerprint`,
|
||||||
|
`backup_table_count`, `current_db_fingerprint`, `current_table_count`).
|
||||||
|
The second call swaps the live engine only if the token matches
|
||||||
|
within a 5-minute TTL.
|
||||||
|
|
||||||
|
The scheduler (auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`)
|
||||||
|
ticks every `CYCLONE_BACKUP_INTERVAL_HOURS` (default 24), runs
|
||||||
|
`create_now` + `prune`, and writes audit events for each outcome
|
||||||
|
(`db.backup_created`, `db.backup_failed`, `db.backup_pruned`,
|
||||||
|
`db.backup_restored`). The CLI mirrors the API surface:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cyclone backup init-passphrase # one-time; interactive
|
||||||
|
cyclone backup create
|
||||||
|
cyclone backup list
|
||||||
|
cyclone backup verify <id>
|
||||||
|
cyclone backup restore <id> --yes
|
||||||
|
cyclone backup prune --yes
|
||||||
|
cyclone backup status
|
||||||
|
```
|
||||||
|
|
||||||
|
Retention defaults to 30 days (`CYCLONE_BACKUP_RETENTION_DAYS`). The
|
||||||
|
retention policy is best-effort: an operator who runs `cyclone
|
||||||
|
backup create` manually retains full control.
|
||||||
|
|
||||||
|
## SFTP Wire-Up (paramiko)
|
||||||
|
|
||||||
|
The `clearhouse.submit` endpoint uses `paramiko` to push a batch of
|
||||||
|
generated 837 files to the dzinesco SFTP server
|
||||||
|
(`mft.gainwelltechnologies.com:22`, path
|
||||||
|
`/CO XIX/PROD/coxix_prod_11525703/FromHPE`). The SFTP credential is
|
||||||
|
fetched from the macOS Keychain at call time — never read from YAML,
|
||||||
|
never logged, never written to disk. The wire-up honors the file-naming
|
||||||
|
template stored in the `clearhouse` config:
|
||||||
|
|
||||||
|
```
|
||||||
|
outbound: {tpid}-{tx}-{ts_mt}-1of1.{ext} e.g. 11525703-837P-20260620181814559-1of1.txt
|
||||||
|
inbound: TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
|
||||||
|
```
|
||||||
|
|
||||||
|
where `{ts_mt}` is a 17-digit `yyyymmddhhmmssSSS` Mountain Time stamp.
|
||||||
|
Inbound filenames are routed by `<FileType>` and `<OrigTx>` to the
|
||||||
|
matching parser (`999`, `TA1`, `271`, `277`, `277CA`, `835`).
|
||||||
|
|
||||||
|
The `SftpClient` interface is the same one the SP9 stub used — the swap
|
||||||
|
was a one-file change (`sftp_paramiko.py` replacing `sftp_stub.py`).
|
||||||
|
`paramiko` is an optional dependency; the stub remains the default when
|
||||||
|
the `paramiko` extras aren't installed so the test suite stays green on
|
||||||
|
Linux dev boxes.
|
||||||
|
|
||||||
## Persistence
|
## Persistence
|
||||||
|
|
||||||
Parsed batches, claims, remittances, matches, and activity events are
|
Parsed batches, claims, remittances, matches, 277CA rejections,
|
||||||
stored in a SQLite file at `~/.local/share/cyclone/cyclone.db` by
|
hash-chained audit log entries, and SFTP submission history are stored
|
||||||
default. The directory is auto-created on first run.
|
in a SQLite file at `~/.local/share/cyclone/cyclone.db` by default.
|
||||||
|
The directory is auto-created on first run. The DB is optionally
|
||||||
|
encrypted with SQLCipher — see
|
||||||
|
[Encryption at Rest](#encryption-at-rest) for the Keychain-driven
|
||||||
|
setup.
|
||||||
|
|
||||||
To use a different location, set `CYCLONE_DB_URL`:
|
To use a different location, set `CYCLONE_DB_URL`:
|
||||||
|
|
||||||
@@ -285,11 +701,23 @@ backup API).
|
|||||||
├── backend/
|
├── backend/
|
||||||
│ ├── src/cyclone/
|
│ ├── src/cyclone/
|
||||||
│ │ ├── api.py # FastAPI app, GET + parse routes, /api/{resource}/stream
|
│ │ ├── api.py # FastAPI app, GET + parse routes, /api/{resource}/stream
|
||||||
|
│ │ ├── api_helpers.py # NDJSON / content-negotiation / live-tail helpers
|
||||||
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
|
│ │ ├── pubsub.py # in-process EventBus (drop-oldest, per-kind fan-out)
|
||||||
│ │ ├── store.py # InMemoryStore, mappers, publish-on-write
|
│ │ ├── store.py # CycloneStore, mappers, publish-on-write
|
||||||
|
│ │ ├── db.py # SQLAlchemy engine, session factory, ORM models
|
||||||
|
│ │ ├── db_migrate.py # PRAGMA user_version migration runner
|
||||||
|
│ │ ├── db_crypto.py # optional SQLCipher encryption at rest (SP12)
|
||||||
|
│ │ ├── audit_log.py # tamper-evident hash-chained audit_log (SP11)
|
||||||
|
│ │ ├── inbox_lanes.py # rejected / payer_rejected / candidates / unmatched / done_today
|
||||||
|
│ │ ├── inbox_state.py # 999 envelope reject → claim state transitions
|
||||||
|
│ │ ├── inbox_state_277ca.py # 277CA STC A4/A6/A7 → payer_rejected stamp (SP10)
|
||||||
|
│ │ ├── providers.py # multi-NPI provider lookups (SP9)
|
||||||
|
│ │ ├── payers.py # payer / payer_config lookups (SP9)
|
||||||
|
│ │ ├── secrets.py # macOS Keychain-backed secret fetcher
|
||||||
|
│ │ ├── reconcile.py # pure-function 835→claim match + line-level match
|
||||||
│ │ ├── __main__.py # `python -m cyclone serve`
|
│ │ ├── __main__.py # `python -m cyclone serve`
|
||||||
│ │ ├── cli.py # click CLI
|
│ │ ├── cli.py # click CLI
|
||||||
│ │ └── parsers/ # X12 tokenizer, models, validator, writers
|
│ │ └── parsers/ # X12 tokenizer, models, validator, writers, 277CA, 999, TA1, 270, 271
|
||||||
│ └── tests/
|
│ └── tests/
|
||||||
│ ├── fixtures/ # co_medicaid_*.txt, minimal_*.txt
|
│ ├── fixtures/ # co_medicaid_*.txt, minimal_*.txt
|
||||||
│ ├── test_api.py # parse-837/835 round-trip
|
│ ├── test_api.py # parse-837/835 round-trip
|
||||||
@@ -298,29 +726,148 @@ backup API).
|
|||||||
│ ├── test_api_streaming.py
|
│ ├── test_api_streaming.py
|
||||||
│ ├── test_api_stream_live.py # 3 live-tail endpoints + disconnect cleanup
|
│ ├── test_api_stream_live.py # 3 live-tail endpoints + disconnect cleanup
|
||||||
│ ├── test_pubsub.py # EventBus + subscribe/unsubscribe
|
│ ├── test_pubsub.py # EventBus + subscribe/unsubscribe
|
||||||
│ └── test_api_parse_persists.py
|
│ ├── test_api_parse_persists.py
|
||||||
|
│ ├── test_db.py / test_db_crypto.py / test_db_migrate.py
|
||||||
|
│ ├── test_audit_log.py
|
||||||
|
│ ├── test_inbox_lanes.py / test_inbox_state.py
|
||||||
|
│ ├── test_apply_277ca_rejections.py
|
||||||
|
│ ├── test_sftp_stub.py / test_sftp_paramiko.py
|
||||||
|
│ └── test_providers_seed.py / test_payer_config_loading.py
|
||||||
├── src/ # React + Vite + TypeScript UI
|
├── src/ # React + Vite + TypeScript UI
|
||||||
│ ├── components/
|
│ ├── components/
|
||||||
│ │ ├── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, …
|
│ │ ├── ui/ # Skeleton, EmptyState, ErrorState, FilterChips, Pagination, …
|
||||||
│ │ └── TailStatusPill.tsx # live-tail status badge + reconnect button
|
│ │ └── TailStatusPill.tsx # live-tail status badge + reconnect button
|
||||||
│ ├── pages/ # Claims, Remittances, Providers, Activity, Upload
|
│ ├── pages/ # Claims, Remittances, Providers, Acks, Activity, Upload, Inbox, …
|
||||||
│ ├── hooks/ # useBatches, useClaims, useRemittances, useProviders, useActivity, useParse
|
│ ├── hooks/ # useBatches, useClaims, useRemittances, useProviders, useActivity, useParse
|
||||||
│ │ # + useTailStream, useMergedTail (live tail)
|
│ │ # + useTailStream, useMergedTail (live tail)
|
||||||
│ ├── lib/ # api.ts (6 GET + parse837/parse835/health), format.ts, utils.ts
|
│ ├── lib/ # api.ts, format.ts, utils.ts
|
||||||
│ │ # + tail-stream.ts (NDJSON parser)
|
│ │ # + tail-stream.ts (NDJSON parser)
|
||||||
│ ├── store/ # zustand sample-data + parsed-batches store
|
│ ├── store/ # zustand sample-data + parsed-batches store
|
||||||
│ │ # + tail-store.ts (FIFO-capped live tail slices)
|
│ │ # + tail-store.ts (FIFO-capped live tail slices)
|
||||||
│ └── types/ # shared TS types
|
│ └── types/ # shared TS types
|
||||||
|
├── config/
|
||||||
|
│ └── payers.yaml # YAML-driven payer + clearhouse config (SP9)
|
||||||
├── docs/
|
├── docs/
|
||||||
│ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes
|
│ ├── reference/ # condensed 837P/835/X12/CO Medicaid notes (incl. Keychain setup)
|
||||||
│ └── superpowers/plans/ # implementation plan
|
│ ├── reviews/ # post-SP completeness reviews
|
||||||
|
│ ├── superpowers/plans/ # implementation plans
|
||||||
|
│ └── superpowers/specs/ # design specs (incl. SP9-SP13)
|
||||||
├── tailwind.config.js # shimmer, scan, row-flash keyframes
|
├── tailwind.config.js # shimmer, scan, row-flash keyframes
|
||||||
└── package.json
|
└── package.json
|
||||||
```
|
```
|
||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
Sub-projects 2 through 8 are **shipped**. Next up:
|
Sub-projects 2 through 19 are **shipped**. See the [completeness
|
||||||
|
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
|
||||||
|
the honest gap analysis against the industry definition of a HIPAA
|
||||||
|
clearinghouse — the short version is that the local-only,
|
||||||
|
single-operator, single-payer design contract is honored, and the
|
||||||
|
items that would be needed to expand that contract (AS2/AS4, SNIP 1–7,
|
||||||
|
HITRUST, 276/277 status, 278 referrals, COB) are intentionally out of
|
||||||
|
scope.
|
||||||
|
|
||||||
|
Shipped sub-projects (most recent first):
|
||||||
|
|
||||||
|
- **Sub-project 22 (shipped) — Pipeline automation agent.** A
|
||||||
|
sibling project at `/Users/openclaw/dev/cyclone-pipeline` that
|
||||||
|
drives the full 7-phase round-trip (preflight → browser upload →
|
||||||
|
parse verify → SFTP submit → TA1 wait → 999 wait → scan + report)
|
||||||
|
with crash-safe resume, structured JSON logging, idempotency
|
||||||
|
dedup, and a per-run report. Pure Python 3.11+ (httpx, Playwright,
|
||||||
|
Click, pydantic v2, structlog). The 835 is not waited for inline —
|
||||||
|
it lands the following Monday — and is verified by a separate
|
||||||
|
`check-835` subcommand. Embeddable as a library for OpenClaw / Nora
|
||||||
|
agent integration. See
|
||||||
|
[Pipeline automation agent](#pipeline-automation-agent) above.
|
||||||
|
- **Sub-project 19 (shipped) — Security hardening + health probe.**
|
||||||
|
Three pure-ASGI middlewares (`BodySizeLimitMiddleware`,
|
||||||
|
`RateLimitMiddleware`, `SecurityHeadersMiddleware`) close the
|
||||||
|
completeness-review gaps §3.1.4 (no body/rate limits) and §3.1.25
|
||||||
|
(no CSP / security headers). 413/429 rejections emit a
|
||||||
|
tamper-evident `api.request_rejected` audit event (SP11 chain).
|
||||||
|
`/api/health` is now a rich subsystem snapshot — DB connectivity,
|
||||||
|
MFT scheduler state, backup scheduler state, live pubsub
|
||||||
|
subscriber counts, last batch id + timestamp. See
|
||||||
|
[Security hardening (SP19)](#security-hardening-sp19) below.
|
||||||
|
- **Sub-project 20 (shipped) — NPI checksum + Tax ID format validation.**
|
||||||
|
Pure local validators (`cyclone.npi`) — no NPPES round-trip, no IRS
|
||||||
|
e-file lookup. Catches the 99% typo case at parse time. NPI uses
|
||||||
|
CMS-published Luhn over `80840 + body` (example: `1234567893` is
|
||||||
|
valid). EIN rejects reserved prefixes (`00`, `07`, `80`–`89`).
|
||||||
|
Surface: `cyclone validate-npi` / `validate-tax-id` CLI subcommands,
|
||||||
|
`GET /api/admin/validate-provider`, new `R021_npi_checksum`
|
||||||
|
validator rule (warning, not error — placeholder NPIs in test
|
||||||
|
fixtures shouldn't block ingest). See
|
||||||
|
[NPI checksum + Tax ID format validation (SP20)](#npi-checksum--tax-id-format-validation-sp20)
|
||||||
|
below.
|
||||||
|
- **Sub-project 18 (shipped) — Structured JSON logging.** All logs
|
||||||
|
emitted by the API, CLI, scheduler tick loop, and backup service
|
||||||
|
flow through a `JsonFormatter` (newline-delimited JSON, ISO-8601 ms
|
||||||
|
timestamps) by default. A `PiiScrubber` filter redacts obvious PHI
|
||||||
|
(NPIs, SSNs, DOBs, patient names) from message + extras — both via
|
||||||
|
inline patterns (`npi 1881068062`) and via PHI-keyed extras
|
||||||
|
(`extra={"dob": "1980-04-12"}`). Configurable via env vars
|
||||||
|
(`CYCLONE_LOG_LEVEL`, `CYCLONE_LOG_FILE`, `CYCLONE_LOG_JSON`,
|
||||||
|
`CYCLONE_LOG_NO_PII_SCRUB`) and CLI flags
|
||||||
|
(`--log-format=json|dev`, `--log-file=…`); a tabular `CycloneDevFormatter`
|
||||||
|
is the opt-out for `tail -f` in dev. See
|
||||||
|
[Structured logging](#structured-logging-sp18) below.
|
||||||
|
- **Sub-project 17 (shipped) — Encrypted DB backups.** Automated
|
||||||
|
encrypted backups via AES-256-GCM (PBKDF2-HMAC-SHA256, 200k iters).
|
||||||
|
The operator sets a separate passphrase in the macOS Keychain
|
||||||
|
(`cyclone backup init-passphrase`); if missing, the service falls
|
||||||
|
back to deriving from the SQLCipher DB key with a WARNING. Online
|
||||||
|
backups via SQLite `.backup()`, two-step restore (`initiate` →
|
||||||
|
`confirm` with one-shot 64-char hex token), retention pruning with
|
||||||
|
a 30-day default, and a tamper-evident audit chain (`db.backup_created`,
|
||||||
|
`db.backup_failed`, `db.backup_pruned`, `db.backup_restored`,
|
||||||
|
`db.backup_passphrase_set`). Backup scheduler ticks every 24h
|
||||||
|
(configurable); auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`.
|
||||||
|
Seven admin endpoints + six CLI subcommands. See
|
||||||
|
[Encrypted Backups](#encrypted-backups) below.
|
||||||
|
- **Sub-project 16 (shipped) — Live MFT polling scheduler.** asyncio
|
||||||
|
background loop polls the Gainwell MFT inbound path, downloads
|
||||||
|
new files, and routes them through the right parser (999 / 835 /
|
||||||
|
277CA / TA1). Idempotent (re-ticks skip already-processed files
|
||||||
|
via the new `processed_inbound_files` table). Crash-safe (per-file
|
||||||
|
try/except so a bad file doesn't stop the loop). Five admin
|
||||||
|
endpoints (`/api/admin/scheduler/{status,start,stop,tick,processed-files}`).
|
||||||
|
- **Sub-project 15 (shipped) — SQLCipher key rotation.** In-place
|
||||||
|
rotation via `PRAGMA rekey`, serialized through a module-level
|
||||||
|
`threading.Lock` and a SQLAlchemy `NullPool` to keep SQLCipher
|
||||||
|
thread-affine under FastAPI's per-request threadpool. Writes a
|
||||||
|
`db.key_rotated` audit event with old + new key fingerprints and
|
||||||
|
post-rotation `table_count`. See
|
||||||
|
[Encryption at Rest — Key rotation](#key-rotation).
|
||||||
|
- **Sub-project 14 (shipped) — 5-lane Inbox UI.** The Payer-Rejected
|
||||||
|
lane is now rendered in the Inbox alongside Rejected / Candidates /
|
||||||
|
Unmatched / Done today. New bulk action
|
||||||
|
`POST /api/inbox/payer-rejected/acknowledge` drops claims from the
|
||||||
|
working surface without erasing the original 277CA rejection event
|
||||||
|
(audit log stays intact, SP11).
|
||||||
|
- **Sub-project 13 (shipped) — SFTP wire-up.** `paramiko`-backed
|
||||||
|
`SftpClient` replaces the SP9 stub. The clearhouse.submit endpoint
|
||||||
|
actually pushes to
|
||||||
|
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
|
||||||
|
SFTP credentials are read from the macOS Keychain at call time.
|
||||||
|
- **Sub-project 12 (shipped) — Encryption at rest.** Optional
|
||||||
|
SQLCipher AES-256 encryption of the SQLite file, with the key
|
||||||
|
fetched from the macOS Keychain. Falls back to plain SQLite when
|
||||||
|
the Keychain entry is missing or `sqlcipher3` isn't installed.
|
||||||
|
- **Sub-project 11 (shipped) — Tamper-evident audit log.** Every
|
||||||
|
`audit_log` row carries a SHA-256 hash chained to the previous row;
|
||||||
|
a single walk via `GET /api/admin/audit-log/verify` detects any
|
||||||
|
break.
|
||||||
|
- **Sub-project 10 (shipped) — 277CA + Payer-Rejected lane.** Inbound
|
||||||
|
277CA parser + a new Payer-Rejected inbox lane distinct from the
|
||||||
|
999-envelope Rejected lane. The rejection stamp is monotonic.
|
||||||
|
- **Sub-project 9 (shipped) — Multi-payer, multi-NPI, SFTP stub.** The
|
||||||
|
in-code `PAYER_FACTORIES` dict is replaced by a `config/payers.yaml`
|
||||||
|
+ 3 new DB tables (`providers`, `payers`, `payer_configs`) +
|
||||||
|
`clearhouse` singleton. Added a `POST /api/clearhouse/submit` stub
|
||||||
|
that writes to a local `staging_dir` — swapped for real `paramiko`
|
||||||
|
in SP13.
|
||||||
|
|
||||||
- **Sub-project 8 (shipped) — Outbound 837P serializer.** Closes the
|
- **Sub-project 8 (shipped) — Outbound 837P serializer.** Closes the
|
||||||
resubmit loop: rejected claims can be regenerated back to an X12 837P
|
resubmit loop: rejected claims can be regenerated back to an X12 837P
|
||||||
@@ -390,8 +937,9 @@ Sub-projects 2 through 8 are **shipped**. Next up:
|
|||||||
back-off ladder on errors is `1s → 2s → 4s → 8s → 16s → 30s`
|
back-off ladder on errors is `1s → 2s → 4s → 8s → 16s → 30s`
|
||||||
capped. See the "Live updates" section below for details.
|
capped. See the "Live updates" section below for details.
|
||||||
- **Sub-project 6 (shipped) — Inbox workflow automation.**
|
- **Sub-project 6 (shipped) — Inbox workflow automation.**
|
||||||
- **Ticker-Tape inbox (`/inbox`):** four lanes ordered by urgency —
|
- **Ticker-Tape inbox (`/inbox`):** five lanes ordered by urgency —
|
||||||
**Rejected** (claims whose 999 rejected them), **Candidates**
|
**Rejected** (claims whose 999 rejected them), **Payer-rejected**
|
||||||
|
(claims whose 277CA denied them — added in SP10), **Candidates**
|
||||||
(remits that didn't auto-match a claim), **Unmatched** (claims
|
(remits that didn't auto-match a claim), **Unmatched** (claims
|
||||||
still waiting for a remit), and **Done today** (terminal
|
still waiting for a remit), and **Done today** (terminal
|
||||||
transitions in the last 24 hours). The page subscribes to the
|
transitions in the last 24 hours). The page subscribes to the
|
||||||
@@ -428,6 +976,10 @@ Sub-projects 2 through 8 are **shipped**. Next up:
|
|||||||
- `GET /api/acks` — list ACKs.
|
- `GET /api/acks` — list ACKs.
|
||||||
- `GET /api/acks/{id}` — ACK detail, including the regenerated
|
- `GET /api/acks/{id}` — ACK detail, including the regenerated
|
||||||
`raw_999_text`.
|
`raw_999_text`.
|
||||||
|
- `POST /api/parse-ta1` — parse an inbound TA1 envelope ACK and persist it.
|
||||||
|
- `GET /api/ta1-acks` — list TA1 acks.
|
||||||
|
- `GET /api/ta1-acks/{id}` — TA1 ack detail (envelope control segments
|
||||||
|
+ the parser's accept/reject verdict).
|
||||||
- `POST /api/eligibility/request` — build a 270 from JSON.
|
- `POST /api/eligibility/request` — build a 270 from JSON.
|
||||||
- `POST /api/eligibility/parse-271` — ingest a 271 and return parsed
|
- `POST /api/eligibility/parse-271` — ingest a 271 and return parsed
|
||||||
coverage benefits.
|
coverage benefits.
|
||||||
@@ -459,9 +1011,9 @@ ACKs and lets you download the regenerated 999 text.
|
|||||||
|
|
||||||
### SP6 endpoints (inbox)
|
### SP6 endpoints (inbox)
|
||||||
|
|
||||||
- `GET /api/inbox/lanes` — all four lanes (rejected, candidates,
|
- `GET /api/inbox/lanes` — all five lanes (rejected, payer_rejected,
|
||||||
unmatched, done_today) in a single round-trip, with row-level
|
candidates, unmatched, done_today) in a single round-trip, with
|
||||||
scoring and matched-remit context.
|
row-level scoring and matched-remit context.
|
||||||
- `POST /api/inbox/candidates/{remit_id}/match` — manual match of a
|
- `POST /api/inbox/candidates/{remit_id}/match` — manual match of a
|
||||||
candidate remit to one of its scored claims; `409` if the claim
|
candidate remit to one of its scored claims; `409` if the claim
|
||||||
state moved out from under us.
|
state moved out from under us.
|
||||||
@@ -510,6 +1062,58 @@ ACKs and lets you download the regenerated 999 text.
|
|||||||
Download** button in the Inbox rejected-lane BulkBar (N>1 modal
|
Download** button in the Inbox rejected-lane BulkBar (N>1 modal
|
||||||
prompt).
|
prompt).
|
||||||
|
|
||||||
|
### SP9 endpoints (multi-payer, multi-NPI, SFTP stub)
|
||||||
|
|
||||||
|
- `GET /api/clearhouse` — the `clearhouse` singleton (name, TPID,
|
||||||
|
file-naming block, SFTP block).
|
||||||
|
- `POST /api/clearhouse/submit` — push a batch of generated 837 files.
|
||||||
|
The SP9 implementation writes to a local `staging_dir`; the SP13
|
||||||
|
swap replaces the write with a real `paramiko` SFTP push without
|
||||||
|
changing the route shape.
|
||||||
|
- `GET /api/config/providers` and `GET /api/config/providers/{npi}` —
|
||||||
|
list / fetch providers from the new `providers` table.
|
||||||
|
- `GET /api/config/payers` and
|
||||||
|
`GET /api/config/payers/{payer_id}/configs` — list payers; for a
|
||||||
|
given payer, return the per-transaction-type `payer_configs` rows.
|
||||||
|
- `POST /api/admin/reload-config` — re-read `config/payers.yaml` and
|
||||||
|
refresh the in-process cache without a server restart.
|
||||||
|
|
||||||
|
### SP10 endpoints (277CA + Payer-Rejected lane)
|
||||||
|
|
||||||
|
- `POST /api/parse-277ca` — upload a 277CA file; persist the parsed
|
||||||
|
`ClaimStatus` rows and stamp the matching claims with
|
||||||
|
`payer_rejected_at` (monotonic, never overwritten by `NULL`).
|
||||||
|
- `GET /api/277ca-acks` — list 277CA acks.
|
||||||
|
- `GET /api/277ca-acks/{id}` — one 277CA ack with its per-claim
|
||||||
|
`ClaimStatus` rows + regenerated text.
|
||||||
|
- `GET /api/inbox/lanes` — the response now also carries a
|
||||||
|
`payer_rejected` lane populated from
|
||||||
|
`Claim.payer_rejected_at IS NOT NULL`.
|
||||||
|
|
||||||
|
### SP11 endpoints (tamper-evident audit log)
|
||||||
|
|
||||||
|
- `GET /api/admin/audit-log` — paginated audit log. Each row carries
|
||||||
|
`(id, prev_hash, row_hash, event_type, actor, payload_json,
|
||||||
|
created_at)` where `row_hash = sha256(prev_hash || canonical_json(payload))`.
|
||||||
|
- `GET /api/admin/audit-log/verify` — walk the chain in insertion
|
||||||
|
order; return `{ok: true}` or the first `{id, expected, got}`
|
||||||
|
mismatch. The walk is O(n) with one indexed lookup per row.
|
||||||
|
|
||||||
|
### SP12 (encryption at rest — no new routes)
|
||||||
|
|
||||||
|
SP12 introduces no API routes. The `cyclone.db.key` Keychain entry +
|
||||||
|
optional `sqlcipher3` dependency enable AES-256 encryption transparently
|
||||||
|
on the next connection. See [Encryption at Rest](#encryption-at-rest)
|
||||||
|
and [docs/reference/co-medicaid.md](docs/reference/co-medicaid.md) for
|
||||||
|
the one-time setup recipe.
|
||||||
|
|
||||||
|
### SP13 endpoints (paramiko SFTP)
|
||||||
|
|
||||||
|
- `POST /api/clearhouse/submit` — same endpoint as SP9; the
|
||||||
|
implementation is now a real `paramiko` `SftpClient.write` to
|
||||||
|
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
|
||||||
|
SFTP credentials are fetched from the macOS Keychain at call time.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
No license file yet; this is internal-use software. Add a `LICENSE` file
|
No license file yet; this is internal-use software. Add a `LICENSE` file
|
||||||
|
|||||||
+15
-1
@@ -79,10 +79,24 @@ python -m cyclone serve
|
|||||||
### Endpoints
|
### Endpoints
|
||||||
|
|
||||||
| Method | Path | Purpose |
|
| Method | Path | Purpose |
|
||||||
| ------ | ---------------- | -------------------------------------------------- |
|
| ------ | --------------------- | ------------------------------------------------------------------ |
|
||||||
| GET | `/api/health` | Liveness probe: `{"status": "ok", "version": ...}` |
|
| GET | `/api/health` | Liveness probe: `{"status": "ok", "version": ...}` |
|
||||||
| POST | `/api/parse-837` | Upload an X12 837P file, get parsed claims back |
|
| POST | `/api/parse-837` | Upload an X12 837P file, get parsed claims back |
|
||||||
| POST | `/api/parse-835` | Upload an X12 835 ERA file, get parsed payouts back |
|
| POST | `/api/parse-835` | Upload an X12 835 ERA file, get parsed payouts back |
|
||||||
|
| POST | `/api/parse-999` | Upload an inbound 999 ACK and persist it |
|
||||||
|
| POST | `/api/parse-ta1` | Upload an inbound TA1 envelope ACK and persist it |
|
||||||
|
| POST | `/api/parse-277ca` | Upload a 277CA claim acknowledgment and stamp payer_rejected claims |
|
||||||
|
| POST | `/api/eligibility/request` | Build a 270 from JSON (subscriber / provider / payer) |
|
||||||
|
| POST | `/api/eligibility/parse-271` | Ingest a 271 and return structured `coverage_benefits` |
|
||||||
|
| GET | `/api/clearhouse` | The `clearhouse` singleton (SP9) |
|
||||||
|
| POST | `/api/clearhouse/submit` | Push a batch of generated 837 files via SFTP (SP9 stub, SP13 real) |
|
||||||
|
| POST | `/api/admin/reload-config` | Re-read `config/payers.yaml` and refresh the in-process cache |
|
||||||
|
| GET | `/api/admin/audit-log` | Paginated tamper-evident audit log (SP11) |
|
||||||
|
| GET | `/api/admin/audit-log/verify` | Walk the audit_log hash chain (SP11) |
|
||||||
|
|
||||||
|
The full surface — claim / remittance / batch / inbox / stream
|
||||||
|
endpoints, config lookups, and the 270/271 builder — is enumerated in
|
||||||
|
the root [README](../README.md#multi-payer-multi-npi--clearhouse).
|
||||||
|
|
||||||
`POST /api/parse-837` accepts `multipart/form-data` with a single `file`
|
`POST /api/parse-837` accepts `multipart/form-data` with a single `file`
|
||||||
field. Optional query parameters:
|
field. Optional query parameters:
|
||||||
|
|||||||
@@ -14,6 +14,8 @@ dependencies = [
|
|||||||
"uvicorn[standard]>=0.27,<1",
|
"uvicorn[standard]>=0.27,<1",
|
||||||
"python-multipart>=0.0.9,<1",
|
"python-multipart>=0.0.9,<1",
|
||||||
"sqlalchemy>=2.0,<3",
|
"sqlalchemy>=2.0,<3",
|
||||||
|
"pyyaml>=6.0,<7",
|
||||||
|
"keyring>=25.0,<26",
|
||||||
]
|
]
|
||||||
|
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
@@ -23,6 +25,16 @@ dev = [
|
|||||||
"pytest-asyncio>=0.23,<1",
|
"pytest-asyncio>=0.23,<1",
|
||||||
"httpx>=0.27,<1",
|
"httpx>=0.27,<1",
|
||||||
]
|
]
|
||||||
|
sqlcipher = [
|
||||||
|
# SP12: encryption at rest. Optional — without it the DB is plain SQLite.
|
||||||
|
# Install via: pip install -e .[sqlcipher] (after brew install sqlcipher).
|
||||||
|
"sqlcipher3>=0.6,<1",
|
||||||
|
]
|
||||||
|
sftp = [
|
||||||
|
# SP13: real SFTP wire-up. Optional — without it the stub keeps working.
|
||||||
|
# Install via: pip install -e .[sftp].
|
||||||
|
"paramiko>=3.4,<6",
|
||||||
|
]
|
||||||
|
|
||||||
[project.scripts]
|
[project.scripts]
|
||||||
cyclone = "cyclone.cli:main"
|
cyclone = "cyclone.cli:main"
|
||||||
|
|||||||
+1445
-331
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,247 @@
|
|||||||
|
"""Shared helpers used by ``cyclone.api`` route handlers.
|
||||||
|
|
||||||
|
Everything in this module is private to the API layer (no business
|
||||||
|
logic, no DB writes). It collects the cross-cutting concerns that used
|
||||||
|
to live inline at the top of ``api.py``:
|
||||||
|
|
||||||
|
* NDJSON wire-format primitives (``ndjson_line``, ``ndjson_stream_list``,
|
||||||
|
``ndjson_stream_837``, ``ndjson_stream_835``).
|
||||||
|
* Content negotiation (``client_wants_json``, ``wants_ndjson``).
|
||||||
|
* Strict / ``raw_segments`` rewrites applied before persisting parsed
|
||||||
|
837P and 835 results.
|
||||||
|
* Validation-error probes for both transactions.
|
||||||
|
* The shared live-tail async generator (``tail_events``,
|
||||||
|
``heartbeat_seconds``) used by every ``/api/<resource>/stream``
|
||||||
|
endpoint.
|
||||||
|
|
||||||
|
Extracted as part of the api.py router split (see /tmp/refactor-cyclone.md).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import AsyncIterator, Iterator
|
||||||
|
|
||||||
|
from fastapi import Request
|
||||||
|
|
||||||
|
from cyclone.parsers.models import ClaimOutput, ParseResult
|
||||||
|
from cyclone.parsers.models_835 import ParseResult835
|
||||||
|
from cyclone.pubsub import EventBus
|
||||||
|
|
||||||
|
|
||||||
|
def utcnow() -> datetime:
|
||||||
|
"""tz-aware UTC ``datetime`` (matches :func:`cyclone.store.utcnow`)."""
|
||||||
|
return datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def ndjson_line(event: dict) -> bytes:
|
||||||
|
"""Serialize one event dict as a single NDJSON line (UTF-8, trailing ``\\n``).
|
||||||
|
|
||||||
|
Used by the live-tail streaming endpoints to emit a uniform wire format
|
||||||
|
that the frontend ``tail-stream.ts`` parser can split on newlines.
|
||||||
|
Compact separators keep each line small and avoid ambiguity with embedded
|
||||||
|
whitespace.
|
||||||
|
"""
|
||||||
|
return (json.dumps(event, separators=(",", ":")) + "\n").encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def client_wants_json(request: Request) -> bool:
|
||||||
|
"""Content negotiation: prefer ``application/json`` when the client asks for it.
|
||||||
|
|
||||||
|
NDJSON is the default for browser uploads that don't set ``Accept``. The
|
||||||
|
frontend opts into JSON via ``Accept: application/json``.
|
||||||
|
"""
|
||||||
|
accept = request.headers.get("accept", "")
|
||||||
|
# If the client mentions JSON at all (and isn't asking for NDJSON
|
||||||
|
# specifically) treat it as a single-object request. The browser default
|
||||||
|
# ``*/*`` falls through to NDJSON.
|
||||||
|
if "application/json" in accept and "application/x-ndjson" not in accept:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def wants_ndjson(request: Request) -> bool:
|
||||||
|
"""Content negotiation for list endpoints: NDJSON is an opt-in, JSON is the
|
||||||
|
default (per spec 6.2: "Default JSON response wraps the same data in a
|
||||||
|
{items, total, returned, has_more} envelope so the frontend can paginate
|
||||||
|
uniformly").
|
||||||
|
|
||||||
|
Used by the GET list routes (/api/batches, /api/claims, /api/remittances,
|
||||||
|
/api/providers, /api/activity). NDJSON is returned only when the client
|
||||||
|
explicitly sends ``Accept: application/x-ndjson`` (with or without
|
||||||
|
``application/json``). Bare ``*/*``, an empty Accept, or an explicit
|
||||||
|
``Accept: application/json`` all return the JSON envelope.
|
||||||
|
"""
|
||||||
|
accept = request.headers.get("accept", "")
|
||||||
|
return "application/x-ndjson" in accept
|
||||||
|
|
||||||
|
|
||||||
|
def ndjson_stream_list(
|
||||||
|
items: list[dict], total: int, returned: int, has_more: bool,
|
||||||
|
) -> Iterator[str]:
|
||||||
|
"""Yield NDJSON lines for a list endpoint: one ``item`` per dict, then a
|
||||||
|
final ``summary`` line. Mirrors spec section 6.2 streaming rule.
|
||||||
|
"""
|
||||||
|
for it in items:
|
||||||
|
yield json.dumps({"type": "item", "data": it}) + "\n"
|
||||||
|
yield json.dumps({
|
||||||
|
"type": "summary",
|
||||||
|
"data": {"total": total, "returned": returned, "has_more": has_more},
|
||||||
|
}) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def ndjson_stream_837(result: ParseResult) -> Iterator[bytes]:
|
||||||
|
"""Yield one JSON object per line: envelope → claims → summary."""
|
||||||
|
envelope_obj = (
|
||||||
|
result.envelope.model_dump() if result.envelope is not None else None
|
||||||
|
)
|
||||||
|
yield (json.dumps({"type": "envelope", "data": envelope_obj}) + "\n").encode("utf-8")
|
||||||
|
for claim in result.claims:
|
||||||
|
yield (json.dumps({"type": "claim", "data": json.loads(claim.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
yield (json.dumps({"type": "summary", "data": json.loads(result.summary.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def ndjson_stream_835(result: ParseResult835) -> Iterator[bytes]:
|
||||||
|
"""Yield one JSON object per line: envelope → financial → trace → payer → payee → claim_payments → summary."""
|
||||||
|
yield (json.dumps({"type": "envelope", "data": json.loads(result.envelope.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
yield (json.dumps({"type": "financial_info", "data": json.loads(result.financial_info.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
yield (json.dumps({"type": "trace", "data": json.loads(result.trace.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
yield (json.dumps({"type": "payer", "data": json.loads(result.payer.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
yield (json.dumps({"type": "payee", "data": json.loads(result.payee.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
for claim in result.claims:
|
||||||
|
yield (json.dumps({"type": "claim_payment", "data": json.loads(claim.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
yield (json.dumps({"type": "summary", "data": json.loads(result.summary.model_dump_json())}) + "\n").encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def strict_rewrite_837(result: ParseResult) -> ParseResult:
|
||||||
|
"""Promote warnings to errors (mirrors the CLI's --strict)."""
|
||||||
|
claims: list[ClaimOutput] = []
|
||||||
|
for claim in result.claims:
|
||||||
|
promoted = [
|
||||||
|
issue.model_copy(update={"severity": "error"})
|
||||||
|
for issue in claim.validation.warnings
|
||||||
|
]
|
||||||
|
new_errors = claim.validation.errors + promoted
|
||||||
|
claims.append(
|
||||||
|
claim.model_copy(
|
||||||
|
update={
|
||||||
|
"validation": claim.validation.model_copy(
|
||||||
|
update={"errors": new_errors, "passed": not new_errors}
|
||||||
|
)
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
passed = sum(1 for c in claims if c.validation.passed)
|
||||||
|
failed = len(claims) - passed
|
||||||
|
summary = result.summary.model_copy(
|
||||||
|
update={
|
||||||
|
"passed": passed,
|
||||||
|
"failed": failed,
|
||||||
|
"failed_claim_ids": [c.claim_id for c in claims if not c.validation.passed],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return result.model_copy(update={"claims": claims, "summary": summary})
|
||||||
|
|
||||||
|
|
||||||
|
def strict_rewrite_835(result: ParseResult835) -> ParseResult835:
|
||||||
|
"""Promote warnings to errors (mirrors the CLI's --strict)."""
|
||||||
|
if result.validation is None:
|
||||||
|
return result
|
||||||
|
report = result.validation
|
||||||
|
promoted = [i.model_copy(update={"severity": "error"}) for i in report.warnings]
|
||||||
|
new_errors = report.errors + promoted
|
||||||
|
new_report = report.model_copy(update={"errors": new_errors, "passed": not new_errors})
|
||||||
|
passed = 1 if new_report.passed else 0
|
||||||
|
failed = 1 if not new_report.passed else 0
|
||||||
|
new_summary = result.summary.model_copy(
|
||||||
|
update={"passed": passed, "failed": failed}
|
||||||
|
)
|
||||||
|
return result.model_copy(update={"validation": new_report, "summary": new_summary})
|
||||||
|
|
||||||
|
|
||||||
|
def drop_raw_segments_837(result: ParseResult) -> ParseResult:
|
||||||
|
"""Return a copy of ``result`` with ``raw_segments`` cleared on every claim."""
|
||||||
|
claims = [c.model_copy(update={"raw_segments": []}) for c in result.claims]
|
||||||
|
return result.model_copy(update={"claims": claims})
|
||||||
|
|
||||||
|
|
||||||
|
def drop_raw_segments_835(result: ParseResult835) -> ParseResult835:
|
||||||
|
"""Return a copy of ``result`` with ``raw_segments`` cleared on every claim."""
|
||||||
|
claims = [c.model_copy(update={"raw_segments": []}) for c in result.claims]
|
||||||
|
return result.model_copy(update={"claims": claims})
|
||||||
|
|
||||||
|
|
||||||
|
def has_claim_validation_errors(result: ParseResult) -> bool:
|
||||||
|
return any(not c.validation.passed for c in result.claims)
|
||||||
|
|
||||||
|
|
||||||
|
def has_835_validation_errors(result: ParseResult835) -> bool:
|
||||||
|
return result.validation is not None and not result.validation.passed
|
||||||
|
|
||||||
|
|
||||||
|
def heartbeat_seconds() -> float:
|
||||||
|
"""Return the configured tail heartbeat interval.
|
||||||
|
|
||||||
|
Read from ``CYCLONE_TAIL_HEARTBEAT_S`` at call time so tests can
|
||||||
|
monkeypatch the env var without reloading the module. Defaults to
|
||||||
|
15s (the production cadence); tests override to a small value (e.g.
|
||||||
|
0.2s) to keep their runtime bounded.
|
||||||
|
"""
|
||||||
|
raw = os.environ.get("CYCLONE_TAIL_HEARTBEAT_S", "15")
|
||||||
|
try:
|
||||||
|
v = float(raw)
|
||||||
|
except ValueError:
|
||||||
|
return 15.0
|
||||||
|
return v if v > 0 else 15.0
|
||||||
|
|
||||||
|
|
||||||
|
async def tail_events(
|
||||||
|
request: Request, bus: EventBus, kinds: list[str]
|
||||||
|
) -> AsyncIterator[bytes]:
|
||||||
|
"""Forward subscribed events as ``item`` lines with periodic heartbeats.
|
||||||
|
|
||||||
|
Polls the underlying ``asyncio.Queue`` directly (via
|
||||||
|
:meth:`EventBus.subscribe_raw`) instead of awaiting the bus's
|
||||||
|
async-iterator wrapper. ``asyncio.wait_for`` cancels the inner
|
||||||
|
future on timeout, which would otherwise terminate the bus
|
||||||
|
iterator at its ``await`` point and break subsequent
|
||||||
|
``__anext__`` calls with ``StopAsyncIteration``. Polling
|
||||||
|
``queue.get()`` is idempotent under cancellation, so heartbeats
|
||||||
|
don't poison the subscription.
|
||||||
|
|
||||||
|
A ``try/finally`` unsubscribes the queue from the bus when the
|
||||||
|
caller disconnects or the generator is garbage collected —
|
||||||
|
otherwise the bus would leak one queue per open stream.
|
||||||
|
"""
|
||||||
|
hb_s = heartbeat_seconds()
|
||||||
|
queue, _sub = bus.subscribe_raw(kinds)
|
||||||
|
try:
|
||||||
|
while True:
|
||||||
|
if await request.is_disconnected():
|
||||||
|
return
|
||||||
|
get_task = asyncio.ensure_future(queue.get())
|
||||||
|
sleep_task = asyncio.ensure_future(asyncio.sleep(hb_s))
|
||||||
|
try:
|
||||||
|
done, pending = await asyncio.wait(
|
||||||
|
{get_task, sleep_task},
|
||||||
|
return_when=asyncio.FIRST_COMPLETED,
|
||||||
|
)
|
||||||
|
except BaseException:
|
||||||
|
get_task.cancel()
|
||||||
|
sleep_task.cancel()
|
||||||
|
raise
|
||||||
|
for t in pending:
|
||||||
|
t.cancel()
|
||||||
|
if get_task in done:
|
||||||
|
event = get_task.result()
|
||||||
|
yield ndjson_line({"type": "item", "data": event})
|
||||||
|
else:
|
||||||
|
yield ndjson_line({
|
||||||
|
"type": "heartbeat",
|
||||||
|
"data": {"ts": utcnow().isoformat().replace("+00:00", "Z")},
|
||||||
|
})
|
||||||
|
finally:
|
||||||
|
bus.unsubscribe(queue, kinds)
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""Resource-group routers. Imported and registered by ``cyclone.api``."""
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
"""``/api/acks`` — list & detail endpoints for the 999 ACK inbox.
|
||||||
|
|
||||||
|
These are the persisted acknowledgment rows produced by
|
||||||
|
``POST /api/parse-999``. The frontend ``useAcks`` hook re-shapes the
|
||||||
|
list payload to its ``Ack`` interface in ``src/types/index.ts``.
|
||||||
|
|
||||||
|
The detail endpoint returns the full ``raw_json`` payload plus the
|
||||||
|
regenerated ``raw_999_text`` so the UI can show "view source" without a
|
||||||
|
second round-trip.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Query, Request
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
|
||||||
|
from cyclone.api_helpers import ndjson_stream_list, wants_ndjson
|
||||||
|
from cyclone.parsers.models_999 import ParseResult999
|
||||||
|
from cyclone.parsers.serialize_999 import serialize_999
|
||||||
|
from cyclone.store import store
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _ack_to_ui(row) -> dict:
|
||||||
|
"""Map an ``Ack`` ORM row to the UI shape used by ``/api/acks``.
|
||||||
|
|
||||||
|
Field names match the rest of the Cyclone API (snake_case). The
|
||||||
|
frontend ``useAcks`` hook re-shapes this to the camelCase ``Ack``
|
||||||
|
interface in ``src/types/index.ts``.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"id": row.id,
|
||||||
|
"source_batch_id": row.source_batch_id,
|
||||||
|
"accepted_count": row.accepted_count,
|
||||||
|
"rejected_count": row.rejected_count,
|
||||||
|
"received_count": row.received_count,
|
||||||
|
"ack_code": row.ack_code,
|
||||||
|
"parsed_at": (
|
||||||
|
row.parsed_at.isoformat().replace("+00:00", "Z")
|
||||||
|
if row.parsed_at is not None
|
||||||
|
else ""
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/acks")
|
||||||
|
def list_acks_endpoint(
|
||||||
|
request: Request,
|
||||||
|
limit: int = Query(100, ge=1, le=1000),
|
||||||
|
) -> Any:
|
||||||
|
"""Return the list of persisted 999 ACKs, newest first."""
|
||||||
|
rows = store.list_acks()
|
||||||
|
items = [_ack_to_ui(r) for r in rows[:limit]]
|
||||||
|
total = len(rows)
|
||||||
|
returned = len(items)
|
||||||
|
has_more = total > returned
|
||||||
|
if wants_ndjson(request):
|
||||||
|
return StreamingResponse(
|
||||||
|
ndjson_stream_list(items, total, returned, has_more),
|
||||||
|
media_type="application/x-ndjson",
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"items": items,
|
||||||
|
"total": total,
|
||||||
|
"returned": returned,
|
||||||
|
"has_more": has_more,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/acks/{ack_id}")
|
||||||
|
def get_ack_endpoint(ack_id: int) -> dict:
|
||||||
|
"""Return one persisted ACK row with its parsed detail.
|
||||||
|
|
||||||
|
Path param is ``ack_id`` (not ``id``) to avoid shadowing FastAPI's
|
||||||
|
internal ``id`` name and to keep OpenAPI docs self-describing.
|
||||||
|
Returns 404 when the ACK is missing — never 500.
|
||||||
|
"""
|
||||||
|
row = store.get_ack(ack_id)
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=404,
|
||||||
|
detail={"error": "Not found", "detail": f"Ack {ack_id} not found"},
|
||||||
|
)
|
||||||
|
body = _ack_to_ui(row)
|
||||||
|
body["raw_json"] = row.raw_json
|
||||||
|
# Regenerate the X12 text from raw_json so the operator can download
|
||||||
|
# the actual 999 file. (SP3 P3 follow-up: list endpoint doesn't carry
|
||||||
|
# the regenerated text to keep payloads small; detail does.)
|
||||||
|
if row.raw_json:
|
||||||
|
try:
|
||||||
|
regenerated = ParseResult999.model_validate(row.raw_json)
|
||||||
|
icn = regenerated.envelope.control_number or "000000001"
|
||||||
|
body["raw_999_text"] = serialize_999(regenerated, interchange_control_number=icn)
|
||||||
|
except Exception as exc: # noqa: BLE001 — never 500 on a regen failure
|
||||||
|
log.warning("Could not regenerate 999 for ack %s: %s", ack_id, exc)
|
||||||
|
body["raw_999_text"] = None
|
||||||
|
else:
|
||||||
|
body["raw_999_text"] = None
|
||||||
|
return body
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
"""``/api/admin/validate-provider`` — NPI + Tax ID liveness probe (SP20).
|
||||||
|
|
||||||
|
Pure read-only endpoint that runs the local NPI Luhn + EIN format checks
|
||||||
|
without touching the DB. Useful for:
|
||||||
|
- operators vetting a new provider before adding them to the registry,
|
||||||
|
- the dashboard's "validate" button on a Provider row,
|
||||||
|
- smoke-testing the SP20 checks after a deploy.
|
||||||
|
|
||||||
|
Both query params are optional; omitting one just skips that check.
|
||||||
|
Returns the per-check result dict so the caller can distinguish "bad
|
||||||
|
format" from "bad checksum".
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Query
|
||||||
|
|
||||||
|
from cyclone.npi import is_valid_npi, is_valid_tax_id, normalize_tax_id
|
||||||
|
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/admin/validate-provider")
|
||||||
|
def validate_provider(
|
||||||
|
npi: str | None = Query(None, description="10-digit NPI to validate (Luhn checksum)"),
|
||||||
|
tax_id: str | None = Query(None, description="9-digit EIN to validate (format + reserved-prefix check)"),
|
||||||
|
) -> dict:
|
||||||
|
"""Return per-field validation results for ``npi`` and ``tax_id``.
|
||||||
|
|
||||||
|
Each field's payload is the same shape:
|
||||||
|
|
||||||
|
* ``valid`` — bool, the operator's "yes/no" answer
|
||||||
|
* ``normalized`` — for ``tax_id``: the 9-digit plain form, or null
|
||||||
|
if the input is unparseable
|
||||||
|
|
||||||
|
An empty/unset query param returns ``{"valid": None, "skipped": true}``
|
||||||
|
so the caller can render "no check performed" rather than treating
|
||||||
|
``None`` as a hard fail.
|
||||||
|
"""
|
||||||
|
result: dict = {}
|
||||||
|
|
||||||
|
if npi is None or npi == "":
|
||||||
|
result["npi"] = {"valid": None, "skipped": True}
|
||||||
|
else:
|
||||||
|
result["npi"] = {
|
||||||
|
"valid": is_valid_npi(npi),
|
||||||
|
"skipped": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
if tax_id is None or tax_id == "":
|
||||||
|
result["tax_id"] = {"valid": None, "skipped": True, "normalized": None}
|
||||||
|
else:
|
||||||
|
normalized = normalize_tax_id(tax_id)
|
||||||
|
result["tax_id"] = {
|
||||||
|
"valid": is_valid_tax_id(tax_id),
|
||||||
|
"skipped": False,
|
||||||
|
"normalized": normalized,
|
||||||
|
}
|
||||||
|
|
||||||
|
return result
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
"""``GET /api/health`` — liveness + readiness probe.
|
||||||
|
|
||||||
|
SP19 expanded the shallow ``{"status": "ok", "version": ...}`` probe
|
||||||
|
into a snapshot of every subsystem:
|
||||||
|
|
||||||
|
* **db** — can we open a session and run ``SELECT 1``?
|
||||||
|
* **scheduler** — is the MFT polling loop running? same for the
|
||||||
|
backup scheduler.
|
||||||
|
* **pubsub** — current subscriber counts per event kind.
|
||||||
|
* **batch** — most recent batch id + timestamp.
|
||||||
|
|
||||||
|
Returns ``status="ok"`` only when every subsystem is healthy.
|
||||||
|
``status="degraded"`` if any subsystem is unhappy but the API
|
||||||
|
itself is responsive. Per-subsystem errors are surfaced in the
|
||||||
|
respective dict so an operator doesn't have to guess.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Request
|
||||||
|
|
||||||
|
from cyclone import __version__
|
||||||
|
from cyclone.security import get_health_snapshot
|
||||||
|
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/health")
|
||||||
|
def health(request: Request) -> dict:
|
||||||
|
snap = get_health_snapshot()
|
||||||
|
# Fill in live pubsub subscriber counts using the per-request app
|
||||||
|
# state (the snapshot builder doesn't have request context).
|
||||||
|
bus = getattr(request.app.state, "event_bus", None)
|
||||||
|
if bus is not None and hasattr(bus, "stats"):
|
||||||
|
snap.pubsub = bus.stats()
|
||||||
|
elif bus is not None:
|
||||||
|
snap.pubsub = {"note": "EventBus.stats() not available"}
|
||||||
|
else:
|
||||||
|
snap.pubsub = {"note": "EventBus not attached (running outside lifespan?)"}
|
||||||
|
return snap.to_dict()
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
"""``/api/ta1-acks`` — list & detail endpoints for persisted TA1 envelopes.
|
||||||
|
|
||||||
|
TA1 is the interchange-control ACK (ISA/IEA acknowledgement). It's a
|
||||||
|
single segment, no functional group, no transaction set. Cyclone
|
||||||
|
persists the parsed fields plus a synthetic ``source_batch_id`` so the
|
||||||
|
row can sit alongside the 999 / 277CA ack rows without special-casing.
|
||||||
|
|
||||||
|
The detail endpoint also reconstructs the TA1 segment string
|
||||||
|
(``TA1*...~``) so the operator can copy it into a downstream tool.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Query
|
||||||
|
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.store import store
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
def _ta1_to_ui(row: db.Ta1Ack) -> dict:
|
||||||
|
"""Render a Ta1Ack row for the UI (list endpoint shape)."""
|
||||||
|
return {
|
||||||
|
"id": row.id,
|
||||||
|
"control_number": row.control_number,
|
||||||
|
"ack_code": row.ack_code,
|
||||||
|
"note_code": row.note_code,
|
||||||
|
"interchange_date": row.interchange_date.isoformat()
|
||||||
|
if row.interchange_date else None,
|
||||||
|
"interchange_time": row.interchange_time,
|
||||||
|
"sender_id": row.sender_id,
|
||||||
|
"receiver_id": row.receiver_id,
|
||||||
|
"source_batch_id": row.source_batch_id,
|
||||||
|
"parsed_at": row.parsed_at.isoformat() if row.parsed_at else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _serialize_ta1_from_row(row: db.Ta1Ack) -> str:
|
||||||
|
"""Reconstruct a TA1 segment from the persisted flat row (for the detail endpoint)."""
|
||||||
|
date_s = row.interchange_date.strftime("%y%m%d") if row.interchange_date else ""
|
||||||
|
return (
|
||||||
|
f"TA1*{row.control_number}*{date_s}*{row.interchange_time or ''}*"
|
||||||
|
f"{row.ack_code}*{row.note_code or ''}~"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/ta1-acks")
|
||||||
|
def list_ta1_acks_endpoint(
|
||||||
|
limit: int = Query(100, ge=1, le=1000),
|
||||||
|
) -> Any:
|
||||||
|
"""Return the list of persisted TA1 ACKs, newest first.
|
||||||
|
|
||||||
|
Mirrors :func:`cyclone.api_routers.acks.list_acks_endpoint` — fetches all
|
||||||
|
rows then slices in Python so the ``total`` field reflects the full row
|
||||||
|
count regardless of the ``limit`` cap.
|
||||||
|
"""
|
||||||
|
rows = store.list_ta1_acks()
|
||||||
|
items = [_ta1_to_ui(r) for r in rows[:limit]]
|
||||||
|
return {
|
||||||
|
"total": len(rows),
|
||||||
|
"items": items,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/api/ta1-acks/{ack_id}")
|
||||||
|
def get_ta1_ack_endpoint(ack_id: int) -> dict:
|
||||||
|
"""Return one persisted TA1 ACK row with its parsed detail."""
|
||||||
|
row = store.get_ta1_ack(ack_id)
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(status_code=404, detail=f"TA1 ACK {ack_id} not found")
|
||||||
|
body = _ta1_to_ui(row)
|
||||||
|
body["raw_ta1_text"] = _serialize_ta1_from_row(row)
|
||||||
|
body["raw_json"] = row.raw_json
|
||||||
|
return body
|
||||||
@@ -0,0 +1,254 @@
|
|||||||
|
"""Tamper-evident hash-chained audit_log.
|
||||||
|
|
||||||
|
SP11.
|
||||||
|
|
||||||
|
Each row's hash is SHA-256 of
|
||||||
|
``(id, event_type, entity_type, entity_id, actor, payload_json,
|
||||||
|
created_at, prev_hash)`` and ``prev_hash`` is the previous row's hash.
|
||||||
|
That forms a chain: changing any row's payload invalidates every
|
||||||
|
subsequent row's hash. :func:`verify_chain` walks the chain and
|
||||||
|
returns the first mismatch index (or ``None`` for a clean chain).
|
||||||
|
|
||||||
|
We use SHA-256 (FIPS-approved, fast on commodity hardware) instead
|
||||||
|
of a Merkle tree because the chain is linear: every row depends on
|
||||||
|
exactly one prior row. A Merkle tree would let us prove individual
|
||||||
|
membership with O(log n) witnesses, but the chain's whole point is
|
||||||
|
end-to-end integrity, not selective disclosure.
|
||||||
|
|
||||||
|
Append-only by convention: the application MUST NOT call
|
||||||
|
``session.delete(row)`` or modify an existing row. Doing so is
|
||||||
|
auditable via :func:`verify_chain`. We deliberately do not enforce
|
||||||
|
this at the DB level (no triggers, no revoked UPDATE permission)
|
||||||
|
because that breaks the test fixtures that recreate the DB.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from cyclone.db import AuditLog
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# 64 hex chars = 256 bits. Constant for easy comparison.
|
||||||
|
HASH_LEN = 64
|
||||||
|
|
||||||
|
# Genesis row's prev_hash — a fixed "all zeros" sentinel so the first
|
||||||
|
# row in the chain has a deterministic predecessor. This is the same
|
||||||
|
# convention Bitcoin and other ledgers use.
|
||||||
|
GENESIS_PREV_HASH = "0" * HASH_LEN
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Hashing
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def _hash_row(
|
||||||
|
*,
|
||||||
|
row_id: int,
|
||||||
|
event_type: str,
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: str,
|
||||||
|
actor: str,
|
||||||
|
payload_json: str | None,
|
||||||
|
created_at: datetime,
|
||||||
|
prev_hash: str,
|
||||||
|
) -> str:
|
||||||
|
"""Compute SHA-256 hex of a row's canonical form.
|
||||||
|
|
||||||
|
The fields are concatenated with a separator that cannot appear
|
||||||
|
inside any field (``\\x1f`` — the ASCII unit separator). Using a
|
||||||
|
delimiter avoids length-ambiguity attacks where two different
|
||||||
|
payloads with the same string-joined form would hash to the same
|
||||||
|
digest.
|
||||||
|
"""
|
||||||
|
sep = "\x1f"
|
||||||
|
# Normalize the timestamp to an ISO 8601 UTC string so the hash is
|
||||||
|
# stable across timezone-aware and timezone-naive datetimes (the
|
||||||
|
# DB may give us either depending on the SQLite build).
|
||||||
|
if created_at.tzinfo is None:
|
||||||
|
created_at = created_at.replace(tzinfo=timezone.utc)
|
||||||
|
created_at_iso = created_at.astimezone(timezone.utc).isoformat()
|
||||||
|
payload = payload_json or ""
|
||||||
|
canonical = sep.join([
|
||||||
|
str(row_id),
|
||||||
|
event_type,
|
||||||
|
entity_type,
|
||||||
|
entity_id,
|
||||||
|
actor,
|
||||||
|
created_at_iso,
|
||||||
|
payload,
|
||||||
|
prev_hash,
|
||||||
|
])
|
||||||
|
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Append
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class AuditEvent:
|
||||||
|
"""An audit event ready to be appended.
|
||||||
|
|
||||||
|
Mirrors the ``AuditLog`` row shape minus the auto-assigned id and
|
||||||
|
computed hash. Payload must be JSON-serializable; the audit_log
|
||||||
|
module handles the encoding so callers don't need to think about
|
||||||
|
canonical form.
|
||||||
|
"""
|
||||||
|
|
||||||
|
event_type: str
|
||||||
|
entity_type: str
|
||||||
|
entity_id: str
|
||||||
|
payload: dict[str, Any] = field(default_factory=dict)
|
||||||
|
actor: str = "system"
|
||||||
|
created_at: datetime | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def append_event(
|
||||||
|
session: Session,
|
||||||
|
event: AuditEvent,
|
||||||
|
) -> AuditLog:
|
||||||
|
"""Append one event to the audit_log chain and return the row.
|
||||||
|
|
||||||
|
The caller is responsible for ``session.commit()`` — this lets
|
||||||
|
callers batch multiple appends into one transaction (e.g., a
|
||||||
|
parser that appends one event per parsed claim).
|
||||||
|
"""
|
||||||
|
# Read the latest hash within the same session so concurrent
|
||||||
|
# appends don't see stale state. SQLite default isolation level
|
||||||
|
# gives us serializable reads for this query; for Postgres we'd
|
||||||
|
# need SELECT ... FOR UPDATE but that's overkill for v1.
|
||||||
|
latest = (
|
||||||
|
session.query(AuditLog)
|
||||||
|
.order_by(AuditLog.id.desc())
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
prev_hash = latest.hash if latest is not None else GENESIS_PREV_HASH
|
||||||
|
|
||||||
|
created_at = event.created_at or datetime.now(timezone.utc)
|
||||||
|
if created_at.tzinfo is None:
|
||||||
|
created_at = created_at.replace(tzinfo=timezone.utc)
|
||||||
|
|
||||||
|
# Canonical payload form: sort_keys + compact separators. This
|
||||||
|
# makes the hash independent of dict insertion order across
|
||||||
|
# Python versions and across API runs.
|
||||||
|
payload_json = json.dumps(event.payload, sort_keys=True, separators=(",", ":")) if event.payload else None
|
||||||
|
|
||||||
|
# Insert the row with a placeholder hash, then UPDATE once we
|
||||||
|
# know the auto-assigned id. SQLite + SQLAlchemy gives us the id
|
||||||
|
# after the INSERT, so we can compute the real hash then.
|
||||||
|
row = AuditLog(
|
||||||
|
event_type=event.event_type,
|
||||||
|
entity_type=event.entity_type,
|
||||||
|
entity_id=event.entity_id,
|
||||||
|
actor=event.actor,
|
||||||
|
payload_json=payload_json,
|
||||||
|
created_at=created_at,
|
||||||
|
prev_hash=prev_hash,
|
||||||
|
hash=GENESIS_PREV_HASH, # placeholder; updated below
|
||||||
|
)
|
||||||
|
session.add(row)
|
||||||
|
session.flush() # populate row.id
|
||||||
|
row.hash = _hash_row(
|
||||||
|
row_id=row.id,
|
||||||
|
event_type=row.event_type,
|
||||||
|
entity_type=row.entity_type,
|
||||||
|
entity_id=row.entity_id,
|
||||||
|
actor=row.actor,
|
||||||
|
payload_json=row.payload_json,
|
||||||
|
created_at=row.created_at,
|
||||||
|
prev_hash=row.prev_hash,
|
||||||
|
)
|
||||||
|
session.flush()
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Verify
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class VerifyResult:
|
||||||
|
"""Outcome of :func:`verify_chain`."""
|
||||||
|
|
||||||
|
ok: bool
|
||||||
|
checked: int
|
||||||
|
first_bad_id: int | None = None
|
||||||
|
reason: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def verify_chain(session: Session) -> VerifyResult:
|
||||||
|
"""Walk the audit_log and verify every row's hash. Returns the first mismatch.
|
||||||
|
|
||||||
|
A clean chain returns ``VerifyResult(ok=True, checked=N)``. A
|
||||||
|
broken chain returns ``ok=False, first_bad_id=X, reason='...'``
|
||||||
|
describing what went wrong (hash mismatch, prev_hash mismatch,
|
||||||
|
or non-monotonic id).
|
||||||
|
|
||||||
|
This is intended to be called by the operator (e.g., a nightly
|
||||||
|
cron job or the admin UI's "Verify Audit Chain" button). It is
|
||||||
|
NOT a fast operation — for a 6-year-old chain with millions of
|
||||||
|
rows, expect seconds-to-minutes. Call it rarely.
|
||||||
|
"""
|
||||||
|
rows = session.query(AuditLog).order_by(AuditLog.id.asc()).all()
|
||||||
|
if not rows:
|
||||||
|
return VerifyResult(ok=True, checked=0)
|
||||||
|
|
||||||
|
expected_prev = GENESIS_PREV_HASH
|
||||||
|
last_id = 0
|
||||||
|
for i, row in enumerate(rows):
|
||||||
|
# Monotonic id check — covers attempted inserts with a
|
||||||
|
# custom id, or accidental out-of-order rows.
|
||||||
|
if row.id <= last_id:
|
||||||
|
return VerifyResult(
|
||||||
|
ok=False, checked=i, first_bad_id=row.id,
|
||||||
|
reason=f"non-monotonic id (previous={last_id}, this={row.id})",
|
||||||
|
)
|
||||||
|
last_id = row.id
|
||||||
|
|
||||||
|
# Recompute the hash from the row's content and compare.
|
||||||
|
expected_hash = _hash_row(
|
||||||
|
row_id=row.id,
|
||||||
|
event_type=row.event_type,
|
||||||
|
entity_type=row.entity_type,
|
||||||
|
entity_id=row.entity_id,
|
||||||
|
actor=row.actor,
|
||||||
|
payload_json=row.payload_json,
|
||||||
|
created_at=row.created_at,
|
||||||
|
prev_hash=row.prev_hash,
|
||||||
|
)
|
||||||
|
if expected_hash != row.hash:
|
||||||
|
return VerifyResult(
|
||||||
|
ok=False, checked=i, first_bad_id=row.id,
|
||||||
|
reason=f"hash mismatch (stored={row.hash[:16]}…, computed={expected_hash[:16]}…)",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Check prev_hash linkage.
|
||||||
|
if row.prev_hash != expected_prev:
|
||||||
|
return VerifyResult(
|
||||||
|
ok=False, checked=i, first_bad_id=row.id,
|
||||||
|
reason=f"prev_hash mismatch (stored={row.prev_hash[:16]}…, expected={expected_prev[:16]}…)",
|
||||||
|
)
|
||||||
|
expected_prev = row.hash
|
||||||
|
|
||||||
|
return VerifyResult(ok=True, checked=len(rows))
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AuditEvent",
|
||||||
|
"GENESIS_PREV_HASH",
|
||||||
|
"HASH_LEN",
|
||||||
|
"VerifyResult",
|
||||||
|
"append_event",
|
||||||
|
"verify_chain",
|
||||||
|
]
|
||||||
@@ -0,0 +1,280 @@
|
|||||||
|
"""SP17 — Encrypted backup primitives.
|
||||||
|
|
||||||
|
This module provides the low-level building blocks the rest of the
|
||||||
|
backup stack uses:
|
||||||
|
|
||||||
|
* ``derive_key`` — PBKDF2-HMAC-SHA256 key derivation (200,000
|
||||||
|
iterations, 32-byte output). The salt is per-backup, not global,
|
||||||
|
so identical passphrases produce different keys per backup.
|
||||||
|
|
||||||
|
* ``encrypt`` / ``decrypt`` — AES-256-GCM authenticated encryption.
|
||||||
|
Output layout: ``salt (16) | nonce (12) | ciphertext | tag (16)``.
|
||||||
|
The GCM tag is appended to the ciphertext by the cryptography
|
||||||
|
library; we don't prepend it.
|
||||||
|
|
||||||
|
* ``fingerprint`` — SHA-256 of a byte string, returned in the
|
||||||
|
``sha256:<hex>`` format we use across the codebase for DB keys and
|
||||||
|
audit events.
|
||||||
|
|
||||||
|
* ``BackupError`` / ``BackupDecryptError`` — typed exceptions so
|
||||||
|
callers can distinguish "wrong passphrase" from "I/O failed".
|
||||||
|
|
||||||
|
The crypto choices are deliberate:
|
||||||
|
|
||||||
|
* **AES-256-GCM** is the modern AEAD standard; the tag authenticates
|
||||||
|
both the ciphertext and the AAD (we pass an empty AAD; the
|
||||||
|
format itself is self-describing).
|
||||||
|
|
||||||
|
* **PBKDF2-HMAC-SHA256 @ 200k iters** is OWASP's 2023+ minimum for
|
||||||
|
PBKDF2-SHA256. Argon2id would be better but adds a C dependency;
|
||||||
|
PBKDF2 is stdlib via the ``cryptography`` package.
|
||||||
|
|
||||||
|
* **Random salt per backup** prevents rainbow-table attacks across
|
||||||
|
the operator's backup set.
|
||||||
|
|
||||||
|
* **Random 96-bit nonce per encryption** is what AES-GCM requires;
|
||||||
|
we use ``os.urandom`` which is a CSPRNG on every platform we run
|
||||||
|
on (macOS, Linux).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import os
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||||
|
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
|
||||||
|
from cryptography.hazmat.primitives import hashes
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Constants
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
# PBKDF2 iterations. OWASP 2023 minimum for PBKDF2-HMAC-SHA256 is
|
||||||
|
# 600,000; we use 200,000 as a balance between security and operator
|
||||||
|
# pain on the first backup creation (each backup does one KDF; on
|
||||||
|
# modern hardware 200k iters takes ~100ms). Bump this constant if you
|
||||||
|
# rotate the format version.
|
||||||
|
KDF_ITERATIONS = 200_000
|
||||||
|
|
||||||
|
# Salt + nonce sizes are AES-GCM / PBKDF2 standards, not negotiable.
|
||||||
|
SALT_LEN = 16
|
||||||
|
NONCE_LEN = 12
|
||||||
|
|
||||||
|
# Output key length for AES-256 = 32 bytes.
|
||||||
|
KEY_LEN = 32
|
||||||
|
|
||||||
|
# Format version. Bump when the on-disk layout changes (e.g. switch
|
||||||
|
# to Argon2id). Decryption reads this off the sidecar's
|
||||||
|
# encryption.kdf_iterations + cipher fields, not the version, so
|
||||||
|
# old backups remain decryptable until manually migrated.
|
||||||
|
FORMAT_VERSION = "v1"
|
||||||
|
|
||||||
|
# Fallback salt for the SQLCipher-key-derived backup key. Used only
|
||||||
|
# when the operator hasn't set a separate backup passphrase in the
|
||||||
|
# Keychain. This is a *constant* on purpose: the SQLCipher key is
|
||||||
|
# already random, so a fixed salt doesn't reduce entropy (the salt's
|
||||||
|
# job is to prevent rainbow tables, which require a *guessable*
|
||||||
|
# password; SQLCipher's key is unguessable).
|
||||||
|
FALLBACK_SALT = b"cyclone-db-backup-fallback-v1"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Exceptions
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class BackupError(Exception):
|
||||||
|
"""Generic backup failure. See BackupDecryptError for crypto errors."""
|
||||||
|
|
||||||
|
|
||||||
|
class BackupDecryptError(BackupError):
|
||||||
|
"""Decryption failed — wrong passphrase, tampered ciphertext, or
|
||||||
|
truncated file. Caller should NOT retry with the same key."""
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Key derivation + encryption
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def derive_key(passphrase: str, salt: bytes) -> bytes:
|
||||||
|
"""Derive a 32-byte AES key from a passphrase + salt.
|
||||||
|
|
||||||
|
Uses PBKDF2-HMAC-SHA256 with :data:`KDF_ITERATIONS` rounds. The
|
||||||
|
passphrase is encoded as UTF-8 bytes; the salt is used verbatim.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
passphrase: The operator's passphrase (any string).
|
||||||
|
salt: Per-backup random bytes of length :data:`SALT_LEN`.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
32 bytes suitable for AES-256-GCM.
|
||||||
|
"""
|
||||||
|
kdf = PBKDF2HMAC(
|
||||||
|
algorithm=hashes.SHA256(),
|
||||||
|
length=KEY_LEN,
|
||||||
|
salt=salt,
|
||||||
|
iterations=KDF_ITERATIONS,
|
||||||
|
)
|
||||||
|
return kdf.derive(passphrase.encode("utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def encrypt(plaintext: bytes, key: bytes) -> bytes:
|
||||||
|
"""AES-256-GCM encrypt with a fresh random 12-byte nonce.
|
||||||
|
|
||||||
|
Returns ``nonce (12) || ciphertext || tag (16)``. The
|
||||||
|
``cryptography`` library appends the tag automatically.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
plaintext: The bytes to encrypt (e.g. the SQLite .backup blob).
|
||||||
|
key: 32-byte AES key from :func:`derive_key`.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The combined nonce+ciphertext+tag blob.
|
||||||
|
"""
|
||||||
|
if len(key) != KEY_LEN:
|
||||||
|
raise BackupError(f"key must be {KEY_LEN} bytes; got {len(key)}")
|
||||||
|
nonce = os.urandom(NONCE_LEN)
|
||||||
|
aesgcm = AESGCM(key)
|
||||||
|
ciphertext = aesgcm.encrypt(nonce, plaintext, associated_data=None)
|
||||||
|
return nonce + ciphertext
|
||||||
|
|
||||||
|
|
||||||
|
def decrypt(blob: bytes, key: bytes) -> bytes:
|
||||||
|
"""AES-256-GCM decrypt. Raises :class:`BackupDecryptError` on auth failure.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
blob: The ``nonce||ciphertext||tag`` bytes from :func:`encrypt`.
|
||||||
|
key: The same 32-byte key used to encrypt.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The original plaintext.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
BackupDecryptError: If the blob is too short, the tag fails to
|
||||||
|
verify (wrong key or tampered ciphertext), or the input is
|
||||||
|
otherwise malformed.
|
||||||
|
"""
|
||||||
|
if len(key) != KEY_LEN:
|
||||||
|
raise BackupError(f"key must be {KEY_LEN} bytes; got {len(key)}")
|
||||||
|
if len(blob) < NONCE_LEN + 16:
|
||||||
|
# 12 (nonce) + 16 (tag) = minimum; no room for ciphertext.
|
||||||
|
raise BackupDecryptError(
|
||||||
|
f"blob too short ({len(blob)} bytes); expected >= {NONCE_LEN + 16}",
|
||||||
|
)
|
||||||
|
nonce = blob[:NONCE_LEN]
|
||||||
|
ciphertext = blob[NONCE_LEN:]
|
||||||
|
aesgcm = AESGCM(key)
|
||||||
|
try:
|
||||||
|
return aesgcm.decrypt(nonce, ciphertext, associated_data=None)
|
||||||
|
except Exception as exc: # cryptography raises InvalidTag
|
||||||
|
raise BackupDecryptError(f"decryption failed: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Fingerprint
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def fingerprint(data: bytes) -> str:
|
||||||
|
"""SHA-256 of ``data`` as ``"sha256:<64-hex-chars>"``."""
|
||||||
|
return "sha256:" + hashlib.sha256(data).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def fingerprint_file(path: "os.PathLike[str] | str") -> str:
|
||||||
|
"""SHA-256 of a file's bytes, streamed. Memory-bounded for big DBs."""
|
||||||
|
h = hashlib.sha256()
|
||||||
|
with open(path, "rb") as f:
|
||||||
|
for chunk in iter(lambda: f.read(1024 * 1024), b""):
|
||||||
|
h.update(chunk)
|
||||||
|
return "sha256:" + h.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Sidecar dataclass
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Sidecar:
|
||||||
|
"""Plaintext metadata written next to each backup file.
|
||||||
|
|
||||||
|
Not required for decryption — it's a manifest an operator
|
||||||
|
consults to decide whether to restore. Kept intentionally
|
||||||
|
tiny so it survives most format rotations.
|
||||||
|
"""
|
||||||
|
|
||||||
|
format_version: str
|
||||||
|
created_at: str # ISO 8601 UTC
|
||||||
|
db_fingerprint: str # "sha256:..."
|
||||||
|
table_count: int
|
||||||
|
size_bytes: int
|
||||||
|
kdf: str # "PBKDF2-HMAC-SHA256"
|
||||||
|
kdf_iterations: int
|
||||||
|
cipher: str # "AES-256-GCM"
|
||||||
|
key_fingerprint: str # "sha256:..." of the derived key
|
||||||
|
|
||||||
|
def to_json(self) -> str:
|
||||||
|
import json
|
||||||
|
return json.dumps(
|
||||||
|
{
|
||||||
|
"format_version": self.format_version,
|
||||||
|
"created_at": self.created_at,
|
||||||
|
"db_fingerprint": self.db_fingerprint,
|
||||||
|
"table_count": self.table_count,
|
||||||
|
"size_bytes": self.size_bytes,
|
||||||
|
"encryption": {
|
||||||
|
"kdf": self.kdf,
|
||||||
|
"kdf_iterations": self.kdf_iterations,
|
||||||
|
"cipher": self.cipher,
|
||||||
|
"key_fingerprint": self.key_fingerprint,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
indent=2,
|
||||||
|
sort_keys=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_json(cls, text: str) -> "Sidecar":
|
||||||
|
import json
|
||||||
|
d = json.loads(text)
|
||||||
|
enc = d.get("encryption") or {}
|
||||||
|
return cls(
|
||||||
|
format_version=d["format_version"],
|
||||||
|
created_at=d["created_at"],
|
||||||
|
db_fingerprint=d["db_fingerprint"],
|
||||||
|
table_count=int(d["table_count"]),
|
||||||
|
size_bytes=int(d["size_bytes"]),
|
||||||
|
kdf=enc.get("kdf", "PBKDF2-HMAC-SHA256"),
|
||||||
|
kdf_iterations=int(enc.get("kdf_iterations", KDF_ITERATIONS)),
|
||||||
|
cipher=enc.get("cipher", "AES-256-GCM"),
|
||||||
|
key_fingerprint=enc.get("key_fingerprint", ""),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Filename helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def backup_filename(timestamp: Optional[datetime] = None) -> str:
|
||||||
|
"""``cyclone-backup-YYYYMMDDTHHMMSSZ-<rand>.bin`` for a UTC timestamp.
|
||||||
|
|
||||||
|
The random suffix is a 4-byte hex string so two backups in the
|
||||||
|
same second don't collide on the ``db_backups`` unique index.
|
||||||
|
"""
|
||||||
|
import secrets as _secrets
|
||||||
|
from datetime import timezone as _tz
|
||||||
|
ts = timestamp or datetime.now(_tz.utc)
|
||||||
|
suffix = _secrets.token_hex(4)
|
||||||
|
return f"cyclone-backup-{ts.strftime('%Y%m%dT%H%M%SZ')}-{suffix}.bin"
|
||||||
|
|
||||||
|
|
||||||
|
def sidecar_filename(bin_filename: str) -> str:
|
||||||
|
"""``<bin_filename>.meta.json``."""
|
||||||
|
return bin_filename + ".meta.json"
|
||||||
@@ -0,0 +1,368 @@
|
|||||||
|
"""SP17 — Backup scheduler.
|
||||||
|
|
||||||
|
Wraps :class:`cyclone.backup_service.BackupService` in an
|
||||||
|
asyncio task, mirroring the MFT scheduler pattern (SP16). A backup
|
||||||
|
tick:
|
||||||
|
|
||||||
|
1. Calls :meth:`BackupService.create_now` to take + encrypt a backup.
|
||||||
|
2. Calls :meth:`BackupService.prune` to apply the retention policy.
|
||||||
|
3. Writes a tamper-evident ``audit_log`` row (SP11) for each outcome.
|
||||||
|
|
||||||
|
The scheduler is OFF by default. Operators opt in via
|
||||||
|
``CYCLONE_BACKUP_AUTOSTART=true``. The poll interval is
|
||||||
|
``CYCLONE_BACKUP_INTERVAL_HOURS`` (default 24).
|
||||||
|
|
||||||
|
Like the MFT scheduler, this is single-asyncio-task — no
|
||||||
|
threading, no APScheduler. All access (start/stop/tick/status)
|
||||||
|
must happen on the same event loop; the FastAPI app satisfies
|
||||||
|
that trivially because endpoints run on the loop.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import traceback
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone.audit_log import AuditEvent, append_event
|
||||||
|
from cyclone.backup_service import BackupService
|
||||||
|
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class BackupTickResult:
|
||||||
|
"""Outcome of a single backup tick (one cycle of create + prune + audit)."""
|
||||||
|
|
||||||
|
started_at: datetime
|
||||||
|
finished_at: Optional[datetime] = None
|
||||||
|
created: Optional[svc_mod.BackupRecord] = None
|
||||||
|
pruned_paths: list[str] = field(default_factory=list)
|
||||||
|
error: Optional[str] = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ok(self) -> bool:
|
||||||
|
return self.error is None and self.created is not None
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"started_at": self.started_at.isoformat(),
|
||||||
|
"finished_at": (
|
||||||
|
self.finished_at.isoformat() if self.finished_at else None
|
||||||
|
),
|
||||||
|
"ok": self.ok,
|
||||||
|
"created": (
|
||||||
|
{
|
||||||
|
"id": self.created.id,
|
||||||
|
"filename": self.created.filename,
|
||||||
|
"size_bytes": self.created.size_bytes,
|
||||||
|
"db_fingerprint": self.created.db_fingerprint,
|
||||||
|
"table_count": self.created.table_count,
|
||||||
|
}
|
||||||
|
if self.created else None
|
||||||
|
),
|
||||||
|
"pruned_paths": list(self.pruned_paths),
|
||||||
|
"error": self.error,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class BackupSchedulerStatus:
|
||||||
|
running: bool
|
||||||
|
interval_hours: float
|
||||||
|
backup_dir: str
|
||||||
|
retention_days: int
|
||||||
|
last_tick: Optional[BackupTickResult] = None
|
||||||
|
tick_count: int = 0
|
||||||
|
total_created: int = 0
|
||||||
|
total_errors: int = 0
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"running": self.running,
|
||||||
|
"interval_hours": self.interval_hours,
|
||||||
|
"backup_dir": self.backup_dir,
|
||||||
|
"retention_days": self.retention_days,
|
||||||
|
"tick_count": self.tick_count,
|
||||||
|
"total_created": self.total_created,
|
||||||
|
"total_errors": self.total_errors,
|
||||||
|
"last_tick": self.last_tick.as_dict() if self.last_tick else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class BackupScheduler:
|
||||||
|
"""Asyncio loop that ticks the BackupService on an interval.
|
||||||
|
|
||||||
|
Lifecycle mirrors :class:`cyclone.scheduler.Scheduler`:
|
||||||
|
|
||||||
|
sched = BackupScheduler(backup_service)
|
||||||
|
await sched.start() # begin ticking
|
||||||
|
await sched.stop() # finish current tick, exit
|
||||||
|
status = sched.status() # snapshot
|
||||||
|
|
||||||
|
Threading: NOT thread-safe. All access must happen on the
|
||||||
|
same event loop. FastAPI endpoints satisfy this automatically.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
service: BackupService,
|
||||||
|
*,
|
||||||
|
interval_hours: float = 24.0,
|
||||||
|
) -> None:
|
||||||
|
self._service = service
|
||||||
|
self._interval_hours = max(0.1, float(interval_hours))
|
||||||
|
self._task: Optional[asyncio.Task[None]] = None
|
||||||
|
self._stop_event = asyncio.Event()
|
||||||
|
self._tick_in_progress = False
|
||||||
|
self._last_tick: Optional[BackupTickResult] = None
|
||||||
|
self._tick_count = 0
|
||||||
|
self._total_created = 0
|
||||||
|
self._total_errors = 0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def service(self) -> BackupService:
|
||||||
|
return self._service
|
||||||
|
|
||||||
|
# ---- Public API -------------------------------------------------------
|
||||||
|
|
||||||
|
async def start(self) -> None:
|
||||||
|
if self._task is not None and not self._task.done():
|
||||||
|
log.info("BackupScheduler already running; start() is a no-op")
|
||||||
|
return
|
||||||
|
self._stop_event.clear()
|
||||||
|
self._task = asyncio.create_task(self._run(), name="backup-scheduler")
|
||||||
|
log.info(
|
||||||
|
"BackupScheduler started",
|
||||||
|
extra={
|
||||||
|
"interval_hours": self._interval_hours,
|
||||||
|
"backup_dir": str(self._service.backup_dir),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
async def stop(self) -> None:
|
||||||
|
if self._task is None or self._task.done():
|
||||||
|
return
|
||||||
|
self._stop_event.set()
|
||||||
|
try:
|
||||||
|
await asyncio.wait_for(self._task, timeout=60)
|
||||||
|
except asyncio.TimeoutError:
|
||||||
|
log.warning("BackupScheduler did not stop within 60s; cancelling")
|
||||||
|
self._task.cancel()
|
||||||
|
try:
|
||||||
|
await self._task
|
||||||
|
except (asyncio.CancelledError, Exception): # noqa: BLE001
|
||||||
|
pass
|
||||||
|
self._task = None
|
||||||
|
log.info("BackupScheduler stopped")
|
||||||
|
|
||||||
|
def status(self) -> BackupSchedulerStatus:
|
||||||
|
return BackupSchedulerStatus(
|
||||||
|
running=self.is_running(),
|
||||||
|
interval_hours=self._interval_hours,
|
||||||
|
backup_dir=str(self._service.backup_dir),
|
||||||
|
retention_days=self._service._retention_days,
|
||||||
|
last_tick=self._last_tick,
|
||||||
|
tick_count=self._tick_count,
|
||||||
|
total_created=self._total_created,
|
||||||
|
total_errors=self._total_errors,
|
||||||
|
)
|
||||||
|
|
||||||
|
def is_running(self) -> bool:
|
||||||
|
return self._task is not None and not self._task.done()
|
||||||
|
|
||||||
|
async def tick(self) -> BackupTickResult:
|
||||||
|
"""Run a single backup tick (create + prune + audit).
|
||||||
|
|
||||||
|
Concurrent ticks are coalesced: if a tick is already in
|
||||||
|
progress, the second caller waits for it. This protects
|
||||||
|
against a slow backup holding up multiple operator-driven
|
||||||
|
``POST /api/admin/backup/tick`` calls.
|
||||||
|
"""
|
||||||
|
while self._tick_in_progress:
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
self._tick_in_progress = True
|
||||||
|
try:
|
||||||
|
result = await self._tick_impl()
|
||||||
|
self._last_tick = result
|
||||||
|
self._tick_count += 1
|
||||||
|
if result.created is not None:
|
||||||
|
self._total_created += 1
|
||||||
|
if result.error is not None:
|
||||||
|
self._total_errors += 1
|
||||||
|
return result
|
||||||
|
finally:
|
||||||
|
self._tick_in_progress = False
|
||||||
|
|
||||||
|
# ---- Internals --------------------------------------------------------
|
||||||
|
|
||||||
|
async def _run(self) -> None:
|
||||||
|
# Stagger the first tick (same rationale as the MFT scheduler).
|
||||||
|
await asyncio.sleep(5)
|
||||||
|
while not self._stop_event.is_set():
|
||||||
|
try:
|
||||||
|
await self.tick()
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
# tick() catches its own exceptions and returns them
|
||||||
|
# in the result. This is the safety net for
|
||||||
|
# programmer errors in the loop body.
|
||||||
|
log.exception("BackupScheduler tick raised", extra={"error": str(exc)})
|
||||||
|
try:
|
||||||
|
await asyncio.wait_for(
|
||||||
|
self._stop_event.wait(),
|
||||||
|
timeout=self._interval_hours * 3600,
|
||||||
|
)
|
||||||
|
except asyncio.TimeoutError:
|
||||||
|
pass # interval elapsed
|
||||||
|
|
||||||
|
async def _tick_impl(self) -> BackupTickResult:
|
||||||
|
started = datetime.now(timezone.utc)
|
||||||
|
result = BackupTickResult(started_at=started)
|
||||||
|
try:
|
||||||
|
create_result = await asyncio.to_thread(self._service.create_now)
|
||||||
|
result.created = create_result.backup
|
||||||
|
# Audit event for the created backup.
|
||||||
|
await asyncio.to_thread(
|
||||||
|
_audit_backup_created,
|
||||||
|
create_result.backup.id,
|
||||||
|
create_result.backup.db_fingerprint,
|
||||||
|
create_result.backup.table_count,
|
||||||
|
"backup-scheduler",
|
||||||
|
)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
log.exception("Backup create failed during tick")
|
||||||
|
result.error = f"create: {type(exc).__name__}: {exc}"
|
||||||
|
await asyncio.to_thread(
|
||||||
|
_audit_backup_failed,
|
||||||
|
f"create: {type(exc).__name__}: {exc}",
|
||||||
|
traceback.format_exc()[-500:],
|
||||||
|
"backup-scheduler",
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
pruned = await asyncio.to_thread(self._service.prune)
|
||||||
|
result.pruned_paths = pruned
|
||||||
|
if pruned:
|
||||||
|
await asyncio.to_thread(
|
||||||
|
_audit_backup_pruned, pruned, "backup-scheduler",
|
||||||
|
)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
log.exception("Backup prune failed during tick")
|
||||||
|
# Don't clobber the create error if there was one.
|
||||||
|
if result.error is None:
|
||||||
|
result.error = f"prune: {type(exc).__name__}: {exc}"
|
||||||
|
await asyncio.to_thread(
|
||||||
|
_audit_backup_failed,
|
||||||
|
f"prune: {type(exc).__name__}: {exc}",
|
||||||
|
traceback.format_exc()[-500:],
|
||||||
|
"backup-scheduler",
|
||||||
|
)
|
||||||
|
|
||||||
|
result.finished_at = datetime.now(timezone.utc)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Audit helpers (run in a thread so the asyncio loop doesn't block)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_backup_created(
|
||||||
|
backup_id: int, db_fingerprint: str, table_count: int, actor: str,
|
||||||
|
) -> None:
|
||||||
|
from cyclone import db
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
try:
|
||||||
|
append_event(s, AuditEvent(
|
||||||
|
event_type="db.backup_created",
|
||||||
|
entity_type="database",
|
||||||
|
entity_id="cyclone.db",
|
||||||
|
actor=actor,
|
||||||
|
payload={
|
||||||
|
"backup_id": backup_id,
|
||||||
|
"db_fingerprint": db_fingerprint,
|
||||||
|
"table_count": table_count,
|
||||||
|
},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
log.exception("Failed to write db.backup_created audit event")
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_backup_failed(
|
||||||
|
reason: str, traceback_tail: str, actor: str,
|
||||||
|
) -> None:
|
||||||
|
from cyclone import db
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
try:
|
||||||
|
append_event(s, AuditEvent(
|
||||||
|
event_type="db.backup_failed",
|
||||||
|
entity_type="database",
|
||||||
|
entity_id="cyclone.db",
|
||||||
|
actor=actor,
|
||||||
|
payload={"reason": reason, "traceback_tail": traceback_tail},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
log.exception("Failed to write db.backup_failed audit event")
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_backup_pruned(deleted_paths: list[str], actor: str) -> None:
|
||||||
|
from cyclone import db
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
try:
|
||||||
|
append_event(s, AuditEvent(
|
||||||
|
event_type="db.backup_pruned",
|
||||||
|
entity_type="database",
|
||||||
|
entity_id="cyclone.db",
|
||||||
|
actor=actor,
|
||||||
|
payload={"deleted_paths": deleted_paths},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
log.exception("Failed to write db.backup_pruned audit event")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Module-level singleton
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
_scheduler: Optional[BackupScheduler] = None
|
||||||
|
|
||||||
|
|
||||||
|
def configure_backup_scheduler(
|
||||||
|
service: BackupService,
|
||||||
|
*,
|
||||||
|
interval_hours: float = 24.0,
|
||||||
|
) -> BackupScheduler:
|
||||||
|
"""Create (or return existing) the module-level BackupScheduler."""
|
||||||
|
global _scheduler
|
||||||
|
if _scheduler is not None:
|
||||||
|
return _scheduler
|
||||||
|
hours = float(
|
||||||
|
os.environ.get("CYCLONE_BACKUP_INTERVAL_HOURS", interval_hours),
|
||||||
|
)
|
||||||
|
_scheduler = BackupScheduler(service, interval_hours=hours)
|
||||||
|
return _scheduler
|
||||||
|
|
||||||
|
|
||||||
|
def get_backup_scheduler() -> BackupScheduler:
|
||||||
|
"""Return the configured BackupScheduler. Raises if not set up."""
|
||||||
|
if _scheduler is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
"backup scheduler not configured; call configure_backup_scheduler() first",
|
||||||
|
)
|
||||||
|
return _scheduler
|
||||||
|
|
||||||
|
|
||||||
|
def reset_backup_scheduler_for_tests() -> None:
|
||||||
|
"""Clear the module-level singleton. Test-only."""
|
||||||
|
global _scheduler
|
||||||
|
_scheduler = None
|
||||||
@@ -0,0 +1,851 @@
|
|||||||
|
"""SP17 — High-level backup coordinator.
|
||||||
|
|
||||||
|
Owns the lifecycle of every backup the operator (or the scheduler)
|
||||||
|
takes:
|
||||||
|
|
||||||
|
* ``create_now`` — runs SQLite's online ``.backup()`` against the
|
||||||
|
live engine, encrypts the bytes with :func:`cyclone.backup.encrypt`,
|
||||||
|
writes a ``.bin`` + ``.meta.json`` pair into the backup directory,
|
||||||
|
and persists a row in ``db_backups``.
|
||||||
|
|
||||||
|
* ``list_backups`` — directory listing joined with ``db_backups`` rows.
|
||||||
|
|
||||||
|
* ``verify`` — decrypts + recomputes SHA-256, compares to the sidecar.
|
||||||
|
|
||||||
|
* ``restore_initiate`` / ``restore_confirm`` — two-step restore so an
|
||||||
|
idle browser tab can't nuke the live DB. The first call returns a
|
||||||
|
``restore_token`` (a random 32-byte hex string) plus a preview
|
||||||
|
(``db_fingerprint``, ``table_count``). The second call swaps the
|
||||||
|
engine only if the token matches.
|
||||||
|
|
||||||
|
* ``prune`` — deletes backups older than ``retention_days``.
|
||||||
|
|
||||||
|
This module is intentionally engine-aware: ``create_now`` reaches
|
||||||
|
into the live SQLAlchemy engine to get a raw SQLite connection and
|
||||||
|
call ``.backup()`` (the only way to take an online consistent
|
||||||
|
snapshot). ``restore`` reaches into :func:`cyclone.db.dispose_engine`
|
||||||
|
+ :func:`cyclone.db.reinit_engine` to swap to the restored file.
|
||||||
|
|
||||||
|
The encryption key is loaded once at construction time:
|
||||||
|
|
||||||
|
* If a backup passphrase is set in the Keychain (``backup.passphrase``
|
||||||
|
account under service ``cyclone``), use it directly with the salt
|
||||||
|
stored in the companion ``backup.salt`` account. The salt must be
|
||||||
|
persisted — a fresh random salt per process would defeat the key.
|
||||||
|
* Otherwise fall back to deriving from the SQLCipher DB key + a fixed
|
||||||
|
salt. Logged at WARNING because this is a degraded-but-still-safe
|
||||||
|
posture.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import secrets as _secrets
|
||||||
|
import shutil
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional, Union
|
||||||
|
|
||||||
|
from sqlalchemy.exc import SQLAlchemyError
|
||||||
|
|
||||||
|
from cyclone import backup as backup_mod
|
||||||
|
from cyclone.backup import BackupError
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# Status values for db_backups.status (mirrored in the ORM).
|
||||||
|
STATUS_PENDING = "pending"
|
||||||
|
STATUS_OK = "ok"
|
||||||
|
STATUS_ERROR = "error"
|
||||||
|
STATUS_PRUNED = "pruned"
|
||||||
|
|
||||||
|
# Where the operator's backup passphrase lives in the Keychain.
|
||||||
|
KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT = "backup.passphrase"
|
||||||
|
|
||||||
|
# Companion account for the salt. Stored as hex. Same value across
|
||||||
|
# processes so the derived key is reproducible — a fresh random salt
|
||||||
|
# per BackupService would defeat the key.
|
||||||
|
KEYCHAIN_BACKUP_SALT_ACCOUNT = "backup.salt"
|
||||||
|
|
||||||
|
# Restore token TTL (seconds). The two-step confirm must complete
|
||||||
|
# within this window or the operator re-runs initiate.
|
||||||
|
RESTORE_TOKEN_TTL_SECONDS = 300
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Result dataclasses
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class BackupRecord:
|
||||||
|
"""Public view of a backup row joined with filesystem state."""
|
||||||
|
|
||||||
|
id: int
|
||||||
|
filename: str
|
||||||
|
backup_dir: str
|
||||||
|
size_bytes: int
|
||||||
|
db_fingerprint: str
|
||||||
|
table_count: int
|
||||||
|
created_at: datetime
|
||||||
|
completed_at: Optional[datetime]
|
||||||
|
status: str
|
||||||
|
error_message: Optional[str]
|
||||||
|
key_fingerprint: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CreateResult:
|
||||||
|
"""Outcome of ``create_now``."""
|
||||||
|
|
||||||
|
backup: BackupRecord
|
||||||
|
sidecar: backup_mod.Sidecar
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class VerifyResult:
|
||||||
|
"""Outcome of ``verify``."""
|
||||||
|
|
||||||
|
backup_id: int
|
||||||
|
filename: str
|
||||||
|
ok: bool
|
||||||
|
expected_fingerprint: str
|
||||||
|
actual_fingerprint: str
|
||||||
|
table_count: int
|
||||||
|
reason: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RestoreInitiateResult:
|
||||||
|
"""Returned by the first call of the two-step restore."""
|
||||||
|
|
||||||
|
backup_id: int
|
||||||
|
filename: str
|
||||||
|
restore_token: str
|
||||||
|
expires_at: datetime
|
||||||
|
db_fingerprint: str
|
||||||
|
table_count: int
|
||||||
|
current_db_fingerprint: str
|
||||||
|
current_table_count: int
|
||||||
|
size_bytes: int
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RestoreConfirmResult:
|
||||||
|
"""Returned by the second call of the two-step restore."""
|
||||||
|
|
||||||
|
backup_id: int
|
||||||
|
filename: str
|
||||||
|
restored_from_fingerprint: str
|
||||||
|
restored_at: datetime
|
||||||
|
new_db_fingerprint: str
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# BackupService
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class BackupService:
|
||||||
|
"""Coordinator for encrypted DB backups.
|
||||||
|
|
||||||
|
Construct once at app startup; share across requests. Not
|
||||||
|
thread-safe for *creation* (the SQLite ``.backup()`` call uses
|
||||||
|
the live engine and is best serialized through the scheduler),
|
||||||
|
but ``list_backups`` / ``prune`` / ``status`` are safe to call
|
||||||
|
concurrently.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
backup_dir: Union[str, Path],
|
||||||
|
*,
|
||||||
|
passphrase: Optional[str] = None,
|
||||||
|
salt: Optional[bytes] = None,
|
||||||
|
retention_days: int = 30,
|
||||||
|
db_url: Optional[str] = None,
|
||||||
|
) -> None:
|
||||||
|
self._backup_dir = Path(backup_dir)
|
||||||
|
self._retention_days = max(1, int(retention_days))
|
||||||
|
self._db_url = db_url
|
||||||
|
# The derived key + its salt. If ``passphrase`` is None we
|
||||||
|
# fall back to deriving from the SQLCipher DB key (with a
|
||||||
|
# WARNING log).
|
||||||
|
#
|
||||||
|
# Salt is per-BackupService-instance and MUST be stable across
|
||||||
|
# processes — otherwise the same passphrase would derive
|
||||||
|
# different keys in different invocations and decrypt would
|
||||||
|
# always fail. Two options:
|
||||||
|
#
|
||||||
|
# 1. Caller passes an explicit ``salt`` (the Keychain flow
|
||||||
|
# reads the persisted salt from backup.salt account).
|
||||||
|
# 2. We accept a None salt here; ``_ensure_key`` then either
|
||||||
|
# uses the persisted salt (if available) or generates one
|
||||||
|
# and persists it on first use.
|
||||||
|
#
|
||||||
|
# Tests typically pass an explicit random salt; production
|
||||||
|
# should always pass the persisted one.
|
||||||
|
self._passphrase = passphrase
|
||||||
|
self._salt = salt
|
||||||
|
self._key: Optional[bytes] = None
|
||||||
|
self._used_fallback = False
|
||||||
|
# Pending restore tokens: token -> (backup_id, expires_at).
|
||||||
|
# A simple in-memory dict is sufficient — the token only
|
||||||
|
# needs to survive between the two API calls in one process.
|
||||||
|
self._pending_restores: dict[str, tuple[int, datetime]] = {}
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
|
||||||
|
# ---- Public API -------------------------------------------------------
|
||||||
|
|
||||||
|
@property
|
||||||
|
def backup_dir(self) -> Path:
|
||||||
|
return self._backup_dir
|
||||||
|
|
||||||
|
@property
|
||||||
|
def key_fingerprint(self) -> str:
|
||||||
|
"""SHA-256 of the current derived key, or "" if not yet derived."""
|
||||||
|
if self._key is None:
|
||||||
|
return ""
|
||||||
|
return backup_mod.fingerprint(self._key)
|
||||||
|
|
||||||
|
def create_now(self) -> CreateResult:
|
||||||
|
"""Take an encrypted backup of the live DB right now.
|
||||||
|
|
||||||
|
Crash-safe: any failure marks the ``db_backups`` row as
|
||||||
|
``error``, removes any partial files from the backup dir, and
|
||||||
|
re-raises the exception.
|
||||||
|
"""
|
||||||
|
# 1. Make sure the backup dir exists.
|
||||||
|
self._backup_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# 2. Allocate a filename + insert a pending row.
|
||||||
|
from cyclone.db import DbBackup # late import — circular otherwise
|
||||||
|
from cyclone.store import store as cycl_store
|
||||||
|
|
||||||
|
filename = backup_mod.backup_filename()
|
||||||
|
created_at = datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
row = cycl_store.add_backup_pending(
|
||||||
|
filename=filename,
|
||||||
|
backup_dir=str(self._backup_dir),
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
# 3. Run SQLite's online .backup() to a temp file.
|
||||||
|
# We use a private path *inside* the backup dir so the
|
||||||
|
# operator can see what crashed if it does.
|
||||||
|
staging_db = self._backup_dir / f".{filename}.staging.db"
|
||||||
|
self._sqlite_backup_to(staging_db)
|
||||||
|
|
||||||
|
# 4. Encrypt.
|
||||||
|
plaintext = staging_db.read_bytes()
|
||||||
|
db_fp = backup_mod.fingerprint(plaintext)
|
||||||
|
key = self._ensure_key()
|
||||||
|
blob = backup_mod.encrypt(plaintext, key)
|
||||||
|
|
||||||
|
# 5. Move encrypted blob into place + write sidecar.
|
||||||
|
target = self._backup_dir / filename
|
||||||
|
target.write_bytes(blob)
|
||||||
|
staging_db.unlink()
|
||||||
|
|
||||||
|
table_count = self._count_tables_in_blob(plaintext)
|
||||||
|
sidecar = backup_mod.Sidecar(
|
||||||
|
format_version=backup_mod.FORMAT_VERSION,
|
||||||
|
created_at=created_at.isoformat(),
|
||||||
|
db_fingerprint=db_fp,
|
||||||
|
table_count=table_count,
|
||||||
|
size_bytes=len(blob),
|
||||||
|
kdf="PBKDF2-HMAC-SHA256",
|
||||||
|
kdf_iterations=backup_mod.KDF_ITERATIONS,
|
||||||
|
cipher="AES-256-GCM",
|
||||||
|
key_fingerprint=backup_mod.fingerprint(key),
|
||||||
|
)
|
||||||
|
sidecar_path = self._backup_dir / backup_mod.sidecar_filename(filename)
|
||||||
|
sidecar_path.write_text(sidecar.to_json())
|
||||||
|
|
||||||
|
# 6. Mark the row as ok.
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(DbBackup, row.id)
|
||||||
|
row.status = STATUS_OK
|
||||||
|
row.size_bytes = len(blob)
|
||||||
|
row.db_fingerprint = db_fp
|
||||||
|
row.table_count = table_count
|
||||||
|
row.completed_at = datetime.now(timezone.utc)
|
||||||
|
s.commit()
|
||||||
|
s.refresh(row)
|
||||||
|
|
||||||
|
record = self._row_to_record(row)
|
||||||
|
log.info(
|
||||||
|
"Backup created",
|
||||||
|
extra={
|
||||||
|
"backup_id": record.id,
|
||||||
|
"backup_filename": record.filename,
|
||||||
|
"size_bytes": record.size_bytes,
|
||||||
|
"db_fingerprint": record.db_fingerprint,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return CreateResult(backup=record, sidecar=sidecar)
|
||||||
|
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
log.exception("Backup create failed")
|
||||||
|
try:
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(DbBackup, row.id)
|
||||||
|
row.status = STATUS_ERROR
|
||||||
|
row.error_message = f"{type(exc).__name__}: {exc}"[:500]
|
||||||
|
row.completed_at = datetime.now(timezone.utc)
|
||||||
|
s.commit()
|
||||||
|
# Best-effort cleanup of any partial files.
|
||||||
|
for p in [
|
||||||
|
self._backup_dir / filename,
|
||||||
|
self._backup_dir / f".{filename}.staging.db",
|
||||||
|
self._backup_dir / backup_mod.sidecar_filename(filename),
|
||||||
|
]:
|
||||||
|
if p.exists():
|
||||||
|
try:
|
||||||
|
p.unlink()
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
log.exception("Failed to mark backup row as error")
|
||||||
|
raise
|
||||||
|
|
||||||
|
def list_backups(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
limit: int = 100,
|
||||||
|
status: Optional[str] = None,
|
||||||
|
) -> list[BackupRecord]:
|
||||||
|
"""List ``db_backups`` rows newest first.
|
||||||
|
|
||||||
|
Joins the filesystem state (presence of ``.bin`` and
|
||||||
|
``.meta.json``) implicitly via :attr:`BackupRecord.status`:
|
||||||
|
a row marked ``pruned`` had its files deleted by the
|
||||||
|
retention policy.
|
||||||
|
"""
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
q = s.query(DbBackup)
|
||||||
|
if status is not None:
|
||||||
|
q = q.filter(DbBackup.status == status)
|
||||||
|
rows = q.order_by(DbBackup.id.desc()).limit(limit).all()
|
||||||
|
return [self._row_to_record(r) for r in rows]
|
||||||
|
|
||||||
|
def verify(self, backup_id: int) -> VerifyResult:
|
||||||
|
"""Decrypt + checksum-verify a backup against its sidecar.
|
||||||
|
|
||||||
|
Does NOT trust the sidecar's ``db_fingerprint`` field alone;
|
||||||
|
recomputes the SHA-256 from the decrypted blob and compares.
|
||||||
|
"""
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(DbBackup, backup_id)
|
||||||
|
if row is None:
|
||||||
|
raise BackupError(f"backup {backup_id} not found")
|
||||||
|
record = self._row_to_record(row)
|
||||||
|
sidecar = self._read_sidecar(record.filename)
|
||||||
|
if sidecar is None:
|
||||||
|
return VerifyResult(
|
||||||
|
backup_id=record.id, filename=record.filename,
|
||||||
|
ok=False, expected_fingerprint="", actual_fingerprint="",
|
||||||
|
table_count=0, reason="sidecar missing",
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
blob = (self._backup_dir / record.filename).read_bytes()
|
||||||
|
except FileNotFoundError:
|
||||||
|
return VerifyResult(
|
||||||
|
backup_id=record.id, filename=record.filename,
|
||||||
|
ok=False,
|
||||||
|
expected_fingerprint=sidecar.db_fingerprint,
|
||||||
|
actual_fingerprint="",
|
||||||
|
table_count=sidecar.table_count,
|
||||||
|
reason="backup file missing",
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
plaintext = backup_mod.decrypt(blob, self._ensure_key())
|
||||||
|
except backup_mod.BackupDecryptError as exc:
|
||||||
|
return VerifyResult(
|
||||||
|
backup_id=record.id, filename=record.filename,
|
||||||
|
ok=False,
|
||||||
|
expected_fingerprint=sidecar.db_fingerprint,
|
||||||
|
actual_fingerprint="",
|
||||||
|
table_count=sidecar.table_count,
|
||||||
|
reason=str(exc),
|
||||||
|
)
|
||||||
|
|
||||||
|
actual_fp = backup_mod.fingerprint(plaintext)
|
||||||
|
return VerifyResult(
|
||||||
|
backup_id=record.id,
|
||||||
|
filename=record.filename,
|
||||||
|
ok=(actual_fp == sidecar.db_fingerprint),
|
||||||
|
expected_fingerprint=sidecar.db_fingerprint,
|
||||||
|
actual_fingerprint=actual_fp,
|
||||||
|
table_count=sidecar.table_count,
|
||||||
|
reason=None if actual_fp == sidecar.db_fingerprint else "fingerprint mismatch",
|
||||||
|
)
|
||||||
|
|
||||||
|
def restore_initiate(self, backup_id: int) -> RestoreInitiateResult:
|
||||||
|
"""First half of the two-step restore.
|
||||||
|
|
||||||
|
Decrypts the backup into a temp file and reads its current
|
||||||
|
``db_fingerprint`` + ``table_count``. Returns a one-shot
|
||||||
|
``restore_token`` the operator must echo back to
|
||||||
|
:meth:`restore_confirm` within 5 minutes.
|
||||||
|
"""
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(DbBackup, backup_id)
|
||||||
|
if row is None:
|
||||||
|
raise BackupError(f"backup {backup_id} not found")
|
||||||
|
if row.status != STATUS_OK:
|
||||||
|
raise BackupError(
|
||||||
|
f"backup {backup_id} status is {row.status!r}; only 'ok' backups can be restored",
|
||||||
|
)
|
||||||
|
record = self._row_to_record(row)
|
||||||
|
|
||||||
|
# Decrypt into a staging file so the confirm step is fast.
|
||||||
|
staging = self._backup_dir / f".restore-{record.filename}.staging.db"
|
||||||
|
try:
|
||||||
|
blob = (self._backup_dir / record.filename).read_bytes()
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise BackupError(f"backup file missing: {record.filename}") from exc
|
||||||
|
try:
|
||||||
|
plaintext = backup_mod.decrypt(blob, self._ensure_key())
|
||||||
|
except backup_mod.BackupDecryptError as exc:
|
||||||
|
raise BackupError(f"decrypt failed: {exc}") from exc
|
||||||
|
staging.write_bytes(plaintext)
|
||||||
|
|
||||||
|
# Snapshot the live DB's fingerprint for the operator's "are
|
||||||
|
# you sure you want to do this?" preview.
|
||||||
|
live_fp, live_count = self._live_fingerprint_and_count()
|
||||||
|
|
||||||
|
token = _secrets.token_hex(32)
|
||||||
|
expires_at = datetime.now(timezone.utc) + timedelta(
|
||||||
|
seconds=RESTORE_TOKEN_TTL_SECONDS,
|
||||||
|
)
|
||||||
|
with self._lock:
|
||||||
|
self._pending_restores[token] = (record.id, expires_at)
|
||||||
|
|
||||||
|
log.info(
|
||||||
|
"Restore initiated",
|
||||||
|
extra={
|
||||||
|
"backup_id": record.id,
|
||||||
|
"token_prefix": token[:8],
|
||||||
|
"expires_at": expires_at.isoformat(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return RestoreInitiateResult(
|
||||||
|
backup_id=record.id,
|
||||||
|
filename=record.filename,
|
||||||
|
restore_token=token,
|
||||||
|
expires_at=expires_at,
|
||||||
|
db_fingerprint=backup_mod.fingerprint(plaintext),
|
||||||
|
table_count=self._count_tables_in_blob(plaintext),
|
||||||
|
current_db_fingerprint=live_fp,
|
||||||
|
current_table_count=live_count,
|
||||||
|
size_bytes=len(plaintext),
|
||||||
|
)
|
||||||
|
|
||||||
|
def restore_confirm(
|
||||||
|
self,
|
||||||
|
backup_id: int,
|
||||||
|
restore_token: str,
|
||||||
|
*,
|
||||||
|
actor: str = "operator",
|
||||||
|
) -> RestoreConfirmResult:
|
||||||
|
"""Second half of the two-step restore.
|
||||||
|
|
||||||
|
Validates the token, copies the decrypted staging file over
|
||||||
|
the live DB path, disposes + reopens the engine. Raises
|
||||||
|
``BackupError`` on any mismatch.
|
||||||
|
"""
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
with self._lock:
|
||||||
|
entry = self._pending_restores.pop(restore_token, None)
|
||||||
|
if entry is None:
|
||||||
|
raise BackupError("restore_token not found (already consumed or never issued)")
|
||||||
|
token_backup_id, expires_at = entry
|
||||||
|
if token_backup_id != backup_id:
|
||||||
|
raise BackupError(
|
||||||
|
f"restore_token was for backup {token_backup_id}, not {backup_id}",
|
||||||
|
)
|
||||||
|
if now > expires_at:
|
||||||
|
raise BackupError(
|
||||||
|
f"restore_token expired at {expires_at.isoformat()}; re-run initiate",
|
||||||
|
)
|
||||||
|
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(DbBackup, backup_id)
|
||||||
|
if row is None:
|
||||||
|
raise BackupError(f"backup {backup_id} disappeared mid-restore")
|
||||||
|
record = self._row_to_record(row)
|
||||||
|
|
||||||
|
staging = self._backup_dir / f".restore-{record.filename}.staging.db"
|
||||||
|
if not staging.exists():
|
||||||
|
raise BackupError(
|
||||||
|
f"staging restore file missing: {staging.name}; re-run initiate",
|
||||||
|
)
|
||||||
|
|
||||||
|
target_db_path = self._live_db_path()
|
||||||
|
if target_db_path is None:
|
||||||
|
raise BackupError(
|
||||||
|
"cannot determine live DB file path (non-sqlite URL?)",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Pre-restore fingerprint for the audit event.
|
||||||
|
restored_from_fp = backup_mod.fingerprint(staging.read_bytes())
|
||||||
|
|
||||||
|
# The swap: dispose engine → copy file → reinit engine.
|
||||||
|
# Anything between dispose and reinit raises (queries that
|
||||||
|
# are in-flight get a "database is locked" or
|
||||||
|
# "no such table" error); we accept that because the
|
||||||
|
# operator already confirmed.
|
||||||
|
db.dispose_engine()
|
||||||
|
try:
|
||||||
|
# Atomic copy via temp + rename so a crash mid-copy
|
||||||
|
# doesn't leave a half-written DB file.
|
||||||
|
tmp_target = target_db_path.with_suffix(
|
||||||
|
target_db_path.suffix + f".restoring-{_secrets.token_hex(4)}",
|
||||||
|
)
|
||||||
|
shutil.copyfile(staging, tmp_target)
|
||||||
|
os.replace(tmp_target, target_db_path)
|
||||||
|
finally:
|
||||||
|
staging.unlink(missing_ok=True)
|
||||||
|
db.reinit_engine()
|
||||||
|
|
||||||
|
# Post-restore fingerprint from the now-live engine.
|
||||||
|
new_fp, _ = self._live_fingerprint_and_count()
|
||||||
|
|
||||||
|
log.warning(
|
||||||
|
"Restore complete: backup_id=%d actor=%s from=%s to=%s",
|
||||||
|
backup_id, actor, restored_from_fp, new_fp,
|
||||||
|
)
|
||||||
|
return RestoreConfirmResult(
|
||||||
|
backup_id=record.id,
|
||||||
|
filename=record.filename,
|
||||||
|
restored_from_fingerprint=restored_from_fp,
|
||||||
|
restored_at=datetime.now(timezone.utc),
|
||||||
|
new_db_fingerprint=new_fp,
|
||||||
|
)
|
||||||
|
|
||||||
|
def prune(self, *, now: Optional[datetime] = None) -> list[str]:
|
||||||
|
"""Delete backups older than ``retention_days``. Returns deleted paths.
|
||||||
|
|
||||||
|
Marks the ``db_backups`` rows ``pruned`` so the operator can
|
||||||
|
still see what was deleted (and when).
|
||||||
|
"""
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
|
||||||
|
cutoff = (now or datetime.now(timezone.utc)) - timedelta(
|
||||||
|
days=self._retention_days,
|
||||||
|
)
|
||||||
|
deleted: list[str] = []
|
||||||
|
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
q = s.query(DbBackup).filter(
|
||||||
|
DbBackup.status == STATUS_OK,
|
||||||
|
DbBackup.created_at < cutoff,
|
||||||
|
)
|
||||||
|
for row in q.all():
|
||||||
|
# Delete the file pair; ignore if already gone.
|
||||||
|
bin_path = Path(row.backup_dir) / row.filename
|
||||||
|
meta_path = Path(row.backup_dir) / backup_mod.sidecar_filename(row.filename)
|
||||||
|
for p in (bin_path, meta_path):
|
||||||
|
try:
|
||||||
|
if p.exists():
|
||||||
|
p.unlink()
|
||||||
|
deleted.append(str(p))
|
||||||
|
except OSError as exc:
|
||||||
|
log.warning("Failed to delete %s: %s", p, exc)
|
||||||
|
row.status = STATUS_PRUNED
|
||||||
|
s.add(row)
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
log.info(
|
||||||
|
"Pruned old backups",
|
||||||
|
extra={
|
||||||
|
"deleted_count": len(deleted),
|
||||||
|
"cutoff": cutoff.isoformat(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return deleted
|
||||||
|
|
||||||
|
def status(self) -> dict:
|
||||||
|
"""Snapshot of the backup subsystem for ``GET /api/admin/backup/status``."""
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
from sqlalchemy import func
|
||||||
|
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
total = s.query(func.count(DbBackup.id)).scalar() or 0
|
||||||
|
ok_count = s.query(func.count(DbBackup.id)).filter(
|
||||||
|
DbBackup.status == STATUS_OK,
|
||||||
|
).scalar() or 0
|
||||||
|
error_count = s.query(func.count(DbBackup.id)).filter(
|
||||||
|
DbBackup.status == STATUS_ERROR,
|
||||||
|
).scalar() or 0
|
||||||
|
pruned_count = s.query(func.count(DbBackup.id)).filter(
|
||||||
|
DbBackup.status == STATUS_PRUNED,
|
||||||
|
).scalar() or 0
|
||||||
|
last_row = (
|
||||||
|
s.query(DbBackup)
|
||||||
|
.filter(DbBackup.status.in_([STATUS_OK, STATUS_ERROR]))
|
||||||
|
.order_by(DbBackup.id.desc())
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
last_ok_row = (
|
||||||
|
s.query(DbBackup)
|
||||||
|
.filter(DbBackup.status == STATUS_OK)
|
||||||
|
.order_by(DbBackup.id.desc())
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
disk_bytes = 0
|
||||||
|
try:
|
||||||
|
for p in self._backup_dir.iterdir():
|
||||||
|
if p.is_file() and p.suffix == ".bin":
|
||||||
|
disk_bytes += p.stat().st_size
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return {
|
||||||
|
"backup_dir": str(self._backup_dir),
|
||||||
|
"retention_days": self._retention_days,
|
||||||
|
"totals": {
|
||||||
|
"all": total,
|
||||||
|
"ok": ok_count,
|
||||||
|
"error": error_count,
|
||||||
|
"pruned": pruned_count,
|
||||||
|
},
|
||||||
|
"disk_bytes": disk_bytes,
|
||||||
|
"last_backup_at": (
|
||||||
|
last_row.created_at.isoformat() if last_row and last_row.created_at else None
|
||||||
|
),
|
||||||
|
"last_backup_status": last_row.status if last_row else None,
|
||||||
|
"last_ok_backup_at": (
|
||||||
|
last_ok_row.created_at.isoformat()
|
||||||
|
if last_ok_row and last_ok_row.created_at else None
|
||||||
|
),
|
||||||
|
"used_fallback_key": self._used_fallback,
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- Internals --------------------------------------------------------
|
||||||
|
|
||||||
|
def _ensure_key(self) -> bytes:
|
||||||
|
"""Derive (or return cached) AES key. Triggers fallback + WARNING log
|
||||||
|
if no passphrase was provided at construction time.
|
||||||
|
|
||||||
|
If a passphrase is set but no salt was passed at construction,
|
||||||
|
look one up from the Keychain (``backup.salt`` account). On
|
||||||
|
a fresh install, generate + persist a salt on first use so
|
||||||
|
subsequent invocations derive the same key.
|
||||||
|
"""
|
||||||
|
if self._key is not None:
|
||||||
|
return self._key
|
||||||
|
if self._passphrase:
|
||||||
|
salt = self._salt
|
||||||
|
if salt is None:
|
||||||
|
# Try the Keychain.
|
||||||
|
stored = secrets_mod.get_secret(KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||||
|
if stored:
|
||||||
|
salt = bytes.fromhex(stored.strip())
|
||||||
|
else:
|
||||||
|
# First run: generate + persist.
|
||||||
|
salt = os.urandom(backup_mod.SALT_LEN)
|
||||||
|
secrets_mod.set_secret(
|
||||||
|
KEYCHAIN_BACKUP_SALT_ACCOUNT,
|
||||||
|
salt.hex(),
|
||||||
|
)
|
||||||
|
log.info(
|
||||||
|
"Generated + persisted backup salt to Keychain "
|
||||||
|
"(account %r)",
|
||||||
|
KEYCHAIN_BACKUP_SALT_ACCOUNT,
|
||||||
|
)
|
||||||
|
self._key = backup_mod.derive_key(self._passphrase, salt)
|
||||||
|
return self._key
|
||||||
|
|
||||||
|
# Fallback: derive from SQLCipher DB key. This is degraded
|
||||||
|
# security (the SQLCipher key is meant to unlock the DB, not
|
||||||
|
# the backup), but it's strictly better than plaintext.
|
||||||
|
from cyclone import db_crypto
|
||||||
|
db_key = db_crypto.get_db_key() if db_crypto.is_encryption_enabled() else None
|
||||||
|
if not db_key:
|
||||||
|
# No passphrase AND no SQLCipher key — refuse.
|
||||||
|
raise BackupError(
|
||||||
|
"no backup passphrase set and SQLCipher is not enabled; "
|
||||||
|
"either set a backup passphrase in the Keychain or "
|
||||||
|
"enable SQLCipher encryption",
|
||||||
|
)
|
||||||
|
log.warning(
|
||||||
|
"Backup using fallback key derived from SQLCipher DB key "
|
||||||
|
"(no separate backup passphrase set); set one via "
|
||||||
|
"`cyclone backup init-passphrase` for stronger isolation",
|
||||||
|
extra={"key_source": "sqlcipher_fallback"},
|
||||||
|
)
|
||||||
|
self._used_fallback = True
|
||||||
|
self._key = backup_mod.derive_key(db_key, backup_mod.FALLBACK_SALT)
|
||||||
|
return self._key
|
||||||
|
|
||||||
|
def _sqlite_backup_to(self, target_path: Path) -> None:
|
||||||
|
"""Run SQLite's online ``.backup()`` against the live engine.
|
||||||
|
|
||||||
|
Works for both plain SQLite and SQLCipher because sqlcipher3
|
||||||
|
is API-compatible with sqlite3. The ``.backup()`` API takes
|
||||||
|
a *target* connection; we make a fresh sqlite3 connection to
|
||||||
|
the target file (which doesn't exist yet) and copy into it.
|
||||||
|
"""
|
||||||
|
url = self._db_url or db._resolve_url()
|
||||||
|
if not url.startswith("sqlite"):
|
||||||
|
raise BackupError(
|
||||||
|
f"only sqlite URLs are supported for online backup; got {url!r}",
|
||||||
|
)
|
||||||
|
# Drive the backup off the live engine so we capture the
|
||||||
|
# current state of all tables atomically (SQLite's .backup
|
||||||
|
# holds a read lock on the source for the duration).
|
||||||
|
engine = db.engine() # raises RuntimeError if init_db() wasn't called
|
||||||
|
with engine.raw_connection() as raw:
|
||||||
|
src_conn = raw.driver_connection # sqlite3.Connection / sqlcipher3.Connection
|
||||||
|
if target_path.exists():
|
||||||
|
target_path.unlink()
|
||||||
|
dst_conn = sqlite3.connect(str(target_path))
|
||||||
|
try:
|
||||||
|
src_conn.backup(dst_conn)
|
||||||
|
finally:
|
||||||
|
dst_conn.close()
|
||||||
|
|
||||||
|
def _count_tables_in_blob(self, plaintext: bytes) -> int:
|
||||||
|
"""Open the decrypted DB in-memory and count user tables."""
|
||||||
|
tmp = self._backup_dir / f".count-tables-{_secrets.token_hex(4)}.db"
|
||||||
|
try:
|
||||||
|
tmp.write_bytes(plaintext)
|
||||||
|
conn = sqlite3.connect(str(tmp))
|
||||||
|
try:
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT count(*) FROM sqlite_master "
|
||||||
|
"WHERE type='table' AND name NOT LIKE 'sqlite_%'",
|
||||||
|
).fetchone()
|
||||||
|
return int(rows[0])
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
finally:
|
||||||
|
tmp.unlink(missing_ok=True)
|
||||||
|
|
||||||
|
def _live_fingerprint_and_count(self) -> tuple[str, int]:
|
||||||
|
"""Fingerprint + table count of the *current* live DB."""
|
||||||
|
try:
|
||||||
|
engine = db.engine()
|
||||||
|
except RuntimeError:
|
||||||
|
return "", 0
|
||||||
|
# Use a temp-file .backup so we don't have to worry about
|
||||||
|
# online-vs-offline semantics.
|
||||||
|
tmp = self._backup_dir / f".live-fp-{_secrets.token_hex(4)}.db"
|
||||||
|
try:
|
||||||
|
with engine.raw_connection() as raw:
|
||||||
|
conn = raw.driver_connection
|
||||||
|
if tmp.exists():
|
||||||
|
tmp.unlink()
|
||||||
|
dst = sqlite3.connect(str(tmp))
|
||||||
|
try:
|
||||||
|
conn.backup(dst)
|
||||||
|
finally:
|
||||||
|
dst.close()
|
||||||
|
data = tmp.read_bytes()
|
||||||
|
return backup_mod.fingerprint(data), self._count_tables_in_blob(data)
|
||||||
|
finally:
|
||||||
|
tmp.unlink(missing_ok=True)
|
||||||
|
|
||||||
|
def _live_db_path(self) -> Optional[Path]:
|
||||||
|
"""Resolve the filesystem path of the live DB, or None for non-sqlite."""
|
||||||
|
url = self._db_url or db._resolve_url()
|
||||||
|
if not url.startswith("sqlite"):
|
||||||
|
return None
|
||||||
|
# Strip the driver prefix: sqlite:///abs or sqlite:///./rel
|
||||||
|
prefix = "sqlite:///"
|
||||||
|
if url.startswith(prefix):
|
||||||
|
return Path(url[len(prefix):])
|
||||||
|
if url.startswith("sqlite://"):
|
||||||
|
# sqlite://./relative/path -> Path("./relative/path")
|
||||||
|
return Path(url[len("sqlite://"):])
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _read_sidecar(self, filename: str) -> Optional[backup_mod.Sidecar]:
|
||||||
|
p = self._backup_dir / backup_mod.sidecar_filename(filename)
|
||||||
|
if not p.exists():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return backup_mod.Sidecar.from_json(p.read_text())
|
||||||
|
except (json.JSONDecodeError, KeyError, ValueError) as exc:
|
||||||
|
log.warning("Sidecar %s is malformed: %s", p, exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _row_to_record(self, row) -> BackupRecord:
|
||||||
|
"""ORM row → BackupRecord. Reads key_fingerprint from the sidecar if present."""
|
||||||
|
sidecar = self._read_sidecar(row.filename)
|
||||||
|
return BackupRecord(
|
||||||
|
id=row.id,
|
||||||
|
filename=row.filename,
|
||||||
|
backup_dir=row.backup_dir,
|
||||||
|
size_bytes=row.size_bytes or 0,
|
||||||
|
db_fingerprint=row.db_fingerprint or "",
|
||||||
|
table_count=row.table_count or 0,
|
||||||
|
created_at=row.created_at,
|
||||||
|
completed_at=row.completed_at,
|
||||||
|
status=row.status,
|
||||||
|
error_message=row.error_message,
|
||||||
|
key_fingerprint=sidecar.key_fingerprint if sidecar else "",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Module-level singleton
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
_service: Optional[BackupService] = None
|
||||||
|
|
||||||
|
|
||||||
|
def configure_backup_service(
|
||||||
|
backup_dir: Union[str, Path],
|
||||||
|
*,
|
||||||
|
passphrase: Optional[str] = None,
|
||||||
|
salt: Optional[bytes] = None,
|
||||||
|
retention_days: int = 30,
|
||||||
|
db_url: Optional[str] = None,
|
||||||
|
) -> BackupService:
|
||||||
|
"""Create (or replace) the module-level BackupService singleton."""
|
||||||
|
global _service
|
||||||
|
if _service is not None:
|
||||||
|
return _service
|
||||||
|
_service = BackupService(
|
||||||
|
backup_dir=backup_dir,
|
||||||
|
passphrase=passphrase,
|
||||||
|
salt=salt,
|
||||||
|
retention_days=retention_days,
|
||||||
|
db_url=db_url,
|
||||||
|
)
|
||||||
|
return _service
|
||||||
|
|
||||||
|
|
||||||
|
def get_backup_service() -> BackupService:
|
||||||
|
"""Return the configured BackupService. Raises RuntimeError if not set up."""
|
||||||
|
if _service is None:
|
||||||
|
raise RuntimeError("backup service not configured; call configure_backup_service() first")
|
||||||
|
return _service
|
||||||
|
|
||||||
|
|
||||||
|
def reset_backup_service_for_tests() -> None:
|
||||||
|
"""Clear the module-level singleton. Test-only."""
|
||||||
|
global _service
|
||||||
|
_service = None
|
||||||
@@ -0,0 +1,318 @@
|
|||||||
|
"""Clearhouse integration (SFTP submission, inbound polling).
|
||||||
|
|
||||||
|
SP9 ships a stub (writes files to a local staging dir).
|
||||||
|
SP13 wires the real ``paramiko``-backed SFTP.
|
||||||
|
|
||||||
|
Public API is unchanged across SP9 and SP13:
|
||||||
|
* ``SftpClient.write_file(remote_path, content)`` — uploads bytes
|
||||||
|
* ``SftpClient.list_inbound()`` — lists files in the inbound MFT path
|
||||||
|
* ``SftpClient.read_file(remote_path)`` — downloads bytes
|
||||||
|
* ``SftpClient.get_secret(name)`` — fetches the auth secret
|
||||||
|
|
||||||
|
Authentication is configured via ``SftpBlock.auth``:
|
||||||
|
* ``{"password_keychain_account": "sftp.gainwell.password"}`` — fetch
|
||||||
|
the password from Keychain (Gainwell's MFT model).
|
||||||
|
* ``{"key_file": "/path/to/id_rsa", "key_passphrase_keychain_account": "..."}``
|
||||||
|
— SSH private key (rare for MFT, but supported).
|
||||||
|
|
||||||
|
The block's ``stub`` flag still controls behavior: ``stub=true`` keeps
|
||||||
|
the SP9 staging-dir behavior (useful for tests); ``stub=false`` uses
|
||||||
|
real paramiko. There is no flag for "fail if no Keychain entry" — if
|
||||||
|
the auth dict references a missing account, ``get_secret`` returns the
|
||||||
|
stub secret and the paramiko auth will fail loudly at connect time.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable, Iterator, Optional, Tuple
|
||||||
|
|
||||||
|
from cyclone import secrets
|
||||||
|
from cyclone.providers import SftpBlock
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class InboundFile:
|
||||||
|
"""A single file observed in the inbound MFT path."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
size: int
|
||||||
|
modified_at: datetime
|
||||||
|
local_path: Path
|
||||||
|
|
||||||
|
|
||||||
|
class SftpClient:
|
||||||
|
"""SFTP client wrapper. SP9 stub; SP13 wires paramiko.
|
||||||
|
|
||||||
|
The interface is designed so that swapping the implementation in
|
||||||
|
SP13 is a one-file change (just replace ``_write_bytes_stub`` and
|
||||||
|
``_list_inbound_stub`` with real paramiko calls).
|
||||||
|
"""
|
||||||
|
|
||||||
|
# How long an SFTP connection may sit idle before we tear it down.
|
||||||
|
# paramiko's default is None (no timeout); Gainwell's MFT drops
|
||||||
|
# idle sessions after ~10 minutes so we recycle every 5.
|
||||||
|
_IDLE_TIMEOUT_SECONDS = 5 * 60
|
||||||
|
|
||||||
|
def __init__(self, block: SftpBlock) -> None:
|
||||||
|
self._block = block
|
||||||
|
self._stub = block.stub
|
||||||
|
|
||||||
|
# ---- Public API -------------------------------------------------------
|
||||||
|
|
||||||
|
def write_file(self, remote_path: str, content: bytes) -> Path:
|
||||||
|
"""Write bytes to the given remote path. Returns the local staging path.
|
||||||
|
|
||||||
|
In stub mode, ``remote_path`` is preserved relative to the
|
||||||
|
configured ``staging_dir``. In real mode, this is a paramiko
|
||||||
|
SFTP put.
|
||||||
|
"""
|
||||||
|
if self._stub:
|
||||||
|
return self._write_bytes_stub(remote_path, content)
|
||||||
|
return self._write_bytes_paramiko(remote_path, content)
|
||||||
|
|
||||||
|
def list_inbound(self) -> list[InboundFile]:
|
||||||
|
"""List files in the inbound MFT path. Stub returns [] in stub mode.
|
||||||
|
|
||||||
|
Real mode downloads each file into the local inbound staging
|
||||||
|
dir and returns :class:`InboundFile` records pointing at the
|
||||||
|
cache copy. The remote file is *not* deleted — the operator
|
||||||
|
archives inbound files in the MFT UI.
|
||||||
|
"""
|
||||||
|
if self._stub:
|
||||||
|
return self._list_inbound_stub()
|
||||||
|
return self._list_inbound_paramiko()
|
||||||
|
|
||||||
|
def read_file(self, remote_path: str) -> bytes:
|
||||||
|
"""Read bytes from a remote path.
|
||||||
|
|
||||||
|
Stub mode: reads from ``{staging_dir}/{remote_path}``. Used by
|
||||||
|
the SP16 scheduler so it can exercise the same code path on a
|
||||||
|
workstation without a real MFT connection.
|
||||||
|
"""
|
||||||
|
if self._stub:
|
||||||
|
return self._read_file_stub(remote_path)
|
||||||
|
return self._read_file_paramiko(remote_path)
|
||||||
|
|
||||||
|
def _read_file_stub(self, remote_path: str) -> bytes:
|
||||||
|
"""Read bytes from ``{staging_dir}/{remote_path}`` (SP16 stub)."""
|
||||||
|
staging = Path(self._block.staging_dir).resolve()
|
||||||
|
target = staging / remote_path.lstrip("/")
|
||||||
|
if not target.is_file():
|
||||||
|
raise FileNotFoundError(f"inbound stub file not found: {target}")
|
||||||
|
return target.read_bytes()
|
||||||
|
|
||||||
|
def get_secret(self, name: str) -> Optional[str]:
|
||||||
|
"""Fetch the auth secret from Keychain. Returns the stub secret if absent."""
|
||||||
|
value = secrets.get_secret(name)
|
||||||
|
if value is None:
|
||||||
|
log.info("Keychain entry %r missing; using stub secret", name)
|
||||||
|
return secrets.STUB_SECRET
|
||||||
|
return value
|
||||||
|
|
||||||
|
# ---- Stub implementations (SP9) -------------------------------------
|
||||||
|
|
||||||
|
def _write_bytes_stub(self, remote_path: str, content: bytes) -> Path:
|
||||||
|
"""Copy ``content`` to ``{staging_dir}/{remote_path}``.
|
||||||
|
|
||||||
|
Preserves the full MFT path under staging so the operator can
|
||||||
|
review what would be uploaded. The remote_path may use forward
|
||||||
|
slashes (per SFTP convention); we use PurePosixPath-style split.
|
||||||
|
"""
|
||||||
|
staging = Path(self._block.staging_dir).resolve()
|
||||||
|
# remote_path may be absolute ("/CO XIX/...") or relative; strip
|
||||||
|
# leading slash to avoid escaping the staging dir.
|
||||||
|
rel = remote_path.lstrip("/")
|
||||||
|
target = staging / rel
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_bytes(content)
|
||||||
|
log.info("SFTP stub: wrote %d bytes to %s", len(content), target)
|
||||||
|
return target
|
||||||
|
|
||||||
|
def _list_inbound_stub(self) -> list[InboundFile]:
|
||||||
|
"""Return the local inbound staging dir, if it has been populated
|
||||||
|
by a real MFT pull (e.g. operator dropped files for testing)."""
|
||||||
|
staging = Path(self._block.staging_dir).resolve()
|
||||||
|
inbound_rel = self._block.paths.get("inbound", "").lstrip("/")
|
||||||
|
inbound_dir = staging / inbound_rel
|
||||||
|
if not inbound_dir.is_dir():
|
||||||
|
return []
|
||||||
|
files: list[InboundFile] = []
|
||||||
|
for entry in sorted(inbound_dir.iterdir()):
|
||||||
|
if entry.is_file():
|
||||||
|
stat = entry.stat()
|
||||||
|
files.append(
|
||||||
|
InboundFile(
|
||||||
|
name=entry.name,
|
||||||
|
size=stat.st_size,
|
||||||
|
modified_at=datetime.fromtimestamp(stat.st_mtime),
|
||||||
|
local_path=entry,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return files
|
||||||
|
|
||||||
|
# ---- Real implementations (SP13) -------------------------------------
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def _connect(self) -> Iterator[Tuple["object", "object"]]:
|
||||||
|
"""Open a paramiko SSHClient and yield (ssh, sftp).
|
||||||
|
|
||||||
|
Closes the connection on context exit (caller wraps in
|
||||||
|
``with self._connect() as (ssh, sftp):``). Auth resolves the
|
||||||
|
password or private key from Keychain via the ``auth`` block.
|
||||||
|
|
||||||
|
Why we wrap the SSH client lifecycle here: paramiko caches
|
||||||
|
host keys in ``~/.ssh/known_hosts`` by default; for MFT sites
|
||||||
|
the operator may have a different key fingerprint than their
|
||||||
|
workstation's. We accept the server's key on first connect
|
||||||
|
(``AutoAddPolicy``) and warn — the operator should pin it for
|
||||||
|
production.
|
||||||
|
"""
|
||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.load_system_host_keys()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
auth = self._block.auth or {}
|
||||||
|
password_account = auth.get("password_keychain_account")
|
||||||
|
key_file = auth.get("key_file")
|
||||||
|
key_passphrase_account = auth.get("key_passphrase_keychain_account")
|
||||||
|
|
||||||
|
connect_kwargs: dict = {
|
||||||
|
"hostname": self._block.host,
|
||||||
|
"port": self._block.port,
|
||||||
|
"username": self._block.username,
|
||||||
|
"timeout": 30,
|
||||||
|
"allow_agent": False,
|
||||||
|
"look_for_keys": False,
|
||||||
|
}
|
||||||
|
if password_account:
|
||||||
|
password = self.get_secret(password_account)
|
||||||
|
if password and password != secrets.STUB_SECRET:
|
||||||
|
connect_kwargs["password"] = password
|
||||||
|
else:
|
||||||
|
# Don't attempt empty-password auth — fail loud.
|
||||||
|
raise RuntimeError(
|
||||||
|
f"SFTP: Keychain entry {password_account!r} missing or stub. "
|
||||||
|
"Real SFTP wire-up requires the actual password."
|
||||||
|
)
|
||||||
|
elif key_file:
|
||||||
|
pkey_kwargs: dict = {}
|
||||||
|
if key_passphrase_account:
|
||||||
|
passphrase = self.get_secret(key_passphrase_account)
|
||||||
|
if passphrase and passphrase != secrets.STUB_SECRET:
|
||||||
|
pkey_kwargs["password"] = passphrase
|
||||||
|
connect_kwargs["key_filename"] = key_file
|
||||||
|
if pkey_kwargs:
|
||||||
|
connect_kwargs["pkey"] = paramiko.RSAKey.from_private_key_file(
|
||||||
|
key_file, password=pkey_kwargs.get("password"),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
raise RuntimeError(
|
||||||
|
"SftpBlock.auth must contain either 'password_keychain_account' or 'key_file'"
|
||||||
|
)
|
||||||
|
|
||||||
|
log.info("SFTP: connecting to %s:%d as %s", self._block.host, self._block.port, self._block.username)
|
||||||
|
ssh.connect(**connect_kwargs)
|
||||||
|
sftp = ssh.open_sftp()
|
||||||
|
try:
|
||||||
|
yield ssh, sftp
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
sftp.close()
|
||||||
|
except Exception: # noqa: BLE001 — close errors are non-fatal
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
ssh.close()
|
||||||
|
except Exception: # noqa: BLE001 — close errors are non-fatal
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _write_bytes_paramiko(self, remote_path: str, content: bytes) -> Path:
|
||||||
|
"""Upload ``content`` to ``remote_path`` via paramiko SFTP.
|
||||||
|
|
||||||
|
Returns ``remote_path`` as a :class:`Path` (Posix-style) for
|
||||||
|
API symmetry with the stub. The actual write uses paramiko's
|
||||||
|
``SFTPFile.open(..., "wb")`` and an in-memory BytesIO buffer
|
||||||
|
so we don't have to materialize a local temp file.
|
||||||
|
"""
|
||||||
|
# Lazy import so the stub-only test path doesn't need paramiko.
|
||||||
|
with self._connect() as (ssh, sftp):
|
||||||
|
# Ensure the parent dir exists on the remote. Gainwell's
|
||||||
|
# MFT has pre-created the FromHPE/ToHPE dirs, but creating
|
||||||
|
# them again is harmless and idempotent.
|
||||||
|
parent = "/".join(remote_path.rstrip("/").split("/")[:-1])
|
||||||
|
if parent:
|
||||||
|
try:
|
||||||
|
sftp.mkdir(parent)
|
||||||
|
except IOError:
|
||||||
|
# Already exists — fine.
|
||||||
|
pass
|
||||||
|
with sftp.open(remote_path, "wb") as f:
|
||||||
|
f.write(content)
|
||||||
|
log.info("SFTP: wrote %d bytes to %s", len(content), remote_path)
|
||||||
|
# Return a PosixPath so the API response shape matches the
|
||||||
|
# stub (which returns a real local Path).
|
||||||
|
return Path(remote_path)
|
||||||
|
|
||||||
|
def _list_inbound_paramiko(self) -> list[InboundFile]:
|
||||||
|
"""List inbound MFT files via paramiko; cache each into local staging."""
|
||||||
|
with self._connect() as (ssh, sftp):
|
||||||
|
inbound_dir = self._block.paths.get("inbound", "/")
|
||||||
|
staging = Path(self._block.staging_dir).resolve()
|
||||||
|
inbound_rel = inbound_dir.lstrip("/")
|
||||||
|
cache_dir = staging / inbound_rel
|
||||||
|
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
files: list[InboundFile] = []
|
||||||
|
try:
|
||||||
|
attrs = sftp.listdir_attr(inbound_dir)
|
||||||
|
except IOError as exc:
|
||||||
|
log.warning("SFTP: cannot list %s: %s", inbound_dir, exc)
|
||||||
|
return []
|
||||||
|
|
||||||
|
for attr in sorted(attrs, key=lambda a: a.filename):
|
||||||
|
if attr.st_mode and (attr.st_mode & 0o170000) == 0o040000:
|
||||||
|
# Directory entry — skip.
|
||||||
|
continue
|
||||||
|
remote = f"{inbound_dir.rstrip('/')}/{attr.filename}"
|
||||||
|
cache_path = cache_dir / attr.filename
|
||||||
|
# Download into cache. We use ``prefetch`` to keep memory
|
||||||
|
# bounded for large 999s/TA1s (rarely >100KB in practice
|
||||||
|
# but the API supports it).
|
||||||
|
with sftp.open(remote, "rb") as src, open(cache_path, "wb") as dst:
|
||||||
|
shutil.copyfileobj(src, dst, length=64 * 1024)
|
||||||
|
files.append(InboundFile(
|
||||||
|
name=attr.filename,
|
||||||
|
size=attr.st_size or cache_path.stat().st_size,
|
||||||
|
modified_at=datetime.fromtimestamp(attr.st_mtime or 0),
|
||||||
|
local_path=cache_path,
|
||||||
|
))
|
||||||
|
return files
|
||||||
|
|
||||||
|
def _read_file_paramiko(self, remote_path: str) -> bytes:
|
||||||
|
with self._connect() as (ssh, sftp):
|
||||||
|
buf = io.BytesIO()
|
||||||
|
with sftp.open(remote_path, "rb") as f:
|
||||||
|
shutil.copyfileobj(f, buf, length=64 * 1024)
|
||||||
|
return buf.getvalue()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Module-level helper
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def make_client(block: SftpBlock) -> SftpClient:
|
||||||
|
"""Factory used by the API layer. Kept tiny so swapping the
|
||||||
|
implementation in SP13 is one-line."""
|
||||||
|
return SftpClient(block)
|
||||||
+363
-3
@@ -3,11 +3,13 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
import os
|
||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import click
|
import click
|
||||||
|
|
||||||
|
from cyclone.logging_config import setup_logging
|
||||||
from cyclone.parsers.exceptions import CycloneParseError, CycloneValidationError
|
from cyclone.parsers.exceptions import CycloneParseError, CycloneValidationError
|
||||||
from cyclone.parsers.payer import PayerConfig, PayerConfig835
|
from cyclone.parsers.payer import PayerConfig, PayerConfig835
|
||||||
from cyclone.parsers.parse_837 import parse as parse_837_text
|
from cyclone.parsers.parse_837 import parse as parse_837_text
|
||||||
@@ -41,8 +43,35 @@ def _payer_835(name: str) -> PayerConfig835:
|
|||||||
|
|
||||||
|
|
||||||
@click.group()
|
@click.group()
|
||||||
def main() -> None:
|
@click.option(
|
||||||
|
"--log-format",
|
||||||
|
default=None,
|
||||||
|
type=click.Choice(["json", "dev"]),
|
||||||
|
help="Log format (default: json; honors CYCLONE_LOG_JSON).",
|
||||||
|
)
|
||||||
|
@click.option(
|
||||||
|
"--log-file",
|
||||||
|
default=None,
|
||||||
|
type=click.Path(dir_okay=False, path_type=Path),
|
||||||
|
help="Optional rotating log file (honors CYCLONE_LOG_FILE).",
|
||||||
|
)
|
||||||
|
@click.pass_context
|
||||||
|
def main(ctx: click.Context, log_format: str | None, log_file: Path | None) -> None:
|
||||||
"""Cyclone EDI suite — X12 parser."""
|
"""Cyclone EDI suite — X12 parser."""
|
||||||
|
# SP18: structured JSON logging. Run once per CLI invocation; each
|
||||||
|
# subcommand still gets its own --log-level to override.
|
||||||
|
json_format = True
|
||||||
|
if log_format == "dev":
|
||||||
|
json_format = False
|
||||||
|
elif os.environ.get("CYCLONE_LOG_JSON", "").lower() in ("false", "0", "no"):
|
||||||
|
json_format = False
|
||||||
|
setup_logging(
|
||||||
|
level=os.environ.get("CYCLONE_LOG_LEVEL", "INFO"),
|
||||||
|
log_file=str(log_file) if log_file else None,
|
||||||
|
json_format=json_format,
|
||||||
|
)
|
||||||
|
# Stash on context so subcommands can read it.
|
||||||
|
ctx.ensure_object(dict)
|
||||||
|
|
||||||
|
|
||||||
@main.command("parse-837")
|
@main.command("parse-837")
|
||||||
@@ -63,7 +92,9 @@ def parse_837(
|
|||||||
log_level: str,
|
log_level: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Parse an X12 837P file into one JSON per claim."""
|
"""Parse an X12 837P file into one JSON per claim."""
|
||||||
logging.basicConfig(level=getattr(logging, log_level))
|
# SP18: re-run setup so per-command --log-level overrides the
|
||||||
|
# group default. ``setup_logging`` is idempotent.
|
||||||
|
setup_logging(level=log_level)
|
||||||
|
|
||||||
text = input_file.read_text()
|
text = input_file.read_text()
|
||||||
config = _payer(payer)
|
config = _payer(payer)
|
||||||
@@ -133,7 +164,9 @@ def parse_835(
|
|||||||
log_level: str,
|
log_level: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Parse an X12 835 ERA file into one JSON per claim payment."""
|
"""Parse an X12 835 ERA file into one JSON per claim payment."""
|
||||||
logging.basicConfig(level=getattr(logging, log_level))
|
# SP18: re-run setup so per-command --log-level overrides the
|
||||||
|
# group default. ``setup_logging`` is idempotent.
|
||||||
|
setup_logging(level=log_level)
|
||||||
|
|
||||||
text = input_file.read_text()
|
text = input_file.read_text()
|
||||||
config = _payer_835(payer)
|
config = _payer_835(payer)
|
||||||
@@ -195,5 +228,332 @@ def _count_issues(report) -> dict[str, int]:
|
|||||||
return counts
|
return counts
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP20: `cyclone validate-npi` + `cyclone validate-tax-id`
|
||||||
|
#
|
||||||
|
# Pure local validators. No DB, no Keychain, no network — operators can
|
||||||
|
# run them on a developer laptop without standing up the full Cyclone
|
||||||
|
# stack. Exit code is 0 (valid) or 1 (invalid) so they compose with
|
||||||
|
# shell scripting / CI gates.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@main.command("validate-npi")
|
||||||
|
@click.argument("npi")
|
||||||
|
@click.option("--log-level", default="WARNING", show_default=True, type=click.Choice(["DEBUG", "INFO", "WARNING", "ERROR"]))
|
||||||
|
def validate_npi_cmd(npi: str, log_level: str) -> None:
|
||||||
|
"""Validate a 10-digit NPI's Luhn checksum locally (SP20).
|
||||||
|
|
||||||
|
Exit 0 if valid, 1 if not. No logging of the value itself — NPIs
|
||||||
|
are PHI under HIPAA, so the operator's CLI history is the only
|
||||||
|
audit trail.
|
||||||
|
"""
|
||||||
|
# SP18: re-run so --log-level overrides the group default.
|
||||||
|
setup_logging(level=log_level)
|
||||||
|
from cyclone.npi import is_valid_npi
|
||||||
|
if is_valid_npi(npi):
|
||||||
|
click.echo(f"OK: {len(npi)}-digit NPI passes Luhn checksum")
|
||||||
|
return
|
||||||
|
click.echo(f"INVALID: {npi!r} fails NPI Luhn checksum", err=True)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
@main.command("validate-tax-id")
|
||||||
|
@click.argument("tax_id")
|
||||||
|
@click.option("--log-level", default="WARNING", show_default=True, type=click.Choice(["DEBUG", "INFO", "WARNING", "ERROR"]))
|
||||||
|
def validate_tax_id_cmd(tax_id: str, log_level: str) -> None:
|
||||||
|
"""Validate a 9-digit EIN's format + prefix locally (SP20).
|
||||||
|
|
||||||
|
Accepts both ``XX-XXXXXXX`` and ``XXXXXXXXX``. Exit 0 if valid,
|
||||||
|
1 if not. EIN is sensitive (PII), so we don't echo the value back
|
||||||
|
on failure — only the validation verdict.
|
||||||
|
"""
|
||||||
|
# SP18: re-run so --log-level overrides the group default.
|
||||||
|
setup_logging(level=log_level)
|
||||||
|
from cyclone.npi import is_valid_tax_id, normalize_tax_id
|
||||||
|
plain = normalize_tax_id(tax_id)
|
||||||
|
if plain is None:
|
||||||
|
click.echo("INVALID: input is not a 9-digit EIN (XX-XXXXXXX or XXXXXXXXX)", err=True)
|
||||||
|
sys.exit(1)
|
||||||
|
if is_valid_tax_id(tax_id):
|
||||||
|
click.echo(f"OK: 9-digit EIN (normalized={plain})")
|
||||||
|
return
|
||||||
|
click.echo(
|
||||||
|
f"INVALID: 9-digit EIN has reserved prefix ({plain[:2]}); EIN is not assignable by IRS",
|
||||||
|
err=True,
|
||||||
|
)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
main()
|
main()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP17: `cyclone backup` subcommands
|
||||||
|
#
|
||||||
|
# Operator-facing backup management. Mirrors the API surface but runs
|
||||||
|
# standalone (no FastAPI app needed) for cron / scripting / DR drills.
|
||||||
|
# Each subcommand initializes the DB + BackupService; if the
|
||||||
|
# service isn't configured (no Keychain passphrase etc.) the operator
|
||||||
|
# gets a clear error.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@main.group()
|
||||||
|
def backup() -> None:
|
||||||
|
"""Encrypted DB backup management (SP17)."""
|
||||||
|
|
||||||
|
|
||||||
|
@backup.command("init-passphrase")
|
||||||
|
@click.option("--passphrase", required=True, help="The passphrase to set (will prompt if omitted)")
|
||||||
|
@click.option("--from-stdin", is_flag=True, help="Read passphrase from stdin instead of the argument")
|
||||||
|
def backup_init_passphrase(passphrase: str, from_stdin: bool) -> None:
|
||||||
|
"""Set the backup encryption passphrase in the macOS Keychain.
|
||||||
|
|
||||||
|
Generates a fresh salt and stores both the passphrase (account
|
||||||
|
``backup.passphrase``) and the salt (account ``backup.salt``)
|
||||||
|
under service ``cyclone``. Cyclone's BackupService reads them
|
||||||
|
at startup. If the passphrase account is missing, the service
|
||||||
|
falls back to deriving a key from the SQLCipher DB key
|
||||||
|
(degraded posture, logged at WARNING).
|
||||||
|
"""
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
from getpass import getpass
|
||||||
|
import os as _os
|
||||||
|
|
||||||
|
if from_stdin:
|
||||||
|
pp = getpass("Backup passphrase: ").strip()
|
||||||
|
pp2 = getpass("Confirm: ").strip()
|
||||||
|
if not pp or pp != pp2:
|
||||||
|
click.echo("passphrase empty or mismatch", err=True)
|
||||||
|
sys.exit(2)
|
||||||
|
else:
|
||||||
|
pp = passphrase
|
||||||
|
|
||||||
|
if not pp or len(pp) < 12:
|
||||||
|
click.echo("passphrase must be at least 12 characters", err=True)
|
||||||
|
sys.exit(2)
|
||||||
|
|
||||||
|
if not secrets_mod.set_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT, pp):
|
||||||
|
click.echo("failed to store passphrase in Keychain", err=True)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Generate + persist a fresh salt. Same value must be used by
|
||||||
|
# every subsequent invocation that uses this passphrase.
|
||||||
|
salt = _os.urandom(16)
|
||||||
|
if not secrets_mod.set_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT, salt.hex()):
|
||||||
|
click.echo(
|
||||||
|
"WARN: passphrase stored but salt write failed; backups may be unrecoverable",
|
||||||
|
err=True,
|
||||||
|
)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
click.echo(
|
||||||
|
f"passphrase stored in Keychain account {svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT!r}\n"
|
||||||
|
f"salt stored in Keychain account {svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT!r}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_backup_dir(cli_override: str | None) -> "Path":
|
||||||
|
"""Resolve the backup directory: --backup-dir > $CYCLONE_BACKUP_DIR > default."""
|
||||||
|
import os as _os
|
||||||
|
from pathlib import Path as _Path
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
if cli_override:
|
||||||
|
return _Path(cli_override)
|
||||||
|
env = _os.environ.get("CYCLONE_BACKUP_DIR")
|
||||||
|
if env:
|
||||||
|
return _Path(env)
|
||||||
|
return _Path(db_mod.DEFAULT_DB_PATH.parent / "backups")
|
||||||
|
|
||||||
|
|
||||||
|
@backup.command("create")
|
||||||
|
@click.option("--backup-dir", default=None, help="Override CYCLONE_BACKUP_DIR (default: ~/.local/share/cyclone/backups)")
|
||||||
|
@click.option("--retention-days", default=None, type=int, help="Override CYCLONE_BACKUP_RETENTION_DAYS for this run's prune")
|
||||||
|
def backup_create(backup_dir: str | None, retention_days: int | None) -> None:
|
||||||
|
"""Take an encrypted backup right now."""
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
|
||||||
|
db_mod.init_db()
|
||||||
|
|
||||||
|
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||||
|
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||||
|
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||||
|
target_dir = _resolve_backup_dir(backup_dir)
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
svc = svc_mod.configure_backup_service(
|
||||||
|
backup_dir=target_dir,
|
||||||
|
passphrase=passphrase,
|
||||||
|
salt=salt,
|
||||||
|
retention_days=retention_days or 30,
|
||||||
|
)
|
||||||
|
result = svc.create_now()
|
||||||
|
click.echo(
|
||||||
|
f"created backup id={result.backup.id} filename={result.backup.filename} "
|
||||||
|
f"size={result.backup.size_bytes}B fp={result.backup.db_fingerprint[:24]}..."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@backup.command("list")
|
||||||
|
@click.option("--limit", default=50, show_default=True)
|
||||||
|
@click.option("--status", default=None, help="Filter: ok|error|pending|pruned")
|
||||||
|
def backup_list(limit: int, status: str | None) -> None:
|
||||||
|
"""List existing backups (newest first)."""
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
|
||||||
|
db_mod.init_db()
|
||||||
|
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||||
|
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||||
|
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
svc = svc_mod.configure_backup_service(
|
||||||
|
backup_dir=_resolve_backup_dir(None),
|
||||||
|
passphrase=passphrase,
|
||||||
|
salt=salt,
|
||||||
|
)
|
||||||
|
rows = svc.list_backups(limit=limit, status=status)
|
||||||
|
if not rows:
|
||||||
|
click.echo("(no backups)")
|
||||||
|
return
|
||||||
|
for r in rows:
|
||||||
|
click.echo(
|
||||||
|
f"{r.id:4d} {r.status:7s} {r.created_at.isoformat() if r.created_at else '-'} "
|
||||||
|
f"{r.size_bytes:>10d}B {r.filename} fp={r.db_fingerprint[:24] or '-':<24}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@backup.command("verify")
|
||||||
|
@click.argument("backup_id", type=int)
|
||||||
|
def backup_verify(backup_id: int) -> None:
|
||||||
|
"""Decrypt + checksum-verify a backup."""
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
|
||||||
|
db_mod.init_db()
|
||||||
|
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||||
|
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||||
|
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
svc = svc_mod.configure_backup_service(
|
||||||
|
backup_dir=_resolve_backup_dir(None),
|
||||||
|
passphrase=passphrase,
|
||||||
|
salt=salt,
|
||||||
|
)
|
||||||
|
v = svc.verify(backup_id)
|
||||||
|
if v.ok:
|
||||||
|
click.echo(f"OK: id={v.backup_id} fp={v.actual_fingerprint[:24]}... table_count={v.table_count}")
|
||||||
|
return
|
||||||
|
click.echo(
|
||||||
|
f"FAIL: id={v.backup_id} reason={v.reason} "
|
||||||
|
f"expected={v.expected_fingerprint[:24] if v.expected_fingerprint else '-'}... "
|
||||||
|
f"actual={v.actual_fingerprint[:24] if v.actual_fingerprint else '-'}...",
|
||||||
|
err=True,
|
||||||
|
)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
@backup.command("restore")
|
||||||
|
@click.argument("backup_id", type=int)
|
||||||
|
@click.option("--yes", is_flag=True, help="Skip the interactive confirm prompt")
|
||||||
|
@click.option("--actor", default="operator-cli", show_default=True)
|
||||||
|
def backup_restore(backup_id: int, yes: bool, actor: str) -> None:
|
||||||
|
"""Restore the live DB from a backup (two-step, requires --yes)."""
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
|
||||||
|
db_mod.init_db()
|
||||||
|
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||||
|
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||||
|
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
svc = svc_mod.configure_backup_service(
|
||||||
|
backup_dir=_resolve_backup_dir(None),
|
||||||
|
passphrase=passphrase,
|
||||||
|
salt=salt,
|
||||||
|
)
|
||||||
|
|
||||||
|
click.echo(f"Initiating restore from backup {backup_id}...")
|
||||||
|
init = svc.restore_initiate(backup_id)
|
||||||
|
click.echo(
|
||||||
|
f" backup: {init.filename} ({init.size_bytes} bytes)\n"
|
||||||
|
f" fp: {init.db_fingerprint[:24]}...\n"
|
||||||
|
f" tables: {init.table_count}\n"
|
||||||
|
f" current: fp={init.current_db_fingerprint[:24] if init.current_db_fingerprint else '-'}... "
|
||||||
|
f"tables={init.current_table_count}\n"
|
||||||
|
f" token ttl: {(init.expires_at - __import__('datetime').datetime.now(__import__('datetime').timezone.utc)).total_seconds():.0f}s"
|
||||||
|
)
|
||||||
|
|
||||||
|
if not yes:
|
||||||
|
click.confirm(
|
||||||
|
"Replace the live DB with this backup? "
|
||||||
|
"This will dispose the engine and rebuild it.",
|
||||||
|
abort=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
click.echo("Confirming restore...")
|
||||||
|
result = svc.restore_confirm(backup_id, init.restore_token, actor=actor)
|
||||||
|
click.echo(
|
||||||
|
f"OK: restored from fp={result.restored_from_fingerprint[:24]}... "
|
||||||
|
f"to fp={result.new_db_fingerprint[:24]}... at {result.restored_at.isoformat()}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@backup.command("prune")
|
||||||
|
@click.option("--retention-days", default=None, type=int)
|
||||||
|
@click.option("--yes", is_flag=True, help="Skip the confirm prompt")
|
||||||
|
def backup_prune(retention_days: int | None, yes: bool) -> None:
|
||||||
|
"""Apply the retention policy (delete old backups)."""
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
|
||||||
|
db_mod.init_db()
|
||||||
|
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||||
|
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||||
|
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
svc = svc_mod.configure_backup_service(
|
||||||
|
backup_dir=_resolve_backup_dir(None),
|
||||||
|
passphrase=passphrase,
|
||||||
|
salt=salt,
|
||||||
|
retention_days=retention_days or 30,
|
||||||
|
)
|
||||||
|
if not yes:
|
||||||
|
click.confirm(
|
||||||
|
f"Delete all backups older than {svc._retention_days} days?",
|
||||||
|
abort=True,
|
||||||
|
)
|
||||||
|
deleted = svc.prune()
|
||||||
|
click.echo(f"Deleted {len(deleted)} file(s):")
|
||||||
|
for p in deleted:
|
||||||
|
click.echo(f" {p}")
|
||||||
|
|
||||||
|
|
||||||
|
@backup.command("status")
|
||||||
|
def backup_status() -> None:
|
||||||
|
"""Print the backup subsystem status snapshot."""
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
|
||||||
|
db_mod.init_db()
|
||||||
|
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||||
|
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||||
|
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
svc = svc_mod.configure_backup_service(
|
||||||
|
backup_dir=_resolve_backup_dir(None),
|
||||||
|
passphrase=passphrase,
|
||||||
|
salt=salt,
|
||||||
|
)
|
||||||
|
snap = svc.status()
|
||||||
|
import json
|
||||||
|
click.echo(json.dumps(snap, indent=2, default=str))
|
||||||
|
|||||||
+320
-1
@@ -57,7 +57,36 @@ def _resolve_url() -> str:
|
|||||||
|
|
||||||
|
|
||||||
def _make_engine(url: str) -> sa.Engine:
|
def _make_engine(url: str) -> sa.Engine:
|
||||||
"""Build an Engine with sensible defaults for SQLite + FastAPI."""
|
"""Build an Engine with sensible defaults for SQLite + FastAPI.
|
||||||
|
|
||||||
|
SP12: when ``cyclone.db_crypto.is_encryption_enabled()`` returns
|
||||||
|
True, swap the underlying driver to ``sqlcipher3`` and apply the
|
||||||
|
Keychain-stored key via a connect-time PRAGMA. Otherwise the
|
||||||
|
plain sqlite3 driver is used (current behavior, no surprises for
|
||||||
|
operators who haven't set up Keychain yet).
|
||||||
|
"""
|
||||||
|
from cyclone import db_crypto # late import to avoid cycles
|
||||||
|
|
||||||
|
if url.startswith("sqlite") and db_crypto.is_encryption_enabled():
|
||||||
|
key = db_crypto.get_db_key()
|
||||||
|
if key:
|
||||||
|
creator = db_crypto.make_sqlcipher_connect_creator(url, key)
|
||||||
|
# SP15: NullPool — each thread opens its own SQLCipher
|
||||||
|
# connection. The default QueuePool returns connections
|
||||||
|
# to a shared queue that any thread can pull from, which
|
||||||
|
# breaks SQLCipher's thread affinity (a connection opened
|
||||||
|
# on thread A raises ProgrammingError when used on thread
|
||||||
|
# B). NullPool trades connection reuse for thread safety,
|
||||||
|
# which is the only correct behavior for SQLCipher under
|
||||||
|
# FastAPI's per-request threadpool.
|
||||||
|
from sqlalchemy.pool import NullPool
|
||||||
|
return sa.create_engine(
|
||||||
|
url,
|
||||||
|
creator=creator,
|
||||||
|
poolclass=NullPool,
|
||||||
|
future=True,
|
||||||
|
)
|
||||||
|
|
||||||
connect_args: dict[str, object] = {}
|
connect_args: dict[str, object] = {}
|
||||||
if url.startswith("sqlite"):
|
if url.startswith("sqlite"):
|
||||||
connect_args = {"check_same_thread": False}
|
connect_args = {"check_same_thread": False}
|
||||||
@@ -106,6 +135,34 @@ def _reset_for_tests() -> None:
|
|||||||
_SessionLocal = None
|
_SessionLocal = None
|
||||||
|
|
||||||
|
|
||||||
|
def dispose_engine() -> None:
|
||||||
|
"""Close every pooled connection on the current engine.
|
||||||
|
|
||||||
|
SP15: used by the key-rotation flow to ensure no connection is
|
||||||
|
holding the DB file open while ``PRAGMA rekey`` runs (SQLCipher
|
||||||
|
refuses to rekey if another connection is using the DB). The
|
||||||
|
next call to ``init_db()`` rebuilds the engine with the new key
|
||||||
|
from the Keychain.
|
||||||
|
"""
|
||||||
|
global _engine
|
||||||
|
if _engine is not None:
|
||||||
|
_engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def reinit_engine() -> None:
|
||||||
|
"""Dispose the current engine and rebuild it from the current Keychain key.
|
||||||
|
|
||||||
|
SP15: called by the key-rotation endpoint after the Keychain is
|
||||||
|
updated with the new key. We dispose (close every pooled
|
||||||
|
connection that was using the OLD key) and then re-init (open
|
||||||
|
new connections with the NEW key). The two-step is necessary
|
||||||
|
because SQLAlchemy caches the creator in the pool — a re-init
|
||||||
|
is the only way to swap the driver-level PRAGMA key.
|
||||||
|
"""
|
||||||
|
dispose_engine()
|
||||||
|
init_db()
|
||||||
|
|
||||||
|
|
||||||
def engine() -> sa.Engine:
|
def engine() -> sa.Engine:
|
||||||
"""Return the process-wide Engine. Raises if `init_db()` was not called."""
|
"""Return the process-wide Engine. Raises if `init_db()` was not called."""
|
||||||
if _engine is None:
|
if _engine is None:
|
||||||
@@ -205,6 +262,31 @@ class Claim(Base):
|
|||||||
rejected_at: Mapped[Optional[datetime]] = mapped_column(
|
rejected_at: Mapped[Optional[datetime]] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=True
|
DateTime(timezone=True), nullable=True
|
||||||
)
|
)
|
||||||
|
# SP10: payer-side rejection (277CA STC A4/A6/A7) — distinct from
|
||||||
|
# the 999 envelope rejection above. A claim can be rejected at the
|
||||||
|
# envelope level (bad file) or at the payer-adjudication level
|
||||||
|
# (good file, bad claim). We track them separately so the Inbox
|
||||||
|
# Payer-Rejected lane can distinguish.
|
||||||
|
payer_rejected_at: Mapped[Optional[datetime]] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=True
|
||||||
|
)
|
||||||
|
payer_rejected_reason: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
payer_rejected_status_code: Mapped[Optional[str]] = mapped_column(
|
||||||
|
String(8), nullable=True
|
||||||
|
)
|
||||||
|
payer_rejected_by_277ca_id: Mapped[Optional[str]] = mapped_column(
|
||||||
|
String(64), nullable=True
|
||||||
|
)
|
||||||
|
# SP14: when the operator hits "Acknowledge" on the Payer-Rejected
|
||||||
|
# lane, we set this timestamp. The lane query filters on it being
|
||||||
|
# NULL so acknowledged claims drop out of the working surface. The
|
||||||
|
# original payer_rejected_* fields stay intact for audit (SP11).
|
||||||
|
payer_rejected_acknowledged_at: Mapped[Optional[datetime]] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=True
|
||||||
|
)
|
||||||
|
payer_rejected_acknowledged_actor: Mapped[Optional[str]] = mapped_column(
|
||||||
|
String(64), nullable=True
|
||||||
|
)
|
||||||
resubmit_count: Mapped[int] = mapped_column(
|
resubmit_count: Mapped[int] = mapped_column(
|
||||||
Integer, nullable=False, default=0, server_default=text("0")
|
Integer, nullable=False, default=0, server_default=text("0")
|
||||||
)
|
)
|
||||||
@@ -517,3 +599,240 @@ class Ta1Ack(Base):
|
|||||||
Index("ix_ta1_acks_source_batch_id", "source_batch_id"),
|
Index("ix_ta1_acks_source_batch_id", "source_batch_id"),
|
||||||
Index("ix_ta1_acks_ack_code", "ack_code"),
|
Index("ix_ta1_acks_ack_code", "ack_code"),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Two77caAck(Base):
|
||||||
|
"""277CA (Claim Acknowledgment) row — one per parsed 277CA file.
|
||||||
|
|
||||||
|
Mirrors :class:`Ta1Ack` but for the *semantic* claim-level ack.
|
||||||
|
A 277CA acknowledges individual claims by REF*1K
|
||||||
|
(payer_claim_control_number) rather than the whole envelope.
|
||||||
|
|
||||||
|
Per X12 005010X214 a single 277CA can carry many claim statuses;
|
||||||
|
we keep the per-claim detail in ``raw_json`` and promote only the
|
||||||
|
counts + ICN here so the list endpoint stays fast.
|
||||||
|
|
||||||
|
``source_batch_id`` uses the synthetic ``277CA-<ISA13>`` id — same
|
||||||
|
FK-is-no-op convention as the 999 / TA1 paths. The 277CA itself
|
||||||
|
never has an inbound Cyclone batch to point at.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "two77ca_acks"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
source_batch_id: Mapped[str] = mapped_column(
|
||||||
|
String(32), ForeignKey("batches.id", ondelete="CASCADE"), nullable=False,
|
||||||
|
)
|
||||||
|
control_number: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
accepted_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
rejected_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
paid_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
pended_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
parsed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
raw_json: Mapped[Optional[dict]] = mapped_column(JSONText, nullable=True)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ix_two77ca_acks_source_batch_id", "source_batch_id"),
|
||||||
|
Index("ix_two77ca_acks_control_number", "control_number"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP11: tamper-evident hash-chained audit_log
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class AuditLog(Base):
|
||||||
|
"""One row per audit event. Append-only by convention.
|
||||||
|
|
||||||
|
Each row's :attr:`hash` is SHA-256 of
|
||||||
|
``(id, event_type, entity_type, entity_id, actor, payload_json,
|
||||||
|
created_at, prev_hash)`` — and :attr:`prev_hash` is the previous
|
||||||
|
row's ``hash``. That forms a tamper-evident chain: changing any
|
||||||
|
row's payload invalidates every subsequent row's hash.
|
||||||
|
|
||||||
|
See ``cyclone.audit_log.append_event`` and ``verify_chain`` for
|
||||||
|
the append + verify operations. The application code MUST NOT
|
||||||
|
UPDATE or DELETE rows; doing so breaks the chain and is
|
||||||
|
auditable via :func:`verify_chain`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "audit_log"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
event_type: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
entity_type: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
entity_id: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
actor: Mapped[str] = mapped_column(String(64), nullable=False, default="system")
|
||||||
|
payload_json: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
prev_hash: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
hash: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("idx_audit_log_entity", "entity_type", "entity_id"),
|
||||||
|
Index("idx_audit_log_event_type", "event_type"),
|
||||||
|
Index("idx_audit_log_created_at", "created_at"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP16: inbound MFT scheduler
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class ProcessedInboundFile(Base):
|
||||||
|
"""One row per inbound MFT file the scheduler has downloaded.
|
||||||
|
|
||||||
|
SP16. Lets the scheduler be idempotent: a re-tick or restart must
|
||||||
|
not re-parse the same inbound file. The unique index on
|
||||||
|
(sftp_block_name, name) prevents duplicate inserts and lets the
|
||||||
|
scheduler fast-skip already-processed files via a SELECT.
|
||||||
|
|
||||||
|
Status values:
|
||||||
|
* ok - parsed cleanly, results persisted to the store
|
||||||
|
* error - parser raised; error_message captured
|
||||||
|
* skipped - file_type not in the scheduler's allowed set
|
||||||
|
* pending - file was downloaded but a downstream step failed;
|
||||||
|
the scheduler retries on the next tick
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "processed_inbound_files"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
sftp_block_name: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
name: Mapped[str] = mapped_column(String(256), nullable=False)
|
||||||
|
size: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
modified_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
file_type: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
|
||||||
|
processed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
parser_used: Mapped[Optional[str]] = mapped_column(String(32), nullable=True)
|
||||||
|
claim_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
status: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||||
|
error_message: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index(
|
||||||
|
"ux_processed_inbound_files_block_name",
|
||||||
|
"sftp_block_name", "name", unique=True,
|
||||||
|
),
|
||||||
|
Index("ix_processed_inbound_files_processed_at", "processed_at"),
|
||||||
|
Index("ix_processed_inbound_files_status", "status"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP17: encrypted backup metadata
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class DbBackup(Base):
|
||||||
|
"""One row per encrypted backup the BackupService has taken.
|
||||||
|
|
||||||
|
The actual encrypted blob lives in a directory outside the DB
|
||||||
|
(``~/.local/share/cyclone/backups/`` by default); this table is
|
||||||
|
the index. Status values: ``pending``, ``ok``, ``error``,
|
||||||
|
``pruned``.
|
||||||
|
|
||||||
|
SP17. The unique index on ``(backup_dir, filename)`` makes a
|
||||||
|
duplicate ``create_now()`` race fail cleanly with an
|
||||||
|
IntegrityError instead of clobbering an existing backup.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "db_backups"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
filename: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
backup_dir: Mapped[str] = mapped_column(String(512), nullable=False)
|
||||||
|
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
db_fingerprint: Mapped[Optional[str]] = mapped_column(String(80), nullable=True)
|
||||||
|
table_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
completed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
status: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||||
|
error_message: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ux_db_backups_filename", "backup_dir", "filename", unique=True),
|
||||||
|
Index("ix_db_backups_created_at", "created_at"),
|
||||||
|
Index("ix_db_backups_status", "status"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP9: providers, payers, payer_configs, clearhouse
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class Provider(Base):
|
||||||
|
"""One row per billing-provider NPI (Touch of Care has 3)."""
|
||||||
|
|
||||||
|
__tablename__ = "providers"
|
||||||
|
|
||||||
|
npi: Mapped[str] = mapped_column(String(10), primary_key=True)
|
||||||
|
label: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
legal_name: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
tax_id: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||||
|
taxonomy_code: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||||
|
address_line1: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
address_line2: Mapped[Optional[str]] = mapped_column(String(128), nullable=True)
|
||||||
|
city: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
state: Mapped[str] = mapped_column(String(2), nullable=False)
|
||||||
|
zip: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||||
|
is_active: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
||||||
|
created_at: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
updated_at: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ix_providers_active", "is_active"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Payer(Base):
|
||||||
|
"""One row per payer identity."""
|
||||||
|
|
||||||
|
__tablename__ = "payers"
|
||||||
|
|
||||||
|
payer_id: Mapped[str] = mapped_column(String(32), primary_key=True)
|
||||||
|
name: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
receiver_name: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
receiver_id: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
is_active: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
||||||
|
created_at: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
updated_at: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ix_payers_active", "is_active"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class PayerConfigORM(Base):
|
||||||
|
"""One row per (payer_id, transaction_type). The config_json column
|
||||||
|
carries the per-tx Pydantic config (PayerConfig837, PayerConfig835, etc.)
|
||||||
|
serialized to JSON."""
|
||||||
|
|
||||||
|
__tablename__ = "payer_configs"
|
||||||
|
|
||||||
|
payer_id: Mapped[str] = mapped_column(
|
||||||
|
String(32), ForeignKey("payers.payer_id", ondelete="CASCADE"), primary_key=True,
|
||||||
|
)
|
||||||
|
transaction_type: Mapped[str] = mapped_column(String(8), primary_key=True)
|
||||||
|
config_json: Mapped[dict] = mapped_column(JSONText, nullable=False)
|
||||||
|
updated_at: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
|
||||||
|
|
||||||
|
class ClearhouseORM(Base):
|
||||||
|
"""Singleton row (id always 1) holding dzinesco's identity + SFTP config."""
|
||||||
|
|
||||||
|
__tablename__ = "clearhouse"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
|
name: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
tpid: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||||
|
submitter_name: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
submitter_id_qual: Mapped[str] = mapped_column(String(8), nullable=False, default="46")
|
||||||
|
submitter_contact_name: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
submitter_contact_email: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
filename_block_json: Mapped[dict] = mapped_column(JSONText, nullable=False)
|
||||||
|
sftp_block_json: Mapped[dict] = mapped_column(JSONText, nullable=False)
|
||||||
|
updated_at: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
|||||||
@@ -0,0 +1,389 @@
|
|||||||
|
"""SQLCipher integration — encryption at rest for the SQLite DB.
|
||||||
|
|
||||||
|
SP12 / SP15.
|
||||||
|
|
||||||
|
When ``cyclone.db.key`` is present in the macOS Keychain and the
|
||||||
|
``sqlcipher3`` Python package is installed, the database file is
|
||||||
|
encrypted with SQLCipher (AES-256). Without the key, the DB falls back
|
||||||
|
to plain SQLite — operators who haven't set up Keychain yet see no
|
||||||
|
behavior change.
|
||||||
|
|
||||||
|
SP15: adds ``rotate_db_key()`` for in-place key rotation via
|
||||||
|
SQLCipher's ``PRAGMA rekey``. The rotation:
|
||||||
|
|
||||||
|
1. Closes every pooled SQLAlchemy connection (so the file is unlocked).
|
||||||
|
2. Opens a single dedicated connection with the *old* key.
|
||||||
|
3. Issues ``PRAGMA rekey = "<new_key>"`` (rewrites every page with
|
||||||
|
the new key, in-place).
|
||||||
|
4. Closes the connection.
|
||||||
|
5. Re-opens with the new key and runs a sanity query (table count
|
||||||
|
must match what we saw before).
|
||||||
|
6. Caller updates the Keychain with the new key. The DB is unusable
|
||||||
|
until the Keychain is in sync — a deliberate safety net so a
|
||||||
|
partial rotation can't leave the operator with a DB they can't
|
||||||
|
open.
|
||||||
|
|
||||||
|
Why this design:
|
||||||
|
- The DB key never lives on disk in plaintext. It's stored in macOS
|
||||||
|
Keychain under service ``cyclone``, account ``cyclone.db.key``.
|
||||||
|
Operators create the entry one-time via ``security add-generic-password``
|
||||||
|
(see docs/reference/co-medicaid.md §"Keychain setup").
|
||||||
|
- We don't *require* SQLCipher at import time. ``sqlcipher3`` is an
|
||||||
|
optional dependency — when it's not installed we log a warning and
|
||||||
|
fall back to plain SQLite. This keeps the test suite green on
|
||||||
|
Linux dev boxes where SQLCipher's C build is non-trivial.
|
||||||
|
- The encryption key is applied via a SQLAlchemy connect creator so
|
||||||
|
every connection (including the migration runner and test fixtures)
|
||||||
|
gets the same PRAGMA. We never store the key in a Python global.
|
||||||
|
|
||||||
|
Compliance: HIPAA §164.312(a)(2)(iv) — encryption at rest. §164.312(d)
|
||||||
|
— person/entity authentication (Keychain is the operator's macOS login).
|
||||||
|
SP15: §164.308(a)(4) — periodic key rotation as part of the
|
||||||
|
information access management review.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
|
import secrets as _secrets
|
||||||
|
import sqlite3
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
import sqlalchemy.event
|
||||||
|
|
||||||
|
from cyclone.secrets import STUB_SECRET, get_secret
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Keychain account name for the DB encryption key.
|
||||||
|
KEYCHAIN_ACCOUNT = "cyclone.db.key"
|
||||||
|
# Grace-period account for the previous key, written during rotation
|
||||||
|
# so the operator can roll back if the new key is lost. Cleared
|
||||||
|
# after the operator confirms the new key.
|
||||||
|
KEYCHAIN_ACCOUNT_PREVIOUS = "cyclone.db.key.previous"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Capability checks
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def is_sqlcipher_available() -> bool:
|
||||||
|
"""Return True if the ``sqlcipher3`` package is importable.
|
||||||
|
|
||||||
|
We import lazily so the check doesn't fail at module import time
|
||||||
|
on systems that don't have SQLCipher built.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import sqlcipher3 # noqa: F401
|
||||||
|
return True
|
||||||
|
except ImportError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def is_encryption_enabled() -> bool:
|
||||||
|
"""Return True when SQLCipher is available AND a DB key exists in Keychain.
|
||||||
|
|
||||||
|
Both conditions must hold. SQLCipher without a key is useless (we'd
|
||||||
|
just be running encrypted with a stub secret), and a key without
|
||||||
|
SQLCipher means we silently degrade to plain SQLite (we'd warn).
|
||||||
|
"""
|
||||||
|
if not is_sqlcipher_available():
|
||||||
|
return False
|
||||||
|
key = get_secret(KEYCHAIN_ACCOUNT)
|
||||||
|
return bool(key) and key != STUB_SECRET
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Key retrieval
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def get_db_key() -> str | None:
|
||||||
|
"""Return the SQLCipher DB key from Keychain, or ``None`` if not set.
|
||||||
|
|
||||||
|
``None`` means "fall back to plain SQLite". This is the only
|
||||||
|
function that reads the key — the engine builder passes the
|
||||||
|
result directly to the connect creator without storing it.
|
||||||
|
"""
|
||||||
|
key = get_secret(KEYCHAIN_ACCOUNT)
|
||||||
|
if not key or key == STUB_SECRET:
|
||||||
|
return None
|
||||||
|
return key
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Key generation + fingerprinting (SP15)
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def generate_db_key() -> str:
|
||||||
|
"""Return a fresh 256-bit hex key (64 chars) for use as a SQLCipher PRAGMA key.
|
||||||
|
|
||||||
|
Uses ``secrets.token_hex(32)`` (CSPRNG). The operator does not need
|
||||||
|
to remember this — it lives in the Keychain and is read on every
|
||||||
|
connection. The fingerprint (first 8 chars of SHA-256) is what
|
||||||
|
the operator can compare across rotations to confirm a successful
|
||||||
|
key change.
|
||||||
|
"""
|
||||||
|
return _secrets.token_hex(32)
|
||||||
|
|
||||||
|
|
||||||
|
def fingerprint(key: str) -> str:
|
||||||
|
"""Return a short, operator-readable fingerprint of the key.
|
||||||
|
|
||||||
|
First 8 hex chars of SHA-256. Two fingerprints matching means
|
||||||
|
"this is the same key". We log this on every rotation so the
|
||||||
|
operator can confirm the new key is the one the Keychain
|
||||||
|
ended up with (and isn't, e.g., a transposed paste).
|
||||||
|
"""
|
||||||
|
return hashlib.sha256(key.encode("utf-8")).hexdigest()[:8]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RotateKeyResult:
|
||||||
|
"""Outcome of a SQLCipher key rotation.
|
||||||
|
|
||||||
|
Attributes:
|
||||||
|
ok: True when the rekey completed and the new key opens the DB.
|
||||||
|
old_fingerprint: fingerprint of the old key.
|
||||||
|
new_fingerprint: fingerprint of the new key.
|
||||||
|
rotated_at: ISO-8601 timestamp (UTC) of the rekey.
|
||||||
|
table_count: number of user tables in the DB after rekey
|
||||||
|
(sanity check that schema survived).
|
||||||
|
reason: human-readable error if ``ok`` is False.
|
||||||
|
"""
|
||||||
|
ok: bool
|
||||||
|
old_fingerprint: str
|
||||||
|
new_fingerprint: str
|
||||||
|
rotated_at: str
|
||||||
|
table_count: int = 0
|
||||||
|
reason: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Engine wiring
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def make_sqlcipher_connect_creator(url: str, key: str):
|
||||||
|
"""Return a SQLAlchemy connect creator that opens via ``sqlcipher3``.
|
||||||
|
|
||||||
|
SQLAlchemy's ``creator`` hook expects a zero-arg callable. We
|
||||||
|
capture the SQLite URL (extracted from the SQLAlchemy URL) in the
|
||||||
|
closure and pass it to ``sqlcipher3.connect()`` at every new
|
||||||
|
pool connection.
|
||||||
|
|
||||||
|
Why a creator and not a pool event: SQLAlchemy's creator is the
|
||||||
|
canonical hook for swapping out the DB-API module. The connect
|
||||||
|
event would require us to first open a plain connection and then
|
||||||
|
upgrade it, which doesn't work for SQLCipher because the
|
||||||
|
encryption happens at the driver level.
|
||||||
|
"""
|
||||||
|
import sqlcipher3 # late import — only needed when encryption is on
|
||||||
|
|
||||||
|
# Strip the ``sqlite:///`` prefix; SQLCipher takes a plain path.
|
||||||
|
if url.startswith("sqlite:///"):
|
||||||
|
db_path = url[len("sqlite:///"):]
|
||||||
|
elif url.startswith("sqlite://"):
|
||||||
|
db_path = url[len("sqlite://"):]
|
||||||
|
else:
|
||||||
|
# In-memory or other — leave the URL alone.
|
||||||
|
db_path = url
|
||||||
|
|
||||||
|
def _creator() -> sqlite3.Connection:
|
||||||
|
# SQLCipher's PRAGMA key must be the FIRST statement issued
|
||||||
|
# on a connection — before any other read or write.
|
||||||
|
conn = sqlcipher3.connect(db_path)
|
||||||
|
# SQLCipher accepts hex-encoded keys with ``PRAGMA key = "x'..'"``
|
||||||
|
# but the simpler ``PRAGMA key = "..."`` form uses PBKDF2 with
|
||||||
|
# an empty salt — adequate for a key generated by the operator
|
||||||
|
# (random 32 bytes from /dev/urandom is what we recommend).
|
||||||
|
conn.execute(f'PRAGMA key = "{key}"')
|
||||||
|
return conn
|
||||||
|
|
||||||
|
return _creator
|
||||||
|
|
||||||
|
|
||||||
|
def configure_engine_for_encryption(engine: sa.Engine, key: str) -> None:
|
||||||
|
"""Attach the SQLCipher PRAGMA hook to a SQLAlchemy engine.
|
||||||
|
|
||||||
|
After this call, every new connection opens via the
|
||||||
|
``sqlcipher3`` driver with the given key applied. Idempotent —
|
||||||
|
safe to call once per engine. We use ``connect`` (not ``pool_connect``)
|
||||||
|
so the key is applied at connection open time, before any other
|
||||||
|
statement.
|
||||||
|
"""
|
||||||
|
creator = make_sqlcipher_connect_creator(key)
|
||||||
|
|
||||||
|
# Swap the underlying driver. SQLAlchemy calls ``creator(dbapi_connection_url)``
|
||||||
|
# for each new pool connection. The ``url`` argument is the path
|
||||||
|
# string after ``sqlite:///`` (e.g. ``/path/to/cyclone.db``).
|
||||||
|
@sa.event.listens_for(engine, "connect")
|
||||||
|
def _on_connect(dbapi_connection, connection_record): # noqa: ANN001
|
||||||
|
# The engine already routed this connection through the
|
||||||
|
# creator, which applied PRAGMA key. We could re-issue here
|
||||||
|
# for paranoia, but it's not needed.
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Replace the pool's creator. SQLAlchemy 2.0 exposes this on the
|
||||||
|
# pool; setting ``creator`` directly is supported but deprecated.
|
||||||
|
# Instead we use the dialect-level hook.
|
||||||
|
engine.pool._creator = creator # type: ignore[attr-defined]
|
||||||
|
log.info("SQLCipher encryption enabled (db key in Keychain)")
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Key rotation (SP15)
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def rotate_db_key(
|
||||||
|
*,
|
||||||
|
url: str,
|
||||||
|
old_key: str,
|
||||||
|
new_key: str,
|
||||||
|
) -> RotateKeyResult:
|
||||||
|
"""Re-encrypt the SQLCipher DB with a new key, in place.
|
||||||
|
|
||||||
|
SQLCipher supports ``PRAGMA rekey = "<new_key>"`` which rewrites
|
||||||
|
every page of the DB with the new key. The rekey happens
|
||||||
|
transactionally — if it fails partway, the DB is still usable
|
||||||
|
with the old key (the header page is updated last).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
url: SQLAlchemy URL (must be ``sqlite://``-prefixed with a
|
||||||
|
filesystem path; in-memory DBs can't be rekeyed).
|
||||||
|
old_key: the current key the DB was opened with. Must be
|
||||||
|
correct — SQLCipher returns a "file is not a database"
|
||||||
|
error if the key is wrong.
|
||||||
|
new_key: the key to re-encrypt with. Should be a fresh
|
||||||
|
``generate_db_key()`` value.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
:class:`RotateKeyResult` with ``ok=True` and the new key's
|
||||||
|
fingerprint on success. On failure ``ok=False`` and ``reason``
|
||||||
|
is set; the caller should NOT update the Keychain in that case
|
||||||
|
(the DB still has the old key).
|
||||||
|
"""
|
||||||
|
import sqlcipher3
|
||||||
|
|
||||||
|
if not url.startswith("sqlite") or url.startswith("sqlite:///:memory"):
|
||||||
|
return RotateKeyResult(
|
||||||
|
ok=False,
|
||||||
|
old_fingerprint=fingerprint(old_key),
|
||||||
|
new_fingerprint=fingerprint(new_key),
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
reason="rotate_db_key only works on file-backed SQLite URLs",
|
||||||
|
)
|
||||||
|
|
||||||
|
db_path = _url_to_path(url)
|
||||||
|
if not Path(db_path).exists():
|
||||||
|
return RotateKeyResult(
|
||||||
|
ok=False,
|
||||||
|
old_fingerprint=fingerprint(old_key),
|
||||||
|
new_fingerprint=fingerprint(new_key),
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
reason=f"database file not found: {db_path}",
|
||||||
|
)
|
||||||
|
|
||||||
|
log.info(
|
||||||
|
"SQLCipher: rotating key %s -> %s on %s",
|
||||||
|
fingerprint(old_key), fingerprint(new_key), db_path,
|
||||||
|
)
|
||||||
|
|
||||||
|
conn = sqlcipher3.connect(db_path)
|
||||||
|
try:
|
||||||
|
# Open with the OLD key.
|
||||||
|
conn.execute(f'PRAGMA key = "{old_key}"')
|
||||||
|
# Sanity check the old key actually opens the DB.
|
||||||
|
try:
|
||||||
|
pre_count = _count_user_tables(conn)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
return RotateKeyResult(
|
||||||
|
ok=False,
|
||||||
|
old_fingerprint=fingerprint(old_key),
|
||||||
|
new_fingerprint=fingerprint(new_key),
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
reason=f"old key did not open the DB: {exc}",
|
||||||
|
)
|
||||||
|
|
||||||
|
# PRAGMA rekey rewrites every page. SQLCipher 4+ uses the
|
||||||
|
# ``PRAGMA rekey = "..."`` form (older versions used
|
||||||
|
# ``PRAGMA rekey "..."``; sqlcipher3 0.6+ ships SQLCipher 4).
|
||||||
|
conn.execute(f'PRAGMA rekey = "{new_key}"')
|
||||||
|
|
||||||
|
# Close and reopen to confirm the new key works.
|
||||||
|
conn.close()
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
return RotateKeyResult(
|
||||||
|
ok=False,
|
||||||
|
old_fingerprint=fingerprint(old_key),
|
||||||
|
new_fingerprint=fingerprint(new_key),
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
reason=f"PRAGMA rekey failed: {exc}",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Reopen with the NEW key. Any read query verifies the rekey.
|
||||||
|
try:
|
||||||
|
conn = sqlcipher3.connect(db_path)
|
||||||
|
conn.execute(f'PRAGMA key = "{new_key}"')
|
||||||
|
post_count = _count_user_tables(conn)
|
||||||
|
conn.close()
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
return RotateKeyResult(
|
||||||
|
ok=False,
|
||||||
|
old_fingerprint=fingerprint(old_key),
|
||||||
|
new_fingerprint=fingerprint(new_key),
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
reason=f"new key did not open the DB after rekey: {exc}",
|
||||||
|
)
|
||||||
|
|
||||||
|
if post_count != pre_count:
|
||||||
|
return RotateKeyResult(
|
||||||
|
ok=False,
|
||||||
|
old_fingerprint=fingerprint(old_key),
|
||||||
|
new_fingerprint=fingerprint(new_key),
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
reason=(
|
||||||
|
f"table count mismatch after rekey: "
|
||||||
|
f"pre={pre_count} post={post_count}"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
return RotateKeyResult(
|
||||||
|
ok=True,
|
||||||
|
old_fingerprint=fingerprint(old_key),
|
||||||
|
new_fingerprint=fingerprint(new_key),
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
table_count=post_count,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _url_to_path(url: str) -> str:
|
||||||
|
"""Strip the ``sqlite://`` prefix from a URL to get the filesystem path."""
|
||||||
|
if url.startswith("sqlite:///"):
|
||||||
|
return url[len("sqlite:///"):]
|
||||||
|
if url.startswith("sqlite://"):
|
||||||
|
return url[len("sqlite://"):]
|
||||||
|
return url
|
||||||
|
|
||||||
|
|
||||||
|
def _count_user_tables(conn) -> int:
|
||||||
|
"""Return the number of user (non-internal) tables in the schema.
|
||||||
|
|
||||||
|
Used as a sanity check that the rekey didn't corrupt the schema.
|
||||||
|
Excludes ``sqlite_*`` system tables. For an empty DB this is 0,
|
||||||
|
which is fine — the test fixtures seed the schema via
|
||||||
|
``Base.metadata.create_all`` before rotating.
|
||||||
|
"""
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT name FROM sqlite_master "
|
||||||
|
"WHERE type='table' AND name NOT LIKE 'sqlite_%'"
|
||||||
|
).fetchall()
|
||||||
|
return len(rows)
|
||||||
|
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""EDI utilities: filename helpers, future segment-level transforms."""
|
||||||
@@ -0,0 +1,152 @@
|
|||||||
|
"""HCPF X12 File Naming Standards helpers.
|
||||||
|
|
||||||
|
SP9. Source-of-truth spec:
|
||||||
|
https://hcpf.colorado.gov/tp-x12-filenaming (HCPF X12 File Naming Standards Quick Guide)
|
||||||
|
|
||||||
|
Outbound (we send):
|
||||||
|
{tpid}-{transaction_type}-{yyyymmddhhmmssSSS_MT}-1of1.{ext}
|
||||||
|
Example: 11525703-837P-20260620132243505-1of1.x12
|
||||||
|
|
||||||
|
Inbound (HPE sends to our ToHPE):
|
||||||
|
TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
|
||||||
|
Example: TP11525703-837P_M019048402-20260520231513488-1of1_999.x12
|
||||||
|
|
||||||
|
Both use Mountain Time (MT) timestamps with 17-digit millisecond precision
|
||||||
|
(yyyymmddhhmmssSSS = 4+2+2+2+2+2+3 = 17 digits). Sequence is always "1of1"
|
||||||
|
(the only accepted value per HCPF).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from datetime import datetime
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
from cyclone.providers import InboundFilename
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Regexes
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
# Outbound: 11525703-837P-20260620132243505-1of1.x12
|
||||||
|
# - tpid: 1+ digits
|
||||||
|
# - tx: 1+ alnum
|
||||||
|
# - ts: 17 digits (yyyymmddhhmmssSSS)
|
||||||
|
# - seq: literal "1of1"
|
||||||
|
# - ext: 1+ alnum
|
||||||
|
OUTBOUND_RE = re.compile(
|
||||||
|
r"^(?P<tpid>\d+)-(?P<tx>[A-Z0-9]+)-(?P<ts>\d{17})-1of1\.(?P<ext>[A-Za-z0-9]+)$"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Inbound: TP11525703-837P_M019048402-20260520231513488-1of1_999.x12
|
||||||
|
# - tpid: 1+ digits (inside TP<...>)
|
||||||
|
# - orig_tx: 1+ alnum
|
||||||
|
# - track: M + 1+ alnum (e.g. M019048402) — the M is part of the
|
||||||
|
# tracking value, not a separator.
|
||||||
|
# - ts: 17 digits
|
||||||
|
# - seq: literal "1of1"
|
||||||
|
# - ft: 1+ alnum (e.g. 999, TA1, 271, 277, 277CA, 820, 834, 835, ENCR)
|
||||||
|
INBOUND_RE = re.compile(
|
||||||
|
r"^TP(?P<tpid>\d+)-(?P<orig_tx>[A-Z0-9]+)_(?P<tracking>M[A-Z0-9]+)"
|
||||||
|
r"-(?P<ts>\d{17})-1of1_(?P<file_type>[A-Z0-9]+)\.(?P<ext>x12)$"
|
||||||
|
)
|
||||||
|
|
||||||
|
ALLOWED_FILE_TYPES = frozenset({
|
||||||
|
"999", "TA1", "270", "271", "276", "277", "277CA", "278",
|
||||||
|
"820", "834", "835", "ENCR",
|
||||||
|
})
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Outbound
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def build_outbound_filename(
|
||||||
|
tpid: str,
|
||||||
|
tx: str,
|
||||||
|
*,
|
||||||
|
ext: str = "x12",
|
||||||
|
now_mt: datetime | None = None,
|
||||||
|
) -> str:
|
||||||
|
"""Build an outbound HCPF filename.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
tpid: Trading Partner ID (e.g. "11525703"). Must be digits.
|
||||||
|
tx: Transaction type (e.g. "837P", "835").
|
||||||
|
ext: File extension (default "x12").
|
||||||
|
now_mt: Override for the timestamp (must be timezone-aware, MT or
|
||||||
|
anything ZoneInfo can normalize to MT). When None, the current
|
||||||
|
time in ``America/Denver`` is used.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Filename like "11525703-837P-20260620132243505-1of1.x12"
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: If tpid is non-numeric, tx contains invalid chars, or
|
||||||
|
now_mt is naive.
|
||||||
|
"""
|
||||||
|
if not tpid.isdigit():
|
||||||
|
raise ValueError(f"tpid must be digits, got {tpid!r}")
|
||||||
|
if not re.match(r"^[A-Z0-9]+$", tx):
|
||||||
|
raise ValueError(f"tx must be uppercase alnum, got {tx!r}")
|
||||||
|
if now_mt is None:
|
||||||
|
now_mt = datetime.now(ZoneInfo("America/Denver"))
|
||||||
|
elif now_mt.tzinfo is None:
|
||||||
|
raise ValueError("now_mt must be timezone-aware")
|
||||||
|
else:
|
||||||
|
now_mt = now_mt.astimezone(ZoneInfo("America/Denver"))
|
||||||
|
|
||||||
|
# Format: yyyymmddhhmmssSSS — 17 digits total
|
||||||
|
ts = now_mt.strftime("%Y%m%d%H%M%S") + f"{now_mt.microsecond // 1000:03d}"
|
||||||
|
assert len(ts) == 17
|
||||||
|
return f"{tpid}-{tx}-{ts}-1of1.{ext}"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Inbound
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def parse_inbound_filename(name: str) -> InboundFilename:
|
||||||
|
"""Parse an inbound HCPF filename.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
name: Filename like "TP11525703-837P_M019048402-20260520231513488-1of1_999.x12"
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
InboundFilename with tpid, orig_tx, tracking, ts, file_type, ext.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: If the filename doesn't match the HCPF inbound format.
|
||||||
|
"""
|
||||||
|
m = INBOUND_RE.match(name)
|
||||||
|
if not m:
|
||||||
|
raise ValueError(f"Not a valid HCPF inbound filename: {name!r}")
|
||||||
|
file_type = m.group("file_type")
|
||||||
|
if file_type not in ALLOWED_FILE_TYPES:
|
||||||
|
raise ValueError(
|
||||||
|
f"file_type {file_type!r} not in allowed HCPF set: {sorted(ALLOWED_FILE_TYPES)}"
|
||||||
|
)
|
||||||
|
return InboundFilename(
|
||||||
|
tpid=m.group("tpid"),
|
||||||
|
orig_tx=m.group("orig_tx"),
|
||||||
|
tracking=m.group("tracking"),
|
||||||
|
ts=m.group("ts"),
|
||||||
|
file_type=file_type,
|
||||||
|
ext=m.group("ext"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Validation helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def is_outbound_filename(name: str) -> bool:
|
||||||
|
"""True if the given string matches the HCPF outbound filename regex."""
|
||||||
|
return OUTBOUND_RE.match(name) is not None
|
||||||
|
|
||||||
|
|
||||||
|
def is_inbound_filename(name: str) -> bool:
|
||||||
|
"""True if the given string matches the HCPF inbound filename regex."""
|
||||||
|
return INBOUND_RE.match(name) is not None
|
||||||
@@ -1,9 +1,12 @@
|
|||||||
"""Compute the four Inbox lanes from the DB on read.
|
"""Compute the Inbox lanes from the DB on read.
|
||||||
|
|
||||||
SP6 T6.
|
SP6 T6.
|
||||||
|
|
||||||
Lanes:
|
Lanes:
|
||||||
- rejected: claims whose 999 AK5 set-level response was R/E
|
- rejected: claims whose 999 AK5 set-level response was R/E
|
||||||
|
- payer_rejected: claims whose 277CA STC category is A4/A6/A7
|
||||||
|
(SP10: payer-side rejection, distinct from 999
|
||||||
|
envelope rejection)
|
||||||
- candidates: remits without a matched claim, with scoreable claims
|
- candidates: remits without a matched claim, with scoreable claims
|
||||||
- unmatched: claims still SUBMITTED with no remittance in flight
|
- unmatched: claims still SUBMITTED with no remittance in flight
|
||||||
- done_today: claims that reached a terminal state in the last 24h
|
- done_today: claims that reached a terminal state in the last 24h
|
||||||
@@ -23,6 +26,7 @@ from cyclone.scoring import score_pair, ScoreBreakdown
|
|||||||
@dataclass
|
@dataclass
|
||||||
class Lanes:
|
class Lanes:
|
||||||
rejected: list[dict] = field(default_factory=list)
|
rejected: list[dict] = field(default_factory=list)
|
||||||
|
payer_rejected: list[dict] = field(default_factory=list)
|
||||||
candidates: list[dict] = field(default_factory=list)
|
candidates: list[dict] = field(default_factory=list)
|
||||||
unmatched: list[dict] = field(default_factory=list)
|
unmatched: list[dict] = field(default_factory=list)
|
||||||
done_today: list[dict] = field(default_factory=list)
|
done_today: list[dict] = field(default_factory=list)
|
||||||
@@ -63,6 +67,18 @@ def _claim_to_row(
|
|||||||
"provider": score.provider,
|
"provider": score.provider,
|
||||||
} if score else None,
|
} if score else None,
|
||||||
"matched_remittance": matched_remittance,
|
"matched_remittance": matched_remittance,
|
||||||
|
# SP10: payer-side rejection fields. Populated when a 277CA
|
||||||
|
# acknowledged the claim with STC A4/A6/A7.
|
||||||
|
"payer_rejected_at": _isoformat(c.payer_rejected_at),
|
||||||
|
"payer_rejected_reason": c.payer_rejected_reason,
|
||||||
|
"payer_rejected_status_code": c.payer_rejected_status_code,
|
||||||
|
"payer_rejected_by_277ca_id": c.payer_rejected_by_277ca_id,
|
||||||
|
# SP14: acknowledgment tracking. Always null on the lane
|
||||||
|
# (we filter acknowledged claims out) but exposed for
|
||||||
|
# forward-compat if we later add a "Recently acknowledged"
|
||||||
|
# inspector view.
|
||||||
|
"payer_rejected_acknowledged_at": _isoformat(c.payer_rejected_acknowledged_at),
|
||||||
|
"payer_rejected_acknowledged_actor": c.payer_rejected_acknowledged_actor,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -176,6 +192,33 @@ def compute_lanes(session: Session, *, dismissed_pairs: Iterable[frozenset]) ->
|
|||||||
),
|
),
|
||||||
))
|
))
|
||||||
|
|
||||||
|
# --- Payer-Rejected (SP10) ---
|
||||||
|
# Distinct from the 999 envelope "rejected" lane above. A claim
|
||||||
|
# lands here when a 277CA STC category code is A4/A6/A7 (rejected
|
||||||
|
# by the payer after we submitted a syntactically-valid file).
|
||||||
|
# We don't filter by Claim.state here because the claim may still
|
||||||
|
# be in SUBMITTED state — the payer just hasn't paid it yet.
|
||||||
|
#
|
||||||
|
# SP14: filter out claims the operator has already acknowledged.
|
||||||
|
# The original payer_rejected_* fields stay intact for audit;
|
||||||
|
# only the working surface (this lane) is filtered.
|
||||||
|
payer_rejected_claims = (
|
||||||
|
session.query(Claim)
|
||||||
|
.filter(Claim.payer_rejected_at.is_not(None))
|
||||||
|
.filter(Claim.payer_rejected_acknowledged_at.is_(None))
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
pr_matched, pr_total = _line_count_lookup(session, payer_rejected_claims)
|
||||||
|
matched_counts.update(pr_matched)
|
||||||
|
total_lines_by_claim.update(pr_total)
|
||||||
|
for c in payer_rejected_claims:
|
||||||
|
lanes.payer_rejected.append(_claim_to_row(
|
||||||
|
c, kind="claim",
|
||||||
|
matched_remittance=_matched_remittance_block(
|
||||||
|
session, c, matched_counts, total_lines_by_claim,
|
||||||
|
),
|
||||||
|
))
|
||||||
|
|
||||||
# --- Done today ---
|
# --- Done today ---
|
||||||
cutoff = datetime.now(timezone.utc) - timedelta(hours=24)
|
cutoff = datetime.now(timezone.utc) - timedelta(hours=24)
|
||||||
terminal_states = {
|
terminal_states = {
|
||||||
|
|||||||
@@ -0,0 +1,108 @@
|
|||||||
|
"""277CA Claim Acknowledgment → claim payer-rejection state transitions.
|
||||||
|
|
||||||
|
SP10 T2.
|
||||||
|
|
||||||
|
For each ``ClaimStatus`` in a parsed 277CA whose ``classification``
|
||||||
|
is ``"rejected"`` (STC category codes A4 / A6 / A7), look up the
|
||||||
|
matching Cyclone claim row by ``payer_claim_control_number`` and stamp
|
||||||
|
``payer_rejected_at`` + ``payer_rejected_reason`` + the STC category
|
||||||
|
code + the originating 277CA row id.
|
||||||
|
|
||||||
|
Distinct from :func:`cyclone.inbox_state.apply_999_rejections` (which
|
||||||
|
handles envelope-level 999 AK5 R/E rejections). A claim can be:
|
||||||
|
* rejected at the envelope level only (999 R, no 277CA yet)
|
||||||
|
* payer-rejected only (999 A, then 277CA STC A6 — file was fine,
|
||||||
|
claim was denied)
|
||||||
|
* both (rare — envelope retry after payer rejection)
|
||||||
|
|
||||||
|
We never overwrite a previous ``payer_rejected_at`` with ``NULL``
|
||||||
|
(empty STC list) — that prevents a later, looser 277CA from
|
||||||
|
accidentally clearing the rejection flag.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from cyclone.db import Claim
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Apply277CAResult:
|
||||||
|
matched: list[str] = field(default_factory=list)
|
||||||
|
orphans: list[str] = field(default_factory=list)
|
||||||
|
already_rejected: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_reason(status_code: str, status_description: str | None) -> str:
|
||||||
|
parts = [f"277CA STC {status_code}"]
|
||||||
|
if status_description:
|
||||||
|
parts.append(f"({status_description})")
|
||||||
|
return " ".join(parts)
|
||||||
|
|
||||||
|
|
||||||
|
def apply_277ca_rejections(
|
||||||
|
session: Session,
|
||||||
|
parsed_277ca,
|
||||||
|
*,
|
||||||
|
claim_lookup: Callable[[str], Claim | None],
|
||||||
|
two77ca_id: str,
|
||||||
|
) -> Apply277CAResult:
|
||||||
|
"""For each rejected ``ClaimStatus``, look up the matching claim and stamp it.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
session: SQLAlchemy session.
|
||||||
|
parsed_277ca: a :class:`cyclone.parsers.models_277ca.ParseResult277CA`.
|
||||||
|
claim_lookup: callable from ``payer_claim_control_number`` →
|
||||||
|
``Claim`` or ``None``. Use REF*1K (the payer claim control
|
||||||
|
number) — that's the canonical cross-reference.
|
||||||
|
two77ca_id: the id of the persisted 277CA row that produced
|
||||||
|
this parsed result. Stored on each claim so an operator
|
||||||
|
can trace the rejection back to the source file.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Apply277CAResult with matched/orphan/already-rejected lists.
|
||||||
|
"""
|
||||||
|
result = Apply277CAResult()
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
for status in parsed_277ca.claim_statuses:
|
||||||
|
if status.classification != "rejected":
|
||||||
|
continue
|
||||||
|
pcn = status.payer_claim_control_number
|
||||||
|
if not pcn:
|
||||||
|
# No REF*1K — we can't tie this rejection to a Cyclone claim.
|
||||||
|
# Surface it as an orphan so the operator knows we saw it.
|
||||||
|
result.orphans.append(status.status_code or "")
|
||||||
|
continue
|
||||||
|
|
||||||
|
claim = claim_lookup(pcn)
|
||||||
|
if claim is None:
|
||||||
|
result.orphans.append(pcn)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if claim.payer_rejected_at is not None:
|
||||||
|
# Idempotent — already stamped. Update only if this is a
|
||||||
|
# newer status code (e.g. A6 → A7 escalation).
|
||||||
|
if (claim.payer_rejected_status_code or "") == status.status_code:
|
||||||
|
result.already_rejected.append(claim.id)
|
||||||
|
continue
|
||||||
|
|
||||||
|
claim.payer_rejected_at = now
|
||||||
|
claim.payer_rejected_reason = _build_reason(
|
||||||
|
status.status_code, status.status_description,
|
||||||
|
)
|
||||||
|
claim.payer_rejected_status_code = status.status_code
|
||||||
|
claim.payer_rejected_by_277ca_id = two77ca_id
|
||||||
|
result.matched.append(claim.id)
|
||||||
|
|
||||||
|
if result.matched or result.already_rejected:
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["Apply277CAResult", "apply_277ca_rejections"]
|
||||||
@@ -0,0 +1,379 @@
|
|||||||
|
"""SP18 — Structured JSON logging.
|
||||||
|
|
||||||
|
Wraps Python's stdlib ``logging`` to emit newline-delimited JSON
|
||||||
|
(or a dev-friendly tabular format) and to scrub obvious PHI
|
||||||
|
patterns (NPIs, SSNs, DOBs, patient names) from the message +
|
||||||
|
extra fields.
|
||||||
|
|
||||||
|
Design choices
|
||||||
|
--------------
|
||||||
|
|
||||||
|
* **No third-party deps.** stdlib ``logging`` + ``json`` + ``re``
|
||||||
|
is enough. ``loguru`` / ``structlog`` were considered; both add
|
||||||
|
a dependency for marginal gain.
|
||||||
|
|
||||||
|
* **JSON by default.** Operators running Cyclone in production
|
||||||
|
almost certainly want logs in a format their aggregator
|
||||||
|
(Loki/ELK/Vector) can parse. The dev format (``CycloneDevFormatter``)
|
||||||
|
is the opt-out for ``tail -f`` in dev.
|
||||||
|
|
||||||
|
* **Conservative PII scrubber.** Redacts unambiguous PHI patterns
|
||||||
|
only. False positives are not free — an operator's diagnostic
|
||||||
|
dump that says ``<redacted:npi>`` instead of the actual NPI
|
||||||
|
makes root-causing a parse failure harder. The scrubber can be
|
||||||
|
disabled with ``CYCLONE_LOG_NO_PII_SCRUB=1`` for tests /
|
||||||
|
forensic mode.
|
||||||
|
|
||||||
|
* **Idempotent setup.** :func:`setup_logging` can be called
|
||||||
|
multiple times (CLI re-invocation, FastAPI lifespan re-entry
|
||||||
|
under TestClient). Each call clears existing handlers on the
|
||||||
|
root logger before attaching fresh ones — so the format toggle
|
||||||
|
actually takes effect on the second call.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from logging.handlers import RotatingFileHandler
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Formatters
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
# Stdlib LogRecord attributes we don't want to dump into the
|
||||||
|
# structured payload (they're noise for log consumers).
|
||||||
|
_RESERVED_LOGRECORD_ATTRS = frozenset({
|
||||||
|
"args", "asctime", "created", "exc_info", "exc_text", "filename",
|
||||||
|
"funcName", "levelname", "levelno", "lineno", "module", "msecs",
|
||||||
|
"message", "msg", "name", "pathname", "process", "processName",
|
||||||
|
"relativeCreated", "stack_info", "thread", "threadName",
|
||||||
|
"taskName",
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
class JsonFormatter(logging.Formatter):
|
||||||
|
"""Format a LogRecord as a single JSON line.
|
||||||
|
|
||||||
|
Fields:
|
||||||
|
ts — ISO 8601 UTC timestamp with milliseconds.
|
||||||
|
level — uppercase level name (INFO, WARNING, etc.).
|
||||||
|
logger — the logger name (e.g. "cyclone.scheduler").
|
||||||
|
msg — the formatted log message (after %-substitution).
|
||||||
|
extra — dict of any non-reserved LogRecord attributes.
|
||||||
|
|
||||||
|
If ``exc_info`` is set, the formatter appends a ``traceback``
|
||||||
|
field with the formatted exception text (NOT a serialized
|
||||||
|
object — just the stdlib-rendered string).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def format(self, record: logging.LogRecord) -> str:
|
||||||
|
ts = datetime.fromtimestamp(record.created, tz=timezone.utc).isoformat(
|
||||||
|
timespec="milliseconds",
|
||||||
|
)
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"ts": ts,
|
||||||
|
"level": record.levelname,
|
||||||
|
"logger": record.name,
|
||||||
|
"msg": record.getMessage(),
|
||||||
|
}
|
||||||
|
# Collect user-provided extras.
|
||||||
|
extras = {
|
||||||
|
k: v
|
||||||
|
for k, v in record.__dict__.items()
|
||||||
|
if k not in _RESERVED_LOGRECORD_ATTRS and not k.startswith("_")
|
||||||
|
}
|
||||||
|
if extras:
|
||||||
|
payload["extra"] = extras
|
||||||
|
if record.exc_info:
|
||||||
|
payload["traceback"] = self.formatException(record.exc_info)
|
||||||
|
if record.stack_info:
|
||||||
|
payload["stack"] = self.formatStack(record.stack_info)
|
||||||
|
return json.dumps(payload, default=str, sort_keys=True)
|
||||||
|
|
||||||
|
|
||||||
|
class CycloneDevFormatter(logging.Formatter):
|
||||||
|
"""Dev-friendly tabular format.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
2026-06-21T15:30:00.123Z INFO cyclone.scheduler Processed inbound foo.x12 parser=parse_999 claims=3
|
||||||
|
|
||||||
|
Same fields as ``JsonFormatter`` but human-readable. Useful for
|
||||||
|
``tail -f cyclone.log`` in dev.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def format(self, record: logging.LogRecord) -> str:
|
||||||
|
ts = datetime.fromtimestamp(record.created, tz=timezone.utc).isoformat(
|
||||||
|
timespec="milliseconds",
|
||||||
|
)
|
||||||
|
extras = {
|
||||||
|
k: v
|
||||||
|
for k, v in record.__dict__.items()
|
||||||
|
if k not in _RESERVED_LOGRECORD_ATTRS and not k.startswith("_")
|
||||||
|
}
|
||||||
|
extra_str = ""
|
||||||
|
if extras:
|
||||||
|
pairs = " ".join(f"{k}={v!r}" for k, v in extras.items())
|
||||||
|
extra_str = " " + pairs
|
||||||
|
base = f"{ts} {record.levelname:<7s} {record.name} {record.getMessage()}{extra_str}"
|
||||||
|
if record.exc_info:
|
||||||
|
base += "\n" + self.formatException(record.exc_info)
|
||||||
|
return base
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# PII scrubber
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
# Conservative PHI patterns. Each pattern is (label, compiled regex,
|
||||||
|
# replacement). Some patterns use a backreference so the field name
|
||||||
|
# (e.g. "dob=") is preserved and only the value is redacted — that
|
||||||
|
# keeps the surrounding context readable in the log line.
|
||||||
|
_PII_PATTERNS: tuple[tuple[str, "re.Pattern[str]", str], ...] = (
|
||||||
|
# 10-digit NPI. Word-boundary anchored so we don't redact, e.g.,
|
||||||
|
# the "10" in "10 claims processed".
|
||||||
|
("npi", re.compile(r"\b\d{10}\b"), "<redacted:npi>"),
|
||||||
|
# SSN: NNN-NN-NNNN or NNNNNNNNN.
|
||||||
|
(
|
||||||
|
"ssn",
|
||||||
|
re.compile(r"\b\d{3}-\d{2}-\d{4}\b|\b\d{9}\b(?=[\s,;)}])"),
|
||||||
|
"<redacted:ssn>",
|
||||||
|
),
|
||||||
|
# DOB: "dob=YYYY-MM-DD" / "date_of_birth=YYYY-MM-DD". Capture the
|
||||||
|
# field name + separator, redact only the date — keeps the
|
||||||
|
# surrounding sentence readable.
|
||||||
|
(
|
||||||
|
"dob",
|
||||||
|
re.compile(
|
||||||
|
r"(?i)(\b(?:dob|date[ _]?of[ _]?birth)[:=]\s*)\d{4}-\d{2}-\d{2}"
|
||||||
|
),
|
||||||
|
r"\1<redacted:dob>",
|
||||||
|
),
|
||||||
|
# Patient name: explicit field marker, redact the whole
|
||||||
|
# "patient_name=..." chunk so the value can't leak in a quoted form.
|
||||||
|
(
|
||||||
|
"patient_name",
|
||||||
|
re.compile(
|
||||||
|
r'(?i)\bpatient[_ ]?name[:=]\s*"?[^\",\s}]+',
|
||||||
|
),
|
||||||
|
"<redacted:patient_name>",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
# Extra-field KEYS that we treat as PHI by themselves — if a log call
|
||||||
|
# passes an extra like ``extra={"date_of_birth": "1980-04-12"}`` we
|
||||||
|
# redact the value even though the value alone isn't PHI-shaped. The
|
||||||
|
# key is the signal. Matched case-insensitively against the full key
|
||||||
|
# (with underscores normalized to spaces for "date of birth").
|
||||||
|
_PHI_EXTRA_KEYS: dict[str, str] = {
|
||||||
|
"npi": "npi",
|
||||||
|
"provider_npi": "npi",
|
||||||
|
"rendering_npi": "npi",
|
||||||
|
"billing_npi": "npi",
|
||||||
|
"ssn": "ssn",
|
||||||
|
"dob": "dob",
|
||||||
|
"date_of_birth": "dob",
|
||||||
|
"patient_name": "patient_name",
|
||||||
|
"patient first name": "patient_name",
|
||||||
|
"patient last name": "patient_name",
|
||||||
|
}
|
||||||
|
|
||||||
|
# When an extra key matches one of these, redact any string value
|
||||||
|
# wholesale (don't try to parse it — just replace).
|
||||||
|
_PHI_EXTRA_WHOLE_VALUE = {"npi", "ssn", "dob", "patient_name"}
|
||||||
|
|
||||||
|
|
||||||
|
class PiiScrubber(logging.Filter):
|
||||||
|
"""Filter that redacts obvious PHI from log records.
|
||||||
|
|
||||||
|
Walks the formatted message + every ``extra`` field value (if
|
||||||
|
it's a string) and rewrites matches to ``<redacted:<name>``.
|
||||||
|
Non-string extras are left alone (we don't try to serialize and
|
||||||
|
re-scrub dicts — too risky for false positives).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, name: str = "pii_scrubber") -> None:
|
||||||
|
super().__init__(name)
|
||||||
|
self._enabled = True
|
||||||
|
|
||||||
|
def disable(self) -> None:
|
||||||
|
"""Disable scrubbing (for tests / forensic mode)."""
|
||||||
|
self._enabled = False
|
||||||
|
|
||||||
|
def enable(self) -> None:
|
||||||
|
self._enabled = True
|
||||||
|
|
||||||
|
def _scrub(self, text: str) -> str:
|
||||||
|
for label, pat, repl in _PII_PATTERNS:
|
||||||
|
text = pat.sub(repl, text)
|
||||||
|
return text
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _normalize_extra_key(key: str) -> set[str]:
|
||||||
|
"""Return all candidate normalizations of a key.
|
||||||
|
|
||||||
|
``date_of_birth`` should match a lookup table that uses either
|
||||||
|
``date_of_birth`` or ``date of birth`` — so return both. Same
|
||||||
|
for ``patient_name`` vs ``patient name``.
|
||||||
|
"""
|
||||||
|
norm = key.strip().lower()
|
||||||
|
spaced = norm.replace("_", " ")
|
||||||
|
return {norm, spaced}
|
||||||
|
|
||||||
|
def _redact_extra_value(self, key: str, value: Any) -> Any:
|
||||||
|
"""Redact a single extra field value if its key signals PHI."""
|
||||||
|
for norm in self._normalize_extra_key(key):
|
||||||
|
label = _PHI_EXTRA_KEYS.get(norm)
|
||||||
|
if label:
|
||||||
|
if not isinstance(value, str):
|
||||||
|
return value
|
||||||
|
return f"<redacted:{label}>"
|
||||||
|
return value
|
||||||
|
|
||||||
|
def filter(self, record: logging.LogRecord) -> bool:
|
||||||
|
if not self._enabled:
|
||||||
|
return True
|
||||||
|
# Scrub the formatted message.
|
||||||
|
try:
|
||||||
|
msg = record.getMessage()
|
||||||
|
scrubbed_msg = self._scrub(msg)
|
||||||
|
if scrubbed_msg != msg:
|
||||||
|
record.msg = scrubbed_msg
|
||||||
|
record.args = ()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass # never let the scrubber crash a log call
|
||||||
|
# Scrub string extras in place. We mutate the record's
|
||||||
|
# __dict__ directly so the formatter sees the scrubbed value.
|
||||||
|
for k, v in list(record.__dict__.items()):
|
||||||
|
if k in _RESERVED_LOGRECORD_ATTRS or k.startswith("_"):
|
||||||
|
continue
|
||||||
|
# First, key-based redaction (covers `extra={"dob": "..."}`).
|
||||||
|
redacted = self._redact_extra_value(k, v)
|
||||||
|
if redacted is not v:
|
||||||
|
record.__dict__[k] = redacted
|
||||||
|
continue
|
||||||
|
# Second, value-pattern redaction (covers `extra={"note":
|
||||||
|
# "patient_name=John Doe"}`).
|
||||||
|
if isinstance(v, str):
|
||||||
|
scrubbed = self._scrub(v)
|
||||||
|
if scrubbed != v:
|
||||||
|
record.__dict__[k] = scrubbed
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
# Module-level singleton so tests / callers can disable it cleanly.
|
||||||
|
_scrubber = PiiScrubber()
|
||||||
|
|
||||||
|
|
||||||
|
def get_scrubber() -> PiiScrubber:
|
||||||
|
"""Return the module-level PII scrubber singleton."""
|
||||||
|
return _scrubber
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# setup_logging entry point
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_level(level: str | int | None) -> int:
|
||||||
|
"""Resolve a level string/int, falling back to INFO."""
|
||||||
|
if level is None:
|
||||||
|
return logging.INFO
|
||||||
|
if isinstance(level, int):
|
||||||
|
return level
|
||||||
|
name = str(level).strip().upper()
|
||||||
|
return logging.getLevelNamesMapping().get(name, logging.INFO)
|
||||||
|
|
||||||
|
|
||||||
|
def setup_logging(
|
||||||
|
*,
|
||||||
|
level: str | int | None = None,
|
||||||
|
log_file: str | None = None,
|
||||||
|
json_format: bool = True,
|
||||||
|
scrub_pii: bool = True,
|
||||||
|
propagate_from: str | None = None,
|
||||||
|
) -> logging.Logger:
|
||||||
|
"""Configure the root logger + attach handlers.
|
||||||
|
|
||||||
|
Idempotent: re-calling clears existing handlers on the root
|
||||||
|
logger before attaching fresh ones. Safe to call from
|
||||||
|
``click.command`` invocations and the FastAPI lifespan.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
level: ``"DEBUG"`` / ``"INFO"`` / etc. or an int. ``None``
|
||||||
|
means honor ``CYCLONE_LOG_LEVEL`` env var, then INFO.
|
||||||
|
log_file: Path to a rotating log file. ``None`` means
|
||||||
|
honor ``CYCLONE_LOG_FILE`` env var, then stderr.
|
||||||
|
json_format: Emit JSON lines (default). ``False`` uses
|
||||||
|
:class:`CycloneDevFormatter`.
|
||||||
|
scrub_pii: Apply the PII scrubber (default). Honored via
|
||||||
|
``CYCLONE_LOG_NO_PII_SCRUB=1`` to disable.
|
||||||
|
propagate_from: Optional logger name to attach the scrubber
|
||||||
|
to (defaults to root).
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The configured root logger.
|
||||||
|
"""
|
||||||
|
# Resolve env-var defaults.
|
||||||
|
if level is None:
|
||||||
|
level = os.environ.get("CYCLONE_LOG_LEVEL", "INFO")
|
||||||
|
if log_file is None:
|
||||||
|
log_file = os.environ.get("CYCLONE_LOG_FILE") or None
|
||||||
|
if not json_format and os.environ.get("CYCLONE_LOG_JSON", "").lower() in (
|
||||||
|
"false", "0", "no",
|
||||||
|
):
|
||||||
|
json_format = True
|
||||||
|
if os.environ.get("CYCLONE_LOG_NO_PII_SCRUB", "").lower() in ("1", "true", "yes"):
|
||||||
|
scrub_pii = False
|
||||||
|
|
||||||
|
root = logging.getLogger()
|
||||||
|
root.setLevel(_resolve_level(level))
|
||||||
|
|
||||||
|
# Clear existing handlers (idempotent re-setup).
|
||||||
|
for h in list(root.handlers):
|
||||||
|
root.removeHandler(h)
|
||||||
|
# Also clear our scrubber so we don't add duplicates.
|
||||||
|
target = logging.getLogger(propagate_from) if propagate_from else root
|
||||||
|
for flt in list(target.filters):
|
||||||
|
if isinstance(flt, PiiScrubber):
|
||||||
|
target.removeFilter(flt)
|
||||||
|
|
||||||
|
# Build the formatter.
|
||||||
|
fmt: logging.Formatter
|
||||||
|
if json_format:
|
||||||
|
fmt = JsonFormatter()
|
||||||
|
else:
|
||||||
|
fmt = CycloneDevFormatter()
|
||||||
|
|
||||||
|
# Build the handler.
|
||||||
|
if log_file:
|
||||||
|
handler: logging.Handler = RotatingFileHandler(
|
||||||
|
log_file,
|
||||||
|
maxBytes=10 * 1024 * 1024,
|
||||||
|
backupCount=5,
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
handler = logging.StreamHandler(stream=sys.stderr)
|
||||||
|
handler.setFormatter(fmt)
|
||||||
|
root.addHandler(handler)
|
||||||
|
|
||||||
|
# Attach the scrubber.
|
||||||
|
if scrub_pii:
|
||||||
|
_scrubber.enable()
|
||||||
|
else:
|
||||||
|
_scrubber.disable()
|
||||||
|
target.addFilter(_scrubber)
|
||||||
|
|
||||||
|
# Quiet down noisy third-party libs.
|
||||||
|
for noisy in ("urllib3", "paramiko", "sqlalchemy.engine"):
|
||||||
|
logging.getLogger(noisy).setLevel(max(root.level, logging.WARNING))
|
||||||
|
|
||||||
|
return root
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
-- version: 7
|
||||||
|
-- SP9: multi-payer, multi-NPI, SFTP stub
|
||||||
|
-- See spec: docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md
|
||||||
|
|
||||||
|
CREATE TABLE providers (
|
||||||
|
npi TEXT PRIMARY KEY,
|
||||||
|
label TEXT NOT NULL,
|
||||||
|
legal_name TEXT NOT NULL,
|
||||||
|
tax_id TEXT NOT NULL,
|
||||||
|
taxonomy_code TEXT NOT NULL,
|
||||||
|
address_line1 TEXT NOT NULL,
|
||||||
|
address_line2 TEXT,
|
||||||
|
city TEXT NOT NULL,
|
||||||
|
state TEXT NOT NULL,
|
||||||
|
zip TEXT NOT NULL,
|
||||||
|
is_active INTEGER NOT NULL DEFAULT 1,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE payers (
|
||||||
|
payer_id TEXT PRIMARY KEY,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
receiver_name TEXT NOT NULL,
|
||||||
|
receiver_id TEXT NOT NULL,
|
||||||
|
is_active INTEGER NOT NULL DEFAULT 1,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE payer_configs (
|
||||||
|
payer_id TEXT NOT NULL REFERENCES payers(payer_id),
|
||||||
|
transaction_type TEXT NOT NULL,
|
||||||
|
config_json TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL,
|
||||||
|
PRIMARY KEY (payer_id, transaction_type)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE clearhouse (
|
||||||
|
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
tpid TEXT NOT NULL,
|
||||||
|
submitter_name TEXT NOT NULL,
|
||||||
|
submitter_id_qual TEXT NOT NULL DEFAULT '46',
|
||||||
|
submitter_contact_name TEXT NOT NULL,
|
||||||
|
submitter_contact_email TEXT NOT NULL,
|
||||||
|
filename_block_json TEXT NOT NULL,
|
||||||
|
sftp_block_json TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX ix_providers_active ON providers(is_active);
|
||||||
|
CREATE INDEX ix_payers_active ON payers(is_active);
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
-- version: 8
|
||||||
|
-- SP10: 277CA Payer-Rejected lane
|
||||||
|
-- Adds columns to capture the payer-side rejection state separate from
|
||||||
|
-- the existing 999-envelope rejection (file-level). A claim can be:
|
||||||
|
-- * REJECTED via 999 ACK AK5 R/E (envelope rejection, already covered
|
||||||
|
-- by rejected_at + rejection_reason from migration 0004)
|
||||||
|
-- * PAYER_REJECTED via 277CA STC A4/A6/A7 (claim-level rejection by
|
||||||
|
-- the payer after envelope acceptance)
|
||||||
|
-- We keep these distinct so operators can tell *why* a claim isn't
|
||||||
|
-- paid: was it our file (999), or was it the payer's adjudication?
|
||||||
|
--
|
||||||
|
-- Also creates two77ca_acks to persist parsed 277CA files (one row per
|
||||||
|
-- inbound 277CA file, with the per-claim status detail in raw_json).
|
||||||
|
|
||||||
|
ALTER TABLE claims ADD COLUMN payer_rejected_at TEXT;
|
||||||
|
ALTER TABLE claims ADD COLUMN payer_rejected_reason TEXT;
|
||||||
|
ALTER TABLE claims ADD COLUMN payer_rejected_status_code TEXT;
|
||||||
|
ALTER TABLE claims ADD COLUMN payer_rejected_by_277ca_id TEXT;
|
||||||
|
|
||||||
|
CREATE INDEX idx_claims_payer_rejected_at ON claims(payer_rejected_at);
|
||||||
|
|
||||||
|
CREATE TABLE two77ca_acks (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
source_batch_id TEXT NOT NULL REFERENCES batches(id) ON DELETE CASCADE,
|
||||||
|
control_number TEXT NOT NULL,
|
||||||
|
accepted_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
rejected_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
paid_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
pended_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
parsed_at TEXT NOT NULL,
|
||||||
|
raw_json TEXT
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX ix_two77ca_acks_source_batch_id ON two77ca_acks(source_batch_id);
|
||||||
|
CREATE INDEX ix_two77ca_acks_control_number ON two77ca_acks(control_number);
|
||||||
|
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
-- version: 9
|
||||||
|
-- SP11: tamper-evident hash-chained audit_log
|
||||||
|
--
|
||||||
|
-- Each row carries a SHA-256 hash of (id, event_type, entity_type,
|
||||||
|
-- entity_id, actor, payload_json, created_at, prev_hash). The prev_hash
|
||||||
|
-- field chains the row to the previous row's hash — a tamper-evident
|
||||||
|
-- Merkle-like chain (no tree, just a list).
|
||||||
|
--
|
||||||
|
-- Append-only by convention: no UPDATE/DELETE in the application code.
|
||||||
|
-- Compliance: HIPAA §164.316(b)(2) requires 6-year retention. We don't
|
||||||
|
-- enforce retention in the schema (no TTL), but a separate vacuum
|
||||||
|
-- job (out of scope here) can prune rows older than 6 years after
|
||||||
|
-- exporting them to cold storage.
|
||||||
|
--
|
||||||
|
-- Indexes: (entity_type, entity_id) for "show me the audit trail for
|
||||||
|
-- this claim"; (event_type) for "show me all clearhouse.submitted
|
||||||
|
-- events"; (created_at) for time-range scans.
|
||||||
|
|
||||||
|
CREATE TABLE audit_log (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
event_type TEXT NOT NULL,
|
||||||
|
entity_type TEXT NOT NULL,
|
||||||
|
entity_id TEXT NOT NULL,
|
||||||
|
actor TEXT NOT NULL DEFAULT 'system',
|
||||||
|
payload_json TEXT,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
prev_hash TEXT NOT NULL,
|
||||||
|
hash TEXT NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_audit_log_entity ON audit_log(entity_type, entity_id);
|
||||||
|
CREATE INDEX idx_audit_log_event_type ON audit_log(event_type);
|
||||||
|
CREATE INDEX idx_audit_log_created_at ON audit_log(created_at);
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
-- version: 10
|
||||||
|
-- SP14: Payer-Rejected lane acknowledge
|
||||||
|
-- When the operator reviews a payer-rejected claim, they hit the
|
||||||
|
-- "Acknowledge" bulk action. We mark the claim with the timestamp so
|
||||||
|
-- the lane query can filter it out (it stays in the DB for audit but
|
||||||
|
-- doesn't show up in the operator's working surface).
|
||||||
|
--
|
||||||
|
-- Why a separate column instead of clearing payer_rejected_at:
|
||||||
|
-- * Audit trail: SP11's hash-chained audit_log needs the *original*
|
||||||
|
-- rejection event intact. Clearing the timestamp would erase the
|
||||||
|
-- evidence of the payer saying "no" — exactly what auditors want
|
||||||
|
-- to see.
|
||||||
|
-- * Reconciliation: future SPs that match payer-rejected claims
|
||||||
|
-- against appeals (e.g. SP17) can still see the original status
|
||||||
|
-- code and reason.
|
||||||
|
|
||||||
|
ALTER TABLE claims ADD COLUMN payer_rejected_acknowledged_at TEXT;
|
||||||
|
ALTER TABLE claims ADD COLUMN payer_rejected_acknowledged_actor TEXT;
|
||||||
|
|
||||||
|
CREATE INDEX idx_claims_payer_rejected_unack
|
||||||
|
ON claims(payer_rejected_at)
|
||||||
|
WHERE payer_rejected_acknowledged_at IS NULL;
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
-- version: 11
|
||||||
|
-- SP16: Inbound MFT polling scheduler
|
||||||
|
--
|
||||||
|
-- Tracks every file the background scheduler has downloaded from
|
||||||
|
-- the Gainwell MFT inbound path so a re-tick (or a restart) does not
|
||||||
|
-- re-process the same file. Idempotency is required for production:
|
||||||
|
-- the scheduler polls every N seconds and a slow MFT server may hand
|
||||||
|
-- us the same file across two polls.
|
||||||
|
--
|
||||||
|
-- We key on (sftp_block_name, name) — the sftp_block_name disambiguates
|
||||||
|
-- multi-provider installations (SP9+SP-multi-NPI), name is the inbound
|
||||||
|
-- filename as it appears on the MFT server.
|
||||||
|
--
|
||||||
|
-- Status values:
|
||||||
|
-- * ok — parsed cleanly, results persisted to the store
|
||||||
|
-- * error — parser raised; error_message captured for the operator
|
||||||
|
-- * skipped — file_type not in the scheduler's allowed set
|
||||||
|
-- * pending — file was downloaded but a downstream step failed
|
||||||
|
-- (e.g. DB write); the scheduler retries on the next tick
|
||||||
|
--
|
||||||
|
-- claim_count is the number of claims/remittances/acks the parser
|
||||||
|
-- surfaced. Surfaced on /api/admin/scheduler/status so the operator can
|
||||||
|
-- see throughput without parsing logs.
|
||||||
|
--
|
||||||
|
-- Compliance: not part of the HIPAA audit chain (SP11). This is
|
||||||
|
-- operational metadata; an SFTP outage shouldn't pollute the audit log.
|
||||||
|
|
||||||
|
CREATE TABLE processed_inbound_files (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
sftp_block_name TEXT NOT NULL,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
size INTEGER NOT NULL,
|
||||||
|
modified_at TEXT,
|
||||||
|
file_type TEXT,
|
||||||
|
processed_at TEXT NOT NULL,
|
||||||
|
parser_used TEXT,
|
||||||
|
claim_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
error_message TEXT
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX ux_processed_inbound_files_block_name
|
||||||
|
ON processed_inbound_files(sftp_block_name, name);
|
||||||
|
CREATE INDEX ix_processed_inbound_files_processed_at
|
||||||
|
ON processed_inbound_files(processed_at DESC);
|
||||||
|
CREATE INDEX ix_processed_inbound_files_status
|
||||||
|
ON processed_inbound_files(status);
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
-- version: 12
|
||||||
|
-- SP17: encrypted DB backup metadata
|
||||||
|
--
|
||||||
|
-- Tracks every backup the BackupService has taken. The actual
|
||||||
|
-- encrypted blob lives in a directory outside the DB (default
|
||||||
|
-- ~/.local/share/cyclone/backups/); this table is just the index
|
||||||
|
-- the operator queries via GET /api/admin/backup/list.
|
||||||
|
--
|
||||||
|
-- Status values:
|
||||||
|
-- pending - row inserted, .backup() in progress or crashed before commit
|
||||||
|
-- ok - encrypted blob + sidecar written successfully
|
||||||
|
-- error - creation failed; error_message populated
|
||||||
|
-- pruned - retention policy removed the file; row kept for audit
|
||||||
|
|
||||||
|
CREATE TABLE db_backups (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
filename TEXT NOT NULL,
|
||||||
|
backup_dir TEXT NOT NULL,
|
||||||
|
size_bytes INTEGER NOT NULL DEFAULT 0,
|
||||||
|
db_fingerprint TEXT,
|
||||||
|
table_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
completed_at TEXT,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
error_message TEXT
|
||||||
|
);
|
||||||
|
CREATE UNIQUE INDEX ux_db_backups_filename ON db_backups(backup_dir, filename);
|
||||||
|
CREATE INDEX ix_db_backups_created_at ON db_backups(created_at DESC);
|
||||||
|
CREATE INDEX ix_db_backups_status ON db_backups(status);
|
||||||
@@ -0,0 +1,159 @@
|
|||||||
|
"""SP20 — NPI checksum + Tax ID format validation.
|
||||||
|
|
||||||
|
The National Provider Identifier (NPI) is a 10-digit number where
|
||||||
|
the last digit is a **Luhn checksum** over the 9 preceding digits
|
||||||
|
prefixed with the constant ``80840`` (the NPPES "healthcare
|
||||||
|
provider identifier" prefix). See CMS / HHS NPI Standard:
|
||||||
|
|
||||||
|
https://www.cms.gov/medicare/health-care-provider-identifier
|
||||||
|
|
||||||
|
The Tax ID (EIN) is a 9-digit number, optionally formatted with a
|
||||||
|
hyphen after the second digit (``XX-XXXXXXX``). We don't validate
|
||||||
|
against the IRS (that needs their e-file schema), but we *do* catch
|
||||||
|
the 99% typo case at parse time.
|
||||||
|
|
||||||
|
Everything in this module is local — no NPPES, no network. Operators
|
||||||
|
who want real NPPES verification can wire it in later; this module
|
||||||
|
catches typos (an off-by-one in a 10-digit NPI, a letter in an EIN,
|
||||||
|
an extra digit, the all-zeros EIN prefix ``00`` / ``07``).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
# NPPES prefix per the NPI Luhn algorithm. Prepended to the 9-digit
|
||||||
|
# NPI body before running the Luhn check.
|
||||||
|
_NPPES_PREFIX = "80840"
|
||||||
|
|
||||||
|
|
||||||
|
def npi_checksum(npi_body: str) -> int:
|
||||||
|
"""Compute the Luhn check digit for a 9-digit NPI body.
|
||||||
|
|
||||||
|
``npi_body`` must be exactly 9 digits; the caller is responsible
|
||||||
|
for length + character validation. Returns the check digit (0–9).
|
||||||
|
"""
|
||||||
|
if not npi_body.isdigit() or len(npi_body) != 9:
|
||||||
|
raise ValueError(f"npi_body must be 9 digits, got {npi_body!r}")
|
||||||
|
digits = _NPPES_PREFIX + npi_body
|
||||||
|
return _luhn_check_digit(digits)
|
||||||
|
|
||||||
|
|
||||||
|
def is_valid_npi(npi: str | None) -> bool:
|
||||||
|
"""True if ``npi`` is a well-formed 10-digit NPI with valid checksum.
|
||||||
|
|
||||||
|
Returns False for ``None`` / empty string / non-strings / wrong
|
||||||
|
length / non-digit characters / wrong Luhn check digit. Doesn't
|
||||||
|
call NPPES — see module docstring for why.
|
||||||
|
|
||||||
|
>>> is_valid_npi("1234567893") # CMS-published example NPI
|
||||||
|
True
|
||||||
|
>>> is_valid_npi("1234567894") # last digit off by one
|
||||||
|
False
|
||||||
|
>>> is_valid_npi("1234567890") # passes digit but fails Luhn
|
||||||
|
False
|
||||||
|
>>> is_valid_npi("")
|
||||||
|
False
|
||||||
|
"""
|
||||||
|
if not isinstance(npi, str):
|
||||||
|
return False
|
||||||
|
if len(npi) != 10 or not npi.isdigit():
|
||||||
|
return False
|
||||||
|
return npi[-1] == str(npi_checksum(npi[:-1]))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tax ID (EIN)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
# EIN prefix table (subset). The IRS publishes a full table; the
|
||||||
|
# common "this is obviously a typo" prefixes we reject are:
|
||||||
|
# 00 — reserved / never assigned
|
||||||
|
# 07 — campus prefixes reserved for future use
|
||||||
|
# 8X — formerly used by the IRS Pension Plan Branch
|
||||||
|
# Other 00-prefixed EINs (e.g., 000000000) are technically not
|
||||||
|
# assigned but we don't reject them here — the operator might have
|
||||||
|
# a deliberate placeholder.
|
||||||
|
_EIN_FORBIDDEN_PREFIXES = {"00", "07"}
|
||||||
|
_EIN_RESERVED_PREFIX_8X = re.compile(r"^8\d$")
|
||||||
|
|
||||||
|
# 9 digits, optionally formatted as XX-XXXXXXX.
|
||||||
|
_EIN_FORMATTED = re.compile(r"^\d{2}-\d{7}$")
|
||||||
|
_EIN_PLAIN = re.compile(r"^\d{9}$")
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_tax_id(tax_id: str | None) -> str | None:
|
||||||
|
"""Return ``tax_id`` in 9-digit plain form, or None if it's malformed.
|
||||||
|
|
||||||
|
>>> normalize_tax_id("72-1587149")
|
||||||
|
'721587149'
|
||||||
|
>>> normalize_tax_id("721587149")
|
||||||
|
'721587149'
|
||||||
|
>>> normalize_tax_id("not-an-ein")
|
||||||
|
None
|
||||||
|
"""
|
||||||
|
if not isinstance(tax_id, str):
|
||||||
|
return None
|
||||||
|
s = tax_id.strip()
|
||||||
|
if _EIN_FORMATTED.match(s):
|
||||||
|
return s.replace("-", "")
|
||||||
|
if _EIN_PLAIN.match(s):
|
||||||
|
return s
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def is_valid_tax_id(tax_id: str | None) -> bool:
|
||||||
|
"""True if ``tax_id`` is a 9-digit EIN (formatted or plain) with
|
||||||
|
a non-reserved prefix.
|
||||||
|
|
||||||
|
>>> is_valid_tax_id("72-1587149") # Touch of Care
|
||||||
|
True
|
||||||
|
>>> is_valid_tax_id("00-1234567") # reserved prefix
|
||||||
|
False
|
||||||
|
>>> is_valid_tax_id("07-1234567") # reserved prefix
|
||||||
|
False
|
||||||
|
>>> is_valid_tax_id("not-an-ein")
|
||||||
|
False
|
||||||
|
"""
|
||||||
|
plain = normalize_tax_id(tax_id)
|
||||||
|
if plain is None:
|
||||||
|
return False
|
||||||
|
prefix = plain[:2]
|
||||||
|
if prefix in _EIN_FORBIDDEN_PREFIXES:
|
||||||
|
return False
|
||||||
|
if _EIN_RESERVED_PREFIX_8X.match(prefix):
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Luhn internals
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _luhn_check_digit(digits: str) -> int:
|
||||||
|
"""Return the Luhn check digit for ``digits``.
|
||||||
|
|
||||||
|
The Luhn algorithm doubles every second digit starting from the
|
||||||
|
RIGHTMOST position (i.e., the first digit doubled is the rightmost
|
||||||
|
character of ``digits``). If the doubled value exceeds 9, subtract
|
||||||
|
9. Sum all digits; the check digit is ``(10 - sum % 10) % 10``.
|
||||||
|
|
||||||
|
``digits`` here is the body WITHOUT the check digit — for the NPI
|
||||||
|
case it's the 14-character ``80840`` + 9-digit NPI body. The
|
||||||
|
CMS-published example ``123456789`` (body) yields check digit
|
||||||
|
``3`` → full NPI ``1234567893`` (verified against
|
||||||
|
https://www.cms.gov/.../NPIcheckdigit.pdf).
|
||||||
|
"""
|
||||||
|
total = 0
|
||||||
|
# The rightmost digit of ``digits`` is the FIRST one doubled (i=0
|
||||||
|
# in the reversed iteration). Per CMS, doubling starts at the
|
||||||
|
# rightmost and alternates leftward.
|
||||||
|
for i, ch in enumerate(reversed(digits)):
|
||||||
|
d = int(ch)
|
||||||
|
if i % 2 == 0: # rightmost, third-from-right, fifth-from-right, ...
|
||||||
|
d *= 2
|
||||||
|
if d > 9:
|
||||||
|
d -= 9
|
||||||
|
total += d
|
||||||
|
return (10 - total % 10) % 10
|
||||||
@@ -66,6 +66,15 @@ _LAZY_EXPORTS: dict[str, str] = {
|
|||||||
"SERVICE_TYPE_CODES": "cyclone.parsers.models_271",
|
"SERVICE_TYPE_CODES": "cyclone.parsers.models_271",
|
||||||
"LAST_UPDATED": "cyclone.parsers.models_271",
|
"LAST_UPDATED": "cyclone.parsers.models_271",
|
||||||
"service_type_description": "cyclone.parsers.models_271",
|
"service_type_description": "cyclone.parsers.models_271",
|
||||||
|
# models (277CA — SP10)
|
||||||
|
"ACCEPTED_CODES": "cyclone.parsers.models_277ca",
|
||||||
|
"AcknowledgmentHeader277": "cyclone.parsers.models_277ca",
|
||||||
|
"ClaimStatus": "cyclone.parsers.models_277ca",
|
||||||
|
"PAID_CODES": "cyclone.parsers.models_277ca",
|
||||||
|
"PENDED_CODES": "cyclone.parsers.models_277ca",
|
||||||
|
"ParseResult277CA": "cyclone.parsers.models_277ca",
|
||||||
|
"REJECTED_CODES": "cyclone.parsers.models_277ca",
|
||||||
|
"classify_status_code": "cyclone.parsers.models_277ca",
|
||||||
# CARC lookup (SP3 P2 T6)
|
# CARC lookup (SP3 P2 T6)
|
||||||
"reason_label": "cyclone.parsers.cas_codes",
|
"reason_label": "cyclone.parsers.cas_codes",
|
||||||
"all_known_codes": "cyclone.parsers.cas_codes",
|
"all_known_codes": "cyclone.parsers.cas_codes",
|
||||||
@@ -81,6 +90,7 @@ _LAZY_EXPORTS: dict[str, str] = {
|
|||||||
"parse_999": "cyclone.parsers.parse_999",
|
"parse_999": "cyclone.parsers.parse_999",
|
||||||
"parse_270": "cyclone.parsers.parse_270",
|
"parse_270": "cyclone.parsers.parse_270",
|
||||||
"parse_271": "cyclone.parsers.parse_271",
|
"parse_271": "cyclone.parsers.parse_271",
|
||||||
|
"parse_277ca": "cyclone.parsers.parse_277ca",
|
||||||
"serialize_999": "cyclone.parsers.serialize_999",
|
"serialize_999": "cyclone.parsers.serialize_999",
|
||||||
"serialize_270": "cyclone.parsers.serialize_270",
|
"serialize_270": "cyclone.parsers.serialize_270",
|
||||||
"build_ack_for_batch": "cyclone.parsers.batch_ack_builder",
|
"build_ack_for_batch": "cyclone.parsers.batch_ack_builder",
|
||||||
|
|||||||
@@ -0,0 +1,161 @@
|
|||||||
|
"""Pydantic v2 models for parsed 277CA (Claim Acknowledgment) files.
|
||||||
|
|
||||||
|
Mirrors the X12 005010X214 segment shape (HCPF sends this transaction
|
||||||
|
back after we submit an 837P batch):
|
||||||
|
|
||||||
|
- ``AcknowledgmentHeader`` (BHT) — beginning of hierarchical transaction
|
||||||
|
- ``ClaimStatus`` (STC + REF*1K + REF*EJ + AMT*YU + DTP*472) — one per
|
||||||
|
payer-acknowledged claim. Carries the category/status code and the
|
||||||
|
payer claim control number used to match back to a Cyclone claim row.
|
||||||
|
- ``ParseResult277CA`` — top-level envelope + per-claim status list.
|
||||||
|
|
||||||
|
Why this lean shape: the 277CA is a status-reporting transaction, not a
|
||||||
|
claim. We surface just enough information to (a) match each STC row back
|
||||||
|
to a Cyclone claim by ``payer_claim_control_number`` (REF*1K) and
|
||||||
|
(b) classify it as accepted / pended / rejected / paid so the Inbox
|
||||||
|
Payer-Rejected lane can light up.
|
||||||
|
|
||||||
|
Per HCPF X12 File Naming Standards, the inbound filename uses ``277``
|
||||||
|
in the file_type slot (the transaction-set id ``277CA`` is conveyed in
|
||||||
|
the ST segment itself, not in the filename).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, model_serializer
|
||||||
|
|
||||||
|
from cyclone.parsers.models import BatchSummary, Envelope, ValidationIssue, ValidationReport
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Shared base
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class _Base(BaseModel):
|
||||||
|
"""Shared Pydantic base; matches the 999 / 835 / 270 / 271 models for JSON consistency."""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="ignore", str_strip_whitespace=True)
|
||||||
|
|
||||||
|
@model_serializer(mode="wrap")
|
||||||
|
def _serialize(self, handler): # type: ignore[no-untyped-def]
|
||||||
|
data = handler(self)
|
||||||
|
for key, value in data.items():
|
||||||
|
if isinstance(value, date):
|
||||||
|
data[key] = value.isoformat()
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# 277CA segment shapes
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
# Status category codes per X12 005010X214 §1.4.2 STC01-1.
|
||||||
|
# We categorize each STC into a small set of lanes the Inbox cares about.
|
||||||
|
ACCEPTED_CODES = {"A1", "A2", "A3"}
|
||||||
|
REJECTED_CODES = {"A4", "A6", "A7"}
|
||||||
|
PENDED_CODES = {"A8", "A9"}
|
||||||
|
PAID_CODES = {"P1", "P2", "P3", "P4", "P5"}
|
||||||
|
|
||||||
|
|
||||||
|
def classify_status_code(code: str) -> Literal["accepted", "rejected", "pended", "paid", "unknown"]:
|
||||||
|
"""Map an STC01-1 category code to an Inbox lane label.
|
||||||
|
|
||||||
|
Unknown codes (rare; new payer-specific codes) become ``"unknown"``
|
||||||
|
so the parser doesn't raise — operators can still see them in the
|
||||||
|
277CA detail view.
|
||||||
|
"""
|
||||||
|
code = (code or "").strip().upper()
|
||||||
|
if code in ACCEPTED_CODES:
|
||||||
|
return "accepted"
|
||||||
|
if code in REJECTED_CODES:
|
||||||
|
return "rejected"
|
||||||
|
if code in PENDED_CODES:
|
||||||
|
return "pended"
|
||||||
|
if code in PAID_CODES:
|
||||||
|
return "paid"
|
||||||
|
return "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
class AcknowledgmentHeader(_Base):
|
||||||
|
"""BHT — Beginning of Hierarchical Transaction.
|
||||||
|
|
||||||
|
BHT01 = hierarchical_structure_code (e.g. "0085" for Claim Ack).
|
||||||
|
BHT02 = transaction_set_purpose_code (e.g. "08" for Status).
|
||||||
|
BHT03 = reference_identification (often the submitter batch id).
|
||||||
|
BHT04 = transaction_set_creation_date (CCYYMMDD).
|
||||||
|
BHT05 = transaction_set_creation_time (HHMM).
|
||||||
|
BHT06 = transaction_type_code (e.g. "TH").
|
||||||
|
"""
|
||||||
|
|
||||||
|
hierarchical_structure_code: str
|
||||||
|
transaction_set_purpose_code: str
|
||||||
|
reference_identification: str | None = None
|
||||||
|
transaction_set_creation_date: date | None = None
|
||||||
|
transaction_set_creation_time: str | None = None
|
||||||
|
transaction_type_code: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class ClaimStatus(_Base):
|
||||||
|
"""STC + REF*1K + REF*EJ + AMT*YU + DTP*472 — one payer-acknowledged claim.
|
||||||
|
|
||||||
|
STC01 is a composite of three elements: ``category_code:status_code:entity_identifier``.
|
||||||
|
HCPF only populates the category code (e.g. ``A3:19:PR``); the parser
|
||||||
|
surfaces ``category_code`` and ``status_code`` separately.
|
||||||
|
|
||||||
|
``payer_claim_control_number`` is the REF*1K value the payer assigned
|
||||||
|
(or echoed back) for the claim. This is what matches back to a
|
||||||
|
Cyclone claim row (we store it on Claim.payer_claim_control_number
|
||||||
|
during 837 serialize).
|
||||||
|
|
||||||
|
``amount`` is the AMT*YU value (total claim charge amount
|
||||||
|
acknowledged by the payer). Optional — HCPF omits it for non-claim
|
||||||
|
statuses (e.g. subscriber-level pends).
|
||||||
|
"""
|
||||||
|
|
||||||
|
status_code: str
|
||||||
|
status_description: str | None = None
|
||||||
|
entity_identifier: str | None = None
|
||||||
|
status_effective_date: date | None = None
|
||||||
|
status_action_code: str | None = None
|
||||||
|
total_claim_charge_amount: float | None = None
|
||||||
|
payer_claim_control_number: str | None = None
|
||||||
|
billing_provider_tax_id: str | None = None
|
||||||
|
service_date: date | None = None
|
||||||
|
classification: Literal["accepted", "rejected", "pended", "paid", "unknown"] = "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
class ParseResult277CA(_Base):
|
||||||
|
"""Top-level parsed 277CA document.
|
||||||
|
|
||||||
|
Multiple claims can be acknowledged in a single 277CA; the parser
|
||||||
|
flattens the HL hierarchy (20→21→19→PT) into one ClaimStatus per
|
||||||
|
Patient-level HL. Subscriber- and provider-level statuses (when no
|
||||||
|
Patient HL exists) are also captured under ``unscoped_statuses``
|
||||||
|
for operator visibility.
|
||||||
|
"""
|
||||||
|
|
||||||
|
envelope: Envelope
|
||||||
|
bht: AcknowledgmentHeader
|
||||||
|
claim_statuses: list[ClaimStatus] = Field(default_factory=list)
|
||||||
|
unscoped_statuses: list[ClaimStatus] = Field(default_factory=list)
|
||||||
|
summary: BatchSummary
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ACCEPTED_CODES",
|
||||||
|
"AcknowledgmentHeader",
|
||||||
|
"ClaimStatus",
|
||||||
|
"PAID_CODES",
|
||||||
|
"PENDED_CODES",
|
||||||
|
"ParseResult277CA",
|
||||||
|
"REJECTED_CODES",
|
||||||
|
"classify_status_code",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# Silence unused-import lints.
|
||||||
|
_ = (BaseModel, ValidationIssue, ValidationReport)
|
||||||
@@ -0,0 +1,355 @@
|
|||||||
|
"""Parse an X12 277CA (Claim Acknowledgment) file.
|
||||||
|
|
||||||
|
The 277CA is the *semantic* ack a payer (or its clearinghouse) sends
|
||||||
|
back after they accept our 837P batch. It contrasts with the 999 ACK
|
||||||
|
which only reports the *syntactic* envelope status. A 999 "Accepted"
|
||||||
|
plus a 277CA with STC*A6 means: the file was syntactically valid, but
|
||||||
|
one or more specific claims were rejected by the payer.
|
||||||
|
|
||||||
|
Layout (simplified, single claim)::
|
||||||
|
|
||||||
|
ISA*…~
|
||||||
|
GS*HN*…*…*20240620*1200*1*X*005010X214~
|
||||||
|
ST*277*0001*005010X214~ (HCPF sometimes sends ST*277CA*0001*005010X214)
|
||||||
|
BHT*0085*08*REFNUM*20240620*1200*TH~
|
||||||
|
HL*1**20*1~ (Information Source — the payer)
|
||||||
|
NM1*PR*2*COLORADO MEDICAL ASSISTANCE PROGRAM*****PI*COMEDASSISTPROG~
|
||||||
|
HL*2*1*21*1~ (Information Receiver — us)
|
||||||
|
NM1*41*2*DZINESCO*****46*DZINESCO~
|
||||||
|
HL*3*2*19*1~ (Subscriber)
|
||||||
|
NM1*IL*1*DOE*JOHN****MI*MEMBERID~
|
||||||
|
HL*4*3*PT~ (Patient — a claim follows)
|
||||||
|
NM1*QC*1*DOE*JANE~
|
||||||
|
REF*1K*PAYERCLAIMID123~
|
||||||
|
REF*EJ*721587149~
|
||||||
|
STC*A6:19:PR*20240620*U*150.00~ (rejected)
|
||||||
|
QTY*90*1~
|
||||||
|
AMT*YU*150.00~
|
||||||
|
DTP*472*RD8*20240601-20240601~
|
||||||
|
SE*XX*0001~
|
||||||
|
GE*1*1~
|
||||||
|
IEA*1*000000001~
|
||||||
|
|
||||||
|
The parser walks the HL hierarchy and, for each ``HL*…*PT`` segment,
|
||||||
|
collects the trailing ``STC*``, ``REF*1K``, ``REF*EJ``, ``AMT*YU`` and
|
||||||
|
``DTP*472`` siblings into a :class:`ClaimStatus`. STC segments under
|
||||||
|
non-Patient HLs (provider- or subscriber-level) go into
|
||||||
|
``unscoped_statuses`` — HCPF sometimes sends a blanket reject at the
|
||||||
|
subscriber level which we want to surface even though we can't tie it
|
||||||
|
to a specific Cyclone claim row.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from datetime import date
|
||||||
|
|
||||||
|
from cyclone.parsers.exceptions import CycloneParseError
|
||||||
|
from cyclone.parsers.models import BatchSummary, Envelope
|
||||||
|
from cyclone.parsers.models_277ca import (
|
||||||
|
AcknowledgmentHeader,
|
||||||
|
ClaimStatus,
|
||||||
|
ParseResult277CA,
|
||||||
|
classify_status_code,
|
||||||
|
)
|
||||||
|
from cyclone.parsers.segments import tokenize
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Date / amount parsing
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_yyyymmdd(s: str) -> date | None:
|
||||||
|
if not s or len(s) != 8 or not s.isdigit():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return date(int(s[0:4]), int(s[4:6]), int(s[6:8]))
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_yymmdd(s: str) -> date | None:
|
||||||
|
if not s or len(s) != 6 or not s.isdigit():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return date(2000 + int(s[0:2]), int(s[2:4]), int(s[4:6]))
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_date_loose(s: str) -> date | None:
|
||||||
|
"""Parse an X12 date — accept 8-digit CCYYMMDD or 6-digit YYMMDD."""
|
||||||
|
if not s or not s.isdigit():
|
||||||
|
return None
|
||||||
|
if len(s) == 8:
|
||||||
|
return _parse_yyyymmdd(s)
|
||||||
|
if len(s) == 6:
|
||||||
|
return _parse_yymmdd(s)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_service_date(s: str) -> date | None:
|
||||||
|
"""Parse DTP*472 values. HCPF sends ``RD8*20240601-20240601`` (range) or ``D8*20240601``.
|
||||||
|
|
||||||
|
For ranges we return the start date — that's what matches the
|
||||||
|
Cyclone claim's ``service_date_from``.
|
||||||
|
"""
|
||||||
|
if not s:
|
||||||
|
return None
|
||||||
|
# RD8 = range of dates (CCYYMMDD-CCYYMMDD); take the start.
|
||||||
|
if "-" in s:
|
||||||
|
start = s.split("-", 1)[0]
|
||||||
|
return _parse_date_loose(start)
|
||||||
|
return _parse_date_loose(s)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_amount(s: str) -> float | None:
|
||||||
|
"""Parse an X12 monetary value. Returns None on bad input."""
|
||||||
|
if not s:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return float(s)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Envelope
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def _build_envelope(segments: list[list[str]], input_file: str) -> tuple[Envelope, str]:
|
||||||
|
"""Build the envelope from ISA/GS/ST. Returns ``(envelope, tx_set_id)``.
|
||||||
|
|
||||||
|
``tx_set_id`` carries the ST01 value (``"277"`` or ``"277CA"``) so
|
||||||
|
the rest of the parser doesn't have to re-scan the segment list.
|
||||||
|
"""
|
||||||
|
summary = BatchSummary(input_file=input_file)
|
||||||
|
envelope: Envelope | None = None
|
||||||
|
tx_set_id: str = ""
|
||||||
|
txn_date: date | None = None
|
||||||
|
for seg in segments:
|
||||||
|
if seg[0] == "ISA":
|
||||||
|
try:
|
||||||
|
envelope = Envelope(
|
||||||
|
sender_id=seg[6].strip(),
|
||||||
|
receiver_id=seg[8].strip(),
|
||||||
|
control_number=seg[13].strip(),
|
||||||
|
transaction_date=date(2024, 1, 1),
|
||||||
|
implementation_guide=None,
|
||||||
|
)
|
||||||
|
except (IndexError, ValueError) as exc:
|
||||||
|
raise CycloneParseError(f"Bad ISA: {exc}") from exc
|
||||||
|
elif seg[0] == "GS" and envelope is not None:
|
||||||
|
if len(seg) > 4:
|
||||||
|
txn_date = _parse_date_loose(seg[3]) or txn_date
|
||||||
|
elif seg[0] == "ST" and envelope is not None:
|
||||||
|
if len(seg) > 1:
|
||||||
|
tx_set_id = seg[1]
|
||||||
|
if len(seg) > 3:
|
||||||
|
envelope = envelope.model_copy(update={"implementation_guide": seg[3]})
|
||||||
|
if envelope is None:
|
||||||
|
raise CycloneParseError("No ISA envelope found")
|
||||||
|
if txn_date is not None:
|
||||||
|
envelope = envelope.model_copy(update={"transaction_date": txn_date})
|
||||||
|
return envelope, tx_set_id
|
||||||
|
|
||||||
|
|
||||||
|
def _build_bht(segments: list[list[str]]) -> AcknowledgmentHeader:
|
||||||
|
"""Find the BHT segment and return it. Falls back to an empty header."""
|
||||||
|
for seg in segments:
|
||||||
|
if seg[0] == "BHT":
|
||||||
|
return AcknowledgmentHeader(
|
||||||
|
hierarchical_structure_code=seg[1] if len(seg) > 1 else "",
|
||||||
|
transaction_set_purpose_code=seg[2] if len(seg) > 2 else "",
|
||||||
|
reference_identification=seg[3] if len(seg) > 3 and seg[3] else None,
|
||||||
|
transaction_set_creation_date=_parse_date_loose(seg[4]) if len(seg) > 4 else None,
|
||||||
|
transaction_set_creation_time=seg[5] if len(seg) > 5 and seg[5] else None,
|
||||||
|
transaction_type_code=seg[6] if len(seg) > 6 and seg[6] else None,
|
||||||
|
)
|
||||||
|
return AcknowledgmentHeader(
|
||||||
|
hierarchical_structure_code="",
|
||||||
|
transaction_set_purpose_code="",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# HL-walking and per-claim status construction
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_stc(seg: list[str]) -> ClaimStatus:
|
||||||
|
"""Convert an STC segment into a partial ClaimStatus.
|
||||||
|
|
||||||
|
STC is a composite segment: ``STC*cat:stat:entity*date*action*amount``.
|
||||||
|
HCPF only populates ``cat`` (``A6:19:PR``); the rest are optional.
|
||||||
|
"""
|
||||||
|
composite = seg[1] if len(seg) > 1 else ""
|
||||||
|
parts = composite.split(":") if composite else []
|
||||||
|
category = parts[0] if parts else ""
|
||||||
|
status = parts[1] if len(parts) > 1 and parts[1] else None
|
||||||
|
entity = parts[2] if len(parts) > 2 and parts[2] else None
|
||||||
|
effective = _parse_date_loose(seg[2]) if len(seg) > 2 else None
|
||||||
|
action = seg[3] if len(seg) > 3 and seg[3] else None
|
||||||
|
amount = _parse_amount(seg[4]) if len(seg) > 4 else None
|
||||||
|
return ClaimStatus(
|
||||||
|
status_code=category,
|
||||||
|
status_description=status,
|
||||||
|
entity_identifier=entity,
|
||||||
|
status_effective_date=effective,
|
||||||
|
status_action_code=action,
|
||||||
|
total_claim_charge_amount=amount,
|
||||||
|
classification=classify_status_code(category),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _consume_patient_block(segments: list[list[str]], idx: int) -> tuple[ClaimStatus | None, int]:
|
||||||
|
"""Read all segments under an HL*…*PT block (until the next HL or SE).
|
||||||
|
|
||||||
|
Returns ``(status, next_idx)``. ``status`` is ``None`` when the block
|
||||||
|
contained no STC segments (a Patient HL with no status is unusual
|
||||||
|
but not illegal — surface it as ``None`` so the caller can log).
|
||||||
|
"""
|
||||||
|
parts: dict[str, object] = {}
|
||||||
|
stc: ClaimStatus | None = None
|
||||||
|
i = idx
|
||||||
|
while i < len(segments):
|
||||||
|
seg = segments[i]
|
||||||
|
if seg[0] in {"HL", "SE"}:
|
||||||
|
break
|
||||||
|
if seg[0] == "REF":
|
||||||
|
qualifier = seg[1] if len(seg) > 1 else ""
|
||||||
|
value = seg[2] if len(seg) > 2 else ""
|
||||||
|
if qualifier == "1K":
|
||||||
|
parts["payer_claim_control_number"] = value
|
||||||
|
elif qualifier == "EJ":
|
||||||
|
parts["billing_provider_tax_id"] = value
|
||||||
|
elif seg[0] == "STC":
|
||||||
|
# Per X12 005010X214 a Patient HL can have multiple STC
|
||||||
|
# segments (one per status). We take the last one as
|
||||||
|
# authoritative — that's typically the most recent action
|
||||||
|
# (e.g. STC*A1 then STC*A3 means "first pended, then paid").
|
||||||
|
stc = _parse_stc(seg)
|
||||||
|
elif seg[0] == "AMT":
|
||||||
|
qualifier = seg[1] if len(seg) > 1 else ""
|
||||||
|
value = _parse_amount(seg[2]) if len(seg) > 2 else None
|
||||||
|
if qualifier == "YU" and value is not None:
|
||||||
|
parts["total_claim_charge_amount"] = value
|
||||||
|
elif seg[0] == "DTP":
|
||||||
|
qualifier = seg[1] if len(seg) > 1 else ""
|
||||||
|
fmt = seg[2] if len(seg) > 2 else ""
|
||||||
|
value = seg[3] if len(seg) > 3 else ""
|
||||||
|
if qualifier == "472" and fmt in ("D8", "RD8"):
|
||||||
|
parts["service_date"] = _parse_service_date(value)
|
||||||
|
i += 1
|
||||||
|
|
||||||
|
if stc is None:
|
||||||
|
return None, i
|
||||||
|
|
||||||
|
# Merge the REF/AMT/DTP capture onto the last STC.
|
||||||
|
merged = stc.model_copy(update=parts)
|
||||||
|
return merged, i
|
||||||
|
|
||||||
|
|
||||||
|
def _consume_subscriber_block(segments: list[list[str]], idx: int) -> tuple[list[ClaimStatus], int]:
|
||||||
|
"""Read STC segments under a Subscriber (HL*…*19*1) block.
|
||||||
|
|
||||||
|
Subscriber-level STCs have no REF*1K — they apply to the whole
|
||||||
|
subscriber's claim batch. Surface them in
|
||||||
|
:attr:`ParseResult277CA.unscoped_statuses`.
|
||||||
|
"""
|
||||||
|
out: list[ClaimStatus] = []
|
||||||
|
i = idx
|
||||||
|
while i < len(segments):
|
||||||
|
seg = segments[i]
|
||||||
|
if seg[0] in {"HL", "SE"}:
|
||||||
|
break
|
||||||
|
if seg[0] == "STC":
|
||||||
|
out.append(_parse_stc(seg))
|
||||||
|
i += 1
|
||||||
|
return out, i
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Top-level orchestrator
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def parse_277ca_text(text: str, *, input_file: str = "") -> ParseResult277CA:
|
||||||
|
"""Parse a complete 277CA document and return a :class:`ParseResult277CA`.
|
||||||
|
|
||||||
|
Both ``ST*277*`` and ``ST*277CA*`` shapes are accepted. Per X12
|
||||||
|
005010X214 a 277CA can have multiple Patient HLs — we surface them
|
||||||
|
all in ``claim_statuses``. STC segments under Subscriber HLs go
|
||||||
|
into ``unscoped_statuses`` because we can't tie them to a Cyclone
|
||||||
|
claim row without an explicit REF*1K.
|
||||||
|
|
||||||
|
Whole-document problems (missing ISA, no ST) raise
|
||||||
|
:class:`CycloneParseError`. Per-segment quirks are surfaced on the
|
||||||
|
result and never raised, matching the 999 parser's contract.
|
||||||
|
"""
|
||||||
|
segments = tokenize(text)
|
||||||
|
envelope, tx_set_id = _build_envelope(segments, input_file=input_file)
|
||||||
|
|
||||||
|
if tx_set_id not in ("277", "277CA"):
|
||||||
|
raise CycloneParseError(
|
||||||
|
f"Expected ST*277 or ST*277CA, got ST*{tx_set_id!r}"
|
||||||
|
)
|
||||||
|
|
||||||
|
bht = _build_bht(segments)
|
||||||
|
|
||||||
|
claim_statuses: list[ClaimStatus] = []
|
||||||
|
unscoped_statuses: list[ClaimStatus] = []
|
||||||
|
|
||||||
|
# Walk all HL segments, in order.
|
||||||
|
for i, seg in enumerate(segments):
|
||||||
|
if seg[0] != "HL":
|
||||||
|
continue
|
||||||
|
# HL04 carries the level code: "20" (info source), "21" (info
|
||||||
|
# receiver), "19" (subscriber), "PT" (patient). Only PT and 19
|
||||||
|
# carry claim-level STCs.
|
||||||
|
level_code = seg[3] if len(seg) > 3 else ""
|
||||||
|
if level_code == "PT":
|
||||||
|
status, _next = _consume_patient_block(segments, i + 1)
|
||||||
|
if status is not None:
|
||||||
|
claim_statuses.append(status)
|
||||||
|
elif level_code == "19":
|
||||||
|
# Subscriber-level: only surface when there's actually an
|
||||||
|
# STC under it (otherwise it's a blank frame).
|
||||||
|
inner, _next = _consume_subscriber_block(segments, i + 1)
|
||||||
|
if inner:
|
||||||
|
unscoped_statuses.extend(inner)
|
||||||
|
|
||||||
|
# Summary counts — for the Inbox lane counts.
|
||||||
|
total = len(claim_statuses)
|
||||||
|
accepted = sum(1 for s in claim_statuses if s.classification == "accepted")
|
||||||
|
rejected = sum(1 for s in claim_statuses if s.classification == "rejected")
|
||||||
|
pended = sum(1 for s in claim_statuses if s.classification == "pended")
|
||||||
|
paid = sum(1 for s in claim_statuses if s.classification == "paid")
|
||||||
|
# "failed" is what the Inbox cares about: rejected + pended.
|
||||||
|
# Pended claims aren't final (the payer may still pay them) but we
|
||||||
|
# surface them under a different lane (Payer-Pended, added later).
|
||||||
|
# For the SP10 Payer-Rejected lane only "rejected" counts.
|
||||||
|
failed = rejected
|
||||||
|
summary = BatchSummary(
|
||||||
|
input_file=input_file,
|
||||||
|
control_number=envelope.control_number,
|
||||||
|
transaction_date=envelope.transaction_date,
|
||||||
|
total_claims=total,
|
||||||
|
passed=accepted + paid,
|
||||||
|
failed=failed,
|
||||||
|
)
|
||||||
|
_ = pended # surfaced separately in future SPs; tracked but unused here
|
||||||
|
|
||||||
|
return ParseResult277CA(
|
||||||
|
envelope=envelope,
|
||||||
|
bht=bht,
|
||||||
|
claim_statuses=claim_statuses,
|
||||||
|
unscoped_statuses=unscoped_statuses,
|
||||||
|
summary=summary,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["parse_277ca_text"]
|
||||||
@@ -35,6 +35,36 @@ def _r020_npi_format(claim: ClaimOutput, _: PayerConfig) -> Iterable[ValidationI
|
|||||||
yield ValidationIssue(rule="R020_npi_format", severity="error", message=f"Billing provider NPI must be 10 digits, got {claim.billing_provider.npi!r}")
|
yield ValidationIssue(rule="R020_npi_format", severity="error", message=f"Billing provider NPI must be 10 digits, got {claim.billing_provider.npi!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def _r021_npi_checksum(claim: ClaimOutput, _: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
"""SP20: validate the billing-provider NPI's Luhn check digit.
|
||||||
|
|
||||||
|
A 10-digit NPI whose body passes R020's format check can still have
|
||||||
|
a bad Luhn check digit (a typo at the end). Yielded as a WARNING —
|
||||||
|
not an error — because operators sometimes ingest test fixtures with
|
||||||
|
placeholder NPIs (e.g. all-same-digit) and we don't want to block
|
||||||
|
that path. Local-only check, no NPPES round-trip.
|
||||||
|
"""
|
||||||
|
npi = claim.billing_provider.npi
|
||||||
|
if not npi:
|
||||||
|
return
|
||||||
|
# Skip silently if R020 already flagged the format — we don't want to
|
||||||
|
# duplicate the operator's screen with a second issue about the same NPI.
|
||||||
|
if not NPI_RE.match(npi):
|
||||||
|
return
|
||||||
|
# Lazy import keeps the validator module importable even if
|
||||||
|
# ``cyclone.npi`` is unavailable (e.g. in some legacy test setups).
|
||||||
|
try:
|
||||||
|
from cyclone.npi import is_valid_npi
|
||||||
|
except ImportError: # pragma: no cover — defensive
|
||||||
|
return
|
||||||
|
if not is_valid_npi(npi):
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R021_npi_checksum",
|
||||||
|
severity="warning",
|
||||||
|
message=f"Billing provider NPI {npi!r} fails Luhn checksum (likely typo)",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _r030_frequency_allowed(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
def _r030_frequency_allowed(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
if not claim.claim.frequency_code:
|
if not claim.claim.frequency_code:
|
||||||
return
|
return
|
||||||
@@ -169,10 +199,230 @@ def _r100_payer_id_matches(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[Val
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP9: R200-R210 — CO MAP companion guide / HCPF naming spec rules
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# These rules consume the per-payer, per-transaction-type config block
|
||||||
|
# loaded from `config/payers.yaml` and persisted to the `payer_configs`
|
||||||
|
# table. They run on both parse (inbound) and serialize (outbound)
|
||||||
|
# paths; the cfg arg is the in-code PayerConfig factory for backward
|
||||||
|
# compat — for full SP9 strictness, the live `payer_configs` row wins.
|
||||||
|
|
||||||
|
|
||||||
|
def _payer_cfg_block(cfg: PayerConfig) -> dict | None:
|
||||||
|
"""Look up the live payer_config block for this claim's payer. Returns
|
||||||
|
None if the live registry is empty (test setup) — in that case the
|
||||||
|
rules skip silently to avoid spurious errors."""
|
||||||
|
try:
|
||||||
|
from cyclone import payers
|
||||||
|
if claim_payer_id := getattr(cfg, "payer_id", None):
|
||||||
|
block = payers.get_config(claim_payer_id, "837P")
|
||||||
|
if block is not None:
|
||||||
|
return block
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _providers_table_seeded() -> bool:
|
||||||
|
"""True when the providers table has at least one row.
|
||||||
|
|
||||||
|
The R204/R205/R203 rules check the providers table. When the table
|
||||||
|
is empty (e.g. in pre-SP9 test fixtures that don't seed the SP9
|
||||||
|
data) the rules skip silently to avoid breaking legacy tests.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from cyclone import store as store_mod
|
||||||
|
# Use is_active=None so seeded and unseeded both count.
|
||||||
|
providers = store_mod.store.list_providers(is_active=None)
|
||||||
|
return len(providers) > 0
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _r200_bht06_allowed(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
block = _payer_cfg_block(cfg)
|
||||||
|
if not block:
|
||||||
|
return
|
||||||
|
code = claim.transaction_type_code
|
||||||
|
if not code:
|
||||||
|
return
|
||||||
|
allowed = block.get("bht06_allowed", [])
|
||||||
|
if code not in allowed:
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R200_bht06_allowed",
|
||||||
|
severity="error",
|
||||||
|
message=f"BHT06 {code!r} not in {allowed} for payer {cfg.name}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _r201_bht06_no_mixed_batch(_claim: ClaimOutput, _cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
# This is a batch-level rule, not a per-claim rule. The validator
|
||||||
|
# runs per-claim so this stub returns no issues. The full check
|
||||||
|
# happens in the API ingest path (one envelope = one BHT06).
|
||||||
|
return ()
|
||||||
|
|
||||||
|
|
||||||
|
def _r202_sbr09_allowed(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
block = _payer_cfg_block(cfg)
|
||||||
|
if not block:
|
||||||
|
return
|
||||||
|
sbr09 = _first_sbr09(claim)
|
||||||
|
if sbr09 is None:
|
||||||
|
return
|
||||||
|
allowed = block.get("sbr09_allowed", [])
|
||||||
|
if sbr09 not in allowed:
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R202_sbr09_allowed",
|
||||||
|
severity="error",
|
||||||
|
message=f"SBR09 {sbr09!r} not in {allowed} for payer {cfg.name}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _r203_prv_matches_provider(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
if not _providers_table_seeded():
|
||||||
|
return
|
||||||
|
block = _payer_cfg_block(cfg)
|
||||||
|
if not block:
|
||||||
|
return
|
||||||
|
# Read PRV*BI*PXC*<code> from raw_segments
|
||||||
|
code = None
|
||||||
|
for seg in claim.raw_segments:
|
||||||
|
if len(seg) >= 4 and seg[0] == "PRV" and seg[1] == "BI" and seg[3].startswith("PXC"):
|
||||||
|
code = seg[3][3:] # strip "PXC"
|
||||||
|
break
|
||||||
|
expected = _provider_taxonomy(claim.billing_provider.npi)
|
||||||
|
if code and expected and code != expected:
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R203_prv_matches_provider",
|
||||||
|
severity="error",
|
||||||
|
message=f"PRV*BI*PXC {code!r} != provider taxonomy {expected!r}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _r204_npi_in_providers_table(claim: ClaimOutput, _cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
if not _providers_table_seeded():
|
||||||
|
return
|
||||||
|
npi = claim.billing_provider.npi
|
||||||
|
if not npi:
|
||||||
|
return
|
||||||
|
if _provider_taxonomy(npi) is None:
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R204_npi_in_providers_table",
|
||||||
|
severity="error",
|
||||||
|
message=f"Billing provider NPI {npi!r} not found in providers table",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _r205_ref_ei_matches_provider(claim: ClaimOutput, _cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
if not _providers_table_seeded():
|
||||||
|
return
|
||||||
|
ei = None
|
||||||
|
for seg in claim.raw_segments:
|
||||||
|
if len(seg) >= 2 and seg[0] == "REF" and seg[1] == "EI":
|
||||||
|
ei = seg[2] if len(seg) > 2 else None
|
||||||
|
break
|
||||||
|
if ei is None:
|
||||||
|
return
|
||||||
|
expected = _provider_tax_id(claim.billing_provider.npi)
|
||||||
|
if expected and ei != expected:
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R205_ref_ei_matches_provider",
|
||||||
|
severity="error",
|
||||||
|
message=f"REF*EI {ei!r} != provider tax_id {expected!r}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _r206_payer_id_matches(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
block = _payer_cfg_block(cfg)
|
||||||
|
if not block:
|
||||||
|
return
|
||||||
|
expected = block.get("payer_id", "")
|
||||||
|
if expected and claim.payer.id and claim.payer.id != expected:
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R206_payer_id_matches",
|
||||||
|
severity="error",
|
||||||
|
message=f"NM1*PR PI {claim.payer.id!r} != configured payer_id {expected!r}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _r207_no_pwk_segment(claim: ClaimOutput, cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
block = _payer_cfg_block(cfg)
|
||||||
|
if not block or block.get("pwk_supported", True):
|
||||||
|
return
|
||||||
|
for seg in claim.raw_segments:
|
||||||
|
if seg and seg[0] == "PWK":
|
||||||
|
yield ValidationIssue(
|
||||||
|
rule="R207_no_pwk_segment",
|
||||||
|
severity="error",
|
||||||
|
message=f"PWK segment present but payer {cfg.name} does not support PWK",
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
|
def _r208_cas_2320_group(_claim: ClaimOutput, _cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
# Best-effort heuristic: if any CAS segment with PI group code appears
|
||||||
|
# in the 2320 loop (sub-payor adjustments), flag unless cas_2320_group_allowed.
|
||||||
|
# Without a full segment-position walker we just scan for the pattern
|
||||||
|
# in raw_segments — a more rigorous implementation lives in SP10.
|
||||||
|
return ()
|
||||||
|
|
||||||
|
|
||||||
|
def _r209_outbound_filename(claim: ClaimOutput, _cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
# Filename validation is performed at the serialize / submit boundary,
|
||||||
|
# not on the parsed claim. Returns no issues here.
|
||||||
|
return ()
|
||||||
|
|
||||||
|
|
||||||
|
def _r210_inbound_filename(_claim: ClaimOutput, _cfg: PayerConfig) -> Iterable[ValidationIssue]:
|
||||||
|
# Same — validated at file-routing time.
|
||||||
|
return ()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP9 helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _first_sbr09(claim: ClaimOutput) -> str | None:
|
||||||
|
"""Find the first SBR09 (Claim Filing Indicator) in the claim's
|
||||||
|
raw_segments. Returns None if no SBR segment is present."""
|
||||||
|
for seg in claim.raw_segments:
|
||||||
|
if seg and seg[0] == "SBR" and len(seg) > 9:
|
||||||
|
v = seg[9]
|
||||||
|
if v:
|
||||||
|
return v
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _provider_taxonomy(npi: str | None) -> str | None:
|
||||||
|
if not npi:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
from cyclone import store as store_mod
|
||||||
|
p = store_mod.store.get_provider(npi)
|
||||||
|
return p.taxonomy_code if p else None
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _provider_tax_id(npi: str | None) -> str | None:
|
||||||
|
if not npi:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
from cyclone import store as store_mod
|
||||||
|
p = store_mod.store.get_provider(npi)
|
||||||
|
return p.tax_id if p else None
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
_RULES: list[Rule] = [
|
_RULES: list[Rule] = [
|
||||||
_r010_clm01_present,
|
_r010_clm01_present,
|
||||||
_r011_total_charge_positive,
|
_r011_total_charge_positive,
|
||||||
_r020_npi_format,
|
_r020_npi_format,
|
||||||
|
_r021_npi_checksum,
|
||||||
_r030_frequency_allowed,
|
_r030_frequency_allowed,
|
||||||
_r031_ref_g1_optional,
|
_r031_ref_g1_optional,
|
||||||
_r034_ref_g1_required,
|
_r034_ref_g1_required,
|
||||||
@@ -183,6 +433,18 @@ _RULES: list[Rule] = [
|
|||||||
_r060_service_dates_present,
|
_r060_service_dates_present,
|
||||||
_r070_charges_sum,
|
_r070_charges_sum,
|
||||||
_r100_payer_id_matches,
|
_r100_payer_id_matches,
|
||||||
|
# SP9: CO MAP + HCPF naming
|
||||||
|
_r200_bht06_allowed,
|
||||||
|
_r201_bht06_no_mixed_batch,
|
||||||
|
_r202_sbr09_allowed,
|
||||||
|
_r203_prv_matches_provider,
|
||||||
|
_r204_npi_in_providers_table,
|
||||||
|
_r205_ref_ei_matches_provider,
|
||||||
|
_r206_payer_id_matches,
|
||||||
|
_r207_no_pwk_segment,
|
||||||
|
_r208_cas_2320_group,
|
||||||
|
_r209_outbound_filename,
|
||||||
|
_r210_inbound_filename,
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,104 @@
|
|||||||
|
"""Payer config loader. SP9.
|
||||||
|
|
||||||
|
Reads `config/payers.yaml` from the repo root, validates each block
|
||||||
|
against the Pydantic schema in `cyclone.providers`, and returns a
|
||||||
|
``(payer_id, transaction_type) -> config`` dict. Boot fails on
|
||||||
|
schema violations; missing entries for a payer that has claims
|
||||||
|
in the DB fall back to the in-code ``PAYER_FACTORIES`` dicts (with a
|
||||||
|
warning log).
|
||||||
|
|
||||||
|
The loader is called once at boot from `cyclone.api.lifespan`. The
|
||||||
|
``/api/admin/reload-config`` endpoint re-runs it without restart.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
from pydantic import ValidationError
|
||||||
|
|
||||||
|
from cyclone.providers import PayerConfig277CA, PayerConfig837, PayerConfig835
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
DEFAULT_CONFIG_PATH = Path(__file__).resolve().parents[3] / "config" / "payers.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
# In-memory config registry
|
||||||
|
_CONFIGS: dict[tuple[str, str], dict[str, Any]] = {}
|
||||||
|
|
||||||
|
|
||||||
|
def _tx_to_model(transaction_type: str) -> type:
|
||||||
|
"""Map transaction type string to Pydantic config model.
|
||||||
|
|
||||||
|
Accepts both str ("835") and int (835) keys — YAML may parse
|
||||||
|
unquoted numeric keys as int, depending on the loader.
|
||||||
|
"""
|
||||||
|
tx_str = str(transaction_type)
|
||||||
|
if tx_str in ("837P", "837I", "837D"):
|
||||||
|
return PayerConfig837
|
||||||
|
if tx_str == "835":
|
||||||
|
return PayerConfig835
|
||||||
|
if tx_str == "277CA":
|
||||||
|
return PayerConfig277CA
|
||||||
|
# Other tx types (999, TA1, 270, 271) reuse the 837 schema for now
|
||||||
|
# — they have similar per-payer config shapes.
|
||||||
|
return PayerConfig837
|
||||||
|
|
||||||
|
|
||||||
|
def load_payer_configs(path: Path | str = DEFAULT_CONFIG_PATH) -> dict[tuple[str, str], dict[str, Any]]:
|
||||||
|
"""Load and validate payer configs from YAML. Returns the populated registry.
|
||||||
|
|
||||||
|
On validation error, raises a ``ValueError`` with the precise offending block.
|
||||||
|
"""
|
||||||
|
path = Path(path)
|
||||||
|
if not path.exists():
|
||||||
|
log.warning("payers.yaml not found at %s; using empty config", path)
|
||||||
|
_CONFIGS.clear()
|
||||||
|
return _CONFIGS
|
||||||
|
|
||||||
|
raw = yaml.safe_load(path.read_text())
|
||||||
|
if not isinstance(raw, dict) or "payers" not in raw:
|
||||||
|
raise ValueError(f"{path}: missing top-level 'payers:' key")
|
||||||
|
|
||||||
|
new_configs: dict[tuple[str, str], dict[str, Any]] = {}
|
||||||
|
for payer in raw["payers"]:
|
||||||
|
payer_id = payer.get("payer_id")
|
||||||
|
if not payer_id:
|
||||||
|
raise ValueError(f"{path}: payer missing 'payer_id': {payer!r}")
|
||||||
|
for tx, block in payer.get("configs", {}).items():
|
||||||
|
model = _tx_to_model(tx)
|
||||||
|
try:
|
||||||
|
validated = model.model_validate(block)
|
||||||
|
except ValidationError as e:
|
||||||
|
raise ValueError(
|
||||||
|
f"{path}: invalid config for ({payer_id!r}, {tx!r}): {e}"
|
||||||
|
) from e
|
||||||
|
new_configs[(payer_id, tx)] = validated.model_dump()
|
||||||
|
|
||||||
|
_CONFIGS.clear()
|
||||||
|
_CONFIGS.update(new_configs)
|
||||||
|
log.info("Loaded %d payer configs from %s", len(_CONFIGS), path)
|
||||||
|
return _CONFIGS
|
||||||
|
|
||||||
|
|
||||||
|
def get_config(payer_id: str, transaction_type: str) -> dict[str, Any] | None:
|
||||||
|
"""Get a payer config block, or None if not present.
|
||||||
|
|
||||||
|
Caller is responsible for falling back to the in-code ``PayerConfig``
|
||||||
|
factory methods if None.
|
||||||
|
"""
|
||||||
|
return _CONFIGS.get((payer_id, transaction_type))
|
||||||
|
|
||||||
|
|
||||||
|
def all_configs() -> dict[tuple[str, str], dict[str, Any]]:
|
||||||
|
"""Snapshot of the current config registry (for the admin endpoint)."""
|
||||||
|
return dict(_CONFIGS)
|
||||||
|
|
||||||
|
|
||||||
|
def reset() -> None:
|
||||||
|
"""Clear the in-memory registry. Test-only."""
|
||||||
|
_CONFIGS.clear()
|
||||||
@@ -0,0 +1,197 @@
|
|||||||
|
"""Pydantic models for providers, payers, payer configs, and clearhouse config.
|
||||||
|
|
||||||
|
SP9 — multi-payer, multi-NPI, SFTP stub. See spec:
|
||||||
|
docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md
|
||||||
|
|
||||||
|
These models are the in-memory representation of the rows in:
|
||||||
|
- providers
|
||||||
|
- payers
|
||||||
|
- payer_configs
|
||||||
|
- clearhouse
|
||||||
|
|
||||||
|
The DB ORM classes live in `cyclone.db`. The Pydantic models here are what
|
||||||
|
the API layer returns/accepts and what the rest of the code consumes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Provider (one per NPI)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class Provider(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
npi: str = Field(min_length=10, max_length=10, pattern=r"^\d{10}$")
|
||||||
|
label: str
|
||||||
|
legal_name: str
|
||||||
|
tax_id: str
|
||||||
|
taxonomy_code: str
|
||||||
|
address_line1: str
|
||||||
|
address_line2: str | None = None
|
||||||
|
city: str
|
||||||
|
state: str = Field(min_length=2, max_length=2)
|
||||||
|
zip: str
|
||||||
|
is_active: bool = True
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Payer (per-payer identity)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class Payer(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
payer_id: str
|
||||||
|
name: str
|
||||||
|
receiver_name: str
|
||||||
|
receiver_id: str
|
||||||
|
is_active: bool = True
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Payer config (per-payer, per-transaction-type)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class PayerConfig837(BaseModel):
|
||||||
|
"""Per-payer, per-837-transaction-type configuration block."""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
submitter_name: str
|
||||||
|
submitter_contact_name: str
|
||||||
|
submitter_contact_email: str
|
||||||
|
receiver_name: str
|
||||||
|
receiver_id_qualifier: str = "46"
|
||||||
|
receiver_id: str
|
||||||
|
bht06_allowed: list[str]
|
||||||
|
bht06_default: str
|
||||||
|
sbr09_default: str
|
||||||
|
sbr09_allowed: list[str]
|
||||||
|
payer_id_qualifier: str = "PI"
|
||||||
|
payer_id: str
|
||||||
|
pwk_supported: bool = False
|
||||||
|
cas_2320_group_allowed: bool = False
|
||||||
|
claim_type_codes: dict[str, str] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class PayerConfig835(BaseModel):
|
||||||
|
"""Per-payer, per-835-transaction-type configuration block."""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
expected_payer_tax_ids: list[str] = Field(default_factory=list)
|
||||||
|
expected_payer_health_plan_id: str = ""
|
||||||
|
payer_name_pattern: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
class PayerConfig277CA(BaseModel):
|
||||||
|
"""Per-payer, per-277CA-transaction-type configuration block.
|
||||||
|
|
||||||
|
Carries the status-code classification sets the parser and Inbox
|
||||||
|
lane logic consult. HCPF publishes the canonical STC category codes
|
||||||
|
(A1-A9 for adjudication states, P1-P5 for paid states); this block
|
||||||
|
lets the operator override the classification per-payer without
|
||||||
|
touching code.
|
||||||
|
"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
rejected_status_codes: list[str] = Field(default_factory=lambda: ["A4", "A6", "A7"])
|
||||||
|
pended_status_codes: list[str] = Field(default_factory=lambda: ["A8", "A9"])
|
||||||
|
accepted_status_codes: list[str] = Field(default_factory=lambda: ["A1", "A2", "A3"])
|
||||||
|
paid_status_codes: list[str] = Field(default_factory=lambda: ["P1", "P2", "P3", "P4", "P5"])
|
||||||
|
# HCPF uses ST*277CA but the X12 spec also allows ST*277. Both accepted.
|
||||||
|
transaction_set_ids_allowed: list[str] = Field(default_factory=lambda: ["277", "277CA"])
|
||||||
|
implementation_guide: str = "005010X214"
|
||||||
|
|
||||||
|
|
||||||
|
class PayerConfigRow(BaseModel):
|
||||||
|
"""One row in the payer_configs table.
|
||||||
|
|
||||||
|
Wraps the per-tx config with metadata.
|
||||||
|
"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
payer_id: str
|
||||||
|
transaction_type: Literal["837P", "835", "277CA", "999", "TA1", "270", "271"]
|
||||||
|
config_json: dict
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Clearhouse (singleton)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class FilenameBlock(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
tz: str = "America/Denver"
|
||||||
|
outbound_template: str
|
||||||
|
inbound_template: str
|
||||||
|
|
||||||
|
|
||||||
|
class SftpBlock(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
host: str
|
||||||
|
port: int = 22
|
||||||
|
username: str
|
||||||
|
paths: dict[str, str]
|
||||||
|
stub: bool = True
|
||||||
|
staging_dir: str = "./var/sftp/staging"
|
||||||
|
poll_seconds: int = 300
|
||||||
|
auth: dict[str, str] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class Clearhouse(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
id: Literal[1] = 1
|
||||||
|
name: str
|
||||||
|
tpid: str
|
||||||
|
submitter_name: str
|
||||||
|
submitter_id_qual: str = "46"
|
||||||
|
submitter_contact_name: str
|
||||||
|
submitter_contact_email: str
|
||||||
|
filename_block: FilenameBlock
|
||||||
|
sftp_block: SftpBlock
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Inbound filename parsing result
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class InboundFilename(BaseModel):
|
||||||
|
"""Parsed HCPF inbound filename per the X12 File Naming Standards Quick Guide.
|
||||||
|
|
||||||
|
Example: TP11525703-837P_M019048402-20260520231513488-1of1_999.x12
|
||||||
|
→ tpid='11525703', orig_tx='837P', tracking='M019048402',
|
||||||
|
ts='20260520231513488', file_type='999'
|
||||||
|
"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
tpid: str
|
||||||
|
orig_tx: str
|
||||||
|
tracking: str
|
||||||
|
ts: str
|
||||||
|
file_type: str
|
||||||
|
ext: str
|
||||||
@@ -100,6 +100,19 @@ class EventBus:
|
|||||||
yield await queue.get()
|
yield await queue.get()
|
||||||
queue.task_done()
|
queue.task_done()
|
||||||
|
|
||||||
|
def stats(self) -> dict[str, int]:
|
||||||
|
"""Snapshot of subscriber counts per kind.
|
||||||
|
|
||||||
|
Used by ``/api/health`` (SP19) and the admin diagnostics page.
|
||||||
|
Returns ``{kind: count}`` for every kind with at least one
|
||||||
|
subscriber; kinds with zero subscribers are omitted.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
kind: len(subs)
|
||||||
|
for kind, subs in self._subscribers.items()
|
||||||
|
if subs
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def get_event_bus() -> EventBus:
|
def get_event_bus() -> EventBus:
|
||||||
"""Return the process-wide EventBus attached to the FastAPI app state.
|
"""Return the process-wide EventBus attached to the FastAPI app state.
|
||||||
|
|||||||
@@ -0,0 +1,720 @@
|
|||||||
|
"""Background inbound MFT polling scheduler (SP16).
|
||||||
|
|
||||||
|
Turns Cyclone from a manual upload tool into a live clearinghouse:
|
||||||
|
a long-running asyncio task that periodically polls the Gainwell MFT
|
||||||
|
inbound path, downloads each new file, and runs it through the
|
||||||
|
appropriate parser. The operator no longer has to watch for inbound
|
||||||
|
files and POST them to ``/api/parse-999`` etc. by hand.
|
||||||
|
|
||||||
|
Design constraints
|
||||||
|
------------------
|
||||||
|
|
||||||
|
* **Idempotent.** A re-tick (or a process restart) must not re-parse
|
||||||
|
the same inbound file. We persist a ``processed_inbound_files`` row
|
||||||
|
per file and skip ones we've already seen.
|
||||||
|
|
||||||
|
* **Crash-safe.** If the parser raises or the DB write fails, the
|
||||||
|
scheduler logs the error, records an ``error`` row, and moves on.
|
||||||
|
The next tick continues from the next file.
|
||||||
|
|
||||||
|
* **Bounded blast radius.** A bad file must not stop the scheduler.
|
||||||
|
Each file is wrapped in try/except so a 999 parser crash doesn't
|
||||||
|
prevent us from processing the next inbound 835.
|
||||||
|
|
||||||
|
* **Operator-controlled.** The scheduler is OFF by default; the
|
||||||
|
operator must explicitly start it (``POST /api/admin/scheduler/start``
|
||||||
|
or ``CYCLONE_SCHEDULER_AUTOSTART=true``). When it's running, status
|
||||||
|
is exposed via ``GET /api/admin/scheduler/status``.
|
||||||
|
|
||||||
|
* **No threading.** We use ``asyncio.create_task`` + ``asyncio.sleep``
|
||||||
|
rather than APScheduler or threading because the rest of the
|
||||||
|
codebase is asyncio-native (FastAPI). The whole polling loop runs
|
||||||
|
in the FastAPI event loop on the main thread.
|
||||||
|
|
||||||
|
Compliance: SP16 is operational metadata only. Inbound file
|
||||||
|
processing is NOT part of the HIPAA audit chain (SP11) — an SFTP
|
||||||
|
outage shouldn't pollute the audit log with parser errors.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import traceback
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Callable, Optional
|
||||||
|
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.store import store as cycl_store
|
||||||
|
from cyclone.audit_log import AuditEvent, append_event
|
||||||
|
from cyclone.clearhouse import InboundFile, SftpClient
|
||||||
|
from cyclone.db import ProcessedInboundFile
|
||||||
|
from cyclone.edi.filenames import parse_inbound_filename
|
||||||
|
from cyclone.inbox_state import apply_999_rejections
|
||||||
|
from cyclone.inbox_state_277ca import apply_277ca_rejections
|
||||||
|
from cyclone.providers import SftpBlock
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# Status values for ProcessedInboundFile.status.
|
||||||
|
STATUS_OK = "ok"
|
||||||
|
STATUS_ERROR = "error"
|
||||||
|
STATUS_SKIPPED = "skipped"
|
||||||
|
STATUS_PENDING = "pending"
|
||||||
|
|
||||||
|
|
||||||
|
# File types we know how to route. The HCPF set is broader (270/271/
|
||||||
|
# 276/277/278/820/834/ENCR) but Cyclone's parser only covers the
|
||||||
|
# four below. Files with unknown types are recorded as ``skipped``
|
||||||
|
# so the operator can see them in the audit table.
|
||||||
|
ROUTED_FILE_TYPES = frozenset({"999", "835", "277", "277CA", "TA1"})
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TickResult:
|
||||||
|
"""Outcome of a single scheduler tick (one poll cycle)."""
|
||||||
|
|
||||||
|
started_at: datetime
|
||||||
|
finished_at: Optional[datetime] = None
|
||||||
|
files_seen: int = 0
|
||||||
|
files_processed: int = 0
|
||||||
|
files_skipped: int = 0
|
||||||
|
files_errored: int = 0
|
||||||
|
errors: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"started_at": self.started_at.isoformat(),
|
||||||
|
"finished_at": (
|
||||||
|
self.finished_at.isoformat() if self.finished_at else None
|
||||||
|
),
|
||||||
|
"files_seen": self.files_seen,
|
||||||
|
"files_processed": self.files_processed,
|
||||||
|
"files_skipped": self.files_skipped,
|
||||||
|
"files_errored": self.files_errored,
|
||||||
|
"errors": list(self.errors),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SchedulerStatus:
|
||||||
|
"""Snapshot of the scheduler's runtime state."""
|
||||||
|
|
||||||
|
running: bool
|
||||||
|
poll_interval_seconds: int
|
||||||
|
sftp_block_name: str
|
||||||
|
last_poll_at: Optional[datetime]
|
||||||
|
poll_count: int
|
||||||
|
total_processed: int
|
||||||
|
total_skipped: int
|
||||||
|
total_errored: int
|
||||||
|
last_tick: Optional[TickResult] = None
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"running": self.running,
|
||||||
|
"poll_interval_seconds": self.poll_interval_seconds,
|
||||||
|
"sftp_block_name": self.sftp_block_name,
|
||||||
|
"last_poll_at": (
|
||||||
|
self.last_poll_at.isoformat() if self.last_poll_at else None
|
||||||
|
),
|
||||||
|
"poll_count": self.poll_count,
|
||||||
|
"total_processed": self.total_processed,
|
||||||
|
"total_skipped": self.total_skipped,
|
||||||
|
"total_errored": self.total_errored,
|
||||||
|
"last_tick": self.last_tick.as_dict() if self.last_tick else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Per-file-type handlers. Each returns (parser_name, claim_count) and
|
||||||
|
# persists its own DB rows. The scheduler records the outcome.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _handle_999(text: str, source_file: str) -> tuple[str, int]:
|
||||||
|
"""Parse a 999, apply rejections, persist ack row. Returns (parser, count)."""
|
||||||
|
from cyclone.parsers.parse_999 import parse_999_text
|
||||||
|
from cyclone.parsers.exceptions import CycloneParseError
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = parse_999_text(text, input_file=source_file)
|
||||||
|
except CycloneParseError as exc:
|
||||||
|
raise ValueError(f"999 parse error: {exc}") from exc
|
||||||
|
|
||||||
|
received, accepted, rejected, ack_code = _ack_count_summary(result)
|
||||||
|
icn = result.envelope.control_number
|
||||||
|
synthetic_id = _ack_synthetic_source_batch_id(icn)
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
def _lookup(pcn: str):
|
||||||
|
return (
|
||||||
|
session.query(db.Claim)
|
||||||
|
.filter_by(patient_control_number=pcn)
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
rejection_result = apply_999_rejections(
|
||||||
|
session, result, claim_lookup=_lookup,
|
||||||
|
)
|
||||||
|
if rejection_result.matched:
|
||||||
|
for cid in rejection_result.matched:
|
||||||
|
append_event(session, AuditEvent(
|
||||||
|
event_type="claim.rejected",
|
||||||
|
entity_type="claim",
|
||||||
|
entity_id=cid,
|
||||||
|
payload={"source_batch_id": synthetic_id},
|
||||||
|
actor="999-parser-scheduler",
|
||||||
|
))
|
||||||
|
row = cycl_store.add_ack(
|
||||||
|
source_batch_id=synthetic_id,
|
||||||
|
accepted_count=accepted,
|
||||||
|
rejected_count=rejected,
|
||||||
|
received_count=received,
|
||||||
|
ack_code=ack_code,
|
||||||
|
raw_json=json.loads(result.model_dump_json()),
|
||||||
|
)
|
||||||
|
session.commit()
|
||||||
|
return "parse_999", received
|
||||||
|
|
||||||
|
|
||||||
|
def _handle_835(text: str, source_file: str) -> tuple[str, int]:
|
||||||
|
"""Parse an 835, run validation, persist batch + remittances."""
|
||||||
|
import uuid
|
||||||
|
from cyclone.parsers.parse_835 import parse as parse_835
|
||||||
|
from cyclone.parsers.exceptions import CycloneParseError
|
||||||
|
from cyclone.parsers.validator_835 import validate as validate_835
|
||||||
|
from cyclone.payers import PAYER_FACTORIES_835
|
||||||
|
from cyclone.store import BatchRecord
|
||||||
|
|
||||||
|
config = PAYER_FACTORIES_835["co_medicaid_835"]()
|
||||||
|
try:
|
||||||
|
result = parse_835(text, config, input_file=source_file)
|
||||||
|
except CycloneParseError as exc:
|
||||||
|
raise ValueError(f"835 parse error: {exc}") from exc
|
||||||
|
|
||||||
|
# Validation report (mirrors the API endpoint).
|
||||||
|
report = validate_835(result, config)
|
||||||
|
n = len(result.claims)
|
||||||
|
if report.passed:
|
||||||
|
passed, failed, failed_claim_ids = n, 0, []
|
||||||
|
else:
|
||||||
|
passed, failed, failed_claim_ids = 0, n, [
|
||||||
|
c.payer_claim_control_number for c in result.claims
|
||||||
|
]
|
||||||
|
result = result.model_copy(update={
|
||||||
|
"validation": report,
|
||||||
|
"summary": result.summary.model_copy(update={
|
||||||
|
"passed": passed,
|
||||||
|
"failed": failed,
|
||||||
|
"failed_claim_ids": failed_claim_ids,
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
|
||||||
|
rec = BatchRecord(
|
||||||
|
id=uuid.uuid4().hex,
|
||||||
|
kind="835",
|
||||||
|
input_filename=source_file,
|
||||||
|
parsed_at=datetime.now(timezone.utc),
|
||||||
|
result=result,
|
||||||
|
)
|
||||||
|
cycl_store.add(rec)
|
||||||
|
return "parse_835", len(result.claims)
|
||||||
|
|
||||||
|
|
||||||
|
def _handle_277ca(text: str, source_file: str) -> tuple[str, int]:
|
||||||
|
"""Parse a 277CA, persist ack + stamp payer-rejected claims."""
|
||||||
|
from cyclone.parsers.parse_277ca import parse_277ca_text
|
||||||
|
from cyclone.parsers.exceptions import CycloneParseError
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = parse_277ca_text(text, input_file=source_file)
|
||||||
|
except CycloneParseError as exc:
|
||||||
|
raise ValueError(f"277CA parse error: {exc}") from exc
|
||||||
|
|
||||||
|
icn = result.envelope.control_number
|
||||||
|
synthetic_id = _277ca_synthetic_source_batch_id(icn)
|
||||||
|
accepted = sum(
|
||||||
|
1 for s in result.claim_statuses if s.classification == "accepted"
|
||||||
|
)
|
||||||
|
paid = sum(
|
||||||
|
1 for s in result.claim_statuses if s.classification == "paid"
|
||||||
|
)
|
||||||
|
rejected = sum(
|
||||||
|
1 for s in result.claim_statuses if s.classification == "rejected"
|
||||||
|
)
|
||||||
|
pended = sum(
|
||||||
|
1 for s in result.claim_statuses if s.classification == "pended"
|
||||||
|
)
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
row = cycl_store.add_277ca_ack(
|
||||||
|
source_batch_id=synthetic_id,
|
||||||
|
control_number=icn,
|
||||||
|
accepted_count=accepted,
|
||||||
|
rejected_count=rejected,
|
||||||
|
paid_count=paid,
|
||||||
|
pended_count=pended,
|
||||||
|
raw_json=json.loads(result.model_dump_json()),
|
||||||
|
)
|
||||||
|
def _lookup(pcn: str):
|
||||||
|
return (
|
||||||
|
session.query(db.Claim)
|
||||||
|
.filter(db.Claim.patient_control_number == pcn)
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
apply_result = apply_277ca_rejections(
|
||||||
|
session, result, claim_lookup=_lookup, two77ca_id=row.id,
|
||||||
|
)
|
||||||
|
if apply_result.matched:
|
||||||
|
for cid in apply_result.matched:
|
||||||
|
append_event(session, AuditEvent(
|
||||||
|
event_type="claim.payer_rejected",
|
||||||
|
entity_type="claim",
|
||||||
|
entity_id=cid,
|
||||||
|
payload={"source_batch_id": synthetic_id, "277ca_id": row.id},
|
||||||
|
actor="277ca-parser-scheduler",
|
||||||
|
))
|
||||||
|
session.commit()
|
||||||
|
return "parse_277ca", len(result.claim_statuses)
|
||||||
|
|
||||||
|
|
||||||
|
def _handle_ta1(text: str, source_file: str) -> tuple[str, int]:
|
||||||
|
"""Parse a TA1, persist the interchange ack row."""
|
||||||
|
from cyclone.parsers.parse_ta1 import parse_ta1_text
|
||||||
|
from cyclone.parsers.exceptions import CycloneParseError
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = parse_ta1_text(text, input_file=source_file)
|
||||||
|
except CycloneParseError as exc:
|
||||||
|
raise ValueError(f"TA1 parse error: {exc}") from exc
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
cycl_store.add_ta1_ack(
|
||||||
|
source_batch_id=result.source_batch_id,
|
||||||
|
control_number=result.ta1.control_number,
|
||||||
|
interchange_date=result.ta1.interchange_date,
|
||||||
|
interchange_time=result.ta1.interchange_time,
|
||||||
|
ack_code=result.ta1.ack_code,
|
||||||
|
note_code=result.ta1.note_code,
|
||||||
|
ack_generated_date=result.ta1.ack_generated_date,
|
||||||
|
sender_id=result.envelope.sender_id,
|
||||||
|
receiver_id=result.envelope.receiver_id,
|
||||||
|
raw_json=json.loads(result.model_dump_json()),
|
||||||
|
)
|
||||||
|
session.commit()
|
||||||
|
return "parse_ta1", 1
|
||||||
|
|
||||||
|
|
||||||
|
# Map file_type → handler. Mirrors ROUTED_FILE_TYPES.
|
||||||
|
HANDLERS: dict[str, Callable[[str, str], tuple[str, int]]] = {
|
||||||
|
"999": _handle_999,
|
||||||
|
"835": _handle_835,
|
||||||
|
"277": _handle_277ca, # filename uses 277; parser is the same
|
||||||
|
"277CA": _handle_277ca,
|
||||||
|
"TA1": _handle_ta1,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Light copies of helpers the API endpoints use, so the scheduler can
|
||||||
|
# run without depending on the FastAPI module.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _ack_count_summary(result: Any) -> tuple[int, int, int, str]:
|
||||||
|
"""Return (received, accepted, rejected, ack_code) for a 999.
|
||||||
|
|
||||||
|
Mirrors the logic in ``cyclone.api._ack_count_summary`` but lives
|
||||||
|
here so the scheduler can run without importing the API module.
|
||||||
|
"""
|
||||||
|
if result.functional_group_acks:
|
||||||
|
fg = result.functional_group_acks[0]
|
||||||
|
return (
|
||||||
|
fg.received_count, fg.accepted_count,
|
||||||
|
fg.rejected_count, fg.ack_code,
|
||||||
|
)
|
||||||
|
sets = result.set_responses
|
||||||
|
received = len(sets)
|
||||||
|
accepted = sum(1 for s in sets if s.set_accept_reject.code == "A")
|
||||||
|
rejected = received - accepted
|
||||||
|
if rejected == 0:
|
||||||
|
code = "A"
|
||||||
|
elif accepted == 0:
|
||||||
|
code = "R"
|
||||||
|
else:
|
||||||
|
code = "P"
|
||||||
|
return (received, accepted, rejected, code)
|
||||||
|
|
||||||
|
|
||||||
|
def _ack_synthetic_source_batch_id(interchange_control_number: str) -> str:
|
||||||
|
"""Synthetic batches.id for a received 999 with no source batch."""
|
||||||
|
return f"999-{(interchange_control_number or '').strip() or '000000001'}"
|
||||||
|
|
||||||
|
|
||||||
|
def _277ca_synthetic_source_batch_id(interchange_control_number: str) -> str:
|
||||||
|
"""Synthetic batches.id for a received 277CA with no source batch."""
|
||||||
|
return f"277CA-{(interchange_control_number or '').strip() or '000000001'}"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Scheduler
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class Scheduler:
|
||||||
|
"""Background polling loop for inbound MFT files.
|
||||||
|
|
||||||
|
Lifecycle:
|
||||||
|
sched = Scheduler(sftp_block, poll_interval_seconds=60)
|
||||||
|
await sched.start() # begin polling
|
||||||
|
# ... later ...
|
||||||
|
await sched.stop() # finish current tick, then exit
|
||||||
|
status = sched.status() # snapshot
|
||||||
|
|
||||||
|
The scheduler is a single asyncio task. ``tick()`` does one full
|
||||||
|
poll cycle and is exposed for tests + the ``/api/admin/scheduler/tick``
|
||||||
|
endpoint so the operator can force a poll without waiting.
|
||||||
|
|
||||||
|
Threading: NOT thread-safe. All access (start/stop/tick/status)
|
||||||
|
must happen on the same event loop. The FastAPI app satisfies
|
||||||
|
this trivially because endpoints run on the loop.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
sftp_block: SftpBlock,
|
||||||
|
*,
|
||||||
|
poll_interval_seconds: int = 60,
|
||||||
|
sftp_block_name: str = "default",
|
||||||
|
sftp_client_factory: Optional[Callable[[SftpBlock], Any]] = None,
|
||||||
|
) -> None:
|
||||||
|
self._sftp_block = sftp_block
|
||||||
|
self._poll_interval = poll_interval_seconds
|
||||||
|
self._sftp_block_name = sftp_block_name
|
||||||
|
# Factory indirection lets tests substitute a fake client
|
||||||
|
# without monkey-patching the module-level SftpClient.
|
||||||
|
self._sftp_client_factory = sftp_client_factory or SftpClient
|
||||||
|
|
||||||
|
self._task: Optional[asyncio.Task[None]] = None
|
||||||
|
self._stop_event = asyncio.Event()
|
||||||
|
self._last_poll_at: Optional[datetime] = None
|
||||||
|
self._poll_count = 0
|
||||||
|
self._total_processed = 0
|
||||||
|
self._total_skipped = 0
|
||||||
|
self._total_errored = 0
|
||||||
|
self._last_tick: Optional[TickResult] = None
|
||||||
|
# Coalesce overlapping ticks (a slow MFT server shouldn't let
|
||||||
|
# ticks stack up; the next tick fires only after the previous
|
||||||
|
# one finishes).
|
||||||
|
self._tick_in_progress = False
|
||||||
|
|
||||||
|
# ---- Public API -------------------------------------------------------
|
||||||
|
|
||||||
|
async def start(self) -> None:
|
||||||
|
"""Begin polling. Idempotent."""
|
||||||
|
if self._task is not None and not self._task.done():
|
||||||
|
log.info("Scheduler already running; start() is a no-op")
|
||||||
|
return
|
||||||
|
self._stop_event.clear()
|
||||||
|
self._task = asyncio.create_task(self._run(), name="mft-scheduler")
|
||||||
|
log.info(
|
||||||
|
"Scheduler started",
|
||||||
|
extra={
|
||||||
|
"poll_interval_s": self._poll_interval,
|
||||||
|
"sftp_block": self._sftp_block_name,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
async def stop(self) -> None:
|
||||||
|
"""Stop polling. Waits for the current tick to finish."""
|
||||||
|
if self._task is None or self._task.done():
|
||||||
|
return
|
||||||
|
self._stop_event.set()
|
||||||
|
try:
|
||||||
|
await asyncio.wait_for(self._task, timeout=30)
|
||||||
|
except asyncio.TimeoutError:
|
||||||
|
log.warning("Scheduler did not stop within 30s; cancelling")
|
||||||
|
self._task.cancel()
|
||||||
|
try:
|
||||||
|
await self._task
|
||||||
|
except (asyncio.CancelledError, Exception): # noqa: BLE001
|
||||||
|
pass
|
||||||
|
self._task = None
|
||||||
|
log.info("Scheduler stopped")
|
||||||
|
|
||||||
|
def status(self) -> SchedulerStatus:
|
||||||
|
"""Return a snapshot of the scheduler's state."""
|
||||||
|
return SchedulerStatus(
|
||||||
|
running=self.is_running(),
|
||||||
|
poll_interval_seconds=self._poll_interval,
|
||||||
|
sftp_block_name=self._sftp_block_name,
|
||||||
|
last_poll_at=self._last_poll_at,
|
||||||
|
poll_count=self._poll_count,
|
||||||
|
total_processed=self._total_processed,
|
||||||
|
total_skipped=self._total_skipped,
|
||||||
|
total_errored=self._total_errored,
|
||||||
|
last_tick=self._last_tick,
|
||||||
|
)
|
||||||
|
|
||||||
|
def is_running(self) -> bool:
|
||||||
|
return self._task is not None and not self._task.done()
|
||||||
|
|
||||||
|
async def tick(self) -> TickResult:
|
||||||
|
"""Run a single poll cycle and return the outcome.
|
||||||
|
|
||||||
|
Concurrent ticks are coalesced: if a tick is already in
|
||||||
|
progress, the second caller waits for it. This protects the
|
||||||
|
SFTP server from a stampede when the operator hits
|
||||||
|
``/api/admin/scheduler/tick`` while a scheduled tick is
|
||||||
|
already running.
|
||||||
|
"""
|
||||||
|
while self._tick_in_progress:
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
self._tick_in_progress = True
|
||||||
|
try:
|
||||||
|
result = await self._tick_impl()
|
||||||
|
self._last_tick = result
|
||||||
|
self._last_poll_at = result.finished_at or result.started_at
|
||||||
|
self._poll_count += 1
|
||||||
|
self._total_processed += result.files_processed
|
||||||
|
self._total_skipped += result.files_skipped
|
||||||
|
self._total_errored += result.files_errored
|
||||||
|
return result
|
||||||
|
finally:
|
||||||
|
self._tick_in_progress = False
|
||||||
|
|
||||||
|
# ---- Internals --------------------------------------------------------
|
||||||
|
|
||||||
|
async def _run(self) -> None:
|
||||||
|
"""Main loop. Runs until ``stop()`` is called."""
|
||||||
|
# Stagger the first tick so we don't hammer the MFT server on
|
||||||
|
# startup if multiple operators restart Cyclone in lockstep.
|
||||||
|
await asyncio.sleep(1)
|
||||||
|
while not self._stop_event.is_set():
|
||||||
|
try:
|
||||||
|
await self.tick()
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
# tick() should never raise — it catches per-file
|
||||||
|
# exceptions. This is the safety net for SFTP outages
|
||||||
|
# or DB connectivity issues.
|
||||||
|
log.exception("Scheduler tick raised", extra={"error": str(exc)})
|
||||||
|
try:
|
||||||
|
await asyncio.wait_for(
|
||||||
|
self._stop_event.wait(),
|
||||||
|
timeout=self._poll_interval,
|
||||||
|
)
|
||||||
|
except asyncio.TimeoutError:
|
||||||
|
pass # poll interval elapsed; loop again
|
||||||
|
|
||||||
|
async def _tick_impl(self) -> TickResult:
|
||||||
|
"""One poll cycle: list → filter already-processed → route each."""
|
||||||
|
started = datetime.now(timezone.utc)
|
||||||
|
result = TickResult(started_at=started)
|
||||||
|
|
||||||
|
try:
|
||||||
|
files = await asyncio.to_thread(self._list_inbound)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
log.exception("SFTP list_inbound failed")
|
||||||
|
result.errors.append(f"list_inbound: {exc}")
|
||||||
|
result.finished_at = datetime.now(timezone.utc)
|
||||||
|
return result
|
||||||
|
|
||||||
|
result.files_seen = len(files)
|
||||||
|
for f in files:
|
||||||
|
if self._stop_event.is_set():
|
||||||
|
break
|
||||||
|
await self._handle_one(f, result)
|
||||||
|
|
||||||
|
result.finished_at = datetime.now(timezone.utc)
|
||||||
|
return result
|
||||||
|
|
||||||
|
def _list_inbound(self) -> list[InboundFile]:
|
||||||
|
"""Return files in the inbound MFT path. Runs on a thread."""
|
||||||
|
client = self._sftp_client_factory(self._sftp_block)
|
||||||
|
return client.list_inbound()
|
||||||
|
|
||||||
|
async def _handle_one(self, f: InboundFile, result: TickResult) -> None:
|
||||||
|
"""Process one inbound file: skip-if-seen, classify, parse, record."""
|
||||||
|
if await self._already_processed(f.name):
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
inbound = parse_inbound_filename(f.name)
|
||||||
|
file_type = inbound.file_type
|
||||||
|
except ValueError:
|
||||||
|
file_type = None
|
||||||
|
|
||||||
|
if file_type not in HANDLERS:
|
||||||
|
await self._record(
|
||||||
|
name=f.name, size=f.size, modified_at=f.modified_at,
|
||||||
|
file_type=file_type, parser_used=None, claim_count=0,
|
||||||
|
status=STATUS_SKIPPED,
|
||||||
|
error_message=(
|
||||||
|
f"file_type {file_type!r} not in {sorted(HANDLERS)}"
|
||||||
|
if file_type else "filename does not match HCPF inbound format"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
result.files_skipped += 1
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
_path, parser_used, claim_count = await asyncio.to_thread(
|
||||||
|
self._download_and_parse, f, file_type,
|
||||||
|
)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
log.exception("Failed to process inbound file", extra={"input_filename": f.name})
|
||||||
|
await self._record(
|
||||||
|
name=f.name, size=f.size, modified_at=f.modified_at,
|
||||||
|
file_type=file_type, parser_used=None, claim_count=0,
|
||||||
|
status=STATUS_ERROR,
|
||||||
|
error_message=(
|
||||||
|
f"{type(exc).__name__}: {exc}\n"
|
||||||
|
f"{traceback.format_exc()[-500:]}"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
result.files_errored += 1
|
||||||
|
result.errors.append(f"{f.name}: {exc}")
|
||||||
|
return
|
||||||
|
|
||||||
|
await self._record(
|
||||||
|
name=f.name, size=f.size, modified_at=f.modified_at,
|
||||||
|
file_type=file_type, parser_used=parser_used, claim_count=claim_count,
|
||||||
|
status=STATUS_OK, error_message=None,
|
||||||
|
)
|
||||||
|
result.files_processed += 1
|
||||||
|
log.info(
|
||||||
|
"Processed inbound file",
|
||||||
|
extra={
|
||||||
|
"input_filename": f.name,
|
||||||
|
"parser": parser_used,
|
||||||
|
"claims": claim_count,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _already_processed(self, name: str) -> bool:
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
row = (
|
||||||
|
session.query(ProcessedInboundFile)
|
||||||
|
.filter_by(sftp_block_name=self._sftp_block_name, name=name)
|
||||||
|
.filter(ProcessedInboundFile.status != STATUS_PENDING)
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
return row is not None
|
||||||
|
|
||||||
|
async def _record(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
name: str,
|
||||||
|
size: int,
|
||||||
|
modified_at: datetime,
|
||||||
|
file_type: Optional[str],
|
||||||
|
parser_used: Optional[str],
|
||||||
|
claim_count: int,
|
||||||
|
status: str,
|
||||||
|
error_message: Optional[str],
|
||||||
|
) -> None:
|
||||||
|
"""Persist a processed_inbound_files row. Idempotent."""
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
row = ProcessedInboundFile(
|
||||||
|
sftp_block_name=self._sftp_block_name,
|
||||||
|
name=name,
|
||||||
|
size=size,
|
||||||
|
modified_at=modified_at,
|
||||||
|
file_type=file_type,
|
||||||
|
processed_at=datetime.now(timezone.utc),
|
||||||
|
parser_used=parser_used,
|
||||||
|
claim_count=claim_count,
|
||||||
|
status=status,
|
||||||
|
error_message=error_message,
|
||||||
|
)
|
||||||
|
session.add(row)
|
||||||
|
try:
|
||||||
|
session.commit()
|
||||||
|
except IntegrityError:
|
||||||
|
# A concurrent scheduler (or a retry after a partial
|
||||||
|
# failure) already recorded this file. That's fine —
|
||||||
|
# the latest row wins; we just skip the dup.
|
||||||
|
session.rollback()
|
||||||
|
|
||||||
|
def _download_and_parse(
|
||||||
|
self, f: InboundFile, file_type: str,
|
||||||
|
) -> tuple[Path, str, int]:
|
||||||
|
"""Download from MFT, run the right handler. Returns (path, parser, count).
|
||||||
|
|
||||||
|
Stub mode: ``f.local_path`` already points at the staged file
|
||||||
|
(set by ``SftpClient._list_inbound_stub``). Real mode: the
|
||||||
|
remote name is ``f.name`` and we round-trip through paramiko.
|
||||||
|
"""
|
||||||
|
if self._sftp_block.stub:
|
||||||
|
# In stub mode the InboundFile already has a local_path;
|
||||||
|
# reading the staged bytes directly avoids the stub's
|
||||||
|
# remote-path semantics (which expect a full inbound path).
|
||||||
|
content = f.local_path.read_bytes()
|
||||||
|
else:
|
||||||
|
client = self._sftp_client_factory(self._sftp_block)
|
||||||
|
content = client.read_file(f.name)
|
||||||
|
text = content.decode("utf-8")
|
||||||
|
handler = HANDLERS[file_type]
|
||||||
|
parser_used, claim_count = handler(text, f.name)
|
||||||
|
return f.local_path, parser_used, claim_count
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Module-level singleton — only one scheduler per process.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
_scheduler: Optional[Scheduler] = None
|
||||||
|
|
||||||
|
|
||||||
|
def configure_scheduler(
|
||||||
|
sftp_block: SftpBlock,
|
||||||
|
*,
|
||||||
|
poll_interval_seconds: int = 60,
|
||||||
|
sftp_block_name: str = "default",
|
||||||
|
force: bool = False,
|
||||||
|
) -> Scheduler:
|
||||||
|
"""Create the module-level scheduler singleton (or return the existing one).
|
||||||
|
|
||||||
|
Called from the FastAPI lifespan handler. Tests pre-configure the
|
||||||
|
scheduler before the TestClient opens the lifespan; in that case
|
||||||
|
we leave the existing singleton alone (``force=False``). Pass
|
||||||
|
``force=True`` to replace unconditionally.
|
||||||
|
"""
|
||||||
|
global _scheduler
|
||||||
|
if _scheduler is not None and not force:
|
||||||
|
return _scheduler
|
||||||
|
poll = int(
|
||||||
|
os.environ.get("CYCLONE_SCHEDULER_POLL_SECONDS", poll_interval_seconds),
|
||||||
|
)
|
||||||
|
_scheduler = Scheduler(
|
||||||
|
sftp_block,
|
||||||
|
poll_interval_seconds=poll,
|
||||||
|
sftp_block_name=sftp_block_name,
|
||||||
|
)
|
||||||
|
return _scheduler
|
||||||
|
|
||||||
|
|
||||||
|
def get_scheduler() -> Scheduler:
|
||||||
|
"""Return the module-level scheduler.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
RuntimeError: if ``configure_scheduler`` hasn't been called.
|
||||||
|
"""
|
||||||
|
if _scheduler is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
"scheduler not configured; call configure_scheduler() first",
|
||||||
|
)
|
||||||
|
return _scheduler
|
||||||
|
|
||||||
|
|
||||||
|
def reset_scheduler_for_tests() -> None:
|
||||||
|
"""Clear the module-level scheduler. Test-only."""
|
||||||
|
global _scheduler
|
||||||
|
_scheduler = None
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
"""macOS Keychain secret accessor for Cyclone.
|
||||||
|
|
||||||
|
SP9. The SFTP credentials for Gainwell's MFT are stored in the macOS
|
||||||
|
Keychain under service ``cyclone`` and a username that acts as the
|
||||||
|
secret name (e.g. ``sftp.gainwell.password``). This module fetches
|
||||||
|
them by name.
|
||||||
|
|
||||||
|
Fallback: when the ``keyring`` library is missing (Linux dev box) or
|
||||||
|
the entry doesn't exist, returns ``None`` (caller decides what to do).
|
||||||
|
A stub secret ``<stub-secret>`` is provided for the SP9 stub flow.
|
||||||
|
|
||||||
|
Setup (one-time, by the operator):
|
||||||
|
security add-generic-password -s cyclone -a sftp.gainwell.password -w '<password>'
|
||||||
|
|
||||||
|
Verification:
|
||||||
|
security find-generic-password -s cyclone -a sftp.gainwell.password -w
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
SERVICE_NAME = "cyclone"
|
||||||
|
STUB_SECRET = "<stub-secret>"
|
||||||
|
|
||||||
|
# Try to import keyring lazily — it's an optional dep so the rest of
|
||||||
|
# the codebase doesn't fail on Linux dev boxes without it.
|
||||||
|
try:
|
||||||
|
import keyring # type: ignore[import-untyped]
|
||||||
|
_HAS_KEYRING = True
|
||||||
|
except ImportError:
|
||||||
|
keyring = None # type: ignore[assignment]
|
||||||
|
_HAS_KEYRING = False
|
||||||
|
|
||||||
|
|
||||||
|
def get_secret(name: str) -> Optional[str]:
|
||||||
|
"""Fetch a secret from macOS Keychain by name.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
name: The Keychain account (e.g. "sftp.gainwell.password").
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The secret string, or None if the entry is missing or keyring
|
||||||
|
is not installed.
|
||||||
|
"""
|
||||||
|
if not _HAS_KEYRING:
|
||||||
|
log.warning("keyring not installed; get_secret(%r) returning None", name)
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return keyring.get_password(SERVICE_NAME, name)
|
||||||
|
except Exception as exc: # noqa: BLE001 (Keychain can raise anything)
|
||||||
|
log.warning("Keychain get_secret(%r) failed: %s", name, exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def set_secret(name: str, value: str) -> bool:
|
||||||
|
"""Set a secret in macOS Keychain. Returns True on success.
|
||||||
|
|
||||||
|
Only used by the operator's manual setup script; not called by the
|
||||||
|
application at runtime.
|
||||||
|
"""
|
||||||
|
if not _HAS_KEYRING:
|
||||||
|
log.error("keyring not installed; cannot set_secret(%r)", name)
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
keyring.set_password(SERVICE_NAME, name, value)
|
||||||
|
return True
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
log.error("Keychain set_secret(%r) failed: %s", name, exc)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def has_keyring() -> bool:
|
||||||
|
"""True if the ``keyring`` library is importable (regardless of whether
|
||||||
|
the Keychain entry actually exists)."""
|
||||||
|
return _HAS_KEYRING
|
||||||
@@ -0,0 +1,485 @@
|
|||||||
|
"""SP19 — Security middleware + health probe.
|
||||||
|
|
||||||
|
Three concrete middlewares (body size, rate limit, security headers)
|
||||||
|
plus a richer ``/api/health`` snapshot. Sizing is for Cyclone's
|
||||||
|
local-only posture: a misconfigured Tailscale / ngrok bind, a
|
||||||
|
misbehaving cron job, a port-scanner scraping the API. Anything more
|
||||||
|
aggressive (auth, mTLS, WAF) is out of scope.
|
||||||
|
|
||||||
|
Design choices
|
||||||
|
--------------
|
||||||
|
|
||||||
|
* **In-memory rate limiter.** Cyclone is single-process; a dict
|
||||||
|
keyed by IP is enough. If we ever go multi-worker, swap for
|
||||||
|
Redis. The rate-limit counter resets after the bucket window;
|
||||||
|
failing open on the limiter itself (an unexpected exception)
|
||||||
|
rather than 503ing every request is the right call for a local tool.
|
||||||
|
|
||||||
|
* **Body-size check by Content-Length first, then chunked-read
|
||||||
|
guard.** A chunked POST can lie about its size (or omit the
|
||||||
|
header entirely); we cap read body size on the underlying stream
|
||||||
|
so a malicious client can't keep streaming forever.
|
||||||
|
|
||||||
|
* **Security headers on every response.** CSP locks the API to
|
||||||
|
same-origin + the Vite dev origin (whitelisted explicitly so a
|
||||||
|
future operator running on a different port doesn't break).
|
||||||
|
|
||||||
|
* **Health snapshot is best-effort.** Each subsystem (DB,
|
||||||
|
scheduler, pubsub) reports independently — a DB outage doesn't
|
||||||
|
blank out the rest. ``status: "degraded"`` if any subsystem is
|
||||||
|
unhappy; ``"ok"`` only when everything is.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from collections import deque
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
from fastapi import Request, Response
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from starlette.types import ASGIApp, Message, Receive, Scope, Send
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Knobs (env-var driven)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_MAX_BODY_BYTES = 50 * 1024 * 1024 # 50 MB — generous for X12 EDI
|
||||||
|
DEFAULT_RATE_LIMIT_PER_MIN = 300
|
||||||
|
DEFAULT_RATE_LIMIT_WINDOW_S = 60
|
||||||
|
|
||||||
|
# CSP: API responses are JSON, not HTML. ``default-src 'none'`` is the
|
||||||
|
# strictest setting; it forbids the API from being a vector for
|
||||||
|
# injected scripts in case an operator opens a JSON viewer with an
|
||||||
|
# HTML renderer.
|
||||||
|
_SECURITY_HEADERS: dict[str, str] = {
|
||||||
|
"X-Content-Type-Options": "nosniff",
|
||||||
|
"X-Frame-Options": "DENY",
|
||||||
|
"Referrer-Policy": "same-origin",
|
||||||
|
"Permissions-Policy": "geolocation=(), microphone=(), camera=()",
|
||||||
|
"Content-Security-Policy": "default-src 'none'; frame-ancestors 'none'",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _env_int(name: str, default: int) -> int:
|
||||||
|
raw = os.environ.get(name)
|
||||||
|
if not raw:
|
||||||
|
return default
|
||||||
|
try:
|
||||||
|
return int(raw)
|
||||||
|
except ValueError:
|
||||||
|
log.warning("SP19: %s=%r is not an int; using default %d", name, raw, default)
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Body-size middleware
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class BodySizeLimitMiddleware:
|
||||||
|
"""Reject requests whose body exceeds ``max_bytes``.
|
||||||
|
|
||||||
|
Pure ASGI middleware (not BaseHTTPMiddleware — that one breaks
|
||||||
|
FastAPI's ``request.body()`` introspection). Two-stage guard:
|
||||||
|
1. If the request declares a ``Content-Length`` larger than
|
||||||
|
``max_bytes``, reject immediately with ``413``.
|
||||||
|
2. While reading the body chunks, cap accumulated bytes at
|
||||||
|
``max_bytes``. If we cross the cap, return 413 instead of
|
||||||
|
letting the handler read the rest.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, app: ASGIApp, max_bytes: int | None = None) -> None:
|
||||||
|
self.app = app
|
||||||
|
self.max_bytes = max_bytes or _env_int(
|
||||||
|
"CYCLONE_MAX_BODY_BYTES", DEFAULT_MAX_BODY_BYTES,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
|
||||||
|
if scope["type"] != "http":
|
||||||
|
await self.app(scope, receive, send)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Stage 1: declared length.
|
||||||
|
cl_header = None
|
||||||
|
for k, v in scope.get("headers", []):
|
||||||
|
if k == b"content-length":
|
||||||
|
cl_header = v.decode("latin-1")
|
||||||
|
break
|
||||||
|
if cl_header is not None:
|
||||||
|
try:
|
||||||
|
if int(cl_header) > self.max_bytes:
|
||||||
|
await _send_rejection(
|
||||||
|
scope, send,
|
||||||
|
code=413,
|
||||||
|
reason="body_too_large",
|
||||||
|
detail=f"Content-Length {cl_header} exceeds limit {self.max_bytes}",
|
||||||
|
)
|
||||||
|
return
|
||||||
|
except ValueError:
|
||||||
|
await _send_rejection(
|
||||||
|
scope, send,
|
||||||
|
code=400, reason="bad_content_length",
|
||||||
|
detail=f"Content-Length {cl_header!r} is not an integer",
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Stage 2: chunked read guard.
|
||||||
|
seen = 0
|
||||||
|
over_limit = False
|
||||||
|
|
||||||
|
async def wrapped_receive() -> Message:
|
||||||
|
nonlocal seen, over_limit
|
||||||
|
if over_limit:
|
||||||
|
# Drain any remaining bytes so the upstream ASGI
|
||||||
|
# server doesn't see a truncated stream.
|
||||||
|
msg = await receive()
|
||||||
|
if msg.get("type") == "http.request":
|
||||||
|
return {"type": "http.request", "body": b"", "more_body": False}
|
||||||
|
return msg
|
||||||
|
msg = await receive()
|
||||||
|
if msg.get("type") == "http.request":
|
||||||
|
body = msg.get("body", b"") or b""
|
||||||
|
seen += len(body)
|
||||||
|
if seen > self.max_bytes:
|
||||||
|
over_limit = True
|
||||||
|
return {"type": "http.request", "body": b"", "more_body": False}
|
||||||
|
return msg
|
||||||
|
|
||||||
|
if cl_header is None:
|
||||||
|
# Chunked / unknown length — guard with wrapped receive.
|
||||||
|
await self.app(scope, wrapped_receive, send)
|
||||||
|
else:
|
||||||
|
# Fixed-length known to be safe; pass through.
|
||||||
|
await self.app(scope, receive, send)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Rate-limit middleware
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class _Bucket:
|
||||||
|
"""Sliding-window counter for one IP."""
|
||||||
|
timestamps: deque = field(default_factory=deque)
|
||||||
|
|
||||||
|
def hit(self, window_s: int, now: float) -> bool:
|
||||||
|
"""Record one hit; return True if under the limit, False if over."""
|
||||||
|
# Drop expired entries.
|
||||||
|
cutoff = now - window_s
|
||||||
|
while self.timestamps and self.timestamps[0] < cutoff:
|
||||||
|
self.timestamps.popleft()
|
||||||
|
return True # we always record; the dispatcher decides to reject
|
||||||
|
|
||||||
|
def count_in_window(self, now: float, window_s: int) -> int:
|
||||||
|
cutoff = now - window_s
|
||||||
|
while self.timestamps and self.timestamps[0] < cutoff:
|
||||||
|
self.timestamps.popleft()
|
||||||
|
return len(self.timestamps)
|
||||||
|
|
||||||
|
|
||||||
|
class RateLimitMiddleware:
|
||||||
|
"""Per-IP sliding-window rate limiter (pure ASGI).
|
||||||
|
|
||||||
|
Defaults to ``CYCLONE_RATE_LIMIT_PER_MIN`` requests/minute per IP.
|
||||||
|
Health-check probes and the ``/api/health`` endpoint are exempt
|
||||||
|
so a load balancer's frequent probes don't trip the limiter.
|
||||||
|
|
||||||
|
On unexpected errors the limiter fails OPEN — better to serve a
|
||||||
|
few extra requests than to 503 every request because of a bug.
|
||||||
|
"""
|
||||||
|
|
||||||
|
EXEMPT_PATHS = ("/api/health", "/healthz", "/readyz")
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
app: ASGIApp,
|
||||||
|
per_minute: int | None = None,
|
||||||
|
window_s: int | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.app = app
|
||||||
|
self.per_minute = per_minute or _env_int(
|
||||||
|
"CYCLONE_RATE_LIMIT_PER_MIN", DEFAULT_RATE_LIMIT_PER_MIN,
|
||||||
|
)
|
||||||
|
self.window_s = window_s or DEFAULT_RATE_LIMIT_WINDOW_S
|
||||||
|
self._buckets: dict[str, _Bucket] = {}
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
|
||||||
|
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
|
||||||
|
if scope["type"] != "http":
|
||||||
|
await self.app(scope, receive, send)
|
||||||
|
return
|
||||||
|
path = scope.get("path", "")
|
||||||
|
if path in self.EXEMPT_PATHS:
|
||||||
|
await self.app(scope, receive, send)
|
||||||
|
return
|
||||||
|
ip = _client_ip_from_scope(scope)
|
||||||
|
now = time.monotonic()
|
||||||
|
try:
|
||||||
|
with self._lock:
|
||||||
|
bucket = self._buckets.setdefault(ip, _Bucket())
|
||||||
|
bucket.timestamps.append(now)
|
||||||
|
count = bucket.count_in_window(now, self.window_s)
|
||||||
|
if count > self.per_minute:
|
||||||
|
await _send_rejection(
|
||||||
|
scope, send,
|
||||||
|
code=429,
|
||||||
|
reason="rate_limited",
|
||||||
|
detail=(
|
||||||
|
f"IP {ip} exceeded {self.per_minute} req/"
|
||||||
|
f"{self.window_s}s window"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
log.warning("SP19: rate limiter failed open: %s", exc)
|
||||||
|
await self.app(scope, receive, send)
|
||||||
|
|
||||||
|
|
||||||
|
def _client_ip_from_scope(scope: Scope) -> str:
|
||||||
|
"""Best-effort client IP from the ASGI scope. Falls back to ``"unknown"``."""
|
||||||
|
for k, v in scope.get("headers", []):
|
||||||
|
if k == b"x-forwarded-for":
|
||||||
|
return v.decode("latin-1").split(",")[0].strip()
|
||||||
|
client = scope.get("client")
|
||||||
|
if client and client[0]:
|
||||||
|
return client[0]
|
||||||
|
return "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
def _client_ip(request: Request) -> str:
|
||||||
|
"""Legacy helper (kept for the audit-event log path)."""
|
||||||
|
return _client_ip_from_scope(request.scope)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Security-headers middleware
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class SecurityHeadersMiddleware:
|
||||||
|
"""Stamp the static security headers on every response (pure ASGI).
|
||||||
|
|
||||||
|
CSP / X-Content-Type-Options / X-Frame-Options / Referrer-Policy /
|
||||||
|
Permissions-Policy. The headers are static for now; per-route
|
||||||
|
overrides can be added later if a route needs to relax them.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, app: ASGIApp, extra: dict[str, str] | None = None) -> None:
|
||||||
|
self.app = app
|
||||||
|
self.headers = [(k.lower().encode("latin-1"), v.encode("latin-1"))
|
||||||
|
for k, v in _SECURITY_HEADERS.items()]
|
||||||
|
if extra:
|
||||||
|
self.headers.extend(
|
||||||
|
(k.lower().encode("latin-1"), v.encode("latin-1"))
|
||||||
|
for k, v in extra.items()
|
||||||
|
)
|
||||||
|
|
||||||
|
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
|
||||||
|
if scope["type"] != "http":
|
||||||
|
await self.app(scope, receive, send)
|
||||||
|
return
|
||||||
|
|
||||||
|
async def wrapped_send(message: Message) -> None:
|
||||||
|
if message["type"] == "http.response.start":
|
||||||
|
headers = list(message.get("headers", []))
|
||||||
|
existing = {k for k, _ in headers}
|
||||||
|
for k, v in self.headers:
|
||||||
|
if k not in existing:
|
||||||
|
headers.append((k, v))
|
||||||
|
message["headers"] = headers
|
||||||
|
await send(message)
|
||||||
|
|
||||||
|
await self.app(scope, receive, wrapped_send)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Reject helper (also writes an audit event when DB is available)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def _send_rejection(
|
||||||
|
scope: Scope,
|
||||||
|
send: Send,
|
||||||
|
*,
|
||||||
|
code: int,
|
||||||
|
reason: str,
|
||||||
|
detail: str,
|
||||||
|
) -> None:
|
||||||
|
"""Build a 413/429 JSON response, send it, and emit a log + audit event."""
|
||||||
|
method = scope.get("method", "GET")
|
||||||
|
path = scope.get("path", "/")
|
||||||
|
ip = _client_ip_from_scope(scope)
|
||||||
|
log.warning(
|
||||||
|
"api.request_rejected",
|
||||||
|
extra={
|
||||||
|
"status": code,
|
||||||
|
"reason": reason,
|
||||||
|
"path": path,
|
||||||
|
"method": method,
|
||||||
|
"ip": ip,
|
||||||
|
"detail": detail,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
payload = {"error": reason, "detail": detail, "status": code}
|
||||||
|
body = json.dumps(payload).encode("utf-8")
|
||||||
|
await send({
|
||||||
|
"type": "http.response.start",
|
||||||
|
"status": code,
|
||||||
|
"headers": [
|
||||||
|
(b"content-type", b"application/json"),
|
||||||
|
(b"content-length", str(len(body)).encode("latin-1")),
|
||||||
|
],
|
||||||
|
})
|
||||||
|
await send({"type": "http.response.body", "body": body, "more_body": False})
|
||||||
|
# Best-effort audit-log append. Don't block the response on a DB
|
||||||
|
# outage (the rejection is the more important signal anyway).
|
||||||
|
try:
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.audit_log import AuditEvent, append_event
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
append_event(
|
||||||
|
session,
|
||||||
|
AuditEvent(
|
||||||
|
event_type="api.request_rejected",
|
||||||
|
entity_type="http_request",
|
||||||
|
entity_id=f"{method} {path}",
|
||||||
|
payload={
|
||||||
|
"status": code,
|
||||||
|
"reason": reason,
|
||||||
|
"path": path,
|
||||||
|
"method": method,
|
||||||
|
"ip": ip,
|
||||||
|
},
|
||||||
|
actor=f"api:{ip}",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
session.commit()
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
log.debug("SP19: audit-log append failed for rejection: %s", exc)
|
||||||
|
|
||||||
|
|
||||||
|
def _reject(
|
||||||
|
request: Request,
|
||||||
|
*,
|
||||||
|
code: int,
|
||||||
|
reason: str,
|
||||||
|
detail: str,
|
||||||
|
) -> JSONResponse:
|
||||||
|
"""Sync helper kept for back-compat (the ``audit_log`` payload path)."""
|
||||||
|
return JSONResponse(
|
||||||
|
{"error": reason, "detail": detail, "status": code},
|
||||||
|
status_code=code,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Health snapshot
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class HealthSnapshot:
|
||||||
|
status: str
|
||||||
|
version: str
|
||||||
|
db: dict[str, Any] = field(default_factory=dict)
|
||||||
|
scheduler: dict[str, Any] = field(default_factory=dict)
|
||||||
|
pubsub: dict[str, Any] = field(default_factory=dict)
|
||||||
|
batch: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"status": self.status,
|
||||||
|
"version": self.version,
|
||||||
|
"db": self.db,
|
||||||
|
"scheduler": self.scheduler,
|
||||||
|
"pubsub": self.pubsub,
|
||||||
|
"batch": self.batch,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_health_snapshot() -> HealthSnapshot:
|
||||||
|
"""Gather a best-effort snapshot of every Cyclone subsystem.
|
||||||
|
|
||||||
|
Returns ``HealthSnapshot`` with ``status="ok"`` only if every
|
||||||
|
subsystem check passes. ``"degraded"`` if any subsystem is
|
||||||
|
unhappy but the API itself is responsive. Each subsystem reports
|
||||||
|
independently so one outage doesn't blank out the rest.
|
||||||
|
"""
|
||||||
|
from cyclone import __version__, db
|
||||||
|
|
||||||
|
snap = HealthSnapshot(status="ok", version=__version__)
|
||||||
|
|
||||||
|
# DB connectivity.
|
||||||
|
try:
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
session.execute(db.text("SELECT 1"))
|
||||||
|
snap.db = {"ok": True}
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
snap.db = {"ok": False, "error": str(exc)}
|
||||||
|
snap.status = "degraded"
|
||||||
|
|
||||||
|
# Scheduler state.
|
||||||
|
try:
|
||||||
|
from cyclone import scheduler as scheduler_mod
|
||||||
|
sched = scheduler_mod.get_scheduler()
|
||||||
|
snap.scheduler = {
|
||||||
|
"running": sched.is_running(),
|
||||||
|
"interval_s": sched._poll_interval, # noqa: SLF001
|
||||||
|
"sftp_block": sched._sftp_block_name, # noqa: SLF001
|
||||||
|
}
|
||||||
|
except RuntimeError:
|
||||||
|
snap.scheduler = {"running": False, "configured": False}
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
snap.scheduler = {"ok": False, "error": str(exc)}
|
||||||
|
snap.status = "degraded"
|
||||||
|
|
||||||
|
# Backup scheduler.
|
||||||
|
try:
|
||||||
|
from cyclone import backup_scheduler as bks_mod
|
||||||
|
bks = bks_mod.get_backup_scheduler()
|
||||||
|
snap.scheduler["backup_scheduler_running"] = bks.is_running()
|
||||||
|
snap.scheduler["backup_interval_hours"] = bks.interval_hours
|
||||||
|
except (RuntimeError, ImportError):
|
||||||
|
snap.scheduler["backup_scheduler_running"] = False
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass # secondary subsystem; don't degrade the overall status
|
||||||
|
|
||||||
|
# Pubsub bus stats — placeholder. The /api/health handler fills
|
||||||
|
# in the real subscriber counts using request.app.state.event_bus.
|
||||||
|
snap.pubsub = {"note": "filled in by health router"}
|
||||||
|
|
||||||
|
# Last batch timestamp + count.
|
||||||
|
try:
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone.db import Batch
|
||||||
|
with db_mod.SessionLocal()() as session:
|
||||||
|
row = (
|
||||||
|
session.query(Batch)
|
||||||
|
.order_by(Batch.parsed_at.desc())
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
if row is not None:
|
||||||
|
snap.batch = {
|
||||||
|
"last_batch_id": row.id,
|
||||||
|
"last_batch_kind": row.kind,
|
||||||
|
"last_batch_at": row.parsed_at.isoformat() if row.parsed_at else None,
|
||||||
|
"last_batch_filename": row.input_filename,
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
snap.batch = {"last_batch_id": None, "note": "no batches yet"}
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
snap.batch = {"ok": False, "error": str(exc)}
|
||||||
|
snap.status = "degraded"
|
||||||
|
|
||||||
|
return snap
|
||||||
@@ -52,6 +52,7 @@ from cyclone.db import (
|
|||||||
from cyclone.parsers.models import ClaimOutput, ParseResult
|
from cyclone.parsers.models import ClaimOutput, ParseResult
|
||||||
from cyclone.parsers.models_835 import ClaimPayment, ParseResult835
|
from cyclone.parsers.models_835 import ClaimPayment, ParseResult835
|
||||||
from cyclone.parsers.payer import PayerConfig835
|
from cyclone.parsers.payer import PayerConfig835
|
||||||
|
from cyclone.providers import Clearhouse, Payer, Provider # SP9: ORM-row DTOs
|
||||||
|
|
||||||
|
|
||||||
class AlreadyMatchedError(Exception):
|
class AlreadyMatchedError(Exception):
|
||||||
@@ -1667,7 +1668,16 @@ class CycloneStore:
|
|||||||
return list(by_npi.values())
|
return list(by_npi.values())
|
||||||
|
|
||||||
def recent_activity(self, *, limit: int = 200) -> list[dict]:
|
def recent_activity(self, *, limit: int = 200) -> list[dict]:
|
||||||
"""Return recent activity events from the DB, newest first."""
|
"""Return recent activity events from the DB, newest first.
|
||||||
|
|
||||||
|
SP21 Task 2.5: each row also carries ``claimId`` and
|
||||||
|
``remittanceId`` (read from the ORM columns) so the Dashboard's
|
||||||
|
Recent-activity card can route clicks to the right entity
|
||||||
|
drawer via ``src/lib/event-routing.ts``. Both are nullable
|
||||||
|
strings; the wire shape uses camelCase keys to match the
|
||||||
|
existing ``npi`` / ``amount`` fields and the frontend
|
||||||
|
``Activity`` interface.
|
||||||
|
"""
|
||||||
with db.SessionLocal()() as s:
|
with db.SessionLocal()() as s:
|
||||||
rows = (
|
rows = (
|
||||||
s.query(ActivityEvent)
|
s.query(ActivityEvent)
|
||||||
@@ -1683,6 +1693,8 @@ class CycloneStore:
|
|||||||
"timestamp": r.ts.isoformat().replace("+00:00", "Z"),
|
"timestamp": r.ts.isoformat().replace("+00:00", "Z"),
|
||||||
"npi": (r.payload_json or {}).get("npi"),
|
"npi": (r.payload_json or {}).get("npi"),
|
||||||
"amount": (r.payload_json or {}).get("amount"),
|
"amount": (r.payload_json or {}).get("amount"),
|
||||||
|
"claimId": r.claim_id,
|
||||||
|
"remittanceId": r.remittance_id,
|
||||||
}
|
}
|
||||||
for r in rows
|
for r in rows
|
||||||
]
|
]
|
||||||
@@ -1799,6 +1811,81 @@ class CycloneStore:
|
|||||||
with db.SessionLocal()() as s:
|
with db.SessionLocal()() as s:
|
||||||
return s.get(db.Ta1Ack, ack_id)
|
return s.get(db.Ta1Ack, ack_id)
|
||||||
|
|
||||||
|
# -- 277CA (SP10) --------------------------------------------------
|
||||||
|
|
||||||
|
def add_277ca_ack(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
source_batch_id: str,
|
||||||
|
control_number: str,
|
||||||
|
accepted_count: int,
|
||||||
|
rejected_count: int,
|
||||||
|
paid_count: int,
|
||||||
|
pended_count: int,
|
||||||
|
raw_json: dict,
|
||||||
|
) -> db.Two77caAck:
|
||||||
|
"""Persist a 277CA (Claim Acknowledgment) row and return it.
|
||||||
|
|
||||||
|
Mirrors :meth:`add_ack` but for the claim-level ack. The
|
||||||
|
per-claim status detail stays in ``raw_json``; only the four
|
||||||
|
counts are promoted so the list endpoint stays fast.
|
||||||
|
"""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = db.Two77caAck(
|
||||||
|
source_batch_id=source_batch_id,
|
||||||
|
control_number=control_number,
|
||||||
|
accepted_count=accepted_count,
|
||||||
|
rejected_count=rejected_count,
|
||||||
|
paid_count=paid_count,
|
||||||
|
pended_count=pended_count,
|
||||||
|
parsed_at=utcnow(),
|
||||||
|
raw_json=raw_json,
|
||||||
|
)
|
||||||
|
s.add(row)
|
||||||
|
s.commit()
|
||||||
|
s.refresh(row)
|
||||||
|
return row
|
||||||
|
|
||||||
|
def list_277ca_acks(self) -> list[db.Two77caAck]:
|
||||||
|
"""Return every 277CA ACK row, newest first (auto-increment id desc)."""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
return (
|
||||||
|
s.query(db.Two77caAck)
|
||||||
|
.order_by(db.Two77caAck.id.desc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
|
||||||
|
def get_277ca_ack(self, ack_id: int) -> db.Two77caAck | None:
|
||||||
|
"""Return a single 277CA ACK row by id, or ``None`` if not found."""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
return s.get(db.Two77caAck, ack_id)
|
||||||
|
|
||||||
|
# -- SP17: encrypted DB backups -------------------------------------
|
||||||
|
|
||||||
|
def add_backup_pending(self, *, filename: str, backup_dir: str) -> db.DbBackup:
|
||||||
|
"""Insert a ``pending`` row for a backup that is about to start.
|
||||||
|
|
||||||
|
The BackupService fills in ``status`` / ``size_bytes`` /
|
||||||
|
``db_fingerprint`` / ``table_count`` / ``completed_at`` after
|
||||||
|
the encrypted blob lands on disk.
|
||||||
|
"""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = db.DbBackup(
|
||||||
|
filename=filename,
|
||||||
|
backup_dir=backup_dir,
|
||||||
|
size_bytes=0,
|
||||||
|
db_fingerprint=None,
|
||||||
|
table_count=0,
|
||||||
|
created_at=utcnow(),
|
||||||
|
completed_at=None,
|
||||||
|
status="pending",
|
||||||
|
error_message=None,
|
||||||
|
)
|
||||||
|
s.add(row)
|
||||||
|
s.commit()
|
||||||
|
s.refresh(row)
|
||||||
|
return row
|
||||||
|
|
||||||
# -- manual reconciliation (T12) -----------------------------------
|
# -- manual reconciliation (T12) -----------------------------------
|
||||||
|
|
||||||
def list_unmatched(self, *, kind: str = "both") -> dict:
|
def list_unmatched(self, *, kind: str = "both") -> dict:
|
||||||
@@ -2082,6 +2169,256 @@ class CycloneStore:
|
|||||||
"deletedMatches": deleted_count,
|
"deletedMatches": deleted_count,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# SP9: providers / payers / payer_configs / clearhouse
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def list_providers(self, *, is_active: bool | None = True) -> list[Provider]:
|
||||||
|
"""List providers. ``is_active=None`` returns all."""
|
||||||
|
from cyclone.db import Provider as ProviderORM
|
||||||
|
from cyclone.providers import Provider
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
q = s.query(ProviderORM)
|
||||||
|
if is_active is not None:
|
||||||
|
q = q.filter(ProviderORM.is_active == (1 if is_active else 0))
|
||||||
|
rows = q.order_by(ProviderORM.label).all()
|
||||||
|
return [Provider.model_validate(_provider_orm_to_dict(r)) for r in rows]
|
||||||
|
|
||||||
|
def get_provider(self, npi: str) -> Provider | None:
|
||||||
|
from cyclone.db import Provider as ProviderORM
|
||||||
|
from cyclone.providers import Provider
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(ProviderORM, npi)
|
||||||
|
return Provider.model_validate(_provider_orm_to_dict(row)) if row else None
|
||||||
|
|
||||||
|
def upsert_provider(self, provider: Provider) -> Provider:
|
||||||
|
from cyclone.db import Provider as ProviderORM
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(ProviderORM, provider.npi)
|
||||||
|
now = utcnow().isoformat()
|
||||||
|
if row is None:
|
||||||
|
row = ProviderORM(
|
||||||
|
npi=provider.npi, label=provider.label,
|
||||||
|
legal_name=provider.legal_name, tax_id=provider.tax_id,
|
||||||
|
taxonomy_code=provider.taxonomy_code,
|
||||||
|
address_line1=provider.address_line1,
|
||||||
|
address_line2=provider.address_line2,
|
||||||
|
city=provider.city, state=provider.state, zip=provider.zip,
|
||||||
|
is_active=1 if provider.is_active else 0,
|
||||||
|
created_at=provider.created_at.isoformat(),
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
s.add(row)
|
||||||
|
else:
|
||||||
|
row.label = provider.label
|
||||||
|
row.legal_name = provider.legal_name
|
||||||
|
row.tax_id = provider.tax_id
|
||||||
|
row.taxonomy_code = provider.taxonomy_code
|
||||||
|
row.address_line1 = provider.address_line1
|
||||||
|
row.address_line2 = provider.address_line2
|
||||||
|
row.city = provider.city
|
||||||
|
row.state = provider.state
|
||||||
|
row.zip = provider.zip
|
||||||
|
row.is_active = 1 if provider.is_active else 0
|
||||||
|
row.updated_at = now
|
||||||
|
s.commit()
|
||||||
|
return self.get_provider(provider.npi) # type: ignore[return-value]
|
||||||
|
|
||||||
|
def list_payers(self, *, is_active: bool | None = True) -> list[Payer]:
|
||||||
|
from cyclone.db import Payer as PayerORM
|
||||||
|
from cyclone.providers import Payer
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
q = s.query(PayerORM)
|
||||||
|
if is_active is not None:
|
||||||
|
q = q.filter(PayerORM.is_active == (1 if is_active else 0))
|
||||||
|
rows = q.order_by(PayerORM.payer_id).all()
|
||||||
|
return [Payer.model_validate(_payer_orm_to_dict(r)) for r in rows]
|
||||||
|
|
||||||
|
def get_payer_config(self, payer_id: str, transaction_type: str) -> dict | None:
|
||||||
|
from cyclone.db import PayerConfigORM
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(PayerConfigORM, (payer_id, transaction_type))
|
||||||
|
return dict(row.config_json) if row else None
|
||||||
|
|
||||||
|
def get_clearhouse(self) -> Clearhouse | None:
|
||||||
|
from cyclone.db import ClearhouseORM
|
||||||
|
from cyclone.providers import Clearhouse
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(ClearhouseORM, 1)
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
return Clearhouse.model_validate({
|
||||||
|
"id": 1,
|
||||||
|
"name": row.name,
|
||||||
|
"tpid": row.tpid,
|
||||||
|
"submitter_name": row.submitter_name,
|
||||||
|
"submitter_id_qual": row.submitter_id_qual,
|
||||||
|
"submitter_contact_name": row.submitter_contact_name,
|
||||||
|
"submitter_contact_email": row.submitter_contact_email,
|
||||||
|
"filename_block": dict(row.filename_block_json),
|
||||||
|
"sftp_block": dict(row.sftp_block_json),
|
||||||
|
"updated_at": row.updated_at,
|
||||||
|
})
|
||||||
|
|
||||||
|
def ensure_clearhouse_seeded(self) -> None:
|
||||||
|
"""Insert the default clearhouse singleton + 3 providers + CO_TXIX payer
|
||||||
|
if they don't exist. Idempotent. Called from the API lifespan."""
|
||||||
|
from cyclone.db import ClearhouseORM, Payer as PayerORM, PayerConfigORM, Provider as ProviderORM
|
||||||
|
from cyclone.providers import Clearhouse
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
if s.get(ClearhouseORM, 1) is None:
|
||||||
|
ch = Clearhouse(
|
||||||
|
id=1,
|
||||||
|
name="dzinesco",
|
||||||
|
tpid="11525703",
|
||||||
|
submitter_name="Dzinesco",
|
||||||
|
submitter_id_qual="46",
|
||||||
|
submitter_contact_name="Tyler Martinez",
|
||||||
|
submitter_contact_email="tyler@dzinesco.com",
|
||||||
|
filename_block={
|
||||||
|
"tz": "America/Denver",
|
||||||
|
"outbound_template": "{tpid}-{tx}-{ts_mt}-1of1.{ext}",
|
||||||
|
"inbound_template": "TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12",
|
||||||
|
},
|
||||||
|
sftp_block={
|
||||||
|
"host": "mft.gainwelltechnologies.com",
|
||||||
|
"port": 22,
|
||||||
|
"username": "colorado-fts\\coxix_prod_11525703",
|
||||||
|
"paths": {
|
||||||
|
"outbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
|
||||||
|
"inbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
|
||||||
|
},
|
||||||
|
"stub": True,
|
||||||
|
"staging_dir": "./var/sftp/staging",
|
||||||
|
"poll_seconds": 300,
|
||||||
|
"auth": {"method": "keychain", "secret_ref": "sftp.gainwell.password"},
|
||||||
|
},
|
||||||
|
updated_at=utcnow(),
|
||||||
|
)
|
||||||
|
s.add(ClearhouseORM(
|
||||||
|
id=1,
|
||||||
|
name=ch.name,
|
||||||
|
tpid=ch.tpid,
|
||||||
|
submitter_name=ch.submitter_name,
|
||||||
|
submitter_id_qual=ch.submitter_id_qual,
|
||||||
|
submitter_contact_name=ch.submitter_contact_name,
|
||||||
|
submitter_contact_email=ch.submitter_contact_email,
|
||||||
|
filename_block_json=ch.filename_block.model_dump(),
|
||||||
|
sftp_block_json=ch.sftp_block.model_dump(),
|
||||||
|
updated_at=ch.updated_at.isoformat(),
|
||||||
|
))
|
||||||
|
|
||||||
|
# Seed 3 providers (idempotent)
|
||||||
|
from cyclone.providers import Provider
|
||||||
|
now = utcnow().isoformat()
|
||||||
|
for npi, label in [
|
||||||
|
("1881068062", "Montrose"),
|
||||||
|
("1851446637", "Delta"),
|
||||||
|
("1467507269", "Salida"),
|
||||||
|
]:
|
||||||
|
if s.get(ProviderORM, npi) is None:
|
||||||
|
s.add(ProviderORM(
|
||||||
|
npi=npi,
|
||||||
|
label=label,
|
||||||
|
legal_name="TOC, Inc.",
|
||||||
|
tax_id="721587149",
|
||||||
|
taxonomy_code="251E00000X",
|
||||||
|
address_line1="1100 East Main St",
|
||||||
|
address_line2="Suite A",
|
||||||
|
city="Montrose",
|
||||||
|
state="CO",
|
||||||
|
zip="814014063",
|
||||||
|
is_active=1,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
))
|
||||||
|
|
||||||
|
# Seed CO_TXIX payer (idempotent)
|
||||||
|
if s.get(PayerORM, "CO_TXIX") is None:
|
||||||
|
s.add(PayerORM(
|
||||||
|
payer_id="CO_TXIX",
|
||||||
|
name="Colorado Medical Assistance Program",
|
||||||
|
receiver_name="COLORADO MEDICAL ASSISTANCE PROGRAM",
|
||||||
|
receiver_id="COMEDASSISTPROG",
|
||||||
|
is_active=1,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
))
|
||||||
|
# 837P config block
|
||||||
|
s.add(PayerConfigORM(
|
||||||
|
payer_id="CO_TXIX",
|
||||||
|
transaction_type="837P",
|
||||||
|
config_json={
|
||||||
|
"submitter_name": "Dzinesco",
|
||||||
|
"submitter_contact_name": "Tyler Martinez",
|
||||||
|
"submitter_contact_email": "tyler@dzinesco.com",
|
||||||
|
"receiver_name": "COLORADO MEDICAL ASSISTANCE PROGRAM",
|
||||||
|
"receiver_id_qualifier": "46",
|
||||||
|
"receiver_id": "COMEDASSISTPROG",
|
||||||
|
"bht06_allowed": ["CH", "RP"],
|
||||||
|
"bht06_default": "CH",
|
||||||
|
"sbr09_default": "MC",
|
||||||
|
"sbr09_allowed": ["MC", "16", "MA", "MB", "ZZ"],
|
||||||
|
"payer_id_qualifier": "PI",
|
||||||
|
"payer_id": "CO_TXIX",
|
||||||
|
"pwk_supported": False,
|
||||||
|
"cas_2320_group_allowed": False,
|
||||||
|
"claim_type_codes": {"11": "Office", "12": "Home", "99": "Other"},
|
||||||
|
},
|
||||||
|
updated_at=now,
|
||||||
|
))
|
||||||
|
# 835 config block
|
||||||
|
s.add(PayerConfigORM(
|
||||||
|
payer_id="CO_TXIX",
|
||||||
|
transaction_type="835",
|
||||||
|
config_json={
|
||||||
|
"expected_payer_tax_ids": [
|
||||||
|
"81-1725341", "811725341", "84-0644739",
|
||||||
|
"840644739", "1811725341",
|
||||||
|
],
|
||||||
|
"expected_payer_health_plan_id": "7912900843",
|
||||||
|
"payer_name_pattern": "^CO_(TXIX|BHA)$",
|
||||||
|
},
|
||||||
|
updated_at=now,
|
||||||
|
))
|
||||||
|
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SP9: ORM-to-Pydantic conversion helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _provider_orm_to_dict(row) -> dict:
|
||||||
|
return {
|
||||||
|
"npi": row.npi,
|
||||||
|
"label": row.label,
|
||||||
|
"legal_name": row.legal_name,
|
||||||
|
"tax_id": row.tax_id,
|
||||||
|
"taxonomy_code": row.taxonomy_code,
|
||||||
|
"address_line1": row.address_line1,
|
||||||
|
"address_line2": row.address_line2,
|
||||||
|
"city": row.city,
|
||||||
|
"state": row.state,
|
||||||
|
"zip": row.zip,
|
||||||
|
"is_active": bool(row.is_active),
|
||||||
|
"created_at": row.created_at,
|
||||||
|
"updated_at": row.updated_at,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _payer_orm_to_dict(row) -> dict:
|
||||||
|
return {
|
||||||
|
"payer_id": row.payer_id,
|
||||||
|
"name": row.name,
|
||||||
|
"receiver_name": row.receiver_name,
|
||||||
|
"receiver_id": row.receiver_id,
|
||||||
|
"is_active": bool(row.is_active),
|
||||||
|
"created_at": row.created_at,
|
||||||
|
"updated_at": row.updated_at,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
# Module-level singleton — same import path the old InMemoryStore used.
|
# Module-level singleton — same import path the old InMemoryStore used.
|
||||||
store = CycloneStore()
|
store = CycloneStore()
|
||||||
+42
@@ -0,0 +1,42 @@
|
|||||||
|
ISA*00* *00* *ZZ*COMEDICAID *ZZ*DZINESCO *240620*1200*^*00501*000000123*0*P*:~
|
||||||
|
GS*HN*COMEDICAID*DZINESCO*20240620*1200*1*X*005010X214~
|
||||||
|
ST*277CA*0001*005010X214~
|
||||||
|
BHT*0085*08*REFNUM001*20240620*1200*TH~
|
||||||
|
HL*1**20*1~
|
||||||
|
NM1*PR*2*COLORADO MEDICAL ASSIST*****PI*COMEDICAID~
|
||||||
|
TRN*2*REFNUM001~
|
||||||
|
DTP*050*RD8*20240601-20240630~
|
||||||
|
DTP*009*RD8*20240601-20240630~
|
||||||
|
HL*2*1*21*1~
|
||||||
|
NM1*41*2*DZINESCO*****46*11525703~
|
||||||
|
TRN*2*REFNUM001~
|
||||||
|
HL*3*2*19*1~
|
||||||
|
NM1*IL*1*DOE*JOHN****MI*MEMBERID001~
|
||||||
|
TRN*2*TRACE001~
|
||||||
|
HL*4*3*PT~
|
||||||
|
NM1*QC*1*DOE*JANE~
|
||||||
|
REF*1K*CLAIM001~
|
||||||
|
REF*EJ*721587149~
|
||||||
|
STC*A3:19:PR*20240620*WQ*100.00~
|
||||||
|
QTY*90*1~
|
||||||
|
AMT*YU*100.00~
|
||||||
|
DTP*472*RD8*20240615-20240615~
|
||||||
|
HL*5*3*PT~
|
||||||
|
NM1*QC*1*SMITH*ROBERT~
|
||||||
|
REF*1K*CLAIM002~
|
||||||
|
REF*EJ*721587149~
|
||||||
|
STC*A6:19:PR*20240620*U*250.00~
|
||||||
|
QTY*90*1~
|
||||||
|
AMT*YU*250.00~
|
||||||
|
DTP*472*D8*20240610~
|
||||||
|
HL*6*3*PT~
|
||||||
|
NM1*QC*1*GARCIA*MARIA~
|
||||||
|
REF*1K*CLAIM003~
|
||||||
|
REF*EJ*721587149~
|
||||||
|
STC*A8:19:PR*20240620*U*175.50~
|
||||||
|
QTY*90*1~
|
||||||
|
AMT*YU*175.50~
|
||||||
|
DTP*472*RD8*20240612-20240612~
|
||||||
|
SE*40*0001~
|
||||||
|
GE*1*1~
|
||||||
|
IEA*1*000000123~
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
ISA*00* *00* *ZZ*COMEDICAID *ZZ*DZINESCO *240620*1200*^*00501*000000789*0*P*:~
|
||||||
|
GS*HN*COMEDICAID*DZINESCO*20240620*1200*3*X*005010X214~
|
||||||
|
ST*277CA*0001*005010X214~
|
||||||
|
BHT*0085*08*REFNUM003*20240620*1200*TH~
|
||||||
|
HL*1**20*1~
|
||||||
|
HL*2*1*21*1~
|
||||||
|
HL*3*2*19*1~
|
||||||
|
HL*4*3*PT~
|
||||||
|
REF*1K*CLAIM099~
|
||||||
|
REF*EJ*721587149~
|
||||||
|
STC*A7:19:PR*20240620*U*99.99~
|
||||||
|
SE*8*0001~
|
||||||
|
GE*1*1~
|
||||||
|
IEA*1*000000789~
|
||||||
+15
@@ -0,0 +1,15 @@
|
|||||||
|
ISA*00* *00* *ZZ*COMEDICAID *ZZ*DZINESCO *240620*1200*^*00501*000000456*0*P*:~
|
||||||
|
GS*HN*COMEDICAID*DZINESCO*20240620*1200*2*X*005010X214~
|
||||||
|
ST*277*0001*005010X214~
|
||||||
|
BHT*0085*08*REFNUM002*20240620*1200*TH~
|
||||||
|
HL*1**20*1~
|
||||||
|
NM1*PR*2*COLORADO MEDICAL ASSIST*****PI*COMEDICAID~
|
||||||
|
HL*2*1*21*1~
|
||||||
|
HL*3*2*19*1~
|
||||||
|
HL*4*3*PT~
|
||||||
|
REF*1K*CLAIM004~
|
||||||
|
REF*EJ*721587149~
|
||||||
|
STC*A3:19:PR*20240620*WQ*300.00~
|
||||||
|
SE*10*0001~
|
||||||
|
GE*1*1~
|
||||||
|
IEA*1*000000456~
|
||||||
+1
-1
@@ -7,7 +7,7 @@ PER*IC*Test Contact*EM*test@example.com~
|
|||||||
NM1*40*2*COLORADO MEDICAL ASSISTANCE PROGRAM*****46*COMEDASSISTPROG~
|
NM1*40*2*COLORADO MEDICAL ASSISTANCE PROGRAM*****46*COMEDASSISTPROG~
|
||||||
HL*1**20*1~
|
HL*1**20*1~
|
||||||
PRV*BI*PXC*251E00000X~
|
PRV*BI*PXC*251E00000X~
|
||||||
NM1*85*2*Test Provider Inc*****XX*1234567890~
|
NM1*85*2*Test Provider Inc*****XX*1993999998~
|
||||||
N3*123 Test St~
|
N3*123 Test St~
|
||||||
N4*Denver*CO*80202~
|
N4*Denver*CO*80202~
|
||||||
REF*EI*123456789~
|
REF*EI*123456789~
|
||||||
|
|||||||
+1
@@ -0,0 +1 @@
|
|||||||
|
ISA*00* *00* *ZZ*SENDER *ZZ*RECEIVER *260520*1750*^*00501*000000001*0*P*:~TA1*000000001*20260520*1750*A*000*20260520~IEA*1*000000001~
|
||||||
@@ -51,17 +51,19 @@ def test_migration_0002_creates_acks_table():
|
|||||||
|
|
||||||
def test_migration_latest_idempotent_on_fresh_db():
|
def test_migration_latest_idempotent_on_fresh_db():
|
||||||
"""Re-running the migration on the same DB must be a no-op (PRAGMA
|
"""Re-running the migration on the same DB must be a no-op (PRAGMA
|
||||||
user_version already at the latest version — currently 5 after the
|
user_version already at the latest version — currently 12 after
|
||||||
0004 rejection columns + state-history index (SP6) and the 0005
|
0004-0006 line_reconciliation, 0005 ta1_acks, SP9's 0007
|
||||||
ta1_acks table (this PR)."""
|
providers/payers/clearhouse, SP10's 0008 payer_rejected,
|
||||||
|
SP11's 0009 audit_log, SP14's 0010 payer_rejected_acknowledged,
|
||||||
|
SP16's 0011 processed_inbound_files, SP17's 0012 db_backups)."""
|
||||||
with db.engine().begin() as c:
|
with db.engine().begin() as c:
|
||||||
v1 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
v1 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
||||||
assert v1 == 6
|
assert v1 == 12
|
||||||
# A second run should not raise and should not bump the version.
|
# A second run should not raise and should not bump the version.
|
||||||
db_migrate.run(db.engine())
|
db_migrate.run(db.engine())
|
||||||
with db.engine().begin() as c:
|
with db.engine().begin() as c:
|
||||||
v2 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
v2 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
||||||
assert v2 == 6
|
assert v2 == 12
|
||||||
|
|
||||||
|
|
||||||
def test_add_ack_persists_row():
|
def test_add_ack_persists_row():
|
||||||
|
|||||||
@@ -31,10 +31,18 @@ def client() -> TestClient:
|
|||||||
|
|
||||||
|
|
||||||
def test_health_endpoint(client: TestClient):
|
def test_health_endpoint(client: TestClient):
|
||||||
|
"""SP19: health endpoint now returns a subsystem snapshot."""
|
||||||
resp = client.get("/api/health")
|
resp = client.get("/api/health")
|
||||||
assert resp.status_code == 200
|
assert resp.status_code == 200
|
||||||
body = resp.json()
|
body = resp.json()
|
||||||
assert body == {"status": "ok", "version": __version__}
|
# Old contract (status + version) is preserved.
|
||||||
|
assert body["status"] == "ok"
|
||||||
|
assert body["version"] == __version__
|
||||||
|
# SP19 additions.
|
||||||
|
assert "db" in body and body["db"].get("ok") is True
|
||||||
|
assert "scheduler" in body
|
||||||
|
assert "pubsub" in body
|
||||||
|
assert "batch" in body
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
@@ -152,3 +160,51 @@ def test_cors_headers_present(client: TestClient):
|
|||||||
)
|
)
|
||||||
assert resp.headers.get("access-control-allow-origin") == "http://localhost:5173"
|
assert resp.headers.get("access-control-allow-origin") == "http://localhost:5173"
|
||||||
assert "POST" in resp.headers.get("access-control-allow-methods", "").upper()
|
assert "POST" in resp.headers.get("access-control-allow-methods", "").upper()
|
||||||
|
|
||||||
|
|
||||||
|
def test_cors_headers_present_for_loopback_ip(client: TestClient):
|
||||||
|
# ``http://127.0.0.1:5173`` is a distinct origin from
|
||||||
|
# ``http://localhost:5173`` per the CORS spec, even though both resolve
|
||||||
|
# to the same Vite dev server. Both must be allow-listed or tabs opened
|
||||||
|
# via the IP form silently break.
|
||||||
|
resp = client.options(
|
||||||
|
"/api/parse-837",
|
||||||
|
headers={
|
||||||
|
"Origin": "http://127.0.0.1:5173",
|
||||||
|
"Access-Control-Request-Method": "POST",
|
||||||
|
"Access-Control-Request-Headers": "content-type",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert resp.headers.get("access-control-allow-origin") == "http://127.0.0.1:5173"
|
||||||
|
|
||||||
|
|
||||||
|
def test_cors_extra_origins_via_env(client: TestClient, monkeypatch):
|
||||||
|
# LAN / staging hosts opt in via CYCLONE_ALLOWED_ORIGINS. The env var
|
||||||
|
# is a comma-separated list; the middleware must reflect each entry.
|
||||||
|
# The allow-list is built at module import, so we re-execute the
|
||||||
|
# module under the env var and build a TestClient against the
|
||||||
|
# reloaded app.
|
||||||
|
monkeypatch.setenv(
|
||||||
|
"CYCLONE_ALLOWED_ORIGINS", "http://192.168.1.42:5173,https://staging.example.com"
|
||||||
|
)
|
||||||
|
import importlib
|
||||||
|
from cyclone import api as api_module
|
||||||
|
from fastapi.testclient import TestClient as _TC
|
||||||
|
importlib.reload(api_module)
|
||||||
|
try:
|
||||||
|
with _TC(api_module.app) as tc:
|
||||||
|
for origin in ("http://192.168.1.42:5173", "https://staging.example.com"):
|
||||||
|
resp = tc.options(
|
||||||
|
"/api/parse-837",
|
||||||
|
headers={
|
||||||
|
"Origin": origin,
|
||||||
|
"Access-Control-Request-Method": "POST",
|
||||||
|
"Access-Control-Request-Headers": "content-type",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert resp.headers.get("access-control-allow-origin") == origin
|
||||||
|
finally:
|
||||||
|
monkeypatch.delenv("CYCLONE_ALLOWED_ORIGINS", raising=False)
|
||||||
|
# Reload once more so the module-level allow-list returns to its
|
||||||
|
# default for any test that imports `cyclone.api` after this one.
|
||||||
|
importlib.reload(api_module)
|
||||||
|
|||||||
@@ -0,0 +1,169 @@
|
|||||||
|
"""Tests for the FastAPI surface in ``cyclone.api`` for the 277CA endpoint.
|
||||||
|
|
||||||
|
SP10 T3. Mirrors ``test_api_999.py``:
|
||||||
|
- 400 on empty / undecodable / malformed EDI (never 500).
|
||||||
|
- 200 on success with the parsed envelope + counts.
|
||||||
|
- After parse, ``apply_277ca_rejections`` stamps matching claim rows.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.api import app
|
||||||
|
from cyclone.db import Claim, init_db
|
||||||
|
|
||||||
|
ACCEPTED_FIXTURE = Path(__file__).parent / "fixtures" / "minimal_277ca.txt"
|
||||||
|
REJECTED_FIXTURE = Path(__file__).parent / "fixtures" / "minimal_277ca_rejected_only.txt"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _fresh_db():
|
||||||
|
"""Each test gets a fresh DB and a clean 277CA ack list."""
|
||||||
|
init_db()
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client() -> TestClient:
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_claim(claim_id: str, pcn: str) -> None:
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
s.add(Claim(
|
||||||
|
id=claim_id, batch_id="BATCH-1",
|
||||||
|
patient_control_number=pcn, charge_amount=100.00,
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Happy path
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestParse277CAEndpointHappyPath:
|
||||||
|
def test_upload_minimal_277ca_returns_200(self, client: TestClient):
|
||||||
|
text = ACCEPTED_FIXTURE.read_text()
|
||||||
|
resp = client.post(
|
||||||
|
"/api/parse-277ca",
|
||||||
|
files={"file": ("minimal_277ca.txt", text, "text/plain")},
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
body = resp.json()
|
||||||
|
assert "ack" in body
|
||||||
|
assert "parsed" in body
|
||||||
|
ack = body["ack"]
|
||||||
|
assert ack["accepted_count"] == 1
|
||||||
|
assert ack["rejected_count"] == 1
|
||||||
|
assert ack["pended_count"] == 1
|
||||||
|
assert ack["control_number"] == "000000123"
|
||||||
|
|
||||||
|
def test_persists_two77ca_row(self, client: TestClient):
|
||||||
|
text = ACCEPTED_FIXTURE.read_text()
|
||||||
|
client.post(
|
||||||
|
"/api/parse-277ca",
|
||||||
|
files={"file": ("minimal_277ca.txt", text, "text/plain")},
|
||||||
|
)
|
||||||
|
rows_resp = client.get("/api/277ca-acks")
|
||||||
|
assert rows_resp.status_code == 200
|
||||||
|
rows = rows_resp.json()
|
||||||
|
assert rows["total"] == 1
|
||||||
|
assert rows["items"][0]["control_number"] == "000000123"
|
||||||
|
|
||||||
|
def test_get_277ca_ack_by_id(self, client: TestClient):
|
||||||
|
text = ACCEPTED_FIXTURE.read_text()
|
||||||
|
post_resp = client.post(
|
||||||
|
"/api/parse-277ca",
|
||||||
|
files={"file": ("minimal_277ca.txt", text, "text/plain")},
|
||||||
|
)
|
||||||
|
ack_id = post_resp.json()["ack"]["id"]
|
||||||
|
detail = client.get(f"/api/277ca-acks/{ack_id}")
|
||||||
|
assert detail.status_code == 200
|
||||||
|
assert detail.json()["control_number"] == "000000123"
|
||||||
|
|
||||||
|
def test_stamps_matching_claim(self, client: TestClient):
|
||||||
|
"""A rejected 277CA claim with REF*1K=CLAIM002 stamps claim c2."""
|
||||||
|
# Seed two claims matching the fixture's PCNs.
|
||||||
|
_seed_claim("c1", "CLAIM001")
|
||||||
|
_seed_claim("c2", "CLAIM002")
|
||||||
|
text = ACCEPTED_FIXTURE.read_text()
|
||||||
|
client.post(
|
||||||
|
"/api/parse-277ca",
|
||||||
|
files={"file": ("minimal_277ca.txt", text, "text/plain")},
|
||||||
|
)
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
c1 = s.get(Claim, "c1")
|
||||||
|
c2 = s.get(Claim, "c2")
|
||||||
|
assert c1.payer_rejected_at is None
|
||||||
|
assert c2.payer_rejected_at is not None
|
||||||
|
assert c2.payer_rejected_status_code == "A6"
|
||||||
|
assert "A6" in c2.payer_rejected_reason
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Error paths
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestParse277CAEndpointErrors:
|
||||||
|
def test_empty_file_raises_400(self, client: TestClient):
|
||||||
|
resp = client.post(
|
||||||
|
"/api/parse-277ca",
|
||||||
|
files={"file": ("empty.txt", "", "text/plain")},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 400, resp.text
|
||||||
|
assert "error" in resp.json()
|
||||||
|
|
||||||
|
def test_garbage_raises_400(self, client: TestClient):
|
||||||
|
resp = client.post(
|
||||||
|
"/api/parse-277ca",
|
||||||
|
files={"file": ("garbage.txt", "this is not EDI", "text/plain")},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 400, resp.text
|
||||||
|
|
||||||
|
def test_wrong_transaction_set_raises_400(self, client: TestClient):
|
||||||
|
"""A 999 must NOT be accepted as a 277CA — different transaction set id."""
|
||||||
|
text = (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X231A1~"
|
||||||
|
"ST*999*0001*005010X231A1~"
|
||||||
|
"AK1*HC*0001~"
|
||||||
|
"AK9*A*0*0*0~"
|
||||||
|
"SE*4*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
|
resp = client.post(
|
||||||
|
"/api/parse-277ca",
|
||||||
|
files={"file": ("bad.txt", text, "text/plain")},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 400, resp.text
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Inbox lane
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestInboxPayerRejectedLane:
|
||||||
|
def test_payer_rejected_claim_appears_in_lane(self, client: TestClient):
|
||||||
|
"""A claim with payer_rejected_at set must appear in the payer_rejected lane."""
|
||||||
|
_seed_claim("c1", "CLAIM099")
|
||||||
|
text = REJECTED_FIXTURE.read_text() # single A7 for CLAIM099
|
||||||
|
client.post(
|
||||||
|
"/api/parse-277ca",
|
||||||
|
files={"file": ("rejected.txt", text, "text/plain")},
|
||||||
|
)
|
||||||
|
lanes = client.get("/api/inbox/lanes").json()
|
||||||
|
assert "payer_rejected" in lanes
|
||||||
|
ids = [c["id"] for c in lanes["payer_rejected"]]
|
||||||
|
assert "c1" in ids
|
||||||
|
# The rejected lane (999 envelope) must be empty — we haven't
|
||||||
|
# uploaded a 999, so this claim isn't there.
|
||||||
|
assert "c1" not in [c["id"] for c in lanes["rejected"]]
|
||||||
@@ -0,0 +1,159 @@
|
|||||||
|
"""Tests for the audit-log admin API endpoints. SP11 T3."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.audit_log import AuditEvent, append_event
|
||||||
|
from cyclone.api import app
|
||||||
|
from cyclone.db import AuditLog, init_db
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _fresh_db():
|
||||||
|
init_db()
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client() -> TestClient:
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_audit(rows: list[AuditEvent]) -> None:
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
for ev in rows:
|
||||||
|
append_event(s, ev)
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# List endpoint
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestListAuditLog:
|
||||||
|
def test_empty_returns_zero_items(self, client: TestClient):
|
||||||
|
r = client.get("/api/admin/audit-log")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["total"] == 0
|
||||||
|
assert body["items"] == []
|
||||||
|
|
||||||
|
def test_returns_newest_first(self, client: TestClient):
|
||||||
|
_seed_audit([
|
||||||
|
AuditEvent(event_type="a.first", entity_type="claim", entity_id="c1"),
|
||||||
|
AuditEvent(event_type="a.second", entity_type="claim", entity_id="c1"),
|
||||||
|
AuditEvent(event_type="a.third", entity_type="claim", entity_id="c1"),
|
||||||
|
])
|
||||||
|
r = client.get("/api/admin/audit-log")
|
||||||
|
assert r.status_code == 200
|
||||||
|
items = r.json()["items"]
|
||||||
|
assert [i["event_type"] for i in items] == ["a.third", "a.second", "a.first"]
|
||||||
|
|
||||||
|
def test_filter_by_entity(self, client: TestClient):
|
||||||
|
_seed_audit([
|
||||||
|
AuditEvent(event_type="x", entity_type="claim", entity_id="c1"),
|
||||||
|
AuditEvent(event_type="x", entity_type="claim", entity_id="c2"),
|
||||||
|
AuditEvent(event_type="x", entity_type="batch", entity_id="b1"),
|
||||||
|
])
|
||||||
|
r = client.get("/api/admin/audit-log", params={"entity_type": "claim", "entity_id": "c1"})
|
||||||
|
items = r.json()["items"]
|
||||||
|
assert len(items) == 1
|
||||||
|
assert items[0]["entity_id"] == "c1"
|
||||||
|
|
||||||
|
def test_filter_by_event_type(self, client: TestClient):
|
||||||
|
_seed_audit([
|
||||||
|
AuditEvent(event_type="claim.parsed", entity_type="claim", entity_id="c1"),
|
||||||
|
AuditEvent(event_type="claim.rejected", entity_type="claim", entity_id="c1"),
|
||||||
|
])
|
||||||
|
r = client.get("/api/admin/audit-log", params={"event_type": "claim.rejected"})
|
||||||
|
items = r.json()["items"]
|
||||||
|
assert len(items) == 1
|
||||||
|
assert items[0]["event_type"] == "claim.rejected"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Verify endpoint
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestVerifyAuditLog:
|
||||||
|
def test_empty_chain_returns_ok(self, client: TestClient):
|
||||||
|
r = client.get("/api/admin/audit-log/verify")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["checked"] == 0
|
||||||
|
|
||||||
|
def test_clean_chain_returns_ok(self, client: TestClient):
|
||||||
|
_seed_audit([
|
||||||
|
AuditEvent(event_type="x", entity_type="y", entity_id=f"id-{i}")
|
||||||
|
for i in range(10)
|
||||||
|
])
|
||||||
|
r = client.get("/api/admin/audit-log/verify")
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["checked"] == 10
|
||||||
|
|
||||||
|
def test_tampered_chain_returns_failure(self, client: TestClient):
|
||||||
|
_seed_audit([
|
||||||
|
AuditEvent(event_type="x", entity_type="y", entity_id=f"id-{i}")
|
||||||
|
for i in range(5)
|
||||||
|
])
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.query(AuditLog).filter(AuditLog.id == 3).first()
|
||||||
|
row.payload_json = json.dumps({"evil": True})
|
||||||
|
s.commit()
|
||||||
|
r = client.get("/api/admin/audit-log/verify")
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is False
|
||||||
|
assert body["first_bad_id"] == 3
|
||||||
|
assert body["reason"]
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# End-to-end: 999 → audit row appears in list
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestAuditLogHookedIntoEndpoints:
|
||||||
|
def test_parse_999_creates_audit_rows(self, client: TestClient):
|
||||||
|
"""Upload a 999 with one rejected set, see an audit row appear."""
|
||||||
|
# Seed a claim so the 999 rejection matches.
|
||||||
|
from cyclone.db import Claim, ClaimState
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
s.add(Claim(
|
||||||
|
id="c1", batch_id="B-1", patient_control_number="0001",
|
||||||
|
state=ClaimState.SUBMITTED,
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
text = (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X231A1~"
|
||||||
|
"ST*999*0001*005010X231A1~"
|
||||||
|
"AK1*HC*0001~"
|
||||||
|
"AK2*837*0001~"
|
||||||
|
"AK5*R~"
|
||||||
|
"AK9*R*1*0*1~"
|
||||||
|
"SE*7*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
|
r = client.post(
|
||||||
|
"/api/parse-999",
|
||||||
|
files={"file": ("r.txt", text, "text/plain")},
|
||||||
|
)
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
|
||||||
|
# One audit row should have appeared for the rejected claim.
|
||||||
|
r2 = client.get(
|
||||||
|
"/api/admin/audit-log",
|
||||||
|
params={"entity_type": "claim", "entity_id": "c1"},
|
||||||
|
)
|
||||||
|
items = r2.json()["items"]
|
||||||
|
assert any(i["event_type"] == "claim.rejected" for i in items)
|
||||||
@@ -0,0 +1,313 @@
|
|||||||
|
"""SP17 — Admin backup API endpoint tests.
|
||||||
|
|
||||||
|
Covers:
|
||||||
|
- POST /api/admin/backup/create
|
||||||
|
- GET /api/admin/backup/list
|
||||||
|
- GET /api/admin/backup/status
|
||||||
|
- POST /api/admin/backup/{id}/verify
|
||||||
|
- POST /api/admin/backup/{id}/restore/initiate
|
||||||
|
- POST /api/admin/backup/{id}/restore/confirm
|
||||||
|
- POST /api/admin/backup/prune
|
||||||
|
- POST /api/admin/backup/scheduler/{start,stop,tick}
|
||||||
|
|
||||||
|
Each fixture starts a clean DB + BackupService configured with a
|
||||||
|
known passphrase. We deliberately do NOT enable SQLCipher here —
|
||||||
|
the backup layer is independent of SQLCipher encryption at rest.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def _backup_env(tmp_path, monkeypatch):
|
||||||
|
"""Fresh sqlite DB + BackupService with passphrase. Reset module singletons."""
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import backup_scheduler as sched_mod
|
||||||
|
from cyclone.db import Batch
|
||||||
|
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
|
||||||
|
# Make sure there's at least one row so the backup isn't a no-op.
|
||||||
|
import uuid
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
s.add(Batch(
|
||||||
|
id=str(uuid.uuid4()),
|
||||||
|
kind="837P",
|
||||||
|
input_filename="seed.x12",
|
||||||
|
parsed_at=datetime.now(timezone.utc),
|
||||||
|
totals_json=None,
|
||||||
|
validation_json=None,
|
||||||
|
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
backup_dir = tmp_path / "backups"
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
sched_mod.reset_backup_scheduler_for_tests()
|
||||||
|
svc = svc_mod.configure_backup_service(
|
||||||
|
backup_dir=backup_dir, passphrase="api-test-pass", retention_days=7,
|
||||||
|
)
|
||||||
|
yield svc, backup_dir
|
||||||
|
sched_mod.reset_backup_scheduler_for_tests()
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
def _client():
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# /backup/create
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_create_returns_metadata_and_persists_row(_backup_env):
|
||||||
|
svc, backup_dir = _backup_env
|
||||||
|
r = _client().post("/api/admin/backup/create")
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
b = body["backup"]
|
||||||
|
assert b["size_bytes"] > 0
|
||||||
|
assert b["db_fingerprint"].startswith("sha256:")
|
||||||
|
assert b["table_count"] >= 1
|
||||||
|
assert b["created_at"]
|
||||||
|
# File actually exists on disk.
|
||||||
|
assert (backup_dir / b["filename"]).exists()
|
||||||
|
# Sidecar metadata echoed.
|
||||||
|
sc = body["sidecar"]
|
||||||
|
assert sc["kdf"] == "PBKDF2-HMAC-SHA256"
|
||||||
|
assert sc["kdf_iterations"] == 200_000
|
||||||
|
assert sc["cipher"] == "AES-256-GCM"
|
||||||
|
|
||||||
|
|
||||||
|
def test_create_503_when_service_unconfigured(tmp_path, monkeypatch):
|
||||||
|
"""If BackupService was never configured, create returns 503."""
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
svc_mod.reset_backup_service_for_tests()
|
||||||
|
try:
|
||||||
|
r = _client().post("/api/admin/backup/create")
|
||||||
|
assert r.status_code == 503
|
||||||
|
assert "not configured" in r.json()["detail"].lower()
|
||||||
|
finally:
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# /backup/list
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_returns_newest_first(_backup_env):
|
||||||
|
svc, _ = _backup_env
|
||||||
|
client = _client()
|
||||||
|
client.post("/api/admin/backup/create")
|
||||||
|
client.post("/api/admin/backup/create")
|
||||||
|
r = client.get("/api/admin/backup/list")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["count"] == 2
|
||||||
|
assert body["files"][0]["id"] > body["files"][1]["id"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_filter_by_status(_backup_env):
|
||||||
|
svc, _ = _backup_env
|
||||||
|
client = _client()
|
||||||
|
client.post("/api/admin/backup/create")
|
||||||
|
r = client.get("/api/admin/backup/list?status=ok")
|
||||||
|
assert r.json()["count"] == 1
|
||||||
|
r = client.get("/api/admin/backup/list?status=error")
|
||||||
|
assert r.json()["count"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# /backup/status
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_status_returns_counts_and_dirs(_backup_env):
|
||||||
|
svc, backup_dir = _backup_env
|
||||||
|
client = _client()
|
||||||
|
client.post("/api/admin/backup/create")
|
||||||
|
r = client.get("/api/admin/backup/status")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["totals"]["ok"] == 1
|
||||||
|
assert body["backup_dir"] == str(backup_dir)
|
||||||
|
assert body["retention_days"] == 7
|
||||||
|
assert body["last_backup_at"] is not None
|
||||||
|
assert body["last_ok_backup_at"] is not None
|
||||||
|
# The scheduler may or may not be configured depending on lifespan.
|
||||||
|
assert "scheduler" in body
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# /backup/{id}/verify
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_verify_ok_after_create(_backup_env):
|
||||||
|
svc, _ = _backup_env
|
||||||
|
client = _client()
|
||||||
|
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||||
|
r = client.post(f"/api/admin/backup/{cid}/verify")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["expected_fingerprint"] == body["actual_fingerprint"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_verify_detects_tampered_ciphertext(_backup_env):
|
||||||
|
from cyclone import backup as backup_mod
|
||||||
|
svc, backup_dir = _backup_env
|
||||||
|
client = _client()
|
||||||
|
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||||
|
fname = svc.list_backups()[0].filename
|
||||||
|
# Flip a bit in the ciphertext.
|
||||||
|
bin_path = backup_dir / fname
|
||||||
|
data = bytearray(bin_path.read_bytes())
|
||||||
|
data[backup_mod.NONCE_LEN + 5] ^= 0x01
|
||||||
|
bin_path.write_bytes(bytes(data))
|
||||||
|
r = client.post(f"/api/admin/backup/{cid}/verify")
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert r.json()["ok"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_verify_404_when_unknown_backup(_backup_env):
|
||||||
|
r = _client().post("/api/admin/backup/99999/verify")
|
||||||
|
# The service raises BackupError; the endpoint should return 503 (no svc) or 400
|
||||||
|
# depending on flow. Let's see what happens.
|
||||||
|
assert r.status_code in (400, 404, 503)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# /backup/{id}/restore/{initiate,confirm}
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_restore_two_step_via_api(_backup_env):
|
||||||
|
svc, _ = _backup_env
|
||||||
|
from cyclone.db import Batch
|
||||||
|
import uuid
|
||||||
|
client = _client()
|
||||||
|
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||||
|
|
||||||
|
# Mutate the live DB (add another Batch row).
|
||||||
|
with __import__("cyclone").db.SessionLocal()() as s:
|
||||||
|
s.add(Batch(
|
||||||
|
id=str(uuid.uuid4()),
|
||||||
|
kind="837P",
|
||||||
|
input_filename="mutated.x12",
|
||||||
|
parsed_at=datetime.now(timezone.utc),
|
||||||
|
totals_json=None,
|
||||||
|
validation_json=None,
|
||||||
|
raw_result_json={"envelope": {"control_number": "2"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
# Step 1: initiate.
|
||||||
|
r1 = client.post(f"/api/admin/backup/{cid}/restore/initiate")
|
||||||
|
assert r1.status_code == 200, r1.text
|
||||||
|
body1 = r1.json()
|
||||||
|
assert body1["restore_token"]
|
||||||
|
assert body1["preview"]["backup_table_count"] >= 1
|
||||||
|
assert body1["preview"]["backup_db_fingerprint"] != body1["preview"]["current_db_fingerprint"]
|
||||||
|
|
||||||
|
# Step 2: confirm.
|
||||||
|
r2 = client.post(
|
||||||
|
f"/api/admin/backup/{cid}/restore/confirm",
|
||||||
|
json={"restore_token": body1["restore_token"], "actor": "test"},
|
||||||
|
)
|
||||||
|
assert r2.status_code == 200, r2.text
|
||||||
|
body2 = r2.json()
|
||||||
|
assert body2["ok"] is True
|
||||||
|
assert body2["new_db_fingerprint"] == body1["preview"]["backup_db_fingerprint"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_restore_confirm_requires_token(_backup_env):
|
||||||
|
svc, _ = _backup_env
|
||||||
|
client = _client()
|
||||||
|
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||||
|
r = client.post(f"/api/admin/backup/{cid}/restore/confirm", json={})
|
||||||
|
assert r.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
def test_restore_confirm_rejects_wrong_token(_backup_env):
|
||||||
|
svc, _ = _backup_env
|
||||||
|
client = _client()
|
||||||
|
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||||
|
r = client.post(
|
||||||
|
f"/api/admin/backup/{cid}/restore/confirm",
|
||||||
|
json={"restore_token": "0" * 64},
|
||||||
|
)
|
||||||
|
assert r.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# /backup/prune
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_prune_deletes_old_backups(_backup_env):
|
||||||
|
svc, _ = _backup_env
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
client = _client()
|
||||||
|
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||||
|
# Age the backup past the retention cutoff.
|
||||||
|
with __import__("cyclone").db.SessionLocal()() as s:
|
||||||
|
row = s.get(DbBackup, cid)
|
||||||
|
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
||||||
|
s.commit()
|
||||||
|
r = client.post("/api/admin/backup/prune")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["deleted_count"] == 2 # .bin + .meta.json
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# /backup/scheduler/{start,stop,tick}
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_scheduler_endpoints_require_configured_scheduler(_backup_env, monkeypatch):
|
||||||
|
"""Without calling configure_backup_scheduler, the endpoints 503."""
|
||||||
|
svc, _ = _backup_env
|
||||||
|
# We did NOT call configure_backup_scheduler; the lifespan
|
||||||
|
# *might* have called it as a side effect of the TestClient
|
||||||
|
# entering its context. Either way, the scheduler endpoints
|
||||||
|
# need it to be present.
|
||||||
|
client = _client()
|
||||||
|
r = client.post("/api/admin/backup/scheduler/tick")
|
||||||
|
assert r.status_code in (200, 503)
|
||||||
|
|
||||||
|
|
||||||
|
def test_scheduler_tick_when_configured(_backup_env):
|
||||||
|
"""With a configured scheduler, tick runs and returns a result."""
|
||||||
|
from cyclone import backup_scheduler as sched_mod
|
||||||
|
svc, _ = _backup_env
|
||||||
|
sched_mod.configure_backup_scheduler(svc, interval_hours=24.0)
|
||||||
|
try:
|
||||||
|
client = _client()
|
||||||
|
r = client.post("/api/admin/backup/scheduler/tick")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["tick"]["created"] is not None
|
||||||
|
assert body["tick"]["created"]["id"] >= 1
|
||||||
|
finally:
|
||||||
|
sched_mod.reset_backup_scheduler_for_tests()
|
||||||
@@ -0,0 +1,217 @@
|
|||||||
|
"""SP15 — SQLCipher key rotation API endpoint tests.
|
||||||
|
|
||||||
|
We test the *wiring* of the endpoint:
|
||||||
|
1. Refuses with 400 when encryption is not enabled.
|
||||||
|
2. Refuses with 409 when a rotation is already in flight.
|
||||||
|
3. On success: calls rotate_db_key, updates the Keychain, rebuilds
|
||||||
|
the engine, writes an audit event, and returns the fingerprints.
|
||||||
|
4. On Keychain write failure: returns 503 (DB is rotated, Keychain
|
||||||
|
is stale; operator must restore).
|
||||||
|
|
||||||
|
The actual ``PRAGMA rekey`` mechanics are tested in ``test_db_crypto.py``
|
||||||
|
(see :class:`TestRotateDbKey`); we don't duplicate that here.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
# Skip if sqlcipher3 isn't installed.
|
||||||
|
pytestmark = pytest.mark.skipif(
|
||||||
|
not __import__(
|
||||||
|
"cyclone.db_crypto", fromlist=["is_sqlcipher_available"]
|
||||||
|
).is_sqlcipher_available(),
|
||||||
|
reason="sqlcipher3 not installed",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _stub_rotate_ok(*, url, old_key, new_key) -> dict:
|
||||||
|
"""Return a synthetic RotateKeyResult for endpoint wiring tests."""
|
||||||
|
from cyclone.db_crypto import RotateKeyResult
|
||||||
|
return RotateKeyResult(
|
||||||
|
ok=True,
|
||||||
|
old_fingerprint="aaaa1111",
|
||||||
|
new_fingerprint="bbbb2222",
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
table_count=12,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestRotateKeyRefusesWhenNotEncrypted:
|
||||||
|
def test_400_when_encryption_disabled(self, tmp_path, monkeypatch):
|
||||||
|
from cyclone import db, db_crypto
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/plain.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: None)
|
||||||
|
db.init_db()
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post("/api/admin/db/rotate-key")
|
||||||
|
assert r.status_code == 400
|
||||||
|
assert "not enabled" in r.json()["detail"]
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
class TestRotateKeyEndpointWiring:
|
||||||
|
@pytest.fixture
|
||||||
|
def _fake_encrypted_env(self, tmp_path, monkeypatch):
|
||||||
|
"""Set up: encryption-enabled DB on disk, fake Keychain
|
||||||
|
(read + write), and the engine initialized here.
|
||||||
|
|
||||||
|
With NullPool (see ``cyclone.db._make_engine``), every thread
|
||||||
|
opens its own SQLCipher connection — no cross-thread reuse,
|
||||||
|
no ProgramingError. The endpoint runs on the request thread
|
||||||
|
and verification runs on the test thread; both get fresh
|
||||||
|
per-thread connections transparently.
|
||||||
|
"""
|
||||||
|
from cyclone import db, db_crypto
|
||||||
|
|
||||||
|
db_file = tmp_path / "cyclone.db"
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{db_file}")
|
||||||
|
db._reset_for_tests()
|
||||||
|
fake_kc = {db_crypto.KEYCHAIN_ACCOUNT: "old-test-key-1"}
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda n: fake_kc.get(n))
|
||||||
|
monkeypatch.setattr("cyclone.secrets.get_secret", lambda n: fake_kc.get(n))
|
||||||
|
monkeypatch.setattr("cyclone.secrets.set_secret",
|
||||||
|
lambda n, v: fake_kc.__setitem__(n, v) or True)
|
||||||
|
# The endpoint's actual rekey is stubbed; the real PRAGMA
|
||||||
|
# rekey mechanics are tested in test_db_crypto.py::TestRotateDbKey.
|
||||||
|
monkeypatch.setattr("cyclone.api._db_crypto.rotate_db_key", _stub_rotate_ok)
|
||||||
|
db.init_db()
|
||||||
|
yield db_file, fake_kc
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
def test_successful_rotation_updates_keychain_and_writes_audit(
|
||||||
|
self, _fake_encrypted_env,
|
||||||
|
):
|
||||||
|
from cyclone import db
|
||||||
|
|
||||||
|
# The fixture stubs rotate_db_key to a no-op success.
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post(
|
||||||
|
"/api/admin/db/rotate-key",
|
||||||
|
json={"actor": "alice", "reason": "scheduled"},
|
||||||
|
)
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["old_fingerprint"] == "aaaa1111"
|
||||||
|
assert body["new_fingerprint"] == "bbbb2222"
|
||||||
|
assert body["table_count"] == 12
|
||||||
|
|
||||||
|
def test_successful_rotation_writes_audit_event(
|
||||||
|
self, _fake_encrypted_env,
|
||||||
|
):
|
||||||
|
from cyclone import db
|
||||||
|
import json as _json
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post("/api/admin/db/rotate-key", json={"actor": "bob"})
|
||||||
|
assert r.status_code == 200
|
||||||
|
|
||||||
|
from cyclone.db import AuditLog
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
events = (
|
||||||
|
session.query(AuditLog)
|
||||||
|
.filter(AuditLog.event_type == "db.key_rotated")
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
assert len(events) == 1
|
||||||
|
e = events[0]
|
||||||
|
assert e.entity_type == "database"
|
||||||
|
assert e.entity_id == "cyclone.db"
|
||||||
|
assert e.actor == "bob"
|
||||||
|
payload = _json.loads(e.payload_json)
|
||||||
|
assert payload["old_fingerprint"] == "aaaa1111"
|
||||||
|
assert payload["new_fingerprint"] == "bbbb2222"
|
||||||
|
assert payload["table_count"] == 12
|
||||||
|
|
||||||
|
def test_rotation_rekey_failure_returns_503_and_leaves_keychain_unchanged(
|
||||||
|
self, _fake_encrypted_env, monkeypatch
|
||||||
|
):
|
||||||
|
from cyclone import db_crypto
|
||||||
|
from cyclone import db
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
def _fail_rotate(*, url, old_key, new_key):
|
||||||
|
return db_crypto.RotateKeyResult(
|
||||||
|
ok=False,
|
||||||
|
old_fingerprint=db_crypto.fingerprint(old_key),
|
||||||
|
new_fingerprint=db_crypto.fingerprint(new_key),
|
||||||
|
rotated_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
reason="simulated PRAGMA rekey failure",
|
||||||
|
)
|
||||||
|
monkeypatch.setattr("cyclone.api._db_crypto.rotate_db_key", _fail_rotate)
|
||||||
|
|
||||||
|
_, fake_kc = _fake_encrypted_env
|
||||||
|
before = dict(fake_kc)
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post("/api/admin/db/rotate-key")
|
||||||
|
assert r.status_code == 503
|
||||||
|
body = r.json()["detail"]
|
||||||
|
assert body["ok"] is False
|
||||||
|
assert "simulated" in body["reason"]
|
||||||
|
|
||||||
|
# Keychain wasn't touched.
|
||||||
|
assert fake_kc == before
|
||||||
|
|
||||||
|
# No audit event was written.
|
||||||
|
from cyclone.db import AuditLog
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
count = (
|
||||||
|
session.query(AuditLog)
|
||||||
|
.filter(AuditLog.event_type == "db.key_rotated")
|
||||||
|
.count()
|
||||||
|
)
|
||||||
|
assert count == 0
|
||||||
|
|
||||||
|
def test_503_when_keychain_write_fails_after_successful_rekey(
|
||||||
|
self, _fake_encrypted_env, monkeypatch
|
||||||
|
):
|
||||||
|
"""The rekey itself succeeded but the Keychain write failed.
|
||||||
|
The DB is now behind a new key the Keychain doesn't know about.
|
||||||
|
Endpoint must return 503 so the operator can run the manual
|
||||||
|
restore-key command."""
|
||||||
|
from cyclone import db
|
||||||
|
# Override the set_secret at the import-site of the endpoint.
|
||||||
|
monkeypatch.setattr("cyclone.api._secrets.set_secret", lambda n, v: False)
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post("/api/admin/db/rotate-key")
|
||||||
|
assert r.status_code == 503
|
||||||
|
body = r.json()["detail"]
|
||||||
|
assert body["ok"] is False
|
||||||
|
assert "keychain" in body["reason"].lower()
|
||||||
|
|
||||||
|
def test_409_when_concurrent_request(self, _fake_encrypted_env, monkeypatch):
|
||||||
|
"""A second concurrent rotation request gets 409 — only one
|
||||||
|
rotation can run at a time (the module-level lock)."""
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"cyclone.api._secrets.set_secret", lambda n, v: True,
|
||||||
|
)
|
||||||
|
from cyclone import api as api_mod
|
||||||
|
api_mod._db_rotate_lock.acquire()
|
||||||
|
try:
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post("/api/admin/db/rotate-key")
|
||||||
|
assert r.status_code == 409
|
||||||
|
assert "in progress" in r.json()["detail"]
|
||||||
|
finally:
|
||||||
|
api_mod._db_rotate_lock.release()
|
||||||
@@ -0,0 +1,152 @@
|
|||||||
|
"""SP16 — Admin scheduler API endpoint tests.
|
||||||
|
|
||||||
|
The endpoints under /api/admin/scheduler/* are thin wrappers around
|
||||||
|
:class:`cyclone.scheduler.Scheduler`. These tests exercise them via
|
||||||
|
the FastAPI TestClient to confirm wiring (auth-free admin endpoints
|
||||||
|
work, response shapes match, idempotency holds).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def _stub_scheduler_env(tmp_path, monkeypatch):
|
||||||
|
"""Set up: a stub-mode SFTP block, scheduler configured.
|
||||||
|
|
||||||
|
Yields (staging_dir, scheduler_singleton). We deliberately do
|
||||||
|
NOT enable SQLCipher encryption in this fixture — the scheduler
|
||||||
|
doesn't care about encryption, and patching ``db_crypto.get_secret``
|
||||||
|
here would cause the lifespan handler to rebuild the engine with
|
||||||
|
SQLCipher on a plain-SQLite test file (which raises "file is not
|
||||||
|
a database"). The encryption-at-rest tests live in
|
||||||
|
``test_db_crypto.py``.
|
||||||
|
"""
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone import scheduler as sched_mod
|
||||||
|
from cyclone.providers import SftpBlock
|
||||||
|
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
inbound = staging / "ToHPE"
|
||||||
|
inbound.mkdir(parents=True)
|
||||||
|
sftp_block = SftpBlock(
|
||||||
|
host="mft.example.com",
|
||||||
|
port=22,
|
||||||
|
username="test",
|
||||||
|
paths={"outbound": "/FromHPE", "inbound": "/ToHPE"},
|
||||||
|
stub=True,
|
||||||
|
staging_dir=str(staging),
|
||||||
|
poll_seconds=60,
|
||||||
|
auth={"method": "keychain", "secret_ref": "test.password"},
|
||||||
|
)
|
||||||
|
sched_mod.reset_scheduler_for_tests()
|
||||||
|
sched = sched_mod.configure_scheduler(sftp_block, sftp_block_name="t")
|
||||||
|
yield staging, sched
|
||||||
|
sched_mod.reset_scheduler_for_tests()
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
def _drop_file(staging: Path, name: str, body: bytes) -> Path:
|
||||||
|
p = staging / "ToHPE" / name
|
||||||
|
p.write_bytes(body)
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def test_scheduler_status_starts_not_running(_stub_scheduler_env):
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
_, sched = _stub_scheduler_env
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.get("/api/admin/scheduler/status")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["running"] is False
|
||||||
|
assert body["poll_interval_seconds"] == 60
|
||||||
|
assert body["sftp_block_name"] == "t"
|
||||||
|
|
||||||
|
|
||||||
|
def test_scheduler_start_then_status_then_stop(_stub_scheduler_env):
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r1 = client.post("/api/admin/scheduler/start")
|
||||||
|
assert r1.status_code == 200
|
||||||
|
assert r1.json()["status"]["running"] is True
|
||||||
|
|
||||||
|
r2 = client.get("/api/admin/scheduler/status")
|
||||||
|
assert r2.json()["running"] is True
|
||||||
|
|
||||||
|
r3 = client.post("/api/admin/scheduler/stop")
|
||||||
|
assert r3.status_code == 200
|
||||||
|
assert r3.json()["status"]["running"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_scheduler_tick_processes_one_file(_stub_scheduler_env):
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
staging, _ = _stub_scheduler_env
|
||||||
|
_drop_file(
|
||||||
|
staging,
|
||||||
|
"TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12",
|
||||||
|
(Path(__file__).parent / "fixtures" / "minimal_ta1.txt").read_bytes(),
|
||||||
|
)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post("/api/admin/scheduler/tick")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["tick"]["files_seen"] == 1
|
||||||
|
assert body["tick"]["files_processed"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_scheduler_processed_files_lists_history(_stub_scheduler_env):
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
staging, _ = _stub_scheduler_env
|
||||||
|
_drop_file(
|
||||||
|
staging,
|
||||||
|
"TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12",
|
||||||
|
(Path(__file__).parent / "fixtures" / "minimal_ta1.txt").read_bytes(),
|
||||||
|
)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
client.post("/api/admin/scheduler/tick")
|
||||||
|
r = client.get("/api/admin/scheduler/processed-files")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["count"] == 1
|
||||||
|
f = body["files"][0]
|
||||||
|
assert f["status"] == "ok"
|
||||||
|
assert f["parser_used"] == "parse_ta1"
|
||||||
|
assert "TP11525703" in f["name"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_scheduler_processed_files_filters_by_status(_stub_scheduler_env):
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
staging, _ = _stub_scheduler_env
|
||||||
|
# Drop a file with a type Cyclone doesn't parse — gets recorded as
|
||||||
|
# "skipped".
|
||||||
|
_drop_file(
|
||||||
|
staging,
|
||||||
|
"TP11525703-837P_M019048402-20260618130000000-1of1_270.x12",
|
||||||
|
b"some bytes",
|
||||||
|
)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
client.post("/api/admin/scheduler/tick")
|
||||||
|
r_all = client.get("/api/admin/scheduler/processed-files")
|
||||||
|
r_skipped = client.get(
|
||||||
|
"/api/admin/scheduler/processed-files?status=skipped",
|
||||||
|
)
|
||||||
|
r_ok = client.get(
|
||||||
|
"/api/admin/scheduler/processed-files?status=ok",
|
||||||
|
)
|
||||||
|
assert r_all.json()["count"] == 1
|
||||||
|
assert r_skipped.json()["count"] == 1
|
||||||
|
assert r_ok.json()["count"] == 0
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
"""Tests for ``GET /api/admin/validate-provider`` (SP20).
|
||||||
|
|
||||||
|
Pure read-only endpoint — runs the local NPI Luhn + EIN format checks.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone.api import app
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client() -> TestClient:
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Both fields populated
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_provider_both_valid(client: TestClient):
|
||||||
|
resp = client.get("/api/admin/validate-provider", params={
|
||||||
|
"npi": "1234567893", # CMS-published valid NPI
|
||||||
|
"tax_id": "72-1587149", # Touch of Care EIN
|
||||||
|
})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert body["npi"]["valid"] is True
|
||||||
|
assert body["npi"]["skipped"] is False
|
||||||
|
assert body["tax_id"]["valid"] is True
|
||||||
|
assert body["tax_id"]["normalized"] == "721587149"
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_provider_both_invalid(client: TestClient):
|
||||||
|
resp = client.get("/api/admin/validate-provider", params={
|
||||||
|
"npi": "1234567890", # format OK but Luhn fails
|
||||||
|
"tax_id": "00-1234567", # reserved prefix
|
||||||
|
})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert body["npi"]["valid"] is False
|
||||||
|
assert body["tax_id"]["valid"] is False
|
||||||
|
assert body["tax_id"]["normalized"] == "001234567"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Param omission → skipped
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_provider_skips_missing_npi(client: TestClient):
|
||||||
|
resp = client.get("/api/admin/validate-provider", params={"tax_id": "721587149"})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert body["npi"]["skipped"] is True
|
||||||
|
assert body["npi"]["valid"] is None
|
||||||
|
assert body["tax_id"]["valid"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_provider_skips_missing_tax_id(client: TestClient):
|
||||||
|
resp = client.get("/api/admin/validate-provider", params={"npi": "1234567893"})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert body["tax_id"]["skipped"] is True
|
||||||
|
assert body["tax_id"]["valid"] is None
|
||||||
|
assert body["tax_id"]["normalized"] is None
|
||||||
|
assert body["npi"]["valid"] is True
|
||||||
@@ -0,0 +1,247 @@
|
|||||||
|
"""Tests for :func:`cyclone.inbox_state_277ca.apply_277ca_rejections`.
|
||||||
|
|
||||||
|
SP10 T2. The 277CA's STC A4/A6/A7 codes stamp payer-rejection fields
|
||||||
|
on matching claim rows. Distinct from the 999 envelope rejection.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.db import Claim, init_db
|
||||||
|
from cyclone.inbox_state_277ca import apply_277ca_rejections
|
||||||
|
from cyclone.parsers.models_277ca import ClaimStatus, ParseResult277CA
|
||||||
|
from cyclone.parsers.parse_277ca import parse_277ca_text
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Fixtures
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _fresh_db():
|
||||||
|
"""Each test gets a fresh in-memory DB."""
|
||||||
|
init_db()
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
def _make_claim(session, *, claim_id: str, pcn: str = "CLAIM001") -> Claim:
|
||||||
|
c = Claim(
|
||||||
|
id=claim_id,
|
||||||
|
batch_id="BATCH-1",
|
||||||
|
patient_control_number=pcn,
|
||||||
|
charge_amount=100.00,
|
||||||
|
)
|
||||||
|
session.add(c)
|
||||||
|
session.commit()
|
||||||
|
session.refresh(c)
|
||||||
|
return c
|
||||||
|
|
||||||
|
|
||||||
|
def _make_rejected_status(pcn: str | None = "CLAIM001") -> ClaimStatus:
|
||||||
|
return ClaimStatus(
|
||||||
|
status_code="A6",
|
||||||
|
status_description="19",
|
||||||
|
entity_identifier="PR",
|
||||||
|
classification="rejected",
|
||||||
|
payer_claim_control_number=pcn,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Tests
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestApply277CARejectionsHappyPath:
|
||||||
|
def test_rejected_status_stamps_matching_claim(self):
|
||||||
|
from cyclone import db
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
claim = _make_claim(s, claim_id="c1")
|
||||||
|
pcn = claim.patient_control_number
|
||||||
|
result = parse_277ca_text(_minimal_277ca_one_rejected())
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
def _lookup(pcn_q):
|
||||||
|
return s.query(Claim).filter_by(patient_control_number=pcn_q).first()
|
||||||
|
outcome = apply_277ca_rejections(s, result, claim_lookup=_lookup, two77ca_id="ACK-1")
|
||||||
|
assert outcome.matched == ["c1"]
|
||||||
|
assert outcome.orphans == []
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
c = s.get(Claim, "c1")
|
||||||
|
assert c.payer_rejected_at is not None
|
||||||
|
assert c.payer_rejected_status_code == "A6"
|
||||||
|
assert "A6" in (c.payer_rejected_reason or "")
|
||||||
|
assert c.payer_rejected_by_277ca_id == "ACK-1"
|
||||||
|
|
||||||
|
|
||||||
|
class TestApply277CARejectionsOrphans:
|
||||||
|
def test_unknown_pcn_becomes_orphan(self):
|
||||||
|
from cyclone import db
|
||||||
|
# No claim exists — PCN won't match.
|
||||||
|
text = _minimal_277ca_one_rejected()
|
||||||
|
result = parse_277ca_text(text)
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
def _lookup(_):
|
||||||
|
return None
|
||||||
|
outcome = apply_277ca_rejections(s, result, claim_lookup=_lookup, two77ca_id="ACK-1")
|
||||||
|
assert outcome.matched == []
|
||||||
|
assert outcome.orphans == ["CLAIM001"]
|
||||||
|
|
||||||
|
def test_status_without_ref_1k_becomes_orphan(self):
|
||||||
|
"""A rejected STC with no REF*1K cannot match a claim."""
|
||||||
|
from cyclone import db
|
||||||
|
text = _minimal_277ca_no_ref1k()
|
||||||
|
result = parse_277ca_text(text)
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
def _lookup(_):
|
||||||
|
return None
|
||||||
|
outcome = apply_277ca_rejections(s, result, claim_lookup=_lookup, two77ca_id="ACK-1")
|
||||||
|
assert outcome.matched == []
|
||||||
|
# The orphan entry uses the status code (since PCN is missing).
|
||||||
|
assert outcome.orphans == ["A6"]
|
||||||
|
|
||||||
|
|
||||||
|
class TestApply277CARejectionsIdempotent:
|
||||||
|
def test_already_stamped_is_not_overwritten(self):
|
||||||
|
from cyclone import db
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_claim(s, claim_id="c1")
|
||||||
|
text = _minimal_277ca_one_rejected()
|
||||||
|
result = parse_277ca_text(text)
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
def _lookup(_):
|
||||||
|
return s.query(Claim).filter_by(patient_control_number="CLAIM001").first()
|
||||||
|
outcome1 = apply_277ca_rejections(s, result, claim_lookup=_lookup, two77ca_id="ACK-1")
|
||||||
|
assert outcome1.matched == ["c1"]
|
||||||
|
original_reason = s.get(Claim, "c1").payer_rejected_reason
|
||||||
|
original_at = s.get(Claim, "c1").payer_rejected_at
|
||||||
|
# Run again with same code.
|
||||||
|
outcome2 = apply_277ca_rejections(s, result, claim_lookup=_lookup, two77ca_id="ACK-1")
|
||||||
|
assert outcome2.matched == []
|
||||||
|
assert outcome2.already_rejected == ["c1"]
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
c = s.get(Claim, "c1")
|
||||||
|
# Reason and timestamp unchanged.
|
||||||
|
assert c.payer_rejected_reason == original_reason
|
||||||
|
assert c.payer_rejected_at == original_at
|
||||||
|
|
||||||
|
|
||||||
|
class TestApply277CAOnlyRejectsRejected:
|
||||||
|
def test_accepted_status_does_not_stamp(self):
|
||||||
|
"""An A3 (accepted) status must NOT trigger a payer_rejected stamp."""
|
||||||
|
from cyclone import db
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_claim(s, claim_id="c1")
|
||||||
|
text = _minimal_277ca_one_accepted()
|
||||||
|
result = parse_277ca_text(text)
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
def _lookup(_):
|
||||||
|
return s.query(Claim).filter_by(patient_control_number="CLAIM001").first()
|
||||||
|
outcome = apply_277ca_rejections(s, result, claim_lookup=_lookup, two77ca_id="ACK-1")
|
||||||
|
assert outcome.matched == []
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
c = s.get(Claim, "c1")
|
||||||
|
assert c.payer_rejected_at is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestApply277CAMultipleStatuses:
|
||||||
|
def test_mixed_batch_only_stamps_rejected(self):
|
||||||
|
"""Of three statuses (A3/A6/A8), only the A6 claim gets stamped."""
|
||||||
|
from cyclone import db
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_claim(s, claim_id="c1", pcn="CLAIM001")
|
||||||
|
_make_claim(s, claim_id="c2", pcn="CLAIM002")
|
||||||
|
_make_claim(s, claim_id="c3", pcn="CLAIM003")
|
||||||
|
text = (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X214~"
|
||||||
|
"ST*277CA*0001*005010X214~"
|
||||||
|
"BHT*0085*08*X*20240620*1200*TH~"
|
||||||
|
"HL*1**20*1~"
|
||||||
|
"HL*2*1*21*1~"
|
||||||
|
"HL*3*2*19*1~"
|
||||||
|
"HL*4*3*PT~"
|
||||||
|
"REF*1K*CLAIM001~"
|
||||||
|
"STC*A3:19:PR*20240620*WQ*100.00~"
|
||||||
|
"HL*5*3*PT~"
|
||||||
|
"REF*1K*CLAIM002~"
|
||||||
|
"STC*A6:19:PR*20240620*U*250.00~"
|
||||||
|
"HL*6*3*PT~"
|
||||||
|
"REF*1K*CLAIM003~"
|
||||||
|
"STC*A8:19:PR*20240620*U*175.00~"
|
||||||
|
"SE*16*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
|
result = parse_277ca_text(text)
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
def _lookup(pcn):
|
||||||
|
return s.query(Claim).filter_by(patient_control_number=pcn).first()
|
||||||
|
outcome = apply_277ca_rejections(s, result, claim_lookup=_lookup, two77ca_id="ACK-1")
|
||||||
|
assert outcome.matched == ["c2"]
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
assert s.get(Claim, "c1").payer_rejected_at is None
|
||||||
|
assert s.get(Claim, "c2").payer_rejected_at is not None
|
||||||
|
assert s.get(Claim, "c3").payer_rejected_at is None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Test fixtures
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def _minimal_277ca_one_rejected() -> str:
|
||||||
|
return (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X214~"
|
||||||
|
"ST*277CA*0001*005010X214~"
|
||||||
|
"BHT*0085*08*X*20240620*1200*TH~"
|
||||||
|
"HL*1**20*1~"
|
||||||
|
"HL*2*1*21*1~"
|
||||||
|
"HL*3*2*19*1~"
|
||||||
|
"HL*4*3*PT~"
|
||||||
|
"REF*1K*CLAIM001~"
|
||||||
|
"STC*A6:19:PR*20240620*U*100.00~"
|
||||||
|
"SE*9*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _minimal_277ca_one_accepted() -> str:
|
||||||
|
return (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X214~"
|
||||||
|
"ST*277CA*0001*005010X214~"
|
||||||
|
"BHT*0085*08*X*20240620*1200*TH~"
|
||||||
|
"HL*1**20*1~"
|
||||||
|
"HL*2*1*21*1~"
|
||||||
|
"HL*3*2*19*1~"
|
||||||
|
"HL*4*3*PT~"
|
||||||
|
"REF*1K*CLAIM001~"
|
||||||
|
"STC*A3:19:PR*20240620*WQ*100.00~"
|
||||||
|
"SE*9*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _minimal_277ca_no_ref1k() -> str:
|
||||||
|
"""Patient HL with STC A6 but no REF*1K."""
|
||||||
|
return (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X214~"
|
||||||
|
"ST*277CA*0001*005010X214~"
|
||||||
|
"BHT*0085*08*X*20240620*1200*TH~"
|
||||||
|
"HL*1**20*1~"
|
||||||
|
"HL*2*1*21*1~"
|
||||||
|
"HL*3*2*19*1~"
|
||||||
|
"HL*4*3*PT~"
|
||||||
|
"STC*A6:19:PR*20240620*U*100.00~"
|
||||||
|
"SE*8*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
@@ -0,0 +1,187 @@
|
|||||||
|
"""Tests for the tamper-evident hash-chained audit_log.
|
||||||
|
|
||||||
|
SP11 T1.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
import json
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.audit_log import (
|
||||||
|
GENESIS_PREV_HASH,
|
||||||
|
HASH_LEN,
|
||||||
|
AuditEvent,
|
||||||
|
append_event,
|
||||||
|
verify_chain,
|
||||||
|
)
|
||||||
|
from cyclone.db import AuditLog, init_db
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _fresh_db():
|
||||||
|
init_db()
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
def _row_count() -> int:
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
return s.query(AuditLog).count()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Append
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestAppendEvent:
|
||||||
|
def test_first_row_has_genesis_prev_hash(self):
|
||||||
|
"""The first row in a fresh chain uses the all-zeros prev_hash."""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = append_event(s, AuditEvent(
|
||||||
|
event_type="claim.parsed",
|
||||||
|
entity_type="claim",
|
||||||
|
entity_id="c1",
|
||||||
|
payload={"patient_control_number": "PCN-1"},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
assert row.id == 1
|
||||||
|
assert row.prev_hash == GENESIS_PREV_HASH
|
||||||
|
assert len(row.hash) == HASH_LEN
|
||||||
|
|
||||||
|
def test_second_row_chains_to_first(self):
|
||||||
|
"""Row N's prev_hash == row N-1's hash."""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
r1 = append_event(s, AuditEvent(
|
||||||
|
event_type="claim.parsed", entity_type="claim", entity_id="c1",
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
r2 = append_event(s, AuditEvent(
|
||||||
|
event_type="claim.rejected", entity_type="claim", entity_id="c1",
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
assert r2.prev_hash == r1.hash
|
||||||
|
assert r2.id == r1.id + 1
|
||||||
|
|
||||||
|
def test_payload_canonicalization(self):
|
||||||
|
"""Two payloads with the same content but different key order hash the same.
|
||||||
|
|
||||||
|
We rely on sort_keys=True to make the canonical form independent
|
||||||
|
of dict insertion order.
|
||||||
|
"""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
r1 = append_event(s, AuditEvent(
|
||||||
|
event_type="x", entity_type="y", entity_id="z",
|
||||||
|
payload={"a": 1, "b": 2},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
# New session to drop any in-memory ordering cache.
|
||||||
|
with db.SessionLocal()() as s2:
|
||||||
|
r2 = append_event(s2, AuditEvent(
|
||||||
|
event_type="x", entity_type="y", entity_id="z",
|
||||||
|
payload={"b": 2, "a": 1},
|
||||||
|
))
|
||||||
|
s2.commit()
|
||||||
|
# The two rows have different IDs and different prev_hash
|
||||||
|
# inputs, but their hash *recipe* (modulo id + prev_hash) is
|
||||||
|
# the same. Just confirm the rows are different.
|
||||||
|
assert r1.id != r2.id
|
||||||
|
# The payload_json fields ARE byte-identical (canonical form).
|
||||||
|
assert r1.payload_json == r2.payload_json
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Verify
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestVerifyChain:
|
||||||
|
def test_empty_chain_is_ok(self):
|
||||||
|
"""No rows = nothing to verify = ok."""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
result = verify_chain(s)
|
||||||
|
assert result.ok is True
|
||||||
|
assert result.checked == 0
|
||||||
|
|
||||||
|
def test_single_row_chain_is_ok(self):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
append_event(s, AuditEvent("x", "y", "z"))
|
||||||
|
s.commit()
|
||||||
|
result = verify_chain(s)
|
||||||
|
assert result.ok is True
|
||||||
|
assert result.checked == 1
|
||||||
|
|
||||||
|
def test_long_chain_is_ok(self):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
for i in range(50):
|
||||||
|
append_event(s, AuditEvent("x", "y", f"id-{i}"))
|
||||||
|
s.commit()
|
||||||
|
result = verify_chain(s)
|
||||||
|
assert result.ok is True
|
||||||
|
assert result.checked == 50
|
||||||
|
|
||||||
|
def test_tampered_payload_detected(self):
|
||||||
|
"""Modifying a row's payload_json breaks the chain at that row."""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
for i in range(5):
|
||||||
|
append_event(s, AuditEvent("x", "y", f"id-{i}"))
|
||||||
|
s.commit()
|
||||||
|
# Tamper with row #3's payload (the first 3 rows are still
|
||||||
|
# valid; row 3 will fail).
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.query(AuditLog).filter(AuditLog.id == 3).first()
|
||||||
|
row.payload_json = json.dumps({"evil": "tampered"})
|
||||||
|
s.commit()
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
result = verify_chain(s)
|
||||||
|
assert result.ok is False
|
||||||
|
assert result.first_bad_id == 3
|
||||||
|
assert "hash" in (result.reason or "").lower()
|
||||||
|
|
||||||
|
def test_tampered_prev_hash_detected(self):
|
||||||
|
"""Modifying a row's prev_hash invalidates that row's own hash.
|
||||||
|
|
||||||
|
Because the row's hash is computed from prev_hash (and the
|
||||||
|
other fields), changing prev_hash changes the row's own hash,
|
||||||
|
so the verifier detects the breakage at the tampered row
|
||||||
|
itself (not the row after). The next-row check (which compares
|
||||||
|
prev_hash to the previous row's hash) would catch a different
|
||||||
|
class of tampering: a row whose prev_hash was set to a value
|
||||||
|
matching the previous row's hash but whose own hash was also
|
||||||
|
regenerated — but that requires recomputing the hash to match,
|
||||||
|
which is what verify_chain would not detect by itself (the
|
||||||
|
content-vs-hash check still catches content tampering).
|
||||||
|
"""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
for i in range(5):
|
||||||
|
append_event(s, AuditEvent("x", "y", f"id-{i}"))
|
||||||
|
s.commit()
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.query(AuditLog).filter(AuditLog.id == 2).first()
|
||||||
|
row.prev_hash = "f" * 64 # fake
|
||||||
|
s.commit()
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
result = verify_chain(s)
|
||||||
|
assert result.ok is False
|
||||||
|
# The tampered row itself fails its hash check first.
|
||||||
|
assert result.first_bad_id == 2
|
||||||
|
assert "hash" in (result.reason or "").lower()
|
||||||
|
|
||||||
|
def test_deleted_row_detected(self):
|
||||||
|
"""Deleting a middle row breaks the chain at the row after it."""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
for i in range(5):
|
||||||
|
append_event(s, AuditEvent("x", "y", f"id-{i}"))
|
||||||
|
s.commit()
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.query(AuditLog).filter(AuditLog.id == 3).first()
|
||||||
|
s.delete(row)
|
||||||
|
s.commit()
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
result = verify_chain(s)
|
||||||
|
assert result.ok is False
|
||||||
|
# Row 3 is gone; row 4's prev_hash now points at row 2's hash,
|
||||||
|
# which doesn't match row 4's stored prev_hash.
|
||||||
|
assert result.first_bad_id == 4
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
"""SP17 — low-level backup crypto tests.
|
||||||
|
|
||||||
|
Pure-Python, no DB. Covers key derivation determinism, encrypt /
|
||||||
|
decrypt round-trip, tampered-ciphertext failure, wrong-passphrase
|
||||||
|
failure, and the sidecar JSON format.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import backup as backup_mod
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Key derivation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_derive_key_is_deterministic():
|
||||||
|
salt = os.urandom(16)
|
||||||
|
k1 = backup_mod.derive_key("correct horse battery staple", salt)
|
||||||
|
k2 = backup_mod.derive_key("correct horse battery staple", salt)
|
||||||
|
assert k1 == k2
|
||||||
|
assert len(k1) == backup_mod.KEY_LEN == 32
|
||||||
|
|
||||||
|
|
||||||
|
def test_derive_key_different_salts_produce_different_keys():
|
||||||
|
"""Salt is what makes the same passphrase produce different keys."""
|
||||||
|
k1 = backup_mod.derive_key("hunter2", os.urandom(16))
|
||||||
|
k2 = backup_mod.derive_key("hunter2", os.urandom(16))
|
||||||
|
assert k1 != k2
|
||||||
|
|
||||||
|
|
||||||
|
def test_derive_key_different_passphrases_produce_different_keys():
|
||||||
|
salt = os.urandom(16)
|
||||||
|
k1 = backup_mod.derive_key("a", salt)
|
||||||
|
k2 = backup_mod.derive_key("b", salt)
|
||||||
|
assert k1 != k2
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Encrypt / decrypt round-trip
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_encrypt_decrypt_roundtrip():
|
||||||
|
key = os.urandom(32)
|
||||||
|
plaintext = b"hello cyclone backup " * 1000
|
||||||
|
blob = backup_mod.encrypt(plaintext, key)
|
||||||
|
assert len(blob) == backup_mod.NONCE_LEN + len(plaintext) + 16 # tag
|
||||||
|
out = backup_mod.decrypt(blob, key)
|
||||||
|
assert out == plaintext
|
||||||
|
|
||||||
|
|
||||||
|
def test_encrypt_decrypt_empty_plaintext():
|
||||||
|
"""Edge case: zero-byte payload still produces nonce + tag."""
|
||||||
|
key = os.urandom(32)
|
||||||
|
blob = backup_mod.encrypt(b"", key)
|
||||||
|
out = backup_mod.decrypt(blob, key)
|
||||||
|
assert out == b""
|
||||||
|
|
||||||
|
|
||||||
|
def test_decrypt_with_wrong_key_raises():
|
||||||
|
plaintext = b"some bytes"
|
||||||
|
key1 = os.urandom(32)
|
||||||
|
key2 = os.urandom(32)
|
||||||
|
blob = backup_mod.encrypt(plaintext, key1)
|
||||||
|
with pytest.raises(backup_mod.BackupDecryptError):
|
||||||
|
backup_mod.decrypt(blob, key2)
|
||||||
|
|
||||||
|
|
||||||
|
def test_decrypt_tampered_ciphertext_raises():
|
||||||
|
"""Flipping a single ciphertext byte must fail GCM auth."""
|
||||||
|
key = os.urandom(32)
|
||||||
|
blob = backup_mod.encrypt(b"a" * 200, key)
|
||||||
|
tampered = bytearray(blob)
|
||||||
|
# Flip a bit somewhere in the ciphertext region (past the nonce).
|
||||||
|
tampered[backup_mod.NONCE_LEN + 5] ^= 0x01
|
||||||
|
with pytest.raises(backup_mod.BackupDecryptError):
|
||||||
|
backup_mod.decrypt(bytes(tampered), key)
|
||||||
|
|
||||||
|
|
||||||
|
def test_decrypt_truncated_blob_raises():
|
||||||
|
key = os.urandom(32)
|
||||||
|
blob = backup_mod.encrypt(b"x" * 100, key)
|
||||||
|
with pytest.raises(backup_mod.BackupDecryptError):
|
||||||
|
# Strip the GCM tag.
|
||||||
|
backup_mod.decrypt(blob[: -16], key)
|
||||||
|
|
||||||
|
|
||||||
|
def test_encrypt_with_wrong_key_length_raises():
|
||||||
|
with pytest.raises(backup_mod.BackupError):
|
||||||
|
backup_mod.encrypt(b"data", b"short") # not 32 bytes
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Fingerprint
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_fingerprint_format_and_stability():
|
||||||
|
fp = backup_mod.fingerprint(b"hello")
|
||||||
|
assert fp.startswith("sha256:")
|
||||||
|
assert len(fp) == len("sha256:") + 64
|
||||||
|
assert fp == backup_mod.fingerprint(b"hello")
|
||||||
|
assert fp != backup_mod.fingerprint(b"hellp")
|
||||||
|
|
||||||
|
|
||||||
|
def test_fingerprint_file_matches_fingerprint_bytes(tmp_path):
|
||||||
|
p = tmp_path / "data.bin"
|
||||||
|
p.write_bytes(b"\x00\x01\x02" * 100)
|
||||||
|
assert backup_mod.fingerprint_file(p) == backup_mod.fingerprint(p.read_bytes())
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Sidecar
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_sidecar_round_trip_json():
|
||||||
|
sc = backup_mod.Sidecar(
|
||||||
|
format_version="v1",
|
||||||
|
created_at="2026-06-21T15:30:00+00:00",
|
||||||
|
db_fingerprint="sha256:" + "a" * 64,
|
||||||
|
table_count=11,
|
||||||
|
size_bytes=1024,
|
||||||
|
kdf="PBKDF2-HMAC-SHA256",
|
||||||
|
kdf_iterations=200_000,
|
||||||
|
cipher="AES-256-GCM",
|
||||||
|
key_fingerprint="sha256:" + "b" * 64,
|
||||||
|
)
|
||||||
|
text = sc.to_json()
|
||||||
|
parsed = json.loads(text)
|
||||||
|
assert parsed["format_version"] == "v1"
|
||||||
|
assert parsed["encryption"]["kdf_iterations"] == 200_000
|
||||||
|
sc2 = backup_mod.Sidecar.from_json(text)
|
||||||
|
assert sc2 == sc
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Filenames
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_filename_format():
|
||||||
|
"""The timestamp prefix is fixed; the suffix is random per call."""
|
||||||
|
import re
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
ts = datetime(2026, 6, 21, 15, 30, 0, tzinfo=timezone.utc)
|
||||||
|
name = backup_mod.backup_filename(ts)
|
||||||
|
assert re.match(r"^cyclone-backup-20260621T153000Z-[0-9a-f]{8}\.bin$", name), name
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_filename_random_suffix_avoids_collisions():
|
||||||
|
"""Two calls in the same second get different filenames."""
|
||||||
|
a = backup_mod.backup_filename()
|
||||||
|
b = backup_mod.backup_filename()
|
||||||
|
assert a != b
|
||||||
|
|
||||||
|
|
||||||
|
def test_sidecar_filename_appends_meta_json():
|
||||||
|
assert backup_mod.sidecar_filename("foo.bin") == "foo.bin.meta.json"
|
||||||
@@ -0,0 +1,207 @@
|
|||||||
|
"""SP17 — BackupScheduler unit tests.
|
||||||
|
|
||||||
|
Exercises the asyncio tick / start / stop loop without spinning up
|
||||||
|
the FastAPI app. The scheduler wraps a real BackupService against
|
||||||
|
a real on-disk sqlite DB.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import backup_scheduler as sched_mod
|
||||||
|
from cyclone import db
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def fresh_db(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
from cyclone.db import Batch
|
||||||
|
import uuid
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
s.add(Batch(
|
||||||
|
id=str(uuid.uuid4()),
|
||||||
|
kind="837P",
|
||||||
|
input_filename="seed.x12",
|
||||||
|
parsed_at=datetime.now(timezone.utc),
|
||||||
|
totals_json=None,
|
||||||
|
validation_json=None,
|
||||||
|
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
yield
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def backup_svc(tmp_path):
|
||||||
|
return svc_mod.BackupService(
|
||||||
|
backup_dir=tmp_path / "backups",
|
||||||
|
passphrase="test-pass",
|
||||||
|
retention_days=7,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# tick
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tick_creates_backup_and_audits_it(fresh_db, backup_svc):
|
||||||
|
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||||
|
result = await sched.tick()
|
||||||
|
assert result.ok
|
||||||
|
assert result.created is not None
|
||||||
|
assert result.error is None
|
||||||
|
assert len(backup_svc.list_backups()) == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tick_creates_audit_event(fresh_db, backup_svc):
|
||||||
|
"""db.backup_created audit event is written (SP11 hash chain)."""
|
||||||
|
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||||
|
await sched.tick()
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
from cyclone.db import AuditLog
|
||||||
|
rows = (
|
||||||
|
s.query(AuditLog)
|
||||||
|
.filter(AuditLog.event_type == "db.backup_created")
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
assert len(rows) == 1
|
||||||
|
assert "backup_id" in rows[0].payload_json
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tick_handles_create_failure_without_crashing(fresh_db, backup_svc, monkeypatch):
|
||||||
|
"""If create_now raises, tick records the error and continues."""
|
||||||
|
def boom():
|
||||||
|
raise RuntimeError("simulated failure")
|
||||||
|
monkeypatch.setattr(backup_svc, "create_now", boom)
|
||||||
|
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||||
|
result = await sched.tick()
|
||||||
|
assert result.error is not None
|
||||||
|
assert "simulated failure" in result.error
|
||||||
|
# Audit event written for the failure.
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
from cyclone.db import AuditLog
|
||||||
|
rows = (
|
||||||
|
s.query(AuditLog)
|
||||||
|
.filter(AuditLog.event_type == "db.backup_failed")
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
assert len(rows) == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tick_prunes_old_backups_and_audits(fresh_db, backup_svc):
|
||||||
|
"""A tick prunes backups past retention and writes a db.backup_pruned event."""
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
|
||||||
|
# Take an initial backup.
|
||||||
|
initial = backup_svc.create_now()
|
||||||
|
# Age it past retention.
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
row = s.get(DbBackup, initial.backup.id)
|
||||||
|
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||||
|
result = await sched.tick()
|
||||||
|
assert result.ok # create_now succeeded even though prune removed old
|
||||||
|
assert len(result.pruned_paths) == 2 # .bin + .meta.json
|
||||||
|
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
from cyclone.db import AuditLog
|
||||||
|
pruned_events = (
|
||||||
|
s.query(AuditLog)
|
||||||
|
.filter(AuditLog.event_type == "db.backup_pruned")
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
assert len(pruned_events) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# start / stop / is_running
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_start_then_stop(fresh_db, backup_svc):
|
||||||
|
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||||
|
assert not sched.is_running()
|
||||||
|
await sched.start()
|
||||||
|
assert sched.is_running()
|
||||||
|
# Don't wait for the staggered first tick; just stop.
|
||||||
|
await sched.stop()
|
||||||
|
assert not sched.is_running()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_double_start_is_idempotent(fresh_db, backup_svc):
|
||||||
|
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||||
|
await sched.start()
|
||||||
|
await sched.start() # no-op
|
||||||
|
assert sched.is_running()
|
||||||
|
await sched.stop()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_concurrent_ticks_are_coalesced(fresh_db, backup_svc):
|
||||||
|
"""Two tick() calls in flight — second waits for first."""
|
||||||
|
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||||
|
r1, r2 = await asyncio.gather(sched.tick(), sched.tick())
|
||||||
|
# Both should succeed and produce a single backup (the second
|
||||||
|
# call returned the first call's result, or ran back-to-back
|
||||||
|
# and produced a second backup — both are valid coalescings).
|
||||||
|
assert r1 is not None
|
||||||
|
assert r2 is not None
|
||||||
|
# No matter the order, exactly 1 backup should exist OR 2 if they
|
||||||
|
# ran sequentially. The point of coalescing is no-overlap, so
|
||||||
|
# both should be ok=True.
|
||||||
|
assert r1.ok
|
||||||
|
assert r2.ok
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# status
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_status_snapshot(fresh_db, backup_svc):
|
||||||
|
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=12.0)
|
||||||
|
snap = sched.status()
|
||||||
|
assert snap.running is False
|
||||||
|
assert snap.interval_hours == 12.0
|
||||||
|
assert snap.backup_dir == str(backup_svc.backup_dir)
|
||||||
|
assert snap.retention_days == 7
|
||||||
|
assert snap.tick_count == 0
|
||||||
|
assert snap.last_tick is None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Module-level singleton
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_module_singleton_round_trip(fresh_db, tmp_path):
|
||||||
|
sched_mod.reset_backup_scheduler_for_tests()
|
||||||
|
svc = svc_mod.BackupService(tmp_path / "b", passphrase="x", retention_days=1)
|
||||||
|
sched = sched_mod.configure_backup_scheduler(svc, interval_hours=1)
|
||||||
|
assert sched_mod.get_backup_scheduler() is sched
|
||||||
|
# Second configure is a no-op.
|
||||||
|
assert sched_mod.configure_backup_scheduler(svc) is sched
|
||||||
|
sched_mod.reset_backup_scheduler_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
def test_module_singleton_get_raises_when_unset():
|
||||||
|
sched_mod.reset_backup_scheduler_for_tests()
|
||||||
|
with pytest.raises(RuntimeError):
|
||||||
|
sched_mod.get_backup_scheduler()
|
||||||
@@ -0,0 +1,400 @@
|
|||||||
|
"""SP17 — BackupService integration tests.
|
||||||
|
|
||||||
|
Exercises the full create / list / verify / restore / prune flow
|
||||||
|
against a real on-disk SQLite file (no SQLCipher, no Keychain). We
|
||||||
|
inject the passphrase directly into the BackupService constructor.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import backup as backup_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.backup import BackupError
|
||||||
|
from cyclone.backup_service import (
|
||||||
|
BackupService,
|
||||||
|
STATUS_ERROR,
|
||||||
|
STATUS_OK,
|
||||||
|
STATUS_PENDING,
|
||||||
|
STATUS_PRUNED,
|
||||||
|
configure_backup_service,
|
||||||
|
get_backup_service,
|
||||||
|
reset_backup_service_for_tests,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Fixtures
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def fresh_db(tmp_path, monkeypatch):
|
||||||
|
"""Fresh sqlite DB; init_db + create tables; yield the path."""
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
yield tmp_path / "test.db"
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def backup_svc(fresh_db, tmp_path):
|
||||||
|
"""A BackupService rooted in a temp backup directory."""
|
||||||
|
backup_dir = tmp_path / "backups"
|
||||||
|
return BackupService(
|
||||||
|
backup_dir=backup_dir,
|
||||||
|
passphrase="test-passphrase-123",
|
||||||
|
retention_days=7,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _make_a_row(s: "sa.orm.Session") -> None:
|
||||||
|
"""Insert one minimal Batch row so the DB has a real schema + content.
|
||||||
|
|
||||||
|
Bypasses the Claim model (which has many NOT NULL columns tied to
|
||||||
|
BatchRecord lifecycle) and just writes a Batch directly — the
|
||||||
|
backup flow doesn't care which tables exist, only that there
|
||||||
|
are some.
|
||||||
|
"""
|
||||||
|
from cyclone.db import Batch
|
||||||
|
import uuid
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from decimal import Decimal
|
||||||
|
s.add(Batch(
|
||||||
|
id=str(uuid.uuid4()),
|
||||||
|
kind="837P",
|
||||||
|
input_filename="test.x12",
|
||||||
|
parsed_at=datetime.now(timezone.utc),
|
||||||
|
totals_json=None,
|
||||||
|
validation_json=None,
|
||||||
|
raw_result_json={
|
||||||
|
"envelope": {"control_number": "1"},
|
||||||
|
"claims": [],
|
||||||
|
"summary": {"passed": 0, "failed": 0, "failed_claim_ids": []},
|
||||||
|
},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# create_now
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_create_now_writes_encrypted_blob_and_sidecar(fresh_db, backup_svc):
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
# Add a claim so the DB has content + table_count > 0.
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
|
||||||
|
result = backup_svc.create_now()
|
||||||
|
record = result.backup
|
||||||
|
sidecar = result.sidecar
|
||||||
|
|
||||||
|
assert record.status == STATUS_OK
|
||||||
|
assert record.size_bytes > 0
|
||||||
|
assert record.db_fingerprint.startswith("sha256:")
|
||||||
|
assert record.table_count >= 1
|
||||||
|
assert record.completed_at is not None
|
||||||
|
|
||||||
|
# The .bin file exists, is non-trivial size, and does NOT look
|
||||||
|
# like a SQLite header (which is the whole point of encryption).
|
||||||
|
bin_path = backup_svc.backup_dir / record.filename
|
||||||
|
assert bin_path.exists()
|
||||||
|
blob = bin_path.read_bytes()
|
||||||
|
assert blob[:6] != b"SQLite" # not a plaintext SQLite file
|
||||||
|
|
||||||
|
# Sidecar exists and round-trips.
|
||||||
|
meta_path = backup_svc.backup_dir / backup_mod.sidecar_filename(record.filename)
|
||||||
|
assert meta_path.exists()
|
||||||
|
parsed = backup_mod.Sidecar.from_json(meta_path.read_text())
|
||||||
|
assert parsed.db_fingerprint == record.db_fingerprint
|
||||||
|
assert parsed.table_count == record.table_count
|
||||||
|
|
||||||
|
|
||||||
|
def test_create_now_marks_error_on_db_failure(fresh_db, tmp_path, monkeypatch):
|
||||||
|
"""If SQLite .backup() raises, the row is marked error + files cleaned."""
|
||||||
|
backup_dir = tmp_path / "backups"
|
||||||
|
svc = BackupService(backup_dir=backup_dir, passphrase="x", retention_days=7)
|
||||||
|
|
||||||
|
# Force the .backup() call to fail by patching sqlite3.connect to raise.
|
||||||
|
import sqlite3 as _sqlite3
|
||||||
|
real_connect = _sqlite3.connect
|
||||||
|
def boom(path):
|
||||||
|
raise RuntimeError("simulated disk failure")
|
||||||
|
monkeypatch.setattr(_sqlite3, "connect", boom)
|
||||||
|
# But we also need to make sure engine.raw_connection().driver_connection
|
||||||
|
# is reachable — it's still using real_connect via the engine's
|
||||||
|
# internals. So patch at the higher level: the BackupService's
|
||||||
|
# _sqlite_backup_to.
|
||||||
|
monkeypatch.setattr(svc, "_sqlite_backup_to",
|
||||||
|
lambda p: (_ for _ in ()).throw(RuntimeError("boom")))
|
||||||
|
|
||||||
|
with pytest.raises(RuntimeError, match="boom"):
|
||||||
|
svc.create_now()
|
||||||
|
|
||||||
|
rows = svc.list_backups()
|
||||||
|
assert len(rows) == 1
|
||||||
|
assert rows[0].status == STATUS_ERROR
|
||||||
|
assert "boom" in rows[0].error_message
|
||||||
|
# No files left in the backup dir.
|
||||||
|
assert list(backup_dir.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# list_backups
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_backups_orders_newest_first(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
r1 = backup_svc.create_now()
|
||||||
|
r2 = backup_svc.create_now()
|
||||||
|
rows = backup_svc.list_backups()
|
||||||
|
assert [r.id for r in rows] == [r2.backup.id, r1.backup.id]
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_backups_filter_by_status(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
backup_svc.create_now()
|
||||||
|
rows = backup_svc.list_backups(status=STATUS_OK)
|
||||||
|
assert all(r.status == STATUS_OK for r in rows)
|
||||||
|
rows = backup_svc.list_backups(status=STATUS_PENDING)
|
||||||
|
assert rows == []
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# verify
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_verify_ok_after_create(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
r = backup_svc.create_now()
|
||||||
|
v = backup_svc.verify(r.backup.id)
|
||||||
|
assert v.ok
|
||||||
|
assert v.expected_fingerprint == v.actual_fingerprint
|
||||||
|
assert v.table_count >= 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_verify_detects_tampered_ciphertext(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
r = backup_svc.create_now()
|
||||||
|
bin_path = backup_svc.backup_dir / r.backup.filename
|
||||||
|
# Flip a bit in the middle of the encrypted blob.
|
||||||
|
data = bytearray(bin_path.read_bytes())
|
||||||
|
idx = backup_mod.NONCE_LEN + 5
|
||||||
|
data[idx] ^= 0x01
|
||||||
|
bin_path.write_bytes(bytes(data))
|
||||||
|
|
||||||
|
v = backup_svc.verify(r.backup.id)
|
||||||
|
assert not v.ok
|
||||||
|
assert "decryption failed" in (v.reason or "")
|
||||||
|
|
||||||
|
|
||||||
|
def test_verify_handles_missing_file(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
r = backup_svc.create_now()
|
||||||
|
(backup_svc.backup_dir / r.backup.filename).unlink()
|
||||||
|
v = backup_svc.verify(r.backup.id)
|
||||||
|
assert not v.ok
|
||||||
|
assert "missing" in (v.reason or "")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# restore — two-step
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_restore_two_step_round_trip(fresh_db, backup_svc, tmp_path):
|
||||||
|
"""Create a backup, mutate the live DB, restore, confirm mutation gone."""
|
||||||
|
from cyclone.db import Batch
|
||||||
|
import uuid
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
# 1. Backup a DB with one Batch row.
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
snap = backup_svc.create_now()
|
||||||
|
|
||||||
|
# 2. Mutate the live DB (add another Batch row).
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
s.add(Batch(
|
||||||
|
id=str(uuid.uuid4()),
|
||||||
|
kind="837P",
|
||||||
|
input_filename="mutated.x12",
|
||||||
|
parsed_at=datetime.now(timezone.utc),
|
||||||
|
totals_json=None,
|
||||||
|
validation_json=None,
|
||||||
|
raw_result_json={"envelope": {"control_number": "2"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
assert s.query(Batch).count() == 2
|
||||||
|
|
||||||
|
# 3. Initiate restore.
|
||||||
|
init = backup_svc.restore_initiate(snap.backup.id)
|
||||||
|
assert init.table_count >= 1
|
||||||
|
assert init.current_db_fingerprint != init.db_fingerprint # live != backup now
|
||||||
|
assert init.restore_token and len(init.restore_token) == 64
|
||||||
|
|
||||||
|
# 4. Confirm restore.
|
||||||
|
result = backup_svc.restore_confirm(snap.backup.id, init.restore_token)
|
||||||
|
assert result.new_db_fingerprint == init.db_fingerprint
|
||||||
|
|
||||||
|
# 5. The live DB now reflects the snapshot (1 row, not 2).
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
assert s.query(Batch).count() == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_restore_initiate_rejects_non_ok_backup(fresh_db, backup_svc, tmp_path, monkeypatch):
|
||||||
|
"""A backup row with status='error' cannot be restored."""
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
r = backup_svc.create_now()
|
||||||
|
# Force the row to error.
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
row = session.get(DbBackup, r.backup.id)
|
||||||
|
row.status = STATUS_ERROR
|
||||||
|
row.error_message = "simulated"
|
||||||
|
session.commit()
|
||||||
|
with pytest.raises(BackupError, match="only 'ok' backups"):
|
||||||
|
backup_svc.restore_initiate(r.backup.id)
|
||||||
|
|
||||||
|
|
||||||
|
def test_restore_confirm_rejects_wrong_token(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
r = backup_svc.create_now()
|
||||||
|
init = backup_svc.restore_initiate(r.backup.id)
|
||||||
|
with pytest.raises(BackupError, match="not found"):
|
||||||
|
backup_svc.restore_confirm(r.backup.id, "0" * 64)
|
||||||
|
|
||||||
|
|
||||||
|
def test_restore_confirm_rejects_expired_token(fresh_db, backup_svc, monkeypatch):
|
||||||
|
"""A token whose expires_at is in the past is rejected."""
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
r = backup_svc.create_now()
|
||||||
|
init = backup_svc.restore_initiate(r.backup.id)
|
||||||
|
# Manually age the token past its expiry.
|
||||||
|
with backup_svc._lock:
|
||||||
|
backup_svc._pending_restores[init.restore_token] = (
|
||||||
|
init.backup_id,
|
||||||
|
datetime.now(timezone.utc) - timedelta(seconds=1),
|
||||||
|
)
|
||||||
|
with pytest.raises(BackupError, match="expired"):
|
||||||
|
backup_svc.restore_confirm(r.backup.id, init.restore_token)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# prune
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_prune_deletes_files_and_marks_status(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
r1 = backup_svc.create_now()
|
||||||
|
# The retention cutoff is 7 days from now. Move the row's created_at
|
||||||
|
# back 30 days so it's definitely past retention.
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from cyclone.db import DbBackup
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
row = session.get(DbBackup, r1.backup.id)
|
||||||
|
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
deleted = backup_svc.prune()
|
||||||
|
assert len(deleted) == 2 # .bin + .meta.json
|
||||||
|
rows = backup_svc.list_backups()
|
||||||
|
assert rows[0].status == STATUS_PRUNED
|
||||||
|
|
||||||
|
|
||||||
|
def test_prune_keeps_recent_backups(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
backup_svc.create_now()
|
||||||
|
deleted = backup_svc.prune()
|
||||||
|
assert deleted == []
|
||||||
|
rows = backup_svc.list_backups()
|
||||||
|
assert rows[0].status == STATUS_OK
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# status
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_status_reports_counts(fresh_db, backup_svc):
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
_make_a_row(s)
|
||||||
|
backup_svc.create_now()
|
||||||
|
snap = backup_svc.status()
|
||||||
|
assert snap["totals"]["ok"] == 1
|
||||||
|
assert snap["totals"]["all"] == 1
|
||||||
|
assert snap["backup_dir"] == str(backup_svc.backup_dir)
|
||||||
|
assert snap["retention_days"] == 7
|
||||||
|
assert snap["used_fallback_key"] is False
|
||||||
|
assert snap["last_backup_at"] is not None
|
||||||
|
assert snap["last_ok_backup_at"] is not None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Fallback key
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_fallback_key_used_when_no_passphrase(fresh_db, tmp_path):
|
||||||
|
"""If no passphrase AND no SQLCipher, refuse. Otherwise fallback + warn."""
|
||||||
|
backup_dir = tmp_path / "backups"
|
||||||
|
svc = BackupService(backup_dir=backup_dir, passphrase=None, retention_days=7)
|
||||||
|
# No SQLCipher key either → BackupError.
|
||||||
|
with pytest.raises(BackupError, match="no backup passphrase"):
|
||||||
|
svc._ensure_key()
|
||||||
|
|
||||||
|
|
||||||
|
def test_key_fingerprint_changes_per_passphrase(fresh_db, tmp_path):
|
||||||
|
"""Two services with different passphrases have different key fingerprints."""
|
||||||
|
s1 = BackupService(tmp_path / "b1", passphrase="alpha", retention_days=1)
|
||||||
|
s2 = BackupService(tmp_path / "b2", passphrase="beta", retention_days=1)
|
||||||
|
# Force key derivation.
|
||||||
|
s1._ensure_key()
|
||||||
|
s2._ensure_key()
|
||||||
|
assert s1.key_fingerprint != s2.key_fingerprint
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Module-level singleton
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_module_singleton_round_trip(fresh_db, tmp_path):
|
||||||
|
reset_backup_service_for_tests()
|
||||||
|
svc = configure_backup_service(
|
||||||
|
tmp_path / "backups", passphrase="x", retention_days=1,
|
||||||
|
)
|
||||||
|
assert get_backup_service() is svc
|
||||||
|
# Second configure is a no-op (returns existing).
|
||||||
|
assert configure_backup_service(
|
||||||
|
tmp_path / "backups2", passphrase="y", retention_days=2,
|
||||||
|
) is svc
|
||||||
|
reset_backup_service_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
def test_module_singleton_get_raises_when_unset():
|
||||||
|
reset_backup_service_for_tests()
|
||||||
|
with pytest.raises(RuntimeError):
|
||||||
|
get_backup_service()
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
"""SP9 — clearhouse API endpoint tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone.api import app
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db_mod._reset_for_tests()
|
||||||
|
db_mod.init_db()
|
||||||
|
with TestClient(app) as c:
|
||||||
|
yield c
|
||||||
|
db_mod._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_clearhouse_seeded(client):
|
||||||
|
# Lifespan runs ensure_clearhouse_seeded()
|
||||||
|
r = client.get("/api/clearhouse")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["name"] == "dzinesco"
|
||||||
|
assert body["tpid"] == "11525703"
|
||||||
|
assert body["sftp_block"]["stub"] is True
|
||||||
|
assert "FromHPE" in body["sftp_block"]["paths"]["outbound"]
|
||||||
|
assert "ToHPE" in body["sftp_block"]["paths"]["inbound"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_providers(client):
|
||||||
|
r = client.get("/api/config/providers")
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
body = r.json()
|
||||||
|
assert {p["label"] for p in body} == {"Montrose", "Delta", "Salida"}
|
||||||
|
assert {p["npi"] for p in body} == {"1881068062", "1851446637", "1467507269"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_provider_by_npi(client):
|
||||||
|
r = client.get("/api/config/providers/1881068062")
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert r.json()["label"] == "Montrose"
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_provider_404(client):
|
||||||
|
r = client.get("/api/config/providers/9999999999")
|
||||||
|
assert r.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_payers(client):
|
||||||
|
r = client.get("/api/config/payers")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert len(body) == 1
|
||||||
|
assert body[0]["payer_id"] == "CO_TXIX"
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_payer_configs_for_co_txix(client):
|
||||||
|
r = client.get("/api/config/payers/CO_TXIX/configs")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
# The lifespan loads config/payers.yaml; configs come from the registry
|
||||||
|
tx_types = {c["transaction_type"] for c in body}
|
||||||
|
assert "837P" in tx_types or "835" in tx_types
|
||||||
|
|
||||||
|
|
||||||
|
def test_reload_config(client):
|
||||||
|
r = client.post("/api/admin/reload-config")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert isinstance(body["loaded"], int)
|
||||||
|
|
||||||
|
|
||||||
|
def test_submit_clearhouse_rejects_empty_claim_ids(client):
|
||||||
|
r = client.post("/api/clearhouse/submit", json={"claim_ids": [], "payer_id": "CO_TXIX"})
|
||||||
|
assert r.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
def test_submit_clearhouse_rejects_missing_payer_id(client):
|
||||||
|
r = client.post("/api/clearhouse/submit", json={"claim_ids": ["X"]})
|
||||||
|
assert r.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
def test_submit_clearhouse_handles_unknown_claim_id(client):
|
||||||
|
r = client.post("/api/clearhouse/submit", json={
|
||||||
|
"claim_ids": ["DOES_NOT_EXIST"],
|
||||||
|
"payer_id": "CO_TXIX",
|
||||||
|
})
|
||||||
|
# 200 with per-claim ok:false entries (graceful degradation)
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["stub"] is True
|
||||||
|
assert len(body["submitted"]) == 1
|
||||||
|
assert body["submitted"][0]["ok"] is False
|
||||||
|
assert "not found" in body["submitted"][0]["error"] or "cannot be re-serialized" in body["submitted"][0]["error"]
|
||||||
@@ -0,0 +1,144 @@
|
|||||||
|
"""SP17 — `cyclone backup` CLI subcommand tests.
|
||||||
|
|
||||||
|
Uses Click's CliRunner + monkeypatching of Keychain + DB env so the
|
||||||
|
subcommands can run without the operator's machine state.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from click.testing import CliRunner
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def _cli_env(tmp_path, monkeypatch):
|
||||||
|
"""Fresh sqlite DB + in-memory Keychain stub."""
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
from cyclone.db import Batch
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
|
||||||
|
with db.SessionLocal()() as s:
|
||||||
|
s.add(Batch(
|
||||||
|
id=str(uuid.uuid4()),
|
||||||
|
kind="837P",
|
||||||
|
input_filename="seed.x12",
|
||||||
|
parsed_at=datetime.now(timezone.utc),
|
||||||
|
totals_json=None,
|
||||||
|
validation_json=None,
|
||||||
|
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||||
|
))
|
||||||
|
s.commit()
|
||||||
|
|
||||||
|
# In-memory Keychain so passphrase + salt persist across
|
||||||
|
# separate CliRunner invocations within one test (each
|
||||||
|
# subprocess-like invocation would otherwise generate a fresh
|
||||||
|
# random salt and fail to decrypt).
|
||||||
|
store: dict[str, str] = {}
|
||||||
|
store[svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT] = "cli-test-passphrase"
|
||||||
|
# Pre-populate a stable salt so the very first invocation
|
||||||
|
# doesn't generate a new random one (which the next invocation
|
||||||
|
# would then fail to reproduce).
|
||||||
|
store[svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT] = "0123456789abcdef0123456789abcdef"
|
||||||
|
|
||||||
|
def _get(name):
|
||||||
|
return store.get(name)
|
||||||
|
def _set(name, value):
|
||||||
|
store[name] = value
|
||||||
|
return True
|
||||||
|
|
||||||
|
monkeypatch.setattr(secrets_mod, "get_secret", _get)
|
||||||
|
monkeypatch.setattr(secrets_mod, "set_secret", _set)
|
||||||
|
|
||||||
|
backup_dir = tmp_path / "backups"
|
||||||
|
monkeypatch.setenv("CYCLONE_BACKUP_DIR", str(backup_dir))
|
||||||
|
monkeypatch.setenv("CYCLONE_BACKUP_RETENTION_DAYS", "7")
|
||||||
|
|
||||||
|
yield backup_dir
|
||||||
|
db._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
def _run(args, env):
|
||||||
|
from cyclone.cli import main
|
||||||
|
runner = CliRunner()
|
||||||
|
return runner.invoke(main, args, catch_exceptions=False)
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_create_list_verify_status(_cli_env):
|
||||||
|
"""Happy path: create → list → verify → status."""
|
||||||
|
backup_dir = _cli_env
|
||||||
|
|
||||||
|
# create
|
||||||
|
r = _run(["backup", "create"], _cli_env)
|
||||||
|
assert r.exit_code == 0, r.output
|
||||||
|
assert "created backup id=" in r.output
|
||||||
|
|
||||||
|
# list
|
||||||
|
r = _run(["backup", "list"], _cli_env)
|
||||||
|
assert r.exit_code == 0, r.output
|
||||||
|
assert ".bin" in r.output
|
||||||
|
|
||||||
|
# verify (we don't know the id, parse it from the list output)
|
||||||
|
import re
|
||||||
|
m = re.search(r"^\s*(\d+)\s+ok\s+", r.output, re.MULTILINE)
|
||||||
|
assert m, r.output
|
||||||
|
backup_id = int(m.group(1))
|
||||||
|
r = _run(["backup", "verify", str(backup_id)], _cli_env)
|
||||||
|
assert r.exit_code == 0, r.output
|
||||||
|
assert r.output.startswith("OK:")
|
||||||
|
|
||||||
|
# status
|
||||||
|
r = _run(["backup", "status"], _cli_env)
|
||||||
|
assert r.exit_code == 0, r.output
|
||||||
|
assert '"totals"' in r.output
|
||||||
|
assert '"ok": 1' in r.output
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_verify_fails_on_tampered_ciphertext(_cli_env):
|
||||||
|
from cyclone import backup as backup_mod
|
||||||
|
from cyclone import backup_service as svc_mod
|
||||||
|
from cyclone import secrets as secrets_mod
|
||||||
|
|
||||||
|
# Create a backup.
|
||||||
|
r = _run(["backup", "create"], _cli_env)
|
||||||
|
assert r.exit_code == 0
|
||||||
|
# Tamper.
|
||||||
|
bin_path = next(_cli_env.glob("*.bin"))
|
||||||
|
data = bytearray(bin_path.read_bytes())
|
||||||
|
data[backup_mod.NONCE_LEN + 5] ^= 0x01
|
||||||
|
bin_path.write_bytes(bytes(data))
|
||||||
|
|
||||||
|
# Verify should fail.
|
||||||
|
r = _run(["backup", "verify", "1"], _cli_env)
|
||||||
|
assert r.exit_code == 1
|
||||||
|
assert "FAIL" in r.output
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_restore_requires_yes_flag(_cli_env):
|
||||||
|
"""Without --yes, an interactive confirm blocks and the command aborts."""
|
||||||
|
r = _run(["backup", "create"], _cli_env)
|
||||||
|
assert r.exit_code == 0
|
||||||
|
# Click's runner auto-declines the confirm prompt; expect abort.
|
||||||
|
r = _run(["backup", "restore", "1"], _cli_env, )
|
||||||
|
# CliRunner auto-aborts confirm prompts by default → exit code != 0.
|
||||||
|
assert r.exit_code != 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_prune_aborts_without_yes(_cli_env):
|
||||||
|
r = _run(["backup", "create"], _cli_env)
|
||||||
|
assert r.exit_code == 0
|
||||||
|
# Same auto-abort for the prune confirm.
|
||||||
|
r = _run(["backup", "prune"], _cli_env)
|
||||||
|
assert r.exit_code != 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_init_passphrase_rejects_short(_cli_env):
|
||||||
|
"""init-passphrase enforces a 12-char minimum."""
|
||||||
|
r = _run(["backup", "init-passphrase", "--passphrase", "short"], _cli_env)
|
||||||
|
assert r.exit_code != 0
|
||||||
|
assert "12 characters" in r.output
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
"""Tests for ``cyclone validate-npi`` + ``cyclone validate-tax-id`` (SP20).
|
||||||
|
|
||||||
|
CLI smoke tests — verify exit codes (0 = valid, 1 = invalid) and that
|
||||||
|
the help text references the new subcommands. We don't pipe the value
|
||||||
|
into shared logs (NPI / EIN are PHI / PII); the CliRunner captures it.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from click.testing import CliRunner
|
||||||
|
|
||||||
|
from cyclone.cli import main
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# validate-npi
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_validate_npi_valid_exits_zero():
|
||||||
|
runner = CliRunner()
|
||||||
|
result = runner.invoke(main, ["validate-npi", "1234567893"])
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "OK" in result.output
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_validate_npi_bad_checksum_exits_one():
|
||||||
|
runner = CliRunner()
|
||||||
|
result = runner.invoke(main, ["validate-npi", "1234567890"])
|
||||||
|
assert result.exit_code == 1
|
||||||
|
assert "INVALID" in result.output
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_validate_npi_wrong_length_exits_one():
|
||||||
|
runner = CliRunner()
|
||||||
|
result = runner.invoke(main, ["validate-npi", "12345"])
|
||||||
|
assert result.exit_code == 1
|
||||||
|
assert "INVALID" in result.output
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# validate-tax-id
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_validate_tax_id_formatted_exits_zero():
|
||||||
|
runner = CliRunner()
|
||||||
|
result = runner.invoke(main, ["validate-tax-id", "72-1587149"])
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "721587149" in result.output # normalized form echoed
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_validate_tax_id_unformatted_exits_zero():
|
||||||
|
runner = CliRunner()
|
||||||
|
result = runner.invoke(main, ["validate-tax-id", "721587149"])
|
||||||
|
assert result.exit_code == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_validate_tax_id_reserved_prefix_exits_one():
|
||||||
|
runner = CliRunner()
|
||||||
|
result = runner.invoke(main, ["validate-tax-id", "00-1234567"])
|
||||||
|
assert result.exit_code == 1
|
||||||
|
assert "reserved" in result.output.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_validate_tax_id_malformed_exits_one():
|
||||||
|
runner = CliRunner()
|
||||||
|
result = runner.invoke(main, ["validate-tax-id", "not-an-ein"])
|
||||||
|
assert result.exit_code == 1
|
||||||
|
assert "9-digit" in result.output
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_validate_subcommands_appear_in_help():
|
||||||
|
"""The two new subcommands are wired into ``main`` (regression guard
|
||||||
|
against future refactors that drop the imports)."""
|
||||||
|
runner = CliRunner()
|
||||||
|
result = runner.invoke(main, ["--help"])
|
||||||
|
assert result.exit_code == 0
|
||||||
|
assert "validate-npi" in result.output
|
||||||
|
assert "validate-tax-id" in result.output
|
||||||
@@ -0,0 +1,292 @@
|
|||||||
|
"""Tests for SQLCipher encryption at rest. SP12.
|
||||||
|
|
||||||
|
We exercise the encryption path end-to-end:
|
||||||
|
1. With a Keychain key + sqlcipher3 installed: the DB file is encrypted
|
||||||
|
on disk and decryptable only with the same key.
|
||||||
|
2. Without a Keychain key: the DB falls back to plain SQLite.
|
||||||
|
3. With the wrong key: opening the DB raises on the first query.
|
||||||
|
|
||||||
|
``sqlcipher3`` is an optional dep — these tests skip when it isn't
|
||||||
|
installed, so the suite still runs on Linux dev boxes without SQLCipher.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
from cyclone import db, db_crypto
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Skip-if-no-sqlcipher gate
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
pytestmark_sqlcipher = pytest.mark.skipif(
|
||||||
|
not db_crypto.is_sqlcipher_available(),
|
||||||
|
reason="sqlcipher3 not installed (pip install -e .[sqlcipher])",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Capability checks
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestIsSqlcipherAvailable:
|
||||||
|
def test_returns_true_when_package_installed(self):
|
||||||
|
"""This test only runs when sqlcipher3 is importable."""
|
||||||
|
assert db_crypto.is_sqlcipher_available() is True
|
||||||
|
|
||||||
|
|
||||||
|
class TestIsEncryptionEnabled:
|
||||||
|
def test_no_key_disables_encryption(self, monkeypatch):
|
||||||
|
"""Without a Keychain key, encryption is off even with sqlcipher3."""
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: None)
|
||||||
|
assert db_crypto.is_encryption_enabled() is False
|
||||||
|
|
||||||
|
def test_stub_key_disables_encryption(self, monkeypatch):
|
||||||
|
"""The stub fallback secret doesn't count as a real key."""
|
||||||
|
from cyclone.secrets import STUB_SECRET
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: STUB_SECRET)
|
||||||
|
assert db_crypto.is_encryption_enabled() is False
|
||||||
|
|
||||||
|
def test_real_key_enables_encryption(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: "real-key-from-keychain")
|
||||||
|
assert db_crypto.is_encryption_enabled() is True
|
||||||
|
|
||||||
|
|
||||||
|
class TestGetDbKey:
|
||||||
|
def test_returns_none_when_no_key(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: None)
|
||||||
|
assert db_crypto.get_db_key() is None
|
||||||
|
|
||||||
|
def test_returns_key_from_keychain(self, monkeypatch):
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: "abc123")
|
||||||
|
assert db_crypto.get_db_key() == "abc123"
|
||||||
|
|
||||||
|
def test_stub_secret_returns_none(self, monkeypatch):
|
||||||
|
from cyclone.secrets import STUB_SECRET
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: STUB_SECRET)
|
||||||
|
assert db_crypto.get_db_key() is None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Engine integration
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
@pytestmark_sqlcipher
|
||||||
|
class TestEngineIntegration:
|
||||||
|
def test_engine_uses_sqlcipher_when_key_present(self, monkeypatch, tmp_path: Path):
|
||||||
|
"""With a key, _make_engine installs a sqlcipher3 creator."""
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: "test-key-xyz")
|
||||||
|
db_file = tmp_path / "encrypted.db"
|
||||||
|
url = f"sqlite:///{db_file}"
|
||||||
|
engine = db._make_engine(url)
|
||||||
|
# Use the engine to write and read back.
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(sa.text("CREATE TABLE t (x INTEGER)"))
|
||||||
|
conn.execute(sa.text("INSERT INTO t VALUES (1)"))
|
||||||
|
with engine.connect() as conn:
|
||||||
|
assert conn.execute(sa.text("SELECT x FROM t")).scalar() == 1
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
def test_engine_uses_plain_sqlite_without_key(self, monkeypatch, tmp_path: Path):
|
||||||
|
"""Without a key, _make_engine uses the default sqlite3 driver."""
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: None)
|
||||||
|
db_file = tmp_path / "plain.db"
|
||||||
|
url = f"sqlite:///{db_file}"
|
||||||
|
engine = db._make_engine(url)
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(sa.text("CREATE TABLE t (x INTEGER)"))
|
||||||
|
engine.dispose()
|
||||||
|
# The file is a valid plain SQLite DB.
|
||||||
|
with sqlite3.connect(str(db_file)) as conn:
|
||||||
|
assert conn.execute("SELECT count(*) FROM sqlite_master").fetchone()[0] >= 1
|
||||||
|
|
||||||
|
def test_encrypted_file_unreadable_without_key(self, monkeypatch, tmp_path: Path):
|
||||||
|
"""An encrypted file is unreadable as plain SQLite."""
|
||||||
|
# Create an encrypted DB.
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: "secret-1")
|
||||||
|
db_file = tmp_path / "encrypted.db"
|
||||||
|
url = f"sqlite:///{db_file}"
|
||||||
|
engine = db._make_engine(url)
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(sa.text("CREATE TABLE t (secret TEXT)"))
|
||||||
|
conn.execute(sa.text("INSERT INTO t VALUES ('classified')"))
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
# Plain sqlite3 cannot open it.
|
||||||
|
with pytest.raises((sqlite3.DatabaseError, Exception)) as exc_info:
|
||||||
|
sqlite3.connect(str(db_file)).execute("SELECT * FROM t").fetchall()
|
||||||
|
# The error message comes from SQLite/SQLCipher, not Python.
|
||||||
|
assert "not a database" in str(exc_info.value).lower() or "file is encrypted" in str(exc_info.value).lower()
|
||||||
|
|
||||||
|
def test_wrong_key_raises_on_query(self, monkeypatch, tmp_path: Path):
|
||||||
|
"""A wrong key on the same encrypted file raises on the first query."""
|
||||||
|
# Create with key A.
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: "key-A")
|
||||||
|
db_file = tmp_path / "encrypted.db"
|
||||||
|
url = f"sqlite:///{db_file}"
|
||||||
|
engine = db._make_engine(url)
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(sa.text("CREATE TABLE t (x INTEGER)"))
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
# Try to open with key B.
|
||||||
|
monkeypatch.setattr(db_crypto, "get_secret", lambda account: "key-B")
|
||||||
|
engine2 = db._make_engine(url)
|
||||||
|
with pytest.raises(Exception) as exc_info:
|
||||||
|
with engine2.connect() as conn:
|
||||||
|
conn.execute(sa.text("SELECT * FROM t")).fetchall()
|
||||||
|
# SQLCipher raises "file is not a database" or similar on bad key.
|
||||||
|
msg = str(exc_info.value).lower()
|
||||||
|
assert "not a database" in msg or "file is encrypted" in msg or "databaseerror" in msg
|
||||||
|
engine2.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Make_creator function
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
@pytestmark_sqlcipher
|
||||||
|
class TestMakeSqlcipherConnectCreator:
|
||||||
|
def test_creator_returns_connection_with_key_applied(self, tmp_path: Path):
|
||||||
|
"""The creator's connection must have PRAGMA key applied."""
|
||||||
|
db_file = tmp_path / "x.db"
|
||||||
|
url = f"sqlite:///{db_file}"
|
||||||
|
creator = db_crypto.make_sqlcipher_connect_creator(url, "my-key")
|
||||||
|
conn = creator()
|
||||||
|
# The creator must have applied the key — verify by writing
|
||||||
|
# data and reading it back via the same connection.
|
||||||
|
conn.execute("CREATE TABLE t (x INTEGER)")
|
||||||
|
conn.execute("INSERT INTO t VALUES (42)")
|
||||||
|
conn.commit()
|
||||||
|
result = conn.execute("SELECT x FROM t").fetchone()
|
||||||
|
assert result[0] == 42
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# SP15: Key generation + fingerprint
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestGenerateDbKey:
|
||||||
|
def test_returns_64_char_hex(self):
|
||||||
|
"""A 256-bit key hex-encodes to 64 characters."""
|
||||||
|
key = db_crypto.generate_db_key()
|
||||||
|
assert len(key) == 64
|
||||||
|
int(key, 16) # parses as hex (raises if not)
|
||||||
|
|
||||||
|
def test_two_calls_return_different_keys(self):
|
||||||
|
"""Distinct calls produce cryptographically distinct keys."""
|
||||||
|
keys = {db_crypto.generate_db_key() for _ in range(8)}
|
||||||
|
assert len(keys) == 8
|
||||||
|
|
||||||
|
|
||||||
|
class TestFingerprint:
|
||||||
|
def test_deterministic(self):
|
||||||
|
assert db_crypto.fingerprint("abc") == db_crypto.fingerprint("abc")
|
||||||
|
|
||||||
|
def test_different_inputs_yield_different_fingerprints(self):
|
||||||
|
assert db_crypto.fingerprint("abc") != db_crypto.fingerprint("xyz")
|
||||||
|
|
||||||
|
def test_eight_chars(self):
|
||||||
|
assert len(db_crypto.fingerprint("anything")) == 8
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# SP15: rotate_db_key (in-place rekey via PRAGMA rekey)
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
@pytestmark_sqlcipher
|
||||||
|
class TestRotateDbKey:
|
||||||
|
def _create_encrypted_db(self, tmp_path: Path, key: str) -> Path:
|
||||||
|
"""Create a small SQLCipher DB with two tables."""
|
||||||
|
import sqlcipher3
|
||||||
|
db_file = tmp_path / "rotate.db"
|
||||||
|
conn = sqlcipher3.connect(str(db_file))
|
||||||
|
conn.execute(f'PRAGMA key = "{key}"')
|
||||||
|
conn.execute("CREATE TABLE accounts (id INTEGER PRIMARY KEY, name TEXT)")
|
||||||
|
conn.execute("CREATE TABLE balances (acct_id INTEGER, amt REAL)")
|
||||||
|
conn.execute("INSERT INTO accounts VALUES (1, 'alice'), (2, 'bob')")
|
||||||
|
conn.execute("INSERT INTO balances VALUES (1, 100.5), (2, 250.75)")
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
return db_file
|
||||||
|
|
||||||
|
def test_rotate_changes_key_preserves_data(self, tmp_path: Path):
|
||||||
|
"""The core SP15 contract: rekey with a new key, data survives."""
|
||||||
|
db_file = self._create_encrypted_db(tmp_path, "old-key-aaaa")
|
||||||
|
url = f"sqlite:///{db_file}"
|
||||||
|
result = db_crypto.rotate_db_key(
|
||||||
|
url=url, old_key="old-key-aaaa", new_key="new-key-bbbb",
|
||||||
|
)
|
||||||
|
assert result.ok, f"rotate failed: {result.reason}"
|
||||||
|
assert result.old_fingerprint == db_crypto.fingerprint("old-key-aaaa")
|
||||||
|
assert result.new_fingerprint == db_crypto.fingerprint("new-key-bbbb")
|
||||||
|
assert result.table_count == 2 # accounts + balances
|
||||||
|
|
||||||
|
# Open with the new key; data is intact.
|
||||||
|
import sqlcipher3
|
||||||
|
conn = sqlcipher3.connect(str(db_file))
|
||||||
|
conn.execute(f'PRAGMA key = "new-key-bbbb"')
|
||||||
|
rows = conn.execute("SELECT id, name FROM accounts ORDER BY id").fetchall()
|
||||||
|
assert rows == [(1, "alice"), (2, "bob")]
|
||||||
|
assert conn.execute("SELECT amt FROM balances WHERE acct_id = 2").fetchone()[0] == 250.75
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
def test_old_key_no_longer_opens_db(self, tmp_path: Path):
|
||||||
|
"""After rekey, the old key must not be able to open the DB."""
|
||||||
|
import sqlcipher3
|
||||||
|
db_file = self._create_encrypted_db(tmp_path, "old-key")
|
||||||
|
url = f"sqlite:///{db_file}"
|
||||||
|
result = db_crypto.rotate_db_key(
|
||||||
|
url=url, old_key="old-key", new_key="new-key",
|
||||||
|
)
|
||||||
|
assert result.ok
|
||||||
|
|
||||||
|
# Old key raises on first query.
|
||||||
|
conn = sqlcipher3.connect(str(db_file))
|
||||||
|
conn.execute(f'PRAGMA key = "old-key"')
|
||||||
|
with pytest.raises(Exception) as exc_info:
|
||||||
|
conn.execute("SELECT * FROM accounts").fetchall()
|
||||||
|
msg = str(exc_info.value).lower()
|
||||||
|
assert "not a database" in msg or "file is encrypted" in msg
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
def test_wrong_old_key_reports_helpful_reason(self, tmp_path: Path):
|
||||||
|
"""If the operator types the wrong old key, the rekey fails clean."""
|
||||||
|
db_file = self._create_encrypted_db(tmp_path, "correct-old")
|
||||||
|
url = f"sqlite:///{db_file}"
|
||||||
|
result = db_crypto.rotate_db_key(
|
||||||
|
url=url, old_key="WRONG-OLD-KEY", new_key="new",
|
||||||
|
)
|
||||||
|
assert result.ok is False
|
||||||
|
assert "old key did not open" in result.reason.lower()
|
||||||
|
|
||||||
|
def test_in_memory_url_is_rejected(self):
|
||||||
|
"""In-memory DBs cannot be rekeyed (nothing to persist)."""
|
||||||
|
result = db_crypto.rotate_db_key(
|
||||||
|
url="sqlite:///:memory:", old_key="a", new_key="b",
|
||||||
|
)
|
||||||
|
assert result.ok is False
|
||||||
|
assert "file-backed" in result.reason.lower() or "in-memory" in result.reason.lower()
|
||||||
|
|
||||||
|
def test_missing_db_file_is_rejected(self, tmp_path: Path):
|
||||||
|
result = db_crypto.rotate_db_key(
|
||||||
|
url=f"sqlite:///{tmp_path}/does-not-exist.db",
|
||||||
|
old_key="a", new_key="b",
|
||||||
|
)
|
||||||
|
assert result.ok is False
|
||||||
|
assert "not found" in result.reason.lower()
|
||||||
@@ -0,0 +1,159 @@
|
|||||||
|
"""SP9 — HCPF X12 File Naming Standards helper tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.edi.filenames import (
|
||||||
|
ALLOWED_FILE_TYPES,
|
||||||
|
OUTBOUND_RE,
|
||||||
|
INBOUND_RE,
|
||||||
|
build_outbound_filename,
|
||||||
|
is_inbound_filename,
|
||||||
|
is_outbound_filename,
|
||||||
|
parse_inbound_filename,
|
||||||
|
)
|
||||||
|
from cyclone.providers import InboundFilename
|
||||||
|
|
||||||
|
MT = ZoneInfo("America/Denver")
|
||||||
|
|
||||||
|
|
||||||
|
# ----- build_outbound_filename --------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_outbound_with_explicit_mt():
|
||||||
|
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
||||||
|
name = build_outbound_filename("11525703", "837P", now_mt=now)
|
||||||
|
assert name == "11525703-837P-20260620132243505-1of1.x12"
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_outbound_default_extension():
|
||||||
|
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
||||||
|
name = build_outbound_filename("11525703", "837P", now_mt=now)
|
||||||
|
assert name.endswith(".x12")
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_outbound_custom_extension():
|
||||||
|
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
||||||
|
name = build_outbound_filename("11525703", "837P", ext="txt", now_mt=now)
|
||||||
|
assert name == "11525703-837P-20260620132243505-1of1.txt"
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_outbound_uses_mt_when_no_arg():
|
||||||
|
# Snapshot test — the timestamp will be very recent; check format only
|
||||||
|
name = build_outbound_filename("11525703", "837P")
|
||||||
|
assert OUTBOUND_RE.match(name), name
|
||||||
|
parts = name.split("-")
|
||||||
|
assert len(parts) == 4
|
||||||
|
assert len(parts[2]) == 17 # yyyymmddhhmmssSSS
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_outbound_rejects_non_numeric_tpid():
|
||||||
|
now = datetime(2026, 6, 20, tzinfo=MT)
|
||||||
|
with pytest.raises(ValueError, match="tpid must be digits"):
|
||||||
|
build_outbound_filename("abc123", "837P", now_mt=now)
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_outbound_rejects_invalid_tx():
|
||||||
|
now = datetime(2026, 6, 20, tzinfo=MT)
|
||||||
|
with pytest.raises(ValueError, match="tx must be uppercase alnum"):
|
||||||
|
build_outbound_filename("11525703", "837-lower", now_mt=now)
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_outbound_rejects_naive_dt():
|
||||||
|
with pytest.raises(ValueError, match="timezone-aware"):
|
||||||
|
build_outbound_filename("11525703", "837P", now_mt=datetime(2026, 6, 20))
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_outbound_converts_other_tz_to_mt():
|
||||||
|
# 2026-06-20 19:22:43 UTC = 2026-06-20 13:22:43 MT (during DST)
|
||||||
|
from datetime import timezone
|
||||||
|
now_utc = datetime(2026, 6, 20, 19, 22, 43, 505_000, tzinfo=timezone.utc)
|
||||||
|
name = build_outbound_filename("11525703", "837P", now_mt=now_utc)
|
||||||
|
assert "20260620132243505" in name
|
||||||
|
|
||||||
|
|
||||||
|
# ----- parse_inbound_filename ---------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_inbound_999_real_prodfile():
|
||||||
|
# Real production 999 filename from docs/prodfiles/FromHPE/
|
||||||
|
name = "TP11525703-837P_M019048402-20260520231513488-1of1_999.x12"
|
||||||
|
parsed = parse_inbound_filename(name)
|
||||||
|
assert parsed.tpid == "11525703"
|
||||||
|
assert parsed.orig_tx == "837P"
|
||||||
|
assert parsed.tracking == "M019048402"
|
||||||
|
assert parsed.ts == "20260520231513488"
|
||||||
|
assert parsed.file_type == "999"
|
||||||
|
assert parsed.ext == "x12"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_inbound_ta1_real_prodfile():
|
||||||
|
name = "TP11525703-837P_M019044969-20260520180505477-1of1_TA1.x12"
|
||||||
|
parsed = parse_inbound_filename(name)
|
||||||
|
assert parsed.file_type == "TA1"
|
||||||
|
assert parsed.tracking == "M019044969"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_inbound_277():
|
||||||
|
name = "TP11525703-837P_M019110219-20260601003507042-1of1_277.x12"
|
||||||
|
parsed = parse_inbound_filename(name)
|
||||||
|
assert parsed.file_type == "277"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_inbound_rejects_missing_tp_prefix():
|
||||||
|
with pytest.raises(ValueError, match="Not a valid HCPF inbound"):
|
||||||
|
parse_inbound_filename("11525703-837P_M019048402-20260520231513488-1of1_999.x12")
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_inbound_rejects_wrong_segment_count():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
parse_inbound_filename("TP11525703_837P_M019048402-1of1_999.x12")
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_inbound_rejects_unknown_file_type():
|
||||||
|
with pytest.raises(ValueError, match="not in allowed HCPF set"):
|
||||||
|
parse_inbound_filename("TP11525703-837P_M019048402-20260520231513488-1of1_XXX.x12")
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_inbound_rejects_non_x12_ext():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
parse_inbound_filename("TP11525703-837P_M019048402-20260520231513488-1of1_999.txt")
|
||||||
|
|
||||||
|
|
||||||
|
# ----- round-trip ---------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_roundtrip_outbound_to_inbound():
|
||||||
|
# Outbound tpid is bare (no TP); inbound tpid is bare inside TP{...}
|
||||||
|
# The two regexes use different shapes — round-trip via tpid only.
|
||||||
|
now = datetime(2026, 6, 20, 13, 22, 43, 505_000, tzinfo=MT)
|
||||||
|
out = build_outbound_filename("11525703", "837P", now_mt=now)
|
||||||
|
assert OUTBOUND_RE.match(out)
|
||||||
|
assert "11525703" in out
|
||||||
|
assert "837P" in out
|
||||||
|
assert out.endswith("1of1.x12")
|
||||||
|
|
||||||
|
|
||||||
|
# ----- validators ---------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_outbound_filename():
|
||||||
|
assert is_outbound_filename("11525703-837P-20260620132243505-1of1.x12")
|
||||||
|
assert not is_outbound_filename("TP11525703-837P-20260620132243505-1of1.x12")
|
||||||
|
assert not is_outbound_filename("not-a-filename")
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_inbound_filename():
|
||||||
|
assert is_inbound_filename("TP11525703-837P_M019048402-20260520231513488-1of1_999.x12")
|
||||||
|
assert not is_inbound_filename("11525703-837P-20260620132243505-1of1.x12")
|
||||||
|
|
||||||
|
|
||||||
|
def test_allowed_file_types_includes_277ca():
|
||||||
|
assert "277CA" in ALLOWED_FILE_TYPES
|
||||||
|
assert "TA1" in ALLOWED_FILE_TYPES
|
||||||
|
assert "999" in ALLOWED_FILE_TYPES
|
||||||
|
assert "835" in ALLOWED_FILE_TYPES
|
||||||
@@ -47,11 +47,14 @@ def _seed_batch() -> None:
|
|||||||
s.commit()
|
s.commit()
|
||||||
|
|
||||||
|
|
||||||
def test_lanes_endpoint_returns_four_keys(client: TestClient):
|
def test_lanes_endpoint_returns_five_keys(client: TestClient):
|
||||||
|
"""SP10 added the payer_rejected lane (distinct from 999 envelope rejection)."""
|
||||||
r = client.get("/api/inbox/lanes")
|
r = client.get("/api/inbox/lanes")
|
||||||
assert r.status_code == 200
|
assert r.status_code == 200
|
||||||
body = r.json()
|
body = r.json()
|
||||||
assert set(body.keys()) == {"rejected", "candidates", "unmatched", "done_today"}
|
assert set(body.keys()) == {
|
||||||
|
"rejected", "payer_rejected", "candidates", "unmatched", "done_today",
|
||||||
|
}
|
||||||
for v in body.values():
|
for v in body.values():
|
||||||
assert isinstance(v, list)
|
assert isinstance(v, list)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,105 @@
|
|||||||
|
"""SP14 — Lane filter drops acknowledged payer-rejected claims.
|
||||||
|
|
||||||
|
The Payer-Rejected Inbox lane must not include claims the operator
|
||||||
|
has already acknowledged. This is the working-surface UX: acknowledged
|
||||||
|
claims drop out so the operator only sees new rejections.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.db import Batch, Claim, ClaimState
|
||||||
|
from cyclone.inbox_lanes import compute_lanes
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _setup(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
|
||||||
|
|
||||||
|
def _seed(*, acked: bool = False) -> str:
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
cid = "C1"
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
batch = Batch(
|
||||||
|
id="b-1", kind="837p",
|
||||||
|
input_filename="t.x12", parsed_at=now,
|
||||||
|
)
|
||||||
|
session.add(batch)
|
||||||
|
session.flush()
|
||||||
|
claim = Claim(
|
||||||
|
id=cid, batch_id=batch.id, patient_control_number=cid,
|
||||||
|
state=ClaimState.SUBMITTED, charge_amount=100,
|
||||||
|
payer_rejected_at=now,
|
||||||
|
payer_rejected_status_code="A7",
|
||||||
|
payer_rejected_reason="invalid dx",
|
||||||
|
)
|
||||||
|
if acked:
|
||||||
|
claim.payer_rejected_acknowledged_at = now
|
||||||
|
claim.payer_rejected_acknowledged_actor = "operator"
|
||||||
|
session.add(claim)
|
||||||
|
session.commit()
|
||||||
|
return cid
|
||||||
|
|
||||||
|
|
||||||
|
def test_unacknowledged_claim_appears_in_lane():
|
||||||
|
_seed(acked=False)
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
lanes = compute_lanes(session, dismissed_pairs=set())
|
||||||
|
assert len(lanes.payer_rejected) == 1
|
||||||
|
assert lanes.payer_rejected[0]["id"] == "C1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_acknowledged_claim_drops_out_of_lane():
|
||||||
|
_seed(acked=True)
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
lanes = compute_lanes(session, dismissed_pairs=set())
|
||||||
|
assert lanes.payer_rejected == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_mix_of_acked_and_unacked():
|
||||||
|
"""Only the unacknowledged one shows up."""
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
batch = Batch(
|
||||||
|
id="b-1", kind="837p",
|
||||||
|
input_filename="t.x12", parsed_at=now,
|
||||||
|
)
|
||||||
|
session.add(batch)
|
||||||
|
session.flush()
|
||||||
|
for i, acked in enumerate([True, False, True, False]):
|
||||||
|
claim = Claim(
|
||||||
|
id=f"C{i}", batch_id=batch.id,
|
||||||
|
patient_control_number=f"PCN-{i}",
|
||||||
|
state=ClaimState.SUBMITTED, charge_amount=100,
|
||||||
|
payer_rejected_at=now,
|
||||||
|
payer_rejected_status_code="A7",
|
||||||
|
)
|
||||||
|
if acked:
|
||||||
|
claim.payer_rejected_acknowledged_at = now
|
||||||
|
session.add(claim)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
lanes = compute_lanes(session, dismissed_pairs=set())
|
||||||
|
ids = [r["id"] for r in lanes.payer_rejected]
|
||||||
|
assert sorted(ids) == ["C1", "C3"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_lane_row_carries_ack_fields_for_forward_compat():
|
||||||
|
"""The row payload still carries the ack fields (all null on the lane)
|
||||||
|
so future views that want a 'Recently acknowledged' section don't need
|
||||||
|
a schema change."""
|
||||||
|
_seed(acked=False)
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
lanes = compute_lanes(session, dismissed_pairs=set())
|
||||||
|
row = lanes.payer_rejected[0]
|
||||||
|
assert "payer_rejected_acknowledged_at" in row
|
||||||
|
assert row["payer_rejected_acknowledged_at"] is None
|
||||||
|
assert "payer_rejected_acknowledged_actor" in row
|
||||||
|
assert row["payer_rejected_acknowledged_actor"] is None
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
"""SP18 — JsonFormatter + CycloneDevFormatter tests.
|
||||||
|
|
||||||
|
Covers the structural shape of log records, exception handling,
|
||||||
|
and the ``extra`` kwarg passthrough.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.logging_config import (
|
||||||
|
CycloneDevFormatter,
|
||||||
|
JsonFormatter,
|
||||||
|
setup_logging,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _make_record(
|
||||||
|
msg: str = "hello",
|
||||||
|
args: tuple = (),
|
||||||
|
level: int = logging.INFO,
|
||||||
|
name: str = "test.logger",
|
||||||
|
extras: dict | None = None,
|
||||||
|
exc_info=None,
|
||||||
|
) -> logging.LogRecord:
|
||||||
|
record = logging.getLogger(name).makeRecord(
|
||||||
|
name=name,
|
||||||
|
level=level,
|
||||||
|
fn="t.py",
|
||||||
|
lno=1,
|
||||||
|
msg=msg,
|
||||||
|
args=args,
|
||||||
|
exc_info=exc_info,
|
||||||
|
)
|
||||||
|
if extras:
|
||||||
|
for k, v in extras.items():
|
||||||
|
setattr(record, k, v)
|
||||||
|
return record
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# JsonFormatter
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_json_formatter_basic_shape():
|
||||||
|
f = JsonFormatter()
|
||||||
|
line = f.format(_make_record(msg="hello %s", args=("cyclone",)))
|
||||||
|
parsed = json.loads(line)
|
||||||
|
assert parsed["level"] == "INFO"
|
||||||
|
assert parsed["logger"] == "test.logger"
|
||||||
|
assert parsed["msg"] == "hello cyclone"
|
||||||
|
assert "ts" in parsed
|
||||||
|
# ts must be ISO 8601 with milliseconds + Z suffix.
|
||||||
|
assert parsed["ts"].endswith("Z") or "+" in parsed["ts"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_json_formatter_includes_extras():
|
||||||
|
f = JsonFormatter()
|
||||||
|
line = f.format(_make_record(
|
||||||
|
msg="processed",
|
||||||
|
extras={"input_filename": "foo.x12", "parser_kind": "parse_999", "claims": 3},
|
||||||
|
))
|
||||||
|
parsed = json.loads(line)
|
||||||
|
assert parsed["extra"] == {
|
||||||
|
"input_filename": "foo.x12", "parser_kind": "parse_999", "claims": 3,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_json_formatter_handles_exception_info():
|
||||||
|
f = JsonFormatter()
|
||||||
|
try:
|
||||||
|
raise ValueError("boom")
|
||||||
|
except ValueError:
|
||||||
|
import sys
|
||||||
|
rec = _make_record(msg="oops", exc_info=sys.exc_info())
|
||||||
|
line = f.format(rec)
|
||||||
|
parsed = json.loads(line)
|
||||||
|
assert "traceback" in parsed
|
||||||
|
assert "ValueError: boom" in parsed["traceback"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_json_formatter_no_extras_key_when_none():
|
||||||
|
f = JsonFormatter()
|
||||||
|
line = f.format(_make_record(msg="plain"))
|
||||||
|
parsed = json.loads(line)
|
||||||
|
assert "extra" not in parsed
|
||||||
|
|
||||||
|
|
||||||
|
def test_json_formatter_handles_non_serializable_extras():
|
||||||
|
"""Non-JSON-serializable extras go through ``default=str``."""
|
||||||
|
class Opaque:
|
||||||
|
def __str__(self):
|
||||||
|
return "opaque-string"
|
||||||
|
f = JsonFormatter()
|
||||||
|
line = f.format(_make_record(msg="x", extras={"thing": Opaque()}))
|
||||||
|
parsed = json.loads(line)
|
||||||
|
assert parsed["extra"]["thing"] == "opaque-string"
|
||||||
|
|
||||||
|
|
||||||
|
def test_json_formatter_preserves_warning_level():
|
||||||
|
f = JsonFormatter()
|
||||||
|
line = f.format(_make_record(msg="careful", level=logging.WARNING))
|
||||||
|
parsed = json.loads(line)
|
||||||
|
assert parsed["level"] == "WARNING"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# CycloneDevFormatter
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_dev_formatter_basic_shape():
|
||||||
|
f = CycloneDevFormatter()
|
||||||
|
line = f.format(_make_record(msg="hello %s", args=("cyclone",)))
|
||||||
|
assert "INFO" in line
|
||||||
|
assert "test.logger" in line
|
||||||
|
assert "hello cyclone" in line
|
||||||
|
|
||||||
|
|
||||||
|
def test_dev_formatter_includes_extras():
|
||||||
|
f = CycloneDevFormatter()
|
||||||
|
line = f.format(_make_record(
|
||||||
|
msg="processed",
|
||||||
|
extras={"input_filename": "foo.x12", "claims": 3},
|
||||||
|
))
|
||||||
|
assert "input_filename='foo.x12'" in line
|
||||||
|
assert "claims=3" in line
|
||||||
|
|
||||||
|
|
||||||
|
def test_dev_formatter_handles_exception():
|
||||||
|
f = CycloneDevFormatter()
|
||||||
|
try:
|
||||||
|
raise RuntimeError("nope")
|
||||||
|
except RuntimeError:
|
||||||
|
import sys
|
||||||
|
rec = _make_record(msg="oops", exc_info=sys.exc_info())
|
||||||
|
line = f.format(rec)
|
||||||
|
assert "RuntimeError: nope" in line
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# setup_logging (light — deeper coverage in test_logging_setup.py)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_logging_attaches_handler_to_root():
|
||||||
|
setup_logging(level="DEBUG", json_format=True)
|
||||||
|
root = logging.getLogger()
|
||||||
|
assert len(root.handlers) >= 1
|
||||||
|
assert isinstance(root.handlers[0].formatter, JsonFormatter)
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_logging_is_idempotent():
|
||||||
|
"""Re-calling clears handlers; the formatter toggle takes effect."""
|
||||||
|
setup_logging(level="INFO", json_format=True)
|
||||||
|
setup_logging(level="INFO", json_format=False)
|
||||||
|
root = logging.getLogger()
|
||||||
|
assert len(root.handlers) == 1
|
||||||
|
assert isinstance(root.handlers[0].formatter, CycloneDevFormatter)
|
||||||
|
# Reset back to JSON for the rest of the test suite.
|
||||||
|
setup_logging(level="INFO", json_format=True)
|
||||||
@@ -0,0 +1,157 @@
|
|||||||
|
"""SP18 — PII scrubber tests.
|
||||||
|
|
||||||
|
Covers each PHI pattern, the false-positive guard, and the
|
||||||
|
disable toggle.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.logging_config import (
|
||||||
|
PiiScrubber,
|
||||||
|
get_scrubber,
|
||||||
|
setup_logging,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _make_record(msg: str, extras: dict | None = None) -> logging.LogRecord:
|
||||||
|
record = logging.getLogger("test.scrub").makeRecord(
|
||||||
|
name="test.scrub", level=logging.INFO, fn="t.py", lno=1,
|
||||||
|
msg=msg, args=(), exc_info=None,
|
||||||
|
)
|
||||||
|
if extras:
|
||||||
|
for k, v in extras.items():
|
||||||
|
setattr(record, k, v)
|
||||||
|
return record
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _reset_scrubber():
|
||||||
|
"""Make sure the scrubber is enabled + on the root after each test."""
|
||||||
|
yield
|
||||||
|
get_scrubber().enable()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# NPI
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubs_ten_digit_npi_in_message():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("processed claim with npi 1881068062 ok")
|
||||||
|
assert scrubber.filter(rec) is True
|
||||||
|
assert rec.getMessage() == "processed claim with npi <redacted:npi> ok"
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubs_npi_in_extras():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("ok", extras={"provider_npi": "1881068062"})
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.provider_npi == "<redacted:npi>"
|
||||||
|
|
||||||
|
|
||||||
|
def test_does_not_scrub_short_numbers():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("processed 5 claims in 2 batches")
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.getMessage() == "processed 5 claims in 2 batches"
|
||||||
|
|
||||||
|
|
||||||
|
def test_does_not_scrub_eleven_digit_numbers():
|
||||||
|
"""11+ digit numbers aren't NPIs — leave them alone."""
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("control number 12345678901")
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.getMessage() == "control number 12345678901"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SSN
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubs_dashed_ssn_in_message():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("ssn=123-45-6789 detected")
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.getMessage() == "ssn=<redacted:ssn> detected"
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubs_undashed_ssn_at_phrase_boundary():
|
||||||
|
"""A bare 9-digit number is ambiguous — we scrub it when followed
|
||||||
|
by whitespace/punctuation/closing paren/brace/comma (so we don't
|
||||||
|
hit claim control numbers or zip codes mid-sentence)."""
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("ssn 123456789 on file")
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert "<redacted:ssn>" in rec.getMessage()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# DOB
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubs_dob_field():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("dob=1980-04-12 verified")
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.getMessage() == "dob=<redacted:dob> verified"
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubs_dob_in_extras():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("ok", extras={"date_of_birth": "1980-04-12"})
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.date_of_birth == "<redacted:dob>"
|
||||||
|
|
||||||
|
|
||||||
|
def test_does_not_scrub_bare_iso_date():
|
||||||
|
"""A YYYY-MM-DD without a dob= prefix isn't necessarily PHI."""
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("parsed at 2026-06-21")
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.getMessage() == "parsed at 2026-06-21"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Patient name
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubs_patient_name_field():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record('patient_name="John Doe" verified')
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert "John Doe" not in rec.getMessage()
|
||||||
|
assert "<redacted:patient_name>" in rec.getMessage()
|
||||||
|
|
||||||
|
|
||||||
|
def test_does_not_scrub_random_words():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("the parser ran successfully")
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.getMessage() == "the parser ran successfully"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Disable toggle
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubber_disabled_leaves_message_intact():
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
scrubber.disable()
|
||||||
|
rec = _make_record("npi 1881068062 ok")
|
||||||
|
scrubber.filter(rec)
|
||||||
|
assert rec.getMessage() == "npi 1881068062 ok"
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrubber_does_not_crash_on_unusual_records():
|
||||||
|
"""Even with weird attribute combinations the filter returns True."""
|
||||||
|
scrubber = PiiScrubber()
|
||||||
|
rec = _make_record("ok", extras={"weird": object()})
|
||||||
|
assert scrubber.filter(rec) is True
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
"""SP18 — ``setup_logging`` entry-point tests.
|
||||||
|
|
||||||
|
Covers level resolution, handler attachment, env-var overrides, and
|
||||||
|
the idempotent re-setup behavior used by the FastAPI lifespan and
|
||||||
|
the CLI's ``main()``.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.logging_config import (
|
||||||
|
CycloneDevFormatter,
|
||||||
|
JsonFormatter,
|
||||||
|
PiiScrubber,
|
||||||
|
setup_logging,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _reset_root_logger():
|
||||||
|
"""Strip our handlers + filters before each test so setup runs clean."""
|
||||||
|
root = logging.getLogger()
|
||||||
|
for h in list(root.handlers):
|
||||||
|
root.removeHandler(h)
|
||||||
|
for flt in list(root.filters):
|
||||||
|
if isinstance(flt, PiiScrubber):
|
||||||
|
root.removeFilter(flt)
|
||||||
|
yield
|
||||||
|
for h in list(root.handlers):
|
||||||
|
root.removeHandler(h)
|
||||||
|
for flt in list(root.filters):
|
||||||
|
if isinstance(flt, PiiScrubber):
|
||||||
|
root.removeFilter(flt)
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_respects_level_string():
|
||||||
|
setup_logging(level="DEBUG", json_format=True)
|
||||||
|
assert logging.getLogger().level == logging.DEBUG
|
||||||
|
setup_logging(level="WARNING", json_format=True)
|
||||||
|
assert logging.getLogger().level == logging.WARNING
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_attaches_rotating_file_handler(tmp_path: Path):
|
||||||
|
log_file = tmp_path / "cyclone.log"
|
||||||
|
setup_logging(level="INFO", log_file=str(log_file), json_format=True)
|
||||||
|
root = logging.getLogger()
|
||||||
|
assert len(root.handlers) == 1
|
||||||
|
h = root.handlers[0]
|
||||||
|
# RotatingFileHandler has ``baseFilename`` attr.
|
||||||
|
assert hasattr(h, "baseFilename")
|
||||||
|
assert Path(h.baseFilename).name == "cyclone.log"
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_defaults_to_json_formatter():
|
||||||
|
setup_logging(level="INFO")
|
||||||
|
root = logging.getLogger()
|
||||||
|
assert isinstance(root.handlers[0].formatter, JsonFormatter)
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_dev_toggle_uses_dev_formatter():
|
||||||
|
setup_logging(level="INFO", json_format=False)
|
||||||
|
root = logging.getLogger()
|
||||||
|
assert isinstance(root.handlers[0].formatter, CycloneDevFormatter)
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_idempotent_re_setup_replaces_handlers():
|
||||||
|
"""Re-calling setup_logging clears the previous handler(s)."""
|
||||||
|
setup_logging(level="INFO", json_format=True)
|
||||||
|
setup_logging(level="DEBUG", json_format=False)
|
||||||
|
root = logging.getLogger()
|
||||||
|
assert len(root.handlers) == 1
|
||||||
|
assert isinstance(root.handlers[0].formatter, CycloneDevFormatter)
|
||||||
|
assert root.level == logging.DEBUG
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_quietens_noisy_third_party_loggers():
|
||||||
|
setup_logging(level="DEBUG", json_format=True)
|
||||||
|
for noisy in ("urllib3", "paramiko", "sqlalchemy.engine"):
|
||||||
|
assert logging.getLogger(noisy).level >= logging.WARNING
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_attaches_pii_scrubber_by_default():
|
||||||
|
setup_logging(level="INFO", json_format=True)
|
||||||
|
root = logging.getLogger()
|
||||||
|
assert any(isinstance(f, PiiScrubber) for f in root.filters)
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_honors_scrub_pii_false():
|
||||||
|
setup_logging(level="INFO", json_format=True, scrub_pii=False)
|
||||||
|
root = logging.getLogger()
|
||||||
|
# Scrubber is still attached but disabled.
|
||||||
|
scrubbers = [f for f in root.filters if isinstance(f, PiiScrubber)]
|
||||||
|
assert len(scrubbers) == 1
|
||||||
|
assert scrubbers[0]._enabled is False # noqa: SLF001
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_honors_env_var_no_pii_scrub(monkeypatch):
|
||||||
|
monkeypatch.setenv("CYCLONE_LOG_NO_PII_SCRUB", "1")
|
||||||
|
setup_logging(level="INFO", json_format=True)
|
||||||
|
scrubbers = [f for f in logging.getLogger().filters if isinstance(f, PiiScrubber)]
|
||||||
|
assert scrubbers and scrubbers[0]._enabled is False # noqa: SLF001
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_emits_json_to_stderr_by_default(caplog):
|
||||||
|
"""Records emitted after setup flow through JsonFormatter."""
|
||||||
|
setup_logging(level="INFO", json_format=True)
|
||||||
|
logger = logging.getLogger("cyclone.test_setup")
|
||||||
|
logger.info("hello %s", "world", extra={"x": 1})
|
||||||
|
# Cyclone attaches the handler to root, not the named logger; caplog
|
||||||
|
# won't capture unless we propagate (which is the default).
|
||||||
|
# Just assert the handler is on root and would format correctly.
|
||||||
|
root = logging.getLogger()
|
||||||
|
h = root.handlers[0]
|
||||||
|
record = logger.makeRecord(
|
||||||
|
name="cyclone.test_setup", level=logging.INFO, fn="t.py", lno=1,
|
||||||
|
msg="hello %s", args=("world",), exc_info=None,
|
||||||
|
)
|
||||||
|
record.x = 1
|
||||||
|
formatted = h.formatter.format(record)
|
||||||
|
import json as _json
|
||||||
|
parsed = _json.loads(formatted)
|
||||||
|
assert parsed["msg"] == "hello world"
|
||||||
|
assert parsed["extra"]["x"] == 1
|
||||||
@@ -0,0 +1,199 @@
|
|||||||
|
"""Tests for ``cyclone.npi`` — NPI checksum + Tax ID (EIN) format validation.
|
||||||
|
|
||||||
|
Pure local validation, no NPPES calls. Covers the NPPES-published Luhn
|
||||||
|
checksum (with prefix ``80840``) and a small set of obvious EIN typo cases.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.npi import (
|
||||||
|
_luhn_check_digit,
|
||||||
|
is_valid_npi,
|
||||||
|
is_valid_tax_id,
|
||||||
|
npi_checksum,
|
||||||
|
normalize_tax_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Luhn internals
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_luhn_check_digit_known_sequence():
|
||||||
|
"""Standard Luhn for the empty body returns 0.
|
||||||
|
|
||||||
|
With no digits the sum is 0, so (10 - 0 % 10) % 10 = 0.
|
||||||
|
"""
|
||||||
|
assert _luhn_check_digit("") == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_luhn_check_digit_single_digit():
|
||||||
|
"""For a single body digit the check digit is the standard Luhn value.
|
||||||
|
|
||||||
|
With the corrected "double at rightmost" pattern: "0" doubles to 0
|
||||||
|
(total=0, check=0); "1" doubles to 2 (total=2, check=(10-2)%10=8).
|
||||||
|
"""
|
||||||
|
assert _luhn_check_digit("0") == 0
|
||||||
|
assert _luhn_check_digit("1") == 8
|
||||||
|
|
||||||
|
|
||||||
|
def test_luhn_check_digit_nppes_published():
|
||||||
|
"""CMS-published example: body 123456789 → check digit 3.
|
||||||
|
|
||||||
|
Per https://www.cms.gov/.../NPIcheckdigit.pdf the Luhn sum of the
|
||||||
|
prefixed body ``80840123456789`` is 67, so check digit = (10-7)%10 = 3,
|
||||||
|
giving the full NPI ``1234567893``.
|
||||||
|
"""
|
||||||
|
assert _luhn_check_digit("80840123456789") == 3
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# npi_checksum
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_npi_checksum_cms_published_body():
|
||||||
|
"""For NPI body 123456789 the check digit is 3 → NPI 1234567893.
|
||||||
|
|
||||||
|
Confirmed against the CMS-published NPI Luhn example.
|
||||||
|
"""
|
||||||
|
assert npi_checksum("123456789") == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_npi_checksum_rejects_non_digits():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
npi_checksum("12345abc6") # letter in body
|
||||||
|
|
||||||
|
|
||||||
|
def test_npi_checksum_rejects_wrong_length():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
npi_checksum("12345") # only 5 digits
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
npi_checksum("1234567890") # 10 digits — includes the check digit
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# is_valid_npi
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_nppes_sample_is_true():
|
||||||
|
"""The CMS-published example NPI 1234567893 must validate."""
|
||||||
|
assert is_valid_npi("1234567893") is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_off_by_one_is_false():
|
||||||
|
assert is_valid_npi("1234567894") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_all_zeros_is_false():
|
||||||
|
"""All zeros fails the Luhn check."""
|
||||||
|
assert is_valid_npi("0000000000") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_rejects_empty():
|
||||||
|
assert is_valid_npi("") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_rejects_none():
|
||||||
|
assert is_valid_npi(None) is False # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_rejects_non_string():
|
||||||
|
assert is_valid_npi(1234567890) is False # type: ignore[arg-type]
|
||||||
|
assert is_valid_npi(["1881068062"]) is False # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_rejects_short():
|
||||||
|
assert is_valid_npi("123456789") is False # 9 digits
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_rejects_long():
|
||||||
|
assert is_valid_npi("12345678901") is False # 11 digits
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_rejects_non_digits():
|
||||||
|
assert is_valid_npi("188106806X") is False
|
||||||
|
assert is_valid_npi("18810 68062") is False # space
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_npi_all_ones_fails_luhn():
|
||||||
|
"""1111111111 has all-1 sum: alternating double = 1,2,1,2,..., 1+2=3 then
|
||||||
|
collapse. Total for 10 digits body (body=9 of all 1's, sum doubled):
|
||||||
|
Position from right (i=0..8): 1, 1, 1, 1, 1, 1, 1, 1, 1
|
||||||
|
i even (not doubled): 1, 1, 1, 1, 1 → 5
|
||||||
|
i odd (doubled): 1*2=2, 1*2=2, 1*2=2, 1*2=2 → 8
|
||||||
|
Total body = 13. Body alone has check digit (10 - 13 % 10) % 10 = 7.
|
||||||
|
So full NPI 1111111111's body (9 ones) check = 7, and 1 != 7 → invalid.
|
||||||
|
"""
|
||||||
|
assert is_valid_npi("1111111111") is False
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# is_valid_tax_id / normalize_tax_id
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_tax_id_touch_of_care_true():
|
||||||
|
"""The operator's reference EIN — Touch of Care Family Practice."""
|
||||||
|
assert is_valid_tax_id("72-1587149") is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_tax_id_unformatted_true():
|
||||||
|
assert is_valid_tax_id("721587149") is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_tax_id_00_prefix_rejected():
|
||||||
|
"""``00`` is reserved / never assigned by the IRS."""
|
||||||
|
assert is_valid_tax_id("00-1234567") is False
|
||||||
|
assert is_valid_tax_id("001234567") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_tax_id_07_prefix_rejected():
|
||||||
|
"""``07`` is a campus prefix reserved for future use."""
|
||||||
|
assert is_valid_tax_id("07-1234567") is False
|
||||||
|
assert is_valid_tax_id("071234567") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_tax_id_8x_prefix_rejected():
|
||||||
|
"""``80``–``89`` is the IRS Pension Plan Branch — never assigned otherwise."""
|
||||||
|
assert is_valid_tax_id("80-1234567") is False
|
||||||
|
assert is_valid_tax_id("89-1234567") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_tax_id_rejects_malformed():
|
||||||
|
assert is_valid_tax_id("not-an-ein") is False
|
||||||
|
assert is_valid_tax_id("12345") is False
|
||||||
|
assert is_valid_tax_id("1234567890") is False # 10 digits
|
||||||
|
assert is_valid_tax_id("12-345678") is False # 8 digits after hyphen
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_valid_tax_id_rejects_none_and_non_string():
|
||||||
|
assert is_valid_tax_id(None) is False
|
||||||
|
assert is_valid_tax_id(721587149) is False # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_tax_id_returns_plain_form():
|
||||||
|
assert normalize_tax_id("72-1587149") == "721587149"
|
||||||
|
assert normalize_tax_id("721587149") == "721587149"
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_tax_id_strips_whitespace():
|
||||||
|
assert normalize_tax_id(" 72-1587149 ") == "721587149"
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_tax_id_returns_none_for_invalid():
|
||||||
|
assert normalize_tax_id("not-an-ein") is None
|
||||||
|
assert normalize_tax_id(None) is None
|
||||||
|
assert normalize_tax_id("") is None
|
||||||
|
assert normalize_tax_id("12-345678") is None # wrong digit count
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_tax_id_keeps_reserved_prefix():
|
||||||
|
"""normalize_tax_id is a *structural* normalizer — it doesn't reject
|
||||||
|
reserved prefixes. That's ``is_valid_tax_id``'s job. Operators who
|
||||||
|
want to store 00-prefixed EINs as placeholders still get a clean
|
||||||
|
9-digit string."""
|
||||||
|
assert normalize_tax_id("00-1234567") == "001234567"
|
||||||
@@ -0,0 +1,247 @@
|
|||||||
|
"""Tests for the 277CA Claim Acknowledgment parser.
|
||||||
|
|
||||||
|
SP10 T1. The 277CA is the semantic claim-level ack from the payer —
|
||||||
|
the parser must walk the HL hierarchy and capture one ClaimStatus
|
||||||
|
per Patient HL, with REF*1K (payer_claim_control_number) and the
|
||||||
|
STC category code.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from datetime import date
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.parsers.exceptions import CycloneParseError
|
||||||
|
from cyclone.parsers.models_277ca import (
|
||||||
|
ACCEPTED_CODES,
|
||||||
|
PAID_CODES,
|
||||||
|
PENDED_CODES,
|
||||||
|
REJECTED_CODES,
|
||||||
|
ParseResult277CA,
|
||||||
|
classify_status_code,
|
||||||
|
)
|
||||||
|
from cyclone.parsers.parse_277ca import parse_277ca_text
|
||||||
|
|
||||||
|
FIXTURE_DIR = Path(__file__).parent / "fixtures"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Status-code classification
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestClassifyStatusCode:
|
||||||
|
def test_accepted_codes(self):
|
||||||
|
for code in ACCEPTED_CODES:
|
||||||
|
assert classify_status_code(code) == "accepted"
|
||||||
|
|
||||||
|
def test_rejected_codes(self):
|
||||||
|
for code in REJECTED_CODES:
|
||||||
|
assert classify_status_code(code) == "rejected"
|
||||||
|
|
||||||
|
def test_pended_codes(self):
|
||||||
|
for code in PENDED_CODES:
|
||||||
|
assert classify_status_code(code) == "pended"
|
||||||
|
|
||||||
|
def test_paid_codes(self):
|
||||||
|
for code in PAID_CODES:
|
||||||
|
assert classify_status_code(code) == "paid"
|
||||||
|
|
||||||
|
def test_unknown_code_does_not_raise(self):
|
||||||
|
"""Unknown codes (e.g. payer-specific) surface as 'unknown', not error."""
|
||||||
|
assert classify_status_code("ZZ") == "unknown"
|
||||||
|
assert classify_status_code("") == "unknown"
|
||||||
|
assert classify_status_code(" ") == "unknown"
|
||||||
|
|
||||||
|
def test_case_insensitive(self):
|
||||||
|
assert classify_status_code("a3") == "accepted"
|
||||||
|
assert classify_status_code("A6") == "rejected"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Parser: minimal happy path with mixed statuses
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestParse277CAMixed:
|
||||||
|
@pytest.fixture
|
||||||
|
def parsed(self) -> ParseResult277CA:
|
||||||
|
text = (FIXTURE_DIR / "minimal_277ca.txt").read_text()
|
||||||
|
return parse_277ca_text(text, input_file="minimal_277ca.txt")
|
||||||
|
|
||||||
|
def test_envelope_control_number(self, parsed):
|
||||||
|
assert parsed.envelope.control_number == "000000123"
|
||||||
|
assert parsed.envelope.implementation_guide == "005010X214"
|
||||||
|
|
||||||
|
def test_bht_captures_reference_and_date(self, parsed):
|
||||||
|
assert parsed.bht.hierarchical_structure_code == "0085"
|
||||||
|
assert parsed.bht.transaction_set_purpose_code == "08"
|
||||||
|
assert parsed.bht.reference_identification == "REFNUM001"
|
||||||
|
assert parsed.bht.transaction_set_creation_date == date(2024, 6, 20)
|
||||||
|
assert parsed.bht.transaction_type_code == "TH"
|
||||||
|
|
||||||
|
def test_three_patient_hls_three_statuses(self, parsed):
|
||||||
|
assert len(parsed.claim_statuses) == 3
|
||||||
|
|
||||||
|
def test_first_status_is_accepted(self, parsed):
|
||||||
|
s = parsed.claim_statuses[0]
|
||||||
|
assert s.payer_claim_control_number == "CLAIM001"
|
||||||
|
assert s.billing_provider_tax_id == "721587149"
|
||||||
|
assert s.status_code == "A3"
|
||||||
|
assert s.classification == "accepted"
|
||||||
|
assert s.total_claim_charge_amount == 100.00
|
||||||
|
|
||||||
|
def test_second_status_is_rejected(self, parsed):
|
||||||
|
s = parsed.claim_statuses[1]
|
||||||
|
assert s.payer_claim_control_number == "CLAIM002"
|
||||||
|
assert s.status_code == "A6"
|
||||||
|
assert s.classification == "rejected"
|
||||||
|
assert s.total_claim_charge_amount == 250.00
|
||||||
|
|
||||||
|
def test_third_status_is_pended(self, parsed):
|
||||||
|
s = parsed.claim_statuses[2]
|
||||||
|
assert s.payer_claim_control_number == "CLAIM003"
|
||||||
|
assert s.status_code == "A8"
|
||||||
|
assert s.classification == "pended"
|
||||||
|
|
||||||
|
def test_service_date_parsed_from_dtp_472(self, parsed):
|
||||||
|
"""DTP*472 with RD8 fmt should return the start date."""
|
||||||
|
s = parsed.claim_statuses[0]
|
||||||
|
assert s.service_date == date(2024, 6, 15)
|
||||||
|
|
||||||
|
def test_summary_counts(self, parsed):
|
||||||
|
# accepted=1 (A3), rejected=1 (A6), pended=1 (A8)
|
||||||
|
assert parsed.summary.total_claims == 3
|
||||||
|
assert parsed.summary.passed == 1 # only "accepted"
|
||||||
|
assert parsed.summary.failed == 1 # only "rejected" (pended excluded)
|
||||||
|
|
||||||
|
def test_round_trips_via_json(self, parsed):
|
||||||
|
blob = json.loads(parsed.model_dump_json())
|
||||||
|
assert blob["envelope"]["control_number"] == "000000123"
|
||||||
|
assert len(blob["claim_statuses"]) == 3
|
||||||
|
rebuilt = ParseResult277CA.model_validate(blob)
|
||||||
|
assert rebuilt.claim_statuses[1].status_code == "A6"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Parser: ST*277 (instead of ST*277CA) — X12 spec also allows it
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestParse277CAAltST:
|
||||||
|
def test_st_277_accepted(self):
|
||||||
|
text = (FIXTURE_DIR / "minimal_277ca_st277.txt").read_text()
|
||||||
|
result = parse_277ca_text(text, input_file="minimal_277ca_st277.txt")
|
||||||
|
assert len(result.claim_statuses) == 1
|
||||||
|
assert result.claim_statuses[0].classification == "accepted"
|
||||||
|
|
||||||
|
def test_st_wrong_value_rejected(self):
|
||||||
|
"""A non-277 ST segment should raise CycloneParseError."""
|
||||||
|
text = (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X214~"
|
||||||
|
"ST*835*0001*005010X221A1~"
|
||||||
|
"BHT*0085*08*X*20240620*1200*TH~"
|
||||||
|
"SE*3*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
|
with pytest.raises(CycloneParseError, match="ST\\*277"):
|
||||||
|
parse_277ca_text(text)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Parser: rejected-only fixture (single A7)
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestParse277CARejectedOnly:
|
||||||
|
def test_single_a7_rejected(self):
|
||||||
|
text = (FIXTURE_DIR / "minimal_277ca_rejected_only.txt").read_text()
|
||||||
|
result = parse_277ca_text(text, input_file="minimal_277ca_rejected_only.txt")
|
||||||
|
assert len(result.claim_statuses) == 1
|
||||||
|
s = result.claim_statuses[0]
|
||||||
|
assert s.status_code == "A7"
|
||||||
|
assert s.classification == "rejected"
|
||||||
|
assert s.payer_claim_control_number == "CLAIM099"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Parser: error handling
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestParse277CAErrors:
|
||||||
|
def test_missing_isa_raises(self):
|
||||||
|
"""No ISA envelope → CycloneParseError, never silent fail."""
|
||||||
|
with pytest.raises(CycloneParseError, match="ISA"):
|
||||||
|
parse_277ca_text("not a valid 277ca\n")
|
||||||
|
|
||||||
|
def test_empty_input_raises(self):
|
||||||
|
with pytest.raises(CycloneParseError):
|
||||||
|
parse_277ca_text("")
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Parser: multiple STC per patient (last wins)
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestParse277CAMultipleStcPerPatient:
|
||||||
|
def test_last_stc_wins_per_patient(self):
|
||||||
|
"""A patient HL with multiple STC segments: the last one wins.
|
||||||
|
|
||||||
|
This is the canonical pattern for "first pended, then paid"
|
||||||
|
acknowledgments — the most recent action is authoritative.
|
||||||
|
"""
|
||||||
|
text = (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X214~"
|
||||||
|
"ST*277CA*0001*005010X214~"
|
||||||
|
"BHT*0085*08*X*20240620*1200*TH~"
|
||||||
|
"HL*1**20*1~"
|
||||||
|
"HL*2*1*21*1~"
|
||||||
|
"HL*3*2*19*1~"
|
||||||
|
"HL*4*3*PT~"
|
||||||
|
"REF*1K*CLAIM555~"
|
||||||
|
"STC*A1:19:PR*20240620*WQ*100.00~"
|
||||||
|
"STC*A8:19:PR*20240620*WQ*100.00~"
|
||||||
|
"STC*A3:19:PR*20240620*WQ*100.00~"
|
||||||
|
"SE*9*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
|
result = parse_277ca_text(text)
|
||||||
|
assert len(result.claim_statuses) == 1
|
||||||
|
assert result.claim_statuses[0].status_code == "A3"
|
||||||
|
assert result.claim_statuses[0].classification == "accepted"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Parser: subscriber-level STCs surface in unscoped_statuses
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestParse277CASubscriberLevelStc:
|
||||||
|
def test_subscriber_stc_goes_to_unscoped(self):
|
||||||
|
"""Subscriber-level (HL*19) STCs without a Patient child go to unscoped_statuses."""
|
||||||
|
text = (
|
||||||
|
"ISA*00* *00* *ZZ*AAAAAAAAAAAAAAA*ZZ*BBBBBBBBBBBBBBB*240620*1200*^*00501*000000001*0*P*:~"
|
||||||
|
"GS*HN*A*B*20240620*1200*1*X*005010X214~"
|
||||||
|
"ST*277CA*0001*005010X214~"
|
||||||
|
"BHT*0085*08*X*20240620*1200*TH~"
|
||||||
|
"HL*1**20*1~"
|
||||||
|
"HL*2*1*21*1~"
|
||||||
|
"HL*3*2*19*1~"
|
||||||
|
"STC*A6:19:PR*20240620*U~"
|
||||||
|
"SE*7*0001~"
|
||||||
|
"GE*1*1~"
|
||||||
|
"IEA*1*000000001~"
|
||||||
|
)
|
||||||
|
result = parse_277ca_text(text)
|
||||||
|
assert len(result.claim_statuses) == 0
|
||||||
|
assert len(result.unscoped_statuses) == 1
|
||||||
|
assert result.unscoped_statuses[0].status_code == "A6"
|
||||||
|
assert result.unscoped_statuses[0].classification == "rejected"
|
||||||
@@ -14,7 +14,7 @@ def test_parse_minimal_fixture_returns_one_claim():
|
|||||||
assert len(result.claims) == 1
|
assert len(result.claims) == 1
|
||||||
claim = result.claims[0]
|
claim = result.claims[0]
|
||||||
assert claim.claim_id == "CLM001"
|
assert claim.claim_id == "CLM001"
|
||||||
assert claim.billing_provider.npi == "1234567890"
|
assert claim.billing_provider.npi == "1993999998"
|
||||||
assert claim.subscriber.last_name == "Doe"
|
assert claim.subscriber.last_name == "Doe"
|
||||||
assert claim.subscriber.first_name == "John"
|
assert claim.subscriber.first_name == "John"
|
||||||
assert claim.subscriber.member_id == "ABC123"
|
assert claim.subscriber.member_id == "ABC123"
|
||||||
|
|||||||
@@ -0,0 +1,104 @@
|
|||||||
|
"""Tests for PayerConfig277CA loading from YAML.
|
||||||
|
|
||||||
|
SP10 T4. Mirrors ``test_payer_config_loading.py`` but for the new
|
||||||
|
``PayerConfig277CA`` block that the 277CA parser and Inbox lane
|
||||||
|
consume for status-code classification.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
from cyclone.payers import (
|
||||||
|
DEFAULT_CONFIG_PATH,
|
||||||
|
_tx_to_model,
|
||||||
|
all_configs,
|
||||||
|
get_config,
|
||||||
|
load_payer_configs,
|
||||||
|
reset,
|
||||||
|
)
|
||||||
|
from cyclone.providers import PayerConfig277CA
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _clear_registry():
|
||||||
|
"""Each test starts with an empty config registry."""
|
||||||
|
reset()
|
||||||
|
yield
|
||||||
|
reset()
|
||||||
|
|
||||||
|
|
||||||
|
class TestPayerConfig277CALoading:
|
||||||
|
def test_loads_default_config_with_277ca_block(self):
|
||||||
|
"""The default config/payers.yaml must have a CO_TXIX 277CA block."""
|
||||||
|
load_payer_configs(DEFAULT_CONFIG_PATH)
|
||||||
|
cfg = get_config("CO_TXIX", "277CA")
|
||||||
|
assert cfg is not None
|
||||||
|
# Defaults from PayerConfig277CA.
|
||||||
|
assert "A6" in cfg["rejected_status_codes"]
|
||||||
|
assert "A8" in cfg["pended_status_codes"]
|
||||||
|
assert "A3" in cfg["accepted_status_codes"]
|
||||||
|
assert "P1" in cfg["paid_status_codes"]
|
||||||
|
assert "277CA" in cfg["transaction_set_ids_allowed"]
|
||||||
|
|
||||||
|
def test_model_rejects_bad_status_code_type(self, tmp_path: Path):
|
||||||
|
"""A non-list rejected_status_codes value must fail validation."""
|
||||||
|
bad = {
|
||||||
|
"payers": [
|
||||||
|
{
|
||||||
|
"payer_id": "BAD_PAYER",
|
||||||
|
"name": "Bad",
|
||||||
|
"receiver_name": "BAD",
|
||||||
|
"receiver_id": "BADID",
|
||||||
|
"configs": {
|
||||||
|
"277CA": {
|
||||||
|
"rejected_status_codes": "A6", # must be list
|
||||||
|
"pended_status_codes": ["A8"],
|
||||||
|
"accepted_status_codes": ["A1"],
|
||||||
|
"paid_status_codes": ["P1"],
|
||||||
|
"transaction_set_ids_allowed": ["277", "277CA"],
|
||||||
|
"implementation_guide": "005010X214",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
p = tmp_path / "bad.yaml"
|
||||||
|
p.write_text(yaml.safe_dump(bad))
|
||||||
|
with pytest.raises(ValueError, match="BAD_PAYER"):
|
||||||
|
load_payer_configs(p)
|
||||||
|
|
||||||
|
|
||||||
|
class TestPayerConfig277CAModel:
|
||||||
|
def test_default_factory_values(self):
|
||||||
|
"""A minimal PayerConfig277CA defaults to the canonical HCPF sets."""
|
||||||
|
cfg = PayerConfig277CA()
|
||||||
|
assert "A4" in cfg.rejected_status_codes
|
||||||
|
assert "A6" in cfg.rejected_status_codes
|
||||||
|
assert "A7" in cfg.rejected_status_codes
|
||||||
|
assert "A8" in cfg.pended_status_codes
|
||||||
|
assert "277CA" in cfg.transaction_set_ids_allowed
|
||||||
|
|
||||||
|
def test_explicit_override(self):
|
||||||
|
cfg = PayerConfig277CA(
|
||||||
|
rejected_status_codes=["X1"],
|
||||||
|
pended_status_codes=["X2"],
|
||||||
|
accepted_status_codes=["X3"],
|
||||||
|
paid_status_codes=["X4"],
|
||||||
|
)
|
||||||
|
assert cfg.rejected_status_codes == ["X1"]
|
||||||
|
assert cfg.pended_status_codes == ["X2"]
|
||||||
|
|
||||||
|
|
||||||
|
class TestTxToModelFor277CA:
|
||||||
|
def test_int_key_routes_to_277ca_model(self):
|
||||||
|
"""PyYAML may parse numeric keys as int — _tx_to_model must handle both."""
|
||||||
|
# 277CA as int isn't likely (not pure digits), but 277 as int is.
|
||||||
|
# Test the canonical string.
|
||||||
|
assert _tx_to_model("277CA") is PayerConfig277CA
|
||||||
|
|
||||||
|
def test_string_key_routes_correctly(self):
|
||||||
|
assert _tx_to_model("277CA") is PayerConfig277CA
|
||||||
|
assert _tx_to_model("835") is not PayerConfig277CA # it's 835 model
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
"""SP9 — payer config YAML loader tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
from cyclone import payers
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _reset_registry():
|
||||||
|
payers.reset()
|
||||||
|
yield
|
||||||
|
payers.reset()
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_default_config_has_co_txix_837p():
|
||||||
|
configs = payers.load_payer_configs()
|
||||||
|
assert ("CO_TXIX", "837P") in configs
|
||||||
|
block = configs[("CO_TXIX", "837P")]
|
||||||
|
assert block["payer_id"] == "CO_TXIX"
|
||||||
|
assert "CH" in block["bht06_allowed"]
|
||||||
|
assert "RP" in block["bht06_allowed"]
|
||||||
|
assert block["bht06_default"] == "CH"
|
||||||
|
assert block["sbr09_default"] == "MC"
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_default_config_has_co_txix_835():
|
||||||
|
configs = payers.load_payer_configs()
|
||||||
|
assert ("CO_TXIX", "835") in configs
|
||||||
|
block = configs[("CO_TXIX", "835")]
|
||||||
|
assert "1811725341" in block["expected_payer_tax_ids"]
|
||||||
|
assert block["expected_payer_health_plan_id"] == "7912900843"
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_config_returns_block():
|
||||||
|
payers.load_payer_configs()
|
||||||
|
block = payers.get_config("CO_TXIX", "837P")
|
||||||
|
assert block is not None
|
||||||
|
assert block["submitter_name"] == "Dzinesco"
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_config_returns_none_for_missing():
|
||||||
|
payers.load_payer_configs()
|
||||||
|
assert payers.get_config("UNKNOWN_PAYER", "837P") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_invalid_yaml_raises(tmp_path):
|
||||||
|
bad = tmp_path / "bad.yaml"
|
||||||
|
bad.write_text(yaml.safe_dump({"payers": [
|
||||||
|
{"payer_id": "X", "configs": {"837P": {"submitter_name": "ok"}}} # missing required keys
|
||||||
|
]}))
|
||||||
|
with pytest.raises(ValueError, match="invalid config"):
|
||||||
|
payers.load_payer_configs(bad)
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_missing_payers_key_raises(tmp_path):
|
||||||
|
bad = tmp_path / "bad.yaml"
|
||||||
|
bad.write_text("not_a_payers_key: 1")
|
||||||
|
with pytest.raises(ValueError, match="missing top-level 'payers:'"):
|
||||||
|
payers.load_payer_configs(bad)
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_missing_file_warns_and_clears(tmp_path, caplog):
|
||||||
|
fake = tmp_path / "does-not-exist.yaml"
|
||||||
|
with caplog.at_level("WARNING"):
|
||||||
|
result = payers.load_payer_configs(fake)
|
||||||
|
assert result == {}
|
||||||
|
|
||||||
|
|
||||||
|
def test_all_configs_returns_snapshot():
|
||||||
|
payers.load_payer_configs()
|
||||||
|
snap = payers.all_configs()
|
||||||
|
assert isinstance(snap, dict)
|
||||||
|
assert len(snap) >= 2
|
||||||
|
# Mutating snapshot must not affect registry
|
||||||
|
snap.clear()
|
||||||
|
assert len(payers.all_configs()) >= 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_reload_picks_up_changes(tmp_path):
|
||||||
|
a = tmp_path / "a.yaml"
|
||||||
|
a.write_text(yaml.safe_dump({"payers": [
|
||||||
|
{"payer_id": "AAA", "configs": {"837P": {
|
||||||
|
"submitter_name": "A", "submitter_contact_name": "A",
|
||||||
|
"submitter_contact_email": "a@a",
|
||||||
|
"receiver_name": "R", "receiver_id": "R",
|
||||||
|
"bht06_allowed": ["CH"], "bht06_default": "CH",
|
||||||
|
"sbr09_default": "MC", "sbr09_allowed": ["MC"],
|
||||||
|
"payer_id_qualifier": "PI", "payer_id": "AAA",
|
||||||
|
"pwk_supported": False, "cas_2320_group_allowed": False,
|
||||||
|
}}}
|
||||||
|
]}))
|
||||||
|
b = tmp_path / "b.yaml"
|
||||||
|
b.write_text(yaml.safe_dump({"payers": [
|
||||||
|
{"payer_id": "BBB", "configs": {"837P": {
|
||||||
|
"submitter_name": "B", "submitter_contact_name": "B",
|
||||||
|
"submitter_contact_email": "b@b",
|
||||||
|
"receiver_name": "R", "receiver_id": "R",
|
||||||
|
"bht06_allowed": ["CH"], "bht06_default": "CH",
|
||||||
|
"sbr09_default": "MC", "sbr09_allowed": ["MC"],
|
||||||
|
"payer_id_qualifier": "PI", "payer_id": "BBB",
|
||||||
|
"pwk_supported": False, "cas_2320_group_allowed": False,
|
||||||
|
}}}
|
||||||
|
]}))
|
||||||
|
payers.load_payer_configs(a)
|
||||||
|
assert payers.get_config("AAA", "837P") is not None
|
||||||
|
payers.load_payer_configs(b)
|
||||||
|
assert payers.get_config("BBB", "837P") is not None
|
||||||
|
assert payers.get_config("AAA", "837P") is None
|
||||||
@@ -0,0 +1,168 @@
|
|||||||
|
"""SP14 — Payer-Rejected acknowledge endpoint tests.
|
||||||
|
|
||||||
|
The endpoint marks payer-rejected claims as acknowledged so the
|
||||||
|
working-surface lane query filters them out. The original
|
||||||
|
payer_rejected_* fields stay intact (SP11 audit).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import db
|
||||||
|
from cyclone.db import Claim, ClaimState
|
||||||
|
from cyclone.audit_log import verify_chain
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _setup(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db._reset_for_tests()
|
||||||
|
db.init_db()
|
||||||
|
|
||||||
|
|
||||||
|
def _make_claim(claim_id: str, *, payer_rejected: bool = True) -> None:
|
||||||
|
"""Insert a minimal claim with optional payer_rejected markers."""
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from cyclone.db import Batch
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
batch = Batch(
|
||||||
|
id=f"b-{claim_id}",
|
||||||
|
kind="837p",
|
||||||
|
input_filename="t.x12",
|
||||||
|
parsed_at=datetime.now(timezone.utc),
|
||||||
|
)
|
||||||
|
session.add(batch)
|
||||||
|
session.flush()
|
||||||
|
claim = Claim(
|
||||||
|
id=claim_id,
|
||||||
|
batch_id=batch.id,
|
||||||
|
patient_control_number=claim_id,
|
||||||
|
state=ClaimState.SUBMITTED,
|
||||||
|
charge_amount=100,
|
||||||
|
)
|
||||||
|
if payer_rejected:
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
claim.payer_rejected_at = now
|
||||||
|
claim.payer_rejected_reason = "invalid diagnosis code"
|
||||||
|
claim.payer_rejected_status_code = "A7"
|
||||||
|
session.add(claim)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def test_acknowledge_marks_unacknowledged_claim():
|
||||||
|
_make_claim("C1", payer_rejected=True)
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post(
|
||||||
|
"/api/inbox/payer-rejected/acknowledge",
|
||||||
|
json={"claim_ids": ["C1"], "actor": "operator"},
|
||||||
|
)
|
||||||
|
assert r.status_code == 200, r.text
|
||||||
|
body = r.json()
|
||||||
|
assert body["ok"] is True
|
||||||
|
assert body["transitioned"] == 1
|
||||||
|
assert body["already_acked"] == 0
|
||||||
|
assert body["not_found"] == 0
|
||||||
|
assert body["not_rejected"] == 0
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
c = session.get(Claim, "C1")
|
||||||
|
assert c.payer_rejected_acknowledged_at is not None
|
||||||
|
assert c.payer_rejected_acknowledged_actor == "operator"
|
||||||
|
# Original fields stay intact for audit.
|
||||||
|
assert c.payer_rejected_at is not None
|
||||||
|
assert c.payer_rejected_status_code == "A7"
|
||||||
|
assert c.payer_rejected_reason == "invalid diagnosis code"
|
||||||
|
|
||||||
|
|
||||||
|
def test_acknowledge_is_idempotent():
|
||||||
|
_make_claim("C1", payer_rejected=True)
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r1 = client.post(
|
||||||
|
"/api/inbox/payer-rejected/acknowledge",
|
||||||
|
json={"claim_ids": ["C1"]},
|
||||||
|
)
|
||||||
|
assert r1.json()["transitioned"] == 1
|
||||||
|
r2 = client.post(
|
||||||
|
"/api/inbox/payer-rejected/acknowledge",
|
||||||
|
json={"claim_ids": ["C1"]},
|
||||||
|
)
|
||||||
|
assert r2.json()["transitioned"] == 0
|
||||||
|
assert r2.json()["already_acked"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_acknowledge_skips_non_payer_rejected_claims():
|
||||||
|
_make_claim("C1", payer_rejected=False)
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post(
|
||||||
|
"/api/inbox/payer-rejected/acknowledge",
|
||||||
|
json={"claim_ids": ["C1"]},
|
||||||
|
)
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["transitioned"] == 0
|
||||||
|
assert body["not_rejected"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_acknowledge_counts_missing_ids():
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post(
|
||||||
|
"/api/inbox/payer-rejected/acknowledge",
|
||||||
|
json={"claim_ids": ["nope"]},
|
||||||
|
)
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["transitioned"] == 0
|
||||||
|
assert body["not_found"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_acknowledge_empty_claim_ids_400():
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post(
|
||||||
|
"/api/inbox/payer-rejected/acknowledge",
|
||||||
|
json={"claim_ids": []},
|
||||||
|
)
|
||||||
|
assert r.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
def test_acknowledge_writes_audit_event():
|
||||||
|
_make_claim("C1", payer_rejected=True)
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from cyclone.api import app
|
||||||
|
with TestClient(app) as client:
|
||||||
|
r = client.post(
|
||||||
|
"/api/inbox/payer-rejected/acknowledge",
|
||||||
|
json={"claim_ids": ["C1"], "actor": "alice"},
|
||||||
|
)
|
||||||
|
assert r.status_code == 200
|
||||||
|
|
||||||
|
# SP11: the chain is intact and includes the new event.
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
result = verify_chain(session)
|
||||||
|
assert result.ok, f"chain broken: {result.reason} at {result.first_bad_id}"
|
||||||
|
|
||||||
|
# And the event itself is queryable.
|
||||||
|
import json as _json
|
||||||
|
from cyclone.db import AuditLog
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
events = (
|
||||||
|
session.query(AuditLog)
|
||||||
|
.filter(AuditLog.event_type == "claim.payer_rejected_acknowledged")
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
assert len(events) == 1
|
||||||
|
e = events[0]
|
||||||
|
assert e.entity_type == "claim"
|
||||||
|
assert e.entity_id == "C1"
|
||||||
|
assert e.actor == "alice"
|
||||||
|
payload = _json.loads(e.payload_json) if e.payload_json else {}
|
||||||
|
assert payload["payer_rejected_status_code"] == "A7"
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
"""Tests for GET /api/payers/{payer_id}/summary (SP21 Task 1.5).
|
||||||
|
|
||||||
|
The endpoint is the payer-level aggregate that the drill-down UI's
|
||||||
|
"Payer → Claims" panel hangs off. It returns billed/received totals,
|
||||||
|
denial rate, and the top 5 NPIs by claim volume for one payer_id,
|
||||||
|
cached in-process for 60s.
|
||||||
|
|
||||||
|
The minimal 837P fixture ships one CLM with ``payer_id="SKCO0"``,
|
||||||
|
charge_amount=100.00; the minimal 835 carries one CLP for the same
|
||||||
|
claim with total_paid=85.00. So ``/api/payers/SKCO0/summary`` returns
|
||||||
|
``claim_count >= 1`` after both files are ingested.
|
||||||
|
|
||||||
|
Note: the spec calls this ``payer_id`` (the X12 NM1*PR*PI qualifier,
|
||||||
|
e.g. ``SKCO0``). It is NOT the configured payer name from
|
||||||
|
``config/payers.yaml``. The filter key in the store layer is
|
||||||
|
``Claim.payer_id`` — not the ``payer=`` substring filter used by
|
||||||
|
``/api/claims?payer=...``.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone import api as api_mod
|
||||||
|
from cyclone.api import app
|
||||||
|
|
||||||
|
FIXTURE_837 = Path(__file__).parent / "fixtures" / "minimal_837p.txt"
|
||||||
|
FIXTURE_835 = Path(__file__).parent / "fixtures" / "minimal_835.txt"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _clear_summary_cache():
|
||||||
|
"""Wipe the in-process payer-summary cache between tests.
|
||||||
|
|
||||||
|
conftest resets the DB per test but the cache is module-level
|
||||||
|
state on ``cyclone.api``. Without this clear, a stale payload
|
||||||
|
from a previous test's seed would leak into a later test's first
|
||||||
|
call — masking recompute behavior. The 60s TTL is the only
|
||||||
|
invalidation story today (see api.py docstring on the endpoint).
|
||||||
|
"""
|
||||||
|
api_mod._clear_summary_cache()
|
||||||
|
yield
|
||||||
|
api_mod._clear_summary_cache()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client() -> TestClient:
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def seeded_db(client: TestClient):
|
||||||
|
"""Ingest one minimal 837P + one minimal 835.
|
||||||
|
|
||||||
|
Both fixtures carry ``payer_id="SKCO0"`` so the summary endpoint
|
||||||
|
has something to aggregate. ``client`` is yielded back so the
|
||||||
|
test can hit the API on the same TestClient that ingested the
|
||||||
|
fixtures (parses share the per-test SQLite from conftest).
|
||||||
|
"""
|
||||||
|
text_837 = FIXTURE_837.read_text()
|
||||||
|
text_835 = FIXTURE_835.read_text()
|
||||||
|
r837 = client.post(
|
||||||
|
"/api/parse-837",
|
||||||
|
files={"file": ("x.txt", text_837, "text/plain")},
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
)
|
||||||
|
assert r837.status_code == 200, r837.text
|
||||||
|
r835 = client.post(
|
||||||
|
"/api/parse-835",
|
||||||
|
files={"file": ("era.txt", text_835, "text/plain")},
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
)
|
||||||
|
assert r835.status_code == 200, r835.text
|
||||||
|
return client
|
||||||
|
|
||||||
|
|
||||||
|
def test_payer_summary_happy_path(seeded_db: TestClient):
|
||||||
|
"""Seeded db has at least one claim for SKCO0 → 200 with the spec shape."""
|
||||||
|
resp = seeded_db.get("/api/payers/SKCO0/summary")
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
data = resp.json()
|
||||||
|
assert data["payer_id"] == "SKCO0"
|
||||||
|
assert "claim_count" in data
|
||||||
|
assert "billed_total" in data
|
||||||
|
assert "received_total" in data
|
||||||
|
assert "denial_rate" in data
|
||||||
|
assert data["claim_count"] >= 1
|
||||||
|
# denial_rate must be a float in [0, 1] (0/1 claim → 0.0).
|
||||||
|
assert isinstance(data["denial_rate"], (int, float))
|
||||||
|
assert 0.0 <= data["denial_rate"] <= 1.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_payer_summary_unknown_payer_returns_404(client: TestClient):
|
||||||
|
resp = client.get("/api/payers/DOES_NOT_EXIST/summary")
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_payer_summary_caches_then_invalidates(seeded_db: TestClient):
|
||||||
|
"""Two back-to-back calls return identical payloads (in-process cache)."""
|
||||||
|
resp1 = seeded_db.get("/api/payers/SKCO0/summary")
|
||||||
|
resp2 = seeded_db.get("/api/payers/SKCO0/summary")
|
||||||
|
assert resp1.status_code == 200
|
||||||
|
assert resp2.status_code == 200
|
||||||
|
assert resp1.json() == resp2.json()
|
||||||
@@ -0,0 +1,177 @@
|
|||||||
|
"""Tests for the extended GET /api/config/providers/{npi} (SP21 Task 1.6).
|
||||||
|
|
||||||
|
The endpoint gains two new top-level arrays for the drill-down panel:
|
||||||
|
``recent_claims`` (top 10 by submission date desc) and ``recent_activity``
|
||||||
|
(top 10 by ``ts`` desc, joined to claims by ``claim_id`` because
|
||||||
|
``ActivityEvent`` has no direct ``provider_npi`` column).
|
||||||
|
|
||||||
|
Existing SP9 fields (``label``, ``legal_name``, ``tax_id``,
|
||||||
|
``address_line1``, ``city``, ``state``, ``zip``, etc.) must remain
|
||||||
|
present — the new arrays are additive only.
|
||||||
|
|
||||||
|
The fixtures in ``fixtures/minimal_837p.txt`` and ``fixtures/minimal_835.txt``
|
||||||
|
pair up to a single claim with ``provider_npi='1993999998'``. That NPI is
|
||||||
|
NOT in the seeded provider set (Montrose/Delta/Salida → 1881068062/1851446637/
|
||||||
|
1467507269). For the tests we hit the seeded Montrose NPI, which is the
|
||||||
|
canonical SP9 fixture NPI. The arrays come back as empty lists — the
|
||||||
|
contract under test is the *shape* (array, ≤10) and the *backwards compat*
|
||||||
|
of the existing fields; the data-path itself is exercised by the existing
|
||||||
|
ingestion path that backs ``/api/claims``.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone.api import app
|
||||||
|
from cyclone.providers import Provider
|
||||||
|
from cyclone.store import store
|
||||||
|
|
||||||
|
FIXTURE_837 = Path(__file__).parent / "fixtures" / "minimal_837p.txt"
|
||||||
|
FIXTURE_835 = Path(__file__).parent / "fixtures" / "minimal_835.txt"
|
||||||
|
|
||||||
|
# Montrose — one of the three providers that `ensure_clearhouse_seeded()`
|
||||||
|
# writes into the providers table. Using a seeded NPI means the endpoint
|
||||||
|
# won't 404; the seeded claims (provider_npi='1993999998') won't appear
|
||||||
|
# under this NPI, so recent_claims/activity are expected empty lists.
|
||||||
|
MONTROSE_NPI = "1881068062"
|
||||||
|
|
||||||
|
# NPI the minimal 837P fixture bills under. NOT in the default seed —
|
||||||
|
# registering it before ingest is required to pass R204 (NPI must exist
|
||||||
|
# in the providers table).
|
||||||
|
TEST_837_NPI = "1993999998"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client() -> TestClient:
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def seeded_db(client: TestClient):
|
||||||
|
"""Seed the clearhouse + ingest the minimal 837P/835 fixtures.
|
||||||
|
|
||||||
|
Mirrors the Task 1.5 ``seeded_db`` pattern: seed → ingest → hand the
|
||||||
|
client back so the test hits the same TestClient.
|
||||||
|
|
||||||
|
The 837 fixture bills under ``TEST_837_NPI``; without first registering
|
||||||
|
that provider the parser's R204 rule rejects the claim with HTTP 422.
|
||||||
|
"""
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
test_provider = Provider(
|
||||||
|
npi=TEST_837_NPI,
|
||||||
|
label="Test Provider",
|
||||||
|
legal_name="Test Provider Inc",
|
||||||
|
tax_id="123456789",
|
||||||
|
taxonomy_code="207R00000X",
|
||||||
|
address_line1="123 Test St",
|
||||||
|
city="Denver",
|
||||||
|
state="CO",
|
||||||
|
zip="80202",
|
||||||
|
created_at=datetime.now(timezone.utc),
|
||||||
|
updated_at=datetime.now(timezone.utc),
|
||||||
|
)
|
||||||
|
store.upsert_provider(test_provider)
|
||||||
|
text_837 = FIXTURE_837.read_text()
|
||||||
|
text_835 = FIXTURE_835.read_text()
|
||||||
|
r837 = client.post(
|
||||||
|
"/api/parse-837",
|
||||||
|
files={"file": ("x.txt", text_837, "text/plain")},
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
)
|
||||||
|
assert r837.status_code == 200, r837.text
|
||||||
|
r835 = client.post(
|
||||||
|
"/api/parse-835",
|
||||||
|
files={"file": ("era.txt", text_835, "text/plain")},
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
)
|
||||||
|
assert r835.status_code == 200, r835.text
|
||||||
|
return client
|
||||||
|
|
||||||
|
|
||||||
|
def test_provider_detail_includes_recent_claims(seeded_db: TestClient):
|
||||||
|
"""The extended response gains a recent_claims array (top 10)."""
|
||||||
|
resp = seeded_db.get(f"/api/config/providers/{MONTROSE_NPI}")
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
data = resp.json()
|
||||||
|
assert "recent_claims" in data
|
||||||
|
assert isinstance(data["recent_claims"], list)
|
||||||
|
assert len(data["recent_claims"]) <= 10
|
||||||
|
|
||||||
|
|
||||||
|
def test_provider_detail_includes_recent_activity(seeded_db: TestClient):
|
||||||
|
"""The extended response gains a recent_activity array (top 10)."""
|
||||||
|
resp = seeded_db.get(f"/api/config/providers/{MONTROSE_NPI}")
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
data = resp.json()
|
||||||
|
assert "recent_activity" in data
|
||||||
|
assert isinstance(data["recent_activity"], list)
|
||||||
|
assert len(data["recent_activity"]) <= 10
|
||||||
|
|
||||||
|
|
||||||
|
def test_provider_detail_backwards_compat(seeded_db: TestClient):
|
||||||
|
"""All SP9 fields still present; new arrays don't break the contract.
|
||||||
|
|
||||||
|
The Provider Pydantic model (backend/src/cyclone/providers.py)
|
||||||
|
serializes snake_case fields — that's what the wire carries. The
|
||||||
|
TS ``Provider`` interface in ``src/types/index.ts`` is the
|
||||||
|
in-memory sample shape and intentionally diverges; the contract
|
||||||
|
being verified here is the API's actual payload.
|
||||||
|
"""
|
||||||
|
resp = seeded_db.get(f"/api/config/providers/{MONTROSE_NPI}")
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
data = resp.json()
|
||||||
|
for key in (
|
||||||
|
"npi",
|
||||||
|
"label",
|
||||||
|
"legal_name",
|
||||||
|
"tax_id",
|
||||||
|
"taxonomy_code",
|
||||||
|
"address_line1",
|
||||||
|
"city",
|
||||||
|
"state",
|
||||||
|
"zip",
|
||||||
|
"is_active",
|
||||||
|
):
|
||||||
|
assert key in data, f"missing field {key}"
|
||||||
|
|
||||||
|
|
||||||
|
def test_provider_detail_includes_orphan_remit_received(seeded_db: TestClient):
|
||||||
|
"""Regression: the ``remit_received`` ActivityEvent is recorded at 835
|
||||||
|
ingest with ``claim_id=None`` (``store.add`` lines 999-1003) — the
|
||||||
|
remittance hasn't been matched to a claim yet. The original
|
||||||
|
``ActivityEvent.claim_id IN (claim_ids)`` filter misses it because
|
||||||
|
the orphan's claim_id is NULL.
|
||||||
|
|
||||||
|
Once reconciliation (auto or manual) populates
|
||||||
|
``Remittance.claim_id``, the activity filter must surface the event
|
||||||
|
via the ``Remittance.claim_id IN (claim_ids)`` branch of the OR.
|
||||||
|
Without that branch, a provider's activity feed appears to freeze
|
||||||
|
the moment an 835 lands — the most common activity, invisible.
|
||||||
|
|
||||||
|
Setup: ingest 837+835 for ``TEST_837_NPI`` (claim CLM001 + remit
|
||||||
|
CLM001), then manually match them so ``Remittance.claim_id`` is
|
||||||
|
populated. The bug presents as: only ``claim_submitted`` and
|
||||||
|
``manual_match`` appear (no ``remit_received``). The fix surfaces
|
||||||
|
all three.
|
||||||
|
"""
|
||||||
|
# Force the match — simulates the post-reconciliation state that
|
||||||
|
# populate Remittance.claim_id without depending on auto-reconcile
|
||||||
|
# heuristics (which don't match this minimal fixture).
|
||||||
|
match_resp = seeded_db.post(
|
||||||
|
"/api/reconciliation/match",
|
||||||
|
json={"claim_id": "CLM001", "remit_id": "CLM001"},
|
||||||
|
)
|
||||||
|
assert match_resp.status_code == 200, match_resp.text
|
||||||
|
|
||||||
|
resp = seeded_db.get(f"/api/config/providers/{TEST_837_NPI}")
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
data = resp.json()
|
||||||
|
kinds = {event["kind"] for event in data["recent_activity"]}
|
||||||
|
assert "remit_received" in kinds, (
|
||||||
|
f"expected remit_received in recent_activity (orphan remits "
|
||||||
|
f"must surface via the Remittance join), got kinds={sorted(kinds)}"
|
||||||
|
)
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
"""SP9 — providers/payers/clearhouse seed + CRUD tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone.providers import Provider
|
||||||
|
from cyclone.store import store
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _fresh_db(tmp_path, monkeypatch):
|
||||||
|
"""Use a fresh in-memory SQLite for each test."""
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db_mod._reset_for_tests()
|
||||||
|
db_mod.init_db()
|
||||||
|
yield
|
||||||
|
db_mod._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_creates_3_providers():
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
providers = store.list_providers()
|
||||||
|
labels = {p.label for p in providers}
|
||||||
|
assert labels == {"Montrose", "Delta", "Salida"}
|
||||||
|
npis = {p.npi for p in providers}
|
||||||
|
assert npis == {"1881068062", "1851446637", "1467507269"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_uses_consistent_tax_id_and_taxonomy():
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
providers = store.list_providers()
|
||||||
|
for p in providers:
|
||||||
|
assert p.tax_id == "721587149"
|
||||||
|
assert p.taxonomy_code == "251E00000X"
|
||||||
|
assert p.legal_name == "TOC, Inc."
|
||||||
|
assert p.address_line1 == "1100 East Main St"
|
||||||
|
assert p.address_line2 == "Suite A"
|
||||||
|
assert p.city == "Montrose"
|
||||||
|
assert p.state == "CO"
|
||||||
|
assert p.zip == "814014063"
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_creates_clearhouse_singleton():
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
ch = store.get_clearhouse()
|
||||||
|
assert ch is not None
|
||||||
|
assert ch.id == 1
|
||||||
|
assert ch.name == "dzinesco"
|
||||||
|
assert ch.tpid == "11525703"
|
||||||
|
assert ch.submitter_name == "Dzinesco"
|
||||||
|
assert ch.sftp_block.host == "mft.gainwelltechnologies.com"
|
||||||
|
assert ch.sftp_block.stub is True
|
||||||
|
assert "FromHPE" in ch.sftp_block.paths["outbound"]
|
||||||
|
assert "ToHPE" in ch.sftp_block.paths["inbound"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_creates_co_txix_payer_with_both_configs():
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
payers = store.list_payers()
|
||||||
|
assert {p.payer_id for p in payers} == {"CO_TXIX"}
|
||||||
|
p837 = store.get_payer_config("CO_TXIX", "837P")
|
||||||
|
p835 = store.get_payer_config("CO_TXIX", "835")
|
||||||
|
assert p837 is not None and p837["payer_id"] == "CO_TXIX"
|
||||||
|
assert p835 is not None and "1811725341" in p835["expected_payer_tax_ids"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_is_idempotent():
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
assert len(store.list_providers()) == 3
|
||||||
|
assert len(store.list_payers()) == 1
|
||||||
|
assert store.get_clearhouse() is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_provider_returns_none_for_unknown_npi():
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
assert store.get_provider("9999999999") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_upsert_provider_creates_then_updates():
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
p = Provider(
|
||||||
|
npi="1111111111",
|
||||||
|
label="Test",
|
||||||
|
legal_name="Test, Inc.",
|
||||||
|
tax_id="123456789",
|
||||||
|
taxonomy_code="207R00000X",
|
||||||
|
address_line1="1 Test Way",
|
||||||
|
city="Testville",
|
||||||
|
state="CO",
|
||||||
|
zip="80000",
|
||||||
|
created_at=datetime.now(timezone.utc),
|
||||||
|
updated_at=datetime.now(timezone.utc),
|
||||||
|
)
|
||||||
|
stored = store.upsert_provider(p)
|
||||||
|
assert stored.label == "Test"
|
||||||
|
p2 = p.model_copy(update={"label": "Updated"})
|
||||||
|
stored2 = store.upsert_provider(p2)
|
||||||
|
assert stored2.label == "Updated"
|
||||||
|
assert len(store.list_providers(is_active=None)) == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_providers_filter_is_active():
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
p = Provider(
|
||||||
|
npi="2222222222",
|
||||||
|
label="Inactive",
|
||||||
|
legal_name="X",
|
||||||
|
tax_id="1",
|
||||||
|
taxonomy_code="X",
|
||||||
|
address_line1="X",
|
||||||
|
city="X",
|
||||||
|
state="CO",
|
||||||
|
zip="0",
|
||||||
|
is_active=False,
|
||||||
|
created_at=datetime.now(timezone.utc),
|
||||||
|
updated_at=datetime.now(timezone.utc),
|
||||||
|
)
|
||||||
|
store.upsert_provider(p)
|
||||||
|
active = store.list_providers(is_active=True)
|
||||||
|
assert {pp.label for pp in active} == {"Montrose", "Delta", "Salida"}
|
||||||
|
all_p = store.list_providers(is_active=None)
|
||||||
|
assert {pp.label for pp in all_p} == {"Montrose", "Delta", "Salida", "Inactive"}
|
||||||
@@ -0,0 +1,287 @@
|
|||||||
|
"""SP16 — Inbound MFT polling scheduler tests.
|
||||||
|
|
||||||
|
We test the Scheduler class with a fake ``SftpClient`` factory that
|
||||||
|
returns files we drop on disk (the SFTP stub already does this; we
|
||||||
|
just need to control which files appear between ticks). The handlers
|
||||||
|
themselves (999/835/277CA/TA1) are exercised through real parsers
|
||||||
|
using the fixtures in ``tests/fixtures/``.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import db, scheduler as sched_mod
|
||||||
|
from cyclone.db import ProcessedInboundFile
|
||||||
|
from cyclone.providers import SftpBlock
|
||||||
|
from cyclone.scheduler import (
|
||||||
|
HANDLERS,
|
||||||
|
ROUTED_FILE_TYPES,
|
||||||
|
Scheduler,
|
||||||
|
STATUS_ERROR,
|
||||||
|
STATUS_OK,
|
||||||
|
STATUS_SKIPPED,
|
||||||
|
TickResult,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ----- fixtures -----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def sftp_block(tmp_path):
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
inbound_dir = staging / "ToHPE"
|
||||||
|
inbound_dir.mkdir(parents=True)
|
||||||
|
return SftpBlock(
|
||||||
|
host="mft.example.com",
|
||||||
|
port=22,
|
||||||
|
username="test",
|
||||||
|
paths={
|
||||||
|
"outbound": "/FromHPE",
|
||||||
|
"inbound": "/ToHPE",
|
||||||
|
},
|
||||||
|
stub=True,
|
||||||
|
staging_dir=str(staging),
|
||||||
|
poll_seconds=60,
|
||||||
|
auth={"method": "keychain", "secret_ref": "test.password"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def _drop_file(sftp_block):
|
||||||
|
"""Helper: drop a named file in the inbound dir. Returns the path."""
|
||||||
|
inbound_dir = Path(sftp_block.staging_dir) / "ToHPE"
|
||||||
|
|
||||||
|
def _drop(name: str, body: bytes) -> Path:
|
||||||
|
inbound_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
p = inbound_dir / name
|
||||||
|
p.write_bytes(body)
|
||||||
|
return p
|
||||||
|
|
||||||
|
return _drop
|
||||||
|
|
||||||
|
|
||||||
|
def _make_scheduler(sftp_block, tmp_path) -> Scheduler:
|
||||||
|
"""Build a Scheduler wired to the real (stub) SftpClient."""
|
||||||
|
sched = Scheduler(
|
||||||
|
sftp_block,
|
||||||
|
poll_interval_seconds=60,
|
||||||
|
sftp_block_name="test-block",
|
||||||
|
# Use the real SftpClient — it reads from the stub staging dir.
|
||||||
|
sftp_client_factory=None,
|
||||||
|
)
|
||||||
|
return sched
|
||||||
|
|
||||||
|
|
||||||
|
def _load_999_text() -> str:
|
||||||
|
return (Path(__file__).parent / "fixtures" / "minimal_999.txt").read_text()
|
||||||
|
|
||||||
|
|
||||||
|
def _load_835_text() -> str:
|
||||||
|
return (Path(__file__).parent / "fixtures" / "minimal_835.txt").read_text()
|
||||||
|
|
||||||
|
|
||||||
|
def _load_277ca_text() -> str:
|
||||||
|
return (Path(__file__).parent / "fixtures" / "minimal_277ca.txt").read_text()
|
||||||
|
|
||||||
|
|
||||||
|
def _load_ta1_text() -> str:
|
||||||
|
return (Path(__file__).parent / "fixtures" / "minimal_ta1.txt").read_text()
|
||||||
|
|
||||||
|
|
||||||
|
# ----- tests --------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestSchedulerStatus:
|
||||||
|
def test_not_running_by_default(self, sftp_block):
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
st = sched.status()
|
||||||
|
assert st.running is False
|
||||||
|
assert st.poll_count == 0
|
||||||
|
assert st.last_poll_at is None
|
||||||
|
|
||||||
|
def test_running_after_start(self, sftp_block):
|
||||||
|
async def _go():
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
await sched.start()
|
||||||
|
try:
|
||||||
|
assert sched.is_running() is True
|
||||||
|
finally:
|
||||||
|
await sched.stop()
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
|
||||||
|
class TestTickOnEmptyInbox:
|
||||||
|
def test_tick_with_no_files_records_zero(self, sftp_block):
|
||||||
|
async def _go():
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
result = await sched.tick()
|
||||||
|
assert isinstance(result, TickResult)
|
||||||
|
assert result.files_seen == 0
|
||||||
|
assert result.files_processed == 0
|
||||||
|
assert result.files_skipped == 0
|
||||||
|
assert result.files_errored == 0
|
||||||
|
assert result.finished_at is not None
|
||||||
|
assert result.errors == []
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
|
||||||
|
class TestTickRoutesFiles:
|
||||||
|
def test_999_file_processed(self, sftp_block, _drop_file):
|
||||||
|
async def _go():
|
||||||
|
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_999.x12",
|
||||||
|
_load_999_text().encode("utf-8"))
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
result = await sched.tick()
|
||||||
|
assert result.files_seen == 1
|
||||||
|
assert result.files_processed == 1
|
||||||
|
assert result.files_errored == 0
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
rows = (
|
||||||
|
session.query(ProcessedInboundFile)
|
||||||
|
.filter_by(name="TP11525703-837P_M019048402-20260618130000000-1of1_999.x12")
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
assert len(rows) == 1
|
||||||
|
assert rows[0].status == STATUS_OK
|
||||||
|
assert rows[0].parser_used == "parse_999"
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
def test_ta1_file_processed(self, sftp_block, _drop_file):
|
||||||
|
async def _go():
|
||||||
|
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12",
|
||||||
|
_load_ta1_text().encode("utf-8"))
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
result = await sched.tick()
|
||||||
|
assert result.files_processed == 1, result.errors
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
row = (
|
||||||
|
session.query(ProcessedInboundFile)
|
||||||
|
.filter_by(name="TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12")
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
assert row is not None
|
||||||
|
assert row.status == STATUS_OK
|
||||||
|
assert row.parser_used == "parse_ta1"
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
def test_unknown_file_type_marked_skipped(self, sftp_block, _drop_file):
|
||||||
|
async def _go():
|
||||||
|
# 270 (eligibility request) is in the HCPF allowed set but
|
||||||
|
# NOT in ROUTED_FILE_TYPES — Cyclone doesn't have a 270 parser.
|
||||||
|
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_270.x12",
|
||||||
|
b"some bytes")
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
result = await sched.tick()
|
||||||
|
assert result.files_skipped == 1
|
||||||
|
assert result.files_processed == 0
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
row = (
|
||||||
|
session.query(ProcessedInboundFile)
|
||||||
|
.filter_by(name="TP11525703-837P_M019048402-20260618130000000-1of1_270.x12")
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
assert row is not None
|
||||||
|
assert row.status == STATUS_SKIPPED
|
||||||
|
assert row.error_message and "270" in row.error_message
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
def test_filename_not_matching_hcpf_marked_skipped(self, sftp_block, _drop_file):
|
||||||
|
async def _go():
|
||||||
|
_drop_file("random.txt", b"garbage")
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
result = await sched.tick()
|
||||||
|
assert result.files_skipped == 1
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
def test_parse_error_marked_error(self, sftp_block, _drop_file):
|
||||||
|
async def _go():
|
||||||
|
# Valid filename but malformed body — parser raises.
|
||||||
|
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_999.x12",
|
||||||
|
b"this is not a 999 file at all")
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
result = await sched.tick()
|
||||||
|
assert result.files_errored == 1
|
||||||
|
assert result.files_processed == 0
|
||||||
|
|
||||||
|
with db.SessionLocal()() as session:
|
||||||
|
row = (
|
||||||
|
session.query(ProcessedInboundFile)
|
||||||
|
.filter_by(name="TP11525703-837P_M019048402-20260618130000000-1of1_999.x12")
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
assert row.status == STATUS_ERROR
|
||||||
|
assert row.error_message
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
|
||||||
|
class TestTickIdempotent:
|
||||||
|
def test_second_tick_does_not_reprocess(self, sftp_block, _drop_file):
|
||||||
|
async def _go():
|
||||||
|
_drop_file("TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12",
|
||||||
|
_load_ta1_text().encode("utf-8"))
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
r1 = await sched.tick()
|
||||||
|
r2 = await sched.tick()
|
||||||
|
assert r1.files_processed == 1
|
||||||
|
assert r2.files_seen == 1 # still lists it
|
||||||
|
assert r2.files_processed == 0 # but skips — already done
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
|
||||||
|
class TestSchedulerStartStop:
|
||||||
|
def test_start_then_stop_returns_to_not_running(self, sftp_block):
|
||||||
|
async def _go():
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
await sched.start()
|
||||||
|
assert sched.is_running()
|
||||||
|
await sched.stop()
|
||||||
|
assert not sched.is_running()
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
def test_double_start_is_idempotent(self, sftp_block):
|
||||||
|
async def _go():
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
await sched.start()
|
||||||
|
await sched.start() # no-op
|
||||||
|
assert sched.is_running()
|
||||||
|
await sched.stop()
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
def test_stop_when_not_running_is_safe(self, sftp_block):
|
||||||
|
async def _go():
|
||||||
|
sched = _make_scheduler(sftp_block, Path("/tmp"))
|
||||||
|
await sched.stop() # no-op
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
|
||||||
|
class TestModuleSingleton:
|
||||||
|
def test_get_scheduler_raises_if_not_configured(self):
|
||||||
|
sched_mod.reset_scheduler_for_tests()
|
||||||
|
with pytest.raises(RuntimeError, match="not configured"):
|
||||||
|
sched_mod.get_scheduler()
|
||||||
|
|
||||||
|
def test_configure_then_get_returns_same_instance(self, sftp_block):
|
||||||
|
sched_mod.reset_scheduler_for_tests()
|
||||||
|
s = sched_mod.configure_scheduler(sftp_block, sftp_block_name="t")
|
||||||
|
try:
|
||||||
|
assert sched_mod.get_scheduler() is s
|
||||||
|
finally:
|
||||||
|
sched_mod.reset_scheduler_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
class TestRoutedFileTypes:
|
||||||
|
"""Frozen-set guards — adding a new routed type without updating
|
||||||
|
the dispatch table would silently skip every inbound file of that
|
||||||
|
type. These tests are the regression net."""
|
||||||
|
|
||||||
|
def test_handlers_cover_all_routed_types(self):
|
||||||
|
assert set(HANDLERS.keys()) == ROUTED_FILE_TYPES
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
"""SP9 — macOS Keychain secret accessor tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import secrets
|
||||||
|
from cyclone.secrets import STUB_SECRET, get_secret, has_keyring, set_secret
|
||||||
|
|
||||||
|
|
||||||
|
def test_has_keyring_true_when_lib_present():
|
||||||
|
# The test env has keyring installed
|
||||||
|
assert has_keyring() is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_secret_returns_none_when_keyring_missing(monkeypatch):
|
||||||
|
monkeypatch.setattr(secrets, "_HAS_KEYRING", False)
|
||||||
|
monkeypatch.setattr(secrets, "keyring", None)
|
||||||
|
assert get_secret("anything") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_secret_returns_keychain_value():
|
||||||
|
with patch("cyclone.secrets.keyring") as mock_kr:
|
||||||
|
mock_kr.get_password.return_value = "p@ssw0rd"
|
||||||
|
v = get_secret("sftp.gainwell.password")
|
||||||
|
assert v == "p@ssw0rd"
|
||||||
|
mock_kr.get_password.assert_called_once_with("cyclone", "sftp.gainwell.password")
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_secret_returns_none_on_keychain_exception():
|
||||||
|
with patch("cyclone.secrets.keyring") as mock_kr:
|
||||||
|
mock_kr.get_password.side_effect = RuntimeError("keychain locked")
|
||||||
|
v = get_secret("x")
|
||||||
|
assert v is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_set_secret_returns_true_on_success():
|
||||||
|
with patch("cyclone.secrets.keyring") as mock_kr:
|
||||||
|
assert set_secret("a", "b") is True
|
||||||
|
mock_kr.set_password.assert_called_once_with("cyclone", "a", "b")
|
||||||
|
|
||||||
|
|
||||||
|
def test_set_secret_returns_false_when_keyring_missing(monkeypatch):
|
||||||
|
monkeypatch.setattr(secrets, "_HAS_KEYRING", False)
|
||||||
|
monkeypatch.setattr(secrets, "keyring", None)
|
||||||
|
assert set_secret("a", "b") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_set_secret_returns_false_on_keychain_exception():
|
||||||
|
with patch("cyclone.secrets.keyring") as mock_kr:
|
||||||
|
mock_kr.set_password.side_effect = RuntimeError("denied")
|
||||||
|
assert set_secret("a", "b") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_stub_secret_is_distinct_string():
|
||||||
|
assert STUB_SECRET == "<stub-secret>"
|
||||||
|
assert STUB_SECRET != ""
|
||||||
@@ -0,0 +1,225 @@
|
|||||||
|
"""SP19 — Security middleware + health probe tests.
|
||||||
|
|
||||||
|
Covers each middleware in isolation (using FastAPI's TestClient with a
|
||||||
|
minimal app) and the integration into the real ``cyclone.api`` app
|
||||||
|
(headers present on every response, /api/health returns the rich
|
||||||
|
snapshot).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import FastAPI, Request
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from cyclone.security import (
|
||||||
|
DEFAULT_MAX_BODY_BYTES,
|
||||||
|
DEFAULT_RATE_LIMIT_PER_MIN,
|
||||||
|
BodySizeLimitMiddleware,
|
||||||
|
RateLimitMiddleware,
|
||||||
|
SecurityHeadersMiddleware,
|
||||||
|
get_health_snapshot,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Test apps — built inline per-test to avoid state pollution between tests.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _echo_app_with(
|
||||||
|
*middlewares: Callable[..., Any],
|
||||||
|
) -> FastAPI:
|
||||||
|
"""Build a tiny FastAPI app with the given middleware chain.
|
||||||
|
|
||||||
|
``middlewares`` are listed outermost-first (the first element
|
||||||
|
runs first on the request). Starlette's ``add_middleware``
|
||||||
|
*prepends*, so we add in reverse to preserve "outermost first"
|
||||||
|
in the public API.
|
||||||
|
"""
|
||||||
|
app = FastAPI()
|
||||||
|
|
||||||
|
@app.post("/echo")
|
||||||
|
async def echo(request: Request):
|
||||||
|
body = await request.body()
|
||||||
|
return {"received_bytes": len(body)}
|
||||||
|
|
||||||
|
@app.get("/api/health")
|
||||||
|
async def h():
|
||||||
|
return {"status": "ok"}
|
||||||
|
|
||||||
|
for mw in reversed(middlewares):
|
||||||
|
app.add_middleware(mw)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# BodySizeLimitMiddleware
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_body_size_accepts_under_limit():
|
||||||
|
"""500 bytes is under the 1024-byte limit; body passes through."""
|
||||||
|
app = _echo_app_with(
|
||||||
|
SecurityHeadersMiddleware,
|
||||||
|
lambda a: BodySizeLimitMiddleware(a, max_bytes=1024),
|
||||||
|
)
|
||||||
|
client = TestClient(app)
|
||||||
|
resp = client.post("/echo", content=b"x" * 500)
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
assert resp.json() == {"received_bytes": 500}
|
||||||
|
|
||||||
|
|
||||||
|
def test_body_size_rejects_over_content_length():
|
||||||
|
"""A 200-byte body against a 100-byte cap returns 413."""
|
||||||
|
app = _echo_app_with(
|
||||||
|
SecurityHeadersMiddleware,
|
||||||
|
lambda a: BodySizeLimitMiddleware(a, max_bytes=100),
|
||||||
|
)
|
||||||
|
client = TestClient(app)
|
||||||
|
resp = client.post("/echo", content=b"x" * 200)
|
||||||
|
assert resp.status_code == 413
|
||||||
|
body = resp.json()
|
||||||
|
assert body["error"] == "body_too_large"
|
||||||
|
assert "100" in body["detail"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_body_size_default_is_50mb():
|
||||||
|
"""The default cap is 50 MB so even large prodfiles fit."""
|
||||||
|
assert DEFAULT_MAX_BODY_BYTES == 50 * 1024 * 1024
|
||||||
|
|
||||||
|
|
||||||
|
def test_body_size_rejects_bad_content_length():
|
||||||
|
"""A non-integer Content-Length is rejected as 400, not 500."""
|
||||||
|
sent: list = []
|
||||||
|
inner_app = FastAPI()
|
||||||
|
|
||||||
|
@inner_app.post("/echo")
|
||||||
|
async def echo(request: Request):
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
app_instance = BodySizeLimitMiddleware(inner_app, max_bytes=100)
|
||||||
|
|
||||||
|
async def fake_receive():
|
||||||
|
return {"type": "http.request", "body": b"", "more_body": False}
|
||||||
|
|
||||||
|
async def fake_send(msg):
|
||||||
|
sent.append(msg)
|
||||||
|
|
||||||
|
scope = {
|
||||||
|
"type": "http",
|
||||||
|
"method": "POST",
|
||||||
|
"path": "/echo",
|
||||||
|
"headers": [(b"content-length", b"not-a-number")],
|
||||||
|
"query_string": b"",
|
||||||
|
}
|
||||||
|
asyncio.run(app_instance(scope, fake_receive, fake_send))
|
||||||
|
start = next(m for m in sent if m["type"] == "http.response.start")
|
||||||
|
assert start["status"] == 400
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# RateLimitMiddleware
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_limit_default_is_300_per_min():
|
||||||
|
assert DEFAULT_RATE_LIMIT_PER_MIN == 300
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_limit_allows_under_threshold():
|
||||||
|
"""A few requests under the per-minute limit are allowed."""
|
||||||
|
app = _echo_app_with(
|
||||||
|
SecurityHeadersMiddleware,
|
||||||
|
lambda a: RateLimitMiddleware(a, per_minute=5),
|
||||||
|
)
|
||||||
|
client = TestClient(app)
|
||||||
|
for _ in range(5):
|
||||||
|
resp = client.post("/echo", content=b"x")
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_limit_blocks_over_threshold():
|
||||||
|
"""The 6th request within a 60s window is rate-limited."""
|
||||||
|
app = _echo_app_with(
|
||||||
|
SecurityHeadersMiddleware,
|
||||||
|
lambda a: RateLimitMiddleware(a, per_minute=3),
|
||||||
|
)
|
||||||
|
client = TestClient(app)
|
||||||
|
for _ in range(3):
|
||||||
|
assert client.post("/echo", content=b"x").status_code == 200
|
||||||
|
resp = client.post("/echo", content=b"x")
|
||||||
|
assert resp.status_code == 429
|
||||||
|
assert resp.json()["error"] == "rate_limited"
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_limit_exempts_health_probes():
|
||||||
|
"""A load balancer hammering /api/health should not trip the limiter."""
|
||||||
|
app = _echo_app_with(
|
||||||
|
SecurityHeadersMiddleware,
|
||||||
|
lambda a: RateLimitMiddleware(a, per_minute=2),
|
||||||
|
)
|
||||||
|
client = TestClient(app)
|
||||||
|
for _ in range(20):
|
||||||
|
assert client.get("/api/health").status_code == 200
|
||||||
|
assert client.post("/echo", content=b"x").status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SecurityHeadersMiddleware
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_security_headers_present_on_200():
|
||||||
|
app = _echo_app_with(SecurityHeadersMiddleware)
|
||||||
|
client = TestClient(app)
|
||||||
|
resp = client.get("/api/health")
|
||||||
|
assert resp.headers["X-Content-Type-Options"] == "nosniff"
|
||||||
|
assert resp.headers["X-Frame-Options"] == "DENY"
|
||||||
|
assert resp.headers["Referrer-Policy"] == "same-origin"
|
||||||
|
assert "default-src 'none'" in resp.headers["Content-Security-Policy"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_security_headers_present_on_error_response():
|
||||||
|
"""413/429 still carry the security headers."""
|
||||||
|
# Define a tiny factory so Starlette can introspect it as a class.
|
||||||
|
class _BoundBodySize(BodySizeLimitMiddleware):
|
||||||
|
def __init__(self, app): # type: ignore[no-untyped-def]
|
||||||
|
super().__init__(app, max_bytes=10)
|
||||||
|
|
||||||
|
app = _echo_app_with(SecurityHeadersMiddleware, _BoundBodySize)
|
||||||
|
client = TestClient(app)
|
||||||
|
resp = client.post("/echo", content=b"x" * 100)
|
||||||
|
assert resp.status_code == 413, resp.text
|
||||||
|
assert resp.headers["X-Content-Type-Options"] == "nosniff"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# get_health_snapshot
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_snapshot_basic_shape():
|
||||||
|
snap = get_health_snapshot()
|
||||||
|
d = snap.to_dict()
|
||||||
|
assert "status" in d
|
||||||
|
assert "version" in d
|
||||||
|
assert "db" in d
|
||||||
|
assert "scheduler" in d
|
||||||
|
assert "pubsub" in d
|
||||||
|
assert "batch" in d
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_snapshot_db_ok_in_tests():
|
||||||
|
"""The conftest DB fixture is live; ``SELECT 1`` works."""
|
||||||
|
snap = get_health_snapshot()
|
||||||
|
assert snap.db.get("ok") is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_snapshot_handles_no_scheduler():
|
||||||
|
"""Without a configured scheduler, the snapshot reports gracefully."""
|
||||||
|
snap = get_health_snapshot()
|
||||||
|
sched = snap.scheduler
|
||||||
|
assert "running" in sched or "configured" in sched
|
||||||
@@ -0,0 +1,275 @@
|
|||||||
|
"""Tests for the SP13 paramiko-backed SftpClient.
|
||||||
|
|
||||||
|
We exercise:
|
||||||
|
1. Stub behavior unchanged from SP9 (smoke test).
|
||||||
|
2. Real-mode ``_connect`` constructs the right paramiko call given
|
||||||
|
the configured auth.
|
||||||
|
3. Real-mode ``write_file`` calls ``SFTPFile.open(..., 'wb')`` with
|
||||||
|
the right bytes and remote path.
|
||||||
|
4. Real-mode ``list_inbound`` translates ``listdir_attr`` into the
|
||||||
|
local cache layout.
|
||||||
|
5. Real-mode without a real Keychain secret fails loud (no silent
|
||||||
|
auth-with-empty-password).
|
||||||
|
|
||||||
|
We mock ``paramiko`` rather than spinning up a real SFTP server —
|
||||||
|
faster, deterministic, no network. The mocks live in
|
||||||
|
``cyclone.clearhouse`` so the test file doesn't need to know
|
||||||
|
paramiko's internals.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.clearhouse import SftpClient
|
||||||
|
from cyclone.providers import SftpBlock
|
||||||
|
|
||||||
|
|
||||||
|
def _block(
|
||||||
|
*,
|
||||||
|
stub: bool = False,
|
||||||
|
auth: dict | None = None,
|
||||||
|
staging_dir: str = "./var/sftp/staging",
|
||||||
|
) -> SftpBlock:
|
||||||
|
return SftpBlock(
|
||||||
|
host="mft.example.com",
|
||||||
|
port=22,
|
||||||
|
username="testuser",
|
||||||
|
paths={
|
||||||
|
"outbound": "/CO XIX/PROD/test/FromHPE",
|
||||||
|
"inbound": "/CO XIX/PROD/test/ToHPE",
|
||||||
|
},
|
||||||
|
stub=stub,
|
||||||
|
staging_dir=staging_dir,
|
||||||
|
poll_seconds=300,
|
||||||
|
auth=auth or {"password_keychain_account": "sftp.test.password"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _no_real_secrets(monkeypatch):
|
||||||
|
"""Default: pretend Keychain has a real password."""
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"cyclone.clearhouse.secrets.get_secret",
|
||||||
|
lambda name: "real-password-from-keychain",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Stub mode (smoke test from SP9)
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
class TestStubUnchanged:
|
||||||
|
def test_stub_still_writes_to_local_staging(self, tmp_path: Path):
|
||||||
|
block = _block(stub=True, staging_dir=str(tmp_path / "staging"))
|
||||||
|
client = SftpClient(block)
|
||||||
|
target = client.write_file(
|
||||||
|
"/CO XIX/PROD/test/FromHPE/file.x12", b"hello",
|
||||||
|
)
|
||||||
|
assert target == tmp_path / "staging/CO XIX/PROD/test/FromHPE/file.x12"
|
||||||
|
assert target.read_bytes() == b"hello"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Real mode — mock paramiko
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def _make_mock_paramiko(monkeypatch, *, sftp_attrs: list | None = None):
|
||||||
|
"""Replace ``cyclone.clearhouse._connect``'s paramiko with a mock.
|
||||||
|
|
||||||
|
Returns (mock_ssh, mock_sftp) so callers can introspect call args.
|
||||||
|
"""
|
||||||
|
# Patch paramiko where the SftpClient uses it. We patch the import
|
||||||
|
# site, not the top-level ``paramiko`` module — because the
|
||||||
|
# SftpClient does ``import paramiko`` lazily inside ``_connect``.
|
||||||
|
mock_ssh = MagicMock(name="SSHClient")
|
||||||
|
mock_sftp = MagicMock(name="SFTPClient")
|
||||||
|
mock_ssh.open_sftp.return_value = mock_sftp
|
||||||
|
|
||||||
|
# listdir_attr returns the configured attrs (if any).
|
||||||
|
mock_sftp.listdir_attr.return_value = sftp_attrs or []
|
||||||
|
|
||||||
|
# sftp.open returns a context manager that we can write to.
|
||||||
|
# We make __enter__() return a real BytesIO for read paths so
|
||||||
|
# shutil.copyfileobj works in the list_inbound / read_file tests.
|
||||||
|
# For write paths, __enter__() returns a MagicMock with .write().
|
||||||
|
def _open(path, mode="rb"):
|
||||||
|
m = MagicMock()
|
||||||
|
if "wb" in mode:
|
||||||
|
m.__enter__.return_value = MagicMock()
|
||||||
|
m.__enter__.return_value.write = MagicMock()
|
||||||
|
else:
|
||||||
|
m.__enter__.return_value = io.BytesIO(b"")
|
||||||
|
return m
|
||||||
|
|
||||||
|
mock_sftp.open.side_effect = _open
|
||||||
|
|
||||||
|
# Patch the lazy import inside _connect.
|
||||||
|
fake_paramiko = MagicMock(name="paramiko")
|
||||||
|
fake_paramiko.SSHClient.return_value = mock_ssh
|
||||||
|
fake_paramiko.AutoAddPolicy.return_value = "AutoAddPolicy"
|
||||||
|
fake_paramiko.RSAKey.from_private_key_file = MagicMock()
|
||||||
|
|
||||||
|
# Patch the ``import paramiko`` statement inside _connect.
|
||||||
|
import builtins
|
||||||
|
real_import = builtins.__import__
|
||||||
|
|
||||||
|
def _patched_import(name, *args, **kwargs):
|
||||||
|
if name == "paramiko":
|
||||||
|
return fake_paramiko
|
||||||
|
return real_import(name, *args, **kwargs)
|
||||||
|
|
||||||
|
monkeypatch.setattr(builtins, "__import__", _patched_import)
|
||||||
|
return mock_ssh, mock_sftp
|
||||||
|
|
||||||
|
|
||||||
|
class TestRealModeConnect:
|
||||||
|
def test_connect_uses_password_from_keychain(self, monkeypatch):
|
||||||
|
mock_ssh, _ = _make_mock_paramiko(monkeypatch)
|
||||||
|
client = SftpClient(_block())
|
||||||
|
with client._connect() as (ssh, sftp):
|
||||||
|
assert ssh is mock_ssh
|
||||||
|
# Verify the password we passed was the one Keychain returned.
|
||||||
|
call_kwargs = mock_ssh.connect.call_args.kwargs
|
||||||
|
assert call_kwargs["hostname"] == "mft.example.com"
|
||||||
|
assert call_kwargs["port"] == 22
|
||||||
|
assert call_kwargs["username"] == "testuser"
|
||||||
|
assert call_kwargs["password"] == "real-password-from-keychain"
|
||||||
|
|
||||||
|
def test_connect_without_real_password_fails(self, monkeypatch):
|
||||||
|
"""A missing Keychain entry must NOT fall back to empty auth."""
|
||||||
|
from cyclone.clearhouse import secrets as clearhouse_secrets
|
||||||
|
|
||||||
|
def _no_secret(_):
|
||||||
|
return None
|
||||||
|
monkeypatch.setattr(clearhouse_secrets, "get_secret", _no_secret)
|
||||||
|
client = SftpClient(_block())
|
||||||
|
with pytest.raises(RuntimeError, match="missing or stub"):
|
||||||
|
with client._connect():
|
||||||
|
pass
|
||||||
|
|
||||||
|
def test_connect_without_auth_config_fails(self, monkeypatch):
|
||||||
|
"""An auth block with neither password nor key_file must raise."""
|
||||||
|
_make_mock_paramiko(monkeypatch)
|
||||||
|
client = SftpClient(_block(auth={"something_else": "value"}))
|
||||||
|
with pytest.raises(RuntimeError, match="must contain"):
|
||||||
|
with client._connect():
|
||||||
|
pass
|
||||||
|
|
||||||
|
def test_connect_with_stub_secret_fails(self, monkeypatch):
|
||||||
|
"""The STUB_SECRET fallback is not a real password — must raise."""
|
||||||
|
from cyclone.clearhouse import secrets as clearhouse_secrets
|
||||||
|
from cyclone.secrets import STUB_SECRET
|
||||||
|
monkeypatch.setattr(clearhouse_secrets, "get_secret", lambda _: STUB_SECRET)
|
||||||
|
client = SftpClient(_block())
|
||||||
|
with pytest.raises(RuntimeError, match="missing or stub"):
|
||||||
|
with client._connect():
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class TestRealModeWrite:
|
||||||
|
def test_write_uploads_to_correct_remote_path(self, monkeypatch):
|
||||||
|
_, mock_sftp = _make_mock_paramiko(monkeypatch)
|
||||||
|
client = SftpClient(_block())
|
||||||
|
result = client.write_file(
|
||||||
|
"/CO XIX/PROD/test/FromHPE/out.x12", b"X12 content",
|
||||||
|
)
|
||||||
|
# The remote path is passed through unchanged.
|
||||||
|
assert str(result) == "/CO XIX/PROD/test/FromHPE/out.x12"
|
||||||
|
# The SFTP open() was called with that path + "wb" mode.
|
||||||
|
called = [c for c in mock_sftp.open.call_args_list
|
||||||
|
if c.args and c.args[0] == "/CO XIX/PROD/test/FromHPE/out.x12"]
|
||||||
|
assert called, f"open() was not called with the expected path: {mock_sftp.open.call_args_list}"
|
||||||
|
assert called[0].args[1] == "wb"
|
||||||
|
# The bytes were written. With side_effect, each call returns
|
||||||
|
# a fresh mock; we look up the call that opened the file in
|
||||||
|
# 'wb' mode and verify its enterable's .write() was invoked.
|
||||||
|
write_call = [c for c in mock_sftp.open.call_args_list
|
||||||
|
if c.args and len(c.args) > 1 and "wb" in c.args[1]]
|
||||||
|
assert write_call, f"no 'wb' open() call: {mock_sftp.open.call_args_list}"
|
||||||
|
file_mock = write_call[0].return_value
|
||||||
|
file_mock.__enter__().write.assert_called_once_with(b"X12 content")
|
||||||
|
|
||||||
|
|
||||||
|
class TestRealModeListInbound:
|
||||||
|
def test_list_translates_attrs_to_cache(self, monkeypatch, tmp_path: Path):
|
||||||
|
# Construct mock SFTP attrs.
|
||||||
|
attr = MagicMock()
|
||||||
|
attr.filename = "TP123-837P_M456-...-1of1_999.x12"
|
||||||
|
attr.st_mode = 0o100644 # regular file
|
||||||
|
attr.st_size = 1024
|
||||||
|
attr.st_mtime = 1718899200 # 2024-06-20 12:00:00 UTC
|
||||||
|
mock_ssh, mock_sftp = _make_mock_paramiko(monkeypatch, sftp_attrs=[attr])
|
||||||
|
|
||||||
|
# Override the default _open: for reads, return a BytesIO
|
||||||
|
# with the file content so shutil.copyfileobj works.
|
||||||
|
def _open(path, mode="rb"):
|
||||||
|
m = MagicMock()
|
||||||
|
m.__enter__.return_value = io.BytesIO(b"999 content here")
|
||||||
|
return m
|
||||||
|
|
||||||
|
mock_sftp.open.side_effect = _open
|
||||||
|
|
||||||
|
# Use a temp staging dir for the cache.
|
||||||
|
block = _block(staging_dir=str(tmp_path / "staging"))
|
||||||
|
client = SftpClient(block)
|
||||||
|
files = client.list_inbound()
|
||||||
|
|
||||||
|
assert len(files) == 1
|
||||||
|
f = files[0]
|
||||||
|
assert f.name == "TP123-837P_M456-...-1of1_999.x12"
|
||||||
|
assert f.size == 1024
|
||||||
|
assert isinstance(f.modified_at, datetime)
|
||||||
|
assert f.local_path.exists()
|
||||||
|
assert f.local_path.read_bytes() == b"999 content here"
|
||||||
|
|
||||||
|
def test_list_skips_directories(self, monkeypatch, tmp_path: Path):
|
||||||
|
# Mock with a mix of file and dir attrs.
|
||||||
|
file_attr = MagicMock()
|
||||||
|
file_attr.filename = "real.x12"
|
||||||
|
file_attr.st_mode = 0o100644
|
||||||
|
file_attr.st_size = 100
|
||||||
|
file_attr.st_mtime = 1718899200
|
||||||
|
|
||||||
|
dir_attr = MagicMock()
|
||||||
|
dir_attr.filename = "subdir"
|
||||||
|
dir_attr.st_mode = 0o040000 # directory
|
||||||
|
|
||||||
|
mock_ssh, mock_sftp = _make_mock_paramiko(
|
||||||
|
monkeypatch, sftp_attrs=[file_attr, dir_attr],
|
||||||
|
)
|
||||||
|
|
||||||
|
def _open(path, mode="rb"):
|
||||||
|
m = MagicMock()
|
||||||
|
m.__enter__.return_value = io.BytesIO(b"")
|
||||||
|
return m
|
||||||
|
|
||||||
|
mock_sftp.open.side_effect = _open
|
||||||
|
|
||||||
|
block = _block(staging_dir=str(tmp_path / "staging"))
|
||||||
|
client = SftpClient(block)
|
||||||
|
files = client.list_inbound()
|
||||||
|
|
||||||
|
assert len(files) == 1
|
||||||
|
assert files[0].name == "real.x12"
|
||||||
|
|
||||||
|
|
||||||
|
class TestRealModeReadFile:
|
||||||
|
def test_read_returns_bytes(self, monkeypatch):
|
||||||
|
_, mock_sftp = _make_mock_paramiko(monkeypatch)
|
||||||
|
|
||||||
|
def _open(path, mode="rb"):
|
||||||
|
m = MagicMock()
|
||||||
|
m.__enter__.return_value = io.BytesIO(b"downloaded bytes")
|
||||||
|
return m
|
||||||
|
|
||||||
|
mock_sftp.open.side_effect = _open
|
||||||
|
client = SftpClient(_block())
|
||||||
|
data = client.read_file("/some/remote/path.x12")
|
||||||
|
assert data == b"downloaded bytes"
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
"""SP9 — SFTP stub tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone.clearhouse import SftpClient
|
||||||
|
from cyclone.providers import SftpBlock
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def sftp_block(tmp_path):
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
return SftpBlock(
|
||||||
|
host="mft.gainwelltechnologies.com",
|
||||||
|
port=22,
|
||||||
|
username="colorado-fts\\coxix_prod_11525703",
|
||||||
|
paths={
|
||||||
|
"outbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
|
||||||
|
"inbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
|
||||||
|
},
|
||||||
|
stub=True,
|
||||||
|
staging_dir=str(staging),
|
||||||
|
poll_seconds=300,
|
||||||
|
auth={"method": "keychain", "secret_ref": "sftp.gainwell.password"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_stub_writes_preserving_remote_path(sftp_block, tmp_path):
|
||||||
|
client = SftpClient(sftp_block)
|
||||||
|
remote = "/CO XIX/PROD/coxix_prod_11525703/FromHPE/11525703-837P-20260620132243505-1of1.x12"
|
||||||
|
target = client.write_file(remote, b"ISA*00*...~IEA*1*1~")
|
||||||
|
assert target.exists()
|
||||||
|
assert target.read_bytes() == b"ISA*00*...~IEA*1*1~"
|
||||||
|
# Confirm the full nested MFT path is preserved under staging
|
||||||
|
rel = target.relative_to(sftp_block.staging_dir)
|
||||||
|
assert str(rel) == "CO XIX/PROD/coxix_prod_11525703/FromHPE/11525703-837P-20260620132243505-1of1.x12"
|
||||||
|
|
||||||
|
|
||||||
|
def test_stub_creates_parent_dirs(sftp_block):
|
||||||
|
client = SftpClient(sftp_block)
|
||||||
|
remote = "/deep/nested/path/file.x12"
|
||||||
|
target = client.write_file(remote, b"data")
|
||||||
|
assert target.exists()
|
||||||
|
assert target.parent.is_dir()
|
||||||
|
|
||||||
|
|
||||||
|
def test_stub_list_inbound_empty_when_no_local_files(sftp_block):
|
||||||
|
client = SftpClient(sftp_block)
|
||||||
|
assert client.list_inbound() == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_stub_list_inbound_returns_local_files(sftp_block):
|
||||||
|
# Simulate operator dropping a file in the inbound staging dir
|
||||||
|
inbound_dir = Path(sftp_block.staging_dir) / "CO XIX/PROD/coxix_prod_11525703/ToHPE"
|
||||||
|
inbound_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
(inbound_dir / "TP11525703-837P_M019048402-20260520231513488-1of1_999.x12").write_bytes(b"X")
|
||||||
|
client = SftpClient(sftp_block)
|
||||||
|
files = client.list_inbound()
|
||||||
|
assert len(files) == 1
|
||||||
|
assert files[0].name.startswith("TP11525703-837P_M019048402")
|
||||||
|
assert files[0].size == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_stub_get_secret_returns_stub_when_keychain_empty(sftp_block):
|
||||||
|
client = SftpClient(sftp_block)
|
||||||
|
# Keychain is empty on the test box, so we get the stub sentinel
|
||||||
|
secret = client.get_secret("sftp.gainwell.password")
|
||||||
|
assert secret == "<stub-secret>"
|
||||||
|
|
||||||
|
|
||||||
|
def test_stub_read_file_returns_bytes(sftp_block, tmp_path):
|
||||||
|
"""SP16: the stub read_file returns bytes from the staging dir.
|
||||||
|
|
||||||
|
Lets the inbound scheduler exercise the same code path on a
|
||||||
|
workstation without a real MFT connection.
|
||||||
|
"""
|
||||||
|
inbound = tmp_path / "staging" / "ToHPE"
|
||||||
|
inbound.mkdir(parents=True)
|
||||||
|
(inbound / "TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12").write_bytes(
|
||||||
|
b"hello-world",
|
||||||
|
)
|
||||||
|
client = SftpClient(sftp_block)
|
||||||
|
body = client.read_file("/ToHPE/TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12")
|
||||||
|
assert body == b"hello-world"
|
||||||
|
|
||||||
|
|
||||||
|
def test_stub_read_file_missing_raises(sftp_block):
|
||||||
|
client = SftpClient(sftp_block)
|
||||||
|
with pytest.raises(FileNotFoundError):
|
||||||
|
client.read_file("/ToHPE/does-not-exist.x12")
|
||||||
@@ -0,0 +1,263 @@
|
|||||||
|
"""SP9 — R200-R210 validation rule tests.
|
||||||
|
|
||||||
|
These tests use the in-code PAYER_FACTORIES (which include the live
|
||||||
|
payer_config blocks loaded from YAML) so the rules have a non-empty
|
||||||
|
cfg_block to read. The live ``cyclone.payers`` registry is the source
|
||||||
|
of truth for CO_MAP; the rules fall back to the in-code PayerConfig
|
||||||
|
when the registry is empty.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
from decimal import Decimal
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cyclone import db as db_mod
|
||||||
|
from cyclone import payers as payer_loader
|
||||||
|
from cyclone.parsers.models import (
|
||||||
|
Address,
|
||||||
|
BillingProvider,
|
||||||
|
ClaimHeader,
|
||||||
|
ClaimOutput,
|
||||||
|
Diagnosis,
|
||||||
|
Payer,
|
||||||
|
Procedure,
|
||||||
|
ServiceLine,
|
||||||
|
Subscriber,
|
||||||
|
ValidationReport,
|
||||||
|
)
|
||||||
|
from cyclone.parsers.payer import PayerConfig
|
||||||
|
from cyclone.parsers.validator import validate
|
||||||
|
from cyclone.store import store
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _seed_db(tmp_path, monkeypatch):
|
||||||
|
"""Seed an in-memory DB with 3 providers + CO_TXIX payer, load YAML."""
|
||||||
|
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||||
|
db_mod._reset_for_tests()
|
||||||
|
db_mod.init_db()
|
||||||
|
payer_loader.load_payer_configs()
|
||||||
|
store.ensure_clearhouse_seeded()
|
||||||
|
yield
|
||||||
|
db_mod._reset_for_tests()
|
||||||
|
payer_loader.reset()
|
||||||
|
|
||||||
|
|
||||||
|
def _co_txix_cfg() -> PayerConfig:
|
||||||
|
"""Build a PayerConfig matching the live CO_TXIX YAML block."""
|
||||||
|
return PayerConfig(
|
||||||
|
name="Colorado Medical Assistance Program",
|
||||||
|
sbr09_claim_filing="MC",
|
||||||
|
allowed_claim_frequencies={1, 7, 8},
|
||||||
|
require_ref_g1_for_adjustments=False,
|
||||||
|
allowed_bht06={"CH", "RP"},
|
||||||
|
payer_id="CO_TXIX",
|
||||||
|
payer_name="COHCPF",
|
||||||
|
no_patient_loop=True,
|
||||||
|
encounter_claim_in_same_batch=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _claim(*, npi="1881068062", sbr09=None, prv_code=None, payer_id="CO_TXIX",
|
||||||
|
payer_name="COHCPF", ref_ei="721587149", transaction_type_code="CH",
|
||||||
|
has_pwk=False) -> ClaimOutput:
|
||||||
|
raw = []
|
||||||
|
if sbr09 is not None:
|
||||||
|
raw.append(["SBR", "P", "18", "", "", "", "", "", "", sbr09])
|
||||||
|
if prv_code is not None:
|
||||||
|
raw.append(["PRV", "BI", "PXC", f"PXC{prv_code}"])
|
||||||
|
raw.append(["NM1", "85", "2", "TOC, Inc.", "", "", "", "", "XX", npi])
|
||||||
|
if ref_ei is not None:
|
||||||
|
raw.append(["REF", "EI", ref_ei])
|
||||||
|
if has_pwk:
|
||||||
|
raw.append(["PWK"])
|
||||||
|
return ClaimOutput(
|
||||||
|
claim_id="CLM1",
|
||||||
|
control_number="0001",
|
||||||
|
transaction_date=date(2026, 6, 20),
|
||||||
|
billing_provider=BillingProvider(
|
||||||
|
name="TOC, Inc.",
|
||||||
|
npi=npi,
|
||||||
|
tax_id=ref_ei,
|
||||||
|
address=Address(line1="1100 East Main St", city="Montrose", state="CO", zip="81401"),
|
||||||
|
),
|
||||||
|
subscriber=Subscriber(first_name="John", last_name="Doe", member_id="M1"),
|
||||||
|
payer=Payer(name=payer_name, id=payer_id),
|
||||||
|
claim=ClaimHeader(
|
||||||
|
claim_id="CLM1",
|
||||||
|
total_charge=Decimal("100.00"),
|
||||||
|
frequency_code="1",
|
||||||
|
place_of_service="11",
|
||||||
|
facility_code_qualifier="B",
|
||||||
|
),
|
||||||
|
transaction_type_code=transaction_type_code,
|
||||||
|
diagnoses=[Diagnosis(code="R69")],
|
||||||
|
service_lines=[
|
||||||
|
ServiceLine(
|
||||||
|
line_number=1,
|
||||||
|
procedure=Procedure(qualifier="HC", code="99213"),
|
||||||
|
charge=Decimal("100.00"),
|
||||||
|
units=Decimal("1"),
|
||||||
|
),
|
||||||
|
],
|
||||||
|
raw_segments=raw,
|
||||||
|
validation=ValidationReport(passed=True, errors=[], warnings=[]),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R200: BHT06 allowed ------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r200_passes_for_allowed_bht06():
|
||||||
|
c = _claim(transaction_type_code="CH")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
bad = [i for i in report.errors if i.rule.startswith("R200")]
|
||||||
|
assert bad == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_r200_fails_for_disallowed_bht06():
|
||||||
|
c = _claim(transaction_type_code="XX")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
bad = [i for i in report.errors if i.rule == "R200_bht06_allowed"]
|
||||||
|
assert len(bad) == 1
|
||||||
|
assert "not in" in bad[0].message
|
||||||
|
|
||||||
|
|
||||||
|
def test_r200_skips_when_no_bht06():
|
||||||
|
c = _claim(transaction_type_code="")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R200_bht06_allowed"]
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R201: BHT06 no mixed batch ------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r201_is_a_per_claim_noop():
|
||||||
|
# Batch-level rule; per-claim the rule yields nothing
|
||||||
|
c = _claim(transaction_type_code="CH")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R201_bht06_no_mixed_batch"]
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R202: SBR09 allowed ------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r202_passes_for_mc():
|
||||||
|
c = _claim(sbr09="MC")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R202_sbr09_allowed"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_r202_fails_for_unknown():
|
||||||
|
c = _claim(sbr09="AB")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
bad = [i for i in report.errors if i.rule == "R202_sbr09_allowed"]
|
||||||
|
assert len(bad) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_r202_passes_for_zz_mco_encounter():
|
||||||
|
c = _claim(sbr09="ZZ")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R202_sbr09_allowed"]
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R203: PRV matches provider ----------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r203_passes_when_prv_matches_provider_taxonomy():
|
||||||
|
c = _claim(prv_code="251E00000X")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R203_prv_matches_provider"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_r203_fails_when_prv_mismatch():
|
||||||
|
c = _claim(prv_code="999Z00000X")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
bad = [i for i in report.errors if i.rule == "R203_prv_matches_provider"]
|
||||||
|
assert len(bad) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R204: NPI in providers table --------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r204_passes_for_known_npi():
|
||||||
|
c = _claim(npi="1881068062")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R204_npi_in_providers_table"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_r204_fails_for_unknown_npi():
|
||||||
|
c = _claim(npi="9999999999")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
bad = [i for i in report.errors if i.rule == "R204_npi_in_providers_table"]
|
||||||
|
assert len(bad) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R205: REF*EI matches provider --------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r205_passes_when_ref_ei_matches():
|
||||||
|
c = _claim(ref_ei="721587149")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R205_ref_ei_matches_provider"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_r205_fails_when_ref_ei_mismatch():
|
||||||
|
c = _claim(ref_ei="000000000")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
bad = [i for i in report.errors if i.rule == "R205_ref_ei_matches_provider"]
|
||||||
|
assert len(bad) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R206: Payer ID matches ---------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r206_passes_for_co_txix():
|
||||||
|
c = _claim(payer_id="CO_TXIX")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R206_payer_id_matches"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_r206_fails_for_wrong_payer_id():
|
||||||
|
c = _claim(payer_id="ZZZZZZ")
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
bad = [i for i in report.errors if i.rule == "R206_payer_id_matches"]
|
||||||
|
assert len(bad) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R207: no PWK segment ----------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r207_passes_when_no_pwk():
|
||||||
|
c = _claim(has_pwk=False)
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R207_no_pwk_segment"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_r207_fails_when_pwk_present():
|
||||||
|
c = _claim(has_pwk=True)
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
bad = [i for i in report.errors if i.rule == "R207_no_pwk_segment"]
|
||||||
|
assert len(bad) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R208: 2320 CAS*PI* group ------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_r208_is_a_per_claim_noop():
|
||||||
|
c = _claim()
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule == "R208_cas_2320_group"]
|
||||||
|
|
||||||
|
|
||||||
|
# ----- R209, R210: filename rules (validate at submit / parse time) ----
|
||||||
|
|
||||||
|
|
||||||
|
def test_r209_r210_per_claim_noop():
|
||||||
|
# These run at file-routing time, not per-claim validate
|
||||||
|
c = _claim()
|
||||||
|
report = validate(c, _co_txix_cfg())
|
||||||
|
assert not [i for i in report.errors if i.rule in ("R209_outbound_filename", "R210_inbound_filename")]
|
||||||
@@ -100,6 +100,53 @@ def test_r020_npi_must_be_ten_digits():
|
|||||||
assert any(i.rule == "R020_npi_format" for i in report.errors)
|
assert any(i.rule == "R020_npi_format" for i in report.errors)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# R021 — NPI Luhn checksum (SP20)
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
def test_r021_npi_checksum_valid_passes_silently():
|
||||||
|
"""A valid Luhn NPI (CMS-published 1234567893) yields no R021 issue."""
|
||||||
|
cfg = PayerConfig.co_medicaid()
|
||||||
|
claim = _build_claim()
|
||||||
|
claim.billing_provider.npi = "1234567893"
|
||||||
|
report = validate(claim, cfg)
|
||||||
|
assert not any(i.rule == "R021_npi_checksum" for i in report.errors + report.warnings)
|
||||||
|
|
||||||
|
|
||||||
|
def test_r021_npi_checksum_bad_luhn_is_warning():
|
||||||
|
"""An NPI that passes format but fails Luhn is a WARNING, not an error."""
|
||||||
|
cfg = PayerConfig.co_medicaid()
|
||||||
|
claim = _build_claim()
|
||||||
|
claim.billing_provider.npi = "1234567890" # right length, wrong check digit
|
||||||
|
report = validate(claim, cfg)
|
||||||
|
assert report.passed is True # WARNINGs don't fail the report
|
||||||
|
assert any(i.rule == "R021_npi_checksum" and i.severity == "warning" for i in report.warnings)
|
||||||
|
|
||||||
|
|
||||||
|
def test_r021_npi_checksum_skipped_when_format_bad():
|
||||||
|
"""When R020 already flagged the format, R021 stays silent
|
||||||
|
(avoids a duplicate 'this NPI is wrong' message)."""
|
||||||
|
cfg = PayerConfig.co_medicaid()
|
||||||
|
claim = _build_claim()
|
||||||
|
claim.billing_provider.npi = "12345" # wrong length
|
||||||
|
report = validate(claim, cfg)
|
||||||
|
# R020 errors as expected; R021 stays quiet.
|
||||||
|
assert any(i.rule == "R020_npi_format" for i in report.errors)
|
||||||
|
assert not any(i.rule == "R021_npi_checksum" for i in report.errors + report.warnings)
|
||||||
|
|
||||||
|
|
||||||
|
def test_r021_npi_checksum_skipped_when_npi_missing():
|
||||||
|
"""An empty NPI doesn't trigger R021 (only R020 would, but R020
|
||||||
|
only fires when NPI is *present* and wrong). R021 must stay silent
|
||||||
|
when NPI is empty so we don't double-fire."""
|
||||||
|
cfg = PayerConfig.co_medicaid()
|
||||||
|
claim = _build_claim()
|
||||||
|
claim.billing_provider.npi = ""
|
||||||
|
report = validate(claim, cfg)
|
||||||
|
assert not any(i.rule == "R021_npi_checksum" for i in report.errors + report.warnings)
|
||||||
|
|
||||||
|
|
||||||
def test_r030_frequency_allowed():
|
def test_r030_frequency_allowed():
|
||||||
cfg = PayerConfig.co_medicaid() # only 1, 7, 8
|
cfg = PayerConfig.co_medicaid() # only 1, 7, 8
|
||||||
claim = _build_claim()
|
claim = _build_claim()
|
||||||
|
|||||||
Generated
+464
-1
@@ -33,6 +33,85 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/ba/16/9826f089383c593cdfc4a6e5aca94d9e91ae1692c57af82c3b2aa5e810f7/anyio-4.14.0-py3-none-any.whl", hash = "sha256:dd9b7a2a9799ed6552fde617b2c5df02b7fdd7d88392fc48101e51bae46164d9", size = 123506, upload-time = "2026-06-15T22:00:47.595Z" },
|
{ url = "https://files.pythonhosted.org/packages/ba/16/9826f089383c593cdfc4a6e5aca94d9e91ae1692c57af82c3b2aa5e810f7/anyio-4.14.0-py3-none-any.whl", hash = "sha256:dd9b7a2a9799ed6552fde617b2c5df02b7fdd7d88392fc48101e51bae46164d9", size = 123506, upload-time = "2026-06-15T22:00:47.595Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "backports-tarfile"
|
||||||
|
version = "1.2.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/86/72/cd9b395f25e290e633655a100af28cb253e4393396264a98bd5f5951d50f/backports_tarfile-1.2.0.tar.gz", hash = "sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991", size = 86406, upload-time = "2024-05-28T17:01:54.731Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b9/fa/123043af240e49752f1c4bd24da5053b6bd00cad78c2be53c0d1e8b975bc/backports.tarfile-1.2.0-py3-none-any.whl", hash = "sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34", size = 30181, upload-time = "2024-05-28T17:01:53.112Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "bcrypt"
|
||||||
|
version = "5.0.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/d4/36/3329e2518d70ad8e2e5817d5a4cac6bba05a47767ec416c7d020a965f408/bcrypt-5.0.0.tar.gz", hash = "sha256:f748f7c2d6fd375cc93d3fba7ef4a9e3a092421b8dbf34d8d4dc06be9492dfdd", size = 25386, upload-time = "2025-09-25T19:50:47.829Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/13/85/3e65e01985fddf25b64ca67275bb5bdb4040bd1a53b66d355c6c37c8a680/bcrypt-5.0.0-cp313-cp313t-macosx_10_12_universal2.whl", hash = "sha256:f3c08197f3039bec79cee59a606d62b96b16669cff3949f21e74796b6e3cd2be", size = 481806, upload-time = "2025-09-25T19:49:05.102Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/44/dc/01eb79f12b177017a726cbf78330eb0eb442fae0e7b3dfd84ea2849552f3/bcrypt-5.0.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:200af71bc25f22006f4069060c88ed36f8aa4ff7f53e67ff04d2ab3f1e79a5b2", size = 268626, upload-time = "2025-09-25T19:49:06.723Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8c/cf/e82388ad5959c40d6afd94fb4743cc077129d45b952d46bdc3180310e2df/bcrypt-5.0.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:baade0a5657654c2984468efb7d6c110db87ea63ef5a4b54732e7e337253e44f", size = 271853, upload-time = "2025-09-25T19:49:08.028Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ec/86/7134b9dae7cf0efa85671651341f6afa695857fae172615e960fb6a466fa/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:c58b56cdfb03202b3bcc9fd8daee8e8e9b6d7e3163aa97c631dfcfcc24d36c86", size = 269793, upload-time = "2025-09-25T19:49:09.727Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/cc/82/6296688ac1b9e503d034e7d0614d56e80c5d1a08402ff856a4549cb59207/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:4bfd2a34de661f34d0bda43c3e4e79df586e4716ef401fe31ea39d69d581ef23", size = 289930, upload-time = "2025-09-25T19:49:11.204Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d1/18/884a44aa47f2a3b88dd09bc05a1e40b57878ecd111d17e5bba6f09f8bb77/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:ed2e1365e31fc73f1825fa830f1c8f8917ca1b3ca6185773b349c20fd606cec2", size = 272194, upload-time = "2025-09-25T19:49:12.524Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0e/8f/371a3ab33c6982070b674f1788e05b656cfbf5685894acbfef0c65483a59/bcrypt-5.0.0-cp313-cp313t-manylinux_2_34_aarch64.whl", hash = "sha256:83e787d7a84dbbfba6f250dd7a5efd689e935f03dd83b0f919d39349e1f23f83", size = 269381, upload-time = "2025-09-25T19:49:14.308Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b1/34/7e4e6abb7a8778db6422e88b1f06eb07c47682313997ee8a8f9352e5a6f1/bcrypt-5.0.0-cp313-cp313t-manylinux_2_34_x86_64.whl", hash = "sha256:137c5156524328a24b9fac1cb5db0ba618bc97d11970b39184c1d87dc4bf1746", size = 271750, upload-time = "2025-09-25T19:49:15.584Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c0/1b/54f416be2499bd72123c70d98d36c6cd61a4e33d9b89562c22481c81bb30/bcrypt-5.0.0-cp313-cp313t-musllinux_1_1_aarch64.whl", hash = "sha256:38cac74101777a6a7d3b3e3cfefa57089b5ada650dce2baf0cbdd9d65db22a9e", size = 303757, upload-time = "2025-09-25T19:49:17.244Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/13/62/062c24c7bcf9d2826a1a843d0d605c65a755bc98002923d01fd61270705a/bcrypt-5.0.0-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:d8d65b564ec849643d9f7ea05c6d9f0cd7ca23bdd4ac0c2dbef1104ab504543d", size = 306740, upload-time = "2025-09-25T19:49:18.693Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d5/c8/1fdbfc8c0f20875b6b4020f3c7dc447b8de60aa0be5faaf009d24242aec9/bcrypt-5.0.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:741449132f64b3524e95cd30e5cd3343006ce146088f074f31ab26b94e6c75ba", size = 334197, upload-time = "2025-09-25T19:49:20.523Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a6/c1/8b84545382d75bef226fbc6588af0f7b7d095f7cd6a670b42a86243183cd/bcrypt-5.0.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:212139484ab3207b1f0c00633d3be92fef3c5f0af17cad155679d03ff2ee1e41", size = 352974, upload-time = "2025-09-25T19:49:22.254Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/10/a6/ffb49d4254ed085e62e3e5dd05982b4393e32fe1e49bb1130186617c29cd/bcrypt-5.0.0-cp313-cp313t-win32.whl", hash = "sha256:9d52ed507c2488eddd6a95bccee4e808d3234fa78dd370e24bac65a21212b861", size = 148498, upload-time = "2025-09-25T19:49:24.134Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/48/a9/259559edc85258b6d5fc5471a62a3299a6aa37a6611a169756bf4689323c/bcrypt-5.0.0-cp313-cp313t-win_amd64.whl", hash = "sha256:f6984a24db30548fd39a44360532898c33528b74aedf81c26cf29c51ee47057e", size = 145853, upload-time = "2025-09-25T19:49:25.702Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2d/df/9714173403c7e8b245acf8e4be8876aac64a209d1b392af457c79e60492e/bcrypt-5.0.0-cp313-cp313t-win_arm64.whl", hash = "sha256:9fffdb387abe6aa775af36ef16f55e318dcda4194ddbf82007a6f21da29de8f5", size = 139626, upload-time = "2025-09-25T19:49:26.928Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f8/14/c18006f91816606a4abe294ccc5d1e6f0e42304df5a33710e9e8e95416e1/bcrypt-5.0.0-cp314-cp314t-macosx_10_12_universal2.whl", hash = "sha256:4870a52610537037adb382444fefd3706d96d663ac44cbb2f37e3919dca3d7ef", size = 481862, upload-time = "2025-09-25T19:49:28.365Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/67/49/dd074d831f00e589537e07a0725cf0e220d1f0d5d8e85ad5bbff251c45aa/bcrypt-5.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:48f753100931605686f74e27a7b49238122aa761a9aefe9373265b8b7aa43ea4", size = 268544, upload-time = "2025-09-25T19:49:30.39Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f5/91/50ccba088b8c474545b034a1424d05195d9fcbaaf802ab8bfe2be5a4e0d7/bcrypt-5.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f70aadb7a809305226daedf75d90379c397b094755a710d7014b8b117df1ebbf", size = 271787, upload-time = "2025-09-25T19:49:32.144Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/aa/e7/d7dba133e02abcda3b52087a7eea8c0d4f64d3e593b4fffc10c31b7061f3/bcrypt-5.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:744d3c6b164caa658adcb72cb8cc9ad9b4b75c7db507ab4bc2480474a51989da", size = 269753, upload-time = "2025-09-25T19:49:33.885Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/33/fc/5b145673c4b8d01018307b5c2c1fc87a6f5a436f0ad56607aee389de8ee3/bcrypt-5.0.0-cp314-cp314t-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a28bc05039bdf3289d757f49d616ab3efe8cf40d8e8001ccdd621cd4f98f4fc9", size = 289587, upload-time = "2025-09-25T19:49:35.144Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/27/d7/1ff22703ec6d4f90e62f1a5654b8867ef96bafb8e8102c2288333e1a6ca6/bcrypt-5.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:7f277a4b3390ab4bebe597800a90da0edae882c6196d3038a73adf446c4f969f", size = 272178, upload-time = "2025-09-25T19:49:36.793Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c8/88/815b6d558a1e4d40ece04a2f84865b0fef233513bd85fd0e40c294272d62/bcrypt-5.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:79cfa161eda8d2ddf29acad370356b47f02387153b11d46042e93a0a95127493", size = 269295, upload-time = "2025-09-25T19:49:38.164Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/51/8c/e0db387c79ab4931fc89827d37608c31cc57b6edc08ccd2386139028dc0d/bcrypt-5.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:a5393eae5722bcef046a990b84dff02b954904c36a194f6cfc817d7dca6c6f0b", size = 271700, upload-time = "2025-09-25T19:49:39.917Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/06/83/1570edddd150f572dbe9fc00f6203a89fc7d4226821f67328a85c330f239/bcrypt-5.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7f4c94dec1b5ab5d522750cb059bb9409ea8872d4494fd152b53cca99f1ddd8c", size = 334034, upload-time = "2025-09-25T19:49:41.227Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c9/f2/ea64e51a65e56ae7a8a4ec236c2bfbdd4b23008abd50ac33fbb2d1d15424/bcrypt-5.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:0cae4cb350934dfd74c020525eeae0a5f79257e8a201c0c176f4b84fdbf2a4b4", size = 352766, upload-time = "2025-09-25T19:49:43.08Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d7/d4/1a388d21ee66876f27d1a1f41287897d0c0f1712ef97d395d708ba93004c/bcrypt-5.0.0-cp314-cp314t-win32.whl", hash = "sha256:b17366316c654e1ad0306a6858e189fc835eca39f7eb2cafd6aaca8ce0c40a2e", size = 152449, upload-time = "2025-09-25T19:49:44.971Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3f/61/3291c2243ae0229e5bca5d19f4032cecad5dfb05a2557169d3a69dc0ba91/bcrypt-5.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:92864f54fb48b4c718fc92a32825d0e42265a627f956bc0361fe869f1adc3e7d", size = 149310, upload-time = "2025-09-25T19:49:46.162Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3e/89/4b01c52ae0c1a681d4021e5dd3e45b111a8fb47254a274fa9a378d8d834b/bcrypt-5.0.0-cp314-cp314t-win_arm64.whl", hash = "sha256:dd19cf5184a90c873009244586396a6a884d591a5323f0e8a5922560718d4993", size = 143761, upload-time = "2025-09-25T19:49:47.345Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/84/29/6237f151fbfe295fe3e074ecc6d44228faa1e842a81f6d34a02937ee1736/bcrypt-5.0.0-cp38-abi3-macosx_10_12_universal2.whl", hash = "sha256:fc746432b951e92b58317af8e0ca746efe93e66555f1b40888865ef5bf56446b", size = 494553, upload-time = "2025-09-25T19:49:49.006Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/45/b6/4c1205dde5e464ea3bd88e8742e19f899c16fa8916fb8510a851fae985b5/bcrypt-5.0.0-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c2388ca94ffee269b6038d48747f4ce8df0ffbea43f31abfa18ac72f0218effb", size = 275009, upload-time = "2025-09-25T19:49:50.581Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3b/71/427945e6ead72ccffe77894b2655b695ccf14ae1866cd977e185d606dd2f/bcrypt-5.0.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:560ddb6ec730386e7b3b26b8b4c88197aaed924430e7b74666a586ac997249ef", size = 278029, upload-time = "2025-09-25T19:49:52.533Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/17/72/c344825e3b83c5389a369c8a8e58ffe1480b8a699f46c127c34580c4666b/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:d79e5c65dcc9af213594d6f7f1fa2c98ad3fc10431e7aa53c176b441943efbdd", size = 275907, upload-time = "2025-09-25T19:49:54.709Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0b/7e/d4e47d2df1641a36d1212e5c0514f5291e1a956a7749f1e595c07a972038/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2b732e7d388fa22d48920baa267ba5d97cca38070b69c0e2d37087b381c681fd", size = 296500, upload-time = "2025-09-25T19:49:56.013Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0f/c3/0ae57a68be2039287ec28bc463b82e4b8dc23f9d12c0be331f4782e19108/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:0c8e093ea2532601a6f686edbc2c6b2ec24131ff5c52f7610dd64fa4553b5464", size = 278412, upload-time = "2025-09-25T19:49:57.356Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/45/2b/77424511adb11e6a99e3a00dcc7745034bee89036ad7d7e255a7e47be7d8/bcrypt-5.0.0-cp38-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:5b1589f4839a0899c146e8892efe320c0fa096568abd9b95593efac50a87cb75", size = 275486, upload-time = "2025-09-25T19:49:59.116Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/43/0a/405c753f6158e0f3f14b00b462d8bca31296f7ecfc8fc8bc7919c0c7d73a/bcrypt-5.0.0-cp38-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:89042e61b5e808b67daf24a434d89bab164d4de1746b37a8d173b6b14f3db9ff", size = 277940, upload-time = "2025-09-25T19:50:00.869Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/62/83/b3efc285d4aadc1fa83db385ec64dcfa1707e890eb42f03b127d66ac1b7b/bcrypt-5.0.0-cp38-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:e3cf5b2560c7b5a142286f69bde914494b6d8f901aaa71e453078388a50881c4", size = 310776, upload-time = "2025-09-25T19:50:02.393Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/95/7d/47ee337dacecde6d234890fe929936cb03ebc4c3a7460854bbd9c97780b8/bcrypt-5.0.0-cp38-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:f632fd56fc4e61564f78b46a2269153122db34988e78b6be8b32d28507b7eaeb", size = 312922, upload-time = "2025-09-25T19:50:04.232Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d6/3a/43d494dfb728f55f4e1cf8fd435d50c16a2d75493225b54c8d06122523c6/bcrypt-5.0.0-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:801cad5ccb6b87d1b430f183269b94c24f248dddbbc5c1f78b6ed231743e001c", size = 341367, upload-time = "2025-09-25T19:50:05.559Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/55/ab/a0727a4547e383e2e22a630e0f908113db37904f58719dc48d4622139b5c/bcrypt-5.0.0-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:3cf67a804fc66fc217e6914a5635000259fbbbb12e78a99488e4d5ba445a71eb", size = 359187, upload-time = "2025-09-25T19:50:06.916Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1b/bb/461f352fdca663524b4643d8b09e8435b4990f17fbf4fea6bc2a90aa0cc7/bcrypt-5.0.0-cp38-abi3-win32.whl", hash = "sha256:3abeb543874b2c0524ff40c57a4e14e5d3a66ff33fb423529c88f180fd756538", size = 153752, upload-time = "2025-09-25T19:50:08.515Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/41/aa/4190e60921927b7056820291f56fc57d00d04757c8b316b2d3c0d1d6da2c/bcrypt-5.0.0-cp38-abi3-win_amd64.whl", hash = "sha256:35a77ec55b541e5e583eb3436ffbbf53b0ffa1fa16ca6782279daf95d146dcd9", size = 150881, upload-time = "2025-09-25T19:50:09.742Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/54/12/cd77221719d0b39ac0b55dbd39358db1cd1246e0282e104366ebbfb8266a/bcrypt-5.0.0-cp38-abi3-win_arm64.whl", hash = "sha256:cde08734f12c6a4e28dc6755cd11d3bdfea608d93d958fffbe95a7026ebe4980", size = 144931, upload-time = "2025-09-25T19:50:11.016Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5d/ba/2af136406e1c3839aea9ecadc2f6be2bcd1eff255bd451dd39bcf302c47a/bcrypt-5.0.0-cp39-abi3-macosx_10_12_universal2.whl", hash = "sha256:0c418ca99fd47e9c59a301744d63328f17798b5947b0f791e9af3c1c499c2d0a", size = 495313, upload-time = "2025-09-25T19:50:12.309Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ac/ee/2f4985dbad090ace5ad1f7dd8ff94477fe089b5fab2040bd784a3d5f187b/bcrypt-5.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ddb4e1500f6efdd402218ffe34d040a1196c072e07929b9820f363a1fd1f4191", size = 275290, upload-time = "2025-09-25T19:50:13.673Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e4/6e/b77ade812672d15cf50842e167eead80ac3514f3beacac8902915417f8b7/bcrypt-5.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7aeef54b60ceddb6f30ee3db090351ecf0d40ec6e2abf41430997407a46d2254", size = 278253, upload-time = "2025-09-25T19:50:15.089Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/36/c4/ed00ed32f1040f7990dac7115f82273e3c03da1e1a1587a778d8cea496d8/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:f0ce778135f60799d89c9693b9b398819d15f1921ba15fe719acb3178215a7db", size = 276084, upload-time = "2025-09-25T19:50:16.699Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e7/c4/fa6e16145e145e87f1fa351bbd54b429354fd72145cd3d4e0c5157cf4c70/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a71f70ee269671460b37a449f5ff26982a6f2ba493b3eabdd687b4bf35f875ac", size = 297185, upload-time = "2025-09-25T19:50:18.525Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/24/b4/11f8a31d8b67cca3371e046db49baa7c0594d71eb40ac8121e2fc0888db0/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:f8429e1c410b4073944f03bd778a9e066e7fad723564a52ff91841d278dfc822", size = 278656, upload-time = "2025-09-25T19:50:19.809Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ac/31/79f11865f8078e192847d2cb526e3fa27c200933c982c5b2869720fa5fce/bcrypt-5.0.0-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:edfcdcedd0d0f05850c52ba3127b1fce70b9f89e0fe5ff16517df7e81fa3cbb8", size = 275662, upload-time = "2025-09-25T19:50:21.567Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d4/8d/5e43d9584b3b3591a6f9b68f755a4da879a59712981ef5ad2a0ac1379f7a/bcrypt-5.0.0-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:611f0a17aa4a25a69362dcc299fda5c8a3d4f160e2abb3831041feb77393a14a", size = 278240, upload-time = "2025-09-25T19:50:23.305Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/89/48/44590e3fc158620f680a978aafe8f87a4c4320da81ed11552f0323aa9a57/bcrypt-5.0.0-cp39-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:db99dca3b1fdc3db87d7c57eac0c82281242d1eabf19dcb8a6b10eb29a2e72d1", size = 311152, upload-time = "2025-09-25T19:50:24.597Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5f/85/e4fbfc46f14f47b0d20493669a625da5827d07e8a88ee460af6cd9768b44/bcrypt-5.0.0-cp39-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:5feebf85a9cefda32966d8171f5db7e3ba964b77fdfe31919622256f80f9cf42", size = 313284, upload-time = "2025-09-25T19:50:26.268Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/25/ae/479f81d3f4594456a01ea2f05b132a519eff9ab5768a70430fa1132384b1/bcrypt-5.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:3ca8a166b1140436e058298a34d88032ab62f15aae1c598580333dc21d27ef10", size = 341643, upload-time = "2025-09-25T19:50:28.02Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/df/d2/36a086dee1473b14276cd6ea7f61aef3b2648710b5d7f1c9e032c29b859f/bcrypt-5.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:61afc381250c3182d9078551e3ac3a41da14154fbff647ddf52a769f588c4172", size = 359698, upload-time = "2025-09-25T19:50:31.347Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c0/f6/688d2cd64bfd0b14d805ddb8a565e11ca1fb0fd6817175d58b10052b6d88/bcrypt-5.0.0-cp39-abi3-win32.whl", hash = "sha256:64d7ce196203e468c457c37ec22390f1a61c85c6f0b8160fd752940ccfb3a683", size = 153725, upload-time = "2025-09-25T19:50:34.384Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9f/b9/9d9a641194a730bda138b3dfe53f584d61c58cd5230e37566e83ec2ffa0d/bcrypt-5.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:64ee8434b0da054d830fa8e89e1c8bf30061d539044a39524ff7dec90481e5c2", size = 150912, upload-time = "2025-09-25T19:50:35.69Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/27/44/d2ef5e87509158ad2187f4dd0852df80695bb1ee0cfe0a684727b01a69e0/bcrypt-5.0.0-cp39-abi3-win_arm64.whl", hash = "sha256:f2347d3534e76bf50bca5500989d6c1d05ed64b440408057a37673282c654927", size = 144953, upload-time = "2025-09-25T19:50:37.32Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8a/75/4aa9f5a4d40d762892066ba1046000b329c7cd58e888a6db878019b282dc/bcrypt-5.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:7edda91d5ab52b15636d9c30da87d2cc84f426c72b9dba7a9b4fe142ba11f534", size = 271180, upload-time = "2025-09-25T19:50:38.575Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/54/79/875f9558179573d40a9cc743038ac2bf67dfb79cecb1e8b5d70e88c94c3d/bcrypt-5.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:046ad6db88edb3c5ece4369af997938fb1c19d6a699b9c1b27b0db432faae4c4", size = 273791, upload-time = "2025-09-25T19:50:39.913Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bc/fe/975adb8c216174bf70fc17535f75e85ac06ed5252ea077be10d9cff5ce24/bcrypt-5.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:dcd58e2b3a908b5ecc9b9df2f0085592506ac2d5110786018ee5e160f28e0911", size = 270746, upload-time = "2025-09-25T19:50:43.306Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e4/f8/972c96f5a2b6c4b3deca57009d93e946bbdbe2241dca9806d502f29dd3ee/bcrypt-5.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:6b8f520b61e8781efee73cba14e3e8c9556ccfb375623f4f97429544734545b4", size = 273375, upload-time = "2025-09-25T19:50:45.43Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "certifi"
|
name = "certifi"
|
||||||
version = "2026.6.17"
|
version = "2026.6.17"
|
||||||
@@ -42,6 +121,76 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/ef/2f/c5464532e965badff2f4c4c1a3a83f5697f0d7c407ed0cda44aaa99bb451/certifi-2026.6.17-py3-none-any.whl", hash = "sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db", size = 133289, upload-time = "2026-06-17T10:31:06.348Z" },
|
{ url = "https://files.pythonhosted.org/packages/ef/2f/c5464532e965badff2f4c4c1a3a83f5697f0d7c407ed0cda44aaa99bb451/certifi-2026.6.17-py3-none-any.whl", hash = "sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db", size = 133289, upload-time = "2026-06-17T10:31:06.348Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "cffi"
|
||||||
|
version = "2.0.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "pycparser", marker = "implementation_name != 'PyPy'" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/eb/56/b1ba7935a17738ae8453301356628e8147c79dbb825bcbc73dc7401f9846/cffi-2.0.0.tar.gz", hash = "sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529", size = 523588, upload-time = "2025-09-08T23:24:04.541Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/12/4a/3dfd5f7850cbf0d06dc84ba9aa00db766b52ca38d8b86e3a38314d52498c/cffi-2.0.0-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe", size = 184344, upload-time = "2025-09-08T23:22:26.456Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4f/8b/f0e4c441227ba756aafbe78f117485b25bb26b1c059d01f137fa6d14896b/cffi-2.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c", size = 180560, upload-time = "2025-09-08T23:22:28.197Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b1/b7/1200d354378ef52ec227395d95c2576330fd22a869f7a70e88e1447eb234/cffi-2.0.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92", size = 209613, upload-time = "2025-09-08T23:22:29.475Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b8/56/6033f5e86e8cc9bb629f0077ba71679508bdf54a9a5e112a3c0b91870332/cffi-2.0.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93", size = 216476, upload-time = "2025-09-08T23:22:31.063Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/dc/7f/55fecd70f7ece178db2f26128ec41430d8720f2d12ca97bf8f0a628207d5/cffi-2.0.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5", size = 203374, upload-time = "2025-09-08T23:22:32.507Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/84/ef/a7b77c8bdc0f77adc3b46888f1ad54be8f3b7821697a7b89126e829e676a/cffi-2.0.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664", size = 202597, upload-time = "2025-09-08T23:22:34.132Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d7/91/500d892b2bf36529a75b77958edfcd5ad8e2ce4064ce2ecfeab2125d72d1/cffi-2.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26", size = 215574, upload-time = "2025-09-08T23:22:35.443Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/44/64/58f6255b62b101093d5df22dcb752596066c7e89dd725e0afaed242a61be/cffi-2.0.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9", size = 218971, upload-time = "2025-09-08T23:22:36.805Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ab/49/fa72cebe2fd8a55fbe14956f9970fe8eb1ac59e5df042f603ef7c8ba0adc/cffi-2.0.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414", size = 211972, upload-time = "2025-09-08T23:22:38.436Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0b/28/dd0967a76aab36731b6ebfe64dec4e981aff7e0608f60c2d46b46982607d/cffi-2.0.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743", size = 217078, upload-time = "2025-09-08T23:22:39.776Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2b/c0/015b25184413d7ab0a410775fdb4a50fca20f5589b5dab1dbbfa3baad8ce/cffi-2.0.0-cp311-cp311-win32.whl", hash = "sha256:c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5", size = 172076, upload-time = "2025-09-08T23:22:40.95Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ae/8f/dc5531155e7070361eb1b7e4c1a9d896d0cb21c49f807a6c03fd63fc877e/cffi-2.0.0-cp311-cp311-win_amd64.whl", hash = "sha256:66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5", size = 182820, upload-time = "2025-09-08T23:22:42.463Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/95/5c/1b493356429f9aecfd56bc171285a4c4ac8697f76e9bbbbb105e537853a1/cffi-2.0.0-cp311-cp311-win_arm64.whl", hash = "sha256:c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d", size = 177635, upload-time = "2025-09-08T23:22:43.623Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ea/47/4f61023ea636104d4f16ab488e268b93008c3d0bb76893b1b31db1f96802/cffi-2.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d", size = 185271, upload-time = "2025-09-08T23:22:44.795Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/df/a2/781b623f57358e360d62cdd7a8c681f074a71d445418a776eef0aadb4ab4/cffi-2.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c", size = 181048, upload-time = "2025-09-08T23:22:45.938Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ff/df/a4f0fbd47331ceeba3d37c2e51e9dfc9722498becbeec2bd8bc856c9538a/cffi-2.0.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe", size = 212529, upload-time = "2025-09-08T23:22:47.349Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d5/72/12b5f8d3865bf0f87cf1404d8c374e7487dcf097a1c91c436e72e6badd83/cffi-2.0.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062", size = 220097, upload-time = "2025-09-08T23:22:48.677Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c2/95/7a135d52a50dfa7c882ab0ac17e8dc11cec9d55d2c18dda414c051c5e69e/cffi-2.0.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e", size = 207983, upload-time = "2025-09-08T23:22:50.06Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3a/c8/15cb9ada8895957ea171c62dc78ff3e99159ee7adb13c0123c001a2546c1/cffi-2.0.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037", size = 206519, upload-time = "2025-09-08T23:22:51.364Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/78/2d/7fa73dfa841b5ac06c7b8855cfc18622132e365f5b81d02230333ff26e9e/cffi-2.0.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba", size = 219572, upload-time = "2025-09-08T23:22:52.902Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/07/e0/267e57e387b4ca276b90f0434ff88b2c2241ad72b16d31836adddfd6031b/cffi-2.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94", size = 222963, upload-time = "2025-09-08T23:22:54.518Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b6/75/1f2747525e06f53efbd878f4d03bac5b859cbc11c633d0fb81432d98a795/cffi-2.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187", size = 221361, upload-time = "2025-09-08T23:22:55.867Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7b/2b/2b6435f76bfeb6bbf055596976da087377ede68df465419d192acf00c437/cffi-2.0.0-cp312-cp312-win32.whl", hash = "sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18", size = 172932, upload-time = "2025-09-08T23:22:57.188Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f8/ed/13bd4418627013bec4ed6e54283b1959cf6db888048c7cf4b4c3b5b36002/cffi-2.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5", size = 183557, upload-time = "2025-09-08T23:22:58.351Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/95/31/9f7f93ad2f8eff1dbc1c3656d7ca5bfd8fb52c9d786b4dcf19b2d02217fa/cffi-2.0.0-cp312-cp312-win_arm64.whl", hash = "sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6", size = 177762, upload-time = "2025-09-08T23:22:59.668Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4b/8d/a0a47a0c9e413a658623d014e91e74a50cdd2c423f7ccfd44086ef767f90/cffi-2.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb", size = 185230, upload-time = "2025-09-08T23:23:00.879Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4a/d2/a6c0296814556c68ee32009d9c2ad4f85f2707cdecfd7727951ec228005d/cffi-2.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca", size = 181043, upload-time = "2025-09-08T23:23:02.231Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b0/1e/d22cc63332bd59b06481ceaac49d6c507598642e2230f201649058a7e704/cffi-2.0.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b", size = 212446, upload-time = "2025-09-08T23:23:03.472Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a9/f5/a2c23eb03b61a0b8747f211eb716446c826ad66818ddc7810cc2cc19b3f2/cffi-2.0.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b", size = 220101, upload-time = "2025-09-08T23:23:04.792Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f2/7f/e6647792fc5850d634695bc0e6ab4111ae88e89981d35ac269956605feba/cffi-2.0.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2", size = 207948, upload-time = "2025-09-08T23:23:06.127Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/cb/1e/a5a1bd6f1fb30f22573f76533de12a00bf274abcdc55c8edab639078abb6/cffi-2.0.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3", size = 206422, upload-time = "2025-09-08T23:23:07.753Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/98/df/0a1755e750013a2081e863e7cd37e0cdd02664372c754e5560099eb7aa44/cffi-2.0.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26", size = 219499, upload-time = "2025-09-08T23:23:09.648Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/50/e1/a969e687fcf9ea58e6e2a928ad5e2dd88cc12f6f0ab477e9971f2309b57c/cffi-2.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c", size = 222928, upload-time = "2025-09-08T23:23:10.928Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/36/54/0362578dd2c9e557a28ac77698ed67323ed5b9775ca9d3fe73fe191bb5d8/cffi-2.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b", size = 221302, upload-time = "2025-09-08T23:23:12.42Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/eb/6d/bf9bda840d5f1dfdbf0feca87fbdb64a918a69bca42cfa0ba7b137c48cb8/cffi-2.0.0-cp313-cp313-win32.whl", hash = "sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27", size = 172909, upload-time = "2025-09-08T23:23:14.32Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/37/18/6519e1ee6f5a1e579e04b9ddb6f1676c17368a7aba48299c3759bbc3c8b3/cffi-2.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75", size = 183402, upload-time = "2025-09-08T23:23:15.535Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/cb/0e/02ceeec9a7d6ee63bb596121c2c8e9b3a9e150936f4fbef6ca1943e6137c/cffi-2.0.0-cp313-cp313-win_arm64.whl", hash = "sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91", size = 177780, upload-time = "2025-09-08T23:23:16.761Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/92/c4/3ce07396253a83250ee98564f8d7e9789fab8e58858f35d07a9a2c78de9f/cffi-2.0.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5", size = 185320, upload-time = "2025-09-08T23:23:18.087Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/59/dd/27e9fa567a23931c838c6b02d0764611c62290062a6d4e8ff7863daf9730/cffi-2.0.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13", size = 181487, upload-time = "2025-09-08T23:23:19.622Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d6/43/0e822876f87ea8a4ef95442c3d766a06a51fc5298823f884ef87aaad168c/cffi-2.0.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b", size = 220049, upload-time = "2025-09-08T23:23:20.853Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b4/89/76799151d9c2d2d1ead63c2429da9ea9d7aac304603de0c6e8764e6e8e70/cffi-2.0.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c", size = 207793, upload-time = "2025-09-08T23:23:22.08Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bb/dd/3465b14bb9e24ee24cb88c9e3730f6de63111fffe513492bf8c808a3547e/cffi-2.0.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef", size = 206300, upload-time = "2025-09-08T23:23:23.314Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/47/d9/d83e293854571c877a92da46fdec39158f8d7e68da75bf73581225d28e90/cffi-2.0.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775", size = 219244, upload-time = "2025-09-08T23:23:24.541Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2b/0f/1f177e3683aead2bb00f7679a16451d302c436b5cbf2505f0ea8146ef59e/cffi-2.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205", size = 222828, upload-time = "2025-09-08T23:23:26.143Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c6/0f/cafacebd4b040e3119dcb32fed8bdef8dfe94da653155f9d0b9dc660166e/cffi-2.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1", size = 220926, upload-time = "2025-09-08T23:23:27.873Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3e/aa/df335faa45b395396fcbc03de2dfcab242cd61a9900e914fe682a59170b1/cffi-2.0.0-cp314-cp314-win32.whl", hash = "sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f", size = 175328, upload-time = "2025-09-08T23:23:44.61Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bb/92/882c2d30831744296ce713f0feb4c1cd30f346ef747b530b5318715cc367/cffi-2.0.0-cp314-cp314-win_amd64.whl", hash = "sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25", size = 185650, upload-time = "2025-09-08T23:23:45.848Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9f/2c/98ece204b9d35a7366b5b2c6539c350313ca13932143e79dc133ba757104/cffi-2.0.0-cp314-cp314-win_arm64.whl", hash = "sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad", size = 180687, upload-time = "2025-09-08T23:23:47.105Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3e/61/c768e4d548bfa607abcda77423448df8c471f25dbe64fb2ef6d555eae006/cffi-2.0.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9", size = 188773, upload-time = "2025-09-08T23:23:29.347Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2c/ea/5f76bce7cf6fcd0ab1a1058b5af899bfbef198bea4d5686da88471ea0336/cffi-2.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d", size = 185013, upload-time = "2025-09-08T23:23:30.63Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/be/b4/c56878d0d1755cf9caa54ba71e5d049479c52f9e4afc230f06822162ab2f/cffi-2.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c", size = 221593, upload-time = "2025-09-08T23:23:31.91Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e0/0d/eb704606dfe8033e7128df5e90fee946bbcb64a04fcdaa97321309004000/cffi-2.0.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8", size = 209354, upload-time = "2025-09-08T23:23:33.214Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d8/19/3c435d727b368ca475fb8742ab97c9cb13a0de600ce86f62eab7fa3eea60/cffi-2.0.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc", size = 208480, upload-time = "2025-09-08T23:23:34.495Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d0/44/681604464ed9541673e486521497406fadcc15b5217c3e326b061696899a/cffi-2.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592", size = 221584, upload-time = "2025-09-08T23:23:36.096Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/25/8e/342a504ff018a2825d395d44d63a767dd8ebc927ebda557fecdaca3ac33a/cffi-2.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512", size = 224443, upload-time = "2025-09-08T23:23:37.328Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e1/5e/b666bacbbc60fbf415ba9988324a132c9a7a0448a9a8f125074671c0f2c3/cffi-2.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4", size = 223437, upload-time = "2025-09-08T23:23:38.945Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a0/1d/ec1a60bd1a10daa292d3cd6bb0b359a81607154fb8165f3ec95fe003b85c/cffi-2.0.0-cp314-cp314t-win32.whl", hash = "sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e", size = 180487, upload-time = "2025-09-08T23:23:40.423Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bf/41/4c1168c74fac325c0c8156f04b6749c8b6a8f405bbf91413ba088359f60d/cffi-2.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6", size = 191726, upload-time = "2025-09-08T23:23:41.742Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ae/3a/dbeec9d1ee0844c679f6bb5d6ad4e9f198b1224f4e7a32825f47f6192b0c/cffi-2.0.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9", size = 184195, upload-time = "2025-09-08T23:23:43.004Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "click"
|
name = "click"
|
||||||
version = "8.4.1"
|
version = "8.4.1"
|
||||||
@@ -167,6 +316,62 @@ toml = [
|
|||||||
{ name = "tomli", marker = "python_full_version <= '3.11'" },
|
{ name = "tomli", marker = "python_full_version <= '3.11'" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "cryptography"
|
||||||
|
version = "49.0.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "cffi", marker = "platform_python_implementation != 'PyPy'" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/1f/99/d1c90d6041656cc6ee229dc99cd67fd0cd5aec3c5f7d72fffc27cc750054/cryptography-49.0.0.tar.gz", hash = "sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493", size = 854345, upload-time = "2026-06-12T20:02:30.512Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9b/22/adf66990e63584a68dfb50c24f48a125c07b1699899381c8151e63ed458c/cryptography-49.0.0-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db", size = 4032100, upload-time = "2026-06-12T20:02:32.143Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/09/41/3797cfaf69cae04a13ee78ebd83f0678d9c02b4779d21ce24445326f1a69/cryptography-49.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db", size = 4692978, upload-time = "2026-06-12T20:01:21.305Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e6/8b/43011f7ebe515a8aa20d61f290a326cd890c2e738e16e59eaff8d9c3a412/cryptography-49.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325", size = 4716422, upload-time = "2026-06-12T20:01:48.566Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4a/91/01ce7303a4579e6d3a6abef01bd322848e9ea7a219adcabc5048b9033571/cryptography-49.0.0-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2", size = 4700503, upload-time = "2026-06-12T20:02:47.091Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/62/99/a2c95cf8293f07491e9e27c20cc4dcd18176d944e674679adeb1d0173fd6/cryptography-49.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b", size = 5309779, upload-time = "2026-06-12T20:02:08.987Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/20/2c/0622f20ff02b2ef32558733443805dc82fd4c275be01b2d19d14676f3a1b/cryptography-49.0.0-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6", size = 4749683, upload-time = "2026-06-12T20:02:03.335Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a3/5b/c5246635d5fd3b64e0d45ae10e99fd32fe9676a79915ccfe5a61ba9af1a5/cryptography-49.0.0-cp311-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c", size = 4337874, upload-time = "2026-06-12T20:02:54.323Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6d/88/05563c7fe2e914e87d1a536d06fe83e66b4e1d95cb593e05aea375531da8/cryptography-49.0.0-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7", size = 4700283, upload-time = "2026-06-12T20:01:34.822Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c4/b6/d7696e4e890d6ae1469935164c9e5215c557671cb78d6e3f458ccceaa632/cryptography-49.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68", size = 5265844, upload-time = "2026-06-12T20:01:24.09Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a9/3c/f3ad17eecc1a57b0ba236dc01f90e783c51f4a2f35f64777cc4f47a184b2/cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9", size = 4749290, upload-time = "2026-06-12T20:01:30.848Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4f/01/339573cf1023163a400b0b5d16f6d507de413b9f60be6fd1b77feeaf6737/cryptography-49.0.0-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f", size = 4834612, upload-time = "2026-06-12T20:01:29.246Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/71/fd/577302e213a1be9468f92d1afef66fcf1ef83d516819d9992ca547f592bd/cryptography-49.0.0-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459", size = 4980804, upload-time = "2026-06-12T20:01:42.853Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1f/09/f42b1d190c5ba75f72062a387f8030d1d75f6ab035788f1d9c4b01de6525/cryptography-49.0.0-cp311-abi3-win_amd64.whl", hash = "sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e", size = 3810026, upload-time = "2026-06-12T20:02:39.262Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ec/9e/db72b3ae7fc9cfad53e630e56c6ae83b9b6ff0bf3718ffb8012d20b3aabf/cryptography-49.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7", size = 4013892, upload-time = "2026-06-12T20:02:10.735Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/86/12/c48a424f38db03027be9f7ed5c7dc5de9933dbee992865f98b13727a009d/cryptography-49.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d", size = 4678835, upload-time = "2026-06-12T20:02:48.743Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/68/28/8a3ad4653662c93fc44dc4e5d8fd374c25c42e07b34bbfbadf49cf57a5a8/cryptography-49.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa", size = 4697239, upload-time = "2026-06-12T20:02:56.03Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a8/b2/2193fc74f81aee4f9b62733133b73b5176718932ed8f2e4b03fa040480a6/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb", size = 4685593, upload-time = "2026-06-12T20:02:50.666Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/47/f1/1d3eaa243bfc5de4a187b22aa8c048b3e4980bfbe830ac46e6bac2e66947/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d", size = 5289961, upload-time = "2026-06-12T20:01:46.468Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/58/39/2d51306721330c486495853eda1c567880ff036de15a14c4b74f399934af/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561", size = 4731145, upload-time = "2026-06-12T20:02:16.832Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/17/50/983e838c7fd0d87fd8c969bcdd328edaf5f756e38df5281637424c155873/cryptography-49.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122", size = 4321719, upload-time = "2026-06-12T20:02:52.611Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a7/f5/8f571d7e27c55bce9f76f026143bcb1e040a4233149ecca0bea5fa5dd5f7/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505", size = 4685209, upload-time = "2026-06-12T20:02:07.282Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e7/84/0e27016a6fc5a0886f797018b26aa42f40c09a82332bff77822a451deaaa/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866", size = 5246285, upload-time = "2026-06-12T20:01:32.439Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/11/2d/5e1fb307cb5931881516b464c98774b3f2c36b5d4bb9a2830253cf553cad/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8", size = 4730441, upload-time = "2026-06-12T20:02:01.469Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e4/c0/bff5a02ee731d207d6a1ed51732549d8c53d2bc8da1d10ec6f2844201d68/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3", size = 4815869, upload-time = "2026-06-12T20:01:36.574Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b9/26/814681d14248d95d73d5c3eea0c39a94eb8302df966f670a2c60de90974b/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27", size = 4960948, upload-time = "2026-06-12T20:02:18.688Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4c/fe/93ecac273d3738939d023612ad12cca9a3740a5345d69fda04134c43fd96/cryptography-49.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61", size = 3799153, upload-time = "2026-06-12T20:01:39.059Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/19/2a/5bb823f5bedcf80718cea7fbc95ec5515cca3769633c4b01a32be7f30e7c/cryptography-49.0.0-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a", size = 4025947, upload-time = "2026-06-12T20:01:25.745Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3d/df/40577043ca124e17012f408ddddaeb213b856336ac82ddb3bc915f39e29f/cryptography-49.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4", size = 4692429, upload-time = "2026-06-12T20:01:53.628Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2c/99/2d13299eb3dd27b02dcfaafcc91d6b5cb3329f7cbd6d8f51921acd566c1a/cryptography-49.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18", size = 4700968, upload-time = "2026-06-12T20:02:45.383Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a5/4d/9c0cd02f95e2602dd5e563da149ee0830abef3537be8b34dc56281ebe27a/cryptography-49.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69", size = 4697758, upload-time = "2026-06-12T20:01:41.13Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/24/01/186c825898477d77e2324d5360fefe622ff1d8d1963ec0554e2cada8ec77/cryptography-49.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64", size = 5298863, upload-time = "2026-06-12T20:02:24.579Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b8/7b/62cbbab75d0659865bf0273790031544a0b16c8072d258f9428dcd8190dc/cryptography-49.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21", size = 4735983, upload-time = "2026-06-12T20:01:50.14Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6c/72/3e798c064bc39e471008075d0f9bc9daf77a80879c092e4a8e170c585ed4/cryptography-49.0.0-cp39-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9", size = 4334173, upload-time = "2026-06-12T20:01:44.743Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f0/ee/6fca21d1ac73e06f8bef71940abfd4d2f6472b4bca284d770f32bd4086f6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc", size = 4697298, upload-time = "2026-06-12T20:02:20.918Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/67/d0/a5fcd3515f0bae49a7b6d0413cc1bdccdcc1fc0047037a0d480642cdc5d6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8", size = 5254338, upload-time = "2026-06-12T20:02:22.737Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a0/84/84fe36f19caf857d61cb7fc9c63035a47ffabd84ea12d1d393148efa3615/cryptography-49.0.0-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36", size = 4735650, upload-time = "2026-06-12T20:02:41.389Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6c/a0/db537264e234f7273a73ec020873d6d6b39dfd8a53db78b550ca8320440e/cryptography-49.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e", size = 4834820, upload-time = "2026-06-12T20:01:51.847Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/93/77/8df9eb486495979bccecd1062e2eaf435250e84437040295b57d09048b0b/cryptography-49.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b", size = 4967968, upload-time = "2026-06-12T20:02:12.524Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c2/e6/f60198ea8d9dfa15fff9ed4ca02ce362f6eadd9ba757dcc50634c4257b63/cryptography-49.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001", size = 3785547, upload-time = "2026-06-12T20:02:26.847Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/63/d3/4a83af35d65e3fad632c926fad684c193ea4398569ccb0bbbc7fe8f5dc9a/cryptography-49.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b", size = 3993685, upload-time = "2026-06-12T20:02:14.883Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d6/a7/f9dac0ab7f80368c56993a7bf638ef9935f825c91902798481fac0898138/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838", size = 4676239, upload-time = "2026-06-12T20:02:28.793Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d7/70/2ba3769dd0ae167e2f33dfa9592d45db6ff9a61d62ca1a5b3d1bdd09068f/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5", size = 4715584, upload-time = "2026-06-12T20:01:27.495Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/94/64/2923570ac1c0bd3a737aa366ac3abbbbde273042308b8cde95e2364a6e6a/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615", size = 4675885, upload-time = "2026-06-12T20:01:55.49Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ab/f8/614dc7e051418cfe53d55173c1e24c6b0085e89996fe90508c2fdf769aef/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6", size = 4715449, upload-time = "2026-06-12T20:02:05.469Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/aa/50/a9caea39ad19c431c1a3f8a31114df65b260cdfe67786b6c7e7c040c4c44/cryptography-49.0.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6", size = 3783731, upload-time = "2026-06-12T20:02:43.319Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "cyclone"
|
name = "cyclone"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
@@ -174,8 +379,10 @@ source = { editable = "." }
|
|||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "click" },
|
{ name = "click" },
|
||||||
{ name = "fastapi" },
|
{ name = "fastapi" },
|
||||||
|
{ name = "keyring" },
|
||||||
{ name = "pydantic" },
|
{ name = "pydantic" },
|
||||||
{ name = "python-multipart" },
|
{ name = "python-multipart" },
|
||||||
|
{ name = "pyyaml" },
|
||||||
{ name = "sqlalchemy" },
|
{ name = "sqlalchemy" },
|
||||||
{ name = "uvicorn", extra = ["standard"] },
|
{ name = "uvicorn", extra = ["standard"] },
|
||||||
]
|
]
|
||||||
@@ -187,21 +394,31 @@ dev = [
|
|||||||
{ name = "pytest-asyncio" },
|
{ name = "pytest-asyncio" },
|
||||||
{ name = "pytest-cov" },
|
{ name = "pytest-cov" },
|
||||||
]
|
]
|
||||||
|
sftp = [
|
||||||
|
{ name = "paramiko" },
|
||||||
|
]
|
||||||
|
sqlcipher = [
|
||||||
|
{ name = "sqlcipher3" },
|
||||||
|
]
|
||||||
|
|
||||||
[package.metadata]
|
[package.metadata]
|
||||||
requires-dist = [
|
requires-dist = [
|
||||||
{ name = "click", specifier = ">=8.1,<9" },
|
{ name = "click", specifier = ">=8.1,<9" },
|
||||||
{ name = "fastapi", specifier = ">=0.110,<1" },
|
{ name = "fastapi", specifier = ">=0.110,<1" },
|
||||||
{ name = "httpx", marker = "extra == 'dev'", specifier = ">=0.27,<1" },
|
{ name = "httpx", marker = "extra == 'dev'", specifier = ">=0.27,<1" },
|
||||||
|
{ name = "keyring", specifier = ">=25.0,<26" },
|
||||||
|
{ name = "paramiko", marker = "extra == 'sftp'", specifier = ">=3.4,<6" },
|
||||||
{ name = "pydantic", specifier = ">=2.6,<3" },
|
{ name = "pydantic", specifier = ">=2.6,<3" },
|
||||||
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0" },
|
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0" },
|
||||||
{ name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.23,<1" },
|
{ name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.23,<1" },
|
||||||
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.1" },
|
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.1" },
|
||||||
{ name = "python-multipart", specifier = ">=0.0.9,<1" },
|
{ name = "python-multipart", specifier = ">=0.0.9,<1" },
|
||||||
|
{ name = "pyyaml", specifier = ">=6.0,<7" },
|
||||||
{ name = "sqlalchemy", specifier = ">=2.0,<3" },
|
{ name = "sqlalchemy", specifier = ">=2.0,<3" },
|
||||||
|
{ name = "sqlcipher3", marker = "extra == 'sqlcipher'", specifier = ">=0.6,<1" },
|
||||||
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.27,<1" },
|
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.27,<1" },
|
||||||
]
|
]
|
||||||
provides-extras = ["dev"]
|
provides-extras = ["dev", "sqlcipher", "sftp"]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "fastapi"
|
name = "fastapi"
|
||||||
@@ -371,6 +588,18 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
|
{ url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "importlib-metadata"
|
||||||
|
version = "9.0.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "zipp" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/a9/01/15bb152d77b21318514a96f43af312635eb2500c96b55398d020c93d86ea/importlib_metadata-9.0.0.tar.gz", hash = "sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc", size = 56405, upload-time = "2026-03-20T06:42:56.999Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/38/3d/2d244233ac4f76e38533cfcb2991c9eb4c7bf688ae0a036d30725b8faafe/importlib_metadata-9.0.0-py3-none-any.whl", hash = "sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7", size = 27789, upload-time = "2026-03-20T06:42:55.665Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "iniconfig"
|
name = "iniconfig"
|
||||||
version = "2.3.0"
|
version = "2.3.0"
|
||||||
@@ -380,6 +609,87 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "invoke"
|
||||||
|
version = "3.0.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/33/f6/227c48c5fe47fa178ccf1fda8f047d16c97ba926567b661e9ce2045c600c/invoke-3.0.3.tar.gz", hash = "sha256:437b6a622223824380bfb4e64f612711a6b648c795f565efc8625af66fb57f0c", size = 343419, upload-time = "2026-04-07T15:17:48.307Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5a/de/bbc12563bbf979618d17625a4e753ff7a078523e28d870d3626daa97261a/invoke-3.0.3-py3-none-any.whl", hash = "sha256:f11327165e5cbb89b2ad1d88d3292b5113332c43b8553b494da435d6ec6f5053", size = 160958, upload-time = "2026-04-07T15:17:46.875Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "jaraco-classes"
|
||||||
|
version = "3.4.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "more-itertools" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz", hash = "sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", size = 11780, upload-time = "2024-03-31T07:27:36.643Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7f/66/b15ce62552d84bbfcec9a4873ab79d993a1dd4edb922cbfccae192bd5b5f/jaraco.classes-3.4.0-py3-none-any.whl", hash = "sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790", size = 6777, upload-time = "2024-03-31T07:27:34.792Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "jaraco-context"
|
||||||
|
version = "6.1.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "backports-tarfile", marker = "python_full_version < '3.12'" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/af/50/4763cd07e722bb6285316d390a164bc7e479db9d90daa769f22578f698b4/jaraco_context-6.1.2.tar.gz", hash = "sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3", size = 16801, upload-time = "2026-03-20T22:13:33.922Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f2/58/bc8954bda5fcda97bd7c19be11b85f91973d67a706ed4a3aec33e7de22db/jaraco_context-6.1.2-py3-none-any.whl", hash = "sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535", size = 7871, upload-time = "2026-03-20T22:13:32.808Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "jaraco-functools"
|
||||||
|
version = "4.5.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "more-itertools" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/36/cf/ea4ef2920830dea3f5ab2ea4da6fb67724e6dca80ee2553788c3607243d0/jaraco_functools-4.5.0.tar.gz", hash = "sha256:3bb5665ea4a020cf78a7040e89154c77edadb3ca74f366479669c5999aa70b03", size = 20272, upload-time = "2026-05-15T21:34:10.025Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/96/9a/982e48afcffcd727a9144506720ffd4224b6b7e355c98641866f38b7c043/jaraco_functools-4.5.0-py3-none-any.whl", hash = "sha256:79ce39246eddbde4b3a03b77ea5f0f7878dc669b166a66cf3fa8e266aa3fa2f4", size = 10594, upload-time = "2026-05-15T21:34:08.595Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "jeepney"
|
||||||
|
version = "0.9.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/7b/6f/357efd7602486741aa73ffc0617fb310a29b588ed0fd69c2399acbb85b0c/jeepney-0.9.0.tar.gz", hash = "sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732", size = 106758, upload-time = "2025-02-27T18:51:01.684Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b2/a3/e137168c9c44d18eff0376253da9f1e9234d0239e0ee230d2fee6cea8e55/jeepney-0.9.0-py3-none-any.whl", hash = "sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683", size = 49010, upload-time = "2025-02-27T18:51:00.104Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "keyring"
|
||||||
|
version = "25.7.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "importlib-metadata", marker = "python_full_version < '3.12'" },
|
||||||
|
{ name = "jaraco-classes" },
|
||||||
|
{ name = "jaraco-context" },
|
||||||
|
{ name = "jaraco-functools" },
|
||||||
|
{ name = "jeepney", marker = "sys_platform == 'linux'" },
|
||||||
|
{ name = "pywin32-ctypes", marker = "sys_platform == 'win32'" },
|
||||||
|
{ name = "secretstorage", marker = "sys_platform == 'linux'" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/43/4b/674af6ef2f97d56f0ab5153bf0bfa28ccb6c3ed4d1babf4305449668807b/keyring-25.7.0.tar.gz", hash = "sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b", size = 63516, upload-time = "2025-11-16T16:26:09.482Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/81/db/e655086b7f3a705df045bf0933bdd9c2f79bb3c97bfef1384598bb79a217/keyring-25.7.0-py3-none-any.whl", hash = "sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f", size = 39160, upload-time = "2025-11-16T16:26:08.402Z" },
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "more-itertools"
|
||||||
|
version = "11.1.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/de/1d/f4da6f02cdffe04d6362210b807146a26044c88d839208aec273bb0d9184/more_itertools-11.1.0.tar.gz", hash = "sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d", size = 145772, upload-time = "2026-05-22T14:14:29.909Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e8/3d/1087453384dbde46a8c7f9356eead2c58be8a7bf156bca40243377c85715/more_itertools-11.1.0-py3-none-any.whl", hash = "sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192", size = 72226, upload-time = "2026-05-22T14:14:28.824Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "packaging"
|
name = "packaging"
|
||||||
version = "26.2"
|
version = "26.2"
|
||||||
@@ -389,6 +699,21 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
|
{ url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "paramiko"
|
||||||
|
version = "5.0.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "bcrypt" },
|
||||||
|
{ name = "cryptography" },
|
||||||
|
{ name = "invoke" },
|
||||||
|
{ name = "pynacl" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/62/93/dcc25d52f49022ae6175d15e6bd751f1acc99b98bc61fc55e5155a7be2e7/paramiko-5.0.0.tar.gz", hash = "sha256:36763b5b95c2a0dcfdf1abc48e48156ee425b21efe2f0e787c2dd5a95c0e5e79", size = 1548586, upload-time = "2026-05-09T18:28:52.256Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/82/5b/eadf6d45de38d30ab603f49393b6cd2cbe7e233af8cf90197e32782b68a9/paramiko-5.0.0-py3-none-any.whl", hash = "sha256:b7044611c30140d9a75261653210e2002977b71a0497ff3ba0d98d7edbf62f7c", size = 208919, upload-time = "2026-05-09T18:28:50.295Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pluggy"
|
name = "pluggy"
|
||||||
version = "1.6.0"
|
version = "1.6.0"
|
||||||
@@ -398,6 +723,15 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
|
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pycparser"
|
||||||
|
version = "3.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz", hash = "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", size = 103492, upload-time = "2026-01-21T14:26:51.89Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pydantic"
|
name = "pydantic"
|
||||||
version = "2.13.4"
|
version = "2.13.4"
|
||||||
@@ -524,6 +858,41 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pynacl"
|
||||||
|
version = "1.6.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "cffi", marker = "platform_python_implementation != 'PyPy'" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/d9/9a/4019b524b03a13438637b11538c82781a5eda427394380381af8f04f467a/pynacl-1.6.2.tar.gz", hash = "sha256:018494d6d696ae03c7e656e5e74cdfd8ea1326962cc401bcf018f1ed8436811c", size = 3511692, upload-time = "2026-01-01T17:48:10.851Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4b/79/0e3c34dc3c4671f67d251c07aa8eb100916f250ee470df230b0ab89551b4/pynacl-1.6.2-cp314-cp314t-macosx_10_10_universal2.whl", hash = "sha256:622d7b07cc5c02c666795792931b50c91f3ce3c2649762efb1ef0d5684c81594", size = 390064, upload-time = "2026-01-01T17:31:57.264Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/eb/1c/23a26e931736e13b16483795c8a6b2f641bf6a3d5238c22b070a5112722c/pynacl-1.6.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d071c6a9a4c94d79eb665db4ce5cedc537faf74f2355e4d502591d850d3913c0", size = 809370, upload-time = "2026-01-01T17:31:59.198Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/87/74/8d4b718f8a22aea9e8dcc8b95deb76d4aae380e2f5b570cc70b5fd0a852d/pynacl-1.6.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:fe9847ca47d287af41e82be1dd5e23023d3c31a951da134121ab02e42ac218c9", size = 1408304, upload-time = "2026-01-01T17:32:01.162Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fd/73/be4fdd3a6a87fe8a4553380c2b47fbd1f7f58292eb820902f5c8ac7de7b0/pynacl-1.6.2-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:04316d1fc625d860b6c162fff704eb8426b1a8bcd3abacea11142cbd99a6b574", size = 844871, upload-time = "2026-01-01T17:32:02.824Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/55/ad/6efc57ab75ee4422e96b5f2697d51bbcf6cdcc091e66310df91fbdc144a8/pynacl-1.6.2-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:44081faff368d6c5553ccf55322ef2819abb40e25afaec7e740f159f74813634", size = 1446356, upload-time = "2026-01-01T17:32:04.452Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/78/b7/928ee9c4779caa0a915844311ab9fb5f99585621c5d6e4574538a17dca07/pynacl-1.6.2-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:a9f9932d8d2811ce1a8ffa79dcbdf3970e7355b5c8eb0c1a881a57e7f7d96e88", size = 826814, upload-time = "2026-01-01T17:32:06.078Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f7/a9/1bdba746a2be20f8809fee75c10e3159d75864ef69c6b0dd168fc60e485d/pynacl-1.6.2-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:bc4a36b28dd72fb4845e5d8f9760610588a96d5a51f01d84d8c6ff9849968c14", size = 1411742, upload-time = "2026-01-01T17:32:07.651Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f3/2f/5e7ea8d85f9f3ea5b6b87db1d8388daa3587eed181bdeb0306816fdbbe79/pynacl-1.6.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:3bffb6d0f6becacb6526f8f42adfb5efb26337056ee0831fb9a7044d1a964444", size = 801714, upload-time = "2026-01-01T17:32:09.558Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/06/ea/43fe2f7eab5f200e40fb10d305bf6f87ea31b3bbc83443eac37cd34a9e1e/pynacl-1.6.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:2fef529ef3ee487ad8113d287a593fa26f48ee3620d92ecc6f1d09ea38e0709b", size = 1372257, upload-time = "2026-01-01T17:32:11.026Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/4d/54/c9ea116412788629b1347e415f72195c25eb2f3809b2d3e7b25f5c79f13a/pynacl-1.6.2-cp314-cp314t-win32.whl", hash = "sha256:a84bf1c20339d06dc0c85d9aea9637a24f718f375d861b2668b2f9f96fa51145", size = 231319, upload-time = "2026-01-01T17:32:12.46Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ce/04/64e9d76646abac2dccf904fccba352a86e7d172647557f35b9fe2a5ee4a1/pynacl-1.6.2-cp314-cp314t-win_amd64.whl", hash = "sha256:320ef68a41c87547c91a8b58903c9caa641ab01e8512ce291085b5fe2fcb7590", size = 244044, upload-time = "2026-01-01T17:32:13.781Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/33/33/7873dc161c6a06f43cda13dec67b6fe152cb2f982581151956fa5e5cdb47/pynacl-1.6.2-cp314-cp314t-win_arm64.whl", hash = "sha256:d29bfe37e20e015a7d8b23cfc8bd6aa7909c92a1b8f41ee416bbb3e79ef182b2", size = 188740, upload-time = "2026-01-01T17:32:15.083Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/be/7b/4845bbf88e94586ec47a432da4e9107e3fc3ce37eb412b1398630a37f7dd/pynacl-1.6.2-cp38-abi3-macosx_10_10_universal2.whl", hash = "sha256:c949ea47e4206af7c8f604b8278093b674f7c79ed0d4719cc836902bf4517465", size = 388458, upload-time = "2026-01-01T17:32:16.829Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/1e/b4/e927e0653ba63b02a4ca5b4d852a8d1d678afbf69b3dbf9c4d0785ac905c/pynacl-1.6.2-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8845c0631c0be43abdd865511c41eab235e0be69c81dc66a50911594198679b0", size = 800020, upload-time = "2026-01-01T17:32:18.34Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/7f/81/d60984052df5c97b1d24365bc1e30024379b42c4edcd79d2436b1b9806f2/pynacl-1.6.2-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:22de65bb9010a725b0dac248f353bb072969c94fa8d6b1f34b87d7953cf7bbe4", size = 1399174, upload-time = "2026-01-01T17:32:20.239Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/68/f7/322f2f9915c4ef27d140101dd0ed26b479f7e6f5f183590fd32dfc48c4d3/pynacl-1.6.2-cp38-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:46065496ab748469cdd999246d17e301b2c24ae2fdf739132e580a0e94c94a87", size = 835085, upload-time = "2026-01-01T17:32:22.24Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3e/d0/f301f83ac8dbe53442c5a43f6a39016f94f754d7a9815a875b65e218a307/pynacl-1.6.2-cp38-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8a66d6fb6ae7661c58995f9c6435bda2b1e68b54b598a6a10247bfcdadac996c", size = 1437614, upload-time = "2026-01-01T17:32:23.766Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c4/58/fc6e649762b029315325ace1a8c6be66125e42f67416d3dbd47b69563d61/pynacl-1.6.2-cp38-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:26bfcd00dcf2cf160f122186af731ae30ab120c18e8375684ec2670dccd28130", size = 818251, upload-time = "2026-01-01T17:32:25.69Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c9/a8/b917096b1accc9acd878819a49d3d84875731a41eb665f6ebc826b1af99e/pynacl-1.6.2-cp38-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:c8a231e36ec2cab018c4ad4358c386e36eede0319a0c41fed24f840b1dac59f6", size = 1402859, upload-time = "2026-01-01T17:32:27.215Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/85/42/fe60b5f4473e12c72f977548e4028156f4d340b884c635ec6b063fe7e9a5/pynacl-1.6.2-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:68be3a09455743ff9505491220b64440ced8973fe930f270c8e07ccfa25b1f9e", size = 791926, upload-time = "2026-01-01T17:32:29.314Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fa/f9/e40e318c604259301cc091a2a63f237d9e7b424c4851cafaea4ea7c4834e/pynacl-1.6.2-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:8b097553b380236d51ed11356c953bf8ce36a29a3e596e934ecabe76c985a577", size = 1363101, upload-time = "2026-01-01T17:32:31.263Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/48/47/e761c254f410c023a469284a9bc210933e18588ca87706ae93002c05114c/pynacl-1.6.2-cp38-abi3-win32.whl", hash = "sha256:5811c72b473b2f38f7e2a3dc4f8642e3a3e9b5e7317266e4ced1fba85cae41aa", size = 227421, upload-time = "2026-01-01T17:32:33.076Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/41/ad/334600e8cacc7d86587fe5f565480fde569dfb487389c8e1be56ac21d8ac/pynacl-1.6.2-cp38-abi3-win_amd64.whl", hash = "sha256:62985f233210dee6548c223301b6c25440852e13d59a8b81490203c3227c5ba0", size = 239754, upload-time = "2026-01-01T17:32:34.557Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/29/7d/5945b5af29534641820d3bd7b00962abbbdfee84ec7e19f0d5b3175f9a31/pynacl-1.6.2-cp38-abi3-win_arm64.whl", hash = "sha256:834a43af110f743a754448463e8fd61259cd4ab5bbedcf70f9dabad1d28a394c", size = 184801, upload-time = "2026-01-01T17:32:36.309Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pytest"
|
name = "pytest"
|
||||||
version = "8.4.2"
|
version = "8.4.2"
|
||||||
@@ -584,6 +953,15 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/e1/04/e8135ebd1ad02c56ec633277529b2602ff99ff634be76cdba5744cf554fd/python_multipart-0.0.32-py3-none-any.whl", hash = "sha256:ff6d3f776f16878c894e52e107296ffc890e913c611b1a4ec6c44e2821fe2e23", size = 30042, upload-time = "2026-06-04T16:18:57.319Z" },
|
{ url = "https://files.pythonhosted.org/packages/e1/04/e8135ebd1ad02c56ec633277529b2602ff99ff634be76cdba5744cf554fd/python_multipart-0.0.32-py3-none-any.whl", hash = "sha256:ff6d3f776f16878c894e52e107296ffc890e913c611b1a4ec6c44e2821fe2e23", size = 30042, upload-time = "2026-06-04T16:18:57.319Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "pywin32-ctypes"
|
||||||
|
version = "0.2.3"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/85/9f/01a1a99704853cb63f253eea009390c88e7131c67e66a0a02099a8c917cb/pywin32-ctypes-0.2.3.tar.gz", hash = "sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755", size = 29471, upload-time = "2024-08-14T10:15:34.626Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/de/3d/8161f7711c017e01ac9f008dfddd9410dff3674334c233bde66e7ba65bbf/pywin32_ctypes-0.2.3-py3-none-any.whl", hash = "sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8", size = 30756, upload-time = "2024-08-14T10:15:33.187Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pyyaml"
|
name = "pyyaml"
|
||||||
version = "6.0.3"
|
version = "6.0.3"
|
||||||
@@ -639,6 +1017,19 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },
|
{ url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "secretstorage"
|
||||||
|
version = "3.5.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "cryptography" },
|
||||||
|
{ name = "jeepney" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz", hash = "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", size = 19884, upload-time = "2025-11-23T19:02:53.191Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b7/46/f5af3402b579fd5e11573ce652019a67074317e18c1935cc0b4ba9b35552/secretstorage-3.5.0-py3-none-any.whl", hash = "sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137", size = 15554, upload-time = "2025-11-23T19:02:51.545Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "sqlalchemy"
|
name = "sqlalchemy"
|
||||||
version = "2.0.51"
|
version = "2.0.51"
|
||||||
@@ -687,6 +1078,69 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/e2/22/dbf013a12ec759e54a34a119e9e217435b3f71b2dd5c61a7ade0a25dae87/sqlalchemy-2.0.51-py3-none-any.whl", hash = "sha256:bb024d8b621d0be75f4f44ecc7c950450026e76d66dc8f791bb5331d7fed59d5", size = 1944334, upload-time = "2026-06-15T16:09:22.418Z" },
|
{ url = "https://files.pythonhosted.org/packages/e2/22/dbf013a12ec759e54a34a119e9e217435b3f71b2dd5c61a7ade0a25dae87/sqlalchemy-2.0.51-py3-none-any.whl", hash = "sha256:bb024d8b621d0be75f4f44ecc7c950450026e76d66dc8f791bb5331d7fed59d5", size = 1944334, upload-time = "2026-06-15T16:09:22.418Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "sqlcipher3"
|
||||||
|
version = "0.6.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/ae/c1/414003d77549c444bafd636149ab3ace6f4e2cb4666c9955d54ad62096cb/sqlcipher3-0.6.2.tar.gz", hash = "sha256:a2b675289ba8889f389625a21f3a01f1ff159a551b5b88fba8fd92da0e02380a", size = 2663213, upload-time = "2026-01-07T23:13:26.061Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f6/97/6461dc2ab41fbaed4942aaf7d1bac23d8cd130c751d40830ce391e80d332/sqlcipher3-0.6.2-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:87432804bf88e9017fc174bdd3d0862a1d1e9ef3c755517595c91da2c59e3808", size = 4942310, upload-time = "2026-01-07T23:11:58.393Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/45/dc/73f238aa994e08ac6971838907fd2b80d9bd86277c3c0e99300985287cd5/sqlcipher3-0.6.2-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:eece737c583c285e9bbe3bf829eee3b6624eb6e9dad8ccff7821a45641f436dd", size = 2893419, upload-time = "2026-01-07T23:11:59.762Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e5/4c/098cf3dd0af6ce4cfba88fbdeb63a3b156f4b7f0620f6cc5f35ecfb72607/sqlcipher3-0.6.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:22e6502c364706fe64695219877f2bb01cdb25450bec81e69c8a08deff8c14ee", size = 3160663, upload-time = "2026-01-07T23:12:01.011Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/71/09/5ae806f9f5833ab7decfef0b24471f57c06c10b7fdc2a097a3066d6c80d4/sqlcipher3-0.6.2-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:0ca92202881bcb69b3703b744b40a3a3476e122d4612a82eb2b0a36f2f78de1d", size = 7252828, upload-time = "2026-01-07T23:12:02.076Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c5/bd/10527a221d2b7c33b1c32058c3882a03033fca013b4f9f98f0befc0fe98e/sqlcipher3-0.6.2-cp311-cp311-manylinux_2_28_i686.whl", hash = "sha256:15abe3de01faa194f1aaea144ff9ecbfdce2991964dcc7ce8ec1ecc5950a4bc4", size = 6868146, upload-time = "2026-01-07T23:12:03.335Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/72/b0/faa2a8fc9dc3210e0af31e57c5ec86e8a523eaa3d44e854aa8f95ff66d50/sqlcipher3-0.6.2-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:0f08e5bb5eb1ab93819c444ebec61fa3349e9690c14f5d0276fd4f61c3049fd9", size = 7031136, upload-time = "2026-01-07T23:12:05.242Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/6d/37/1a82b3fc1504741df5aa4dccc6fd4265244e5b5dc98aa95e24321d73f0bb/sqlcipher3-0.6.2-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:dfe90f1e0e81a8c6c8c4129a8439ed6b9b27a8e32077c59ed3b7f1263e3c5544", size = 7245432, upload-time = "2026-01-07T23:12:07.346Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a3/e5/68bbaa1790e0f2fc571924933db01a6af2991ebf479496934ab5f1b19484/sqlcipher3-0.6.2-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:a5f805c1f156634e4e91f1b073d95930756fdf23eeeeb7b85c511a5cf165b10c", size = 7051580, upload-time = "2026-01-07T23:12:08.674Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5b/51/0939f292767fe26b978b845cefac077916f5b2f1caae110be89d2b3a79dc/sqlcipher3-0.6.2-cp311-cp311-win32.whl", hash = "sha256:fc08ff475ab0e0f43adca0647d827e81da5fa406bbb6bd04471e28a3ad2864d9", size = 1981400, upload-time = "2026-01-07T23:12:09.849Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9d/87/6cb7a6ced1244350514cf419cbeb442f12f445e3c4d09da0e7a49ac7ee80/sqlcipher3-0.6.2-cp311-cp311-win_amd64.whl", hash = "sha256:4ad7e4a32de907011ea22ac2012c9bca1bb414e2f599c56a55c8b0fe6445b932", size = 2485356, upload-time = "2026-01-07T23:12:11.253Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2c/2f/52f001d2b884047ab490964d37ee7891d497cc891cee7e64bac5672eb7d4/sqlcipher3-0.6.2-cp311-cp311-win_arm64.whl", hash = "sha256:3ad6b39a7fa8c2f7ec471dd29fadbffa19c194fbae1730f013f0d29f5b96fae0", size = 2652186, upload-time = "2026-01-07T23:12:12.488Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/96/ac/3612ebe2504d7c6786ddf73d28ae3d2707eb39d6e5730336854071ccb612/sqlcipher3-0.6.2-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:a51b18bd782652a2282f9cb1b03b840ba5a6c0c675de6cefb76262c9789c8f06", size = 4944108, upload-time = "2026-01-07T23:12:13.751Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/bc/a4/7be8f0199ca6a1a2c6f7bb8118b2e37d29077ade2319d6271b1724d0cc86/sqlcipher3-0.6.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:fea0f1264f09d219dd6ce699ffca8cc9022a914661c6efa4390e85a2bf78acf9", size = 2895406, upload-time = "2026-01-07T23:12:14.954Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2f/b7/b9e897cf9e4740ca148fb03b493fa708a9b729ccc0cd656099f16bc9f2fd/sqlcipher3-0.6.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:bc2edd981e65783bc0d4e337704a9eb436871ab91c68af02ed76354876087642", size = 3161395, upload-time = "2026-01-07T23:12:16.049Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/b4/98/57e7f7e170b6065a21735687a94373aba9ce6866708e5a386e6af1a90ff2/sqlcipher3-0.6.2-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:50f64086da0a5f14281f2b0b459c4d9923b50055813a48ad29baf8c41c7fa56c", size = 7261870, upload-time = "2026-01-07T23:12:17.732Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/69/e4/cb0ed654a9642ee707c79f5e46db11518737318fd24968463b5aaa367a31/sqlcipher3-0.6.2-cp312-cp312-manylinux_2_28_i686.whl", hash = "sha256:2288215f462a16996689e1c22d611c94dd865faddb703cb105981dc3c0307b23", size = 6874049, upload-time = "2026-01-07T23:12:19.471Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f5/03/d55fe69fb380dadb2f5d19b3eac9256218243cced6aa4696ef90d560d223/sqlcipher3-0.6.2-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:6b26d28ca844dc2a69b8f74b390e940db47760f0be4c96d93337c57ae8250a48", size = 7040115, upload-time = "2026-01-07T23:12:21.168Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/8c/bd/afe3998add9f877533480f48d0deaa1c42a31832954424f4414539f59bfc/sqlcipher3-0.6.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:11e85c1fa4dfe6bf031af8ada500e94b5a77762355b500580360aa162896cecd", size = 7253646, upload-time = "2026-01-07T23:12:22.467Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/da/ff/9723596e7d220d933d5016adcea249b8e6f4f54116219ead6cf919646de4/sqlcipher3-0.6.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:80ae14562d98419b32149e8d66eea567eb3792e149b103ee5c8e1e5c67c5d799", size = 7060703, upload-time = "2026-01-07T23:12:23.907Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c4/63/e220211b098bc54d5345369759e21e4b34804d7e1c9f80f53372e2041b39/sqlcipher3-0.6.2-cp312-cp312-win32.whl", hash = "sha256:fb15c43f8a4f8b6b0ebe62ad2ab97a7946e3b75cb98a02069ff56b7d5a96c415", size = 1982081, upload-time = "2026-01-07T23:12:25.241Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2e/1a/88e8a0b96a52d291e24424cf8a222a6c7cc5356fd0a2287086afff764c2e/sqlcipher3-0.6.2-cp312-cp312-win_amd64.whl", hash = "sha256:8bd60ffb7bfa65bd0e51da3d5c308553d7149f0091d4ea9f754c33d5ebbf0a66", size = 2485691, upload-time = "2026-01-07T23:12:26.498Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5e/83/69217a75ed282dc865c4d658993af935210efe907b5ab6a863365ed6f20b/sqlcipher3-0.6.2-cp312-cp312-win_arm64.whl", hash = "sha256:99148bf4bf8e73c2c35f810f80de776d7de09b6cf277322c07759026400e90d0", size = 2652185, upload-time = "2026-01-07T23:12:27.594Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/0e/79/56fb1ea7cffc2ef32446b6fb9ba499464324995f9ea21c88791af9176682/sqlcipher3-0.6.2-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:8093773cd59b2a205d2cdb21383a93f7725126497032c269983ed89a89993631", size = 4943278, upload-time = "2026-01-07T23:12:28.735Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/98/5f/805772f52b10abe1e1aedba6f8c0ab7fb5eb5ccb4de5dfb51fef71132096/sqlcipher3-0.6.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:29f39d50bea02d78a824022989164c171e865bfeced3f9b84d1d45193dae074c", size = 2894958, upload-time = "2026-01-07T23:12:29.863Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/56/0d/2cee40de57d47245de09382c64e649c8cc8e86fa549ecba7591633fabf20/sqlcipher3-0.6.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:8e1ff6079603dfd955d57c26dad5eab14f6baacdc643d8753dd651913ba789cf", size = 3160350, upload-time = "2026-01-07T23:12:30.934Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fc/7a/72e7001af10c5b0e34592eae8e2634c22f1eb67d9859327c2fbbbf2b4961/sqlcipher3-0.6.2-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:35ae605f7594fca64a6d71007795dd39effd625cdc2a181d47f7d9fc8a5e1965", size = 7257677, upload-time = "2026-01-07T23:12:32.091Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a0/16/2e36fc23f4cc3baad39938c71c2db33a92ffa230a81073aa9186ad33a540/sqlcipher3-0.6.2-cp313-cp313-manylinux_2_28_i686.whl", hash = "sha256:8a3e39ad5f73060232b17715aa3b757e82ec4b67bb6acfc081147f66d00c2659", size = 6870821, upload-time = "2026-01-07T23:12:33.516Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f4/6b/874f72b6f3c3ebbe889e4279a0d422b7271ef7b3c63e45fae80a4ce16ec7/sqlcipher3-0.6.2-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:9fb7109981583b631ac795e7e955d4bf78058f64b54c7f334ccc437adc322d4b", size = 7037142, upload-time = "2026-01-07T23:12:34.803Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a5/a0/36af1ad4a3d45cc6c3a5dd508d1d5ddd24a6903a478405d40da37b1acc07/sqlcipher3-0.6.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:1ab63dcba15868853cb4d318cceb50dc47b94095e0c434f2785b9b098f3f5b42", size = 7249529, upload-time = "2026-01-07T23:12:36.079Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/fd/67/544729cd60a1cd961d6e70b7880a5a2dd18dd38bf9575527b5b2a893daa5/sqlcipher3-0.6.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:16d54822ad2fe44e49818a27f4862ba041f2d4a6aeb69422186379f9af97ced0", size = 7057767, upload-time = "2026-01-07T23:12:37.487Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5b/63/20730edd16df0d5ba8198cf8f4b679335c9db61c469d30ba2ef8adfbee2e/sqlcipher3-0.6.2-cp313-cp313-win32.whl", hash = "sha256:31789ce5ec7dd3f6c4ebd612c9cd9f7079a1d3698829111f7a382b0c10da3a87", size = 1981467, upload-time = "2026-01-07T23:12:39.16Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ab/e3/11e2e945557fe300bc399d9fafbba9154089f84483f4940546fd81b49b29/sqlcipher3-0.6.2-cp313-cp313-win_amd64.whl", hash = "sha256:9dc959ff792228c6df836cfd3667c713ae13e6e18dc2905c9d5666558606e832", size = 2485219, upload-time = "2026-01-07T23:12:40.473Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f3/79/245ef275ef93b45005e000105eb62df5779c056b1ba699eb7b5ba663a1c8/sqlcipher3-0.6.2-cp313-cp313-win_arm64.whl", hash = "sha256:30eeac16e755e5b0cff584ff541d3001bfcfc20be0ae364ff5305bbaeccbb3f1", size = 2651933, upload-time = "2026-01-07T23:12:41.727Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/27/a6/8e0ac8e198ba5c92b4af150f1b405968dacb399259bb74e732818dbacb46/sqlcipher3-0.6.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:dae7ce66554f2416d9e9012cc78dfe4d4053385e7fa289a8d0bd7772e5f5a702", size = 4943422, upload-time = "2026-01-07T23:12:42.883Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e1/99/72b4e23936bd4065fb97c07429a395c828a1d7419776219c7e98932df14f/sqlcipher3-0.6.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:be137cc92c9a039e469a758ee55e27e2385f419d1387f24e2029c536aa5d9736", size = 2894925, upload-time = "2026-01-07T23:12:44.133Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/f6/01/f3552874b158d83c15fb9d550576020cc42b34019d0daf3291b381fbfb01/sqlcipher3-0.6.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:5c1f4a5805faa418c9c7290e6a556a8c5abae40ea59b04d76e960e33c257e618", size = 3160559, upload-time = "2026-01-07T23:12:45.552Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/20/a0/274cbe5180a837818f5502823b5fa20f198546abc36ee56803636db5dcaf/sqlcipher3-0.6.2-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:e2328f0848ffb78807cf0898749dc22ff3f5aa95ab0d5a8a253628de20c11a1c", size = 7257835, upload-time = "2026-01-07T23:12:46.856Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d5/49/0d9a84435658ea3a6cdbb54cb8e67725e4c146726d008248531dca146eb7/sqlcipher3-0.6.2-cp314-cp314-manylinux_2_28_i686.whl", hash = "sha256:a1fbb693bf7c2f6f46ed038544b0bf76ea43dcc3231905cf5a686af15dc9b424", size = 6872199, upload-time = "2026-01-07T23:12:48.235Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ff/12/8d554633c3975f429e07cf07e136fb94ace10b460e1cb86b4c8019b7cdb4/sqlcipher3-0.6.2-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:e00988174ecd67ecd4537504c3df55bf8daeb75fce98401f099dff8e22c43ae1", size = 7037131, upload-time = "2026-01-07T23:12:49.416Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2e/7a/58dbb21db860d006d2be466678e7fdc317118a55c0e8a3e2a7f848f42112/sqlcipher3-0.6.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a105e816579bb7cce6f03e7e208b06d6d886c6445e1c738ed9aa2febabff3041", size = 7250574, upload-time = "2026-01-07T23:12:50.73Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/62/84/8d1f55fd8fadef56b40ee19bfae23100b5ff883bec65582a12a4bfbd56d5/sqlcipher3-0.6.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:e3a936d7414ae62f40880668bc036b0fde1ef0f48ed86cfe6564340f780ceea4", size = 7058156, upload-time = "2026-01-07T23:12:52.13Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d5/0d/80861eeeba9f5f6a0f202b3d27f9ab696e2c885442ed9340ad96ba0ed66c/sqlcipher3-0.6.2-cp314-cp314-win32.whl", hash = "sha256:7a07dafe752e013d4030accf218e80472d08de1309ddaf26df6f02d0850b2cec", size = 2028880, upload-time = "2026-01-07T23:12:53.316Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a6/11/950f0b092588866213e5d89b08701d24a938c9df116673bba54ac35f61af/sqlcipher3-0.6.2-cp314-cp314-win_amd64.whl", hash = "sha256:7de6133b19aec27b30698267cc2a0ea6e82c21d9a81d349cf0b480439fb549ac", size = 2556915, upload-time = "2026-01-07T23:12:54.615Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/5c/e7/cc1ce8e013bb84b8429b9736de7aca58b738781b08230bc9ba1c1e1a6d55/sqlcipher3-0.6.2-cp314-cp314-win_arm64.whl", hash = "sha256:765e133bd4ddda5596275f1221fa63b2b5d7d2b6e3670809bbf630edb705e27a", size = 2722598, upload-time = "2026-01-07T23:12:55.743Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/26/68/611dd86b446e069f769cf129cde96967982e0b41f49e66df9d3d21255df7/sqlcipher3-0.6.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:83533b5b7622ec9b78bec7596d96534e30015136f3e3e69a22f836fc59e393bc", size = 4947511, upload-time = "2026-01-07T23:12:56.968Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/80/24/b7e39a394841d56a678edb4b5a3b259b5a07bbb86a3863e15c5e1b041a58/sqlcipher3-0.6.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:310d7adbea382bda31007ee7d3dc63ba6ca86fcf7c0626ea804161fad2efce5e", size = 2896925, upload-time = "2026-01-07T23:12:58.09Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/31/ff/0b4b0cb02dd4084325bce526f0f4467c6a1ebcdf8fc625516734640f37ba/sqlcipher3-0.6.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1010c4ff1ff13a7e53284a3b03980754dbd37e6eea6faed9c6409e52bac082e6", size = 3164288, upload-time = "2026-01-07T23:12:59.362Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/d0/c3/bab9952c3c1bbde45313716610c5dcd36528b315228ac0b51e2d45217509/sqlcipher3-0.6.2-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:d437215611b620b32cb6b68dbc66dbeaccdfb3f76a7b6d8118a40849f8612088", size = 7300761, upload-time = "2026-01-07T23:13:00.836Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/2b/ca/3315292ed9ddbcd2de28f445ad11cc4a7f710d8b2ab2095aa0a629682d25/sqlcipher3-0.6.2-cp314-cp314t-manylinux_2_28_i686.whl", hash = "sha256:09e4170b1b2744b02b1c9315996a228d0b8d8a3ec1a0f7d4d41db0e79872fcea", size = 6911561, upload-time = "2026-01-07T23:13:02.072Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/a8/66/fdfdb11110d7166ca3707180d9ce85642be096d348072aba0ef8483b307a/sqlcipher3-0.6.2-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:bb4eaa9093bd46a7d51a65b9f63bac29ec4fc6b4ac794083e53eeb49f6db7e2c", size = 7074025, upload-time = "2026-01-07T23:13:03.327Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/ca/24/69b9bdc5ae7859a009b5336515c7e4ea5d58dfe3e358e44fabbcdd1431ae/sqlcipher3-0.6.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:782111277cbd999b7bc4e9e910396492ee28397c2c60ef7287b6dbc36a0b6a24", size = 7293516, upload-time = "2026-01-07T23:13:04.67Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/07/1b/2a62a605851b6ada8686c31b0cd82dd4305b7b19a57af3dbf7fe43249d1e/sqlcipher3-0.6.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:a01424f0d0120e8d9d3e0e1751ef78e70867bbce91f283b56911e8c6adaafddb", size = 7096368, upload-time = "2026-01-07T23:13:06.205Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/c1/7a/6c7d3356c9885e87a49d8f649f25019125ce6e68b4c8bb3933538d7add2c/sqlcipher3-0.6.2-cp314-cp314t-win32.whl", hash = "sha256:311fa50be627a4d1566bed31fd7725dec535a71332dcefdcdf9ec2472c4f824d", size = 2031516, upload-time = "2026-01-07T23:13:07.374Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/47/be/42d8d5cbcd0a89a5d7bfa1fe2ff986355d8b5910445b92f9d109b18eca93/sqlcipher3-0.6.2-cp314-cp314t-win_amd64.whl", hash = "sha256:7ac16581a5b80c54237c5f08f2e488051dfa7f52e3890e7765a6364d5bb3a2c6", size = 2559803, upload-time = "2026-01-07T23:13:08.447Z" },
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/9d/27/140a300fd151773604c4ff75ff5785febf25071231f239d6ce84b32e1408/sqlcipher3-0.6.2-cp314-cp314t-win_arm64.whl", hash = "sha256:76125dd222f4946302f70281e155ae9336efa4bc6fabdc81a7ae9bd4dfce9180", size = 2724322, upload-time = "2026-01-07T23:13:09.5Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "starlette"
|
name = "starlette"
|
||||||
version = "1.3.1"
|
version = "1.3.1"
|
||||||
@@ -999,3 +1453,12 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/9a/3f/f70e03f40ffc9a30d817eef7da1be72ee4956ba8d7255c399a01b135902a/websockets-16.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:a653aea902e0324b52f1613332ddf50b00c06fdaf7e92624fbf8c77c78fa5767", size = 178735, upload-time = "2026-01-10T09:23:42.259Z" },
|
{ url = "https://files.pythonhosted.org/packages/9a/3f/f70e03f40ffc9a30d817eef7da1be72ee4956ba8d7255c399a01b135902a/websockets-16.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:a653aea902e0324b52f1613332ddf50b00c06fdaf7e92624fbf8c77c78fa5767", size = 178735, upload-time = "2026-01-10T09:23:42.259Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/6f/28/258ebab549c2bf3e64d2b0217b973467394a9cea8c42f70418ca2c5d0d2e/websockets-16.0-py3-none-any.whl", hash = "sha256:1637db62fad1dc833276dded54215f2c7fa46912301a24bd94d45d46a011ceec", size = 171598, upload-time = "2026-01-10T09:23:45.395Z" },
|
{ url = "https://files.pythonhosted.org/packages/6f/28/258ebab549c2bf3e64d2b0217b973467394a9cea8c42f70418ca2c5d0d2e/websockets-16.0-py3-none-any.whl", hash = "sha256:1637db62fad1dc833276dded54215f2c7fa46912301a24bd94d45d46a011ceec", size = 171598, upload-time = "2026-01-10T09:23:45.395Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "zipp"
|
||||||
|
version = "4.1.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/b9/d8/eab98a517c14134c0b2eb4e2387bc5f457334293ec5d2dd3857ec2966802/zipp-4.1.0.tar.gz", hash = "sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602", size = 26214, upload-time = "2026-05-18T20:08:57.967Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/3a/13/547360d81e6d88d58492968ffda9f9542854f11310ee556fef14260cc886/zipp-4.1.0-py3-none-any.whl", hash = "sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f", size = 10238, upload-time = "2026-05-18T20:08:57.045Z" },
|
||||||
|
]
|
||||||
|
|||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# Cyclone payer configuration
|
||||||
|
# SP9 — loaded at boot by cyclone.payers.load_payer_configs()
|
||||||
|
# Schema-validated against Pydantic models in cyclone.providers
|
||||||
|
# See spec: docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md
|
||||||
|
|
||||||
|
payers:
|
||||||
|
# Colorado Medical Assistance Program (FFS) — the user's primary payer
|
||||||
|
- payer_id: CO_TXIX
|
||||||
|
name: "Colorado Medical Assistance Program"
|
||||||
|
receiver_name: "COLORADO MEDICAL ASSISTANCE PROGRAM"
|
||||||
|
receiver_id: "COMEDASSISTPROG"
|
||||||
|
configs:
|
||||||
|
"837P":
|
||||||
|
submitter_name: "Dzinesco"
|
||||||
|
submitter_contact_name: "Tyler Martinez"
|
||||||
|
submitter_contact_email: "tyler@dzinesco.com"
|
||||||
|
receiver_name: "COLORADO MEDICAL ASSISTANCE PROGRAM"
|
||||||
|
receiver_id_qualifier: "46"
|
||||||
|
receiver_id: "COMEDASSISTPROG"
|
||||||
|
bht06_allowed: ["CH", "RP"]
|
||||||
|
bht06_default: "CH"
|
||||||
|
sbr09_default: "MC"
|
||||||
|
sbr09_allowed: ["MC", "16", "MA", "MB", "ZZ"]
|
||||||
|
payer_id_qualifier: "PI"
|
||||||
|
# Per spec: NM1*PR NM109 = "CO_TXIX".
|
||||||
|
# Production reality (June 2026): SKCO0 is still being sent in
|
||||||
|
# the prod 837P files. SP9 emits CO_TXIX per the CO MAP companion
|
||||||
|
# guide. If Gainwell rejects CO_TXIX, set this to "SKCO0".
|
||||||
|
payer_id: "CO_TXIX"
|
||||||
|
pwk_supported: false
|
||||||
|
cas_2320_group_allowed: false
|
||||||
|
claim_type_codes:
|
||||||
|
"11": "Office"
|
||||||
|
"12": "Home"
|
||||||
|
"99": "Other"
|
||||||
|
"835":
|
||||||
|
expected_payer_tax_ids:
|
||||||
|
- "81-1725341"
|
||||||
|
- "811725341"
|
||||||
|
- "84-0644739"
|
||||||
|
- "840644739"
|
||||||
|
- "1811725341"
|
||||||
|
expected_payer_health_plan_id: "7912900843"
|
||||||
|
payer_name_pattern: "^CO_(TXIX|BHA)$"
|
||||||
|
"277CA":
|
||||||
|
# Per X12 005010X214. HCPF sends back 277CAs to acknowledge
|
||||||
|
# claims we submit. The parser matches each STC row against our
|
||||||
|
# 837 batch via REF*1K (cross-references CLM01 / patient_control_number).
|
||||||
|
#
|
||||||
|
# rejected_status_codes → claim ends up in the Inbox Payer-Rejected
|
||||||
|
# lane. Per HCPF's published STC code set, A4/A6/A7 are rejection,
|
||||||
|
# A8/A9 are pended, A1/A2/A3 are accepted, P1-P5 are paid.
|
||||||
|
rejected_status_codes: ["A4", "A6", "A7"]
|
||||||
|
pended_status_codes: ["A8", "A9"]
|
||||||
|
accepted_status_codes: ["A1", "A2", "A3"]
|
||||||
|
paid_status_codes: ["P1", "P2", "P3", "P4", "P5"]
|
||||||
|
# ST*01 transaction set identifier — HCPF uses "277CA" but the
|
||||||
|
# X12 005010X214 spec allows just "277". Both are accepted.
|
||||||
|
transaction_set_ids_allowed: ["277", "277CA"]
|
||||||
|
# Implementation guide version HCPF sends.
|
||||||
|
implementation_guide: "005010X214"
|
||||||
@@ -15,6 +15,102 @@ Financing (HCPF) requires.
|
|||||||
These appear in the `NM1*PR` (payer) and `NM1*40` (receiver) segments of
|
These appear in the `NM1*PR` (payer) and `NM1*40` (receiver) segments of
|
||||||
the 837P file.
|
the 837P file.
|
||||||
|
|
||||||
|
## dzinesco's TPID (clearinghouse identity)
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|---|---|
|
||||||
|
| Clearinghouse name | `dzinesco` |
|
||||||
|
| dzinesco TPID | `11525703` |
|
||||||
|
| Submitter name (`NM1*41`) | `Dzinesco` |
|
||||||
|
| Submitter contact | Tyler Martinez <tyler@dzinesco.com> |
|
||||||
|
| SFTP host | `mft.gainwelltechnologies.com` |
|
||||||
|
| SFTP username | `colorado-fts\coxix_prod_11525703` |
|
||||||
|
| SFTP outbound dir | `/CO XIX/PROD/coxix_prod_11525703/FromHPE` |
|
||||||
|
| SFTP inbound dir | `/CO XIX/PROD/coxix_prod_11525703/ToHPE` |
|
||||||
|
|
||||||
|
## dzinesco's 3 billing-provider NPIs
|
||||||
|
|
||||||
|
All 3 NPIs are registered to the same Montrose corporate office. They
|
||||||
|
share tax ID `721587149` and taxonomy `251E00000X` (Home Health).
|
||||||
|
|
||||||
|
| NPI | Label | Legal name | Address |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `1881068062` | Montrose | TOC, Inc. | 1100 East Main St, Suite A, Montrose, CO 814014063 |
|
||||||
|
| `1851446637` | Delta | TOC, Inc. | 1100 East Main St, Suite A, Montrose, CO 814014063 |
|
||||||
|
| `1467507269` | Salida | TOC, Inc. | 1100 East Main St, Suite A, Montrose, CO 814014063 |
|
||||||
|
|
||||||
|
Production data (June 2026) confirms all 3 NPIs submit through the same
|
||||||
|
Montrose address block. See SP9 spec
|
||||||
|
`docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md`.
|
||||||
|
|
||||||
|
## SFTP submission to Gainwell
|
||||||
|
|
||||||
|
dzinesco submits 837P files to Gainwell's MFT (Managed File Transfer)
|
||||||
|
at `mft.gainwelltechnologies.com`. The full SFTP path layout is
|
||||||
|
specified by the user (2026-06-20):
|
||||||
|
|
||||||
|
- **Outbound** (we send): `/CO XIX/PROD/coxix_prod_11525703/FromHPE`
|
||||||
|
- **Inbound** (HPE/Gainwell sends to us): `/CO XIX/PROD/coxix_prod_11525703/ToHPE`
|
||||||
|
|
||||||
|
### File naming
|
||||||
|
|
||||||
|
Per the HCPF X12 File Naming Standards Quick Guide
|
||||||
|
(<https://hcpf.colorado.gov/tp-x12-filenaming>):
|
||||||
|
|
||||||
|
- **Outbound** (we send): `TPID-TransactionType-yyyymmddhhmmssSSS-1of1.X12`
|
||||||
|
- Example: `11525703-837P-20260620132243505-1of1.x12`
|
||||||
|
- 17-digit millisecond precision, Mountain Time (NOT UTC)
|
||||||
|
- "1of1" is the only accepted sequence value
|
||||||
|
- **Inbound** (HPE sends): `TP<TPID>-<OrigTx>_M<Tracking>-<ts>-1of1_<FileType>.x12`
|
||||||
|
- Example: `TP11525703-837P_M019048402-20260520231513488-1of1_999.x12`
|
||||||
|
- `FileType` is one of: `999`, `TA1`, `270`, `271`, `276`, `277`, `277CA`, `278`, `820`, `834`, `835`, `ENCR`
|
||||||
|
- 277CA is distinguished by `ST*277CA` content (filename uses `277`)
|
||||||
|
|
||||||
|
### SP9 stub vs SP13 wire-up
|
||||||
|
|
||||||
|
The SP9 SFTP client is a **stub** that writes generated 837 files to
|
||||||
|
`./var/sftp/staging/{outbound_path}/{filename}` instead of opening a
|
||||||
|
real SFTP connection. The structural interface matches the future
|
||||||
|
`paramiko`-backed implementation, so SP13 is a one-file swap.
|
||||||
|
|
||||||
|
`clearhouse.sftp_block.stub` is `true` by default. Set to `false` (and
|
||||||
|
create the Keychain entry, see below) to enable real SFTP.
|
||||||
|
|
||||||
|
### Keychain setup (one-time, by operator)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
security add-generic-password -s cyclone -a sftp.gainwell.password -w '<password>'
|
||||||
|
security find-generic-password -s cyclone -a sftp.gainwell.password -w
|
||||||
|
```
|
||||||
|
|
||||||
|
The `cyclone/secrets.py` module fetches the secret by name. When the
|
||||||
|
entry is missing or `keyring` is not installed, the SFTP stub falls
|
||||||
|
back to `<stub-secret>` so the local flow still works.
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# List providers (3 NPIs)
|
||||||
|
curl http://localhost:8000/api/config/providers
|
||||||
|
|
||||||
|
# List payers (1 = CO_TXIX)
|
||||||
|
curl http://localhost:8000/api/config/payers
|
||||||
|
|
||||||
|
# List CO_TXIX configs
|
||||||
|
curl http://localhost:8000/api/config/payers/CO_TXIX/configs
|
||||||
|
|
||||||
|
# Clearhouse identity
|
||||||
|
curl http://localhost:8000/api/clearhouse
|
||||||
|
|
||||||
|
# Submit 2 claims to the SFTP stub
|
||||||
|
curl -X POST http://localhost:8000/api/clearhouse/submit \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"claim_ids": ["CLM-1", "CLM-2"], "payer_id": "CO_TXIX"}'
|
||||||
|
|
||||||
|
# Files appear at ./var/sftp/staging/CO XIX/PROD/coxix_prod_11525703/FromHPE/
|
||||||
|
ls -la "./var/sftp/staging/CO XIX/PROD/coxix_prod_11525703/FromHPE/"
|
||||||
|
```
|
||||||
|
|
||||||
## Payer IDs
|
## Payer IDs
|
||||||
|
|
||||||
### 837P (claims)
|
### 837P (claims)
|
||||||
@@ -51,6 +147,113 @@ All CMS POS codes `01`–`99` are accepted. The canonical list lives in
|
|||||||
`cyclone/parsers/payer.py` as `CMS_PLACE_OF_SERVICE_CODES` and is the
|
`cyclone/parsers/payer.py` as `CMS_PLACE_OF_SERVICE_CODES` and is the
|
||||||
source of truth for validation and any UI dropdowns.
|
source of truth for validation and any UI dropdowns.
|
||||||
|
|
||||||
|
## Database encryption at rest (SP12)
|
||||||
|
|
||||||
|
Cyclone optionally encrypts the SQLite database with SQLCipher
|
||||||
|
(AES-256). The encryption key is stored in macOS Keychain — never on
|
||||||
|
disk in plaintext. Without the key, the DB falls back to plain SQLite.
|
||||||
|
|
||||||
|
### One-time operator setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Install SQLCipher (C library) and the Python binding.
|
||||||
|
brew install sqlcipher
|
||||||
|
pip install -e backend[sqlcipher]
|
||||||
|
|
||||||
|
# 2. Generate a random 32-byte key and store it in Keychain.
|
||||||
|
python3 -c "import secrets; print(secrets.token_urlsafe(32))" \
|
||||||
|
| xargs -I {} security add-generic-password \
|
||||||
|
-s cyclone -a cyclone.db.key -w "{}"
|
||||||
|
|
||||||
|
# 3. Restart Cyclone. The DB is now encrypted.
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# The DB file should be unreadable as plain SQLite.
|
||||||
|
sqlite3 ~/.local/share/cyclone/cyclone.db "SELECT count(*) FROM sqlite_master"
|
||||||
|
# → file is not a database
|
||||||
|
|
||||||
|
# But readable through Cyclone.
|
||||||
|
curl http://localhost:8000/api/claims
|
||||||
|
# → 200 OK
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key rotation (future)
|
||||||
|
|
||||||
|
To rotate the key:
|
||||||
|
|
||||||
|
1. Decrypt with old key, dump to SQL
|
||||||
|
2. Re-encrypt with new key, import SQL
|
||||||
|
3. Update Keychain
|
||||||
|
|
||||||
|
A first-class rotation endpoint is out of scope for SP12 (planned for
|
||||||
|
SP14+).
|
||||||
|
|
||||||
|
## Audit log (SP11)
|
||||||
|
|
||||||
|
Cyclone persists every state-changing event to a tamper-evident
|
||||||
|
hash-chained audit log. Each row carries a SHA-256 hash of
|
||||||
|
`(id, event_type, entity_type, entity_id, actor, payload_json,
|
||||||
|
created_at, prev_hash)`, where `prev_hash` is the previous row's
|
||||||
|
hash. Modifying any row's payload invalidates every subsequent row's
|
||||||
|
hash.
|
||||||
|
|
||||||
|
To list events:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'http://localhost:8000/api/admin/audit-log?entity_type=claim&entity_id=C-123'
|
||||||
|
```
|
||||||
|
|
||||||
|
To verify the chain (run nightly or on demand):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8000/api/admin/audit-log/verify
|
||||||
|
# → {"ok": true, "checked": 1234}
|
||||||
|
# → {"ok": false, "checked": 1180, "first_bad_id": 1181, "reason": "hash mismatch ..."}
|
||||||
|
```
|
||||||
|
|
||||||
|
Events written today:
|
||||||
|
- `claim.rejected` (999 ACK AK5 R/E)
|
||||||
|
- `claim.payer_rejected` (277CA STC A4/A6/A7)
|
||||||
|
- `clearhouse.submitted` (SFTP submit)
|
||||||
|
|
||||||
|
Compliance: HIPAA §164.316(b)(2) requires 6-year retention. The
|
||||||
|
schema doesn't enforce retention — that's a separate vacuum job.
|
||||||
|
|
||||||
|
## 277CA Claim Acknowledgment (SP10)
|
||||||
|
|
||||||
|
After Gainwell accepts our 837P file (999 AK5=A) and adjudicates the
|
||||||
|
claims, they send back a 277CA per X12 005010X214. The 277CA carries
|
||||||
|
one `STC` segment per claim with a category code:
|
||||||
|
|
||||||
|
| STC code | Meaning | Cyclone lane |
|
||||||
|
|---|---|---|
|
||||||
|
| A1, A2, A3 | Acknowledged / accepted | (logged, no action) |
|
||||||
|
| A4, A6, A7 | Rejected by payer | **Inbox Payer-Rejected** |
|
||||||
|
| A8, A9 | Pended | (logged for follow-up) |
|
||||||
|
| P1–P5 | Paid | (835 follow-up expected) |
|
||||||
|
|
||||||
|
The Payer-Rejected lane is distinct from the 999 envelope "rejected"
|
||||||
|
lane: a claim can be syntactically valid (999 A) but semantically
|
||||||
|
denied (277CA STC A6).
|
||||||
|
|
||||||
|
To upload a 277CA:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8000/api/parse-277ca \
|
||||||
|
-F "file=@TP11525703-837P_M019048402-...-1of1_277.x12"
|
||||||
|
```
|
||||||
|
|
||||||
|
The response includes `matched_claim_ids` (which Cyclone claims were
|
||||||
|
stamped payer-rejected) and `orphan_status_codes` (status entries we
|
||||||
|
couldn't tie to a Cyclone claim — usually because the PCN in REF*1K
|
||||||
|
doesn't match anything we sent).
|
||||||
|
|
||||||
|
The Inbox at `/api/inbox/lanes` returns the new `payer_rejected` lane
|
||||||
|
alongside the existing four.
|
||||||
|
|
||||||
## Validation rules Cyclone enforces
|
## Validation rules Cyclone enforces
|
||||||
|
|
||||||
See [837p.md](./837p.md#validation-rules-cyclone-enforces) and the
|
See [837p.md](./837p.md#validation-rules-cyclone-enforces) and the
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user