docs: surface SP14 (Payer-Rejected UI + acknowledge) + SP15 (key rotation) #2

2026-06-21T01:08:01-06:00

tyler commented

2026-06-21 01:08:01 -06:00

What

Surface SP14 (5-lane Inbox UI + Payer-Rejected acknowledge action) and SP15 (SQLCipher key rotation) in the README. Both are merged into main but were not yet documented; the previous doc pass covered up to SP13.

The previous doc PR (#1) was effectively merged via d4f6fdd and is now closed.

Changes

Inbox

Endpoint table now includes POST /api/inbox/payer-rejected/acknowledge (idempotent, audit-logged, returns transitioned / already_acked / not_found / not_rejected counts).

Encryption at Rest

New ### Key rotation sub-section explaining the POST /api/admin/db/rotate-key handler, the NullPool choice (SQLCipher thread affinity), the module-level threading.Lock, the error code mapping (409 / 400 / 503).
New endpoint row in the SP12 endpoint table.

Tamper-Evident Audit Log

Lead paragraph now mentions Payer-Rejected acknowledgements and SQLCipher key rotations as part of the audit-logged event set.

Roadmap

Bumped "shipped 2-13" to "shipped 2-15".
Added SP14 and SP15 entries to the shipped list at the top.
Added ### SP14 endpoints and ### SP15 endpoints reference sections at the end.

Project layout

Added api_routers/ (health.py, acks.py, ta1_acks.py) — FastAPI APIRouter subpackage extracted from api.py.
Updated api.py line to note it now mounts api_routers/*.

Verification

pytest --collect-only in backend/ collects 759 tests (up from 733 at the last doc pass).
git diff --check clean.
No code changes; no tests touched.
PR #1 closed (the work it contained was merged directly as d4f6fdd).

## What Surface SP14 (5-lane Inbox UI + Payer-Rejected acknowledge action) and SP15 (SQLCipher key rotation) in the README. Both are merged into main but were not yet documented; the previous doc pass covered up to SP13. The previous doc PR (#1) was effectively merged via `d4f6fdd` and is now closed. ## Changes **Inbox** - Endpoint table now includes `POST /api/inbox/payer-rejected/acknowledge` (idempotent, audit-logged, returns `transitioned` / `already_acked` / `not_found` / `not_rejected` counts). **Encryption at Rest** - New `### Key rotation` sub-section explaining the `POST /api/admin/db/rotate-key` handler, the `NullPool` choice (SQLCipher thread affinity), the module-level `threading.Lock`, the error code mapping (`409` / `400` / `503`). - New endpoint row in the SP12 endpoint table. **Tamper-Evident Audit Log** - Lead paragraph now mentions Payer-Rejected acknowledgements and SQLCipher key rotations as part of the audit-logged event set. **Roadmap** - Bumped "shipped 2-13" to "shipped 2-15". - Added SP14 and SP15 entries to the shipped list at the top. - Added `### SP14 endpoints` and `### SP15 endpoints` reference sections at the end. **Project layout** - Added `api_routers/` (health.py, acks.py, ta1_acks.py) — FastAPI APIRouter subpackage extracted from `api.py`. - Updated `api.py` line to note it now mounts `api_routers/*`. ## Verification - `pytest --collect-only` in `backend/` collects **759 tests** (up from 733 at the last doc pass). - `git diff --check` clean. - No code changes; no tests touched. - PR #1 closed (the work it contained was merged directly as `d4f6fdd`).

tyler added 7 commits 2026-06-21 01:08:02 -06:00

refactor(sp): extract shared API helpers from api.py into api_helpers.py 6233df1270

First checkpoint of the architecture satisfaction loop. Cyclone's api.py
was a 2281-line god-module with 14 cross-cutting helpers inlined next
to the @app route declarations. This commit moves them to a dedicated
cyclone.api_helpers module:

- NDJSON wire format: ndjson_line, ndjson_stream_837, ndjson_stream_835,
  ndjson_stream_list.
- Content negotiation: client_wants_json, wants_ndjson.
- Strict / raw_segments rewrites: strict_rewrite_837, strict_rewrite_835,
  drop_raw_segments_837, drop_raw_segments_835.
- Validation probes: has_claim_validation_errors, has_835_validation_errors.
- Live-tail generator: tail_events, heartbeat_seconds, utcnow.

api.py re-imports them under the original underscore-prefixed names so
every route call site stays unchanged. claims_stream, remittances_stream,
and activity_stream remain exposed at cyclone.api (test_api_stream_live
imports them directly).

Verifies byte-identical NDJSON wire format, content negotiation rules,
and the tail_events async-generator semantics (deliberately polls the
EventBus queue rather than awaiting its async iterator, so heartbeats
don't poison the bus subscription).

Live-tested: GET /api/health, /api/claims, /api/remittances,
/api/activity, /api/acks, /api/providers, /api/inbox/lanes,
/api/inbox/payer-rejected/acknowledge, and the /api/claims/stream
NDJSON tail all return expected codes / payload.

Backend pytest: 29 failures identical to baseline (pre-existing
secrets env, serialize_837, db_crypto env, prodfile env failures),
700 passed. Frontend npm test: 357/357 passing.

See /tmp/refactor-cyclone.md for the full checkpoint log and the plan
for the next step (splitting api.py routes into FastAPI APIRouters).

Autoreview: /tmp/grok-review-local.md (0 bugs, 1 suggestion, 4 nits —
all addressed: dead asyncio/os imports removed, dead Any import
removed, duplicate utcnow import dropped, trailing newline added).

feat(sp15): SQLCipher key rotation via PRAGMA rekey 47902fd6b2

Adds in-place key rotation for the encrypted DB at rest (HIPAA
sec.164.308(a)(4) - periodic key rotation).

- db_crypto.rotate_db_key(): opens with old key, issues PRAGMA rekey,
  reopens with new key, verifies schema survived (table-count sanity).
- db_crypto.generate_db_key(): fresh 256-bit CSPRNG hex key.
- db_crypto.fingerprint(): SHA-256[:8] of a key, for the operator to
  compare across rotations.
- db.dispose_engine() + db.reinit_engine(): SP15 plumbing. The
  rotation endpoint disposes the pooled connections (SQLCipher
  refuses to rekey while another connection holds the file), runs
  the rekey, then rebuilds the engine with the new key from the
  Keychain.
- API: POST /api/admin/db/rotate-key with module-level threading.Lock
  to serialize rotations. 400 when encryption not enabled, 409 when
  a rotation is already in flight, 503 on rekey or Keychain failure
  with a reason that tells the operator what to do next.
- Engine uses NullPool when SQLCipher is enabled: the default
  QueuePool returns connections to a shared queue that any thread
  can pull from, which breaks SQLCipher's thread affinity. NullPool
  trades connection reuse for thread safety, the only correct
  behavior under FastAPI's per-request threadpool.
- Audit event db.key_rotated with old/new fingerprints and
  table_count, written after the engine is rebuilt so the new key
  proves it can take new writes.
- previous key is stashed to a second Keychain account so the
  operator can roll back if the new key turns out to be broken.

Tests:
- test_db_crypto.py: 12 new tests for generate/fingerprint/rekey
  mechanics (5 require SQLCipher at runtime; skipped otherwise).
- test_api_rotate_key.py: 6 new tests for endpoint wiring
  (encryption-required, Keychain update, audit event, rekey-failure
  rollback, Keychain-write-failure 503, concurrent-rotation 409).

merge: SP15 SQLCipher key rotation into main ab00909715

docs: sync READMEs with SP14 (Payer-Rejected lane) + endpoint inventory d4f6fdd49c

refactor(api): split health + acks + ta1_acks routes into FastAPI APIRouters a63ba5e88c

Step 2 (first half) of the architecture satisfaction loop. api.py
shrank from 2595 to 2452 lines (-143) by extracting three read-only
resource groups into cyclone.api_routers:

- health.py: GET /api/health (1 endpoint)
- acks.py: GET /api/acks, GET /api/acks/{ack_id} (2 endpoints)
- ta1_acks.py: GET /api/ta1-acks, GET /api/ta1-acks/{ack_id} (2 endpoints)

Each router owns its endpoint bodies + the small UI-shape helper that
goes with them (_ack_to_ui, _ta1_to_ui, _serialize_ta1_from_row). The
helpers stay in the router file rather than being shared because each
is only used by its own endpoints.

api.py now ends the app-wiring section with three include_router()
calls. The new package is named cyclone.api_routers (not
cyclone.api.routers) to avoid the Python package-vs-same-named-module
ambiguity that would shadow the existing cyclone.api module.

Verifies: 41 targeted tests (test_acks, test_health, test_api_gets)
pass, full pytest is 8 failed / 735 passed / 16 skipped — identical
to clean main baseline. Live curl against the running server:
GET /api/health -> 200, GET /api/acks -> 200, GET /api/ta1-acks -> 200.

See /tmp/refactor-cyclone.md for the full plan.

refactor(api): trailing newlines + hoist acks.py parser imports to top-level 1267a341e3

Self-review nits from the router-split commit:
- All four new files lacked a trailing newline (PEP 8 / POSIX).
- acks.py was lazily importing ParseResult999 / serialize_999 inside
  get_ack_endpoint. Hoist to module-level — there's no import cycle
  (acks.py does not import from cyclone.api), so the imports are safe
  to do once.

No behavior change. Targeted tests (test_acks + test_health + test_api_gets)
still pass 41/41.

docs: surface SP14 (Payer-Rejected UI + acknowledge) + SP15 (key rotation) d25f00ac58

The README's most recent merge was SP13. Since then two more sub-projects
landed in main:

- SP14 (5c9365e + 8a65baa) — 5-lane Inbox UI with the Payer-Rejected
  lane rendered alongside Rejected/Candidates/Unmatched/Done today,
  and a bulk Acknowledge action that drops claims from the working
  surface without erasing the original 277CA rejection event. New
  endpoint POST /api/inbox/payer-rejected/acknowledge (idempotent,
  audit-logged). Migration 0010.

- SP15 (47902fd + ab00909) — SQLCipher key rotation in place via
  PRAGMA rekey. New endpoint POST /api/admin/db/rotate-key, serialized
  through a module-level threading.Lock. Switches the SQLAlchemy
  engine to NullPool when SQLCipher is enabled (QueuePool breaks
  SQLCipher thread affinity under FastAPI's per-request threadpool).
  Writes a db.key_rotated audit event with old + new key
  fingerprints and post-rotation table_count.

What this PR does:

- Inbox section + Inbox endpoint table: document the Payer-Rejected
  acknowledge action.
- Encryption at Rest: add a 'Key rotation' sub-section that explains
  the rotate-key handler, the NullPool choice, the threading.Lock,
  and the error code mapping (409/400/503).
- Tamper-Evident Audit Log: note that key rotations + Payer-Rejected
  acknowledgements are part of the audit-logged event set.
- Project layout: add api_routers/ (health.py, acks.py, ta1_acks.py)
  as the FastAPI APIRouter subpackage extracted from api.py.
- Roadmap: bump 'shipped 2-13' to 'shipped 2-15'; add SP14 and SP15
  entries to the shipped list; add SP14 + SP15 endpoint reference
  sections at the end.

Verification:

- pytest --collect-only collects 759 tests (up from 733 at the
  previous doc pass).
- git diff --check clean.
- No code changes; no tests touched.

tyler closed this pull request

2026-06-21 01:10:01 -06:00

tyler referenced this issue from a commit

2026-07-07 12:26:35 -06:00

fix(sp37-followup): use CliRunner in test_submit_batch_cli_help

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: tyler/cyclone#2