docs(spec,plan): SP30 Dashboard Recent batches widget

2026-07-02 14:08:30 -06:00
parent 1d2572e624
commit bde3060e9e
2 changed files with 293 additions and 0 deletions
@@ -0,0 +1,193 @@
+# SP30 — Dashboard "Recent batches" widget implementation plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Add a single Dashboard card that lists the 5 most recently parsed batches with one-line outcome per batch (status icon, filename, billed total, accepted/rejected/pending counts, top rejection reason). The operator can spot a bad batch — full envelope reject, sudden rejection spike — the moment they log in, without clicking through to `/batches`.
+
+**Architecture:** Extend the existing `GET /api/batches` list response with billing-outcome fields (one batched SQL aggregate per request, no N+1). Extend the frontend `BatchSummary` TS type with the new fields (all optional — backwards compatible). Build a new `RecentBatchesWidget` component that mirrors the existing Dashboard "Recent activity" Card chrome. Wire it into the Dashboard between the KPI tile row and the Activity row. Use the existing `useBatches(limit)` hook (queryKey `["batches", 5]`). Click → navigate to `/batches?batch=ID` opening the existing BatchDrawer.
+
+**Tech Stack:** Python 3.11+, FastAPI, SQLAlchemy 2.x, SQLite (encrypted via SQLCipher), React 18 + TypeScript + Vite, TanStack Query, lucide-react.
+
+---
+
+## File map
+
+| File | Change |
+|---|---|
+| `backend/src/cyclone/api.py` | Add `_batch_summary_billing_outcomes` helper. Extend `list_batches` handler to attach the new fields per item. |
+| `backend/tests/test_api_gets.py` | +2 tests (`test_batches_includes_billing_outcome`, `test_batches_835_kind_returns_zero_billed_and_no_rejection`). |
+| `src/lib/api.ts` | +6 optional fields on `BatchSummary`. |
+| `src/components/RecentBatchesWidget.tsx` | NEW component. |
+| `src/pages/Dashboard.tsx` | Insert widget between KPI tiles and Activity row. |
+| `src/components/RecentBatchesWidget.test.tsx` | NEW, 3 tests. |
+| `src/pages/Dashboard.test.tsx` | +1 test (widget renders + click navigates). |
+
+---
+
+## Step 1: Backend — extend `list_batches` with billing-outcome fields
+
+- [ ] Open `backend/src/cyclone/api.py` and read `list_batches` at lines 1769-1808 plus the existing helpers at 1740-1764. Note the pattern.
+- [ ] Add a new helper `_batch_summary_billing_outcomes(records)` near the existing helpers (after line 1764). It:
+  1. Opens a single `db.SessionLocal()() as s` session.
+  2. Collects batch ids from `records` via `[r.id for r in records]`. Returns `{}` if empty.
+  3. Runs one GROUP BY query:
+     ```python
+     from sqlalchemy import func
+     from cyclone.db import Claim, ClaimState
+     rows = (
+         s.query(Claim.batch_id, Claim.state, func.count(Claim.id), func.sum(Claim.charge_amount))
+         .filter(Claim.batch_id.in_(batch_ids))
+         .group_by(Claim.batch_id, Claim.state)
+         .all()
+     )
+     ```
+  4. Buckets into a `{batch_id: {"accepted": int, "rejected": int, "pending": int, "billed": float}}` dict. Define the state buckets:
+     - `accepted` = `PAID, RECEIVED, RECONCILED, PARTIAL`
+     - `rejected` = `REJECTED, DENIED, REVERSED`
+     - `pending` = `SUBMITTED, DRAFT` (everything else is "other" — not surfaced on the widget)
+  5. Runs the rejection-reason probe (only when there's at least one rejected claim across all batches — skip the query otherwise):
+     ```python
+     rejected_states = (ClaimState.REJECTED, ClaimState.DENIED, ClaimState.REVERSED)
+     rej = (
+         s.query(Claim.batch_id, Claim.rejection_reason, Claim.payer_rejected_status_code)
+         .filter(Claim.batch_id.in_(batch_ids),
+                 ((Claim.state.in_(rejected_states)) | (Claim.payer_rejected_status_code.in_(("A4","A6","A7")))))
+         .order_by(Claim.rejected_at.desc().nullslast())
+         .all()
+     )
+     ```
+     Bucket into `{batch_id: {"top_rejection_reason": str|null, "has_payer_reject": bool}}`.
+  6. Truncate `top_rejection_reason` to 60 chars with `…` suffix.
+  7. Returns `{batch_id: {accepted, rejected, pending, billed, top_rejection_reason, has_problem}}` where `has_problem = rejected > 0 or has_payer_reject`.
+- [ ] Update `list_batches` handler (line 1783-1793) to:
+  1. Call `_batch_summary_billing_outcomes(records)` after `records = store.list(limit=limit)`.
+  2. For each item in the constructed `items` list, look up `outcomes = outcomes_by_id.get(r.id, {})` and add: `acceptedCount`, `rejectedCount`, `pendingCount`, `billedTotal`, `topRejectionReason`, `hasProblem` (defaulting sensibly when missing — e.g. `acceptedCount=0`, `hasProblem=False`).
+- [ ] Confirm the existing `test_batches_*` tests in `test_api_gets.py` still pass (the new fields are added, none removed — existing assertions on `kind`, `inputFilename`, `claimIds` are untouched).
+
+## Step 2: Backend tests
+
+- [ ] Open `backend/tests/test_api_gets.py`. Add the 2 tests after the existing 4 `test_batches_*` tests (after line 101).
+- [ ] **`test_batches_includes_billing_outcome`** — seed one batch via direct ORM insert (use the `_add_batch` / `_add_claim` helper pattern from `test_dashboard_kpis.py:36-91` — these helpers don't exist in `test_api_gets.py` so inline them):
+  ```python
+  from datetime import datetime, timezone
+  from decimal import Decimal
+  from cyclone.db import Batch, Claim, ClaimState, db as db_module
+  
+  with db_module.SessionLocal()() as s:
+      s.add(Batch(id="b-out", kind="837p", input_filename="out.edi",
+                   parsed_at=datetime(2026, 7, 1, 12, 0, tzinfo=timezone.utc),
+                   totals_json={"total_claims": 3},
+                   validation_json={"passed": True, "warnings": [], "errors": []},
+                   raw_result_json={"_": "stub"}))
+      s.add(Claim(id="C-paid", batch_id="b-out", patient_control_number="PCN-paid",
+                  charge_amount=Decimal("100.00"), state=ClaimState.PAID,
+                  raw_json={"subscriber": {"first_name":"Jane","last_name":"Doe"},
+                            "payer": {"name":"CO_TXIX"}, "billing_provider": {"npi":"1234567893"},
+                            "service_lines": []}))
+      s.add(Claim(id="C-rej", batch_id="b-out", patient_control_number="PCN-rej",
+                  charge_amount=Decimal("50.00"), state=ClaimState.REJECTED,
+                  rejection_reason="999 AK5 R",
+                  raw_json={"subscriber": {"first_name":"Jane","last_name":"Doe"},
+                            "payer": {"name":"CO_TXIX"}, "billing_provider": {"npi":"1234567893"},
+                            "service_lines": []}))
+      s.add(Claim(id="C-sub", batch_id="b-out", patient_control_number="PCN-sub",
+                  charge_amount=Decimal("75.00"), state=ClaimState.SUBMITTED,
+                  raw_json={"subscriber": {"first_name":"Jane","last_name":"Doe"},
+                            "payer": {"name":"CO_TXIX"}, "billing_provider": {"npi":"1234567893"},
+                            "service_lines": []}))
+      s.commit()
+  resp = client.get("/api/batches")
+  item = resp.json()["items"][0]
+  assert item["acceptedCount"] == 1
+  assert item["rejectedCount"] == 1
+  assert item["pendingCount"] == 1
+  assert item["billedTotal"] == 225.0
+  assert item["topRejectionReason"] == "999 AK5 R"
+  assert item["hasProblem"] is True
+  ```
+- [ ] **`test_batches_835_kind_returns_zero_billed_and_no_rejection`** — seed one 835 via `/api/parse-835` (uses existing fixture pattern in `test_batches_claim_ids_empty_for_835` at lines 85-101), assert the new fields are zeroed out: `acceptedCount=0, rejectedCount=0, pendingCount=0, billedTotal=0, topRejectionReason=None, hasProblem=False`.
+- [ ] Run: `cd backend && .venv/bin/pytest tests/test_api_gets.py -v`. Both new tests pass; existing 4 still pass.
+
+## Step 3: Frontend types — extend `BatchSummary`
+
+- [ ] Open `src/lib/api.ts` line 245-259. Add 6 optional fields to the `BatchSummary` interface:
+  ```ts
+  /** Number of claims on the batch whose state is paid / received / partial / reconciled. */
+  acceptedCount?: number;
+  /** Number of claims whose state is rejected / denied / reversed. */
+  rejectedCount?: number;
+  /** Number of claims whose state is submitted / draft. */
+  pendingCount?: number;
+  /** Sum of charge_amount across all claims on the batch (837P only; 0 for 835). */
+  billedTotal?: number;
+  /** Most recent rejection_reason on the batch (truncated to 60 chars). Null when no rejections. */
+  topRejectionReason?: string | null;
+  /** True when the batch has rejections OR any 277CA A4/A6/A7 payer-reject claim. */
+  hasProblem?: boolean;
+  ```
+
+## Step 4: Frontend component — `RecentBatchesWidget`
+
+- [ ] Create `src/components/RecentBatchesWidget.tsx`. Build per the spec's row markup (Card chrome mirroring Dashboard.tsx:252-287).
+- [ ] Add `data-testid="recent-batches-widget"` on the outer `<Card>` for Playwright + the Dashboard integration test.
+- [ ] Import `Layers`, `CheckCircle2`, `AlertTriangle` from `lucide-react`.
+- [ ] Empty state: "No batches yet." centered with `text-[13px] text-muted-foreground px-2 py-6`.
+- [ ] Row markup per spec — `role="button" tabIndex={0}` + `onClick` + `onKeyDown` for Enter key.
+
+## Step 5: Frontend wire-up — Dashboard
+
+- [ ] Open `src/pages/Dashboard.tsx`. Add imports:
+  ```ts
+  import { useBatches } from "@/hooks/useBatches";
+  import { RecentBatchesWidget } from "@/components/RecentBatchesWidget";
+  ```
+- [ ] Inside the `Dashboard` component (after `const activity = activityQuery.data?.items ?? [];` on line 81), add:
+  ```ts
+  const RECENT_BATCHES_LIMIT = 5;
+  const recentBatchesQuery = useBatches(RECENT_BATCHES_LIMIT);
+  ```
+- [ ] Insert a new `<section>` BETWEEN line 245 (`</section>` for KPI tiles) and line 247 (`{/* Activity + Top providers */}`):
+  ```tsx
+  <section
+    className="animate-fade-in-up"
+    style={{ animationDelay: `${sectionBase + 200}ms` }}
+  >
+    <RecentBatchesWidget
+      batches={recentBatchesQuery.data ?? []}
+      onRowClick={(id) => navigate(`/batches?batch=${encodeURIComponent(id)}`)}
+    />
+  </section>
+  ```
+
+## Step 6: Frontend tests — component + Dashboard
+
+- [ ] Create `src/components/RecentBatchesWidget.test.tsx` with 3 tests (empty state, 2-batch render, 835 row). Mirror the patterns from `src/components/ActivityFeed.test.tsx` (if present) or `src/pages/Batches.test.tsx`.
+- [ ] Open `src/pages/Dashboard.test.tsx`. Read its existing setup. Add `SP30: Recent batches widget renders + click navigates to /batches?batch=ID`:
+  1. Mock `api.listBatches` to resolve with 2 batches.
+  2. Render `<Dashboard />` inside `MemoryRouter + QueryClientProvider`.
+  3. Wait for the widget's testid.
+  4. Click the first row's `<li role="button">`.
+  5. Assert URL is `/batches?batch=ID`.
+- [ ] Run: `npx vitest run src/components/RecentBatchesWidget.test.tsx src/pages/Dashboard.test.tsx` — all green.
+
+## Step 7: Per-tree verification
+
+- [ ] `cd backend && .venv/bin/pytest` — full backend suite passes (no regressions).
+- [ ] `npx vitest run` — full frontend suite passes (no regressions).
+- [ ] `npm run typecheck` — 0 errors in SP30 files (existing baseline errors in `Upload.tsx` are acceptable).
+- [ ] `npm run lint` — 0 errors in SP30 files.
+
+## Step 8: Atomic merge + deploy
+
+- [ ] Commit: `feat(sp30): Dashboard Recent batches widget with billing outcome`.
+- [ ] `git checkout main && git merge --no-ff sp30-recent-batches-widget -m "merge: SP30 Dashboard Recent batches widget into main"`.
+- [ ] `docker compose build backend frontend` — rebuild both images (backend has the new endpoint fields; frontend has the new widget).
+- [ ] `docker compose up -d backend frontend` — restart.
+- [ ] Playwright smoke test:
+  1. Login
+  2. Navigate to `/`
+  3. Wait for `[data-testid="recent-batches-widget"]`
+  4. Screenshot `/tmp/dashboard-sp30.png`
+  5. Click top row → assert URL changes
+  6. Screenshot `/tmp/dashboard-sp30-clicked.png`
+- [ ] Show user the screenshots + the curl response from `/api/batches` confirming the new fields.
@@ -0,0 +1,100 @@
+# SP30 — Dashboard "Recent batches" widget: glance how the last few billed
+
+> **Status:** Approved, 2026-07-02.
+> **Branch:** `sp30-recent-batches-widget`
+> **Spec → plan → implement → atomic merge into main.**
+
+## Goal
+
+The Dashboard already shows six-month KPIs (billed, denial rate, etc.) but the operator has no way to see **how the last few batches actually billed** without clicking through to `/batches`. Add a single Dashboard card that lists the 5 most recently parsed batches with one-line outcome per batch — so the operator can spot a batch that landed entirely rejected (or has a sudden spike in rejections) at a glance, the moment they log in.
+
+## Scope
+
+### Backend — 1 file modified (`backend/src/cyclone/api.py`)
+
+**`list_batches` handler (`api.py:1769-1808`):**
+- Extend each returned item with billing-outcome fields, computed via a single batched SQL query per request (no N+1):
+  - `acceptedCount` (int)
+  - `rejectedCount` (int)
+  - `pendingCount` (int)
+  - `billedTotal` (float)
+  - `topRejectionReason` (string|null, truncated to 60 chars)
+  - `hasProblem` (bool — true when `rejectedCount > 0` or any 277CA A4/A6/A7 claim on the batch)
+- New helper `_batch_summary_billing_outcomes(records)` runs one GROUP BY query and one rejection-reason probe. Mirrors the existing `_batch_summary_claim_count` / `_batch_summary_claim_ids` helpers (`api.py:1740-1764`).
+
+### Backend — 1 test file extended (`backend/tests/test_api_gets.py`)
+
+**+2 tests after the existing `test_batches_*` block at lines 47-101:**
+- `test_batches_includes_billing_outcome` — seed one batch with 3 claims (1 paid $100, 1 rejected $50 reason="999 AK5 R", 1 submitted $75); `GET /api/batches`; assert `acceptedCount=1, rejectedCount=1, pendingCount=1, billedTotal=225.0, topRejectionReason="999 AK5 R", hasProblem=True`.
+- `test_batches_835_kind_returns_zero_billed_and_no_rejection` — seed one 835 batch; assert `acceptedCount=0, rejectedCount=0, billedTotal=0, topRejectionReason=null, hasProblem=False`.
+
+Reuses the `seeded_store` fixture (line 27) and the direct ORM-insert pattern from `test_dashboard_kpis.py:36-91`.
+
+### Frontend — 1 type extension + 1 new component + 1 layout wire-up
+
+**`src/lib/api.ts:245-259`** — extend `BatchSummary` with 6 optional fields:
+```
+acceptedCount?, rejectedCount?, pendingCount?,
+billedTotal?, topRejectionReason?, hasProblem?
+```
+Optional preserves backward compat with existing tests in `Upload.history.test.tsx`, `Batches.test.tsx`, `BatchDiff.test.tsx`.
+
+**`src/components/RecentBatchesWidget.tsx`** (NEW, ~120 LOC):
+- Card chrome mirroring the Dashboard "Recent activity" Card.
+- Header: `<Layers />` icon + "Recent batches" title + counter chip "{N} latest".
+- Empty state: "No batches yet." (matches ActivityFeed).
+- Row markup: status icon (CheckCircle2 / AlertTriangle) + filename + (kind · time · top rejection reason) + billed/accepted breakdown (or "N payments" for 835).
+- Click → `onRowClick(id)`; Enter key → same. Mirrors `Dashboard.tsx:300-307`.
+
+**`src/pages/Dashboard.tsx`** — insert the widget as a full-width row BETWEEN the KPI tile row (line 245) and the "Activity + Top providers" grid (line 247):
+- `const recentBatchesQuery = useBatches(5);` — queryKey `["batches", 5]` (distinct from `["batches", null]` used by Upload).
+- `animationDelay: ${sectionBase + 200}ms` so it lands after the KPI stagger and before the Activity row.
+- onRowClick → `navigate('/batches?batch=' + id)`.
+
+### Frontend — 1 test file added + 1 extended
+
+**`src/components/RecentBatchesWidget.test.tsx`** (NEW, 3 tests):
+- empty state
+- 2-batch render (one clean, one with rejections) — verifies icon tint, billed total, accepted count, click + Enter key handler
+- 835 row renders "N payments" not $
+
+**`src/pages/Dashboard.test.tsx`** (+1 test):
+- widget renders, row click navigates to `/batches?batch=ID`.
+
+## Decisions
+
+- **D1 — widget shows 5 batches, not "today only" or "last 24h".** 5 is the natural at-a-glance cap. Hard-coded constant `RECENT_BATCHES_LIMIT = 5` in the page.
+- **D2 — backend returns billing outcome on EVERY `/api/batches` row, not a separate `/api/batches/summary` endpoint.** The query cost is bounded by the existing `limit` and is one extra GROUP BY — no N+1. Returning on the list endpoint means future surfaces (Batches page, Upload History) can use the same fields without another endpoint.
+- **D3 — `hasProblem` drives the visible status, not `rejectedCount > 0` directly.** 277CA `A4/A6/A7` payer-rejected claims also count as problems. Mirrors the Inbox `rejected + payer_rejected` aggregation.
+- **D4 — 835 (ERA) rows show "N payments" instead of a $ figure.** The `Remittance` table has no batch-level `total_charge` aggregate; payments count is the honest, glanceable signal.
+- **D5 — full-width row above the Activity grid, not a 2/3 + 1/3 split.** The widget is the operator's primary "how's it going?" affordance; full width reads as the headline.
+
+## Out of scope (deferred)
+
+- Trend sparkline per batch (last N parse throughput).
+- Click-to-filter the widget itself by status.
+- Live-tail `batch_parsed` event wiring (no such event exists yet).
+- A "missing-ack-alarm" surfacing on claims SUBMITTED >24h with zero 999 acks (separate UX concern).
+
+## Tech Stack
+
+Python 3.11+, FastAPI, SQLAlchemy 2.x, SQLite (encrypted via SQLCipher), React 18 + TypeScript + Vite, TanStack Query, lucide-react. Same as SP29.
+
+## File footprint
+
+**Estimated 7 files changed, ~270 LOC.**
+
+- Backend modified (1): `backend/src/cyclone/api.py` (+~60 LOC).
+- Backend test extended (1): `backend/tests/test_api_gets.py` (+~70 LOC).
+- Frontend type extended (1): `src/lib/api.ts` (+~8 LOC).
+- Frontend component added (1): `src/components/RecentBatchesWidget.tsx` (NEW, ~120 LOC).
+- Frontend layout wire-up (1): `src/pages/Dashboard.tsx` (+~20 LOC).
+- Frontend test added (1): `src/components/RecentBatchesWidget.test.tsx` (NEW, ~150 LOC).
+- Frontend test extended (1): `src/pages/Dashboard.test.tsx` (+~40 LOC).
+
+## Risks
+
+- **Aggregate query cost on `/api/batches`** — bounded by the existing `limit` (default 100, capped at 1000); one GROUP BY ≤60k rows is sub-50ms on SQLite.
+- **Backwards compat of `BatchSummary`** — new fields are optional, so every existing test that constructs `BatchSummary` continues to pass.
+- **`topRejectionReason` truncation** — capped at 60 chars in the backend; tested with one assertion on a long reason.
+- **Dashboard layout choreography** — widget uses `animationDelay: sectionBase + 200ms`; tested visually in the Playwright smoke test.