hotfix(acks): paginate /api/acks and surface server-side aggregates

The Acks page silently capped at 100 of 1056 rows in the operator's
DB: the eyebrow read `${items.length} on file` (not the server's
`total`), the KPI strip summed from the 100 visible items, and the
endpoint accepted no `offset` — so the user had no signal that 956
more acks existed. Same class of bug on the Activity page (cap=200,
no 'X of Y' hint).

Backend
- /api/acks (acks.py): add `offset`, bump `le=1000`→`le=5000`,
  slice `rows[offset:offset+limit]`, return server-side
  `aggregates` (accepted/rejected/received summed over the full
  row set, not the page) so the KPI strip reflects every persisted
  999 instead of just the visible 50.
- /api/activity list + stream (api.py): bump `le=500`→`le=5000` so
  the page can ask for a denser snapshot.
- /api/277ca-acks (api.py): bump `le=1000`→`le=5000` for
  consistency.
- /api/ta1-acks: left at `le=1000` — TA1s aren't shipped today and
  the structural fix (offset + aggregates) wasn't applied, so a
  larger cap would just make the same latent silent-failure easier
  to hit. (TODO: fold in the same shape when Gainwell starts
  shipping TA1s.)

Frontend
- listAcks (api.ts): accept `offset`, surface `aggregates`,
  adapt wire `*_count` keys to the in-page
  `accepted`/`rejected`/`received` shape so the page can use
  `data.aggregates` as a drop-in for the page-local fallback
  accumulator.
- useAcks (hooks): pass `offset` through; return type carries
  `aggregates`.
- Acks.tsx: add `page` state (PAGE_SIZE=50), use `data.total`
  for eyebrow + watermark (not `items.length`), use
  `data.aggregates` for the KPI strip (with in-page fallback
  accumulator on first paint), render `<Pagination>` when
  `totalCount > PAGE_SIZE`. Footer row reads "N rows on file"
  instead of "N rows".
- ActivityLog.tsx: bump `limit: 200`→`limit: 500`, eyebrow reads
  "Activity · showing N most recent" to make the bounded-window
  semantics honest (the endpoint doesn't expose a true total — it
  reports events matching the current kind/since filter, capped at
  the request limit).

Tests
- test_acks.py: 4 new tests pin the fix:
  1. `offset` walks the full set; `has_more` flips at the
     boundary.
  2. `aggregates` reflects the full row set, not the page (the
     silent-failure pin) — and stays stable across page slices.
  3. `limit` cap of 5000 is enforced (422 above it).
  4. `offset` past the end returns an empty page with stable
     aggregates (a stale UI page state across a row count change
     must not 500 or zero the KPIs).

Live smoke-verified: /api/acks?limit=2&offset=0 vs ?offset=2 return
the expected row slices, aggregates stable at 15/10/15 for 5 seeded
rows, /api/acks?limit=10000 rejected with 422.

Triage note: the TA1 section (`Ta1AcksSection`, lines 609-616 of
Acks.tsx) has the same latent silent-failure pattern (page-sums
KPIs, no offset on /api/ta1-acks). Left untouched because the
empty-state copy says Gainwell doesn't ship TA1s today and the
larger structural fix belongs in a follow-up.
This commit is contained in:
Nora
2026-06-29 11:19:10 -06:00
parent 4592bca372
commit 4c05c6527b
7 changed files with 265 additions and 26 deletions
+3 -3
View File
@@ -1020,7 +1020,7 @@ async def parse_277ca_endpoint(
@app.get("/api/277ca-acks", dependencies=[Depends(matrix_gate)])
def list_277ca_acks_endpoint(
limit: int = Query(100, ge=1, le=1000),
limit: int = Query(100, ge=1, le=5000),
) -> Any:
"""Return the list of persisted 277CA ACKs, newest first."""
rows = store.list_277ca_acks()
@@ -2315,7 +2315,7 @@ def list_activity(
request: Request,
kind: str | None = Query(None),
since: str | None = Query(None),
limit: int = Query(200, ge=1, le=500),
limit: int = Query(200, ge=1, le=5000),
) -> Any:
events = store.recent_activity(limit=limit)
if kind is not None:
@@ -2342,7 +2342,7 @@ async def activity_stream(
request: Request,
kind: str | None = Query(None),
since: str | None = Query(None),
limit: int = Query(50, ge=1, le=500),
limit: int = Query(50, ge=1, le=5000),
) -> StreamingResponse:
"""Stream Activity events as NDJSON: snapshot first, then live events.
+20 -4
View File
@@ -66,14 +66,29 @@ def _ack_to_ui(row) -> dict:
@router.get("/api/acks")
def list_acks_endpoint(
request: Request,
limit: int = Query(100, ge=1, le=1000),
limit: int = Query(100, ge=1, le=5000),
offset: int = Query(0, ge=0),
) -> Any:
"""Return the list of persisted 999 ACKs, newest first."""
"""Return the list of persisted 999 ACKs, newest first.
``limit`` caps the page size; ``offset`` lets the UI walk the
full set without holding it all in memory. ``aggregates`` is
summed over the *full* row set (not the page) so the KPI strip
on the Acks page reflects every persisted 999, not just the
visible 50. Without server-side aggregates the page would
silently under-report (silent-failure mode) once the row count
exceeds the page size.
"""
rows = store.list_acks()
items = [_ack_to_ui(r) for r in rows[:limit]]
items = [_ack_to_ui(r) for r in rows[offset : offset + limit]]
total = len(rows)
returned = len(items)
has_more = total > returned
has_more = offset + returned < total
aggregates = {
"accepted_count": sum(r.accepted_count or 0 for r in rows),
"rejected_count": sum(r.rejected_count or 0 for r in rows),
"received_count": sum(r.received_count or 0 for r in rows),
}
if wants_ndjson(request):
return StreamingResponse(
ndjson_stream_list(items, total, returned, has_more),
@@ -84,6 +99,7 @@ def list_acks_endpoint(
"total": total,
"returned": returned,
"has_more": has_more,
"aggregates": aggregates,
}
+151
View File
@@ -128,3 +128,154 @@ def test_get_ack_returns_row_when_present():
assert fetched.id == row.id
assert fetched.source_batch_id == "b-1"
assert fetched.raw_json == {"hello": "world"}
# ---------------------------------------------------------------------------
# Hotfix 2026-06-29: acks list endpoint silently capped at limit=100
# without exposing the true total or full-set aggregates, so the Acks page
# rendered "100 on file" and KPI totals summed from the page only when the
# row count exceeded 100. These tests pin the new offset param + the
# server-side `aggregates` field so the silent-failure mode can't regress.
# ---------------------------------------------------------------------------
def _seed_acks(n: int) -> list[int]:
"""Insert n ack rows with deterministic per-row counts. Returns the ids."""
_make_batch("b-hotfix")
ids: list[int] = []
for i in range(n):
row = store.add_ack(
source_batch_id="b-hotfix",
accepted_count=i + 1,
rejected_count=i,
received_count=i + 1,
ack_code="A",
raw_json={"order": i},
)
ids.append(row.id)
return ids
def test_list_acks_endpoint_pagination_offsets_correctly():
"""`offset` walks the full set, `has_more` flips at the boundary."""
from fastapi.testclient import TestClient
from cyclone.api import app
from cyclone.auth.users import create
from cyclone.db import SessionLocal
with SessionLocal()() as s:
create(s, username="u", password="p", role="admin")
s.commit()
client = TestClient(app)
client.post("/api/auth/login", json={"username": "u", "password": "p"})
_seed_acks(5)
r = client.get("/api/acks?limit=2&offset=0")
d = r.json()
assert d["total"] == 5
assert d["returned"] == 2
assert d["has_more"] is True
assert len(d["items"]) == 2
r = client.get("/api/acks?limit=2&offset=4")
d = r.json()
assert d["total"] == 5
assert d["returned"] == 1
assert d["has_more"] is False
assert len(d["items"]) == 1
def test_list_acks_endpoint_aggregates_reflect_full_set():
"""`aggregates` sums over the full row set, not the page.
Without this the Acks KPI strip silently under-reports once the
row count exceeds the page size — the silent-failure the operator
flagged on 2026-06-29.
"""
from fastapi.testclient import TestClient
from cyclone.api import app
from cyclone.auth.users import create
from cyclone.db import SessionLocal
with SessionLocal()() as s:
create(s, username="u", password="p", role="admin")
s.commit()
client = TestClient(app)
client.post("/api/auth/login", json={"username": "u", "password": "p"})
# 5 rows: accepted = 1+2+3+4+5 = 15, rejected = 0+1+2+3+4 = 10,
# received = same as accepted.
_seed_acks(5)
# Page 1 of 2 — full set is 5 rows, page only shows 2, but aggregates
# must reflect all 5.
r = client.get("/api/acks?limit=2&offset=0")
d = r.json()
assert d["total"] == 5
assert d["returned"] == 2
assert d["aggregates"]["accepted_count"] == 15
assert d["aggregates"]["rejected_count"] == 10
assert d["aggregates"]["received_count"] == 15
# Page 2 must return identical aggregates — page slice mustn't shift them.
r = client.get("/api/acks?limit=2&offset=2")
d = r.json()
assert d["aggregates"]["accepted_count"] == 15
assert d["aggregates"]["rejected_count"] == 10
assert d["aggregates"]["received_count"] == 15
def test_list_acks_endpoint_limit_cap_is_5000():
"""The validator still enforces an upper bound so a client can't
request 1,000,000 rows and OOM the SQLite-backed list call."""
from fastapi.testclient import TestClient
from cyclone.api import app
from cyclone.auth.users import create
from cyclone.db import SessionLocal
with SessionLocal()() as s:
create(s, username="u", password="p", role="admin")
s.commit()
client = TestClient(app)
client.post("/api/auth/login", json={"username": "u", "password": "p"})
r = client.get("/api/acks?limit=10000")
assert r.status_code == 422 # FastAPI validation error
r = client.get("/api/acks?limit=5000")
assert r.status_code == 200
def test_list_acks_endpoint_offset_past_end_returns_empty_page():
"""`offset` past the row count must yield an empty page, not 500.
Pinning this so a future refactor that introduces streaming or
cursor-based pagination can't accidentally error or 500 when the
UI holds stale page state across a row count change.
"""
from fastapi.testclient import TestClient
from cyclone.api import app
from cyclone.auth.users import create
from cyclone.db import SessionLocal
with SessionLocal()() as s:
create(s, username="u", password="p", role="admin")
s.commit()
client = TestClient(app)
client.post("/api/auth/login", json={"username": "u", "password": "p"})
_seed_acks(3)
r = client.get("/api/acks?limit=2&offset=99")
assert r.status_code == 200
d = r.json()
assert d["total"] == 3
assert d["returned"] == 0
assert d["has_more"] is False
assert d["items"] == []
# Aggregates must still reflect the full 3-row set on the empty page —
# otherwise a stale UI page state would silently zero out the KPI strip.
assert d["aggregates"]["accepted_count"] == 1 + 2 + 3
assert d["aggregates"]["rejected_count"] == 0 + 1 + 2
assert d["aggregates"]["received_count"] == 1 + 2 + 3
+8 -3
View File
@@ -1,5 +1,5 @@
import { useQuery } from "@tanstack/react-query";
import { api, type PaginatedResponse } from "@/lib/api";
import { api, type AckAggregates, type PaginatedResponse } from "@/lib/api";
import type { Ack } from "@/types";
/**
@@ -7,9 +7,14 @@ import type { Ack } from "@/types";
* intentionally has no in-memory fallback (there is no zustand
* sample-data path for ACKs in v1 — the UI is empty until the
* backend serves real rows).
*
* The response carries `aggregates` summed over the *full* row set
* (not just the visible page) so the KPI strip on the Acks page
* reflects every persisted 999 — without this, totals silently
* under-report once the row count exceeds the page size.
*/
export function useAcks(params: { limit?: number } = {}) {
return useQuery<PaginatedResponse<Ack>>({
export function useAcks(params: { limit?: number; offset?: number } = {}) {
return useQuery<PaginatedResponse<Ack> & { aggregates: AckAggregates }>({
queryKey: ["acks", params],
queryFn: () => api.listAcks(params),
enabled: api.isConfigured,
+30 -1
View File
@@ -160,6 +160,19 @@ export interface PaginatedResponse<T> {
has_more: boolean;
}
/**
* Server-side aggregates returned alongside `/api/acks` — summed over
* the *full* persisted row set (not just the visible page). The
* `accepted` / `rejected` / `received` keys match the in-page `totals`
* shape so the page can use `data.aggregates` as a drop-in for the
* page-local fallback accumulator.
*/
export interface AckAggregates {
accepted: number;
rejected: number;
received: number;
}
/**
* Lightweight summary used by the GET /api/batches list endpoint. Distinct
* from the parser's `BatchSummary` (which lives in `@/types` and carries the
@@ -753,21 +766,37 @@ function mapAck(row: RawAckRow): Ack {
};
}
async function listAcks(params: { limit?: number } = {}): Promise<PaginatedResponse<Ack>> {
async function listAcks(
params: { limit?: number; offset?: number } = {},
): Promise<
PaginatedResponse<Ack> & {
aggregates: AckAggregates;
}
> {
if (!isConfigured) throw notConfiguredError();
const query: Record<string, unknown> = {};
if (params.limit !== undefined) query.limit = params.limit;
if (params.offset !== undefined) query.offset = params.offset;
const body = await authedFetch<{
items: RawAckRow[];
total: number;
returned: number;
has_more: boolean;
aggregates: { accepted_count: number; rejected_count: number; received_count: number };
}>(`/api/acks${qs(query)}`);
return {
items: body.items.map(mapAck),
total: body.total,
returned: body.returned,
has_more: body.has_more,
// Adapt the wire-format `*_count` keys to the in-page `totals` shape
// so the page can treat server-side and client-side-fallback objects
// interchangeably. See useAcks.ts for the matching AckAggregates type.
aggregates: {
accepted: body.aggregates.accepted_count,
rejected: body.aggregates.rejected_count,
received: body.aggregates.received_count,
},
};
}
+41 -13
View File
@@ -16,6 +16,7 @@ import { KpiCard } from "@/components/KpiCard";
import { PageHeader } from "@/components/PageHeader";
import { KeyboardCheatsheet } from "@/components/KeyboardCheatsheet";
import { AckDrawer } from "@/components/AckDrawer";
import { Pagination } from "@/components/ui/pagination";
import { useAckDrawerUrlState } from "@/hooks/useAckDrawerUrlState";
import { useAcks } from "@/hooks/useAcks";
import { useTa1Acks } from "@/hooks/useTa1Acks";
@@ -25,6 +26,8 @@ import { fmt } from "@/lib/format";
import { cn } from "@/lib/utils";
import type { Ack, Ta1Ack } from "@/types";
const PAGE_SIZE = 50;
/**
* 999 ACK register. The page reads the persisted 999 Implementation
* Acknowledgments produced in response to 837P ingests (or parsed
@@ -39,8 +42,18 @@ import type { Ack, Ta1Ack } from "@/types";
* instrument view of the same data.
*/
export function Acks() {
const { data, isLoading, isError, error, refetch } = useAcks({ limit: 100 });
const [page, setPage] = useState(1);
const { data, isLoading, isError, error, refetch } = useAcks({
limit: PAGE_SIZE,
offset: (page - 1) * PAGE_SIZE,
});
const items = data?.items ?? [];
const totalCount = data?.total ?? 0;
// Server-side aggregates — sum over the *full* ACK set, not just the
// visible page. Without this, the KPI strip silently under-reports
// once the row count exceeds PAGE_SIZE (the "silent failure" the
// operator flagged on 2026-06-29 when 1056 acks showed as 100).
const aggregates = data?.aggregates;
// SP21 Phase 5 Task 5.3: drill-down from an acks row into
// AckDrawer. The hook reads `?ack=` off `window.location.search`
// so deep links restore the open ack on reload.
@@ -75,15 +88,16 @@ export function Acks() {
// -----------------------------------------------------------------
// Aggregate metrics for the KPI strip + hero copy.
// Sourced from the server-side `aggregates` field so the KPI strip
// reflects every persisted 999, not just the visible page. If
// `aggregates` hasn't loaded yet (first page), fall back to summing
// the visible items so the cards still display a meaningful value.
// -----------------------------------------------------------------
const totals = items.reduce(
(acc, a) => ({
accepted: acc.accepted + a.acceptedCount,
rejected: acc.rejected + a.rejectedCount,
received: acc.received + a.receivedCount,
}),
{ accepted: 0, rejected: 0, received: 0 },
);
const totals = aggregates ?? {
accepted: items.reduce((s, a) => s + a.acceptedCount, 0),
rejected: items.reduce((s, a) => s + a.rejectedCount, 0),
received: items.reduce((s, a) => s + a.receivedCount, 0),
};
const acceptRate =
totals.received > 0
? Math.round((totals.accepted / totals.received) * 1000) / 10
@@ -91,11 +105,16 @@ export function Acks() {
// The ghost watermark carries the on-file count, scaled like the
// Dashboard/Upload/Reconciliation watermarks (clamp 72140px, 4.5%
// opacity, right-anchored).
const watermark = items.length > 0 ? items.length.toLocaleString() : "999";
// opacity, right-anchored). Use the server-reported total (not the
// page length) so the watermark tells the truth when there are more
// than PAGE_SIZE rows on file.
const watermark = totalCount > 0 ? totalCount.toLocaleString() : "999";
// Tone for the status pill: green when nothing rejected, amber when
// the register holds a partial/rejected payload.
// `anyRejected` is a "page" property, not a "totals" property — the
// status pill summarizes the visible register, so it's OK to scope
// it to the current page.
const anyRejected = items.some(
(a) => a.ackCode === "R" || a.ackCode === "E",
);
@@ -147,7 +166,7 @@ export function Acks() {
<div className="relative z-10 flex items-end justify-between gap-4 sm:gap-6 flex-wrap">
<div className="min-w-0">
<PageHeader
eyebrow={`999 ACKs · EDI reference · ${items.length} on file`}
eyebrow={`999 ACKs · EDI reference · ${totalCount.toLocaleString()} on file`}
title={
<>
Acknowledgments,{" "}
@@ -407,6 +426,14 @@ export function Acks() {
</Table>
</div>
)}
{totalCount > PAGE_SIZE ? (
<Pagination
page={page}
pageSize={PAGE_SIZE}
total={totalCount}
onPageChange={setPage}
/>
) : null}
</div>
</CardContent>
</Card>
@@ -441,7 +468,8 @@ export function Acks() {
</div>
<div className="flex items-center gap-3 mono text-[10.5px] uppercase tracking-[0.18em] text-muted-foreground/60">
<span>
{items.length} {items.length === 1 ? "row" : "rows"}
{totalCount.toLocaleString()}{" "}
{totalCount === 1 ? "row" : "rows"} on file
</span>
<span className="text-muted-foreground/30" aria-hidden>
·
+12 -2
View File
@@ -68,7 +68,13 @@ export function ActivityLog() {
const { data, isLoading, isError, error, refetch } = useActivity({
kind: apiKind,
since: sinceIso,
limit: 200,
// 500 keeps the wire size manageable (and the operator can rely
// on the live tail to surface new events as they land). The
// activity list endpoint doesn't expose a true total — `data.total`
// reflects events matching the current kind/since filter, capped
// at the request limit — so the eyebrow below uses "Showing N
// most recent" instead of "X of Y" to avoid a misleading ratio.
limit: 500,
});
const allItems = data?.items ?? [];
@@ -127,7 +133,11 @@ export function ActivityLog() {
return (
<div className="space-y-6 lg:space-y-8 animate-fade-in">
<PageHeader
eyebrow="Activity"
eyebrow={
allItems.length > 0
? `Activity · showing ${allItems.length.toLocaleString()} most recent`
: "Activity"
}
title={<>Activity <span className="display italic text-muted-foreground">log</span></>}
subtitle="Every claim submission, denial, payment, and provider event, in reverse chronological order. Auto-refreshes every 30s."
actions={