cyclone/docs/superpowers/specs/2026-07-02-cyclone-ack-live-tail-design.md

SP25 — Ack Live-Tail: Design Spec

Date: 2026-07-02 Status: Draft, awaiting user sign-off Branch: sp25-ack-live-tail Aesthetic direction: No new visual language. The Acks page already has its dark-instrument chrome (hero / KPI strip / register / TA1 section / footer). We add a single <TailStatusPill> next to the existing "Envelope accepted / Mixed envelope" badge in the hero — same component, same semantic states, used identically on Claims / Remittances / Activity. No new color, type, layout, or motion decisions.

1. Scope

When the SFTP poller (or a manual upload) drops a TA1 or 999 file, every copy of the /acks page should reflect the new row within one heartbeat — no manual refresh, no polling. The mechanism is the same one Claims / Remittances / Activity already use: the store publishes a per-resource event after the row commits, a streaming NDJSON endpoint fans that event out, the page's useTailStream hook appends it to a zustand slice, and useMergedTail dedups against the page's initial JSON fetch.

The store already does this for claim_written / remittance_written / activity_recorded. SP25 extends the same pattern to the three ack tables (999, TA1, 277CA). 277CA is a no-UI side effect — the store still publishes the event so any future 277CA page can subscribe without a backend change. 999 and TA1 also get stream endpoints and frontend wiring because the Acks page already renders both.

The handler-side if event_bus is not None: publish(...) block — the broken stub that started this work — disappears along with the event_bus kwarg from handle_999 / handle_ta1 / handle_277ca / handle_835. The TODO(sp27-task-6) comments go away because the design they were deferring is no longer the design.

Out of scope (explicit, each is its own future increment if requested):

• A new 277CA page and useTwo77caAcks hook. The store publishes the event for future use; no page subscribes today.
• Backfilling historical acks into the live store. The snapshot-on-reconnect already covers "first time the page mounts" so the operator never sees an empty page when the DB has rows.
• Reformatting the Acks page hero, KPI strip, or sectioning. We add a pill; we do not redesign the page.
• A unified "all acks" stream endpoint. The page renders 999 and TA1 in distinct sections with different shapes; one stream per resource is the right call, mirroring the existing pattern.
• Any scheduler change. The scheduler already calls the store; once the store publishes, the scheduler benefits without further code.
• A 277CA audit-event publish path. claim.payer_rejected and the SP27 Task 13 follow-up are out of scope.

2. Goals

1. The Acks page reflects new acks within one heartbeat. The operator sees the same "appears on its own" behavior they already get on Claims / Remittances / Activity.
2. The store owns the publish, not the handler. Mirrors the existing three live resources. Removes the sync-handler-meets-async-EventBus design friction that was the source of the original bug.
3. The event payload matches the list-endpoint shape. The stream-endpoint snapshot and the live-event item are interchangeable; a downstream consumer that snapshots the page over HTTP and one that subscribes to the stream see the same row shape for the same row id.
4. Both write paths fire the event. The SFTP poller AND the manual /api/parse-999 / /api/parse-ta1 / /api/parse-277ca endpoints both end up in the store's add_*_ack methods, so both fire the event without any caller-specific plumbing.
5. A failed publish never rolls back the row. Persisted state is the source of truth; the event is the cache-invalidation hint. Match the existing claim_written semantics.
6. No DB migration. The events are pure pubsub; no new tables, no new columns, no new indexes.

3. Threat model

The auth boundary is HTTP (login required, bcrypt + HttpOnly session cookie); file-system threats remain the local-only threat model (SQLCipher at rest, macOS Keychain). The new stream endpoints sit behind Depends(matrix_gate) like every other /api/* endpoint — operators must be authenticated to receive live ack updates. The store's add_*_ack methods take event_bus as an explicit parameter (not a global) so the publish path is unit-testable without a real bus, and so a future move of the bus out of process does not require touching the store API.

4. Locked decisions

4.1 Publish from the store, not the handler

The handler-side if event_bus is not None: publish(...) block is removed. The store's add_999_ack / add_ta1_ack / add_277ca_ack methods take event_bus: EventBus | None and call _sync_publish(...) after the row commits, exactly the way store.write.add_record does for the three existing live resources. The handlers stop accepting event_bus; they just call the store and return. The TODO(sp27-task-6) comments go away because the sync/async gap they were deferring no longer exists at the call site.

Rationale: the handler is def (sync) and EventBus.publish is async. The current stub is a silent bug — publish(...) returns an unawaited coroutine and the warning gets swallowed by the try/except. Publishing from the store uses _sync_publish, which is the project's blessed bridge between sync write paths and the async bus. One change, one place.

4.2 Three event names, full to_ui_* payload

The three new event names are ack_received (999), ta1_ack_received (TA1), and two77ca_ack_received (277CA). Each payload is the full to_ui_* row shape that the matching list endpoint would return for the same row id — to_ui_ack for 999, to_ui_ta1_ack for TA1. (277CA's payload is the same shape its list endpoint at GET /api/277ca-acks returns; that serializer already lives in api.py near the endpoint and moves to store/ui.py along with the other two for ownership consistency.)

Rationale: the existing <entity>_written events carry the full row shape, and cyclone-store makes the event-payload-must-match-list-endpoint contract explicit. Two acks in flight through the same channel become trivially comparable; a future live-tail consumer that does its own dedup does not need a separate refetch to see the canonical row.

4.3 Two new stream endpoints, registered in api.py

/api/acks/stream subscribes to ["ack_received"]. /api/ta1-acks/stream subscribes to ["ta1_ack_received"]. Both follow the same wire format as the three existing live-tail endpoints (application/x-ndjson, snapshot → snapshot_end → live items, 15-second heartbeats, 30-second stall threshold). Both sit behind Depends(matrix_gate). The new endpoints live in api.py next to /api/claims/stream / /api/remittances/stream / /api/activity/stream because the live-tail triplet is currently inlined there; moving the acks streams into the api_routers/acks.py / api_routers/ta1_acks.py modules is a separate refactor.

4.4 _to_ui mappers move to store/ui.py

_ack_to_ui (currently in api_routers/acks.py) becomes to_ui_ack in store/ui.py. _ta1_to_ui (currently in api_routers/ta1_acks.py) becomes to_ui_ta1_ack in store/ui.py. The 277CA serializer moves alongside. store/ui.py already houses to_ui_claim_from_orm / to_ui_remittance_from_orm / to_ui_provider per the SP21 split map; the ack serializers are the obvious home. The api_routers/ modules import the renamed mappers from the store. The store's add_*_ack methods import them from the same module. One source of truth.

4.5 Frontend TailResource gains two members

TailResource in src/lib/tail-stream.ts becomes "claims" | "remittances" | "activity" | "acks" | "ta1_acks". useTailStream gains two dispatch cases in its resource switch. useTailStore gains two keyed-by-id slices (acks, ta1_acks) with the same FIFO cap as claims / remittances. useMergedTail adds two slice selectors that read in arrival order from ackOrder / ta1AckOrder arrays. The 277CA case is intentionally absent — no page consumes it today, and YAGNI says we add it when the page exists.

4.6 Acks page subscribes via the same hook triplet

Acks.tsx calls useTailStream("acks") and useMergedTail("acks", data?.items ?? []) for the 999 register, then useTailStream("ta1_acks") and useMergedTail("ta1_acks", ta1Items) inside the Ta1AcksSection. Each stream mounts its own <TailStatusPill> so the operator sees a live / connecting / stalled indicator independently for 999s and TA1s. The 999 pill sits in the hero next to the existing "Envelope accepted / Mixed envelope" badge (the two read on the same scanline so the operator's eye lands on both at once). The TA1 pill sits in the TA1 section header next to the "Envelope acks" eyebrow.

The page continues to use the existing useAcks / useTa1Acks TanStack Query hooks for the initial JSON fetch — the tail subscription is an additive append layer, never a replacement (per the original live-tail spec's locked decisions).

4.7 Handler event_bus kwarg removed; parse endpoints thread event_bus to the store

handle_999 / handle_ta1 / handle_277ca / handle_835 drop the event_bus: Optional[object] = None parameter. They drop the if event_bus is not None: ... block too. The handlers do the parse + persist as today; the store does the publish.

The FastAPI /api/parse-999 / /api/parse-ta1 / /api/parse-277ca endpoints (which today inline the parse + persist logic) keep their inline code but gain event_bus=request.app.state.event_bus on the call to store.add_*_ack(...). The migration of those inline bodies to call the handler functions is out of scope for SP25 — it is the SP27 Task 6 follow-up and is independent of this bug fix.

5. Test impact

Per the cyclone-tests skill, every backend change ships a pytest case and every frontend change ships a vitest sibling.

Backend:

• backend/tests/test_store_acks_publish.py (new) — three red-green-red tests asserting add_999_ack / add_ta1_ack / add_277ca_ack publish the right event kind with a payload that round-trips through to_ui_ack / to_ui_ta1_ack / the 277CA serializer. Includes a "publish failure does not roll back the row" test.
• backend/tests/test_api_ack_stream.py (new) — integration tests for /api/acks/stream and /api/ta1-acks/stream: snapshot shape, snapshot_end ordering, live event lands within one cycle, heartbeat cadence, 404 / 401 paths.
• backend/tests/test_api_999.py and test_api_ta1.py — existing tests keep passing (regression). A new case asserts that a successful parse fires the event on app.state.event_bus.

Frontend:

• src/hooks/useTailStream.test.ts (new) — extends the existing dispatch test with cases for "acks" and "ta1_acks".
• src/hooks/useMergedTail.test.ts (new or extended) — slice-selector tests for the new resources.
• src/store/tail-store.test.ts (new) — addAck / addTa1Ack FIFO eviction, dedup, and the reset("acks") / reset("ta1_acks") paths.
• src/pages/Acks.test.tsx — existing tests keep passing. Two new cases: "live event lands a new row in the 999 table" and "live event lands a new row in the TA1 table" (both mock streamTail via vi.mock("@/lib/tail-stream", ...)).
• src/lib/tail-stream.test.ts (new or extended) — KNOWN_TYPES includes the new event types (or, more precisely, the wire format parser already treats item / snapshot_end / heartbeat / item_dropped / error symmetrically across resources, so the new resource URLs are the only thing that needs to land).

6. Branch / commit / merge shape

Per the SP-N increment flow (see cyclone-spec):

• Branch: sp25-ack-live-tail off main.
• Spec: this file, committed as docs(spec): design for SP25 ack live-tail.
• Plan: docs/superpowers/plans/2026-07-02-cyclone-ack-live-tail.md, committed as docs(plan): plan for SP25 ack live-tail.
• Implementation commits: all feat(sp25): ... on the branch.
• PR title: SP25 Ack live-tail.
• Merge: single atomic merge commit merge: SP25 ack live-tail into main (no squash, no rebase).