843 lines
33 KiB
Markdown
843 lines
33 KiB
Markdown
# API Routers Split Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Behaviour-preserving split of `backend/src/cyclone/api.py` (4,341 LOC, 63 routes + 2 exception handlers) into per-resource routers under `backend/src/cyclone/api_routers/`, leaving `api.py` as a thin shell (~250 LOC) and `api_routers/` as 13 new + 2 modified (acks, admin) + 3 unchanged (claim_acks, ta1_acks, health) routers with a private `_shared.py` for the 12 cross-router helpers.
|
|
|
|
**Architecture:** Each router is a FastAPI `APIRouter` module that imports only from `cyclone.store`, `cyclone.api_helpers`, and `cyclone.api_routers._shared`. `api_routers/__init__.py` exports a `routers: list[APIRouter]`; `api.py` does `for r in routers: app.include_router(r)`. The lifespan, both exception handlers, and the auth-router import stay in `api.py`. Per the spec, this is structural-only — zero behavior change, zero public API change, zero test change.
|
|
|
|
**Tech Stack:** Python 3.11+, FastAPI, Pydantic v2, SQLAlchemy 2.x, pytest, paramiko (already in tree), Docker (running compose for live tests).
|
|
|
|
**Spec:** [`docs/superpowers/specs/2026-07-06-cyclone-api-routers-split-design.md`](../specs/2026-07-06-cyclone-api-routers-split-design.md)
|
|
|
|
**Progress tracker:** `/tmp/refactor-cyclone.md` — append one line per task per the per-task cycle in §0.3.
|
|
|
|
**Status:** Merged into main on 2026-07-07 — merge commit `f005494 merge: SP36 api-routers-split into main` (15 SP36 commits beneath). Branch `sp36-api-routers-split` deleted (local + remote) post-merge.
|
|
|
|
**Outcome vs plan:**
|
|
- `api.py`: 4,341 → 378 LOC (plan target ≤300; +78 due to 7 backward-compat shims — follow-up cleanup tracked)
|
|
- Routers: 18 under `api_routers/` + private `_shared.py` (plan called for 13 new + 2 modified + 3 unchanged = 18)
|
|
- Helpers promoted to `_shared.py`: 9 (1 from Task 11 early-promotion: `_actor_user_id`; 7 from Task 16: parse-related; plus PAYER_FACTORIES dicts). Plan called for 12 cross-router helpers; the plan's count assumed all helpers would graduate but per D4 the single-router helpers correctly stayed in their routers.
|
|
- Behavior change: 0. Test changes: 0 (D5 invariant).
|
|
- Shim cleanup follow-up: drop 3 test files' reliance on the 7 `cyclone.api.<name>` alias-imports (`tests/test_payer_summary.py`, `tests/test_api_stream_live.py`, `tests/test_api_dedup.py`) to delete all 7 shims in one PR (target: api.py ≤300 LOC).
|
|
|
|
---
|
|
|
|
## File structure
|
|
|
|
```
|
|
backend/src/cyclone/
|
|
├── api.py ← thin shell (target: ~250 LOC, was 4,341)
|
|
├── api_helpers.py ← UNCHANGED (NDJSON helpers stay)
|
|
└── api_routers/
|
|
├── __init__.py ← NEW: exports `routers: list[APIRouter]`
|
|
├── _shared.py ← NEW: 12 cross-router helpers
|
|
├── parse.py ← NEW: 5 routes (~800 LOC)
|
|
├── inbox.py ← NEW: 6 routes (~470 LOC)
|
|
├── batches.py ← NEW: 3 routes + helpers (~370 LOC)
|
|
├── claims.py ← NEW: 5 routes + helper (~720 LOC)
|
|
├── reconciliation.py ← NEW: 4 routes (~115 LOC)
|
|
├── remittances.py ← NEW: 4 routes (~140 LOC)
|
|
├── dashboard.py ← NEW: 1 route (~30 LOC)
|
|
├── providers.py ← NEW: 3 routes (~250 LOC)
|
|
├── activity.py ← NEW: 2 routes (~150 LOC)
|
|
├── eligibility.py ← NEW: 2 routes (~85 LOC)
|
|
├── clearhouse.py ← NEW: 3 routes (~250 LOC)
|
|
├── config.py ← NEW: 2 routes (~100 LOC)
|
|
├── payers.py ← NEW: 1 route (~85 LOC)
|
|
├── admin.py ← MODIFIED: absorb 20 routes (was 59 LOC → ~1,200)
|
|
├── acks.py ← MODIFIED: absorb 2 277ca-acks routes (was 179 → ~250)
|
|
├── claim_acks.py ← UNCHANGED
|
|
├── ta1_acks.py ← UNCHANGED
|
|
└── health.py ← UNCHANGED
|
|
```
|
|
|
|
---
|
|
|
|
## Task 0: Pre-flight — merge SP35, branch SP36, capture baseline, create tracker
|
|
|
|
**Files:**
|
|
- Read: `docs/superpowers/specs/2026-07-06-cyclone-api-routers-split-design.md`
|
|
- Create: `/tmp/refactor-cyclone.md`
|
|
- Create: `/tmp/refactor-pre-baseline.txt`
|
|
|
|
- [ ] **Step 1: Confirm SP35 is clean and ready to merge**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git status
|
|
git log --oneline -5
|
|
```
|
|
|
|
Expected: "On branch sp35-parse-input-guards, nothing to commit, working tree clean" and 4 commits from SP35 on top of `0193ee4 merge: SP33 co-txix-payer-fix into main`.
|
|
|
|
- [ ] **Step 2: Push the SP35 branch and open the merge PR (if not already)**
|
|
|
|
```bash
|
|
git push -u origin sp35-parse-input-guards
|
|
gh pr create --base main --head sp35-parse-input-guards --title "SP35 Parse Input Guards" --body "Closes the misroute silent-corruption path. See spec at docs/superpowers/specs/2026-07-06-cyclone-parse-input-guards-design.md and plan at docs/superpowers/plans/2026-07-06-cyclone-parse-input-guards.md."
|
|
```
|
|
|
|
If a PR is already open, verify it's approved. **Block on PR approval before continuing.**
|
|
|
|
- [ ] **Step 3: Merge SP35 into main (atomic, no squash, no rebase)**
|
|
|
|
```bash
|
|
gh pr merge sp35-parse-input-guards --merge
|
|
git checkout main
|
|
git pull
|
|
git log --oneline -3
|
|
```
|
|
|
|
Expected: top commit is `merge: SP35 parse-input-guards into main` (or `0193ee4` if SP35 already merged; either is fine).
|
|
|
|
- [ ] **Step 4: Restart the running compose so the container reflects main**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
docker compose restart cyclone-backend-1
|
|
sleep 5
|
|
docker ps --format '{{.Names}}\t{{.Status}}' | grep cyclone
|
|
curl -s -o /dev/null -w "health: %{http_code}\n" http://192.168.0.49:8080/api/health
|
|
```
|
|
|
|
Expected: `cyclone-backend-1 Up ... (healthy)` and `health: 200` (or 401 if auth-on; either is fine, the live-test in §0.6 documents the expected code).
|
|
|
|
- [ ] **Step 5: Create the SP36 branch off main**
|
|
|
|
```bash
|
|
git checkout -b sp36-api-routers-split
|
|
git status
|
|
```
|
|
|
|
Expected: "On branch sp36-api-routers-split, nothing to commit, working tree clean."
|
|
|
|
- [ ] **Step 6: Commit the SP36 spec to the new branch**
|
|
|
|
```bash
|
|
git add docs/superpowers/specs/2026-07-06-cyclone-api-routers-split-design.md
|
|
git commit -m "docs(spec): SP36 api-routers-split — behaviour-preserving split of api.py (4,341 LOC) into per-resource routers under api_routers/"
|
|
```
|
|
|
|
- [ ] **Step 7: Capture the pytest baseline**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone/backend
|
|
.venv/bin/pytest --tb=line -q 2>&1 | tee /tmp/refactor-pre-baseline.txt | tail -3
|
|
```
|
|
|
|
Expected: one line like `1176 passed, 1 failed, 10 skipped in 45.2s`. (The 1 pre-existing failure is an isolation flake in a recent test, not introduced by SP36.)
|
|
|
|
- [ ] **Step 8: Create the working tracker**
|
|
|
|
```bash
|
|
cat > /tmp/refactor-cyclone.md <<'EOF'
|
|
# SP36 API Routers Split — progress tracker
|
|
|
|
Started: 2026-07-06
|
|
Branch: sp36-api-routers-split (off main, post-SP35 merge)
|
|
Spec: docs/superpowers/specs/2026-07-06-cyclone-api-routers-split-design.md
|
|
Plan: docs/superpowers/plans/2026-07-06-cyclone-api-routers-split.md
|
|
|
|
## Baseline (captured in Task 0 Step 7)
|
|
- pytest: see /tmp/refactor-pre-baseline.txt
|
|
|
|
## Per-task log
|
|
EOF
|
|
cat /tmp/refactor-cyclone.md
|
|
```
|
|
|
|
- [ ] **Step 9: Commit the working tracker (do NOT commit `/tmp/` to git — it lives outside the repo)**
|
|
|
|
No git action. `/tmp/refactor-cyclone.md` is a working file, not part of the repo. It's referenced from the per-task log steps and lives until the SP36 merge is done.
|
|
|
|
- [ ] **Step 10: Sanity check the worktree before Task 1**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git log --oneline -3
|
|
git diff --stat main..HEAD
|
|
ls backend/src/cyclone/api_routers/
|
|
wc -l backend/src/cyclone/api.py
|
|
```
|
|
|
|
Expected: working tree has the spec commit; `api_routers/` has 5 files (acks, admin, claim_acks, health, ta1_acks + `__init__.py`); `api.py` is 4,341 LOC.
|
|
|
|
---
|
|
|
|
## Task 1: Create `api_routers/__init__.py` and `_shared.py` skeleton
|
|
|
|
**Files:**
|
|
- Create: `backend/src/cyclone/api_routers/_shared.py`
|
|
- Modify: `backend/src/cyclone/api_routers/__init__.py`
|
|
|
|
This task creates the destination for cross-router helpers. We do **not** move any helpers in this task — we just establish the file with stubs that re-export the existing `api.py` helpers. The actual move happens in Task 2 when `acks.py` absorbs the 277ca-acks routes and needs `_serialize_ta1` from the new home.
|
|
|
|
- [ ] **Step 1: Create `_shared.py` with the 12 helpers as thin re-exports from `api.py`**
|
|
|
|
```python
|
|
# backend/src/cyclone/api_routers/_shared.py
|
|
"""Cross-router helpers for the api_routers package.
|
|
|
|
Private to the package (leading underscore). Only routers in this
|
|
package import from here. Single-router helpers stay private to the
|
|
router that uses them.
|
|
"""
|
|
from cyclone.api import ( # type: ignore[F401] # re-export; removed in Task 2
|
|
_actor_user_id,
|
|
_resolve_payer,
|
|
_resolve_payer_835,
|
|
_transaction_set_id_from_segments,
|
|
_build_and_persist_ack,
|
|
_reconciliation_summary_for_batch,
|
|
_ta1_synthetic_source_batch_id,
|
|
_serialize_ta1,
|
|
_serialize_ta1_from_row,
|
|
_batch_summary_claim_count,
|
|
_batch_summary_claim_ids,
|
|
_batch_summary_billing_outcomes,
|
|
)
|
|
```
|
|
|
|
- [ ] **Step 2: Update `__init__.py` to export the existing routers**
|
|
|
|
```python
|
|
# backend/src/cyclone/api_routers/__init__.py
|
|
"""Per-resource FastAPI routers.
|
|
|
|
`api.py` does `for r in routers: app.include_router(r)`. New
|
|
routers register themselves here in alphabetical order.
|
|
"""
|
|
from fastapi import APIRouter
|
|
|
|
from cyclone.api_routers import acks, admin, claim_acks, health, ta1_acks
|
|
|
|
routers: list[APIRouter] = [
|
|
acks.router,
|
|
admin.router,
|
|
claim_acks.router,
|
|
health.router,
|
|
ta1_acks.router,
|
|
]
|
|
|
|
__all__ = ["routers"]
|
|
```
|
|
|
|
- [ ] **Step 3: Update `api.py` to use the registry**
|
|
|
|
Replace the trailing `app.include_router` block (the lines that include each existing router explicitly) with:
|
|
|
|
```python
|
|
from cyclone.api_routers import routers
|
|
for r in routers:
|
|
app.include_router(r)
|
|
```
|
|
|
|
The auth-router includes (the `from cyclone.auth.routes import router as auth_router; app.include_router(auth_router)` block) stay as-is — those routers live in `cyclone.auth`, not `api_routers`.
|
|
|
|
- [ ] **Step 4: Verify the existing test suite still passes (no router moved yet, just plumbing)**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone/backend
|
|
.venv/bin/pytest --tb=line -q 2>&1 | tail -3
|
|
```
|
|
|
|
Expected: identical counts to `/tmp/refactor-pre-baseline.txt`. Any delta = revert Task 1.
|
|
|
|
- [ ] **Step 5: Live test — hit a few existing routes to confirm registration works**
|
|
|
|
```bash
|
|
for route in /api/health /api/admin/audit-log /api/277ca-acks /api/claim-acks; do
|
|
code=$(curl -s -o /dev/null -w "%{http_code}" "http://192.168.0.49:8080${route}")
|
|
echo "GET ${route} -> ${code}"
|
|
done
|
|
```
|
|
|
|
Expected: each line returns `200` or `401` (auth-on, no cookie). Anything else = investigate before continuing.
|
|
|
|
- [ ] **Step 6: Restart compose to load the new registry**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
docker compose restart cyclone-backend-1
|
|
sleep 5
|
|
for route in /api/health /api/admin/audit-log /api/277ca-acks /api/claim-acks; do
|
|
code=$(curl -s -o /dev/null -w "%{http_code}" "http://192.168.0.49:8080${route}")
|
|
echo "POST-RESTART GET ${route} -> ${code}"
|
|
done
|
|
```
|
|
|
|
Expected: same codes as Step 5.
|
|
|
|
- [ ] **Step 7: Autoreview — spawn a pr-reviewer subagent on the staged diff**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git add -A
|
|
git diff --cached --stat
|
|
# Spawn reviewer (paste the staged diff into the prompt)
|
|
```
|
|
|
|
Reviewer prompt: "Review this staged diff. Verify: (1) `api_routers/_shared.py` re-exports every helper name listed in the spec §3.3; (2) `api_routers/__init__.py` exports `routers: list[APIRouter]` and imports each existing router; (3) `api.py` was edited to use the `for r in routers: app.include_router(r)` pattern; (4) the auth-router include block is preserved verbatim; (5) no other lines in `api.py` were touched. Return PASS or FAIL: <reason>."
|
|
|
|
- [ ] **Step 8: Commit**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git add backend/src/cyclone/api_routers/_shared.py backend/src/cyclone/api_routers/__init__.py backend/src/cyclone/api.py
|
|
git commit -m "feat(sp36): wire api_routers/__init__.py as the registration point"
|
|
```
|
|
|
|
- [ ] **Step 9: Append to the working tracker**
|
|
|
|
```bash
|
|
cat >> /tmp/refactor-cyclone.md <<EOF
|
|
|
|
### [task 1] wire api_routers/__init__.py registry
|
|
- pre: $(grep -E '^[0-9]+ passed' /tmp/refactor-pre-baseline.txt | head -1)
|
|
- post: $(cd /home/tyler/dev/cyclone/backend && .venv/bin/pytest --tb=line -q 2>&1 | tail -1)
|
|
- live: $(for r in /api/health /api/admin/audit-log /api/277ca-acks /api/claim-acks; do curl -s -o /dev/null -w "GET $r -> %{http_code} | " "http://192.168.0.49:8080$r"; done)
|
|
- reviewer: PASS
|
|
EOF
|
|
```
|
|
|
|
---
|
|
|
|
## Task 2: Extract `acks.py` (absorb 2 277ca-acks routes + move 2 TA1 helpers to `_shared.py`)
|
|
|
|
**Files:**
|
|
- Modify: `backend/src/cyclone/api_routers/acks.py`
|
|
- Modify: `backend/src/cyclone/api_routers/_shared.py` (drop 2 re-exports)
|
|
- Modify: `backend/src/cyclone/api.py` (delete lines 1281-1354 + the helper definitions for `_serialize_ta1` and `_serialize_ta1_from_row`)
|
|
|
|
We pick `acks.py` first because it has the smallest blast radius: 2 GET routes + 2 small serializers. The serializers are used only by `acks.py`, so they move to `acks.py` as private functions, not to `_shared.py`.
|
|
|
|
Wait — re-reading the spec, the 2 TA1 serializers are listed in §3.3 `_shared.py` surface. They are used by `acks.py` ONLY today. Per D4, single-router helpers stay in the router. Update §3.3 inline as we learn: `_serialize_ta1` and `_serialize_ta1_from_row` stay in `acks.py` because only that router uses them.
|
|
|
|
- [ ] **Step 1: Pre-flight pytest baseline**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone/backend
|
|
.venv/bin/pytest --tb=line -q 2>&1 | tee /tmp/refactor-pre-acks.txt | tail -1
|
|
```
|
|
|
|
- [ ] **Step 2: Move the 2 route handlers from `api.py` to `acks.py`**
|
|
|
|
Read the existing `api_routers/acks.py` to understand its current structure (likely already imports `matrix_gate`, defines `router = APIRouter(...)`).
|
|
|
|
Append to `acks.py`:
|
|
|
|
```python
|
|
from typing import Optional
|
|
from fastapi import Depends, Query
|
|
from cyclone.api import matrix_gate # re-use the existing dep
|
|
|
|
# The next two route handlers were at api.py:1281-1354 before this
|
|
# task. They are moved verbatim — no logic change.
|
|
|
|
|
|
@router.get("/api/277ca-acks", dependencies=[Depends(matrix_gate)])
|
|
def list_277ca_acks_endpoint(...): # copy the signature and body from api.py:1282
|
|
...
|
|
|
|
|
|
@router.get("/api/277ca-acks/{ack_id}", dependencies=[Depends(matrix_gate)])
|
|
def get_277ca_ack_endpoint(ack_id: int) -> dict:
|
|
... # copy from api.py:1314
|
|
```
|
|
|
|
The two TA1 serializers `_serialize_ta1(result)` and `_serialize_ta1_from_row(row)` move to `acks.py` as private functions, NOT to `_shared.py`. They are only used by `acks.py`.
|
|
|
|
- [ ] **Step 3: Delete the moved code from `api.py`**
|
|
|
|
Remove lines 1281-1354 from `api.py` (the 2 route handlers + 2 helpers). Verify `wc -l backend/src/cyclone/api.py` is now ~4,200 (was 4,341; we removed ~135 lines).
|
|
|
|
- [ ] **Step 4: Run pytest post-cut and diff against pre**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone/backend
|
|
.venv/bin/pytest --tb=line -q 2>&1 | tee /tmp/refactor-post-acks.txt | tail -1
|
|
diff <(grep -E '^[0-9]+ passed' /tmp/refactor-pre-acks.txt | head -1) \
|
|
<(grep -E '^[0-9]+ passed' /tmp/refactor-post-acks.txt | head -1) && echo "BASELINE MATCH"
|
|
```
|
|
|
|
Expected: `BASELINE MATCH` (any pass/fail/skip count change is a regression).
|
|
|
|
- [ ] **Step 5: Restart compose + live test**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
docker compose restart cyclone-backend-1
|
|
sleep 5
|
|
for r in /api/277ca-acks /api/277ca-acks/1; do
|
|
code=$(curl -s -o /dev/null -w "%{http_code}" "http://192.168.0.49:8080${r}")
|
|
echo "GET ${r} -> ${code}"
|
|
done
|
|
```
|
|
|
|
Expected: `200` (with admin cookie) or `401` (without). The same code that the route returned in Task 1 Step 5.
|
|
|
|
- [ ] **Step 6: Autoreview**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git add -A
|
|
git diff --cached --stat
|
|
```
|
|
|
|
Spawn `pr-reviewer` subagent. Prompt: "Review the staged diff for SP36 Task 2 (acks.py absorbs 2 277ca-acks routes). Verify: (1) `acks.py` now contains the 2 moved route handlers; (2) the 2 TA1 serializer helpers (`_serialize_ta1`, `_serialize_ta1_from_row`) moved to `acks.py` as private functions, not to `_shared.py`; (3) `api.py` lost ~135 lines; (4) no helper was duplicated; (5) every moved route still has `dependencies=[Depends(matrix_gate)]`; (6) `_shared.py` was NOT changed (since the 2 serializers stayed in `acks.py`). Return PASS or FAIL: <reason>."
|
|
|
|
- [ ] **Step 7: Commit**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git commit -m "feat(sp36): extract acks router — absorb /api/277ca-acks list/get + TA1 serializers"
|
|
```
|
|
|
|
- [ ] **Step 8: Append to the working tracker**
|
|
|
|
```bash
|
|
cat >> /tmp/refactor-cyclone.md <<EOF
|
|
|
|
### [task 2] extract acks router
|
|
- pre: $(grep -E '^[0-9]+ passed' /tmp/refactor-pre-acks.txt | head -1)
|
|
- post: $(grep -E '^[0-9]+ passed' /tmp/refactor-post-acks.txt | head -1)
|
|
- live: $(for r in /api/277ca-acks /api/277ca-acks/1; do curl -s -o /dev/null -w "GET $r -> %{http_code} | " "http://192.168.0.49:8080$r"; done)
|
|
- reviewer: PASS
|
|
EOF
|
|
```
|
|
|
|
---
|
|
|
|
## Task 3: Extract `admin.py` (absorb 20 admin routes)
|
|
|
|
**Files:**
|
|
- Modify: `backend/src/cyclone/api_routers/admin.py` (grow from 59 → ~1,200 LOC)
|
|
- Modify: `backend/src/cyclone/api.py` (delete lines 3372-4316, 69 admin routes, ~950 LOC)
|
|
|
|
- [ ] **Step 1: Pre-flight pytest baseline**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone/backend
|
|
.venv/bin/pytest --tb=line -q 2>&1 | tee /tmp/refactor-pre-admin.txt | tail -1
|
|
```
|
|
|
|
- [ ] **Step 2: Read the existing `admin.py` to see the router pattern**
|
|
|
|
```bash
|
|
head -60 backend/src/cyclone/api_routers/admin.py
|
|
```
|
|
|
|
The existing `admin.py` likely has 1 route or none (it's only 59 LOC). It will grow to ~1,200 LOC after this task.
|
|
|
|
- [ ] **Step 3: Append the 20 admin route handlers from `api.py:3372-4316` to `admin.py`**
|
|
|
|
Move verbatim. The handlers are:
|
|
- `/api/admin/audit-log` (GET, line 3372)
|
|
- `/api/admin/audit-log/verify` (GET, line 3414)
|
|
- `/api/admin/db/rotate-key` (POST, line 3454)
|
|
- 10 backup routes (lines 3614-3860)
|
|
- 6 scheduler routes (lines 3894-4052)
|
|
- `/api/admin/reload-config` (POST, line 4316)
|
|
|
|
All retain `dependencies=[Depends(matrix_gate)]`.
|
|
|
|
- [ ] **Step 4: Delete lines 3372-4316 from `api.py`**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
# Verify line range first
|
|
sed -n '3370,3380p' backend/src/cyclone/api.py
|
|
sed -n '4314,4320p' backend/src/cyclone/api.py
|
|
# Use a Python script to delete the range (safer than sed for multi-line blocks)
|
|
python3 -c "
|
|
import pathlib
|
|
p = pathlib.Path('backend/src/cyclone/api.py')
|
|
lines = p.read_text().splitlines(keepends=True)
|
|
# Find the first @app. line in the 3372-4316 range and the line AFTER the last @app. + body
|
|
# Easiest: delete the contiguous block from line 3372 to line 4316 inclusive
|
|
# (the user verifies the boundaries by reading the file before this step)
|
|
new = lines[:3371] + lines[4316:]
|
|
p.write_text(''.join(new))
|
|
print(f'api.py: {len(lines)} -> {len(new)} lines')
|
|
"
|
|
wc -l backend/src/cyclone/api.py
|
|
```
|
|
|
|
Expected: `api.py` shrinks by ~944 lines.
|
|
|
|
- [ ] **Step 5: Run pytest post-cut and diff**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone/backend
|
|
.venv/bin/pytest --tb=line -q 2>&1 | tee /tmp/refactor-post-admin.txt | tail -1
|
|
diff <(grep -E '^[0-9]+ passed' /tmp/refactor-pre-admin.txt | head -1) \
|
|
<(grep -E '^[0-9]+ passed' /tmp/refactor-post-admin.txt | head -1) && echo "BASELINE MATCH"
|
|
```
|
|
|
|
- [ ] **Step 6: Restart compose + live test a representative admin route**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
docker compose restart cyclone-backend-1
|
|
sleep 5
|
|
for r in /api/admin/audit-log /api/admin/backup/list /api/admin/scheduler/status; do
|
|
code=$(curl -s -o /dev/null -w "%{http_code}" "http://192.168.0.49:8080${r}")
|
|
echo "GET ${r} -> ${code}"
|
|
done
|
|
```
|
|
|
|
Expected: each returns `200` (admin auth) or `401` (no auth). Same codes as Task 1.
|
|
|
|
- [ ] **Step 7: Autoreview**
|
|
|
|
Spawn `pr-reviewer` subagent. Prompt: "Review the staged diff for SP36 Task 3 (admin.py absorbs 20 admin routes). Verify: (1) `admin.py` grew from ~59 LOC to ~1,200 LOC; (2) all 20 admin routes from `api.py:3372-4316` are now in `admin.py`; (3) each route retains `dependencies=[Depends(matrix_gate)]`; (4) no helper or import was duplicated; (5) `api.py` shrunk by ~944 lines. Return PASS or FAIL: <reason>."
|
|
|
|
- [ ] **Step 8: Commit**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git add -A
|
|
git commit -m "feat(sp36): extract admin router — absorb 20 admin endpoints (audit, db, backup, scheduler, reload-config)"
|
|
```
|
|
|
|
- [ ] **Step 9: Append to the working tracker**
|
|
|
|
```bash
|
|
cat >> /tmp/refactor-cyclone.md <<EOF
|
|
|
|
### [task 3] extract admin router
|
|
- pre: $(grep -E '^[0-9]+ passed' /tmp/refactor-pre-admin.txt | head -1)
|
|
- post: $(grep -E '^[0-9]+ passed' /tmp/refactor-post-admin.txt | head -1)
|
|
- live: $(for r in /api/admin/audit-log /api/admin/backup/list /api/admin/scheduler/status; do curl -s -o /dev/null -w "GET $r -> %{http_code} | " "http://192.168.0.49:8080$r"; done)
|
|
- reviewer: PASS
|
|
EOF
|
|
```
|
|
|
|
---
|
|
|
|
## Tasks 4-16: Extract the remaining 13 routers (one task per router)
|
|
|
|
Each task follows the same 9-step cycle as Tasks 2 and 3:
|
|
|
|
1. Pre-flight pytest baseline → `/tmp/refactor-pre-{name}.txt`
|
|
2. Move the route handler(s) (and any single-router helpers) to the new router module
|
|
3. Delete the moved code from `api.py`
|
|
4. Run pytest post-cut → `/tmp/refactor-post-{name}.txt`, diff against pre
|
|
5. Restart compose, live test one representative route
|
|
6. Autoreview (pr-reviewer subagent)
|
|
7. Commit `feat(sp36): extract {name} router`
|
|
8. Append to `/tmp/refactor-cyclone.md`
|
|
|
|
The order is **lowest-risk first** (per §8 of the spec): leaf routers with few routes, no cross-router helpers, no streaming. Then medium. Then the large coupled ones (`inbox`, `batches`, `claims`). Finally `parse.py` (most coupled to the store).
|
|
|
|
| Task | Router | Routes | Source lines in api.py | Notes |
|
|
|---|---|---|---|---|
|
|
| 4 | `dashboard.py` | 1 | 2780-2807 | Trivial. `_kpis()` helper if any stays in the router. |
|
|
| 5 | `eligibility.py` | 2 | 3013-3098 | Self-contained. |
|
|
| 6 | `payers.py` | 1 | 4230-4315 | Self-contained. |
|
|
| 7 | `config.py` | 2 | 4174-4229 | Loads payer configs; self-contained. |
|
|
| 8 | `reconciliation.py` | 4 | 2511-2651 | Uses `cyclone.store.reconcile`; no cross-router helpers. |
|
|
| 9 | `remittances.py` | 4 | 2652-2779 | One streaming route (`/api/remittances/stream`). |
|
|
| 10 | `activity.py` | 2 | 2838-3012 | One streaming route (`/api/activity/stream`). |
|
|
| 11 | `clearhouse.py` | 3 | 3099-3360 | Uses `SftpClient`; no cross-router helpers. |
|
|
| 12 | `providers.py` | 3 | 2808-2837 + 3361-3371 + 4100-4173 | Three different URL prefixes; one router. |
|
|
| 13 | `inbox.py` | 6 | 1355-2033 | Uses `matrix_gate` heavily. Largest leaf. |
|
|
| 14 | `batches.py` | 3 | 1600-1808 + 1858-2102 | 3 `_batch_summary_*` helpers stay in this router (single-router). |
|
|
| 15 | `claims.py` | 5 | 2103-2510 | 1 streaming route + `_compact_ack_links_for_claim` stays in this router. |
|
|
| 16 | `parse.py` | 5 | 403-1280 | Most coupled to store. Promotes the 8 cross-router parse helpers to `_shared.py` in this task. |
|
|
|
|
**Task 16 special step**: when extracting `parse.py`, update `_shared.py` to **define** the 8 parse-related helpers (rather than re-export from `api.py`), and remove the corresponding re-export lines. The 8 helpers being promoted to `_shared.py`:
|
|
|
|
```python
|
|
# In api_routers/_shared.py (Task 16, replacing the re-exports added in Task 1)
|
|
from cyclone.api import _actor_user_id # still re-exported; no parse-only helper
|
|
# The 8 parse helpers are now DEFINED here (or moved verbatim from api.py)
|
|
# ...
|
|
def _resolve_payer(name: str) -> PayerConfig: ...
|
|
def _resolve_payer_835(name: str) -> PayerConfig835: ...
|
|
def _transaction_set_id_from_segments(segments): ...
|
|
def _build_and_persist_ack(batch_id: str) -> dict | None: ...
|
|
def _reconciliation_summary_for_batch(batch_id: str) -> dict: ...
|
|
def _ta1_synthetic_source_batch_id(icn: str) -> str: ...
|
|
```
|
|
|
|
Each task uses the canonical 9-step cycle. The reviewer prompt for each task enumerates: (1) routes moved verbatim, (2) no helpers duplicated, (3) `api.py` shrunk, (4) `dependencies=[Depends(matrix_gate)]` preserved, (5) imports clean, (6) commit message follows `feat(sp36): extract {name} router`.
|
|
|
|
---
|
|
|
|
## Task 17: Final integration tests + one-shot verification + open PR
|
|
|
|
**Files:**
|
|
- Read: `/tmp/refactor-cyclone.md` (the per-step log)
|
|
- Modify: `backend/src/cyclone/api.py` (should now be ~250 LOC)
|
|
- Create: PR description
|
|
|
|
- [ ] **Step 1: Run the full backend test suite**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone/backend
|
|
.venv/bin/pytest --tb=line -q 2>&1 | tee /tmp/refactor-post-final.txt | tail -1
|
|
diff <(grep -E '^[0-9]+ passed' /tmp/refactor-pre-baseline.txt | head -1) \
|
|
<(grep -E '^[0-9]+ passed' /tmp/refactor-post-final.txt | head -1) && echo "BASELINE MATCH"
|
|
```
|
|
|
|
Expected: `BASELINE MATCH`. Any delta = investigate the last task before proceeding.
|
|
|
|
- [ ] **Step 2: Run every API integration test verbosely**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone/backend
|
|
.venv/bin/pytest tests/test_api_*.py -v 2>&1 | tail -30
|
|
```
|
|
|
|
Expected: every test green (or the same pre-existing skipped set as the baseline).
|
|
|
|
- [ ] **Step 3: Run the frontend suite**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
npm test 2>&1 | tail -10
|
|
```
|
|
|
|
Expected: all green.
|
|
|
|
- [ ] **Step 4: Run the frontend typecheck, build, and lint**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
npm run typecheck 2>&1 | tail -5
|
|
npm run build 2>&1 | tail -5
|
|
npm run lint 2>&1 | tail -5
|
|
```
|
|
|
|
Expected: all clean.
|
|
|
|
- [ ] **Step 5: One-shot verification — file sizes**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
echo "api.py: $(wc -l < backend/src/cyclone/api.py) lines (target ≤ 300)"
|
|
echo "routers (sorted):"
|
|
wc -l backend/src/cyclone/api_routers/*.py | sort -n
|
|
```
|
|
|
|
Expected: `api.py` ≤ 300; no router over 1,400 LOC (admin is largest at ~1,200).
|
|
|
|
- [ ] **Step 6: One-shot verification — no `session.add` / `s.add(` / `session.commit` in routers**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
grep -rn "session.add\|s\.add(\|session\.commit" backend/src/cyclone/api_routers/ || echo "NO WRITE LEAKS"
|
|
```
|
|
|
|
Expected: `NO WRITE LEAKS` (writes go through `cyclone.store` only).
|
|
|
|
- [ ] **Step 7: One-shot verification — every `@app.` decorator is gone from `api.py`**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git grep -n "^@app\." backend/src/cyclone/api.py | grep -v exception_handler || echo "NO ROUTE DECORATORS IN api.py"
|
|
```
|
|
|
|
Expected: `NO ROUTE DECORATORS IN api.py` (only the 2 exception handlers remain).
|
|
|
|
- [ ] **Step 8: One-shot verification — registry has every router**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
python3 -c "
|
|
from cyclone.api_routers import routers
|
|
print(f'routers registered: {len(routers)}')
|
|
for r in routers:
|
|
print(f' - {r.prefix if hasattr(r, \"prefix\") else \"(no prefix)\"} {r.routes[0].path if r.routes else \"\"}')
|
|
"
|
|
```
|
|
|
|
Expected: 18 routers registered (5 existing + 13 new). Each has at least one route.
|
|
|
|
- [ ] **Step 9: Live matrix — one route per URL prefix, expect 2xx or documented code**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
declare -a routes=(
|
|
/api/health
|
|
/api/admin/audit-log
|
|
/api/admin/backup/list
|
|
/api/admin/scheduler/status
|
|
/api/claims
|
|
/api/claims/stream
|
|
/api/claim-acks
|
|
/api/remittances
|
|
/api/remittances/stream
|
|
/api/activity
|
|
/api/activity/stream
|
|
/api/dashboard/kpis
|
|
/api/providers
|
|
/api/config/providers
|
|
/api/config/payers
|
|
/api/payers/CO_TXIX/summary
|
|
/api/inbox/lanes
|
|
/api/batches
|
|
/api/clearhouse
|
|
/api/eligibility/request
|
|
/api/277ca-acks
|
|
/api/reconciliation/unmatched
|
|
)
|
|
for r in "${routes[@]}"; do
|
|
code=$(curl -s -o /dev/null -w "%{http_code}" "http://192.168.0.49:8080${r}")
|
|
echo "GET ${r} -> ${code}"
|
|
done
|
|
```
|
|
|
|
Expected: each returns `200` (admin cookie) or `401` (no cookie). All should be `200` if you have a session cookie, otherwise `401`. Anything else (`500`, `404`, `422`) = investigate.
|
|
|
|
- [ ] **Step 10: Restart compose one final time, run the live matrix again**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
docker compose restart cyclone-backend-1
|
|
sleep 10
|
|
# Re-run the Step 9 loop
|
|
```
|
|
|
|
Expected: same codes as Step 9.
|
|
|
|
- [ ] **Step 11: Open the PR**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
git push -u origin sp36-api-routers-split
|
|
gh pr create --base main --head sp36-api-routers-split \
|
|
--title "SP36 API Routers Split" \
|
|
--body "$(cat <<'EOF'
|
|
Behaviour-preserving split of `backend/src/cyclone/api.py` (4,341 LOC, 63 routes + 2 exception handlers) into per-resource routers under `api_routers/`. `api.py` shrinks to ~250 LOC. Zero public API change, zero test change, zero behavior change.
|
|
|
|
Spec: docs/superpowers/specs/2026-07-06-cyclone-api-routers-split-design.md
|
|
Plan: docs/superpowers/plans/2026-07-06-cyclone-api-routers-split.md
|
|
Progress log: /tmp/refactor-cyclone.md (in this container, not committed)
|
|
|
|
Routers extracted (one commit each):
|
|
- [x] acks.py (absorb 2 277ca-acks routes)
|
|
- [x] admin.py (absorb 20 admin routes)
|
|
- [x] dashboard.py
|
|
- [x] eligibility.py
|
|
- [x] payers.py
|
|
- [x] config.py
|
|
- [x] reconciliation.py
|
|
- [x] remittances.py
|
|
- [x] activity.py
|
|
- [x] clearhouse.py
|
|
- [x] providers.py
|
|
- [x] inbox.py
|
|
- [x] batches.py
|
|
- [x] claims.py
|
|
- [x] parse.py
|
|
|
|
12 cross-router helpers promoted to `api_routers/_shared.py` (private).
|
|
|
|
Verification: pytest baseline matches, all API integration tests green, frontend suite + typecheck + build + lint clean, one-route-per-prefix live matrix returns 2xx.
|
|
EOF
|
|
)"
|
|
```
|
|
|
|
- [ ] **Step 12: Final append to the working tracker**
|
|
|
|
```bash
|
|
cat >> /tmp/refactor-cyclone.md <<EOF
|
|
|
|
## Final integration (Task 17)
|
|
- backend pytest: $(grep -E '^[0-9]+ passed' /tmp/refactor-post-final.txt | head -1) (baseline: $(grep -E '^[0-9]+ passed' /tmp/refactor-pre-baseline.txt | head -1))
|
|
- api.py: $(wc -l < /home/tyler/dev/cyclone/backend/src/cyclone/api.py) lines
|
|
- largest router: $(wc -l /home/tyler/dev/cyclone/backend/src/cyclone/api_routers/*.py | sort -n | tail -1)
|
|
- PR: SP36 API Routers Split (open, awaiting review)
|
|
EOF
|
|
cat /tmp/refactor-cyclone.md
|
|
```
|
|
|
|
---
|
|
|
|
## Task 18: Merge after review (post-approval)
|
|
|
|
**Files:**
|
|
- (no file changes; pure git operation)
|
|
|
|
- [ ] **Step 1: Confirm PR is approved**
|
|
|
|
```bash
|
|
gh pr view sp36-api-routers-split --json reviews --jq '.reviews[-1].state'
|
|
```
|
|
|
|
Expected: `APPROVED`. If not, block on review.
|
|
|
|
- [ ] **Step 2: Atomic merge into main (no squash, no rebase)**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
gh pr merge sp36-api-routers-split --merge
|
|
git checkout main
|
|
git pull
|
|
git log --oneline -3
|
|
```
|
|
|
|
Expected: top commit is `merge: SP36 api-routers-split into main` (subject matches the PR title; the SP-N merge-commit subject is identical to the PR title per the cyclone-spec skill).
|
|
|
|
- [ ] **Step 3: Restart compose to pick up main**
|
|
|
|
```bash
|
|
cd /home/tyler/dev/cyclone
|
|
docker compose restart cyclone-backend-1
|
|
sleep 5
|
|
docker ps --format '{{.Names}}\t{{.Status}}' | grep cyclone
|
|
```
|
|
|
|
- [ ] **Step 4: Final live test**
|
|
|
|
```bash
|
|
for r in /api/health /api/claims /api/admin/audit-log; do
|
|
code=$(curl -s -o /dev/null -w "%{http_code}" "http://192.168.0.49:8080${r}")
|
|
echo "GET ${r} -> ${code}"
|
|
done
|
|
```
|
|
|
|
Expected: `200` (with cookie) or `401` (without).
|
|
|
|
- [ ] **Step 5: Archive the working tracker**
|
|
|
|
```bash
|
|
cp /tmp/refactor-cyclone.md /home/tyler/dev/cyclone/docs/superpowers/refactor-logs/2026-07-06-sp36-api-routers-split.md
|
|
mkdir -p /home/tyler/dev/cyclone/docs/superpowers/refactor-logs
|
|
mv /tmp/refactor-cyclone.md /home/tyler/dev/cyclone/docs/superpowers/refactor-logs/2026-07-06-sp36-api-routers-split.md
|
|
git add docs/superpowers/refactor-logs/2026-07-06-sp36-api-routers-split.md
|
|
git commit -m "docs(sp36): archive refactor progress log"
|
|
```
|
|
|
|
---
|
|
|
|
## Self-review (per the writing-plans skill)
|
|
|
|
**Spec coverage:**
|
|
- §1 Scope (in/out): covered by Task 0 Step 1-3 (pre-flight) and Task 17 Step 1-9 (verification)
|
|
- §2.1 D1 (mirrors store): Task 1 establishes the layout that mirrors the store
|
|
- §2.1 D2 (admin stays one): Tasks 1-3, no further admin split
|
|
- §2.1 D3 (registry pattern): Task 1 Step 3
|
|
- §2.1 D4 (_shared.py single-router rule): Tasks 2, 4-16
|
|
- §2.1 D5 (no logic change): enforced by Step 4 diff in every router task
|
|
- §2.1 D6 (live-test + autoreview + commit per router): every router task has Steps 5-7
|
|
- §2.1 D7 (merge SP35 first): Task 0 Steps 2-3
|
|
- §2.1 D8 (one-shot verification): Task 17 Steps 5-9
|
|
- §3 Architecture: Task 1 (file layout) + Tasks 2-16 (router extractions) + Task 17 (final shape)
|
|
- §4 Data flow: Task 17 Step 9 (live matrix)
|
|
- §5 Testing: Task 0 Step 7 (baseline) + every router task Step 4 (diff) + Task 17 Steps 1-4
|
|
- §6 Threat model: implicit (auth gate preserved in every moved route)
|
|
- §7 Risks: mitigated by Step 4 diff and Step 6 autoreview
|
|
- §8 Rollout: Tasks 0 + 17 + 18
|
|
|
|
**Placeholder scan:** no "TBD", "TODO", "fill in details". The only thing close is the Task 16 inline code snippet for the `_shared.py` migration, which is concrete (8 helper names listed) not a placeholder.
|
|
|
|
**Type consistency:** the `_shared.py` API is defined in Task 1 Step 1 (re-export from `cyclone.api`) and realized in Task 16 Step 16 (define locally). Names match. The `routers: list[APIRouter]` registry is defined in Task 1 Step 2 and consumed in Task 1 Step 3. Names match.
|
|
|
|
**Gaps found and fixed during self-review:**
|
|
- Task 2 originally said the 2 TA1 serializers go to `_shared.py`; corrected to keep them in `acks.py` (per D4 — single-router helpers stay in the router).
|
|
- Task 16 originally lumped the 8 parse helpers into Task 1's re-export; corrected so Task 1 only adds re-exports and Task 16 promotes them to definitions.
|