cyclone/docs/superpowers/specs/2026-06-21-cyclone-pipeline-agent-design.md

# Sub-project 22 — Cyclone Pipeline Agent: Design Spec

**Date:** 2026-06-21
**Status:** Draft (awaiting user review)
**Branch:** `sp22-pipeline-agent` (not yet created; will branch from `main`)
**Aesthetic direction:** n/a (backend automation; no UI surface)

## 1. Scope

A production-grade agent (sibling Python package `cyclone-pipeline`) that
automates the full submission lifecycle of an EDI 837P file through Cyclone
to the Gainwell MFT, and watches for the inbound TA1 + 999 ACKs that
confirm acceptance. The agent exists so a single human operator can drop a
file in and walk away; the agent handles the browser upload, SFTP
submission, ACK polling, and report generation end to end, and writes a
self-contained run directory that OpenClaw / Nora / any other automation
can read.

**In scope:**

- Sibling Python package at `/Users/openclaw/dev/cyclone-pipeline/` (no
  shared imports with the cyclone app; talks to cyclone exclusively over
  HTTP).
- CLI surface: `run <file>`, `run-batch <dir>`, `status <run-id>`,
  `resume <run-id>`, `check-835 <run-id>`.
- Library surface: `CyclonePipeline` class with `run()`, `run_batch()`,
  `resume()`, `check_835()` for embedding in OpenClaw / Nora.
- 7-phase state machine: pre-flight → upload (browser) → verify parse →
  SFTP submit → wait TA1 → wait 999 → scan HTML + report. The "scan
  HTML" and "final report" steps are combined as phase 7 (the HTML
  scan populates the report; the report write is phase 7's
  postcondition).
- Per-run output folder with `report.{md,json}`, `run.log`,
  `run.state.json`, and `screenshots/` with stable filenames.
- Crash-safe resume: `run.state.json` is written after each phase's
  postcondition; `resume` skips completed phases.
- Headless + visible browser modes; default visible so screenshots are
  useful for debugging.
- Defensive reliability: typed exceptions, 3× retry with backoff on
  idempotent operations, timeouts per phase, screenshots on failure,
  Keychain-failure guidance.
- Test suite: `pytest` + `pytest-asyncio` + `respx` (httpx mock) +
  `pytest-playwright` for the small browser surface. Non-browser tests
  run on every PR; Playwright tests gated to `main`.

**Out of scope (deferred):**

- **835 waiting within a single run.** CO Medicaid 835 remittances land
  the following Monday, on the payment cycle. A run that blocks for up
  to 7 days is impractical. The agent defers 835 detection to a separate
  `check-835 <run-id>` subcommand that the operator runs on/after
  Monday. The report always carries the `check-835` recipe.
- **277CA polling.** CO Medicaid does not use 277CA claim-level ACKs;
  the agent never polls for them. (The `GET /api/277ca-acks` endpoint
  exists in the API surface but is not exercised by this agent.)
- **Watch-folder / daemon mode.** The agent is on-demand. A cron job that
  invokes `cyclone-pipeline run-batch /inbox/*.txt --strict` on a 5-min
  schedule achieves the same effect with no daemon in the loop.
- **Multi-payer support.** The agent targets CO Medicaid (the only
  configured payer in `config/payers.yaml`). A new payer means new
  payer-specific regex / file-naming / SFTP block — out of scope for v1.
- **Patient peek or other drill-down features.** This is a backend
  automation; no UI changes.
- **Editing the audit log or any existing cyclone data.** The agent
  only reads + triggers; it never writes to cyclone's DB except through
  the published APIs.

## 2. Locked decisions

### 2.1 Architecture

API-first + minimal Playwright shell. The browser is used only for the
upload page (drop zone, Parse button, progress bar). Everything else
(verification, SFTP submit, scheduler control, ACK polling, claim-state
checks) hits the FastAPI directly via `httpx.AsyncClient`.

Rationale: the upload page is the only surface in cyclone with a real
interactive flow. Clearinghouse submission is API-only; the MFT
scheduler is API-only; ACK retrieval is API-only. A pure-Playwright
approach would be 3–5x more code and considerably flakier on the
API-driven phases, with no benefit.

### 2.2 Sibling package, not an extra

The agent lives at `/Users/openclaw/dev/cyclone-pipeline/` with its own
`pyproject.toml`, not as a new optional extra on the existing cyclone
package. Rationale: the agent has Playwright as a hard dependency; we
do not want Playwright in cyclone's dev env for developers who never
invoke the agent. Separate package = clean separation of concerns.

### 2.3 Real round-trip, but only the parts that are time-bounded

The agent waits for **TA1 (envelope ACK)** and **999 (file-level ACK)**.
It does **not** wait for 277CA (CO Medicaid doesn't use it) or 835
(payment cycle is next Monday). 835 detection is a separate, fast
follow-up command that the operator runs on Monday.

Rationale: blocking a run for up to 7 days is impractical and ties up
operator attention. The TA1 + 999 receipt is the unambiguous "your file
was accepted by the payer at the file level" signal; the 835 is
eventually-consistent and doesn't need a tight loop.

### 2.4 CO Medicaid / Gainwell specifics

- **SFTP server:** `mft.gainwelltechnologies.com:22`
- **Inbound path:** `/CO XIX/PROD/coxix_prod_11525703/FromHPE`
- **Outbound file naming:** `11525703-837P-{yyyymmddhhmmssSSS}-1of1.txt`
  (17-digit Mountain Time timestamp; matches the `clearhouse` config's
  `outbound` template in `config/payers.yaml`).
- **Inbound ACKs (in observed order):**
  1. TA1 (seconds to minutes) — envelope-level
  2. 999 (minutes) — file-level
  3. HTML (sometimes) — non-X12 artifact, lands in inbound, not parsed
  4. 835 (next Monday) — remit
  5. 277CA — *never used by CO Medicaid*

### 2.5 Failure semantics

- **Upload retries:** 3× attempts; each attempt is a fresh page load +
  file re-attach + click. Screenshots on every attempt.
- **SFTP submit retries:** 3× attempts with `1s → 2s → 4s` backoff. The
  submit endpoint is POST and is *not* auto-retried after a network
  timeout (would double-submit); the retry covers `409 Conflict` /
  `503 Service Unavailable` only.
- **TA1 timeout (5 min):** if no TA1 arrives, the run is **soft-fail**.
  The submission is "in flight" — the report says so and recommends
  checking Gainwell directly.
- **999 timeout (15 min):** same — soft-fail. Most production runs
  complete this phase in < 2 min.
- **999 received with status `R` (rejected):** hard-fail. The 999's
  rejection segments surface in the report with full context.
- **HTML detected in inbound:** warn in the report; do not fail.
- **835 not present at end of run:** expected; reported as
  "deferred — expected next Monday" with the `check-835` recipe.
- **Process crash (SIGKILL, power loss, etc.):** the next `resume` skips
  completed phases by reading `run.state.json`. No work is lost except
  the in-flight phase.

### 2.6 Idempotency

- The MFT outbound filename embeds a millisecond Mountain Time
  timestamp, so back-to-back submissions to the same input file are
  unique on the SFTP server. No `Idempotency-Key` header is needed.
- The agent captures the new `claim_id[]` set from phase 3 (verify
  parse) and refuses to SFTP-submit if any of those claim IDs already
  appear in a prior `processed_inbound_files` row within a configurable
  dedup window (default 24h). `--force` overrides the dedup.
- The `check-835` command is fully idempotent: querying
  `GET /api/remittances?since=<date>` is read-only.

### 2.7 Output shape

Every run writes a dated folder under `./runs/`. The folder is
self-contained — anyone reading it has the full picture without needing
the live cyclone instance.

```
runs/2026-06-21-1430-001/
├── run.log                    # structured JSON, structlog
├── run.state.json             # for resume
├── report.md                  # human-readable summary
├── report.json                # machine-readable, OpenClaw/Nora-friendly
└── screenshots/
    ├── 01-preflight.png
    ├── 02-upload-1.png        # attempts are numbered
    ├── 02-upload-2.png
    ├── 03-parse-complete.png
    ├── 04-submit.png
    ├── 05-ta1.png
    ├── 06-999.png
    ├── 07-inbound-scan.png    # HTML detection screenshot
    └── 99-final-inbox.png     # final state of the Inbox page
```

On failure, the `finally` block adds `screenshots/FAIL-{phase}.png` and
a `failure.md` with the traceback + phase + remediation hint.

## 3. Module map

| File | Role |
|---|---|
| `pyproject.toml` | Project metadata; declares httpx, playwright, click, pydantic, structlog; dev deps: pytest, pytest-asyncio, respx, pytest-playwright |
| `README.md` | Install, run, embed-in-agent examples |
| `src/cyclone_pipeline/__init__.py` | Public API re-exports |
| `src/cyclone_pipeline/__main__.py` | `python -m cyclone_pipeline run …` |
| `src/cyclone_pipeline/cli.py` | Click CLI: `run`, `run-batch`, `status`, `resume`, `check-835` |
| `src/cyclone_pipeline/pipeline.py` | `CyclonePipeline` orchestrator; 7-phase state machine |
| `src/cyclone_pipeline/api_client.py` | `httpx.AsyncClient` wrapper with typed methods |
| `src/cyclone_pipeline/browser.py` | Playwright `UploadPage` class for the upload UI |
| `src/cyclone_pipeline/waiters.py` | Reusable wait primitives (claim state, ACK arrival) |
| `src/cyclone_pipeline/state.py` | `RunState` dataclass + `run.state.json` writer/reader |
| `src/cyclone_pipeline/screenshots.py` | Dated folder + stable filename helper |
| `src/cyclone_pipeline/report.py` | `report.md` + `report.json` writer |
| `src/cyclone_pipeline/selectors.py` | Centralized Playwright selectors with fallbacks |
| `src/cyclone_pipeline/exceptions.py` | Typed errors: `UploadError`, `ParseError`, `SubmitError`, `AckTimeoutError`, `SchedulerNotRunningError`, `IdempotencyError` |
| `src/cyclone_pipeline/logging_setup.py` | `structlog` config matching cyclone's SP18 JSON style |
| `src/cyclone_pipeline/check_835.py` | The deferred 835 detector (pure API, no browser) |
| `tests/conftest.py` | Shared fixtures: fake API server, mock browser, fake clock |
| `tests/test_api_client.py` | Typed responses, retry behavior, timeouts (respx) |
| `tests/test_waiters.py` | Backoff, timeout, predicate resolution (fake clock) |
| `tests/test_browser.py` | safe_click, attach_file, progress wait (pytest-playwright) |
| `tests/test_pipeline.py` | End-to-end orchestration, state machine, resume (mocks) |
| `tests/test_report.py` | Markdown + JSON shape, screenshot paths (golden files) |
| `tests/test_cli.py` | `run`, `run-batch`, `resume`, `check-835` (Click's CliRunner) |
| `tests/test_check_835.py` | `check-835` happy path + "835 not yet present" path |
| `tests/fixtures/sample_837p.edi` | A small valid 837P file for tests (copied from cyclone's `tests/fixtures/minimal_837p.txt`) |

## 4. Pipeline phases

The state machine in `pipeline.py` advances through 7 phases. Each phase
has: `precondition`, `execute`, `postcondition`, `retry_policy`, and a
side-effect of writing `run.state.json` + a screenshot at the end of a
successful attempt.

### Phase 1 — Pre-flight (API)

- `GET /api/health` → expect `status == "ok"` and `db.ok == true`.
- `GET /api/admin/scheduler/status` → record scheduler state
  (`running` / `stopped`). If `stopped`, the agent will call
  `POST /api/admin/scheduler/tick` per-iteration in phase 5/6 instead
  of relying on the polling loop.
- Verify the browser-side URL is reachable: `HEAD {browser_base}/`
  via `httpx` (cheap, no rendering). If the frontend dev server isn't
  up, fail-fast with `BrowserNotReachableError` rather than discovering
  this in phase 2.
- Capture the API base URL + browser base URL for the report.
- Screenshot the Upload page in its empty state →
  `screenshots/01-preflight.png`.

### Phase 2 — Upload (Playwright)

- `chromium.launch(headless=VISIBLE)` (configurable).
- `page.goto(f"{browser_base}/upload")`.
- `expect(drop_zone).to_be_visible(timeout=10s)`.
- `set_input_files("input[type=file]", file_path)` (the hidden input
  behind the drop zone; `set_input_files` works on hidden inputs).
- `expect(file_name_display).to_have_text(file.name)`.
- `safe_click("button:has-text('Parse')")`.
- `wait_for_progress_complete()` — polls the progress bar
  (`text_content` containing `"100%"`) or the success toast
  (`text_content` containing `"Parsed N claims"`), whichever comes
  first, with a 5 min ceiling.
- Screenshots: `screenshots/02-upload-{attempt}.png` per attempt.

### Phase 3 — Verify parse (API)

- The browser-side toast gives the parsed claim count; capture it.
- `GET /api/claims?since=<phase1_start_ts>` → filter to claims whose
  `submission_date >= phase1_start_ts`; these are the new ones.
- Collect the set of new `claim_id` values.
- If 0 new claims after a 30s poll, fail with `ParseError`.

### Phase 4 — SFTP submit (API)

- Dedup check: query `GET /api/admin/scheduler/processed-files` for
  the past 24h; if any outbound filename pattern matches a recent
  submission, fail with `IdempotencyError` unless `--force`.
- `POST /api/clearhouse/submit` with `{payer_id: "CO_TXIX",
  transaction_type: "837P"}`. The body is small; the server picks
  the batch from the newly-parsed claims.
- Expect 200; if 5xx with a "keychain" hint, raise `SubmitError` with
  the macOS Keychain setup recipe inline.
- Screenshot: `screenshots/04-submit.png` (the Inbox page right after
  submit, showing the batch in the lane).

### Phase 5 — Wait for TA1 (API poll)

- `GET /api/ta1-acks?since=<phase4_ts>` in a loop, every 5s, with a
  5 min ceiling.
- Match by `isa13` (the control number in the submitted interchange
  envelope; surfaced in the TA1 record).
- If scheduler is `stopped`, call `POST /api/admin/scheduler/tick`
  between polls to manually drive the MFT fetch.
- On hit: capture `ta1_id`, `accepted` (vs `rejected`), screenshot the
  TA1 detail in the Acks page → `screenshots/05-ta1.png`.
- On timeout: soft-fail; record `ta1_status: "timeout"`.

### Phase 6 — Wait for 999 (API poll)

- `GET /api/acks?since=<phase4_ts>` in a loop, every 10s, with a
  15 min ceiling.
- Match by `original_batch_id` (or, if not available, by
  `functional_group_control_number` from the submitted file's GS06).
- On hit: capture `ack_id`, `status` (`A` = accepted, `R` = rejected,
  `E` = rejected with errors), the rejection segment details if
  rejected.
- On `A`: screenshot the Acks page → `screenshots/06-999.png`.
- On `R`: hard-fail; record `ack.status == "rejected"`, the rejection
  segments, and the recommended remediation in the report.
- On timeout: soft-fail; record `ack_999_status: "timeout"`.

### Phase 7 — Scan for HTML + final report (API + local)

- `GET /api/admin/scheduler/processed-files?since=<phase4_ts>` — list
  inbound files the MFT scheduler saw during the run. If any has
  `file_type == "html"` or an unrecognized extension, log a WARN,
  capture the filename, and add it to the report's "Detected artifacts"
  section.
- Generate `report.md` and `report.json` (see §5).
- Screenshot the Inbox page → `screenshots/99-final-inbox.png`.
- Emit a one-line summary on stdout for shell pipelines:
  `RESULT: PASS  run=2026-06-21-1430-001  ta1=TA1-42  999=999-43  835=deferred`.

## 5. Output format

### 5.1 `report.md` (human)

```markdown
# Cyclone pipeline run — 2026-06-21 14:30 PDT

| Field         | Value                              |
|---------------|------------------------------------|
| Run id        | 2026-06-21-1430-001                |
| Input         | /path/to/axiscare-837p.txt (12.4 KB) |
| Result        | PASS — TA1 + 999 received, 835 deferred |
| Duration      | 2m 14s                             |

## Phase summary

| Phase                  | Status | Duration | Detail                                  |
|------------------------|--------|----------|-----------------------------------------|
| 1. Pre-flight          | OK     | 0.4s     | health.ok, scheduler running            |
| 2. Upload              | OK     | 2.1s     | 1 attempt, 17 claims parsed             |
| 3. Verify parse        | OK     | 0.8s     | claim_ids: CLM-001 … CLM-017            |
| 4. SFTP submit         | OK     | 4.3s     | filename 11525703-837P-20260621143012345-1of1.txt |
| 5. Wait TA1            | OK     | 12s      | ack id: TA1-42, accepted                |
| 6. Wait 999            | OK     | 1m 14s   | ack id: 999-43, status Accepted         |
| 7. Scan HTML + report  | WARN   | 0.2s     | 1 HTML file in inbound, skipped; report.md + report.json written |

## Inbound artifacts detected

- 1 HTML file (`gainwell_status_20260621.htm`) — not parsed, not a failure.

## Expected next steps

- **835 expected Monday** (2026-06-28; computed as next Monday in
  Mountain Time at the moment of report generation; if today is
  Monday before noon, expected same day, otherwise next Monday).
  Re-run: `cyclone-pipeline check-835 2026-06-21-1430-001`

## Artifacts

- screenshots/01-preflight.png
- screenshots/02-upload-1.png
- screenshots/03-parse-complete.png
- screenshots/04-submit.png
- screenshots/05-ta1.png
- screenshots/06-999.png
- screenshots/99-final-inbox.png
```

### 5.2 `report.json` (machine)

```json
{
  "run_id": "2026-06-21-1430-001",
  "input": {
    "path": "/path/to/axiscare-837p.txt",
    "size_bytes": 12700,
    "sha256": "…"
  },
  "result": "pass",
  "result_detail": "TA1 + 999 received, 835 deferred",
  "started_at": "2026-06-21T14:30:00-06:00",
  "finished_at": "2026-06-21T14:32:14-06:00",
  "duration_s": 134,
  "phases": [
    {"n": 1, "name": "preflight", "status": "ok", "duration_s": 0.4},
    {"n": 2, "name": "upload",    "status": "ok", "duration_s": 2.1, "attempts": 1},
    {"n": 3, "name": "verify_parse", "status": "ok", "duration_s": 0.8, "claim_ids": ["CLM-001", "…"]},
    {"n": 4, "name": "submit",    "status": "ok", "duration_s": 4.3, "filename": "11525703-837P-…"},
    {"n": 5, "name": "wait_ta1",  "status": "ok", "duration_s": 12, "ta1_id": "TA1-42", "accepted": true},
    {"n": 6, "name": "wait_999",  "status": "ok", "duration_s": 74, "ack_999_id": "999-43", "status": "A"},
    {"n": 7, "name": "scan_and_report", "status": "warn", "duration_s": 0.2, "html_files": ["gainwell_status_20260621.htm"], "report_paths": {"md": "report.md", "json": "report.json"}}
  ],
  "artifacts": {
    "screenshots": ["01-preflight.png", "…", "99-final-inbox.png"],
    "report_md": "report.md",
    "report_json": "report.json",
    "run_log": "run.log"
  },
  "next_steps": {
    "check_835": "cyclone-pipeline check-835 2026-06-21-1430-001",
    "expected_835_by": "2026-06-28"
  }
}
```

On soft-fail (TA1 or 999 timeout) or hard-fail (999 rejected), the
`result` is `"soft_fail"` or `"hard_fail"`, the offending phase gets
`status: "timeout"` / `"rejected"`, and a `failure` block is added
with the traceback, recommended remediation, and the `failure.md` path.

### 5.3 Exit codes

| Code | Meaning |
|------|---------|
| 0    | PASS — TA1 + 999 both received, both accepted |
| 2    | SOFT_FAIL — TA1 or 999 timed out; submission in flight, manual check needed |
| 3    | HARD_FAIL — 999 rejected, upload failed irrecoverably, or submit refused |
| 4    | USAGE — bad CLI args, missing file, backend unreachable after retries |

This mirrors common CI conventions so OpenClaw / Nora / shell scripts
can branch on `$?` without parsing JSON.

## 6. Library API (for OpenClaw / Nora)

```python
from cyclone_pipeline import CyclonePipeline, RunResult

async def main():
    pipeline = CyclonePipeline(
        api_base="http://127.0.0.1:8000",
        browser_base="http://127.0.0.1:5173",
        run_dir="./runs",
        headless=False,
        timeouts={"ta1_s": 300, "ack_999_s": 900},
    )

    result: RunResult = await pipeline.run(file_path="axiscare-837p.txt")
    # result.outcome: "pass" | "soft_fail" | "hard_fail" | "usage"
    # result.exit_code: int (0/2/3/4; mirrors CLI exit codes in §5.3)
    # result.report: RunReport (Pydantic model; serializes to report.json)
    # result.run_id: str

    # Or batch:
    results = await pipeline.run_batch(file_paths=[...], strict=False)

    # Or resume:
    result = await pipeline.resume(run_id="2026-06-21-1430-001")

    # Or check 835 later:
    ack = await pipeline.check_835(run_id="2026-06-21-1430-001")
    # ack.found, ack.remittance_id, ack.total_paid, ...
```

`RunResult` is a Pydantic model so OpenClaw / Nora can serialize it
straight to JSON without any extra glue.

## 7. Tests

| Test file | Strategy |
|---|---|
| `tests/test_api_client.py` | `respx` mock; verify typed responses, retry-on-idempotent, no-retry-on-POST, timeout behavior |
| `tests/test_waiters.py` | Inject fake clock + fake state; verify backoff math, timeout, predicate resolution |
| `tests/test_browser.py` | `pytest-playwright` against a Vite dev server spun up in CI; covers `safe_click`, `set_input_files`, progress wait |
| `tests/test_pipeline.py` | Mock both `api_client` and `browser`; drive fake "happy path" through every phase + "phase 5 timeout" + "phase 3 parse error" + "phase 6 reject" |
| `tests/test_report.py` | Golden-file comparison for `report.md` and `report.json` |
| `tests/test_cli.py` | Click's `CliRunner`; covers all 5 subcommands + error paths |
| `tests/test_check_835.py` | Mock `GET /api/remittances?since=…`; happy path + "835 not yet present" path |

CI: GitHub Actions matrix on Python 3.11 + 3.12. Non-Playwright tests
run on every PR; Playwright tests gated to `main` push (the Vite
dev-server + chromium setup is too slow for every PR).

Coverage target: 90% line coverage on `pipeline.py`, 85% on the rest.
`browser.py` is exempt from the strict target (Playwright's own
assertion failures are the real test).

## 8. Files to add

- `pyproject.toml` — package metadata, deps, dev deps, scripts entry
- `README.md` — install, run, embed-in-agent examples
- 14 source files under `src/cyclone_pipeline/` listed in §3
- 8 test files under `tests/` listed in §3
- `tests/fixtures/sample_837p.edi` — small valid 837P for tests
  (copied from cyclone's `tests/fixtures/minimal_837p.txt`)

Total: 24 files (1 `pyproject.toml`, 1 `README.md`, 14 src `.py`, 8
test `.py`, 1 fixture).

## 9. Files NOT touched

- No changes to the cyclone repo. The agent is a strict HTTP consumer
  of cyclone's published API.
- No changes to cyclone's DB schema, API, CLI, or UI.
- No new dependencies added to cyclone's `pyproject.toml`.

## 10. Future work (explicit non-goals for v1)

- **Watch-folder daemon** — `cyclone-pipeline watch /inbox/*.txt` that
  runs continuously and submits files as they appear. Achievable with
  the current `run-batch` + a 1-line shell loop; the daemon is just UX.
- **Multi-payer support** — adding a new payer (e.g., a second
  Medicaid MCO) means new payer-specific SFTP block, file naming
  template, and ID-matching strategy. Worth doing once a second payer
  is on the books.
- **Slack / email notification on completion** — `report.json` is the
  integration point; a 50-LOC notifier could post a summary. v1 just
  prints the one-line summary and writes the file.
- **Pushing reports to S3 / a share** — same as above; `report.json`
  is the integration point.
- **Web UI for run history** — out of scope; OpenClaw / Nora can read
  the `runs/` directory and surface the JSON.
- **277CA polling for payers that use it** — straightforward to add
  behind a `--payer-config` flag once a second payer is configured.
  277CA support in cyclone is already shipped; only the agent's wait
  list changes.

## 11. Open questions (none for v1)

None. The design is locked pending user review of this spec.