docs(sp25): RUNBOOK.md — enable SFTP polling for real Gainwell MFT

docs/RUNBOOK.md

@@ -0,0 +1,113 @@
+# Cyclone Operator Runbook
+
+Procedures for running Cyclone in production. The end-user README
+covers dev setup; this doc is for an operator bringing up SFTP polling
+on a fresh host.
+
+## SP25 — Enable SFTP Polling for Real Gainwell MFT
+
+The inbound MFT polling scheduler ships in SP16 and is wired to real
+paramiko SFTP in SP13. To turn it on for `mft.gainwelltechnologies.com`:
+
+### Prerequisites
+
+- Python 3.11+ with the `cyclone` package installed (`pip install -e .[dev,sftp]`).
+- The macOS `keyring` library is optional. On a Linux server or Docker
+  container, the MFT password is supplied via a plain env var (no
+  Keychain dependency).
+- Outbound TCP/22 to `mft.gainwelltechnologies.com`.
+
+### Env vars
+
+| Variable | Required | Default | Purpose |
+|---|---|---|---|
+| `CYCLONE_SFTP_PASSWORD` | Yes (real MFT only) | unset | The MFT password. Stripped of whitespace; empty values are treated as unset. |
+| `CYCLONE_SCHEDULER_AUTOSTART` | No | unset (falsy) | When `1`/`true`/`yes`, the scheduler starts polling on API launch. |
+| `CYCLONE_SCHEDULER_POLL_SECONDS` | No | `60` | Seconds between poll cycles. The Gainwell MFT server doesn't push — we pull. |
+
+### First-time setup
+
+1. **Set the password env var.** On the server that runs Cyclone:
+
+   ```bash
+   export CYCLONE_SFTP_PASSWORD='the-actual-password'
+   ```
+
+   For systemd / Docker, persist this in the service file or
+   `docker-compose.yml` (use a `_FILE` companion or a `secrets:`
+   block; see your platform's docs).
+
+2. **Confirm the clearhouse SFTP block is configured.** `GET /api/clearhouse` should return a row. If not, run the lifespan once to seed it (any API launch will do).
+
+3. **Flip stub → false and point at real MFT.** Authenticated as an admin:
+
+   ```bash
+   # GET the current row, modify the sftp_block, PATCH it back.
+   # The endpoint requires a full Clearhouse body, so always
+   # round-trip through GET to get the right updated_at + filename_block.
+   curl -s http://127.0.0.1:8000/api/clearhouse -b cookies.txt > /tmp/ch.json
+   # Edit /tmp/ch.json — set sftp_block.stub to false and adjust host/port/paths/auth.
+   # The minimum change for real-MFT mode is:
+   #   jq '.sftp_block.stub = false
+   #       | .sftp_block.host = "mft.gainwelltechnologies.com"
+   #       | .sftp_block.auth = {"password_keychain_account": "sftp.gainwell.password"}' \
+   #     /tmp/ch.json > /tmp/ch-patched.json
+   curl -X PATCH http://127.0.0.1:8000/api/clearhouse \
+     -H 'Content-Type: application/json' \
+     -b cookies.txt \
+     --data @/tmp/ch-patched.json
+   ```
+
+   The endpoint hot-reloads the scheduler. No API restart required.
+
+4. **Start polling.** Either:
+
+   - Set `CYCLONE_SCHEDULER_AUTOSTART=true` and restart the API.
+   - Or trigger manually:
+
+     ```bash
+     curl -X POST http://127.0.0.1:8000/api/admin/scheduler/start -b cookies.txt
+     ```
+
+### Verification
+
+```bash
+# Is the loop running?
+curl http://127.0.0.1:8000/api/admin/scheduler/status -b cookies.txt
+
+# Force one poll cycle right now:
+curl -X POST http://127.0.0.1:8000/api/admin/scheduler/tick -b cookies.txt
+
+# What files has the scheduler seen?
+curl 'http://127.0.0.1:8000/api/admin/scheduler/processed-files?limit=20' -b cookies.txt
+
+# Only the errors?
+curl 'http://127.0.0.1:8000/api/admin/scheduler/processed-files?status=error' -b cookies.txt
+```
+
+A successful first poll shows rows in `processed-files` with
+`status=ok` and a non-zero `claim_count` for 835/277CA files.
+
+### Troubleshooting
+
+| Symptom | Likely cause | Fix |
+|---|---|---|
+| `status=error` rows with `AuthenticationException` | Wrong password in `CYCLONE_SFTP_PASSWORD` | Re-export with the correct value, then PATCH /api/clearhouse or restart. |
+| `status=error` rows with `IOError: [Errno 111] Connection refused` | Outbound TCP/22 blocked, or wrong host/port | Check the firewall; confirm `host` in `GET /api/clearhouse`. |
+| Zero rows in `processed-files` after a tick | Inbound MFT dir is empty (HPE hasn't pushed yet) — not an error | Wait. Trigger `tick` again later. |
+| `RuntimeError: SFTP: Keychain entry ... missing or stub` | `get_secret()` fell through to Keychain (Linux + missing env var) | Set `CYCLONE_SFTP_PASSWORD`. |
+| `RuntimeError: SftpBlock.auth must contain ...` | PATCH set `stub=false` without an `auth` block | PATCH again with a complete `auth` dict. |
+| Scheduler never starts (`running: false`) | `CYCLONE_SCHEDULER_AUTOSTART` not set, no manual `start` call | Either set autostart or `POST /api/admin/scheduler/start`. |
+
+### macOS dev box variant
+
+If you prefer the Keychain over env vars on macOS:
+
+```bash
+security add-generic-password -s cyclone -a sftp.gainwell.password -w '<password>'
+security find-generic-password -s cyclone -a sftp.gainwell.password -w   # verify
+```
+
+The env var is the highest-priority lookup; the Keychain is the
+fallback. Setting both means the env var wins. To force the Keychain,
+unset the env var for that shell.