Cyclone Operator Runbook

Procedures for running Cyclone in production. The end-user README covers dev setup; this doc is for an operator bringing up SFTP polling on a fresh host.

SP25 — Enable SFTP Polling for Real Gainwell MFT

The inbound MFT polling scheduler ships in SP16 and is wired to real paramiko SFTP in SP13. To turn it on for mft.gainwelltechnologies.com:

Prerequisites

Python 3.11+ with the cyclone package installed (pip install -e .[dev,sftp]).
The macOS keyring library is optional. On a Linux server or Docker container, the MFT password is supplied via a plain env var (no Keychain dependency).
Outbound TCP/22 to mft.gainwelltechnologies.com.

Env vars

Variable	Required	Default	Purpose
`CYCLONE_SFTP_PASSWORD`	Yes (real MFT only)	unset	The MFT password. Stripped of whitespace; empty values are treated as unset.
`CYCLONE_SFTP_PASSWORD_FILE`	No	unset	Path to a file containing the MFT password. Highest-priority lookup in `secrets.get_secret()`. Standard Docker-secrets pattern. See "Docker secrets variant" below.
`CYCLONE_SCHEDULER_AUTOSTART`	No	unset (falsy)	When `1`/`true`/`yes`, the scheduler starts polling on API launch.
`CYCLONE_SCHEDULER_POLL_SECONDS`	No	`60`	Seconds between poll cycles. The Gainwell MFT server doesn't push — we pull.

First-time setup

Set the password env var. On the server that runs Cyclone:
```
export CYCLONE_SFTP_PASSWORD='the-actual-password'
```
For systemd / Docker, persist this in the service file or docker-compose.yml (use a _FILE companion or a secrets: block; see your platform's docs).
Confirm the clearhouse SFTP block is configured. GET /api/clearhouse should return a row. If not, run the lifespan once to seed it (any API launch will do).

Flip stub → false and point at real MFT. Authenticated as an admin:

# GET the current row, modify the sftp_block, PATCH it back.
# The endpoint requires a full Clearhouse body, so always
# round-trip through GET to get the right updated_at + filename_block.
curl -s http://127.0.0.1:8000/api/clearhouse -b cookies.txt > /tmp/ch.json
# Edit /tmp/ch.json — set sftp_block.stub to false and adjust host/port/paths/auth.
# The minimum change for real-MFT mode is:
#   jq '.sftp_block.stub = false
#       | .sftp_block.host = "mft.gainwelltechnologies.com"
#       | .sftp_block.auth = {"password_keychain_account": "sftp.gainwell.password"}' \
#     /tmp/ch.json > /tmp/ch-patched.json
curl -X PATCH http://127.0.0.1:8000/api/clearhouse \
  -H 'Content-Type: application/json' \
  -b cookies.txt \
  --data @/tmp/ch-patched.json

The endpoint hot-reloads the scheduler. No API restart required.

Start polling. Either:
- Set CYCLONE_SCHEDULER_AUTOSTART=true and restart the API.
- Or trigger manually:
```
curl -X POST http://127.0.0.1:8000/api/admin/scheduler/start -b cookies.txt
```

Verification

# Is the loop running?
curl http://127.0.0.1:8000/api/admin/scheduler/status -b cookies.txt

# Force one poll cycle right now:
curl -X POST http://127.0.0.1:8000/api/admin/scheduler/tick -b cookies.txt

# What files has the scheduler seen?
curl 'http://127.0.0.1:8000/api/admin/scheduler/processed-files?limit=20' -b cookies.txt

# Only the errors?
curl 'http://127.0.0.1:8000/api/admin/scheduler/processed-files?status=error' -b cookies.txt

A successful first poll shows rows in processed-files with status=ok and a non-zero claim_count for 835/277CA files.

Troubleshooting

Symptom	Likely cause	Fix
`status=error` rows with `AuthenticationException`	Wrong password in `CYCLONE_SFTP_PASSWORD`	Re-export with the correct value, then PATCH /api/clearhouse or restart.
`status=error` rows with `IOError: [Errno 111] Connection refused`	Outbound TCP/22 blocked, or wrong host/port	Check the firewall; confirm `host` in `GET /api/clearhouse`.
Zero rows in `processed-files` after a tick	Inbound MFT dir is empty (HPE hasn't pushed yet) — not an error	Wait. Trigger `tick` again later.
`RuntimeError: SFTP: Keychain entry ... missing or stub`	`get_secret()` fell through to Keychain (Linux + missing env var)	Set `CYCLONE_SFTP_PASSWORD`.
`RuntimeError: SftpBlock.auth must contain ...`	PATCH set `stub=false` without an `auth` block	PATCH again with a complete `auth` dict.
Scheduler never starts (`running: false`)	`CYCLONE_SCHEDULER_AUTOSTART` not set, no manual `start` call	Either set autostart or `POST /api/admin/scheduler/start`.

macOS dev box variant

If you prefer the Keychain over env vars on macOS:

security add-generic-password -s cyclone -a sftp.gainwell.password -w '<password>'
security find-generic-password -s cyclone -a sftp.gainwell.password -w   # verify

The env var is the highest-priority lookup; the Keychain is the fallback. Setting both means the env var wins. To force the Keychain, unset the env var for that shell.

Docker secrets variant (SP26)

For the SP23 Docker stack, mount the MFT password as a file rather than embedding it in docker-compose.yml. The compose file already declares the cyclone_sftp_password secret and wires CYCLONE_SFTP_PASSWORD_FILE: "/run/secrets/cyclone_sftp_password" on the backend service. Create the file once on the host:

sudo install -m 0600 -o root -g root /dev/null /etc/cyclone/secrets/sftp_password
echo -n 'the-actual-password' | sudo tee /etc/cyclone/secrets/sftp_password > /dev/null
sudo chmod 0600 /etc/cyclone/secrets/sftp_password

Then docker compose up -d. The backend's secrets.get_secret() will read the file on the next scheduler tick — no env-var export, no docker-compose.yml edit with the password in it. The file takes precedence over the plain CYCLONE_SFTP_PASSWORD env var; setting both means the file wins.

If the mounted file is missing or unreadable (typo in path, container started without the secret mount), the scheduler surfaces a RuntimeError at the next tick that names the env var and the missing path — this is intentional, so a silent fall-through doesn't mask a real misconfiguration.

6.2 KiB Raw Blame History