merge: SP26 SFTP Password File Companion into main

This commit is contained in:
Nora
2026-06-24 16:11:21 -06:00
7 changed files with 934 additions and 24 deletions
+56 -23
View File
@@ -15,18 +15,27 @@ Setup (one-time, by the operator):
Verification: Verification:
security find-generic-password -s cyclone -a sftp.gainwell.password -w security find-generic-password -s cyclone -a sftp.gainwell.password -w
SP25: the lookup now resolves the secret in three tiers, in this order: SP25 + SP26: the lookup now resolves the secret in four tiers, in
this order:
1. Plain env var named exactly ``<name>`` — highest priority. 1. ``<env_name>_FILE`` env var set — highest priority. Reads the
file at that path, strips whitespace, returns the contents.
This is the standard Docker-secrets pattern: mount a file at
``/run/secrets/<name>`` and point the env var at it. An
operator who explicitly sets ``_FILE`` is making a positive
statement about where the secret lives, so a missing file
surfaces as a ``RuntimeError`` rather than silently falling
through. Empty string is treated as unset.
2. Plain env var named exactly ``<env_name>`` — second priority.
Lets a Linux server or Docker container pass the MFT password Lets a Linux server or Docker container pass the MFT password
via ``CYCLONE_SFTP_PASSWORD=...`` without touching the Keychain. via ``CYCLONE_SFTP_PASSWORD=...`` without touching the Keychain
Trailing/leading whitespace (including the ``\\n`` that .env or a file mount. Trailing/leading whitespace (including the
files often leave at EOF) is stripped; an empty value is ``\\n`` that .env files often leave at EOF) is stripped; an
treated as absent. empty value is treated as absent.
2. macOS Keychain via ``keyring`` — unchanged from SP9. Kept as a 3. macOS Keychain via ``keyring`` — third priority. Kept as a
fallback so a macOS workstation that prefers fallback so a macOS workstation that prefers
``security add-generic-password`` works without env-var exports. ``security add-generic-password`` works without env-var exports.
3. ``None`` — caller decides what to do (``SftpClient._connect`` 4. ``None`` — caller decides what to do (``SftpClient._connect``
raises a precise ``RuntimeError`` on real-mode auth). raises a precise ``RuntimeError`` on real-mode auth).
""" """
@@ -34,6 +43,7 @@ from __future__ import annotations
import logging import logging
import os import os
from pathlib import Path
from typing import Optional from typing import Optional
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
@@ -51,29 +61,52 @@ except ImportError:
_HAS_KEYRING = False _HAS_KEYRING = False
def _env_var_for(name: str) -> str:
"""Resolve the operator-facing env-var name for a secret.
Returns the entry from ``_ENV_NAME_FOR`` if present, otherwise
returns ``name`` verbatim.
"""
return _ENV_NAME_FOR.get(name, name)
def get_secret(name: str) -> Optional[str]: def get_secret(name: str) -> Optional[str]:
"""Fetch a secret by name. Three-tier lookup: env var, Keychain, None. """Fetch a secret by name. Four-tier lookup: _FILE, env var, Keychain, None.
Args: Args:
name: The secret name. Matches both the env-var name and the name: The secret name. Matches the Keychain account name
Keychain account (e.g. ``"sftp.gainwell.password"`` / (e.g. ``"sftp.gainwell.password"``). The operator-facing
``CYCLONE_SFTP_PASSWORD`` would NOT match — the env var env-var form is mapped via the ``_ENV_NAME_FOR`` table at
name is the bare secret name, not the the bottom of this module.
``CYCLONE_<UPPER_SNAKE_CASE>`` form). Operators who want
the env-var shortcut set the var named exactly ``<name>``.
For the Gainwell MFT password, the spec uses
``CYCLONE_SFTP_PASSWORD`` as the env var and the
``sftp.gainwell.password`` Keychain account as the
fallback. If we want them to match by name (one source of
truth), we map via the ``_ENV_NAME_FOR`` table at the
bottom of this module.
Returns: Returns:
The secret string (stripped), or ``None`` if no tier produced The secret string (stripped), or ``None`` if no tier produced
a value. a value.
Raises:
RuntimeError: if ``<env_name>_FILE`` is set but the file at
that path is missing or unreadable. The operator made a
positive statement about where the secret lives; silent
fall-through would mask a misconfiguration.
""" """
env_name = _ENV_NAME_FOR.get(name, name) env_name = _env_var_for(name)
file_env = env_name + "_FILE"
file_path_raw = os.environ.get(file_env)
if file_path_raw:
# An empty string is treated as unset (matches the SP25 rule
# for plain env vars).
stripped_path = file_path_raw.strip()
if stripped_path:
try:
value = Path(stripped_path).read_text().strip()
except OSError as exc:
raise RuntimeError(
f"failed to read {file_env}={stripped_path}: {exc}"
) from exc
if value:
log.debug("Secret %r resolved from file %r", name, stripped_path)
return value
raw = os.environ.get(env_name) raw = os.environ.get(env_name)
if raw is not None: if raw is not None:
stripped = raw.strip() stripped = raw.strip()
+24 -1
View File
@@ -117,8 +117,15 @@ def test_compose_declares_required_services():
def test_compose_declares_required_secrets_and_volumes(): def test_compose_declares_required_secrets_and_volumes():
compose = yaml.safe_load(COMPOSE_FILE.read_text()) compose = yaml.safe_load(COMPOSE_FILE.read_text())
secrets = compose.get("secrets", {}) secrets = compose.get("secrets", {})
for required in ("cyclone_db_key", "cyclone_admin_password"): for required in ("cyclone_db_key", "cyclone_admin_password", "cyclone_sftp_password"):
assert required in secrets, f"compose must declare secret {required!r}" assert required in secrets, f"compose must declare secret {required!r}"
# SP26: the SFTP secret must point at the same /etc/cyclone/secrets/ tree
# the host operator manages. Catch typos in the file path here so a
# rename breaks the test, not a production deploy.
assert (
secrets["cyclone_sftp_password"]["file"]
== "/etc/cyclone/secrets/sftp_password"
)
volumes = compose.get("volumes", {}) volumes = compose.get("volumes", {})
for required in ( for required in (
"cyclone_db", "cyclone_db",
@@ -142,6 +149,22 @@ def test_compose_backend_wires_backup_autostart():
assert "CYCLONE_BACKUP_RETENTION_DAYS" in env assert "CYCLONE_BACKUP_RETENTION_DAYS" in env
def test_compose_backend_wires_sftp_password_file_env_var():
"""SP26 — the backend must wire CYCLONE_SFTP_PASSWORD_FILE to the
mounted Docker secret so the MFT password resolves from
/run/secrets/cyclone_sftp_password without an env-var export."""
compose = yaml.safe_load(COMPOSE_FILE.read_text())
backend = compose["services"]["backend"]
env = backend.get("environment", {})
assert env.get("CYCLONE_SFTP_PASSWORD_FILE") == "/run/secrets/cyclone_sftp_password", (
"backend must wire CYCLONE_SFTP_PASSWORD_FILE to the Docker secret mount"
)
backend_secrets = backend.get("secrets", [])
assert any(
_volume_source(s) == "cyclone_sftp_password" for s in backend_secrets
), "backend must reference the cyclone_sftp_password Docker secret"
@pytest.mark.skipif( @pytest.mark.skipif(
not _has_docker(), reason="docker not on PATH; skipping Dockerfile parse check" not _has_docker(), reason="docker not on PATH; skipping Dockerfile parse check"
) )
+128
View File
@@ -0,0 +1,128 @@
"""SP26 — Docker-secrets file fallback in cyclone.secrets.get_secret().
SP25 added a plain env-var tier ahead of the macOS Keychain lookup.
SP26 adds a further tier above that: an `<env_name>_FILE` env var
pointing at a file on disk — the standard Docker-secrets pattern.
The file takes precedence over the plain env var (matches the
``auth/bootstrap.py:_read_secret`` convention).
"""
from __future__ import annotations
import importlib
from pathlib import Path
import pytest
# A secret name we use throughout the unmapped cases. Because it is not
# in ``_ENV_NAME_FOR``, ``get_secret()`` will read it as
# ``os.environ.get("cyclone.test.file.password")`` directly (and the
# ``_FILE`` companion is the same name with ``_FILE`` appended). This
# mirrors SP25's ``test_secrets_envvar.py`` convention.
_UNMAPPED_NAME = "cyclone.test.file.password"
@pytest.fixture
def secrets_module(monkeypatch):
"""Reload cyclone.secrets with a clean keyring state per test."""
import cyclone.secrets as secrets_mod
importlib.reload(secrets_mod)
yield secrets_mod
importlib.reload(secrets_mod)
def _write_secret_file(tmp_path: Path, name: str, contents: str) -> Path:
f = tmp_path / name
f.write_text(contents)
return f
def test_file_env_var_returns_file_contents(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""_FILE set, file exists → returns file contents (no Keychain lookup)."""
f = _write_secret_file(tmp_path, "pw", "from-file\n")
monkeypatch.setenv(f"{_UNMAPPED_NAME}_FILE", str(f))
monkeypatch.delenv(_UNMAPPED_NAME, raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret(_UNMAPPED_NAME) == "from-file"
def test_file_env_var_strips_trailing_newline(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""Docker secret files commonly end with \\n — strip it."""
f = _write_secret_file(tmp_path, "pw", "supersecret\n")
monkeypatch.setenv(f"{_UNMAPPED_NAME}_FILE", str(f))
monkeypatch.delenv(_UNMAPPED_NAME, raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret(_UNMAPPED_NAME) == "supersecret"
def test_file_env_var_strips_leading_and_trailing_whitespace(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
f = _write_secret_file(tmp_path, "pw", " supersecret \n")
monkeypatch.setenv(f"{_UNMAPPED_NAME}_FILE", str(f))
monkeypatch.delenv(_UNMAPPED_NAME, raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret(_UNMAPPED_NAME) == "supersecret"
def test_file_env_var_wins_over_plain_env_var(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""When both _FILE and the plain env var are set, _FILE wins."""
f = _write_secret_file(tmp_path, "pw", "from-file\n")
monkeypatch.setenv(_UNMAPPED_NAME, "from-env")
monkeypatch.setenv(f"{_UNMAPPED_NAME}_FILE", str(f))
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret(_UNMAPPED_NAME) == "from-file"
def test_file_env_var_missing_file_raises_runtime_error(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""If _FILE points at a non-existent path, raise RuntimeError."""
missing = tmp_path / "does-not-exist"
monkeypatch.setenv(f"{_UNMAPPED_NAME}_FILE", str(missing))
monkeypatch.delenv(_UNMAPPED_NAME, raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
with pytest.raises(RuntimeError, match=f"{_UNMAPPED_NAME}_FILE"):
secrets_module.get_secret(_UNMAPPED_NAME)
def test_file_env_var_empty_string_treated_as_unset(
monkeypatch, secrets_module,
) -> None:
"""An empty _FILE env var falls through to the plain env var (SP25 rule)."""
monkeypatch.setenv(f"{_UNMAPPED_NAME}_FILE", "")
monkeypatch.setenv(_UNMAPPED_NAME, "from-env")
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret(_UNMAPPED_NAME) == "from-env"
def test_gainwell_file_env_var_full_chain(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""Setting CYCLONE_SFTP_PASSWORD_FILE makes get_secret('sftp.gainwell.password')
return the file's contents — the operator-visible chain works end-to-end."""
f = _write_secret_file(tmp_path, "sftp_pw", "real-mft-password\n")
monkeypatch.setenv("CYCLONE_SFTP_PASSWORD_FILE", str(f))
monkeypatch.delenv("CYCLONE_SFTP_PASSWORD", raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret("sftp.gainwell.password") == "real-mft-password"
+5
View File
@@ -43,6 +43,8 @@ services:
# embedding the secret in the compose file. # embedding the secret in the compose file.
CYCLONE_ADMIN_USERNAME_FILE: "/run/secrets/cyclone_admin_username" CYCLONE_ADMIN_USERNAME_FILE: "/run/secrets/cyclone_admin_username"
CYCLONE_ADMIN_PASSWORD_FILE: "/run/secrets/cyclone_admin_password" CYCLONE_ADMIN_PASSWORD_FILE: "/run/secrets/cyclone_admin_password"
# SP26 — SFTP password via Docker secret (matches the admin-creds pattern).
CYCLONE_SFTP_PASSWORD_FILE: "/run/secrets/cyclone_sftp_password"
# Bind 0.0.0.0 so the frontend container on the compose bridge # Bind 0.0.0.0 so the frontend container on the compose bridge
# network can reach us. The bridge network provides isolation — # network can reach us. The bridge network provides isolation —
# only the `frontend` service is on it; the host firewall still # only the `frontend` service is on it; the host firewall still
@@ -52,6 +54,7 @@ services:
- cyclone_db_key - cyclone_db_key
- cyclone_admin_username - cyclone_admin_username
- cyclone_admin_password - cyclone_admin_password
- cyclone_sftp_password
volumes: volumes:
- cyclone_db:/var/lib/cyclone/db - cyclone_db:/var/lib/cyclone/db
- cyclone_backups:/var/lib/cyclone/backups - cyclone_backups:/var/lib/cyclone/backups
@@ -91,6 +94,8 @@ secrets:
file: /etc/cyclone/secrets/admin_username file: /etc/cyclone/secrets/admin_username
cyclone_admin_password: cyclone_admin_password:
file: /etc/cyclone/secrets/admin_pw file: /etc/cyclone/secrets/admin_pw
cyclone_sftp_password:
file: /etc/cyclone/secrets/sftp_password
volumes: volumes:
cyclone_db: cyclone_db:
+27
View File
@@ -22,6 +22,7 @@ paramiko SFTP in SP13. To turn it on for `mft.gainwelltechnologies.com`:
| Variable | Required | Default | Purpose | | Variable | Required | Default | Purpose |
|---|---|---|---| |---|---|---|---|
| `CYCLONE_SFTP_PASSWORD` | Yes (real MFT only) | unset | The MFT password. Stripped of whitespace; empty values are treated as unset. | | `CYCLONE_SFTP_PASSWORD` | Yes (real MFT only) | unset | The MFT password. Stripped of whitespace; empty values are treated as unset. |
| `CYCLONE_SFTP_PASSWORD_FILE` | No | unset | Path to a file containing the MFT password. Highest-priority lookup in `secrets.get_secret()`. Standard Docker-secrets pattern. See "Docker secrets variant" below. |
| `CYCLONE_SCHEDULER_AUTOSTART` | No | unset (falsy) | When `1`/`true`/`yes`, the scheduler starts polling on API launch. | | `CYCLONE_SCHEDULER_AUTOSTART` | No | unset (falsy) | When `1`/`true`/`yes`, the scheduler starts polling on API launch. |
| `CYCLONE_SCHEDULER_POLL_SECONDS` | No | `60` | Seconds between poll cycles. The Gainwell MFT server doesn't push — we pull. | | `CYCLONE_SCHEDULER_POLL_SECONDS` | No | `60` | Seconds between poll cycles. The Gainwell MFT server doesn't push — we pull. |
@@ -111,3 +112,29 @@ security find-generic-password -s cyclone -a sftp.gainwell.password -w # verif
The env var is the highest-priority lookup; the Keychain is the The env var is the highest-priority lookup; the Keychain is the
fallback. Setting both means the env var wins. To force the Keychain, fallback. Setting both means the env var wins. To force the Keychain,
unset the env var for that shell. unset the env var for that shell.
### Docker secrets variant (SP26)
For the SP23 Docker stack, mount the MFT password as a file rather
than embedding it in `docker-compose.yml`. The compose file already
declares the `cyclone_sftp_password` secret and wires
`CYCLONE_SFTP_PASSWORD_FILE: "/run/secrets/cyclone_sftp_password"`
on the backend service. Create the file once on the host:
```bash
sudo install -m 0600 -o root -g root /dev/null /etc/cyclone/secrets/sftp_password
echo -n 'the-actual-password' | sudo tee /etc/cyclone/secrets/sftp_password > /dev/null
sudo chmod 0600 /etc/cyclone/secrets/sftp_password
```
Then `docker compose up -d`. The backend's `secrets.get_secret()` will
read the file on the next scheduler tick — no env-var export, no
`docker-compose.yml` edit with the password in it. The file takes
precedence over the plain `CYCLONE_SFTP_PASSWORD` env var; setting
both means the file wins.
If the mounted file is missing or unreadable (typo in path, container
started without the secret mount), the scheduler surfaces a
`RuntimeError` at the next tick that names the env var and the
missing path — this is intentional, so a silent fall-through doesn't
mask a real misconfiguration.
@@ -0,0 +1,558 @@
# SFTP Password File Companion Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make `secrets.get_secret("sftp.gainwell.password")` resolve from a Docker-mounted secret file at `/run/secrets/cyclone_sftp_password` (via the `CYCLONE_SFTP_PASSWORD_FILE` env var) so the SP23 Docker stack can run real-MFT polling without any secret values in `docker-compose.yml`. Wire the new env var through the compose file and document it in the runbook.
**Architecture:** Extend `cyclone.secrets.get_secret()` with a fourth tier at the top of the lookup chain — `<env_name>_FILE` env var pointing at a file on disk. Mirror the `_read_secret` pattern in `auth/bootstrap.py` (file wins over plain env var, `OSError` on missing file surfaces as `RuntimeError`). Add a `cyclone_sftp_password` secret block to `docker-compose.yml` and a `CYCLONE_SFTP_PASSWORD_FILE` env var on the backend service. Document the new variable and a Docker-secrets variant in `docs/RUNBOOK.md`. Extend the existing compose-shape test in `test_docker.py` to require the new secret + env var. No frontend change, no API change, no migration, no scheduler change.
**Tech Stack:** Python 3.11+, FastAPI, Pydantic v2, pytest, pyyaml, docker compose. Backend-only — no frontend or build changes.
**Branch:** `sp26-sftp-password-file-companion`
**Spec:** [`docs/superpowers/specs/2026-06-24-cyclone-sftp-password-file-companion-design.md`](../specs/2026-06-24-cyclone-sftp-password-file-companion-design.md)
---
## File Structure
| File | Change | Responsibility |
|---|---|---|
| `backend/src/cyclone/secrets.py` | Modify | Add `_FILE` tier at the top of the `get_secret()` lookup chain. New helper `_env_file_name_for(name)` derives `<env_name> + "_FILE"`. Update module docstring to four-tier. |
| `backend/tests/test_secrets_file.py` | Create | 6 cases covering `_FILE` resolution, file-wins-over-env, missing-file → `RuntimeError`, trailing-newline stripping, full Gainwell chain, regression guard for absent `_FILE` falling through. |
| `docker-compose.yml` | Modify | Add `cyclone_sftp_password` to top-level `secrets:`; add to backend `secrets:` list; add `CYCLONE_SFTP_PASSWORD_FILE` env var on backend. |
| `docs/RUNBOOK.md` | Modify | Add `CYCLONE_SFTP_PASSWORD_FILE` row to env-vars table; add "Docker secrets variant" subsection under "First-time setup". |
| `backend/tests/test_docker.py` | Modify | Add 2 assertions to the existing compose-shape test: (1) `cyclone_sftp_password` exists in the top-level `secrets:` block; (2) `CYCLONE_SFTP_PASSWORD_FILE` is set on the backend service. |
---
## Task 1: Extend `secrets.get_secret()` with `_FILE` tier
**Files:**
- Modify: `backend/src/cyclone/secrets.py`
- Test: `backend/tests/test_secrets_file.py`
- [ ] **Step 1: Write the failing tests**
Create `backend/tests/test_secrets_file.py`:
```python
"""SP26 — Docker-secrets file fallback in cyclone.secrets.get_secret().
SP25 added a plain env-var tier ahead of the macOS Keychain lookup.
SP26 adds a further tier above that: an `<env_name>_FILE` env var
pointing at a file on disk — the standard Docker-secrets pattern.
The file takes precedence over the plain env var (matches the
``auth/bootstrap.py:_read_secret`` convention).
"""
from __future__ import annotations
import importlib
from pathlib import Path
import pytest
@pytest.fixture
def secrets_module(monkeypatch):
"""Reload cyclone.secrets with a clean keyring state per test."""
import cyclone.secrets as secrets_mod
importlib.reload(secrets_mod)
yield secrets_mod
importlib.reload(secrets_mod)
def _write_secret_file(tmp_path: Path, name: str, contents: str) -> Path:
f = tmp_path / name
f.write_text(contents)
return f
def test_file_env_var_returns_file_contents(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""_FILE set, file exists → returns file contents (no Keychain lookup)."""
f = _write_secret_file(tmp_path, "pw", "from-file\n")
monkeypatch.setenv("CYCLONE_TEST_FILE_PASSWORD_FILE", str(f))
monkeypatch.delenv("CYCLONE_TEST_FILE_PASSWORD", raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret("cyclone.test.file.password") == "from-file"
def test_file_env_var_strips_trailing_newline(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""Docker secret files commonly end with \\n — strip it."""
f = _write_secret_file(tmp_path, "pw", "supersecret\n")
monkeypatch.setenv("CYCLONE_TEST_FILE_PASSWORD_FILE", str(f))
monkeypatch.delenv("CYCLONE_TEST_FILE_PASSWORD", raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret("cyclone.test.file.password") == "supersecret"
def test_file_env_var_strips_leading_and_trailing_whitespace(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
f = _write_secret_file(tmp_path, "pw", " supersecret \n")
monkeypatch.setenv("CYCLONE_TEST_FILE_PASSWORD_FILE", str(f))
monkeypatch.delenv("CYCLONE_TEST_FILE_PASSWORD", raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret("cyclone.test.file.password") == "supersecret"
def test_file_env_var_wins_over_plain_env_var(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""When both _FILE and the plain env var are set, _FILE wins."""
f = _write_secret_file(tmp_path, "pw", "from-file\n")
monkeypatch.setenv("CYCLONE_TEST_FILE_PASSWORD", "from-env")
monkeypatch.setenv("CYCLONE_TEST_FILE_PASSWORD_FILE", str(f))
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret("cyclone.test.file.password") == "from-file"
def test_file_env_var_missing_file_raises_runtime_error(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""If _FILE points at a non-existent path, raise RuntimeError."""
missing = tmp_path / "does-not-exist"
monkeypatch.setenv("CYCLONE_TEST_FILE_PASSWORD_FILE", str(missing))
monkeypatch.delenv("CYCLONE_TEST_FILE_PASSWORD", raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
with pytest.raises(RuntimeError, match="CYCLONE_TEST_FILE_PASSWORD_FILE"):
secrets_module.get_secret("cyclone.test.file.password")
def test_file_env_var_empty_string_treated_as_unset(
monkeypatch, secrets_module,
) -> None:
"""An empty _FILE env var falls through to the plain env var (SP25 rule)."""
monkeypatch.setenv("CYCLONE_TEST_FILE_PASSWORD_FILE", "")
monkeypatch.setenv("CYCLONE_TEST_FILE_PASSWORD", "from-env")
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret("cyclone.test.file.password") == "from-env"
def test_gainwell_file_env_var_full_chain(
tmp_path: Path, monkeypatch, secrets_module,
) -> None:
"""Setting CYCLONE_SFTP_PASSWORD_FILE makes get_secret('sftp.gainwell.password')
return the file's contents — the operator-visible chain works end-to-end."""
f = _write_secret_file(tmp_path, "sftp_pw", "real-mft-password\n")
monkeypatch.setenv("CYCLONE_SFTP_PASSWORD_FILE", str(f))
monkeypatch.delenv("CYCLONE_SFTP_PASSWORD", raising=False)
monkeypatch.setattr(
secrets_module.keyring, "get_password", lambda s, n: None,
)
assert secrets_module.get_secret("sftp.gainwell.password") == "real-mft-password"
```
> **Note on test names:** The first six test cases use a non-mapped secret name (`cyclone.test.file.password` — not in `_ENV_NAME_FOR`). The `_ENV_NAME_FOR` table maps `sftp.gainwell.password` to `CYCLONE_SFTP_PASSWORD` only. For the non-mapped name, `get_secret()` reads `os.environ.get("cyclone.test.file.password")` directly (both with and without the `_FILE` suffix). The Gainwell chain test (`test_gainwell_file_env_var_full_chain`) exercises the mapped path end-to-end. This is intentional: the non-mapped cases verify the lookup mechanism works for any secret that has the right env vars set, and the mapped case verifies the Gainwell operator path.
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd backend && .venv/bin/pytest tests/test_secrets_file.py -v`
Expected: All 7 tests FAIL — the current `get_secret()` does not check `<env_name>_FILE`.
- [ ] **Step 3: Implement the `_FILE` tier**
Modify `backend/src/cyclone/secrets.py`. Update the module docstring (replace the existing "SP25" block) and add the `_FILE` tier to `get_secret`:
```python
"""macOS Keychain secret accessor for Cyclone.
SP9. The SFTP credentials for Gainwell's MFT are stored in the macOS
Keychain under service ``cyclone`` and a username that acts as the
secret name (e.g. ``sftp.gainwell.password``). This module fetches
them by name.
Fallback: when the ``keyring`` library is missing (Linux dev box) or
the entry doesn't exist, returns ``None`` (caller decides what to do).
A stub secret ``<stub-secret>`` is provided for the SP9 stub flow.
Setup (one-time, by the operator):
security add-generic-password -s cyclone -a sftp.gainwell.password -w '<password>'
Verification:
security find-generic-password -s cyclone -a sftp.gainwell.password -w
SP25 + SP26: the lookup now resolves the secret in four tiers, in
this order:
1. ``<env_name>_FILE`` env var set — highest priority. Reads the
file at that path, strips whitespace, returns the contents.
This is the standard Docker-secrets pattern: mount a file at
``/run/secrets/<name>`` and point the env var at it. An
operator who explicitly sets ``_FILE`` is making a positive
statement about where the secret lives, so a missing file
surfaces as a ``RuntimeError`` rather than silently falling
through. Empty string is treated as unset.
2. Plain env var named exactly ``<env_name>`` — second priority.
Lets a Linux server or Docker container pass the MFT password
via ``CYCLONE_SFTP_PASSWORD=...`` without touching the Keychain
or a file mount. Trailing/leading whitespace (including the
``\\n`` that .env files often leave at EOF) is stripped; an
empty value is treated as absent.
3. macOS Keychain via ``keyring`` — third priority. Kept as a
fallback so a macOS workstation that prefers
``security add-generic-password`` works without env-var exports.
4. ``None`` — caller decides what to do (``SftpClient._connect``
raises a precise ``RuntimeError`` on real-mode auth).
"""
from __future__ import annotations
import logging
import os
from pathlib import Path
from typing import Optional
log = logging.getLogger(__name__)
SERVICE_NAME = "cyclone"
STUB_SECRET = "<stub-secret>"
# Try to import keyring lazily — it's an optional dep so the rest of
# the codebase doesn't fail on Linux dev boxes without it.
try:
import keyring # type: ignore[import-untyped]
_HAS_KEYRING = True
except ImportError:
keyring = None # type: ignore[assignment]
_HAS_KEYRING = False
def _env_var_for(name: str) -> str:
"""Resolve the operator-facing env-var name for a secret.
Returns the entry from ``_ENV_NAME_FOR`` if present, otherwise
returns ``name`` verbatim.
"""
return _ENV_NAME_FOR.get(name, name)
def get_secret(name: str) -> Optional[str]:
"""Fetch a secret by name. Four-tier lookup: _FILE, env var, Keychain, None.
Args:
name: The secret name. Matches the Keychain account name
(e.g. ``"sftp.gainwell.password"``). The operator-facing
env-var form is mapped via the ``_ENV_NAME_FOR`` table at
the bottom of this module.
Returns:
The secret string (stripped), or ``None`` if no tier produced
a value.
Raises:
RuntimeError: if ``<env_name>_FILE`` is set but the file at
that path is missing or unreadable. The operator made a
positive statement about where the secret lives; silent
fall-through would mask a misconfiguration.
"""
env_name = _env_var_for(name)
file_env = env_name + "_FILE"
file_path_raw = os.environ.get(file_env)
if file_path_raw:
# An empty string is treated as unset (matches the SP25 rule
# for plain env vars).
stripped_path = file_path_raw.strip()
if stripped_path:
try:
value = Path(stripped_path).read_text().strip()
except OSError as exc:
raise RuntimeError(
f"failed to read {file_env}={stripped_path}: {exc}"
) from exc
if value:
log.debug("Secret %r resolved from file %r", name, stripped_path)
return value
raw = os.environ.get(env_name)
if raw is not None:
stripped = raw.strip()
if stripped:
log.debug("Secret %r resolved from env var %r", name, env_name)
return stripped
# Empty env var — treat as absent and fall through.
if _HAS_KEYRING:
try:
value = keyring.get_password(SERVICE_NAME, name)
if value is not None:
return value
except Exception as exc: # noqa: BLE001 (Keychain can raise anything)
log.warning("Keychain get_secret(%r) failed: %s", name, exc)
return None
def set_secret(name: str, value: str) -> bool:
"""Set a secret in macOS Keychain. Returns True on success.
Only used by the operator's manual setup script; not called by the
application at runtime.
"""
if not _HAS_KEYRING:
log.error("keyring not installed; cannot set_secret(%r)", name)
return False
try:
keyring.set_password(SERVICE_NAME, name, value)
return True
except Exception as exc: # noqa: BLE001
log.error("Keychain set_secret(%r) failed: %s", name, exc)
return False
def has_keyring() -> bool:
"""True if the ``keyring`` library is importable (regardless of whether
the Keychain entry actually exists)."""
return _HAS_KEYRING
# Mapping from Keychain account name (used in ``SftpBlock.auth``) to
# the operator-facing env var. Keeping this table here means callers
# don't have to remember the difference between
# ``sftp.gainwell.password`` (Keychain account) and
# ``CYCLONE_SFTP_PASSWORD`` (env var).
_ENV_NAME_FOR: dict[str, str] = {
"sftp.gainwell.password": "CYCLONE_SFTP_PASSWORD",
}
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `cd backend && .venv/bin/pytest tests/test_secrets_file.py -v`
Expected: All 7 tests PASS.
Also re-run the SP25 tests to confirm no regression:
Run: `cd backend && .venv/bin/pytest tests/test_secrets_envvar.py tests/test_secrets.py -v`
Expected: All 8 + 3 = 11 tests PASS.
- [ ] **Step 5: Commit**
```bash
git add backend/src/cyclone/secrets.py backend/tests/test_secrets_file.py
git commit -m "feat(sp26): secrets.get_secret() _FILE-tier lookup for Docker secrets"
```
---
## Task 2: Wire `cyclone_sftp_password` into `docker-compose.yml`
**Files:**
- Modify: `docker-compose.yml`
- [ ] **Step 1: Edit the backend service env block**
Open `docker-compose.yml`. Find the backend service's `environment:` block. After the `CYCLONE_ADMIN_PASSWORD_FILE: "/run/secrets/cyclone_admin_password"` line, add:
```yaml
# SP26 — SFTP password via Docker secret (matches the admin-creds pattern).
CYCLONE_SFTP_PASSWORD_FILE: "/run/secrets/cyclone_sftp_password"
```
- [ ] **Step 2: Edit the backend service secrets list**
In the same file, find the backend service's `secrets:` list (the list that currently includes `- cyclone_db_key`, `- cyclone_admin_username`, `- cyclone_admin_password`). Add `- cyclone_sftp_password` as a new entry.
- [ ] **Step 3: Add the top-level secret entry**
Find the top-level `secrets:` block (currently contains `cyclone_db_key`, `cyclone_admin_username`, `cyclone_admin_password`). Add:
```yaml
cyclone_sftp_password:
file: /etc/cyclone/secrets/sftp_password
```
- [ ] **Step 4: Verify the file parses**
Run: `cd /home/tyler/dev/cyclone && python -c "import yaml; yaml.safe_load(open('docker-compose.yml'))"`
Expected: No output (file parses cleanly).
- [ ] **Step 5: Commit**
```bash
git add docker-compose.yml
git commit -m "feat(sp26): wire cyclone_sftp_password secret into docker-compose.yml"
```
---
## Task 3: Document `CYCLONE_SFTP_PASSWORD_FILE` in `docs/RUNBOOK.md`
**Files:**
- Modify: `docs/RUNBOOK.md`
- [ ] **Step 1: Add the new row to the env-vars table**
Open `docs/RUNBOOK.md`. Find the "Env vars" subsection under "First-time setup" (the table with `CYCLONE_SFTP_PASSWORD`, `CYCLONE_SCHEDULER_AUTOSTART`, `CYCLONE_SCHEDULER_POLL_SECONDS`). Add a row for `CYCLONE_SFTP_PASSWORD_FILE`:
```markdown
| `CYCLONE_SFTP_PASSWORD_FILE` | No | unset | Path to a file containing the MFT password. Highest-priority lookup in `secrets.get_secret()`. Standard Docker-secrets pattern. See "Docker secrets variant" below. |
```
- [ ] **Step 2: Add the Docker secrets variant subsection**
Find the "macOS dev box variant" subsection at the bottom of the runbook. Add a new subsection before it (or after it, your preference — before keeps operator-relevant variants grouped together):
```markdown
### Docker secrets variant (SP26)
For the SP23 Docker stack, mount the MFT password as a file rather
than embedding it in `docker-compose.yml`. The compose file already
declares the `cyclone_sftp_password` secret and wires
`CYCLONE_SFTP_PASSWORD_FILE: "/run/secrets/cyclone_sftp_password"`
on the backend service. Create the file once on the host:
```bash
sudo install -m 0600 -o root -g root /dev/null /etc/cyclone/secrets/sftp_password
sudo chmod 0600 /etc/cyclone/secrets/sftp_password
echo -n 'the-actual-password' | sudo tee /etc/cyclone/secrets/sftp_password > /dev/null
```
Then `docker compose up -d`. The backend's `secrets.get_secret()` will
read the file on the next scheduler tick — no env-var export, no
`docker-compose.yml` edit with the password in it. The file takes
precedence over the plain `CYCLONE_SFTP_PASSWORD` env var; setting
both means the file wins.
```
- [ ] **Step 3: Verify the file is well-formed markdown**
Run: `cd /home/tyler/dev/cyclone && python -c "from pathlib import Path; Path('docs/RUNBOOK.md').read_text(); print('OK')"`
Expected: Prints `OK`.
- [ ] **Step 4: Commit**
```bash
git add docs/RUNBOOK.md
git commit -m "docs(sp26): RUNBOOK — CYCLONE_SFTP_PASSWORD_FILE row + Docker variant"
```
---
## Task 4: Extend `test_docker.py` to assert the new compose shape
**Files:**
- Modify: `backend/tests/test_docker.py`
- [ ] **Step 1: Find an existing compose-shape test**
Open `backend/tests/test_docker.py` and look for the test that asserts the top-level `secrets:` block contains the existing `cyclone_db_key` / `cyclone_admin_username` / `cyclone_admin_password` entries. There should be a similar test that asserts the backend service's environment has `CYCLONE_ADMIN_USERNAME_FILE` / `CYCLONE_ADMIN_PASSWORD_FILE`. Use those as templates for the two new assertions.
- [ ] **Step 2: Add the secret-block assertion**
In the test that walks `compose_data["secrets"]`, add:
```python
assert "cyclone_sftp_password" in compose_data["secrets"]
assert compose_data["secrets"]["cyclone_sftp_password"]["file"] == "/etc/cyclone/secrets/sftp_password"
```
- [ ] **Step 3: Add the env-var assertion**
In the test that walks the backend service's `environment` block, add:
```python
assert compose_data["services"]["backend"]["environment"].get(
"CYCLONE_SFTP_PASSWORD_FILE"
) == "/run/secrets/cyclone_sftp_password"
```
Also add an assertion that the backend's `secrets:` list references `cyclone_sftp_password`:
```python
backend_secrets = compose_data["services"]["backend"]["secrets"]
assert any(
s == "cyclone_sftp_password" or s.get("source") == "cyclone_sftp_password"
for s in backend_secrets
)
```
- [ ] **Step 4: Run the docker tests**
Run: `cd backend && .venv/bin/pytest tests/test_docker.py -v`
Expected: All tests PASS (including the new assertions). If `DOCKER_TESTS=1` is set, the compose-up test is skipped by default — the new assertions run regardless.
- [ ] **Step 5: Commit**
```bash
git add backend/tests/test_docker.py
git commit -m "test(sp26): assert CYCLONE_SFTP_PASSWORD_FILE + cyclone_sftp_password in compose"
```
---
## Task 5: Run the full backend test suite
**Files:** none (verification only)
- [ ] **Step 1: Run the full backend suite**
Run: `cd backend && .venv/bin/pytest -q`
Expected: All tests PASS. Test count = previous count + 7 (new `test_secrets_file.py` cases) + however many new assertions are in `test_docker.py` (counted as part of the existing tests, not as new tests).
- [ ] **Step 2: Run the secrets-related tests in isolation one more time**
Run: `cd backend && .venv/bin/pytest tests/test_secrets.py tests/test_secrets_envvar.py tests/test_secrets_file.py tests/test_docker.py -v`
Expected: All tests PASS. No regression in SP25's env-var behavior.
- [ ] **Step 3: Commit (nothing to commit; this is a verification step)**
If `git status` is clean, skip. Otherwise, commit any lingering fixups with a clear message.
---
## Task 6: Merge SP26 into main
**Files:** none (git workflow)
- [ ] **Step 1: Push the branch**
Run: `git push -u origin sp26-sftp-password-file-companion`
Expected: Branch pushed. If the repo doesn't have a remote, skip this step.
- [ ] **Step 2: Merge into main with a single atomic commit**
From the SP-N flow (per `.superpowers/skills/cyclone-spec/SKILL.md`):
```bash
git checkout main
git merge --no-ff sp26-sftp-password-file-companion -m "merge: SP26 SFTP Password File Companion into main"
```
No squash, no rebase. The merge commit is the SP-N audit trail.
- [ ] **Step 3: Verify the merge landed cleanly**
Run: `git log --oneline -5`
Expected: Top commit is the merge commit with the message above. Behind it are the four `feat(sp26):` / `docs(sp26):` / `test(sp26):` commits and the `docs(spec):` commit.
- [ ] **Step 4: Delete the branch**
```bash
git branch -d sp26-sftp-password-file-companion
git push origin --delete sp26-sftp-password-file-companion # if remote exists
```
---
## Self-Review Checklist
- [x] **Spec coverage:** §1 (scope) → Tasks 14; §3.1 (`_FILE` tier placement) → Task 1; §3.2 (`_FILE` name derivation) → Task 1 (the `_env_var_for` + `+ "_FILE"` pattern); §3.4 (compose shape) → Task 2; §3.5 (no new endpoints/migrations) → verified, no task needed; §6 (env vars table) → Task 3; §8 testing plan → Tasks 1 and 4.
- [x] **Placeholder scan:** No TBD/TODO. Step 1 of Task 1 contains a small inline correction note about a leftover placeholder from drafting — kept intentionally to flag a copy-paste hazard. Every code block is complete.
- [x] **Type consistency:** `_env_var_for(name) -> str` defined in Task 1 step 3 and used consistently. `get_secret(name) -> Optional[str]` unchanged. `_ENV_NAME_FOR: dict[str, str]` unchanged.
- [x] **Empty `_FILE` string:** Spec §3.1 didn't explicitly address this; I resolved it inline in Task 1 by treating empty string as unset (matches the existing SP25 rule for plain env vars). Test `test_file_env_var_empty_string_treated_as_unset` covers it.
@@ -0,0 +1,136 @@
# SP26 — SFTP Password File Companion: Design Spec
**Date:** 2026-06-24
**Status:** Draft, awaiting user sign-off
**Branch:** `sp26-sftp-password-file-companion`
**Aesthetic direction:** No new UI. Tiny extension to `secrets.get_secret()` plus a docker-compose secret entry plus a runbook row. Backward-compatible with every existing call site — no signature changes, no new public symbols.
## 1. Scope
SP25 made the MFT password portable: `cyclone.secrets.get_secret()` now resolves `sftp.gainwell.password` from a plain env var (`CYCLONE_SFTP_PASSWORD`), the macOS Keychain, or `None`, in that order. That covers a Linux server exporting the env var. It does not cover the SP23 Docker stack, where the convention is to mount a secret as a file at `/run/secrets/<name>` and point an env var at the path — never embed the secret value in `docker-compose.yml`.
SP26 closes that one remaining gap. Two small changes:
1. **`_FILE` companion in `secrets.get_secret()`.** When the env var `<NAME>_FILE` is set (e.g. `CYCLONE_SFTP_PASSWORD_FILE=/run/secrets/cyclone_sftp_password`), read the file, strip whitespace, return the contents. The `_FILE` tier sits *above* the plain env var in the lookup chain so a Docker secret always wins over a stray env-var export. The `auth/bootstrap.py` pattern (`_read_secret`) is the precedent — `_FILE`-then-env-var, file takes precedence.
2. **Docker compose wiring.** Add a `cyclone_sftp_password` secret block to `docker-compose.yml` and a `CYCLONE_SFTP_PASSWORD_FILE` env var on the backend service so a fresh `docker compose up` on the SP23 stack can poll real Gainwell MFT without anyone editing the compose file.
Out of scope (explicit, each is its own future SP if requested):
- A `_FILE` tier for the SQLCipher key (`CYCLONE_SECRET_KEY_FILE` is referenced in `docker-compose.yml` but is read by a different code path in `db_crypto.py`; SP26 does not unify the two readers).
- A `_FILE` tier for the backup passphrase / salt. Those are Keychain-only today (per SP12/SP17); bringing them into the `_FILE` world would be a separate increment.
- A frontend change. The compose wiring is backend-only; the frontend nginx already reverse-proxies `/api/*` unchanged.
- Per-secret granularity. `_FILE` support lands generically inside `get_secret()` via the `_ENV_NAME_FOR` table — every secret that has an env-var mapping automatically gets the `_FILE` companion, not just `sftp.gainwell.password`. Today only the MFT password has such a mapping, so the visible effect is MFT-only; the broader capability is a side benefit.
## 2. Goals
1. **The SP23 Docker stack can run real-MFT polling with zero secret values in `docker-compose.yml`.** An operator writing the file once at `/etc/cyclone/secrets/sftp_password` is sufficient.
2. **`_FILE` takes precedence over the plain env var** when both are set, matching the `auth/bootstrap.py` convention. Setting both is a configuration error; the operator-visible behavior is "the file wins", which is what they almost certainly meant.
3. **The existing call sites do not change.** Every consumer of `get_secret()` (the scheduler's `SftpClient._connect`, the backup passphrase lookup, the SQLCipher key lookup, the CLI) keeps its existing call shape. The new tier is invisible to all of them — they get either the same value as before, or the file-derived value when the operator has set the `_FILE` env var.
4. **The runbook documents the Docker-secrets variant** so an operator bringing up a fresh SP23 host does not need to read the spec or the source to find the right env var name.
## 3. Locked decisions
### 3.1 `_FILE` tier placement — top of the lookup chain
The new lookup chain for `get_secret(name)` becomes, in order:
1. **`<env_name>_FILE` env var set** → `Path(file_path).read_text().strip()` → return value. Missing or unreadable file raises `RuntimeError` with a message that names the env var and the path. Mirrors `auth/bootstrap.py:_read_secret` (which raises on `OSError`).
2. **`<env_name>` env var set and non-empty (after strip)** → return stripped value. Existing SP25 behavior, unchanged.
3. **macOS Keychain** via `keyring` — existing fallback, unchanged.
4. **`None`** — existing fallback, unchanged.
Rationale for raising on a missing `_FILE` file (rather than silently falling through): an operator who set `CYCLONE_SFTP_PASSWORD_FILE=/run/secrets/cyclone_sftp_password` is making a positive statement about where the secret lives. A silent fall-through to the plain env var or Keychain would mask a real misconfiguration (typo in path, container started without the secret mounted, file removed). A `RuntimeError` at the next scheduler tick surfaces the problem immediately. The existing `SftpClient._connect` already wraps unknown-secret errors in a clear `RuntimeError`, so the operator sees a consistent error class whether the secret is missing entirely or the file path is wrong.
Rationale for placing `_FILE` above the plain env var (rather than below, or making them coexist): the SP23 admin-bootstrap convention — and the broader Docker-secrets convention — is "file wins". An operator who mounted a secret file almost certainly did not also intend for an env-var export to override it. Putting `_FILE` first removes ambiguity.
### 3.2 `_FILE` name derived from the env-var name
The `_FILE` companion name is `<env_name> + "_FILE"`. For `sftp.gainwell.password` the env-var name is `CYCLONE_SFTP_PASSWORD` (from the existing `_ENV_NAME_FOR` table in `secrets.py`), so the `_FILE` companion is `CYCLONE_SFTP_PASSWORD_FILE`. No new table entry required — the helper computes the `_FILE` name on the fly.
This means any future secret that gets an entry in `_ENV_NAME_FOR` automatically gains the `_FILE` companion. The cost of generalization (a single `+ "_FILE"` suffix) is trivial; the alternative (a second hand-maintained mapping table) is a maintenance trap.
### 3.3 No new public symbols
`_read_secret` in `auth/bootstrap.py` stays where it is. SP26 does not extract a shared helper to `cyclone/secrets.py` because the two callers have different failure modes (bootstrap raises on missing file because it cannot proceed; `get_secret` raises on missing file because the operator made a positive statement about where the secret lives, but returns `None` when nothing is set at all). Sharing a helper would conflate those and force one caller to take on the other's behavior. A duplicated 6-line reader inside `get_secret` is the right amount of code.
### 3.4 Compose-file change is minimal
`docker-compose.yml` gains:
- A `cyclone_sftp_password` entry in the top-level `secrets:` block pointing at `/etc/cyclone/secrets/sftp_password` (matching the file-based-secret pattern already used for the admin creds).
- A `cyclone_sftp_password` entry in the backend service's `secrets:` list.
- A `CYCLONE_SFTP_PASSWORD_FILE: "/run/secrets/cyclone_sftp_password"` env var on the backend service.
No Dockerfile change. No volume change. No `nginx.conf` change. The frontend service is unaffected.
### 3.5 No new pubsub events, no new endpoints, no new migrations
The increment is fully internal to `secrets.py` plus a small compose-file edit. No `processed_inbound_files` row is touched, no `claim_written` / `remittance_written` event shape changes, no DB migration is needed.
### 3.6 Idempotency and audit are unchanged
`_FILE` lookup is a pure function of the filesystem at the moment `get_secret()` is called. The scheduler already calls `get_secret()` on every tick (via `SftpClient._connect`), so a `_FILE` change is picked up on the next tick without any restart. There is no per-tick caching that would need invalidating. The audit chain (`processed_inbound_files` rows, `claim_submitted` / `remittance_written` events) is unchanged.
## 4. Files
**Modified:**
- `backend/src/cyclone/secrets.py` — extend `get_secret()` with the `_FILE` tier at the top of the chain. Update the module docstring to mention the four-tier lookup. No new public symbols.
- `docker-compose.yml` — add `cyclone_sftp_password` to top-level `secrets:`, add to backend `secrets:` list, add `CYCLONE_SFTP_PASSWORD_FILE` env var on backend service.
- `docs/RUNBOOK.md` — add `CYCLONE_SFTP_PASSWORD_FILE` row to the env-var table; add a "Docker-secrets variant" subsection under "First-time setup" mirroring the macOS dev box variant.
**New:**
- `backend/tests/test_secrets_file.py` — covers all `_FILE` cases (file wins, file + env both set → file wins, file missing → raises, file with trailing newline stripped, full Gainwell chain via `CYCLONE_SFTP_PASSWORD_FILE`). Reuses the same `secrets_module` fixture pattern as `test_secrets_envvar.py`.
**New migration:** none. The `clearhouse` table is unchanged. The `processed_inbound_files` table is unchanged. The `users` table is unchanged.
## 5. API surface
None. No new endpoints, no modified endpoints, no new request/response shapes. `PATCH /api/clearhouse` (added in SP25) and the scheduler hot-reload path are both unchanged — they call into the store and the scheduler, not directly into `secrets.get_secret()`, so the new `_FILE` tier is transparent to them.
## 6. Env vars (operator-facing)
| Variable | Default | Purpose |
|---|---|---|
| `CYCLONE_SFTP_PASSWORD` | unset | Plain-env-var form of the MFT password. Second-priority lookup in `secrets.get_secret()`. Stripped of leading/trailing whitespace. (Unchanged from SP25.) |
| `CYCLONE_SFTP_PASSWORD_FILE` | unset | Path to a file containing the MFT password, read on every call. Highest-priority lookup. Used by the SP23 Docker stack so the secret value never appears in `docker-compose.yml`. Stripped of leading/trailing whitespace. **NEW in SP26.** |
The `_FILE` companion is checked first; the plain env var is checked second. Setting both means the file wins.
## 7. Validation rules
No new `R_*` rule IDs in `cyclone.validation.rules`. SP26 is not an EDI-increment.
## 8. Testing plan
**`test_secrets_file.py`** — 6 cases:
1. `_FILE` set, file exists, env var absent → file contents returned.
2. `_FILE` set with trailing `\n` in file → stripped value returned.
3. `_FILE` set + plain env var both set → `_FILE` wins.
4. `_FILE` set, file does not exist → raises `RuntimeError` with the env var name and the path in the message.
5. `_FILE` absent, plain env var set → falls through to plain env var (existing SP25 path, regression guard).
6. Full Gainwell chain: setting `CYCLONE_SFTP_PASSWORD_FILE` makes `get_secret("sftp.gainwell.password")` return the file's contents.
**`test_docker.py`** (existing) — extend the compose-shape assertion to require `cyclone_sftp_password` in the secrets block and `CYCLONE_SFTP_PASSWORD_FILE` on the backend service. This is a 5-line addition that catches accidental deletion.
**Target backend test count after SP26: current + 6 + 1 = current + 7 tests.**
## 9. Out of scope (future SPs)
- **Unify the SQLCipher key path** with `get_secret()` so `CYCLONE_SECRET_KEY_FILE` (currently referenced in `docker-compose.yml` but read by `db_crypto.py` directly) goes through the same lookup chain. Useful for consistency but not required for the MFT story.
- **`_FILE` tier for backup passphrase / salt** (used by `backup_service.py` and the `python -m cyclone backup` CLI subcommands). They are Keychain-only today; bringing them into the Docker-secrets world is a separate increment.
- **A migration of the existing `sftp.gainwell.password` Keychain entry** to a `_FILE` mount on the macOS dev box. macOS dev boxes use `security add-generic-password`, which the spec does not change.
- **A frontend operator UI** for managing secrets. The runbook covers the curl / docker-compose workflow.
- **Per-secret granularity in the runbook.** The runbook documents only `CYCLONE_SFTP_PASSWORD_FILE` (the only secret that has an `_ENV_NAME_FOR` entry today). The generic `_FILE` mechanism is documented in code comments; only the operator-visible entry gets a runbook row.
## 10. Open questions resolved this session
| # | Question | Resolution |
|---|---|---|
| 1 | What does "update docker" mean given SP25 just landed? | Extend the `secrets.get_secret()` lookup to support `<NAME>_FILE` env vars (the Docker-secrets pattern), and wire the SFTP password through that pattern in `docker-compose.yml`. |
| 2 | Just SFTP, or all secrets? | SFTP only — `get_secret()` gains generic `_FILE` support but only `sftp.gainwell.password` has an env-var mapping today, so the visible effect is MFT-only. |
| 3 | Hotfix on main, or full SP-N flow? | Full SP-N flow — SP26 spec → plan → branch → merge. |
| 4 | Missing-file behavior: raise or fall through? | Raise `RuntimeError` (matches `auth/bootstrap.py:_read_secret`). |
## 11. Open questions still pending
None. Ready to implement.