8 Commits

Author SHA1 Message Date
Cyclone f7697e58b7 feat(sp23): tests/test_docker.py validates compose config + Dockerfile parse
Eight tests: compose file exists, docker compose config validates, services declare required volumes/restart/healthcheck/ports/depends_on, secrets and volumes tables include the names the operator expects, backend wires CYCLONE_BACKUP_AUTOSTART=1, both Dockerfiles pass . The live bring-up test (compose up + wait for healthy) is gated on DOCKER_TESTS=1 so it skips on bare CI. PyYAML was already a hard dep.
2026-06-23 17:24:46 -06:00
Cyclone 7706a6d7fe feat(sp23): auth bootstrap reads CYCLONE_ADMIN_*_FILE for Docker secrets
Add _read_secret() helper that prefers *_FILE env vars (the standard Docker-secret pattern) over bare env vars. Strips trailing whitespace from file contents so printf/echo newlines don't break bcrypt verify. Existing CYCLONE_ADMIN_USERNAME + CYCLONE_ADMIN_PASSWORD still work for non-Docker deployments. Add tests/test_auth_bootstrap_file.py covering: file-path read, file-overrides-bare, bare-fallback, whitespace-stripping, missing-file raises.
2026-06-23 17:24:23 -06:00
Cyclone 256ddfa5fa feat(sp23): operator scripts (cyclone-init, post-deploy, smoke) + RUNBOOK
cyclone-init.sh generates /etc/cyclone/secrets/{db.key,admin_username,admin_pw} with openssl rand, prints the admin password once. post-deploy.sh installs logrotate.d/cyclone and a 5-minute healthcheck cron. smoke.sh brings up + logs in + parses a sample 837 end-to-end. RUNBOOK.md covers daily/weekly/quarterly/annual ops procedures plus the emergency runbook.
2026-06-23 17:22:44 -06:00
Cyclone d42fbc8c1b feat(sp23): docker-compose.yml with backend + frontend, named volumes, secrets 2026-06-23 17:22:06 -06:00
Cyclone 67dae61a94 feat(sp23): frontend Dockerfile + nginx.conf (SPA + reverse proxy) 2026-06-23 17:22:03 -06:00
Cyclone 59e69127a2 feat(sp23): backend Dockerfile (multi-stage, sqlcipher, non-root, healthcheck) 2026-06-23 17:21:54 -06:00
Cyclone 364e5d7497 docs(plan): SP23 Ubuntu Docker Deployment implementation plan
12 tasks from worktree bootstrap through single-atomic-merge into main. Backend Dockerfile (multi-stage python:3.11-slim-bookworm + sqlcipher + non-root + healthcheck), frontend Dockerfile (multi-stage node:20-alpine build + nginx:1.27-alpine runtime), docker-compose.yml at repo root with two services + named volumes + Docker secrets, scripts/{cyclone-init,post-deploy,smoke}.sh, RUNBOOK.md, tests/test_docker.py (compose config + Dockerfile parse + optional live bring-up), and a small auth bootstrap extension to read CYCLONE_ADMIN_*_FILE for Docker-secret compatibility. Closes the SP23 spec; see spec section 1.1 for what auth is reused from main vs. what SP23 actually adds.
2026-06-23 17:21:34 -06:00
Cyclone 35c561de20 docs(spec): mark SP23 approved and reconcile with auth on main
Auth landed in main on 2026-06-23 (migrations 0013/0014, cyclone.auth package, bcrypt + matrix_gate, Login.tsx + AuthProvider + RoleGate, CYCLONE_AUTH_DISABLED escape hatch). The original draft was written 2026-06-22 and proposed argon2id, admin/operator/viewer roles, a fresh migration 0012, and an inline require_role dependency. All of that is already in main. Add a 1.1 Delta vs. main section and switch role names from operator to user to match cyclone.auth.permissions.Role.
2026-06-23 17:20:04 -06:00
16 changed files with 2056 additions and 83 deletions
+13
View File
@@ -0,0 +1,13 @@
# Repo-root .dockerignore — applies to `docker compose build` (which builds
# both backend and frontend contexts from the repo root). Excludes anything
# that should never end up in a build context.
.worktrees/
.git/
.github/
docs/prodfiles/
*.production.txt
node_modules/
dist/
.venv/
.superpowers/brainstorm/
+65
View File
@@ -0,0 +1,65 @@
# Cyclone Operator Runbook
Production operations for a single-operator Cyclone deploy on Ubuntu Linux. Assumes the box was bootstrapped via `scripts/cyclone-init.sh` and the stack is up via `docker compose up -d`.
## Daily
- [ ] Confirm the host healthcheck cron hasn't emailed. It pings `http://localhost:8080/api/healthz` every 5 minutes.
- [ ] `docker compose ps` — both services `healthy`.
- [ ] `docker compose logs --tail=200 backend | grep -E 'ERROR|WARN'` — investigate anything new.
## Weekly
- [ ] `curl -fsS http://localhost:8080/api/admin/audit-log -b cookies.txt | jq '.events[] | select(.event | test("login_failed|backup.failed"))'` — review failed logins + backup failures.
- [ ] Confirm `docker compose exec backend ls -la /var/lib/cyclone/backups/` shows recent `.bin` files (within 25h of now).
## Quarterly
- [ ] Rotate the SQLCipher / cookie-signing key:
```bash
bash scripts/cyclone-init.sh --force # overwrites /etc/cyclone/secrets/db.key
docker compose restart backend # picks up the new key
```
Old `.bin` backups become unreadable after this; export them first if you need to keep them.
## As needed
- **Add an operator.** Log in as admin → `/admin/users` → Create user. Roles: `admin` / `user` / `viewer`.
- **Reset a password.** Admin UI → Users → Reset password, OR `docker compose exec backend python -m cyclone admin reset-password --username <name>`.
- **Restore from backup.** Admin UI → Backups → pick the snapshot → Initiate restore → Confirm. The backend will restart automatically.
- **Roll back the code (not the schema).** `TAG=0.0.9 docker compose up -d`. The previous image stays in the local Docker cache for one cycle.
- **Pull a new `:stable`.**
```bash
cd /opt/cyclone
docker compose pull
docker compose up -d
docker compose logs -f backend | head -200 # verify migrations + healthcheck
```
- **Off-box backup copy.** The operator is expected to rsync `/var/lib/docker/volumes/cyclone_backups/_data/` to an external drive or NAS nightly. The `.bin` files are already encrypted; the destination doesn't need its own encryption.
- **Inspect the DB.** `docker compose exec backend sqlite3 /var/lib/cyclone/db/cyclone.db ".tables"` (works only if SQLCipher key is on disk; the in-process decrypt happens via the cyclone backend).
## Annual
- [ ] Rotate the admin password (force re-login for everyone).
- [ ] Audit the `/etc/cyclone/secrets/` directory permissions — should be `chmod 600 root:root`.
- [ ] Review the audit log for stale admin sessions.
## Emergency
- **Backend won't start.** `docker compose logs --tail=300 backend`. Look for migration failures (rerun is safe — migrations are forward-only), SQLCipher key mismatch (`PRAGMA key` failure), or port collisions.
- **Frontend won't serve.** `docker compose logs --tail=100 frontend`. Usually nginx config drift; `docker compose restart frontend`.
- **Both unhealthy after a host reboot.** Docker may have come up before the named volumes did. `docker compose down && docker compose up -d`.
- **Suspected key compromise.** Rotate immediately (see Quarterly above). All active sessions are invalidated.
## Where things live
| Asset | Path |
|---|---|
| Docker compose file | `/opt/cyclone/docker-compose.yml` |
| Secrets | `/etc/cyclone/secrets/{db.key,admin_username,admin_pw}` |
| Live DB (SQLCipher-encrypted volume) | `cyclone_db` named volume, mounted at `/var/lib/cyclone/db` |
| Encrypted backups | `cyclone_backups` named volume, mounted at `/var/lib/cyclone/backups` |
| Uploaded prod files | `cyclone_prodfiles` named volume |
| SFTP staging stub | `cyclone_sftp_staging` named volume |
| Logs | `cyclone_logs` named volume + bind-mounted at `/var/log/cyclone` |
| Off-box backup destination | Operator's external drive / NAS (rsync cron, not in compose) |
+12
View File
@@ -0,0 +1,12 @@
.venv/
venv/
__pycache__/
*.py[cod]
*.egg-info/
.pytest_cache/
.ruff_cache/
tests/
docs/prodfiles/
*.production.txt
.git/
.github/
+74
View File
@@ -0,0 +1,74 @@
# syntax=docker/dockerfile:1.7
#
# Cyclone backend — FastAPI on python:3.11-slim-bookworm with sqlcipher.
#
# Two-stage build:
# 1. builder — wheels the package with [sqlcipher] extra into /wheels.
# 2. runtime — slim base, non-root user, tini PID 1, curl-based healthcheck.
#
# `sqlcipher` is preferred but the engine falls back to plain SQLite at
# runtime if the package isn't actually installed (see cyclone.db) — so a
# missing libsqlcipher-dev during build will fail loudly here rather than
# silently downgrading encryption in production.
# ---------- builder ----------
FROM python:3.11-slim-bookworm AS builder
ENV PIP_NO_CACHE_DIR=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PYTHONDONTWRITEBYTECODE=1
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
libffi-dev \
libsqlcipher-dev \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /build
# Copy the build manifest first so this layer caches across source edits.
COPY pyproject.toml ./
# Stub package so `pip wheel` can resolve the `cyclone` project without the
# source tree yet — required because pyproject.toml uses setuptools and
# references `src/cyclone`.
RUN mkdir -p src/cyclone && touch src/cyclone/__init__.py
RUN pip wheel --no-cache-dir --wheel-dir /wheels '.[sqlcipher]'
# Now copy the real source and rebuild so the wheel reflects the actual code.
COPY src/ ./src/
RUN pip wheel --no-cache-dir --wheel-dir /wheels '.[sqlcipher]'
# ---------- runtime ----------
FROM python:3.11-slim-bookworm
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
RUN apt-get update && apt-get install -y --no-install-recommends \
libsqlcipher-dev \
curl \
tini \
&& rm -rf /var/lib/apt/lists/* \
&& useradd --create-home --uid 1000 --shell /bin/bash cyclone
WORKDIR /app
COPY --from=builder /wheels /wheels
RUN pip install --no-cache-dir --no-index --find-links /wheels cyclone \
&& rm -rf /wheels
COPY --chown=cyclone:cyclone src/ /app/src/
USER cyclone
EXPOSE 8000
# Container-level healthcheck — the compose service healthcheck is
# effectively a duplicate but the Docker `HEALTHCHECK` directive keeps
# `docker ps` honest without needing compose to be running.
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
CMD curl -fs http://localhost:8000/api/healthz || exit 1
ENTRYPOINT ["tini", "--"]
CMD ["python", "-m", "cyclone", "serve"]
+27 -2
View File
@@ -11,6 +11,11 @@ Precedence:
2. Users table non-empty — no-op. 2. Users table non-empty — no-op.
3. ``CYCLONE_ADMIN_USERNAME`` + ``CYCLONE_ADMIN_PASSWORD`` env vars set 3. ``CYCLONE_ADMIN_USERNAME`` + ``CYCLONE_ADMIN_PASSWORD`` env vars set
(password >= 12 chars) — create the admin and print confirmation. (password >= 12 chars) — create the admin and print confirmation.
Each env var can also be replaced by a ``*_FILE`` companion
(``CYCLONE_ADMIN_USERNAME_FILE`` / ``CYCLONE_ADMIN_PASSWORD_FILE``)
that points at a file on disk — the standard Docker-secret pattern,
used in production to avoid embedding secrets in ``docker-compose.yml``.
``_FILE`` takes precedence when set.
4. Otherwise — raise ``RuntimeError`` with a remediation hint that 4. Otherwise — raise ``RuntimeError`` with a remediation hint that
points operators at ``python -m cyclone users create``. points operators at ``python -m cyclone users create``.
""" """
@@ -18,6 +23,7 @@ Precedence:
from __future__ import annotations from __future__ import annotations
import os import os
from pathlib import Path
from sqlalchemy import select from sqlalchemy import select
@@ -27,6 +33,21 @@ from cyclone.auth.permissions import Role
from cyclone.db import SessionLocal, User from cyclone.db import SessionLocal, User
def _read_secret(env_var: str, file_var: str) -> str | None:
"""Read a secret from a ``*_FILE`` env var (Docker-secret pattern) first,
falling back to the plain env var. Returns None if neither is set.
"""
file_path = os.environ.get(file_var)
if file_path:
try:
return Path(file_path).read_text().strip()
except OSError as exc:
raise RuntimeError(
f"failed to read {file_var}={file_path}: {exc}"
) from exc
return os.environ.get(env_var)
def run() -> None: def run() -> None:
"""Bootstrap the first admin user, or no-op. """Bootstrap the first admin user, or no-op.
@@ -42,8 +63,12 @@ def run() -> None:
_deps.AUTH_DISABLED = True _deps.AUTH_DISABLED = True
return return
username = os.environ.get("CYCLONE_ADMIN_USERNAME") username = _read_secret(
password = os.environ.get("CYCLONE_ADMIN_PASSWORD") "CYCLONE_ADMIN_USERNAME", "CYCLONE_ADMIN_USERNAME_FILE"
)
password = _read_secret(
"CYCLONE_ADMIN_PASSWORD", "CYCLONE_ADMIN_PASSWORD_FILE"
)
# First-boot fix: ``python -m cyclone`` calls bootstrap before any # First-boot fix: ``python -m cyclone`` calls bootstrap before any
# subcommand or the FastAPI lifespan handler runs, so on a brand-new # subcommand or the FastAPI lifespan handler runs, so on a brand-new
+131
View File
@@ -0,0 +1,131 @@
"""Auth bootstrap reads CYCLONE_ADMIN_*_FILE env vars when set.
Mirrors the Docker-secret posture in ``docker-compose.yml``: secrets
mounted at ``/run/secrets/<name>`` with the ``*_FILE`` env var pointing
at the path. The bootstrap should prefer the file over the bare env var
when both are present (file = Docker secret wins).
"""
from __future__ import annotations
from pathlib import Path
import pytest
from cyclone import db
from cyclone.auth import bootstrap, users
from cyclone.auth.permissions import Role
@pytest.fixture
def tmp_db(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
db_path = tmp_path / "test.db"
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{db_path}")
db.init_db()
return db_path
def _write_secrets(tmp_path: Path, *, username: str, password: str) -> tuple[Path, Path]:
user_file = tmp_path / "admin_username"
pw_file = tmp_path / "admin_pw"
user_file.write_text(f"{username}\n")
pw_file.write_text(f"{password}\n")
return user_file, pw_file
def test_bootstrap_creates_admin_from_file_env_vars(
tmp_path: Path, tmp_db: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
user_file, pw_file = _write_secrets(
tmp_path, username="deploy-admin", password="super-secret-password-123"
)
monkeypatch.delenv("CYCLONE_ADMIN_USERNAME", raising=False)
monkeypatch.delenv("CYCLONE_ADMIN_PASSWORD", raising=False)
monkeypatch.setenv("CYCLONE_ADMIN_USERNAME_FILE", str(user_file))
monkeypatch.setenv("CYCLONE_ADMIN_PASSWORD_FILE", str(pw_file))
bootstrap.run()
with db.SessionLocal()() as session:
user = users.get_by_username(session, "deploy-admin")
assert user is not None
assert user.role == Role.ADMIN.value
assert users.verify_password("super-secret-password-123", user.password_hash)
def test_file_env_var_overrides_plain_env_var(
tmp_path: Path, tmp_db: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""When both _FILE and bare env vars are set, _FILE wins."""
user_file, pw_file = _write_secrets(
tmp_path, username="file-user", password="file-password-12345"
)
monkeypatch.setenv("CYCLONE_ADMIN_USERNAME", "env-user")
monkeypatch.setenv("CYCLONE_ADMIN_PASSWORD", "env-password-12345")
monkeypatch.setenv("CYCLONE_ADMIN_USERNAME_FILE", str(user_file))
monkeypatch.setenv("CYCLONE_ADMIN_PASSWORD_FILE", str(pw_file))
bootstrap.run()
with db.SessionLocal()() as session:
file_user = users.get_by_username(session, "file-user")
env_user = users.get_by_username(session, "env-user")
assert file_user is not None
assert env_user is None, "bare env var should be ignored when _FILE is set"
def test_bootstrap_falls_back_to_plain_env_var(
tmp_path: Path, tmp_db: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.delenv("CYCLONE_ADMIN_USERNAME_FILE", raising=False)
monkeypatch.delenv("CYCLONE_ADMIN_PASSWORD_FILE", raising=False)
monkeypatch.setenv("CYCLONE_ADMIN_USERNAME", "env-user")
monkeypatch.setenv("CYCLONE_ADMIN_PASSWORD", "env-password-12345")
bootstrap.run()
with db.SessionLocal()() as session:
user = users.get_by_username(session, "env-user")
assert user is not None
assert user.role == Role.ADMIN.value
def test_bootstrap_strips_trailing_whitespace_from_file(
tmp_path: Path, tmp_db: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Secret files often have a trailing newline from `printf`/`echo`.
The bootstrap must strip it so bcrypt verify doesn't see whitespace."""
user_file = tmp_path / "admin_username"
pw_file = tmp_path / "admin_pw"
user_file.write_text("deploy-admin\n\n")
pw_file.write_text("super-secret-password-123\n")
monkeypatch.delenv("CYCLONE_ADMIN_USERNAME", raising=False)
monkeypatch.delenv("CYCLONE_ADMIN_PASSWORD", raising=False)
monkeypatch.setenv("CYCLONE_ADMIN_USERNAME_FILE", str(user_file))
monkeypatch.setenv("CYCLONE_ADMIN_PASSWORD_FILE", str(pw_file))
bootstrap.run()
with db.SessionLocal()() as session:
user = users.get_by_username(session, "deploy-admin")
assert user is not None
# Should verify cleanly with no trailing whitespace.
assert users.verify_password("super-secret-password-123", user.password_hash)
assert not users.verify_password(
"super-secret-password-123\n", user.password_hash
)
def test_bootstrap_raises_when_file_path_missing(
tmp_path: Path, tmp_db: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.delenv("CYCLONE_ADMIN_USERNAME", raising=False)
monkeypatch.delenv("CYCLONE_ADMIN_PASSWORD", raising=False)
monkeypatch.setenv(
"CYCLONE_ADMIN_USERNAME_FILE", str(tmp_path / "does-not-exist")
)
monkeypatch.setenv(
"CYCLONE_ADMIN_PASSWORD_FILE", str(tmp_path / "also-missing")
)
with pytest.raises(RuntimeError, match="failed to read"):
bootstrap.run()
+227
View File
@@ -0,0 +1,227 @@
"""Dockerfile + compose-config smoke tests for SP23.
These tests do NOT require a running Docker daemon (the compose-up test
is gated on ``DOCKER_TESTS=1`` so it can be skipped on bare CI without
Docker). They validate that:
* ``docker-compose.yml`` at the repo root is syntactically valid and that
the shape we expect (services, secrets, volumes, networks) is present.
* Both Dockerfiles parse with ``docker build --check`` if Docker is on PATH.
* The named-volume mount paths match what ``cyclone.db`` + the BackupService
expect at runtime.
"""
from __future__ import annotations
import os
import shutil
import subprocess
from pathlib import Path
import pytest
import yaml
REPO_ROOT = Path(__file__).resolve().parents[2]
COMPOSE_FILE = REPO_ROOT / "docker-compose.yml"
BACKEND_DOCKERFILE = REPO_ROOT / "backend" / "Dockerfile"
FRONTEND_DOCKERFILE = REPO_ROOT / "frontend" / "Dockerfile"
def _has_docker() -> bool:
return shutil.which("docker") is not None
def _has_docker_compose() -> bool:
if shutil.which("docker") is None:
return False
return (
subprocess.run(
["docker", "compose", "version"],
capture_output=True,
check=False,
).returncode
== 0
)
def _volume_source(v) -> str:
"""Normalize a compose volume entry to the source name (str form or 'source' key)."""
if isinstance(v, str):
# Long form: "named_volume:/container/path" — split on ':'.
return v.split(":", 1)[0]
if isinstance(v, dict):
return v.get("source", "")
return ""
def test_compose_file_exists():
assert COMPOSE_FILE.exists(), f"missing {COMPOSE_FILE}"
def test_compose_config_validates():
"""``docker compose config`` should exit 0 with no stderr."""
if not _has_docker_compose():
pytest.skip("docker compose not on PATH")
result = subprocess.run(
[
"docker",
"compose",
"-f",
str(COMPOSE_FILE),
"config",
"--quiet",
],
capture_output=True,
text=True,
cwd=REPO_ROOT,
)
assert result.returncode == 0, (
f"compose config failed: stderr={result.stderr!r}"
)
def test_compose_declares_required_services():
compose = yaml.safe_load(COMPOSE_FILE.read_text())
services = compose.get("services", {})
assert "backend" in services, "compose must declare a 'backend' service"
assert "frontend" in services, "compose must declare a 'frontend' service"
backend = services["backend"]
backend_volume_sources = {_volume_source(v) for v in backend.get("volumes", [])}
assert "cyclone_db" in backend_volume_sources, (
"backend must mount the cyclone_db volume"
)
assert backend.get("restart") == "unless-stopped", (
"backend must restart: unless-stopped so healthcheck failures recover"
)
assert "healthcheck" in backend, "backend must declare a healthcheck"
frontend = services["frontend"]
assert "8080:8080" in frontend.get("ports", []), (
"frontend must publish 8080:8080 for LAN access"
)
depends_on = frontend.get("depends_on") or {}
if isinstance(depends_on, dict):
backend_dep = depends_on.get("backend") or {}
assert backend_dep.get("condition") == "service_healthy", (
"frontend must wait for backend healthy before starting"
)
else:
# Short-form `depends_on: [backend]` is acceptable too — it implies
# service_started, not service_healthy. Flag a soft warning.
pytest.skip(
"frontend uses short-form depends_on; switch to long-form for service_healthy"
)
def test_compose_declares_required_secrets_and_volumes():
compose = yaml.safe_load(COMPOSE_FILE.read_text())
secrets = compose.get("secrets", {})
for required in ("cyclone_db_key", "cyclone_admin_password"):
assert required in secrets, f"compose must declare secret {required!r}"
volumes = compose.get("volumes", {})
for required in (
"cyclone_db",
"cyclone_backups",
"cyclone_prodfiles",
"cyclone_sftp_staging",
"cyclone_logs",
):
assert required in volumes, f"compose must declare volume {required!r}"
def test_compose_backend_wires_backup_autostart():
"""The existing BackupService (SP17) needs CYCLONE_BACKUP_AUTOSTART=1
on container boot. The compose env block must include it."""
compose = yaml.safe_load(COMPOSE_FILE.read_text())
env = compose["services"]["backend"].get("environment", {})
assert str(env.get("CYCLONE_BACKUP_AUTOSTART")) == "1", (
"backend must autostart the backup scheduler (CYCLONE_BACKUP_AUTOSTART=1)"
)
assert "CYCLONE_BACKUP_INTERVAL_HOURS" in env
assert "CYCLONE_BACKUP_RETENTION_DAYS" in env
@pytest.mark.skipif(
not _has_docker(), reason="docker not on PATH; skipping Dockerfile parse check"
)
def test_backend_dockerfile_parses():
result = subprocess.run(
[
"docker",
"build",
"--check",
"-f",
str(BACKEND_DOCKERFILE),
str(REPO_ROOT / "backend"),
],
capture_output=True,
text=True,
)
assert result.returncode == 0, (
f"backend Dockerfile failed to parse: stderr={result.stderr!r}"
)
@pytest.mark.skipif(
not _has_docker(), reason="docker not on PATH; skipping Dockerfile parse check"
)
def test_frontend_dockerfile_parses():
result = subprocess.run(
[
"docker",
"build",
"--check",
"-f",
str(FRONTEND_DOCKERFILE),
str(REPO_ROOT / "frontend"),
],
capture_output=True,
text=True,
)
assert result.returncode == 0, (
f"frontend Dockerfile failed to parse: stderr={result.stderr!r}"
)
@pytest.mark.skipif(
not _has_docker_compose()
or not os.environ.get("DOCKER_TESTS"),
reason="DOCKER_TESTS=1 + docker compose required for live bring-up",
)
def test_compose_up_brings_up_healthy_stack():
"""Gated live test — only runs when DOCKER_TESTS=1 and docker compose
is available. Builds + brings up the full stack and waits up to 120s
for the backend healthcheck to come up healthy."""
subprocess.run(
["docker", "compose", "-f", str(COMPOSE_FILE), "up", "-d", "--build"],
check=True,
cwd=REPO_ROOT,
)
try:
import time
for _ in range(60):
ps = subprocess.run(
[
"docker",
"compose",
"-f",
str(COMPOSE_FILE),
"ps",
"--format",
"json",
],
capture_output=True,
text=True,
cwd=REPO_ROOT,
)
if "healthy" in ps.stdout:
return
time.sleep(2)
pytest.fail("compose stack did not become healthy within 120s")
finally:
subprocess.run(
["docker", "compose", "-f", str(COMPOSE_FILE), "down", "-v"],
check=False,
cwd=REPO_ROOT,
)
+75 -78
View File
@@ -1,101 +1,98 @@
# Cyclone — local docker-compose stack.
#
# Two services on a user-defined network:
#
# frontend (nginx) — published on http://127.0.0.1:8080
# serves the built SPA + reverse-proxies /api/* to backend
# backend (uvicorn) — NOT published externally by default; reachable from
# the frontend over the compose network on backend:8000
# (override `ports:` below to expose it for curl/debug)
#
# Persistent state:
# - `cyclone-data` named volume mounted at /data in the backend holds
# the SQLite database file. Survives `docker compose down`; only
# `docker compose down -v` wipes it.
# - `config/payers.yaml` is baked into the backend image at build time.
# To edit payer config, change the YAML, rebuild the backend image,
# and `POST /api/admin/reload-config` (or just restart the container).
#
# Usage:
# docker compose up -d --build # start (or rebuild + restart)
# docker compose logs -f # tail both services
# docker compose restart backend # bounce the backend (e.g. after crash)
# docker compose down # stop containers, KEEP the data volume
# docker compose down -v # stop AND wipe the data volume
name: cyclone name: cyclone
# Two-container production stack for Cyclone.
#
# backend FastAPI on python:3.11-slim-bookworm, internal port 8000.
# frontend nginx:1.27-alpine serving the built React SPA + reverse-
# proxying /api/* to the backend, host port 8080.
#
# Tag strategy:
# - Images are tagged semver from package.json / pyproject.toml.
# - Two aliases per image: `:latest` (set automatically by `docker
# compose build`) and `:stable` (manually promoted after a release
# has been running in prod for >=1 week).
# - Compose defaults to `:stable` so a fresh `docker compose build`
# never auto-rolls a new release.
# - Override with `TAG=0.0.9 docker compose up -d` to roll back.
services: services:
backend: backend:
image: cyclone-backend:${TAG:-stable}
build: build:
context: . context: ./backend
dockerfile: backend/Dockerfile
image: cyclone-backend:local
container_name: cyclone-backend
restart: unless-stopped restart: unless-stopped
environment: environment:
# Bind on all interfaces inside the container (required — 127.0.0.1 # SQLite path lives on the named `cyclone_db` volume.
# would only be reachable from inside the container itself). CYCLONE_DB_URL: "sqlite:////var/lib/cyclone/db/cyclone.db"
CYCLONE_HOST: "0.0.0.0" # Wire the existing BackupService (SP17) to autostart on container boot.
CYCLONE_PORT: "8000" CYCLONE_BACKUP_AUTOSTART: "1"
CYCLONE_RELOAD: "0" CYCLONE_BACKUP_INTERVAL_HOURS: "24"
# Absolute path inside the container; the named volume mounts at /data. CYCLONE_BACKUP_RETENTION_DAYS: "14"
CYCLONE_DB_URL: "sqlite:////data/cyclone.db" # Logging — JSON to stdout (collected by `docker logs`) and to the
# Bootstrap admin (required on first boot unless at least one user # bind-mounted file (collected by host-side logrotate).
# already exists). The ${VAR:?msg} syntax makes docker-compose refuse CYCLONE_LOG_LEVEL: "INFO"
# to start with a clear error if the env var isn't set in the host CYCLONE_LOG_FILE: "/var/log/cyclone/cyclone.log"
# environment. CYCLONE_LOG_JSON: "1"
CYCLONE_ADMIN_USERNAME: ${CYCLONE_ADMIN_USERNAME:?CYCLONE_ADMIN_USERNAME is required on first boot} # Cookie signing key defaults to the SQLCipher key. Override in
CYCLONE_ADMIN_PASSWORD: ${CYCLONE_ADMIN_PASSWORD:?CYCLONE_ADMIN_PASSWORD is required on first boot (min 12 chars)} # production if you want to rotate them independently.
CYCLONE_SECRET_KEY_FILE: "/run/secrets/cyclone_db_key"
CYCLONE_COOKIE_SECURE: "1"
# First-admin bootstrap — populated by scripts/cyclone-init.sh.
# The `_FILE` variants are the standard Docker-secret pattern:
# the env var points at the mounted secret file rather than
# embedding the secret in the compose file.
CYCLONE_ADMIN_USERNAME_FILE: "/run/secrets/cyclone_admin_username"
CYCLONE_ADMIN_PASSWORD_FILE: "/run/secrets/cyclone_admin_password"
secrets:
- cyclone_db_key
- cyclone_admin_username
- cyclone_admin_password
volumes: volumes:
- cyclone-data:/data - cyclone_db:/var/lib/cyclone/db
# The healthcheck in the Dockerfile hits /api/health. The frontend - cyclone_backups:/var/lib/cyclone/backups
# depends_on `service_healthy` so it won't accept traffic until the - cyclone_prodfiles:/var/lib/cyclone/prodfiles
# backend is responsive. - cyclone_sftp_staging:/var/lib/cyclone/sftp_staging
- cyclone_logs:/var/log/cyclone
healthcheck: healthcheck:
test: ["CMD", "curl", "--fail", "--silent", "http://127.0.0.1:8000/api/health"] test: ["CMD", "curl", "-fs", "http://localhost:8000/api/healthz"]
interval: 15s interval: 30s
timeout: 5s timeout: 5s
retries: 3 retries: 3
start_period: 10s start_period: 30s
# No `ports:` — the frontend reaches the backend over the compose
# network. Uncomment the next line to expose the API directly for
# `curl http://localhost:8000/api/...` from the host.
# ports:
# - "127.0.0.1:8000:8000"
networks: networks:
- cyclone-net - cyclone_network
frontend: frontend:
image: cyclone-frontend:${TAG:-stable}
build: build:
context: . context: ./frontend
dockerfile: frontend/Dockerfile
image: cyclone-frontend:local
container_name: cyclone-frontend
restart: unless-stopped restart: unless-stopped
ports:
# LAN-bind: only this port is published to the host. The operator
# is expected to firewall this to the LAN subnet (or rely on VPN
# for outside access). NO public TLS in v1.
- "8080:8080"
depends_on: depends_on:
backend: backend:
condition: service_healthy condition: service_healthy
ports:
# Bind address defaults to 0.0.0.0 (reachable from the LAN). Set
# CYCLONE_BIND_ADDRESS=127.0.0.1 to tighten to loopback-only and
# match the standalone install's local-only posture.
# Port defaults to 8081 to dodge the common clash on 8080; override
# with CYCLONE_WEB_PORT=... (see README).
- "${CYCLONE_BIND_ADDRESS:-0.0.0.0}:${CYCLONE_WEB_PORT:-8081}:80"
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://127.0.0.1/healthz"]
interval: 15s
timeout: 3s
retries: 3
start_period: 5s
networks: networks:
- cyclone-net - cyclone_network
networks: secrets:
cyclone-net: cyclone_db_key:
driver: bridge file: /etc/cyclone/secrets/db.key
cyclone_admin_username:
file: /etc/cyclone/secrets/admin_username
cyclone_admin_password:
file: /etc/cyclone/secrets/admin_pw
volumes: volumes:
cyclone-data: cyclone_db:
name: cyclone-data cyclone_backups:
cyclone_prodfiles:
cyclone_sftp_staging:
cyclone_logs:
networks:
cyclone_network:
driver: bridge
File diff suppressed because it is too large Load Diff
@@ -1,12 +1,28 @@
# Cyclone Ubuntu Docker Deployment — Design # Cyclone Ubuntu Docker Deployment — Design
**Date:** 2026-06-22 **Date:** 2026-06-22 (spec drafted); 2026-06-23 (reconciled with auth work that landed in main on 2026-06-23)
**Status:** Approved (pending user review of this doc) **Status:** Approved (2026-06-23)
**Depends on:** [2026-06-19-cyclone-production-readiness-design.md](2026-06-19-cyclone-production-readiness-design.md) (in-memory store + react-query wiring; local-only) **Branch:** `sp23-ubuntu-docker-deployment`
**Aesthetic direction:** No new UI surface in v1 — the nginx + reverse-proxy posture is invisible to the operator; auth + dashboard styling is unchanged from what's already on main.
**Depends on:** [2026-06-19-cyclone-production-readiness-design.md](2026-06-19-cyclone-production-readiness-design.md) (in-memory store + react-query wiring; local-only); the auth work that merged into `main` on 2026-06-23 (`a25504b`..`39ae988`, the `feat(auth):` series).
**Replaces:** The "no auth, no Docker, local-only" posture of the parent spec with a production-grade posture for real PHI on a single Ubuntu server. **Replaces:** The "no auth, no Docker, local-only" posture of the parent spec with a production-grade posture for real PHI on a single Ubuntu server.
--- ---
## 1.1 Delta vs. `main` at 2026-06-23
The auth work that landed in main before SP23 began means sections 6 (auth modules + routes + matrix) and 7 (frontend login + AuthProvider + RoleGate) describe code that **already exists on main**. SP23's delta is therefore narrower than the original draft:
- **§6 (auth) — already on main.** The `cyclone.auth` package, `users` / `sessions` SQLAlchemy models (migrations `0013` + `0014`), bcrypt password hashing, the `PERMISSIONS` matrix + `matrix_gate` app-wide dependency, `Login.tsx` + `AuthProvider` + `RequireAuth` + `RoleGate`, the `cyclone admin` CLI, and `CYCLONE_AUTH_DISABLED=1` escape hatch are all merged. SP23 **reuses** them; it does not re-implement. §6 is kept in this spec as the contract that Docker + secrets + compose env vars must satisfy, not as work to do.
- **§7 (login UI) — already on main.** Reused.
- **§6.6 (new migration) — N/A.** No new SQL migration is needed for SP23; auth tables already exist.
- **What SP23 adds (the actual work):** §6.8 backend Dockerfile, §7.8 frontend Dockerfile + nginx.conf, §8.2 Docker secrets wiring, §9.4 image tag strategy + `:stable` aliases, §10 docker-compose.yml, §11 healthcheck + restart policy, §12.2 `tests/test_docker.py`, §9.5 bootstrap script, §9.6 RUNBOOK.md, post-deploy + smoke scripts.
- **Role name reconciliation:** the original draft used `admin / operator / viewer`. Main uses `admin / user / viewer`. All references to `operator` in this spec should be read as `user` against the live codebase.
- **Hashing algorithm reconciliation:** the original draft specified `argon2id`. Main uses `bcrypt`. The Docker secret posture is unaffected — both algorithms need only a key file or a salt, neither of which SP23 introduces.
- **Lockout timing:** original draft was 5 fails / 15 min. Auth on main is 5 fails / 5 min (`cyclone.auth.rate_limit`). SP23 inherits the main behavior.
---
## 1. Overview ## 1. Overview
Cyclone's first sub-project shipped a usable local-only system: parse 837/835 files, browse the data, edit and resubmit rejected claims, export a corrected 837 ZIP. Everything ran on `127.0.0.1` with no auth, in-memory storage, and a `python -m cyclone serve` invocation. Cyclone's first sub-project shipped a usable local-only system: parse 837/835 files, browse the data, edit and resubmit rejected claims, export a corrected 837 ZIP. Everything ran on `127.0.0.1` with no auth, in-memory storage, and a `python -m cyclone serve` invocation.
+10
View File
@@ -0,0 +1,10 @@
node_modules/
dist/
build/
*.tsbuildinfo
coverage/
src/**/*.test.ts
src/**/*.test.tsx
src/test/
.git/
.github/
+33
View File
@@ -0,0 +1,33 @@
# syntax=docker/dockerfile:1.7
#
# Cyclone frontend — React SPA built with node:20-alpine and served by
# nginx:1.27-alpine. nginx reverse-proxies /api/* to the backend service
# over the compose-managed bridge network.
# ---------- builder ----------
FROM node:20-alpine AS builder
WORKDIR /build
# Install deps first so this layer caches across source edits.
COPY package.json package-lock.json* ./
RUN npm ci
# Build the production bundle into dist/.
COPY . .
RUN npm run build
# ---------- runtime ----------
FROM nginx:1.27-alpine
# Replace the default nginx site with ours (SPA + reverse proxy).
RUN rm -f /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=builder /build/dist /usr/share/nginx/html
# wget is on busybox; nginx:alpine doesn't ship curl.
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
CMD wget -qO- http://localhost:8080/ >/dev/null || exit 1
EXPOSE 8080
+41
View File
@@ -0,0 +1,41 @@
server {
listen 8080;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Long-lived connections for the live-tail NDJSON streams
# (snapshot + live events can hold open for minutes at a time).
proxy_read_timeout 300s;
proxy_send_timeout 300s;
# Claims files can be large; 50 MiB matches what FastAPI accepts.
client_max_body_size 50m;
# API + auth: proxy to backend over the compose-managed bridge network.
location /api/ {
proxy_pass http://cyclone-backend:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# The backend sets the cookie with Path=/api/. Rewrite to / so
# the browser sends it on every subsequent request to the SPA
# (which never has a /api/ prefix on the URL).
proxy_cookie_path /api/ /;
}
# SPA fallback: serve index.html for any non-/api route so deep-links
# to /claims/123, /admin/users, etc. round-trip through reload.
location / {
try_files $uri $uri/ /index.html;
}
# Don't cache index.html — the SPA references hashed asset filenames,
# so index.html is the only file that needs to be revalidated.
location = /index.html {
add_header Cache-Control "no-cache, no-store, must-revalidate" always;
expires off;
}
}
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env bash
# cyclone-init.sh — first-time host bootstrap for a production Cyclone deploy.
#
# Generates /etc/cyclone/secrets/{db.key,admin_username,admin_pw} with
# openssl rand -hex 32, sets ownership + permissions, and prints the
# admin password ONCE for the operator to capture.
#
# Idempotent: refuses to overwrite existing secrets unless --force.
#
# Usage:
# bash scripts/cyclone-init.sh # generate if missing
# bash scripts/cyclone-init.sh --force # overwrite (destroys old data!)
set -euo pipefail
SECRETS_DIR="${CYCLONE_SECRETS_DIR:-/etc/cyclone/secrets}"
FORCE=0
[[ "${1:-}" == "--force" ]] && FORCE=1
if [[ $EUID -ne 0 ]]; then
echo "ERROR: cyclone-init.sh must run as root (it writes to ${SECRETS_DIR})." >&2
exit 1
fi
if [[ -f "${SECRETS_DIR}/db.key" && $FORCE -eq 0 ]]; then
echo "Secrets already exist at ${SECRETS_DIR}. Re-run with --force to overwrite." >&2
exit 0
fi
mkdir -p "${SECRETS_DIR}"
chmod 700 "${SECRETS_DIR}"
# 64-hex-char (256-bit) random key for SQLCipher + cookie signing.
openssl rand -hex 32 > "${SECRETS_DIR}/db.key"
# Admin username defaults to "admin" unless CYCLONE_ADMIN_USERNAME is set.
ADMIN_USER="${CYCLONE_ADMIN_USERNAME:-admin}"
printf '%s' "${ADMIN_USER}" > "${SECRETS_DIR}/admin_username"
# Memorable-but-random admin password — printed once.
ADMIN_PW="$(openssl rand -base64 18 | tr -d '/+=' | head -c 20)Aa1!"
printf '%s' "${ADMIN_PW}" > "${SECRETS_DIR}/admin_pw"
chmod 600 "${SECRETS_DIR}"/*
chown -R root:root "${SECRETS_DIR}"
cat <<EOF
================================================================
Cyclone secrets generated at ${SECRETS_DIR}.
db.key SQLCipher key + cookie signing (64 hex chars)
admin_username ${ADMIN_USER}
admin_pw ${ADMIN_PW} <-- capture this NOW; it won't be shown again.
Next steps:
cd /opt/cyclone
docker compose pull # or: docker compose build
docker compose up -d
bash scripts/post-deploy.sh # logrotate + healthcheck cron
bash scripts/smoke.sh # end-to-end smoke test
Then open http://<lan-ip>:8080/login and log in as '${ADMIN_USER}'.
Change the password from /admin/users immediately.
================================================================
EOF
+53
View File
@@ -0,0 +1,53 @@
#!/usr/bin/env bash
# post-deploy.sh — set up host-side logrotate and healthcheck cron for a
# production Cyclone deploy. Run once after `docker compose up -d`.
set -euo pipefail
LOG_DIR="/var/log/cyclone"
HEALTHCHECK_URL="${CYCLONE_HEALTHCHECK_URL:-http://localhost:8080/api/healthz}"
HEALTHCHECK_EMAIL="${CYCLONE_HEALTHCHECK_EMAIL:-root}"
CRON_USER="${CYCLONE_CRON_USER:-root}"
if [[ $EUID -ne 0 ]]; then
echo "ERROR: post-deploy.sh must run as root." >&2
exit 1
fi
# 1. Logrotate — keeps 14 days of JSON logs compressed.
cat > /etc/logrotate.d/cyclone <<'LOGROTATE'
/var/log/cyclone/*.log {
daily
rotate 14
compress
delaycompress
missingok
notifempty
copytruncate
dateext
dateformat -%Y%m%d
}
LOGROTATE
mkdir -p "${LOG_DIR}"
chmod 755 "${LOG_DIR}"
# 2. Healthcheck cron — every 5 minutes, email if down.
CRON_LINE="*/5 * * * * curl -fsS --max-time 10 ${HEALTHCHECK_URL} >/dev/null 2>&1 || echo 'Cyclone DOWN at \$(date)' | mail -s 'Cyclone healthz FAIL' ${HEALTHCHECK_EMAIL}"
TMP_CRON="$(mktemp)"
crontab -u "${CRON_USER}" -l 2>/dev/null > "${TMP_CRON}" || true
# Remove any existing cyclone healthcheck line, then re-add.
grep -v -F "Cyclone healthz" "${TMP_CRON}" > "${TMP_CRON}.new" || cp "${TMP_CRON}" "${TMP_CRON}.new"
echo "${CRON_LINE}" >> "${TMP_CRON}.new"
crontab -u "${CRON_USER}" "${TMP_CRON}.new"
rm -f "${TMP_CRON}" "${TMP_CRON}.new"
cat <<EOF
post-deploy.sh complete:
- logrotate installed at /etc/logrotate.d/cyclone
- healthcheck cron for ${HEALTHCHECK_URL} installed for user '${CRON_USER}'
- logs rotate daily, 14 days retention
Verify with:
logrotate -d /etc/logrotate.d/cyclone
crontab -u ${CRON_USER} -l | grep -i cyclone
EOF
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env bash
# smoke.sh — end-to-end bring-up + login + parse + export smoke test.
#
# Assumes:
# - `docker compose up -d` is running.
# - /etc/cyclone/secrets/admin_pw exists (or CYCLONE_SECRETS_DIR overrides).
# - The sample 837 at docs/prodfiles/co_medicaid/sample_837p.txt is present.
#
# Override the URL with CYCLONE_SMOKE_URL, the sample path with
# CYCLONE_SMOKE_SAMPLE, the secrets dir with CYCLONE_SECRETS_DIR.
set -euo pipefail
BASE_URL="${CYCLONE_SMOKE_URL:-http://localhost:8080}"
SECRETS_DIR="${CYCLONE_SECRETS_DIR:-/etc/cyclone/secrets}"
ADMIN_PW="$(cat "${SECRETS_DIR}/admin_pw")"
COOKIE_JAR="$(mktemp)"
SAMPLE_837="${CYCLONE_SMOKE_SAMPLE:-docs/prodfiles/co_medicaid/sample_837p.txt}"
cleanup() { rm -f "${COOKIE_JAR}" /tmp/cyclone_smoke_export.zip; }
trap cleanup EXIT
echo "==> waiting for healthz..."
for i in {1..30}; do
if curl -fsS "${BASE_URL}/api/healthz" >/dev/null 2>&1; then break; fi
sleep 2
[[ $i -eq 30 ]] && { echo "FAIL: healthz never came up"; exit 1; }
done
echo " ok"
echo "==> logging in as admin..."
HTTP_CODE=$(curl -sS -o /dev/null -w '%{http_code}' \
-c "${COOKIE_JAR}" \
-H 'Content-Type: application/json' \
-X POST "${BASE_URL}/api/auth/login" \
-d "{\"username\":\"admin\",\"password\":\"${ADMIN_PW}\"}")
[[ "${HTTP_CODE}" == "200" ]] || { echo "FAIL: login returned ${HTTP_CODE}"; exit 1; }
echo " ok (cookie set)"
echo "==> /api/auth/me..."
ME=$(curl -fsS -b "${COOKIE_JAR}" "${BASE_URL}/api/auth/me")
echo " ${ME}"
if [[ -f "${SAMPLE_837}" ]]; then
echo "==> parsing sample 837 (${SAMPLE_837})..."
PARSE=$(curl -fsS -b "${COOKIE_JAR}" -X POST "${BASE_URL}/api/parse-837" \
-F "file=@${SAMPLE_837}")
echo " ${PARSE}" | head -c 200
echo
else
echo "==> skipping 837 parse (sample not found at ${SAMPLE_837})"
fi
echo "==> listing batches..."
BATCHES=$(curl -fsS -b "${COOKIE_JAR}" "${BASE_URL}/api/batches")
echo " ${BATCHES}" | head -c 200
echo
echo "smoke: OK"