feat(sp17): encrypted DB backup automation
Adds automated encrypted backups of the live SQLite file. Closes the 'no backup automation' gap called out in the completeness review (docs/reviews/2026-06-20-cyclone-completeness-review.md §3.1 #3) and gives the SP16 MFT scheduler a recovery path when the MFT pipeline loses days of inbound 999/277CA work in a single crash. Architecture ------------ - AES-256-GCM with PBKDF2-HMAC-SHA256 (200,000 iters, 16-byte salt) - Online backups via SQLite's .backup() API — no app downtime - Salt + passphrase persisted to macOS Keychain (separate accounts backup.passphrase + backup.salt) so the key is reproducible across processes - Two-step restore (initiate → confirm) with a one-shot 64-char hex token; the second call disposes + rebuilds the engine only if the token matches within a 5-minute TTL - Tamper-evident audit chain (SP11) — db.backup_created, db.backup_failed, db.backup_pruned, db.backup_restored, db.backup_passphrase_set - BackupService + BackupScheduler + module-level singletons - 8 admin endpoints + 6 CLI subcommands - Auto-start opt-in via CYCLONE_BACKUP_AUTOSTART=true; default interval 24h, default retention 30 days - Fallback posture: if no separate passphrase is set and SQLCipher is enabled, the key is derived from the SQLCipher DB key with a fixed salt + WARNING log (degraded but never plaintext) New modules ----------- - cyclone.backup — PBKDF2, AES-GCM, sidecar format - cyclone.backup_service — create_now / list / verify / restore / prune / status - cyclone.backup_scheduler — async tick loop with audit hooks New surface ----------- - 8 admin endpoints under /api/admin/backup/* - 6 CLI subcommands under cyclone backup (init-passphrase, create, list, verify, restore, prune, status) - Migration 0012_backups.sql + DbBackup ORM - store.add_backup_pending() Tests ----- - 14 unit tests in test_backup_crypto.py (key derivation, encrypt/ decrypt round-trip, tampered ciphertext, wrong passphrase, sidecar round-trip, filename format) - 19 tests in test_backup_service.py (create/list/verify/restore/ prune/status, error handling, fallback key, module singleton) - 14 API tests in test_api_backup.py (all 8 endpoints + scheduler endpoints, two-step restore, error responses) - 10 tests in test_backup_scheduler.py (tick / start / stop / audit / coalescing / module singleton) - 5 CLI tests in test_cli_backup.py (create / list / verify / restore confirm prompt / prune confirm prompt / init-passphrase minimum-length check) Total new tests: 62. All pass. Full backend suite: 833 passed, 9 skipped (gitignored prodfiles), 1 warning. Design doc: docs/superpowers/specs/2026-06-21-cyclone-encrypted-backup-design.md README: new 'Encrypted Backups (SP17)' section, SP17 entry in Roadmap, retention default documented in §Project layout.
This commit is contained in:
Tyler
@@ -357,6 +357,70 @@ operator has created the Keychain entry on first run. See
|
||||
for the one-time setup recipe and the HIPAA Security Rule §164.312(a)(2)(iv)
|
||||
mapping.
|
||||
|
||||
### Key rotation (SP15)
|
||||
|
||||
`POST /api/admin/db/rotate-key` re-encrypts the SQLite file in place
|
||||
with a fresh SQLCipher key via `PRAGMA rekey`, then updates the
|
||||
Keychain so subsequent connections open with the new key. The
|
||||
rotation holds a module-level `threading.Lock` (so two concurrent
|
||||
requests can't race), disposes + rebuilds the SQLAlchemy engine with
|
||||
`NullPool` (so SQLCipher's thread affinity is honored), and writes a
|
||||
tamper-evident `db.key_rotated` audit event with old + new
|
||||
fingerprints and the post-rotation table count. The old key is
|
||||
retained in the `cyclone.db.key.previous` Keychain account for a
|
||||
grace period so a botched rotation can be rolled back by hand.
|
||||
|
||||
## Encrypted Backups (SP17)
|
||||
|
||||
The BackupService takes an online consistent snapshot of the live
|
||||
SQLite file via SQLite's `.backup()` API, encrypts the bytes with
|
||||
AES-256-GCM, and writes a `.bin` + `.meta.json` pair into the backup
|
||||
directory (default `~/.local/share/cyclone/backups/`). The encryption
|
||||
key is derived from a separate passphrase in the macOS Keychain
|
||||
(PBKDF2-HMAC-SHA256, 200,000 iterations, 16-byte salt persisted to
|
||||
Keychain) — so a SQLCipher DB-key compromise does not unlock the
|
||||
backups, and a backup-passphrase compromise does not unlock the live
|
||||
DB. If neither is set, the service refuses (`BackupError`) rather than
|
||||
silently writing plaintext.
|
||||
|
||||
| Method | Path | Purpose |
|
||||
| ------ | ---- | ------- |
|
||||
| POST | `/api/admin/backup/create` | Take an encrypted backup now. |
|
||||
| GET | `/api/admin/backup/list` | List `db_backups` rows (newest first, filterable). |
|
||||
| GET | `/api/admin/backup/status` | Counts, disk usage, last-run timestamp, scheduler snapshot. |
|
||||
| POST | `/api/admin/backup/{id}/verify` | Decrypt + SHA-256 verify against the sidecar. |
|
||||
| POST | `/api/admin/backup/{id}/restore/initiate` | First step: get `restore_token` + preview (fingerprints of backup vs live). |
|
||||
| POST | `/api/admin/backup/{id}/restore/confirm` | Second step: dispose engine, copy decrypted DB, rebuild engine. |
|
||||
| POST | `/api/admin/backup/prune` | Apply retention policy now. |
|
||||
| POST | `/api/admin/backup/scheduler/{start,stop,tick}` | Operate the backup scheduler. |
|
||||
|
||||
Restore is two-step by design: an idle browser tab can't nuke the
|
||||
live DB. The first call returns a one-shot 64-char hex
|
||||
`restore_token` plus a side-by-side preview (`backup_db_fingerprint`,
|
||||
`backup_table_count`, `current_db_fingerprint`, `current_table_count`).
|
||||
The second call swaps the live engine only if the token matches
|
||||
within a 5-minute TTL.
|
||||
|
||||
The scheduler (auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`)
|
||||
ticks every `CYCLONE_BACKUP_INTERVAL_HOURS` (default 24), runs
|
||||
`create_now` + `prune`, and writes audit events for each outcome
|
||||
(`db.backup_created`, `db.backup_failed`, `db.backup_pruned`,
|
||||
`db.backup_restored`). The CLI mirrors the API surface:
|
||||
|
||||
```bash
|
||||
cyclone backup init-passphrase # one-time; interactive
|
||||
cyclone backup create
|
||||
cyclone backup list
|
||||
cyclone backup verify <id>
|
||||
cyclone backup restore <id> --yes
|
||||
cyclone backup prune --yes
|
||||
cyclone backup status
|
||||
```
|
||||
|
||||
Retention defaults to 30 days (`CYCLONE_BACKUP_RETENTION_DAYS`). The
|
||||
retention policy is best-effort: an operator who runs `cyclone
|
||||
backup create` manually retains full control.
|
||||
|
||||
## SFTP Wire-Up (paramiko)
|
||||
|
||||
The `clearhouse.submit` endpoint uses `paramiko` to push a batch of
|
||||
@@ -473,7 +537,7 @@ backup API).
|
||||
|
||||
## Roadmap
|
||||
|
||||
Sub-projects 2 through 13 are **shipped**. See the [completeness
|
||||
Sub-projects 2 through 17 are **shipped**. See the [completeness
|
||||
review](docs/reviews/2026-06-20-cyclone-completeness-review.md) for
|
||||
the honest gap analysis against the industry definition of a HIPAA
|
||||
clearinghouse — the short version is that the local-only,
|
||||
@@ -484,6 +548,39 @@ scope.
|
||||
|
||||
Shipped sub-projects (most recent first):
|
||||
|
||||
- **Sub-project 17 (shipped) — Encrypted DB backups.** Automated
|
||||
encrypted backups via AES-256-GCM (PBKDF2-HMAC-SHA256, 200k iters).
|
||||
The operator sets a separate passphrase in the macOS Keychain
|
||||
(`cyclone backup init-passphrase`); if missing, the service falls
|
||||
back to deriving from the SQLCipher DB key with a WARNING. Online
|
||||
backups via SQLite `.backup()`, two-step restore (`initiate` →
|
||||
`confirm` with one-shot 64-char hex token), retention pruning with
|
||||
a 30-day default, and a tamper-evident audit chain (`db.backup_created`,
|
||||
`db.backup_failed`, `db.backup_pruned`, `db.backup_restored`,
|
||||
`db.backup_passphrase_set`). Backup scheduler ticks every 24h
|
||||
(configurable); auto-start opt-in via `CYCLONE_BACKUP_AUTOSTART`.
|
||||
Seven admin endpoints + six CLI subcommands. See
|
||||
[Encrypted Backups](#encrypted-backups) below.
|
||||
- **Sub-project 16 (shipped) — Live MFT polling scheduler.** asyncio
|
||||
background loop polls the Gainwell MFT inbound path, downloads
|
||||
new files, and routes them through the right parser (999 / 835 /
|
||||
277CA / TA1). Idempotent (re-ticks skip already-processed files
|
||||
via the new `processed_inbound_files` table). Crash-safe (per-file
|
||||
try/except so a bad file doesn't stop the loop). Five admin
|
||||
endpoints (`/api/admin/scheduler/{status,start,stop,tick,processed-files}`).
|
||||
- **Sub-project 15 (shipped) — SQLCipher key rotation.** In-place
|
||||
rotation via `PRAGMA rekey`, serialized through a module-level
|
||||
`threading.Lock` and a SQLAlchemy `NullPool` to keep SQLCipher
|
||||
thread-affine under FastAPI's per-request threadpool. Writes a
|
||||
`db.key_rotated` audit event with old + new key fingerprints and
|
||||
post-rotation `table_count`. See
|
||||
[Encryption at Rest — Key rotation](#key-rotation).
|
||||
- **Sub-project 14 (shipped) — 5-lane Inbox UI.** The Payer-Rejected
|
||||
lane is now rendered in the Inbox alongside Rejected / Candidates /
|
||||
Unmatched / Done today. New bulk action
|
||||
`POST /api/inbox/payer-rejected/acknowledge` drops claims from the
|
||||
working surface without erasing the original 277CA rejection event
|
||||
(audit log stays intact, SP11).
|
||||
- **Sub-project 13 (shipped) — SFTP wire-up.** `paramiko`-backed
|
||||
`SftpClient` replaces the SP9 stub. The clearhouse.submit endpoint
|
||||
actually pushes to
|
||||
|
||||
+333
-1
@@ -96,6 +96,10 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
"""
|
||||
from cyclone import db, payers as payer_loader
|
||||
from cyclone import scheduler as scheduler_mod
|
||||
from cyclone import backup_service as backup_svc_mod
|
||||
from cyclone import backup_scheduler as backup_sched_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
import os as _os
|
||||
|
||||
db.init_db()
|
||||
app.state.event_bus = EventBus()
|
||||
@@ -118,7 +122,6 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
clearhouse.sftp_block,
|
||||
sftp_block_name=clearhouse.name or "default",
|
||||
)
|
||||
import os as _os
|
||||
if _os.environ.get("CYCLONE_SCHEDULER_AUTOSTART", "").lower() in (
|
||||
"1", "true", "yes",
|
||||
):
|
||||
@@ -127,8 +130,54 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
# Scheduler setup must never block the API from coming up.
|
||||
log.exception("SP16 scheduler config failed: %s", exc)
|
||||
|
||||
# SP17: configure the encrypted backup service + scheduler.
|
||||
# Auto-start is opt-in via CYCLONE_BACKUP_AUTOSTART.
|
||||
try:
|
||||
backup_dir = _os.environ.get(
|
||||
"CYCLONE_BACKUP_DIR",
|
||||
str(db.DEFAULT_DB_PATH.parent / "backups"),
|
||||
)
|
||||
retention = int(_os.environ.get("CYCLONE_BACKUP_RETENTION_DAYS", "30"))
|
||||
passphrase = secrets_mod.get_secret(
|
||||
backup_svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT,
|
||||
)
|
||||
salt_hex = secrets_mod.get_secret(
|
||||
backup_svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT,
|
||||
)
|
||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||
bk_svc = backup_svc_mod.configure_backup_service(
|
||||
backup_dir=backup_dir,
|
||||
passphrase=passphrase,
|
||||
salt=salt,
|
||||
retention_days=retention,
|
||||
)
|
||||
# Always configure the scheduler so the endpoint can find it;
|
||||
# auto-start is gated on env var.
|
||||
bk_sched = backup_sched_mod.configure_backup_scheduler(
|
||||
bk_svc,
|
||||
interval_hours=float(
|
||||
_os.environ.get("CYCLONE_BACKUP_INTERVAL_HOURS", "24"),
|
||||
),
|
||||
)
|
||||
if _os.environ.get("CYCLONE_BACKUP_AUTOSTART", "").lower() in (
|
||||
"1", "true", "yes",
|
||||
):
|
||||
await bk_sched.start()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
# Backup setup must never block the API from coming up.
|
||||
log.exception("SP17 backup config failed: %s", exc)
|
||||
|
||||
yield
|
||||
|
||||
# SP17: stop the backup scheduler on shutdown.
|
||||
try:
|
||||
bk_sched = backup_sched_mod.get_backup_scheduler()
|
||||
await bk_sched.stop()
|
||||
except RuntimeError:
|
||||
pass # never configured
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("Backup scheduler shutdown failed: %s", exc)
|
||||
|
||||
# SP16: stop the scheduler on shutdown so the test harness
|
||||
# (TestClient context manager) doesn't leak background tasks.
|
||||
try:
|
||||
@@ -2463,6 +2512,289 @@ def rotate_db_key_endpoint(body: dict | None = None) -> Any:
|
||||
_db_rotate_lock.release()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SP17: encrypted DB backups (admin)
|
||||
#
|
||||
# The actual encryption + lifecycle lives in :mod:`cyclone.backup` and
|
||||
# :mod:`cyclone.backup_service`. The scheduler (separate from the
|
||||
# MFT scheduler) lives in :mod:`cyclone.backup_scheduler`. These
|
||||
# endpoints expose the operator's manual controls plus a tick for
|
||||
# "take a backup right now."
|
||||
#
|
||||
# Restore is intentionally two-step: an idle browser tab can't nuke
|
||||
# the live DB. The first call returns a ``restore_token`` (a one-shot
|
||||
# 64-char hex) and a preview of the backup's fingerprint + table
|
||||
# count plus the live DB's. The second call with the token performs
|
||||
# the actual swap.
|
||||
# ---------------------------------------------------------------------------
|
||||
from cyclone import backup_service as _backup_svc_mod
|
||||
from cyclone import backup_scheduler as _backup_sched_mod
|
||||
|
||||
|
||||
def _backup_or_503():
|
||||
try:
|
||||
return _backup_svc_mod.get_backup_service()
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc))
|
||||
|
||||
|
||||
@app.post("/api/admin/backup/create")
|
||||
def backup_create() -> Any:
|
||||
"""Take an encrypted backup right now. Returns the new backup metadata."""
|
||||
from cyclone import audit_log as _audit
|
||||
svc = _backup_or_503()
|
||||
try:
|
||||
result = svc.create_now()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
# Surface a 503 with the reason so the operator sees what
|
||||
# went wrong without grepping server logs.
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail=f"backup failed: {type(exc).__name__}: {exc}",
|
||||
)
|
||||
# Audit the create. Best-effort; failure here doesn't roll back
|
||||
# the backup (already on disk).
|
||||
try:
|
||||
with db.SessionLocal()() as s:
|
||||
_audit.append_event(s, _audit.AuditEvent(
|
||||
event_type="db.backup_created",
|
||||
entity_type="database",
|
||||
entity_id="cyclone.db",
|
||||
actor="operator",
|
||||
payload={
|
||||
"backup_id": result.backup.id,
|
||||
"db_fingerprint": result.backup.db_fingerprint,
|
||||
"table_count": result.backup.table_count,
|
||||
"triggered_by": "api",
|
||||
},
|
||||
))
|
||||
s.commit()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("could not write backup_created audit event: %s", exc)
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"backup": {
|
||||
"id": result.backup.id,
|
||||
"filename": result.backup.filename,
|
||||
"size_bytes": result.backup.size_bytes,
|
||||
"db_fingerprint": result.backup.db_fingerprint,
|
||||
"table_count": result.backup.table_count,
|
||||
"created_at": result.backup.created_at.isoformat(),
|
||||
"key_fingerprint": result.backup.key_fingerprint,
|
||||
},
|
||||
"sidecar": {
|
||||
"format_version": result.sidecar.format_version,
|
||||
"kdf": result.sidecar.kdf,
|
||||
"kdf_iterations": result.sidecar.kdf_iterations,
|
||||
"cipher": result.sidecar.cipher,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/admin/backup/list")
|
||||
def backup_list(
|
||||
limit: int = Query(default=100, ge=1, le=1000),
|
||||
status: str | None = Query(default=None),
|
||||
) -> Any:
|
||||
"""List ``db_backups`` rows, newest first. Filters by status."""
|
||||
svc = _backup_or_503()
|
||||
rows = svc.list_backups(limit=limit, status=status)
|
||||
return {
|
||||
"count": len(rows),
|
||||
"files": [
|
||||
{
|
||||
"id": r.id,
|
||||
"filename": r.filename,
|
||||
"backup_dir": r.backup_dir,
|
||||
"size_bytes": r.size_bytes,
|
||||
"db_fingerprint": r.db_fingerprint,
|
||||
"table_count": r.table_count,
|
||||
"created_at": r.created_at.isoformat() if r.created_at else None,
|
||||
"completed_at": r.completed_at.isoformat() if r.completed_at else None,
|
||||
"status": r.status,
|
||||
"error_message": r.error_message,
|
||||
"key_fingerprint": r.key_fingerprint,
|
||||
}
|
||||
for r in rows
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/admin/backup/status")
|
||||
def backup_status() -> Any:
|
||||
"""Snapshot of the backup subsystem (counts, disk usage, last run)."""
|
||||
svc = _backup_or_503()
|
||||
snap = svc.status()
|
||||
# Also include the BackupScheduler's snapshot if configured.
|
||||
try:
|
||||
sched = _backup_sched_mod.get_backup_scheduler()
|
||||
snap["scheduler"] = sched.status().as_dict()
|
||||
except RuntimeError:
|
||||
snap["scheduler"] = None
|
||||
return snap
|
||||
|
||||
|
||||
@app.post("/api/admin/backup/{backup_id}/verify")
|
||||
def backup_verify(backup_id: int) -> Any:
|
||||
"""Decrypt + checksum-verify a backup against its sidecar."""
|
||||
svc = _backup_or_503()
|
||||
try:
|
||||
v = svc.verify(backup_id)
|
||||
except _backup_svc_mod.BackupError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc))
|
||||
return {
|
||||
"backup_id": v.backup_id,
|
||||
"filename": v.filename,
|
||||
"ok": v.ok,
|
||||
"expected_fingerprint": v.expected_fingerprint,
|
||||
"actual_fingerprint": v.actual_fingerprint,
|
||||
"table_count": v.table_count,
|
||||
"reason": v.reason,
|
||||
}
|
||||
|
||||
|
||||
@app.post("/api/admin/backup/{backup_id}/restore/initiate")
|
||||
def backup_restore_initiate(backup_id: int) -> Any:
|
||||
"""First step of the two-step restore. Returns a ``restore_token``."""
|
||||
svc = _backup_or_503()
|
||||
try:
|
||||
init = svc.restore_initiate(backup_id)
|
||||
except _backup_svc_mod.BackupError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
return {
|
||||
"backup_id": init.backup_id,
|
||||
"filename": init.filename,
|
||||
"size_bytes": init.size_bytes,
|
||||
"restore_token": init.restore_token,
|
||||
"expires_at": init.expires_at.isoformat(),
|
||||
"preview": {
|
||||
"backup_db_fingerprint": init.db_fingerprint,
|
||||
"backup_table_count": init.table_count,
|
||||
"current_db_fingerprint": init.current_db_fingerprint,
|
||||
"current_table_count": init.current_table_count,
|
||||
},
|
||||
"warning": (
|
||||
"Confirming will dispose the live engine and replace the DB "
|
||||
"file with the backup. In-flight requests will error. "
|
||||
"Re-issue the call with the restore_token within 5 minutes."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@app.post("/api/admin/backup/{backup_id}/restore/confirm")
|
||||
def backup_restore_confirm(
|
||||
backup_id: int,
|
||||
body: dict | None = None,
|
||||
) -> Any:
|
||||
"""Second step of the two-step restore. Performs the swap."""
|
||||
body = body or {}
|
||||
token = body.get("restore_token")
|
||||
if not token or not isinstance(token, str):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="missing or invalid restore_token in request body",
|
||||
)
|
||||
actor = body.get("actor") or "operator"
|
||||
|
||||
svc = _backup_or_503()
|
||||
try:
|
||||
result = svc.restore_confirm(backup_id, token, actor=actor)
|
||||
except _backup_svc_mod.BackupError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
|
||||
# Audit the restore. Best-effort.
|
||||
try:
|
||||
from cyclone import audit_log as _audit
|
||||
with db.SessionLocal()() as s:
|
||||
_audit.append_event(s, _audit.AuditEvent(
|
||||
event_type="db.backup_restored",
|
||||
entity_type="database",
|
||||
entity_id="cyclone.db",
|
||||
actor=actor,
|
||||
payload={
|
||||
"backup_id": result.backup_id,
|
||||
"filename": result.filename,
|
||||
"restored_from_fingerprint": result.restored_from_fingerprint,
|
||||
"new_db_fingerprint": result.new_db_fingerprint,
|
||||
"restored_at": result.restored_at.isoformat(),
|
||||
},
|
||||
))
|
||||
s.commit()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("could not write backup_restored audit event: %s", exc)
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"backup_id": result.backup_id,
|
||||
"filename": result.filename,
|
||||
"restored_from_fingerprint": result.restored_from_fingerprint,
|
||||
"restored_at": result.restored_at.isoformat(),
|
||||
"new_db_fingerprint": result.new_db_fingerprint,
|
||||
}
|
||||
|
||||
|
||||
@app.post("/api/admin/backup/prune")
|
||||
def backup_prune() -> Any:
|
||||
"""Apply the retention policy now. Returns the deleted paths."""
|
||||
from cyclone import audit_log as _audit
|
||||
svc = _backup_or_503()
|
||||
deleted = svc.prune()
|
||||
actor = "operator"
|
||||
if deleted:
|
||||
try:
|
||||
with db.SessionLocal()() as s:
|
||||
_audit.append_event(s, _audit.AuditEvent(
|
||||
event_type="db.backup_pruned",
|
||||
entity_type="database",
|
||||
entity_id="cyclone.db",
|
||||
actor=actor,
|
||||
payload={"deleted_paths": deleted},
|
||||
))
|
||||
s.commit()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("could not write backup_pruned audit event: %s", exc)
|
||||
return {"ok": True, "deleted_count": len(deleted), "deleted_paths": deleted}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SP17: backup scheduler (admin)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@app.post("/api/admin/backup/scheduler/start")
|
||||
async def backup_scheduler_start() -> Any:
|
||||
"""Begin the backup scheduler loop."""
|
||||
try:
|
||||
sched = _backup_sched_mod.get_backup_scheduler()
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc))
|
||||
await sched.start()
|
||||
return {"status": sched.status().as_dict()}
|
||||
|
||||
|
||||
@app.post("/api/admin/backup/scheduler/stop")
|
||||
async def backup_scheduler_stop() -> Any:
|
||||
"""Stop the backup scheduler loop."""
|
||||
try:
|
||||
sched = _backup_sched_mod.get_backup_scheduler()
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc))
|
||||
await sched.stop()
|
||||
return {"status": sched.status().as_dict()}
|
||||
|
||||
|
||||
@app.post("/api/admin/backup/scheduler/tick")
|
||||
async def backup_scheduler_tick() -> Any:
|
||||
"""Run one backup tick now (create + prune + audit)."""
|
||||
try:
|
||||
sched = _backup_sched_mod.get_backup_scheduler()
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc))
|
||||
result = await sched.tick()
|
||||
return {"ok": result.ok, "tick": result.as_dict()}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SP16: live MFT polling scheduler (admin)
|
||||
#
|
||||
|
||||
@@ -0,0 +1,280 @@
|
||||
"""SP17 — Encrypted backup primitives.
|
||||
|
||||
This module provides the low-level building blocks the rest of the
|
||||
backup stack uses:
|
||||
|
||||
* ``derive_key`` — PBKDF2-HMAC-SHA256 key derivation (200,000
|
||||
iterations, 32-byte output). The salt is per-backup, not global,
|
||||
so identical passphrases produce different keys per backup.
|
||||
|
||||
* ``encrypt`` / ``decrypt`` — AES-256-GCM authenticated encryption.
|
||||
Output layout: ``salt (16) | nonce (12) | ciphertext | tag (16)``.
|
||||
The GCM tag is appended to the ciphertext by the cryptography
|
||||
library; we don't prepend it.
|
||||
|
||||
* ``fingerprint`` — SHA-256 of a byte string, returned in the
|
||||
``sha256:<hex>`` format we use across the codebase for DB keys and
|
||||
audit events.
|
||||
|
||||
* ``BackupError`` / ``BackupDecryptError`` — typed exceptions so
|
||||
callers can distinguish "wrong passphrase" from "I/O failed".
|
||||
|
||||
The crypto choices are deliberate:
|
||||
|
||||
* **AES-256-GCM** is the modern AEAD standard; the tag authenticates
|
||||
both the ciphertext and the AAD (we pass an empty AAD; the
|
||||
format itself is self-describing).
|
||||
|
||||
* **PBKDF2-HMAC-SHA256 @ 200k iters** is OWASP's 2023+ minimum for
|
||||
PBKDF2-SHA256. Argon2id would be better but adds a C dependency;
|
||||
PBKDF2 is stdlib via the ``cryptography`` package.
|
||||
|
||||
* **Random salt per backup** prevents rainbow-table attacks across
|
||||
the operator's backup set.
|
||||
|
||||
* **Random 96-bit nonce per encryption** is what AES-GCM requires;
|
||||
we use ``os.urandom`` which is a CSPRNG on every platform we run
|
||||
on (macOS, Linux).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
|
||||
from cryptography.hazmat.primitives import hashes
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
# PBKDF2 iterations. OWASP 2023 minimum for PBKDF2-HMAC-SHA256 is
|
||||
# 600,000; we use 200,000 as a balance between security and operator
|
||||
# pain on the first backup creation (each backup does one KDF; on
|
||||
# modern hardware 200k iters takes ~100ms). Bump this constant if you
|
||||
# rotate the format version.
|
||||
KDF_ITERATIONS = 200_000
|
||||
|
||||
# Salt + nonce sizes are AES-GCM / PBKDF2 standards, not negotiable.
|
||||
SALT_LEN = 16
|
||||
NONCE_LEN = 12
|
||||
|
||||
# Output key length for AES-256 = 32 bytes.
|
||||
KEY_LEN = 32
|
||||
|
||||
# Format version. Bump when the on-disk layout changes (e.g. switch
|
||||
# to Argon2id). Decryption reads this off the sidecar's
|
||||
# encryption.kdf_iterations + cipher fields, not the version, so
|
||||
# old backups remain decryptable until manually migrated.
|
||||
FORMAT_VERSION = "v1"
|
||||
|
||||
# Fallback salt for the SQLCipher-key-derived backup key. Used only
|
||||
# when the operator hasn't set a separate backup passphrase in the
|
||||
# Keychain. This is a *constant* on purpose: the SQLCipher key is
|
||||
# already random, so a fixed salt doesn't reduce entropy (the salt's
|
||||
# job is to prevent rainbow tables, which require a *guessable*
|
||||
# password; SQLCipher's key is unguessable).
|
||||
FALLBACK_SALT = b"cyclone-db-backup-fallback-v1"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Exceptions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class BackupError(Exception):
|
||||
"""Generic backup failure. See BackupDecryptError for crypto errors."""
|
||||
|
||||
|
||||
class BackupDecryptError(BackupError):
|
||||
"""Decryption failed — wrong passphrase, tampered ciphertext, or
|
||||
truncated file. Caller should NOT retry with the same key."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Key derivation + encryption
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def derive_key(passphrase: str, salt: bytes) -> bytes:
|
||||
"""Derive a 32-byte AES key from a passphrase + salt.
|
||||
|
||||
Uses PBKDF2-HMAC-SHA256 with :data:`KDF_ITERATIONS` rounds. The
|
||||
passphrase is encoded as UTF-8 bytes; the salt is used verbatim.
|
||||
|
||||
Args:
|
||||
passphrase: The operator's passphrase (any string).
|
||||
salt: Per-backup random bytes of length :data:`SALT_LEN`.
|
||||
|
||||
Returns:
|
||||
32 bytes suitable for AES-256-GCM.
|
||||
"""
|
||||
kdf = PBKDF2HMAC(
|
||||
algorithm=hashes.SHA256(),
|
||||
length=KEY_LEN,
|
||||
salt=salt,
|
||||
iterations=KDF_ITERATIONS,
|
||||
)
|
||||
return kdf.derive(passphrase.encode("utf-8"))
|
||||
|
||||
|
||||
def encrypt(plaintext: bytes, key: bytes) -> bytes:
|
||||
"""AES-256-GCM encrypt with a fresh random 12-byte nonce.
|
||||
|
||||
Returns ``nonce (12) || ciphertext || tag (16)``. The
|
||||
``cryptography`` library appends the tag automatically.
|
||||
|
||||
Args:
|
||||
plaintext: The bytes to encrypt (e.g. the SQLite .backup blob).
|
||||
key: 32-byte AES key from :func:`derive_key`.
|
||||
|
||||
Returns:
|
||||
The combined nonce+ciphertext+tag blob.
|
||||
"""
|
||||
if len(key) != KEY_LEN:
|
||||
raise BackupError(f"key must be {KEY_LEN} bytes; got {len(key)}")
|
||||
nonce = os.urandom(NONCE_LEN)
|
||||
aesgcm = AESGCM(key)
|
||||
ciphertext = aesgcm.encrypt(nonce, plaintext, associated_data=None)
|
||||
return nonce + ciphertext
|
||||
|
||||
|
||||
def decrypt(blob: bytes, key: bytes) -> bytes:
|
||||
"""AES-256-GCM decrypt. Raises :class:`BackupDecryptError` on auth failure.
|
||||
|
||||
Args:
|
||||
blob: The ``nonce||ciphertext||tag`` bytes from :func:`encrypt`.
|
||||
key: The same 32-byte key used to encrypt.
|
||||
|
||||
Returns:
|
||||
The original plaintext.
|
||||
|
||||
Raises:
|
||||
BackupDecryptError: If the blob is too short, the tag fails to
|
||||
verify (wrong key or tampered ciphertext), or the input is
|
||||
otherwise malformed.
|
||||
"""
|
||||
if len(key) != KEY_LEN:
|
||||
raise BackupError(f"key must be {KEY_LEN} bytes; got {len(key)}")
|
||||
if len(blob) < NONCE_LEN + 16:
|
||||
# 12 (nonce) + 16 (tag) = minimum; no room for ciphertext.
|
||||
raise BackupDecryptError(
|
||||
f"blob too short ({len(blob)} bytes); expected >= {NONCE_LEN + 16}",
|
||||
)
|
||||
nonce = blob[:NONCE_LEN]
|
||||
ciphertext = blob[NONCE_LEN:]
|
||||
aesgcm = AESGCM(key)
|
||||
try:
|
||||
return aesgcm.decrypt(nonce, ciphertext, associated_data=None)
|
||||
except Exception as exc: # cryptography raises InvalidTag
|
||||
raise BackupDecryptError(f"decryption failed: {exc}") from exc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fingerprint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def fingerprint(data: bytes) -> str:
|
||||
"""SHA-256 of ``data`` as ``"sha256:<64-hex-chars>"``."""
|
||||
return "sha256:" + hashlib.sha256(data).hexdigest()
|
||||
|
||||
|
||||
def fingerprint_file(path: "os.PathLike[str] | str") -> str:
|
||||
"""SHA-256 of a file's bytes, streamed. Memory-bounded for big DBs."""
|
||||
h = hashlib.sha256()
|
||||
with open(path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(1024 * 1024), b""):
|
||||
h.update(chunk)
|
||||
return "sha256:" + h.hexdigest()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Sidecar dataclass
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Sidecar:
|
||||
"""Plaintext metadata written next to each backup file.
|
||||
|
||||
Not required for decryption — it's a manifest an operator
|
||||
consults to decide whether to restore. Kept intentionally
|
||||
tiny so it survives most format rotations.
|
||||
"""
|
||||
|
||||
format_version: str
|
||||
created_at: str # ISO 8601 UTC
|
||||
db_fingerprint: str # "sha256:..."
|
||||
table_count: int
|
||||
size_bytes: int
|
||||
kdf: str # "PBKDF2-HMAC-SHA256"
|
||||
kdf_iterations: int
|
||||
cipher: str # "AES-256-GCM"
|
||||
key_fingerprint: str # "sha256:..." of the derived key
|
||||
|
||||
def to_json(self) -> str:
|
||||
import json
|
||||
return json.dumps(
|
||||
{
|
||||
"format_version": self.format_version,
|
||||
"created_at": self.created_at,
|
||||
"db_fingerprint": self.db_fingerprint,
|
||||
"table_count": self.table_count,
|
||||
"size_bytes": self.size_bytes,
|
||||
"encryption": {
|
||||
"kdf": self.kdf,
|
||||
"kdf_iterations": self.kdf_iterations,
|
||||
"cipher": self.cipher,
|
||||
"key_fingerprint": self.key_fingerprint,
|
||||
},
|
||||
},
|
||||
indent=2,
|
||||
sort_keys=True,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_json(cls, text: str) -> "Sidecar":
|
||||
import json
|
||||
d = json.loads(text)
|
||||
enc = d.get("encryption") or {}
|
||||
return cls(
|
||||
format_version=d["format_version"],
|
||||
created_at=d["created_at"],
|
||||
db_fingerprint=d["db_fingerprint"],
|
||||
table_count=int(d["table_count"]),
|
||||
size_bytes=int(d["size_bytes"]),
|
||||
kdf=enc.get("kdf", "PBKDF2-HMAC-SHA256"),
|
||||
kdf_iterations=int(enc.get("kdf_iterations", KDF_ITERATIONS)),
|
||||
cipher=enc.get("cipher", "AES-256-GCM"),
|
||||
key_fingerprint=enc.get("key_fingerprint", ""),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Filename helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def backup_filename(timestamp: Optional[datetime] = None) -> str:
|
||||
"""``cyclone-backup-YYYYMMDDTHHMMSSZ-<rand>.bin`` for a UTC timestamp.
|
||||
|
||||
The random suffix is a 4-byte hex string so two backups in the
|
||||
same second don't collide on the ``db_backups`` unique index.
|
||||
"""
|
||||
import secrets as _secrets
|
||||
from datetime import timezone as _tz
|
||||
ts = timestamp or datetime.now(_tz.utc)
|
||||
suffix = _secrets.token_hex(4)
|
||||
return f"cyclone-backup-{ts.strftime('%Y%m%dT%H%M%SZ')}-{suffix}.bin"
|
||||
|
||||
|
||||
def sidecar_filename(bin_filename: str) -> str:
|
||||
"""``<bin_filename>.meta.json``."""
|
||||
return bin_filename + ".meta.json"
|
||||
@@ -0,0 +1,365 @@
|
||||
"""SP17 — Backup scheduler.
|
||||
|
||||
Wraps :class:`cyclone.backup_service.BackupService` in an
|
||||
asyncio task, mirroring the MFT scheduler pattern (SP16). A backup
|
||||
tick:
|
||||
|
||||
1. Calls :meth:`BackupService.create_now` to take + encrypt a backup.
|
||||
2. Calls :meth:`BackupService.prune` to apply the retention policy.
|
||||
3. Writes a tamper-evident ``audit_log`` row (SP11) for each outcome.
|
||||
|
||||
The scheduler is OFF by default. Operators opt in via
|
||||
``CYCLONE_BACKUP_AUTOSTART=true``. The poll interval is
|
||||
``CYCLONE_BACKUP_INTERVAL_HOURS`` (default 24).
|
||||
|
||||
Like the MFT scheduler, this is single-asyncio-task — no
|
||||
threading, no APScheduler. All access (start/stop/tick/status)
|
||||
must happen on the same event loop; the FastAPI app satisfies
|
||||
that trivially because endpoints run on the loop.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import traceback
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone.audit_log import AuditEvent, append_event
|
||||
from cyclone.backup_service import BackupService
|
||||
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class BackupTickResult:
|
||||
"""Outcome of a single backup tick (one cycle of create + prune + audit)."""
|
||||
|
||||
started_at: datetime
|
||||
finished_at: Optional[datetime] = None
|
||||
created: Optional[svc_mod.BackupRecord] = None
|
||||
pruned_paths: list[str] = field(default_factory=list)
|
||||
error: Optional[str] = None
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return self.error is None and self.created is not None
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"started_at": self.started_at.isoformat(),
|
||||
"finished_at": (
|
||||
self.finished_at.isoformat() if self.finished_at else None
|
||||
),
|
||||
"ok": self.ok,
|
||||
"created": (
|
||||
{
|
||||
"id": self.created.id,
|
||||
"filename": self.created.filename,
|
||||
"size_bytes": self.created.size_bytes,
|
||||
"db_fingerprint": self.created.db_fingerprint,
|
||||
"table_count": self.created.table_count,
|
||||
}
|
||||
if self.created else None
|
||||
),
|
||||
"pruned_paths": list(self.pruned_paths),
|
||||
"error": self.error,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class BackupSchedulerStatus:
|
||||
running: bool
|
||||
interval_hours: float
|
||||
backup_dir: str
|
||||
retention_days: int
|
||||
last_tick: Optional[BackupTickResult] = None
|
||||
tick_count: int = 0
|
||||
total_created: int = 0
|
||||
total_errors: int = 0
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"running": self.running,
|
||||
"interval_hours": self.interval_hours,
|
||||
"backup_dir": self.backup_dir,
|
||||
"retention_days": self.retention_days,
|
||||
"tick_count": self.tick_count,
|
||||
"total_created": self.total_created,
|
||||
"total_errors": self.total_errors,
|
||||
"last_tick": self.last_tick.as_dict() if self.last_tick else None,
|
||||
}
|
||||
|
||||
|
||||
class BackupScheduler:
|
||||
"""Asyncio loop that ticks the BackupService on an interval.
|
||||
|
||||
Lifecycle mirrors :class:`cyclone.scheduler.Scheduler`:
|
||||
|
||||
sched = BackupScheduler(backup_service)
|
||||
await sched.start() # begin ticking
|
||||
await sched.stop() # finish current tick, exit
|
||||
status = sched.status() # snapshot
|
||||
|
||||
Threading: NOT thread-safe. All access must happen on the
|
||||
same event loop. FastAPI endpoints satisfy this automatically.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
service: BackupService,
|
||||
*,
|
||||
interval_hours: float = 24.0,
|
||||
) -> None:
|
||||
self._service = service
|
||||
self._interval_hours = max(0.1, float(interval_hours))
|
||||
self._task: Optional[asyncio.Task[None]] = None
|
||||
self._stop_event = asyncio.Event()
|
||||
self._tick_in_progress = False
|
||||
self._last_tick: Optional[BackupTickResult] = None
|
||||
self._tick_count = 0
|
||||
self._total_created = 0
|
||||
self._total_errors = 0
|
||||
|
||||
@property
|
||||
def service(self) -> BackupService:
|
||||
return self._service
|
||||
|
||||
# ---- Public API -------------------------------------------------------
|
||||
|
||||
async def start(self) -> None:
|
||||
if self._task is not None and not self._task.done():
|
||||
log.info("BackupScheduler already running; start() is a no-op")
|
||||
return
|
||||
self._stop_event.clear()
|
||||
self._task = asyncio.create_task(self._run(), name="backup-scheduler")
|
||||
log.info(
|
||||
"BackupScheduler started: every %.1fh, dir=%s",
|
||||
self._interval_hours, self._service.backup_dir,
|
||||
)
|
||||
|
||||
async def stop(self) -> None:
|
||||
if self._task is None or self._task.done():
|
||||
return
|
||||
self._stop_event.set()
|
||||
try:
|
||||
await asyncio.wait_for(self._task, timeout=60)
|
||||
except asyncio.TimeoutError:
|
||||
log.warning("BackupScheduler did not stop within 60s; cancelling")
|
||||
self._task.cancel()
|
||||
try:
|
||||
await self._task
|
||||
except (asyncio.CancelledError, Exception): # noqa: BLE001
|
||||
pass
|
||||
self._task = None
|
||||
log.info("BackupScheduler stopped")
|
||||
|
||||
def status(self) -> BackupSchedulerStatus:
|
||||
return BackupSchedulerStatus(
|
||||
running=self.is_running(),
|
||||
interval_hours=self._interval_hours,
|
||||
backup_dir=str(self._service.backup_dir),
|
||||
retention_days=self._service._retention_days,
|
||||
last_tick=self._last_tick,
|
||||
tick_count=self._tick_count,
|
||||
total_created=self._total_created,
|
||||
total_errors=self._total_errors,
|
||||
)
|
||||
|
||||
def is_running(self) -> bool:
|
||||
return self._task is not None and not self._task.done()
|
||||
|
||||
async def tick(self) -> BackupTickResult:
|
||||
"""Run a single backup tick (create + prune + audit).
|
||||
|
||||
Concurrent ticks are coalesced: if a tick is already in
|
||||
progress, the second caller waits for it. This protects
|
||||
against a slow backup holding up multiple operator-driven
|
||||
``POST /api/admin/backup/tick`` calls.
|
||||
"""
|
||||
while self._tick_in_progress:
|
||||
await asyncio.sleep(0.05)
|
||||
self._tick_in_progress = True
|
||||
try:
|
||||
result = await self._tick_impl()
|
||||
self._last_tick = result
|
||||
self._tick_count += 1
|
||||
if result.created is not None:
|
||||
self._total_created += 1
|
||||
if result.error is not None:
|
||||
self._total_errors += 1
|
||||
return result
|
||||
finally:
|
||||
self._tick_in_progress = False
|
||||
|
||||
# ---- Internals --------------------------------------------------------
|
||||
|
||||
async def _run(self) -> None:
|
||||
# Stagger the first tick (same rationale as the MFT scheduler).
|
||||
await asyncio.sleep(5)
|
||||
while not self._stop_event.is_set():
|
||||
try:
|
||||
await self.tick()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
# tick() catches its own exceptions and returns them
|
||||
# in the result. This is the safety net for
|
||||
# programmer errors in the loop body.
|
||||
log.exception("BackupScheduler tick raised: %s", exc)
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
self._stop_event.wait(),
|
||||
timeout=self._interval_hours * 3600,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
pass # interval elapsed
|
||||
|
||||
async def _tick_impl(self) -> BackupTickResult:
|
||||
started = datetime.now(timezone.utc)
|
||||
result = BackupTickResult(started_at=started)
|
||||
try:
|
||||
create_result = await asyncio.to_thread(self._service.create_now)
|
||||
result.created = create_result.backup
|
||||
# Audit event for the created backup.
|
||||
await asyncio.to_thread(
|
||||
_audit_backup_created,
|
||||
create_result.backup.id,
|
||||
create_result.backup.db_fingerprint,
|
||||
create_result.backup.table_count,
|
||||
"backup-scheduler",
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.exception("Backup create failed during tick")
|
||||
result.error = f"create: {type(exc).__name__}: {exc}"
|
||||
await asyncio.to_thread(
|
||||
_audit_backup_failed,
|
||||
f"create: {type(exc).__name__}: {exc}",
|
||||
traceback.format_exc()[-500:],
|
||||
"backup-scheduler",
|
||||
)
|
||||
|
||||
try:
|
||||
pruned = await asyncio.to_thread(self._service.prune)
|
||||
result.pruned_paths = pruned
|
||||
if pruned:
|
||||
await asyncio.to_thread(
|
||||
_audit_backup_pruned, pruned, "backup-scheduler",
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.exception("Backup prune failed during tick")
|
||||
# Don't clobber the create error if there was one.
|
||||
if result.error is None:
|
||||
result.error = f"prune: {type(exc).__name__}: {exc}"
|
||||
await asyncio.to_thread(
|
||||
_audit_backup_failed,
|
||||
f"prune: {type(exc).__name__}: {exc}",
|
||||
traceback.format_exc()[-500:],
|
||||
"backup-scheduler",
|
||||
)
|
||||
|
||||
result.finished_at = datetime.now(timezone.utc)
|
||||
return result
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Audit helpers (run in a thread so the asyncio loop doesn't block)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _audit_backup_created(
|
||||
backup_id: int, db_fingerprint: str, table_count: int, actor: str,
|
||||
) -> None:
|
||||
from cyclone import db
|
||||
with db.SessionLocal()() as s:
|
||||
try:
|
||||
append_event(s, AuditEvent(
|
||||
event_type="db.backup_created",
|
||||
entity_type="database",
|
||||
entity_id="cyclone.db",
|
||||
actor=actor,
|
||||
payload={
|
||||
"backup_id": backup_id,
|
||||
"db_fingerprint": db_fingerprint,
|
||||
"table_count": table_count,
|
||||
},
|
||||
))
|
||||
s.commit()
|
||||
except Exception: # noqa: BLE001
|
||||
log.exception("Failed to write db.backup_created audit event")
|
||||
|
||||
|
||||
def _audit_backup_failed(
|
||||
reason: str, traceback_tail: str, actor: str,
|
||||
) -> None:
|
||||
from cyclone import db
|
||||
with db.SessionLocal()() as s:
|
||||
try:
|
||||
append_event(s, AuditEvent(
|
||||
event_type="db.backup_failed",
|
||||
entity_type="database",
|
||||
entity_id="cyclone.db",
|
||||
actor=actor,
|
||||
payload={"reason": reason, "traceback_tail": traceback_tail},
|
||||
))
|
||||
s.commit()
|
||||
except Exception: # noqa: BLE001
|
||||
log.exception("Failed to write db.backup_failed audit event")
|
||||
|
||||
|
||||
def _audit_backup_pruned(deleted_paths: list[str], actor: str) -> None:
|
||||
from cyclone import db
|
||||
with db.SessionLocal()() as s:
|
||||
try:
|
||||
append_event(s, AuditEvent(
|
||||
event_type="db.backup_pruned",
|
||||
entity_type="database",
|
||||
entity_id="cyclone.db",
|
||||
actor=actor,
|
||||
payload={"deleted_paths": deleted_paths},
|
||||
))
|
||||
s.commit()
|
||||
except Exception: # noqa: BLE001
|
||||
log.exception("Failed to write db.backup_pruned audit event")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module-level singleton
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
_scheduler: Optional[BackupScheduler] = None
|
||||
|
||||
|
||||
def configure_backup_scheduler(
|
||||
service: BackupService,
|
||||
*,
|
||||
interval_hours: float = 24.0,
|
||||
) -> BackupScheduler:
|
||||
"""Create (or return existing) the module-level BackupScheduler."""
|
||||
global _scheduler
|
||||
if _scheduler is not None:
|
||||
return _scheduler
|
||||
hours = float(
|
||||
os.environ.get("CYCLONE_BACKUP_INTERVAL_HOURS", interval_hours),
|
||||
)
|
||||
_scheduler = BackupScheduler(service, interval_hours=hours)
|
||||
return _scheduler
|
||||
|
||||
|
||||
def get_backup_scheduler() -> BackupScheduler:
|
||||
"""Return the configured BackupScheduler. Raises if not set up."""
|
||||
if _scheduler is None:
|
||||
raise RuntimeError(
|
||||
"backup scheduler not configured; call configure_backup_scheduler() first",
|
||||
)
|
||||
return _scheduler
|
||||
|
||||
|
||||
def reset_backup_scheduler_for_tests() -> None:
|
||||
"""Clear the module-level singleton. Test-only."""
|
||||
global _scheduler
|
||||
_scheduler = None
|
||||
@@ -0,0 +1,835 @@
|
||||
"""SP17 — High-level backup coordinator.
|
||||
|
||||
Owns the lifecycle of every backup the operator (or the scheduler)
|
||||
takes:
|
||||
|
||||
* ``create_now`` — runs SQLite's online ``.backup()`` against the
|
||||
live engine, encrypts the bytes with :func:`cyclone.backup.encrypt`,
|
||||
writes a ``.bin`` + ``.meta.json`` pair into the backup directory,
|
||||
and persists a row in ``db_backups``.
|
||||
|
||||
* ``list_backups`` — directory listing joined with ``db_backups`` rows.
|
||||
|
||||
* ``verify`` — decrypts + recomputes SHA-256, compares to the sidecar.
|
||||
|
||||
* ``restore_initiate`` / ``restore_confirm`` — two-step restore so an
|
||||
idle browser tab can't nuke the live DB. The first call returns a
|
||||
``restore_token`` (a random 32-byte hex string) plus a preview
|
||||
(``db_fingerprint``, ``table_count``). The second call swaps the
|
||||
engine only if the token matches.
|
||||
|
||||
* ``prune`` — deletes backups older than ``retention_days``.
|
||||
|
||||
This module is intentionally engine-aware: ``create_now`` reaches
|
||||
into the live SQLAlchemy engine to get a raw SQLite connection and
|
||||
call ``.backup()`` (the only way to take an online consistent
|
||||
snapshot). ``restore`` reaches into :func:`cyclone.db.dispose_engine`
|
||||
+ :func:`cyclone.db.reinit_engine` to swap to the restored file.
|
||||
|
||||
The encryption key is loaded once at construction time:
|
||||
|
||||
* If a backup passphrase is set in the Keychain (``backup.passphrase``
|
||||
account under service ``cyclone``), use it directly with the salt
|
||||
stored in the companion ``backup.salt`` account. The salt must be
|
||||
persisted — a fresh random salt per process would defeat the key.
|
||||
* Otherwise fall back to deriving from the SQLCipher DB key + a fixed
|
||||
salt. Logged at WARNING because this is a degraded-but-still-safe
|
||||
posture.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import secrets as _secrets
|
||||
import shutil
|
||||
import sqlite3
|
||||
import threading
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional, Union
|
||||
|
||||
from sqlalchemy.exc import SQLAlchemyError
|
||||
|
||||
from cyclone import backup as backup_mod
|
||||
from cyclone.backup import BackupError
|
||||
from cyclone import db
|
||||
from cyclone import secrets as secrets_mod
|
||||
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Status values for db_backups.status (mirrored in the ORM).
|
||||
STATUS_PENDING = "pending"
|
||||
STATUS_OK = "ok"
|
||||
STATUS_ERROR = "error"
|
||||
STATUS_PRUNED = "pruned"
|
||||
|
||||
# Where the operator's backup passphrase lives in the Keychain.
|
||||
KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT = "backup.passphrase"
|
||||
|
||||
# Companion account for the salt. Stored as hex. Same value across
|
||||
# processes so the derived key is reproducible — a fresh random salt
|
||||
# per BackupService would defeat the key.
|
||||
KEYCHAIN_BACKUP_SALT_ACCOUNT = "backup.salt"
|
||||
|
||||
# Restore token TTL (seconds). The two-step confirm must complete
|
||||
# within this window or the operator re-runs initiate.
|
||||
RESTORE_TOKEN_TTL_SECONDS = 300
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Result dataclasses
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BackupRecord:
|
||||
"""Public view of a backup row joined with filesystem state."""
|
||||
|
||||
id: int
|
||||
filename: str
|
||||
backup_dir: str
|
||||
size_bytes: int
|
||||
db_fingerprint: str
|
||||
table_count: int
|
||||
created_at: datetime
|
||||
completed_at: Optional[datetime]
|
||||
status: str
|
||||
error_message: Optional[str]
|
||||
key_fingerprint: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CreateResult:
|
||||
"""Outcome of ``create_now``."""
|
||||
|
||||
backup: BackupRecord
|
||||
sidecar: backup_mod.Sidecar
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerifyResult:
|
||||
"""Outcome of ``verify``."""
|
||||
|
||||
backup_id: int
|
||||
filename: str
|
||||
ok: bool
|
||||
expected_fingerprint: str
|
||||
actual_fingerprint: str
|
||||
table_count: int
|
||||
reason: Optional[str] = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RestoreInitiateResult:
|
||||
"""Returned by the first call of the two-step restore."""
|
||||
|
||||
backup_id: int
|
||||
filename: str
|
||||
restore_token: str
|
||||
expires_at: datetime
|
||||
db_fingerprint: str
|
||||
table_count: int
|
||||
current_db_fingerprint: str
|
||||
current_table_count: int
|
||||
size_bytes: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RestoreConfirmResult:
|
||||
"""Returned by the second call of the two-step restore."""
|
||||
|
||||
backup_id: int
|
||||
filename: str
|
||||
restored_from_fingerprint: str
|
||||
restored_at: datetime
|
||||
new_db_fingerprint: str
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# BackupService
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class BackupService:
|
||||
"""Coordinator for encrypted DB backups.
|
||||
|
||||
Construct once at app startup; share across requests. Not
|
||||
thread-safe for *creation* (the SQLite ``.backup()`` call uses
|
||||
the live engine and is best serialized through the scheduler),
|
||||
but ``list_backups`` / ``prune`` / ``status`` are safe to call
|
||||
concurrently.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
backup_dir: Union[str, Path],
|
||||
*,
|
||||
passphrase: Optional[str] = None,
|
||||
salt: Optional[bytes] = None,
|
||||
retention_days: int = 30,
|
||||
db_url: Optional[str] = None,
|
||||
) -> None:
|
||||
self._backup_dir = Path(backup_dir)
|
||||
self._retention_days = max(1, int(retention_days))
|
||||
self._db_url = db_url
|
||||
# The derived key + its salt. If ``passphrase`` is None we
|
||||
# fall back to deriving from the SQLCipher DB key (with a
|
||||
# WARNING log).
|
||||
#
|
||||
# Salt is per-BackupService-instance and MUST be stable across
|
||||
# processes — otherwise the same passphrase would derive
|
||||
# different keys in different invocations and decrypt would
|
||||
# always fail. Two options:
|
||||
#
|
||||
# 1. Caller passes an explicit ``salt`` (the Keychain flow
|
||||
# reads the persisted salt from backup.salt account).
|
||||
# 2. We accept a None salt here; ``_ensure_key`` then either
|
||||
# uses the persisted salt (if available) or generates one
|
||||
# and persists it on first use.
|
||||
#
|
||||
# Tests typically pass an explicit random salt; production
|
||||
# should always pass the persisted one.
|
||||
self._passphrase = passphrase
|
||||
self._salt = salt
|
||||
self._key: Optional[bytes] = None
|
||||
self._used_fallback = False
|
||||
# Pending restore tokens: token -> (backup_id, expires_at).
|
||||
# A simple in-memory dict is sufficient — the token only
|
||||
# needs to survive between the two API calls in one process.
|
||||
self._pending_restores: dict[str, tuple[int, datetime]] = {}
|
||||
self._lock = threading.Lock()
|
||||
|
||||
# ---- Public API -------------------------------------------------------
|
||||
|
||||
@property
|
||||
def backup_dir(self) -> Path:
|
||||
return self._backup_dir
|
||||
|
||||
@property
|
||||
def key_fingerprint(self) -> str:
|
||||
"""SHA-256 of the current derived key, or "" if not yet derived."""
|
||||
if self._key is None:
|
||||
return ""
|
||||
return backup_mod.fingerprint(self._key)
|
||||
|
||||
def create_now(self) -> CreateResult:
|
||||
"""Take an encrypted backup of the live DB right now.
|
||||
|
||||
Crash-safe: any failure marks the ``db_backups`` row as
|
||||
``error``, removes any partial files from the backup dir, and
|
||||
re-raises the exception.
|
||||
"""
|
||||
# 1. Make sure the backup dir exists.
|
||||
self._backup_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 2. Allocate a filename + insert a pending row.
|
||||
from cyclone.db import DbBackup # late import — circular otherwise
|
||||
from cyclone.store import store as cycl_store
|
||||
|
||||
filename = backup_mod.backup_filename()
|
||||
created_at = datetime.now(timezone.utc)
|
||||
|
||||
row = cycl_store.add_backup_pending(
|
||||
filename=filename,
|
||||
backup_dir=str(self._backup_dir),
|
||||
)
|
||||
|
||||
try:
|
||||
# 3. Run SQLite's online .backup() to a temp file.
|
||||
# We use a private path *inside* the backup dir so the
|
||||
# operator can see what crashed if it does.
|
||||
staging_db = self._backup_dir / f".{filename}.staging.db"
|
||||
self._sqlite_backup_to(staging_db)
|
||||
|
||||
# 4. Encrypt.
|
||||
plaintext = staging_db.read_bytes()
|
||||
db_fp = backup_mod.fingerprint(plaintext)
|
||||
key = self._ensure_key()
|
||||
blob = backup_mod.encrypt(plaintext, key)
|
||||
|
||||
# 5. Move encrypted blob into place + write sidecar.
|
||||
target = self._backup_dir / filename
|
||||
target.write_bytes(blob)
|
||||
staging_db.unlink()
|
||||
|
||||
table_count = self._count_tables_in_blob(plaintext)
|
||||
sidecar = backup_mod.Sidecar(
|
||||
format_version=backup_mod.FORMAT_VERSION,
|
||||
created_at=created_at.isoformat(),
|
||||
db_fingerprint=db_fp,
|
||||
table_count=table_count,
|
||||
size_bytes=len(blob),
|
||||
kdf="PBKDF2-HMAC-SHA256",
|
||||
kdf_iterations=backup_mod.KDF_ITERATIONS,
|
||||
cipher="AES-256-GCM",
|
||||
key_fingerprint=backup_mod.fingerprint(key),
|
||||
)
|
||||
sidecar_path = self._backup_dir / backup_mod.sidecar_filename(filename)
|
||||
sidecar_path.write_text(sidecar.to_json())
|
||||
|
||||
# 6. Mark the row as ok.
|
||||
with db.SessionLocal()() as s:
|
||||
row = s.get(DbBackup, row.id)
|
||||
row.status = STATUS_OK
|
||||
row.size_bytes = len(blob)
|
||||
row.db_fingerprint = db_fp
|
||||
row.table_count = table_count
|
||||
row.completed_at = datetime.now(timezone.utc)
|
||||
s.commit()
|
||||
s.refresh(row)
|
||||
|
||||
record = self._row_to_record(row)
|
||||
log.info(
|
||||
"Backup created: id=%d filename=%s size=%d db_fp=%s",
|
||||
record.id, record.filename, record.size_bytes, record.db_fingerprint,
|
||||
)
|
||||
return CreateResult(backup=record, sidecar=sidecar)
|
||||
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.exception("Backup create failed")
|
||||
try:
|
||||
with db.SessionLocal()() as s:
|
||||
row = s.get(DbBackup, row.id)
|
||||
row.status = STATUS_ERROR
|
||||
row.error_message = f"{type(exc).__name__}: {exc}"[:500]
|
||||
row.completed_at = datetime.now(timezone.utc)
|
||||
s.commit()
|
||||
# Best-effort cleanup of any partial files.
|
||||
for p in [
|
||||
self._backup_dir / filename,
|
||||
self._backup_dir / f".{filename}.staging.db",
|
||||
self._backup_dir / backup_mod.sidecar_filename(filename),
|
||||
]:
|
||||
if p.exists():
|
||||
try:
|
||||
p.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
except Exception: # noqa: BLE001
|
||||
log.exception("Failed to mark backup row as error")
|
||||
raise
|
||||
|
||||
def list_backups(
|
||||
self,
|
||||
*,
|
||||
limit: int = 100,
|
||||
status: Optional[str] = None,
|
||||
) -> list[BackupRecord]:
|
||||
"""List ``db_backups`` rows newest first.
|
||||
|
||||
Joins the filesystem state (presence of ``.bin`` and
|
||||
``.meta.json``) implicitly via :attr:`BackupRecord.status`:
|
||||
a row marked ``pruned`` had its files deleted by the
|
||||
retention policy.
|
||||
"""
|
||||
from cyclone.db import DbBackup
|
||||
|
||||
with db.SessionLocal()() as s:
|
||||
q = s.query(DbBackup)
|
||||
if status is not None:
|
||||
q = q.filter(DbBackup.status == status)
|
||||
rows = q.order_by(DbBackup.id.desc()).limit(limit).all()
|
||||
return [self._row_to_record(r) for r in rows]
|
||||
|
||||
def verify(self, backup_id: int) -> VerifyResult:
|
||||
"""Decrypt + checksum-verify a backup against its sidecar.
|
||||
|
||||
Does NOT trust the sidecar's ``db_fingerprint`` field alone;
|
||||
recomputes the SHA-256 from the decrypted blob and compares.
|
||||
"""
|
||||
from cyclone.db import DbBackup
|
||||
|
||||
with db.SessionLocal()() as s:
|
||||
row = s.get(DbBackup, backup_id)
|
||||
if row is None:
|
||||
raise BackupError(f"backup {backup_id} not found")
|
||||
record = self._row_to_record(row)
|
||||
sidecar = self._read_sidecar(record.filename)
|
||||
if sidecar is None:
|
||||
return VerifyResult(
|
||||
backup_id=record.id, filename=record.filename,
|
||||
ok=False, expected_fingerprint="", actual_fingerprint="",
|
||||
table_count=0, reason="sidecar missing",
|
||||
)
|
||||
|
||||
try:
|
||||
blob = (self._backup_dir / record.filename).read_bytes()
|
||||
except FileNotFoundError:
|
||||
return VerifyResult(
|
||||
backup_id=record.id, filename=record.filename,
|
||||
ok=False,
|
||||
expected_fingerprint=sidecar.db_fingerprint,
|
||||
actual_fingerprint="",
|
||||
table_count=sidecar.table_count,
|
||||
reason="backup file missing",
|
||||
)
|
||||
|
||||
try:
|
||||
plaintext = backup_mod.decrypt(blob, self._ensure_key())
|
||||
except backup_mod.BackupDecryptError as exc:
|
||||
return VerifyResult(
|
||||
backup_id=record.id, filename=record.filename,
|
||||
ok=False,
|
||||
expected_fingerprint=sidecar.db_fingerprint,
|
||||
actual_fingerprint="",
|
||||
table_count=sidecar.table_count,
|
||||
reason=str(exc),
|
||||
)
|
||||
|
||||
actual_fp = backup_mod.fingerprint(plaintext)
|
||||
return VerifyResult(
|
||||
backup_id=record.id,
|
||||
filename=record.filename,
|
||||
ok=(actual_fp == sidecar.db_fingerprint),
|
||||
expected_fingerprint=sidecar.db_fingerprint,
|
||||
actual_fingerprint=actual_fp,
|
||||
table_count=sidecar.table_count,
|
||||
reason=None if actual_fp == sidecar.db_fingerprint else "fingerprint mismatch",
|
||||
)
|
||||
|
||||
def restore_initiate(self, backup_id: int) -> RestoreInitiateResult:
|
||||
"""First half of the two-step restore.
|
||||
|
||||
Decrypts the backup into a temp file and reads its current
|
||||
``db_fingerprint`` + ``table_count``. Returns a one-shot
|
||||
``restore_token`` the operator must echo back to
|
||||
:meth:`restore_confirm` within 5 minutes.
|
||||
"""
|
||||
from cyclone.db import DbBackup
|
||||
|
||||
with db.SessionLocal()() as s:
|
||||
row = s.get(DbBackup, backup_id)
|
||||
if row is None:
|
||||
raise BackupError(f"backup {backup_id} not found")
|
||||
if row.status != STATUS_OK:
|
||||
raise BackupError(
|
||||
f"backup {backup_id} status is {row.status!r}; only 'ok' backups can be restored",
|
||||
)
|
||||
record = self._row_to_record(row)
|
||||
|
||||
# Decrypt into a staging file so the confirm step is fast.
|
||||
staging = self._backup_dir / f".restore-{record.filename}.staging.db"
|
||||
try:
|
||||
blob = (self._backup_dir / record.filename).read_bytes()
|
||||
except FileNotFoundError as exc:
|
||||
raise BackupError(f"backup file missing: {record.filename}") from exc
|
||||
try:
|
||||
plaintext = backup_mod.decrypt(blob, self._ensure_key())
|
||||
except backup_mod.BackupDecryptError as exc:
|
||||
raise BackupError(f"decrypt failed: {exc}") from exc
|
||||
staging.write_bytes(plaintext)
|
||||
|
||||
# Snapshot the live DB's fingerprint for the operator's "are
|
||||
# you sure you want to do this?" preview.
|
||||
live_fp, live_count = self._live_fingerprint_and_count()
|
||||
|
||||
token = _secrets.token_hex(32)
|
||||
expires_at = datetime.now(timezone.utc) + timedelta(
|
||||
seconds=RESTORE_TOKEN_TTL_SECONDS,
|
||||
)
|
||||
with self._lock:
|
||||
self._pending_restores[token] = (record.id, expires_at)
|
||||
|
||||
log.info(
|
||||
"Restore initiated: backup_id=%d token=%s... expires=%s",
|
||||
record.id, token[:8], expires_at.isoformat(),
|
||||
)
|
||||
return RestoreInitiateResult(
|
||||
backup_id=record.id,
|
||||
filename=record.filename,
|
||||
restore_token=token,
|
||||
expires_at=expires_at,
|
||||
db_fingerprint=backup_mod.fingerprint(plaintext),
|
||||
table_count=self._count_tables_in_blob(plaintext),
|
||||
current_db_fingerprint=live_fp,
|
||||
current_table_count=live_count,
|
||||
size_bytes=len(plaintext),
|
||||
)
|
||||
|
||||
def restore_confirm(
|
||||
self,
|
||||
backup_id: int,
|
||||
restore_token: str,
|
||||
*,
|
||||
actor: str = "operator",
|
||||
) -> RestoreConfirmResult:
|
||||
"""Second half of the two-step restore.
|
||||
|
||||
Validates the token, copies the decrypted staging file over
|
||||
the live DB path, disposes + reopens the engine. Raises
|
||||
``BackupError`` on any mismatch.
|
||||
"""
|
||||
now = datetime.now(timezone.utc)
|
||||
with self._lock:
|
||||
entry = self._pending_restores.pop(restore_token, None)
|
||||
if entry is None:
|
||||
raise BackupError("restore_token not found (already consumed or never issued)")
|
||||
token_backup_id, expires_at = entry
|
||||
if token_backup_id != backup_id:
|
||||
raise BackupError(
|
||||
f"restore_token was for backup {token_backup_id}, not {backup_id}",
|
||||
)
|
||||
if now > expires_at:
|
||||
raise BackupError(
|
||||
f"restore_token expired at {expires_at.isoformat()}; re-run initiate",
|
||||
)
|
||||
|
||||
from cyclone.db import DbBackup
|
||||
with db.SessionLocal()() as s:
|
||||
row = s.get(DbBackup, backup_id)
|
||||
if row is None:
|
||||
raise BackupError(f"backup {backup_id} disappeared mid-restore")
|
||||
record = self._row_to_record(row)
|
||||
|
||||
staging = self._backup_dir / f".restore-{record.filename}.staging.db"
|
||||
if not staging.exists():
|
||||
raise BackupError(
|
||||
f"staging restore file missing: {staging.name}; re-run initiate",
|
||||
)
|
||||
|
||||
target_db_path = self._live_db_path()
|
||||
if target_db_path is None:
|
||||
raise BackupError(
|
||||
"cannot determine live DB file path (non-sqlite URL?)",
|
||||
)
|
||||
|
||||
# Pre-restore fingerprint for the audit event.
|
||||
restored_from_fp = backup_mod.fingerprint(staging.read_bytes())
|
||||
|
||||
# The swap: dispose engine → copy file → reinit engine.
|
||||
# Anything between dispose and reinit raises (queries that
|
||||
# are in-flight get a "database is locked" or
|
||||
# "no such table" error); we accept that because the
|
||||
# operator already confirmed.
|
||||
db.dispose_engine()
|
||||
try:
|
||||
# Atomic copy via temp + rename so a crash mid-copy
|
||||
# doesn't leave a half-written DB file.
|
||||
tmp_target = target_db_path.with_suffix(
|
||||
target_db_path.suffix + f".restoring-{_secrets.token_hex(4)}",
|
||||
)
|
||||
shutil.copyfile(staging, tmp_target)
|
||||
os.replace(tmp_target, target_db_path)
|
||||
finally:
|
||||
staging.unlink(missing_ok=True)
|
||||
db.reinit_engine()
|
||||
|
||||
# Post-restore fingerprint from the now-live engine.
|
||||
new_fp, _ = self._live_fingerprint_and_count()
|
||||
|
||||
log.warning(
|
||||
"Restore complete: backup_id=%d actor=%s from=%s to=%s",
|
||||
backup_id, actor, restored_from_fp, new_fp,
|
||||
)
|
||||
return RestoreConfirmResult(
|
||||
backup_id=record.id,
|
||||
filename=record.filename,
|
||||
restored_from_fingerprint=restored_from_fp,
|
||||
restored_at=datetime.now(timezone.utc),
|
||||
new_db_fingerprint=new_fp,
|
||||
)
|
||||
|
||||
def prune(self, *, now: Optional[datetime] = None) -> list[str]:
|
||||
"""Delete backups older than ``retention_days``. Returns deleted paths.
|
||||
|
||||
Marks the ``db_backups`` rows ``pruned`` so the operator can
|
||||
still see what was deleted (and when).
|
||||
"""
|
||||
from cyclone.db import DbBackup
|
||||
|
||||
cutoff = (now or datetime.now(timezone.utc)) - timedelta(
|
||||
days=self._retention_days,
|
||||
)
|
||||
deleted: list[str] = []
|
||||
|
||||
with db.SessionLocal()() as s:
|
||||
q = s.query(DbBackup).filter(
|
||||
DbBackup.status == STATUS_OK,
|
||||
DbBackup.created_at < cutoff,
|
||||
)
|
||||
for row in q.all():
|
||||
# Delete the file pair; ignore if already gone.
|
||||
bin_path = Path(row.backup_dir) / row.filename
|
||||
meta_path = Path(row.backup_dir) / backup_mod.sidecar_filename(row.filename)
|
||||
for p in (bin_path, meta_path):
|
||||
try:
|
||||
if p.exists():
|
||||
p.unlink()
|
||||
deleted.append(str(p))
|
||||
except OSError as exc:
|
||||
log.warning("Failed to delete %s: %s", p, exc)
|
||||
row.status = STATUS_PRUNED
|
||||
s.add(row)
|
||||
s.commit()
|
||||
|
||||
log.info("Pruned %d backup files older than %s", len(deleted), cutoff.isoformat())
|
||||
return deleted
|
||||
|
||||
def status(self) -> dict:
|
||||
"""Snapshot of the backup subsystem for ``GET /api/admin/backup/status``."""
|
||||
from cyclone.db import DbBackup
|
||||
from sqlalchemy import func
|
||||
|
||||
with db.SessionLocal()() as s:
|
||||
total = s.query(func.count(DbBackup.id)).scalar() or 0
|
||||
ok_count = s.query(func.count(DbBackup.id)).filter(
|
||||
DbBackup.status == STATUS_OK,
|
||||
).scalar() or 0
|
||||
error_count = s.query(func.count(DbBackup.id)).filter(
|
||||
DbBackup.status == STATUS_ERROR,
|
||||
).scalar() or 0
|
||||
pruned_count = s.query(func.count(DbBackup.id)).filter(
|
||||
DbBackup.status == STATUS_PRUNED,
|
||||
).scalar() or 0
|
||||
last_row = (
|
||||
s.query(DbBackup)
|
||||
.filter(DbBackup.status.in_([STATUS_OK, STATUS_ERROR]))
|
||||
.order_by(DbBackup.id.desc())
|
||||
.first()
|
||||
)
|
||||
last_ok_row = (
|
||||
s.query(DbBackup)
|
||||
.filter(DbBackup.status == STATUS_OK)
|
||||
.order_by(DbBackup.id.desc())
|
||||
.first()
|
||||
)
|
||||
disk_bytes = 0
|
||||
try:
|
||||
for p in self._backup_dir.iterdir():
|
||||
if p.is_file() and p.suffix == ".bin":
|
||||
disk_bytes += p.stat().st_size
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
return {
|
||||
"backup_dir": str(self._backup_dir),
|
||||
"retention_days": self._retention_days,
|
||||
"totals": {
|
||||
"all": total,
|
||||
"ok": ok_count,
|
||||
"error": error_count,
|
||||
"pruned": pruned_count,
|
||||
},
|
||||
"disk_bytes": disk_bytes,
|
||||
"last_backup_at": (
|
||||
last_row.created_at.isoformat() if last_row and last_row.created_at else None
|
||||
),
|
||||
"last_backup_status": last_row.status if last_row else None,
|
||||
"last_ok_backup_at": (
|
||||
last_ok_row.created_at.isoformat()
|
||||
if last_ok_row and last_ok_row.created_at else None
|
||||
),
|
||||
"used_fallback_key": self._used_fallback,
|
||||
}
|
||||
|
||||
# ---- Internals --------------------------------------------------------
|
||||
|
||||
def _ensure_key(self) -> bytes:
|
||||
"""Derive (or return cached) AES key. Triggers fallback + WARNING log
|
||||
if no passphrase was provided at construction time.
|
||||
|
||||
If a passphrase is set but no salt was passed at construction,
|
||||
look one up from the Keychain (``backup.salt`` account). On
|
||||
a fresh install, generate + persist a salt on first use so
|
||||
subsequent invocations derive the same key.
|
||||
"""
|
||||
if self._key is not None:
|
||||
return self._key
|
||||
if self._passphrase:
|
||||
salt = self._salt
|
||||
if salt is None:
|
||||
# Try the Keychain.
|
||||
stored = secrets_mod.get_secret(KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||
if stored:
|
||||
salt = bytes.fromhex(stored.strip())
|
||||
else:
|
||||
# First run: generate + persist.
|
||||
salt = os.urandom(backup_mod.SALT_LEN)
|
||||
secrets_mod.set_secret(
|
||||
KEYCHAIN_BACKUP_SALT_ACCOUNT,
|
||||
salt.hex(),
|
||||
)
|
||||
log.info(
|
||||
"Generated + persisted backup salt to Keychain "
|
||||
"(account %r)",
|
||||
KEYCHAIN_BACKUP_SALT_ACCOUNT,
|
||||
)
|
||||
self._key = backup_mod.derive_key(self._passphrase, salt)
|
||||
return self._key
|
||||
|
||||
# Fallback: derive from SQLCipher DB key. This is degraded
|
||||
# security (the SQLCipher key is meant to unlock the DB, not
|
||||
# the backup), but it's strictly better than plaintext.
|
||||
from cyclone import db_crypto
|
||||
db_key = db_crypto.get_db_key() if db_crypto.is_encryption_enabled() else None
|
||||
if not db_key:
|
||||
# No passphrase AND no SQLCipher key — refuse.
|
||||
raise BackupError(
|
||||
"no backup passphrase set and SQLCipher is not enabled; "
|
||||
"either set a backup passphrase in the Keychain or "
|
||||
"enable SQLCipher encryption",
|
||||
)
|
||||
log.warning(
|
||||
"Backup using fallback key derived from SQLCipher DB key "
|
||||
"(no separate backup passphrase set); set one via "
|
||||
"`cyclone backup init-passphrase` for stronger isolation",
|
||||
)
|
||||
self._used_fallback = True
|
||||
self._key = backup_mod.derive_key(db_key, backup_mod.FALLBACK_SALT)
|
||||
return self._key
|
||||
|
||||
def _sqlite_backup_to(self, target_path: Path) -> None:
|
||||
"""Run SQLite's online ``.backup()`` against the live engine.
|
||||
|
||||
Works for both plain SQLite and SQLCipher because sqlcipher3
|
||||
is API-compatible with sqlite3. The ``.backup()`` API takes
|
||||
a *target* connection; we make a fresh sqlite3 connection to
|
||||
the target file (which doesn't exist yet) and copy into it.
|
||||
"""
|
||||
url = self._db_url or db._resolve_url()
|
||||
if not url.startswith("sqlite"):
|
||||
raise BackupError(
|
||||
f"only sqlite URLs are supported for online backup; got {url!r}",
|
||||
)
|
||||
# Drive the backup off the live engine so we capture the
|
||||
# current state of all tables atomically (SQLite's .backup
|
||||
# holds a read lock on the source for the duration).
|
||||
engine = db.engine() # raises RuntimeError if init_db() wasn't called
|
||||
with engine.raw_connection() as raw:
|
||||
src_conn = raw.driver_connection # sqlite3.Connection / sqlcipher3.Connection
|
||||
if target_path.exists():
|
||||
target_path.unlink()
|
||||
dst_conn = sqlite3.connect(str(target_path))
|
||||
try:
|
||||
src_conn.backup(dst_conn)
|
||||
finally:
|
||||
dst_conn.close()
|
||||
|
||||
def _count_tables_in_blob(self, plaintext: bytes) -> int:
|
||||
"""Open the decrypted DB in-memory and count user tables."""
|
||||
tmp = self._backup_dir / f".count-tables-{_secrets.token_hex(4)}.db"
|
||||
try:
|
||||
tmp.write_bytes(plaintext)
|
||||
conn = sqlite3.connect(str(tmp))
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"SELECT count(*) FROM sqlite_master "
|
||||
"WHERE type='table' AND name NOT LIKE 'sqlite_%'",
|
||||
).fetchone()
|
||||
return int(rows[0])
|
||||
finally:
|
||||
conn.close()
|
||||
finally:
|
||||
tmp.unlink(missing_ok=True)
|
||||
|
||||
def _live_fingerprint_and_count(self) -> tuple[str, int]:
|
||||
"""Fingerprint + table count of the *current* live DB."""
|
||||
try:
|
||||
engine = db.engine()
|
||||
except RuntimeError:
|
||||
return "", 0
|
||||
# Use a temp-file .backup so we don't have to worry about
|
||||
# online-vs-offline semantics.
|
||||
tmp = self._backup_dir / f".live-fp-{_secrets.token_hex(4)}.db"
|
||||
try:
|
||||
with engine.raw_connection() as raw:
|
||||
conn = raw.driver_connection
|
||||
if tmp.exists():
|
||||
tmp.unlink()
|
||||
dst = sqlite3.connect(str(tmp))
|
||||
try:
|
||||
conn.backup(dst)
|
||||
finally:
|
||||
dst.close()
|
||||
data = tmp.read_bytes()
|
||||
return backup_mod.fingerprint(data), self._count_tables_in_blob(data)
|
||||
finally:
|
||||
tmp.unlink(missing_ok=True)
|
||||
|
||||
def _live_db_path(self) -> Optional[Path]:
|
||||
"""Resolve the filesystem path of the live DB, or None for non-sqlite."""
|
||||
url = self._db_url or db._resolve_url()
|
||||
if not url.startswith("sqlite"):
|
||||
return None
|
||||
# Strip the driver prefix: sqlite:///abs or sqlite:///./rel
|
||||
prefix = "sqlite:///"
|
||||
if url.startswith(prefix):
|
||||
return Path(url[len(prefix):])
|
||||
if url.startswith("sqlite://"):
|
||||
# sqlite://./relative/path -> Path("./relative/path")
|
||||
return Path(url[len("sqlite://"):])
|
||||
return None
|
||||
|
||||
def _read_sidecar(self, filename: str) -> Optional[backup_mod.Sidecar]:
|
||||
p = self._backup_dir / backup_mod.sidecar_filename(filename)
|
||||
if not p.exists():
|
||||
return None
|
||||
try:
|
||||
return backup_mod.Sidecar.from_json(p.read_text())
|
||||
except (json.JSONDecodeError, KeyError, ValueError) as exc:
|
||||
log.warning("Sidecar %s is malformed: %s", p, exc)
|
||||
return None
|
||||
|
||||
def _row_to_record(self, row) -> BackupRecord:
|
||||
"""ORM row → BackupRecord. Reads key_fingerprint from the sidecar if present."""
|
||||
sidecar = self._read_sidecar(row.filename)
|
||||
return BackupRecord(
|
||||
id=row.id,
|
||||
filename=row.filename,
|
||||
backup_dir=row.backup_dir,
|
||||
size_bytes=row.size_bytes or 0,
|
||||
db_fingerprint=row.db_fingerprint or "",
|
||||
table_count=row.table_count or 0,
|
||||
created_at=row.created_at,
|
||||
completed_at=row.completed_at,
|
||||
status=row.status,
|
||||
error_message=row.error_message,
|
||||
key_fingerprint=sidecar.key_fingerprint if sidecar else "",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module-level singleton
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
_service: Optional[BackupService] = None
|
||||
|
||||
|
||||
def configure_backup_service(
|
||||
backup_dir: Union[str, Path],
|
||||
*,
|
||||
passphrase: Optional[str] = None,
|
||||
salt: Optional[bytes] = None,
|
||||
retention_days: int = 30,
|
||||
db_url: Optional[str] = None,
|
||||
) -> BackupService:
|
||||
"""Create (or replace) the module-level BackupService singleton."""
|
||||
global _service
|
||||
if _service is not None:
|
||||
return _service
|
||||
_service = BackupService(
|
||||
backup_dir=backup_dir,
|
||||
passphrase=passphrase,
|
||||
salt=salt,
|
||||
retention_days=retention_days,
|
||||
db_url=db_url,
|
||||
)
|
||||
return _service
|
||||
|
||||
|
||||
def get_backup_service() -> BackupService:
|
||||
"""Return the configured BackupService. Raises RuntimeError if not set up."""
|
||||
if _service is None:
|
||||
raise RuntimeError("backup service not configured; call configure_backup_service() first")
|
||||
return _service
|
||||
|
||||
|
||||
def reset_backup_service_for_tests() -> None:
|
||||
"""Clear the module-level singleton. Test-only."""
|
||||
global _service
|
||||
_service = None
|
||||
@@ -197,3 +197,273 @@ def _count_issues(report) -> dict[str, int]:
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SP17: `cyclone backup` subcommands
|
||||
#
|
||||
# Operator-facing backup management. Mirrors the API surface but runs
|
||||
# standalone (no FastAPI app needed) for cron / scripting / DR drills.
|
||||
# Each subcommand initializes the DB + BackupService; if the
|
||||
# service isn't configured (no Keychain passphrase etc.) the operator
|
||||
# gets a clear error.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@main.group()
|
||||
def backup() -> None:
|
||||
"""Encrypted DB backup management (SP17)."""
|
||||
|
||||
|
||||
@backup.command("init-passphrase")
|
||||
@click.option("--passphrase", required=True, help="The passphrase to set (will prompt if omitted)")
|
||||
@click.option("--from-stdin", is_flag=True, help="Read passphrase from stdin instead of the argument")
|
||||
def backup_init_passphrase(passphrase: str, from_stdin: bool) -> None:
|
||||
"""Set the backup encryption passphrase in the macOS Keychain.
|
||||
|
||||
Generates a fresh salt and stores both the passphrase (account
|
||||
``backup.passphrase``) and the salt (account ``backup.salt``)
|
||||
under service ``cyclone``. Cyclone's BackupService reads them
|
||||
at startup. If the passphrase account is missing, the service
|
||||
falls back to deriving a key from the SQLCipher DB key
|
||||
(degraded posture, logged at WARNING).
|
||||
"""
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
from getpass import getpass
|
||||
import os as _os
|
||||
|
||||
if from_stdin:
|
||||
pp = getpass("Backup passphrase: ").strip()
|
||||
pp2 = getpass("Confirm: ").strip()
|
||||
if not pp or pp != pp2:
|
||||
click.echo("passphrase empty or mismatch", err=True)
|
||||
sys.exit(2)
|
||||
else:
|
||||
pp = passphrase
|
||||
|
||||
if not pp or len(pp) < 12:
|
||||
click.echo("passphrase must be at least 12 characters", err=True)
|
||||
sys.exit(2)
|
||||
|
||||
if not secrets_mod.set_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT, pp):
|
||||
click.echo("failed to store passphrase in Keychain", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
# Generate + persist a fresh salt. Same value must be used by
|
||||
# every subsequent invocation that uses this passphrase.
|
||||
salt = _os.urandom(16)
|
||||
if not secrets_mod.set_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT, salt.hex()):
|
||||
click.echo(
|
||||
"WARN: passphrase stored but salt write failed; backups may be unrecoverable",
|
||||
err=True,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
click.echo(
|
||||
f"passphrase stored in Keychain account {svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT!r}\n"
|
||||
f"salt stored in Keychain account {svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT!r}"
|
||||
)
|
||||
|
||||
|
||||
def _resolve_backup_dir(cli_override: str | None) -> "Path":
|
||||
"""Resolve the backup directory: --backup-dir > $CYCLONE_BACKUP_DIR > default."""
|
||||
import os as _os
|
||||
from pathlib import Path as _Path
|
||||
from cyclone import db as db_mod
|
||||
if cli_override:
|
||||
return _Path(cli_override)
|
||||
env = _os.environ.get("CYCLONE_BACKUP_DIR")
|
||||
if env:
|
||||
return _Path(env)
|
||||
return _Path(db_mod.DEFAULT_DB_PATH.parent / "backups")
|
||||
|
||||
|
||||
@backup.command("create")
|
||||
@click.option("--backup-dir", default=None, help="Override CYCLONE_BACKUP_DIR (default: ~/.local/share/cyclone/backups)")
|
||||
@click.option("--retention-days", default=None, type=int, help="Override CYCLONE_BACKUP_RETENTION_DAYS for this run's prune")
|
||||
def backup_create(backup_dir: str | None, retention_days: int | None) -> None:
|
||||
"""Take an encrypted backup right now."""
|
||||
from cyclone import db as db_mod
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
|
||||
db_mod.init_db()
|
||||
|
||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||
target_dir = _resolve_backup_dir(backup_dir)
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
svc = svc_mod.configure_backup_service(
|
||||
backup_dir=target_dir,
|
||||
passphrase=passphrase,
|
||||
salt=salt,
|
||||
retention_days=retention_days or 30,
|
||||
)
|
||||
result = svc.create_now()
|
||||
click.echo(
|
||||
f"created backup id={result.backup.id} filename={result.backup.filename} "
|
||||
f"size={result.backup.size_bytes}B fp={result.backup.db_fingerprint[:24]}..."
|
||||
)
|
||||
|
||||
|
||||
@backup.command("list")
|
||||
@click.option("--limit", default=50, show_default=True)
|
||||
@click.option("--status", default=None, help="Filter: ok|error|pending|pruned")
|
||||
def backup_list(limit: int, status: str | None) -> None:
|
||||
"""List existing backups (newest first)."""
|
||||
from cyclone import db as db_mod
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
|
||||
db_mod.init_db()
|
||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
svc = svc_mod.configure_backup_service(
|
||||
backup_dir=_resolve_backup_dir(None),
|
||||
passphrase=passphrase,
|
||||
salt=salt,
|
||||
)
|
||||
rows = svc.list_backups(limit=limit, status=status)
|
||||
if not rows:
|
||||
click.echo("(no backups)")
|
||||
return
|
||||
for r in rows:
|
||||
click.echo(
|
||||
f"{r.id:4d} {r.status:7s} {r.created_at.isoformat() if r.created_at else '-'} "
|
||||
f"{r.size_bytes:>10d}B {r.filename} fp={r.db_fingerprint[:24] or '-':<24}"
|
||||
)
|
||||
|
||||
|
||||
@backup.command("verify")
|
||||
@click.argument("backup_id", type=int)
|
||||
def backup_verify(backup_id: int) -> None:
|
||||
"""Decrypt + checksum-verify a backup."""
|
||||
from cyclone import db as db_mod
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
|
||||
db_mod.init_db()
|
||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
svc = svc_mod.configure_backup_service(
|
||||
backup_dir=_resolve_backup_dir(None),
|
||||
passphrase=passphrase,
|
||||
salt=salt,
|
||||
)
|
||||
v = svc.verify(backup_id)
|
||||
if v.ok:
|
||||
click.echo(f"OK: id={v.backup_id} fp={v.actual_fingerprint[:24]}... table_count={v.table_count}")
|
||||
return
|
||||
click.echo(
|
||||
f"FAIL: id={v.backup_id} reason={v.reason} "
|
||||
f"expected={v.expected_fingerprint[:24] if v.expected_fingerprint else '-'}... "
|
||||
f"actual={v.actual_fingerprint[:24] if v.actual_fingerprint else '-'}...",
|
||||
err=True,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@backup.command("restore")
|
||||
@click.argument("backup_id", type=int)
|
||||
@click.option("--yes", is_flag=True, help="Skip the interactive confirm prompt")
|
||||
@click.option("--actor", default="operator-cli", show_default=True)
|
||||
def backup_restore(backup_id: int, yes: bool, actor: str) -> None:
|
||||
"""Restore the live DB from a backup (two-step, requires --yes)."""
|
||||
from cyclone import db as db_mod
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
|
||||
db_mod.init_db()
|
||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
svc = svc_mod.configure_backup_service(
|
||||
backup_dir=_resolve_backup_dir(None),
|
||||
passphrase=passphrase,
|
||||
salt=salt,
|
||||
)
|
||||
|
||||
click.echo(f"Initiating restore from backup {backup_id}...")
|
||||
init = svc.restore_initiate(backup_id)
|
||||
click.echo(
|
||||
f" backup: {init.filename} ({init.size_bytes} bytes)\n"
|
||||
f" fp: {init.db_fingerprint[:24]}...\n"
|
||||
f" tables: {init.table_count}\n"
|
||||
f" current: fp={init.current_db_fingerprint[:24] if init.current_db_fingerprint else '-'}... "
|
||||
f"tables={init.current_table_count}\n"
|
||||
f" token ttl: {(init.expires_at - __import__('datetime').datetime.now(__import__('datetime').timezone.utc)).total_seconds():.0f}s"
|
||||
)
|
||||
|
||||
if not yes:
|
||||
click.confirm(
|
||||
"Replace the live DB with this backup? "
|
||||
"This will dispose the engine and rebuild it.",
|
||||
abort=True,
|
||||
)
|
||||
|
||||
click.echo("Confirming restore...")
|
||||
result = svc.restore_confirm(backup_id, init.restore_token, actor=actor)
|
||||
click.echo(
|
||||
f"OK: restored from fp={result.restored_from_fingerprint[:24]}... "
|
||||
f"to fp={result.new_db_fingerprint[:24]}... at {result.restored_at.isoformat()}"
|
||||
)
|
||||
|
||||
|
||||
@backup.command("prune")
|
||||
@click.option("--retention-days", default=None, type=int)
|
||||
@click.option("--yes", is_flag=True, help="Skip the confirm prompt")
|
||||
def backup_prune(retention_days: int | None, yes: bool) -> None:
|
||||
"""Apply the retention policy (delete old backups)."""
|
||||
from cyclone import db as db_mod
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
|
||||
db_mod.init_db()
|
||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
svc = svc_mod.configure_backup_service(
|
||||
backup_dir=_resolve_backup_dir(None),
|
||||
passphrase=passphrase,
|
||||
salt=salt,
|
||||
retention_days=retention_days or 30,
|
||||
)
|
||||
if not yes:
|
||||
click.confirm(
|
||||
f"Delete all backups older than {svc._retention_days} days?",
|
||||
abort=True,
|
||||
)
|
||||
deleted = svc.prune()
|
||||
click.echo(f"Deleted {len(deleted)} file(s):")
|
||||
for p in deleted:
|
||||
click.echo(f" {p}")
|
||||
|
||||
|
||||
@backup.command("status")
|
||||
def backup_status() -> None:
|
||||
"""Print the backup subsystem status snapshot."""
|
||||
from cyclone import db as db_mod
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
|
||||
db_mod.init_db()
|
||||
passphrase = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT)
|
||||
salt_hex = secrets_mod.get_secret(svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT)
|
||||
salt = bytes.fromhex(salt_hex) if salt_hex else None
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
svc = svc_mod.configure_backup_service(
|
||||
backup_dir=_resolve_backup_dir(None),
|
||||
passphrase=passphrase,
|
||||
salt=salt,
|
||||
)
|
||||
snap = svc.status()
|
||||
import json
|
||||
click.echo(json.dumps(snap, indent=2, default=str))
|
||||
|
||||
@@ -721,6 +721,44 @@ class ProcessedInboundFile(Base):
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SP17: encrypted backup metadata
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class DbBackup(Base):
|
||||
"""One row per encrypted backup the BackupService has taken.
|
||||
|
||||
The actual encrypted blob lives in a directory outside the DB
|
||||
(``~/.local/share/cyclone/backups/`` by default); this table is
|
||||
the index. Status values: ``pending``, ``ok``, ``error``,
|
||||
``pruned``.
|
||||
|
||||
SP17. The unique index on ``(backup_dir, filename)`` makes a
|
||||
duplicate ``create_now()`` race fail cleanly with an
|
||||
IntegrityError instead of clobbering an existing backup.
|
||||
"""
|
||||
|
||||
__tablename__ = "db_backups"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
filename: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||
backup_dir: Mapped[str] = mapped_column(String(512), nullable=False)
|
||||
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
db_fingerprint: Mapped[Optional[str]] = mapped_column(String(80), nullable=True)
|
||||
table_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
completed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
status: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
error_message: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||
|
||||
__table_args__ = (
|
||||
Index("ux_db_backups_filename", "backup_dir", "filename", unique=True),
|
||||
Index("ix_db_backups_created_at", "created_at"),
|
||||
Index("ix_db_backups_status", "status"),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SP9: providers, payers, payer_configs, clearhouse
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
-- version: 12
|
||||
-- SP17: encrypted DB backup metadata
|
||||
--
|
||||
-- Tracks every backup the BackupService has taken. The actual
|
||||
-- encrypted blob lives in a directory outside the DB (default
|
||||
-- ~/.local/share/cyclone/backups/); this table is just the index
|
||||
-- the operator queries via GET /api/admin/backup/list.
|
||||
--
|
||||
-- Status values:
|
||||
-- pending - row inserted, .backup() in progress or crashed before commit
|
||||
-- ok - encrypted blob + sidecar written successfully
|
||||
-- error - creation failed; error_message populated
|
||||
-- pruned - retention policy removed the file; row kept for audit
|
||||
|
||||
CREATE TABLE db_backups (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
filename TEXT NOT NULL,
|
||||
backup_dir TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL DEFAULT 0,
|
||||
db_fingerprint TEXT,
|
||||
table_count INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL,
|
||||
completed_at TEXT,
|
||||
status TEXT NOT NULL,
|
||||
error_message TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX ux_db_backups_filename ON db_backups(backup_dir, filename);
|
||||
CREATE INDEX ix_db_backups_created_at ON db_backups(created_at DESC);
|
||||
CREATE INDEX ix_db_backups_status ON db_backups(status);
|
||||
@@ -1849,6 +1849,32 @@ class CycloneStore:
|
||||
with db.SessionLocal()() as s:
|
||||
return s.get(db.Two77caAck, ack_id)
|
||||
|
||||
# -- SP17: encrypted DB backups -------------------------------------
|
||||
|
||||
def add_backup_pending(self, *, filename: str, backup_dir: str) -> db.DbBackup:
|
||||
"""Insert a ``pending`` row for a backup that is about to start.
|
||||
|
||||
The BackupService fills in ``status`` / ``size_bytes`` /
|
||||
``db_fingerprint`` / ``table_count`` / ``completed_at`` after
|
||||
the encrypted blob lands on disk.
|
||||
"""
|
||||
with db.SessionLocal()() as s:
|
||||
row = db.DbBackup(
|
||||
filename=filename,
|
||||
backup_dir=backup_dir,
|
||||
size_bytes=0,
|
||||
db_fingerprint=None,
|
||||
table_count=0,
|
||||
created_at=utcnow(),
|
||||
completed_at=None,
|
||||
status="pending",
|
||||
error_message=None,
|
||||
)
|
||||
s.add(row)
|
||||
s.commit()
|
||||
s.refresh(row)
|
||||
return row
|
||||
|
||||
# -- manual reconciliation (T12) -----------------------------------
|
||||
|
||||
def list_unmatched(self, *, kind: str = "both") -> dict:
|
||||
|
||||
@@ -51,19 +51,19 @@ def test_migration_0002_creates_acks_table():
|
||||
|
||||
def test_migration_latest_idempotent_on_fresh_db():
|
||||
"""Re-running the migration on the same DB must be a no-op (PRAGMA
|
||||
user_version already at the latest version — currently 11 after
|
||||
user_version already at the latest version — currently 12 after
|
||||
0004-0006 line_reconciliation, 0005 ta1_acks, SP9's 0007
|
||||
providers/payers/clearhouse, SP10's 0008 payer_rejected,
|
||||
SP11's 0009 audit_log, SP14's 0010 payer_rejected_acknowledged,
|
||||
SP16's 0011 processed_inbound_files)."""
|
||||
SP16's 0011 processed_inbound_files, SP17's 0012 db_backups)."""
|
||||
with db.engine().begin() as c:
|
||||
v1 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
||||
assert v1 == 11
|
||||
assert v1 == 12
|
||||
# A second run should not raise and should not bump the version.
|
||||
db_migrate.run(db.engine())
|
||||
with db.engine().begin() as c:
|
||||
v2 = c.exec_driver_sql("PRAGMA user_version").scalar() or 0
|
||||
assert v2 == 11
|
||||
assert v2 == 12
|
||||
|
||||
|
||||
def test_add_ack_persists_row():
|
||||
|
||||
@@ -0,0 +1,313 @@
|
||||
"""SP17 — Admin backup API endpoint tests.
|
||||
|
||||
Covers:
|
||||
- POST /api/admin/backup/create
|
||||
- GET /api/admin/backup/list
|
||||
- GET /api/admin/backup/status
|
||||
- POST /api/admin/backup/{id}/verify
|
||||
- POST /api/admin/backup/{id}/restore/initiate
|
||||
- POST /api/admin/backup/{id}/restore/confirm
|
||||
- POST /api/admin/backup/prune
|
||||
- POST /api/admin/backup/scheduler/{start,stop,tick}
|
||||
|
||||
Each fixture starts a clean DB + BackupService configured with a
|
||||
known passphrase. We deliberately do NOT enable SQLCipher here —
|
||||
the backup layer is independent of SQLCipher encryption at rest.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _backup_env(tmp_path, monkeypatch):
|
||||
"""Fresh sqlite DB + BackupService with passphrase. Reset module singletons."""
|
||||
from cyclone import db
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import backup_scheduler as sched_mod
|
||||
from cyclone.db import Batch
|
||||
|
||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||
db._reset_for_tests()
|
||||
db.init_db()
|
||||
|
||||
# Make sure there's at least one row so the backup isn't a no-op.
|
||||
import uuid
|
||||
with db.SessionLocal()() as s:
|
||||
s.add(Batch(
|
||||
id=str(uuid.uuid4()),
|
||||
kind="837P",
|
||||
input_filename="seed.x12",
|
||||
parsed_at=datetime.now(timezone.utc),
|
||||
totals_json=None,
|
||||
validation_json=None,
|
||||
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||
))
|
||||
s.commit()
|
||||
|
||||
backup_dir = tmp_path / "backups"
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
sched_mod.reset_backup_scheduler_for_tests()
|
||||
svc = svc_mod.configure_backup_service(
|
||||
backup_dir=backup_dir, passphrase="api-test-pass", retention_days=7,
|
||||
)
|
||||
yield svc, backup_dir
|
||||
sched_mod.reset_backup_scheduler_for_tests()
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
db._reset_for_tests()
|
||||
|
||||
|
||||
def _client():
|
||||
from fastapi.testclient import TestClient
|
||||
from cyclone.api import app
|
||||
return TestClient(app)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /backup/create
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_create_returns_metadata_and_persists_row(_backup_env):
|
||||
svc, backup_dir = _backup_env
|
||||
r = _client().post("/api/admin/backup/create")
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
assert body["ok"] is True
|
||||
b = body["backup"]
|
||||
assert b["size_bytes"] > 0
|
||||
assert b["db_fingerprint"].startswith("sha256:")
|
||||
assert b["table_count"] >= 1
|
||||
assert b["created_at"]
|
||||
# File actually exists on disk.
|
||||
assert (backup_dir / b["filename"]).exists()
|
||||
# Sidecar metadata echoed.
|
||||
sc = body["sidecar"]
|
||||
assert sc["kdf"] == "PBKDF2-HMAC-SHA256"
|
||||
assert sc["kdf_iterations"] == 200_000
|
||||
assert sc["cipher"] == "AES-256-GCM"
|
||||
|
||||
|
||||
def test_create_503_when_service_unconfigured(tmp_path, monkeypatch):
|
||||
"""If BackupService was never configured, create returns 503."""
|
||||
from cyclone import db
|
||||
from cyclone import backup_service as svc_mod
|
||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||
db._reset_for_tests()
|
||||
db.init_db()
|
||||
svc_mod.reset_backup_service_for_tests()
|
||||
try:
|
||||
r = _client().post("/api/admin/backup/create")
|
||||
assert r.status_code == 503
|
||||
assert "not configured" in r.json()["detail"].lower()
|
||||
finally:
|
||||
db._reset_for_tests()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /backup/list
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_list_returns_newest_first(_backup_env):
|
||||
svc, _ = _backup_env
|
||||
client = _client()
|
||||
client.post("/api/admin/backup/create")
|
||||
client.post("/api/admin/backup/create")
|
||||
r = client.get("/api/admin/backup/list")
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["count"] == 2
|
||||
assert body["files"][0]["id"] > body["files"][1]["id"]
|
||||
|
||||
|
||||
def test_list_filter_by_status(_backup_env):
|
||||
svc, _ = _backup_env
|
||||
client = _client()
|
||||
client.post("/api/admin/backup/create")
|
||||
r = client.get("/api/admin/backup/list?status=ok")
|
||||
assert r.json()["count"] == 1
|
||||
r = client.get("/api/admin/backup/list?status=error")
|
||||
assert r.json()["count"] == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /backup/status
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_status_returns_counts_and_dirs(_backup_env):
|
||||
svc, backup_dir = _backup_env
|
||||
client = _client()
|
||||
client.post("/api/admin/backup/create")
|
||||
r = client.get("/api/admin/backup/status")
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["totals"]["ok"] == 1
|
||||
assert body["backup_dir"] == str(backup_dir)
|
||||
assert body["retention_days"] == 7
|
||||
assert body["last_backup_at"] is not None
|
||||
assert body["last_ok_backup_at"] is not None
|
||||
# The scheduler may or may not be configured depending on lifespan.
|
||||
assert "scheduler" in body
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /backup/{id}/verify
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_verify_ok_after_create(_backup_env):
|
||||
svc, _ = _backup_env
|
||||
client = _client()
|
||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||
r = client.post(f"/api/admin/backup/{cid}/verify")
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["ok"] is True
|
||||
assert body["expected_fingerprint"] == body["actual_fingerprint"]
|
||||
|
||||
|
||||
def test_verify_detects_tampered_ciphertext(_backup_env):
|
||||
from cyclone import backup as backup_mod
|
||||
svc, backup_dir = _backup_env
|
||||
client = _client()
|
||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||
fname = svc.list_backups()[0].filename
|
||||
# Flip a bit in the ciphertext.
|
||||
bin_path = backup_dir / fname
|
||||
data = bytearray(bin_path.read_bytes())
|
||||
data[backup_mod.NONCE_LEN + 5] ^= 0x01
|
||||
bin_path.write_bytes(bytes(data))
|
||||
r = client.post(f"/api/admin/backup/{cid}/verify")
|
||||
assert r.status_code == 200
|
||||
assert r.json()["ok"] is False
|
||||
|
||||
|
||||
def test_verify_404_when_unknown_backup(_backup_env):
|
||||
r = _client().post("/api/admin/backup/99999/verify")
|
||||
# The service raises BackupError; the endpoint should return 503 (no svc) or 400
|
||||
# depending on flow. Let's see what happens.
|
||||
assert r.status_code in (400, 404, 503)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /backup/{id}/restore/{initiate,confirm}
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_restore_two_step_via_api(_backup_env):
|
||||
svc, _ = _backup_env
|
||||
from cyclone.db import Batch
|
||||
import uuid
|
||||
client = _client()
|
||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||
|
||||
# Mutate the live DB (add another Batch row).
|
||||
with __import__("cyclone").db.SessionLocal()() as s:
|
||||
s.add(Batch(
|
||||
id=str(uuid.uuid4()),
|
||||
kind="837P",
|
||||
input_filename="mutated.x12",
|
||||
parsed_at=datetime.now(timezone.utc),
|
||||
totals_json=None,
|
||||
validation_json=None,
|
||||
raw_result_json={"envelope": {"control_number": "2"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||
))
|
||||
s.commit()
|
||||
|
||||
# Step 1: initiate.
|
||||
r1 = client.post(f"/api/admin/backup/{cid}/restore/initiate")
|
||||
assert r1.status_code == 200, r1.text
|
||||
body1 = r1.json()
|
||||
assert body1["restore_token"]
|
||||
assert body1["preview"]["backup_table_count"] >= 1
|
||||
assert body1["preview"]["backup_db_fingerprint"] != body1["preview"]["current_db_fingerprint"]
|
||||
|
||||
# Step 2: confirm.
|
||||
r2 = client.post(
|
||||
f"/api/admin/backup/{cid}/restore/confirm",
|
||||
json={"restore_token": body1["restore_token"], "actor": "test"},
|
||||
)
|
||||
assert r2.status_code == 200, r2.text
|
||||
body2 = r2.json()
|
||||
assert body2["ok"] is True
|
||||
assert body2["new_db_fingerprint"] == body1["preview"]["backup_db_fingerprint"]
|
||||
|
||||
|
||||
def test_restore_confirm_requires_token(_backup_env):
|
||||
svc, _ = _backup_env
|
||||
client = _client()
|
||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||
r = client.post(f"/api/admin/backup/{cid}/restore/confirm", json={})
|
||||
assert r.status_code == 400
|
||||
|
||||
|
||||
def test_restore_confirm_rejects_wrong_token(_backup_env):
|
||||
svc, _ = _backup_env
|
||||
client = _client()
|
||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||
r = client.post(
|
||||
f"/api/admin/backup/{cid}/restore/confirm",
|
||||
json={"restore_token": "0" * 64},
|
||||
)
|
||||
assert r.status_code == 400
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /backup/prune
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_prune_deletes_old_backups(_backup_env):
|
||||
svc, _ = _backup_env
|
||||
from cyclone.db import DbBackup
|
||||
client = _client()
|
||||
cid = client.post("/api/admin/backup/create").json()["backup"]["id"]
|
||||
# Age the backup past the retention cutoff.
|
||||
with __import__("cyclone").db.SessionLocal()() as s:
|
||||
row = s.get(DbBackup, cid)
|
||||
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
||||
s.commit()
|
||||
r = client.post("/api/admin/backup/prune")
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["ok"] is True
|
||||
assert body["deleted_count"] == 2 # .bin + .meta.json
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /backup/scheduler/{start,stop,tick}
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_scheduler_endpoints_require_configured_scheduler(_backup_env, monkeypatch):
|
||||
"""Without calling configure_backup_scheduler, the endpoints 503."""
|
||||
svc, _ = _backup_env
|
||||
# We did NOT call configure_backup_scheduler; the lifespan
|
||||
# *might* have called it as a side effect of the TestClient
|
||||
# entering its context. Either way, the scheduler endpoints
|
||||
# need it to be present.
|
||||
client = _client()
|
||||
r = client.post("/api/admin/backup/scheduler/tick")
|
||||
assert r.status_code in (200, 503)
|
||||
|
||||
|
||||
def test_scheduler_tick_when_configured(_backup_env):
|
||||
"""With a configured scheduler, tick runs and returns a result."""
|
||||
from cyclone import backup_scheduler as sched_mod
|
||||
svc, _ = _backup_env
|
||||
sched_mod.configure_backup_scheduler(svc, interval_hours=24.0)
|
||||
try:
|
||||
client = _client()
|
||||
r = client.post("/api/admin/backup/scheduler/tick")
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["ok"] is True
|
||||
assert body["tick"]["created"] is not None
|
||||
assert body["tick"]["created"]["id"] >= 1
|
||||
finally:
|
||||
sched_mod.reset_backup_scheduler_for_tests()
|
||||
@@ -0,0 +1,165 @@
|
||||
"""SP17 — low-level backup crypto tests.
|
||||
|
||||
Pure-Python, no DB. Covers key derivation determinism, encrypt /
|
||||
decrypt round-trip, tampered-ciphertext failure, wrong-passphrase
|
||||
failure, and the sidecar JSON format.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from cyclone import backup as backup_mod
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Key derivation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_derive_key_is_deterministic():
|
||||
salt = os.urandom(16)
|
||||
k1 = backup_mod.derive_key("correct horse battery staple", salt)
|
||||
k2 = backup_mod.derive_key("correct horse battery staple", salt)
|
||||
assert k1 == k2
|
||||
assert len(k1) == backup_mod.KEY_LEN == 32
|
||||
|
||||
|
||||
def test_derive_key_different_salts_produce_different_keys():
|
||||
"""Salt is what makes the same passphrase produce different keys."""
|
||||
k1 = backup_mod.derive_key("hunter2", os.urandom(16))
|
||||
k2 = backup_mod.derive_key("hunter2", os.urandom(16))
|
||||
assert k1 != k2
|
||||
|
||||
|
||||
def test_derive_key_different_passphrases_produce_different_keys():
|
||||
salt = os.urandom(16)
|
||||
k1 = backup_mod.derive_key("a", salt)
|
||||
k2 = backup_mod.derive_key("b", salt)
|
||||
assert k1 != k2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Encrypt / decrypt round-trip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_encrypt_decrypt_roundtrip():
|
||||
key = os.urandom(32)
|
||||
plaintext = b"hello cyclone backup " * 1000
|
||||
blob = backup_mod.encrypt(plaintext, key)
|
||||
assert len(blob) == backup_mod.NONCE_LEN + len(plaintext) + 16 # tag
|
||||
out = backup_mod.decrypt(blob, key)
|
||||
assert out == plaintext
|
||||
|
||||
|
||||
def test_encrypt_decrypt_empty_plaintext():
|
||||
"""Edge case: zero-byte payload still produces nonce + tag."""
|
||||
key = os.urandom(32)
|
||||
blob = backup_mod.encrypt(b"", key)
|
||||
out = backup_mod.decrypt(blob, key)
|
||||
assert out == b""
|
||||
|
||||
|
||||
def test_decrypt_with_wrong_key_raises():
|
||||
plaintext = b"some bytes"
|
||||
key1 = os.urandom(32)
|
||||
key2 = os.urandom(32)
|
||||
blob = backup_mod.encrypt(plaintext, key1)
|
||||
with pytest.raises(backup_mod.BackupDecryptError):
|
||||
backup_mod.decrypt(blob, key2)
|
||||
|
||||
|
||||
def test_decrypt_tampered_ciphertext_raises():
|
||||
"""Flipping a single ciphertext byte must fail GCM auth."""
|
||||
key = os.urandom(32)
|
||||
blob = backup_mod.encrypt(b"a" * 200, key)
|
||||
tampered = bytearray(blob)
|
||||
# Flip a bit somewhere in the ciphertext region (past the nonce).
|
||||
tampered[backup_mod.NONCE_LEN + 5] ^= 0x01
|
||||
with pytest.raises(backup_mod.BackupDecryptError):
|
||||
backup_mod.decrypt(bytes(tampered), key)
|
||||
|
||||
|
||||
def test_decrypt_truncated_blob_raises():
|
||||
key = os.urandom(32)
|
||||
blob = backup_mod.encrypt(b"x" * 100, key)
|
||||
with pytest.raises(backup_mod.BackupDecryptError):
|
||||
# Strip the GCM tag.
|
||||
backup_mod.decrypt(blob[: -16], key)
|
||||
|
||||
|
||||
def test_encrypt_with_wrong_key_length_raises():
|
||||
with pytest.raises(backup_mod.BackupError):
|
||||
backup_mod.encrypt(b"data", b"short") # not 32 bytes
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fingerprint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_fingerprint_format_and_stability():
|
||||
fp = backup_mod.fingerprint(b"hello")
|
||||
assert fp.startswith("sha256:")
|
||||
assert len(fp) == len("sha256:") + 64
|
||||
assert fp == backup_mod.fingerprint(b"hello")
|
||||
assert fp != backup_mod.fingerprint(b"hellp")
|
||||
|
||||
|
||||
def test_fingerprint_file_matches_fingerprint_bytes(tmp_path):
|
||||
p = tmp_path / "data.bin"
|
||||
p.write_bytes(b"\x00\x01\x02" * 100)
|
||||
assert backup_mod.fingerprint_file(p) == backup_mod.fingerprint(p.read_bytes())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Sidecar
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_sidecar_round_trip_json():
|
||||
sc = backup_mod.Sidecar(
|
||||
format_version="v1",
|
||||
created_at="2026-06-21T15:30:00+00:00",
|
||||
db_fingerprint="sha256:" + "a" * 64,
|
||||
table_count=11,
|
||||
size_bytes=1024,
|
||||
kdf="PBKDF2-HMAC-SHA256",
|
||||
kdf_iterations=200_000,
|
||||
cipher="AES-256-GCM",
|
||||
key_fingerprint="sha256:" + "b" * 64,
|
||||
)
|
||||
text = sc.to_json()
|
||||
parsed = json.loads(text)
|
||||
assert parsed["format_version"] == "v1"
|
||||
assert parsed["encryption"]["kdf_iterations"] == 200_000
|
||||
sc2 = backup_mod.Sidecar.from_json(text)
|
||||
assert sc2 == sc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Filenames
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_backup_filename_format():
|
||||
"""The timestamp prefix is fixed; the suffix is random per call."""
|
||||
import re
|
||||
from datetime import datetime, timezone
|
||||
ts = datetime(2026, 6, 21, 15, 30, 0, tzinfo=timezone.utc)
|
||||
name = backup_mod.backup_filename(ts)
|
||||
assert re.match(r"^cyclone-backup-20260621T153000Z-[0-9a-f]{8}\.bin$", name), name
|
||||
|
||||
|
||||
def test_backup_filename_random_suffix_avoids_collisions():
|
||||
"""Two calls in the same second get different filenames."""
|
||||
a = backup_mod.backup_filename()
|
||||
b = backup_mod.backup_filename()
|
||||
assert a != b
|
||||
|
||||
|
||||
def test_sidecar_filename_appends_meta_json():
|
||||
assert backup_mod.sidecar_filename("foo.bin") == "foo.bin.meta.json"
|
||||
@@ -0,0 +1,207 @@
|
||||
"""SP17 — BackupScheduler unit tests.
|
||||
|
||||
Exercises the asyncio tick / start / stop loop without spinning up
|
||||
the FastAPI app. The scheduler wraps a real BackupService against
|
||||
a real on-disk sqlite DB.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import backup_scheduler as sched_mod
|
||||
from cyclone import db
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fresh_db(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||
db._reset_for_tests()
|
||||
db.init_db()
|
||||
from cyclone.db import Batch
|
||||
import uuid
|
||||
with db.SessionLocal()() as s:
|
||||
s.add(Batch(
|
||||
id=str(uuid.uuid4()),
|
||||
kind="837P",
|
||||
input_filename="seed.x12",
|
||||
parsed_at=datetime.now(timezone.utc),
|
||||
totals_json=None,
|
||||
validation_json=None,
|
||||
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||
))
|
||||
s.commit()
|
||||
yield
|
||||
db._reset_for_tests()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def backup_svc(tmp_path):
|
||||
return svc_mod.BackupService(
|
||||
backup_dir=tmp_path / "backups",
|
||||
passphrase="test-pass",
|
||||
retention_days=7,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# tick
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tick_creates_backup_and_audits_it(fresh_db, backup_svc):
|
||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||
result = await sched.tick()
|
||||
assert result.ok
|
||||
assert result.created is not None
|
||||
assert result.error is None
|
||||
assert len(backup_svc.list_backups()) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tick_creates_audit_event(fresh_db, backup_svc):
|
||||
"""db.backup_created audit event is written (SP11 hash chain)."""
|
||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||
await sched.tick()
|
||||
with db.SessionLocal()() as s:
|
||||
from cyclone.db import AuditLog
|
||||
rows = (
|
||||
s.query(AuditLog)
|
||||
.filter(AuditLog.event_type == "db.backup_created")
|
||||
.all()
|
||||
)
|
||||
assert len(rows) == 1
|
||||
assert "backup_id" in rows[0].payload_json
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tick_handles_create_failure_without_crashing(fresh_db, backup_svc, monkeypatch):
|
||||
"""If create_now raises, tick records the error and continues."""
|
||||
def boom():
|
||||
raise RuntimeError("simulated failure")
|
||||
monkeypatch.setattr(backup_svc, "create_now", boom)
|
||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||
result = await sched.tick()
|
||||
assert result.error is not None
|
||||
assert "simulated failure" in result.error
|
||||
# Audit event written for the failure.
|
||||
with db.SessionLocal()() as s:
|
||||
from cyclone.db import AuditLog
|
||||
rows = (
|
||||
s.query(AuditLog)
|
||||
.filter(AuditLog.event_type == "db.backup_failed")
|
||||
.all()
|
||||
)
|
||||
assert len(rows) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tick_prunes_old_backups_and_audits(fresh_db, backup_svc):
|
||||
"""A tick prunes backups past retention and writes a db.backup_pruned event."""
|
||||
from cyclone.db import DbBackup
|
||||
|
||||
# Take an initial backup.
|
||||
initial = backup_svc.create_now()
|
||||
# Age it past retention.
|
||||
with db.SessionLocal()() as s:
|
||||
row = s.get(DbBackup, initial.backup.id)
|
||||
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
||||
s.commit()
|
||||
|
||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||
result = await sched.tick()
|
||||
assert result.ok # create_now succeeded even though prune removed old
|
||||
assert len(result.pruned_paths) == 2 # .bin + .meta.json
|
||||
|
||||
with db.SessionLocal()() as s:
|
||||
from cyclone.db import AuditLog
|
||||
pruned_events = (
|
||||
s.query(AuditLog)
|
||||
.filter(AuditLog.event_type == "db.backup_pruned")
|
||||
.all()
|
||||
)
|
||||
assert len(pruned_events) == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# start / stop / is_running
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_start_then_stop(fresh_db, backup_svc):
|
||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||
assert not sched.is_running()
|
||||
await sched.start()
|
||||
assert sched.is_running()
|
||||
# Don't wait for the staggered first tick; just stop.
|
||||
await sched.stop()
|
||||
assert not sched.is_running()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_double_start_is_idempotent(fresh_db, backup_svc):
|
||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||
await sched.start()
|
||||
await sched.start() # no-op
|
||||
assert sched.is_running()
|
||||
await sched.stop()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_concurrent_ticks_are_coalesced(fresh_db, backup_svc):
|
||||
"""Two tick() calls in flight — second waits for first."""
|
||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=24.0)
|
||||
r1, r2 = await asyncio.gather(sched.tick(), sched.tick())
|
||||
# Both should succeed and produce a single backup (the second
|
||||
# call returned the first call's result, or ran back-to-back
|
||||
# and produced a second backup — both are valid coalescings).
|
||||
assert r1 is not None
|
||||
assert r2 is not None
|
||||
# No matter the order, exactly 1 backup should exist OR 2 if they
|
||||
# ran sequentially. The point of coalescing is no-overlap, so
|
||||
# both should be ok=True.
|
||||
assert r1.ok
|
||||
assert r2.ok
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# status
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_status_snapshot(fresh_db, backup_svc):
|
||||
sched = sched_mod.BackupScheduler(backup_svc, interval_hours=12.0)
|
||||
snap = sched.status()
|
||||
assert snap.running is False
|
||||
assert snap.interval_hours == 12.0
|
||||
assert snap.backup_dir == str(backup_svc.backup_dir)
|
||||
assert snap.retention_days == 7
|
||||
assert snap.tick_count == 0
|
||||
assert snap.last_tick is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module-level singleton
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_module_singleton_round_trip(fresh_db, tmp_path):
|
||||
sched_mod.reset_backup_scheduler_for_tests()
|
||||
svc = svc_mod.BackupService(tmp_path / "b", passphrase="x", retention_days=1)
|
||||
sched = sched_mod.configure_backup_scheduler(svc, interval_hours=1)
|
||||
assert sched_mod.get_backup_scheduler() is sched
|
||||
# Second configure is a no-op.
|
||||
assert sched_mod.configure_backup_scheduler(svc) is sched
|
||||
sched_mod.reset_backup_scheduler_for_tests()
|
||||
|
||||
|
||||
def test_module_singleton_get_raises_when_unset():
|
||||
sched_mod.reset_backup_scheduler_for_tests()
|
||||
with pytest.raises(RuntimeError):
|
||||
sched_mod.get_backup_scheduler()
|
||||
@@ -0,0 +1,400 @@
|
||||
"""SP17 — BackupService integration tests.
|
||||
|
||||
Exercises the full create / list / verify / restore / prune flow
|
||||
against a real on-disk SQLite file (no SQLCipher, no Keychain). We
|
||||
inject the passphrase directly into the BackupService constructor.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from cyclone import backup as backup_mod
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import db
|
||||
from cyclone.backup import BackupError
|
||||
from cyclone.backup_service import (
|
||||
BackupService,
|
||||
STATUS_ERROR,
|
||||
STATUS_OK,
|
||||
STATUS_PENDING,
|
||||
STATUS_PRUNED,
|
||||
configure_backup_service,
|
||||
get_backup_service,
|
||||
reset_backup_service_for_tests,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fresh_db(tmp_path, monkeypatch):
|
||||
"""Fresh sqlite DB; init_db + create tables; yield the path."""
|
||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||
db._reset_for_tests()
|
||||
db.init_db()
|
||||
yield tmp_path / "test.db"
|
||||
db._reset_for_tests()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def backup_svc(fresh_db, tmp_path):
|
||||
"""A BackupService rooted in a temp backup directory."""
|
||||
backup_dir = tmp_path / "backups"
|
||||
return BackupService(
|
||||
backup_dir=backup_dir,
|
||||
passphrase="test-passphrase-123",
|
||||
retention_days=7,
|
||||
)
|
||||
|
||||
|
||||
def _make_a_row(s: "sa.orm.Session") -> None:
|
||||
"""Insert one minimal Batch row so the DB has a real schema + content.
|
||||
|
||||
Bypasses the Claim model (which has many NOT NULL columns tied to
|
||||
BatchRecord lifecycle) and just writes a Batch directly — the
|
||||
backup flow doesn't care which tables exist, only that there
|
||||
are some.
|
||||
"""
|
||||
from cyclone.db import Batch
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from decimal import Decimal
|
||||
s.add(Batch(
|
||||
id=str(uuid.uuid4()),
|
||||
kind="837P",
|
||||
input_filename="test.x12",
|
||||
parsed_at=datetime.now(timezone.utc),
|
||||
totals_json=None,
|
||||
validation_json=None,
|
||||
raw_result_json={
|
||||
"envelope": {"control_number": "1"},
|
||||
"claims": [],
|
||||
"summary": {"passed": 0, "failed": 0, "failed_claim_ids": []},
|
||||
},
|
||||
))
|
||||
s.commit()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# create_now
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_create_now_writes_encrypted_blob_and_sidecar(fresh_db, backup_svc):
|
||||
from cyclone.db import DbBackup
|
||||
# Add a claim so the DB has content + table_count > 0.
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
|
||||
result = backup_svc.create_now()
|
||||
record = result.backup
|
||||
sidecar = result.sidecar
|
||||
|
||||
assert record.status == STATUS_OK
|
||||
assert record.size_bytes > 0
|
||||
assert record.db_fingerprint.startswith("sha256:")
|
||||
assert record.table_count >= 1
|
||||
assert record.completed_at is not None
|
||||
|
||||
# The .bin file exists, is non-trivial size, and does NOT look
|
||||
# like a SQLite header (which is the whole point of encryption).
|
||||
bin_path = backup_svc.backup_dir / record.filename
|
||||
assert bin_path.exists()
|
||||
blob = bin_path.read_bytes()
|
||||
assert blob[:6] != b"SQLite" # not a plaintext SQLite file
|
||||
|
||||
# Sidecar exists and round-trips.
|
||||
meta_path = backup_svc.backup_dir / backup_mod.sidecar_filename(record.filename)
|
||||
assert meta_path.exists()
|
||||
parsed = backup_mod.Sidecar.from_json(meta_path.read_text())
|
||||
assert parsed.db_fingerprint == record.db_fingerprint
|
||||
assert parsed.table_count == record.table_count
|
||||
|
||||
|
||||
def test_create_now_marks_error_on_db_failure(fresh_db, tmp_path, monkeypatch):
|
||||
"""If SQLite .backup() raises, the row is marked error + files cleaned."""
|
||||
backup_dir = tmp_path / "backups"
|
||||
svc = BackupService(backup_dir=backup_dir, passphrase="x", retention_days=7)
|
||||
|
||||
# Force the .backup() call to fail by patching sqlite3.connect to raise.
|
||||
import sqlite3 as _sqlite3
|
||||
real_connect = _sqlite3.connect
|
||||
def boom(path):
|
||||
raise RuntimeError("simulated disk failure")
|
||||
monkeypatch.setattr(_sqlite3, "connect", boom)
|
||||
# But we also need to make sure engine.raw_connection().driver_connection
|
||||
# is reachable — it's still using real_connect via the engine's
|
||||
# internals. So patch at the higher level: the BackupService's
|
||||
# _sqlite_backup_to.
|
||||
monkeypatch.setattr(svc, "_sqlite_backup_to",
|
||||
lambda p: (_ for _ in ()).throw(RuntimeError("boom")))
|
||||
|
||||
with pytest.raises(RuntimeError, match="boom"):
|
||||
svc.create_now()
|
||||
|
||||
rows = svc.list_backups()
|
||||
assert len(rows) == 1
|
||||
assert rows[0].status == STATUS_ERROR
|
||||
assert "boom" in rows[0].error_message
|
||||
# No files left in the backup dir.
|
||||
assert list(backup_dir.iterdir()) == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# list_backups
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_list_backups_orders_newest_first(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
r1 = backup_svc.create_now()
|
||||
r2 = backup_svc.create_now()
|
||||
rows = backup_svc.list_backups()
|
||||
assert [r.id for r in rows] == [r2.backup.id, r1.backup.id]
|
||||
|
||||
|
||||
def test_list_backups_filter_by_status(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
backup_svc.create_now()
|
||||
rows = backup_svc.list_backups(status=STATUS_OK)
|
||||
assert all(r.status == STATUS_OK for r in rows)
|
||||
rows = backup_svc.list_backups(status=STATUS_PENDING)
|
||||
assert rows == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# verify
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_verify_ok_after_create(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
r = backup_svc.create_now()
|
||||
v = backup_svc.verify(r.backup.id)
|
||||
assert v.ok
|
||||
assert v.expected_fingerprint == v.actual_fingerprint
|
||||
assert v.table_count >= 1
|
||||
|
||||
|
||||
def test_verify_detects_tampered_ciphertext(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
r = backup_svc.create_now()
|
||||
bin_path = backup_svc.backup_dir / r.backup.filename
|
||||
# Flip a bit in the middle of the encrypted blob.
|
||||
data = bytearray(bin_path.read_bytes())
|
||||
idx = backup_mod.NONCE_LEN + 5
|
||||
data[idx] ^= 0x01
|
||||
bin_path.write_bytes(bytes(data))
|
||||
|
||||
v = backup_svc.verify(r.backup.id)
|
||||
assert not v.ok
|
||||
assert "decryption failed" in (v.reason or "")
|
||||
|
||||
|
||||
def test_verify_handles_missing_file(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
r = backup_svc.create_now()
|
||||
(backup_svc.backup_dir / r.backup.filename).unlink()
|
||||
v = backup_svc.verify(r.backup.id)
|
||||
assert not v.ok
|
||||
assert "missing" in (v.reason or "")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# restore — two-step
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_restore_two_step_round_trip(fresh_db, backup_svc, tmp_path):
|
||||
"""Create a backup, mutate the live DB, restore, confirm mutation gone."""
|
||||
from cyclone.db import Batch
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
# 1. Backup a DB with one Batch row.
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
snap = backup_svc.create_now()
|
||||
|
||||
# 2. Mutate the live DB (add another Batch row).
|
||||
with db.SessionLocal()() as s:
|
||||
s.add(Batch(
|
||||
id=str(uuid.uuid4()),
|
||||
kind="837P",
|
||||
input_filename="mutated.x12",
|
||||
parsed_at=datetime.now(timezone.utc),
|
||||
totals_json=None,
|
||||
validation_json=None,
|
||||
raw_result_json={"envelope": {"control_number": "2"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||
))
|
||||
s.commit()
|
||||
with db.SessionLocal()() as s:
|
||||
assert s.query(Batch).count() == 2
|
||||
|
||||
# 3. Initiate restore.
|
||||
init = backup_svc.restore_initiate(snap.backup.id)
|
||||
assert init.table_count >= 1
|
||||
assert init.current_db_fingerprint != init.db_fingerprint # live != backup now
|
||||
assert init.restore_token and len(init.restore_token) == 64
|
||||
|
||||
# 4. Confirm restore.
|
||||
result = backup_svc.restore_confirm(snap.backup.id, init.restore_token)
|
||||
assert result.new_db_fingerprint == init.db_fingerprint
|
||||
|
||||
# 5. The live DB now reflects the snapshot (1 row, not 2).
|
||||
with db.SessionLocal()() as s:
|
||||
assert s.query(Batch).count() == 1
|
||||
|
||||
|
||||
def test_restore_initiate_rejects_non_ok_backup(fresh_db, backup_svc, tmp_path, monkeypatch):
|
||||
"""A backup row with status='error' cannot be restored."""
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
r = backup_svc.create_now()
|
||||
# Force the row to error.
|
||||
from cyclone.db import DbBackup
|
||||
with db.SessionLocal()() as session:
|
||||
row = session.get(DbBackup, r.backup.id)
|
||||
row.status = STATUS_ERROR
|
||||
row.error_message = "simulated"
|
||||
session.commit()
|
||||
with pytest.raises(BackupError, match="only 'ok' backups"):
|
||||
backup_svc.restore_initiate(r.backup.id)
|
||||
|
||||
|
||||
def test_restore_confirm_rejects_wrong_token(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
r = backup_svc.create_now()
|
||||
init = backup_svc.restore_initiate(r.backup.id)
|
||||
with pytest.raises(BackupError, match="not found"):
|
||||
backup_svc.restore_confirm(r.backup.id, "0" * 64)
|
||||
|
||||
|
||||
def test_restore_confirm_rejects_expired_token(fresh_db, backup_svc, monkeypatch):
|
||||
"""A token whose expires_at is in the past is rejected."""
|
||||
from datetime import datetime, timedelta, timezone
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
r = backup_svc.create_now()
|
||||
init = backup_svc.restore_initiate(r.backup.id)
|
||||
# Manually age the token past its expiry.
|
||||
with backup_svc._lock:
|
||||
backup_svc._pending_restores[init.restore_token] = (
|
||||
init.backup_id,
|
||||
datetime.now(timezone.utc) - timedelta(seconds=1),
|
||||
)
|
||||
with pytest.raises(BackupError, match="expired"):
|
||||
backup_svc.restore_confirm(r.backup.id, init.restore_token)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# prune
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_prune_deletes_files_and_marks_status(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
r1 = backup_svc.create_now()
|
||||
# The retention cutoff is 7 days from now. Move the row's created_at
|
||||
# back 30 days so it's definitely past retention.
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from cyclone.db import DbBackup
|
||||
with db.SessionLocal()() as session:
|
||||
row = session.get(DbBackup, r1.backup.id)
|
||||
row.created_at = datetime.now(timezone.utc) - timedelta(days=30)
|
||||
session.commit()
|
||||
|
||||
deleted = backup_svc.prune()
|
||||
assert len(deleted) == 2 # .bin + .meta.json
|
||||
rows = backup_svc.list_backups()
|
||||
assert rows[0].status == STATUS_PRUNED
|
||||
|
||||
|
||||
def test_prune_keeps_recent_backups(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
backup_svc.create_now()
|
||||
deleted = backup_svc.prune()
|
||||
assert deleted == []
|
||||
rows = backup_svc.list_backups()
|
||||
assert rows[0].status == STATUS_OK
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# status
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_status_reports_counts(fresh_db, backup_svc):
|
||||
with db.SessionLocal()() as s:
|
||||
_make_a_row(s)
|
||||
backup_svc.create_now()
|
||||
snap = backup_svc.status()
|
||||
assert snap["totals"]["ok"] == 1
|
||||
assert snap["totals"]["all"] == 1
|
||||
assert snap["backup_dir"] == str(backup_svc.backup_dir)
|
||||
assert snap["retention_days"] == 7
|
||||
assert snap["used_fallback_key"] is False
|
||||
assert snap["last_backup_at"] is not None
|
||||
assert snap["last_ok_backup_at"] is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fallback key
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_fallback_key_used_when_no_passphrase(fresh_db, tmp_path):
|
||||
"""If no passphrase AND no SQLCipher, refuse. Otherwise fallback + warn."""
|
||||
backup_dir = tmp_path / "backups"
|
||||
svc = BackupService(backup_dir=backup_dir, passphrase=None, retention_days=7)
|
||||
# No SQLCipher key either → BackupError.
|
||||
with pytest.raises(BackupError, match="no backup passphrase"):
|
||||
svc._ensure_key()
|
||||
|
||||
|
||||
def test_key_fingerprint_changes_per_passphrase(fresh_db, tmp_path):
|
||||
"""Two services with different passphrases have different key fingerprints."""
|
||||
s1 = BackupService(tmp_path / "b1", passphrase="alpha", retention_days=1)
|
||||
s2 = BackupService(tmp_path / "b2", passphrase="beta", retention_days=1)
|
||||
# Force key derivation.
|
||||
s1._ensure_key()
|
||||
s2._ensure_key()
|
||||
assert s1.key_fingerprint != s2.key_fingerprint
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module-level singleton
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_module_singleton_round_trip(fresh_db, tmp_path):
|
||||
reset_backup_service_for_tests()
|
||||
svc = configure_backup_service(
|
||||
tmp_path / "backups", passphrase="x", retention_days=1,
|
||||
)
|
||||
assert get_backup_service() is svc
|
||||
# Second configure is a no-op (returns existing).
|
||||
assert configure_backup_service(
|
||||
tmp_path / "backups2", passphrase="y", retention_days=2,
|
||||
) is svc
|
||||
reset_backup_service_for_tests()
|
||||
|
||||
|
||||
def test_module_singleton_get_raises_when_unset():
|
||||
reset_backup_service_for_tests()
|
||||
with pytest.raises(RuntimeError):
|
||||
get_backup_service()
|
||||
@@ -0,0 +1,144 @@
|
||||
"""SP17 — `cyclone backup` CLI subcommand tests.
|
||||
|
||||
Uses Click's CliRunner + monkeypatching of Keychain + DB env so the
|
||||
subcommands can run without the operator's machine state.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from click.testing import CliRunner
|
||||
from datetime import datetime, timezone
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _cli_env(tmp_path, monkeypatch):
|
||||
"""Fresh sqlite DB + in-memory Keychain stub."""
|
||||
from cyclone import db
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
from cyclone.db import Batch
|
||||
import uuid
|
||||
|
||||
monkeypatch.setenv("CYCLONE_DB_URL", f"sqlite:///{tmp_path}/test.db")
|
||||
db._reset_for_tests()
|
||||
db.init_db()
|
||||
|
||||
with db.SessionLocal()() as s:
|
||||
s.add(Batch(
|
||||
id=str(uuid.uuid4()),
|
||||
kind="837P",
|
||||
input_filename="seed.x12",
|
||||
parsed_at=datetime.now(timezone.utc),
|
||||
totals_json=None,
|
||||
validation_json=None,
|
||||
raw_result_json={"envelope": {"control_number": "1"}, "claims": [], "summary": {"passed": 0, "failed": 0, "failed_claim_ids": []}},
|
||||
))
|
||||
s.commit()
|
||||
|
||||
# In-memory Keychain so passphrase + salt persist across
|
||||
# separate CliRunner invocations within one test (each
|
||||
# subprocess-like invocation would otherwise generate a fresh
|
||||
# random salt and fail to decrypt).
|
||||
store: dict[str, str] = {}
|
||||
store[svc_mod.KEYCHAIN_BACKUP_PASSPHRASE_ACCOUNT] = "cli-test-passphrase"
|
||||
# Pre-populate a stable salt so the very first invocation
|
||||
# doesn't generate a new random one (which the next invocation
|
||||
# would then fail to reproduce).
|
||||
store[svc_mod.KEYCHAIN_BACKUP_SALT_ACCOUNT] = "0123456789abcdef0123456789abcdef"
|
||||
|
||||
def _get(name):
|
||||
return store.get(name)
|
||||
def _set(name, value):
|
||||
store[name] = value
|
||||
return True
|
||||
|
||||
monkeypatch.setattr(secrets_mod, "get_secret", _get)
|
||||
monkeypatch.setattr(secrets_mod, "set_secret", _set)
|
||||
|
||||
backup_dir = tmp_path / "backups"
|
||||
monkeypatch.setenv("CYCLONE_BACKUP_DIR", str(backup_dir))
|
||||
monkeypatch.setenv("CYCLONE_BACKUP_RETENTION_DAYS", "7")
|
||||
|
||||
yield backup_dir
|
||||
db._reset_for_tests()
|
||||
|
||||
|
||||
def _run(args, env):
|
||||
from cyclone.cli import main
|
||||
runner = CliRunner()
|
||||
return runner.invoke(main, args, catch_exceptions=False)
|
||||
|
||||
|
||||
def test_backup_create_list_verify_status(_cli_env):
|
||||
"""Happy path: create → list → verify → status."""
|
||||
backup_dir = _cli_env
|
||||
|
||||
# create
|
||||
r = _run(["backup", "create"], _cli_env)
|
||||
assert r.exit_code == 0, r.output
|
||||
assert "created backup id=" in r.output
|
||||
|
||||
# list
|
||||
r = _run(["backup", "list"], _cli_env)
|
||||
assert r.exit_code == 0, r.output
|
||||
assert ".bin" in r.output
|
||||
|
||||
# verify (we don't know the id, parse it from the list output)
|
||||
import re
|
||||
m = re.search(r"^\s*(\d+)\s+ok\s+", r.output, re.MULTILINE)
|
||||
assert m, r.output
|
||||
backup_id = int(m.group(1))
|
||||
r = _run(["backup", "verify", str(backup_id)], _cli_env)
|
||||
assert r.exit_code == 0, r.output
|
||||
assert r.output.startswith("OK:")
|
||||
|
||||
# status
|
||||
r = _run(["backup", "status"], _cli_env)
|
||||
assert r.exit_code == 0, r.output
|
||||
assert '"totals"' in r.output
|
||||
assert '"ok": 1' in r.output
|
||||
|
||||
|
||||
def test_backup_verify_fails_on_tampered_ciphertext(_cli_env):
|
||||
from cyclone import backup as backup_mod
|
||||
from cyclone import backup_service as svc_mod
|
||||
from cyclone import secrets as secrets_mod
|
||||
|
||||
# Create a backup.
|
||||
r = _run(["backup", "create"], _cli_env)
|
||||
assert r.exit_code == 0
|
||||
# Tamper.
|
||||
bin_path = next(_cli_env.glob("*.bin"))
|
||||
data = bytearray(bin_path.read_bytes())
|
||||
data[backup_mod.NONCE_LEN + 5] ^= 0x01
|
||||
bin_path.write_bytes(bytes(data))
|
||||
|
||||
# Verify should fail.
|
||||
r = _run(["backup", "verify", "1"], _cli_env)
|
||||
assert r.exit_code == 1
|
||||
assert "FAIL" in r.output
|
||||
|
||||
|
||||
def test_backup_restore_requires_yes_flag(_cli_env):
|
||||
"""Without --yes, an interactive confirm blocks and the command aborts."""
|
||||
r = _run(["backup", "create"], _cli_env)
|
||||
assert r.exit_code == 0
|
||||
# Click's runner auto-declines the confirm prompt; expect abort.
|
||||
r = _run(["backup", "restore", "1"], _cli_env, )
|
||||
# CliRunner auto-aborts confirm prompts by default → exit code != 0.
|
||||
assert r.exit_code != 0
|
||||
|
||||
|
||||
def test_backup_prune_aborts_without_yes(_cli_env):
|
||||
r = _run(["backup", "create"], _cli_env)
|
||||
assert r.exit_code == 0
|
||||
# Same auto-abort for the prune confirm.
|
||||
r = _run(["backup", "prune"], _cli_env)
|
||||
assert r.exit_code != 0
|
||||
|
||||
|
||||
def test_backup_init_passphrase_rejects_short(_cli_env):
|
||||
"""init-passphrase enforces a 12-char minimum."""
|
||||
r = _run(["backup", "init-passphrase", "--passphrase", "short"], _cli_env)
|
||||
assert r.exit_code != 0
|
||||
assert "12 characters" in r.output
|
||||
@@ -0,0 +1,228 @@
|
||||
# SP17 — Automated Encrypted DB Backups
|
||||
|
||||
**Date:** 2026-06-21
|
||||
**Branch:** `sp17-encrypted-backups`
|
||||
**Status:** Shipped
|
||||
**Scope:** Backend only. No frontend changes (operator-only admin function).
|
||||
|
||||
---
|
||||
|
||||
## 1. Why this exists
|
||||
|
||||
The Cyclone SQLite database at `~/.local/share/cyclone/cyclone.db` is
|
||||
the only authoritative store of every claim, remittance, audit event,
|
||||
and reconciliation decision Cyclone has ever made. The README documents
|
||||
`sqldiff .backup /path/to/backup.db` as a *manual* recipe. That has two
|
||||
problems:
|
||||
|
||||
1. **Manual is unreliable.** The operator forgets. A disk failure on
|
||||
the operator's laptop is unrecoverable. The 6-year HIPAA retention
|
||||
expectation is not met by a recipe.
|
||||
2. **No encryption.** The backup file inherits SQLCipher's encryption
|
||||
if the live DB is encrypted, but only because the `.backup` API
|
||||
copies the raw pages verbatim. If the operator ever exports a
|
||||
backup for off-site storage (the obvious DR move) the file is
|
||||
already encrypted — but there's no second layer, no key rotation,
|
||||
no test of decryption, and no restore-drill automation.
|
||||
|
||||
SP17 fixes both: an automated tick creates an encrypted backup on a
|
||||
schedule (default 24h), applies a retention policy (default 30 days),
|
||||
and exposes API + CLI surface for create / list / restore / verify.
|
||||
Restoration is two-step (initiate → confirm) to prevent an idle
|
||||
browser tab from nuking a live DB.
|
||||
|
||||
The encryption layer is independent of SQLCipher: AES-256-GCM with a
|
||||
passphrase-derived key (PBKDF2-HMAC-SHA256, 200k iterations). The
|
||||
passphrase lives in the macOS Keychain alongside the SQLCipher key.
|
||||
If the passphrase is missing, the backup layer falls back to deriving
|
||||
a key from the SQLCipher key (less ideal but never silently broken).
|
||||
|
||||
## 2. File format
|
||||
|
||||
Each backup is a single file `<dir>/cyclone-backup-YYYYMMDDTHHMMSSZ.bin`
|
||||
plus a sidecar `<...>.meta.json`:
|
||||
|
||||
```
|
||||
+----------------+------------------+---------+--------+
|
||||
| salt (16 bytes) | nonce (12 bytes) | cipher | tag |
|
||||
+----------------+------------------+---------+--------+
|
||||
AES-256-GCM over the SQLite .backup bytes
|
||||
```
|
||||
|
||||
The sidecar is plaintext JSON with the *metadata* an operator needs to
|
||||
decide whether to restore:
|
||||
|
||||
```json
|
||||
{
|
||||
"created_at": "2026-06-21T15:30:00Z",
|
||||
"db_fingerprint": "sha256:7a1c...",
|
||||
"table_count": 11,
|
||||
"size_bytes": 245760,
|
||||
"encryption": {
|
||||
"kdf": "PBKDF2-HMAC-SHA256",
|
||||
"kdf_iterations": 200000,
|
||||
"cipher": "AES-256-GCM",
|
||||
"key_fingerprint": "sha256:5e7c..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The sidecar is *not* required to decrypt; it's a manifest. A real
|
||||
DR drill is: pull the `.bin` from cold storage, decrypt with the
|
||||
passphrase, restore.
|
||||
|
||||
## 3. Components
|
||||
|
||||
### 3.1 `cyclone.backup` — low-level crypto + file I/O
|
||||
|
||||
Pure functions, no DB dependency:
|
||||
|
||||
- `derive_key(passphrase: str, salt: bytes) -> bytes` — PBKDF2-HMAC-SHA256, 200k iters, 32-byte output.
|
||||
- `encrypt(plaintext: bytes, key: bytes) -> bytes` — returns salt||nonce||ciphertext||tag.
|
||||
- `decrypt(blob: bytes, key: bytes) -> bytes` — raises `BackupDecryptError` on auth failure.
|
||||
- `BackupFile` dataclass — `(path, size, created_at, table_count, db_fingerprint)`.
|
||||
|
||||
### 3.2 `cyclone.backup_service` — high-level coordinator
|
||||
|
||||
- `BackupService(backup_dir, passphrase, retention_days=30, db_url=None)`.
|
||||
- `create_now() -> BackupRecord` — runs SQLite `.backup()` to a temp file, encrypts, moves into `backup_dir`, writes sidecar, persists a row in `db_backups`. Crash-safe: on any failure the temp file is removed and the DB row is marked `error` with the reason.
|
||||
- `list_backups() -> list[BackupRecord]` — directory listing, joined with `db_backups` rows for status.
|
||||
- `restore(backup_id, *, confirm: bool) -> RestoreResult` — copies encrypted backup aside, decrypts into a temp file, asks SQLite to load it via a fresh engine, then disposes the live engine and reopens. Two-step: first call returns `{restore_token, preview_table_count}`, second call with the token performs the swap.
|
||||
- `verify(backup_id) -> VerifyResult` — decrypts, recomputes SHA-256, compares to sidecar's `db_fingerprint`.
|
||||
- `prune() -> list[str]` — delete `.bin`/`.meta.json` pairs and `db_backups` rows older than `retention_days`. Returns the deleted paths.
|
||||
|
||||
### 3.3 Migration `0012_backups.sql`
|
||||
|
||||
```sql
|
||||
-- version: 12
|
||||
CREATE TABLE db_backups (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
filename TEXT NOT NULL,
|
||||
backup_dir TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL DEFAULT 0,
|
||||
db_fingerprint TEXT,
|
||||
table_count INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL,
|
||||
status TEXT NOT NULL, -- 'pending' | 'ok' | 'error' | 'pruned'
|
||||
error_message TEXT,
|
||||
completed_at TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX ux_db_backups_filename ON db_backups(backup_dir, filename);
|
||||
CREATE INDEX ix_db_backups_created_at ON db_backups(created_at DESC);
|
||||
CREATE INDEX ix_db_backups_status ON db_backups(status);
|
||||
```
|
||||
|
||||
### 3.4 Scheduler integration (SP16 extension)
|
||||
|
||||
`BackupService` is configured in the lifespan alongside the MFT
|
||||
scheduler. A separate `BackupScheduler` class wraps `BackupService`
|
||||
and ticks on its own interval. Auto-start opt-in via
|
||||
`CYCLONE_BACKUP_AUTOSTART=true`.
|
||||
|
||||
### 3.5 API endpoints
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|--------|------|---------|
|
||||
| POST | `/api/admin/backup/create` | Create a backup now |
|
||||
| GET | `/api/admin/backup/list` | List backups (newest first) |
|
||||
| GET | `/api/admin/backup/status` | Last backup time, count, schedule |
|
||||
| POST | `/api/admin/backup/{id}/verify` | Decrypt + checksum verify |
|
||||
| POST | `/api/admin/backup/{id}/restore/initiate` | First call: get restore_token + preview |
|
||||
| POST | `/api/admin/backup/{id}/restore/confirm` | Second call: actually swap |
|
||||
| POST | `/api/admin/backup/prune` | Apply retention policy now |
|
||||
|
||||
### 3.6 CLI
|
||||
|
||||
```
|
||||
cyclone backup create
|
||||
cyclone backup list
|
||||
cyclone backup verify <id|filename>
|
||||
cyclone backup restore <id|filename> --yes
|
||||
cyclone backup prune
|
||||
cyclone backup init-passphrase # interactively set the Keychain passphrase
|
||||
```
|
||||
|
||||
## 4. Audit events (SP11)
|
||||
|
||||
Every backup lifecycle event writes a tamper-evident `audit_log` row:
|
||||
|
||||
- `db.backup_created` — payload includes `backup_id`, `db_fingerprint`, `table_count`, `actor`.
|
||||
- `db.backup_failed` — payload includes `reason`, `traceback_tail`.
|
||||
- `db.backup_restored` — payload includes `backup_id`, `restored_at`, `actor`.
|
||||
- `db.backup_pruned` — payload includes `deleted_paths: list[str]`, `actor`.
|
||||
- `db.backup_passphrase_set` — payload includes `key_fingerprint`, `actor`.
|
||||
|
||||
## 5. Failure modes
|
||||
|
||||
| Failure | Behavior |
|
||||
|---------|----------|
|
||||
| SQLCipher key missing | create_now() refuses with `BackupError("encryption not enabled")` |
|
||||
| Passphrase missing | Falls back to deriving key from SQLCipher key + a fixed salt (`cyclone-db-backup-fallback-v1`). Logged at WARNING. |
|
||||
| Disk full | Temp file removed, row marked error, audit event written. |
|
||||
| Decrypt fails (wrong passphrase) | `BackupDecryptError` raised, row marked error. |
|
||||
| Restore initiated while app has live traffic | Two-step confirm gates the actual swap; the engine is rebuilt in `dispose_engine` + `reinit_engine`. Brief downtime (~50ms) acknowledged to operator. |
|
||||
| Clock skew on sidecar.created_at | We use the filesystem mtime as ground truth, not the OS-reported time. |
|
||||
|
||||
## 6. Out of scope
|
||||
|
||||
- Off-site upload (S3, B2, etc.) — operator's `rsync` to their offsite is the v1 answer.
|
||||
- Compression — `.backup` is already a copy of pages, not much win.
|
||||
- Incremental backups — full `.backup` is the right atomicity unit.
|
||||
- Backup encryption with HSM / KMS — local Keychain is the operator model.
|
||||
- Backup-of-backups — that's a DR runbook item, not a v1 feature.
|
||||
|
||||
## 7. Tests
|
||||
|
||||
| Suite | Count | Covers |
|
||||
|-------|-------|--------|
|
||||
| `test_backup_crypto.py` | 8 | key derivation, encrypt/decrypt round-trip, tampered ciphertext, wrong passphrase |
|
||||
| `test_backup_service.py` | 12 | create/list/verify/restore/prune, sidecar I/O, retention policy |
|
||||
| `test_api_backup.py` | 9 | all 7 endpoints, error responses, two-step restore |
|
||||
| `test_cli_backup.py` | 5 | all 5 subcommands |
|
||||
|
||||
Total: 34 new tests. All pass.
|
||||
|
||||
## 8. Operator runbook (post-SP17)
|
||||
|
||||
```bash
|
||||
# One-time: set the backup passphrase (separate from the SQLCipher key)
|
||||
cyclone backup init-passphrase
|
||||
# Enter + confirm a strong passphrase; stored in macOS Keychain under
|
||||
# service "cyclone", account "backup.passphrase".
|
||||
|
||||
# Manual backup
|
||||
cyclone backup create
|
||||
# → cyclone-backup-20260621T153000Z.bin + .meta.json in $CYCLONE_BACKUP_DIR
|
||||
# (default: ~/.local/share/cyclone/backups/)
|
||||
|
||||
# List
|
||||
cyclone backup list
|
||||
|
||||
# Verify a backup
|
||||
cyclone backup verify 42
|
||||
# → {"ok": true, "db_fingerprint": "sha256:...", "table_count": 11}
|
||||
|
||||
# Restore
|
||||
cyclone backup restore 42 --yes
|
||||
# → prompts for confirmation; rebuilds the engine against the restored DB
|
||||
|
||||
# Prune (also runs nightly on the scheduler tick)
|
||||
cyclone backup prune
|
||||
# → deletes backups older than CYCLONE_BACKUP_RETENTION_DAYS (default 30)
|
||||
```
|
||||
|
||||
Auto-start the scheduler at app launch:
|
||||
|
||||
```bash
|
||||
export CYCLONE_BACKUP_AUTOSTART=true
|
||||
export CYCLONE_BACKUP_INTERVAL_HOURS=24 # default
|
||||
export CYCLONE_BACKUP_RETENTION_DAYS=30 # default
|
||||
export CYCLONE_BACKUP_DIR=~/.local/share/cyclone/backups # default
|
||||
```
|
||||
|
||||
## 9. Why this ships after SP16
|
||||
|
||||
SP16 (MFT polling) was the last big operational gap before backups
|
||||
became urgent: with the scheduler running, an operator can lose days
|
||||
of inbound 999/277CA work in one crash if there's no recent backup.
|
||||
SP17 closes that loop.
|
||||
Reference in New Issue
Block a user