feat(water-log): Smartsheet sync integration
Deploy to route.crispygoat.com / deploy (push) Successful in 5m34s

Per-brand Smartsheet integration that pushes new water_log_entries to a
configured sheet. Config UI lives at /admin/water-log/settings.

Database (0093_water_smartsheet_sync.sql):
- water_smartsheet_config: one row per brand; AES-256-GCM encrypted API
  token (BYTEA), sheet id, column mapping JSONB, sync_frequency
  (realtime | every_15_minutes | hourly), enable flag, last-sync
  metadata. RLS via existing current_brand_id() pattern.
- water_smartsheet_sync_queue: one row per entry awaiting sync;
  tracks status / attempts / exponential backoff. UNIQUE(brand_id,
  entry_id) for idempotent enqueue.
- water_smartsheet_sync_log: append-only audit of every sync attempt
  (success or failure); backs the Recent Activity panel in the UI.

Server:
- src/lib/crypto.ts: AES-256-GCM encrypt/decrypt + token mask helper.
  Reads SMARTSHEET_TOKEN_ENC_KEY. Throws on missing/wrong-size key.
- src/lib/smartsheet.ts: typed REST wrapper (no SDK; the official
  smartsheet npm has Node 22 compat issues). getSheetMeta, addRow,
  findRowByColumn + typed SmartsheetApiError.
- src/services/smartsheet-sync.ts: syncEntryToSmartsheet (one entry),
  drainSyncQueue (batch with backoff), runScheduledSync (cron entry).
  Sequential processing stays well under Smartsheet's 300 req/min.
  Max 5 attempts, then row stays 'failed' permanently.
- src/actions/water-log/smartsheet.ts: 6 server actions. Token never
  returned in plaintext — UI gets maskedToken only. Permission check
  matches the rest of the water-log module (can_manage_water_log).
- src/app/api/water-log/smartsheet-sync/route.ts: POST cron route,
  bearer auth via SMARTSHEET_CRON_SECRET (falls back to CRON_SECRET).
  Optional {brandId} body for manual retry.
- vercel.json: added */15 * * * * cron entry. Single entry covers
  both 15-min and hourly frequencies (route filters per-brand).

UI:
- src/components/admin/water-log/SmartsheetIntegrationCard.tsx:
  self-contained client component with useReducer. Sections: enable
  toggle, sheet URL/ID + API token + Test Connection, sync frequency
  radio group, column mapping dropdowns (populated after Test),
  Recent Activity panel.
- src/app/admin/water-log/settings/page.tsx: mounted the card as new
  '§ 05 — Integrations' section after the existing admin-PIN settings.
  Card takes brandId as a prop (CLAUDE.md 'Brand ID Threading').

Hooks:
- src/actions/water-log/field.ts::submitWaterEntry: calls
  triggerSyncForEntry(brandId, entryId) in a fire-and-forget void
  after the entry insert. Sync failures NEVER bubble to the field
  worker.

Env vars (.env.example):
- SMARTSHEET_TOKEN_ENC_KEY (REQUIRED, 32 random bytes base64)
- SMARTSHEET_CRON_SECRET (optional bearer)
- SMARTSHEET_SYNC_TIMEOUT_MS (optional, default 8000)

Deploy:
1. openssl rand -base64 32 → SMARTSHEET_TOKEN_ENC_KEY in .env.local
   AND Vercel dashboard
2. npm run migrate:one 93  (pushes 0093_water_smartsheet_sync.sql)
3. Push triggers .gitea/workflows/deploy.yml

Rollback: remove cron entry, DROP the 3 tables, revert field.ts +
settings page hooks, git revert the merge commit. See MEMORY.md
for full rollback story + sharp edges (no key rotation, slug URLs).
This commit is contained in:
Nora
2026-07-03 14:42:15 -06:00
parent 3b0d0c07e3
commit fc70344dab
13 changed files with 2794 additions and 1 deletions
+125
View File
@@ -0,0 +1,125 @@
/**
* AES-256-GCM token encryption for at-rest secrets.
*
* Used by the Smartsheet integration (and any future third-party token
* storage) to keep API tokens encrypted in Postgres. AES-256-GCM gives
* us confidentiality + integrity in one primitive: a unique 12-byte
* IV per encrypt, a 16-byte auth tag that detects tampering.
*
* Key management:
* - Single key per environment, sourced from `SMARTSHEET_TOKEN_ENC_KEY`
* (32 random bytes, base64-encoded).
* - Generate with: `openssl rand -base64 32`
* - Storing the key in env vars matches the codebase convention
* (`STRIPE_SECRET_KEY`, `NEON_AUTH_COOKIE_SECRET`, etc.).
* - Key rotation is *not* supported in this version — if the key
* changes, existing encrypted blobs become unreadable and admins
* must re-enter tokens. Phase 2 should add a `key_version` column
* on the config table to support zero-downtime rotation.
*
* Why AES-256-GCM?
* - Built into Node's `crypto` module — zero dependencies.
* - Authenticated encryption — we detect (and refuse to decrypt)
* ciphertext that was tampered with.
* - Faster than AES-CBC + HMAC and easier to use correctly.
*/
import "server-only";
import { createCipheriv, createDecipheriv, randomBytes } from "node:crypto";
const ALGO = "aes-256-gcm";
const IV_LEN = 12; // GCM standard
const TAG_LEN = 16; // GCM standard
const KEY_LEN = 32; // 256 bits
export type EncryptedToken = {
/** Ciphertext (raw bytes; not base64). Stored as BYTEA. */
cipher: Buffer;
/** 12-byte IV, unique per encrypt. Stored as BYTEA. */
iv: Buffer;
/** 16-byte GCM auth tag. Stored as BYTEA. */
authTag: Buffer;
};
function getKey(): Buffer {
const raw = process.env.SMARTSHEET_TOKEN_ENC_KEY;
if (!raw || raw.length === 0) {
if (process.env.NODE_ENV !== "production") {
throw new Error(
"SMARTSHEET_TOKEN_ENC_KEY is not set. Generate one with `openssl rand -base64 32` and add it to .env.local.",
);
}
throw new Error(
"SMARTSHEET_TOKEN_ENC_KEY is not set in production. This is required — refusing to fall back to an insecure default.",
);
}
const key = Buffer.from(raw, "base64");
if (key.length !== KEY_LEN) {
throw new Error(
`SMARTSHEET_TOKEN_ENC_KEY must decode to exactly ${KEY_LEN} bytes (got ${key.length}). Regenerate with \`openssl rand -base64 32\`.`,
);
}
return key;
}
/**
* Encrypt a plaintext token (e.g. a Smartsheet API access token).
*
* Returns a triple of Buffers that should each be persisted to a
* separate BYTEA column. Never log the plaintext or the auth tag.
*/
export function encryptToken(plaintext: string): EncryptedToken {
if (typeof plaintext !== "string" || plaintext.length === 0) {
throw new Error("encryptToken: plaintext must be a non-empty string");
}
const key = getKey();
const iv = randomBytes(IV_LEN);
const cipher = createCipheriv(ALGO, key, iv);
const ciphertext = Buffer.concat([
cipher.update(plaintext, "utf8"),
cipher.final(),
]);
const authTag = cipher.getAuthTag();
return { cipher: ciphertext, iv, authTag };
}
/**
* Decrypt a token previously produced by `encryptToken`.
*
* Throws if the auth tag verification fails (tampered ciphertext).
* Throws if the env key is missing/wrong size (key rotation break).
*/
export function decryptToken(token: EncryptedToken): string {
if (
!Buffer.isBuffer(token.cipher) ||
!Buffer.isBuffer(token.iv) ||
!Buffer.isBuffer(token.authTag)
) {
throw new Error("decryptToken: invalid token shape");
}
if (token.iv.length !== IV_LEN) {
throw new Error(`decryptToken: iv must be ${IV_LEN} bytes`);
}
if (token.authTag.length !== TAG_LEN) {
throw new Error(`decryptToken: authTag must be ${TAG_LEN} bytes`);
}
const key = getKey();
const decipher = createDecipheriv(ALGO, key, token.iv);
decipher.setAuthTag(token.authTag);
const plaintext = Buffer.concat([
decipher.update(token.cipher),
decipher.final(), // throws on auth-tag mismatch
]);
return plaintext.toString("utf8");
}
/**
* Mask a token for UI display. Returns "••••abcd" where "abcd" is the
* last 4 characters of the plaintext. For tokens < 4 chars, returns
* "••••" (no tail). Used by the Settings card to show "we have a token
* saved" without revealing it.
*/
export function maskToken(plaintext: string): string {
if (!plaintext || plaintext.length === 0) return "";
if (plaintext.length <= 4) return "••••";
return `••••${plaintext.slice(-4)}`;
}
+324
View File
@@ -0,0 +1,324 @@
/**
* Smartsheet REST API wrapper (no SDK).
*
* We use direct REST instead of the official `smartsheet` npm SDK
* because:
* - The SDK has Node 22 compatibility issues (top-level `await` in
* a CommonJS module) and pulls in ~1.4MB of transitive deps
* (`request`, `request-promise`, `lodash`, etc.).
* - REST covers our 3 endpoints cleanly: get sheet metadata, add a
* row, search a sheet by column value.
* - Smartsheet's own docs recommend REST for simple integrations.
*
* Endpoints used (all docs at https://smartsheet.redoc.ly):
* - GET /2.0/sheets/{sheetId} — sheet metadata + columns
* - POST /2.0/sheets/{sheetId}/rows — add a row
* - GET /2.0/sheets/{sheetId} — full sheet (used for find-by-column)
*
* Auth:
* - Bearer token via `Authorization: Bearer <token>` header.
* - 300 requests/minute per token. Our drain caps at 50 rows per
* cron run, well under the limit.
*
* Errors:
* - `SmartsheetApiError` is thrown for any non-2xx response. It
* carries `status` and `body` for the caller to log.
*/
import "server-only";
const BASE_URL = "https://api.smartsheet.com";
const DEFAULT_TIMEOUT_MS = Number.parseInt(
process.env.SMARTSHEET_SYNC_TIMEOUT_MS ?? "8000",
10,
);
export type SmartsheetColumn = {
/** Numeric column ID, as a string (Smartsheet IDs overflow JS numbers). */
id: string;
/** Display name in the sheet header row. */
title: string;
/** Column type — TEXT_NUMBER, DATE, PICKLIST, etc. We treat all as text on write. */
type: string;
/** Column index in the sheet (0-based). */
index: number;
};
export type SmartsheetSheetMeta = {
id: string;
name: string;
columns: SmartsheetColumn[];
};
export type SmartsheetCell = {
columnId: string;
value: string | number | boolean | null;
};
export class SmartsheetApiError extends Error {
readonly status: number;
readonly body: string;
readonly code: number | null;
constructor(status: number, body: string, message?: string) {
super(
message ??
`Smartsheet API error ${status}: ${body.slice(0, 200)}${
body.length > 200 ? "…" : ""
}`,
);
this.name = "SmartsheetApiError";
this.status = status;
this.body = body;
// Smartsheet returns { errorCode, message, ... } on failure
try {
const parsed = JSON.parse(body) as { errorCode?: number };
this.code = parsed.errorCode ?? null;
} catch {
this.code = null;
}
}
}
async function request<T>(
method: "GET" | "POST",
path: string,
token: string,
body?: unknown,
): Promise<T> {
const url = `${BASE_URL}${path}`;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), DEFAULT_TIMEOUT_MS);
let res: Response;
try {
res = await fetch(url, {
method,
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
Accept: "application/json",
"User-Agent": "RouteCommerce-WaterLog/1.0",
},
body: body ? JSON.stringify(body) : undefined,
signal: controller.signal,
// Disable Next.js fetch caching for API calls — we always want
// fresh data from Smartsheet.
cache: "no-store",
});
} catch (err) {
if ((err as { name?: string }).name === "AbortError") {
throw new SmartsheetApiError(
408,
`Smartsheet request timed out after ${DEFAULT_TIMEOUT_MS}ms`,
);
}
throw new SmartsheetApiError(
0,
`Network error contacting Smartsheet: ${(err as Error).message ?? err}`,
);
} finally {
clearTimeout(timer);
}
if (!res.ok) {
const errBody = await res.text();
throw new SmartsheetApiError(res.status, errBody);
}
// Some 204s return no body; guard.
const text = await res.text();
if (!text) return undefined as unknown as T;
return JSON.parse(text) as T;
}
/**
* Extract the numeric sheet ID from either:
* - A bare ID: "123456789012345"
* - A URL: "https://app.smartsheet.com/sheets/abc123?..."
*
* Smartsheet share URLs include an opaque slug (e.g. "abc123"). We
* can't resolve that slug without an API call (the sheet meta call
* will fail with 404 anyway), so we just take the last path segment
* and pass it as the sheet ID. If the user pasted a slug URL, the
* "Test Connection" UI will surface the 404 and prompt for a numeric
* ID instead. (This matches what most Smartsheet integrations do.)
*/
export function extractSheetId(input: string): string {
const trimmed = input.trim();
if (!trimmed) {
throw new Error("Sheet ID or URL is required");
}
// Plain numeric ID
if (/^\d+$/.test(trimmed)) return trimmed;
// URL form
try {
const url = new URL(trimmed);
const segments = url.pathname.split("/").filter(Boolean);
// Find the segment after "sheets"
const sheetsIdx = segments.findIndex((s) => s === "sheets");
if (sheetsIdx >= 0 && segments[sheetsIdx + 1]) {
return segments[sheetsIdx + 1];
}
// Last segment as a fallback
if (segments.length > 0) {
return segments[segments.length - 1];
}
} catch {
// Not a URL — fall through
}
// Treat as opaque ID; API will surface 404 if it's not numeric.
return trimmed;
}
/**
* Fetch sheet metadata (columns, name) — used by the "Test Connection"
* button to populate the column-mapping dropdowns.
*
* Calls `GET /2.0/sheets/{sheetId}?pageSize=1` — we only need the
* header row (column definitions), not the row data, so a 1-row
* response keeps the payload tiny.
*/
export async function getSheetMeta(
sheetId: string,
token: string,
): Promise<SmartsheetSheetMeta> {
if (!sheetId) throw new Error("sheetId is required");
if (!token) throw new Error("token is required");
type ApiResponse = {
result: {
id: number;
name: string;
columns: Array<{
id: number;
title: string;
type: string;
index: number;
}>;
};
};
const data = await request<ApiResponse>(
"GET",
`/2.0/sheets/${encodeURIComponent(sheetId)}?pageSize=1`,
token,
);
return {
id: String(data.result.id),
name: data.result.name,
columns: data.result.columns.map((c) => ({
id: String(c.id),
title: c.title,
type: c.type,
index: c.index,
})),
};
}
/**
* Add a row to a sheet. `cells` is mapped to Smartsheet's
* `[{columnId, value, strict?}]` shape. Boolean values are coerced
* to "TRUE"/"FALSE" (Smartsheet wants strings for PICKLIST columns).
*
* Returns the new row's `id` and `rowNumber`.
*/
export async function addRow(
sheetId: string,
token: string,
cells: SmartsheetCell[],
): Promise<{ rowId: string; rowNumber: number }> {
if (!sheetId) throw new Error("sheetId is required");
if (!token) throw new Error("token is required");
if (cells.length === 0) throw new Error("addRow: cells must not be empty");
const toCellValue = (
v: string | number | boolean | null,
): string | boolean | number => {
if (v === null || v === undefined) return "";
if (typeof v === "boolean") return v;
return v;
};
type ApiResponse = {
result: { id: number; rowNumber: number };
};
const data = await request<ApiResponse>(
"POST",
`/2.0/sheets/${encodeURIComponent(sheetId)}/rows`,
token,
{
rows: [
{
cells: cells.map((c) => ({
columnId: Number(c.columnId),
value: toCellValue(c.value),
})),
},
],
},
);
return {
rowId: String(data.result.id),
rowNumber: data.result.rowNumber,
};
}
/**
* Find an existing row by a column value. Used for dedup — if a row
* already exists for the given `entry_id`, skip the insert and
* record the existing row ID in the queue.
*
* Implementation: scans the sheet's `columnId` column for `value`.
* This is O(n) on sheet size; acceptable for typical water-log sheets
* (< 10k rows/yr) but would need a search endpoint for huge sheets.
* Smartsheet does not expose a server-side "find by column" endpoint.
*/
export async function findRowByColumn(
sheetId: string,
token: string,
columnId: string,
value: string,
): Promise<{ rowId: string; rowNumber: number } | null> {
if (!sheetId || !token || !columnId) return null;
// We use `pageSize=1` is useless for search; fetch the whole sheet.
// For sheets up to ~10k rows this is fine. For larger, admin should
// switch to a sheet-summary API or a backend search.
type ApiResponse = {
rows: Array<{
id: number;
rowNumber: number;
cells: Array<{ columnId: number; value?: string | number | boolean | null }>;
}>;
};
let allRows: ApiResponse["rows"] = [];
let page = 1;
const PAGE_SIZE = 500;
// Bound the search to 5 pages (2,500 rows) to avoid runaway fetches.
const MAX_PAGES = 5;
while (page <= MAX_PAGES) {
const data = await request<ApiResponse>(
"GET",
`/2.0/sheets/${encodeURIComponent(sheetId)}?pageSize=${PAGE_SIZE}&page=${page}`,
token,
);
if (data.rows.length === 0) break;
allRows = allRows.concat(data.rows);
if (data.rows.length < PAGE_SIZE) break;
page += 1;
}
for (const row of allRows) {
const cell = row.cells.find((c) => String(c.columnId) === String(columnId));
if (cell && String(cell.value ?? "") === String(value)) {
return { rowId: String(row.id), rowNumber: row.rowNumber };
}
}
return null;
}