feat(water-log): Smartsheet sync integration
Deploy to route.crispygoat.com / deploy (push) Successful in 5m34s
Deploy to route.crispygoat.com / deploy (push) Successful in 5m34s
Per-brand Smartsheet integration that pushes new water_log_entries to a
configured sheet. Config UI lives at /admin/water-log/settings.
Database (0093_water_smartsheet_sync.sql):
- water_smartsheet_config: one row per brand; AES-256-GCM encrypted API
token (BYTEA), sheet id, column mapping JSONB, sync_frequency
(realtime | every_15_minutes | hourly), enable flag, last-sync
metadata. RLS via existing current_brand_id() pattern.
- water_smartsheet_sync_queue: one row per entry awaiting sync;
tracks status / attempts / exponential backoff. UNIQUE(brand_id,
entry_id) for idempotent enqueue.
- water_smartsheet_sync_log: append-only audit of every sync attempt
(success or failure); backs the Recent Activity panel in the UI.
Server:
- src/lib/crypto.ts: AES-256-GCM encrypt/decrypt + token mask helper.
Reads SMARTSHEET_TOKEN_ENC_KEY. Throws on missing/wrong-size key.
- src/lib/smartsheet.ts: typed REST wrapper (no SDK; the official
smartsheet npm has Node 22 compat issues). getSheetMeta, addRow,
findRowByColumn + typed SmartsheetApiError.
- src/services/smartsheet-sync.ts: syncEntryToSmartsheet (one entry),
drainSyncQueue (batch with backoff), runScheduledSync (cron entry).
Sequential processing stays well under Smartsheet's 300 req/min.
Max 5 attempts, then row stays 'failed' permanently.
- src/actions/water-log/smartsheet.ts: 6 server actions. Token never
returned in plaintext — UI gets maskedToken only. Permission check
matches the rest of the water-log module (can_manage_water_log).
- src/app/api/water-log/smartsheet-sync/route.ts: POST cron route,
bearer auth via SMARTSHEET_CRON_SECRET (falls back to CRON_SECRET).
Optional {brandId} body for manual retry.
- vercel.json: added */15 * * * * cron entry. Single entry covers
both 15-min and hourly frequencies (route filters per-brand).
UI:
- src/components/admin/water-log/SmartsheetIntegrationCard.tsx:
self-contained client component with useReducer. Sections: enable
toggle, sheet URL/ID + API token + Test Connection, sync frequency
radio group, column mapping dropdowns (populated after Test),
Recent Activity panel.
- src/app/admin/water-log/settings/page.tsx: mounted the card as new
'§ 05 — Integrations' section after the existing admin-PIN settings.
Card takes brandId as a prop (CLAUDE.md 'Brand ID Threading').
Hooks:
- src/actions/water-log/field.ts::submitWaterEntry: calls
triggerSyncForEntry(brandId, entryId) in a fire-and-forget void
after the entry insert. Sync failures NEVER bubble to the field
worker.
Env vars (.env.example):
- SMARTSHEET_TOKEN_ENC_KEY (REQUIRED, 32 random bytes base64)
- SMARTSHEET_CRON_SECRET (optional bearer)
- SMARTSHEET_SYNC_TIMEOUT_MS (optional, default 8000)
Deploy:
1. openssl rand -base64 32 → SMARTSHEET_TOKEN_ENC_KEY in .env.local
AND Vercel dashboard
2. npm run migrate:one 93 (pushes 0093_water_smartsheet_sync.sql)
3. Push triggers .gitea/workflows/deploy.yml
Rollback: remove cron entry, DROP the 3 tables, revert field.ts +
settings page hooks, git revert the merge commit. See MEMORY.md
for full rollback story + sharp edges (no key rotation, slug URLs).
This commit is contained in:
@@ -0,0 +1,125 @@
|
||||
/**
|
||||
* AES-256-GCM token encryption for at-rest secrets.
|
||||
*
|
||||
* Used by the Smartsheet integration (and any future third-party token
|
||||
* storage) to keep API tokens encrypted in Postgres. AES-256-GCM gives
|
||||
* us confidentiality + integrity in one primitive: a unique 12-byte
|
||||
* IV per encrypt, a 16-byte auth tag that detects tampering.
|
||||
*
|
||||
* Key management:
|
||||
* - Single key per environment, sourced from `SMARTSHEET_TOKEN_ENC_KEY`
|
||||
* (32 random bytes, base64-encoded).
|
||||
* - Generate with: `openssl rand -base64 32`
|
||||
* - Storing the key in env vars matches the codebase convention
|
||||
* (`STRIPE_SECRET_KEY`, `NEON_AUTH_COOKIE_SECRET`, etc.).
|
||||
* - Key rotation is *not* supported in this version — if the key
|
||||
* changes, existing encrypted blobs become unreadable and admins
|
||||
* must re-enter tokens. Phase 2 should add a `key_version` column
|
||||
* on the config table to support zero-downtime rotation.
|
||||
*
|
||||
* Why AES-256-GCM?
|
||||
* - Built into Node's `crypto` module — zero dependencies.
|
||||
* - Authenticated encryption — we detect (and refuse to decrypt)
|
||||
* ciphertext that was tampered with.
|
||||
* - Faster than AES-CBC + HMAC and easier to use correctly.
|
||||
*/
|
||||
import "server-only";
|
||||
import { createCipheriv, createDecipheriv, randomBytes } from "node:crypto";
|
||||
|
||||
const ALGO = "aes-256-gcm";
|
||||
const IV_LEN = 12; // GCM standard
|
||||
const TAG_LEN = 16; // GCM standard
|
||||
const KEY_LEN = 32; // 256 bits
|
||||
|
||||
export type EncryptedToken = {
|
||||
/** Ciphertext (raw bytes; not base64). Stored as BYTEA. */
|
||||
cipher: Buffer;
|
||||
/** 12-byte IV, unique per encrypt. Stored as BYTEA. */
|
||||
iv: Buffer;
|
||||
/** 16-byte GCM auth tag. Stored as BYTEA. */
|
||||
authTag: Buffer;
|
||||
};
|
||||
|
||||
function getKey(): Buffer {
|
||||
const raw = process.env.SMARTSHEET_TOKEN_ENC_KEY;
|
||||
if (!raw || raw.length === 0) {
|
||||
if (process.env.NODE_ENV !== "production") {
|
||||
throw new Error(
|
||||
"SMARTSHEET_TOKEN_ENC_KEY is not set. Generate one with `openssl rand -base64 32` and add it to .env.local.",
|
||||
);
|
||||
}
|
||||
throw new Error(
|
||||
"SMARTSHEET_TOKEN_ENC_KEY is not set in production. This is required — refusing to fall back to an insecure default.",
|
||||
);
|
||||
}
|
||||
const key = Buffer.from(raw, "base64");
|
||||
if (key.length !== KEY_LEN) {
|
||||
throw new Error(
|
||||
`SMARTSHEET_TOKEN_ENC_KEY must decode to exactly ${KEY_LEN} bytes (got ${key.length}). Regenerate with \`openssl rand -base64 32\`.`,
|
||||
);
|
||||
}
|
||||
return key;
|
||||
}
|
||||
|
||||
/**
|
||||
* Encrypt a plaintext token (e.g. a Smartsheet API access token).
|
||||
*
|
||||
* Returns a triple of Buffers that should each be persisted to a
|
||||
* separate BYTEA column. Never log the plaintext or the auth tag.
|
||||
*/
|
||||
export function encryptToken(plaintext: string): EncryptedToken {
|
||||
if (typeof plaintext !== "string" || plaintext.length === 0) {
|
||||
throw new Error("encryptToken: plaintext must be a non-empty string");
|
||||
}
|
||||
const key = getKey();
|
||||
const iv = randomBytes(IV_LEN);
|
||||
const cipher = createCipheriv(ALGO, key, iv);
|
||||
const ciphertext = Buffer.concat([
|
||||
cipher.update(plaintext, "utf8"),
|
||||
cipher.final(),
|
||||
]);
|
||||
const authTag = cipher.getAuthTag();
|
||||
return { cipher: ciphertext, iv, authTag };
|
||||
}
|
||||
|
||||
/**
|
||||
* Decrypt a token previously produced by `encryptToken`.
|
||||
*
|
||||
* Throws if the auth tag verification fails (tampered ciphertext).
|
||||
* Throws if the env key is missing/wrong size (key rotation break).
|
||||
*/
|
||||
export function decryptToken(token: EncryptedToken): string {
|
||||
if (
|
||||
!Buffer.isBuffer(token.cipher) ||
|
||||
!Buffer.isBuffer(token.iv) ||
|
||||
!Buffer.isBuffer(token.authTag)
|
||||
) {
|
||||
throw new Error("decryptToken: invalid token shape");
|
||||
}
|
||||
if (token.iv.length !== IV_LEN) {
|
||||
throw new Error(`decryptToken: iv must be ${IV_LEN} bytes`);
|
||||
}
|
||||
if (token.authTag.length !== TAG_LEN) {
|
||||
throw new Error(`decryptToken: authTag must be ${TAG_LEN} bytes`);
|
||||
}
|
||||
const key = getKey();
|
||||
const decipher = createDecipheriv(ALGO, key, token.iv);
|
||||
decipher.setAuthTag(token.authTag);
|
||||
const plaintext = Buffer.concat([
|
||||
decipher.update(token.cipher),
|
||||
decipher.final(), // throws on auth-tag mismatch
|
||||
]);
|
||||
return plaintext.toString("utf8");
|
||||
}
|
||||
|
||||
/**
|
||||
* Mask a token for UI display. Returns "••••abcd" where "abcd" is the
|
||||
* last 4 characters of the plaintext. For tokens < 4 chars, returns
|
||||
* "••••" (no tail). Used by the Settings card to show "we have a token
|
||||
* saved" without revealing it.
|
||||
*/
|
||||
export function maskToken(plaintext: string): string {
|
||||
if (!plaintext || plaintext.length === 0) return "";
|
||||
if (plaintext.length <= 4) return "••••";
|
||||
return `••••${plaintext.slice(-4)}`;
|
||||
}
|
||||
@@ -0,0 +1,324 @@
|
||||
/**
|
||||
* Smartsheet REST API wrapper (no SDK).
|
||||
*
|
||||
* We use direct REST instead of the official `smartsheet` npm SDK
|
||||
* because:
|
||||
* - The SDK has Node 22 compatibility issues (top-level `await` in
|
||||
* a CommonJS module) and pulls in ~1.4MB of transitive deps
|
||||
* (`request`, `request-promise`, `lodash`, etc.).
|
||||
* - REST covers our 3 endpoints cleanly: get sheet metadata, add a
|
||||
* row, search a sheet by column value.
|
||||
* - Smartsheet's own docs recommend REST for simple integrations.
|
||||
*
|
||||
* Endpoints used (all docs at https://smartsheet.redoc.ly):
|
||||
* - GET /2.0/sheets/{sheetId} — sheet metadata + columns
|
||||
* - POST /2.0/sheets/{sheetId}/rows — add a row
|
||||
* - GET /2.0/sheets/{sheetId} — full sheet (used for find-by-column)
|
||||
*
|
||||
* Auth:
|
||||
* - Bearer token via `Authorization: Bearer <token>` header.
|
||||
* - 300 requests/minute per token. Our drain caps at 50 rows per
|
||||
* cron run, well under the limit.
|
||||
*
|
||||
* Errors:
|
||||
* - `SmartsheetApiError` is thrown for any non-2xx response. It
|
||||
* carries `status` and `body` for the caller to log.
|
||||
*/
|
||||
import "server-only";
|
||||
|
||||
const BASE_URL = "https://api.smartsheet.com";
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = Number.parseInt(
|
||||
process.env.SMARTSHEET_SYNC_TIMEOUT_MS ?? "8000",
|
||||
10,
|
||||
);
|
||||
|
||||
export type SmartsheetColumn = {
|
||||
/** Numeric column ID, as a string (Smartsheet IDs overflow JS numbers). */
|
||||
id: string;
|
||||
/** Display name in the sheet header row. */
|
||||
title: string;
|
||||
/** Column type — TEXT_NUMBER, DATE, PICKLIST, etc. We treat all as text on write. */
|
||||
type: string;
|
||||
/** Column index in the sheet (0-based). */
|
||||
index: number;
|
||||
};
|
||||
|
||||
export type SmartsheetSheetMeta = {
|
||||
id: string;
|
||||
name: string;
|
||||
columns: SmartsheetColumn[];
|
||||
};
|
||||
|
||||
export type SmartsheetCell = {
|
||||
columnId: string;
|
||||
value: string | number | boolean | null;
|
||||
};
|
||||
|
||||
export class SmartsheetApiError extends Error {
|
||||
readonly status: number;
|
||||
readonly body: string;
|
||||
readonly code: number | null;
|
||||
|
||||
constructor(status: number, body: string, message?: string) {
|
||||
super(
|
||||
message ??
|
||||
`Smartsheet API error ${status}: ${body.slice(0, 200)}${
|
||||
body.length > 200 ? "…" : ""
|
||||
}`,
|
||||
);
|
||||
this.name = "SmartsheetApiError";
|
||||
this.status = status;
|
||||
this.body = body;
|
||||
// Smartsheet returns { errorCode, message, ... } on failure
|
||||
try {
|
||||
const parsed = JSON.parse(body) as { errorCode?: number };
|
||||
this.code = parsed.errorCode ?? null;
|
||||
} catch {
|
||||
this.code = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function request<T>(
|
||||
method: "GET" | "POST",
|
||||
path: string,
|
||||
token: string,
|
||||
body?: unknown,
|
||||
): Promise<T> {
|
||||
const url = `${BASE_URL}${path}`;
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), DEFAULT_TIMEOUT_MS);
|
||||
|
||||
let res: Response;
|
||||
try {
|
||||
res = await fetch(url, {
|
||||
method,
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Content-Type": "application/json",
|
||||
Accept: "application/json",
|
||||
"User-Agent": "RouteCommerce-WaterLog/1.0",
|
||||
},
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
signal: controller.signal,
|
||||
// Disable Next.js fetch caching for API calls — we always want
|
||||
// fresh data from Smartsheet.
|
||||
cache: "no-store",
|
||||
});
|
||||
} catch (err) {
|
||||
if ((err as { name?: string }).name === "AbortError") {
|
||||
throw new SmartsheetApiError(
|
||||
408,
|
||||
`Smartsheet request timed out after ${DEFAULT_TIMEOUT_MS}ms`,
|
||||
);
|
||||
}
|
||||
throw new SmartsheetApiError(
|
||||
0,
|
||||
`Network error contacting Smartsheet: ${(err as Error).message ?? err}`,
|
||||
);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
const errBody = await res.text();
|
||||
throw new SmartsheetApiError(res.status, errBody);
|
||||
}
|
||||
|
||||
// Some 204s return no body; guard.
|
||||
const text = await res.text();
|
||||
if (!text) return undefined as unknown as T;
|
||||
return JSON.parse(text) as T;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract the numeric sheet ID from either:
|
||||
* - A bare ID: "123456789012345"
|
||||
* - A URL: "https://app.smartsheet.com/sheets/abc123?..."
|
||||
*
|
||||
* Smartsheet share URLs include an opaque slug (e.g. "abc123"). We
|
||||
* can't resolve that slug without an API call (the sheet meta call
|
||||
* will fail with 404 anyway), so we just take the last path segment
|
||||
* and pass it as the sheet ID. If the user pasted a slug URL, the
|
||||
* "Test Connection" UI will surface the 404 and prompt for a numeric
|
||||
* ID instead. (This matches what most Smartsheet integrations do.)
|
||||
*/
|
||||
export function extractSheetId(input: string): string {
|
||||
const trimmed = input.trim();
|
||||
if (!trimmed) {
|
||||
throw new Error("Sheet ID or URL is required");
|
||||
}
|
||||
// Plain numeric ID
|
||||
if (/^\d+$/.test(trimmed)) return trimmed;
|
||||
|
||||
// URL form
|
||||
try {
|
||||
const url = new URL(trimmed);
|
||||
const segments = url.pathname.split("/").filter(Boolean);
|
||||
// Find the segment after "sheets"
|
||||
const sheetsIdx = segments.findIndex((s) => s === "sheets");
|
||||
if (sheetsIdx >= 0 && segments[sheetsIdx + 1]) {
|
||||
return segments[sheetsIdx + 1];
|
||||
}
|
||||
// Last segment as a fallback
|
||||
if (segments.length > 0) {
|
||||
return segments[segments.length - 1];
|
||||
}
|
||||
} catch {
|
||||
// Not a URL — fall through
|
||||
}
|
||||
// Treat as opaque ID; API will surface 404 if it's not numeric.
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch sheet metadata (columns, name) — used by the "Test Connection"
|
||||
* button to populate the column-mapping dropdowns.
|
||||
*
|
||||
* Calls `GET /2.0/sheets/{sheetId}?pageSize=1` — we only need the
|
||||
* header row (column definitions), not the row data, so a 1-row
|
||||
* response keeps the payload tiny.
|
||||
*/
|
||||
export async function getSheetMeta(
|
||||
sheetId: string,
|
||||
token: string,
|
||||
): Promise<SmartsheetSheetMeta> {
|
||||
if (!sheetId) throw new Error("sheetId is required");
|
||||
if (!token) throw new Error("token is required");
|
||||
|
||||
type ApiResponse = {
|
||||
result: {
|
||||
id: number;
|
||||
name: string;
|
||||
columns: Array<{
|
||||
id: number;
|
||||
title: string;
|
||||
type: string;
|
||||
index: number;
|
||||
}>;
|
||||
};
|
||||
};
|
||||
|
||||
const data = await request<ApiResponse>(
|
||||
"GET",
|
||||
`/2.0/sheets/${encodeURIComponent(sheetId)}?pageSize=1`,
|
||||
token,
|
||||
);
|
||||
|
||||
return {
|
||||
id: String(data.result.id),
|
||||
name: data.result.name,
|
||||
columns: data.result.columns.map((c) => ({
|
||||
id: String(c.id),
|
||||
title: c.title,
|
||||
type: c.type,
|
||||
index: c.index,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Add a row to a sheet. `cells` is mapped to Smartsheet's
|
||||
* `[{columnId, value, strict?}]` shape. Boolean values are coerced
|
||||
* to "TRUE"/"FALSE" (Smartsheet wants strings for PICKLIST columns).
|
||||
*
|
||||
* Returns the new row's `id` and `rowNumber`.
|
||||
*/
|
||||
export async function addRow(
|
||||
sheetId: string,
|
||||
token: string,
|
||||
cells: SmartsheetCell[],
|
||||
): Promise<{ rowId: string; rowNumber: number }> {
|
||||
if (!sheetId) throw new Error("sheetId is required");
|
||||
if (!token) throw new Error("token is required");
|
||||
if (cells.length === 0) throw new Error("addRow: cells must not be empty");
|
||||
|
||||
const toCellValue = (
|
||||
v: string | number | boolean | null,
|
||||
): string | boolean | number => {
|
||||
if (v === null || v === undefined) return "";
|
||||
if (typeof v === "boolean") return v;
|
||||
return v;
|
||||
};
|
||||
|
||||
type ApiResponse = {
|
||||
result: { id: number; rowNumber: number };
|
||||
};
|
||||
|
||||
const data = await request<ApiResponse>(
|
||||
"POST",
|
||||
`/2.0/sheets/${encodeURIComponent(sheetId)}/rows`,
|
||||
token,
|
||||
{
|
||||
rows: [
|
||||
{
|
||||
cells: cells.map((c) => ({
|
||||
columnId: Number(c.columnId),
|
||||
value: toCellValue(c.value),
|
||||
})),
|
||||
},
|
||||
],
|
||||
},
|
||||
);
|
||||
|
||||
return {
|
||||
rowId: String(data.result.id),
|
||||
rowNumber: data.result.rowNumber,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find an existing row by a column value. Used for dedup — if a row
|
||||
* already exists for the given `entry_id`, skip the insert and
|
||||
* record the existing row ID in the queue.
|
||||
*
|
||||
* Implementation: scans the sheet's `columnId` column for `value`.
|
||||
* This is O(n) on sheet size; acceptable for typical water-log sheets
|
||||
* (< 10k rows/yr) but would need a search endpoint for huge sheets.
|
||||
* Smartsheet does not expose a server-side "find by column" endpoint.
|
||||
*/
|
||||
export async function findRowByColumn(
|
||||
sheetId: string,
|
||||
token: string,
|
||||
columnId: string,
|
||||
value: string,
|
||||
): Promise<{ rowId: string; rowNumber: number } | null> {
|
||||
if (!sheetId || !token || !columnId) return null;
|
||||
// We use `pageSize=1` is useless for search; fetch the whole sheet.
|
||||
// For sheets up to ~10k rows this is fine. For larger, admin should
|
||||
// switch to a sheet-summary API or a backend search.
|
||||
type ApiResponse = {
|
||||
rows: Array<{
|
||||
id: number;
|
||||
rowNumber: number;
|
||||
cells: Array<{ columnId: number; value?: string | number | boolean | null }>;
|
||||
}>;
|
||||
};
|
||||
|
||||
let allRows: ApiResponse["rows"] = [];
|
||||
let page = 1;
|
||||
const PAGE_SIZE = 500;
|
||||
// Bound the search to 5 pages (2,500 rows) to avoid runaway fetches.
|
||||
const MAX_PAGES = 5;
|
||||
|
||||
while (page <= MAX_PAGES) {
|
||||
const data = await request<ApiResponse>(
|
||||
"GET",
|
||||
`/2.0/sheets/${encodeURIComponent(sheetId)}?pageSize=${PAGE_SIZE}&page=${page}`,
|
||||
token,
|
||||
);
|
||||
if (data.rows.length === 0) break;
|
||||
allRows = allRows.concat(data.rows);
|
||||
if (data.rows.length < PAGE_SIZE) break;
|
||||
page += 1;
|
||||
}
|
||||
|
||||
for (const row of allRows) {
|
||||
const cell = row.cells.find((c) => String(c.columnId) === String(columnId));
|
||||
if (cell && String(cell.value ?? "") === String(value)) {
|
||||
return { rowId: String(row.id), rowNumber: row.rowNumber };
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
Reference in New Issue
Block a user