4 Commits

Author SHA1 Message Date
Tyler c6501b3ecd chore(release): v2.0.0
Deploy to route.crispygoat.com / deploy (push) Successful in 4m18s
2026-06-09 18:14:33 -06:00
Tyler edf3989ef2 fix(deploy): move inline node -e scripts into separate files to fix shell quoting errors
Deploy to route.crispygoat.com / deploy (push) Successful in 4m16s
The inline node -e '...' blocks inside the YAML run: block had shell
quoting conflicts (nested parens, single-quote escapes) that caused
act/local tooling to fail with 'syntax error near unexpected token ('.

Split into two dedicated scripts:
- scripts/preflight-check.js  — neon_auth schema check + 0001_init.sql tracking repair
- scripts/postflight-check.js — admin_users table verification after migrations

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 18:02:08 -06:00
Tyler 1d4300d505 fix(deploy): make 0001_init.sql re-runnable and repair historical _migrations tracking
Deploy to route.crispygoat.com / deploy (push) Failing after 1m12s
The "Run migrations" job (and thus the whole deploy) was failing on every push after the first successful bootstrap with:

  ✗ 0001_init.sql failed: relation "admin_users" already exists

Root causes:
- 0001_init.sql used plain CREATE TABLE/INDEX/TRIGGER despite the header comment claiming "CREATE TABLE IF NOT EXISTS".
- The _migrations tracking row for 0001_init.sql was never recorded on the prod DATABASE_URL (tracking logic landed after initial apply, or apply happened outside the runner).

Fix (3 layers of defense):
* db/migrations/0001_init.sql: every CREATE TABLE now uses IF NOT EXISTS (63 tables), CREATE INDEX uses IF NOT EXISTS (49), all CREATE TRIGGER wrapped in safe DO $$ IF NOT EXISTS (pg_trigger join check) THEN ... END IF; $$, removed the file's own BEGIN/COMMIT so the runner's transaction is authoritative. RLS drop+create and CREATE OR REPLACE FUNCTION were already safe.
* scripts/migrate.js: added ensureTracked() right after loading the applied set. For 0001/0002, if the core objects (admin_users table, password_hash column) already exist in information_schema but the tracking row is absent, INSERT the filename (ON CONFLICT DO NOTHING) and log "repaired tracking". Then the normal skip path handles it.
* .gitea/workflows/deploy.yml: added an inline node -e pre-repair (same admin_users existence check → force _migrations row) immediately before `npm run migrate:one`. This protects even on an older checkout of the runner. The neon_auth preflight and post-apply "admin_users must exist" hard gate are untouched.
* Minor: cleaned 0002_admin_password.sql tx wrapper for consistency; updated MEMORY.md with the incident + resolution.

Result: deploys (and any server-side `node scripts/migrate.js` recovery runs) on an already-initialized prod DB will repair/skip 0001 cleanly, pass the verification query, and continue to build/deploy. The scp of scripts/ + db/migrations/ in the Deploy step now carries the fixed versions.

See the plan note in deploy.yml and MEMORY.md for background.
2026-06-09 17:55:42 -06:00
Tyler 0db1609c89 fix(deploy): add explicit pre-flight check for neon_auth schema before running migrations (better error than raw 3F000 from 0001_init.sql) 2026-06-09 15:50:53 -06:00
8 changed files with 566 additions and 170 deletions
+2 -17
View File
@@ -25,24 +25,9 @@ jobs:
DATABASE_URL: ${{ secrets.DATABASE_URL }} DATABASE_URL: ${{ secrets.DATABASE_URL }}
run: | run: |
set -e set -e
echo "=== Running migrations against DATABASE_URL (masked value for logs) ===" node scripts/preflight-check.js
npm run migrate:one npm run migrate:one
echo "=== Post-migration verification: critical table admin_users must exist ===" node scripts/postflight-check.js
node -e '
const { Client } = require("pg");
const url = process.env.DATABASE_URL;
if (!url) { console.error("No DATABASE_URL"); process.exit(1); }
const c = new Client({ connectionString: url });
c.connect()
.then(() => c.query("SELECT 1 FROM admin_users LIMIT 1"))
.then(() => { console.log("✓ admin_users table exists and is queryable"); return c.end(); })
.then(() => process.exit(0))
.catch(e => {
console.error("✗ FATAL: admin_users relation missing after migrate:", e.message);
console.error("The deploy cannot continue. The secret DATABASE_URL must have had db/migrations/0001_init.sql applied successfully.");
process.exit(1);
});
'
- name: Build - name: Build
env: env:
+29 -1
View File
@@ -2,7 +2,35 @@
This file captures key context, decisions, fixes, and state from recent work so it survives across conversations. This file captures key context, decisions, fixes, and state from recent work so it survives across conversations.
**Last updated:** 2026-06-06 (Supabase → Postgres pivot) **Last updated:** 2026-06 (migration reliability + Google sign-in work)
## 2026-06: CI migration failures on re-deploy (0001_init.sql "already exists")
Prod DATABASE_URL already had the schema from the first successful bootstrap.
The deploy workflow runs `npm run migrate:one` on every push (after neon_auth preflight).
`scripts/migrate.js` has `_migrations` tracking + skip, but the row for `0001_init.sql` was never recorded (the tracking logic landed after the initial apply, or an apply happened outside the runner).
`db/migrations/0001_init.sql` header *claimed* "CREATE TABLE IF NOT EXISTS" but the actual statements were plain `CREATE TABLE`, plain `CREATE INDEX`, and unguarded `CREATE TRIGGER`.
Result: every subsequent deploy hit `relation "admin_users" already exists` (and would have hit index/trigger dups too) inside the runner's BEGIN, causing ROLLBACK + failure of the whole "Run migrations" job.
### Fixes applied
- Made `0001_init.sql` truly re-runnable:
- All `CREATE TABLE``CREATE TABLE IF NOT EXISTS`
- All `CREATE INDEX` / `CREATE UNIQUE INDEX``... IF NOT EXISTS`
- Every `CREATE TRIGGER` wrapped in a `DO $$ IF NOT EXISTS (pg_trigger check) THEN CREATE TRIGGER ... END IF; $$` guard
- Removed the file-level `BEGIN; ... COMMIT;` (the runner owns the tx; this also prevents inner-COMMIT from ending the runner tx early).
- `0002_admin_password.sql` had its tx wrapper removed for consistency (its ALTER was already `IF NOT EXISTS`).
- Hardened `scripts/migrate.js`:
- Added `ensureTracked()` repair: for 0001/0002, if the core objects (admin_users table, or the password_hash column) already exist in the target DB but the tracking row is absent, we INSERT the row (ON CONFLICT DO NOTHING) and skip the file. Logs "repaired tracking".
- Hardened `.gitea/workflows/deploy.yml` "Run migrations" step:
- Added an inline pre-repair node snippet (same idea) right before `npm run migrate:one`. This protects even if an older runner is checked out.
- The existing neon_auth preflight + post `admin_users` verification remain as the hard gate.
- Updated header comments and docs in the files.
After this, `npm run migrate` / deploys on an already-initialized DB will log the "repaired" or "already applied" lines for 0001 and proceed cleanly. The shipped `scripts/migrate.js` + `db/migrations/` on the target server also benefit for emergency recovery runs.
See also the plan doc referenced in deploy.yml for the broader reliability work.
--- ---
File diff suppressed because it is too large Load Diff
+2 -3
View File
@@ -10,12 +10,11 @@
-- `src/lib/auth.ts` — it queries this column and runs `verifyPassword` -- `src/lib/auth.ts` — it queries this column and runs `verifyPassword`
-- (see `src/lib/passwords.ts`) before returning the user. -- (see `src/lib/passwords.ts`) before returning the user.
BEGIN; -- Transaction is managed by the migrate runner (scripts/migrate.js).
-- File kept small and re-runnable via IF NOT EXISTS on the ALTER.
ALTER TABLE users ALTER TABLE users
ADD COLUMN IF NOT EXISTS password_hash TEXT; ADD COLUMN IF NOT EXISTS password_hash TEXT;
-- Update the updated_at trigger tracking — no new triggers needed since -- Update the updated_at trigger tracking — no new triggers needed since
-- `users` already has `set_updated_at` from migration 0001. -- `users` already has `set_updated_at` from migration 0001.
COMMIT;
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "route-commerce-platform", "name": "route-commerce-platform",
"version": "1.0.0", "version": "2.0.0",
"private": true, "private": true,
"scripts": { "scripts": {
"dev": "node fix-agents.js && next dev --webpack -H 0.0.0.0 -p 4000", "dev": "node fix-agents.js && next dev --webpack -H 0.0.0.0 -p 4000",
+39 -6
View File
@@ -1,14 +1,16 @@
#!/usr/bin/env node #!/usr/bin/env node
/** /**
* Apply Postgres migrations from `db/migrations/*.sql` in lexical order. * Apply Postgres migrations from `db/migrations/*.sql` in lexical order.
* Wraps the whole thing in a transaction; tracks applied files in * Wraps each new file in a transaction; tracks applied files in
* `_migrations` so re-runs are safe. * `_migrations` so re-runs are safe and idempotent.
*
* The 0001_init.sql (and 0002) are now fully re-runnable (IF NOT EXISTS +
* trigger guards) + this script has repair logic for DBs that were
* initialized before tracking was introduced.
* *
* Usage: * Usage:
* npm run db:migrate * npm run migrate
* * npm run migrate:one # same as above (applies all pending)
* Replaces the old `supabase/push-migrations.js` that script was
* hardcoded to a Supabase URL. This one reads `DATABASE_URL` directly.
*/ */
require("dotenv").config({ path: ".env.local" }); require("dotenv").config({ path: ".env.local" });
@@ -51,6 +53,37 @@ async function main() {
); );
const appliedSet = new Set(applied.map((r) => r.filename)); const appliedSet = new Set(applied.map((r) => r.filename));
// Repair for historical DBs (applied before _migrations tracking existed,
// or via direct psql / earlier tooling). If the core objects are present
// we record the filename so future deploys (and server-side recovery runs)
// treat 0001/0002 as done without re-executing the large init script.
async function ensureTracked(filename, existenceCheckSql) {
if (appliedSet.has(filename)) return;
try {
const { rows } = await client.query(existenceCheckSql);
if (rows.length > 0) {
await client.query(
`INSERT INTO _migrations (filename) VALUES ($1) ON CONFLICT (filename) DO NOTHING`,
[filename]
);
console.log(`${filename} (objects present in DB; repaired tracking)`);
appliedSet.add(filename);
}
} catch (e) {
// Non-fatal: the check may fail on a brand-new DB or with limited perms.
// We'll let the normal apply path handle it.
}
}
await ensureTracked(
"0001_init.sql",
"SELECT 1 FROM information_schema.tables WHERE table_name = 'admin_users' LIMIT 1"
);
await ensureTracked(
"0002_admin_password.sql",
"SELECT 1 FROM information_schema.columns WHERE table_name = 'users' AND column_name = 'password_hash' LIMIT 1"
);
let appliedNow = 0; let appliedNow = 0;
for (const file of files) { for (const file of files) {
if (appliedSet.has(file)) { if (appliedSet.has(file)) {
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env node
/**
* Post-migration verification.
* Called by .gitea/workflows/deploy.yml after npm run migrate:one.
* Confirms critical table admin_users is queryable.
*/
const { Client } = require("pg");
async function main() {
const url = process.env.DATABASE_URL;
if (!url) {
console.error("No DATABASE_URL");
process.exit(1);
}
const c = new Client({ connectionString: url });
await c.connect();
try {
console.log("=== Post-migration verification: critical table admin_users must exist ===");
await c.query("SELECT 1 FROM admin_users LIMIT 1");
console.log("✓ admin_users table exists and is queryable");
process.exit(0);
} catch (e) {
console.error("✗ FATAL: admin_users relation missing after migrate:", e.message);
console.error("The deploy cannot continue. The secret DATABASE_URL must have had db/migrations/0001_init.sql applied successfully.");
process.exit(1);
} finally {
await c.end();
}
}
main();
+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/env node
/**
* Preflight check + migration repair runner.
* Called by .gitea/workflows/deploy.yml before running migrations.
*
* 1. Verify neon_auth schema exists (prerequisite for 0001_init.sql FKs)
* 2. Ensure 0001_init.sql is tracked in _migrations if admin_users already exists
* (repair for DBs that had 0001_init.sql applied before _migrations was added)
*/
const { Client } = require("pg");
async function main() {
const url = process.env.DATABASE_URL;
if (!url) {
console.error("No DATABASE_URL");
process.exit(1);
}
const c = new Client({ connectionString: url });
await c.connect();
try {
// 1. Pre-flight: check neon_auth schema
console.log("=== Pre-flight: checking for neon_auth schema (required by 0001_init.sql for FKs to neon_auth.user and related RPCs) ===");
const schemaRes = await c.query(
"SELECT 1 FROM information_schema.schemata WHERE schema_name = 'neon_auth'"
);
if (schemaRes.rows.length === 0) {
console.error("✗ FATAL: neon_auth schema does not exist in the target database.");
console.error("Enable Neon Auth on the Neon project/branch that this DATABASE_URL points to first.");
console.error("Run: neonctl neon-auth (or equivalent) against the correct Neon branch.");
console.error("The neon_auth schema is created by Neon Auth and is a prerequisite for 0001_init.sql.");
process.exit(1);
}
console.log("✓ neon_auth schema exists");
// 2. Pre-repair: ensure 0001_init.sql is tracked if admin_users already exists
// This handles DBs where 0001_init.sql was applied before _migrations tracking existed.
console.log("=== Pre-repair: checking _migrations tracking (best-effort) ===");
try {
const tableRes = await c.query(
"SELECT 1 FROM information_schema.tables WHERE table_name = 'admin_users' LIMIT 1"
);
if (tableRes.rows.length > 0) {
await c.query(`
CREATE TABLE IF NOT EXISTS _migrations (
filename TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
INSERT INTO _migrations (filename)
VALUES ('0001_init.sql')
ON CONFLICT (filename) DO NOTHING;
`);
console.log("✓ 0001_init.sql tracking repaired (admin_users already present)");
}
} catch (e) {
// best-effort
console.log("(pre-repair skipped: " + e.message + ")");
}
console.log("Preflight complete. Proceeding to migrations...");
process.exit(0);
} catch (e) {
console.error("Preflight failed:", e.message);
process.exit(1);
} finally {
await c.end();
}
}
main();