tyler/route-commerce
1
0
Fork 1
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

Selfhost/migrate #1

Open
tyler wants to merge 21 commits from TOC/route-commerce:selfhost/migrate into main
pull from: TOC/route-commerce:selfhost/migrate
merge into: tyler:main
Conversation 0 Commits 21 Files Changed 203
+158 -129
1 changed files with 158 additions and 129 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit cbc8958b55 - Show all commits
docs/SUPABASE_DUMP_GUIDE.md
+158 -129
View File
@@ -1,176 +1,205 @@
# Supabase Dump & Restore Guide # Supabase Dump & Restore Guide
This guide is the runbook for when your brother removes the Supabase This guide is the runbook for capturing the real Supabase schema and data
spend cap and we can finally connect to the live Supabase project to and restoring it to a local Postgres database. It documents the exact
pull the real schema + data. steps that worked when the spend cap was removed on 2026-06-05.
## Prerequisites ## Connection (this works from the dev box)
- Brother has removed the Supabase spend cap (project status: ACTIVE_HEALTHY) The Supabase project `wnzkhezyhnfzhkhiflrp` (route-commerce) is hosted in
- Supabase project ref: `wnzkhezyhnfzhkhiflrp` (from CLAUDE.md) **East US (North Virginia)**. The dev box has no IPv6, so we must use
- Supabase DB password: `YLKzP9jz2yqop7jr` (user-provided) the **Supavisor pooler** (IPv4 only) — and it's on **`aws-1`**, not `aws-0`.
- Local Postgres running on `127.0.0.1:5432` (user: `routecommerce`,
password: `routecommerce_dev_password`, db: `route_commerce`)
## Connection Test
The Supabase hostname has NO IPv4 address from this dev box. You
must use the **Supabase Supavisor pooler** (port 6543) which resolves
over IPv4. From your home network, you can also use the direct
hostname on port 5432.
```bash ```bash
# Direct hostname (home network only — dev box has no IPv6) # Working pooler URL (from supabase/.temp/pooler-url)
psql "postgres://postgres.wnzkhezyhnfzhkhiflrp:YLKzP9jz2yqop7jr@db.wnzkhezyhnfzhkhiflrp.supabase.co:5432/postgres" postgresql://postgres.wnzkhezyhnfzhkhiflrp:YLKzP9jz2yqop7jr@aws-1-us-east-1.pooler.supabase.com:5432/postgres
# Pooler (works from dev box — IPv4 only)
psql "postgres://postgres.wnzkhezyhnfzhkhiflrp:YLKzP9jz2yqop7jr@aws-0-us-east-1.pooler.supabase.com:6543/postgres"
``` ```
If neither works after the cap is removed, check: The `aws-0-us-east-1.pooler.supabase.com` hostname resolves over IPv4
1. `getent hosts db.wnzkhezyhnfzhkhiflrp.supabase.co` — should return IPv4 but Supavisor returns "tenant/user not found" because the project lives
2. Supabase dashboard → Settings → API → "Direct connection" string on `aws-1`. The correct region number is non-obvious — always check
`supabase/.temp/pooler-url` for the actual endpoint.
## Capture Schema (no data) The direct hostname `db.wnzkhezyhnfzhkhiflrp.supabase.co` only resolves
over IPv6, which is unreachable from this dev box.
## pg_dump version
Supabase runs **PostgreSQL 17.6**. Local pg must be ≥ 17 to dump cleanly.
The dev box had pg 16 by default — install pg 17 client:
```bash ```bash
# From your home network, with the password set: echo "deb [signed-by=/usr/share/keyrings/pgdg.gpg] http://apt.postgresql.org/pub/repos/apt/ noble-pgdg main" \
PGPASSWORD="YLKzP9jz2yqop7jr" pg_dump \ | sudo tee /etc/apt/sources.list.d/pgdg.list
--host=db.wnzkhezyhnfzhkhiflrp.supabase.co \ sudo apt-get update
sudo apt-get install -y --allow-unauthenticated postgresql-client-17
# Use /usr/lib/postgresql/17/bin/pg_dump explicitly (or update PATH)
```
## Capture Schema
```bash
export PGPASSWORD="YLKzP9jz2yqop7jr"
export PATH="/usr/lib/postgresql/17/bin:$PATH"
mkdir -p supabase/captured
pg_dump \
--host=aws-1-us-east-1.pooler.supabase.com \
--port=5432 \ --port=5432 \
--username=postgres \ --username=postgres.wnzkhezyhnfzhkhiflrp \
--dbname=postgres \ --dbname=postgres \
--schema-only \ --schema-only \
--no-owner \ --no-owner \
--no-privileges \ --no-privileges \
--no-acl \ --no-acl \
--file=supabase/captured_schema.sql --exclude-schema=auth \
--exclude-schema=storage \
--exclude-schema=realtime \
--exclude-schema=supabase_functions \
--exclude-schema=graphql \
--exclude-schema=graphql_public \
--exclude-schema=pgsodium \
--exclude-schema=pgsodium_masks \
--exclude-schema=extensions \
--exclude-schema=pgbouncer \
--exclude-schema=supabase_migrations \
--exclude-schema=net \
--exclude-schema=vault \
--file=supabase/captured/captured_schema.sql
``` ```
This produces a SQL file with all CREATE TABLE, CREATE FUNCTION, Captured schema is ~540KB, 65 tables, 252 functions.
CREATE INDEX, etc. statements. May be 50-200 MB depending on the
function count. **Compress it before committing:** ## Capture Data
```bash ```bash
gzip supabase/captured_schema.sql # → captured_schema.sql.gz pg_dump \
``` --host=aws-1-us-east-1.pooler.supabase.com \
## Capture Data (no schema)
```bash
PGPASSWORD="YLKzP9jz2yqop7jr" pg_dump \
--host=db.wnzkhezyhnfzhkhiflrp.supabase.co \
--port=5432 \ --port=5432 \
--username=postgres \ --username=postgres.wnzkhezyhnfzhkhiflrp \
--dbname=postgres \ --dbname=postgres \
--data-only \ --data-only \
--no-owner \ --no-owner \
--no-privileges \ --no-privileges \
--no-acl \ --no-acl \
--disable-triggers \ --disable-triggers \
--file=supabase/captured_data.sql --exclude-schema=auth \
--exclude-schema=storage \
--exclude-schema=realtime \
--exclude-schema=supabase_functions \
--exclude-schema=graphql \
--exclude-schema=graphql_public \
--exclude-schema=pgsodium \
--exclude-schema=pgsodium_masks \
--exclude-schema=extensions \
--exclude-schema=pgbouncer \
--exclude-schema=supabase_migrations \
--exclude-schema=net \
--exclude-schema=vault \
--file=supabase/captured/captured_data.sql
``` ```
Use `--disable-triggers` so the dump skips triggers that might fire Captured data is ~130KB for this project's current size.
on INSERT and slow things down. Compress:
## Restore to Local DB (PostgreSQL 16)
PostgreSQL 16 doesn't support `transaction_timeout` (added in PG 17) and
doesn't have the `supabase_vault` extension. Pre-create the stub objects:
```bash ```bash
gzip supabase/captured_data.sql
```
## Restore to Local DB
```bash
cd /path/to/route-commerce
export PGPASSWORD=routecommerce_dev_password export PGPASSWORD=routecommerce_dev_password
psql -U routecommerce -h 127.0.0.1 -d route_commerce <<'SQL'
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
GRANT ALL ON SCHEMA public TO routecommerce;
# 1. Wipe the synthesized schema -- extensions schema (referenced by dump as extensions.uuid_generate_v4())
psql -U routecommerce -h 127.0.0.1 -d route_commerce \ CREATE SCHEMA extensions;
-c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;" CREATE EXTENSION IF NOT EXISTS "uuid-ossp" SCHEMA extensions;
CREATE EXTENSION IF NOT EXISTS "pgcrypto" SCHEMA extensions;
CREATE EXTENSION IF NOT EXISTS "pg_stat_statements" SCHEMA extensions;
# 2. Apply the real schema -- Stub functions in extensions schema (real Supabase has pgcrypto.crypt etc.)
zcat supabase/captured_schema.sql.gz | \ CREATE OR REPLACE FUNCTION extensions.uuid_generate_v4()
psql -U routecommerce -h 127.0.0.1 -d route_commerce \ RETURNS uuid LANGUAGE sql AS $$ SELECT gen_random_uuid(); $$;
-v ON_ERROR_STOP=1 2>&1 | tee /tmp/schema_restore.log CREATE OR REPLACE FUNCTION extensions.gen_salt(text)
RETURNS text LANGUAGE sql AS $$ SELECT '$2a$06$' || repeat('A', 53); $$;
CREATE OR REPLACE FUNCTION extensions.crypt(text, text)
RETURNS text LANGUAGE sql AS $$ SELECT $1; $$;
# 3. Apply the data GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA extensions TO routecommerce;
zcat supabase/captured_data.sql.gz | \
psql -U routecommerce -h 127.0.0.1 -d route_commerce \ -- auth stub (we excluded auth schema from dump, but RLS policies reference auth.uid())
-v ON_ERROR_STOP=1 2>&1 | tee /tmp/data_restore.log CREATE SCHEMA IF NOT EXISTS auth;
CREATE TABLE IF NOT EXISTS auth.users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
email TEXT,
raw_user_meta_data JSONB,
created_at TIMESTAMPTZ DEFAULT now(),
updated_at TIMESTAMPTZ DEFAULT now()
);
CREATE OR REPLACE FUNCTION auth.uid() RETURNS UUID
LANGUAGE sql STABLE AS $$ SELECT NULL::UUID; $$;
GRANT USAGE ON SCHEMA auth TO routecommerce;
GRANT ALL ON auth.users TO routecommerce;
GRANT EXECUTE ON FUNCTION auth.uid() TO routecommerce;
-- Drop Supabase-specific publications (we don't have realtime / vault)
DROP PUBLICATION IF EXISTS supabase_realtime;
DROP PUBLICATION IF EXISTS supabase_realtime_messages_publication;
SQL
``` ```
Strip the PG17-only `SET transaction_timeout` line from the dump:
```bash
sed -i 's/^SET transaction_timeout = 0;$/-- transaction_timeout requires PG17 (we are on PG16); skipped/' \
supabase/captured/captured_schema.sql
```
Apply schema and data (idempotent — re-runs are safe):
```bash
psql -U routecommerce -h 127.0.0.1 -d route_commerce -v ON_ERROR_STOP=0 \
-f supabase/captured/captured_schema.sql 2>&1 | tail -20
psql -U routecommerce -h 127.0.0.1 -d route_commerce -v ON_ERROR_STOP=0 \
-f supabase/captured/captured_data.sql 2>&1 | tail -20
```
Most errors on re-run are "already exists" — that's expected because the
dump is idempotent for CREATE statements (we used `IF NOT EXISTS` where
possible, but pg_dump doesn't add it for everything).
## Verify ## Verify
```bash ```bash
# Should show ~70+ tables (not just the 70 from the synthesized schema) psql -U routecommerce -h 127.0.0.1 -d route_commerce -c "SELECT count(*) FROM pg_tables WHERE schemaname='public';"
psql -U routecommerce -h 127.0.0.1 -d route_commerce -c "\dt" # Expect: 65
# Should show actual data, not 0 rows everywhere psql -U routecommerce -h 127.0.0.1 -d route_commerce -c "SELECT count(*) FROM pg_proc WHERE pronamespace=(SELECT oid FROM pg_namespace WHERE nspname='public');"
psql -U routecommerce -h 127.0.0.1 -d route_commerce -c "SELECT count(*) FROM brands;" # Expect: ~250+
psql -U routecommerce -h 127.0.0.1 -d route_commerce -c "SELECT count(*) FROM products;"
psql -U routecommerce -h 127.0.0.1 -d route_commerce -c "SELECT count(*) FROM orders;" psql -U routecommerce -h 127.0.0.1 -d route_commerce -c "SELECT name, slug, created_at FROM brands ORDER BY created_at;"
# Expect 5 brands: Tuxedo Corn, Indian River Direct, Sunrise Farms, Green Valley Organics, Orchard Fresh
``` ```
## Clean Up ## What Didn't Work (don't try these)
```bash | Approach | Why it failed |
# Once verified, drop the synthesized schema (no longer needed) |---|---|
rm supabase/synthesized/000_base_schema.sql | `psql ... db.wnzkhezyhnfzhkhiflrp.supabase.co:5432` | Hostname is IPv6-only, dev box has no IPv6 |
git add -A | `aws-0-us-east-1.pooler.supabase.com` | "tenant/user not found" — wrong region (project is on `aws-1`) |
git commit -m "remove synthesized schema (real dump now in place)" | `pg_dump 16.x` against PG 17 server | "server version mismatch" — fatal error |
``` | `supabase db dump --linked` | Requires Docker (we don't have it) |
## Schema-Per-Brand Restructure (FUTURE, not in this dump) ## Notes
The user wants each brand in its own Postgres schema: - The captured data includes 3 test brands (Sunrise, Green Valley, Orchard)
- `brand_tuxedo` — products, orders, stops, etc. for Tuxedo created on 2026-06-03 — these were test data the user added during the
- `brand_indian_river_direct` — products, orders, stops, etc. for IRD migration work, not real customers. They can be deleted safely.
- Tuxedo Corn and Indian River Direct are the real production brands.
This is a major refactor and is **NOT** part of the dump-and-restore. - The schema dump is committed to git at `supabase/captured/captured_schema.sql.gz`
Plan for it as a follow-up phase once the data is loaded into shared for reproducibility. The data dump is gitignored (regenerate as needed).
tables and the app is verified working. - After dump, the local PostgREST needs a schema cache reload — restart
the postgrest process or it'll serve stale metadata for ~30 seconds.
The refactor will involve:
1. Creating schemas `brand_tuxedo`, `brand_indian_river_direct`
2. Moving brand-specific tables into the appropriate schema
3. Updating all RPC SECURITY DEFINER functions to set
`search_path` based on caller brand
4. Updating server actions to set `search_path` per request
5. Updating PostgREST config to expose schemas
6. Testing brand isolation thoroughly
## Troubleshooting
### "password authentication failed"
Password has been rotated. Check Supabase dashboard → Settings →
Database → Reset password. The user-provided password
`YLKzP9jz2yqop7jr` may need to be reset.
### "could not translate host name"
Dev box has no IPv6. Use the pooler hostname
`aws-0-us-east-1.pooler.supabase.com` or run from a machine with IPv6.
### "permission denied for schema auth"
The dump includes the `auth` schema which is Supabase-managed.
Filter it out with: `--exclude-schema=auth --exclude-schema=storage
--exclude-schema=realtime --exclude-schema=supabase_functions`.
### "relation already exists"
You forgot step 1 (DROP SCHEMA public CASCADE). The data restore
runs `INSERT` and may not recreate tables — only the schema dump
does that.
### "out of memory" during data restore
The data file is huge. Use `--single-transaction` to keep
Postgres from spilling to disk:
```bash
zcat supabase/captured_data.sql.gz | \
psql -U routecommerce -h 127.0.0.1 -d route_commerce \
--single-transaction \
-v ON_ERROR_STOP=0 2>&1 | tee /tmp/data_restore.log
```
### Data dump is too big to commit
Don't commit it. The data file is in `.gitignore` and gets
re-loaded on first run via a one-shot script.