Batching all migrations into one psql session caused a hard error to put
Postgres into an aborted transaction state, blocking every subsequent
migration. Back to per-file calls with PAGER= still set to prevent hang.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
psql was launching less/more for query output in the CI shell, then
blocking indefinitely on "Press RETURN to continue". Also batch all
numbered migrations into one psql session instead of one process per file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dynamic port leasing was the root cause of all the "address already in use"
failures. Production now always uses 3010; dev still uses 3001. No more
.postgrest-port file, no more port scanning loop, no more race conditions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Multiple --filter name= flags are ANDed in docker ps, so combining
route_commerce_ and route-commerce- in one call matched nothing and
left db/minio running. Split into two calls, dedup with sort -u.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docker compose down was silently failing (|| true) when the compose file
was stale, leaving db/minio running on the old network subnet. The new
postgrest container then couldn't bind because Docker tried to reuse the
same subnet with conflicting container IPs.
Now: rm -f all containers matching route_commerce_/route-commerce- by name
first (works regardless of compose file state), then remove the network
explicitly so Docker allocates a clean subnet on the next up.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The three hardcoded pkill lines only covered ports 3011-3013. docker-proxy
can linger after compose down, causing "address already in use" when the new
stack races to bind the same port. Replace with a single pattern covering all
30xx ports and bump sleep to 5s.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The Start Docker stack step was racing itself: docker compose down
errors were silently swallowed (|| true), leaving the previous
postgrest container in a half-torn-down state. fuser -k would kill
the listener, ss would see 3011 as free, the script would pick 3011,
and then docker compose up would fail to bind because the port was
in TIME_WAIT (or a stale docker-proxy listener was still holding it).
Three changes:
1. Don't swallow docker compose down errors. If the previous stack
isn't actually down, fail loudly so we can investigate.
2. Explicitly remove any surviving route_commerce_postgrest container
and kill any leftover docker-proxy listeners for ports 3011-3013
before picking a port.
3. Verify the postgrest container is gone after the cleanup. If it's
not, exit 1 instead of continuing into a guaranteed bind failure.
4. Port-picking loop now retries with a 1s sleep on each port (handles
TIME_WAIT settling), and uses a stricter ss grep that matches
127.0.0.1:<port> anchored on whitespace to avoid false positives
from substring matches (e.g. :30110 matching :3011).
Reproduces the failure mode from the run on 2026-06-06: postgrest
failed to bind 127.0.0.1:3011 with 'address already in use' even
though the script had just reported it as free.
Previous attempts hardcoded 3011, but something on tyler's host is
holding 3011 too. Instead of guessing ports, make the deploy probe
for the first free port starting from 3011 and thread that choice
through the rest of the workflow.
How it works:
1. Start Docker stack frees 3001 + the previous deploy's chosen port
(read from .postgrest-port) so the lowest free port can be picked
back up
2. Loops from 3011 upward, checking ss -tln until it finds a free
port in the 3011-30200 range
3. Writes the chosen port to .postgrest-port in the workspace
4. Sets NEXT_PUBLIC_API_URL based on that port
5. Writes POSTGREST_HOST_PORT and NEXT_PUBLIC_API_URL to the .env
that docker compose reads
6. Build and Deploy steps read .postgrest-port to set their
NEXT_PUBLIC_API_URL dynamically — no more hardcoded 3011
The dev PostgREST (started outside docker compose for local development)
and the production PostgREST (started by docker compose up) were both
trying to bind 127.0.0.1:3001. Even with fuser -k in the deploy cleanup,
something on tyler's host (likely a process supervisor restarting the
dev PostgREST) keeps reclaiming 3001 between our check and the
docker bind.
Move the production PostgREST to host port 3011, keeping the container's
internal port at 3001 (PostgREST's default). This lets dev and prod
coexist on the same host without fighting over a port.
Update NEXT_PUBLIC_API_URL to http://localhost:3011 in:
- Build step env (so the build bakes in the right URL)
- Start Docker stack .env append (so docker compose + app see the URL)
- Deploy step .env.production writing (so runtime app uses 3011)
The previous attempt's `docker compose down` removed the project's
network, but on the subsequent `docker compose up`, the postgrest
container was being recreated rather than created from scratch, and
its network attachment still pointed to the now-orphaned old network
ID. The new db/minio containers attached to the freshly-created
network fine; only postgrest failed with 'network ... not found'.
`-d --force-recreate` tells docker compose to throw away any
existing container state and build from scratch, eliminating stale
network references.
On hosts with leftover dev processes or stuck containers from prior
attempts, docker compose fails with 'address already in use' when
trying to bind PostgREST to 127.0.0.1:3001. The previous deploy
attempt actually got as far as starting Postgres and MinIO, then
PostgREST failed at port binding — leaving the Postgres container
running, which would persist for the next attempt and keep
re-occupying 3001 indirectly.
Add a defensive pre-check: if 3001 is bound, kill the holder
(fuser -k) and remove any docker container publishing 3001, then
docker compose down --remove-orphans to clear stuck state. Sleep 3
to let the kernel release the port.
Same first-deploy bootstrap pattern as .env.example: 'Start Docker stack'
needs docker-compose.yml at $APP_DIR for 'docker compose up', and
'Apply migrations' needs supabase/migrations/ at $APP_DIR for psql.
Neither was copied there until the 'Deploy' step runs at the end of
the job — so the first deploy on a fresh host fails before any
runtime file is ever placed.
Each step now seeds the file/dir it needs from the workspace before
using it. The 'Deploy' step's later copy remains as a refresh for
subsequent deploys.
The Start Docker stack step does `cd $APP_DIR && [ -f .env ] || cp .env.example .env`
but never ensured .env.example existed in APP_DIR. The rsync in the Deploy step
also doesn't copy it. The first deploy on a fresh host therefore failed with
'cp: cannot stat .env.example' before ever bringing up the stack.
Fix: copy .env.example from the workspace into APP_DIR if it isn't already
there, before the cd.
tyler
tyler