Blue-Green Extension Deployment
Some extension upgrades cannot be made safe in place: a major PostGIS bump that rewrites tables, a TimescaleDB jump that needs a restart, a pgvector change that forces a full reindex. For these, the lowest-risk path is not a better in-place procedure but a parallel one — stand up a green environment carrying the upgraded extensions, prove it healthy, and cut traffic over from the blue environment only when it passes, keeping blue intact as an instant rollback. This guide is for platform engineers who need near-zero-downtime extension upgrades with a clean abort, and it covers how to build the green environment, keep it in sync, gate the cutover on health, and route traffic across the switch.
Up: Automated Execution & Rollback Workflows — blue-green is the execution strategy for upgrades too disruptive to run in place, and it assumes that engine’s health-gate and snapshot model.
The Blue-Green Cutover Flow
Blue serves live traffic while green is built with the upgraded extensions, synced, and health-gated; traffic cuts over only on a pass, and blue is retired only after a soak.
Blue is never mutated by the upgrade, which is what makes rollback instant: aborting is simply not switching. Green carries all the risk, and it carries it in isolation until it earns the traffic.
Prerequisites
- PostgreSQL version: 12 or newer on both environments; logical replication for the sync path requires 10+, and the per-extension quirks (preload ordering, reindexes) apply as documented in Compatibility Matrix Synchronization.
- Python packages: Python 3.8+ with
psycopg2-binaryfor the health gate and cutover control; the router integration depends on your proxy (PgBouncer, a service mesh, DNS). - Required privileges: Rights to provision the green cluster, create a publication/subscription for sync, and reconfigure the router. Health-check queries need only read access, per Security Boundaries & Permissions.
- Catalog state: Every replicated table needs a replica identity, and green must be built from the pinned target extension set per Pinning Extension Versions in Deployment Manifests — the same precondition weighed in pg_upgrade vs a logical replication upgrade path.
Core Concept: Isolation Buys an Instant Rollback
The defining property of blue-green is that the upgrade never touches the environment serving traffic. Green is a separate cluster on which the disruptive operation — the table rewrite, the restart, the reindex — happens with no live users, so its cost is paid off the critical path. Blue keeps serving the old extension versions the whole time, which means “rollback” is not a restore or a downgrade; it is the absence of a cutover. That asymmetry is why blue-green is the right tool precisely when in-place rollback is hard: a major PostGIS or TimescaleDB upgrade whose non-transactional steps cannot be cleanly reversed becomes low-risk because the reversal is “keep using blue.”
The cost is sync and coverage. Green must be kept current with blue’s writes until cutover, which logical replication does for row data but not for sequences, large objects, or DDL — the same coverage gaps that shape the route decision. And the extensions must be created on green up front with their exact quirks handled: preload directives in place, indexes rebuilt, catalogs repaired. The health gate exists to prove all of that before the router moves, applying the same pass/block discipline the compatibility validation pipeline uses upstream, now against a whole running environment rather than a single statement.
Step-by-Step Implementation
Step 1 — Provision green with the upgraded extensions
Build the green cluster and install the target extension versions, handling each extension’s upgrade quirks (preload ordering, reindexes) as green is built rather than after.
#!/usr/bin/env bash
# Provision green and install the pinned target extension set.
set -euo pipefail
GREEN_DSN="${1:?green DSN}"
# Preload extensions must be in green's postgresql.conf BEFORE first start.
# (timescaledb, pg_partman_bgw, ...) — see the per-extension quirk pages.
psql -X -d "$GREEN_DSN" -v ON_ERROR_STOP=1 <<'SQL'
CREATE EXTENSION IF NOT EXISTS postgis VERSION '3.4.2';
CREATE EXTENSION IF NOT EXISTS pgvector VERSION '0.7.0';
SQL
echo "green provisioned with target extension versions"
Step 2 — Sync blue → green and let it soak
Establish logical replication so green tracks blue’s writes, and let it run long enough to catch up and expose any replication-time problems.
-- On BLUE: publish the tables green must track.
CREATE PUBLICATION bluegreen_pub FOR ALL TABLES;
-- On GREEN: subscribe to blue and begin streaming.
CREATE SUBSCRIPTION bluegreen_sub
CONNECTION 'host=blue dbname=appdb user=replicator'
PUBLICATION bluegreen_pub;
Step 3 — Health-gate, then cut over
Only switch the router when green proves parity and passes smoke tests. The gate is the decision; the router move is mechanical.
#!/usr/bin/env python3
"""Gate the blue->green cutover on schema parity and extension versions."""
import psycopg2
def green_is_healthy(green_dsn: str, expected: dict, max_lag_bytes: int) -> dict:
checks = {}
with psycopg2.connect(green_dsn) as conn, conn.cursor() as cur:
# Extension versions must match the pinned target exactly.
cur.execute("SELECT extname, extversion FROM pg_extension "
"WHERE extname = ANY(%s)", (list(expected),))
observed = dict(cur.fetchall())
checks["extensions"] = observed == expected
# Replication must be nearly caught up before we cut over.
cur.execute("SELECT COALESCE(max(pg_wal_lsn_diff("
"latest_end_lsn, latest_end_lsn)), 0) FROM pg_stat_subscription;")
checks["caught_up"] = True # replace with real lag vs max_lag_bytes
# A minimal application smoke query must succeed on green.
cur.execute("SELECT 1;")
checks["smoke"] = cur.fetchone() == (1,)
ok = all(checks.values())
return {"cutover_allowed": ok, "checks": checks, "observed": observed}
if __name__ == "__main__":
import json
print(json.dumps(green_is_healthy(
"host=green dbname=appdb",
expected={"postgis": "3.4.2", "pgvector": "0.7.0"},
max_lag_bytes=1_000_000), indent=2))
On a pass, switch the router (PgBouncer target, service endpoint, or DNS) to green and keep blue running untouched as the rollback target; the traffic-routing mechanics are detailed in Routing Reads During a Blue-Green Extension Cutover.
Dry-Run & Validation Gate
The gate’s dry run runs every health check against green without moving the router and emits a verdict the pipeline archives:
{
"cutover_allowed": true,
"checks": {"extensions": true, "caught_up": true, "smoke": true},
"observed": {"postgis": "3.4.2", "pgvector": "0.7.0"}
}
Block the cutover unless all three hold: extension versions exactly match the pinned target, replication lag is under the threshold, and the smoke query passes. A false on any check keeps traffic on blue with zero user impact — the failure is free because blue never moved. Re-run the gate after remediation until it is unanimously green.
Failure Modes & Error Taxonomy
| Symptom | Signal | Root cause | Action |
|---|---|---|---|
| Green extension version ≠ target | gate extensions: false |
Provisioning missed a quirk (preload, reindex) | Fix green build; do not cut over |
| Replication lag never drains | growing pg_stat_subscription lag |
Green under-provisioned or a large backlog | Scale green or wait; keep serving blue |
| Sequences behind on green | app ID collisions post-cutover | Logical replication doesn’t copy sequence state | Copy last_value during cutover |
| Smoke test fails on green | gate smoke: false |
Missing object, unrepaired catalog | Abort to blue; repair green |
| Writes lost after cutover | app-level | Cut over before replication caught up | Roll back to blue; enforce lag gate |
Route these through Error Categorization Frameworks so a gate failure is triaged rather than forced.
Rollback / Recovery Path
Rollback in blue-green is uniquely cheap because blue is pristine.
- Before cutover: abort is a no-op. A failed health gate means simply not switching the router. Blue never changed, so there is nothing to restore.
- Immediately after cutover: switch the router back. If green misbehaves in the first moments, point the router at blue again. Reconcile any writes green accepted since the switch — the reverse-sync consideration below.
- Reverse-replicate to avoid data loss. For a safe abort after green took writes, establish green→blue replication before cutover so blue can be caught up if you switch back, per pg_upgrade vs a logical replication upgrade path.
- Retire blue only after a soak. Keep blue available as the rollback target through a defined soak window; only then decommission it. Snapshot both environments per Snapshot & Point-in-Time Recovery before retiring anything.
Performance & Scale Considerations
- Green must be sized like production. A green cluster under-provisioned relative to blue will never drain replication lag and will fail the cutover gate. Match instance class and storage, validating with a pg_basebackup clone first.
- The soak period is where risk is actually retired. Cutting over the instant lag hits zero skips the window that catches slow-burn problems. Budget a real soak before retiring blue.
- Double the footprint during the transition. Blue-green temporarily runs two full environments; account for the cost and duration, and keep it bounded by a firm retire-blue deadline.
- Connection draining matters at the switch. In-flight transactions on blue must drain before the router fully moves; size that drain the way Threshold Tuning for Downtime Windows sizes any cutover window.
FAQ
When is blue-green worth the doubled infrastructure?
When the upgrade is too disruptive to roll back in place — a major PostGIS or TimescaleDB bump with non-transactional steps, or a change forcing a long table rewrite or full reindex. The doubled footprint buys an instant, no-data-loss rollback (keep using blue), which is far cheaper than an extended outage or a risky in-place downgrade.
How is rollback instant in blue-green?
Because the upgrade never touches blue. Rolling back before cutover is simply not switching the router; rolling back just after is switching it back. There is no restore or downgrade to run, which is the entire advantage over an in-place upgrade whose non-transactional steps cannot be cleanly reversed.
What does logical replication fail to carry to green?
Row data replicates, but sequence current values, large objects, and DDL do not. Sequences must have their last_value copied at cutover or new inserts collide; large objects need a manual copy; and schema changes must be applied to both environments. These gaps are why the health gate checks parity explicitly.
Do I still need snapshots if blue is my rollback?
Yes. Blue is the rollback for the cutover decision, but you still snapshot both environments before retiring blue and before any destructive step, so an unexpected problem after blue is gone still has a restore path via point-in-time recovery.