How to configure automatic failover between two Debian VPS with PostgreSQL replication and avoid split‑brain

By Kevin Marcondes · Published on Oct 28, 2026 · 8 min read

Want to learn how to configure automatic failover between two Debian VPS with PostgreSQL replication and avoid split‑brain? The real challenge is maintaining quorum for promotion, reliable checks, and a stable entry point (VIP/DNS). Here’s a practical architecture with a witness, critical replication parameters, healthcheck scripts, and an incident playbook to lower risk and recovery time.

The real problem: two VPS, network partitions and split‑brain risk

With only two nodes, any loss of connectivity can make each side think it’s the primary. Without quorum, split‑brain can occur and lead to logical data corruption.

To have reliable automatic failover, combine failure detection, coordination with consensus (or a witness) and a mechanism to move the traffic entry point (VIP/DNS). Without these, automation becomes a gamble.

• Detection: healthchecks that validate primary status and write capability.
• Coordination: witness/DCS to break ties for promotion decisions.
• Traffic switch: VIP via Keepalived/VRRP or DNS with low TTL.

Recommended safe architecture (2 nodes + witness + VIP)

Use two Debian VPS (primary and standby) and a lightweight witness or a consensus service (DCS) — e.g., etcd/Consul when used by a HA tool. The VIP lives on the healthy primary and follows failover.

Create a private tunnel between the VPS for replication and health checks. This reduces exposure and helps latency. If needed, follow this site‑to‑site WireGuard walkthrough: how to set up WireGuard site-to-site.

Implement logical fencing: if the primary loses contact with the witness, it must relinquish the VIP and refuse writes. This guardrail drastically reduces split‑brain risk.

• Primary: accepts writes and advertises the VIP.
• Standby: replicates and can be promoted automatically.
• Witness/DCS: provides quorum for promotion decisions.
• VIP via Keepalived/VRRP: stable traffic entry point.
• Private channel: WireGuard tunnel between the VPS.

Replication: choices and essential PostgreSQL settings

Native streaming replication solves most cases. Choose synchronous (RPO≈0, higher write latency) or asynchronous (lower latency, small risk of loss on failover).

Tune parameters for stability and predictability during failover and rejoin. The more explicit your WAL and checkpoint limits, the fewer surprises during incidents.

Synchronize credentials and configs in a controlled way. Use migrations and a rollback plan to avoid divergence between nodes during structural changes.

• synchronous_commit and synchronous_standby_names (quorum or single sync).
• wal_keep_size and replication slots to avoid drops.
• primary_conninfo with SSL/TLS on the replication link.
• hot_standby=on to allow reads on the standby.
• checkpoint_timeout and max_wal_size aligned with your write volume.

Orchestration tools: compare before you choose

Automating failover requires reliable orchestration. Evaluate priorities: simplicity, features, VIP/DNS integration and operational learning curve. The right choice depends on your team and environment.

Safe VIP healthcheck: advertise only with quorum

A fragile healthcheck paves the way to split‑brain. The script below exits successfully only if the node is primary and can talk to the witness/DCS. In Keepalived, use it as a vrrp_script. Never advertise the VIP without confirming quorum.

Adjust PG_URI, WITNESS_HOST/PORT and the required number of synchronous standbys according to your RPO policy and topology.

#!/usr/bin/env bash
set -euo pipefail

PG_URI="${PG_URI:-postgresql://[email protected]/postgres}"
WITNESS_HOST="${WITNESS_HOST:-10.0.0.10}"
WITNESS_PORT="${WITNESS_PORT:-2380}"  # etcd (use 8300 p/ Consul)
REQUIRED_SYNC="${REQUIRED_SYNC:-0}"

# 1) É primário e gravável?
if ! psql "$PG_URI" -tAc "SELECT NOT pg_is_in_recovery();" | grep -qx "t"; then
  exit 1
fi

# 2) Consegue falar com o witness/DCS? Evita VIP em partição isolada
if ! timeout 1 bash -c ">/dev/tcp/$WITNESS_HOST/$WITNESS_PORT"; then
  exit 1
fi

# 3) (Opcional) Garante standby(s) síncrono(s) prontos se exigir RPO≈0
if [ "$REQUIRED_SYNC" -gt 0 ]; then
  READY=$(psql "$PG_URI" -tAc "SELECT count(*) FROM pg_stat_replication WHERE sync_state IN ('sync','quorum');")
  if [ "$READY" -lt "$REQUIRED_SYNC" ]; then
    exit 1
  fi
fi

exit 0

Incident playbook: safe promotion, return and rejoin

Define a short, tested runbook. When the primary fails, the orchestration promotes the standby and the VIP moves. If there’s suspected partition, force the old primary to refuse writes and release the VIP. On recovery, rejoin it as a replica following the correct timeline.

Keep consistent backups to support PITR. See practical approaches for incremental encrypted S3 backups and PITR without downtime: incremental encrypted S3 backups and PITR without downtime.

• Failure detected: VIP moves and the standby is promoted automatically.
• Partition: old primary without VIP and refusing writes until quorum is confirmed.
• Return: rejoin the recovered node as a standby on the winning timeline.
• Schema changes: perform via migrations with a maintenance window and rollback.

DR tests: validate failover without interrupting business

Test in a controlled window. Validate VIP switchover, write continuity and consistency after simulated switchover/failover. Take a snapshot/backup before the exercise. Audit state before and after and record lessons in the runbook.

Harden the host posture to reduce false positives and collateral failures. See the VPS security checklist for guidance: VPS Linux security checklist.

• Create/update a test table before the switchover.
• Trigger switchover/failover using the chosen orchestration tool.
• Confirm the old primary became a standby and data persisted.

# No primário atual
psql -tAc "SELECT pg_is_in_recovery();"
psql -c "CREATE TABLE IF NOT EXISTS ha_test(id int primary key); INSERT INTO ha_test VALUES (1) ON CONFLICT DO NOTHING;"

# Execute o switchover/failover pela orquestração

# No novo primário
psql -tAc "SELECT pg_is_in_recovery(); SELECT count(*) FROM ha_test;"

Anti split‑brain checklist and failover timings

Focus on timings and promotion conditions. The checklist below helps reduce risks in a two‑node plus witness setup. Review it periodically based on incidents and load growth. Short timeouts cause unintended failovers; long timeouts delay recovery.

• VIP healthcheck passes only if: actual primary + witness reachable.
• Coherent timeouts: detection (3–10s), promotion (10–30s), VIP advertisement (≥ detection).
• If using synchronous replication: ensure at least 1 standby is ready (quorum).
• Avoid restart loops; use exponential backoff in services.
• Monitor replication lag and WAL replay on the standby.
• Document rejoin steps and cleanup for old timelines.

Frequently asked questions

Do I need a third node to avoid split‑brain?

In two‑VPS scenarios, a third element (witness or DCS) is the safest way to avoid split‑brain. It provides quorum for promotion decisions. Without quorum, network partitions can create two primaries and corrupt data.

Can I achieve RPO≈0 with only two VPS?

Yes, by using synchronous replication with one standby. This reduces the risk of data loss but increases write latency and sensitivity to network issues. Ensure robust healthchecks and an accessible witness before advertising the VIP, otherwise downtime may increase.

Which is better: Patroni, repmgr or scripts with Keepalived?

It depends on priorities. Patroni provides mature automation with a DCS. repmgr is lighter and works well with a witness. Keepalived with scripts gives fine control but requires building anti split‑brain guardrails. Consider team experience and environment complexity.

How do I safely test failover?

Take a snapshot/backup, create a test table and perform a scheduled switchover. Verify the VIP movement, check pg_is_in_recovery on both nodes and confirm data persistence. Monitor replication lag and logs. Update your runbook after the exercise.

What to do when the old primary returns after failover?

Do not promote it automatically. Remove its VIP and write access, confirm the winning timeline and rejoin it as a standby. HA tools offer safe rejoin commands; avoid manual repointing without validating timelines to prevent divergence or data loss.

See also

• Intermittent 502 Bad Gateway in Docker Compose with Nginx & PHP‑FPM: how to diagnose and fix timeouts and connection overloads
• How to diagnose and capture packets for intermittent network issues between Docker containers on Debian
• How to detect and fix duplicate SKU inconsistencies between WooCommerce and your ERP without losing orders
• How to reduce build time and avoid duplicate artifacts in CI for large PHP monoliths