Infrastructure

Proxmox VE for production: from homelab to real infrastructure.


Proxmox VE is an excellent hypervisor platform โ€” mature, open-source, actively developed, and based on Debian with KVM and LXC. It's also something of a rite of passage: many self-hosters run it as a homelab, then discover that the configuration choices that are fine for a homelab will quietly ruin a production deployment in ways that only manifest at the worst moments.

This post is for people making the jump. It covers the specific decisions that differ between a homelab Proxmox setup and one you'd stake a production workload on: storage, networking, backup, and the HA cluster configuration that most guides skip over or get wrong.

Storage: the most consequential decision

Proxmox supports a range of storage backends. The choice you make here affects performance, reliability, snapshot behaviour, and how you'll recover from hardware failure. The three backends you'll actually use in production:

ZFS (local)

Best default for single-node

ZFS gives you checksumming (detects silent data corruption), copy-on-write snapshots, built-in RAID (RAIDZ), and compression. Proxmox has first-class ZFS support โ€” you can snapshot and roll back VMs from the UI. ARC (Adaptive Replacement Cache) needs RAM: budget 1GB RAM per TB of usable storage as a starting point.

โœ“ Checksumming โœ“ Snapshots โœ“ Compression โœ“ Self-healing
โœ— RAM-hungry โœ— Not NFS/shared by default
LVM-thin

Simpler, less RAM

LVM thin provisioning is fast and works well with SSDs. No checksumming, no compression, but also no ZFS ARC overhead. Snapshots work but are less reliable under write load than ZFS. Good choice when RAM is constrained or when you're running on hardware where ZFS isn't practical.

โœ“ Low overhead โœ“ Fast โœ“ Snapshots work
โœ— No checksumming โœ— Snapshot reliability under load
Ceph (shared)

Required for live migration

Ceph provides a distributed storage pool across multiple Proxmox nodes. Live VM migration (moving a running VM to another node with no downtime) requires shared storage โ€” Ceph is the native option. It requires a minimum of three nodes, a dedicated 10GbE network for replication, and adds significant operational complexity. Don't use Ceph unless you need live migration or multi-node storage.

โœ“ Live migration โœ“ Redundancy โœ“ Scales horizontally
โœ— Needs 3+ nodes โœ— Dedicated network required โœ— Complex
NFS / SMB (external)

Backups and ISO storage

External NFS is the right answer for backup storage (pointing Proxmox Backup Server or vzdump at a NAS) and for storing ISOs and templates. Don't run production VM disks on NFS unless you have a very well-configured NAS with 10GbE and jumbo frames โ€” the latency and reliability characteristics at scale are hard to manage.

โœ“ Cheap โœ“ Simple โœ“ Good for backups/ISOs
โœ— Latency for VM disks โœ— Single point of failure

The homelab mistake: Using a single ZFS pool with spindles for both OS/VM disks and as backup target. In production, VM disks and backup storage should be on separate hardware. When the pool fills, or when a resilver (RAIDZ rebuild) is running at full speed, VM disk I/O suffers badly. Keep them separate.

Network configuration: the 3am failure mode

Proxmox nodes communicate cluster state over Corosync, a heartbeat protocol that's remarkably unforgiving about network reliability. The two most common production failures we see:

Split-brain from a single management NIC

Default Proxmox installation puts management traffic, VM traffic, and Corosync all on the same interface. A brief network hiccup โ€” a switch restart, a cable problem, a NIC driver update โ€” can cause a cluster node to lose heartbeat and trigger an unexpected quorum decision. In a 2-node cluster, this means one node gets fenced (STONITH) and its VMs restart on the other node. Not catastrophic, but not planned downtime either.

For production, Corosync should have a dedicated network link. The standard configuration is:

# /etc/network/interfaces (on each node)
# Management / API interface
auto eno1
iface eno1 inet static
  address 10.0.1.11/24
  gateway 10.0.1.1

# VM traffic (Linux bridge)
auto vmbr0
iface vmbr0 inet manual
  bridge-ports eno2
  bridge-stp off
  bridge-fd 0

# Corosync / cluster link (dedicated NIC or VLAN)
auto eno3
iface eno3 inet static
  address 10.0.10.11/24
  # No gateway - this is a private cluster link

# In /etc/pve/corosync.conf:
# link0 { bindnetaddr: 10.0.10.11 }

MTU mismatch taking down storage

If you're using jumbo frames (MTU 9000) for Ceph or NFS, every switch in the path must be configured for jumbo frames. A single switch with default MTU 1500 in the path will cause intermittent storage failures that look like drive errors, Ceph OSD timeouts, or mysterious VM pauses. Test with ping -s 8972 -M do 10.0.20.1 (8972 = 9000 - 28 byte IP+ICMP header) from each node to verify the full path supports your MTU before deploying workloads.

Proxmox Backup Server: do not skip this

vzdump โ€” Proxmox's built-in backup tool โ€” works and is fine for ad-hoc backups. For production, Proxmox Backup Server (PBS) is a separate appliance you should run. PBS adds:

  • Deduplication: Changed-block backups using a content-addressed store. A 100GB VM with a 5GB daily change produces roughly 5GB of new backup data, not 100GB. This makes daily backups practical even with large VMs.
  • Verification: PBS verifies backup integrity regularly โ€” not just the backup run, but the ongoing store. You'll know if backup data is corrupt before you need to restore.
  • Encryption: Client-side encryption before data leaves the Proxmox node. The PBS server never sees plaintext โ€” useful if PBS is on separate infrastructure or collocated with untrusted parties.
  • Restore granularity: You can restore individual files from a VM backup without restoring the full disk. This matters when someone deletes a single database file and needs it back in the next 30 minutes.

Run PBS on a separate machine โ€” even a low-spec one works, since PBS is mostly I/O bound. A Raspberry Pi 4 with a large USB drive is not the right answer; a small x86 machine with a hardware RAID card and 4+ drives is. The PBS node should be on a different physical failure domain than your Proxmox cluster nodes โ€” different power circuit at minimum, different rack if you have it.

High availability: what the GUI doesn't tell you

Proxmox HA requires a quorum mechanism and a way to fence (forcibly shut down) nodes. Without fencing, a node that appears dead but isn't could restart VMs that the cluster has already restarted on another node โ€” you'd have the same VM running twice, potentially corrupting data.

STONITH (Shoot The Other Node In The Head) is the fencing mechanism. Proxmox supports several STONITH agents: IPMI/iDRAC/iLO (hardware management interfaces), power distribution units with managed outlets, and hosted environments via API. For production on physical hardware:

  • Every server must have an IPMI/BMC interface configured and reachable from the other cluster nodes
  • The IPMI network should be isolated from the main management network (different VLAN)
  • Test fencing manually before you need it โ€” pvecm status and then force a node offline and verify the fence fires

The 2-node cluster trap: Proxmox recommends 3+ nodes for HA because quorum requires a majority. A 2-node cluster cannot achieve quorum after one node fails โ€” both nodes have exactly half the votes. The solution is a third corosync member called a QDevice โ€” a lightweight process running on a separate machine (even a Raspberry Pi or a cloud VM) that provides the third vote for quorum without hosting any VMs.

What this looks like in practice

A minimal production-grade Proxmox deployment from 47Network Studio typically looks like this: two physical nodes (minimum), each with local ZFS on NVMe for VM disks, Proxmox Backup Server on a third machine with spinning RAIDZ for backup storage, a QDevice on a fourth lightweight machine for quorum, and a dedicated 1GbE link for Corosync heartbeat separate from management and VM traffic. Ceph is added only when live migration is a genuine requirement โ€” it adds real operational overhead that's worth it only when the benefit is clear.

This setup handles a single node failure gracefully: VMs restart on the surviving node, backups remain intact and verifiable, and the QDevice ensures quorum is maintained throughout. It's not a hyperscaler setup, but it's reliable production infrastructure at a size that's actually manageable by a small team.


โ† Back to Blog Hardware Services โ†’