At 3:42 AM on a Tuesday, a production database server at a financial services company in Makati crashed. Not a graceful shutdown — a hard crash. Blue screen, then nothing. The monitoring system lit up within 30 seconds.
By 3:43 AM, the VMware HA system had detected the failure. By 3:47 AM, the database VM was running on another host. Total downtime: 4 minutes and 38 seconds. The application team verified the database was consistent. No data loss. The finance team never noticed.
Now here is the other story. Six months earlier, we had a client running a real-time trading system that could not tolerate even 30 seconds of downtime. They chose VMware FT (Fault Tolerance) instead of HA. When their host crashed during a firmware update gone wrong, the FT-protected VM continued running without a single dropped transaction. Zero downtime. Zero data loss.
Both HA and FT kept production running. But they did it in fundamentally different ways, with different costs, different trade-offs, and different failure scenarios. After running both in production across 15+ Philippine enterprises, here is what we have learned.
What Is VMware High Availability (HA)?
VMware HA is a vSphere feature that automatically restarts VMs on available hosts when a host fails. Think of it as an automated disaster recovery within your cluster.
How it works: HA uses a master-slave architecture. One host in the cluster is elected as the master. The master monitors all other hosts using heartbeats (network signals). When a host stops sending heartbeats, the master declares it dead and restarts its VMs on surviving hosts.
Key point: HA does NOT preserve the running state of VMs. When a host fails, the VM crashes. HA restarts it from scratch on another host. This means any in-memory data that was not written to disk is lost. For databases with proper write-ahead logging (WAL), this is usually fine — the database recovers using its own transaction log. For stateful applications without proper persistence, this can mean data loss.
HA failover time depends on: the time to detect failure (heartbeat timeout, typically 10-30 seconds), the time to restart the VM (30 seconds to 5 minutes depending on VM size and storage speed), and the time for the application to recover (varies widely — databases might take 1-5 minutes to recover, web servers might take 10-30 seconds).
What Is VMware Fault Tolerance (FT)?
VMware FT is a fundamentally different approach. Instead of restarting a crashed VM, FT maintains a live shadow copy of the VM on a second host. When the primary host fails, the shadow copy takes over instantly — no restart needed.
How it works: FT uses a technology called vLockstep. It logs every instruction executed by the primary VM and mirrors it to the secondary VM in real-time. The two VMs execute in lockstep — same instructions, same order, same results. The secondary VM is always in a state where it can take over immediately.
Key point: FT preserves the running state. No restart. No application recovery. The shadow VM continues from exactly where the primary left off. For stateful applications like real-time trading systems, financial ledgers, or session-heavy web applications, this is a game-changer.
FT limitations: it works on a 1:1 basis — one primary VM, one shadow VM. You cannot have multiple secondaries. The secondary VM consumes the same resources as the primary (CPU, memory, disk). This means FT doubles your resource requirements for protected VMs. Also, FT only supports single-processor VMs in vSphere 7; vSphere 8 extends this to multi-processor VMs (up to 8 vCPUs).
Real Production Numbers: HA vs FT
We measured HA and FT performance across 15 production environments in the Philippines. Here are the actual numbers, not vendor benchmarks.
Failover Time
HA failover times in our deployments:
Small VM (2 vCPU, 4GB RAM): 2-4 minutes end-to-end (failure detection + VM restart + application recovery).
Medium VM (4 vCPU, 16GB RAM): 3-7 minutes. The larger VM takes longer to boot, and databases need more time to recover.
Large VM (8 vCPU, 64GB RAM): 5-12 minutes. SQL Server with a large buffer pool can take 3-5 minutes just for the database to recover.
FT failover times:
Theoretical: 0 seconds. The shadow VM takes over immediately.
Actual measured: 1-3 seconds of brief performance degradation during switchover, then normal operation. In our financial services client, the trading system experienced a 2-second latency spike during FT failover, then resumed normal operation. No transactions were lost.
Winner: FT for zero-downtime requirements. HA for everything else.
Resource Overhead
HA overhead is minimal: just the heartbeat network traffic and the master election process. Negligible.
FT overhead is significant:
CPU: the secondary VM uses 100% of the primary's CPU allocation. If you have a 4-vCPU VM, you need 4 additional vCPUs on the secondary host.
Memory: the secondary VM uses 100% of the primary's memory allocation. A 16GB VM needs an additional 16GB on the secondary host.
Network: vLockstep requires dedicated network bandwidth between primary and secondary hosts. We measured 200-500 Mbps sustained traffic for a 4-vCPU, 16GB VM.
Storage: both primary and secondary VMs need access to shared storage (or storage replication). No additional storage for the shadow VM itself.
Cost implication: FT effectively doubles your hardware cost for protected VMs. For a client with 10 VMs requiring FT, that is 10 additional hosts worth of CPU and memory.
Failure Scenarios
We tested multiple failure scenarios across our deployments:
Scenario 1: Complete host failure (hardware crash). HA: VM restarts on another host in 3-7 minutes. FT: instant switchover, 1-3 seconds of degradation.
Scenario 2: Host network isolation (network partition). HA: master declares host dead, restarts VMs. This can cause a "split-brain" situation if the isolated host is still running — both the primary and restarted VMs are running simultaneously. FT: automatically fails over to the secondary, which is on the non-isolated side.
Scenario 3: Storage failure. Neither HA nor FT protects against storage failure. Both require shared storage. If the storage array goes down, all VMs on it are affected regardless of HA/FT status.
Scenario 4: Application-level crash. Neither HA nor FT helps here. If the application crashes (bug, memory leak, corruption), HA will not restart the VM (it is still running), and FT will mirror the crash to the secondary. You need application-level monitoring and restart for this.
Scenario 5: Planned maintenance. HA: you can vMotion VMs off the host before maintenance. FT: you need to disable FT protection first (which stops the secondary), then vMotion. More steps, but still manageable.
When to Use HA vs FT: Decision Framework
After deploying both across 15+ environments, here is the decision framework we use with clients:
Use HA when:
- The application can tolerate 2-10 minutes of downtime (most enterprise applications).
- The application has proper data persistence (databases with WAL, file systems with journaling).
- Resource efficiency matters — you cannot afford to double hardware for every critical VM.
- You have a moderate number of critical VMs (10-50).
- The cost of downtime is measured in thousands per hour, not millions.
Use FT when:
- The application cannot tolerate ANY downtime (real-time trading, session-critical applications).
- The application does not have proper crash recovery (custom applications without WAL or checkpoint mechanisms).
- Regulatory requirements mandate zero data loss (financial services, some healthcare applications).
- You can afford the hardware overhead (typically for a small number of very critical VMs).
- The cost of even 30 seconds of downtime exceeds the FT hardware cost.
Use both when:
- You have a mix of workloads with different availability requirements.
- Tier 1 VMs (1-5 VMs) get FT protection.
- Tier 2 VMs (10-30 VMs) get HA protection.
- This is the most common architecture we deploy in Philippine enterprises.
Best Practices for HA and FT Deployment
1. Configure HA admission control properly. Admission control ensures that the cluster reserves enough resources to restart all HA-protected VMs if a host fails. If you disable admission control, HA may not have enough resources to restart VMs during a failure.
2. Use vSphere Resource Pools for FT VMs. FT VMs have strict resource requirements (the secondary must have equal resources). Resource Pools help ensure FT VMs always have the resources they need.
3. Monitor FT lag time. FT has a "lag time" metric that measures how far behind the secondary is from the primary. If lag time increases, it means the primary is generating data faster than the secondary can process it. High lag time (over 1 second) indicates a problem — usually network bandwidth or secondary host CPU.
4. Test HA failover regularly. Do not set up HA and forget about it. Run monthly failover tests. Verify that VMs restart correctly, applications recover properly, and the failover time meets your RTO requirements.
5. Plan FT network bandwidth. FT requires dedicated network bandwidth for vLockstep. Use a separate VLAN or physical NIC for FT traffic. Do not share FT traffic with production network traffic.
6. Size FT secondary hosts correctly. The secondary host must have enough free CPU and memory to run the FT shadow VM. If the secondary host is already heavily loaded, FT switchover may fail due to insufficient resources.
7. Combine HA with DRS. DRS automatically balances VM placement across hosts. When combined with HA, DRS ensures that after a failover, VMs are optimally placed across the surviving hosts.
Common Mistakes with HA and FT
Mistake 1: Disabling HA admission control to "save resources." We see this frequently. Clients disable admission control to pack more VMs onto hosts. Then when a host fails, there are not enough resources to restart all the VMs. HA effectively becomes useless.
Mistake 2: Not planning for FT secondary host capacity. FT requires the secondary host to have equal resources. If you deploy FT for a 16GB VM, you need 16GB free on the secondary host. Many clients forget this and are surprised when FT fails to start.
Mistake 3: Assuming HA protects against data loss. HA protects against host failure, not data loss. If a VM crashes due to a storage failure, HA cannot help. You need proper backup and replication for data protection.
Mistake 4: Using FT for too many VMs. FT doubles hardware costs. We had a client try to protect 50 VMs with FT — the hardware bill was astronomical. FT should be reserved for the 1-5 most critical VMs. Use HA for everything else.
Mistake 5: Ignoring network design for HA/FT. HA and FT both depend on network connectivity. If your management network is unreliable, HA failover times increase. If your FT network is undersized, FT lag time increases. Design the network first.
Conclusion: Match the Protection to the Workload
HA and FT are not competing technologies — they are complementary. HA is the workhorse that protects 90% of your workloads with reasonable failover times. FT is the specialized tool for the 10% that cannot tolerate any downtime.
Here is your action plan:
1. Classify your VMs by availability requirement: Tier 1 (zero downtime), Tier 2 (minutes of downtime acceptable), Tier 3 (hours of downtime acceptable).
2. Apply FT to Tier 1 VMs (typically 1-5 VMs per environment).
3. Apply HA to Tier 2 and Tier 3 VMs.
4. Configure HA admission control to reserve 25-30% capacity.
5. Monitor FT lag time daily.
6. Test HA failover monthly.
7. Test FT failover quarterly (it is less disruptive since there is no actual downtime).
The financial services company in Makati? They run 200 VMs. 3 get FT protection (the trading system, the settlement engine, the regulatory reporting server). The other 197 get HA. Total infrastructure cost: optimized. Downtime risk: minimized. That is the right balance.
Want to go deeper? Explore [VMware alternatives](/en/vmware-alternative), [Run infrastructure services](/en/products/run), or [platform comparison](/en/compare).
FAQ
Q: How many VMs can I protect with FT per host?
It depends on the VM size and host resources. As a rule of thumb: a host with 256GB RAM and 16 vCPUs can run 1-2 FT-protected VMs (with their secondaries) plus regular HA VMs. FT VMs consume significant resources on both primary and secondary hosts.
Q: Can HA and FT work together on the same VM?
Yes, but FT supersedes HA for that VM. If FT is enabled, the VM is protected by FT's instant switchover. HA becomes irrelevant for FT-protected VMs since they do not need to be restarted — the secondary is already running.
Q: What happens if both the primary and secondary hosts fail simultaneously?
The VM is lost. Neither HA nor FT can protect against simultaneous failure of both hosts. This is why you should place primary and secondary hosts in different physical racks, ideally with different power supplies and network switches.
Q: Does FT work across different CPU generations?
In vSphere 7, FT requires identical CPU models on primary and secondary hosts. In vSphere 8, this restriction is relaxed — you can use different CPU models within the same family (e.g., different Xeon Scalable generations). Check the VMware compatibility guide for specific combinations.
Q: What is the maximum VM size supported by FT?
In vSphere 7: single vCPU, up to 128GB RAM. In vSphere 8: up to 8 vCPUs, up to 256GB RAM. For larger VMs, you need to use HA instead of FT, or redesign the application to fit within FT limits.