Part 7: VCF Storage — vSAN OSA vs ESA Deep Dive

vSAN in VCF: The Storage Engine

vSAN is the mandatory storage platform in VMware Cloud Foundation. All VMs in a VCF domain run on vSAN datastores — there is no option to use external SAN or NAS as the primary storage in a VCF deployment. Understanding vSAN storage architectures is therefore essential for VCF capacity planning, hardware selection, and performance expectations.

VCF 9.1 supports two distinct vSAN architectures: the original OSA (Original Storage Architecture) and the newer, NVMe-only ESA (Express Storage Architecture). Choosing between them at deployment time is critical — you cannot convert between them without data migration.

vSAN OSA — Original Storage Architecture

OSA is the traditional vSAN architecture, available since vSAN 5.5. It uses a tiered design with a fast cache layer and a larger capacity layer.

OSA Disk Group Design

In OSA, storage is organized into disk groups. Each host can have 1–5 disk groups, and each disk group contains:

• 1 cache disk — High-endurance NVMe or SAS/SATA SSD. Acts as read cache (70%) and write buffer (30%). Must be sized at minimum 10% of the total capacity tier size in the disk group.
• 1–7 capacity disks — SAS SSD, SATA SSD, SAS HDD, or NVMe drives. These store the actual data.

How OSA Read/Write Works

• Writes: All writes go to the cache disk first (write buffer / journal). Periodically, data is destaged from cache to capacity disks.
• Reads: Frequently accessed data is served from the cache disk (read cache hit). Cold data is read directly from capacity disks.
• Cache disk failure: If the cache disk fails, the entire disk group is offline. All VMs with data in that disk group trigger resync. This is why high-endurance cache SSDs are critical — cache disk DWPD (Drive Writes Per Day) should be 10+ for production.

OSA Storage Policies

Storage policies control redundancy and performance per VM:

• FTT (Failures to Tolerate): Number of host failures the storage policy can tolerate. FTT=1 requires minimum 3 hosts. FTT=2 requires minimum 5 hosts. FTT=3 requires minimum 7 hosts.
• RAID-1 (Mirroring): FTT=1 RAID-1 writes 2 copies of data. FTT=2 RAID-1 writes 3 copies. Simple but consumes the most raw capacity.
• RAID-5/6 (Erasure Coding): FTT=1 RAID-5 uses 4 hosts and consumes 1.33x raw capacity (vs 2x for RAID-1 FTT=1). FTT=2 RAID-6 uses 6 hosts and consumes 1.5x raw capacity. Better capacity efficiency but higher CPU overhead.

vSAN ESA — Express Storage Architecture

ESA is the next-generation vSAN architecture introduced in vSAN 8.0. It redesigns the storage stack from the ground up to take advantage of NVMe performance characteristics.

ESA Storage Pool Design

ESA eliminates disk groups entirely and replaces them with a single storage pool per host:

• All NVMe drives on a host join a single storage pool — no artificial tiering
• No separate cache tier — ESA uses all NVMe drives for both reads and writes directly
• Minimum 4 NVMe drives per host (for ESA storage pool)
• NVMe drives must be NVMe PCIe Gen 3/4/5 or U.2 NVMe — check the VCG for specific models

How ESA Read/Write Works

• Writes: Writes go directly to the NVMe drives in the storage pool using a log-structured approach. No cache destaging cycle.
• Reads: Data is read directly from NVMe drives. No cache-miss penalty since all drives are NVMe-speed.
• Compression: ESA applies inline compression to all data before writing — enabled by default. On compressible data (text, logs, databases), this effectively multiplies capacity 2–4x with minimal CPU overhead (ESA uses hardware offload where available).
• Deduplication: ESA deduplication is per-pool and cross-VM — dramatically more efficient than OSA’s disk-group-scoped dedup.

ESA RAID-5/6 Performance Advantage

One of ESA’s most significant advantages is that RAID-5 and RAID-6 erasure coding can be applied at write time (online) rather than as a background operation. In OSA, RAID-5 data is initially written as RAID-1 and later converted — creating two write amplifications. ESA writes RAID-5 inline, eliminating this overhead and delivering RAID-5 performance parity with RAID-1.

OSA vs ESA: Feature Comparison

When to Choose OSA vs ESA

• Choose OSA if: You have existing SAS/SATA SSD hardware, budget constraints prevent all-NVMe, or you’re running less I/O intensive workloads (file servers, dev environments).
• Choose ESA if: You’re doing a new deployment, running VDI, databases (SQL, Oracle), Kubernetes, or any latency-sensitive workload; or if raw capacity efficiency via compression is a priority.

vSAN Health Monitoring in VCF

vSAN Health Service (accessible via vCenter → vSAN → Skyline Health) continuously monitors cluster health across 200+ health checks including: disk capacity thresholds, network latency between hosts, storage policy compliance, and NVMe controller firmware compatibility. In VCF, SDDC Manager also surfaces vSAN health alerts in its dashboard.

Broadcom Documentation

• vSAN ESA Concepts — TechDocs
• vSAN OSA Requirements
• vSAN Storage Policies
• vSAN Compatibility Guide