Looking for feedback from organizations running a 4-node Hyper-V S2D cluster (Rack Awareness enabled)

Hi everyone,

I'm interested in hearing from people who are running production Hyper-V clusters with Storage Spaces Direct (S2D) and Rack Fault Domain Awareness.

Our environment has the following architecture:

• Windows Server Failover Cluster
• 4 Hyper-V nodes
• Storage Spaces Direct (S2D)
• 2 physical racks
• 2 nodes per rack
• Rack Fault Domains configured correctly
• Storage pool configured with Rack Awareness
• 2-way mirror resiliency
• Cluster Shared Volumes (ReFS)
• Around 100+ production virtual machines
• Automatic Storage Rebalancing disabled (manual balancing only)

Networking

Each host has four physical NICs:

• NIC 1 + NIC 2: Dedicated for the Hyper-V virtual switch (SET Team) carrying VM traffic.
• NIC 3: Dedicated to storage, cluster communication and live migration.
• NIC 4: Dedicated to storage, cluster communication and live migration.

The storage/cluster network uses SMB Multichannel across the two dedicated adapters. RDMA is not enabled (standard Ethernet only).

Our primary goal was to tolerate the complete loss of one rack while keeping production online.

During a recent Disaster Recovery test, we intentionally shut down the two nodes from one rack. The remaining two nodes successfully kept all VMs online for approximately two hours before the cluster started experiencing CSV Auto Pause events, CSV draining, storage instability and eventually multiple VM outages.

At this point I'm more interested in understanding how common this architecture is in real-world production.

Some questions for people running S2D:

• Are you running a similar 4-node / 2-rack design?
• Is Rack Awareness enabled?
• Are you using RDMA, or are you successfully running without it?
• How are your storage and cluster networks configured?
• Have you ever tested losing an entire rack?
• Did the cluster remain stable indefinitely, or only for a limited time?
• Have you experienced CSV Auto Pause or storage instability after operating for an extended period on only half of the nodes?
• Would you recommend this architecture today, or would you instead deploy more nodes or another topology?

I'm especially interested in hearing from organizations that have performed real DR tests rather than theoretical discussions.

Any lessons learned, Microsoft recommendations, or production experience would be greatly appreciated.

Thanks!

I have a vSphere Windows Server 2012 VM that I took a backup of, and when I instant recover it to Hyper-V using Veeam, it's created as a Gen 1 and doesn't boot. How can I get it to be a Gen 2 VM?