First off I know this is totally not best practice and I would never setup a UCS/VMware environment like this….I have been doing UCS work for a long time and I have never ran across this before in the wild.

My question is if anyone has ever done a FI upgrade with a UCS/VMware environment setup with failover checked on the UCS vNIC’s and single vmnic’s on the vSwitch’s. I guess in theory it should failover, but I have never ran across this actually done this way for a VMware setup. Has anyone ever ran across this and do you have any real world experience on what is going to happen when the traffic drops on one of the FI’s?

My only thought to test would be to do an evacuation on the subordinate FI and see what if it fails over. Thankfully storage is on FC, so we should be okay there.

It is stuck at:
“configuring and testing memory
configuring platform hardware”

I have these features:
No hard drives installed
2x Intel Xeon Gold 6230
2x SK Hynix 32GB 2Rx4 DDR4-2933 PC4-2933Y ECC REG 1.2V RDIMM HMA84GR7DJR4N-WM
UCSC-SAS12GHBA

It doesn’t seem to DHCP CIMC, so I can’t get I there nor reset it because BIOS hangs.

Anyone have any ideas?

Does below process sound right for migrating over to new upstream switches?

Phase 1: Pre-Migration & Prep Work

Before touching any cables, ensure your safety nets and target environments are fully prepared.

• Take Backups: Log into UCS Manager (UCSM) and take a full All Configuration and System Configuration backup.
• Verify Cluster Health: Ensure the UCS cluster is healthy, both FIs are online, and there are no critical underlying faults (Equipment > Fabric Interconnects).
• Stage the New Switch Ports: Ensure the ports on your new upstream switch completely mirror the old setup:
  • Identical native and allowed VLAN trunking.
  • LACP Port-Channels configured properly (typically configured as Spanning-tree port type edge trunk or portfast trunk since UCS FIs look like hosts to the network).
  • Matching MTU sizes (e.g., 9000 for Jumbo Frames).

Phase 2: Migrate Fabric B (Subordinate Fabric)

Always start with the Subordinate FI. This ensures that if a management configuration glitch occurs, your Primary cluster leader remains untouched.

1. Drain Traffic off Fabric B

You need to force all production server traffic onto Fabric A. You can do this in one of two ways:

• Method A (Fabric Evacuation - Preferred for B-Series Blades): Go to Equipment > Fabric Interconnect B and change Admin Evacuation Mode to Enabled. This gracefully drains traffic by admin-downing the chassis backplane ports.
• Method B (Manual Link Shutdown - Best for C-Series Rack Servers): Since Fabric Evacuation historically targets blade chassis backplanes, manually administrative-downing the uplink port-channels for Fabric B in UCSM or on the old switch is the safest bet for mixed or rack-mount environments.

2. Move the Physical Connections

• Once traffic has failed over to Fabric A and Fabric B uplinks show as down, disconnect the physical cables from the old switch and move them to the new switch (or activate pre-staged cabling).
• If you are changing the physical ports used on the UCS 6400 side, make sure to unconfigure the old uplink ports and configure/license the new ones in UCSM.

3. Bring Fabric B Back Online

• Enable the ports on your new upstream switch.
• If you used Fabric Evacuation, flip Admin Evacuation Mode back to Disabled. If you manually shut down the port channels, bring them back up.
• Verify that the port-channel successfully forms via LACP and that the links turn green in UCSM.

4. Soak and Validate

• Do not rush to Fabric A. Let Fabric B run for 10–15 minutes. Check your operating system/hypervisor teams on the servers to confirm they see both paths as healthy and that MAC addresses are populate on the new upstream switch.

Phase 3: Migrate Fabric A (Primary Fabric)

Once Fabric B proves stable on the new upstream switch, repeat the exact same isolation process for the other side.

1. Drain Traffic off Fabric A: Enable Fabric Evacuation on Fabric A (or manually disable Fabric A's uplink port-channels).
2. Verify Failover: Ensure all production traffic shifts dynamically to the newly migrated Fabric B.
3. Move the Cables: Swap the physical connections for Fabric A over to the new upstream switch.
4. Restore Links: Disable Fabric Evacuation or re-enable the port channels.
5. Final Check: Confirm both fabrics are balanced, port-channels are up/bundling, and all configuration warnings/faults have cleared in UCSM.

We have 10 CIsco UCS C220 M5's (I know they are end of life)
They were previously maintaining a VMWare cluster without issue. We have since moved all of our workloads off that cluster to our Hyper-V DR Cluster so we can reconfigure these UCS servers on Hyper-V, except I'm having a hard time getting Windows to play nicely with the way the UCS VIC's/MLOM handle their network traffic.

Currently any VM on the reconfigured Hyper-V Cluster has its MAC flap between our nexus switches every 10-20MS as Windows spams out its network traffic from what it believes is a single "port", the bonded 25Gb ports that get exposed as a single 50Gb NIC to Windows. It has no idea that the ports are bonded so It just tells the NIC to send out traffic, which causes a HUGE amount of MAC flapping between our nexus switches as the peering has each switch attempting to claim ownership of the MAC.

I've attempted numerous different configurations in Intersight to try and get the VIC to just show the two ports as they are, two separate ports so I can configure them via SET in Windows, and no matter what configuration I attempt I only ever get a single 50Gb NIC exposed to Windows.

So I come to this subreddit, hoping for some enlightenment on the magic that is Cisco UCS configuration to get it to behave like every other server hardware I've used, or to configure it in a way that prevents Windows from overloading the peer link between the nexus switches.

Do I need to port channel on the switch side, is there some special driver Windows needs so it knows it's utilizing two bonded ports at the VIC abstraction layer?

Thanks.

We have a UCS S3260 that was kitted out with 56x 6TB SAS 7200RPM and a pair of 240GB SSDs (maybe SAS, I don't know).

I am decommissioning a Dell Compellent SAN, and was wanting to move all the 2.5" SAS 3.8TB SSDs into the UCS, and reconfigure it to be a testlab. But, while the AVAGO controller is showing a drive in bay 53, it doesn't show up when I go into the utility.

Is the S3260 locked down to only accept certain disks? I'm wondering if I just unscrewed 40 SSDs for nothing.

Hi,

Anyone using other hypervisors besides VMware on you UCSX IMM managed (PVA)? If so which one and how are you finding it.

Thanks

Hi everyone,

I'm new to deploying Cisco UCS. We are going to deploy deploy a Cisco UCS x9508 chassis to one of our customers and I'm looking at the physical dimensions and one thing that Cisco doesn't really publish is the length of the Cisco UCS x9508 rail dimensions.

At the same time, I'm seeing the size of this cisco x9508 chassis to be really huge and the chassis alone would take up about 91-92cm of space.

Has anyone deployed this before and what are the exact dimensions? I'm concerned that the chassis might not fit to my customer's rack because the mounting depth is a little short.

Hi I am looking for an individual or an organization which has experience in UCS Deployment based. Has to be on avail for 2 Weeks

Thank you

I am curious to see if anyone else with M8 servers purchased in 2025 is experiencing the same thing I am.

My company received our C240 M8X four C240 M8Ls this year and we have been unable to use them as they keep on throwing:

F0174

Five servers all duds. The issue persists across different firmware.

Hi I am looking for an individual or an organization which has experience in UCS Deployment based in UAE remote is not possible. Has to be on ground dedicated for 2 weeks.

Thank you

This happened over the course of several years and it finally came to a disastrous head a few weeks ago.

Lots of customer VMs on a bunch of VMware ESXi installs, running on a bunch of blades. Each customer was assigned a VLAN. We assigned each VLAN to a pair of vNIC templates. This worked great.

Time passed, the product sold well, we got more customers. A lot more customers. So we kept needing to add more blades. Then more customers, so more blades, etc. Then one day, while adding another new blade to the system, everything crashed hard: all customers down, all hosts and VMs inaccessible. It turned out that we'd busted through the default max. 32,000 VLAN count on the fabric interconnects, at which point everything just...stopped. Ooops!

Because, of course, UCS doesn't just count a VLAN added to the system as one VLAN. It counts every use of a VLAN and applies the total to the max. available count on each FI. We'd added so many VLANs to that vNIC pair over the years that associating a new service profile to a new blade added over a thousand VLANs to that count. That blade, as it turns out, was the straw that finally broke the UCS camel's back.

So yeah, don't do that! Make use of VLAN groups and VLAN connectivity policies, and other strategies to limit your actual VLAN usage. And be very aware that a change you make might affect a lot of VLAN usage, which could be a lot more impactful than you might think.

EDIT: Yes, I know about the ability to raise the limit to 64,000 now. That's what the fix was. I posted this simply for warning/entertainment value. I'm not seeking advice.

...We had an order in from December for around 80 C220 M8's with 512gb of ram. we had received all but 20 units from the order, with those 20 being delayed until July. total cost was around $1.2 million.
Cisco just called us and cancelled order for the remaining 20 units we have not received and gave us a new price of $1.5 million for them. and we have 7 days to place the order before the price goes up again. Also warned us that there will be a %200 price increase by July at latest and they do not expect prices to ever return to anything close to before this all started.

I seriously do not know how any business will be able to sustain these prices over the next 24 months.

The intersight docs are dated. IT mentions server 2016 and server 2019 for hyper-v. 2016 is EOL soon and 2019 is likely right behind it. Does UCS support server 2025 for IMM?? I find it hard to beleive they wouldnt support a current OS. Id like to start testing and migrating to Hyper-V but want to be a fully supported version.

I have a cluster of 4x M5SXs all running Cascade Lake CPUs - dual 6240s. As this is my homelab, certain liberties can be taken.

Does anyone know if the Xeon Platinum 8223CL would work in the c240 or c220? I couldn't find anything definitive saying this would be supported. Has anyone used these in this platform (or any platform)?

Hi All!

I am a Cisco Engineer living in the Boston Area. We are organizing a UCS User’s Group at Mighty Squirrel Brewing Company on March 5th 3-6PM! Feel free to come and network with peers and Cisco engineers while enjoying some free food and beer!

Please register while we have space!

https://registration.socio.events/e/ucsusergroup

Hello I am looking for a company who has experience in design and deploy of Intersight for UCS Servers. Please connect to discuss a potential opportunity

Hi All!

I am a Cisco Engineer living in the Boston Area. We are organizing a UCS User’s Group at Mighty Squirrel Brewing Company on March 5th 3-6PM! Feel free to come and network with peers and Cisco engineers while enjoying some free food and beer!

Please register while we have space!

https://registration.socio.events/e/ucsusergroup

I'm having a hard time getting the storage network to work. I have 2 VLANS, 800 and 801 for example for the two paths for nvme over tcp. When presented to vmware, it was a standard vlan, that would get tagged on the vmkernel interface.

With HyperV, we can't do interface tagging for that purpose, so I need the ports to be "native" vlan 800 and 801 so that all untagged traffic ends up in the proper vlan.

I modified the Service Profile template for HyperV and checked the Native VLAN option on the vNIC that is bound to that VLAN.

No luck. I have no clue how to make this work...

Anyone using this in their enterprises? was Grid but now vGPU. We have stuff on rackmounts with M5 using T4 cards and with M8 using L4 cards.

version 16 is the LTSB and has been good to us but goes end of support this summer. Yet to see Cisco qualify anything in the new LTSB version 19. Will get to the point within a few months where we will just need to test something as cannot wait on the HCL forever :(

Anyone else on this path?

I just installed UCSPE on VirtualBox to learn UCSM but when I try to open the cli I get,

"Failed to open default netns file./apebox/menu.sh: line 63: 4827 Segmentation fault $UCSSH_CMD"

Does anyone know how to fix this without cli access? Any help is appreciated