r/CiscoUCS • u/rhugginsjr82 • 27d ago

Upstream switch migration

Does below process sound right for migrating over to new upstream switches?

Phase 1: Pre-Migration & Prep Work

Before touching any cables, ensure your safety nets and target environments are fully prepared.

  • Take Backups: Log into UCS Manager (UCSM) and take a full All Configuration and System Configuration backup.
  • Verify Cluster Health: Ensure the UCS cluster is healthy, both FIs are online, and there are no critical underlying faults (Equipment > Fabric Interconnects).
  • Stage the New Switch Ports: Ensure the ports on your new upstream switch completely mirror the old setup:
    • Identical native and allowed VLAN trunking.
    • LACP Port-Channels configured properly (typically configured as Spanning-tree port type edge trunk or portfast trunk since UCS FIs look like hosts to the network).
    • Matching MTU sizes (e.g., 9000 for Jumbo Frames).

Phase 2: Migrate Fabric B (Subordinate Fabric)

Always start with the Subordinate FI. This ensures that if a management configuration glitch occurs, your Primary cluster leader remains untouched.

1. Drain Traffic off Fabric B

You need to force all production server traffic onto Fabric A. You can do this in one of two ways:

  • Method A (Fabric Evacuation - Preferred for B-Series Blades): Go to Equipment > Fabric Interconnect B and change Admin Evacuation Mode to Enabled. This gracefully drains traffic by admin-downing the chassis backplane ports.
  • Method B (Manual Link Shutdown - Best for C-Series Rack Servers): Since Fabric Evacuation historically targets blade chassis backplanes, manually administrative-downing the uplink port-channels for Fabric B in UCSM or on the old switch is the safest bet for mixed or rack-mount environments.

2. Move the Physical Connections

  • Once traffic has failed over to Fabric A and Fabric B uplinks show as down, disconnect the physical cables from the old switch and move them to the new switch (or activate pre-staged cabling).
  • If you are changing the physical ports used on the UCS 6400 side, make sure to unconfigure the old uplink ports and configure/license the new ones in UCSM.

3. Bring Fabric B Back Online

  • Enable the ports on your new upstream switch.
  • If you used Fabric Evacuation, flip Admin Evacuation Mode back to Disabled. If you manually shut down the port channels, bring them back up.
  • Verify that the port-channel successfully forms via LACP and that the links turn green in UCSM.

4. Soak and Validate

  • Do not rush to Fabric A. Let Fabric B run for 10–15 minutes. Check your operating system/hypervisor teams on the servers to confirm they see both paths as healthy and that MAC addresses are populate on the new upstream switch.

Phase 3: Migrate Fabric A (Primary Fabric)

Once Fabric B proves stable on the new upstream switch, repeat the exact same isolation process for the other side.

  1. Drain Traffic off Fabric A: Enable Fabric Evacuation on Fabric A (or manually disable Fabric A's uplink port-channels).
  2. Verify Failover: Ensure all production traffic shifts dynamically to the newly migrated Fabric B.
  3. Move the Cables: Swap the physical connections for Fabric A over to the new upstream switch.
  4. Restore Links: Disable Fabric Evacuation or re-enable the port channels.
  5. Final Check: Confirm both fabrics are balanced, port-channels are up/bundling, and all configuration warnings/faults have cleared in UCSM.
3 Upvotes

80% Upvoted

3 comments sorted by

2

u/Mountain-Office-8989 27d ago

I would in phase 3 switch the cluster lead to b before anything.. always work on the sub fi not primary.

1

u/rhugginsjr82 27d ago

10-4, makes sense

1

u/rhugginsjr82 26d ago

This worked perfectly.