How to stretch a VCF Cluster: A Comprehensive Guide

Published by Valentin on

Introduction

In today’s blog post, we’ll discuss the steps to stretch a Management Cluster in VMware Cloud Foundation (VCF). If you’re like me, you might have found VMware’s documentation a bit unclear and lacking practical examples. After some trial and error, I found the best way to accomplish this task. Below, I’ve detailed the procedure to help you do it without the frustration.

Preparation

Before you start the stretching process, ensure you have met all the prerequisites:

  1. Deploy Half of the Cluster: Make sure half of the management cluster is already deployed.
  2. SDDC Manager: Ensure your Software-Defined Data Center (SDDC) Manager is working properly.
  3. vCenter Health: Verify that the vCenter is in good health.
  4. NSX Cluster: Confirm that your NSX cluster is operational.

You also need to determine which network profile you are using:

  • Profile-1: 4 NICs on 1 Distributed Virtual Switch (DVS)
  • Profile-2 & Profile-3: 2 NICs on 2 DVS

Step 1: Create a Network Pool in SDDC

  • Name the Network Pool: Based on the Availability Zone, e.g., NP-M1-AZB.
  • Configure Network Type: Determine which types of networks are required (vMotion, vSAN, etc.).
  • Set Up the VLAN ID, MTU, Network Information: Configure the VLAN ID, MTU, subnet mask, default gateway, and IP range.

Step 2: Commission Hosts for Stretching

  • Commission hosts to be used for the stretching operation.
  • Don’t forget to update the SSL certificate with the host’s Fully Qualified Domain Name (FQDN).

Step 3: Deploy a vSAN Witness Cluster

  • Deploy the vSAN Witness cluster on a third site.
  • Add the vSAN Witness to your inventory.

JSON Writing

For stretching your VCF cluster, you will need to prepare and validate a JSON configuration file.

Step 1: Retrieve the Host and Cluster IDs

  • Use the API Explorer to grab host and cluster IDs.
    • Get Host IDs: API – /v1/hosts,
      • Filter with status: UNASSIGNED_USEABLE
    • Get Cluster ID: API – /v1/cluster

Step 2: Write the Stretch JSON File

The JSON has four main parts:

  1. ESXi Host Declaration
  2. Network Profile Configuration and Mapping
  3. Object Definition (IP Pool, Uplink Profile, vSAN)
  4. vSAN Witness Details

Example JSON Profile 1:

If you’re using Profile 1 (4 NICs on 1 DVS):

  • NSX Profile: One network profile is configured, and two uplinks are mapped on a single DVS.
  • JSON Configuration:
{
"clusterStretchSpec": {
  "hostSpecs": [
   {
    "hostname": "host1.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink3",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink4",
       "vdsName": "VDS-01",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   },
   {
    "hostname": "host3.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	   "moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink3",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink4",
       "vdsName": "VDS-01",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   },
   {
    "hostname": "host5.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink3",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink4",
       "vdsName": "VDS-01",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   },
   {
    "hostname": "host7.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink3",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink4",
       "vdsName": "VDS-01",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   },
   {
    "hostname": "host9.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink3",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink4",
       "vdsName": "VDS-01",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   }
  ],
  "isEdgeClusterConfiguredForMultiAZ": true,
  "networkSpec": {
   "networkProfiles": [
    {
     "isDefault": false,
     "name": "NP-AZB-01",
     "nsxthostSwitchConfigs": [
      {
       "ipAddressPoolName": "IPPool-HtepAZB-01",
       "uplinkProfileName": "UpLinkProfile-VDS-01-AZB",
       "vdsName": "VDS-01",
       "vdsUplinkToNsxUplink": [
        {
         "nsxUplinkName": "uplink-1",
         "vdsUplinkName": "uplink1"
        },
	    {
         "nsxUplinkName": "uplink-2",
         "vdsUplinkName": "uplink2"
        }
		]
	  }
     ]
    }
   ],
   "nsxClusterSpec": {
    "ipAddressPoolsSpec": [
     {
      "description": "M01 AZB host TEP Pool",
      "name": "IPPool-HtepAZB-01",
      "subnets": [
       {
        "cidr": "X.X.X.X/X",
        "gateway": "X.X.X.X",
        "ipAddressPoolRanges": [
         {
          "end": "X.X.X.X",
          "start": "X.X.X.X"
         }
        ]
       }
      ]
     }
    ],
    "uplinkProfiles": [
     {
      "name": "UP-VDS-01-AZB",
      "teamings": [
       {
        "activeUplinks": [
         "uplink-1",
         "uplink-2"
        ],
        "name": "DEFAULT",
        "policy": "LOADBALANCE_SRCID",
        "standByUplinks": []
       }
      ],
      "transportVlan": 9999
     }
    ]
   }
  },
  "witnessSpec": {
   "fqdn": "witness.vcollin.private",
   "vsanCidr": "X.X.X.X/X",
   "vsanIp": "X.X.X.X"
  },
  "witnessTrafficSharedWithVsanTraffic": false
}
}

Example JSON for Profile 2 and 3

For Profiles 2 and 3 (2 NICs on 2 DVS):

  • NSX Profiles: Two NSX network profiles are needed, each linking to separate DVS.
{
"clusterStretchSpec": {
  "hostSpecs": [
   {
    "hostname": "host1.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink1",
       "vdsName": "VDS-02",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink2",
       "vdsName": "VDS-02",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   },
   {
    "hostname": "host3.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	   "moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink1",
       "vdsName": "VDS-02",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink2",
       "vdsName": "VDS-02",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   },
   {
    "hostname": "host5.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink1",
       "vdsName": "VDS-02",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink2",
       "vdsName": "VDS-02",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   },
   {
    "hostname": "host7.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink1",
       "vdsName": "VDS-02",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink2",
       "vdsName": "VDS-02",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   },
   {
    "hostname": "host9.vcollin.private",
    "hostNetworkSpec": {
     "networkProfileName": "NP-AZB-01",
     "vmNics": [
      {
       "id": "vmnic0",
       "uplink": "uplink1",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
	  {
       "id": "vmnic1",
       "uplink": "uplink1",
       "vdsName": "VDS-02",
	"moveToNvds": false
      },
	  {
       "id": "vmnic2",
       "uplink": "uplink2",
       "vdsName": "VDS-01",
	"moveToNvds": false
      },
      {
       "id": "vmnic3",
       "uplink": "uplink2",
       "vdsName": "VDS-02",
	"moveToNvds": false
      }
     ]
    },
    "id": "ID Grabbed with API: Get /v1/hosts",
    "licenseKey": "XXXXX-XXXXX-XXXX-XXXXX-XXXXX"
   }
  ],
  "isEdgeClusterConfiguredForMultiAZ": true,
  "networkSpec": {
   "networkProfiles": [
    {
     "isDefault": false,
     "name": "NP-AZB-01",
     "nsxthostSwitchConfigs": [
      {
       "ipAddressPoolName": "IPPool-HtepAZB-01",
       "uplinkProfileName": "UP-VDS-01-AZB",
       "vdsName": "VDS-01",
       "vdsUplinkToNsxUplink": [
        {
         "nsxUplinkName": "uplink-1",
         "vdsUplinkName": "uplink1"
        },
	    {
         "nsxUplinkName": "uplink-2",
         "vdsUplinkName": "uplink2"
        }
	]
	  },
	  {
	   "uplinkProfileName": "UP-VDS-02-AZB",
	   "vdsName": "VDS-02",
	   "vdsUplinkToNsxUplink": [
          {
           "nsxUplinkName": "uplink-1",
           "vdsUplinkName": "uplink1"
          },
	  {
           "nsxUplinkName": "uplink-2",
           "vdsUplinkName": "uplink2"
          }
	   ]
	  }
     ]
    }
   ],
   "nsxClusterSpec": {
    "ipAddressPoolsSpec": [
     {
      "description": "M01 AZB host TEP Pool",
      "name": "IPPool-HtepAZB-01",
      "subnets": [
       {
        "cidr": "X.X.X.X/X",
        "gateway": "X.X.X.X",
        "ipAddressPoolRanges": [
         {
          "end": "X.X.X.X",
          "start": "X.X.X.X"
         }
        ]
       }
      ]
     }
    ],
    "uplinkProfiles": [
     {
      "name": "UP-VDS-01-AZB",
      "teamings": [
       {
        "activeUplinks": [
         "uplink-1",
         "uplink-2"
        ],
        "name": "DEFAULT",
        "policy": "LOADBALANCE_SRCID",
        "standByUplinks": []
       }
      ],
      "transportVlan": 9999
     },
     {
      "name": "UP-VDS-02-AZB",
      "teamings": [
       {
        "activeUplinks": [
         "uplink-1",
         "uplink-2"
        ],
        "name": "DEFAULT",
        "policy": "LOADBALANCE_SRCID",
        "standByUplinks": []
       }
      ]
	 }
    ]
   }
  },
  "witnessSpec": {
   "fqdn": "witness.vcollin.private",
   "vsanCidr": "X.X.X.X/X",
   "vsanIp": "X.X.X.X"
  },
  "witnessTrafficSharedWithVsanTraffic": false
}
}

Step 3: Validate the JSON Configuration

You can validate the JSON with the API Command:

  •  POST /v1/clusters/{id}/validations

Where the id is the ID of the cluster that you store in your notepad

You must modify your JSON by adding “ClusterUpdateSpec”:

REM: This validation will only validate the syntax and the argument, not the content

Normally, you will receive a response: SUCCEEDED

Step 4: Stretch the cluster

Now that you have validate the JSON, you can stretch cluster by running the API CALL:

  • PATCH /v1/clusters/{id}

Where the id is the ID of the cluster that you store in your notepad

After successfully stretching the cluster, you can proceed with deploying the Edge cluster in NSX and continue your VMware Cloud Foundation deployment.

Conclusion

Stretching a VMware Cloud Foundation management cluster can be complex, but with the right preparation and a clear step-by-step guide, it’s achievable. By following these steps, you can confidently manage your VCF infrastructure and expand to multiple availability zones.

Source:

Cheers,
Valentin

Post Views: 155
Categories: SDDCVCFvSAN
Tags:

2 Comments

Amith · 17 November 2024 at 14:41

Hi, Thank you for this article. I’m working with VMware Cloud Foundation and have a question about vSAN stretched clusters in a management domain with multiple availability zones. My current setup:

– Management domain with AZ1 and AZ2
– AZ1 and AZ2 are in different subnets
– Planning to implement a vSAN stretched cluster

My questions:

– When management VMs failover from AZ1 to AZ2, do I need to manually change their IP addresses?
– If manual IP changes are required, is there a way to automate this process during failover?
– Are there any best practices or recommended configurations to handle IP addressing across different subnets in this scenario?

I’d appreciate any insights or experiences you can share. Thanks in advance for your help!

Reply

    Valentin · 18 November 2024 at 21:28

    Hey Amith,

    Thanks for your question.

    In fact, you do not need to change the IP if the network is stretched across the site.
    If the network is not, the VMs will lost network on the second site, you do not have the possibility to avoid that a network is located only on a site as the DVS belong to the cluster.

    By default, in a VCF deployement only VMs Network (where is located vCenter), Uplinks networks A and B (BGP) need to be stretch.

    When your domain (Management or VI) will be createn and fully functionnal, the Geneve transport zone will allow you to create “stretch” network on top of routed network (ESXi and Edges TEP).
    If you used mostly Geneve networks you will never needs to stretch any networks over sites and you will have a real network redundancy.

    Does it answer to your question?

    Reply

Leave a Reply

Avatar placeholder

Your email address will not be published. Required fields are marked *

Related Posts

NSX

Deploy NSX Edge Cluster without Tier-0/1 in VMware Cloud Foundation 5.1

Deploy NSX Edge Cluster without Tier-0/1 in VMware Cloud Foundation 5.1. Validate with POST /v1/edge-clusters/validations, deploy with POST /v1/edge-clusters.

vSAN

vSAN OSA cache tier capacity upgrade to 1.6TB

Unlock enhanced performance in your vSAN infrastructure with vSphere 8.0's expanded cache tier capacity. Upgrade now for optimized IO operations!

HCI

HCI vs Traditional Storage: 7 Common Misconceptions

Discover the truth behind Hyper-Converged Infrastructure (HCI) in this comprehensive post. Authored by an expert consultant, it addresses prevalent misconceptions surrounding HCI technology. Unveil the nuances between Traditional Storage and HCI Storage, exploring performance, latency, management complexity, monitoring, use cases, resilience, and more. Gain insights into why HCI, particularly vSAN, is increasingly becoming the preferred choice for enterprise storage solutions. Don't miss out on expert recommendations and valuable guidance for implementing HCI solutions seamlessly.