Part 3: VMWare NSX Cluster preparation and Controllers Installation

In this post will cover the Controller deployment, VTEP for VXLAN configuration, Cluster preparation and Transport zone creation which are basic for VXLAN to work.

To make it easy this series is split in to 5 parts.

Part-1 : VMWare NSX Manager 6.4.x Installation and Initial Configuration

Part 2: VMWare NSX 6.4.x Micro-Segmentation Configuration steps

Part 3: VMWare NSX Cluster preparation and Controllers Installation

Part 4: VMWare NSX Logical Switching and DLR Configuration

Part 5: VMWare NSX EDGE and OSPF Routing Configuration

Introduction

Below diagram illustrates the Distributed switch design and VTEP setup

Below diagram illustrates theĀ  VTEP setup

Deploying NSX Controllers

Login to vCenter web client – Network and security – Installation and upgrade – Under Management – Select NSX Controller Nodes – Click on ADD to deploy Controller nodes.

Provide the IP and login password for NSX Manager.

 

Provide the Network port group to which Controllers will be connected to.

Please Note vCenter , NSX Manager and Controllers should be on same vLan


Click ADD on IP pool – Provide the IP pool details of the Controllers as shown below.

 


Select the Pool


Provide the Details for the NSX Controller and Finish.


Controller will be deployed. For production environments its recommended to deploy 3 controllers.

Please note if you deploy one for lab it will work, but don’t deploy 3 and remove 2 that will bring the cluster down and controllers will not work. Add the controllers at any point in time in future.


Controller is successfully deployed.

Preparing the Clusters for NSX

Login to vCenter web client – Network and security – Installation and upgrade – Host preparation – select the cluster and click Install to install the NSX VIB’s on the ESXi Hosts as shown below.

Configuring VTEP for VXLAN

Before Creating VTEP for VXLAN encapsulation, we need to make sure the ESXi hosts are part of the distributed switch. If not please add the hosts to distributed switch as shown below.

vCenter web client – Networking- select vDS – Add and manage hosts – Add Hosts.


Click on + new hosts to add hosts.


Select Manage Physical adapters.


Assign the physical vmNIC of the ESXi hosts to vDS uplinks as shown below.


Review the uplinks are added.


Review and finish


Now Select Host preparation and click on VXLAN – Configure as shown below.


Create new IP Pool for the VXLAN , create one as shown below.

Please note use separate vLAN for VTEP IP Pool.


Select the Distributed switch and provide MTU as minimum 1600 or more based on your network switches configuration.

Teaming Policy : Failover will act as active – standby and its easy for operations and torubleshooting. If you need mote VTEP through put then use multi VTEP’s. In my case i am using only one vtep for each ESXi host.

Review and save

After few minutes all VTEPS will be configured and IP’s will be shown as below.


When we prepare VTEP for VXLAN as port group in VDS will be created with required settings and MTU automatically.


Teaming policy is auto set to failover


Creating Segment ID and transport Zone

Now we need to provide segment ID’s and Transport Zones

Click on Edit on segment ID as shown below


Provide your segment ID, which is typically like your vLan ID . provide anything more than 5000 as till 4096 used for vLans.


Click on Transport Zone – Add as shown below.


Provide transport zone name and select Unicast , select cluster and click ADD.

Please note unicast doesn’t require any config on physical switch level like PIM and all, other two need it.

Testing VXLAN

Once it is done. Make sure your network switches are configured with MTUĀ  1600or more.

Connect to an ESXi host and try to ping the VTEP of other hosts using vmkping command as shown below. This will make sure its pinging with MTU 1572.

Below are the complete list of posts part of the series

Part-1 : VMWare NSX Manager 6.4.x Installation and Initial Configuration

Part 2: VMWare NSX 6.4.x Micro-Segmentation Configuration steps

Part 3: VMWare NSX Cluster preparation and Controllers Installation

Part 4: VMWare NSX Logical Switching and DLR Configuration

Part 5: VMWare NSX EDGE and OSPF Routing Configuration

Hope this post is useful. Leave your comments below.

Siva Sankar

Siva Sankar works as Solution Architect in Abu Dhabi with primary focus on SDDC, Automation,Network Virtualization, Digital Workspace, VDI, HCI and Virtualization products from VMWare, Citrix and Microsoft.

6 thoughts on “Part 3: VMWare NSX Cluster preparation and Controllers Installation

  • January 2, 2020 at 9:44 am
    Permalink

    I have a problem with the communication between two hosts

    ——————————————————————————————————————
    [root@esxi01:~] vmkping ++netstack=vxlan -s 1572 -d 192.168.71.102
    PING 192.168.71.102 (192.168.71.102): 1572 data bytes

    — 192.168.71.102 ping statistics —
    3 packets transmitted, 0 packets received, 100% packet loss
    [root@esxi01:~] vmkping ++netstack=vxlan -s 1572 -d 192.168.71.101
    PING 192.168.71.101 (192.168.71.101): 1572 data bytes
    1580 bytes from 192.168.71.101: icmp_seq=0 ttl=64 time=0.058 ms
    1580 bytes from 192.168.71.101: icmp_seq=1 ttl=64 time=0.053 ms
    1580 bytes from 192.168.71.101: icmp_seq=2 ttl=64 time=0.505 ms

    — 192.168.71.101 ping statistics —
    3 packets transmitted, 3 packets received, 0% packet loss
    round-trip min/avg/max = 0.053/0.205/0.505 ms
    [root@esxi01:~]
    —————————————————————————————————————

    How should I solve this issue?

    Reply
    • January 5, 2020 at 9:49 pm
      Permalink

      Please check the Port MTU on the physical switches on the hosts which are not working.

      Reply
  • May 15, 2020 at 7:27 am
    Permalink

    Hi Siva

    Excellent post!

    There is vlan in your “configure vxlan networking” screenshot.

    which vlan is that ? or is it VNI ?

    Can you please explain it ? if you can explain packet flow between VM A and VM B in same cluster and when in they belong to diff cluster it will be of great help.

    one more question , what should be gateway of vtep subnet ?

    Reply
    • May 16, 2020 at 1:23 pm
      Permalink

      Dear

      The vLan is for VTEP’s. The VTEP’s will do the encapsulation and decapsulation for your vxlan traffic. the VTEP vlan is a typical vLan which gateway can be on TOR or coreswitch or firewall depends on your network design.
      The packet flow between the VM’s in same or diff clusters will be taken care by VTEP’s as the ESXI hosts only knows vteps. so the packet will be encapsulated at source and sent to vtep ip of destination host where VM B is there and it will be decapsulated and handed over to the VM.

      thanks,

      Reply
  • October 27, 2020 at 8:33 pm
    Permalink

    Hi Siva,
    If I were to add a new cluster (for example like adding the SS-CLU-02 cluster in your article), the only things I should add are the ESGs (Edge and DLR) on the same network as the SS-CLU-01 cluster (so that the VMs can communicate with each other), as well as add the cluster to the Transport Zone.
    VTEP’s network / vlan configuration remains the same, right?

    Thanks for your info.

    Reply

Leave a Reply

Your email address will not be published.

Show Buttons
Hide Buttons