Tag Archives: NSX

VMware Released NSX for vSphere 6.2.3

VMware released NSX for vSphere 6.2.3 last month with many Changes and also includes a number of bug fixes in the previous version of NSX.


Here are Changes introduced in NSX vSphere 6.2.3:-

  • Logical Switching and Routing
    • NSX Hardware Layer 2 Gateway Integration: expands physical connectivity options by integrating 3rd-party hardware gateway switches into the NSX logical network
    • New VXLAN Port 4789 in NSX 6.2.3 and later: Before version 6.2.3, the default VXLAN UDP port number was 8472. See the NSX Upgrade Guide for details.
  • Networking and Edge Services
    • New Edge DHCP Options: DHCP Option 121 supports static route option, which is used for DHCP server to publish static routes to DHCP client; DHCP Options 66, 67, 150 supports DHCP options for PXE Boot; and DHCP Option 26 supports configuration of DHCP client network interface MTU by DHCP server.
    • Increase in DHCP Pool, static binding limits: The following are the new limit numbers for various form factors: Compact: 2048; Large: 4096; Quad large: 4096; and X-large: 8192.
    • Edge Firewall adds SYN flood protection: Avoid service disruptions by enabling SYN flood protection for transit traffic. Feature is disabled by default, use the NSX REST API to enable it.
    • NSX Edge — On Demand Failover: Enables users to initiate on-demand failover when needed.
    • NSX Edge — Resource Reservation: Reserves CPU/Memory for NSX Edge during creation. You can change the default CPU and memory resource reservation percentages using this API. The CPU/Memory percentage can be set to 0 percent each to disable resource reservation.PUT https://<NSXManager>/api/4.0/edgePublish/tuningConfiguration
    • Change in NSX Edge Upgrade Behavior: Replacement NSX Edge VMs are deployed before upgrade or redeploy. The host must have sufficient resources for four NSX Edge VMs during the upgrade or redeploy of an Edge HA pair. Default value for TCP connection timeout is changed to 21600 seconds from the previous value of 3600 seconds.
    • Cross VC NSX — Universal Distributed Logical Router (DLR) Upgrade: Auto upgrade of Universal DLR on secondary NSX Manager, once upgraded on primary NSX Manager.
    • Flexible SNAT / DNAT rule creation: vnicId no longer needed as an input parameter; removed requirement that the DNAT address must be the address of an NSX Edge VNIC.
    • NSX Edge VM (ESG, DLR) now shows both Live Location and Desired Location. NSX Manager and NSX APIs including GET api/4.0/edges//appliances now return configuredResourcePool and configuredDataStore in addition to current location.
    • Edge Firewall adds SYN flood protection: Avoid service disruptions by enabling SYN flood protection for transit traffic. Feature is disabled by default, use the NSX REST API to enable it.
    • NSX Manager exposes the ESXi hostname on which the 3rd-party VM Series firewall SVM is running to improve operational manageability in large-scale environments.
    • NAT rule now can be applied to a VNIC interface and not only an IP address.

For complete details please refer release note :- http://pubs.vmware.com/Release_Notes/en/nsx/6.2.3/releasenotes_nsx_vsphere_623.html

Thank you and Keep sharing 🙂

Network Virtualization with VMware NSX – Part 8

Let’s back into NSX mode again 🙂 In my last blog Network Virtualization with VMware NSX – Part 7 discussed about Network Address Translation (NAT) and Load Balancing with NSX Edge Gateway. Here in Network Virtualization with VMware NSX – Part 8 will discuss about High Availability of the NSX Edge.

High Availability

High Availability (HA) ensures that NSX Edge appliance is always available by installing an active pair of Edges on your virtualized infrastructure. We can enable HA either when installing NSX Edge appliance or after installing NSX Edge appliance.

The primary NSX Edge appliance is in the Active State and the Secondary Appliance is in Standby State. NSX Edge replicates the configuration of the primary appliance to the standby appliance. VMware recommends create the primary and secondary appliances on separate datastores. If you create the primary and secondary appliances on the same datastore, the datastore must be shared across all hosts in the cluster for the HA appliance pair to be deployed on different ESX hosts.

All NSX Edge services run on the active appliance. The primary appliance maintains a heartbeat with the standby appliance and sends service updates through an internal interface. If a heartbeat is not received from the primary appliance within the specified time (default value is 15 seconds), the primary appliance is declared dead. The standby appliance moves to the active state, takes over the interface configuration of the primary appliance, and starts the NSX Edge services that were running on the primary appliance. After switch over Load Balancer and VPN services need to re-establish TCP connection with NSX Edge, so service is disrupted for a short while. Logical switch connections and firewall sessions are synched between the primary and standby appliances, so there is no service disruption during switch over.

If the NSX Edge appliance fails and a bad state is reported, high availability force-synchronizes the failed appliance to revive it. When the appliance is revived, it takes on the configuration of the now active appliance and stays in a standby state. If the NSX Edge appliance is dead, you must delete the appliance and add an appliance.

NSX Edge ensures that the two HA NSX Edge virtual machines are not on the same ESX host even after you use DRS and vMotion (unless you manually vMotion them to the same host).

Now let’s verify HA settings and Configure High Availability for NSX Edge :-

1. Login to the web Client –> Home –> Networking and Security –> NSX Edges –> Double click either Logical Router or NSX Edge Services Router.HA1

2. It will open up the selected device. Click Manage –> Settings –> Configuration –> And under HA Configuration you can see HA Status is DISABLED. Same way you can check for Logical Router.HA2

3. Same can be verify from Management Cluster where we have deployed NSX Edge appliances. you can see in the below screenshot that only one instance of Edge Services Router (Edge Services Router-0) and One instance of Logical Router (Logical-Router-0) is running.HA3

4. Now let’s enabled HA for NSX Edge. Click Manage –> Settings –> Configuration –> And under HA Configuration –> Click Change.HA4

5. Change HA Configuration window will open up, Select HA Status –> Enable, Select vNIC, enter Declare Dead Time (Default is 15 Seconds), And enter the management IP for Heartbeat for both nodes and Click OK.HA5

6. It will take few seconds and you can see HA Status under HA Configuration is showing now Enabled.HA6

7. Let’s go to Management Cluster to see the number of Nodes. Now you can see that there are two instances up and running. Edge Services Router (Edge Services Router-0 and Edge Services Router-1)HA7

8. That’s it. Now NSX Edge Services Router is running is HA mode, If Active node will fail standby node will take over after 15 seconds. Same way we can enable HA for Logical Router. I have added screenshot for Logical Router.HA8



HA119. Once you have enabled HA for NSX Edge. You can putty to NSX edge and verify the Active Node and Standby Node by running Show Service highavailability command. Let me connect to and run this command to verify.

You can see in below result that This node (vshield-edge-4-0) is Active and vshield-edge-4-1 is peer host means Standby Node.HA14

10. Now let’s shut down the vshield-edge-4-0 and run the Show Service highavailability command again.

Now you can see in below result that vshield-edge-4-1 is Active and vshield-edge-4-0 is unreachable.HA15

11. Now let’s Power On the vshield-edge-4-0 and run the command again.

Now you can see in below result that vshield-edge-4-1 is Active and vshield-edge-4-0 is peer host means Standby Node.HA16

That’s It !! This is how we can enable HA and test failover for NSX Edge.

Thank You and Keep sharing :)


Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5

Network Virtualization with VMware NSX – Part 6

Network Virtualization with VMware NSX – Part 7

Network Virtualization with VMware NSX – Part 8

Network Virtualization with VMware NSX – Part 7

In my last blog Network Virtualization with VMware NSX – Part 6 we have discussed about Static and Dynamic routing. Here In the Network Virtualization with VMware NSX – Part 7 will be discus about Network Address Translation (NAT) and Load Balancing with NSX Edge Gateway.

Network Address Translation (NAT)

Network Address Translation (NAT) is the process where a network device assigns a public address to a computer (or group of computers) inside a private network. The main use of NAT is to limit the number of public IP addresses an organization or company must use, for both economy and security purposes.

Three blocks of IP addresses are reserved for private use and these Private IP addresses cannot be advertised in the public Internet. to, to, and  192.168.0 0 to

The private addressing scheme works well for computers that only have to access resources inside the network, like workstations needing access to file servers and printers. Routers inside the private network can route traffic between private addresses with no trouble. However, to access resources outside the network, like the Internet, these computers have to have a public address in order for responses to their requests to return to them. This is where NAT comes into play.NAT1

Another exam is public Cloud™ where there are multiple tenant running their workload with Private IP address range. Hosts assigned with private IP addresses cannot communicate with other hosts through the Internet. The solution to this problem is to use network address translation (NAT) with private addressing.

NAT2NSX Edge provides network address translation (NAT) service to assign a public address to a computer or group of computers in a private network. NSX Edge service supports two types of NAT:- SNAT and DNAT

Source NAT (SNAT) is used to translate a private internal IP address into a public IP address for outbound traffic. The below picture depict that NSX Edge gateway is translating Test-Network using addresses through to This technique is called masquerading where multiple Private IP Address are translating into Single host IP Address.

NAT3Destination NAT (DNAT) commonly used to publish a service located in a private network on a publicly accessible IP address. The below picture depict that NSX Edge NAT is publishing the Web Server on an external network as The rule translates the destination IP address in the inbound packet to an internal IP address and forwards the packet.

NAT4Configuring Network Address Translation (SNAT and DNAT) on an NSX Edge Services Gateway:-

1. Connect to vCenter Server through vSphere Web Client —> Click Home tab –> Inventories –> Networking & Security –> NSX Edges –> and Double Click NSX Edge.NAT5

2. Under NSX Edge router –> click Manage tab –> click NAT tab –> and click the green plus sign (+) and select Add DNAT Rule or Add SNAT Rule whichever you would like to add.NAT6

3. In the Add DNAT Rule dialog box, Select Uplink-Interface from the Applied On drop-down menu. Enter the Public IP address in the Original IP/Range text box and enter the destination Translated IP/Range. And select Enabled the DNAT rule check box and Click OK to add the rule.NAT7

4. click Publish Changes to add the rule.NAT8

5. Once rules pushed you can see one rule has been added to the rule list.NAT9

6. To test the Connectivity Using the Destination NAT Translation, putty to NSX Egde router with Admin account and run command  to begin capturing packets on the Transit-Interface.

debug packet display interface vNic_1 port_80 or debug packet display interface vNic_0 icmp

1st command will capture packets on interface 1 for TCP port 80 and 2nd command will capture packets on interface 0 for ICMP protocol.

Same way we can Add SNAT rules for outgoing traffic.


NSX Edge Load Balancer

Load Balancing is another network service available within NSX that can be natively enabled on the NSX Edge device. The two main drivers for deploying a load balancer are scaling out an application (Load is distributed across multiple backend servers) as well as improving its high-availability characteristics (Servers or applications that fail are automatically removed from the pool).LB1

The NSX Edge load balancer distributes incoming service requests evenly among multiple servers in such a way that the load distribution is transparent to users. Load balancing thus helps in achieving optimal resource use, maximizing throughput, minimizing response time, and avoiding overload. NSX Edge provides load balancing up to layer 7.

Note :- The NSX platform can be integrate load-balancing services offered by 3rd party vendors as well.

NSX Edge offers support for two types of deployment: One-arm mode (called proxy mode) and Inline mode (called transparent mode)

One-arm mode (called proxy mode)

The one-arm load balancer has several advantages and disadvantages. The advantages are that the design is simple and can be deployed easily. The main disadvantage is that you must have a load balancer per segment, leading to a large number of load balancers.

So when you design and deploy you need to see both the factors and choose which mode is fitting to your requirement.LB2

Inline mode (called transparent mode)

The advantage of using Inline mode is that the client IP address is preserved because the proxies are not doing source NAT. This design also requires fewer load balancers because a single NSX Edge instance can service multiple segments.
With this configuration, you cannot have a distributed router because the Web servers must point at the NSX Edge instance as the default gateway.

LB3Configuring Load Balancing with NSX Edge Gateway

1. Connect to vCenter Server through vSphere Web Client —> Click Home tab –> Inventories –> Networking & Security –> NSX Edges –> and Double Click NSX Edge.

2.  Under the Manage tab, click Load Balancer. In the load balancer category panel, select Global Configuration.

3. Under Load balancer global configuration –> Click Edit to open the Edit load balancer global configuration page, In the Edit load balancer global configuration page check the Enable Loadbalancer box and Click OK.LB4

4. Once Loan balancer has been Enabled, you can see the Green Tick mark for Enable Loadbalancer.LB5

5. Next We need to create Application Profiles, In the load balancer category panel, select Application Profiles –> Click the green plus sign (+) to open the New Profile dialog box.

6. In the New Profile dialog box, Enter the Name –> Select Protocol Type (HTTPS)           –>  Select the Enable SSL Passthrough check box and click OK.LB6

7. Once Application Profile has been created you can see Profile ID and name under box.LB7

8. Next we have to Create a Server Pool. I am going to create a round-robin server pool that contains the two Web server virtual machines as members providing HTTPS.

9. In the load balancer category panel, select Pools –> Click the green plus sign (+) to open the New Pool dialog box.

10. In the New Pool dialog box, Enter the Server Pool Name in the text box –> Select Algorithm – ROUND-ROBIN –> And Below Members, click the green plus sign (+) to open the New Member dialog box, and add all web servers as members.LB8

11. Once all members has been added into Server Pool verify and Click OK.LB9

12. Once Pools has been added you can see the Pool ID, Pool Name with Configured Algorithm under the box.LB10

13. Next we need to Create a Virtual Server. select Virtual Servers –> click the green plus sign (+) to open the New Virtual Server dialog box.

14. In the New Virtual Server dialog box, select Enabled box –> Enter the Virtual Server name –> enter the IP Address of the Interface –> Select protocol (HTTPS) –> Port Number for HTTPS (443) –> Select the Pool name and Application Profile created earlier and Click OK.LB11

15. Once done you can see Virtual Server Name with all configured details under the box. LB12

That’s It 🙂 This is how we can configure NAT and Load balancer using NSX Edge.

Thank You and Keep sharing :)


Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5

Network Virtualization with VMware NSX – Part 6

Network Virtualization with VMware NSX – Part 7

Network Virtualization with VMware NSX – Part 8

Network Virtualization with VMware NSX – Part 5

In Network Virtualization with VMware NSX – Part 4 we discussed Configuring and Deploying an NSX Distributed Router. Here in Network Virtualization with VMware NSX – Part 5 will discuss about VXLAN to VLAN Layer 2 Bridging, Configure and Deploy an NSX Edge Gateway, Configure Routes (Static Routing) on the NSX Edge Gateway and on the Distributed Router.

VXLAN to VLAN Layer 2 Bridging

A VXLAN to VLAN bridge enables direct Ethernet connectivity between virtual machines in a logical switch, and virtual machines in a distributed port group, This connectivity is called layer 2 bridging.

We can create a layer 2 bridge between a logical switch and a VLAN, which enables to migrate virtual workloads to physical devices with no effect on IP addresses. A logical network can leverage a physical gateway and access existing physical network and security resources by bridging the logical switch broadcast domain to the VLAN broadcast domain. Bridging can also be used in a migration strategy where you might be using P2V and you do not want to change subnets.

Note:- VXLAN to VXLAN bridging or VLAN to VLAN bridging is not supported. Bridging between different data centers is also not supported. All participants of the VLAN and VXLAN bridge must be in the same data center.

NSX Edge Services Gateway

The services gateway gives you access to all NSX Edge services such as firewall, NAT, DHCP, VPN, load balancing, and high availability. You can install multiple NSX Edge services gateway virtual appliances in a datacenter. Each NSX Edge virtual appliance can have a total of ten uplink and internal network interfaces.


NSX Edge logical router provides East-West and NSX Edge Services Gateway provide North-South Routing.

NSX Edge Services Gateway Sizing:-

NSX Edge can be deployed in four different configurations.ESG-2When we deploy NSX Edge gateway we need to choose right size as per load/requirements. We can also covert size of ESG later from Compact to Large, X-large or Quad Large. as you can in picture.

ESG20Note :- A service interruption might occur when the old NSX Edge gateway instance is removed and the new NSX Edge gateway instance is redeployed with new size or when we convert size of ESG.

NSX Edge Services Gateway features:-

ESG-3For resiliency and high-availability NSX Edge Services Gateway can be deployed as a pair of Active/Standby units (HA Mode).

When we deploy ESG/DLR in HA mode NSX Manager deploy the pair of NSX Edges/DLR on different hosts (anti-affinity rule). Heartbeat keepalives are exchanged every second between the active and standby edge instances to monitor each other’s health status.

If the ESXi server hosting the active NSX Edge fails, at the expiration of a “Declare Dead Time” timer, the standby node takes over the active duties. The default value for this timer is 15 seconds, but it can be tuned down (via UI or API calls) to 6 seconds.

The NSX Manager also monitors the state of health of the deployed NSX Edges, so it ensures to restart the failed unit on another ESXi host.

The NSX Edge appliance supports static and dynamic routing (OSPF, IS-IS, BGP, and Route redistribution).

Deploy NSX Edge gateway and Configure the static routing:

1. Connect to vCenter Server through vSphere Web Client —> Click Home tab –> Inventories –> Networking & Security and  select NSX Edges.ESG12. Click the green plus sign (+) to open the New NSX Edge dialog box. On the Name and description page, select Edge Services Gateway. (If you want to Enable HA for ESG select the Enable High Availability check box or leave it unchecked). Enter the Name of ESG as per your company standard and click Next.ESG23. On the CLI credentials page, enter the password for ESG in the password text box. Check Enable SSH Access box to enable SSH access for ESG appliance.             Note:- Password length must be at-least 12 characters. ESG1-P

ESG34. Select the Datacenter where you want to deploy this appliance. Select Appliance Size depending on your requirement we can also convert to any Size later as well. Check Enable auto rule generation to automatically generate service rules to allow flow of control traffic.

Under NSX Edge Appliances, click the green plus sign (+) to open the Add NSX Edge Appliance dialog box.ESG45. In Add NSX Edge Appliance dialog box select the Cluster and Datastore to deploy NSX Edge Appliance in the required location and designated datastore. And Click OK.

ESG56. verify all the settings on Configure deployment page and Click Next.

ESG67. On the Configure Interfaces page,click the green plus sign (+) to open the Add NSX Edge Interface dialog box

ESG78. Enter the Interface Name in the Name text box, choose Type, Click the Connected To –> Select link and choosed the required Distributed Port group. Click the green plus sign (+) under Configure Subnets to add subnet for the Interface.

ESG89. In the Add Subnet dialog box, click the green plus sign (+) to add an IP address field. Enter required IP address ( in the IP Address text box and click OK to confirm the entry. Enter the subnet prefix length (24) in the Subnet prefix length text box and click OK.

ESG910. verify all the settings on Add NSX Edge Interface dialog box and Click OK.

ESG1011. Repeat steps 7-10 to add all required interfaces for ESG and Click Next.




ESG1412. Once all Interfaces has been added verify settings on Configure Interfaces dialog box and Click Next.

ESG1513. On the Default gateway settings page, selec the Configure Default Gateway check box. Verify that the vNIC selection is Uplink-Interface. and  Enter the DG address ( in the Gateway IP text box and Click Next.

ESG1614. On the Firewall and HA page, Select the Configure Firewall default policy check box. and Default Traffic Policy Accept. You can see that Configure HA parameters are gray out because we have not checked the Enable High Availability check box in step 2. And Click Next.

ESG1715. On the Ready to Complete dialog box verify all the settings (if you want to change any settings go back and change that)  and click Finish to complete the deployment for NSX Edge.

ESG1816. It will take few minutes to complete the deployment. Now under NSX Edges you can see that it is showing Deployed.

ESG1917. Double Click on the NSX Edge and can see the configuration settings as we choosed while deploying this.

esg1-ppNow Will Configure Static Routes on the NSX Edge Gateway:-

1. Double Click on the NSX Edge to browse NSX Edge –> Click on the Manage tab –> click Routing and select Static Routes. And Click the green plus sign (+) to open the Add Static Route dialog box.ESG-SR12. Select the interface connected to DLR which is (Transit-Interface), Enter the network ID with Subnet Mask ( for which you want to add Routing and Next Hop Address for configured Network (in my case and click OK.

ESG-SR23. After every settings or Modification need to Publish Changes. Click on Publish Changes.

ESG-SR34. Once Publishing finished you can see entry under Static Routes.


Configure Static Routes on the Distributed Router:-

1.Under Networking & Security –> NSX Edges –> double-click the Distributed Router entry to manage that object.ESG19

DLR-SR12. After browsing DLR  Click on the Manage and Routing tab. In the routing category panel select Static Routes and Click the Green Plus Sign (+) to add static Routes on DLR.


3. Select the interface connected to ESG which is (Transit-Interface), Enter the network ID with Subnet Mask ( for which you want to add Routing and Next Hop Address for configured Network (in my case and click OK.

DLR-SR34. After every settings or Modification need to Publish Changes. Click on Publish Changes. Once done you can see Static routes in the Static Routes lists.


Once Static Routing has been done will be able to ping the Logical switch network with External network. e.g external Network to 3 logical switch network created in part 2


That’s it. We are done with Deploying NSX Distributed Router and NSX Edge Services Gateway and also how to Configure Static Routing on DLR and ESG. 

In the next part (Network Virtualization with VMware NSX – Part 6) will discuss how to Configure Dynamic Routing on NSX Edge Appliances and NSX Distributed Router.

Thank you and stay tuned for next part. Keep sharing the knowledge 🙂

Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5

VMware NSX- How to Delete/Remove NSX Logical Switch

Recently i was trying to Remove/Delete one of the NSX logical switch in my lab. While trying to Remove/Delete logical switch got this error:

er1 er2As per error message some of the resources still connected to this logical Switch, that’s why getting error: DB-Tier resources are still in use. So we need to remove any connected Virtual Machines from this logical switch and then we’ll be able to Remove/Delete this NSX logical Switch.

So first thing we need to check what are the VMs / Resources utilizing the NSX logical Switch.

Connect to VC ( vSphere Web Client) –> Networking and Security –> Logical Switches    –> and from right pane double click the logical switch we are trying to Remove.er3As you can see in below screen that one Virtual Machine is connected to DB-Tier NSX logical Switch. We have two option to remove this VM from this logical Switch.

1. Migrate Virtual Machine to another port group or,

2. delete the Virtual NIC card from the VM (which is not good practice).

er4So here we are going to migrate DB-sv-01a VM from DB-Tier logical switch to another Virtual Machine port group.

er5 er6Now after migrating VM we try to Remove/Delete this NSX Logical Switch.


er1Now we are able to Remove/Delete this NSX Logical Switch.


That’s All. I hope this will be informative for others. Thank you !!

Network Virtualization with VMware NSX – Part 3

In the Network Virtualization with VMware NSX – Part 2 we have discussed about NSX Controller Cluster, How to Deploy the NSX Controller Instances, Create IP Pool, and Install Network Virtualization Components ( Prepare Hosts) on vSphere Hosts.

In this part will discuss about Logical Switch Networks and VXLAN Overlays.

Before Discussing VXLAN let’s discuss bit about Virtual LAN (VLAN):-

A VLAN is a group of devices on one or more LANs that are configured to communicate as if they were attached to the same wire, when in fact they are located on a number of different LAN segments. Because VLANs are based on logical instead of physical connections, they are extremely flexible.

VLANs address scalability, security, and network management by enabling a switch to serve multiple virtual subnets from its LAN ports.

VLAN Split switches into separate virtual switches (Broadcast Domains). Only members of a virtual LAN (VLAN) can see that VLAN’s traffic. Traffic between VLANs must go through a router.

By default, all ports on a switch are in a single broadcast domain. VLANs enable a single switch to serve multiple switching domains. The forwarding table on the switch is partitioned between all ports belonging to a common VLAN. All ports on a Switch by default part of single and default VLAN 0 and this default VLAN is called the Native VLAN.

Virtual Extensible LAN (VXLAN) enables you to create a logical network for your virtual machines across different networks. You can create a layer 2 network on top of your layer 3 networks.

VXLAN is an Ethernet in IP overlay technology, where the original layer 2 frame is encapsulated in a User Datagram Protocol (UDP) packet and delivered over a transport network. This technology provides the ability to extend layer 2 networks across layer 3 boundaries and consume capacity across clusters. The VXLAN adds 50 to 54 bytes of information to the frame, depending on whether VLAN tagging is used. VMware recommends increasing the MTU to at least 1,600 bytes to support NSX.

A VXLAN Number Identifier (VNI) is a 24-bit number that gets added to the VXLAN frame. The 24-bit address space theoretically enables up to 16 million VXLAN networks. Each VXLAN network is an isolated logical network.  VMware NSX™ starts with VNI 5000.

A Virtual Tunnel End Point (VTEP) is an entity that encapsulates an Ethernet frame in a VXLAN frame or de-encapsulates a VXLAN frame and forwards the inner Ethernet frame.

VXLAN Frame :-

VXLAN1The top frame is the original frame from the virtual machines, minus the Frame Check Sequence (FCS), encapsulated in a VXLAN frame. A new FCS is created by the VTEP to include the entire VXLAN frame. The VLAN tag in the layer 2 Ethernet frame exists if the port group that your VXLAN VMkernel port is connected to has an associated VLAN number. When the port group is associated with a VLAN number, the port group tags the VXLAN frame with that VLAN number.

VXLAN Replication Modes:-

Three modes of traffic replication exist: two modes are based on VMware NSX Controller™ based and one mode is based on data plane.

vxlan1Unicast has no physical network requirements apart from the MTU. All traffic is replicated by the VTEPs. In NSX, the default mode of traffic replication is unicast.  Unicast has Higher overhead on the source VTEP and UTEP.

Multicast mode uses the VTEP as a proxy. In multicast, the VTEP never goes to the NSX Controller instance. As soon as the VTEP receives the broadcast traffic, the VTEP multicasts the traffic to all devices. Multicast has lowest overhead on the source VTEP.

Hybrid mode is not the default mode of operation in NSX for vSphere, but is important for larger scale operations. Also the configuration overhead or complexity of L2 IGMP is significantly lower than multicast routing.

In the Network Virtualization with VMware NSX – Part 2 we have configured/Prepared Hosts so now let’s Configure VXLAN on the ESXi Hosts.

1. Connect to vCenter using web client.

2. Click Networking & Security and then click Installation.

3. Click the Host Preparation tab and under VXLAN column Click Configure to start Configuring VXLAN on the ESXi Hosts.

vxlan24. In the Configure VXLAN networking dialog box, Select Switch, VLAN, Set MTU to 1600, for VMKNic IP Addressing if you have created IP Pool choose existing IP from from list or Click IP Pool to create New Pool And Click OK.


vxlan45. It will take few minutes to configure depending upon number of Hosts into Cluster. If an error is indicated, it is a transitory condition that occurs early in the process of applying the VXLAN configuration to the cluster. The vSphere Web Client interface has not updated to display the actual status. Click Refresh to update the console.

vxlan56. Repeat the steps to configure all the clusters. Once Configuration done on all clusters.Verify that the VXLAN status is Enabled with a green check mark.

vxlan67.  Once VXLAN Configuration done for all the clusters and VXLAN status is Enabled with a green check mark. Click the Logical Network Preparation tab and verify that VXLAN Transport is selected. In the Clusters and Hosts list,expand each of the clusters and confirm the host has a vmk# interface created with IP Address from the IP Pool we have created for each.

vxlan7Once We have finished Configuring VXLAN and Verified VXLAN configuration for all the clusters. Next need to Configure the VXLAN ID Pool to identify VXLAN networks:-

1.  On the Logical Network Preparation tab, click the Segment ID button and Click Edit to open the Segment ID pool dialog box to configure ID Pool.

2. Enter the Segment ID Pool and Click Ok to complete. VMware NSX™ starts with VNI ID from 5000.

vxlan8Next we need to Configure a Global Transport Zone:-

A transport zone specifies the hosts and clusters that are associated with logical switches created in the zone. Hosts in a transport zone are automatically added to the logical switches that you create. This process is very similar to manually adding hosts to VMware vSphere Distributed Switch.

1. On the Logical Network Preparation tab, click Transport Zones and Click the green plus sign to open the New Transport Zone dialog box.

vxlan92.  Enter the Name for Transport Zone and Select Control Plane Mode. select Clusters to Add to the Transport Zone and Click OK to complete the creation.




NSX Logical Switching

The Logical Switching capability in the NSX platform provides customers the ability to spin up isolated logical L2 networks with the same flexibility and agility, as it is to spin up virtual machines. Endpoints, both virtual and physical, can then connect to those logical segments and establish connectivity independently from the specific location where
they are deployed in the data center network. This is possible because of the decoupling between network infrastructure and logical networks provided by NSX network virtualization. Each logical switch gets its own unique VNI.

The deployment of the NSX Virtualization components can help to the agile and flexible creation of applications with their required network connectivity and services. A typical example is the creation of a multi-tier application.

LS11Configure Logical Switch Networks

We need to create logical switches for the all required networks (e.g. Transit, Web-Tier, App-Tier, and DB-Tier networks as per above picture.)
1. Connect to vCenter Server using web Client and Click Networking and Security and Select Logical Switches,  In the left navigation pane.

LS12. Click the Green plus sign to open the New Logical Switch dialog box. Enter the Logical Switch Name and  Select the Global Transport Zone we had created earlier, Choose the Control Plane Mode and Click OK to complete the Switch creation.

ls23. Wait for the update to complete and confirm Transit-Network appears with a status of Normal. Repeat steps to create all required Logical Switches and all are Normal.

LS3Once Logical Switches has been created we need to Migrate Virtual Machines to Logical Switches:-

1. In the left pane under Networking & Security and select Logical Switches. In the center pane, select the logical Switch e.g. Web-Tier –> Right Click the Choose Add VM..

LS42. Select Virtual Machines you want to add to the Logical Switch and Click Next.

LS53.  Select the VNIC you want to add to the Network and Click Next.

LS64. In the Ready to complete box verify the settings and  Click Finish to Complete adding VMs to desired Network.

LS75. To verify that VMs have been added to Logical Switch, Double Click the Logical Switch.

LS36. Click Related Objects and Virtual Machines tab and you can the list of VMs added to this specific Logical Switch.

LS87. Repeat the same steps for all the Logical Switches to Add VMs. Once done try to ping VMs in same switch and between Switch.

Now you can only ping VMs connected in the same Switch. To communicate with VMs in another Switch we need to configure Routing. Which will discuss in next Part.


Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5

– See more at: http://virtualcloudsolutions.info/?p=829#sthash.YMq7IeEE.dpuf

Please share if useful …..Thank You 🙂

Network Virtualization with VMware NSX – Part 2

We have finished NSX Manager Deployment and Configuration in Network Virtualization with VMware NSX – Part 1. So let’s start with Deploying and Configuring NSX Manager Components.

NSX Controller Cluster

The Controller cluster in the NSX platform is the control plane component that is responsible in managing the switching and routing modules in the hypervisors. The controller cluster consists of controller nodes that manage specific logical switches. The use of controller cluster in managing VXLAN based logical switches eliminates the need for multicast support from the physical network infrastructure.

NSX Controller stores four types of tables:

  • The ARP table
  • The MAC table
  • VTEP (VXLAN Tunnel End Point) Table
  • Routing table

Note :- VMware recommends to add three controllers for scale and redundancy. But as of Now NSX Manager only support Max 3 Nodes Cluster. Even if you deploy 4th NSX Controller it will not show in the NSX Controller Nodes list.

Let’s Deploy the First NSX Controller Instance:-

1. Log in to the vCenter Server through Web Client and Click Networking & Security.

NSXM262. In the left navigation pane, Select Installation.

NSXC23. On the Management Tab under NSX Controller nodes you can see there is no node listed. To Add First NSX Controller Node Click the GREEN PLUS Sign (+).

4. Add Controller dialog box will be appear. Provide all required details (NSX Manager Name, Datacenter, Cluster Name, Datastore to hold node, ESXi host name, Select the network port group to connect the node, In the IP Pool you can select existing IP Pool or Create New pool by choosing New IP Pool option, enter and confirm Password for NSX Controller Nodes.) and Click OK to deploy First NSX Controller Node.

Note:- Password option will only appear for the First NSX Controller Node deployment for 2nd and 3rd node same Password will be used so there will not be password field.

NSXC35. Monitor the Deployment until the status change from Deploying to Normal. It will take few minutes to complete the Deployment.

NSXC76. Repeat the steps 3 and 4 to Add 2 more NSX Controller Nodes.

NSXC8Note:- You will notice my controllers are not 1,2, &, 3.  That is because my controllers deployment got failed because of some misconfiguration on IP Pools and  few i have deleted just to test something. That’s why you can see my controller name as 15,16 & 17. This is BUG with NSX 6.0 when you add new NSX Controller Node it will start from next number what you have last deployed even got fails or you deleted.

7. To verify that NSX Controller Nodes have been Deployed and working fine. Go to the Management Cluster where we have Deployed all three nodes.

NSX controller nodes are deployed as virtual appliances from the NSX Manager UI. Each appliance is characterized by an IP address used for all control-plane interactions and by specific settings (4 vCPUs, 4GB of RAM) that cannot currently be modified.

NSXC9 8. We can also PUTTY each of the controller to check the Status/Roles/Connections/Startup-nodes.

NSXC13We have deployed and verified NSX Controller nodes. All 3 have been Deployed up and running fine.


Now we need to Install Network Virtualization Components/ Prepare ESXi Hosts :-

NSX installs three vSphere Installation Bundles (VIB) that enable NSX functionality to the host. One VIB enables the layer 2 VXLAN functionality, 2nd VIB enables the distributed router, and the 3rd VIB enables the distributed firewall. After adding the VIBs to a distributed switch, that distributed switch is called VMware NSX Virtual Switch. 


Note :- To remove the VIBs from the ESXi Host, the ESXi host requires a reboot.

You install the network infrastructure components in your virtual environment on a per-cluster level for each vCenter server, which deploys the required software on all hosts in the cluster. When a new host is added to this cluster, the required software is automatically installed on the newly added host. After the network infrastructure is installed on a cluster, Logical Firewall is enabled on that cluster.

As you can see in below screen under Firewall  that it is showing Not Enabled. When the installation is complete, the Installation Status column displays 6.0 and the Firewall column displays Enabled. Both columns have a green check mark as well.

NSXC16Let’s Install Network Virtualization Components now Cluster Now:-

1. Connect to vCenter using web client.

2. Click Networking & Security and then click Installation.

3. Click the Host Preparation tab.

4. For each cluster, Click Install and Click YES to Start installation for Cluster.


NSXC175. Monitor the installation until the Installation Status column displays a green check mark.


NSXC19Troubleshooting:- If the Installation Status column displays a red warning icon and says Not Ready, click Resolve. Clicking Resolve might result in a reboot of the host. If the installation is still not successful, click the warning icon. All errors are displayed. Take the required action and click Resolve again.


Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5

Thank You!

Network Virtualization with VMware NSX – Part1

Overview of VMware NSX

VMware NSX is a network virtualization platform that enables you to build a rich set of logical networking services such as Logical Switching, Logical Routing, Logical Firewall, Logical Load Balancer, Logical Virtual Private Network (VPN). NSX enables you to start with your existing network and server hardware in the data center. NSX adds nothing to the physical switching environment. NSX exists in the ESXi environment and is independent of the network hardware.

NSX is a software networking and security virtualization platform that delivers the operational model of a virtual machine for the network. Virtual networks reproduce the Layer2 – Layer7 network model in software. By virtualizing the network, NSX delivers a new operational model for networking that breaks through current physical network barriers and enables data center operators to achieve better speed and agility with reduced costs.

With VMware NSX, virtualization now delivers for networking what it has already delivered for compute and storage. In much the same way that server virtualization programmatically creates, snapshots, deletes and restores software-based virtual machines (VMs), VMware NSX network virtualization programmatically creates, snapshots, deletes, and restores software-based virtual networks.

NSX can be configured through the vSphere Web Client, a command line interface (CLI), and REST API.

An NSX-v deployment consists of a data plane, control plane and management plane:


NSX Functional Services

NSX provides a faithful reproduction of network & security services in software. e.g.


Preparing for Installation

NSX has the following requirements:

  • vCenter Server 5.5 or later
  • ESXi 5.0 or later for each server
  • VMware Tools

NSX requires below ports for installation and daily operations:

  • 443 between the ESXi hosts, vCenter Server, and NSX Manager.
  • 443 between the REST client and NSX Manager.
  • TCP 902 and 903 between the vSphere Web Client and ESXi hosts.
  • TCP 80 and 443 to access the NSX Manager management user interface and initialize the vSphere and NSX Manager connection.
  • TCP 1234 Communication between ESXi Host and NSX Controller Clusters
  • TCP 22 for CLI troubleshooting.

NSX Manager

The NSX Manager is the centralized management component of NSX, and runs as a virtual appliance on an ESXi host. Each NSX Manager manages a single vCenter Server environment. The NSX Manager requires connectivity to the vCenter Server, ESXi host, and NSX Edge instances, vShield Endpoint module, and NSX Data Security virtual machine. NSX components can communicate over routed connections as well as different LANs.

The NSX Manager virtual machine is packaged as an Open Virtualization Appliance (OVA) file, which allows you to use the vSphere Web Client to import the NSX Manager into the datastore and virtual machine inventory.

In the NSX for vSphere architecture, the NSX Manager is tightly connected to the vCenter server managing the compute infrastructure. In fact, there is a 1:1 relationship between the NSX Manager and vCenter and upon installation the NSX Manager registers with vCenter and injects a plugin into the vSphere Web Client for consumption within the Web management platform.

NSX Manager Components Plugin and Integration inside vSphere Web Client :-


Note :- You can install the NSX Manager in a different vCenter than the one that the NSX Manager will be interoperating with. A single NSX Manager serves a single vCenter Server environment only.

Note :- Each NSX virtual appliance includes VMware Tools. Do not upgrade or uninstall the version of VMware Tools included with a NSX virtual appliance.

Deploy NSX Manager Virtual Appliance :-

1. Download the NSX Manager Open Virtualization Appliance (OVA) from https://my.vmware.com/web/vmware/downloads.

NSX22. Under Networking & Security section click Download Product for VMware NSX.

NSX33. Select your Version and click Go to Downloads.

NSX44. On the Download VMware NSX for vSphere 6.X Window click Download Now to start downloading of the NSX Manager Open Virtualization Appliance (OVA) file.

5. Place the NSX Manager Open Virtualization Appliance (OVA) file in a location accessible to your vCenter server and ESXi hosts.

6. Log in to the vSphere Web Client where do you want to Import/Run the NSX Manager.

7. Right-click the Cluster/Host where you want to install NSX Manager and select Deploy OVF Template.

NSX158. If this is the first time you are deploying an OVF file, It will ask you to download the Client Integration Plug-in. Click on Download the Client Integration Plug-in link to download and install. (Close all browser before installation and once completed Log in to the vSphere Web Client again and navigate to the host where you were installing NSX Manager.)

NSX169. On the Select Source window Click Browse to locate the folder on your computer that contains the NSX Manager OVA file, Select the OVA click Open and click Next.


NSXM310. It will take few seconds to validate the OVA. Once validated click Next to continue

NSXM411. Review the OVF template details and click Next.

NSXM612. Click Accept to accept the VMware license agreements and click Next.

NSXM713. Name the NSX Manager and select the location for the NSX Manager that you are installing and Click Next.

NSXM814. Select Storage and Click Next.

NSXM915. On the Setup networks page, confirm that the NSX Manager adapter has been mapped to the correct host network and click Next.

NSXM1016. On the Customized template page, specify the Passwords, Network Properties, DNS, NTP and SSH and Click Next.

NSXM1117. On the Ready to complete page, review the NSX Manager settings, Check the Power On after Deployment and click Finish.

NSXM12The NSX Manager is installed as a virtual machine in the inventory. Once deployment of NSX manager finished we need to Log In to the NSX Manager Virtual Appliance and Configure the NSX Manager.

Log In to the NSX Manager Virtual Appliance:-

1. Open the Web browser window and type the Name/IP address assigned to the NSX Manager. For example, https://nsxmanager.vdca550.com (In my case). Accept the security certificate. The NSX Manager login screen appears.

2. Use User name admin and the password you set during installation. If you had not set a password during installation, type default as the password and Click Log In.

NSXM133. Below is Home Screen of the NSX Manager. As you can see from here we can Manage Appliance Settings, Manage vCenter Registration, Backup and Restore of NSX Manager, and Upgrade NSX Manager Appliance.

NSXM144. Click on the View Summary to View and Configure the NSX Manager.

NSXM155. Click on the Mange Tab. From General Setting you can configure Time (NTP) and Syslog server Settings. Click Edit to enter the details and click ok.

NSXM16Time (NTP) Settings:-

NSXM17Syslog Server Settings:-

NSXM186. Click on Network. You can Review/Edit NSX Manager Network Settings and DNS Server settings for NSX Manager. Click on Edit to Edit the settings and click OK.


NSXM197. Click on SSL Certificates option to configure the SSL Certificate for NSX Manager.

8. Click on Backups and Restore option to take or scheduled Back of NSX manager Data.

NSXM21Note :- Currently there is no option to have multiple NSX managers for redundancy, So Backup is very critical for NSX Manger. In the case of NSX Manager failure you need to Deploy New NSX Manger and Restore the configuration from last backup.

9. To Upgrade your NSX Manager Appliance to latest version Download the Upgrade bundle from VMware website first and then from Upgrade Option in NSX Manager you can Upgrade to latest version. Click Upgrade in the Upgrade NSX Management Service –> Click Browse to select the Upgrade bundle and Click Upgrade to start the upgrade.


NSXM2410. Last and Important Option is NSX Management Service. Click on NSX Management Service –> Under vCenter Server Section click Configure to Register vCenter Server with NSX Manager. Enter vCenter Server Name, User Name and Password and Click OK to Add/Register vCenter Server with NSX Manager.

NSXM2511. Once vCenter Server registration done with NSX Manager We can connect to vCenter Server and verify that Networking & Security Icon under Inventories List.

NSXM2612. Click on the Networking & Security to open up the NSX Home page.

NSXM27And now we are all set to start the use of NSX features.

In the Next Part will discuss Installing and Configuring NSX Components …Please leave your Questions/Comments/Suggestions..Thank you !! 

Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5