Tag Archives: NSX Controller

How to Configure centralized logging for the NSX Manager 6.x.x, NSX Controllers and NSX Edge devices

In my previous article discussed about VMware NSX Manager 6.x.x Backup and Restore and in this article I am going to discuss how to Configure centralized logging for the NSX Manager 6.x.x, NSX Controllers and NSX Edge devices.

In the production environment it is always recommenced to have remote log collector server configured, so that NSX Manager 6.x.x, NSX Controllers and NSX Edge devices sends all audit logs and system events from  NSX Manager 6.x.x, NSX Controllers and NSX Edge devices to the syslog server. This will be handy to troubleshoot or to get the final RCA in the event of any issue.

Let’s start with configuring syslog server for NSX Manager:-

1. Login to the VMware NSX Manager Virtual Appliance with Admin account.b1
2. Go to Manage –> General –> Click Edit in the Syslog Server section.p13. Provide Syslog Server, Port and Protocol details in the syslog server window and Click OK to test and save the  settings.p2

4. Once it is saved. It will show the settings like below.p3This is how we can configure Syslog server for NSX Manager.


Next is how to Configure Syslog Server for VMware NSX controllers :-

For NSX Controllers only supported method to configure syslog Server is through the NSX API. And using Rest API we need to push Syslog Server details on all the NSX controllers one by one.

Before we go ahead and push the syslog server on NSX controllers through REST API, We need to enable/Add REST API client to the browser. You can search for Rest API Client for the browser for Chrome or Mozilla and Add to the Browser.

api1

api2

Once you are done with adding REST API plug-in to your browser. There are couple of thing that needs to be remember.

REST API requests requires Authentication  header and Content-Type as application/xml to send HTTP body.

api4

Now we are ready to send the request body to configure Syslog Server for NSX controllers.

Open the Rest Client to set the request body to configure Syslog for NSX for vSphere Controllers. Make sure you have selected the Method as POST and URL as https://<NSX Manager IP>/api/2.0/vdn/controller/{controller-id}/syslog where controller-id is the name of NSX controller and can be found on the NSX Installation page.

HTTP Request body has to be this:

<controllerSyslogServer>
<syslogServer>x.x.x.x</syslogServer>
<port>514</port>
<protocol>UDP</protocol>
<level>INFO</level>
</controllerSyslogServer>

api3

This is how we can configure Syslog Server on NSX Controllers. If you want to DELETE the Syslog exporter use below request:-

Method :- DELETE and URL:- https://<NSX-Manager-IP>/api/2.0/vdn/controller/{controller-ID}/syslog.


How to configure Syslog Server for Distributed Logical Router.

1.  Login to vCenter Server using vSphere Web Client and choose Networking and Security –> NSX Edges –> and Double click on Logical Router.lrs1

2. Under Manage –> Settings –> Configuration click on Change under Syslog Servers.LRs2

3. Enter the Syslog Server and Protocol details in the Edit Syslog Server Configuration page and Click OK.LRs3

4. Now we can see Syslog is configured and ready to send all the logs to Remote Server.LRs4


How to configure Syslog Server for NSX Edge.

1.  Login to vCenter Server using vSphere Web Client and choose Networking and Security –> NSX Edges –> and Double click on NSX Edge.DRS1

2. Under Manage –> Settings –> Configuration click on Change under Syslog Servers.DRS2

3. Enter the Syslog Server and Protocol details and Click OK.

DRS4

That’s All. This is how you can configure Syslog Server for NSX Manager, NSX Controllers and NSX Edges.

Thank you and Happy learning 🙂

 

Network Virtualization with VMware NSX – Part 5

In Network Virtualization with VMware NSX – Part 4 we discussed Configuring and Deploying an NSX Distributed Router. Here in Network Virtualization with VMware NSX – Part 5 will discuss about VXLAN to VLAN Layer 2 Bridging, Configure and Deploy an NSX Edge Gateway, Configure Routes (Static Routing) on the NSX Edge Gateway and on the Distributed Router.

VXLAN to VLAN Layer 2 Bridging

A VXLAN to VLAN bridge enables direct Ethernet connectivity between virtual machines in a logical switch, and virtual machines in a distributed port group, This connectivity is called layer 2 bridging.

We can create a layer 2 bridge between a logical switch and a VLAN, which enables to migrate virtual workloads to physical devices with no effect on IP addresses. A logical network can leverage a physical gateway and access existing physical network and security resources by bridging the logical switch broadcast domain to the VLAN broadcast domain. Bridging can also be used in a migration strategy where you might be using P2V and you do not want to change subnets.

Note:- VXLAN to VXLAN bridging or VLAN to VLAN bridging is not supported. Bridging between different data centers is also not supported. All participants of the VLAN and VXLAN bridge must be in the same data center.

NSX Edge Services Gateway

The services gateway gives you access to all NSX Edge services such as firewall, NAT, DHCP, VPN, load balancing, and high availability. You can install multiple NSX Edge services gateway virtual appliances in a datacenter. Each NSX Edge virtual appliance can have a total of ten uplink and internal network interfaces.

ESG-1

NSX Edge logical router provides East-West and NSX Edge Services Gateway provide North-South Routing.

NSX Edge Services Gateway Sizing:-

NSX Edge can be deployed in four different configurations.ESG-2When we deploy NSX Edge gateway we need to choose right size as per load/requirements. We can also covert size of ESG later from Compact to Large, X-large or Quad Large. as you can in picture.

ESG20Note :- A service interruption might occur when the old NSX Edge gateway instance is removed and the new NSX Edge gateway instance is redeployed with new size or when we convert size of ESG.

NSX Edge Services Gateway features:-

ESG-3For resiliency and high-availability NSX Edge Services Gateway can be deployed as a pair of Active/Standby units (HA Mode).

When we deploy ESG/DLR in HA mode NSX Manager deploy the pair of NSX Edges/DLR on different hosts (anti-affinity rule). Heartbeat keepalives are exchanged every second between the active and standby edge instances to monitor each other’s health status.

If the ESXi server hosting the active NSX Edge fails, at the expiration of a “Declare Dead Time” timer, the standby node takes over the active duties. The default value for this timer is 15 seconds, but it can be tuned down (via UI or API calls) to 6 seconds.

The NSX Manager also monitors the state of health of the deployed NSX Edges, so it ensures to restart the failed unit on another ESXi host.

The NSX Edge appliance supports static and dynamic routing (OSPF, IS-IS, BGP, and Route redistribution).

Deploy NSX Edge gateway and Configure the static routing:

1. Connect to vCenter Server through vSphere Web Client —> Click Home tab –> Inventories –> Networking & Security and  select NSX Edges.ESG12. Click the green plus sign (+) to open the New NSX Edge dialog box. On the Name and description page, select Edge Services Gateway. (If you want to Enable HA for ESG select the Enable High Availability check box or leave it unchecked). Enter the Name of ESG as per your company standard and click Next.ESG23. On the CLI credentials page, enter the password for ESG in the password text box. Check Enable SSH Access box to enable SSH access for ESG appliance.             Note:- Password length must be at-least 12 characters. ESG1-P

ESG34. Select the Datacenter where you want to deploy this appliance. Select Appliance Size depending on your requirement we can also convert to any Size later as well. Check Enable auto rule generation to automatically generate service rules to allow flow of control traffic.

Under NSX Edge Appliances, click the green plus sign (+) to open the Add NSX Edge Appliance dialog box.ESG45. In Add NSX Edge Appliance dialog box select the Cluster and Datastore to deploy NSX Edge Appliance in the required location and designated datastore. And Click OK.

ESG56. verify all the settings on Configure deployment page and Click Next.

ESG67. On the Configure Interfaces page,click the green plus sign (+) to open the Add NSX Edge Interface dialog box

ESG78. Enter the Interface Name in the Name text box, choose Type, Click the Connected To –> Select link and choosed the required Distributed Port group. Click the green plus sign (+) under Configure Subnets to add subnet for the Interface.

ESG89. In the Add Subnet dialog box, click the green plus sign (+) to add an IP address field. Enter required IP address (192.168.100.3) in the IP Address text box and click OK to confirm the entry. Enter the subnet prefix length (24) in the Subnet prefix length text box and click OK.

ESG910. verify all the settings on Add NSX Edge Interface dialog box and Click OK.

ESG1011. Repeat steps 7-10 to add all required interfaces for ESG and Click Next.

ESG12

ESG11

ESG13

ESG1412. Once all Interfaces has been added verify settings on Configure Interfaces dialog box and Click Next.

ESG1513. On the Default gateway settings page, selec the Configure Default Gateway check box. Verify that the vNIC selection is Uplink-Interface. and  Enter the DG address (192.168.100.2) in the Gateway IP text box and Click Next.

ESG1614. On the Firewall and HA page, Select the Configure Firewall default policy check box. and Default Traffic Policy Accept. You can see that Configure HA parameters are gray out because we have not checked the Enable High Availability check box in step 2. And Click Next.

ESG1715. On the Ready to Complete dialog box verify all the settings (if you want to change any settings go back and change that)  and click Finish to complete the deployment for NSX Edge.

ESG1816. It will take few minutes to complete the deployment. Now under NSX Edges you can see that it is showing Deployed.

ESG1917. Double Click on the NSX Edge and can see the configuration settings as we choosed while deploying this.

esg1-ppNow Will Configure Static Routes on the NSX Edge Gateway:-

1. Double Click on the NSX Edge to browse NSX Edge –> Click on the Manage tab –> click Routing and select Static Routes. And Click the green plus sign (+) to open the Add Static Route dialog box.ESG-SR12. Select the interface connected to DLR which is (Transit-Interface), Enter the network ID with Subnet Mask (172.16.0.0/24) for which you want to add Routing and Next Hop Address for configured Network (in my case 192.168.10.2) and click OK.

ESG-SR23. After every settings or Modification need to Publish Changes. Click on Publish Changes.

ESG-SR34. Once Publishing finished you can see entry under Static Routes.

ESG-SR4

Configure Static Routes on the Distributed Router:-

1.Under Networking & Security –> NSX Edges –> double-click the Distributed Router entry to manage that object.ESG19

DLR-SR12. After browsing DLR  Click on the Manage and Routing tab. In the routing category panel select Static Routes and Click the Green Plus Sign (+) to add static Routes on DLR.

DLR-SR2

3. Select the interface connected to ESG which is (Transit-Interface), Enter the network ID with Subnet Mask (192.168.110.0/24) for which you want to add Routing and Next Hop Address for configured Network (in my case 192.168.10.1) and click OK.

DLR-SR34. After every settings or Modification need to Publish Changes. Click on Publish Changes. Once done you can see Static routes in the Static Routes lists.

DLR-SR4

Once Static Routing has been done will be able to ping the Logical switch network with External network. e.g external Network 192.168.110.10 to 3 logical switch network created in part 2 172.16.0.0/24.

esg1-2

That’s it. We are done with Deploying NSX Distributed Router and NSX Edge Services Gateway and also how to Configure Static Routing on DLR and ESG. 

In the next part (Network Virtualization with VMware NSX – Part 6) will discuss how to Configure Dynamic Routing on NSX Edge Appliances and NSX Distributed Router.

Thank you and stay tuned for next part. Keep sharing the knowledge 🙂

Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5

Network Virtualization with VMware NSX – Part 3

In the Network Virtualization with VMware NSX – Part 2 we have discussed about NSX Controller Cluster, How to Deploy the NSX Controller Instances, Create IP Pool, and Install Network Virtualization Components ( Prepare Hosts) on vSphere Hosts.

In this part will discuss about Logical Switch Networks and VXLAN Overlays.

Before Discussing VXLAN let’s discuss bit about Virtual LAN (VLAN):-

A VLAN is a group of devices on one or more LANs that are configured to communicate as if they were attached to the same wire, when in fact they are located on a number of different LAN segments. Because VLANs are based on logical instead of physical connections, they are extremely flexible.

VLANs address scalability, security, and network management by enabling a switch to serve multiple virtual subnets from its LAN ports.

VLAN Split switches into separate virtual switches (Broadcast Domains). Only members of a virtual LAN (VLAN) can see that VLAN’s traffic. Traffic between VLANs must go through a router.

By default, all ports on a switch are in a single broadcast domain. VLANs enable a single switch to serve multiple switching domains. The forwarding table on the switch is partitioned between all ports belonging to a common VLAN. All ports on a Switch by default part of single and default VLAN 0 and this default VLAN is called the Native VLAN.

Virtual Extensible LAN (VXLAN) enables you to create a logical network for your virtual machines across different networks. You can create a layer 2 network on top of your layer 3 networks.

VXLAN is an Ethernet in IP overlay technology, where the original layer 2 frame is encapsulated in a User Datagram Protocol (UDP) packet and delivered over a transport network. This technology provides the ability to extend layer 2 networks across layer 3 boundaries and consume capacity across clusters. The VXLAN adds 50 to 54 bytes of information to the frame, depending on whether VLAN tagging is used. VMware recommends increasing the MTU to at least 1,600 bytes to support NSX.

A VXLAN Number Identifier (VNI) is a 24-bit number that gets added to the VXLAN frame. The 24-bit address space theoretically enables up to 16 million VXLAN networks. Each VXLAN network is an isolated logical network.  VMware NSX™ starts with VNI 5000.

A Virtual Tunnel End Point (VTEP) is an entity that encapsulates an Ethernet frame in a VXLAN frame or de-encapsulates a VXLAN frame and forwards the inner Ethernet frame.

VXLAN Frame :-

VXLAN1The top frame is the original frame from the virtual machines, minus the Frame Check Sequence (FCS), encapsulated in a VXLAN frame. A new FCS is created by the VTEP to include the entire VXLAN frame. The VLAN tag in the layer 2 Ethernet frame exists if the port group that your VXLAN VMkernel port is connected to has an associated VLAN number. When the port group is associated with a VLAN number, the port group tags the VXLAN frame with that VLAN number.

VXLAN Replication Modes:-

Three modes of traffic replication exist: two modes are based on VMware NSX Controller™ based and one mode is based on data plane.

vxlan1Unicast has no physical network requirements apart from the MTU. All traffic is replicated by the VTEPs. In NSX, the default mode of traffic replication is unicast.  Unicast has Higher overhead on the source VTEP and UTEP.

Multicast mode uses the VTEP as a proxy. In multicast, the VTEP never goes to the NSX Controller instance. As soon as the VTEP receives the broadcast traffic, the VTEP multicasts the traffic to all devices. Multicast has lowest overhead on the source VTEP.

Hybrid mode is not the default mode of operation in NSX for vSphere, but is important for larger scale operations. Also the configuration overhead or complexity of L2 IGMP is significantly lower than multicast routing.

In the Network Virtualization with VMware NSX – Part 2 we have configured/Prepared Hosts so now let’s Configure VXLAN on the ESXi Hosts.

1. Connect to vCenter using web client.

2. Click Networking & Security and then click Installation.

3. Click the Host Preparation tab and under VXLAN column Click Configure to start Configuring VXLAN on the ESXi Hosts.

vxlan24. In the Configure VXLAN networking dialog box, Select Switch, VLAN, Set MTU to 1600, for VMKNic IP Addressing if you have created IP Pool choose existing IP from from list or Click IP Pool to create New Pool And Click OK.

vxlan3

vxlan45. It will take few minutes to configure depending upon number of Hosts into Cluster. If an error is indicated, it is a transitory condition that occurs early in the process of applying the VXLAN configuration to the cluster. The vSphere Web Client interface has not updated to display the actual status. Click Refresh to update the console.

vxlan56. Repeat the steps to configure all the clusters. Once Configuration done on all clusters.Verify that the VXLAN status is Enabled with a green check mark.

vxlan67.  Once VXLAN Configuration done for all the clusters and VXLAN status is Enabled with a green check mark. Click the Logical Network Preparation tab and verify that VXLAN Transport is selected. In the Clusters and Hosts list,expand each of the clusters and confirm the host has a vmk# interface created with IP Address from the IP Pool we have created for each.

vxlan7Once We have finished Configuring VXLAN and Verified VXLAN configuration for all the clusters. Next need to Configure the VXLAN ID Pool to identify VXLAN networks:-

1.  On the Logical Network Preparation tab, click the Segment ID button and Click Edit to open the Segment ID pool dialog box to configure ID Pool.

2. Enter the Segment ID Pool and Click Ok to complete. VMware NSX™ starts with VNI ID from 5000.

vxlan8Next we need to Configure a Global Transport Zone:-

A transport zone specifies the hosts and clusters that are associated with logical switches created in the zone. Hosts in a transport zone are automatically added to the logical switches that you create. This process is very similar to manually adding hosts to VMware vSphere Distributed Switch.

1. On the Logical Network Preparation tab, click Transport Zones and Click the green plus sign to open the New Transport Zone dialog box.

vxlan92.  Enter the Name for Transport Zone and Select Control Plane Mode. select Clusters to Add to the Transport Zone and Click OK to complete the creation.

vxlan10

vxlan11

———————————————————————————————————-

NSX Logical Switching

The Logical Switching capability in the NSX platform provides customers the ability to spin up isolated logical L2 networks with the same flexibility and agility, as it is to spin up virtual machines. Endpoints, both virtual and physical, can then connect to those logical segments and establish connectivity independently from the specific location where
they are deployed in the data center network. This is possible because of the decoupling between network infrastructure and logical networks provided by NSX network virtualization. Each logical switch gets its own unique VNI.

The deployment of the NSX Virtualization components can help to the agile and flexible creation of applications with their required network connectivity and services. A typical example is the creation of a multi-tier application.

LS11Configure Logical Switch Networks

We need to create logical switches for the all required networks (e.g. Transit, Web-Tier, App-Tier, and DB-Tier networks as per above picture.)
1. Connect to vCenter Server using web Client and Click Networking and Security and Select Logical Switches,  In the left navigation pane.

LS12. Click the Green plus sign to open the New Logical Switch dialog box. Enter the Logical Switch Name and  Select the Global Transport Zone we had created earlier, Choose the Control Plane Mode and Click OK to complete the Switch creation.

ls23. Wait for the update to complete and confirm Transit-Network appears with a status of Normal. Repeat steps to create all required Logical Switches and all are Normal.

LS3Once Logical Switches has been created we need to Migrate Virtual Machines to Logical Switches:-

1. In the left pane under Networking & Security and select Logical Switches. In the center pane, select the logical Switch e.g. Web-Tier –> Right Click the Choose Add VM..

LS42. Select Virtual Machines you want to add to the Logical Switch and Click Next.

LS53.  Select the VNIC you want to add to the Network and Click Next.

LS64. In the Ready to complete box verify the settings and  Click Finish to Complete adding VMs to desired Network.

LS75. To verify that VMs have been added to Logical Switch, Double Click the Logical Switch.

LS36. Click Related Objects and Virtual Machines tab and you can the list of VMs added to this specific Logical Switch.

LS87. Repeat the same steps for all the Logical Switches to Add VMs. Once done try to ping VMs in same switch and between Switch.

Now you can only ping VMs connected in the same Switch. To communicate with VMs in another Switch we need to configure Routing. Which will discuss in next Part.

======================================================

Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5

– See more at: http://virtualcloudsolutions.info/?p=829#sthash.YMq7IeEE.dpuf

Please share if useful …..Thank You 🙂

Network Virtualization with VMware NSX – Part 2

We have finished NSX Manager Deployment and Configuration in Network Virtualization with VMware NSX – Part 1. So let’s start with Deploying and Configuring NSX Manager Components.

NSX Controller Cluster

The Controller cluster in the NSX platform is the control plane component that is responsible in managing the switching and routing modules in the hypervisors. The controller cluster consists of controller nodes that manage specific logical switches. The use of controller cluster in managing VXLAN based logical switches eliminates the need for multicast support from the physical network infrastructure.

NSX Controller stores four types of tables:

  • The ARP table
  • The MAC table
  • VTEP (VXLAN Tunnel End Point) Table
  • Routing table

Note :- VMware recommends to add three controllers for scale and redundancy. But as of Now NSX Manager only support Max 3 Nodes Cluster. Even if you deploy 4th NSX Controller it will not show in the NSX Controller Nodes list.

Let’s Deploy the First NSX Controller Instance:-

1. Log in to the vCenter Server through Web Client and Click Networking & Security.

NSXM262. In the left navigation pane, Select Installation.

NSXC23. On the Management Tab under NSX Controller nodes you can see there is no node listed. To Add First NSX Controller Node Click the GREEN PLUS Sign (+).

4. Add Controller dialog box will be appear. Provide all required details (NSX Manager Name, Datacenter, Cluster Name, Datastore to hold node, ESXi host name, Select the network port group to connect the node, In the IP Pool you can select existing IP Pool or Create New pool by choosing New IP Pool option, enter and confirm Password for NSX Controller Nodes.) and Click OK to deploy First NSX Controller Node.

Note:- Password option will only appear for the First NSX Controller Node deployment for 2nd and 3rd node same Password will be used so there will not be password field.

NSXC35. Monitor the Deployment until the status change from Deploying to Normal. It will take few minutes to complete the Deployment.

NSXC76. Repeat the steps 3 and 4 to Add 2 more NSX Controller Nodes.

NSXC8Note:- You will notice my controllers are not 1,2, &, 3.  That is because my controllers deployment got failed because of some misconfiguration on IP Pools and  few i have deleted just to test something. That’s why you can see my controller name as 15,16 & 17. This is BUG with NSX 6.0 when you add new NSX Controller Node it will start from next number what you have last deployed even got fails or you deleted.

7. To verify that NSX Controller Nodes have been Deployed and working fine. Go to the Management Cluster where we have Deployed all three nodes.

NSX controller nodes are deployed as virtual appliances from the NSX Manager UI. Each appliance is characterized by an IP address used for all control-plane interactions and by specific settings (4 vCPUs, 4GB of RAM) that cannot currently be modified.

NSXC9 8. We can also PUTTY each of the controller to check the Status/Roles/Connections/Startup-nodes.

NSXC13We have deployed and verified NSX Controller nodes. All 3 have been Deployed up and running fine.

=======================================

Now we need to Install Network Virtualization Components/ Prepare ESXi Hosts :-

NSX installs three vSphere Installation Bundles (VIB) that enable NSX functionality to the host. One VIB enables the layer 2 VXLAN functionality, 2nd VIB enables the distributed router, and the 3rd VIB enables the distributed firewall. After adding the VIBs to a distributed switch, that distributed switch is called VMware NSX Virtual Switch. 

NSXC16

Note :- To remove the VIBs from the ESXi Host, the ESXi host requires a reboot.

You install the network infrastructure components in your virtual environment on a per-cluster level for each vCenter server, which deploys the required software on all hosts in the cluster. When a new host is added to this cluster, the required software is automatically installed on the newly added host. After the network infrastructure is installed on a cluster, Logical Firewall is enabled on that cluster.

As you can see in below screen under Firewall  that it is showing Not Enabled. When the installation is complete, the Installation Status column displays 6.0 and the Firewall column displays Enabled. Both columns have a green check mark as well.

NSXC16Let’s Install Network Virtualization Components now Cluster Now:-

1. Connect to vCenter using web client.

2. Click Networking & Security and then click Installation.

3. Click the Host Preparation tab.

4. For each cluster, Click Install and Click YES to Start installation for Cluster.

NSXC15

NSXC175. Monitor the installation until the Installation Status column displays a green check mark.

NSXC18

NSXC19Troubleshooting:- If the Installation Status column displays a red warning icon and says Not Ready, click Resolve. Clicking Resolve might result in a reboot of the host. If the installation is still not successful, click the warning icon. All errors are displayed. Take the required action and click Resolve again.

=============================================================

Other NSX Parts:-

Network Virtualization with VMware NSX – Part 1

Network Virtualization with VMware NSX – Part 2

Network Virtualization with VMware NSX – Part 3

Network Virtualization with VMware NSX – Part 4

Network Virtualization with VMware NSX – Part 5

Thank You!