A Walk Through the NSX Design Guide (Part 1)

In my personal journey to understand NSX, I've found the greatest understanding to come from the NSX Design Guide. As it so happens, when I decided to write this blog (Dec 30, 2022), I noticed VMware released a new design guide on December 5th. This gave the perfect opportunity to write new blog posts as they relate to topics in the design guide. My intention is to write a series of blog post that give concrete examples corresponding to the section headings in the design guide. The reader will hopefully get a better understanding of the ideas being conveyed in the design guide, and I get to flesh out my understanding of the topic by explaining it. 

The NSX design guide can be found here: https://communities.vmware.com/t5/VMware-NSX-Documents/VMware-NSX-T-Reference-Design/ta-p/2778093?attachment-id=110654


Starting on page 18, we've got an introduction to the NSX Architecture Components.



To start, we've got the NSX management cluster. NSX can function with only a single NSX manager, but for production support, you must have 3 NSX managers in a cluster. Lets have a look at the management cluster from the perspective of my vCenter Server.

In a production environment, you'll probably see NSX managers using a larger VM form factor. Here, I've used the smallest form factor suitable for a home lab. And since we're getting into specifics, let me specify my nsx and vcenter versions:

vCenter Server:  7.0.3 build 19717403
ESXi: 7.0.2 build 18426014
NSX: 3.2.0.0.0.19067070

If you want a more in-depth understanding of nsx installation, check out the NSX install guide here:

https://docs.vmware.com/en/VMware-NSX-T-Data-Center/3.2/nsxt_32_install.pdf

Also consider the VMware interoperability guide here to make sure your NSX and vCenter (and ESXi) instances are compatible:

https://interopmatrix.vmware.com/Interoperability?col=2,&row=912,


Assuming your NSX infrastructure is healthy, the most common method to interface with NSX will be the NSX web GUI. To access the web GUI, browse to the ip address of one of the NSX manager ip addresses, or URL, or the VIP address:


After login, it looks like this:


The other access method is via SSH. There are two user accounts of consequence, root login and an admin login. Each will give the user a different prompt and different commands. Lets take a look:

Here I've logged in as root. It gives the user access to Linux prompt. A common reason why you may want to login as root is to view some of the NSX specific log files and you are familiar with a Linux command line:





Alternatively, there is the admin access. Its possible to change from root privlidge to admin and vice versa. While I'm already in root access, the easiest way to change to admin access is type "su admin" to drop to admin user access:

I can get to root from admin privilege using "st en". I believe this is short for "start engineering" mode. In admin privilege mode, the most common command you'll run is "get cluster status". This will allow you to validate some of the topics discussed in the design guide heading "NSX Architecture Components" .



You can check this same information in the NSX GUI, but the output will be a little delayed relative to cli output. Interestingly, you get a bit more information in the GUI. I've highlighted the fact that it also shows how many transport nodes this specific manager is responsible for. Each NSX manager will handle management and control plane for 1/3 of the total transport nodes. 

Lastly for this blog post, I'll talk about the management plane and control plane from the perspective of the ESXi hosts. When an ESXi host has NSX installed on it, we say that that host is prepared for NSX. Preparation is the process of installing NSX on the host. I'm starting here with NSX already installed and will show how you can validate the NSX management and control planes of an ESXi host. 

First, you'll need to SSH to the host. Next, verify the version of the host and verify that NSX is installed.  


What you should see if NSX is installed on the host is a list of vibs starting with the name nsx. Think of a vib as a driver. I went over them in a previous blog post. When you've verified that NSX installed, be aware that you'll have access to an NSX specific cli tool called 'nsxcli'.


The ESXi hosts form their management and control plane connections to the nsx manager and controller on port 1234 and 1235 respectively. Using command "esxcli network ip connection list | grep 1234" or "1235", I can check the connection of the management plane and control plane. Below, you can see my ESXi host has a management plane connection to each of the 3 hosts, but only a single control plane connection to one manager.



In subsequent blog posts, I'll cover additional topics in the NSX Design guide. As a final point to illustrate the kinds of things NSX manager can show from a CLI stand point, I'll leave a list of commands available from the "get" prompt. This output was generated from typing "get ?". For a more comprehensive listing of all cli commands, perform a Google search for ""NSX-T cli guide".

From my NSX Lab:

nsx-t-manager1> get version
Thu Jan 05 2023 UTC 18:22:30.595
VMware NSX Software, Version 3.2.0.0.0.19067081
Product Version 3.2.0.0.0.19067070

nsx-t-manager1> get
  all                  All items
  arp-table            ARP entries
  auth-policy          Authentication policy
  capture              Packet capture
  certificate          X509 certificate
  cgroup               Container grouping object
  cgroups              Container grouping objects
  cli-output           CLI output
  cli-timeout          CLI timeout
  clock                Manage the system clock
  cluster              NSX cluster
  command              Command history
  configuration        Configuration details
  cpu-stats            CPU statistics
  data                 Datum(s) for nestdb-pigeon
  domain-object        Logical object representation for controller
  domain-objects       Logical objects representation for controller
  eula                 End User License Agreement
  file                 File
  files                Files
  filesystem-stats     Filesystem statistics
  firewall             Firewall configuration
  gateway              Gateway
  gateways             Gateways
  group                Multicast group address
  groups               IGMP groups
  hostname             System's network name
  interface            Interface configuration
  interfaces           Interface status and configuration
  last                 Show last N lines of output
  log-file             Log file
  logging-servers      Syslog logging servers
  logical-router       Logical router
  logical-routers      Logical routers
  logical-switch       Logical switch
  logical-switch-port  Logical switch port
  logical-switches     Logical switches
  managers             NSX managers
  memory-stats         Memory statistics
  name-servers         Name servers
  network-stats        Show system network stats
  node                 Node
  nodes                Nodes
  ntp-server           NTP server
  ntp-servers          NTP servers
  processes            System processes
  realization-status   The fulfillment status of a desired entity
  receivers            Transport nodes connected to nestdb-pigeon
  route                IP routing table
  routes               IP routing table
  routing-domain       Routing domain
  routing-domains      Routing domains
  search-domains       DNS search domains
  segment              Segment
  segment-port         Segment port
  segments             Segments
  service              Node service
  services             Node services
  site-replicator      Controller module to sync states among sites.
  snmp                 SNMP service
  sockets              Open IP sockets
  stretched            Stretched configurations used in multiple sites
  support-bundle       Support bundle
  transport-node       Transport node
  transport-nodes      Transport nodes
  upgrade              Upgrade
  upgrade-bundle       Node Upgrade bundle
  uptime               Show system uptime information
  user                 Configure system passwords
  version              System version
  vif                  VIF
  vtep-group           Vtep group
  vtep-groups          Vtep groups


Comments

Post a Comment

Popular posts from this blog

VXLAN versus GENEVE (NSX-V vs. NSX-T)

Image
  August 14th, 2021 With the ramp-up of NSX-T overlay networks and transition away from NSX-V overlay networks, it's a good time to look at one of the fundamental differences between them. NSX-V uses VXLAN as its encapsulation protocol while NSX-T uses the more recent GENEVE encapsulation protocol. Each require the physical networking devices have their MTU adjusted to 1600 bytes or greater. We'll take a detailed look at why that is. First, the basics: VXLAN is: short for Virtual eXtensible Local Area Network Is defined in rfc 7348 - https://datatracker.ietf.org/doc/html/rfc7348 Uses UDP port 4789 8 byte header GENEVE is: short for Generic Network Virtualization Encapsulation Is defined in rfc 8926 -  https://datatracker.ietf.org/doc/html/rfc8926 Uses UDP port 6081 16 byte header From the prospective of the physical network, an overlay network is essentially an application. NSX-T is an application using well known UDP port 6081. Switching perspectives to the overlay's
Read more

"Twice NAT" with NSX-T T0 Gateway

Image
  Network address translation, or more commonly NAT, is most often used to do source NAT or destination NAT. In rare instances, SNAT or DNAT isn't enough to get ip packets to their destination. Enter twice nat. While twice NAT isn't a function I've needed often, it was a function drilled into me while studying for various Cisco exams. I used to watch Christian Matei explain it over at INE and it really helped me to understand what twice NAT is and how to configure it on Cisco devices. Previous to my time working with NSX-T, I'd only utilized twice NAT once. A few years back, a customer wanted to build and IPsec tunnel from the organization HQ to a branch office. The private ip space used by HQ and the branch used overlapping ip space. We used twice NAT to NAT the source and destination ip address in a single NAT rule ( similar to this guy ) before shoving it all into an IPsec tunnel. Previous to working with NSX, circumstances requiring twice NAT were uncommon.  While w
Read more

Packet Capture Network Traffic Inside ESXi Hypervisor

Image
The three components of computer infrastructure are considered compute, storage, and networking. When applications don't work properly, its often the case that networking is the first component to receive blame for failed application performance. In the case of virtualized workloads (i.e. applications running on virtual machines), we are fortunate to have a powerful troubleshooting tool which allows us to packet capture network traffic in an ESXi hypervisor. For this blog, I'm going to use an Hands On Lab (HOL) from the VMware website to demonstrate the process of doing a quick live packet capture. The output of the packet capture will be sent to the shell of the hypervisor. There are ways to sent the network traffic to a file so it can be extracted and analyzed by a packet analyzer tool, but that will be for another post. First, lets start with heading over to the HOL website:  https://labs.hol.vmware.com/ For this lab, I'm going to type NSX in the search bar to find
Read more