Port Model
This section includes the following topics:
Viewing Ports from the VSE
The Cisco Nexus1000VE differentiates between virtual and physical ports on each of the VSEs. Figure 10-1 shows how ports on the Cisco Nexus1000VE switch are bound to physical and virtual VMware ports within a VSE.
Figure 10-1 VSE View of Ports
On the virtual side of the switch, three layers of ports are mapped together:
-
Virtual NICs—Three types of Virtual NICs are in VMware. The virtual NIC (vnic) is part of the VM and represents the physical port of the host that is plugged into the switch. The virtual kernel NIC (VTEP) is used by the hypervisor for management, VMotion, iSCSI, network file system (NFS), and other network access needed by the kernel. This interface carries the IP address of the hypervisor itself and is also bound to a virtual Ethernet port. The vswif (not shown) appears only in CoS-based systems and is used as the VMware management port. Each type maps to a virtual Ethernet port within the Cisco Nexus1000VE.
-
Virtual Ethernet Ports (VEth)—A vEth port is a port on the Cisco Nexus 1000V. The Cisco Nexus 1000V has a flat space of vEth ports 0..N. The virtual cable plugs into these vEth ports that are moved to the host running the VM.
Virtual Ethernet ports are assigned to port groups.
-
Local Virtual Ethernet Ports (lveth)—Each host has a number of local vEth ports. These ports are dynamically selected for vEth ports that are needed on the host.
These local ports do not move and are addressable by the module/port number method.
On the physical side of the switch, from bottom to top, is the following:
-
Each physical NIC in VMware is represented by an interface called a vmnic. The vmnic number is allocated during VMware installation, or when a new physical NIC is installed, and remains the same for the life of the host.
Viewing Ports from the VSM
Figure 10-2 shows the VSM view ports.
Figure 10-2 VSM View of Ports
Port Types
The following types of ports are available:
-
vEths can be associated with any one of the following:
– VNICs of a Virtual Machine on the ESX host.
– VTEPs of the ESX Host
– VSWIFs of an ESX COS Host.
-
Eths (physical Ethernet interfaces)—Correspond to the outside-trunk interface of the VSEs.
For more information about Layer 2 switching, see the
Cisco Nexus 1000VE Layer 2 Switching Configuration Guide
.
Layer 2 Switching Problems
This section describes how to troubleshoot Layer 2 problems and lists troubleshooting commands. This section includes the following topics:
Verifying a Connection Between VSE Ports
You can verify a connection between two vEth ports on a VSE.
Step 1 View the state of the VLANs associated with the port. If the VLAN associated with a port is not active, the port may be down. In this case, you must create the VLAN and activate it.
switch#
show vlan
v
lan-id
Step 2 View the state of the ports on the VSM.
switch#
show interface brief
Step 3 Display the ports that are present on the VSE, their local interface indices, VLAN, type (physical or virtual), port mode and port name.
switch#
module vse module-number execute vemcmd show port
The key things to look for in the output are as follows:
-
State of the port.
-
CBL.
-
Mode.
-
Attached device name.
-
The LTL of the port that you are trying to troubleshoot. It will help you to identify the interface quickly in other VSE commands where the interface name is not displayed.
-
Make sure that the state of the port is up. If not, verify the configuration of the port on the VSM.
Step 4 View the VLANs and port lists on a particular VSE.
switch#
module vse module-number execute vemcmd show bd
If you are trying to verify that a port belongs to a particular VLAN, make sure that you see the port name or LTL in the port list of that VLAN.
Verifying a Connection Between VSEs
You can verify a connection between vEth ports on two separate VSEs.
Step 1 Log in to the upstream switch and make sure that the port is configured to allow the VLAN that you are looking for.
switch
# show running-config interface gigabitEthernet 1/38Building configuration... Current configuration : 161 bytes interface GigabitEthernet1/38 description Srvr-100:vmnic1 switchport trunk allowed vlan 1,60-69,231-233
As this output shows, VLANs 1,60-69, 231-233 are allowed on the port. If a particular VLAN is not in the allowed VLAN list, make sure to add it to the allowed VLAN list of the port.
Isolating Traffic Interruptions
You can isolate the cause for no traffic passing across VMs on different VSEs.
Step 1 Inside the VM, verify that the Ethernet interface is up.
ifconfig –a
If not, delete that NIC from the VM, and add another NIC.
Step 2 Using any sniffer tool, verify that ARP requests and responses are received on the VM interface.
Step 3 On the upstream switch, look for the association between the IP and MAC address:
debug arp
show arp
ARP packet debugging is on 11w4d: RARP: Rcvd RARP req for 0050.56b7.3031 11w4d: RARP: Rcvd RARP req for 0050.56b7.3031 11w4d: RARP: Rcvd RARP req for 0050.56b7.4d35 11w4d: RARP: Rcvd RARP req for 0050.56b7.52f4 11w4d: IP ARP: rcvd req src 10.78.1.123 0050.564f.3586, dst 10.78.1.24 Vlan3002 11w4d: RARP: Rcvd RARP req for 0050.56b7.3031 Protocol Address Age (min) Hardware Addr Type Interface Internet 10.78.1.72 - 001a.6464.2008 ARPA Internet 7.114.1.100 - 0011.bcac.6c00 ARPA Vlan140 Internet 41.0.0.1 - 0011.bcac.6c00 ARPA Vlan410 Internet 7.61.5.1 - 0011.bcac.6c00 ARPA Vlan1161 Internet 10.78.1.5 - 0011.bcac.6c00 ARPA Vlan3002 Internet 7.70.1.1 - 0011.bcac.6c00 ARPA Vlan700 Internet 7.70.3.1 - 0011.bcac.6c00 ARPA Vlan703 Internet 7.70.4.1 - 0011.bcac.6c00 ARPA Vlan704 Internet 10.78.1.1 0 0011.bc7c.9c0a ARPA Vlan3002 Internet 10.78.1.15 0 0050.56b7.52f4 ARPA Vlan3002 Internet 10.78.1.123 0 0050.564f.3586 ARPA Vlan3002
Step 4 You have completed this procedure.
Layer 2 Switching Troubleshooting Commands
You can use the commands in this section to troubleshoot problems related to the Layer 2 MAC address configuration.
|
|
show mac address-table
|
Displays the MAC address table to verify all MAC addresses on all VSEs controlled by the VSM.
See
Example 10-1 on page 10-7
.
|
show mac address-table module
module-number
|
Displays all the MAC addresses on the specified VSE.
|
show mac address-table static
HHHH.WWWW.HHHH
|
Displays the MAC address table static entries.
See
Example 10-2 on page 10-8
.
|
show mac address-table address
HHHH.WWWW.HHHH
|
Displays the interface on which the MAC address specified is learned or configured.
-
For dynamic MAC addresses, if the same MAC address appears on multiple interfaces, each of them is displayed separately.
-
For static MAC addresses, if the same MAC address appears on multiple interfaces, only the entry on the configured interface is displayed.
|
show mac address-table static | inc veth
|
Displays the static MAC address of vEthernet interfaces in case a VSE physical port learns a dynamic MAC addrress and the packet source is in another VSE on the same VSM.
See
Example 10-3 on page 10-8
.
|
show running-config vlan
vlan-id
|
Displays VLAN information in the running configuration.
|
show vlan
[
all-ports
|
brief
|
id
vlan-id
|
name
name
|
dot1q tag native
]
|
Displays VLAN information as specified. See
Example 10-4 on page 10-8
.
|
show vlan summary
|
Displays a summary of VLAN information.
|
show interface brief
|
Displays a table of interface states.
See
Example 10-5 on page 10-9
.
|
module vse module-numbe
r
execute vemcmd
show port
|
On the VSE, displays the port state on a particular VSE.
This command can only be used from the VSE.
See
Example 10-6 on page 10-9
.
|
module vse module-numbe
r
execute vemcmd show bd
|
For the specified VSE, displays its VLANs and their port lists
.
See
Example 10-7 on page 10-10
.
|
module vse module-number execute vemcmd show trunk
|
For the specified VSE, displays the VLAN state on a trunk port.
-
If a VLAN is forwarding (active) on a port, its CBL state should be 1.
-
If a VLAN is blocked, its CBL state is 0.
See
Example 10-8 on page 10-11
.
|
module vse module-number execute vemcmd show l2 vlan-id
|
For the specified VSE, displays the VLAN forwarding table for a specified VLAN.
See
Example 10-9 on page 10-11
.
|
show interface
interface_id
mac
|
Displays the MAC addresses and the burn-in MAC address for an interface.
|
Example 10-1
show mac address-table Command
Note The Cisco Nexus 1000V MAC address table does not display multicast MAC addresses.
Tip The “Module” indicates the VSE on which this MAC address is seen.
The “N1KV Internal Port” refers to an internal port created on the VSE. This port is used for control and management of the VSE and is not used for forwarding packets.
switch# show mac address-table VLAN MAC Address Type Age Port Mod ---------+-----------------+-------+---------+------------------------------+--- 1 0002.3d23.7802 static 0 N1KV Internal Port 3 1 0002.3d33.7802 static 0 N1KV Internal Port 3 1 0002.3d43.7802 static 0 N1KV Internal Port 3 1 0002.3d63.7802 static 0 N1KV Internal Port 3 1 0002.3d83.7802 static 0 N1KV Internal Port 3 222 0050.56b8.7584 static 0 Veth2 3 222 d48c.b5bc.fe01 dynamic 0 Eth3/1 3 223 0050.56b8.0375 static 0 Veth1 3 3968 0002.3d83.7802 static 0 N1KV Internal Port 3 3970 0002.3d83.7802 static 0 N1KV Internal Port 3 3971 0002.3d83.7802 static 0 N1KV Internal Port 3 3972 0002.3d83.7802 static 0 N1KV Internal Port 3 1 0002.3d23.7803 static 0 N1KV Internal Port 4 1 0002.3d33.7803 static 0 N1KV Internal Port 4 1 0002.3d43.7803 static 0 N1KV Internal Port 4 1 0002.3d63.7803 static 0 N1KV Internal Port 4 1 0002.3d83.7803 static 0 N1KV Internal Port 4 222 0050.56b8.8ce8 static 0 Veth3 4 223 0050.56b8.99b6 static 0 Veth4 4 3968 0002.3d83.7803 static 0 N1KV Internal Port 4 3970 0002.3d83.7803 static 0 N1KV Internal Port 4 3971 0002.3d83.7803 static 0 N1KV Internal Port 4 3972 0002.3d83.7803 static 0 N1KV Internal Port 4
Example 10-2 show mac address-table address Command
Tip This command shows all interfaces on which a MAC is learned dynamically.
In this example, the same MAC appears on Eth3/1 and Eth4/1.
switch# show mac address-table address 0050.568d.5a3f VLAN MAC Address Type Age Port Module ---------+-----------------+-------+---------+------------------------------+--------- 342 0050.568d.5a3f dynamic 0 Eth3/3 3 342 0050.568d.5a3f dynamic 0 Eth4/3 4
Total MAC Addresses: 1
Example 10-3 show mac address-table static | inc veth Command
switch# show mac address-table static | inc veth 460 0050.5678.ed16 static 0 Veth2 3 460 0050.567b.1864 static 0 Veth1 4
Example 10-4 show vlan Command
Tip This command shows the state of each VLAN created on the VSM.
---- -------------------------------- --------- ------------------------------- 1 default active Eth3/1, Eth4/1 ---- -------------------------------- --------- ------------------------------- ---- -------------------------------- --------- ------------------------------- ------------------------------------------------------------------------------- Primary Secondary Type Ports ------- --------- --------------- -------------------------------------------
Example 10-5 show interface brief Command
switch# show interface brief -------------------------------------------------------------------------------- Port VRF Status IP Address Speed MTU -------------------------------------------------------------------------------- mgmt0 -- up 172.23.232.163 1000 1500 -------------------------------------------------------------------------------- Ethernet VLAN Type Mode Status Reason Speed Port -------------------------------------------------------------------------------- Eth3/1 1 eth trunk up none 10G Eth4/1 1 eth trunk up none 10G -------------------------------------------------------------------------------- Vethernet VLAN/ Type Mode Status Reason MTU Module -------------------------------------------------------------------------------- Veth1 223 virt access up none 1500 3 Veth2 222 virt access up none 1500 3 Veth3 222 virt access up none 1500 4 Veth4 223 virt access up none 1500 4 -------------------------------------------------------------------------------- Port VRF Status IP Address Speed MTU -------------------------------------------------------------------------------- control0 -- up -- 1000 1500 NOTE : * Denotes ports on modules which are currently offline on VSM
Example 10-6 module vse
module-
number
execute vemcmd show port Command
Tip Look for the state of the port.
siwtch# module vse 3 execute vemcm show port LTL VSM Port Admin Link State PC-LTL SGID Vem Port Type ORG svcpath Owner 21 Eth3/1 UP UP F/B* 0 eth1 0 0 dpdk-outside 53 Veth2 UP UP FWD 0 test-vm1.eth1 0 0 test-vm1 54 Veth1 UP UP FWD 0 test-vm2.eth1 0 0 test-vm2 * F/B: Port is BLOCKED on some of the vlans. One or more vlans are either not created or not in the list of allowed vlans for this port. Please run "vemcmd show port vlans" to see the details.
Example 10-7 module vse
module-numbe
r
execute vemcmd show bd Command
Tip If a port belongs to a particular VLAN, the port name or LTL should be in the port list for the VLAN.
switch# module vse 3 execute vemcmd show bd BD 1, vdc 1, vlan 1, swbd 1, table-id 0, 1 ports, "" BD 2, vdc 1, vlan 3972, swbd 3972, table-id 0, 0 ports, "" BD 3, vdc 1, vlan 3970, swbd 3970, table-id 0, 0 ports, "" BD 4, vdc 1, vlan 3968, swbd 3968, table-id 0, 1 ports, "" BD 5, vdc 1, vlan 3971, swbd 3971, table-id 0, 1 ports, "" BD 6, vdc 1, vlan 222, swbd 222, table-id 0, 2 ports, "" BD 7, vdc 1, vlan 220, swbd 220, table-id 0, 1 ports, "" BD 8, vdc 1, vlan 221, swbd 221, table-id 0, 1 ports, "" BD 9, vdc 1, vlan 223, swbd 223, table-id 0, 2 ports, ""
Example 10-8 module vse
module-number
execute vemcmd show trunk Command
Tip If a VLAN is active on a port, its CBL state should be 1.
If a VLAN is blocked, its CBL state is 0.
switch# module vse 3 execute vemcmd show trunk Trunk port 6 native_vlan 1 CBL 1 vlan(1) cbl 1, vlan(3972) cbl 1, vlan(3970) cbl 1, vlan(3968) cbl 1, vlan(3971) cbl 1, vlan(222) cbl 1, vlan(220) cbl 1, vlan(221) cbl 1, vlan(223) cbl 1, vlan(224) cbl 1, vlan(225) cbl 1, vlan(226) cbl 1, vlan(227) cbl 1, vlan(228) cbl 1, vlan(229) cbl 1, Trunk port 16 native_vlan 1 CBL 1 vlan(1) cbl 1, vlan(3972) cbl 1, vlan(3970) cbl 1, vlan(3968) cbl 1, vlan(3971) cbl 1, vlan(222) cbl 1, vlan(220) cbl 1, vlan(221) cbl 1, vlan(223) cbl 1, vlan(224) cbl 1, vlan(225) cbl 1, vlan(226) cbl 1, vlan(227) cbl 1, vlan(228) cbl 1, vlan(229) cbl 1, Trunk port 21 native_vlan 1 CBL 0 vlan(222) cbl 1, vlan(220) cbl 1, vlan(221) cbl 1, vlan(223) cbl 1, vlan(224) cbl 1, vlan(225) cbl 1, vlan(226) cbl 1, vlan(227) cbl 1, vlan(228) cbl 1, vlan(229) cbl 1, switch# module vse 3 execute vemcmd show l2 switch# module vse 3 execute vemcmd show l2 222 Bridge domain 6 brtmax 4096, brtcnt 2, timeout 300 Flags: P - PVLAN S - Secure D - Drop R - Router-mac Type MAC Address LTL timeout Flags PVLAN Dynamic d4:8c:b5:bc:fe:01 21 1 Static 00:50:56:b8:75:84 53 0
Example 10-9 module vse
module-number
execute vemcmd show l2 Command
~ # module vse 5 execute vemcmd show l2 Bridge domain 115 brtmax 1024, brtcnt 2, timeout 300
Dynamic MAC 00:50:56:bb:49:d9 LTL 16 timeout 0
Dynamic MAC 00:02:3d:42:e3:03 LTL 10 timeout 0
Limitations and Restrictions
A syslog is generated if one of the following configurations exists when you try to disable automatic static MAC learning for MS-NLB because they do not support this feature:
-
PVLAN port
-
Ports configured with unknown unicast flood blocking (UUFB)
Disabling Automatic Static MAC Learning on a vEthernet Interface
You must disable automatic static MAC learning before you can successfully configure NLB on a vEthernet (vEth) interface.
In interface configuration mode enter the following commands:
switch(config)# int veth 1 switch(config-if)# no mac auto-static-learn
In port profile configuration mode enter the following commands:
switch(config)# port-profile type vethernet ms-nlb switch(config-port-prof)# no mac auto-static-learn
Checking Status on a VSM
If the NLB unicast mode configuration does not function, check the status of the Virtual Supervisor Module (VSM).
Confirm that the
no mac auto-static-learn
command is listed in the vEth and/or port profile configurations.
Step 1 In interface configuration mode, generate the VSM status.
switch(config-if)# show running-config int veth1 inherit port-profile vm59 description Fedora117, Network Adapter 2 vmware dvport 32 dvswitch uuid "ea 5c 3b 50 cd 00 9f 55-41 a3 2d 61 84 9e 0e c4"
Step 2 In port profile configuration mode, generate the VSM status.
switch(config-if)# show running-config port-profile ms-nlb port-profile type vethernet ms-nlb switchport access vlan 59
Checking the Status on a VSE
If the NLB unicast mode configuration does not function, check the status of the Virtual Ethernet Module (VSE). Check the following:
-
Confirm that the MS-NLB vEths are disabled.
-
Confirm that the MS-NLB shared-MAC (starting with 02:BF) is not listed in the Layer 2 (L2) MAC table.
Step 1 Generate the VSE status.
~ # vemcmd show port auto-smac-learning LTL VSM Port Auto Static MAC Learning
Step 2 Generate the Layer 2 MAC address table for VLAN 59.
Bridge domain 15 brtmax 4096, brtcnt 6, timeout 300
VLAN 59, swbd 59, ""
Flags: P - PVLAN S - Secure D - Drop
Type MAC Address LTL timeout Flags PVLAN Dynamic 00:15:5d:b4:d7:02 305 4 Dynamic 00:15:5d:b4:d7:04 305 25 Dynamic 00:50:56:b3:00:96 51 4 Dynamic 00:50:56:b3:00:94 305 5 Dynamic 00:0b:45:b6:e4:00 305 5 Dynamic 00:00:5e:00:01:0a 51 0