Correctly adding MTU for NAS fileshares

When adding NFS datastores in ESXi, if NAS is used with jumbo frames, remember to set the MTU correctly at all the levels:

  • NAS interfaces
  • NAS switch ports
  • Intermediate switches [if any]
  • Uplink NIC switch port
  • vSwitch
  • vmk virtual interface

From an ESXi perspective, the latter two are added as follows.

Verify the NIC names:

~ # vmkchdev -l | grep -i nic
000:002:00.0 8086:100f 15ad:0750 vmkernel vmnic0
000:002:02.0 8086:100f 15ad:0750 vmkernel vmnic1
000:002:03.0 8086:100f 15ad:0750 vmkernel vmnic2
000:002:04.0 8086:100f 15ad:0750 vmkernel vmnic3
000:002:05.0 8086:100f 15ad:0750 vmkernel vmnic4
000:002:06.0 8086:100f 15ad:0750 vmkernel vmnic5

Verify the current MTUs:

~ # esxcfg-nics -l
Name    PCI           Driver      Link Speed     Duplex MAC Address       MTU    Description
vmnic0  0000:02:00.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:41 1500   Intel Corporation PRO/1000 MT Single Port Adapter
vmnic1  0000:02:02.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:4b 1500   Intel Corporation PRO/1000 MT Single Port Adapter
vmnic2  0000:02:03.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:55 1500   Intel Corporation PRO/1000 MT Single Port Adapter
vmnic3  0000:02:04.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:5f 1500   Intel Corporation PRO/1000 MT Single Port Adapter
vmnic4  0000:02:05.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:69 1500   Intel Corporation PRO/1000 MT Single Port Adapter
vmnic5  0000:02:06.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:73 1500   Intel Corporation PRO/1000 MT Single Port Adapter

~ # esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         128         3           128               1500    vmnic0

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  Management Network    0        1           vmnic0

Create a new switch, set the MTU, and the uplinks:

~ # esxcfg-vswitch -a vSwitch1
~ # esxcfg-vswitch -m 9000 vSwitch1
~ # esxcfg-vswitch -A NFS vSwitch1
~ # esxcfg-vswitch -L vmnic4 vSwitch1
~ # esxcfg-vswitch -L vmnic5 vSwitch1

~ # esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         128         3           128               1500    vmnic0

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  Management Network    0        1           vmnic0

Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch1         128         4           128               9000    vmnic4,vmnic5

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  NFS                   0        1           vmnic4,vmnic5

Add the NFS portgroup, either with DHCP or static IP:

~ # esxcfg-vmknic -a -i DHCP -m 9000 -p NFS
~ # esxcfg-vmknic -a -i 10.16.123.123 -n 255.255.255.0 -m 9000 -p NFS
~ # esxcfg-vmknic -l
Interface  Port Group/DVPort   IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type
vmk0       Management Network  IPv4      192.168.0.123                           255.255.255.0   192.168.0.255   00:0c:29:95:93:41 1500    65535     true    DHCP
vmk1       NFS                 IPv4      10.16.123.123                           255.255.255.0   10.16.123.255   00:50:56:7e:b1:88 9000    65535     true    STATIC

Add the NFS share:

~ # esxcfg-nas -a -o 10.16.123.100 -s /vol/someshare/somedir friendlyname_01
Connecting to NAS volume: friendlyname_01
friendlyname_01 created and connected.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: