jeremYprieS.com

I was just reading Scott’s posting regarding ESX Server and the Native VLAN.  It got me started on a post to describe my logic regarding vSwitch0 design for ESX host building. 

I can tell you that I suggest native VLAN use for many ESX installations.  I suggest this for ease of building the server.  It is not required and many network guys hate this.  I understand, but it is a convenience during the building process.  Depending on the environment, it may be a requirement (lights out data center for example).

A sample vSwitch0 network looks something like this:

The matching Cisco configuration would be something like this:

interface GigabitEthernet4/17
switchport trunk allowed vlan 15,254
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk native vlan 15
no ip address
spanning-tree portfast trunk

It is important to note that for all port groups that need VLAN 15, you must use VLAN 0 (native/un-tagged).  A VLAN either tagged or not tagged, but not both.  I am not sure if it is a Cisco thing or a VMware thing, but if you were to add a port group to this vSwitch and tag it VLAN 15, the traffic will not work.

Build Problems

I have been unsuccessful building an ESX host using kickstart on a trunk without the use of the native vlan.  The problem is that there is no vlanid option on the boot prompt.  After we read the ks file in, we might be able to immediately start tagging packets using the –vlanid= parameter on the network line in your ks file.  It would look something like this:

network --device eth4 --bootproto static --ip 192.168.15.107 --netmask 255.255.255.0 --gateway 192.168.15.1 --nameserver 192.168.18.5 --hostname esx01.whatever.com --addvmportgroup=0 --vlanid=15

As I understand it, immediately after reading in the ks file, the network stack will be reloaded with the settings configured here.  But you have to get to the file first. 

To deal with this, I see three options.

• Put the kickstart cfg file on local media (floppy, cdrom, usb, …).
• Use a “build port” temporarily.  Usually a port on the patch panel in the server rack that is an “access” port only used to build the host.  You will need to apply the tagging to the service console post build.
• Use the native VLAN option.

Of those options I typically prefer to use the native VLAN.  So far, I have not found it to be a problem.

Building is still frustrating

I have found that even with the native VLAN, I still have too many problems with network.  It’s more so that the build environment does not have many troubleshooting tools.  You have to rely on the messages on tty 3 and 4; if they don’t fly off your screen.  Then to rerun to build, you have to wait for post and such.  Also, usually waiting for a network person to check or change settings in between each attempt.

In an effort to minimize frustration and increase productivity, I now recommend using the Ultimate Deployment Appliance in VMware player or workstation on a laptop.   For me this virtual appliance ties some pieces together, such as a web server, dhcp and tftp for PXE.  It has limited ESX support, which is fine because I use the exact same %post scripts as I did in the past. 

Here is the high level build process:

• Take your laptop to the data center and plug a cat5 cable between your laptop and the back of the server.
• Hard code the IP on your laptop and UDA.  Put it on the same subnet as where the service console will be.  Be sure to undo this before plugging into the real network.  Also, be sure to turn off the dhcp server whenever you are not using it to be sure that you are not breaking anything on the real network.
• Create a “template” for your server in UDA
• PXE boot the new ESX server and select the template name.

Here is the best part.  After the build is complete, unplug your laptop and plug in the real network.  Now you can use the full suite of tools to troubleshoot the network.  You get fancy newfangled fancy tools like ping and such.

So, to go full circle, if you are using UDA with a laptop, you don’t really need to use the native VLAN for the service console.  If you want to tag, then post-build, you will need to add the VLAN tag for the service console.  I personally dislike manual tasks and prefer to use the native VLAN all the time.

jeremy esx3