Configuring and testing VXLAN within a nested environment

In this post I'm going to show you how I managed to get VXLAN working (Which is required for NSX) within my test environment that I'm currently using to learn more about NSX.  My homelab currently consists of:

3 x HP ML310e G8
1 x RouterBOARD RB751
1 x Linksys SRW2016 16 Gigabit Switch
1 x Synology DS412+

Within the Production cluster I run all my home production services but I also run a Nested ESXi lab that I previously used for vCNS and now NSX as per the screenshot below:

My Test cluster contains three hosts and the vCenter Server Appliance.  I installed NSX manager outside of this cluster as I didn't want it to take up the nested lab resources.  My Test cluster looks like this:

I've deployed a single NSX controller in this cluster as I'm not too bothered about redundancy and would rather save the resources for VM's.  So, now onto the blog post.  In order for VXLAN to work you need a dedicated VLAN and the MTU needs to be set to 1600 end to end.  First step was to enable jumbo frames on my Linksys SRW2016 switch.  This simply involved ticking a box and rebooting the switch.  This was a pain as I had to power down all VM's and hosts but I also took the opportunity to upgrade my Synology DS412+ to the latest firmware.  To enable jumbo frames on the Linksys switch you simply browse to the admin interface, select Admin and then tick the option next to Jumbo Frames and then reboot the switch:

Next step was to create a new VLAN dedicated for VXLAN traffic and ensure the MTU was set to 1600.  I use a RouterBOARD RB751 as my router / Layer3 switch which I highly recommend.  This was simply a case of creating a new VLAN, I used 15, setting the MTU to 1600 and assigning it to the bridge-local interface:

It was then a case of creating the same VLAN within the Linksys switch and passing this VLAN down the trunks to the three ESXi hosts.  Once the VLAN was presented to the three ESXi hosts I then needed to set the MTU on both the Production and Test cluster Distributed Virtual Switchs as by default they are set to 1500.  To do this simply go to Networking, right click on the DVS and select Edit Settings.  From there, select Advanced and then change the MTU to 1600

It's then a case of configuring VXLAN for the cluster using VLAN 15, ensuring the MTU is set to 1600 and configuring an IP pool for the VMK nics:

In order to test network connectivity from one host to another you can use the following command after SSH'ing into the host:

ping ++netstack=vxlan -d -s 1600 -I vmk5 <IP ADDRESS>

In my case when pining another host with the MTU set to 1600 it failed:

After reading this article by Kamau Wanguhu (Blog | Twitter) I found the the actual VXLAN packet size is 1572.  I then pinged another host with the new MTU size and it worked fine:

So, hopefully that's VXLAN all configured in my NSX Test cluster.  Big thanks to Dan Watson (Twitter) and Geordy Korte (Blog | Twitter) for their help.  I'm sure I'll be harassing you more and more  in the coming months.

Getting started with NSX

This week I've been on an internal training course with VMware on NSX.  In case you haven't heard of NSX, this is a network virtualisation solution offered by VMware (My Employer).  I've already deployed NSX in a nested lab but this blog post is just going to quickly show you my current setup and how to grant specific user accounts admin rights within NSX and also how to stop accidentally locking yourself out of access to vCenter with the distributed firewall.  Expect more blog posts on NSX as I become more familiar with it and use it in the lab.  Firstly, This is how my physical lab looks like which hosts all my home production VM's:

As you can see I run a nested environment consisting of three virtual ESXi hosts and the vCenter Server Appliance.  The NSX Manager is running outside of the test cluster as I didn't want to consume resourced within the nested environment.  The nested environment simply has a three hosts, an NSX controller and a few simple web servers:

Once you've deployed NSX manager and deployed your first controller your probably going to want to grant your admin account permissions within NSX so you don't have to keep logging is as the local admin of the vCenter Server Appliance (This is just the way I like to work).  To grant your admin account access, simply log into the web client and select Networking & Security from the menu option:

From the NSX menu screen click on NSX Managers:

Click on the NSX manager link that you want to make changes to:

Click on the Manage tab and then the Users tab and simply click the green plus:

Enter the user or group name of the active directory object that you wish to add and click Next:

Select the role for the user or group and click Next:

Set the scope limit and click Finish:

You should now be able to access NSX Manager as the user or the user of the group that you just added.  Another operation that you should probably perform is to exclude vCenter from having any policies being applied to it as you can quite easily lock yourself out.  In my case it shouldn't be an issue as the vCenter Appliance does not reside in the same cluster as the one I'm going to be applying policies to but if yours does then simply browse to the NSX Manager, select Manage and then Exclusion List:

Click on the green plus and then form the drop down menu, select your vCenter server object.  As mentioned, in my case the vCenter does reside in the same cluster so I should be OK.  Once added simply click OK:

So, that's as far as I am at the moment.  I need to make some physical changes to my switch to allow jumbo frames (or MTU of 1600) for the VXLAN traffic and unfortunately I can't do that remotely without taking down the entire lab and my access so I'll be doing that over the weekend.

Great North Run 2014 Sponsorship

It’s that time again when everyone is asking for sponsorship for the Great North Run in September so I National Autistic Society.  I already have my place secured so I don’t need to raise a specific amount to guarantee entry.  I’m doing this more for personal reasons as a great friend’s son was recently diagnosed with Autism and it also runs in my family.  I’m also doing this for a little bit of financial motivation (More on that later).  I’ve always wanted to complete the Great North Run in under 1h 30m but have failed twice so far, once by around 7 minutes and last year by around 2.  In my defence last year I was training for a marathon and just used the Great North Run as another training session.  If you know me I’m definately not built for running but do have the knack and more importantly, enjoy it.  In order to complete this half marathon distance in under 1h 30m involves an average pace of 6m 51s and currently my last 5 mile run averaged around 7m 40s so I’ve got some serious training to do.

So, what is the financial motivation for this?  Well,  For every £100 donated I’ll stump up £50 (Up to a maximum of £250) if I fail to beat 1h 30m.  I’ll be going off the official time by the Great North Run organisers and will post my time up when finished.  So, to sum it all up:

If I get £300 worth of donations and I fail, then I’ll top that up with £150
If I get £700 worth of donations and I fail, then I’ll top that up with £250
If I get £130 worth of donations and I fail, then I’ll top that up with £100

You get the idea.  So dig deep everyone and pray that I fail.  Once again, for the people who know me, I hate loosing and I’m tight so bring it on!!

So, if you'd link to sponsor me then just follow the URL below and leave a "Hope you fail" message:

http://www.justgiving.com/Michael-Armstrong5

Thanks for taking the time to read this and hopefully donating.

Michael

North East VMUG - 07/08/2014

Registration for the next North East VMUG event is now open and you can register here.  The event will be held on Thursday 7th August and will now take place at the International Centre for Life.  There are two community sessions planned as well as sessions from VMware on the vCloud Hybrid Service and PernixData on their FVP solution.  The agenda will be as follows:

11:00 - EUC Round Table
12:00 - Lunch and Registration
12:30 - Welcome and Agenda - VMUG Leadership
12:45 - Community Presentation - Duncan Simpson
13:45 - Pernix Data Presentation
14:30 - Break
14:45 - Community Presentation - James Rankin
15:45 - vCHS - Dave Hill
16:45 - Closing Statement and Raffle
17:00 - vBeers

Big thanks to PernixData and Zerto for sponsoring these events so they can be free to attend.

Gold Sponson:

Silver Sponson: