Monthly Archives: November 2013
Thanks once again to Tom Verhaeg for this great scenario.
The voice team has recently setup Cisco Unity. The VoIP administrator sends you an e-mail. To comply with Cisco best practices, the Cisco Unity VM needs to have CPU affinity set. You really don’t like this, but the VoIP administrator and your boss insist. Make it happen……..
Damn, this really isn’t a fun thing to do. CPU affinity restricts a VM only to run on specific cores / processors that you specify. There may be some requirements for this (such as the above), but overall you shouldn’t do it. This breaks NUMA architecture, and more important, Fully Automated DRS! To support this, the DRS level should either be manual or partially automated.
The process itself isn’t that complicated. Edit the settings of the VM and go to the resources tab. Under advanced CPU, you find the option for CPU affinity.
If you do not see the Scheduling Affinity piece on a DRS-Cluster host, you are running DRS in fully automated mode. You can set DRS to manual for this VM by going to the cluster settings, and under DRS select Virtual Machine options. Set the DRS mode for this VM to either disabled, manual or partially automated.
Your recent work on the new portgroup was top notch! Now, the network administrators have some new requirements. You currently use one vNIC for the DvS. A second pNIC has been connected to the network and you have been tasked with adding it to the DvS. Also ensure that the DvS_StorageNetwork Port Group only uses the new pNIC and does VLAN tagging on VLAN ID 20.
Another networking objective. Whoohoo! Allright, let us first check out the current network adapters available on the host:
Allright, so vmnic2 is the one that we can add to the DvS_AMS01. Go over to the networking view (Ctrl + Shift + N) and edit the settings of your DvS. We first need to check if the DvS allows for 2 uplinks, instead of just 1.
And check this out! It’s still set to 1. This is a good one to remember for the exam, on the DvS object itself, you configure the maximum number of physical adapters (also called uplink ports) per host. So set that one to 2 and let’s continue with adding vmnic2 to the DvS.
Since the host is already connected to the DvS, click the DvS and select Manage Hosts. You will find your host, and you can add the second nic.
You could also do this from the hosts and clusters view, do whatever works for you.
Now that we have added that pNIC to the DvS, we need to create the DvS_StorageNetwork port group. Remember that we need to do VLAN tagging on VLAN ID 20 here. Create the new port group now, it’s settings should look like this:
Now, for the last part: As ESXi does load balancing by default (originating port ID based) we will now have load balancing on the DvS_ProductionNetwork, which is great, but not what we need for the Storage Network.
Open up the settings of that port group and go to the Teaming and Failover section.
Both uplink ports are now under Active Uplinks. Let’s review real quick what the options are:
Active Uplinks – actively being used for traffic flow
Standby Uplinks – will only become active until a failure occurs on one of the active uplinks
Unused Uplinks – this adapter will never be used for this port group
We need to ensure that it will never use this uplink, so move the dvUplink1 over to the Unused Uplinks. It should then look like this:
First off I want to thank Tom Verhaeg (blog/twitter) for providing this scenario. Tom had gotten in contact with myself and wanted to do what he can to help our with the 8 weeks of #VCAP series as he is going through a similar type process as me in studying for the VCAP5-DCA. So props to Tom for taking the time and initiative to give back. Hopefully we see more from him in the coming weeks! Even better for myself as I can run through some scenarios that I didn't make up 🙂 Be sure to follow Tom on Twitter and check out his blog Thanks for the help Tom!!!
Your company leverages the full Enterprise Plus licensing and has set up a Distributed vSwitch. Recently, the number of ports needed on a particular portgroup exceeded the number configured. You are tasked with creating a new Portgroup, called DvS_ProductionNetwork which only connects the running VM’s and also functions when vCenter is down.
Off we go again. So, let’s recall. There are 3 different options of port binding on a DvS.
Static binding – Which creates a port group with a manual set number of ports. A port is assigned whenever a vNIC is added to a VM. You can connect a vNIC static binding only through vCenter.
Dynamic binding (Deprecated in vSphere 5.0!) – A port is assigned to a vNIC when the VM is powered on, and it’s vNIC is in a connected state. You can connect this dynamic binding only through vCenter.
Empheral binding – A port is assigned to a vNIC when the VM is powered on, and it’s vNIC is in a connected state. This binding method allows the bypass of vCenter, allowing you to manage virtual machine networking when vCenter is down.
So, that’s the one we need! Empheral binding! Luckily, it’s quite simple to configure. Hop over to the networking inventory (Ctrl + Shift + N) and create the new port group. Give it a name and leave the number of ports on the default of 128.
Now edit the settings of this port group, and select the Empheral binding under the port binding dropdown. Also note, that the number of ports is greyed out now.
Have you ever wondered what your plan of attack will be in the event you need to recover your virtual infrastructure and the applications running on it? How long before users will be able to resume their work? Well, PHD Virtual, makers of PHD Virtual Backup have put the power in your hands to figure out just what that number is. Today they have released a free, yes FREE tool dubbed the RTA calculator which can provide visibility into your organizations recover time will be.
The tool itself runs on Windows, and simply points to your vCenter instance. From there you have a wizard driven interface where you select which VMs you would like to analyze and set a boot order. Now the magic happens. RTA will take a snapshot of the selected VMs and utilizes link clones to create a copy of of these. Lastly, it calculates the total time it will take for you to boot that group of VMs, leaving you with a nice estimate on your RTA (Recovery Time Actual).
You can go and grab your free copy of the RTA calculator here – and for those that may be interested in seeing it in action there's a pretty cool video embedded below.
Hopefully you all enjoyed the last scenario based post because you are about to get another one 🙂 Kind of a different take on covering the remaining skills from the storage section, section 1. So, here we go!
A coworker has come to you complaining that every time he performs storage related functions from within the vSphere client, VMware kicks off these long running rescan operations. He's downright sick of seeing them and wants them to stop, saying he will rescan when he feels the need to, rather than having vSphere decide when to do it. Make it happen!
So, quite the guy your coworker, thinking he's smarter than the inner workings of vSphere but luckily we have a way we can help him. And also the functions we are going to perform are also part of the VCAP blueprint as well – coincidence? Either way, the answer to our coworkers prayers is something called vCenter Server storage filters and there are 4 of them, explained below…
RDM Filter (config.vpxd.filter.rdmFilter) – filters out LUNs that are already mapped as an RDM
VMFS Filter (config.vpxd.filter.vmfsFilter) – filters out LUNs that are already used as a VMFS datastore
Same Hosts and Transports Filter (config.vpxd.filter.sameHostsAndTransporstFilter) – Filters out LUNS that cannot be used as a datastore extent
Host Rescan Filter (config.vpxd.filter.hostRescanFilter) – Automatically rescans storage adapters after storage-related management functions are performed.
As you might of concluded it's the Host Rescan Filter that we will need to setup. Also, you may have concluded that these are advanced vCenter Server settings, judging by the config.vpxd prefixes. What is conclusive is that all of these settings are enabled by default – so if we need to disable one, such as the Host Rescan Filter, we will need to set the corresponding key to false. Another funny thing is that we won't see these setup by default. Basically they are silently enabled. Anyways, let's get on to solving our coworkers issue.
Head into the advanced settings of vCenter Server (Home-vCenter Server Settings->Advanced Options). From here, disabling the host rescan filter is as easy as adding the config.vpxd.filter.hostRescanFilter and false values to the text boxes near the bottom of the screen and clicking 'Add' – see below
You work for the mayors office in the largest city in Canada. The mayor himself has told you that he installed some SSD into a host last night and it is showing as mpx.vmhba1:C0:T0:L0 – but not being picked up as SSD! You mention that you think that is simply SAS disks but he persists it isn't (what is this guy on crack :)). Either way, you are asked if there is anything you can do to somehow 'trick' vSphere into thinking that this is in fact an SSD.
Ok, so this one isn't that bad really, a whole lot of words for one task. Although most SSD devices will be tagged as SSD by default there are times when they aren't. Obviously this datastore isn't an SSD device, but the thing is we can tag it as SSD if we want to. To start, we need to find the identifier of the device we wish to tag. This time I'm going to run esxcfg-scsidevs to do so (with -c to show a compact display).
From there I'll grab the UUID of the device I wish to tag, in my case mpx.vmhba1:C0:T0:L0 – (crazy Rob Ford). Now if I have a look at that device with the esxcli command I can see that it is most certainly not ssd.
esxcli storage core device list -d mpx.vmhba1:C0:T0:L0
esxcli storage nmp device list -d mpx.vmhba1:C0:T0:L0
esxcli storage nmp satp rule add -s VMW_SATP_LOCAL -d mpx.vmhba1:C0:T0:L0 -o enable_ssd
And from here we need to reclaim the device
esxcli storage core claiming reclaim -d mpx.vmhba1:C0:T0:L0
And, another look at our listing out of the device should now show us that we are dealing with a device that is SSD.
esxcli storage core device list -d mpx.vmhba1:C0:T0:L0
And that's all for now 🙂
So my 8 weeks of #VCAP is quickly turning into just under 4 weeks of #VCAP so as I attempt to learn and practice everything on the blueprint you might find that I'm jumping around quite a bit. Also, I thought I would try presenting myself with a scenario with this post. Now all of the prep for the scenario is made by myself, therefore it's a pretty simple thing for me to solve, but none the less it will help get me into the act of reading a scenario and performing the tasks that are on it. So, this post will cover a bunch of random storage skills listed in Objective 1 of the blueprint – without ado, the scenario
Let's say we've been tasked with the following. We have an iSCSI datastore (iSCSI2) which utlizes iSCSI port bonding to provide multiple paths to our array. We want to change the default PSP for iSCSI2 from mru to fixed, and set the preferred path to travel down CO:T1:L0 – only one problem, C0:T1:L0 doesn't seem to be available at the moment. Fix the issues with C0:T1:L0 and change the PSP on iSCSI2 and set the preferred path.
Alright, so to start this one off let's have a look first why we can't see that second path to our datastore. If browsing through the GUI you aren't even seeing the path at all, the first place I would look at is claimrules (now how did I know that 🙂 ) and make sure that the path isn't masked away – remember the LUN Masking section. So ssh on into your host and run the following command.
esxcli storage core claimrule list
As you can see from my output lun masking is most certainly the cause of why we can't see the path. Rule 5001 loads the MASK_PATH plugin on the exact path that is in question. So, do you remember from the LUN Masking post how we get rid of it? If not, we are going to go ahead and do it here again.
First step, we need to remove that rule. That's done using the following command.
esxcli storage core claimrule remove -r 5001
Now that its gone we can load that current list into runtime with the following command
esxcli storage core claimrule load
But we aren't done yet! Instead of waiting for the next reclaim to happen or the next reboot, let's go ahead and unclaim that path from the MASK_PATH plugin. Again, we use esxcli to do so
esxcli storage core claiming unclaim -t location -A vmhba33 -C 0 -T 1 -L 0
And rescan that hba in question – why not just do it via command line since we are already there…
And voila – flip back into your Manage Paths section of iSCSI2 and you should see both paths are now available. Now we can move on to the next task, which is switching the PSP on iSCSI2 from MRU to Fixed. Now we will be doing this a bit later via the command line, and if you went into the GUI to check your path status, and since we are only doing it on one LUN we probably can get away with simply changing this via the vSphere Client. Honestly, it's all about just selecting a dropdown at this point – see below.
I circled the 'Change' button on this screenshot because it's pretty easy to simply select from the drop down and go and hit close. Nothing will happen until you actually press 'Change' so don't forget that. Also, remember, PSP is done on a per-host basis. So if you have more than one host and the VCAP didn't specify to do it on only one host, you will have to go and duplicate everything you did on the other host. Oh, and setting the preferred path is as easy as right-clicking the desired path and marking it as preferred. And, this scenario is completed!
The storage team thanks you very much for doing that but requirements have changed and they now wish for all of the iSCSI datastores, both current and any newly added datastores, to utilize the Round Robin PSP. How real life is that, people changing their mind 🙂
No problem you might say! We can simply change the PSP on each and every iSCSI datastore – not a big deal, there's only three of them. Well, you could do this, but the question specifically mentions that we need to have the PSP set to Round Robin on all newly added iSCSI datastores as well, so there's a bit of command line work we have to do. And, since we used the vSphere Client to set the PSP in the last scenario, we'll do it via command line in this one.
First up, let's switch over our existing iSCSI datastores (iSCSI1, iSCSI2, iSCSI3). To do this we will need their identifier which we can get from the GUI, however since we are doing the work inside the CLI, why not utilize it to do the mappings. To have a look at identifiers and their corresponding datastore names we can run the following
As you can see there are three datastores we will be targeting here. The identifier that we need will be the first string field listed beginning with t10 and ending with :1 (although we don't need the :1). Once you have the string identifier of the device we want to alter we can change its' PSP with the following command.
esxcli storage nmp device set -d t10.FreeBSD_iSCSI_Disk______000c299f1aec010_________________ -P VMW_PSP_RR
So, just do this three times, once for each datastore. Now, to handle any newly added datastores to defaulr to round robin we need to first figure out what SATP the iSCSI datastores are utilizing, then associate the VMW_PSP_RR PSP to it. We can use the following command to see which SATP is associated with our devices.
esxcli storage nmp device list
As you can see, our iSCSI datastores are being claimed by the VMW_SATP_DEFAULT_AA SATP. So, our next step would be to associate the VMW_PSP_RR PSP with this SATP – I know, crazy acronyms! To do that we can use the following command.
esxcli storage nmp satp set -s VMW_SATP_DEFAULT_AA -P VMW_PSP_RR
This command will ensure that any newly added iSCSI datastores claimed by the default AA SATP will get the round robin PSP.
At this point we are done this scenario but while I was doing this I realized there might be a quicker way to to change those PSP's on our existing LUNs. If we set associate our SATP with our PSP first then we can simply utilized the following command on each of our datastores to force them to change their PSP back to default (which will be RR since we just changed it).
esxcli storage nmp device set -d t10.FreeBSD_iSCSI_Disk______000c299f1aec010_________________ -E
Of course we have to run this on each datastore as well – oh, and on every host 😉
Big Joe, your coworker just finished reading a ton of vSphere related material because his poor little SQL server on his iSCSI datastore just isn't cutting it in terms of performance. He read some best practices which stated that the max IOPs for the Round Robin policy should be changed to 1. He requested that you do so for his datastore (iSCSI1). The storage team has given you the go ahead but said not to touch any of the other datastores or your fired.
Nice, so there is really only one thing to do in this scenario – change our default max IOPs setting for the SCSI1 device. So, first off, let's get our identifier for SCSI1
Once we have our identifier we can take a look on the roundrobin settings for that device with the following command
esxcli storage nmp psp roundrobin deviceconfig get -d t10.FreeBSD_iSCSI_Disk______000c299f1aec000_________________
As we can see, the IOOperation Limit is 1000, meaning it will send 1000 IOPs down each path before switching to the next. The storage team is pretty adamant we switch this to 1, so let's go ahead and do that with the following command.
esxcli storage nmp psp roundrobin deviceconfig set -d t10.FreeBSD_iSCSI_Disk______000c299f1aec000_________________ -t iops -I 1
Basically what we define with the above command is that we will change that 1000 to 1, and specify that the type of switching we will use is iops (-t). This could also be set with a -t bytes and entering the number of bytes to send before switching.
So, that's basically it for this post! Let me know if you like the scenario based posts over me just rambling on about how to do a certain task! I've still got lots more to cover so I'd rather put it out there in a format that you all prefer! Use the comments box below! Good Luck!