Tag Archives: Powercli

Using PowerCLI to set multipath policy based on array model

powerclilogoGoing through all of your hosts and setting your multipathing policy manually through either the C# or the vSphere Web Client can certainly be a trying task.  Sure, if you have 2 or 3 hosts it might be alright, but anything above that and things start to get pretty monotonous really fast!  Also we have all known that if you need to perform the same task more than once it’s always better to script it – Firstly it creates consistency across your environment, secondly it saves you time, and thirdly it helps the economy as you are out buying coffee when normally you would be at your desk clicking….

So all that leads me to the following script…  For the most part the script does two things – first it gets a list of all of your LUNS associated with a host or group of hosts, second, it sets up your desired multipathing policy on those luns based on the model of array presenting them as well as marks paths as preferred if applicable…  That being said, I’ve only included a few different models inside the script below, to see a full list of models you can run “esxcli storage nmp satp rule list”  from the esxcli to list all of the models and integrate them into the script as you see fit.

So first off you can see that the script accepts a parameter – if you pass the string fixall with the call of the script it will just continue without prompts, otherwise, it asks you for a y/n answer.   Also, when setting a path policy to fixed you might want to specify a preferred path – you can see that I’ve set this up as vmhba1, just be sure to customize the script to your liking for your environment…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
#fixall - if equals autoyes then don't prompt for y/n
param ($fixall)
 
$vcenter = "vcenterserver"
$username = "username"
$password = "password"
 
$hostcontainer = "cluster or datacenter containing hosts to check"
 
Write-Host "Connecting to vCenter Server..."
$success = connect-viserver $vcenter -username $username -Password $password
 
$vmhosts = get-vmhost -Location $hostcontainer
 
foreach ($vmhost in $vmhosts)
{
  Write-Host "Processing " $vmhost.Name"......" -ForegroundColor white -BackgroundColor Red
 
  #get all luns of type 'disk' from host
  $luns = get-ScsiLun -VMhost $vmhost -LunType disk
 
  foreach ($lun in $luns)
  {
    $multipath = $lun.MultipathPolicy
    $canname = $lun.CanonicalName
    $model = $lun.Model
    # if lun belongs to EVA
    if ($model -eq "HSV400")
    {
      $defaultpolicy = "roundrobin"
      if ($multipath -ne $defaultpolicy)
      {
        #get matching datastore name of lun
        $ds = Get-Datastore | Where-Object {$_.Extensiondata.Info.vmfs.Extent.DiskName -eq "$canname"}
        $dsname = $ds.Name
        Write-Host "$dsname" -foregroundcolor red -backgroundcolor yellow -nonewline; Write-Host " is on the EVA and is currently set to " -nonewline; Write-Host "$multipath" -ForegroundColor yellow -nonewline; Write-Host " where it should be " -nonewline; Write-Host "Round Robin" -ForegroundColor green -NoNewline; Write-Host " - Fix? " -NoNewline;
        if ($fixall -eq "autoyes")
        {
          $fix = "y"
        }
        else
        {
          $fix = Read-Host " y/[N]:"
        }
        if ($fix -eq "y")
        {
          $lun | set-ScsiLun -MultipathPolicy $defaultpolicy
          Write-Host "***Complete***" -Foregroundcolor green
        }
      }
    }
    #if lun belongs to an MSA
    elseif ($model -eq "MSA1000 VOLUME")
    {
      $defaultpolicy = "fixed"
      $prefPathHBA = "vmhba1"
      if ($multipath -ne $defaultpolicy)
      {
        #get corresponding datastore name of lun
        $ds = Get-Datastore | Where-Object {$_.Extensiondata.Info.vmfs.Extent.DiskName -eq "$canname"}
        $dsname = $ds.Name
        Write-Host "$dsname" -foregroundcolor red -backgroundcolor yellow -nonewline; Write-Host " is on the MSA and is currently set to " -nonewline; Write-Host "$multipath" -ForegroundColor yellow -nonewline; Write-Host " where it should be " -nonewline; Write-Host "Fixed" -ForegroundColor green -NoNewline; Write-Host " - Fix? " -NoNewline;
        if ($fixall -eq "autoyes")
        {
          $fix = "y"
        }
        else
        {
          $fix = Read-Host " y/[N]:"
        } 
        if ($fix -eq "y")
        {
          $lunpath = Get-ScsiLunPath -scsilun $lun | where-object {$_.ExtensionData.Adapter -like "*$prefPathHBA"}
          $lun | set-ScsiLun -MultipathPolicy $defaultpolicy -PreferredPath $lunpath -confirm:$false
        }
      }
    }
  }
}

Again I don’t claim to be a Powershell/PowerCLI scriptmaster, so if you see any performance enhancements, problems, suggestions, concerns please use the comments section below to let me know… – Anyone can write a script, but it takes a community to make it a great script.

My First vCenter Orchestrator Workflow – Part 4 – A look at the Powershell Plug-in

Orchestrate-FacilitateAlright!  So far throughout this series of posts we have installed, configured and setup vCenter Orchestrator; as well as created and tested the integration between both vCO and the vSphere Web Client.  So what’s next in my venture?  Well if you can remember back to Part 1 of the series I set out on a quest to develop a workflow that would take an un-configured host and go through a series of tasks to create datastores, rename networks, setup hostname, etc…  Basically configure it the way I wanted.  Now there are built-in workflows in vCO to do this, however I had already done this work inside of a PowerShell script – so my main goal was to get vCO to execute that script.  And this leads us to part 4!

So first off as I said above there are a ton of great built in workflows inside of vCO that can do pretty much anything.  Some things however, such as the ability to run a Powershell script, are missing!  Thankfully there is a thriving developers community built around vCO and it’s SDKs allowing third party vendors to release their own sets of workflows in the form of a plug-in.  So as you probably guessed by now we will need to install a plugin to handle our Powershell script execution.

This plugin is called the vCenter Orchestrator Plug-in for Microsoft Windows Powershell and can be found over on VMware’s site.  You will need to download this file.  Once you have your hands on the vmoapp file we need to go into the vCenter Orchestrator Configuration page (https://IP_OF_vCO:8283/) to install it.  Select Plug-ins from the left hand menu, then browse down to the Install new plug-in section.  Upload your file and click ‘Upload and install’, accept the EULA and away you go… If you browse back to the main Plug-ins page you will notice that it states the plugin will be installed at the next server startup; so if you wish you can do this task manually by going to Startup Options -> Restart Service.

vCO-InstallPlugin

At this point we can fire up our vCO client.  Having a look in the Library of workflows we can now see a PowerShell folder.  Expanding this out further let’s us see some of the workflows that come packaged with the plug-in.  So we now have a few configuration steps we need to go through before executing scripts with this plug-in.

First off we need setup a PowerShell host.  A PowerShell host is simply a Windows machine with Powershell installed located somewhere in your environment that will handle the storing and executing of the scripts.  There is a great article on all of the steps here but I will go through explaining them all as well.   Keep in mind this might not be the most secure of ways to go about everything – but this is simply a lab so I’m not too worried – for those concerned w/ security implications check out the full lengthy thrilling winrm read on the msdn 🙂   

So, once you have chosen which machine you would like to be your Powershell host go ahead and drop to a command prompt on that machine and run the following list of commands.

  • To create a winrm listener and open any required firewall ports
  • winrm quickconfig
  • To enable kerberos authentication
  • winrm set winrm/config/service/auth @{Kerberos=”true”}
  • Allow transfer of unencrypted data – told you wasn’t the most secure 🙂
  • winrm set winrm/config/service @{AllowUnencrypted=”true”}
  • Up the max memory per shell – I needed to do this to get things working
  • winrm set winrm/config/winrs @{MaxMemoryPerShellMB=”2048″}

And there we are!  Things should be setup fine now on our host end to allow the connections from our vCO server.  Now we need to configure kerberos on the vCO end of things.  To do this you will need to console into your vCO server and get on the command line there – hope your comfortable with vi 🙂

Basically we need to create a file under /opt/vmo/jre/lib/security called krb5.conf – to do so simply

cd /opt/vmo/jre/lib/security
vi krb5.conf

This file will need to look as follows – obviously change to fit your environment – if you happen to be using an default configuration of the autolab then you are good just to copy the following…

[libdefaults]
default_realm = LAB.LOCAL
udp_preferences_limit = 1
 
[realms]
LAB.LOCAL = {
kdc = dc.lab.local
default_domain = LAB.LOCAL
}
 
[domain_realms]
.LAB.LOCAL=LAB.LOCAL
LAB.LOCAL=LAB.LOCAL

Once you are done simply hit :wq to save and exit.  In order to grant vCO permission to access our newly created krb5 file you will also need to change ownership of the file using the following command

chown vco.vco krb5.conf

And, in order for changes to take affect, restart the vCO service – you can do this from the configuration page->Startup Options or simply

service vcod restart

Whew!  Ok!  Getting somewhere….Now we need to add the host into our configuration of the Powershell plug-in.   I find the easiest way to do this is to simply run the ‘Add a Powershell Host’ workflow however you can develop the script/code on the fly to add the host during your own workflow if you want.   The way I have documented however is as follows…

Browse to the PowerShell plug-in configuration by expanding Library->PowerShell->Configuration in the workflows view of your vCO Client.  Right click ‘Add a PowerShell host’ and select Start Workflow.

vCO-AddHost1

Here we need to add a few key pieces of information.  Enter a name for your PowerShell host – this can be whatever you wish, then enter the HOSTNAME of the host you setup (kerberos will not work with IP)   and click Next.

vCO-AddHost2

The only item that needs to be changed on the Host Type screen is Authentication; be sure Kerberos is selected and click ‘Next’.

vCO-AddHost3

In the User Credentials screen you have a couple of options in regards to Session mode.  Again since this is a lab environment I selected Shared Session, however if you were looking at a more production type deployment you might want to explore the Per User settings which will execute commands under the credentials of the current user connected to vCO.  Enter in some credentials with access to vCO and click ‘Submit’.  The workflow should kick off and you can watch it flow through the different elements.  When done, hopefully you have a nice green check mark beside it!

Now that we have successfully installed the Powershell plug-in and added a Powershell host lets go ahead and create a script on our Powershell host to run in order to make sure everything works.  Although my end goal is to completely configure a host –  for simplicity sake I’ll use a small snippet of the complete script.  So I have placed the following script on my PowerShell host to simply add a new VM Network to vSwith0.  As you can see the script is pretty simple, takes two parameters, a host and a location code.  The location code is used to represent uniquness in my environment and is actually used multiple times in the complete script for the hostname, datastores, etc….

1
2
3
4
5
6
param ($hosttoconfigure, $locationcode)
add-pssnapin VMware.VimAutomation.Core
$username = "Host Username"
$password = "Host Password"
$success = Connect-VIServer $hosttoconfigure -username $username -Password $password
Get-VirtualSwitch -Name vSwitch0 | New-VirtualPortGroup -Name "$locationcode-VM_Network" -confirm:$false

So let’s get started on our workflow to call this script.  Again, create a new workflow and call it what you will.   Once you are in the workflow editor move to the Inputs tab.  Here we will get the information from the user that we require in the script.  So go head and add a couple of inputs to the workflow, one being hosttoconfigure (Type VC.HostSystem) and locationcode (Type String).

vCO-SetupParameters

By selecting our host parameter and clicking the eye icon we can set the ‘Show in Inventory’ presentation property on the host input.  This will basically allow the user to browse through their vCenter to select the host when running the workflows, so might as well go ahead and do that as well.

vCO-PresentationProperty

The Powershell workflow that we will be using is called ‘Invoke and external script’ and requires 3 parameters; One, a Powershell Host which we have setup previously, the second is the script to execute, and the third is the arguments to pass to the script.

vCO-PS-SetupParameters

Once on the Schema tab go head and drag the ‘Invoke an external script’ workflow into yours.   When you see the parameter setup black bar pop up click ‘Setup’ and we can have a look at the parameters.

vCO-PS-PromoteParameters

Through this screen we can basically promote or link the parameters from the Script workflow to our main workflow.  You can see that I’ve done a few things.  One, I’ve set the host parameter to a value (same as creating an attribute) and browsed through my inventory to select my Powershell host that we created previously.  Secondly I manually inputted the path to my Powershell script in the externalScript parameter.  Thirdly I selected to skip the arguments parameter at this time.  I’ve done this because I need to pass the hostname as well as the locationcode parameter from my main workflow to this parameter which we will do with a separate element called a Scriptable task.  Go ahead and click ‘Promote’.

vCO-PS-ScriptableTask

So now we have our main input parameters setup (locationcode and hosttoconfigure) as well as our script setup and ready to go.  We just need to tackle that arguments parameter.  As I said we will do this with a separate Scriptable Task workflow so go ahead and drag that into your schema, just before your Invoke External Script.   A scriptable task is just that, kind of a blank canvas to do any sort of scripting (javascript) that you desire.

vCO-PS-ScriptableTask-SetupInVars

So let’s go ahead and click the pencil icon to edit our scriptable task.  There are few IN parameters that we are going to need inside of our script; The first is hosttoconfigure and the second is locationcode.

vCO-PS-ScriptableTask-SetupOutVars

Now move on to the OUT parameters.  We will need to create a new out parameter to store our arguments, so click the link for a new parameter and create a new workflow attribute called scriptArgs of type string.

Now we need to move to the Scripting tab.  This is where we can concatenate a property of the hosttoconfigure parameter and the locationcode to populate our newly created scriptArgs variable.  Honestly it’s a one-liner.  I’m sure there is a way to create this workflow without even requiring this whole Scriptable task but this is the way I got to to work (again, my FIRST vCO workflow).  Copy/Paste the following into your scripting tab.

scriptArgs = hosttoconfigure.Name + " " + locationcode;

Ok, you can close out of your Scriptable task workflow now.  We are almost there.  All that we have left to do is map the External Script workflow paramater arguments to our Scriptable task elements parameter scriptArgs.  Thankfully this is an easy thing to do.

vCO-PS-MapVars

Once again go into your External Script element and click on the Visual Binding tab.  Here, as shown above, we simply need to drag the scriptArgs in attribute over to the arguments in parameter.

vCO-PS-AllDONE

Guess what!  Your done!  If everything has been setup properly and I’ve done a good job at explaining what to do then you should be able to Save and Close your workflow and actually run it on a host (a test host of course).  The result – you should get a VM Network setup on vSwitch0 – oh and a pretty green checkmark beside your workflow 🙂

At this point I’m going to cut off this post since it’s getting pretty long.  That being said I’m not going to end the series!  I’m always looking at ways to better automate things and you know, having users type in that location code just isn’t good enough for me, a lot of room for error there; they could type it in wrong, do the same location twice, on and on and on…  And hey, I have a database already setup with my location codes in it and whether or not they have already been deployed.  So, stay tuned for Part 5 of my series where we will go through configuring the SQL plug-in and setting up our location input parameter to be populated by an SQL table – as well as update that same table on completion of our script.

My first vCenter Orchestrator Workflow

My wishlist for the vSphere Web Client .Next

Genie-1First off I want to state that by no means do I have anything to do with VMware and/or the development of any of the following.  Also, VMware has nothing to do with me and to my knowledge are not working on any of the following items, but you never know 🙂

So with that out of the way lets get started.  I’ve had enough time to use the web client provided with vSphere 5.1 to know that it is most certainly a powerful tool.  As of late I’ve been experimenting with adding my own vCenter Orchestrator workflows and executing these from the Web Client – This is, along with some other features of the web client that I have blogged about (tagging and searching),  very cool and very nice to have.  That being said, I’m a geek, and I’m never satisfied, always wanting more – so here’s what I call a wishlist for the WebClient.Next.

vCOps / Capacity IQ overprosioned VMs identification.

vm-elephantAlthough in many environments almost every single VM may be over provisioned I think it would still be nice to be able to quickly identify those that are heavily over provisioned (maybe by a metric? that the user defines).  So maybe the elephant is a bit much, but to have the VM icon change, or an alarm raised directly within the web client may help to drive home the importance of right sizing your VMs.  It may expose this information to more people, more application owners that may have access to vCenter but not vCOPs.  But hey, why stop there?  Why not make some recommendations when editing the said VMs settings on where that “right-size” is.  Maybe another little row on the RAM selection to identify vCOps recommendations, or maybe a simple note in the CPU selection window.

vcopsram

More vCOps integration

vcopsreports2I’m not going to lie, it has been fabulous being able to view those vCOps badges from within the web client.  The less I need to go to another tab the better 🙂  That being said I want more!!!  It would be nice to have some of the vCOps reports available from within the web client and mapped to inventory items contextually….  Just as we can now start a vCenter Orchestrator workflow by right-clicking on a cluster it would be nice to generate a capacity report in a similar fashion.

 

More vendor plugins

plugObviously this one isn’t on VMware, I believe that the foundation is already there, but I’d love to see more and more vendors and partners take advantage of it and develop plugins for the web client.  For instance, those backup vendors that have had integration within the c# client, as well as those that haven’t had any integration at all.  I can certainly see advantages to right clicking a VM and being able to directly add it to a backup job.  I understand that vDP already has this, but there are a lot of people using third party solutions….

***UPDATE***

The longer I wait to publish this the more of my items are being delivered.  Veeam has just announced their support and integration into the vSphere Web Client in the next release of Backup and Replication.

Update Manager and Single Host Management

Yup, this is one that I’m positive is on the list for almost everyone I know.  We certainly need a way to perform those Update Manager scans and remediations from within the Web Client.  Having to switch back and forth between clients, well, how do I put it, kinda sucks!  Also the ability to manage a single host.  I’m sure we are not the only customer in the world that has those one-off hosts that are not connected to a vCenter – it would certainly be nice to be able to have the same interface available to manage these as well.  This one I would assume is a shoe in as there will be no more C# client next release.

Tags into PowerCLI

powerclilogoOK OK, this one really has nothing to do with the web client but still, something I wanted to stress.  The ability to assign tags to inventory items in vSphere 5.1 is one of the most useful and practical features of the release in my opinion.  It’s easier to sort through massive amounts of VMs, tag them by service, location, sla, however you really want.  Certainly granting the ability to tag and retrieve VMs by tag from PowerCLI can only complement that feature as there are a lot of scripts I find myself wanting to write, but being held back by the absence of and tag cmdlets.

 

Speaking of Tags and Update Manager

Wouldn’t it be nice to remediate hosts based on how they have been tagged?  Hmm, OK, well, maybe that isn’t that useful…BUT, what would be cool is having the ability to schedule certain VMware Tools/Compatibility updates based on  a virtual machine tag.  I’ve had countless times where I’ve needed to move certain VMs into a temporary folder in order to have them participate in a VMware tools upgrade that following night, then, move them back and try to remember where they were. By simply tagging these VMs and running my scheduled tasks against the tags would make this process a little easier.

I’m sure I had more, however over the weeks I’ve been compiling this list I’ve just forgotten to write some of them down…  With that said, is there anything you are looking for in vSphere.Next?  Comments, concerns, opinions, rants, all are welcome below….

Creating Roles and adding Active Directory Users & Groups to the role through PowerCLI

As of late I’ve been working on a PowerCLI script to assist me with the deployment of around 50 or so ESXi hosts.  For the most part the PowerCLI cmdlets have been pretty self explanatory and pretty simple to use.  Really I just need to setup some DNS, Hostnames, Networks, Datastores, NTP settings, and permissions…you know, everything you would normally do when setting up a host.  The problem I’ve ran into is when I hit that permissions section.

Essentially what I wanted to do was create a new role containing only those permissions that I wanted then simply assign an active directory group to that role.  So the role part, not too bad, pretty simple really…I used the following…

New-VIRole -Name "Elevated VM User" -Privilege (Get-VIPrivilege -ID "VirtualMachine.Interact")
Get-VIRole "Elevated VM User" | Set-VIRole -AddPrivilege (Get-VIPrivilege -ID "ScheduledTask.Run")
Get-VIRole "Elevated VM User" | Set-VIRole -AddPrivilege (Get-VIPrivilege -ID "ScheduledTask.Create")
Get-VIRole "Elevated VM User" | Set-VIRole -AddPrivilege (Get-VIPrivilege -Name "Power")
Get-VIRole "Elevated VM User" | Set-VIRole -AddPrivilege (Get-VIPrivilege -Name "Maintenance")

Basically this just creates my new role (Elevated VM User) and adds a handful of permissions to it.  In order to find out the name for all the priveleges I simply just ran Get-VIPrivelege -Role ‘Admin’.

So now that I had my role created I just need to add an AD Account to that role.  A lot of the documentation led me to the New-VIPermission cmdlet – sounds easy, but running the following New-VIPermission -Role ‘Elevated VM User’ -Principal ‘DOMAIN\Username’ produced nothing but the following errors for me….

New-VIPermission : 9/12/2012 2:47:40 PM New-VIPermission Could not find VIAccount with name ‘Domain\Username’.
At line:1 char:17
+ New-VIPermission <<<< -Role ‘Admin’ -Principal ‘Domain\Username’
+ CategoryInfo : ObjectNotFound: (Domain\Username:String) [New-VIPermission], VimException
+ FullyQualifiedErrorId : Core_ObnSelector_SelectObjectByNameCore_ObjectNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.PermissionManagement.NewVIPermission

New-VIPermission : 9/12/2012 2:47:40 PM New-VIPermission Value cannot be found for the mandatory parameter Entity
At line:1 char:17
+ New-VIPermission <<<< -Role ‘Admin’ -Principal ‘Domain\Username’
+ CategoryInfo : NotSpecified: (:) [New-VIPermission], VimException
+ FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.PermissionManagement.NewVIPermission

Now it seams as if me passing the string of ‘Domain\Username’ is not satisfying whatever object it is that -Principal is looking for.  So after a many googling and twittering the following code is how I ended up assigning Domain\Username to the role I created.

$groupName = "Domain\Username"
$currhost = get-vmhost HOSTNAME | % {Get-View $_.Id}
$authmgr = Get-View AuthorizationManager-ha-authmgr
$perm = New-Object VMware.VIM.Permission
$perm.Principal = $groupName
$perm.group = $true
$perm.propagate = $true
$perm.RoleId = ($authmgr.RoleList | where {$_.Name -eq "Elevated VM User"}).RoleId
$authmgr.SetEntityPermissions($currhost.parent,$perm)

And this worked perfect!  As I’ve always said I’m sure there are a million ways to do something and if you think your way is more efficient or better than mine, please share!  This is my first crack at it so all suggestions and comments are always welcome 🙂

Getting rid of those VM Memory Limits (you know, the ones you didn’t know existed)

OK so here’s a quick one.  I know there are many other posts out there about this, i’ve come across multiple ones but again for my own documentation and for the mere fact that usually writing about something helps me to remember it I’m going to throw one more out into the mix.

I’ve occasionally come across a bunch of my VMs which seem for some reason to have a memory limit set on them.  BTW, vCheck is a great tool for reporting on these.   We all know the dangers of memory limits (swapping, ballooning, etc) and to tell you the truth I’m not sure why or how they got set.  I’m assuming I have a template somewhere within my environment which has a limit imposed on it.  Either way, there is a quick and easy way to get rid of those limits using PowerCLI.

First off I wanted to see which VMs had lmits imposed on them from within a certain cluster in my environment.  The following command does just that.

Get-VM -Location CLUSTERNAME | Get-VMResourceConfiguration 
| where {$_.MemLimitMB -ne '-1'} | Select VM,MemLimitMB

Furthermore, If you do have some VMs that you want to have a memory limit set on and not returned here and you have some soft of consistency within their name (ie. Veeam Replica’s) you can do the following to exclude them…

Get-VM -Location CLUSTERNAME | where {$_.Name -notlike '*Replica'} 
| Get-VMResourceConfiguration | where {$_.MemLimitMB -ne '-1'} 
| Select VM,MemLimitMB

You should be left with a listing of your VMs which have a memory limit set on them…

To set these VMs limit to unlimited, just pipe in the Set-VMResourceConfiguration cmdlet as follows

Get-VM -Location CLUSTERNAME | Get-VMResourceConfiguration 
| where {$_.MemLimitMB -ne '-1'} 
| Set-VMResourceConfiguration -MemLimitMB $null

Voila!  No more memory limits…as you can imagine you can do the same with CPULimitMhz.  FYI, if you want to learn more about PowerShell or PowerCLI or scripting you are in the wroooooonnnnggg place, I know what I know, but most examples I get from the following people and you should really follow Alan Renouf (blog/twitter), Luc Dekens (blog/twitter), Jake Robinson (blog/twitter), William Lam (blog/twitter) Eric Wright (blog/twitter), Josh Atwell (blog/twitter) or Hal Rottenberg (blog/twitter).  I’m sure I’m missing a few, these are just the ones I can remember off the top of my head, feel free to recommend some more CMDLET rock stars to me in the comments below…. 🙂

Practise makes perfect! More PowerCLI APC Powerchute Network Shutdown Goodness (Now with Power On!)

Picture this, a quaint little city in Eastern Ontario. It’s been at least a couple of months since we have really had any rain. My tomato plants are dying, my lawn is completely brown, but the datacentre, it’s chugging along nicely…Then it hits us, the first rainfall (of any real value) of the summer. Finally maybe things will start to grow again. Chatter around the office turns to the weather yet again, people are smiling, happy, can’t wait to get home see if maybe there was a chance that their grass might have turned a slight shade of green…and then, nothing but darkness and the faint sound of our neighbouring businesses generator kicking on…!

Are you kidding me? It rains once this summer and it knocks the power out? Wow! No big deal though right? We have our Powerchute Network Shutdown all configured to peel through and shut-down all of our physical hardware, and a while back I wrote this nifty little script to shutdown the virtual infrastructure, no problem!

And thus the title for this blog post – Practise makes Perfect! Turns out that I could have been a little more efficient in my script. Initially I was looping through hosts one by one shutting off VMs, then waiting, then checking, then going through for another pass, and then moving on to the next host… Well, with 10 minutes left on battery I don’t have the time to complete this process on our 8 production hosts, let alone an environment of any type of size. Needless to say the VMs and hosts that didn’t get the time of the day with the script fell hard. Thankfully, there was no corruption or issues resulting from it, but still, I don’t like things not going as planned . When I’m standing in the datacenter I like to see all of the blinky lights shut off before those air conditioners stop pumping out coolness.. So out of my failures I give you the new and improved shutdown script (now with Power ON). You can head straight here to pull down both the power off and power on scripts, but they are explained in a bit more detail below as there is a little bit of configuration to get the power on functionality to work….  As well a big thanks goes out to a fellow Toronto VMUGer (VMUGite ??) Eric Wright (blog / twitter ) and his ‘The Art of Shutdown‘ post a couple of months back.  Eric wrote a great script to shut things down and even has a great little ‘test’ mode in his version.  I got a few lines of code and ideas from Eric’s script so go check it out as well

Power Off the VMs

So first off is the shutdown script – you can get the complete version here.

Just a few notes regarding my updates to the script.

if ($keyword -ne $mysecret) 
{ 
    Write-Host "You haven't passed the proper detonation sequence...ABORTING THE SCRIPT" -ForegroundColor red
    exit
}

So first off you see here that if you call the script without a parameter or with a parameter that doesn’t match our secret keyword inside the script the whole thing aborts.  This is to stop us from simply double clicking this or accidenttally unleashing a disaster amongst ourselves.

Get-VM -Location $cluster | where-object {$_.PowerState -eq "PoweredOn" } | Select Name | Export-CSV c:\Scripts\Shutdown\PowereredOnVMGuests.csv

Another important addition to the script.  Basically this dumps a list of the powered on VMs to a csv file.  This will be used in the partner power on script once power has been restored.

The rest of the script is pretty straight forward.  Basically sets DRS to manual mode, disables HA, gracefully shuts down the VMs (any without VMware Tools installed get a hard shutdown), waits a couple of minutes and does another pass, then procedes to shutdown the hosts.  Again, the complete poweroff script can be downloaded here.

Power On the VMs

So all is good, VMs got powered off, but what happens when power comes back up.  Even if you do get your hosts powered back on they aren’t going to turn on your VMs automatically.  Now I know there is a start and stop with host setting but when you are in a DRS cluster this doesn’t do you much good as it is a host setting and VMs move between hosts.  Thus the power on script!  So this script will sit on the same box as your shutdown script.  It needs to be outside of your virtual infrastructure.  My vCenter Server is a physical box so I have placed both the shutdown and the power on scripts there.  Also, you will need to have this script be triggered on start-up for this machine.  This is simply do to the fact that I don’t want to be running any scripts in the middle of the night if power is restored…I’d rather walk in to a fully functional datacenter the next morning 🙂

The full script can be downloaded here but below are a few explanations of what happens….

if (Test-Path $filename)

The complete script is wrapped in this if statement.  Meaning if the power off dump file doesn’t exist then the script will not execute.

    while ((Get-Service vpxd).Status -ne "Running")
    {
        Write-Host "." -nonewline
        Sleep 2
        Write-Host "." -nonewline
        Sleep 2
        Write-Host "." -nonewline
        Sleep 2 
    }

This repeating loop basically waits for the vCenter Service to be started before the script can continue.  Since this script should be executed during startup we may have to spend some time here waiting for vCenter to get all its ducks in a row 🙂

    foreach ($iVM in $myImportantVMs)
    {
        Write-Host "Powering on $iVM ..." -nonewline
        Start-VM $iVM
        Sleep 5
        Write-Host "DONE" -Foregroundcolor Green   
    }

Before reading the dump file and getting into the thick of everything we have the option to power on some of the more important VMs first.  So if you have any mail servers, dhcp servers, dns servers, it would be a good idea to put them into $myImportantVMs to have them started first. Once these VMs are on a similar type function will power on the rest of the VMs in the csv file, one by one, every 5 seconds.  You can set the sleep command to whatever you want but 5 seems good for me.

    rename-item "$fileName" "$nameOnly-$DateStamp$extOnly"

This is nearing the end of the script and basically appends the date to the csv file.  This prevents the script from running on next startup – unless you have another power outage of course.

So there you have it – a complete power off script and a complete power on script.  I know I’ve rambled a little bit in the post but I’m just in that sort of mood.  If you need a hand setting things up or have any questions, concerns, improvements, criticism don’t hesitate to leave them in the comments.  I’ll get back to you….  And yes, my tomato plants are all dead!