Monthly Archives: February 2015

Tech Field Day – #VFD4 – VMTurbo Putting a price on resources!

VMTurboLogoSmVMTurbo closed off the first day at VFD4 in Austin, Texas with an overview and deep dive into their flagship product Operations Manager.  This was one of the presentations that I was most looking forward to as my fellow Toronto VMUG Co-Leader, fellow Canadian and good friend Eric Wright was involved in it, and for the first time I got to see Eric on the “other side of the table” speaking for a vendor.

 Disclaimer: As a Virtualization Field Day 4 delegate all of my flight, travel, accommodations, eats, and drinks are paid for.  However I do not receive any compensation nor am I required to write anything in regards to the event or the sponsors.  This is done at my own discretion.

Demand-Driven Control for the Software-Defined Universe

Eric started off by prompting everyone’s thoughts around what exactly Operations Manager is– not by talking about the product or what it can do, but by briefly explaining a motto that VMTurbo has been built around – Demand Driven Control for the Software Defined Universe.  I know, it’s a long one but in essence it’s something that is lacking within the industry.  With the Software Defined X being introduced into our data centers, it has brought with it many benefits, and perhaps the biggest being control – we can now have software controlling our storage, software controlling our network, and in the case of automation, we have software controlling our software.  And as Eric pointed out this control is great, but useless if there is no real consistency or leverage behind whatever is controlling it – and in fact, having the demand, having our infrastructure be the driving factor behind this control is truly the answer.  VMTurbo’s Operations Manager is a product that helps us along our path to the Demand-Driven Control for the Software-Defined Universe and it does so in it’s own unique way…

Desired State – Datacenter Nirvana

herethereBefore we get into VMTurbo’s unique take on operations management I first want to talk a little bit about desired state.  Looking after a virtual datacenter we are always looking to bring our VMs, our applications, our workloads into what we consider a desired state.  This desired state essentially combines both availability and performance together, all while maximizing the efficiency of the resources and infrastructure that we have to work with.  We begin to see anomalies and performance issues when we veer away from this desired state, and traditionally, we, as administrators are tasked with bring our workloads back into that desired state.  VMTurbo states that this is where the problem lies – this human interaction both takes time – time for humans to find out about this shift, as well as time for humans to try and put the puzzle back together and get back to the desired state.  VMTurbo essentially takes the human interaction out of this equation – allowing software, in this case Operations Manager to both detect the shift from desired state but also, and more importantly take action towards moving your environment back to the desired state – thus the “control” part of the Demand-Driven Control.

And the Demand-Driven part?

This is where we see the uniqueness of VMTurbo’s Operations Manager shine through.  With Operations Manager in control, making the decisions of what VMs should run where, etc. it needs a way to look holistically at your environment.  How it does this is by taking an economic model and applying that to your infrastructure, essentially turning your datacenter into a supply chain.  Every entity of your environment either supplies or demands resources, and just as in economics when there are a lot of resources available, things are a bit cheaper.  As resources go down, things begin to get a lot more expensive.


So in terms of a VM demanding resources Operations Manager calculates the cost of those resources, again, holistically across your entire environment to determine just how those resources should be provisioned.  Think of adding more disk to a VM –  you need to look at where the disk will come from, how expanding that disk will affect other consumers (VMs) on the same datastore, how the extra capacity will affect other suppliers such as your storage array, your LUN, etc.  Operations Manager calculates all of this information in real time to determine how to best provision that storage capacity to the VM and takes action if need be to free up resources or create more supply, all while maintaining the desired state of all of your applications.

Operations Manager also goes deeper than just the VM when determining who it’s buyers are.  Through the use of WMI, SNMP, or by simply importing metrics from a third-party tools Operations Manager is able to discover services inside of your operating systems and throw them into the crazy economic market as well.  Think of things like Tomcat servers, Java Heaps, SQL Server, etc.  These are processes that may affect the demand for memory, and without insight into them making a recommendation for more memory on a VM isn’t going to help anything.  By taking these granular metrics and statistics from inside of your VMs operating system, Operations Manager can give a complete recommendation or action that will best suit your application, VM, and entire infrastructure.

It still does all the other stuff

Now VMTurbo’s supply chain model definitely sets it apart from other monitoring tools.  Also, the fact that Operations Manager can take action automatically also is a big plus when comparing the product to other tools but  you may be asking yourself what about all of the other stuff that most monitoring tools do today?  Well, Operations Manager does that as well.  Items such as right-sizing a VM, taking away or granting CPU to a VM, placement, capacity planning, etc.  Operations Manager does all of this an in fact it also applies these actions to its supply chain model, allowing the software to see just how granting another 2 vCPUs to a VM will “disrupt” the market and decide whether or not that change is “worth it”.  Operations Manager also has some decent networking functionality built-in as well.  By figuring out which VMs are “chatty” or communicating each other often, Operations Manager can make the recommendation to move these VMs onto the same host, eliminating any performance degradation or latency that could occur by having the communication move out across your network.

When VMTurbo takes action it does so in a manner of a recommendation or an action – meaning we can have the software recommend the changes to the user or we can have the software go ahead and take care of the issues itself.  Honestly this is a personal preference and I can see customers probably using a mix of both.  When calculating these recommendations and actions Operations Manager also places a transaction cost on any move it makes.  What this does is alleviate VMs from essentially bouncing back and forth between hosts trying to achieve their desired state.

Operations Manager really looks like a slick product which takes a different stance on monitoring and healing your infrastructure.  Having the application that is doing the watching do the actual doing makes sense to me, eliminating the need for the human interaction which in turn eliminates risk and certainly increases the time it takes to get back to desired state.  And I know I’ve specifically geared this post towards vSphere but honestly VMTurbo supports just about everything – think OpenStack, Azure, Hyper-V, AWS, vCloud – it’s got them all covered.  If your interest has at all peaked I encourage you to watch all of the VMTurbo #VFD4 videos here – or better yet, get yourself a trial version and try it out yourself.  Oh, and this just in – get your name in on a home-lab giveaway they are having in respect to their newest launch.

Tech Field Day – VFD4 – StorMagic A VSA living on the edge

StorMagic_Wordmark_Black_RGB_hiBefore we get too far into this post let’s first get some terminology straight.  StorMagic refers to your remote or branch offices as the “edge” – This might help when reading through a lot of their marketing material as sometimes I tend to relate “edge” to networking, more specifically entry/exit points.

 Disclaimer: As a Virtualization Field Day 4 delegate all of my flight, travel, accommodations, eats, and drinks are paid for.  However I do not receive any compensation nor am I required to write anything in regards to the event or the sponsors.  This is done at my own discretion.

StorMagic, a UK based company was founded in 2006 and set forth to develop a software based storage appliance that enterprise can use to solve one big issue – shared storage at the edge.  StorMagic is another one of those companies that presented at VFD4 that really had a strong sense for who their target markets are – they aren’t looking to go into the data center (although there is no reason that they can’t), they aren’t looking to become the end all be all of enterprise storage (although I’m sure they would love that) – they simply provide a shared, highly available storage solution for those companies that tend to have many remote branch offices with a couple of ESXi (or Hyper-V) hosts.  On the second day of VFD4 in a conference room at SolarWinds, StorMagic stood up and explained how their product, SvSAN can solve these issues.

Another VSA but not just another VSA

Choosing between deploying a traditional SAN vs. a VSA is a pretty easy thing to do – most the times it comes down to the shear fact that you simply don’t have enough room at your remote site to deploy a complete rack of infrastructure, nor do you have the resources on site to manage the complexity of a SAN – so a VSA presents itself as a perfect fit.  With that said there are a ton of VSA’s out on the market today so what is setting StorMagic apart from all the other players in the space?  Why would I chose SvSAN over any other solution?  To answer these questions let’s put ourselves into the shoes of a customer in StorMagic’s target market – a distributed enterprise with anywhere from 10 to 10000 remote “edge” offices.

One of the driving forces behind SvSAN’s marketing material is the fact that you can setup your active/active shared storage solution with as little as 2 nodes. 2 – Two.  Most VSA vendors require at least a three node deployment, and justifiably they do this to prevent a scenario called split-brain.  Split-brain is a scenario where nodes within a clustered environment become partitioned, with each surviving node thinking it’s active, which results in a not so appealing situation.  So how does StorMagic prevent split-brain scenarios with only two nodes?  The answer lies in a heartbeat mechanism called the Neutral Storage Host (NSH).   The NSH is recommended and designed to run centrally, with one NSH supporting multiple SvSAN clusters.  Think one NSH supporting 100 remote SvSAN sites.   The NSH communicates back and forth with the SvSAN nodes in order to determine who is up and who is down, thus being the “tie breaker” if you will in the event the nodes become partitioned.   That said the NSH is an important piece to the SvSAN puzzle and it doesn’t necessarily need to run centralized.  For those sites that don’t have good or any bandwidth, the NSH can be run on any Windows, Linux, Raspberry Pi device you want, locally at the site.  Beyond the heartbeat mechanisms of the NSH, SvSAN also does a multitude of things locally between the two nodes to prevent split brain as well.  It can utilize any one of its networks, be it management, iSCSI, or mirroring network to determine and prevent nodes from becoming partitioned.  So with all this what advantages come from not requiring that third node of compute within the cluster – well, one less VMware license, one less piece of hardware you have to buy and one less piece of infrastructure you need to monitor, troubleshoot and backup – which can add up to a pretty hefty weight in loonies if you have 10000 remote sites.


Aside from lowering our infrastructure requirements SvSAN brings a lot of enterprise functionality to your remote sites.  It acts in an active/active fashion, synchronously replicating writes between each node.  When a second node of SvSAN is introduced, a second path to the VMs storage is presented to our hosts.  If at any time one host fails, the other host containing the mirrored data can pick up where it left off, which essentially allows VMware HA to take our VMs that were running on local storage on the failed host, and restart them on the surviving host using local storage.   While the failed node is gone, the surviving SvSAN journals and writes meta data around the changes that occur in the environment, minimizing the time that it will take to re-synchronize when the original node returns.  That said the original node isn’t required for re-synchronization – the benefits of the SvSAN architecture allow for the second node to come up on different hardware or even different storage.  This newly added node will be automatically configured, setup and re-synchronized into the cluster, same goes for the third, the fourth, the fifth node and so on, with just a few clicks.

As far as storage goes, SvSAN can take whatever local or network storage you have presented to the host and use that as their datastore.  The appliance itself sits on a datastore local to the host, somewhere in the terms of 100GB – from there, the remaining storage can be passed straight up to SvSAN in a JBOD, RDM, or “vmdk on a datastore” fashion.  SvSAN also gives us the ability to create different storage tiers, presenting different datastores to your hosts depending on the type of disk presented, be it SATA, SAS, etc.  In terms of SSD, SvSAN supports either running your VMs directly on solid state datastores, or you can carve up SSD tier to be used as a write-back cache to help accelerate some of those slower tiers of storage.


In terms of management, StorMagic is fully integrated into the vSphere Web Client via a plug-in.  From what I’ve seen, all of the tasks and configuration that you need to perform are done through very slick, wizard driven menus within the plug-in, and for the most part StorMagic has automated a lot of the configuration for you.  When adding new nodes into the VSA cluster, vSwitches, network configurations, iSCSI multipathing – they are all setup and applied for you – when recovering existing nodes, surviving VSA’s can push configuration and IQN identifiers down to the new nodes, making the process of coming out of a degraded state that much faster.

Wait speaking of VMware

VMware LogoWorse transition ever but hey, who better to validate your solution than one of the hypervisors that you run on.  As of Feb 4th, VMware and StorMagic have announced a partnership which basically allows customers to couple the new vSphere ROBO licensing with a license for SvSAN as well.  Having VMware, who took a shot at their own VSA in the past (ugh, remember that!) chose your product as one they bundle their ROBO solutions with has to be a big push of confidence for both StorMagic and their potential customers.  You can read more about the partnership and offering here – having both products bundled together is a great move on StorMagic’s part IMO as it can really help push both adoption and recognition within the VSA market.

Should I spend my loonies on this?

IMO StorMagic has a great product in SvSAN – They have done a great job in stating who their target market is and who they sell to – and defending questions to no end with that market in mind.  HA and continuous up time is very important to those enterprises that have distributed architecture.  They’ve placed these workloads at the “edge” of their business for a reason, they need the low latency, and honestly, the “edge” is where a company makes their money so why not protect it.  With that said I see no reason why an SMB or mid market business wouldn’t use this within their primary data center and/or broom closet and I feel they could really benefit by maybe focusing some of their efforts in that space – but that’s just my take, and the newly coupled VMware partnership, combining SvSAN with the ROBO licenses kind of de-validates my thinking and validates that of StorMagic – so what do I know Smile. Either way I highly recommend checking out StorMagic and SvSAN for yourself – you can get a 60 day trial on their site and you can find the full library of their VFD4 videos here.

Share All Of The Content – Automation around sharing community blogs on Twitter

sharememeAttending an X Field Day event for me has been awesome – there are a ton of perks, you get to hear deep-dives directly from vendors, usually involving CTO/CEO/Founder type people.  You get to meet an amazing panel of fellow delegates and develop friendships.  But aside from all this there is one huge benefit that usually goes un-blogged; and that is you get to hear stories and experiences from Stephen Foskett himself – and he has a lot of them.  One in particularly caught my attention – as he explained all of the behind the scenes automation that occurs in terms of building the TFD webpages and sharing all this information out on his respective social media channels.  So, as soon as we, as a delegate click ‘Publish’ there is a ton of IFTTT recipes, Zapier functionality and custom scripts that takes our posts, tags relevant vendors/authors and kicks off the craziness that is Foskett’s brain in code.  It’s really quite amazing.  So amazing that I thought I might try my hand at my own.  Now I am by no means at the level that Mr. Foskett is at – but it’s a start – and here’s how it goes…

My Brain

So what I set out to accomplish was simple.  I wanted to be able to flag, Digg, or star (chose whatever terminology for the RSS reader of your choice) blog posts as I read them – ones that I thought were awesome – From there, the posts would be transformed into a tweet, mentioning the author and sent out on my twitter account at a later time, keeping in mind I would need to buffer these as I could be “digging” 20 posts at a time.

My Brain on Code

So here is how I accomplished that task of taking those random ideas from my brain and transforming them into code.  There is a few pre-reqs and different technologies used, so ensure you have these if you plan on duplicating this.

  • Twitter – yeah, you need a Twitter account.
  • Twitter App – you need to setup an app within that Twitter account – this is what will allow the rest of this craziness to actually connect into your account and send the tweet – we will go over the how to on this
  • Google Drive – The core scripting behind all of this is done in Google Script in behind a Google Spreadsheet – so, you will need a Google account.
  • Digg – I use this as my RSS reader so if you are following along step by step you might want to set this up.  If you use another, I’m sure you can figure out how to get your favorite posts from the reader into Delicious
  • Delicious – I implemented this as a middle man between my RSS reader and the Google Spreadsheet simply due to the fact that I may want to share out content that isn’t inside of my RSS reader.  I can easily add any content into Delicious.
  • IFTTT – You will also need an If This Then That account setup as we will be using recipes to move content from Digg into Delicious, and further more from Delicious into the Google Spreadsheet.  I use IFTTT for a ton of other “stuff” to make my life easier.  You should already have an account here setup Smile

So, the concept is as follows

  1. I “digg” a post within digg – IFTTT then takes that post and creates a Delicous bookmark with a specific tag “ShareOut”  I could also just use the Delicious chrome plug-in to tag any current site that I’m on as a bookmark as well.
  2. IFTTT then takes any new public bookmarks with the “ShareOut” tag that I have created and adds them into a Google Spreadsheet, containing the blog Title, URL, and a “0” indicating that I haven’t processed this yet.
  3. The spreadsheet contains a “CheckNew” trigger/function, which runs every x minutes/hours.  This takes any rows with a “0” (new ones from Delicious) and transforms them into a tweet, shortening the URL, ensuring we are under the 140 characters and adding the authors Twitter handle.  It then places this tweet into the next blank row on the “Tweets” sheet and updates the processed column to “1”
  4. The spreadsheet contains a “SendTweet” trigger/function, which runs once an hour – this simply takes the first tweet on the “Tweets” sheet and tweets it out, then deletes it from the spreadsheet, allowing the next Tweet to move up in the queue and be processed in the next hour, repeat, repeat, repeat.  Every hour until the sheet is empty.

So let’s set this stuff up.

First up we need the spreadsheet created within Google Drive – you can copy mine if you like as it already has a sheet which maps the top 100 vBloggers (and some more) blogs to Twitter (this is the only way I could figure out how to map Twitter handles to blogs).  Either way it will need to be setup the same if you create it new.

Next, setup the recipes in IFTTT to copy Diggs to Delicious and subsequently Delicious to Google Spreadsheet –  IFTTT is pretty easy to use so  I’ll leave  it up to you to figure out how to get the data to Delicious and the spreadsheet.  Just ensure if you are using “Add Row to Spreadsheet” as a that – that you populate all three columns in the spreadsheet in the following order (Title, URL, 0) – The “0” needs to added in order for the Google Script to function properly.  Let me know if you need a hand.

Now we need to setup an app to allow Google script to send the Tweets for us.  Log into your Twitter account and head to .  Once there click the “Create New App” button in the top right hand corner.  Most of the information you put here doesn’t matter, with the exception of the Callback URL – this needs to be “”


Once created, click on the Permissions tab and ensure that the access is set to “Read and Write”


Now we need to get the Consumer Key and Consumer Secret – This is on the “Keys and Access Tokens” tab and we will need to copy to use within our Google Script later, so shove it over to notepad or vi or something Smile


Now we are done with Twitter and it’s time to get into the good stuff!  Open up your Google Spreadsheet and make sure you have the following sheets within it.

  • Sheet1 – named exactly this and it will be a place holder for the information coming from IFTTT
  • BlogTwitter – This sheet contains all of the Blog->Twitter handle mappings.
  • Tweets – This sheet will be a place holder for our Tweets

Again, feel free to simply copy my spreadsheet – it may make it easier and already has the BlogTwitter sheet populated.

As far as just setting up the sheets with the above names in the same order there is nothing we really need to do on the Spreadsheet – it’s the code behind we really need.  To get here, select Tools->Script Editor.  When the dialog appears select “Blank Project” under “Create New Script For”.  If you copied my sheet you will simply be brought into an already existing blank project.

Before we can get started with the code there are a couple of things we need to do.  Since I use the Google URL shortening service you will need to enable this in the project resources.  This is done under Resources->Advanced Google Services”  Find the URL Shortener API and switch it to On.  You will also need to turn this service on within the Google Developers Console – the link to do so is right within that same dialog – go do that.

shortener enable

So, as far as the code goes I’m just going to dump it all right here so you can just copy/paste all of it – I’ll explain a few things about it underneath.

function sendTweet(){
var oauth = false;
function authTwitter(){
var oauthConfig = UrlFetchApp.addOAuthService('twitter');
var requestData = {
'method': 'POST',
'oAuthServiceName': 'twitter',
'oAuthUseToken': 'always'
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("Tweets");
var tweet = sheet.getActiveCell().getValue();
var encodedTweet = encodeURIComponent(tweet);
if (tweet!='') {
if (!oauth) {
oauth = true;
UrlFetchApp.fetch('' + encodedTweet, requestData);
function checkNew()
var ss = SpreadsheetApp.getActiveSpreadsheet();
var iftsheet = ss.getSheetByName("Sheet1");
var values = iftsheet.getDataRange().getValues();
for(var i=0, iLen=values.length; i<iLen; i++) {
if(values[i][2] != "1") {
// get Twitter Hangle
var urlPath = values[i][1].split("/");
var baseURL = urlPath[2];
var twitterHandle = findTwitterHandle(baseURL);
// get other data
var postTitle = values[i][0];
var postURLLong = values[i][1];
var URLShort = getShortenedUrl(postURLLong);
// build tweet string
var myTweet = buildTweet(postTitle, URLShort, twitterHandle);
// place variable in next available row on Tweets spreadsheet
var targetSheet = ss.getSheetByName("Tweets");
var lastRow = targetSheet.getLastRow() + 1;
var targetCol = "A" + lastRow.toString();
values[i][2] = "1"
function getShortenedUrl(url){
var longUrl = UrlShortener.newUrl();
var shortUrl = UrlShortener.Url.insert(longUrl);
return shortUrl.getId();
function buildTweet(postTitle, postURL, twitterHandle)
var tweet = "[Shared] " + postTitle + " - " + postURL;
if (typeof twitterHandle != "undefined")
tweet += " via " + twitterHandle;
var tweetlength = tweet.length;
if (tweetlength > 140)
var charsToTrim = tweetlength - 135;
postTitle = postTitle.substr(0, postTitle.length-charsToTrim);
tweet = "[Shared] " + postTitle + " - " + postURL;
if (typeof twitterHandle != "undefined")
tweet += " via " + twitterHandle;
return tweet;
function findTwitterHandle(blogurl) {
var twitterSheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("BlogTwitter");
var values = twitterSheet.getDataRange().getValues();
for(var i=0, iLen=values.length; i<iLen; i++) {
if(values[i][0] == blogurl) {
return values[i][1];

Copy and paste all of the above code into the blank file that was created.  First up, remember that consumer key and secret from Twitter – yeah, they will need to go in their respective spots in lines 3/4.  That is really all of the edits to code you need to make.  If you look at the script there are a few different functions.  checkNew() (Line 45) – this is what takes the data from IFTTT on Sheet1 and transforms it into tweet-format, then places it on the “Tweets” sheet.  You can see it calls out some other functions which shorten the URL and ensure the tweet is under 140 characters, as well as flag a 0 to 1 on Sheet1 (ensures that we don’t’ tweet the same thing twice).   The sendTweet() function (Line 1) – this takes whatever is in A1 on the Tweets sheet and, you guessed it, tweets it.  When its done, row 1 is deleted, allowing the next tweet to move into A1 and be processed next time the function runs,

To test, put some data in the first sheet, Column A – the blog post title, Column B, the blog post URL, and a “0” in column C.  You can do this manually for now if you want or if you have the IFTTT recipes setup let them take charge.

Then, at the top of the script editor change the function drop-down to “checkNew” and click “Run”.  If you don’t see the functions listed within the drop-down you may need to “File-Save” and completely close out both the script editor and the spreadsheet.


Once this has complete its’ run you should be able to flip back to the spreadsheet and see a Tweet sitting inside A1 of the “Tweets” sheet.  The data from “Sheet1” should also be updated with 1 flag.

From here it’s a matter of flipping back to the Script Editor and running the “sendTweet” function the same way you did the “checkNew”.  You will most likely be prompted to authorize Google as well as authorize Twitter during this process.  Go ahead and do that.  Sometimes I have found that the first time you run it you will need to authorize Google, then you will need to run it again to authorize Twitter.  Once both applications are authorized your tweet should have been sent out!

So this is all great, everything is working.  Now to setup triggers for these functions as running them manually doesn’t make much sense.  To get into your triggers select Resources->All your triggers from the script editor.


As you can see I have set up two.  One that runs my checkNew function every 30 minutes – if it finds a newly inserted row from IFTTT it will build the tweet for me.  The other, sendTweet runs once an hour – this will simply take one of the tweets and send it out!  This way, if there are tweets available it will tweet just one per hour, so I don’t flood everyone with tweets!

And that’s that!  I know this was a long winded post but I wanted to convey what I was trying to accomplish and how I did it!  Hopefully some of you find it useful if you are looking to deploy a similar type solution.  There is a ton of great content out there and this is just my part on helping to spread the good word within the virtualization community.

If you are trying to get through this and need a little help don’t hesitate to reach out – I’m more than willing to help!  If you have any ideas on how to better this process I”d love to hear them as well.  The part currently nagging at me is that manual BlogTwitter sheet – not sure how this part could be automated or bettered but if you have ideas let me know…  Thanks for reading!  And sharing Smile