Upgrading to vCenter Server 5.1 and SSO Database connection has failed.

So this post isn’t so much as a how to upgrade to vSphere 5.1 post as it is a simple outline of some of the gotchya’s that I ran into. And not so much during the actual upgrade of vCenter, but with the introduction of the vCenter Single Sign On service. vCenter SSO was introduced in 5.1 in order to act as an authentication broker as well as a security token exchange provider enabling you to authenticate to the SSO service once and then pass those tokens to various other VMware solutions that utilize the SSO components such as Orchestrator.

Honestly, all of the information that you find here (aside from some of the SQL Server tasks) you can find within VMware’s documentation set. That being said I’m going to throw it out here anyways since sometimes I find it easier to follow a blog post rather than a 500 page pdf. Also, this post will really only apply if you are using the embedded SQL Express database for your current vCenter Server, you shouldn’t experience these issues if using an external db.

So first off even though I wanted to install all components on the same machine I opted to go with each individual install rather than the “Simple Install”. I think I’ve read somewhere to do this but can’t remember where, either way, that’s what I did.

Anyways, SSO in itself requires a database, a separate database from your original vCenter database. Now VMware does provide you with the SQL scripts in order to create that database as well as the roles and users that go along with it. You can find these buried within the vCenter Server install media at “Single Sign On\DBScripts\SSOServer\Schema\mssql”. If you don’t have SQL Server Management Studio you might as well download and install that as well, as I”m not going to be touching on SQL command line at all. So, the first script you need to run is rsaIMSLiteMSSQLSetupTablespaces.sql – simply open this script up in SSMS, change the <CHANGE ME> to the directories you wish to store your mdf/ldf database files in. If you don’t know you can always right click on your vCenter database and have a look at where its’ files are located, with the default install of MSSQL Express it’s normally C:\Program Files (x86)\Microsoft SQL Server\MSSQL.1\MSSQL\DATA\ –

Once you have changed these to your preferred locations simply execute the query, this should create the new SSO DB for you as well as some tables within it. As for the users, they are located in that same directory on the install media and is titled rsaIMSLiteMSSQLSetupUsers.sql. Again, load this script into SSMS, replace <CHANGE ME> with your desired passwords for these database users and execute.

Also in accordance with the VMware documentation you need to be sure your SQL server is running under mixed authentication, at the time mine was only running under Windows Authentication.  This can be done by right-clicking your server inside of management studio, selecting properties and modifying the Server authentication section under the Security tab.

So with all of these prerequisites met I went along my merry way of upgrading my vCenter Server.  Everything was fine until I got to the point in the installation which setup the vSphere SSO database.  I entered in my server name, the users that I had created earlier yet still ended up getting the following error “Database connection has failed.  You can refer to the vm-sso-javalib.log in the system temporary folder for more information.”

After a frenzy of googling and reading I tracked down the issue to being that of the default of MS SQL Express utilizing dynamic ports.  You can read more about dynamic ports here, it’s certainly not my place to try and explain them, what I will try to explain though is how to get your SSO database connected.  So we need to change our SQL instance from using dynamic ports to a static port.  In the end it’s actually quite easy.  First off you need to start your MS SQL Server Instance Configuration (should be in your start menu).  First you need to go to the network configuration of the server, then double click on tcp/ip.  From here a second dialog box will appear.  Scroll all the way down to the bottom to the IPAll section.  Simply enter the port you wish to use in the TCP Port section.  I chose 1433 (the default SQL port and the one specified in the database connection screen during the SSO setup).  Really you could chose any port, so long as they match.  After changing this value and doing another quick restart on the SQL Service I was able to complete my installation of vSphere Single Sign-On.

Needless to say it took a lot of work to get the first prerequisite of vCenter Server 5.1 up and running, and honestly the rest of the installations (Inventory Service and vCenter Server itself) went flawless.  .  Either way I thought I would throw up the issues I ran into and how I resolved them in the case it might help someone else.  As always, comments are most certainly encouraged in the box below.