Category Archives: Posts

VMCE v9 Study Guide – Module 4 – Configuration Backup and Global Notification Settings

VMCE LogoFinishing off Module 4 of the Veeam VMCE v9 Study Guide we will take a look at configuration backups, along with what can be set in terms of global notifications within Veeam Backup & Replication version 9.

Configuration Backup and Restore

A configuration backup essentially takes our Veeam Backup & Replication database and saves it to a backup file on the repository.  The database data is then written to a set of xml files and archived into a (.bco) format. If for any reason our backup server experiences a failure we can simply reinstall a new backup server and quickly restore the old configuration.  We could also use this backup to deploy another Veeam Backup server in the same environment.  If you plan on migrating configuration data to another server be sure to stop and disable all running jobs before creating the backup or sessions may fail after restoring.

A configuration backup contains the following information

  • Backup Infrastructure Components and objects – all hosts, servers, proxies, repositories, Wan accelerators, jobs, global configuration settings, etc..
  • Backups – Backups, replicas and backup copies (information regarding the backups, not the backups themselves)
  • Sessions – historical session information
  • Tapes – libraries connected to the server

By default Veeam will create a configuration backup daily and store it in the default backup repository.  That said, it’s best to redirect this to a different repository that doesn’t reside on the backup server itself.  When you create a new repo, Veeam will offer to store the config backup on it, clicking yes will redirect NEW configuration backups to this repository.  Old configuration backups REMAIN on the default repository.

If you have created a password within the password manager on the backup server Veeam will enforce that you encrypt the configuration backup.  If you do not encrypt the configuration backup and there is a password present, Veeam will disable the configuration backup job.  Also, without encryption the credentials will not be backed up with passwords within the configuration backup – you would need to enter all of the passwords again upon restore.

There are a couple of options when it comes to restoring

  • Data Restore
    • useful if the database gets corrupted, the SQL server hosting the database becomes corrupt or you deploy the database on a new SQL server, rolling back to a point in time or restoring data to a new database on the same SQL server.
  • Data Migration
    • used when you want to move the backup server and the configuration database to another location.
  • If you forget your encryption password need for the restore you have the following option
    • If the backup server is connected to Enterprise Manager you will be presented with a I forgot the password link.
    • need to have enterprise or enterprise plus, and enterprise manager connected to the backup server
    • Veeam will launch the encryption key restore wizard, at the request step a key will be generated, this can be copied or emailed.
    • Within Enterprise Manager go to Configuration-> Key Management and click Password Recovery, and paste the key that was generated.
    • Once the response is generated, copy or email that key.
    • Back in the Encryption key restore wizard enter the copied response, upon completion VBR will apply them to the encrypted backup file and unlock all content within it.
  • Backup and replica catalogs along with session history are optional when restoring a configuration backup
  • Veeam can automatically setup your powershell policy for you during restore
  • Veeam can back up existing databases before restoring over top of them.
  • You can specify new passwords for the backed up credentials if they have changed between the backup and restore times.
  • After a restore has completed a components upgrade will be checked and ran.
  • After a restore has completed VBR can perform a sync operation for backup/replicas created on the server and tape libraries connected to it.  This is ran if
    • you restored a database from a backup created on 7.0 in restore mode
    • you restored a database created with 8.0 in restore mode and selected to restore data from the backup and replica catalog.

You should also follow the below pre-reqs before restoring a configuration backup

  • Stop all running jobs
  • Check version of backup server.  For instance v9 can restore configuration backups from 7 update 4, 8, and 9

 

Global Notification Settings

Veeam Backup & Replication can be setup to send out some alerts and notifications globally – some of which can be overridden on a per job status, but this section will just focus on global notifications.

Setting up notifications settings within Veeam is done through the Options option of the main menu on the email tab.  From here we can specify things such as the smtp server to use, it’s port and authentication methods.  We can also customize what our notification settings in terms of jobs look like for instance

  • to – who the email goes to, anyone setup in this global area will receive notifications about every job ran on the system.  Can be left empty if you wish as we can define additional emails to get notifications on a per job basis
  • Subject – contains the following variables for use %time% (completion time), %jobName%, %jobResult%, %VMCount% and %issues% (number of VMs with warning or failed status).
  • We can choose whether to notify on success, warning, and/or failure.
  • Suppress notifications till the last retry

Aside from job messages we can also setup other notifications from VBR on the notifications tab such as

  • Low Disk Space – Veeam will check disk space on datastore and target repository and include a warning message if it is below a certain threshold (warning is in the job session details).  The threshold is in terms of percent on the backup storage, and in terms of GB on the datastore details.
  • Support Expiration – By default, Veeam will warn  all email recipients about the support expiration  up to 14 days before it expires.  This is included in every email notification sent from Veeam.  This can be disabled here.
  • Update Notifications – When enabled Veeam will automatically check for new product version and patches from the Veeam website.

SNMP

Veeam can also send SNMP traps with the status of the jobs performed on the backup server.  SNMP traps can be sent to 5 different destinations.  From the SNMP tab input your receiver and community information and setting up your service properties with the Windows SNMP service are requirements to make this happen.  Then, from within your job you simply check the Send SNMP notifications for this job check box within the Notification tab of the Job Options.

VMCE v9 Study Guide – Module 4 – WAN Accelerators and Managing Network Traffic

VMCE LogoI didn’t really see WAN Acceleration mentioned anywhere within the course description of the VMCE class, so I decided this might be the best place to fit it in since we will be talking about managing network traffic in Module 4.  That said, I’m sure the topic will be brought up again in later modules, however let’s go over what we can here!

 

Wan Acceleration

WAN Acceleration is Veeam’s answer to help optimize VM traffic that will be going over the WAN.  It does this by deploying at least 2 WAN Accelerators on 64 bit Windows Servers, one located at the source, and one located at the target.  If you remember back to Module 3 we spoke a bit about WAN Acceleration so some of this may be a repeat, however its good to know for the exam.

Configuring WAN Accelerations happens in the following way

  • Configure Source side WAN Accelerator, then the target.
    • Launch the New WAN Accelerator wizard from the Backup Infrastructure view
    • From the Server step
      • specify the Windows Server you wish to use for the accelerator
      • provide a description
      • Traffic Port – Specify network port used for source to target communication – defaults to 6165
      • Streams – Number of connections that must be used to transmit data (defaults to 5).  Keep in mind as this number increases so will the bandwidth and accelerators resources it requires.  Applies only to the source WAN Accelerator.
    • Cache – location of service files and global cache
      • Folder – Path o location where service files (for source and target) or Global Cache (target only) must be stored.  Defaults to c:\VeeamWAN.  It’s also best not to nest these deep in the file system as service file names can be very long, no use in making them longer.
      • Size – Specify a size for the target WAN Accelerator according to the sizing best practices – we will go over this below
    • Review
      • Review components to be installed (data mover service, WAN Accelerator service) and click ‘Next’ to finish.

Clearing/Populating Global Cache

These process can all be accomplished by right clicking on the WAN Accelerator within the Wan Accelerators node in the Backup Infrastructure view and selecting the desired operation (process explained below)

WAN Accelerator Sizing

As mentioned above there are some best practices we need to take when correctly sizing how much space we need for WAN Accelerators, both source and target.

Source WAN Accelerator

  • Veeam analyzes data blocks that will go to target and digests them, these are stored in our source accelerator.
  • Size of cache on source accelerator depends on the capacity of all our source VM disks.
    • Every 1TB of data requires 20GB of cache space.  IE if you have 4TB of VM disks you are backing up, you should provide 80GB of cache on the source accelerator.
    • There is no global cache on the source, only the digest metadata is stored here.  Global is just for target accelerators.

Target WAN Accelerator

  • This is where our global cache is stored.
    • Global Cache is basically a library that holds data blocks that go from source to target.
    • Populated fully on the first cycle of a job.
    • If a new data block is constantly sent across the WAN, it will be added to the global cache.
    • If an already cached block is not sent over the WAN after a period of time, it will be removed from the global cache.
    • If a periodic check deems a block in the global cache is corrupt, it will remove it.
    • Global cache can copy blocks stored from one source accelerator folder to another source accelerator folder if they are the same, meaning if we have two locations each replicating a Windows 2012 server, we can simply copy blocks from the first cache to the second cache without having to send them across the WAN.
    • The Global Cache can be pre-populated without actually running the job.
      • Useful on the first run of a job so all data blocks don’t need to be copied
      • Useful if the cache becomes corrupt to prevent all data blocks to be copied again.  This requires you to clean the cache first
      • Encrypted backups are not used for population
      • You cannot start any jobs using the accelerator while the cache is being populated.
      • Veeam uses data blocks stored in specified repositories to populate the cache – only OS blocks are copied.
        • That said if there is other accelerator cache already located in the target, it will match OSs from the source repository and copy these blocks directly from the already existing cache folders if they exist.
      • Copied to a default cache folder, when a remote job starts Veeam renames this to the source accelerator used in the job.
  • Recommended to provide 10GB of cache per every type of OS utilized. (defaults to 100GB, so 10 OSes).  IE – say we backup 10 VMs (1xWin7, 6xWin2008, 3xWin2010) we should provide at least 30GB (3 OS types x 10GB).
  • If the Digests data on the source accelerator is missing or for some reason cannot be used, the target accelerator will have to re-calculate this, therefore, will require space to do so.  Therefore the same rule of source sizing applies also to target, in addition to the OS type cache allocated.  IE those 10 VMs also occupy 4TB of space we will need to add 80GB (20GB/TB * 4)  more cache space in addition to our OS cache.  So 80GB for digest calculation and 30GB per OS caching = 110GB total.
  • All this said, Global Cache is calculated per source accelerator.  Within Veeam we have the ability to apply a many to one situation, meaning many source accelerators running through 1 target accelerator.  This changes our cache size exponentially depending on the number of source accelerators.  The formula is as follows
    • Total Cache Size = (number of source accelerators) * ( Size of target WAN accelerators properties [10gb/OS]) + 20GB/TB of source data.
    • Let’s say we add a second source accelerator to our example we have been using.  The second accelerator has 1TB of source data spread across 2 OS types (Linux, Server 2003).  We would end up with the following for a global cache size
      • Total Cache size = 2(we have two source accelerators) * 50GB (5 OS types [Linux, server 2003, server 2008, server 2012, win7) at 10GB per) + 100GB ( 5TB of source data spread across the 2 source locations)
      • 2 * 50GB + 100GB = Total Cache Size of 200GB
  • With all of this, if you have the space it’s best to add as much as you can in order to obtain more efficient acceleration as it would be able to hold more repeating data blocks.

Data Block Verification

Veeam calculates checksums on blocks being transferred between source and target to help ensure that no corrupt blocks are stored in the global cache.  This works in the following way

  • Before sending, Veeam calculates a checksum on the block
  • When the target receives the block it re-calculates this checksum (before it is even written to cache).
  • The checksums are compared, if there is a difference, the target sends a request for the source to resend, upon receiving the block again, it is written to the global cache.

WAN Acceleration works in the following way

  • If using a backup copy job, Veeam uncompressed backup file to analyze content
  • Source accelerators analyzes data blocks and creates file with digest for blocks.
  • Veeam compresses data and sends it to the target
  • Target populates global cache with blocks from the copied file
  • On the next job cycle, source analyzes data blocks in the file that need transferred and creates digests just for these blocks
  • Source compares new digests with old – if duplicate blocks are found the file is not copied over the WAN.  Instead, the target will pull this file from the global cache
  • Also, restore points already existing on the target side are analyzed – if there is a duplicate located in them, the target will take them directly from the restore points.

Managing Network Traffic

Before we get into some of the ways we can throttle and manage our network manually, let’s have a look at a couple different ways Veeam manages network disconnects automatically.

Data Transport on WAN Disconnect

This type of reconnection attempt exists only on jobs who utilize WAN accelerators.  Basically if a connection drops while we are transferring VM data between accelerators VBR will pick up and resume the job from the point where the connection was lost when services are restored, rather than starting all over again.  When the connection is restored, VBR will initiate a new transfer process, this time writing data to a new working snapshot.  If the connection drops multiple times, veeam will only keep 2 working snapshots on the VM by merging previous ones together.  Once all data has made its way to the target, all snapshots are merged and a new restore point is created.

Resume on Disconnect

This process handles network disconnects not applying to accelerators, and handles disconnects between backup server, proxies, and repositories (storing replica metadata).  VBR will attempt to reestablish the connection every 15 seconds for 30 minutes, picking up right where it left off.

Network Traffic Throttling Rules

Network throttling rules are setup and enforced globally on the backup server.  They essentially limit the maximum throughput of traffic going from source to target.  They are set with a pair of IP addresses, source ip, and target ip.  If a component within the backup infrastructure fall into the specified source and target IP range, the rule is applied to them.  The steps to set them up are as follows…

  • Select Network Traffic from the Main Menu and click ‘Add ‘ in the Global Network Traffic Rules section.
  • In the source ip range, specify a range of IPs representing the source components
  • In the target IP range, specify a range of IPs representing the target components.
  • Select the box to Throttle Network traffic
    • Specify the maximum speed that must be used to transfer VM data to in the Throttle to field
  • In the Apply throttling we can set up a schedule in which this rule will apply, or have it apply all the time.
    • If a rule has overlapping schedules, the rule with the lowest maximum speed will apply
  • Network Data Encryption is also setup in this same manner with the Encrypt network traffic checkbox.  More on network encryption below

Managing Data Transfer Connections

By default Veeam uses 5 TCP/IP connections to transfer data from source to target.  This may cause network traffic to be heavy if multiple jobs run at the same time.  This can also be changed in the Global Network Traffic Rules settings using the ‘Use multiple upload streams per job’ selection box.

Enabling Network Encryption

By default Veeam encrypts data with 256-AES flowing to/from public IPs, however you may want to have encryption between your local/remote source and targets.  Again this is done in the Global Network Traffic Rules window by clicking add.  It’s the same process as setting up throttling rules (above), however checking the ‘Use Network Encryption’ box.

Specifying priority networks for transfer

VBR gives you the ability to specify what networks you want to send your VM data on.  This is useful if you have some sort of backup network or non-production network that is utilized for backup data.  Again from the Global Network Traffic Rules section we set this up

  • Click on Networks
  • Select to ‘Prefer the following networks for backup and replication traffic’ and click ‘Add’
  • Specify a network in a CIDR notation or mask
  • VBR will failover to the production network if for some reason the preferred networks are unavailable.

VMCE v9 Study Guide – Module 4 – Adding Backup Repositories

VMCE LogoIf you can recall in Module 3 we discussed the three types of backup repositories in VBR; Simple, Scale-Out and those backed by rotated drives.  Now let’s go over how to add and configure each type as we continue on with Module 4 of the VMCE v9 Study Guide!

 

Adding Simple Backup Repositories

If we can remember back to Module 3 we actually have 4 sub types of simple backup repositories; Microsoft Windows, Linux, Shared CIFS, or Deduplicated Storage Appliances.

There are a number of prerequisites we must meet depending on the type of repository we are adding, listed below

  • Linux repositories
    • Requires SSH daemon installed and configured as well as SCP utility available on the Linux server hosting the repository
  • EMC Data Domain – note without meeting these requirements you can still add DD as a CIFS share, however you will not be able to leverage any DD Boost technology.
    • Must be running DD OS 5.4 or later
    • DD Boost license must be installed and DD Boost enabled and configured
    • Must have a gateway server added to the VBR environment
  • ExaGrid
    • Must be firmware 4.7 or later
    • Must follow ExaGrid best practices to set up
  • HPE StoreOnce – without meeting these requirements you can still add HPE StoreOnce as a shared folder, however in this case VBR will perform the deduplication.
    • Must be running firmware 3.13.1 or later
    • Must have StoreOnce Catalyst license
    • Must use Catalyst as a backup target  and configured to work with Low Bandwidth mode (primary and secondary transfer policy)
    • Must have a gateway server added to the VBR Environment
    • Client account used to connect must have access permission on the Catalyst store where backup data resides

Different options will appear in the wizard depending on the type of repository we are adding, however the process of adding it is somewhat the same.

  • From the Backup Infrastructure View right-click the Backup Repositories node and select Add Backup Repository
  • Name – specify the FQDN or IP address, as well as a description for the backup repository.
  • Type – select the type of repository you want to add.
    • Microsoft Windows server
      • Server – Select the Windows server you would like to use from the drop down.  If the server hasn’t already been added you can do so by clicking Add New.  Clicking Populate will populate a list of disk storage connected to the server.
    • Linux Server
      • Server – Select the Linux server you would like to use from the drop down.  If the server hasn’t already been added you can do so by clicking Add New.  Clicking Populate will populate a list of disk storage connected to the server.
    • Shared Folder
      • In the shared folder field, specify the UNC path to the folder you want to use.
      • If the share requires credentials, select the ‘This share requires access credentials’ and provide credentials.
      • If you have a fast connection between the source and backup repository we can leave the gateway server at automatic selection.  This will automatically chose a gateway server randomly per job session.  If the connection is slower or over a WAN we can explicitly specify which gateway server to use.
    • Deduplicated Storage Appliance
      • Deduplicated Storage – Select either EMC, ExaGrid, or HP StoreOnce
        • Data Domain
          • Specify the connection settings to the data domain.  If connecting over FC select ‘User Fibre Channel’ and enter a Data Domain Fibre Channel server in the domain server name field.
          • Specify credentials supporting DD Boost
          • Select whether to use in flight encryption.
          • Specify a Gateway server or leave set to automatic if connection is fast.  If the DataDomain is connected over FC you must explicitly define gateway server and said server must have access to the Data Domain appliance over FC.
        • ExaGrid
          • From the Repository server drop down select the ExaGrid appliance you wish to use.  If it isn’t added you must add it with the ‘Add New’ button.
        • StoreOnce
          • Specify your connection settings to the StoreOnce appliance, and selecting ‘Use FC’ if connecting over Fibre Channel.
          • Specify credentials having access to the Catalyst store where you wish to store the backups
          • Select whether to automatically chose a gateway server or explicitly define one.  Again, if using FC you must explicitly define a GW server and it must have access to the FC StoreOnce appliance.
  • Repository – this is where we specify where on the selected repository we wish to store our backups, as well as load control settings.  Again this may be different depending on what type of repository we are adding
    • Location – specify a path to the folder to store backups in.  For DataDomain click Browse and select a location – for StoreOnce, select a Catalyst store from the list.  For Windows/Linux, specify a path.
    • Load Control – limits the number of concurrent tasks and data ingestion rate.  The limiting of read and write data rates applies to the combined rate of both.
      • Advanced presents a number of additional settings to place on the repository.
        • Align backup file data blocks – Veeam will align VM data saved to a backup file at a 4kb block boundary.  Provides better dedup but can result in wasted space depending on storage level of fragmentation.
        • Decompress backup data blocks before storing – This will decompress data before storing it, even if compression is enabled.  A setting that is useful for utilizing compression on a job with deduplication appliances as a target
        • This Repository is Backed by rotated hard drives. – if you plan on using rotated drives.
        • User per-VM backup files – recommended if you use a dedup storage appliance or a repository supporting multiple streams.  Will write data with several streams, one VM per backup file per stream.
        • Deduplicating storage appliances supported by Veeam have the following recommendations
          • Data Domain
            • Align backup file blocks – disabled
            • Decompress Backup data blocks – enabled
            • backed by rotated drives – disabled
            • User Per-VM Backup Files – enabled
          • ExaGrid
            • Align backup file blocks – disabled
            • Decompress Backup data blocks – disabled
            • Backed by Rotated Drives – Disabled
            • Use Per-VM Backup Files – Enabled
            • Limit max concurrent tasks – 1
          • StoreOnce
            • Align backup file blocks – disabled
            • Decompress Backup data blocks – enabled
            • backed by rotated drives – disabled
            • User Per-VM Backup Files – enabled
  • Specify Mount Server settings.
    • From the server list select a mount server to use with the backup repository.  If the desired one is not there we can add it at this point by selecting ‘Add New’
    • Enable vPower NFS server – enforces repository accessible by Veeam vPower NFS, for SureBackup Jobs, virtual labs etc.
      • Folder – specify a folder to be used as the vPower NFS root folder
    • Mount server will not be deployed until after the repository has been fully configured.
    • Ports – allows us to customize the network ports used by the vPower NFS service.  By default these are…
      • RPC port: 6161
      • Mount Port: 1058
      • vPower NFS port: 2049
  • Review settings
    • Here you can review your settings and complete.  There is a couple other options.  If the repository already contains backup files we can select to Import these automatically.  If so, they will display under our Imported Backups.  If there is also guest index files located on the repository we can chose to import these indexes as well.
  • Apply settings and watch as VBR updates the status on all the subtasks it performs

Adding a Scale-Out Backup Repository

Before we get into the process of adding a Scale-Out Backup Repository it’s best to have a little review of some of the requirements and limitations associated with them.  We went over this in Module 3, but for memory purposes let’s list a few of them below…

  • Only Available in Enterprise and Enterprise Plus – Enterprise is limited to 1 SOBR with 3 extents only.
  • If license is downgraded to standard with a SOBR present you will not be able to back up to it, but will be able to perform restores.
  • Cannot use SOBR as a target for Config Backups, Replication jobs, VM Copy Jobs or Endpoint jobs.  If repository contains data from any of these unsupported jobs you will need to retarget the jobs at another repository AND REMOVE DATA from the repository

To add a SOBR right-click on the ‘Scale-out Backup Repositories’ node on the Backup Infrastructure view and select ‘Add Scale-out Backup Repository’ and follow the following configuration steps.

  • Name – Add a name and description for the SOBR
  • Extents – Click ‘Add’ to select the backup repositories that you wish to add as an extent to this SOBR.
    • Advanced Options on this screen include whether to Use Per-VM backup files, and whether or not to perform a full backup when a required extent is offline.  This basically means that if an extent that contains previous files from a backup chain is offline, Veeam will create a full backup file instead of a scheduled incremental.
  • Extents – If we have selected a repository that is already used by jobs of a supported type (backup jobs) or already has supported backup files on it such as VeeamZIP backups you will be prompted to update the jobs/backup to point to the new repository.  Need to click yes here to continue with the creation.
  • Policy – this is where we specify our backup placement policy.  If you can remember back to Module 3 we have two
    • Data locality – stores backup files that belong to the same chain together – full/incremental on the same extent.  Any new backup chains associated, for example a new full and incremental chain could be on the same extent or another extent, so long as the individual full/incremental are together.
    • Performance – stores full and incremental on different extents allowing read/write streams to be optimized to different underlying disks.
      • Performance allows you to restrict which types of backups can be stored on a specific extent in the Advanced settings.  We could place full backups on extent1, and incremental on extent2.  By default, Veeam stores both on the same extents, so long as they are from different chains.
  • Summary – review details and click finish

Extending a SOBR is just a matter of going back into the SOBR properties and adding more extents during the extents step.

Removing extents from a SOBR requires a bit more work as they may contain backup files already. To remove an extent we must follow the following steps

  • Put extent in maintenance mode
    • Click on your SOBR name in the Backup Infrastructure view
    • From the extent list, right-click the desired extent and select ‘Maintenance Mode’
  • evacuate backups from the extent
    • Click on your SOBR name in the Backup Infrastructure View
    • Right click the desired extent and select ‘Evacuate Backups’
  • remove extent from SOBR
    • From within the properties screen of your SOBR select the desired extent and click ‘Remove’
      • Note, if you skipped the ‘Evacuate Backups’ step you will be prompted to do so here.  If you chose not to, you may end up breaking the chain of some restore points.

Adding Backup Repositories with Rotated Drives

Before adding a rotated drive backup repository first attach your external drive to the windows or Linux server you wish to add as a repository and launch the ‘Add New Backup Repository’ wizard, following the below configuration and instructions…

  • Give the repository a name and description
  • Select which server to use as the repository
  • On the server section, click ‘Advanced’ and select ‘This Repository is backed up by rotated hard drives’ and select the volume of your external drive.
  • Follow all other instructions to complete the Simple Backup Repository addition.

Cohesity 3.0 – One platform for all your secondary storage!

logo-cohesity-darkAfter just over half a year of making their 1.0 product generally available Cohesity, a company based out of Santa Clara have announced version 3.0 of their flagship secondary storage products DataProtect and DataPlatform.  I had the chance to take a 1:1 briefing with Cohesity to check out what’s new and find out just what they define secondary storage as and thought I’d try and share my thoughts around the new features and overall solution from Cohesity here…

What is secondary storage?

Before we get too in-depth around the features and benefits of the Cohesity platforms its nice to stop and take a look at just what secondary storage is.  Quite simply, Cohesity sees secondary storage as any storage hosting data that isn’t “mission critical”, and surprisingly they are also discovering that this non “mission critical” data takes up the majority of an organizations overall capacity.  As show below we can see that data such as backups, test/dev, file shares, etc.…  These all fit into the secondary storage profile – data that is rarely used, fragmented and complex to manage, data that Cohesity defines as “Dark Data”

storgageiceberg

All of this “Dark Data” can become a bit of a challenge to manage and maintain – We end up with numerous backups that we don’t touch, we have many appliances and servers within our datacenter performing various functions such as deduplication, compression, analytics, etc.  All of these moving pieces within our datacenter each come with their own cost, their own hardware footprint, and for the most part have no way of interfacing with each other, nor do they have the ability to scale all together.  This is where Cohesity makes it’s play – simplifying secondary storage within your datacenter

Cohesity – All your secondary storage – One Hyperconverged platform

Cohesity moves into the datacenter and aims to eliminate all of those secondary storage silos.  They do this by consolidating your backups, file shares, test/dev copies, etc. and moving them all on to a Cohesity appliance.  To get the data there, Cohesity first leverages their DataProtect platform.  DataProtect provides the means of backup, using seamless integration into your vSphere environment Cohesity starts performing the role of your backup infrastructure.  Utilizing user create polices based on SLA requirements, Cohesity begins on loading your backup data, adhering to specified RPOs, retention policies etc.  From there DataProtect also adds the ability to offload to cloud for archival purposes. Think in terms of offloading certain restore points or aged backup files to Amazon, Azure, or Google.   Once the data resides on a Cohesity appliance a number of benefits are presented to their customers; think analytics, being able to get a Google-like search throughout all of your secondary data, looking for pre-defined templates such as social security numbers or credit card numbers.  DataPlatform also provides the ability to leverage copy data management to quickly spark up exact, isolated copies of our production environment directly on the Cohesity appliance.  This allows for things such as patch management testing, application testing, or development environments to be deployed in a matter of minutes utilizing flash-accelerated technologies on the appliance itself.

dataplatform

Integrating all of these services into one common platform for sure has its benefits – lowering TCO for one, not having to pony up for support and licensing for 4 different platforms is the first thing that comes to mind.  But beyond that it provides savings in terms of OpEx as well – no more do we have to learn how to operate and configure different pieces of software within our environment dealing with our secondary storage.  No more do we have to spend the time copying data between solutions in order to perform various functions and analytics on it.  We can just use one appliance to do it all, scaling as we need by adding nodes into the cluster, and in turn, receiving more compute, memory, and storage capacity, thus increasing performance of the secondary storage environment overall.

So what’s new in 3.0?

As I mentioned before this is Cohesity’s third release in just over half a year.   We saw 1.0 GA in October of 2015, 2.0 not long after that added replication, cloning and SMB support in February of this year, and now we have 3.0 hitting the shelves with the following improvements and features…

  • Physical Windows/Linux Support – perhaps the biggest feature within 3.0 is the ability to now protect our physical Windows and Linux servers with DataProtect.  The same policy based engine can now process those physical servers we have in our environment and allow us to leverage all of the analytics and search capabilities on the data that we have always had.
  • VMware SQL/Exchange/SharePoint Support – As we all know in the world of IT it’s really the application that matters.  3.0 provides the ability to perform application aware backups on our virtualized SQL, Exchange, and SharePoint servers in order to ensure we are getting consistent and reliable backups, which can be restored to any point-in-time, or restoration of individual application objects as well.  3.0 also adds the ability to provide source-side deduplication for these application-aware backups, meaning only unique blocks of data are transferred into the Cohesity platform during a database backup.
  • Search and recovery from Cloud – 3.0 also brings us the ability to perform search capabilities on our data that has been archived to cloud, but more importantly, perform granular object level recovery on that cloud archived data as well.  Meaning the cost of moving data out of the cloud should decrease as we are just moving the data we need.
  • Performance Enhancements – Utilizing a technology based upon parallel ingest, Cohesity can now spread the load of ingesting individual VMs across all the nodes within its’ cluster – resulting in not only a capacity increase when you scale, but also a performance increase.  Also, they have done much work around their file access services, basically doubling the amount of IOPs and throughput.

And to top it all off, Best of VMworld

bovmw

A huge congrats to Cohesity on the announcement revolving around 3.0 but an even huger congrats goes out for the “Best of VMworld 2016” within the Data Protection Category!  If you want to learn more I definitely recommend checking out Cohesity here, or, if you happen to be at VMworld you have a couple more days to drop in and say Hi at booth #827!

VMCE v9 Study Guide Module 4 – Initial Configuration Adding Windows/Linux servers and Backup Proxies

VMCE LogoFinally we are moving on to Module 4 of the Veeam VMCE v9 Study Guide.  In Module 3 we took a look at all of the core components that are required in order to make Veeam Backup & Replication work – in this module we will go one step further and discuss some of the options and features we have when we go through the process of adding these into our Veeam Backup Server

Adding Microsoft Windows Servers

Windows Servers are used for a variety of different roles within VBR.  Before we can assign these roles to the servers however we need to add them into our VBR configuration.  Adding Windows Servers is done through the Backup Infrastructure View on the Microsoft Servers Node (under Managed Servers).  When adding a Microsoft Windows server you need to ensure first that file and printer sharing is enabled on the server – if it isn’t, VBR will be unable to deploy the Veeam Installer service or the Veeam Data Mover service to the server.  To add a Windows server, right-click the ‘Windows Servers’ node and select ‘Add Server’ and follow the following steps and configurations…

  • If prompted, meaning if you used an ‘Add Server’ from anywhere else, select ‘Microsoft Windows’ as your desired server type.
  • Server Name – Specify the servers fqdn or an ip address.  You can also add a description here for future reference.  The default description simply states who added the server and when.
  • Credentials – If you have already stored credentials in VBR and they are valid for this server go ahead and select them.  If not, you are able to click ‘Add’ at this point to add a new set of credentials.  These credentials will be used to deploy both the installer service and the data mover service on the Windows server.
  • Ports – We can also customize any network ports if we would like with this button.  By default the services that may get deployed on a Windows server use the following ports.
    • Veeam Installer Service – 6160
    • Veeam Data Mover Service – 6162
    • Veeam vPower NFS Service – 6161
    • Veeam WAN Accelerator Service – 6164
    • Veeam Mount Server – 6170
  • Ports – Still within this screen we have some Data Transfer options.  The range of ports displayed (default 2500-5000) are used for transmission channels between the source and target servers, with each task utilizing one port.  If you have a small environment, or don’t expect a lot of data traffic you can scale this down to a smaller range of ports.  Just remember that one port = one concurrent task.
  • Ports – Preferred TCP – Also within this screen we can see the ‘Preferred TCP connection role’ section.   This is used if this Windows server is being deployed outside of a NATed environment.  If it was, this server would not be able to initiate a connection to another server on the other side of the NAT.  If this is the case, select the ‘Run server on this side’ checkbox to reverse the direction of the connection.
  • Review – simply shows the status of the options selected.
  • Apply – At this step we can review and monitor the steps that VBR has taken to successfully add the Windows Server.

Adding a Linux Server

Before we can add a Linux Backup Repository we must first add a Linux server into our VBR environment.  Just as with Windows, this is done on the Backup Infrastructure view by right clicking the Linux Server node and selecting Add Server.  The following steps and configurations apply to the addition of Linux servers.

  • Name – provide the FQDN or IP address of the Linux Server – an optional Description can also be specified at this point.
  • SSH Connection – Veeam will deploy the required components to a Linux server through an ssh connection.  At this step we need to provide some credentials that can connect to our desired Linux Server.  If you already have credentials setup we can simply select them from the drop down, or click ‘Add’ to create a new set of credentials.  Note, both username/password and Identity/Pubkey authentication is supported for the ssh credentials.
  • SSH Connection – The advanced section on this screen allows us to further configure how we would like components deployed.  We can specify an ssh timeout value if we please.  By default this is 20000 ms, meaning if a task targeted at this server is inactive after 20000ms, VBR will automatically terminate said task.  Just as with Windows we have the ability to adjust our Data Transfer Options as well, either scaling up or down the port range and in turn scaling up/down our maximum concurrent tasks.  Also, like Windows, we see the ability to select ‘Run server on this side’ if we are deploying outside of a NATed environment.
  • When we move to the next screen we may be prompted to trust the SSH key fingerprint. When we do this, the fingerprint is saved to the Veeam configuration database.  The fingerprint is then used during every communication task between Veeam components and this Linux server to help prevent man in the middle attacks.  If this key gets updated on the Linux server, you will need to return to this servers settings within Veeam and run through the wizard again in order to trust the new fingerprint.
  • After clicking ‘Finish’ we are done.

Adding a VMware Backup Proxy

We already know that our Backup Proxy is used to process and deliver traffic to either another proxy or backup repository.  By building out multiple proxies we are able to split the load across them and in the same time take the data mover load off of our Veeam Backup Server.   Adding a VMware backup proxy is performed through the Backup Infrastructure view on the Backup Proxies node from within the VBR Console with the following steps and configuration options

  • Right-click the Backup Proxies node and select ‘Add VMware Backup Proxy’
  • Server – Chose Server – Select the Windows server you wish to assign the proxy role to – if you haven’t already added your server to the backup infrastructure you are able to select ‘Add New’ at this point to go through the process of Adding a new Windows Server (See above).
  • Server – Description – We also have the option of creating a description here as well, by default this just states who and when added the backup proxy.
  • Server – Transport mode – Select your desired transport mode, meaning how you would like the proxy to read/write the data.    By default , VBR will scan the proxy configuration and it’s connection to datastores in order to determine an optimal transport mode for it, which will be selected automatically upon reaching this screen.  If we need to override this we can by clicking ‘Chose’.   Our options here are Direct Storage Access, Virtual Appliance, or Network.  See Module 3 for more information about how each of these transport mode works.  From within the Options section of our Transport Mode selection we can specify additional options for whichever mode we have selected.
    • For Direct Storage Access and Virtual Appliance modes we can choose to either failover to network mode (default) or not.
    • For Network Mode we can choose to transfer VM data over an encrypted SSL connection by selecting ‘Enable host to proxy traffic encryption in Network mode’.
  • Server – Connected Datastores – Allows us to specify which datastores this proxy has a direct SAN or NFS connection to.  By default Veeam will detect all datastores that the proxy has access to, however if you wanted to limit certain proxies to certain datastores you can do so here.
  • Server – Max Concurrent Tasks – We can specify here the number of tasks that the backup proxy will be able to run conccurrently.  At any time if this number is exceeeded no new tasks will start until one has completed.  Keep in mind that Veeam requires 1 CPU core for 1 task, as well as increasing concurrent tasks has the potential to flood network traffic throughput as well.
  • Traffic Rules – The traffic rules section allows us to utilize throttling rules in order to limit the OUTBOUND traffic rate for the proxy.  These help to manage bandwidth and minimize impact on the network.  These rules are created globally within VBR and will only display here if the proxy ip happens to fall within the range the rule applies to. To view the globally set traffic rules we can click on the ‘Manage network traffic rules’ link below the table displayed or click ‘View’ to view a single rule.  We will go over the traffic rules in a bit more details when we cover off global settings of VBR.
  • Summary – After reviewing the summary select ‘Finish’

At anytime you can come back to the Backup Proxies node and right-click a Backup Proxy to edit it.  We can also Disable Backup Proxies on an individual basis.  When disabled a backup proxy will not be used in any backup jobs that can select it.  If you want to remove a backup proxy that is possible as well.  That said, if the Backup Proxy is explicitly selected in a job, meaning the job does not automatically select proxies, then you will first need to delete the reference to this proxy in the job before the proxy can be removed.  Removing a backup proxy only removes it from the Backup Proxies node, the server will remain in the Windows Servers node.

Adding a Hyper-V Off host proxy

By default, MS Hyper-V hosts perform the role of a proxy – this is called on-host mode.  That said they take up resources that may be needed to run your actual production environment so its best to add Off Host proxies.  We discussed these a bit in Module 3, and if you remember they have the following prerequisites.

  • Windows Server 2008 R2 or higher with Hyper-V role of 2008 R2 or higher installed
  • Must be connected to the shared storage
  • Hardware VSS providers must be installed on host (supplied by vendor)
  • If using CSV, the Hyper-V off host proxy must not be a part of the cluster it is backing up.
  • If backing up SMB3, the local system account on off host proxy must have full access permissions to the file share and must be in the same domain, or in a trusted domain.

To add a Hyper-V off host proxy you need to add the backup proxy role to a Microsoft Windows server within the backup infrastructure utilizing the ‘New Hyper-V Off-Host Backup Proxy’ wizard and the following configuration…

  • Server – select a Windows server to assign the role to, if not listed you can add new at this point.  You can also add a description.  By default, Veeam will automatically detect the connected volumes however if you would like to specify which volumes you want this host to work with you can do so using the ‘Connected Volumes Choose…’ button.  We can also specify the Maximum Concurrent Tasks for this proxy, keeping in mind each proxy requires 1 CPU.
  • In the Traffic Rules selection we can select any rules that will apply to our off host proxy to limit its OUTBOUND traffic rate.  These rules are not created here, they are created globally and only those rules that are applicable to the IP of our proxy are listed.  You can move into the global rules by clicking ‘Manage Network Traffic Rules’ link.
  • Review the summary of task and click ‘Next’ to finish deploying the proxy.

Veeam announces new Availability Platform; moves into SaaS space with Office 365 Backup!

Today during Veeam’s “Next Big Thing” event they announced a new all-encompassing Availability Platform; by leveraging and adding new features already existing products (Veeam Backup & Replication, Veeam Cloud Connect and Veeam ONE) along with tying in some newly announced products (Veeam Backup for Office 365, Veeam Availability Console), and adding in some new feature-packed versions of their products supporting physical systems (Veeam Agents for Linux/Windows) Veeam is set to deliver an all-encompassing product to customers of any size, small or enterprise, ensuring that all their data is protected and available no matter where it may reside.

veeamavailabilityplatform

Although the event was entitled “Next Big Thing” it really should have been plural (Things) as a lot was announced, released, and talked about.  If we look at the above graphical representation of the platform we see a number of products that we may not recognize; ie The Veeam Availability Console, Veeam Availability Orchestrator, Veeam Agents???  You may not recognize these, some are new, some are re branded, let me try to summarize all the announcements as best I can…

Veeam Backup for Office 365

So this one isn’t even shown in the platform graphic but hey, no point in beating around the bush here – this is probably the announcement I’m most excited about.  As a customer I was ecstatic when Veeam announced their support for Microsoft Exchange – as an admin, I could now process my Exchange backups and perform granular restores right down to item level such as individual messages right back into my co-workers mailboxes!  It was awesome!  Then, something happened – the way organizations started thinking about delivering email changed – Being in education it was a pretty easy decision to simply move into Office 365 – the price was right 🙂  No longer do we have to maintain 7 or 8 servers just to run our email system – put it in the cloud, set it and forget it!  That said, being in the cloud is great and all – but when those high level executives accidentally delete that important email where do you think they will run to?  No matter what as IT we will still be the ones responsible, and in some cases, the ones who take the blame if we can’t restore something – it doesn’t matter that it’s in the cloud or its out of our hands – it’s an IT issue!

o365

That’s why when Veeam announced support today for Office 365 I immediately started perusing around looking for some sort of beta list!  Bringing the same functionality that they have for on premises exchange environments to Office 356 is awesome!  Want to use the explorers? Sure!  Need to restore individual emails/mailboxes/folders?  Veeam Backup for Microsoft Office 365 is aimed to be released Q4 of this year, but here is the best part – if you are a Veeam Availability Suite customer or a Veeam Backup & Replication Enterprise Plus customer you can get your first three-year subscription to this product absolutely free.  For those running Enterprise or Standard don’t feel ignored – you can pick up a free 1 year subscription!

Veeam Availability Console

2 years ago at VeeamON we saw Veeam Endpoint Backup announced – a free product that we could use to back up our Windows endpoints.  There was always some “give” within the support for the product as the messaging was always “Backup your client endpoints AND those few SERVERS you have still running physical workloads”.  Although we initially saw some integration into Veeam Backup & Replication there was never really a true management interface to handle these backups or deploy configurations to endpoints we wanted to process.  This is where our Veeam Availability Console comes into play – think of this as, dare I say, the single pane of glass to manage your Veeam environment, both VBR jobs as well as jobs from the Veeam Agents for Windows and Linux- whether these workloads and backups are on-premises, or in the cloud!

vac

The Veeam Availability Console is a cloud-enabled platform, allowing both enterprises and service providers to streamline their Veeam deployments, and manage all of those remote environments, providing the framework for managing all licensed components of the Veeam Availability Platform.  Think of managing your physical and virtual backups, backups from VMs running in the cloud, and being able to restore these to your environment, or directly to an Azure instance!

As far as who this is targeted at service providers comes to mind – those Veeam Cloud Connect providers certainly can benefit from this!  But aside from the obvious Veeam is making this available to enterprise deployments as well.  For those with a lot of endpoints or a lot of distributed deployments of Veeam Backup & Replication this can be a great fit into their organization, providing that single place to go to manage all of you remote and branch office deployments, essentially making YOU a Veeam Cloud Connect provider for your business!  Veeam Availability Console is expected to be released Q1 2017!

Veeam Agent for Windows/Linux

Staying with the theme of physical support we saw the Veeam Endpoint Backup product get a face lift today as well – to keep up with its Linux counterpart, Veeam Endpoint Backup will now be known as Veeam Agent for Windows.  That said rebrand/renames, not to exciting – new features and subsequent versions are – so let’s talk about those!  Veeam Agent for Windows/Linux will now come packaged in three different versions – the free version as it stands today will remain their – always free, however Veeam has added a Workstation version along with a Server version to compliment the functionality provided.  Cleverly, Workstation will target those looking to back up, well, workstations and Server will support those looking to back up servers, adding certain features to the new versions to provide enterprise functionality into the products.  Think of things like Application Aware processing to get those consistent backups, transaction log processing to protect those physical SQL servers, Guest file indexing to provide a fast search capability for finding and restoring files.  These are the types of features that will now be available in either the Workstation or Server versions of the Veeam Agents.  Along with those features we also see a couple of new benefits in the newly released versions; the first being the Configuration and Management API – Veeam Agents licensed with Workstation or Server will now expose an API allowing customers to centrally deploy the products, complete with a backup job configured to their endpoints and servers. (Think management from the Availability Console here).  Also we see a backup cache – meaning, backups can be run and end users can stay protected even if their backup target or repository isn’t within reach.  Think of your CEO on a plane if you will, working on a very important (yet very boring) spreadsheet.  They make some changes and somehow end up losing the file – Veeam Agent for Windows could still process this backup from 15000 feet, just caching it locally on the workstation while the target was offline, and in turn moving it to the repository when it does become available.  Meaning we are protected even when we are remote!  A small but mighty feature that I’m sure will save a lot of headaches for a lot of IT admins.

veeameditions

Also, as with any paid version of a product we now see complete enterprise technical support for the Veeam agents!  Veeam hasn’t forgotten about that Free product either – along with adding features to the Workstation and Server versions we see some new enhancements to the Free edition as well – Windows 2016 support, Direct restore to Azure, and Direct restore to Hyper-V just to name a few.  Veeam Agents will be licensed per agent, with an annual subscription model!  We can expect the Linux and Windows agent to be released in November and December of this year respectively!

Veeam Availability Orchestrator

Although Veeam Availability Orchestrator (VAO) has already been announced we’ve yet to see any sort of glimpse into what the product can do.  Today that all changed.  We saw how VAO can take those DR plans that we have in place and essentially test, execute, and maintain them for us.  VAO is truly a multi hyper-visor DR machine for your organization that provides a lot of features needed to be successful when you need to be the most.

Utilizing technologies such as vPower and SureBackup/SureReplica VAO can non disruptively test our disaster recovery plan and workflow – eliminating the need for time-consuming, expensive, manual processes and ensuring things will work just as you planned.

vao

In terms of documentation have you updated your DR plan every single time you add a new service or VM, do you ensure that all the steps are properly changed when you change something within your environment?  If you answered yes then I praise you but I know I surely have not – I’ll revisit it during that quarterly review time scheduled on my calendar and just hope nothing happens between now and then – not the best strategy!  VAO solves this issue by automatically producing DR documentation, dynamically and on the fly, ensuring you always have the most up to date documentation and are in complete compliance with your DR requirements when “push comes to shove”!  VAO, which will be licensed per VM with an annual subscription is targeted to hit the market sometime in Q1 2017 with a beta sometime next month.  Be the first to know here.

But what about the Veeam Availability Suite?

Oh yeah – less we forget these products!  Veeam has been slowly announcing features for their next release of their flagship software, Veeam Backup & Replication v9.5.  We have already been notified of integration into Nimble Arrays, Direct Restore to Azure, full Windows Server 2016 support and enhanced VMware vCloud Director integration but today Veeam announced perhaps some of the most interesting and exciting features to ship with version 9.5!

ReFS Integration for VBR – as we all know ReFS is Microsoft’s “next gen” file-system, with version 3.0 set to ship with Server 2016 when it’s released!  To be honest I’ve not done enough homework on ReFS to delve deep into details of how it works but what I do know is that it includes a number of automatic integrity checks and data scrubbing operations built into the filesystem, as well as some interesting features when it comes to failure and redundancy.  But, the feature most useful to Veeam customers will be based around how ReFS provides and allocate on write model for disk updates.  Think of your repositories here – when using NTFS as an underlying repository when creating a synthetic full, Veeam actually creates a new full backup file out of previous backup chains (full and incrementals) on disk without having to transfer production datastore data.  To do this, it needs space, it needs space to create a temporary full backup file and merge incrementals into it, almost duplicating the size on disk required.  ReFS handles this a bit differently – utilizing APIs provided by Microsoft, and integration into the filesystem provided by Veeam, Veeam is able to leverage ReFS in a way that i can move metadata pointers around, eliminating the need to actually duplicate data, both saving capacity and increasing performance DRAMATICALLY  when creating synthetic full backups.  Backup & Replication v9.5 introduces this technology as fast cloning!!!  And I know I’ve mentioned a Windows specific feature here, but since it’s a feature implemented on the repository, both Hyper-V and VMware customers will be able to take advantage of this!

Enterprise Scalability Enhancements – Many enhancements have been made to the VBR processing engine, providing even more backup and VM restore acceleration technologies helping you to get to that infamous low RTPO Veeam provides.

Veeam ONE charge back – Veeam ONE has always done a great job on reporting on resource consumption and capacity planning!  Now with 9.5 we will see charge back functionality built into the product!  The charge back will be available across all platforms Veeam ONE supports, bringing it to a Hyper-V, VMware, or vCloud Director environment near you!

v10 feature revealed – wait what!?!  We haven’t even seen v9.5 released yet!  Veeam seem to be starting to reveal more of a long-term strategy here!  Anyways, we have seen yet another storage integration provided by Veeam, this time in v10, and with IBM.  Tech previews of v10 will be available this coming May at VeeamON in New Orleans!

Release date – Perhaps the most important piece of information – VBR 9.5 will be here October 2016!!!  Be the first to know when it breaks into the market by signing up here.

Needless to say there were a lot of announcements today!  In the days to come I’m sure we will see more and more technical details around these products, how the work, how they will be priced and when they will come out – but for now if you want to see the announcements yourself I recommend taking a look at the Veeam blog!  Thanks for reading!