VMCE v9 Study Guide – Module 5 – Creating Backup Jobs

Finally, we get into the heart and soul of Veeam Backup & Replication; creating backup jobs. In this section we will kick off Module 5 and take a look at creating backup jobs from within Veeam Backup & Replication.

Before we go through the process of creating a backup job however there are a number of options and settings that we should completely understand!

Veeam Backup File types

.vbk – full backup file

.vib – forward incremental changes

.vrb – reverse incremental changes

.vbm – backup metadata

.vlb – transaction log backup

.vsb – synthetic backup used when creating virtual synthesized full backups for tape.

.bco – configuration backup

Backup Methods

Veeam Backup provides 3 methods for storing backups; Reverse Incremental Backup, Forward  Incremental Backup, Forever Forward incremental backup.

Reverse Incremental Backup

• Consists of the last full backup and a set of reverse incremental backups to roll back in time.
• Allows you to immediately restore a VM using the most recent state without having to chain together incremental.  When restoring from a later restore point, Veeam applies the required reverse incremental backup into the full backup and restores the VM from the full backup.
• Performs as follows
  • During the first run Veeam creates a full backup
  • During subsequent runs, Veeam copies only blocks that have changed and injects these into the full backup file, resulting in the full backup always containing the most recent state.  During this, Veeam takes the blocks that are being replaced and builds a reverse incremental file with them and adds this to the backup chain.
  • Veeam checks the retention policy of the job.  If there is an outdated restore point it removes this from the backup chain

Forever Forward Incremental Backup

• Consists of the first full backup, and subsequent incrementals following it.
• Processed as follows
  • During the first run, Veeam creates a full backup
  • During subsequent runs, Veeam copies only changed VM data and saves these as an incremental backup file.
  • After adding the new restore point Veeam checks the retention period for the job.  If an outdated restore point is detected it transforms the backup chain as follows
    • Takes the restore point immediately following the full backup and injects those blocks into the full backup, thus moving the full backup ahead one restore point.
    • Deletes the restore point that was injected into the full backup

Forward Incremental Backup

• produces a backup chain consisting of a full backup and set of incrementals following it.  In addition to this, synthetic or active full backups are created that split the backup chain into smaller chunks of chains.
• Processed as follows
  • During the first run, Veeam creates a full backup file
  • During subsequent runs, Veeam copies only changed blocks and saves them as an incremental backup.
  • On days where a synthetic or active full backup is scheduled Veeam does the following
    • Active Full Backups
      • Veeam completes another full backup using data from the production datastore
    • Synthetic Full Backups
      • Veeam takes the incremental that was created in step 2, along with all other incrementals in the chain and merges them with the original full backup, creating a second full backup on disk.
      • Veeam then deletes the incremental that was created within step 2 as it is redundant data and already injected into the synthetic full.
      • This new synthetic full is now used as a starting point for future incrementals
  • The retention policy is then checked for the job and processed as follows
    • Veeam looks for outdated restore points and then looks to see if it is possible to delete them
    • Veeam needs to maintain a full backup and subsequent incrementals in order to restore to a certain point in time, therefore if a full backup has expired but some of its incrementals have not, Veeam will wait till the next full backup has expired in order to delete the first chain of backups
    • At times you may have more restore points on disk then expected.

Transforming incrementals into reverse incrementals

If we are creating synthetic full backups we can additionally chose to transform our forward incrementals to reverse incrementals.  In this case, Veeam will take the existing chain full and incrementals and transform it to reverse incremental restore points, allowing us to only keep one full backup on disk at a time, with reverse incremental restore points before it, and incremental restore points after.

Retention Policy for Deleted VMs

• Applied only to reverse incremental, forever forward incremental, and forward incremental with synthetic full and transform
• By default, VMs backups are set to remain on disk.
• Space is not actually removed, it is flagged to be overwritten by other backups.

Storage Level Corruption Guard (Health Check for Backup Files)

VBR can periodically perform health checks for the latest restore points within backup files.  During this process VBR performs a CRC and hash check for both the metadata and blocks within the backup file to verify integrity.  This process is performed once a day when the health check is scheduled – if the backup session runs twice on the same day, it will not be run a second time.  The process of a health check is as follows

• If corrupt blocks are detected in the metadata for a full backup file, Veeam marks the chain starting from this full as corrupted and triggers a health check retry
  • During the rety Veeam will transfer data blocks of the complete VM from the source datastore, creating a new full backup file.
• If corrupt blocks are found in the meta data for an incremental file, Veeam removes the information about this incremental in the restore point configuration.
  • During the retry, Veeam transports incremental data relatively from the latest valid restore point – again, obtaining its data from the source datastore, creating a new incremental backup file on the repository.
• If corrupt blocks are found in the actual backup file itself, full or incremental, Veeam marks these blocks as corrupt.
  • During the retry data blocks that were corrupt are transferred from the source datastore directly into the latest restore point.

Compression and Deduplication

Veeam provides 5 options in terms of compression

• None – no compression – recommended when storing backups on storage devices that already have hardware compression
• Dedupe-Friendly – optimized compression level for low CPU usage
• Optimal – recommended compression level.  Best ratio between size and time.
• High – Provides additional 10% over Optimal, but has 10x higher CPU usage
• Extreme – smallest size of backup file but most demanding performance wise – recommended to have a proxy with at least 6 caores.

Deduplication allows us to save space as well in VBR and provides us with 4 options

• Local Target (16TB) – recommended for backup jobs that may produce large (16TB+) backup files.  Uses 4MB block size in order to lower metadata size.
• Local Target – recommended for backup to SAN, DAS, or local storage – uses 1MB block size.
• LAN Target – recommended for NAS – uses 512 k block size.
• WAN Target – recommended for backup over slower WAN – uses 256K block size

When we change compression settings for existing jobs any new settings will not be applied to already existing backups.  Compression will only be applied to new backup files created

When we change deduplication settings previously created files will not be affected.  New files created will not be deduplicated until we create an active full backup.

How do we do it?

The process of creating a backup job is as follows

• There are a number of ways to kick off the Job wizard.
  • From the Jobs node in the Backup & Replication view right-click on jobs and select Backup->VMware (or Hyper-V)
  • From the Home tab, click Backup Job and then VMware
  • From the Virtual Machines view select the VM and right-click selecting either Add to Backup Job->Name of existing job or Add to Backup Job->New job.
• Give the job a name and a description
• In the Virtual Machines step of the wizard we need to select the VMs and VM containers we would like to backup.  If we select a contain, all child objects belonging to that container will be backed up – any change in our environment, like a new VM being added to that container will automatically be picked up by the job and processed.  To add our VMs select ‘Add’
  • We will now have a number of different views; Hosts and Clusters, VMs and Templates, Datastores and VMs, and Tags.
  • Select the desired VM/VM Containers and click ‘Add’
• Here we can also exclude different VMs or disks…  Note, VM log files are automatically excluded to help reduce the size of the backup file and increase efficiency.
  • VMs from VM containers
    • From the VMs tab, click Add and select which VMs you wish to exclude from the job.  You will be presented with same multiple views as you were when you were choosing VMs to be backed up
  • Specific VM disks
    • On the disks tab select the VM in the list and click ‘Edit’.  If you are backing up a container or by tags you may need to Add the VM first by clicking ‘Add’ to manually place it in the list.
    • Chose which disks you would like to exclude from the VM – You can exclude all disks, 0:0 disks (system disks) or browse through a list of custom IDE/SCSI/Sata disks.
    • If you wish you Veeam can remove these disks from the VMs configuration file so you are able to power on the VM when restoring.  To do so, select the ‘Remove excluded disks from VM configuration’ checkbox.
  • VM templates
    • On the templates tab clear the Backup VM templates checkbox
    • If you wish so, you can alternatively clear the Exclude templates from incremental backup checkbox to only process templates in the full backup file.
• Still on the Virtual Machines section we have the option to re-order the processing of VMs by selecting the VM and using the Up/Down buttons.  That said, if you are backing up a VM from a container or using tags you will not be able to do this as they are processed randomly.  To do so, you would need to add them as standalone VMs.  Also take note, with Parallel processing enabled you may find unexpected results with the processing order.  In cases where resources for a higher priority VM are not fully available but enough resources for a lower priority VM are, the lower VM may be processed first.
• On the storage tab we will specify which backup proxy we would like to use, which repository to backup up to, the retention policy, any secondary destinations for the backups as well as any advanced settings we’d like to apply…
  • Backup Proxy has a couple different options
    • Automatic – Veeam will detect backup proxies that have access to the source datastore and automatically assign one to process VMs within the job.  This is done on a per VM basis.  Before processing a new VM, Veeam analyzes the available proxies, looking at their transport modes and current workload to select the optimal proxy for that VM
    • Use selected – This allows you to explicitly state which backup proxies can be used within the job.  At least two selections are recommended for HA or network loss.
  • Backup Repository – Select a destination repository to store the backups on.
    • If you already have a backup stored on the repository you can map to those already existing files.  To do so select the Map Backup link below the repository selection.
  • Retention Policy – Specify the number of restore points that you wish to store on the repository, keeping in mind all of the information we went through in the Retention sections above.
  • Secondary Destination – Allows for us to archive our backups to a secondary destination, either a backup repository or tape.  When this option is selected we will see an additional step appear.   In the additional step we link this backup job to another backup copy job or backup to tape job.
  • Advanced.  There are a number of advanced settings we can set on the storage step of the backup job as well.  Note, after configuring all of these settings you can save them as the default settings for future jobs.
    • Backup Settings
      • Select the desired backup method  to store the backup chain; Reverse Incremental, Incremental w/ Synthetic or active full or Forever forward incremental.  See earlier in the post on each backup type.
      • Select whether or not to create a synthetic full – synthetic full will build a full backup out of the restore points already located on the backup repository.  Also select the Days to create the synthetic full on.  If creating a synthetic full you can also specify whether to transform any previous backup chains to rollbacks
      • Select whether to create an active full backup (retrieves all source data from the datastore), and specify to create it monthly or weekly and specify days.
    • Maintenance Settings
      • Storage Level Corruption guard – check to periodically perform a health check on the latest restore point in the backup chain.  Helps to prevent a corrupted restore point.  If the health check discovers a corrupt block it starts the health check retry and transfers the data from the source datastore to a new backup file or the latest backup file, depending on the scenario – explained above.
      • Full Backup File Maintenance
        • Remove deleted VMs data after – specifies the number of days to keep backup data for VMs that have been deleted from the production environment
        • Defragement and compact full backup file – check to perform and schedule a compact operation.   Compact will create a new empty file and copy data blocks from the full backup to it.  If the backup files contains deleted VMs, they will be removed.  More info above.
    • Storage Settings
      • Data Reduction
        • Enable Inline data deduplication – clear the checkbox to disable deduplication.
        • Exclude swap file blocks – By default Veeam looks at the NTFS MFT file to identify the blocks that make up hiberfil.sys and pagefile.sys and excludes them from the backup.  Clear this checkbox if you prefer to back these files up
        • Exclude delete file blocks – By default Veeam does not copy file blocks that were deleted ormarked as dirty.  If you would rather copy these, clear this checkbox
      • Compression Level – Select the desired compression level for the job (None, Dedup-Friendly, Optimal, High, or Extreme).
      • Storage Optimization  – Select the type of backup target you plan to use; Local Target, LAN target, or WAN target.  Veeam will use different data block size to optimize backup performance.
      • Encryption
        • Check and provide a password to encrypt the backups on disk.
        • If you enable this on an existing job a full backup will be created on the next run and it and all subsequent incrementals will be encrypted.
    • Notification Settings
      • Can be used to send SNMP notifications for the job, to customize notifications on a per job basis rather than utilizing the global settings.
      • Can also set backup details to a VM attribute of choosing, either overwriting or appending.
    • vSphere Settings
      • Choose whether to enable VMware tools quiescence.
      • Choose whether to use Change block tracking and whether to enable CBT on existing VMs.
    • Integration Settings
      • Chose whether to enable backup from storage snapshots and whether to limit processed VM count per storage snapshot to a certain number.  You can also chose to failover to a standard backup if Veeam fails to create the storage snapshot.  If using NetApp you will also have the option to failover to primary storage snapshots.
    • Script Settings
      • Job Scripts
        • Chose to run a script before and/or after the job and specify a path to the script.
        • Chose when to run them – every X backup sessions or on selected days only.
• Guest Processing handles the following options
  • Application Aware Processing
    • Enable/Disable application aware processing.
    • Clicking Applications gives you the following options on a per-VM basis.
      • General
        • Chose whether to require successful VSS application processing, try application processing but ignore failures or disable application processing
        • Chose how to handle transaction logs if this is a SQL server, Oracle, or Exchange VM, either process the logs (additional settings will be required on the SQL tab) or perform a copy only of the logs (logs will just be copied in backup as a normal backup would copy files)
      • SQL – additional settings for logs if you chose process from last step.
        • How to handle transaction logs, either truncate, don’t truncate, or backup logs periodically
        • If last option is chosen we need to specify the interval to backup transaction logs (in minutes) and a retention policy, either in days or until the corresponding image level backup that the logs are attached to is deleted.
        • We can also specify which server to use as a log shipping server, the server that transports the logs.  We can let Veeam automatically figure this out or specify a specific set of Log Shipping servers.  Log shipping servers are just any Windows servers added to our Backup Infrastructure.
      • Oracle
        • Oracle Account – specify a user account that has SYSDBA rights on the database.  If you chose to use guest credentials, Veeam will use the account setup within the guest processing section to connect.
        • Log Action – here we chose what to do with our logs, we can set it to not truncate logs at all, truncate logs older than X number of hours, or truncate logs over X number of GB.
        • Log Backup inteval – set the transaction log backup interval in minutes – default is 15.
        • Retention  – specify how long to keep the logs, either last X number of days or until the corresponding image level backup is deleted.
        • Log shipping server – just as with SQL we can either let Veeam automatically pick a log shippiong server or set this to a specific set.
      • File Exclusions – we are able to exclude certain files, folders or filetypes on a per VM basis.
        • First select to either disable file level exclusions, Exclude certain files and folders, or include only certain files and folders.
          • If excluding or including we can click add to add certain files or folders.  We have the option here to use environment variables, such as %windir% or file masks like * and ?
      • Scripts – we can use this section if we plan to back VMs up that do not support VSS in order to run scripts to obtain application consistent backups.
        • Script Processing Mode – select wehter to require successful script execution, ignore script execution failuer or disable script execution
        • Windows Scripts – path to windows scripts
        • Linux Scripts – path to Linux scripts
  • VM Guest OS file indexing – can select from the following options on a per-VM basis; Disable Indexing, Index everything, Index everything ecept and specify files/folders.
    • If indexing Linux OS, several components need to be installed on the VM.  mlocate, gzip, and tar.  Veeam will prompt to install these if they are not found.
  • Guest OS Credentials
    • Select default credentials to use to deploy runtime processes.  Any VMs that don’t have credentials explicitly assigned to them will use these credentials.
  • Guest Interaction Proxy
    • Specify a Guest Interaction Proxy to use or let Veeam Automatically choose one.
• Schedule
  • Run job automatically
    • Daily – runs job daily at a selected time on selected days
    • Monthly – runs job monthly at a selected time on selected days
    • Periodically every – runs job periodically ever X number of hours/minutes or continuosly to continuosly run job
    • After this job – Allows you to chain this job to the ending of another.  This will only be ran if the first job is started by a schedule – manually running the first job will not run this job.
  • Automatic Retry
    • Select to retry VM X number of times with X number of minutes in between each try upon job failure
  • Backup Window
    • Can set the job to terminate if it exceeds a specified backup window
• Summary – review settings and save job.  You do have the option to immediately execute the job here as well.