I’ve taken a little break from the VMCE Study Guide creation but truth be told participating in the #VDM30in30 will probably help get me back into the swing of things. Today we will continue along with the guide, jumping ahead slightly to Module 9 and tape support. I’ve not used tape at all within Veeam so this was definitely a good excercise for me to go through before going and writing the VMCE certification.
Requirements and Configurations
As far as support goes, Veeam Backup & Replication supports the following devices…
- LTO3 or later tape libraries
- Physical and Standalone drives, along with virtual tape libraries
- Partitions of physical or VTLs which are presented to VBR
- Drives can be connected via FC, SAS, or SCSI when connecting directly, iSCSI remote connections are also supported.
- If multiple driver installation modes are supported for the storage device, then the driver that supports multiple open handles must be utilized.
When determining which block size to use, VBR will scan all the drives within the same library or all the standalone drives which are connected to one tape server. It compares all of the available sizes which the drives support and then uses the maximum size available that is supported among all of them.
Deploying tape support within Veeam
The Veeam tape environment contains a number of components required to make it work…
Veeam Tape Server
The first step in configuring VBR to utilize tape is deploying a tape server. A tape server, like a lot of other Veeam Infrastructure components can be any Windows-based machine that has been added to VBR, including the Backup Server itself in small environments. For larger environments, to balance traffic load with a lot of data processing, or to configure remote data archiving a dedicated tape server should be utilized. Once connected to the tape server, Tape devices are automatically discovered by VBR. Keep in mind, although we can connect many different tape devices to one tape server, we can only connect the tape server to one Veeam Backup Server. That said, if we remove a tape server from one instance of VBR and connect it to another, VBR will recognize all of the library settings automatically, however you will need to reconfigure jobs to point to the new library.
Veeam Backup Database
Veeam catalogs all of the information about its archived data and stores this in the Veeam Backup Database. Tapes will stay in the database until you manually remove it. The information available about the tape, such as backups which are written to it is always available, even if the tape isn’t inserted into the library. Having all off this information stored “offline” allows administrators to quickly find the location of required information to restore. When the restore task is started, VBR will prompt administrators to insert the required tape. The following catalogs are utilized by VBR
- Tape Catalog – contains information in regards to files and folders which have been archived to tape with the File To Tape jobs, along with backup files which have been produced by Backup to Tape jobs. This information is available within the FILES view.
- Backup Catalog – This contains information in regards to the VMs that have backups archived to tape media with Backup To Tape jobs. This is available by selecting Backups->Tape.
Veeam supports the following technologies within a tape infrastructure
- Media Pools
- Logical container created by VBR to organize and administer tapes
- Tapes must be placed into a media pool before we can utilize it, therefore we must have at least one manually created custom media pool before we can write data to tape. We can have as many custom media pools as required. A media pool can be a target for any number of tape jobs as well. A custom media pool can have a number of different rules associated with it.
- Tape Replenishment – can manually specify particular tapes, or have the pool pull from the “Free” pool as required.
- Media Sets – assign sets of tapes to hold data from a particular time frame.
- Data Retention – Choose the period for which the data on the tapes will be protected from overwriting.
- Parallel Processing – allows the media pool to use multiple tape devices simultaneously. Can process several tape jobs or split data within one job across different drives.
- Encryption – Software encryption
- Export tapes to vault
- Another type of media pool is a GFS media pool.
- used to store data to tape in the GFS format, common for long time archiving.
- Contains 4 predefined media sets tied to a tiered retention scheme; Weekly, Monthly Quarterly, Yearly
- Predefined media sets can be disabled if you do not need them, for example if you don’t keep quarterly jobs we can disable the Quarterly media set.
- One GFS media pool can support multiple tape GFS jobs.
- GFS media pools contain only FULL backups
- There are a number of “service” media pools which are created by default in VBR. These cannot be modified or removed in any way.
- Free – all newly introduced tapes are placed into this pool. Used to replenish any custom pools when needed.
- Unrecognized – contains tapes that have been loaded to a device but require further identification from the user. Normally require the user to run an inventory or catalog job.
- Imported – contains non-empty tapes that have been identified by the tape catalog job.
- Retired – contains tapes that have been retired and reached the maximum number of re-writes. Also can contain tapes that have had some sort of mechanical malfunction.In order to create backups in the GFS (Grandfather, father, son) mode, we must create a GFS Media Pool
- Once a tape has been allocated to a custom media pool it wall always be tied to it. When a tape with expired data comes back online, VBR automatically places it into its corresponding custom media pool.
- When a tape is manually moved to another media pool it will be marked as free and data lost
- Media pools are global, meaning we can use tapes in different libraries without the need to re catalog them – meaning we can restore data from tapes inserted into any tape library, regardless of the media pool they belong to
- The only exception to this rule is when hardware encryption is utilized. When this is utilized we must insert the encrypted tape into a device with the same encryption standard.
- If a media pool (custom or GFS) uses multiple tape libraries we are able to set up Tape Library Failover. The failover order is user defined, and the job always reverts back to the primary library on the next run. One tape library within the media pool will always be marked as the primary however can failover to the next library if the following occurs.
- The tape library is offline
- The tape library has no free tapes
- The tape library as no free drives.
- Media Sets
- A media set is a set of tapes that is used for continuously writing backup data.
- Media sets always starts with a free tape, and data blocks are always appended to the previous one and rely on the following configuration options available
- Create a new media set for each backup session – VBR will produce a separate set of tapes for each backup session.
- Starting a new media set for a certain period of time – Can define a new set of tapes for a given time frame, for instance, each week use a different set of tapes. – normally used to move certain backup timeframes offsite.
- Always continuing one media set – Will continuously use tapes in the media set, not splitting archives on to separate sets of tapes – Normally used when tapes are not exported from libraries.
- Media sets are convenient if we need to move certain timeframes of data offsite as it allows us to ensure certain backup retention points are on certain tapes. That said, depending on the configuration it can cause inefficiencies in your tape utilization.
- Some instances will cause VBR to forcibly start a new media set, overriding any media pool settings, for instance
- a required tape is offline
- a required tape is hardware or software protected
- a library is offline and all drives are busy.
- Backup Sets
- Backup sets are a set of files that have been archived to tape during one tape job session. If there is a large amount of data being archived to tape, a backup set can most certainly span multiple tapes.
- Media Vaults
- Media Vaults are logical containers that help to organize offline tapes.
- Created by end user.
- Basically allow you to specify that certain tapes belong to a vault, meaning they have been moved offsite, etc.
- Allows for a convenient representation of the tapes available within VBR, and those that are offline and in storage.
- Can be set to be moved to a vault automatically after a tape goes offline.
- Tapes can be moved between vaults.
- Show up as “offline” within their media pool.
- VBR stores the following information in regards to tapes in a media vault
- What tapes are stored in a particular location
- Which tapes have expired and can be overwritten
- Which tapes comprise a particular media set that is needed for a restore.
- Media Vaults normally are created to mimic physical storage locations.
- Data Retention
- Configured on the media pool and applied to all tapes which belong to it.
- Can choose between the following policies
- Never Overwrite Data
- Define a particular time period to protect data
- Not to protect data at all.
- If a tape contains several backup sets, it will not be expired until the backup set with the longest retention period expires.
- Tape Protection
- Software option to prohibit overwriting, erasing, or appending data to a protected tape
- This OVERRIDES any Data Retention settings on the media pool
- Can be overridden at any time, at which point the data retention settings will be applied again.
- When tapes are protected the following are disabled; appending data to tapes, erasing tapes, marking tapes as free, and removing tapes from the catalog.
- If using forward or reverse incremental backup chains you must ensure that the retention period for the tape archive is not less than that of the backup to disk job.
The Virtual Synthesized Full Backup for Tapes
When utilizing the Forever incremental backup chain you may end up being left with long chains of increments. Having these long chains of increments can require loading a large number of tapes during a restore depending on how it needs to chain the data together. To help combat this Veeam allows for creating a Virtual Synthesized Full Backup. Basically Veeam will periodically synthesize a full backup using the respective incremental located on the repository directly on the tape (Directly on tape, not requiring additional repository space). Some requirements/limitations of the Virtual Synthesized full backup are
- The source backup job must not have a scheduled synthetic or active full. If it does, the tape will not run the virtual full even if it is configured.
- The virtual full is created automatically once a week if the job produces forever forward incremental – the day of week this happens is customizable, but it cannot be disabled.
The Synthesized Virtual Full Backup process is as follows…
- On the day the tape job runs it creates a (.vsb.) file and stores this on the backup repository – the vsb file contains only pointers to data blocks inside the files of the backup chain located on disk.
- Depending on the file pointers, the tape job detects what backup chain and what data blocks located on the repository are required to synthesize a full backup. It then takes these blocks and writes the directly to tape as a full backup file (.vbk)
- Once the full backup has been written, the vsb file is removed from the repository.
Backing up to tape with Veeam.
Veeam contains a couple of different jobs that will write data to tape
- Backup To Tape Job
- Archives Veeam backups created on disk to tape
- Doesn’t create new backups, simply locates already existing backups produced by both backup/backup copy jobs and copies them to tape.
- One tape job can have an unlimited number of source jobs linked to it.
- Backup jobs can have the following source jobs
- Backup Jobs – when a backup job is a source it scans the proper repositories and picks restore points created by the backup job. When the configuration of the source backup job is changed, the tape job automatically updates to reflect those changes
- Backup Copy Jobs – When using a backup copy job as a source the tape job will copy the full backup file once, then, only increments after that. Since the latest restore point of a backup copy jobs are marked as active until another restore point has been created, the backup to tape job will not copy them. Therefore, backup to tape jobs with a backup copy job as the source will always be one restore point behind.
- Backup Repositories – When a backup repository is used as a source for a Backup to Tape job, Veeam will archive new Veeam Backup files (created by the same backup server) as soon as they appear on disk. Explicit windows can be set for the tape job instead of continuously monitoring and archive in the background.
- Endpoint Backup Jobs – Backup files created by endpoint backup jobs can also be used as the source to a Backup to Tape job.
- Linked Jobs – Rather than linking backup jobs to tape jobs you can also setup a tape device as a secondary destination in your backup job. In this case, Veeam waits for the backup job to create a new backup file and then copies it to tape.
- Backup to tape works as follows
- Backup to Tape job addresses the backup catalog to detect backups matching job criteria
- Files are queued for archiving
- Veeam connects to the data movers and starts transfer process
- source data move gets data from repository, target data mover sends data to tape
- Tape job retrieves a list of tapes for writing from the media pool.
- Data is processed and Backup Catalog is updated.
- GFS Tape Jobs
- GFS jobs can use any backup job as a source that is processed in forever forward incremental, forward incremental, or reversed incremental.
- Full backups are processed depending on when the individual media sets within the media pool have been setup.
- If two periods overlap, the backup with the longest retention is archived.
- If the source job creates more than one full backup within 24 hours the tape job only copies the first one.
- If the restore point has been locked, GFS job will wait up to 7 days for it to become available.
- If the job is manually cancelled it will automatically start again the next day (00:00)
- If there is a technical issue the GFS job will archive the two most recent missed backups for each media set once it has been fixed.
- File to Tape jobs
- File to Tape jobs allow you to copy any Windows or Linux files to tape.
- Can be used to process Veeam backups as well, but treats them as simple files rather than backups.
- File To Tape compresses sources files to files which have been stored on tape and copies just the changes. Both full and incremental modes can be used with the File To Tape job.
- Can copy files from any server which has been added to the VBR infrastructure
- File to tape jobs are more efficient when copying fewer larger files than a lot of smaller files
- jobs will timeout if they fail to complete within three weeks.
- The process of File to Tape is as follows
- Job detects files that match the job criteria
- Files are queued for archive
- If it is a first run or scheduled full, all files are queued
- If it is an incremental, the tape catalog is addressed to detect if any data has been modified since the last backup.
- Connects to the data mover and begins the transfer
- Tape job addresses media pools associated with the job, media pool allots proper tapes for writing.
- Catalog is updated as job is processing with new data.
Restoring from tape
Veeam supports a few different options for restoring from tape
- Restore VM backups to repository or disk
- Allows to copy VM backups from tape to a disk or repository of your choice.
- The restored backup is registered within VBR console so you are able to perform any other type of restore operation to it later on.
- You can restore from any restore point on the tape, at which VBR will restore the VM to the selected point in time.
- Restore VM to Virtual Infrastructure
- Process includes all of the options that a restore VM from disk does
- Can select a restore point, change VM configuration settings, etc.
- It first copies the VM to a temporary location such as a repository or folder on disk and then performs a standard VM restore back to your virtual environment.
- Restore Files from Tape
- This is used for restoring files and folders that were backed up using the Files to Tape backup job
- Can be restored directly to their original location or to another location.
- All ownership and permissions are preserved.
- Can be restored from any restore point on tape.
No matter which restore method is selected, Veeam will scan the backup catalog and get all of the required tapes that need to be available to perform the process. If all the tapes are online Veeam will begin the restore – if some are offline, VBR will list the names of the required tapes and wait for them to come online.
Interesting and testable tidbits around tape
- Tape helps administrators implement the 3-2-1 rule; 3 copies of your data, 2 types of media, 1 copy offsite.
- VBR uses MTF (Microsoft Tape Format) to write data to the tape
- VBR DOES NOT support WORM (Write Once, Read Many)
- When determining a block size, Veeam will use the maximum size available supported within the tape library.
- Veeam does not write data directly to tape – the backups are first created on disk and then copied to the tape.
- Synthesized Virtual Full backups are written directly to tape, utilizing full and incrementals from the repository
- Synthesized Virtual full backups utilize a data pointer file on the repository with a .vsb extension
- Synthesized virtual full backups will not run if the source job has an active/synthesized full scheduled
- If a synthesized virtual full backup is scheduled the same day that a full backup has run, it simply copies the full backup to tape without generating the .vsb file.
- Tape protection overrides any Data Retention settings which have been set on the media pool
- Parallel processing cannot be enabled for jobs writing to a GFS Media Pool
- Backup to Tape Jobs which use Backup Copy jobs as the source will always be one restore point behind since the latest restore point from a backup copy job is marked as active and Backup to Tape jobs will not process active restore points.
- Tape Servers automatically get the Data Mover service installed on them, just like proxies.
