Understanding Veeam and ReFS block cloning

With the introduction of Windows Server 2016 we saw a third iteration of the Resilient File System (ReFS) included.  Veeam was quick to jump on board with ReFS, leveraging a Microsoft provided block clone API in order to decrease your backup windows and increase your backup performance.  Now, like many I was eager to see just how this all worked – and I was quick to set up a repository within Veeam formatted with ReFS.  And the claims held true – I had a few jobs that performed synthetic full backups which used to take hours and hours on NTFS which were instantly decreased down to a matter of minutes!

So needless to say I was happy with the results – but being who I am I’m always left wondering what exactly is going on underneath the hood – how this “block clone” API can take a process that took so long and do the same thing within minutes.  So below, is my learning and understanding of ReFS and just how all this is happening…

So yeah, block clone…

When Veeam is creating its’ synthetic full backups it’s basically instructing the underlying file system to create a new full backup using the previous increments and full within the chain – so basically we are just copying a bunch of data.  Now in NTFS we actually have to perform reads and writes to the data in order to do this – which is why we see those longer build times.  In ReFS instead, we use the block/fast clone functionality.  How this differs is we are not actually copying the data, instead, ReFS performs lower-cost copies of the metadata which points to the original data.  What we are left with is multiple files which share the same logical cluster on the filesystem.  If at any time an application attempts to write over the original data an allocate on write method is deployed.  Essentially, the region is duplicated, all metadata is updated accordingly and the write is performed to the newly duplicated region.  So, in Veeams case, our synthetic full can just point to the latest incremental or full blocks in order to construct itself.  It’s kind of like having a synthetic full made up of just pointer blocks.

If we take all of this and apply it to our Veeam environment, our synthetic full can basically just point to the latest incremental or full blocks in order to construct itself.  It’s kind of like having a synthetic full made up of just pointer blocks.

Looking at the illustration above we can see that the traditional synthetic full is an actual duplicate of the data, whereas the ReFS synthetic full is just pointers to other data.  So that’s great right?  We can perform our synthetic fulls in hardly any time but the next question that came to my mind is around the pointers – I mean, if we just have pointers to the older pieces of data how would this synthetic full help us in the event that the old data became corrupt?  Traditionally we would be ok as we would have had another actual copy!

We know that Veeam has some tools to combat corruption with their schedulable health checks, but ReFS also has a few tricks up its sleeve as well to protect this type of corruption from occurring as well…  ReFS regularly generates checksums for both it’s metadata and file data.  It’s these checksums that are used to validate the integrity of the data before access is granted to it.  Coupling this with the resilient mirroring features available ReFS is essentially able to take corrective measures in the event that data becomes corrupt.  And don’t worry if you aren’t using the resilient mirror features – I’m sure not a lot of companies are when it comes to storing backup data – Veeam has their built-in health checks and Storage Level Corruption Guard to protect against corruption on the repository.

So in the end, moving to ReFS brings some cool space savings into play – as well as what I contribute to be the best feature, more performance when generating those synthetic fulls, thus a shorter backup window.  Now, the debate is still out on whether one should use dedup or ReFS block cloning.  Really they are two separate features even though they “kind of” accomplish the same thing.  ReFS essentially gives us the space savings that deduplication can, without the overhead deduplication usually comes with – not to mention that we can also use ReFS with other Veeam features such as encryption and compression whereas deduplication doesn’t make sense to use with these.   Deduplication, however, can provide space savings on a global level – allowing us to save space by not duplicating blocks on not just our synthetic full backups, but all backups including active fulls and incrementals.   All that said this might not even be a debate as Microsoft has announced plans to include some sort of deduplication support for ReFS in the future.    Also keep in mind that the block clone functionality doesn’t just work when copying files into the volume – the applications actually have to be designed around utilizing this block clone API, just like Veeam Backup and Replication is.  So that’s that – I know this has been written about before but wanted to throw my two cents out there – if not for anything buy my own learning experience!  Thanks for reading!