Backup And Recovery: The Benefits Of Multiple De-Duplication Policies

Data de-duplication is changing the way that IT departments protect their data. It allows users to expand their use of disk for backup and introduces the idea of replicating backup sets over a conventional WAN as a practical part of a disaster recovery program. There are several different approaches to data de-duplication—some technologies compare specific files to find repeated data between versions, others use a block-based approach that expands the data comparison to a more global scale. When it comes to effectiveness, several of the approaches seem to be able to reduce disk capacity and network bandwidth requirements by 90% or more - powerful savings that allow users to retain more data on disk, improve their restore performance, leverage networks for replication, reclaim floor space in the datacenter, and reduce power and cooling requirements.

There are also two different approaches to when de-duplication activity is carried out, and that choice can have a significant impact on how effectively de-duplication works with specific data protection tasks. The basic issue is whether data is de-duplicated 1) during the backup process (during ingest) or 2) after the backup (in a post process). Each methodology has advantages, but these have been largely obscured because until recently each vendor only offered one choice. That meant that the discussions, often heated, focused on which method was "the best" and why the other methodology was less effective. The reality is that there is no single best approach—different approaches have different results and the right way to think about the problem is to see which specific kinds of backup jobs are best matched to the different methodologies.

It is important for the reader to know that some of Quantum's disk-based de-duplication products, the ones designed for larger and more complex backup environments, give users a choice of different de-duplication methods. While we believe that a choice between different de-duplication policies is likely to benefit most users, we have tried to describe the approaches as objectively as possible to allow the reader to decide whether or not a single approach might make more sense in their specific case.

Read more...