Down With Duplicate Data!

By Jacob Cherian for Beyond The Blocks - Wednesday, July 20, 2016


At Reduxio, we are delivering major improvements in data storage performance and economics while at the same time reimagining data management and protection from the ground up. As we wrote in a previous blog post, an example of how we are reimagining data management and protection is our BackDating™ technology, which enables users to recover data from any second in the past, making snapshots obsolete. Another key technology that allows us to deliver the significant improvements in storage performance and economics is our NoDup data reduction technology.

NoDup is unique in the market today because it is the only data reduction technology that is able to dedupe and compress data inline on storage systems with multiple media types, reducing data across all these media types simultaneously and continuously. In addition, NoDup:

  • De-duplicates and compresses data in real-time as it is being ingested into the system, so that data reduction is applied before data is placed in memory cache.
  • Is global, checking for duplicate data across all tiers, volumes, clones, and historical data used for BackDating – increasing data reduction savings by increasing the probability that duplicate data will be found and removed.
  • Is always on. While many other storage vendors recommend that users turn off deduplication and compression in order to avoid a reduction in storage system performance, NoDup stays on continuously without impacting performance.

By changing the way data storage deduplication and compression takes place, NoDup is able to deliver dramatic improvements in storage efficiency and density. For example, because NoDup is applied as data is ingested, data in the storage system’s memory cache is de-duplicated and compressed. This increases the effective cache size by the factor of the savings ratio. While this improves storage system performance in general, it can have a particularly strong impact on performance in virtualization environments – both desktop and server. In these environments the large reduction in data duplication can allow for the majority of the IO to be served from the memory cache, resulting in significant improvements in application performance and user experience

Because NoDup works globally across data in a system, it results in much higher data reduction savings than solutions where the scope of data reduction may be just a volume or a collection of volumes. This means that a system that nominally has 40 terabytes of storage can effectively store as much as 120 terabytes of data. This cross volume nature of NoDup can be particularly helpful in development and test environments, where users might have multiple, but slightly different, versions of what are essentially the same application (and their associated databases).

In addition, while many flash based systems use dedupe and compression to reduce the amount of data written to SSDs, the higher savings achieved from global data reduction, deduplication and a compressed cache in a NoDup system further reduces overall use of the flash devices, reducing wear and extending flash life.

When designing NoDup we consciously chose to have data reduction turned on all the time. This improves system performance since NoDup increases the amount of an application’s data that can be served from flash and reduces the amount of access required to serve a given amount of data. With NoDup the measured impact of the additional CPU cycles to perform real-time deduplication and compression is miniscule and are easily outweighed by the benefits. In addition, NoDup’s data reduction speeds up Reduxio’s Tier-X™ tiering technology, since there is less data that needs to be moved between tiers. As a result, the system is able to react faster to changing workloads.

For too long, data storage vendors have clung to the status quo. The result has been that, despite many innovations in processing, networking and media technologies, data storage efficiency and performance have failed to advance at the pace that they could have. However, with new digital services, mobile devices and Internet of Things (IoT) devices increasing enterprises’ data storage capacity and performance needs, the days in which data storage vendors can rest on their laurels has ended. The time has come for a revolution in data storage – and with technologies like NoDup, BackDating and TierX, Reduxio is at the forefront of the movement to again bring real innovation to the data management and protection market.

Interested in learning more about Reduxio’s always-on, in-line, in-memory deduplication and compression? Download the Product Brief for NoDup by clicking below.

NoDup Product Brief 


Jacob Cherian

Written by Jacob Cherian

Jacob has a solid track record of product and technology innovation, and defining technology and product strategy across multiple products. Extensive experience with translating technology to products, and defining and building new products.

Want to comment on this blog post?