In the mid to late 2000, I worked at a small startup that offered a block-level backup and restore solution. It was back in the days when HW snapshots were only available on high-end storage arrays (most customers still used DAS, or mid/low-end SAN). At that time, Microsoft DPM was also a new technology that had just come out and so we were using software snapshots and Copy-On-Write technology to create the point-in-time backups.
Back then, most customers were backing up to tape about once a day, and this was considered a standard acceptable practice. Thus, the major challenge we encountered when selling the product to customers was how to change their perception about having daily backups, and teach them that they could create multiple backups (snapshots) per day–even during working hours.
Another novel concept that we needed to introduce was that they could run backups at the volume level, according to the criticality of each volume, and not just at the machine level. It took customers some time to accept these changes, but those who did really liked the solution and enjoyed a more granular RPO than traditional daily backups.
Continuous Data Protection Improves RPO
At some point, we added Continuous Data Protection (CDP) as another feature which offered an even better RPO. CDP captured every write to the disk and enabled recovery to any second in time. Implementing this as part of a software backup product sitting in the I/O path worked well enough, and was effective for use in some critical applications. But CDP also came with a few limitations.
The main issue was the performance hit on the production machines. When it became too high, the collection of CDP data had to be stopped, and the solution had to wait for the next snapshot to resync and resume backup.
Still, CDP was a major buzzword at that time, and many backup companies offered it. Some vendors offered true Continuous Data Protection while others had CDP products that were essentially “near-CDP” or “CDP-like,” offering highly granular backups/snapshots (every hour or few hours).
In 2008, the vendor I worked for was acquired by one of the largest companies in the world. As a result, the CDP solution was discontinued after a few years, although parts of the technology were still integrated into some of their products.
In the last several years, I noticed that we rarely hear about true CDP solutions and that made me wonder why is this happening?
The Problems of Continuous Data Protection
As time passed, the use of CDP revealed some significant drawbacks that led companies to explore other solutions. For one, customers started using faster and faster storage especially with the heavy deployment of flash.
This means that more data is being written every second, so the amount of data that changes between snapshots that are running at the same interval as before, has also become bigger. In case issues arise during the process, more data will be potentially lost. Hence the need for something that would minimize data loss.
I was able to find only one or two solutions that offer CDP. This got me wondering why we don’t see more solutions that allow users to recover to any point in time and minimize the data loss?.
There are two major problems that stem from CDP, and I believe that this is why CDP is not common anymore. These issues are actually similar to those I experienced with the product I was working on in my previous company:
- CDP creates extra load on the network and the production server. It needs to use either split write or something like Copy-on-Write on the server. In both instances, we double the network traffic. In some implementations, we also add extra load and latency to the server since it needs to perform Copy-on-Write, or manage the write to two places. In most cases, the servers that require CDP are running the critical applications and are under heavy load to begin with, so adding this extra load is not even an option.
- Any disruption to the CDP flow could make it unusable, which means it wasn’t a solid backup and recovery solution to begin with. On top of that, keeping CDP data for a long period of time consumed a lot of storage that was not always available. As a result, CDP data was kept only for a short period of time, thus lowering its utility.
Based on the above issues, I believe that the main reason we hardly see CDP anymore is not because there is no need for it, but because it’s easier to not use it at all than deal with the problems it creates.
These problems have gotten worse with the introduction of flash storage. Since CDP is part of backup, the data is stored on slower, secondary storage. The use of split writes or technologies like Copy-on-Write creates additional latency when writing to this slower storage, and this becomes even more noticeable when flash storage is used as the primary.
Since the primary storage is the one that receives all the writes, shouldn’t it make more sense that it also keeps all the writes, thus enabling customers to benefit from being able to recover to any point in the past?
The Future of Continuous Data Protection
Businesses continue to strive to minimize RPO and improve recovery times, but in reality, very few organizations are able to do this. Achieving such an environment usually requires a very complex infrastructure and a tremendous sustained orchestration of technological components and processes which creates complex IT environments.
At Reduxio, we believe that integrated secondary storage and data protection built into a Primary Storage Platform can address the challenges earlier described, and can help drive technological and economic benefits for organizations. With that in mind, we’ve built a down-to-the-second data recovery solution that eliminates the need to schedule snapshots and allows data to be consistent across volumes.
So if you want to recover to any point in the past you can do just that with BackDatingTM. BackDating is no CDP. Reduxio’s storage system tracks all the writes and manages the capacity and retention of data to allow recovery to any second in the past. Any saved data that is no longer needed to fulfill the defined retention is automatically removed. Every volume is protected by default and policies manage the granularity of protection; meaning, as time passes, second level recovery can be reduced to hourly, daily, weekly, and monthly.
You also don’t need to worry about adding load to the servers or the network since all the data is stored in the storage array. Data can be recovered using cloning or in place revert (no need to create a snapshot ahead of time, just select the second in time and create a clone).
We’re not the only ones enthusiastic about letting more people know about our technology. Analyst firms such as Forrester Research have positioned Reduxio as a “stable innovator” disrupting integrated secondary storage and data protection. You can read the report here.
Now if you simply want to know more about Backdating and what’s in store for Data Protection, then I invite you to subscribe to our blog.