Like Knighthood, all things grow more complex
In the early days of Knighthood, it was a simple matter to become a knight. Save enough money, buy a horse (or four, knighthood was rough on horses) a suit of armor and some weapons, stake a claim and have those within the boundaries of your claim build you a manor house, or a castle. Suddenly you were a knight, collecting taxes, meting out justice, and lording over all you surveyed. Actually lording over all you could ride to and back in a single day, much more than that had to wait for a more complex system. In the era of the early middle ages, with the Roman Empire collapsing throughout western Europe, this model brought many benefits to all people. While these were not the shining knights of chivalry by a very long shot, they were the protectors of the peasantry and the bane of robbers and ne’er-do-wells.
As time went on, the system of knighthood became increasingly complex, and this low cost of entry disappeared. The rules governing Knights became more complex as the protection of the people was clarified and the needs of outsiders – kings and religious leaders – became a burden upon the knighthood. Eventually peerage, family lines, and aristocracy grew out of what was essentially a “to the strongest go the spoils” system, and all would agree this was better, for it strengthened the protections and limited the powers of those in control.
Like Knighthood of old, simple data replication has found itself fertile ground for increasing complexity. When replication first came about, it was a fast and easy way to keep an up-to-date copy of your file-level data on a second machine. It didn’t take long though before people started finding problems with this scheme. Copying files was great, but you couldn’t easily replicate databases. This introduced products like Oracle Streams and SQL DTS with bulk copy to the replication mix. Copying data within a datacenter didn’t do you much good in the cases of power outages and disasters, so replication over the WAN became an issue. Recovery Point Objectives (the time lag introduced by replication) started out in terms of days and hours because that was a massive improvement over backups, which were in terms of weeks and days. But the time delay acceptable to most IT departments quickly shrank to be expressed in terms of minutes and seconds, increasing the traffic being transferred by increasing the frequency of communications. The sheer volume of what is being transferred went up, both in terms of user data, and in terms of the ever-increasing number of VMs floating around your organization.
And like the system of external controls instituted upon knighthood to focus the knights on more chivalrous and less death-defying behavior, so it goes with replication. As more and more requirements built up for replication to be as useful as possible, tools and techniques have developed to make replication more focused on getting your data where it needs to be, when it needs to be.
Today we have de-duplication tools, both on-disk and in-stream, we have compression for transmission of data over slower connections, the ability to move VMs has seen its own set of tools developed to speed and enhance the experience of moving VMs, databases have their own replication mechanisms that can be optimized by most of the above techniques, there are secure tunnels for getting the information from data center one to data center two… The list goes on.
Replication is not yet done growing up, it still introduces unexpected behaviors in many deployments and has many options that are not always well understood – like whether you are doing synchronization, controller-agent, or something in-between, the RPO settings and what they mean to transaction integrity, etc. But we’ve come a long way, and at the current rate of advance, it looks like soon we will have the Camelot of replication.
Of course, once knighthood had become standardized as the system of Nobility, then the entire ecosystem started to change… So expect that replication will adapt, for unlike knighthood, it is not something we can do without, but like knighthood, it will mature, be taken for granted, and other things will attract attention. In fact, this process is already well underway in most organizations… Don’t you just expect that your replication works correctly? Hopefully you’re not tilting at windmills.
But hey, you only have to buy two replication appliances, not four, so we’ve come a long way from knighthood, right?
