September 28, 2017
Speed is of the essence in BDR, which is why IT leaders tend to focus on using the best technology possible to recover when disaster hits. Yet teams can be so intent on storing and managing data that they forget to focus on the people required for recovery. Just as composing a detailed incident response plan is a critical part of BDR, leaders must train their staff – not just on the actual BDR solution, but on every aspect of data protection. For fast, efficient and guaranteed recovery, employees must be prepared for any kind of crisis.
Step 1: Define staff roles and responsibilities.
Reliable recovery can’t rest in one person’s hands. You’ll need to train a full bench of designated BDR staff who understand both the solution and DR plan. Even if you only need a few people for the actual recovery process, you’ll want multiple people trained in case a valuable point person is out sick or on vacation when a crisis hits. This includes employees in other geographic regions and people outside the team, such as external partners, who play a role in the recovery process.
You can identify these people by outlining the staff members involved:
Before the disaster: Who’s in charge of creating and testing backups? Who manages long-term data archiving? Who’s in control of security? Every component that might impact your recovery after a disaster should be accounted for here.
During the disaster: Who knows what systems are mission critical and should be restored first? Does the team know how to restore all of the business processes and systems? You’ll also need to think of each kind of disaster that might occur, from hurricanes to breaches to server failures to accidental errors – and identify whose help is needed for each one.
After the disaster: Who will be responsible for returning to normal? Who will clean up the aftermath?
Don’t just name the chief players in each area – everyone from your CIO to your project managers will need to be trained on their specific responsibilities to keep the recovery process on track.
Step 2: Conduct a training analysis by testing your staff on their recovery jobs.
Testing is a critical part of every DR plan – without it, you won’t know where your plan will break down in action. The same principle applies to staff: if you don’t put them through their paces, you won’t know who’s ready for a real-life disaster and who’s going to choke.
Whether you conduct a tabletop exercise or a drill, have the team run through a mock disaster to assess their readiness. Some might be misguided on what their role is or who to contact; some might not be familiar with the software, processes and other teams involved in getting back online. You might also realize that some people have been tasked with the wrong responsibilities.
Run through different disasters and start with first response actions. Does the staff know how to control the disaster, communicate with customers and employees, and contact the chain of command? What is the process for failing over and who understands how to do it? How successful is the team in resuming normal business operations?
Test the team not just on how well they execute their roles, but how long it takes them. Identify the staff members who knew what to do and those who fumbled. You’ll know who needs training.
Step 3: Train the team.
When it comes to training recommendations, there won’t be a one size fits all. Outside partners and vendors may benefit from a video or guide explaining their roles. Your team may need a reminder or process change. And some may need to be trained or retrained on every aspect of the BDR lifecycle, whether it’s testing backups or understanding your failover process.
At some point, however, you may want to ask yourself if the team needs too much training when it comes to your BDR solution. If failing over involves a dozen steps that no one can remember without reading instructions, if you need to speak to six different vendors to understand why the solution isn’t working, then chances are your team is not the problem – it’s your vendor. Today’s best BDR solutions keep it streamlined and simple so you can failover with just a click, making recovery fast and easy enough for anyone to do. If your current system is too complex, you’re better off upgrading than repeatedly training a changing team on a difficult process.
As you move through training, be sure to create clear and complete documentation for your recovery process. There’s no time for phone calls or long explanations during a crisis, so supplement your training with some solid resources that help all relevant players understand what they need to do, who they need to contact, and in what order. You won’t be able to predict when or what your next disaster is – but you’ll know you can rely on your team when it matters most.