15/08/2014 Working with partners during incidents
Charlie discusses how you can co-operate with clients to achieve the best results.
Today I have been training a client’s staff on how to manage an incident. We went through the initial response to the incident, dealing with the immediate aftermath. It was then on to invoking the plan and discussing what sort of incidents would cause it to be invoked. After that we moved on to managing the incident and setting up the incident team. One of the questions from a student was “should you invite your customers to be part of the management team?" To clarify - all those attending the training manage operations on behalf of their customer and usually work on a customer’s site. They are an integral part of their operations.
I thought this week I would share some thoughts on working with partners and customers during incidents.
1. If you manage a site on behalf of your customer and are responsible for writing and implementing the business continuity plan, it is a good idea to invite your customer to have a seat on the incident management team. This allows for joint working and you are not second-guessing what the customer would want during the recovery. Having joint decision-making means there is much less chance of the customers suing you after the event, as they were intermittently involved in the decision-making process.
2. As a member of your incident management team they should be included in exercises and training. Having trained together, there will be much greater understanding of each other’s needs in and it allows any issues to be sorted out without the pressure of being in the middle of a major incident.
3. The only time when it may be difficult to have your customer as a member of your incident team is when you are in dispute with the client. This could be a situation where they feel your actions caused the incident or where a failure of your procedures or a member of your staff contributed to causing the incident. The clients or your lawyers may feel that it is no longer appropriate to have clients personnel in your incident room. Before suggesting your client joins you in the incident room it may be worth discussing this with your lawyer as you many need to have protocols or ways of working developed. This could include the type of incidents or instances when the client would be asked to leave the team or there may be subjects agreed in advance which would only be discussed when your staff are present. Having these ways of working agreed in advance may save both parties embarrassment and also make each party more confident in working together, knowing that the ways of working have been agreed by senior managers and lawyers.
4. One of the areas where close co-operation in advance is required, is agreeing of the recovery requirements and RTO (Recovery Time Objectives) of the operation. A while ago I worked with a client whose customer was very insistent that their operation was recovered within 24 hours and this was written into the contract. For a long time the client was very happy with the business continuity plan although it was very vague on how the operation was going to be recovered. In looking at the plan in more detail we reviewed the strategy. The only way the customer was going to get a 24 hour recovery was for our client to purchase £20,000 worth of racking and have it on standby at an alternative location. Our client went back to the customer to ask them to pay for the £20,000 racking or to push the RTO to 4 days when some racking could be purchased and assembled with no up-front cost. Unsurprisingly the client changed the RTO rather than pay the £20,000 cost. I would not be surprised if many contracts have unrealistic views of the speed of recovery and write unachievable RTOs into contracts. As it is rare that the plan is implemented then unrealistic or impossible RTOs can sit in contracts unchallenged for years. My lesson from this episode is that you have a conversation with your client in advance about what is possible and the contract may need to be changed to make the recovery possible. It is a disservice to both parties if the designated RTO for an operation is 24 hours and the senior manager thinks the recovery is in place, while those at site level know this is impossible with the current arrangements.
Like all things in business continuity the more preparation you can do in advance and the closer you work with your partners, the better the response will be on the day.