Gavin’s Digital Diner

I woke up when I heard the snap, crackle, pop. A tree had fallen in the woods and I had heard it. Whatever the philosophical implications, the actual effect was that my power went out.

“Damn,” I muttered, shaking off a sense of déjà vu. It was early morning, Sunday, December 23. The winds were howling — gusts to 90 mph, or so the weather channel had predicted the evening before. Trees were snapping and cracking like Rice Krispies. “Damn,” I muttered, shaking off a sense of déjà vu. The earlier the hour, the more limited my vocabulary. “Damn,” I muttered.

I glanced at the clock. Now on battery — having achieved true cosmic Zen harmony with its VCR brethren — it was happily flashing 12:00, 12:00, 12:00. “Damn,” I muttered, switching my gaze to my backup alarm clock. It read 7:30 am.

It’s was a Sunday, the day before Christmas Eve, and I was without electricity. I thought to myself: “now, you couldn’t pick a better day to challenge the fading infrastructure of a once-great industrial state.” I then told myself to shut-up and stop being so pedantic — fading infrastructure, indeed. “Damn.”

Me and disasters, well, we’re on friendly terms. Part of my job is disaster planning, “Disasters R Us.” I get to ponder global pandemics, earthquakes, deep-fat-fried-Twinkies, extensive power outages, staff gone wild, management run amok, nuclear winter, printer jams, and printer jellies. It’s all part of the job. Since ICT is now central to people’s days, lives, and work, it’s my job to figure out what to do when things go pear-shaped.

Consequently, I try to have all the appropriate responses already lined up, ready to go. I was a Boy Scout; I am prepared. In the case of my Christmas Eve scenario I was ready to leap into appropriate action.

In this case, I pulled the covers over my head and tried to adjust the position of my patented personal heating unit (AKA Tanzy the Dog). Moving a sleeping dog is impossible, by the way. Dogs control space, time, and gravity. They can change their weight and size, at will. And, they’re also very fond of tennis balls. It was Sunday, for Pete’s sake, a holiday, during a holiday week, no less. I was going to sleep in whether I wanted to or not. Now that’s an entirely appropriate disaster response.

Experts have lots to say about disaster preparedness (pesky bastards); most of it is either unintelligible or shrouded in $10 words and $100 phrases that don’t mean squat. Personally, I have four simple elements: Avoidance, honest and realistic objectives, maximum flexibility, and clear communications. Let me explain:

1. Avoid the disaster in the first place — this is the most important rule and it’s the one that most people ignore. I prefer avoiding a disaster to living through one. Given a choice, spend your resources on avoidance. I like redundancy (hence the backup alarm clock). In this case, I should have bought that standby generator I lusted after when the world was going to end in 1999.
2. For those cases where avoidance fails you, at least be honest in your planning. Identify the processes and resources that are truly critical, and I mean truly. Having all your phones work is nice, having one that works is critical; having AC in the summer is nice, having heat in the Michigan winter is critical. Moreover, once identified, develop realistic and cost effective recovery objectives. Simply, that means figure out how you might easily resurrect the critical bits, cheaply, quickly, and without much fuss.
3. When you’re planning, admit you can’t know the future and don’t try. The objective is to give your future self maximum flexibility. Remember “Murphy’s Law of Combat” — No plan ever survives first contact intact. Designing a plan that tells you exactly what to do is stupid. Instead, design so you can roll with the punches. Be prepared — plan — to adapt, improvise, and overcome. Have options, have backups, have redundancy, and think on your feet. Worry only about the critical stuff (see item 2, above) and give yourself lots of lots of options.
4. Finally, plan on communicating. Plan to (and have systems that allow you to) talk to folks — your clients, your staff, your mother. Stress clear communications in your planning and in your responses. This is absolutely crucial and this is where most folks screw it up. All the planning in the world won’t do you a lick of good if nobody knows about it, or if nobody knows what to do. Have a backup communications plan, and have a backup of that.

Back to my wee disaster…That morning, Tanzy’s canine gravity control was in top form. She was immovable. I made due, wondering all the while how such a medium-sized dog could be such a huge dog, or is it the other way ’round? Nevertheless, she’s a great heater (dogs have a body temperature between 101°F and 102°F).

We get 4 or 5 major “outages” a year. Obviously, I’m on an alien flight path and the damn di-lithium crystals are playing havoc with the flux capacitors again. Perhaps, you say, I’ve been watching too much of the SciFi Channel? Maybe, it’s my karma. Nevertheless, we’re prepared. Batteries, flashlights, a small battery powered lantern, 10 or 20 gallons of bottled water. Duct Tape. We’re prepared, I like the process. I’m often tempted to put together the survival kit from “Dr. Strangelove”:

Survival kit contents check. In them you’ll find: one forty-five caliber automatic; two boxes of ammunition; four days’ concentrated emergency rations; one drug issue containing antibiotics, morphine, vitamin pills, pep pills, sleeping pills, tranquilizer pills; one miniature combination Russian phrase book and Bible; one hundred dollars in rubles; one hundred dollars in gold; nine packs of chewing gum; one issue of prophylactics; three lipsticks; three pair of nylon stockings. “Shoot, a fella’ could have a pretty good weekend in Vegas with all that stuff.”  (Slim Pickens speaking as Major T.J. “King” Kong from “Dr. Strangelove,” 1964).

Admittedly, my supplies are slightly different. (The Russian phrase book seemed superfluous and I couldn’t decide about the lipstick.) Nevertheless, we’re prepared. We’re prepared because when the power goes out, it’s not just the lights. We lose all niceties like water, phone, toilets, heat, internet, TV, garage door opener, and all my ‘puters. Prepared does not mean pleasant.

At 9:30 am I got up. The power was still off, and I started wishing again that I had succumbed to the siren call of Y2K and bought a standby generator. I made conscious decision to revisit that decision — avoidance is the best defense.

With great trepidation I realized it was time to brave Detroit-Edison’s (DTE) voice-mail-hell — the living example of how not to design your customer / disaster communications systems. So I woke my wife.

DTE’s integrated voice response (IVR) was designed by someone with the communications skills of an illiterate monkey. Perhaps, I’m insulting the monkey. It was designed to obfuscate, not communicate, to placate not to inform. Even that was done badly. During a disaster, you don’t want to trifle with people. You want a clear message, with clear instructions.

This particular IVR was not only irrelevant and condescending but inconsistent. Seemingly randomly, the option to press the keypad would disappear, and it would only accept a voice response. Moreover, its voice recognition system was lousy, only understanding if you mimicked it, imitating its lilting prosody, forcing my wife to sing “I don’t have one” when asked about a second contact number. After endless (and slightly chilly) minutes of punching buttons and singing into the phone, we heard a hopeful note: “please hold on while I transfer you to a customer service representative.”

If the journey was endless, the letdown was immediate. The system quickly interjected: “Actually, there are no customer service representatives available right now.” “Actually,” I thought? What strange phrasing. “Actually, I’m an idiot for actually getting my hopes up.”

“Please enter or say your telephone number and a customer representative will call you back,” said the IVR. To that, my wife sang a few more bars of “Yes,” “No,” and “I don’t have one,” it hung up. I went to gather firewood.

All in all, we must have called at least 20 times between Sunday morning and Monday morning, Christmas Eve. No one ever called back. Finally, on Christmas, we reached a human who told us our worries would be over by 5:30 pm, a repair crew was on its way. At 11:00 pm, still in the dark, I realized we’d been stood up again. I stoked the fire and vowed to get a generator. Two days later, the crew arrived.

I learned something. The DTE systems for both reporting problems and checking on the status of problems just did nothing except to force us to sing into the phone — Voicemail, the quickest way of telling your customers that frankly, you don’t give a damn. The web site (via laptop and blackberry) was designed solely with marketing in mind. It was also unusable via handheld.

Let me tell you, trying to navigate through buttons, icons, pictures, and swirly glossy things with a handheld does not make a good marketing impression. This struck me as another example of poor communications planning. Folks, pay attention, ’cause this is a lesson for me too:

In any sort of a disaster situation, most likely the tool you’ll have at hand is a cell phone. Moreover, the tool your customers, users, staff, or family will have is also a cell phone. Design your information systems, your communications systems, and your response systems with that simple fact in mind. Keep things simple and to the point, and communicate often.

This old dog learned from the experience. Girded by the experience, I decided that my various disaster recovery plans needed some rethinking. I needed some communications redundancy — and I needed to make things work via handheld and cell phone, the telephone is a simple and resilient communications resource, ubiquitous in its reach and adoption.

In the end, I went with a simple out-dial system, something that could be turned on or off at the flip of a switch, something off-site, third-party, and something that I didn’t have to maintain. In this case, a pre-programmable phone-tree service called “Call-em-All.” For next to nothing a year, it provides a turn-key automatic phone tree that can, at the drop of a hat (or crash of a tree limb), automatically call a preset list of folks and deliver a message. They even have nonprofit rates.

Be smart. When you plan for disasters, plan to communicate. Plan to communicate about its duration, impact, and consequences. People want to know what’s up, and they worry if they don’t know. Do that and you’ll weather it. Planning is easy; communications can be tough.

Note: Most of this was written in the dark, on a laptop. The laptop was charged using our hybrid Mercury Mariner (it is one giant battery after all) and the draft was uploaded using a wireless connection via my Blackberry (also charged off the hybrid).

Trackback URI | Comments RSS