When Good Hypotheses Go Bad

I’ve written recently about the necessity of hypotheses, whether you’re writing or troubleshooting. When you craft a hypothesis, it’s based on some preconceived notion you have that you plan to test. When your hypotheses are tested, sometimes they are found wanting. It’s tempting to discard your failed hypotheses and simply move on to the next, but even a failed hypothesis can have a purpose.

Imagine for a moment that you’re sitting in front of a user’s computer, helping them out with some pesky problem. Suddenly it’s the end of the day, you’ve tried everything in your repertoire and you’re calling it quits when the user looks at you and says, “I thought it was kinda weird you tried all that. Bob did everything you did last week and he couldn’t figure it out either.” Gee, thanks, Bob! There’s not even a ticket from last week, nor did he mention talking with this user. How many hours did you just waste that you could have saved if you knew none of it would work?

Bob spent hours crafting and testing his hypotheses, but he discarded all of them, straight to the circular file. You then proceeded to craft and test many of the same hypotheses which, of course, failed again. If only there was some way we could learn from our failures… Wait, there is!

Let’s take a quick look at another example, a scientific hypothesis. A researcher crafts a hypothesis and spends $100,000 to gather preliminary data that can be submitted for a grant worth $2,000,000. If the preliminary data looks good, great – well on the way to two million in funding. If it doesn’t pan out and the hypothesis is shot, $100,000 just went down the drain. That’s the nature of science. But…

A few years go by. Another researcher comes up with the same brilliant idea and sets out to collect some preliminary data for around $100,000. Whoops, the hypothesis isn’t that brilliant, doesn’t work out, and the scientist wasted time and money. Now science is out $200,000 on this failed hypothesis. If only she had known that someone else had tried this before, but there was nothing in the literature to indicate that someone had. She publishes her data in a journal and the next scientist who thinks they have it made can see what the results will look like before investing time and money in the idea. Good money isn’t thrown after bad money anymore.

You can help those after you (including future-you) if you take some time to record your hypotheses and how they failed. You don’t necessarily have to go into great detail, though scientific papers obviously require more rigor, often just a sentence or two will work. “Traceroutes were failing at the firewall, but a packet capture on the data port showed the traffic leaving the firewall,” or, “The AC fan wouldn’t start and the capacitor looked like it might be bad, but I swapped it out for my spare cap and it still won’t start.” If it’s a really spectacular failure – something that was ohhhhh-so-close to working, or a real subtle failure – maybe it’s worthy of a full blog article.

Make sure to store this information somewhere it will be found by someone who is likely to need it. In Bob’s case, this is what the ticketing system was there for, so that others can see his previous work on an asset or for a user. At home, you might keep a journal or put a note in the margins of the AC manual. For public consumption, you might write a blog article or submit your research results to a journal. Anywhere that will help prevent someone in the future from having to waste resources to rediscover the failed hypothesis.

Try and make this part of your habit when researching and troubleshooting. State your hypothesis, test the hypothesis, and record any failures before proceeding with successes. Don’t be a Bob!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s