Subscribe to 3 Ideas in 2 Minutes

Premortem Analysis: How to Anticipate Failure

Sophocles, the Ancient Greek tragedian, once said that he had no intention to suffer twice; in reality, and in retrospect. The bigger our failure, the bigger our remorse over what we should’ve done and what could’ve been. If you always wanted the ability to anticipate impending mistakes, you might want to give Premortem Analysis a try. It’s a method to avoid both, the regret and the failure itself. It hinges on a simple yet crucial change in perspective. Here’s how it works.

What Is a Premortem Analysis?

Premortem Analysis is based on the familiar concept of a post-mortem, or autopsy. During an autopsy, a pathologist examines a body to determine the cause of death. When a Premortem Analysis is conducted, nobody has died. At least not yet. The goal is to contemplate the reasons for potential failure before it happens. This is done by challenging the thinking that goes into making a decision prior to its implementation. With the benefit of something akin to proactive hindsight, changes can be made to avert failure.

The analytical method was first developed by cognitive psychologist Gary Klein. In business circles, Premortem Analyses are applied to projects and business decisions. In the intelligence community, analysts use it to re-evaluate an assessment before the reports are written or advice is passed on. In everyday life, there’s no reason why it couldn’t be used to double-check any larger life decision. After you finished learning for that important exam. Or before your family buys that new house.

Let’s take a look at two versions of Premortem Analysis. First, a bare-bones approach close to what Gary Klein proposed. Second, a structured evolution of the method proposed by two former intelligence analysts, Richards Heuer and Randolph Pherson.

A Basic Premortem Analysis

When doing a Premorten Analysis, participants assume a project has failed and ask themselves what went wrong. Such a preemptive autopsy can be done alone. However, it’s best to leverage the diversity of thought that comes with involving the whole group.

In terms of timing, Gary Klein suggested conducting the analysis at the very beginning of the project. This way, the whole process could be improved. But there’s also a case to be made to initiate a Premortem Analysis towards the end. After participants have acquired a high-resolution picture of what they’re trying to achieve. But before they’re personally and emotionally invested in the decision because it’s already been implemented. In other words, right before the group is about to come to an agreement.

Whichever timing we choose, a Premortem Analysis shouldn’t be done as a tick-the-box exercise. We can achieve this by leaving enough time for planning and decisions to be altered in case the analysis uncovers serious problems.

Benefits

When considering the benefits of the method, it’s worth looking at the reasons why projects fail. According to Klein, it’s not necessarily that those involved do not see disaster coming. It could be that many people are “reluctant to speak up about their reservations during the all-important planning phase”. This makes Premortem Analysis an elegant method to avoid groupthink.

Groupthink is a psychological phenomenon coined by psychologist Irving Janis in 1971. We could summarise it like this: The more friendly group members are with each other, the more likely they will sacrifice independent critical thinking on the altar of group cohesion. This results in the group inadvertently suppressing dissent. Group consensus becomes a crutch. Potential causes for failure remain unspoken.

Premortem Analysis prompts a change in perspective as it leverages group cohesion against itself. The project and the people are the same but the questions are different. Now, dissent with the group’s impending decision is not only encouraged. It’s legitimised. This plays into the hands of those who were afraid to speak up. Before, criticising a group’s work or judgement meant going against the presumed consensus. Now, critiquing the planning process aligns with the group’s goal.

That isn’t to say we don’t already intuitively think about how our judgement might be wrong. The benefit lies in its deliberate slow-thinking form.

Method

The task to initiate a Premortem Analysis is pretty straightforward. Wait for the moment when it becomes clear that the group finalises a project. Assemble your team and picture the following scenario:

We’re about to finalise this project. It looks like we’re all happy with the outcome. But let’s be sure about this. Imagine yourselves a few weeks, months or years in the future. Our decision has turned out to be spectacularly mistaken. Our project has failed. What went wrong?

Group members are then given time to put on their hindsight hats. Preferably, everyone gets a day or at least a few hours to ponder the question. They essentially engage in what could be described as a personal meditation on all the ways things can go sideways. The products of this phase are personal lists of potential causes for failure. This can be done in an unrestricted way. Or people can be given pointers to get their creative juices flowing. Here are a few ways to do it:

  • A quick and simple method is to have participants come up with five potential reasons for failure on the spot. Once they’re done and the most obvious answers are off their minds, prompt them to come up with five more. However, this can put a lot of pressure on people who work best by thinking things through thoroughly and in private.
  • Randy Pherson suggests encouraging participants to see the questions through their own life experiences. Who hasn’t failed spectacularly in their life in one way or the other? Perhaps a personal lesson learned is applicable to the project at hand.
  • A similar approach would be to prepare a folder of failures specific to an organisation or industry. Whenever something goes wrong, even if it’s just a close call, write it down. Regularly updated and browsed at the right time, it can serve as a reminder of what not to do (again). More broadly, anecdotes around the Law of Unintended Consequences and so-called cobra effects can prompt participants to think differently about their work. (I’ll have essays about those phenomena out soon.)

Once the brainstorming phase is done, individual results are collated and categorised into groups of similar potential errors. Finally, it’s time to review the original decision and make changes based on the findings. What has been learned and what actions need to be taken?

Premortem Analysis and Structured Self-Critique

It would be perfectly fine to end the analysis here and implement a decision. Unless we’re keen to challenge our analysis even more. Remember the two intel analysts I mentioned in the beginning? Richards Heuer and Randy Pherson have taken the Premortem one step further with a method called Structured Self-Critique. The technique was originally developed by Heuer. In his Handbook of Analytic Tools & Techniques, which you’ll also find on my list of books on critical thinking, Pherson combined the two.

Benefits

As Klein implied, human error in judgement is a major reason why projects fail. Think of all the informal fallacies, misapplied heuristics and other barriers to critical thinking we face. The Structured Self-Critique acknowledges the flaws in human reasoning but also our ability to pull ourselves out of it. To a degree. In any case, the difference to Premortem Analysis is twofold.

First, the Structured Self-Critique is more straightforward in the goal of being a critical evaluation of human decision-making. Second, the process is much more guided, covering all aspects of an assessment experience has shown to be relevant. Pherson considers it a distinct method that nevertheless complements Premortem Analysis. Consequently, we’d jump into it right after we’ve finalised our Premortem results.

Method

The Structured Self-Critique turns every participant into a devil’s advocate once again. Here’s how Pherson suggests you phrase the task for the group:

Reemphasise that all analysts in the group are now wearing a black hat. They are now critics, not advocates, and will be judged by their ability to find weaknesses in the previous analysis.

As opposed to the basic Premortem Analysis, the Self-Critique comes with a purposeful structure. Since this method originates from analysts questioning their analytical judgements, I’ve adapted it slightly:

  1. Sources of uncertainty: Do we expect a single solution or answer or are there alternatives? Is the situation fluid or static? Have we considered basic decision-making principles such as Ockham’s Razor and Chesterton’s Fence?
  2. Analytic Process: To what extent were alternative hypotheses, assumptions and opinions considered? Would different decision-model yield different, more accurate results?
  3. Critical assumptions: If turned out to be wrong, which of our original assumptions would have the biggest impact?
  4. Evidence: What’s the quality of our evidence? What evidence was missing, anomalous or rejected and why? What evidence would it take to change our minds about our current approach? Is there any evidence we’d expect to see if we made the right decision?
  5. Information gaps: What information was unavailable and why? Is there any information we didn’t consider because we didn’t think it was relevant? If so, why?
  6. Out of our control: How might social, economical, political or environmental changes impact our decision?
  7. Expertise: To what extent might we be lacking the expertise to come to a true conclusion or make the right decision? To what extent are we acting based on unconscious incompetence?
  8. Deception: Have we been deceived by an adversary or competitor into making the wrong call?

When participants have answered the questions, the (supposedly) easy phase of criticism ends again. The team switches back to a constructive phase in which they consider what was learned from the exercise. What must be done to strengthen decisions or address weaknesses and risks?

The Limits of Premortem & Self-Critique

Clearly, the success of the method depends on the quality of the participants’ ideas. People are rewarded to criticise the group’s thinking by design. However, when reasons become personal or incriminating to colleagues they may still fall under the table.

Still, I appreciate the combination of the two methods. Participants can increasingly get used to critiquing a team’s planning and judgement, including their own. This is particularly true when both techniques are used on a regular basis. Because every unspoken idea that surfaces during the session has the potential to save the day.

Premortem and Self-Critique Case Study

Let’s apply both, the basic Premortem Analysis and Structured Self-Critique, to a high-stakes everyday decision:

As first-time home buyers, should we buy and move into a new house in Australia’s Blue Mountains?

Premortem Analysis

We begin our family brainstorming session. Get the kids. Invite your friends. Because who doesn’t love a good analytical session on a Sunday afternoon? We’ll be using the quick version. So from the top of their heads, everyone jots down five reasons how the decision could turn out to be wrong. Here’s what it could look like.

  1. We’ve lost our jobs and our son’s summer job can’t pay the mortgage alone.
  2. The foundations of the house turned out to be seriously damaged, resulting in costly repairs. Have you seen one of those house-flipping shows lately?
  3. There are no children in the neighbourhood to make friends with, so our kids are terribly unhappy.
  4. A natural disaster irreparably damaged our new house, rendering us homeless.
  5. The next martial arts gym turned out to be 100 km away.

I think the brief list shows that the value add lies in the different perspectives everyone involved brings to the table. For example, while the parents might focus on the potential financial disaster, the kids might have their free time activities on their minds. But we’re not done yet.

Structured Self-Critique

Our fun analytical Sunday session isn’t over. There’s still coffee to drink, cake to eat and self-critique to be levelled. Here’s what a, admittedly crude, hypothetical second phase could look like for our house conundrum:

  1. Sources of uncertainty: We’ve not inspected that many houses. This might not be the right one. Perhaps there are better options or we should postpone moving altogether.
  2. The process of analysis and decision-making: We didn’t really talk this through with the kids. Even though one reason we want to move was to give them more space to play.
  3. Critical assumptions: We just assumed the kids would find friends anywhere. Or that we would keep our jobs and be able to pay the mortgage. We don’t have a plan if even one of us is unable to work.
  4. Evidence: Let’s go over our observations from the house inspection again. The only reason we think the plumbing of the 30-year-old house is okay was that the real estate agent told us so. If we learned that the plumbing was in need of repair, it would change our minds instantly.
  5. Information gaps: There was no time to have the foundations of the house thoroughly checked. Christ, now that I think of it, we only inspected this house for ten minutes and now want to spend hundreds of thousands of dollars on it.
  6. Out of our control: The Blue Mountains are an area prone to bushfires. Our potential new home might very well be at risk.
  7. Expertise: Perhaps we’re simply lacking the knowledge and expertise about home ownership and mortgages to come to a proper conclusion.
  8. Deception: The real estate agent certainly has a motive to make us believe this was a good buy. That doesn’t mean she’s been deceiving us. Still, spending some extra dollars on an independent expert might be a good idea.

Some of the brainstormed reasons for failure and self-criticisms can be easily addressed. A quick Google search or a second inspection will do. Other reasons may have already been adequately addressed so they can be safely dismissed. Again others require more time and investment. As such they have more impact on the viability of our initial conclusion. But it’s only in the final analysis that the family’s decision can be affirmed or reconsidered.

I’m not going to lie, doing this for the first time is a lot of extra work. But when it comes to high-stakes decisions, I dare say it’s better than riding a dead horse, the awkward state you’re in when your project has died and nobody wants to acknowledge it.

Closing Thoughts

It’s much more enjoyable to contemplate failure than to experience it. It’s easier on the nerves, too. But we should be realistic. Conducting a Premortem Analysis and Structured Self-Critique is no guarantee to avoid making wrong decisions altogether.

Even if correctly identified, not all potential factors for failure can be mitigated. There will always be trade-offs and residual risks. A Black Swan might still be lurking in the shadows. But in knowing that we did everything in our power to anticipate and prepare for the unexpected, there’s a better chance we only suffer once.