Get New Ideas by Email. Join 3k+ Readers.

Black Swan Theory: How to Predict the Unpredictable

There’s no such thing as a black swan. That was the consensus in Europe for centuries. Over the years, black swans became synonymous with impossible events. Until 1697, when a Dutch explorer ran into them in today’s Western Australia. The meaning of the term changed and it became a metaphor for something that was unlikely, but certainly possible. Now it was understood that a single piece of evidence could disprove any claim of impossibility. However, this brought up an entirely new challenge, especially for organisations dealing with risks: How can you predict the seemingly unpredictable? Today, this concept is known as the Black Swan Theory.

What Is the Black Swan Theory?

The Black Swan Theory is a metaphor for an unpredictable and highly improbable event that has severe consequences if it does occur. In its current form, it was developed by essayist and former risk analyst Nassim Nicholas Taleb. This is how he defined the concept in his 2007 book The Black Swan: The Impact of the Highly Improbable:

A black swan is an unpredictable event that is beyond what is normally expected of a situation and has potentially severe consequences. Black swan events are characterised by their extreme rarity, their severe impact, and the widespread insistence they were obvious in hindsight.

The concept of the Black Swan has been applied in a variety of fields. Let’s look at three different applications and examples.

Applications of the Black Swan Theory

First, we take the perspective of an intelligence analyst. Then we explore the point of view of a master negotiator. Finally, we’re going to circle back to Taleb’s very own interpretation. How does each of them attempt to cope with the unpredictable?

1. Black Swans In Intelligence Analysis

Broadly speaking, intelligence analysts deal in knowledge and foreknowledge. As I detailed in my essay about the Intelligence Cycle, they collect and analyse data to describe, explain, evaluate or predict events. The latter is the supreme discipline of the profession. Criminal intelligence, for example, may try to get a sense of where the next burglary will most likely happen; to allocate limited resources more efficiently.

In a national security context, analysts may aim to predict and prevent an impending terror attack. By definition, the unknown unknowns of the Black Swan Theory are the hardest to predict. 9/11 is a much-cited example of a rare, unlikely and unforeseen event. One that changed the world as we knew it, but appeared preventable with the benefit of hindsight. If only we had gathered more data or paid better attention to what we already knew.

Black Swans hang like a Sword of Damocles over national security analysts. As intelligence analysis researcher Kristian J. Wheaton writes:

The black swan has become a metaphor for the limits of the forecasting sciences. At its best, it is a warning against overconfidence in intelligence analysis. At its worst (and far too often it is at its worst), the black swan is an excuse for not having wrung every last bit of uncertainty out of an estimate before we make it.

In other words, Black Swans can act as a driver of thorough analysis. But also as a Get Out of Jail Free card to avoid accountability for not having done your due diligence.

Anticipating Black Swans

One way the intel community tries to bring more transparency and validity to future estimates is by using so-called Structured Analytic Techniques. Premortem Analysis and Deception Detection are two examples of such methods. As the name suggests, they attempt to take the guesswork out of the exercise as much as possible. The US Government’s Tradecraft Primer on SATs and the Handbook of Analytic Tools & Techniques are probably the most noteworthy resources on SATs. You’ll find them on my reading list with books on intelligence analysis.

But how do SATs try to address Black Swans? Say you wanted to predict the winner of the next football world cup. You could use the Foresight Structured Analytic Technique which guides you through a process that looks roughly like this:

  1. Based on past and current data and experience, key assumptions about the teams’ performances are collected, condensed and grouped to generate hypotheses about future developments. One might be that Argentina has momentum from recent wins and Lionel Messi is at the top of his game internationally. Ideally, assumptions and hypotheses are sourced from a broad range of sports data, analysts and football experts.
  2. The hypotheses are then used to create different future scenarios of potential world cup outcomes. These can be told in stories that illustrate how things might develop. An Underdog Scenario might see key players of all top teams being injured while a dark horse team scores a few lucky goals. Ideally, the scenarios represent both, expected developments and highly improbable ones.
  3. In the last step, indicators are generated to enable the analyst to track if and when we’re moving towards one of the scenarios. Ideally, these indicators are observable in the real world, valid and reliable. For instance, we could track the health status of key football stars.

This technique is certainly more useful and transparent than just hazarding a guess. Though, it’s not perfect. Obviously.

The Limits of Structured Analytic Techniques

While there’s an infinite number of hypotheses and scenarios that could be created, eventually a selection has to be made on which one to consider and monitor via indicators. Our Black Swan might still fall through the cracks and Germany may win the cup a fifth time.

More precisely, intel analysis researcher Stephen Coulthart sees the lack of a “stopping rule” in hypothesis generation as a major problem. Nobody knows when to stop generating hypotheses and scenarios to avoid Black Swans. Even with hundreds of possible outcomes on your radar, the question is which ones we should pay attention to, act on or – in the case of our world cup example – bet on.

In sum, intelligence analysts cope with unknown unknowns by attempting to reduce uncertainties and anticipate future developments. Now, what about Black Swan Theory in a negotiation context?

2. Black Swans in Negotiation

The second conceptualisation of the Black Swan Theory originated in the world of hostage negotiation. It’s no coincidence that Chris Voss, former FBI negotiator and author of Never Split the Difference, named his business after our elusive water bird. For Chris and The Black Swan Group, negotiation is all about uncovering that one piece of information that will fundamentally change the dynamic of personal exchange and lead to a deal.

A Bank Heist Gone Wrong

Chris’ prime example of a Black Swan in negotiation is the story of a hostage situation in Rochester, New York in 1981. Back then a man called William Griffin took nine bank employees hostage after he had already killed several others. Griffin gave the police an ultimatum. He wanted a shoot-out at 3 pm. Otherwise, he’d start killing hostages.

At that time, it was both unheard of and unthinkable that a criminal would actually murder a hostage at the end of a deadline. Until that day in June when Griffin shot one of his captives to provoke being shot by the police. Suicide by cop. The unexpected had happened.

In hindsight, Griffin’s background and behaviour made his intent obvious. It goes to show: In the world of negotiation, whether in law enforcement, business or personal exchanges, Black Swans are a very personal phenomenon. People’s actions may seem erratic or crazy and not fit into a pre-conceived frame of reference. In reality, it may be a manifestation of someone’s unique circumstances and unspoken needs.

Spotting Black Swans Before They Happen

This insight serves as the springboard for Voss’ strategy to uncover Black Swans. I summarise it in three points:

  1. Your knowledge, expertise and experience are the ground you stand on. Use it as a guide to discover the unknown unknowns. The more you know about negotiation or human behaviour in general and your counterpart in particular, the better your chances of identifying a piece of information that gives you the upper hand.
  2. Black Swans live in people’s worldviews. Understanding their most cherished beliefs, religion, ideology, emotions and motivations is key. If you’re unable to comprehend the world according to your counterpart, if you can’t steelman their case, you’re less likely to discover that one piece of the puzzle that makes the difference.
  3. The solution is to pay close attention to every aspect of your counterpart’s world and communication and to establish rapport on shared common ground. Observe how they behave in an unguarded moment. Arrange face-to-face meetings to enable you to dissect their verbal and non-verbal cues.

What differentiates Voss’ strategy from our intel analysis scenario is that negotiations are less abstract and much more targeted. Negotiators seem less in the business of prediction and more in the business of emotional forensics. Usually, there’s a specific deal you want to make, and a specific person you deal with and the exchange is rather dynamic. This gives a negotiator a better chance to probe for and discover previously unexpected relevant info.

Though, like intel analysts, Voss works on the assumption that the unknown unknowns can indeed be uncovered. You just have to ask the right questions. Here to disagree is the father of the Black Swan Theory, Nassim Nicholas Taleb.

3. Black Swan Theory According to Nassim Taleb

Taleb wouldn’t be Taleb if he didn’t have a unique and principled take on Black Swans. Allow him to remind us of the three characteristics: “rarity, extreme ‘impact’, and retrospective (though not prospective) predictability”. The latter is where he deviates from the analyst and the negotiator.

A Thanksgiving Surprise

Black Swan Theory and the Turkey Problem
The Turkey Problem

Taleb’s famous example is that of a Thanksgiving Turkey enjoying its life among humans. Year after year, it is fed and treated exceptionally well; building its firm belief that humans are quite awesome. Until one day in November when our turkey’s worldview will suddenly collapse. The point is that our turkey did not have the means to predict what was going to happen, making his encounter with the butcher a true Black Swan event.

According to Taleb’s reading, our analysts and negotiators from above are chasing false Black Swans. These are improbable events that could’ve been picked up on with better data, analysis, training or technique. If only there hadn’t been an oversight, lapse in judgement, ignorance or even willful blindness. Genuine Black Swans, however, cannot possibly be predicted even with all facts on the table. So, according to Taleb’s standards, preventable events aren’t Black Swans in the first place.

Dealing With the Inevitable Black Swan

To a degree, this is all semantics and branding. Though the question remains: How are we supposed to deal with the inevitable? Taleb suggests the following approach, which I’ve tried to adapt for personal use:

  1. Do not waste your time attempting to predict the unpredictable. Black Swans, properly understood, are inevitable outliers. You might draw conclusions based on the latest unforeseen incident (such as 9/11). The next outlier, however, will look very different and render you unprepared once again.
  2. Instead of focusing on the precise, prepare for the general phenomenon of the negative impact of Black Swans. By building robustness and anti-fragility. It’s better if fragile parts break early while they’re still small so they can be rebuilt stronger and withstand the negative impact of the improbable.
  3. Black Swans are dependent on your personal circumstances and perspective. The turkey did not see his fate coming. The butcher did. So avoid being the turkey by knowing your vulnerabilities and dependencies. Put yourself into a position where you have options and risks are more calculable. You’ll be less surprised on Thanksgiving Day.

Taleb’s rigorous interpretation holds true to the origins of the Black Swan Theory. Europeans were in no position to know about black swans until they encountered one. The indigenous population of Australia, however, had been well aware of their existence for years.

The future tends to turn out differently than expected. You can’t even predict with certainty what will happen in the next five minutes. There’s always a (minor) detail that causes our predictions to be rendered obsolete. Past experiences can only give us a remote idea of future experiences. In the realm of knowing and not knowing, the unpredictable remains, well, unpredictable.

Closing Thoughts

It seems like there are two schools of thought when it comes to the Black Swan Theory. The first one treats them like a solvable puzzle. The second school of thought treats them like an unsolvable mystery. In a sense, it comes down to the question if we should even try to predict the unpredictable. But perhaps that’s just semantics.

Inadvertently or not, both schools of thought end up coping similarly with Black Swans. They aim at reducing uncertainty and surprise effects by emphasising anticipation over prediction. They put on a holistic lens rather than obsessing about specific nightmare scenarios. Similar to High Reliability Organizations, they seem to pursue a balance between being in the detail and drawing conclusions for the big picture.

There’s a sense of humility in that attitude. Who would’ve thought that paying attention, being open-minded, thinking critically and having a healthy relationship with potential failure could help us navigate the world and reduce the chance of being thrown for a loop by a Black Swan? That was entirely unpredictable.