The 7 Steps of Highly Effective Super-forecasters

Forecasting is the science of predicting future events.

Predicting the future can hold immense power, especially in business and investments where every decision hinges on a thesis — a bet on what’s to come.

Take Jeff Bezos as an example, when he started Amazon in 1994, the world around him was saying that the internet was a “fad” and it wouldn’t last. But, Jeff’s thesis was that the internet was just beginning and people would want to shop online. It turns out he was right.

However, this isn’t always the case. Humans often get it wrong.

Blockbuster ignored the rise of online streaming, and their thesis was that DVDs would continue to be in high demand. It turns out they were wrong.

Forecasting is a delicate art that requires skill which gets better with practice. This article has been inspired by Philip Tetlock and Dan Gardner who wrote the book “‘Super-forecasting.’’ The book studied 1000’s of forecasters to discover what separates the best from the average.

We’ll begin by breaking down why humans suck at forecasting and the mindsets super-forecasters have, before breaking down a 7 step process you can use to get better at making predictions at the end.

Humans suck at forecasting

Our brains aren’t wired to predict. Daniel Kahneman, the author of thinking fast and slow, highlights why. His research concluded that the brain operates on two systems.

System 1 is our initial response without deliberate thinking, and it is designed to jump to conclusions with little evidence. This system saves us energy. Imagine the habits you’ve formed, like when you drive to work you don’t have to think about the directions, it’s wired into your system 1 thinking.

On the other hand, system 2 kicks in when it’s required to deliberately think about a situation. Imagine you’re driving in a foreign city and trying to get from one side to the other, your system 2 will have to kick in.

Our reactions to a situation or question are initially driven by our system 1, and then followed by our system 2 if required. However, our system 1 will always form a bias that our system 2 is inclined to follow. This irrational decision making process tends to lead to predictions based on bias, rather than numbers and probabilities.

To put its impact into perspective, a dart throwing chimp has the same chance of hitting the board as the average forecasting expert does on predicting correctly.

While researching the book, Philip set up a government funded forecasting tournament. The tournament allowed ordinary people to compete against professional forecasters. And guess what? Ordinary people beat the professional forecasters, by a lot.

Philip discovered a subset of superforecasters within the group. But, they weren’t what you would typically expect. One superforecaster was a former ballroom dancer and another was a retired computer programmer.

What Philip ultimately found was that knowledge did matter in forecasting, but only to a certain point. What mattered most was the habits and mindsets forecasters had built, rather than if they’d had the best education.

The super-forecaster mindset

Super-forecasters employ systematic, rational methods to answer questions, clashing head-on with our natural intuitions. This shift in thinking requires new habits and practice. However, the results will extend beyond your everyday life, impacting how we make decisions and communicate.

At the core, super-forecasters readily use and analyse numerical data. To make accurate predictions, data is used to understand patterns and then make a numerical prediction. Without it, you might as well be tossing darts with a chimp.

Having a numerical outlook creates the foundation of probabilistic thinkers, who think in terms of probabilities, rather than fate. This approach challenges the notion of ‘everything happens for a reason’ and replaces it with the understanding that every outcome arises from a complex interplay of probabilities. Even your seemingly destined encounter with your partner can be viewed through this lens, as a convergence of chance encounters and individual choices leading to a specific outcome.

A common counterargument against probabilistic thinking hinges on the concept of black swan events (highly unpredictable events outside the realm of normal expectations). The recent hurricane that happened in Mexico in late October ’23 serves as a great example of a black swan event. Despite extensive pre-storm analysis, this event caught everyone by surprise, leading to tragic consequences. Hurricane prediction models are based on historic data of events, without taking variable factors from climate change into account. The warming seas, a known variable, likely contributed to the hurricane’s unexpected intensity by powering its energy. This highlights how probabilistic thinking can acknowledge, if not fully predict, the impact of such ever-evolving variables.

Super-forecasters always approach predictions with first principles thinking. This means meticulously breaking down the problem into its known and unknown components, then scrutinising each one to uncover the truth.

Imagine you’re tasked with predicting Joe Biden’s chances of winning the next US election. Super-forecasters would start by identifying key components: his performance across various functions, public opinion polls, and his stance on major geopolitical issues. Then, they wouldn’t just gather information on these components, but actively question them: Are the polls unbiased? Do they capture the entire electorate? How important are policy stances compared to other factors? This deep interrogation of both known and unknown aspects is the hallmark of first principles thinking.

To learn more about first principles thinking, read my article here.

When questioning our assumptions, embracing multiple perspectives is crucial. Humans tend to view life through their own lens, clouded by biases and preconceived assumptions. We should behave like a dragonfly instead. Like humans, dragonflies have two eyes, but theirs work very differently. Depending on the species, there can be as many as thirty thousand lenses in a single eye, giving it a unique perspective. Consider information from all angles, even those we disagree with, to develop accurate predictions.

Super-forecasters’ work doesn’t stop at the initial prediction. In fact, it’s just the beginning. Super-forecasters update their predictions regularly in calculated increments to respond to new information. Imagine predicting a 31% chance of Biden winning the election. But then, he delivers a poor performance in the final presidential debate, as confirmed by media analysis and public opinion. How would you adjust your prediction? The key lies in finding the right balance between overreacting and underreacting. Every update requires the same critical approach to information, diligently questioning and removing biases to reach the most accurate estimate.

Super-forecasters gather information from all angles, from experts to sceptics, historical trends to social media threads. Every data point is scrutinised, every assumption questioned. It’s not about “will it happen?” but “how likely is it?”, and their approach yields predictions that beat guesswork.

The 7 super-forecasting steps

While predicting the future might seem like peering into a cloudy crystal ball, super-forecasting offers a practical framework to crack the code.

So, how do these elusive super-forecasters go about making accurate predictions? Let’s break down the 7 steps:

Step 1: Unpack the Question

Start by dissecting the question like a surgeon analysing an intricate organ. Break it down into its core components, like the different factors influencing the outcome. Think of it as deconstructing a Lego castle into individual bricks.

For example, predicting the success of a new tech startup would involve analysing elements like market demand, product innovation, competitor landscape, and funding potential.

Step 2: Known vs. Unknown

Identify the known elements with solid evidence and data. These are your building blocks. Then, meticulously scrutinise the unknowns. Are there hidden assumptions lurking within? Question every variable, like a detective interrogating a witness.

Is the market demand accurately measured? Are competitor threats accurately assessed? Remember, untangling the unknowns is crucial for a clear forecasting picture.

Step 3: Outside View

Shift your perspective to the outside view. Compare your specific question to similar situations in a broader context.

For example, compare the predicted startup’s market opportunity to the growth trajectory of similar startups in the past. This “helicopter view” helps identify patterns and common pitfalls, reducing the uniqueness bias of your initial analysis.

Step 4: Inside View

Now zoom back in for the inside view. Analyse the unique characteristics of your specific case.

What sets this startup apart from its peers? Are there innovative features with disruptive potential? This deep dive ensures you don’t overlook crucial differentiators that could propel the startup to success.

Step 5: Crowdsourcing Wisdom

No one has a monopoly on foresight. Gather insights from others, like prediction markets and diverse viewpoints. Look at how others are analysing the same question and compare their arguments with your own. This cross-pollination of ideas helps break out of echo chambers and uncover blind spots.

Step 6: Dragonfly Synthesis

Just like a dragonfly’s multifaceted eyes offer a panoramic view, synthesise all the gathered information into a comprehensive picture. Weigh the known and unknown, the outside and inside views, and the collective wisdom of others. This holistic analysis forms the foundation for your final prediction.

Step 7: Precise Judgement

Finally, express your prediction with clarity and precision. Don’t settle for vague pronouncements like “it will be successful.” Instead, assign a concrete probability based on your meticulous analysis. Remember, a 60% chance of success carries more weight than a nebulous “likely to work.”

Remember, super-forecasting is a continuous process. As new information emerges, update your predictions with calculated adjustments, avoiding overreactions or under-reactions. By diligently applying these 7 steps, you can transform from a dart-throwing chimp into a seasoned predictor, navigating the uncertain future with greater clarity and confidence.

Conclusion

Predicting the future can feel like reading tea leaves, but this article offers a practical guide. This method isn’t magic, it’s about breaking down questions like a detective and then weaving together all the clues using logic and diverse perspectives. Imagine analysing a new business idea like a dragonfly with a thousand eyes, considering market trends, competitor threats, and unique strengths. Finally, instead of guesswork, you assign a clear probability of success based on your detective work.

Superforecasting’s power goes beyond predicting. Its principles can transform how we lead, make decisions, and even bring up our children. Just like a CEO using data to steer their company or a strategist mapping out scenarios with foresight, we can all apply this approach to prepare for whatever tomorrow brings.

If you’re enjoying this article, make sure to subscribe to my newsletter here where I post every Friday UK morning.