Optimism Bias and Strategic Misrepresentation: The Overly Optimistic Picture

How we can counter these to empower change through informed decision-making and realistic forecasts

Happy Tuesday, Transformation Friends. Another week, another opportunity to go Beyond the Status Quo.

This week’s article will start a series looking more into the human side of transformation. We’ll examine some cognitive biases affecting our transformation efforts for the next few weeks.

First, some quick level-setting.

A bias is a leaning or preference for or against someone or something, often formed unfairly or unreasonably.

As humans, we all have biases, whether conscious or unconscious.

Cognitive biases are consistent mistakes in thinking that happen when we misunderstand the information we encounter. They occur when we are processing and interpreting information. There are more than 175 different types of cognitive bias.

Today, we’ll start by looking at optimism bias: when we think things will turn out better than past experiences suggest.

We will also look at a related concept called strategic misrepresentation: intentionally playing up positives and downplaying negatives to make things seem more favourable.

Unfortunately, as we’ll see today, these factors negatively impact our transformation initiatives by creating unrealistic expectations. I’m sure you’ve encountered both of these in your professional lives and seen their impact.

So, grab your morning coffee and let’s dive in.

Optimism Bias and Strategic Misrepresentation

Let’s start with Optimism Bias. Flyvbjerg (2006) defines it as,

a cognitive predisposition found with most people to judge future events in a more positive light than is warranted by actual experience.

Essentially, people think things will turn out better than they usually do. We see the future through rose-coloured glasses.

Optimism bias happens for many reasons. When part we are part of, especially leading, an initiative, we are inclined to hope for the best and believe in our ability to succeed, making us overlook risks.

Now, Flyvbjerg defines strategic misrepresentation as,

the tendency to deliberately and systematically distort or misstate information for strategic purposes.

Simply put, it’s justifying the bending of truth to achieve goals. For example, someone might twist the facts to make a project look more appealing to get funding.

These tendencies lead us to overvalue benefits, undervalue time and costs, and ignore risks. They significantly contribute to Flyvbjerg’s Iron Law of Megaprojects:

"Over budget, over time, under benefits, over and over again."

Flyvbjerg’s extensive research on over 16,000 projects across 136 countries reveals that 99.5% of megaprojects face delays, budget overruns, or benefit shortfalls—leaving only a slim 0.5% that meet their time, budget, and benefit projections (Flyvbjerg, 2023).

The Impacts

Painting an overly optimistic picture influences our projects significantly. Here's how:

Inaccurate Forecasts and the Planning Fallacy

Optimism bias and strategic misrepresentation create a significant gap between forecasts and reality (Prater et al., 2017). This leads to the "planning fallacy," an assumption that everything will go as planned, setting unrealistic expectations from the outset through underestimated costs, time, and inflated benefits. (Flyvbjerg, 2006).

Persisting Accuracy Issues

What's striking is that despite advancements in our project management approaches and control frameworks, the accuracy of project performance forecasts hasn’t really changed. Research shows this is fueled by underestimation at the baseline, largely due to optimism bias (Prater et al., 2017).

The Winner’s Curse

The winner’s curse, originating from competitive scenarios, like auctions, suggests that the winner, willing to pay the most, likely overvalued the item. This concept extends to projects, particularly procurement processes, where the lowest bidder wins but may undervalue the project, compromising quality and schedule (Meyer, 2014). Also, plans or options that seem the most cost-effective and beneficial are usually selected, fostering optimism.

Persistent Optimism

Meyer (2014) discusses two types of optimism bias impacting projects.

1. In-project optimism bias makes decision-makers don’t stop failing projects, rather they stick with them, wrongly believing they can make up for lost time and overspending.

2. Post-project optimism bias makes decision-makers expect better results than they first thought. This optimism grows even when there are signs it shouldn’t and success becomes not only meeting project goals but also living up to management’s high hopes.

Mitigation

Optimism isn’t all bad; it fuels motivation and resilience. However, a balanced approach mixing optimism with realism is vital for accurate forecasts and realistic expectations (Lovallo & Kahneman, 2003).

Prater’s 2017 literature review indicates that the top-recommended mitigation method for optimism bias and consequently strategic misrepresentation is Flyvbjerg’s Reference Class Forecasting.

Reference Class Forecasting and Kahneman’s Outside View

Reference Class Forecasting is based on Kahneman’s “Outside View,” popularized his book Thinking Fast and Slow (Kahneman, 2017). It contrasts with the “Inside View” which bases predictions on the specific details of a problem. The outside view, instead, predicts based on the outcomes of similar past situations.

For example, when predicting the time and cost of a website, we can either look at the specifics of the website and the design team (inside view) or the time and cost for similar websites (outside view).

Flyvbjerg’s approach uses the latter idea. It forecasts based on comparable past projects (the reference class). It creates a range of possibilities for things like time, cost, and benefits, based on the reference class. Then, by understanding the particular characteristics of our project versus those of the reference class, we use an empirical approach to forecast time, cost and benefits. This approach minimizes the influence of optimism bias and strategic misrepresentation (Flyvbjerg, 2006).

Embracing this approach requires a shift in mindset. The inside view has an appeal as it’s more detailed and allows us to think that our project is a special snowflake (Lovallo & Kahneman, 2003). Think again. In his book How Big Things Get Done, Flyvbjerg dedicates an entire chapter on why we need to think of all projects as “one of these” as a key to getting our forecasts right and managing risks (Flyvbjerg and Gartner, 2023). Incorporating insights from similar past projects yields more realistic forecasts.

Balancing Realism and Optimism Within Roles

Within organizations, role differentiation is key. Decision-making roles should lean towards realism, while action-guiding roles can incorporate more optimism (Lovallo & Kahneman, 2003). This balance nurtures an atmosphere that promotes innovative thinking alongside grounded decision-making.

Wrap up

Today, we’ve examined how realism and optimism impact our initiative and play a significant role in making decisions. We’ve also looked at methods to manage optimism in our work, Flyvbjerg’s Reference Class Forecasting, and ensuring we balance optimism in different roles.

As we end today’s post, let’s take some time to think:

1. Personal Reflection: How do optimism bias and strategic misrepresentation manifest in your environment?

2. The Inside View: Does your environment lean more on the inside view for information and forecasting?

3. Outside View: How might you incorporate more of the outside view into your environment?

Until next time, stay curious and I’ll see you Beyond the Status Quo.

References

Flyvbjerg, B. (2006). From Nobel Prize to project management: Getting risks right. Project management journal, 37(3), 5-15.

Flyvbjerg, B., & Gardner, D. (2023). How Big Things Get Done: The Surprising Factors that Determine the Fate of Every Project, from Home Renovations to Space Exploration and Everything in Between. Signal.

Kahneman, D., & Lovallo, D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk taking. Management science, 39(1), 17-31.

Kahneman, D. (2017). Thinking, fast and slow.

Lovallo, D., & Kahneman, D. (2003). Delusions of success. Harvard business review, 81(7), 56-63.

Meyer, W. G. (2014). The effect of optimism bias on the decision to terminate failing projects. Project Management Journal, 45(4), 7-20.

Prater, J., Kirytopoulos, K., & Ma, T. (2017). Optimism bias within the project management context: A systematic quantitative literature review. International Journal of Managing Projects in Business, 10(2), 370-385.