Lessons in decision-making: Confident people aren't always correct (except when they are)

Organizations thrive or fail based on the strategic decisions that stream from top leaders. Image: Shutterstock

Sometimes, the loudest, most confident voice in the room might indeed be the best decision-maker. Other times, the person who understands that they don’t know the answer—and therefore holds back in a discussion—may be wiser.

Whether groups and organizations reach good decisions depends on whether the right people are confident, suggests recent research by Thomas Graeber, assistant professor at Harvard Business School. His work tested the effects of meta-cognition—essentially, whether more skilled people are also more confident than the less skilled—and the benefits of “self-selection,” or what happens when people make decisions based on their confidence about how strongly to volunteer their own opinion in group decisions.

Organizations thrive or fail based on the strategic decisions that stream from top leaders. Should the company launch a new product or upgrade an existing one? Is it time to revisit outdated branding or stick with an established image? Executives aspire to project confidence in their decisions, lest they seem incompetent.

However, Graeber’s research suggests that leaders who are well-calibrated about what they know and don’t know, acknowledge their blind spots, and engage others in the decision process might have better results. When confidence is badly calibrated—such as in situations where people who are wrong are erroneously confident—outcomes are worse than the sum of individual decisions.

“In teams, and organizations more broadly, self-selection often plays a critical role: Individual team members decide whether to speak up and volunteer their opinion on a topic, whether they ask for someone else’s help or tend to purely rely on their own assessment,” explains Graeber. The outcomes “depend on whether the right people are confident or not.”

Graeber cowrote the paper, which was published in American Economic Review in July 2023, with Benjamin Enke, an associate professor in economics at Harvard University, and Ryan Oprea, an economics professor at the University of California, Santa Barbara.

How does one measure confidence?

In the first phase of the study, the team invited more than 2,000 people to perform 15 classic cognitive bias tasks, including:

•     The “knapsack problem”—a strategic reasoning challenge that invites people to fill a bag with the right combination of weights to maximize the load without breaking it.
•     A coin flipping task that demonstrates the “gambler’s fallacy,” the faulty tendency to think that previous events influence a future random event.
•     A bidding task to show how people tend to pay more for an item in an auction than it’s worth because of emotions and other factors, a bias called the “winner’s curse.”

Through this effort, the researchers collected more than 70,000 decisions.

In the second part of the study, the test subjects participated in simple auctions, betting, and committees in which they could bid, bet on, or vote for the accuracy of their own solution to each task. These decisions served to find out whether the aggregate outcomes that emerge from people self-selecting into participating more or less aggressively in an auction, market, or voting process are better or worse than the average of everyone’s individual decisions. How robustly people participate was strongly predicted by people’s confidence in the correctness of their answer.

Whether auctions, betting markets, or committees created improvements was strongly predicted by the degree of the “confidence calibration”: are subjects with the correct answer more confident than those with incorrect answers, equally confident, or even less confident? When the right people are confident, they dominate the outcomes of markets or organizations, which is beneficial overall. But when the wrong people are more confident, wrong ideas dominate aggregate outcomes, making things worse overall.

Also read: Too many choices: A model to predict open-ended consumer decisions

That manager who loves to talk

While Graeber and his coauthors called for more research on meta-cognition, their most important finding has immediate implications: People are often poorly calibrated about whether they are right and wrong, and knowing when to pause and question your intuition or the bravado of a particularly overzealous employee might be a critical soft skill. But how does a manager find those right people and allow their decisions to be influential?

First, assess the calibration of an employee’s confidence. “Do those people who appear most confident also make the fewest mistakes? Do mishaps often happen even though the people involved seemed highly confident in what they did?” asks Graeber.

Then, evaluate whether, among which people, and in which tasks surprises happen.

“Let’s suppose a corporate decision is made, and it turns out badly. An undesirable outcome will be more surprising to the more confident people; people who had low confidence in the outcome to begin with wouldn’t be surprised,” says Graeber.

“The exact same argument goes for a surprisingly good outcome: If everybody were highly confident that things would go badly, but then they unexpectedly turned out much better, people should be surprised. Surprises are a sign that confidence is badly calibrated.”

Graeber and his colleagues say that further study of the calibration of confidence may be of “first-order importance” for understanding behavioral economics’ influence on social science.

“Although behavioral economists have put great energies into studying how nudges, frames, familiarity, and learning influence biases themselves, we know next to nothing about how these same drivers of choices influence meta-cognition,” they write.

[This article was provided with permission from Harvard Business School Working Knowledge.]