Socializing an idea? Experiments can help convince your audience

“Decision-makers can and should view experiment results with a critical eye,” Karp explains.

What the ‘smark’ knows

In recent years, a number of studies have evaluated the merit of using scientific approaches to business decision-making. There’s a growing consensus that testing the validity of ideas can help a company decide how to reshape its business model, guide product innovation and investments, and steer training and hiring decisions. These include randomized controlled trials, iterative hypothesis testing, focus groups, and crowdfunded product development.

But some studies indicate that those designing and conducting experiments don’t always objectively set out to gather as much information as possible to help managers reach decisions. Instead, experiment designers often try to influence managers to reach certain conclusions to secure a strategic advantage—be that more resources or better access to a new technology.

Prior to her recent study, Karp says she personally observed the early stages of 13 pilot programs and experiments tied to the adoption of new technologies or other innovations at companies. In each case, she says, the business experiments aimed to produce results that reflected the views of the people overseeing them.

Karp leans on the language of professional wrestling to explain the phenomenon. For example, the “kayfabe”—a match that’s being staged—might be a presentation to a group of investors. A decision-maker is influencing a “smark”—or a smart mark—who is aware that the experiment is rigged as part of the staged performance. However, the smark still benefits from access to the information.

Also read: Understanding decision-making: Inherent risk preferences

When bias is ‘a feature, not a bug’

Karp devised a model to examine how a founder might use an experiment to convince a venture capitalist to invest. The founder could structure the experiment to favor a particular investment—a situation that the investor would expect in a pitch.

The study’s findings confirm what Karp had been observing in the field: Experimental processes shift from the objective to the subjective during the early stages of designing experiments. 

  Karp likened the bias inherent in business experiments to venture capitalists listening to funding pitches from entrepreneurs. The audience understands that persuasion is ultimately “a feature, not a bug" of the process of receiving information and reaching critical decisions, as one venture capitalist explained to her.

“The venture capitalists know that the experiment is likely designed to persuade,” says Karp. “The better you are at influencing—the venture capitalist knows that; they are looking for people who know how to play the game.”

Persuasion has some plusses

The researchers are not trying to dissuade executives from conducting experiments before making key business decisions, Karp says, stressing how critical it is for companies to carefully research and analyze options when confronted with challenges. Nor are researchers encouraging decision-makers to pass judgment on those whose experiments almost inevitably veer from objective analysis to subjective persuasion.

Instead, decision-makers need to accept that those designing and conducting experiments are human—and that no experiment is perfect nor free from some form of bias. Experiments designed to persuade their audiences are still worthwhile, Karp says, as long as decision-makers understand their limits.

Indeed, storytelling has become a powerful communication skill for leaders today. The right experiment might lead to a compelling narrative that helps a startup land its next investment or a company use the right technology vendor, the research indicates.