A chatbot might be able to write emails that sound human, but can the technology respond to staff questions just like the boss would? Research by Prithwiraj Choudhury shows that while a chatbot may save leaders time, employees might not view the communications as credible
If a chatbot can Slack convincingly in the boss’s voice, will employees follow orders once they realize the CEO is actually a machine?
A novel two-part study finds that an artificial intelligence (AI) chatbot trained to write like a technology company’s CEO responded to questions so believably that many employees thought the answers came from the boss himself. But there’s a caveat: When employees perceived that a response came from AI—even if it didn’t—they rated those responses as “less helpful” than those they thought came from the CEO, demonstrating a classic case of “algorithm aversion.”
It's an AI-age twist on the classic Turing Test, developed by British computer scientist Alan Turing in 1950 to judge whether machines could exhibit “intelligence.” Called the “Wade Test,” after the CEO of the company the researchers studied, the analysis is among the first to showcase AI’s ability to replicate the unique characteristics of a specific person’s writing style, says Prithwiraj Choudhury, the Lumry Family Associate Professor of Business Administration at Harvard Business School.
“We trained an algorithm to write using the same words and phrases and punctuation meter and grammar and mistakes and abbreviations the CEO uses,” Choudhury says. “What that tells us, generally, is that at least technologically, we can create a writing bot for any one of us.”
Generative AI stands to make workplaces more efficient by automating busy leaders’ routine tasks—such as the electronic communication that takes 24 percent of a CEO’s time, studies show. In theory, this would allow the executive to devote more time to strategic planning, for example. Yet the new research indicates that there’s still a long way to go before humans cede the craft of writing to machines—if that ever happens in an organizational context.
This article was provided with permission from Harvard Business School Working Knowledge.