The first large-scale study of a ChatGPT-like assistant in the workplace finds that it can benefit less experienced employees — and make customers happier
Providing workers with a generative AI tool similar to ChatGPT can lead to more productive workers, happier customers, and higher employee retention.
Image: gremlin/Getty Images
Since generative AI went mainstream a year ago, it has inspired an equal measure of hype and fear. Boosters of tools such as ChatGPT and DALL-E predict that they will transform our economy, while skeptics worry about their potential to produce inaccurate or harmful results and ultimately replace workers. But until recently, no one had tested what really happens when companies unleash generative AI at scale in real workplaces.
The first such study, released as a National Bureau of Economic Research working paper earlier this year, found the best-case scenario: Providing workers with a generative AI tool similar to ChatGPT can lead to more productive workers, happier customers, and higher employee retention.
Researchers studied nearly 5,200 customer support agents at a Fortune 500 software firm who gained access to a generative AI-based assistant in a phased rollout between November 2020 and February 2021. During support chats, the generative AI tool shared real-time recommendations with operators, suggesting how to respond to customers and supplying links to internal documents about technical issues.
Compared to a group of workers operating without the tool, those who had help from the chatbot were 14% more productive, on average, based on the number of issues they resolved per hour. The AI-supported agents ended conversations faster, handled more chats per hour, and were slightly more successful in resolving problems. Notably, the effect was largest for the least skilled and least experienced workers, who saw productivity gains of up to 35%.
Big Gains With Fewer Pains
This piece originally appeared in Stanford Business Insights from Stanford Graduate School of Business. To receive business ideas and insights from Stanford GSB click here: (To sign up: https://www.gsb.stanford.edu/insights/about/emails)