Augmented reality is changing how blue collar workers are trained, one headset at a time

Technology has played a central role in white-collar workplaces for decades, with a screen in front of every employee; now devices like Microsoft HoloLens is putting technology in the hands of workers who don't sit behind a desk

By Karen Weise

Published: Jul 11, 2019

Augmented reality is changing how blue collar workers are trained, one headset at a time

Toby Bouska Jr. assembles semitruck cabs, with the help of Microsoft HoloLens, at Kenworth Truck Co. in Chillicothe, Ohio, May 29, 2019. High-tech devices have played a central role in white-collar workplaces for decades but now, companies like Microsoft see a multibillion-dollar opportunity to get more personal technology to workers who don't sit behind a desk. Image: Andrew Spear/The New York Times

CHILLICOTHE, Ohio — When Toby Bouska Jr. started assembling cabs for Kenworth semitrucks last year, he learned the ropes by observing longtime workers at the factory. But it wasn’t exactly engaging, and he didn’t get much practice doing the job himself.

“It’s them doing the job, and you just have to watch,” said Bouska, 21, who works at Kenworth’s plant in Chillicothe. “I’m not really good at just sitting there watching.”

But then his managers had him train in a new way: with a high-tech headset. They gave him a Microsoft HoloLens, a device that blends digital imagery with the real world. When he wore the headset, it overlaid digital arrows and diagrams over the parts he was looking at, helping to guide his work.

“With the HoloLens, it’s just you and the directions,” Bouska said. He said he had picked up his first new task in about 20 minutes.

After the success with Bouska’s training, Kenworth’s parent company, Paccar, has ordered 50 of the devices. Five will be coming to the Chillicothe plant, which employs more than 2,000 workers, and the manager plans to use them to train employees on at least two dozen tasks.

High-tech devices have played a central role in white-collar workplaces for decades, with a screen in front of nearly every face, and employees’ days spent on email, spreadsheets and video conferences. Now, companies like Microsoft see a multibillion-dollar opportunity to get more personal technology, including the HoloLens, in the hands of workers who don’t sit behind a desk.

The new push goes beyond tools to perform a particular task, like a clerk ringing up a customer with a tablet or a robot moving materials around a factory. It is meant to integrate the tools into the corporate life of a company, like training, scheduling and regular communications. The efforts are enabled by cloud computing, which makes it easier to deliver information via a smartphone app or a mixed-reality headset.

Google is going after the market, as is Salesforce, with its acquisition of Quip, which makes programs for worker productivity. Plenty of niche tech products target particular industries and tasks.

Perhaps no tech company, though, is more aggressive than Microsoft at pursuing so-called frontline or firstline workers, who do the actual production, sales and service work for customers. Microsoft built much of its riches on supplying technology to businesses and is pushing a variety of products to the workers.

Microsoft estimates that 2 billion frontline workers have access to fast internet connections and are, in theory, potential customers. In a call with investors this year, Satya Nadella, Microsoft’s chief executive, said selling products for firstline workers expanded the market Microsoft could tap into.

Emma Williams, a Microsoft executive who develops productivity tools for various industries, like health care and retail, said there was a big, open playing field.

“With firstline workers, it’s pretty fascinating,” she said. “There really isn’t a large incumbent.”

Technology already surrounds many frontline workers, but many of these workers do not even have a corporate email account, so they create workarounds to communicate. Team Inc., a company that performs maintenance and repairs at industrial sites like refineries and pipelines, realized last year that almost half of its field technicians used personal email accounts and cellphones to communicate, said Tracy Terrell, Team’s chief information officer. The company’s leaders decided that was too risky.

“We don’t want to be emailing to Yahoo accounts or sending technical information via text, because that’s kind of our trade secret,” Terrell said.

Any new tool has to be as easy as consumer apps for workers to adopt them, he said. His company’s field technicians can have little patience for impractical solutions, so his team has been easing them into a version of Microsoft Teams, a messaging platform, designed specifically for firstline workers.

It took a year to get everyone to sign up for the corporate logins needed for Teams. And because old habits die hard, the system automatically sends the technicians a text message to check the app when they have a message to read. There, managers in the office can communicate directly with people in the field. Some technicians have created group chats, like one for mechanical bolt specialists, to help troubleshoot repairs.

More than 500,000 organizations use Teams, though Microsoft doesn’t break down how many use the version designed for frontline workers. That version can sync with programs that schedule workers, letting them swap shifts, or only send messages to people working at a particular time.

“There is no question this will work,” Brad Reback, an analyst at the investment bank Stifel, said of the push to reach frontline workers. “The speed at which companies decide to roll it out? We’ll see.”

While Teams is already gaining traction, the HoloLens effort will most likely take years to develop, because of the investment required to buy the headset and put it to use.

The latest version of the device is more tailored to workers. Microsoft made the headset better balanced so workers can wear it longer. It also made the HoloLens easier for an in-house IT department to deploy, like letting workers log in to it with their regular corporate account and password and building basic applications for key uses.

One, called Remote Assist, lets a worker in the field interact with a specialist somewhere else. ZF, a large automotive supplier, has been using the HoloLens to help with plant maintenance.

In the past, when something broke in South Carolina, an expert might fly in from ZF’s headquarters in Germany to fix it. Now, the German expert can look at exactly what a factory technician wearing a HoloLens sees and help troubleshoot a problem.

“You can circle it and say, ‘This is what I’m talking about,’ ” said Robert Copeland, who leads the HoloLens adoption for ZF.

When Paccar experimented with an earlier version of the HoloLens, it was bulky and required developers with skills in building video games to make the right applications.

“The initial reaction for us was, ‘Cool gizmo, but so what?’ ” said Rod Spencer, who runs the Kenworth plant in Chillicothe. “We couldn’t figure out how to get it applied in the real world.”

Spencer said his view had changed with the new version and the corporate tools that Microsoft built, like an application that makes it easier to build step-by-step training.

Bouska thinks most of his colleagues will warm to donning a large headset in training, though a few may resist.

“Some,” he said, “still use flip phones.”