Why Apple and Google's virus alert apps had limited success

Based on software from Apple and Google, the state’s smartphone app uses Bluetooth signals to detect users who come into close contact. If a user later tests positive, the person can anonymously notify other app users whom the person may have crossed paths within restaurants, on trains or elsewhere.

Cavey immediately downloaded the app. But after testing positive for the virus in February, she was unable to get the special verification code she needed from the state to warn others, she said, even after calling Colorado’s health department three times.

“They advertise this app to make people feel good,” Cavey said, adding that she had since deleted the app, called CO Exposure Notifications, in frustration. “But it’s not really doing anything.”

The Colorado health department said it had improved its process and now automatically issues the verification codes to every person in the state who tests positive.

When Apple and Google announced last year that they were working together to create a smartphone-based system to help stem the virus, their collaboration seemed like a game changer. Human contact tracers were struggling to keep up with spiking virus caseloads, and the trillion-dollar rival companies — whose systems run 99% of the world’s smartphones — had the potential to quickly and automatically alert far more people.

Soon Austria, Switzerland and other nations introduced virus apps based on the Apple-Google software, as did some two dozen American states, including Alabama and Virginia. To date, the apps have been downloaded more than 90 million times, according to an analysis by Sensor Tower, an app research firm.

But some researchers say the companies’ product and policy choices limited the system’s usefulness, raising questions about the power of Big Tech to set global standards for public health tools.

Computer scientists have reported accuracy problems with the Bluetooth technology used to detect proximity between smartphones. Some users have complained of failed notifications. And there is little rigorous research to date on whether the apps’ potential to accurately alert people of virus exposures outweighs potential drawbacks — like falsely warning unexposed people, over-testing or failing to detect users exposed to the virus.

“It is still an open question whether or not these apps are assisting in real contact tracing, are simply a distraction, or whether they might even cause problems,” Stephen Farrell and Doug Leith, computer science researchers at Trinity College in Dublin, wrote in a report in April on Ireland’s virus alert app.

In the United States, some public health officials and researchers said the apps had demonstrated modest but important benefits. In Colorado, more than 28,000 people have used the technology to notify contacts of possible virus exposures. In California, which introduced a virus-tracking app called CA Notify in December, about 65,000 people have used the system to alert other app users, the state said.

“Exposure notification technology has shown success,” said Dr. Christopher Longhurst, the chief information officer of UC San Diego Health, which manages California’s app. “Whether it’s hundreds of lives saved or dozens or a handful, if we save lives, that’s a big deal.”

In a joint statement, Apple and Google said: “We’re proud to collaborate with public health authorities and provide a resource — which many millions of people around the world have enabled — that has helped protect public health.”

Based in part on ideas developed by Singapore and by academics, Apple and Google’s system incorporated privacy protections that gave health agencies an alternative to more invasive apps. Unlike virus-tracing apps that continuously track users’ whereabouts, the Apple and Google software relies on Bluetooth signals, which can estimate the distance between smartphones without needing to know people’s locations. And it uses rotating ID codes — not real names — to log app users who come into close contact for 15 minutes or more.

Some health agencies predicted last year that the tech would be able to notify users of virus exposures faster than human contact tracers. Others said they hoped the apps could warn commuters who sat next to an infected stranger on a bus, train or plane — at-risk people whom contact tracers would not typically be able to identify.

“Everyone who uses the app is helping to keep the virus under control,” Chancellor Angela Merkel of Germany said last year in a video promoting the country’s alert system, called Corona-Warn-App.

But the apps never received the large-scale efficacy testing typically done before governments introduce public health interventions like vaccines. And the software’s privacy features — which prevent government agencies from identifying app users — have made it difficult for researchers to determine whether the notifications helped hinder virus transmission, said Michael T. Osterholm, the director of the Center for Infectious Disease Research and Policy at the University of Minnesota.

“The apps played virtually no role at all in our being able to investigate outbreaks that occurred here,” Osterholm said.

Some limitations emerged even before the apps were released. For one thing, some researchers note, exposure notification software inherently excludes certain vulnerable populations, such as elderly people who cannot afford smartphones. For another thing, they say, the apps may send out false alarms because the system is not set up to incorporate mitigation factors like whether users are vaccinated, wearing masks or sitting outside.

Proximity detection in virus alert apps can also be inconsistent. Last year, a study on Google’s system for Android phones conducted on a light-rail tram in Dublin reported that the metal walls, flooring and ceilings distorted Bluetooth signal strength to such a degree that the chance of accurate proximity detection would be “similar to that of triggering notifications by randomly selecting” passengers.

Such glitches have irked early adopters like Kimbley Craig, the mayor of Salinas, California. Last December, when virus rates there were spiking, she said, she downloaded the state’s exposure notification app on her Android phone and soon after tested positive for COVID-19. But after she entered the verification code, she said, the system failed to send an alert to her partner, whom she lives with and who had also downloaded the app.

“If it doesn’t pick up a person in the same household, I don’t know what to tell you,” Craig said.

In a statement, Steph Hannon, Google’s senior director of product management for exposure notifications, said that there were “known challenges with using Bluetooth technology to approximate the precise distance between devices” and that the company was continuously working to improve accuracy.

The companies’ policies have also influenced usage trends. In certain U.S. states, for instance, iPhone users can activate the exposure notifications with one click — by simply turning on a feature on their settings — but Android users must download a separate app.

As a result, about 9.6 million iPhone users in California had turned on the notifications as of May 10, the state said, far outstripping the 900,000 app downloads on Android phones.

Google said it had built its system for states to work on the widest range of devices and be deployed as quickly as possible.

Some public health experts acknowledged that the exposure alert system was an experiment in which they, and the tech giants, were learning and incorporating improvements as they went along.

One issue they discovered early on: To hinder false alarms, states verify positive test results before a person can send out exposure notifications. But local labs can sometimes take days to send test results to health agencies, limiting the ability of app users to quickly alert others.

In Alabama, for instance, the state’s GuideSafe virus alert app has been downloaded about 250,000 times, according to Sensor Tower. But state health officials said they had been able to confirm the positive test results of only 1,300 app users. That is a much lower number than health officials would have expected, they said, given that more than 10% of Alabamians have tested positive for the coronavirus.

“The app would be a lot more efficient if those processes were less manual and more automated,” said Dr. Scott Harris, who oversees the Alabama Department of Public Health.

Colorado, which automatically issues the verification codes to people who test positive, has reported higher usage rates. And in California, UC San Diego Health has set up a dedicated help line that app users can call if they did not receive their verification codes.

Longhurst, the medical center’s chief information officer, said the California app had proved useful as part of a larger statewide public health push that also involved mask-wearing and virus testing.

“It’s not a panacea,” he said. But “it can be an effective part of a pandemic response.”