Forbes India 15th Anniversary Special

Disruption machine: Top-secret startup Magic Leap

Top-secret startup Magic Leap has raised a record-breaking amount of money in pursuit of a radical concept: Blending the digital and the physical. "Mixed reality" will transform the way we work, play, shop and see—and entire industries will be changed forever

Published: Dec 19, 2016 07:02:10 AM IST
Updated: Dec 15, 2016 04:15:12 PM IST

Disruption machine: Top-secret startup Magic Leap
Magic Leap founder Rony Abovitz: Making the unreal real
Image: Jamel Toppin for Forbes

The hottest ticket in tech is an invitation to a banal South Florida business park, indistinguishable on the outside from countless other office buildings that dot America’s suburban landscape. Inside, it’s a whole different story. A different reality, in fact. Humanoid robots walk down the halls, and green reptilian monsters hang out in the lounge. Cartoon fairies turn the lights on and off. War machines, 75 feet tall, patrol the parking lot.

Even the office equipment does the impossible. The high-definition television hanging on the wall seems perfectly normal. Until it vanishes. A moment later, it reappears in the middle of the room. Incredibly, it is now levitating in mid-air. Get as close as you’d like, check it out from different angles. It’s 80 inches diagonal, tuned to ESPN—and there is nothing holding it up.

The TV looks real, but it is not. All these wonders are illusions, conjured into being through the lenses of a “mixed reality” (MR)headset—the arcane invention of the startup called Magic Leap.

Like any good magician, founder and CEO Rony Abovitz, 45, keeps his cards close to his chest. Magic Leap has operated in extreme secrecy since it was founded in 2011. Only a few people got to see its technology, even fewer knew how it worked, and all of them were buried under so many non-disclosure agreements that they could barely admit the company existed.

Yet massive amounts of money were flowing down to Dania Beach, Florida, a town of 30,000 just south of Fort Lauderdale. To date, Magic Leap has raised nearly $1.4 billion in venture capital, including $794 million this past February, reportedly the largest C round in history. Seemingly every blue-chip tech investor has a chunk, including Andreessen Horo­witz, Kleiner Perkins, Google, JPMorgan, Fidelity and Alibaba, plus there’s backing from less conventional sources such as Warner Bros and Legendary Entertainment, the maker of films like Godzilla and Jurassic World. Magic Leap was valued at $4.5 billion in its latest round of financing. If Abovitz has held on to just 22 percent of the company—which he denies—he’s a billionaire.

Disruption machine: Top-secret startup Magic Leap
Evil robots invade your office in Weta Workshop’s Dr Grordbort game

That cascade of money sparked strange rumours within tech circles: Magic Leap was doing something with holograms, or with lasers, or had invented some reality-warping machine the size of a building that would never, could never, be commercialised. The lack of hard information further fuelled the whispers. Magic Leap, after all, has never released a product. It has never given a public demonstration of a product, never announced a product, never explained the proprietary “lightfield” technology (see box on page 91) that powers its product.

But now the company is coming out of the shadows. In a rare interview, Abovitz says Magic Leap has spent a billion dollars perfecting a prototype and has begun constructing manufacturing lines in Florida, ahead of a release of a consumer version of its technology. When it arrives—best guess is within the next 18 months—it could usher in a new era of computing, a next-generation interface we’ll use for decades to come. “We are building a new kind of contextual computer,” Abovitz says. “We’re doing something really, really different.”

Magic Leap’s innovation isn’t just a high-tech display—it’s a disruption machine. This technology could affect every business that uses screens or computers and many that don’t. It could kill the $120 billion market for flat-panel displays and shake the $1 trillion global consumer-electronics business to its core. The applications are profound. Throw out your PC, your laptop and your mobile phone, because the computing power you need will be in your glasses, and they can make a display appear anywhere, at any size you like.

For that matter, they can make anything appear, like directions to your next meeting, drawn in bright yellow arrows along the roads of your town. You’ll be able to see what that new couch you’re thinking of buying looks like in your living room, from every conceivable angle, under every lighting condition, without leaving your home. Even the least mechanically inclined will be able to repair their automobiles, with an interactive programme highlighting exactly which part needs to be replaced and alerting you if you’re doing it wrong. And Magic Leap is positioned to profit from every interaction: Not just from the hardware and software it will sell but also, one imagines, from the torrent of data it could collect, analyse—and resell.  

“It’s hard to think of an area that doesn’t completely change,” Abovitz says.

You have likely tried virtual reality (VR)—Sony, Google, Samsung and Facebook have all debuted VR products in the past 12 months. VR is an immersive computer-generated simulation, now used mostly for videogames, whereby VR headsets mask and replace the real world.

Disruption machine: Top-secret startup Magic Leap
R2-D2 and C-3PO appear in a short video made by ILMxLAB and Magic Leap

You may have also tried augmented reality (AR), in which digital content is overlaid on a physical environment. One of the biggest digital fads of 2016 made AR mainstream: In July, mobile app developer Niantic released Pokémon Go, a game that uses smartphone cameras to make animated monsters appear to exist in the real world—or at least on your phone’s screen.

Neither a VR game nor Pokémon Go can do what Magic Leap’s “mixed reality” does. VR takes you to another place. AR can make a Pikachu appear in your living room. Mixed reality keeps you where you are—and makes that Pikachu come to life.

How does it do it? The centrepiece of Magic Leap’s technology is a head-mounted display, but the final product should fit into a pair of spectacles. When you’re wearing the device, it doesn’t block your view of the world; the hardware projects an image directly onto your retina through an optics system built into a piece of semi-transparent glass (the product won’t fry your eyeballs; it’s replicating the way we naturally observe the world instead of forcing you to stare at a screen). The hardware also constantly gathers information, scanning the room for obstacles, listening for voices, tracking eye movements and watching hands.

As a result, mixed-reality objects are aware of their environment and have the ability to interact with the real world. On Magic Leap’s hardware, a Pokémon might escape capture by ducking behind your couch or, assuming you live in a “smart” home, turning off your lights and hiding in the dark.

In one of its demos, the Magic Leap team shows off a computer-generated “virtual interactive human”, life-size and surprisingly realistic. Abovitz and his team imagine virtual people (or animals or anything else) as digital assistants—think Siri on steroids, except with a physical presence that makes her easier to work with and harder to ignore. Ask your virtual assistant to deliver a message to a co-worker and it might walk out of your office, reappear beside your colleague’s desk via his or her own MR headset and deliver the message in person.

In a mixed reality world, computing power isn’t confined to a gadget on your desk. It’s something that you can link to any object, real or virtual, giving it awareness of its location, intelligence about its purpose and insight on how you might want to use it. “Think of it as the future state of computing,” Abovitz says, “where the world is your desktop.” First we had mainframes, then PCs, then mobile devices. If Magic Leap has its way, the next generation will be virtual.

“This is not about entertainment or just playing videogames,” says Thomas Tull, the billionaire founder of Legendary Entertainment. “This is a different way of interacting with the world, a new generation of computers. I think this will end up being a very, very important company.”

Rony Abovitz has always surrounded himself with visions of a high-tech future. Born in 1971 to Israeli immigrants living in Cleveland, he grew up fascinated with computers and science fiction. “My generation are the children of Steve Jobs and George Lucas,” he says. “We grew up on that, and our brains all got scrambled. My friends and I all wanted to be Luke Skywalker and defeat the Death Star and build C-3PO.”

When he was 11, the family moved to South Florida; he started high school at 13, a year early. After graduation, he was accepted at MIT but chose the University of Miami, staying close to home. He received a bachelor’s degree in mechanical engineering in 1994 and a master’s in biomedical engineering two years later—and then he started thinking about Star Wars again.

Disruption machine: Top-secret startup Magic Leap
Download your email from thin air in this image shot directly through Magic Leap’s prototype headset

Abovitz co-founded his first company, Z-KAT, in 1997. “I decided to build the medical droid from Star Wars because I thought I couldn’t build an X-wing fighter, because that wasn’t something I could explain to my parents after graduating,” he says. In 2004, Abovitz and several of his co-founders spun out Z-KAT’s robotics group into a new company, Mako Surgical, that made robotic arms to assist doctors in orthopaedic surgeries. Demand for the droids was high, and in 2008, the company went public, raising $51 million.

Working full-time at Mako and married with a young daughter, Abovitz found an outlet for his creative side in a project he called Hour Blue—his own fictional fantasy world, an alien planet full of fantastic creatures like talking robots and flying whales. In 2010, he started a new company, Magic Leap Studios, to develop the project as a graphic novel series and a feature film franchise.

“I was the only employee, and it was literally in my garage,” Abovitz says. “My mom made a piece of canvas with some coloured letters on it that said, ‘Magic Leap Studios’.” For help on the project, he used some of the cash he’d banked from Mako to hire Weta Workshop, the New Zealand-based special effects and creative-development shop best known for its work on the Lord of the Rings film trilogy, to develop imagery based on his story ideas and flesh out the world. Meanwhile, inspired by sci-fi novels like William Gibson’s Neuromancer and Vernor Vinge’s Rainbows End, Abovitz became frustrated that the augmented and virtual reality he’d read about in fiction wasn’t available in the real world and began thinking about how to make it so.

“It was a unique moment. Reality and science fiction began to merge,” says Richard Taylor, CEO of Weta Workshop and a member of Magic Leap’s board. “The fictitious technologies that we were creating for Hour Blue were developing in parallel to the real-world applications of augmented technology that Rony was beginning to explore.”

In 2011, Magic Leap Studios changed focus and became Magic Leap Inc, with Abovitz hiring a small team to help him develop this idea of mixed reality. Before long the company had working prototypes.

“The first time we had a single pixel in space, where we could move it around the room, we were so excited,” Abovitz says. “Other people were like, ‘What is that? That’s just a dot.’ But we knew. I knew at that point this was going to work.”

He also knew he was going to need a lot more money. Abovitz had initially funded the company with his proceeds from Mako’s IPO. After Mako was acquired in 2013 by medical device manufacturer Stryker Corp for $1.7 billion, he invested some of those proceeds as well.

Fortunately, the technology sold itself. “When we’d talk to people about what we were doing, they didn’t believe us,” Abovitz says. “Then they’d fly in and go, ‘Oh... you’ve actually made these things happen’. That was the dynamic of everyone who invested; [they went from] ‘This is impossible’ to ‘We want in’.”

In February 2014, Magic Leap announced it had raised more than $50 million in seed funding from private investors. Eight months later, the company closed a Google-led $542 million series B funding round.

“We invested in Magic Leap because we believe their lightfield technology is the next big inflection point in technology after the PC, the web and the smartphone,” says James Joaquin of Obvious Ventures, a San Francisco-based venture capital fund he co-founded with Twitter’s Evan Williams and a third partner. “It has the potential to transform multiple global economic categories, including entertainment, education and productivity.”

Magic Leap’s record-breaking $794 million series C round, announced in February 2016, was led by Alibaba and included follow-on investments from Google and Qualcomm Ventures, plus new money from Fidelity Investments, JPMorgan, Morgan Stanley and T Rowe Price. “I don’t think it would be mischaracterising it to say there was a frenzy,” Thomas Tull says. “This company has had absolutely no problem attracting blue-chip investors, and it’s extraordinarily well capitalised at this point.”

Disruption machine: Top-secret startup Magic Leap
Image: Peter Yang / August

Rony Abovitz doesn’t seem the captain-of-industry type—unless your model mogul is Willy Wonka. Abovitz lights up like Roald Dahl’s genius chocolatier as he leads a tour through Magic Leap’s new headquarters in Plantation, Florida, a campus more befitting his vision than the nondescript offices 15 minutes away in Dania Beach that he’s vacating. He points out “cool” machines, admires tools and invites guests to climb ladders to check out high-end air filters in the ductwork. People comment on what a nice guy he is as often as they mention his intellect.

He’s also liable to get lost in his work. On a recent Friday afternoon he was missing in action half an hour after a tour of the new headquarters was scheduled to start. Eventually one of his executives discovered he had been on-site the whole time, sitting in his car in the parking lot, absentmindedly engrossed in a telephone call.

Magic Leap broke ground on the new, 259,000-square-foot campus in October 2015 and expects the majority of its current 850 employees to move in before the end of this year. The rest of the workers are scattered in nine offices around the globe. Some groups are already in place at the new site, including a machine shop and several engineering teams. It’s important to Abovitz to keep critical development teams together as part of an “agile hardware” model, which has allowed the company to produce “literally many hundreds of iterations” of its headset prototype.

The company is also building manufacturing facilities on the Plantation campus. “This is the most spaceshippy part of Magic Leap,” Abovitz says as he leads a group into the production line areas: A series of long, self-contained, modular bays lined up like submarines in a port. Each line can be activated as needed, ramping up production from thousands of units a year to more than a million.

Abovitz wants Magic Leap to stay in Florida; one of the benefits of manufacturing there is that it allows the company to keep its secrets. If it were headquartered in Northern California, that would be nearly impossible, given Silicon Valley’s job-hopping culture and well-lubricated rumour mill. Of course, he would also have an easier time hiring, but Magic Leap’s technology has been a strong draw, pulling people in from the Valley and other tech hubs.  

Naturally, Abovitz isn’t the only entrepreneur who sees opportunity in this field. In just the past 12 months, $2.3 billion has been invested in virtual reality and augmented reality as venture firms follow the money. According to International Data Corp, worldwide revenues for the augmented and virtual reality market will grow from $5.2 billion this year to more than $162 billion in 2020.

With that kind of growth, all the big names are scrambling to get a piece of the action. Google already dipped its toes into AR in 2013 with its ill-fated Google Glass, a pair of spectacles that made a virtual computer screen appear to float in front of the user. The project stalled in beta amid privacy and safety concerns, but Google’s investment in Magic Leap indicates its ongoing interest. “We knew from some of our earliest discussions with Rony and the team that we wanted to help them accelerate their vision into reality,” says Don Harrison, vice president of corporate development at Google.

Apple is working on AR as well, but it’s unclear whether it is developing its own headset or adding capabilities to the iPhone. Silicon Valley startups like Meta (which has raised $73 million) and Atheer (a reported $23 million) are working on their own AR headsets and would presumably be natural acquisition candidates if they are successful.

For now, however, Magic Leap’s biggest competitor is Microsoft, which announced an augmented reality headset called HoloLens in 2014. A pre-production version, the HoloLens Development Edition, shipped to an unspecified number of Microsoft hardware and software developers in March of this year, and a consumer version could be released sometime in 2017. “Microsoft has a big advantage in business because of their relationships,” says Gartner research analyst Brian Blau. “They’re deep into business, and that’s exactly where they positioned HoloLens.”

So now that it has a production line, what’s Magic Leap’s timing for entering the market? “Soon-ish,” says Abovitz, coyly.

Still, with Microsoft expected to release the consumer version of its HoloLens augmented reality headset sometime next year, it’s clear that Magic Leap’s gear must get onto shelves in the same window or it will risk losing ground to its biggest competitor. And with pre-orders for Meta’s headset priced around $1,000, expect Magic Leap’s new goggles to be in the same ballpark.

Eventually Magic Leap sees its greatest impact in business applications, especially medical imaging and retail (imagine “trying on” garments at home, seamlessly). But as with most technologies, entertainment offerings will lead the way. Magic Leap is developing much of its content in-house and has already hired several well-known videogame designers, cartoonists, artists and writers. Neal Stephenson, author of Snow Crash, a seminal 1992 novel about virtual reality, is Magic Leap’s chief futurist, working out of the company’s Seattle office on an undisclosed game.

Other content is coming from Abovitz’s allies at Weta Workshop, which operates a 25-person lab with Magic Leap in New Zealand. Its first project, Dr Grordbort’s Invaders, is an action-oriented game set in a steampunk alternate universe: The player wields a ray gun and fights off evil robots that seem to break through the walls of his or her living room and fly around the house.

In June, Magic Leap also announced a strategic partnership with ILMxLAB, the immersive entertainment division of Lucasfilm, and opened a joint research lab on Lucasfilm’s San Francisco campus. “It’s like being in the earliest days of cinema,” says Vicki Dobbs Beck, the head of ILMxLAB.

The collaboration has already produced several mixed-reality experiences set in the Star Wars universe, including one revealed at the time of the announcement that features the droids C-3P0 and R2-D2, and another, as-yet-unrevealed action sequence set in The Empire Strikes Back’s iconic opening Battle of Hoth. And it has brought Rony Abovitz full circle. The man who became an entrepreneur because he secretly wanted to build X-wing fighters is now actually doing it.

(This story appears in the 23 December, 2016 issue of Forbes India. To visit our Archives, click here.)