In 2014, Mark Zuckerberg bought Oculus VR for a couple billion dollars with the premise virtual reality was to become the foundation of personal computing.
In 2026, virtual reality is really starting to dig its roots into those foundations with OpenXR and Flatpaks. Operating systems based around VR headsets with eye tracking as a key feature are now receiving updates from Google, Valve and Apple.
New walled gardens are building up fast even as old ones fall down. Valve is coming for gaming, Google is relying on Android APKs, and Apple is building out a new kind of live sports and TV experience, all of it with VR as the display for the entire landscape.
Earlier this week many hundreds of people lost their jobs as Meta announced the most dramatic course-correction to its strategy yet. Even though VR’s future has never been brighter, the weight of Meta’s shift might lead some to believe “Oculus VR” here was a “grand misadventure” and virtual reality is dead, again.
That couldn’t be further from reality. If you care about the future you should have been reading UploadVR yesterday.
As I look across the last 10 years and try to piece together a picture of how Meta ended up here, I find one key technology conspicuously absent from almost all their headset and glasses designs, save for the failed Quest Pro.
Here’s a look at why the absence of eye tracking limits VR’s scale and Zuckerberg’s ambition for a new social network clouded the Oculus vision.
Eye Tracking In 2017
In 2017 I attended a pair of eye tracking demos at GDC, one of them inside Valve’s booth. From these demos I started to realize “just how empowering eye tracking will be for VR software designers.”
“The additional information [eye tracking] provides will allow creators to make games that are fundamentally different from the current generation,” I wrote. “It was like I had been suddenly handed a superpower and I naturally started using it as such — because it was fun. It is up to designers to figure out how much skill will be involved in achieving a particular task when the game knows exactly what you’re interested in at any given moment.”
Architecting an entire VR platform over a decade without a solid plan for default implementation of eye tracking is a study in long-term vision meeting short-term execution.
“Apple’s eye tracking is really nice,” Zuckerberg noted on Instagram in 2024 after saying he tried Apple’s headset. “We actually had those sensors back in Quest Pro. We took them out for Quest 3, and we’re gonna bring them back in the future.”
You could see this tiny comment as one of the first public acknowledgments in which he is starting to realize something is deeply wrong with his current strategy.
When Facebook stopped selling Oculus Go it acknowledged the company wouldn’t ever make another 3dof headset. The same thing should have happened after the Quest Pro launched in 2022 with eye tracking. It didn’t. By the time Meta will ship a second VR headset with eye tracking, roughly five years will have passed from the first. The company is probably all in on full-body codec avatars being their prize for drawing you into their vision of the metaverse now, after Apple stole their initial thunder with FaceTime and Personas powered by good eye tracking.
I believe we now have evidence that VR headsets that can’t see what you want by following the intent of your eyes aren’t serious contenders as platform plays. Valve, Google, and Apple are all centered on the technology in their latest headsets for slightly different reasons. When you pull back far, you can see that Steam Frame’s DK1 was the HTC Vive in 2016 and Valve Index was its DK2 from 2019.
Valve decided SteamOS in VR is ready for prime time in 2026 with Steam Frame’s consumer release, following Apple deciding 2024 was the time to launch Vision Pro. Both use eye tracking to do key things for users.
From Rift To Quest
For Zuckerberg’s organization, the ramping investments over the last decade would build the necessary technologies for a complete computing platform, starting with just a few billion to acquire the development team behind the Oculus Rift. Michael Abrash left Valve to found a modern Xerox PARC within Zuckerberg’s larger organization, drawn by the commitment to invest in costly long-term research and development.
Meta built those technologies in a fairly public way by showing work as it went, both in sharing research and selling products. Solid ideas like Oculus Medium during this early period were spun out and continued at places like Adobe.
Starting in 2020, Facebook tried forcing the linking of its accounts to the use of Quest headsets and, in early 2021, it tried advertising in virtual reality. VR users quickly rejected both efforts.
Facebook’s executives embarked on a rebranding effort to Meta alongside a new accounts system developed as a fresh start for Zuckerberg’s new computing platform in headsets and glasses. By the end of 2021, Facebook was Meta.
Quest 2 was selling well. There was a well-curated store, their hand tracking was quickly approaching state of the art, and there was no credible competition in the United States shipping a standalone VR headset. Any stink associated with Facebook was being put behind Meta with Zuckerberg’s bold new vision of the “Metaverse.”
And a high-end Quest Pro with eye tracking was still coming in late 2022.
A Grand Misadventure?
“Setting out to build the metaverse is not actually the best way to wind up with the metaverse,” warned technical guide John Carmack in 2021. “The metaverse is a honeypot trap for architecture astronauts… Mark Zuckerberg has decided now is the time to build the metaverse….my worry is we could spend years and thousands of people possibly and wind up with things that didn’t contribute all that much to the ways that people are actually using the devices and hardware today…we need to concentrate on actual products rather than technology, architecture, or initiatives.”
In 2022 Carmack left Meta as he “wearied of the fight” and, four years later, thousands have departed as leadership reshaped the company in the form of VR and AR technologies. Until the layoffs in 2026, Meta’s leadership and design failures didn’t reek of the failure Carmack specifically warned about. Now they do.
After laying off the vast majority of the game developers Meta hired and tasking the rest to “Horizon” initiatives, do we see Beat Saber and Population: One become a last ditch effort to keep Horizon Worlds alive? Meta’s latest move in December, as some of the first Steam Frame kits arrived with devs, was to delist Population: One from the Steam store, noting that it was a move to stop “unfair play” by cheaters using the openness of a PC to break the multiplayer experience.
The grand misadventure here was the entire Horizon Worlds effort, attempting to force a social network by brute force onto the wrong technology at the wrong time in the wrong way. 2026 represents a reset of Meta’s efforts, certainly, but the question is exactly how far back in this timeline Meta needs to go to figure out what went wrong, and which structural changes need to take place to fix it?
Gaming Studios Instead Of Eye Tracking
Meta acquired Beat Saber in November of 2019 and, over the next several years, doubled down multiple times by hiring dozens of developers skilled in the use of Oculus Touch controllers. Some of these decisions were set against unusual behavior patterns due to a generationally significant pandemic keeping people home near their headsets.
During the 2016-2018 period, NextVR streamed NBA games live to VR headsets, a startup called Spaces opened a walk around Terminator VR attraction, and the first decent eye tracking was demonstrated in consumer VR hardware. Apple released a headset that combined all those technologies mentioned from 2017 in a 2024 product.
At Meta, someone made decisions to ship headsets without eye tracking after shipping a single headset that tracks eyes. They have their reasons, but whatever they are may be the cause of Meta losing some of the lead in VR that was bought with Oculus in 2014. Whatever is going on with Meta’s decision-making process, leadership tried to rectify it by the end of 2025 with the hiring of a key executive from Apple.
Now Meta faces a world where it might increase production for its non-VR glasses products. Meanwhile, Apple, Google, Samsung, and Valve ship or plan to ship VR headsets with eye tracking.
From Real To Virtual And Virtual To Real
Imagine two types of eyewear at opposite ends of this particular continuum from Paul Milgram’s seminal 1994 paper. One at the right is a relatively heavy VR headset that is essentially all display. The other at left is a pair of ultra lightweight frames with no display. Today Meta ships Quest 3, 3S, and Ray-Ban glasses in each of these categories, and they all lack eye tracking.
Apple ships only the Vision Pro with eye tracking today and it is a $3,500 device not many people have tried. The headset does a little magic trick with this chart. It is rooted at the right edge of the chart, but software defaults to starting you at the left side. Turn the dial on the headset and the world can shift from your environment being fully “real” to fully “virtual” across the whole continuum.
Apple is surely readying something to secure the left side of the chart. When they launch, what features will they focus on and how might Apple and Meta eyewear differ?
Pointing Cameras In The Wrong Directions
If Vision Pro is a spatial computer I want Apple’s answer to the Meta Ray-Ban glasses to function more like a spatial mouse. No display and all input.
Apple could take the sensors for tracking hand movements and eye movements from Vision Pro and put that technology into slim frames with Bluetooth and battery. Thin clear glasses can gather the same eye and finger input as a big enclosed VR headset. It’s difficult, surely, but it’s more useful than putting in a display system for one eye. The differentiating feature would be a universal remote for everything that’s so impossibly advanced it could feel like magic almost everywhere.
- You should be able to operate an iPad or Apple TV, and maybe other Apple devices, the same way you do the menu of a Vision Pro. Just pinch and drag in the open air. Ex: While washing the dishes with your hands and watching a movie on your iPad, you look over and pause a movie without touching the tablet with your dirty hands.
- You should be able to run your finger along any flat surface as a virtual trackpad for any computer you’re looking at.
- You should be able to touch type on any surface.
Google told me touch typing on any surface would be a solved problem in a couple of years at the end of 2024. In such a focused design, Apple could conceivably replace the mouse, trackpad and keyboard with eyewear at the opposite end of the spectrum from Vision Pro. I mean that literally because display-free means you only ever see the real environment through a pair of frames, and yet the glasses still track your eye movements the same as they do in Vision Pro.
In Apple Vision Pro, eye tracking is used to target what you’re looking at so that when you “mouse click” by pinching your fingers together the whole system responds to exactly what you want in that moment. It’s also used to drive the included Persona avatars and even the outward-facing display system showing recreated eyeballs to external viewers. That’s a lot of technology, weight and expense Apple introduced in Vision Pro to fully enclose a person in a focused virtual location represented as an Apple home environment, and then anywhere else along the mixed reality spectrum using a dial and software.
None of that seems like a mass market need unless you had an experience in 2017 that instantly made you feel like a superhero in a VR headset. Why do VR headsets need eye tracking? For the same reason a computer needs a mouse. It is how you tell the computer what you want in a graphical user interface, even if you still need something else to select what you’re looking at.
The camera Meta placed in multiple generations of glasses faces outward for photo capture, rather than inward at the eyes for intent detection. Meta moved fast, but it made the wrong decisions in the wrong order.
When I leave the house, I could grab a pair of glasses to listen to music, take calls through my phone, and control all my other computers more easily on the go. And when I get home, or I need to do real work, I enter my Apple Vision Pro (or future Meta headset or Steam Frame).
Apple doesn’t need to make camera glasses. It doesn’t need a social network. And it could kill the mouse and keyboard while fulfilling the exact words Michael Abrash at Meta said to me in 2022.
Meta is likely aiming to get back on track with an ultralight headset that does what Apple Vision Pro does with avatars, eye tracking, focus and movies. As Meta looks to the future and figures out which brands to leave in the past I have just one suggestion.
There is one brand in Meta’s arsenal with a very close association to both eyes and headsets that gamers love.
Just call the next headset Oculus.
It’s cleaner.
Source link
#Mark #Zuckerberg #Fumbled #Oculus
