Meta’s Smart Wristband Can Control Devices Like Tom Cruise in ‘Minority Report’

In an iconic scene in the cyberpunk classic Minority Report, the protagonist dons specialized gloves and uses a variety of hand gestures to display and manipulate different tabs on a wall-sized screen—without ever physically touching it.

Now the film’s sci-fi technology is coming to the real world. This week, Meta revealed a wristband that decodes finger movements using electrical signals in the wrist. The movements are familiar to anyone with a smartphone: Pinching, swiping, tapping, and even writing.

An onboard computer translates these signals into commands on a laptop screen. Without training or calibration, users tackled a range of tests, like moving a cursor to a target, playing a Pacman-like game, and writing words and phrases—“hello world”—by drawing their index fingers across a tabletop.

Meta has long teased a muscle-reading wristband, with an early version that could translate computer clicks. The new device has broader capability. Powered by neural networks and trained on data from over 6,000 volunteers, the wristband achieved up to 90 percent accuracy in some tests. On average, participants could write roughly 21 words per minute, and they improved as they became more familiar with the device.

“To our knowledge, this is the highest level of cross-participant performance achieved by a neuromotor interface,” wrote the team in a paper describing the work.

The prototype wristband is “off-the-shelf” and comes in multiple sizes, making it a more consumer-viable product. The team hopes to integrate it into Meta’s AR and VR devices. The device could also be an affordable way to reconnect people with hand paralysis, spinal cord injury, or other motor challenges to the digital world.

Evolution of Controllers

As computers have advanced, so have the ways we connect with them.

Users controlled early computers with mechanical knobs. Then came the keyboard, first invented in the late 1800s, and still a staple today. More recently, touchscreens have forever changed computers—to the point younger generations instinctively swipe on paper magazines.

These days, we don’t even need to use our hands.

Advances in AI and voice recognition make it possible to talk to your phone instead of typing. But Meta thinks there’s still room for improvement. Voice commands can be drowned out in loud environments, and they may be impractical (or annoying) in public.

Instead of touch or voice, Mea is tapping into our body’s electrical signals. Every time we swipe, scroll, or pinch, our nerves send electrical signals to wrist and finger muscles and command them to move in highly accurate and specific ways. It’s possible to decode the brain’s instructions for movement by listening in on these signals.

Surface electromyography (sEMG) uses electrodes on the skin to capture and amplify the electrical chatter. The technology is already used in prosthetic limbs and stroke rehabilitation. It’s less invasive and more flexible than implanted devices, but also less precise. Most sEMG setups need to be carefully fine-tuned for each wearer and recalibrated if transferred to another person, making the technology hard to scale up for a general consumer crowd.

Despite this, Meta saw the technology’s potential.

The team sought to design a wearable with an intuitive, accessible, easy-to-use interface that didn’t intrude in everyday life. The device also needed to be useful for multiple types of usage—switching apps, rearranging tabs, or editing documents—and comfortable enough to wear all day.

They settled on a wristband. People already wear watches and bracelets, so a wrist device might be easier to adopt and more socially acceptable. And crucially, signals captured from the wrist could be used to decode finger motions, enabling a wide variety of gesture controls.

Power in Numbers

The device includes several loosely linked electrode blocks and a processor that looks like a small iPod. The gaps provide flexibility to orient the electrodes toward wrist muscles—rather than sitting above bones—and make the device easier to slip on and off. The processor churns through data in real time and sends decoded commands to a computer via Bluetooth.

To make sure anyone can use the wristband, the team trained its onboard neural network on data collected from thousands of people doing multiple tasks—sliding a cursor to a target, performing a variety of finger gestures, and writing on a hard surface.

The team then invited new volunteers to test the device on the same three tasks. Everyone improved with experience, especially when given coaching from a supervisor—for example, “swipe faster” or “write more continuously.” By the end, participants were able to track objects in roughly twice the amount of time as using a MacBook trackpad and write roughly 21 words per minute—slower than the average 36 words per minute on a smartphone keypad.

The speeds don’t sound impressive, but the participants had far less time using the wristband compared to the two other highly familiar “daily drivers.” And more experiments found personalization boosted performance.

“While generic models allow a neuromotor interface to be used with little to no setup, performance can be improved for a particular individual by personalizing the generic model to data from that participant,” wrote the authors.

Adding just 20 minutes of personalized data to the generic model boosted performance by 16 percent on average. It would take a hefty 14,000 minutes of additional generic data to yield a similar bump. Tailoring the model was especially helpful for volunteers with relatively poor performance. Although no longer off-the-shelf per se, future generations of the device could potentially incorporate personalized data and “learn” a person’s motor intricacies over time.

The sEMG approach opens other interaction possibilities, like detecting a gesture’s force and linking it to specific functions. Decoding up-and-down movements, rather than only in a horizontal ones, could further broaden the device’s utility. Adding buzzes and other haptic feedback could make the wristband feel like an extension of the user’s own body—increasing the sense of immersion when controlling smartphones, laptop, or AR/VR glasses.

“Over time, sEMG could revolutionize how we interact with our devices, help people with motor disabilities gain new levels of independence while improving their quality of life, and unlock new possibilities for HCI [human-computer interfaces] that we haven’t even dreamt of yet,” wrote Meta in a blog.