Why Apple is probably working on a heads-up display

Every once in a while, I like to get pie-in-the-sky here. Cast my net way out at what the future might hold, what big changes might be around the corner. Obviously, it’s hard to predict with any certainty what Apple’s moves might be next year, much less several years down the road, but sometimes if you peer very closely at what the company is doing, you start to see some interesting confluences. 

Recently, I’ve become convinced that Apple is working on some sort of wireless heads-up display—call them smart glasses, or smart goggles, if you must. It’s not a product that I think is coming anytime soon; the technology required to deliver the device that Apple would surely like to produce is still some ways off. But looking at many of the technologies that Apple has been devoting a lot of time and energy to in the past few years, it’s not hard to see a glimmer, if you will, in the company’s eye.

The independent Apple Watch

Let’s start with an analogy—remember those from your SAT days? The Apple Watch is to the iPhone as the iPhone was to the Mac.

Back when the iPhone first came out, it relied on a personal computer. If you wanted to get your media onto it, you had to sync it via iTunes. Same went for your contacts or your calendar. You even needed to set up the whole device via iTunes.

Over time, the iPhone became more and more independent, to the point where it didn’t require a computer anymore. Nowadays, plenty of people rely on an iPhone to the exclusion of other devices. 

The Apple Watch hasn’t quite hit that same point yet, but it’s clearly heading in that direction. With the recent addition of LTE support, Apple is making strides in turning the Watch into a fully functional device in its own right. It’s not there yet, but it’s also not hard to imagine a future where an Apple Watch could serve as your primary device for many everyday tasks, including communications, payments, appointments and calendaring, and so on. Even just a couple of years’ worth of Apple Watch updates have shown us how increasingly powerful the device can get—and just look at the same curve for the iPhone, where the device’s current benchmarks are beating many Macs. 

But though you may some day be wearing your powerful computing device on your wrist, there are certain limitations imposed by the device’s form factor. Chief among them, the small size of the display—which obviously provides an opportunity for just the sort of heads-up display I’m imagining. 

Attention, please

Let’s bear in mind that Apple is generally not a first mover in categories. It’s not hard to see why: take a look at the reception given to the now defunct Google Glass. This is a touchy area where the possible technology has to be balanced with not only aesthetics and design but also with privacy concerns. Apple has been steadfast in its position on privacy, and certainly has the design chops to produce something that looks better—or at least less conspicuous—than Google Glass.

But let’s look at the technology for a second. Because design, of course, isn’t just how something looks, but how it works. And while simply mirroring elements of an Apple Watch to a heads-up display could provide a very basic sort of interface, such a device would obviously have to be more than that. After all, you don’t want to have to bring up your wrist every time you want to do a simple action like dismiss a notification from your display.

Apple’s already shown some hints of how it might handle this with the new camera technology in the iPhone X. For example, the iPhone X doesn’t unlock—and thus display you the text of your notifications—until you give it your attention. In the same way, a hypothetical Apple display could pop up an unobtrusive icon noting you have received a message, but not display the text of said message until or unless you focused your attention on it.

Reality, augmented 

While an attention-based interface could provide one way of interacting with such a device, however, I think there’s another clue to be garnered from a technology that Apple has been pushing of late: augmented reality.

At present, perhaps the biggest impediment to AR across the board is having to hold your device. It’s cumbersome and limiting to have to peer at a display on a small device in front of you in order to experience the expansiveness of augmented reality. A larger mixed reality display that’s always in front of your eyes but leaves your hands free, à la Microsoft’s Holo Lens, seems like the clear direction of evolution.

Moreover, having the ability to overlay virtual elements on top of the real world could provide an ideal way to extend a user interface. Rather than limiting the user to only real world controls, virtual controls could be used for interacting with applications or perhaps even entering text on a virtual keyboard. (Seems annoying, sure, but who really wanted to type on glass when the iPhone was released? Now we’re all used to it.) With such a device, the whole world could become our interface—instead of looking into a phone or a screen, we look outwards to the environment around us.

Looking at the state of the art, we’re a long way from such a device coming to fruition. And, like any tech project, it might never happen. But the seeds have been planted, and technology is always moving forward, so it’s best to keep one’s eyes open.