Microsoft shared some additional details about the components inside its augmented reality HoloLens headset during the company’s Build conference, then provided further details in a follow-up blog post on the Microsoft Devices blog.
Like a traditional PC, HoloLens contains a CPU and GPU, said Alex Kipman, a technical fellow in Microsoft’s operating system group, at Build on Thursday. But the headset also uses a custom built “holographic processing unit” to handle data coming from the many sensors contained in the device.
The HoloLens includes microphones to capture sound and voice commands and cameras to map the space around a person, Kipman said.
“The advanced sensors in HoloLens capture information about what you’re doing and the environment you’re in,” Todd Holmdahl wrote on the Microsoft Devices blog. “This is done through the inertial measurement unit (IMU) which includes an accelerometer, gyroscope, and magnetometer. It’s this IMU, coupled with head tracking cameras, which enables HoloLens to understand where your head is and how it’s moving.”
This means the headset will be taking in large amounts of data, such as what a person is looking at and reaching for as well as the environment around them, like the location of a table, he said. HoloLens doesn’t immerse a person in a total virtual environment. Instead, when a person looks through the headset’s lens, they see their surroundings with holographic images placed in them.
Using a third processor frees up the CPU and GPU to run apps created by developers, Kipman said.
Microsoft didn’t share any details on how HoloLens functions without hooking up to a PC or smartphone.
“All of the infrastructure is in the top of the device,” Kipman said, adding that HoloLens is “a stand-alone fully untethered holographic computer.”
The APIs HoloLens uses to incorporate gesture control and gaze are found on Windows 10 so any program built on Microsoft’s universal app platform will run on the headset, he said. And the speech engine used to process voice commands in HoloLens is the same one behind Cortana, Microsoft’s voice-controlled digital assistant, he said.
The cameras in HoloLens can be used to capture still images and video. The camera can also be used to take a mixed reality pictures that combines the virtual and real environments, Kipman said. For example, a person wearing HoloLens could take a photo of someone in the real world interacting with a 3D object, he said.
Developers looking for access to HoloLens’ sensors are out of luck. They’ll only have the ability to access the sensor APIs, Kipman said. Allowing developers to modify the sensors wouldn’t create a uniform experience across all HoloLens, he said. If developers created programs to work only on HoloLens with sensors modified in a certain way, the software in the app store wouldn’t be consistent, Kipman said.
This article has been updated with details from Microsoft’s blog post.