Like many, I followed the coverage of CES from the Las Vegas Convention Center as the world's tech giants showed off the principles and ideas that will be driving the industry forwards over the next year and beyond. The rise of various realities - be they mixed, virtual, actual, augmented or otherwise - stood out, although many of the solutions on show were leaning towards demonstrations of techniques and hardware.
To get out of the hype phase, the world needs more solutions that are available and making a practical difference today. For the 'fully enclosed' experience where your eyes and ears perceive only the virtual world, equipment like the HTC Vive and the PlayStation VR headset have taken the first steps in bringing modern gaming into the virtual world. While gaming is the obvious application for new realities, there are other approaches.
Last month I was invited to put the Microsoft HoloLens through its paces and experience Redmond's view of this new wave of computing. I spent time in Microsoft's HoloLens suite in London using the mixed reality glasses, spoke to Microsoft team members, and met partners who are already seeing the financial benefits of using HoloLens in the real world.
Leila Martine, Microsoft's Director of New Device Experiences, introduced me not just to the HoloLens, but to the thinking behind the almost cyberpunk smart glasses.
"This lives between two worlds, the physical world and the virtual world. The human world is physical, and the computer world is the virtual. How do you get back information from the virtual world into the real world? HoloLens."
Running Windows 10 Holographic edition, the essence of HoloLens is that it overlays information so it is 'in the room' with the wearer. As a standalone computer it doesn't need to be connected to a desktop or laptop computer, it is genuinely standalone. The built-in sensors map the environment it is in so there is no need for any external positioning devices. It also maps the user, so it can see where you are looking and register hand gestures that act as the control inputs.
Sound also plays an important part, with the 3D speaker system not covering the ears - this allows sounds from the real world to be heard with no interference, while the HoloLens uses binaural audio to create spatial effects and mix in the sound from the digital reality, telling the brain which physical location the virtual sound is allegedly coming from.
To bring this all together, Microsoft designed and fabricated the Holographic Processing Unit. The HPU sits alongside the CPU and the GPU in the unit, and s key to the smooth experience of HoloLens. There's a lot going on here, but does it actually work?
Next: I try it on and someone hands me a screwdriver...
Now it's time to try HoloLens myself. The Developer Edition of HoloLens started shipping last year in the United States for a price of $3,000. Q4 2016 saw an expansion of the program into the United Kingdom, along with Ireland, France, Germany, Australia and New Zealand).
The unit itself is not subtle at all. It is bulky and substantial, you are going to feel the weight of it on your head. What is nice is that it doesn't feel constrictive - the headband has a ratchet mechanism to tighten around your head and there's very little weight or pressure on the bridge of your nose. The technology is distributed around the band, and it's comfortable to wear for long periods.
It helps that the unit is standalone You don't need to be wired into another device, there's no laptop lurking in your backpack to drive this thing, it is a genuine portable computer. It might not have the traditional keyboard and screen, but there is a screen (in front of your eyes), a keyboard (that hovers in the virtual world ahead of you) and you can use your fingers with various pointing and pinching actions much like a mouse.
After booting up the device I ran the first demo application. This was to be a virtual cadaver that would show off various systems of the human body. As I started to look at a skeleton, I walked around, moved towards it, and my brain simply accepted that the skeleton had a physical presence in the room. As a teaching aid this is impressive, especially when you can walk into the skeleton and look around at the bones normally hidden away. Neither was it restricted to a skeleton; the lymph system, blood vessels, muscles and tendons were all programmed in and could be stripped away or put on show as required.
The demonstration also revealed a little bit of a gotcha with HoloLens' screen. Even though the smokey glass wrapped around my face, the field of vision is a much smaller rectangle in front of me. The size is comparable to looking at a large laptop screen from a comfortable distance. There are moments where the edges of the display will cut off what you are looking at, which can be a distraction if you are in 'look around in awe' mode but when I started doing some task with the HoloLens my perception 'zoomed in' to this smaller area and it all felt genuine.
HoloLens works when your brain simply gives up thinking 'computer image' and thinks 'that thing is in the room'. That happens really quickly, and even with some fast head turns, tilts, and shaking movements, the HoloLens system never glitches or lost its keen sense of positioning. The built-in sensors just worked, and it wasn't long before I got my first dizzying 'woah' moment.
Typically, it was in a star field. Tucked into the end of another demo, the HoloLens user is asked to walk around and look at the galaxy. You can insert your own favourite science-fiction reference in here, but the kicker was this.
"Look down", I was told.
Around knee height was our solar system, with the orbit of each planet drawn out. Being able to duck down, look between the planets, see the eccentric orbit of Pluto, the comparable distances familiar from textbooks... HoloLens had won me over.
Now it was time to do some work.
Next: Someone hands me a screwdriver...
If there's one thing my family know to never do, it's to let me near tools or any sort of DIY. It's normally a recipe for confusion, stress and constant explanations of what to do and not do. So Microsoft's team handed me a screwdriver, pointed to a broken light switch and fuse box... and left the room.
Oh and I was also left with the HoloLens, so I wasn't really alone. It was time for another sneaky demo of the capability of HoloLens. Installed on the headset was Skype. That allowed the team to call me remotely (okay, next door, but it could have been miles away) using the standard Skype client for Windows 10. Once connected for a video chat, I could see 'the support technician' (come on, play along) in a floating window that I could move around my environment, while the technician could see my view as relayed by the HoloLens cameras,a nd exactly where I was looking thanks to the eye tracking.
I can speak, what I can see is being relayed back to base, and both my hands are free to do the work. But here's the delightful thing about this reality - the technician could get me to look at the toolbox and draw an arrow on his screen to show me what tool was needed to test for a live current. He draws an arrow on his screen (naturally a touchscreen Surface machine) and that arrow appears in the HoloLens screen... and no matter where I turn, look, or walk in the room, it stays pointing at the tester tool. So too does the highlighted 'test this wire' arrow, which is added next.
Yes I managed to fix the wire... with some help. No, Microsoft didn't electrocute me. And yes, suddenly the practicality of strapping thousands of dollars to a head becomes clear.
While many people will focus on games and leisure activities, HoloLens is already proving to be a practical tool for enterprise partners. As an educational tool it can create models and examples far more detailed and useful than real life. For product designers it can show virtual objects in the real world - and one object can be in a shared space so multiple HoloLens users can see and work on the same object. It can turn one single person into the eyes and ears of an entire team - and when you consider that the International Space Station has a HoloLens on board to help Ground Control assist Astronauts through procedures and give those on the ground a feeling of physicality, the sky really is no longer the limit.
Now... does it all work outside of the demo room?
Next: Keeping people moving with HoloLens...
"We move people."
Javier Sesma Sanchez is the General Manager of thyssenkrupp's Elevator Innovation Center, and has a huge amount of experience in bringing different technologies to the problem of keeping people moving. "Seventy percent of people live in cities and we see more and more people moving each day. We are much less flexible with the downtime of an elevator."
The average elevator needs servicing between three and five times a year, and worldwide thyssenkrupp's downtime is 190 million hours. Part of Sanchez's role is to find technology that can reduce downtime. HoloLens is undergoing trials in the field to see if can contribute to this mission with 60 HoloLens units in use in New York and another 60 units in Europe. These are being used to help with training, in main branches to demonstrate their effectiveness, and in the field to reduce repair time.
Rather like my experience with the electrical rewiring, a thyssenkrupp technical who arrives on scene is not alone. Previously they could have used a hands-free mobile kit, or perhaps a tablet to look up information. With HoloLens that changes. It brings the eyes and ears of the technician to a support center ...that could be anywhere in the world. Getting in touch with an expert in a specific area of elevator repair means using HoloLens to put them with the technical at the repair, no matter where they are based.
Traditional methods of training requires technicians to come to a central location to use fixed assists. With HoloLens they can train from anywhere, and they can start out working on virtual assets.
There's also the simple fact that the HoloLens runs on Windows 10. That offers immediate integration with thyssenkrupp's current software. As well as using Skype to call back to a support center, HoloLens can keep Outlook open but used hands-free, and information assets such as circuit diagrams can be placed literally next to the wiring looms a technician is working on. "The latter was probably the most surprising to us," Sanchez highlights. " Circuit diagrams are much harder to remember than where mechanical parts go, at that moment HoloLens clicked into place."
How successful is the trial? "Our target is to drop downtime down by fifty percent where HoloLens is used. But in time saved, HoloLens testing shows a two-hour repair can be cut to twenty minutes."
Like many of the hardware solutions using augmented, virtual, mixed, or other realities, HoloLens is still at an early stage of its development. What is key is that even with the larger size and limitations that the current hardware has, it is able to deliver compelling solutions that address real-world issues. HoloLens isn't about flashy games, immersive experiences, or looking sexy, it's about blending the virtual and the physical worlds to solve practical problems.
Microsoft's experience with the Enterprise markets has resulted in the development of a wearable technology that is making a difference today. And as HoloLens makes a difference, Microsoft gathers more knowledge and information about how it is used, it gets the hardware into the hands of more developers and clients, and no doubt channels all of that into making the next-generation smaller, lighter and more powerful.
There was a lot of noise about mixed reality at CES, but the truth is... it's already here.
Now read how Microsoft is fighting in 2017's smartphone world...
PS: It was Colin Edwards who handed me the screwdriver... blame him.
