What it's like to play Super Mario in AR, and why it matters
Mario on a HoloLens is weird and tiring -- and it's also the future.
- Nearly 20 years writing about tech, and over a decade reviewing wearable tech, VR, and AR products and apps
I'm on the corner of Madison Square Park, slipping a Microsoft HoloLens headset over my face. It's my first time trying one outdoors. I'm already conspicuous with a massive visor-helmet on. Now I'm jumping over invisible things, and suddenly I'm a sidewalk obstacle.
One man watches from a bench. He smiles, claps. Pumps his fist, calls it the future.
A video of an augmented reality version of Super Mario Bros. in Central Park, complete with pop-up Goombas and pipes rising from the road, went viral earlier this summer. We tracked down Abhishek Singh, who created the video. The demo is real -- it's done on a HoloLens, captured through the headset (you can look at his other work here). I was lucky enough to try it for myself along with Singh, and it's pretty close to madness.
Viewing a wall through the HoloLens and, inset, what I was doing.
The project lacks the polish of other high-profile projects like Microsoft's Minecraft in HoloLens, but it's nevertheless an important step in augmented reality. The technology, which overlays digtal images on top of the real world, is poised for a big step forward in the coming months thanks to Apple's big push in the next version of iOS. And you've already seen bits of AR in Pokemon Go and in SnapChat filters.
Singh's project brings a familiar aspect to this new area with the use of Mario.
Singh mapped all of the first level, or "World 1-1," into 3D. You're not looking at Mario from the side like the classic Nintendo game -- you are Mario and ahead of you is the level from a first-person perspective. Yes, that means a course that stretches out over dozens of meters in a walkthrough that's really about stomping occasional critters and hopping to collect mushroom power-ups.
That was enough for me -- my jumping ability is a far cry from Mario's. The app adjusts to height and jump capabilities, thankfully.
Of course, through HoloLens, the field of view is a lot smaller than it seems in the video. But sure enough, ghostly Mario things were floating around me, relatively mapped against the park (and also, the long hallways of CNET's offices). It works better indoors, because bright sunlight dulls the projections.
Singh's Mario is a pretty linear experience, but it goes farther than I expected. The level kept going, from one end of Madison Square Park to the other, with hills and blocks around me, and pixel-clouds up above. I eventually found the castle and the flag sprouting out of a nearby parked car. Sadly, that meant I couldn't jump on it. Though I was tempted.
Trying on Hololens at the office with Singh.
The HoloLens' ability to map 3D objects onto the real world gets messy -- passing people walk through and overlap, things don't always line up. That's the current state of AR location-mapping in a nutshell.
It's not an authorized Nintendo app; in fact, Singh hasn't talked directly to Nintendo. It's a project he's worked on independently, and as a result, you can't download it on HoloLens. Instead, it's an experiment to see where AR and mixed reality games could go next.
I talked to Singh about the future of AR on phones , and he's already working on applying his ideas to Apple's ARKit on iOS 11 . It wouldn't be head-mounted, but it could be the training wheels we all need to get used to weird world-mapped games.
Pokemon Go was just the beginning. First, the future will be handheld. Then maybe later on, strapped on our faces. Heaven help us all.
iHate: CNET looks at how intolerance is taking over the internet.
It's Complicated: This is dating in the age of apps. Having fun yet? These stories get to the heart of the matter.