San Francisco looks different through an iPhone XS Max — it's still a city, but now there are floating balls of fabric, speech bubbles, and words popping out from the trees and buildings.
For more than two hours on Sunday, I ambled through the streets of San Francisco, taking in several art pieces scattered around the city by Apple. But the art wasn't actually physically on the ground — instead, it was digital art, mere 1s and 0s, attached to several significant locations around the city, and viewed through the camera and display of an Apple iPhone.
The walk, which launched on Saturday, is a new program at Apple stores called "AR[T]," which is a play on words on augmented reality, a technology that uses cameras and machine learning to place digital objects in the real world. Apple has developed software for iPhones called ARKit. Apple CEO Tim Cook has called the technology "big and profound."
The hope among technologists is that augmented reality can be the next big computing platform, and companies like Facebook, Google, and Microsoft are also investing heavily in augmented reality technologies.
"I think in AR's early days when it's still trying to find its footing, user traction and killer apps, these types of organized initiatives by tech giants can slowly push the ball forward," Mike Boland, chief analyst of ARtillery Intelligence said. "Apple is particular has invested a lot in AR and is banking certain parts of its future hardware lineup on AR, so these ARt walks are both a move to accelerate AR traction and to continue feeling out the demand signals and what will resonate with consumers."
"I have been working in augmented reality for the past 3 years and I am convinced it will be the medium in which we will experience most of the arts in the future," said artist Sebastian Errazuriz, who was not involved in the project with Apple, but who has made similar public art in augmented reality.
"Augmented reality will prove to be as huge an invention as electricity. We will all live augmented reality lives," he continued.
The walk
The AR[T] walk I attended started from Apple's flagship store in Union Square in San Francisco. It's available at six Apple stores in major cities, including New York, Hong Kong, and Paris. Over about two hours, we walked from the store to an alley, across Market Street to Yerba Buena Gardens, and to a historic church and back.
I was lucky to be able to make a reservation on Apple's website — most sessions through the end of the month are already full. Of the six sessions scheduled at the store for next weekend, all are booked as of Monday.
However, when I arrived, there was plenty of space. There were only five of us when we departed the store, including a store employee who came along for the walk on his free time. We were led by two Apple employees with a separate store manager checking in during the walk to troubleshoot our path.
If there are open slots in a walk, Apple stores will accommodate people who are there, an Apple representative said, so even if you can't get a reservation, it still may be worth seeing if there are open spaces.
The walk is guided by an Apple store employee and everyone gets to borrow an iPhone XS Max and pair of Beats Solo headphones for the trip. You can't use your personal iPhone. Each one of the the Apple-loaned iPhones has a non-public AR[T] app installed that enables you to access the experiences.
There's a bit of a ritual to see the art. The tour guide turned each experience on and off from an iPad Pro, and also led discussions about what we saw.
We'd walk to a location, like Maiden Lane, a cute little alley in downtown San Francisco. Then, we were told to find and point our phones at a "marker" like a sign, which allows the phone to place digital objects and creatures in the real world by giving the phone a point to orient the graphics around. Once you scan the marker — feeling a little haptic pulse when it's locked in — then you get a few minutes to walk around and experience the art.
At the end of each art piece, the guide asked us to all put our loaner phones together, then the guide would press buttons on his iPad, and all the phones would shut off, turn to black, and we'd be asked to put it in our pockets so we could walk to the next location.
The art
The draw for the ART walks is six pieces of art selected by the New Museum, a modern art museum based in New York. The art is often interactive and sometimes challenging, although at times you're reminded that augmented reality is a nascent medium that artists and other creators are still learning how to use.
Seven artists, including Nick Cave, John Giorno and Pipilotti Rist, made six pieces for the walk. The first piece we saw was by Cave, and it included creating and customizing a floating ball of digital cloth that walks with you down the alley. At the end, you point your phone towards the sky, and through its screen you can see a giant man with a bowl for his head standing on top of a building. Bowl Man sucks up all the cloth balls, and he changes color.
Another piece, by Cao Fei, places a little factory on the ground with a series of conveyor belts moving boxes into the distance. That piece was designed to be interactive — we could pinch or stretch individual boxes, or press a switch that reversed the flow of the boxes.
Other pieces were less successful. One of them, called "Now at the Dawn of My Life" by Giorno, involved the words from a poem floating on a rainbow pathway through a park, which was difficult to navigate on a crowded Sunday while looking through a smartphone. Our guide mentioned we didn't want to be the people engrossed in our phones running into other people, but that's exactly who we were as we tried to experience the art.
Apple's ART walk isn't the first time that a tech company has sponsored a piece of digital art placed into the real world through augmented reality technology. In 2017, Snap placed a giant Jeff Koons balloon animal in Central Park that could be viewed through Snapchat. That artwork was "vandalized" by Errazuriz, the graffiti artist, who placed a similar balloon animal in the same spot covered with graffiti tags, arguing at the time that corporations "should pay rent" in augmented reality and that citizens "should choose to approve what can be geo-tagged to our digital public and private space."
"As the technology becomes ubiquitous during the next couple of years, it will be important to stay alert to the permissions and responsibilities we allow for geo-located AR content in the future. So far Apple has proven trustworthy and responsible, I believe in them and their mission," Errazuriz said to CNBC on Monday.
The walks aren't the only augmented reality-related programming Apple is scheduling at its stores. There's a Cave piece that's available in all stores, and a separate class to learn how to make augmented reality experiences. The walks will run at least through the end of 2019.
