By Jason Snell
September 25, 2017 4:00 AM PT
ARKit goes to the ballpark with MLB At Bat
Note: This story has not been updated for several years.
Apple showed Major League Baseball Advanced Media’s take on augmented reality on stage at its media event earlier this month, but AR is always better in person, when you can see the reality that’s been augmented. That’s why last week MLBAM and Apple invited me (and a bunch of other tech and sports journalists) to the ballpark to check out the future of augmented reality at sporting events.
The technology is still in the early days—MLB says that it won’t reach the hands of consumers until a new version of the MLB At Bat app reaches devices for the beginning of the 2018 season. But it’s definitely on the way, powered by iOS 11’s ARKit technologies and MLB’s rich trove of Statcast data.
If you don’t know about Statcast, here’s the deal: Every major-league ballpark is equipped with imaging equipment that allows MLB to measure, at a rate of 60 frames per second, the position of every player on the field, as well as the location of the ball. It’s a technological revolution that is allowing teams and researchers alike to understand aspects of baseball that were previously thought to be unmeasurable, because they go beyond traditional stats that simply measure the outcomes of individual plays.
That data is available in real time—and it’s being tapped by the MLB At Bat app to power its augmented-reality view. Sitting at AT&T Park in San Francisco, we were able to look at an iPad pointed at the field and see floating icons with pictures of each player on the field—and the icons that moved as the players moved. Tapping on the shortstop’s icon added a colored shape indicating his fielding range, the area where he’d be expected to stop a ball and make an out. When a runner took a lead, the app could display the length of his lead.
Early in the game, Joe Panik of the Giants lined a triple to the wall in right-center field. After the play was over, the app drew the arc of the ball’s path on the screen, as well as the trajectory of the return throw by the outfielder. All the information was overlaid on the live camera view of AT&T Park thanks to Apple’s ARKit technology.
The truth is, Statcast is a fire hose. The challenge for the MLB At Bat app developers is to figure out what sorts of data are interesting and useful to someone at the ballpark—and how to deliver it to them in a way that’s easy to use and understand. Also, an alert when a foul ball is coming your direction so you can look up from your phone might be nice.
MLB’s other challenge will be making sure that its augmented-reality views will work in every conceivable section of every ballpark. It turns out that aligning the app to the field itself isn’t that hard — baseball diamonds are blessedly symmetrical, even if stadiums aren’t. The challenge is that different seats can have very different points of view. If you’re behind home plate, you’ll get a very different view than if you’re sitting atop the Green Monster at Fenway Park. So far they’ve tested the app in Oakland, San Francisco, and Los Angeles, but it’ll need to be checked out at every location.
I have to wonder if perhaps there are some augmented-reality applications for this technology that go beyond people at the ballpark. Imagine erecting a virtual ballpark on your dining-room table and being able to play back game events. All the technology to do that exists today, too—it’s just a matter of figuring out the right way to build it. But there’s no doubt that, once again, MLB’s apps will be pushing these new technologies to the limit.
If you appreciate articles like this one, support us by becoming a Six Colors subscriber. Subscribers get access to an exclusive podcast, members-only stories, and a special community.