Siri: What Apple needs to do to improve its voice-activated digital assistant

The HomePod is coming Friday, and with it, even more attention is being paid to Siri, Apple’s voice assistant that serves as its primary interface. The early HomePod reviews are in, and most of them suggest the device is an excellent speaker that’s hampered by Siri’s limitations.

Apple

I haven’t used a HomePod yet, so I can’t speak to that, but as someone with a constellation of Apple devices, it does seem to me that Siri could stand to use some improvement. (Couldn’t we all?) So let’s leave the details of the HomePod aside for the moment and think bigger. Where does Siri need to go from here?

Fix the annoyances

When I asked a bunch of dedicated Apple users what they’d like to see changed in Siri, I (unsurprisingly) stepped into a long list of grievances. Nobody is more aware of the faults of a piece of software than its most dedicated users. And yes, one of the ways Siri could evolve is by sanding off some of its roughest edges.

I was surprised that several people told me that they’re annoyed by Siri’s cutesy responses to their commands. Apple apparently thinks it’s delightful when Siri makes little jokes as it answers your questions. If this all comes down to personal preference, perhaps the answer is an option that a user can set to force Siri to get down to business. It’s an option that I’ve taken advantage of in the excellent weather app Carrot Weather, which by default displays weather information with a hefty dose of attitude. Some people love it, but I hate it—and fortunately, I can just turn it off. Some people don’t want personality in their personal assistants.

As has been exposed in initial reviews of the HomePod, Siri doesn’t support multiple named timers, which is a problem—especially if you’re cooking. The single most used feature of our Amazon Echo is timers, believe it or not. It’s incredibly useful to be able to call out with your voice as you’re moving pots around on the stove and chopping vegetables, and Siri would benefit from this feature across all its platforms. (Let’s also toss in the ability to chain commands, something that Alexa also fails at. I should be able to say, “set a potato timer for 20 minutes and a bread timer for 40 minutes” and have it do the right thing, rather than making those into separate commands.)

Michael Brown

Depending on the names of the people in your household, the activation phrase “Hey Siri” might also be accidentally triggered with a frustrating frequency. How about an alternative activation phrase or two? (Amazon, for instance, lets you activate the Echo by saying “Computer,” “Amazon,” or “Echo” instead of “Alexa.”)

Another frustration is Siri’s inability to be used without an internet connection. Back before Siri existed, the iPhone offered extremely limited voice control, letting you use your voice to control music playback. If I’m off the internet, shouldn’t I still be able to instruct my phone to play a specific playlist? Or open a app? It doesn’t seem quite right.

Context is for kings

Apple has reduced the chances of a person who isn’t you activating Siri on your devices by adding voice training. What it hasn’t been able to do yet is allow multiple people to train Siri—and allow Siri to recognize who is speaking and tailor its responses accordingly.

With a collection of personal devices—iPhone, iPad, Apple Watch, Mac—this is less of a big deal. With the HomePod, which is not a personal device (unless you live alone?), it’s a killer. At the very least, Siri should be able to detect when the owner of the linked account is speaking and provide more data to them than to anyone else. But ideally, every speaker should be able to tie in with their own account and related device, so if my wife asks Siri to add an item to her calendar or to-do list, it does so—rather than adding that item to mine.

In general, Siri (and, to be fair, most of its competition) is really bad at understanding context. This is a hard problem to solve, but it’s something that needs to be solved. Siri should know who it’s talking to, not just in order to associate an Apple ID with that person, but to remember the context of previous conversations.

Some of this involves basic follow-up information: If I ask Siri for a baseball score and I’ve got the MLB app installed, maybe it should ask me if I want to listen to the game? Or if it doesn’t offer, maybe if I say “can you play the audio of that game,” Siri remembers what game we just talked about and kicks off the MLB stream?

And half an hour later, if I say “what’s the score of that game,” Siri should know what I mean, what the score was the last time I asked, and give me an update. If I send my wife a message by voice while I’m driving, I’d like to be able to ask Siri ten minutes later if she’s seen it. Or even, at the time, ask Siri to send the message and alert me when she’s seen it.

Extend access to apps and web services

The most powerful feature of Amazon Echo is not in Amazon’s platform itself, but in the fact that Amazon has made a large number of partnerships with third-party apps and services, as well as throwing the gates of Alexa wide open via a third-party skills market.

Apple has tentatively introduced extremely light support for Siri from third-party apps via SiriKit. Given the power of iOS apps and the popularity of the app store, SiriKit could use a major expansion. Obvious features like media playback—you still can’t control apps like Overcast or Pandora via Siri—are still missing.

Apple

But Siri’s palette can be dramatically expanded by opening up to web services, not just apps resident on an iOS device. In true Apple fashion, such a move wouldn’t come via a wide-open skills market like Amazon’s. Instead, I assume Siri web integrations would need to be approved by Apple before being placed in the Siri equivalent of the App Store.

Think about how powerful your iOS devices are because of the third-party apps on them. Now imagine those devices limited to Apple apps and a very small selection of ride-sharing, list-making, and workout-tracking apps. That’s where Siri is right now. My Amazon Echo has access to an amazing selection of services, some blessed by Amazon, others not. Are many of them garbage? Absolutely. Apple doesn’t roll that way. But as with the App Store, there’s a way to open up Siri to industrious third-party developers and services without sacrificing control over the platform.

Finally, consider how much more powerful Siri could be on our devices if the users could use it to perform tasks that we care about. Apple bought the iOS automation app Workflow a while back; wouldn’t it be great if those of us who were so inclined could use a Workflow-like system to train Siri to perform specific workflows for us?

The introduction of Siri kicked off the modern era of voice assistants, but these are still the early days. It doesn’t really matter who’s ahead or behind right now, because we will look back on today’s crop of voice assistants in a few years and laugh at how incapable they were. The challenge for Apple is to continue moving forward with Siri, making it more powerful and flexible. That way users of Apple products can have confidence in talking to their devices, knowing they’ll receive help in return—not frustration.