Samsung's Bixby assistant needs to grow up. Fast (hands-on)
Analysis: This three-in-one app for the Galaxy S8 is very much in beta mode.
Samsung's new Bixby assistant for the Galaxy S8 and S8 Plus (and beyond) isn't exactly like Siri, Google Voice Search, Amazon Alexa or Microsoft Cortana, and that may confound you. Especially when Google Assistant also lives on both S8 phones.
So what is it? I find it's easier to think of Bixby as an app in three parts -- Bixby Voice, Bixby Home, Bixby Vision -- and one that's very firmly in beta mode. In fact, Samsung says it will work on Bixby until the two S8 phones arrive in stores on April 21, and even long after. So what I saw during my hands-on time with the Galaxy S8 and S8 Plus may not be as sophisticated as the final version is at launch.
Samsung's new digital pal might not even come preloaded on the phone -- the company says it will update us on Bixby's progress -- but it could arrive as a software update, with more updates definitely rolling out over time. In other words, Bixby may have a slow start, and become more versatile as Samsung adds support for third-party partners and new Bixby skills.
Tired of typing? Bixby can use your words to controls settings on your phone.
(Bixby is all homegrown so far. Samsung says it will eventually integrate the AI it bought when it acquired Viv Labs in 2016, but we don't know when.)
But how is Bixby now?
I dig into that more below, but let me start by saying that some demos I saw worked well and others were pretty buggy. There was a lot we heard about (like linking up with Uber to hail a ride) that we didn't actually get to see -- and none of us were able to try out Bixby's voice controls for ourselves.
A little bugginess is expected when you see prefinal software like this Bixby beta, so I can cut Samsung some slack here. It takes time to bring complex, multi-part software together in a smooth, cohesive way. But it's also risky. Showing off an ambitious project before it's truly complete can either rally people behind promising software, or alienate them if they aren't feeling the spark.
Why is Bixby important. Because AI assistants are on the rise in phones and in the smart home in a big way. Amazon, Google, Apple, Huawei and HTC are all hoping to strike a chord with their own different versions. Samsung needs its big Bixby bet to keep up with the curve, or maybe even speed ahead. A three-pronged Bixby is an ambitious way to do that, but the beta software reveals some potential cracks that Samsung will need to firmly fill in.
Bixby's three faces: Voice, Vision, Home
Bixby Voice is the most like Siri and Google's voice-command assistants. But instead of looking up a sports score or how long it will take you to cycle to Antarctica, you'll only be able to control phone settings -- things like adjusting screen brightness and toggling Wi-Fi, or opening the camera to take a selfie. Samsung has said that anything you can do with touch on the phone, you can do with voice -- but we're not sure how that works (or doesn't) with an internet search query. It sounds like there's an inconsistency there. You can launch Bixby with a wake word or press and hold the dedicated button to use it like a walkie-talkie.
Bixby Home is the second part. Like Google Now, it'll serve you "cards" (blocks of images or text) with the weather, news headlines, your step count, a YouTube video you're interested in. This will link with third-party apps like Facebook and Twitter. You can customize this area, and Bixby is supposed to also be predictive -- if you call the same person at the same time often enough, Bixby Home will nudge you in that direction. You get to it by swiping to the left of the home screen.
Finally, Bixby Vision uses the camera to recognize objects and words. Hold up a candy bar, business card or shampoo bottle, and much like Amazon's Flow feature, Bixby Vision will flutter to life. It can identify text to extract and share and objects to research or purchase. It reads QR codes and translates text using Google Translate. A partnership with Vivino pulls up tasting notes when you scan wine. It'll recognize landmarks (even from a picture, Samsung says). Sounds familiar? Bixby Vision is a lot like Google Goggles, as well as apps we've seen on other phones. (Anyone else remember Bing Vision for Lumia phones?)
The problem with all of these hands-on demos is that the prefinal software often completed only part of a task, or failed to identify something correctly. For example, text extraction (also known as optical character recognition, or OCR) is hard to get right. But it's not very satisfying to capture half a business card. Or translate a handful of words from another language. Or identify a landmark, the Golden Gate Bridge, as the wrong one (we used a color photo in our indoor demo).
Samsung Galaxy S8 launch
Also, with Bixby and Google competing on the phone for your attention, you may not always remember which one to trigger when you want to do something specific: Is it Bixby through the side key, Google Assistant through the home button, Bixby Vision through the camera, or Bixby Home with a screen swipe? It's too soon to say if this will feel natural or confusing.
High stakes game
It's absolutely possible that Samsung will ease Bixby's growing pains by the time the S8 and S8 Plus arrive on April 21. At that time, I'll test Bixby thoroughly, not just to see how far it's come in three weeks, but also to see how well it does what it promises on a finalized device.
But Samsung has the burden of proof right now. Fail and Bixby will be a laughing stock. Succeed and Samsung can wiggle a little more from Google's Android grasp. Moreover, Samsung can use Bixby as a platform for an ecosystem that exists well beyond phones, tablets and laptops. Bixby on Samsung refrigerators, washing machines, TVs and perhaps a standalone speaker like the Amazon Echo or Google Home could own a smart home and all the phones in it.
Either way, Samsung is asking for a leap of faith. And for a company clawing its way back from the Note 7 meltdown, a leadership scandal in the company's highest echelons and a bad track record with making its own software, this doesn't feel like the right time to ask buyers for favors.
It feels like a time to be confident -- and complete. After April 21, we'll have a much better sense of Bixby's ability to be both.