On the May 8th edition of Closing Bell on CNBC, venture capitalist Chamath Palihapitiya, founder and CEO of Social Capital, created quite a stir in enterprise artificial intelligence (AI) circles, when he took on
“Watson is a joke, just to be completely honest,” Palihapitiya said. “I think what IBM is excellent at is using their sales and marketing infrastructure to convince people who have asymmetrically less knowledge to pay for something.”
IBM quickly parried. “Watson is not a consumer gadget but the A.I. platform for real business. Watson is in clinical use in the U.S. and 5 other countries. It has been trained on 6 types of cancers with plans to add 8 more this year,” IBM told CNBC. “Beyond oncology, Watson is in use by nearly half of the top 25 life sciences companies, major manufacturers for IoT applications, retail and financial services firms, and partners like GM,
The next day, Palihapitiya walked back his comments – somewhat. “I probably should have been more careful with my words,” he told CNBC, but he added that “there are a lot of people building extremely meaningful solutions who are probably getting somewhat out-marketed” by IBM – including one or more of his firm’s own portfolio companies.
Clearly, IBM sees no humor in Watson’s progress in the marketplace, and given Palihapitiya is building competitors to Watson, he may simply be talking smack. Nevertheless, the challenges IBM is facing with its bet-the-company gamble on Watson are well-known, as I discussed in a recent article on the company.
Now that Palihapitiya has poked a stick at Big Blue, it’s time to take another look at Watson, joke or not.
The Blogosphere Responds to the ‘Joke’ Comment
A number of bloggers and other pundits joined the fray. “I like that ‘asymmetrically less knowledge.’ It suggests that the PR firms, the paid consultants who flog the word ‘cognitive,’ and the torrent of odd ball conference talks are smoke and mirrors,” blogged Enterprise search expert Stephen E Arnold, Managing Partner, ArnoldIT.com. “Reporting five years of declining revenue puts hyperbole in context. IBM is simply trying too hard to push Watson into everything from recipes to healthcare. The financial reports tell me that the bet is not working.”
Not all bloggers sided with Palihapitiya, however. André M. König, Co-Founder at Opentopic (an IBM partner), added his two cents. “Well I agree that IBM is a formidable marketing machine, only to be outmatched by their corporate boldness and technological innovation,” König wrote. “If you call IBM Watson a joke you call the hundreds of companies and startups that have built on it a joke.”
Another important angle on this controversy is from the investor perspective, as IBM’s long history as a blue chip is now under pressure – and its bet on Watson is big enough to affect its stock price long-term.
“Big Blue has been working on Watson for years now and investors have hoped, for some time now, that this unit would drive growth that would replace sales from the shrinking legacy businesses,” blogged Nicholas Ward for Seeking Alpha. “Time will tell, but recent growth doesn't paint a pretty picture.”
At least one writer for the healthcare press agreed. “Lately, much of the press for Watson has been bad,” wrote David H. Freedman, Executive Editor at Global HealthCare Insights, for Technology Review. “As IBM’s revenue has swooned and its stock price has seesawed, analysts have been questioning when Watson will actually deliver much value.”
Fallout from the M.D. Anderson Debacle
In February 2017, M.D. Anderson Cancer Center canceled a promising, but troubled contract with IBM for its Watson platform. “The breakup with M.D. Anderson seemed to show IBM choking on its own hype about Watson,” Freedman added. “The University of Texas, which runs M.D. Anderson, announced it had shuttered the project, leaving the medical center out $39 million in payments to IBM—for a project originally contracted at $2.4 million.”
It’s unclear, however, what the root of the problem was for M.D. Anderson. “Most of the criticism of Watson, even from M.D. Anderson, doesn’t seem rooted in any particular flaw in the technology. Instead, it’s a reaction to IBM’s overly optimistic claims of how far along Watson would be by now,” Freedman added. “After four years it had not produced a tool for use with patients that was ready to go beyond pilot tests.”
The medical community was similarly concerned about Watson’s shortcomings at M.D. Anderson. “A university audit of the project exposed many procurement problems, cost overruns, and delays. Although the audit took no position on Watson’s scientific basis or functional capabilities, it did describe challenges with assimilating Watson into the hospital setting,” said Charlie Schmidt, writing for the Journal of the National Cancer Institute. “Experts familiar with Watson’s applications in oncology describe problems with the system’s ability to digest written case reports, doctors’ notes, and other text-heavy information generated in medical care.”
Watson’s Fundamental Shortcomings
IBM wowed the world with Watson’s 2011 Jeopardy! win – but in the intervening years, IBM hasn’t released a new version of the platform. Instead of a dramatically improved ‘Watson 2.0,’ IBM has settled for ongoing, but limited improvements in the technology.
The rest of the AI technology community, in contrast, has been innovating rapidly, and Palihapitiya’s smack-talking aside, VC-funded startups are now running circles around Watson’s weaknesses.
And weakness there are. Watson requires many months of laborious training, as experts must feed vast quantities of well-organized data into the platform for it to be able to draw any useful conclusions. And then it can only draw conclusions based upon the body of data, or ‘corpus’ (plural: ‘corpora’) that it has been trained on.
The ‘well-organized’ requirement is especially challenging for Watson, as unprepared data sets are typically insufficient. As a result, Watson customers must hire teams of expert consultants to prepare the data sets, a time-consuming and extraordinarily expensive process.
Watson also cannot make connections across different corpora, and thus is unable to glean even basic insights outside each corpus. For example, training Watson on oncology will give it no insights into heart disease – a failing that dramatically limits its use in clinical settings.
A team of
Sohn et al. didn’t call out IBM Watson by name, but it’s clear from the context of their article who the target was. “A health care organization that relies on a single EHR [Electronic Health Record] vendor’s analytic solutions, as well as its own legacy analytics infrastructure created before the era of big data, may see limited progress,” they continued.
They then characterize such limited progress, differentiating the various uses Watson (or similar platforms) might offer. “While many machine-learning solutions are not yet mature and sophisticated enough to support complex clinical decisions, machine learning can be effectively deployed today to reduce more routine, time-consuming, and resource-intensive tasks, allowing freed-up personnel to be redeployed to support higher-end work.”
The gentlemen also identify where they feel the most useful innovations in AI for healthcare will come from – and Watson is most assuredly not on the list. “Leading machine-learning solutions, both general and health-specific, are evolving rapidly and are likely to come from both start-up and established technology companies as well as innovative health systems. Many of these solutions (
Where Watson Should Have Been by Now
As the M.D. Anderson fiasco illustrates, IBM fell into the trap of over-promising and under-delivering. “IBM claimed in 2013 that ‘a new era of computing has emerged’ and gave Forbes the impression that Watson ‘now tackles clinical trials’ and would be in use with patients in just a matter of months,” Freedman noted.
As to whether Watson will ever be useful in clinical situations? “This is hard,” opined Stephen Kraus, a partner at Bessemer Venture Partners. “It’s not happening today, and it might not be happening in five years. And it’s not going to replace doctors.”
In fact, given the exponential rate of change in technology innovation, the Jeopardy!-winning Watson of 2011 should now be available on our PCs or even our smartphones. “It was only five years between Deep Blue in 1997, which was a specialized supercomputer, and Deep Fritz in 2002, which ran on eight personal computers, and did about as well,” pointed out renowned futurist Ray Kurzweil, Owner, Kurzweil Technologies, Inc. in 2011, after the Jeopardy! game aired.
Placing Kurzweil’s comments into today’s context, we can conclude that not only do we lack Jeopardy!-playing horsepower on our phones, the current version of Watson should be orders of magnitude more advanced than it is now.
Instead, IBM is ceding whatever AI leadership it purported to have to a new crop of far more innovative startups and other AI firms willing to reinvent themselves as the inexorable pace of innovation continues unabated – and that’s no joke.
Intellyx publishes the Agile Digital Transformation Roadmap poster, advises companies on their digital transformation initiatives, and helps vendors communicate their agility stories. As of the time of writing, none of the organizations mentioned in this article are Intellyx customers. Image credit: Atomic Taco.
