Google’s Custom Chip may be the Biggest I/O Announcement

Near the end of Google’s two hour I/O keynote, they snuck an announcement in which may be the biggest deal of the whole event. Google officially announced a project they have been working on for over a year — building their own custom ASIC designed specifically for machine learning applications. Google calls this chip a Tensor Processing Unit and it is tailored for their TensorFlow machine learning intelligence.

From a server standpoint, it appears Google is tiptoeing toward vertical data center hardware to run their software in the same way Apple designs their hardware to run their software. In this blog outlining the TPU, Google says:

“But one thing we know to be true at Google: great software shines brightest with great hardware underneath.”

A statement similar to one used frequently by Apple quoting Alan Kay who said, “People who are really serious about software should make their own hardware.” I’ve often said I believe Apple designing their own SoCs in the A-family series processors is one of the most brilliant things they have done to differentiate and give them an edge over their competition. Similarly, this move from Google is looking to add similar elements of differentiation when it comes to artificial intelligence. I view what Apple is doing in SoCs custom-tuned to their software to yield a distinct advantage and Google making custom chips to give them an edge in Artifical Intelligence in the same strategic light.

The benefit to Google in making their own custom ASIC is their software will yield significant performance benefits. Google states:

“We’ve been running TPUs inside our data centers for more than a year, and have found them to deliver an order of magnitude better-optimized performance per watt for machine learning. This is roughly equivalent to fast-forwarding technology about seven years into the future (three generations of Moore’s Law).”

If that is true, it is pretty impressive. This ASIC, essentially a co-processor specifically designed for their machine learning applications, was powering the AlphaGo according to their blog.

Many did not understand what it meant when Apple started designing their own custom CPU/GPU solutions by utilizing the customization flexibility offered by ARM and Imagination Technologies and what it would yield in terms of competitive advantage. Similarly, this custom silicon work by Google may be underestimated in its long term potential.

AI in some shape and form is clearly an important part of our future. Google is looking to lead here and I expect even more attempts at verticalization through the hardware and software stack in order to lead in Artificial Intelligence. I remain convinced that, in many areas of personal computing, custom silicon will play a much more important role in helping competitors differentiate going forward than at any other time in this industry’s history. Keep an eye on the few companies with the talent to build custom chips for their own proprietary applications.