Oracle open sources GraphPipe, a new standard for machine learning models
Machine learning is expected to transform industries. However, its adoption in the enterprise has been slower than some might expect because it's difficult for organizations to deploy and manage machine learning technology on their own. Part of the challenge is that machine learning models are often trained and deployed using bespoke techniques, making it difficult to deploy models across servers or within different departments.
Oracle is aiming to solve this challenge with a new open source, high-performance standard network protocol for transmitting tensor data. The new standard, called GraphPipe, should make it easier for enterprises to deploy and query machine learning models from any framework.
GraphPipe is designed to solve three particular challenges: First, there's no standard for model serving APIs, meaning business applications typically need a bespoke client to communicate with a deployed model. Next, building a model server is simply hard, and there are few out-of-the-box solutions for deployment. Lastly, the solutions that enterprises typically use now -- like a python-JSON API -- don't provide the performance needed for critical business applications.
GraphPipe includes: a set of flatbuffer definitions, guidelines for serving models consistently according to the flatbuffer definitions, examples for serving models from various machine learning frameworks, and client libraries for querying models served via GraphPipe.
With these tools, a business should be able to deploy a model across multiple servers, or create an ensemble of models from different frameworks, with a common protocol. GraphPipe should also help deploy machine learning for IoT applications that rely on running models remotely.
GraphPipe is available on Oracle's GitHub.