TensorFlow is commodity software

As you already know, Alphabet’s Google open sourced TensorFlow, a machine learning library. A lot of news articles were written about this. Some of them got it all wrong, depicting it as a marvelous black box that will miraculously provide your organization with brilliant Artificial Intelligence capabilities, out of thin air.

Others were straight to the point: Google open sourced TensorFlow because it is not what gives them competitive advantage and also because they will benefit from this. TensorFlow is commodity software. I will step back to explain what TensorFlow is and why I see it as the commodity part of Google’s operation.

TensorFlow is part framework and part a distributed computing engine. The framework provides an API to express machine learning algorithms using directed graphs as their representation. The TensorFlow paper highlights that its development was driven by the findings they had with DistBelief (their previous proprietary machine learning library) and that Artificial Neural Networks were one of the focus of the research (Google Research, 2015, pages 1-2).

Artificial Neural Networks (ANNs) are, as the name implies, based on biological neural networks¹. ANNs are formed by layers of interconnected nodes. The connections between these nodes are activated when an input value is strong enough to activate a node. This interaction repeats until a node in the output layer is activated, which is the result that the network thinks it is right for the given input.

For example, a neural network to recognize handwritten digits scanned in 28x28 pixels images is shown in the following image:

A three-layer neural network that recognizes handwritten digits (Nielsen, Michael A., 2015)

The input layer have a node for each pixel of the 28x28 pixel image. Each node is given a value ranging from 0.0 (white) to 1.0 (black) and the range represents the shades of gray. When a node in the input layer is given a value, it tries to activate the hidden layer and so on until a node in the output layer is activated. The activated node in the output layer indicates the digit that the neural network thinks it is.

This capability to infer a digit is possible by training the neural network. By feeding the neural network with a training data set, it is possible to query it later with an image that it was not present in the data set. The neural network then infers the result based on past experience.

Besides the existence of different neural networks models, training them is also hard. The model type and training type and rate matters. You can end with a neural network that overgeneralize the inferences or that don’t learn anything. Building them requires competencies not just in the algorithms as well in math and statistics.

Also important is to know what you want to achieve by building a machine learning model. What kind of capability or enhancement you want to provide for your product or service. Leveraging machine learning to enhance a product or service ressembles domain modeling. It is context-sensitive, specific to a business. It requires you to formulate your hypotheses, collect data, formulate the models and learn from the results². Tweak, rinse and repeat. And for sure, you need data.

And that’s why Google open sourced it. They turned TensorFlow a commodity because it has a supporting role in its activities. As pointed by, Rob Landley, commoditization happens when markets mature:

As any market matures, it commoditizes. Once-proprietary advances become generally understood, are improved upon, and become obsolete. The rapid pace of information technology has accelerated this cycle as well.

This happened before in the IT industry: from the commoditization of computer hardware (IBM PC), operating systems (Unix, Windows, Linux), SQL and NoSQL databases (MySQL/PostgreSQL, MongoDB/Redis), messaging brokers (RabbitMQ, ActiveMQ) to Big Data distributed storage and computing systems (Hadoop, Spark).

In the previous commoditization cycle, the Big Data cycle, Hadoop emerged as its flagship software. Inspired by Google’s File System and MapReduce papers, Hadoop reigned for some years and was pointed by an ex-Google engineer as more sophisticated than the company’s proprietary software³.

Maybe Google did not took advantages from the sustaining innovations introduced in Hadoop but their proprietary software enabled them to build a gigantic infrastructure based on commodity hardware and to scale its systems in an unprecedented way.

And here is where the commoditization of TensorFlow becomes more evident: Google already owns a monopoly on data and has a massive infrastructure. They crunch massive amounts of data in their machine learning models in a scale that no other company does. This poses a great barrier for competitors. As Steve Blank (2003) points out “attacking a competitor with a monopoly in an existing market requires three times the competitor’s spending”. To create services to compete against Google’s core services is then a huge endeavor that only some companies can tackle.

In this scenario, open sourcing TensorFlow⁴ is hugely beneficial for Google. It will bring a stream of sustaining innovations through contributions from different parties, leveraging the Artificial Intelligence community knowledge⁵. For the community and the market, it will raise interest in the subject, not to forget that it “borne from real-world experience in conducting research and deploying more than one hundred machine learning projects throughout a wide range of Google products and services” (Google Research, 2015, page 14). Indeed, it was a huge development effort and it is a great contribution to the ecosystem.

To wrap up, I quote Rob Landley again:

Commoditization does not mean that innovation ceases in that niche. On the contrary, innovation is greatly helped by the modular nature of a commodity marketplace: it’s easier to upgrade to a better mousetrap when your mousetrap didn’t come bundled with the house. Commoditization merely means that consumers have a general idea what they want and multiple suppliers know how to give it to them. A mature market has a usable baseline and a frame of reference. If it was static and boring, the commodities market wouldn’t be so active.

References

• Nielsen, Michael A. 2015. Neural Networks and Deep Learning. The book is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported
• Google Research, 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (preliminary whitepaper)
• Blank, Steve. 2003. The Four Steps to the Epiphany (2nd edition)