As you already know, Alphabet’s Google open sourced TensorFlow, a machine learning library. A lot of news articles were written about this. Some of them got it all wrong, depicting it as a marvelous black box that will miraculously provide your organization with brilliant Artificial Intelligence capabilities, out of thin air.
Others were straight to the point: Google open sourced TensorFlow because it is not what gives them competitive advantage and also because they will benefit from this. TensorFlow is commodity software. I will step back to explain what TensorFlow is and why I see it as the commodity part of Google’s operation.
TensorFlow is part framework and part a distributed computing engine. The framework provides an API to express machine learning algorithms using directed graphs as their representation. The TensorFlow paper highlights that its development was driven by the findings they had with DistBelief (their previous proprietary machine learning library) and that Artificial Neural Networks were one of the focus of the research (Google Research, 2015, pages 1-2).
Artificial Neural Networks (ANNs) are, as the name implies, based on biological neural networks1. ANNs are formed by layers of interconnected nodes. The connections between these nodes are activated when an input value is strong enough to activate a node. This interaction repeats until a node in the output layer is activated, which is the result that the network thinks it is right for the given input.
For example, a neural network to recognize handwritten digits scanned in 28x28 pixels images is shown in the following image:
A three-layer neural network that recognizes handwritten digits (Nielsen, Michael A., 2015)
The input layer have a node for each pixel of the 28x28 pixel image. Each node is given a value ranging from 0.0 (white) to 1.0 (black) and the range represents the shades of gray. When a node in the input layer is given a value, it tries to activate the hidden layer and so on until a node in the output layer is activated. The activated node in the output layer indicates the digit that the neural network thinks it is.
This capability to infer a digit is possible by training the neural network. By feeding the neural network with a training data set, it is possible to query it later with an image that it was not present in the data set. The neural network then infers the result based on past experience.
Besides the existence of different neural networks models, training them is also hard. The model type and training type and rate matters. You can end with a neural network that overgeneralize the inferences or that don’t learn anything. Building them requires competencies not just in the algorithms as well in math and statistics.
Also important is to know what you want to achieve by building a machine learning model. What kind of capability or enhancement you want to provide for your product or service. Leveraging machine learning to enhance a product or service ressembles domain modeling. It is context-sensitive, specific to a business. It requires you to formulate your hypotheses, collect data, formulate the models and learn from the results2. Tweak, rinse and repeat. And for sure, you need data.
And that’s why Google open sourced it. They turned TensorFlow a commodity because it has a supporting role in its activities. As pointed by, Rob Landley, commoditization happens when markets mature:
As any market matures, it commoditizes. Once-proprietary advances become generally understood, are improved upon, and become obsolete. The rapid pace of information technology has accelerated this cycle as well.
This happened before in the IT industry: from the commoditization of computer hardware (IBM PC), operating systems (Unix, Windows, Linux), SQL and NoSQL databases (MySQL/PostgreSQL, MongoDB/Redis), messaging brokers (RabbitMQ, ActiveMQ) to Big Data distributed storage and computing systems (Hadoop, Spark).
In the previous commoditization cycle, the Big Data cycle, Hadoop emerged as its flagship software. Inspired by Google’s File System and MapReduce papers, Hadoop reigned for some years and was pointed by an ex-Google engineer as more sophisticated than the company’s proprietary software3.
Maybe Google did not took advantages from the sustaining innovations introduced in Hadoop but their proprietary software enabled them to build a gigantic infrastructure based on commodity hardware and to scale its systems in an unprecedented way.
And here is where the commoditization of TensorFlow becomes more evident: Google already owns a monopoly on data and has a massive infrastructure. They crunch massive amounts of data in their machine learning models in a scale that no other company does. This poses a great barrier for competitors. As Steve Blank (2003) points out “attacking a competitor with a monopoly in an existing market requires three times the competitor’s spending”. To create services to compete against Google’s core services is then a huge endeavor that only some companies can tackle.
In this scenario, open sourcing TensorFlow4 is hugely beneficial for Google. It will bring a stream of sustaining innovations through contributions from different parties, leveraging the Artificial Intelligence community knowledge5. For the community and the market, it will raise interest in the subject, not to forget that it “borne from real-world experience in conducting research and deploying more than one hundred machine learning projects throughout a wide range of Google products and services” (Google Research, 2015, page 14). Indeed, it was a huge development effort and it is a great contribution to the ecosystem.
To wrap up, I quote Rob Landley again:
Commoditization does not mean that innovation ceases in that niche. On the contrary, innovation is greatly helped by the modular nature of a commodity marketplace: it’s easier to upgrade to a better mousetrap when your mousetrap didn’t come bundled with the house. Commoditization merely means that consumers have a general idea what they want and multiple suppliers know how to give it to them. A mature market has a usable baseline and a frame of reference. If it was static and boring, the commodities market wouldn’t be so active.
- Nielsen, Michael A. 2015. Neural Networks and Deep Learning. The book is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported
- Google Research, 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (preliminary whitepaper)
- Blank, Steve. 2003. The Four Steps to the Epiphany (2nd edition)
I’m not by any means specialist in the Artificial Intelligence subject neither in neural networks. I implemented some machine learning algorithms, specifically clustering (to detect, for example, duplicated contacts in a SaaS CRM) and classifiers (for credit analysis in a Bank’s customer portfolio). I took the care to provide references where anyone can look for authoritative experience. ↩
Yes, I almost described the Build-Measure-Learn loop from the Lean Startup model. It is worth approaching technology adoption in this way: define your experiment, set your expected outcomes, build and the check the results. Start small. Technology adoption alone is meaningless. ↩
The claims can be seen as an ex-employee whining but I like to highlight that Hadoop had massive investments for its development, initially from Yahoo! and then from the various companies that provides services around it (HortonWorks, Cloudera, AWS, IBM) and, from some big users (Facebook, Twitter, LinkedIn). So, it is feasible that the diversity in contributions coming from multiple vendors and users can result in software of superior quality. ↩
The distributed computing part of TensorFlow is not available in the open source release yet. I doubt this is because they want to secure competitive advantage. I believe it is just a matter of time for them to decouple it from Google’s specific code. ↩
Want to discuss this post? Reach me at Twitter!