Unreliable Machine Learning

Machine Learning

By Vint Cerf

I’m not an expert in machine learning, so this essay is likely to expose this fact. Machine learning has been in the news increasingly in the past few years with the occasional spectacular headline, such as “AlphaGo Beats World Champion Go Player!” or “Self-Driving Cars Pass Two Million Miles on the Road!” We can’t help being impressed by the successes of neural networks and their uncanny ability to perform tasks that in some cases surpass human capacity. So how does this work and why am I worried about it?

John Giannandrea, Google’s Senior Vice President for Search, Research, and Machine Intelligence, likens neural networks to a box with about a million dials on it. As the box is exposed to inputs, it makes choices or decisions, and is given feedback as to the correctness or desirability of its output. As these lessons proceed, the box adjusts the dials until it performs to the trainer’s satisfaction. On the other hand, if someone said, “What happens if you adjust THIS dial by 2 percent?” it’s not clear that anyone (including the trainer) would have a credible answer. This is, of course, where my general lack of knowledge about these mechanisms presumably shows. My impression is that if such devices are put into operation and make a poor decision under some set of conditions, the only option is to analyze the conditions leading to the bad choice and to invent new training sequences to get the system to magically adjust its dials for that case. After which, someone then needs to test to see that other inputs still produce satisfactory results.

There’s something mildly unsettling about this — it seems to suggest that we don’t have a theory of operation that allows us to make direct adjustments in a confident way. When in doubt, Google! So I did, and discovered a very cogent paper from nearly 20 years ago on the subject of neural networks and Bayesian methods. My interpretation of this paper, as applied to my worry about theory of operation, is that the Bayesian methods can be applied to the operation of the neural network to corral some of the side effects of overfitting in consequence of limited input data. While this doesn’t offer a mechanism for direct adjustment of the neural model parameters, short of training, it does seem to improve the neural network’s performance. Presumably, some progress has been made in the past 20 years so that Bishop’s paper doesn’t represent the state of the art, although I found it oddly comforting.

If we presume that neural networks will find their way into an increasing variety of processes, we might wonder how these systems will behave when they interact with each other in the real world. Reading the headlines again, my impression is that there are a number of companies out to invent self-driving cars, Google included. It would be wrong to assume that these cars rely solely on neural methods for their operation. A great deal of hard information is available for navigation, state information (traffic lights, for example), detection of moving and fixed objects, collision avoidance, and so on. If there’s a significant neural component to the operation of these cars, we might wonder how they will interact with each other.

In the stock market, programmed trading is now a major component of all the trades done and multiple parties run these systems. I’ve been told that they sometimes behave like young kids playing soccer: they all run after the ball, leading to various kinds of instability. I wonder about streets filled with self-driving cars, each running its own brand of neural network. Each will have been trained with different scenarios and each might have its own “blind spots” with regard to particular situations. Will the training feedback be based on accidents? “Back to driver training for you, Robot 45792B!”

As with all my essays, I hope that I provoke some discussion so that I can learn, as well. Of course, I might provoke outrage among those who know a lot more than I do on this topic, but that’s the risk you take in writing for public consumption.