Tackling the complex challenges of today's competitive environment requires savvy, speed, decisiveness, and all the qualities of a quick witted professional. Unfortunately, there are often instances when even our best is not enough. The history of machines has been marked by these times when a solution to a challenge required a new device or invention. The first computers opened a world of possibilities. Massive number crunching used to be the job of teams of people, often women, who labored to produce table after table of calculations. Now we have a tiny box (or not so tiny) that can do the work that once took days in mere milliseconds. But for all this power there are still challenges that require more than the classical analytical approach.
We have a debate raging here at FTL about whether "learning" is the right word for what the machine is actually doing. My colleague pragmatically prefers the term "self-calibrating" over "machine-learning," partly from an outright refusal to attribute living qualities to an artifact. I try to remain impartial to the philosophical implications and focus on more practical applications. Like it or not, we have a new tool in the world of programming. We have found ways to let the system program itself.
There are many methods in the machine learning toolbox, each with their own strengths and weaknesses. Artificial Neural Networks are often sold as a miracle thinking machine, when the mathematics behind them reveals them to be essentially glorified function approximators. Perhaps that is all learning really is: analyze the function to the point where you can approximate it yourself: "Watch me! Now you try…"
My favorite machine learning method happens to be the Support Vector Machine (SVM). Perhaps because at it's heart it is basically attempting to drive a wedge between a group of data points. Even if the wedge is a hyper-plane and the data exists in thousands (or millions) of dimensions it is still just a classifier, albeit a very good one. I feel like I get it...
Decision trees, Naïve Bayes, Markov Decision Process, the list goes on, but how to know which is best for the task at hand? The answer is you have to learn for yourself. Despite our sophisticated tools and algorithms we still have to rely on our own minds to make the meta-level decisions based on the available data. After all, a human was responsible for inventing the tool in the first place, often in response to a specific challenge presented by the data. A quick image search of "machine learning roadmap" produced a whole array of complex flow-charts for deciding the best way to tackle the problem. In the end the best way (perhaps the only way) is to experiment for yourself. In science we don't assume, we test.
For all the scaremongering about artificial intelligence taking over, it seems to me that we will never reach a point where a machine doesn't need a human to watch over it and make the "big decisions." Solving complex problems requires a good team and we all rely on others to survive. We need machines to do the nasty work of sifting through piles of numbers, but without a person to do the serendipitous thinking that machine wouldn't exist. It will always come down to skill and experience.