Big Data – Useless If You Don’t Know What You Are Doing

In a wide-ranging interview posted last week on IEEE-Spectrum, “Machine Learning Maestro” Michael Jordan discussed the current state of Big Data; its strengths, its shortcomings and the ways it can be misused. Here are a few excerpted quotes which highlight the key issues, in my mind, that people must be aware of when they use Big Data.

“When you have large amounts of data, your appetite for hypotheses tends to get even larger. And if it’s growing faster than the statistical strength of the data, then many of your inferences are likely to be false.”

“I think data analysis can deliver inferences at certain levels of quality. But we have to be clear about what levels of quality. We have to have error bars around all our predictions. That is something that’s missing in much of the current machine learning literature.”

“It’s not a year or two. It will take decades to get right. We are still learning how to do big data well.”

“Spectrum: What adverse consequences might await the big-data field if we remain on the trajectory you’re describing?

Michael Jordan: The main one will be a “big-data winter.” After a bubble, when people invested and a lot of companies overpromised without providing serious analysis, it will bust. And soon, in a two- to five-year span, people will say, “The whole big-data thing came and went. It died. It was wrong.” I am predicting that. It’s what happens in these cycles when there is too much hype, i.e., assertions not based on an understanding of what the real problems are or on an understanding that solving the problems will take decades, that we will make steady progress but that we haven’t had a major leap in technical progress.”

Okay, so here’s the deal for marketers:

  • Using Big Data successfully to identify customers and prospects is totally dependent on
    • the strength and reliability of your data; and
    • your ability to ask the right questions
  • At this point, Big Data is another thing in your marketing tool box. It’s still in its infancy.
  • Implementation still rules. Big Data is useless unless you create, execute and deliver marketing campaigns that resonate with customers and prospects, driving them to buy your products and services.