Big Data and the Efficacy of that Treasured “Gut Feeling”

If you Google “benefits, Big Data” you’re rewarded with a landslide of links pointing to various companies, consultants and vendors extolling the virtues of this new technology trend.

One of the more restrained assessments comes from McKinsey & Company research, which indicates that we are now at an inflection point where the ability to capture, store and analyze the massive amounts of data available to business, government and consumers has become an economic necessity. Referencing the McKinsey research, an article in the Ivey Business Journal states that, like other past trends in IT innovation, Big Data has the power to “transform our lives.”

But before throwing caution to the winds and proclaiming a new era of enlightened decision making, it might be wise to heed the words of Accenture Managing Director Jeanne Harris as reported in a recent Wall Street Journal blog authored by Deborah Gage.

Using the latest storage, databases and analytical tools, corporate managers can now, at least in theory, make better, faster and more informed decisions, she (Harris) says, but she doesn’t see that happening,” Gage writes. “A quarter of managers still say they make decisions not with data, but with their “gut”–and of the ones that do claim to use data, 40% used their gut to make their last big decision. Older managers grew up in a time when data, if you could find it, wasn’t reliable, she says. ‘They grew to trust their gut, because they had no other option.’ The problem now, Harris has decided, is no longer technology—it’s a shortage of people who understand enough about math and data science to evaluate their companies’ options and speak to their managers in a language that managers understand.

In the parlance of Daniel Kahneman, the Nobel Prize winning author of Thinking, Fast and Slow, this is a problem involving System 1 and System 2 thinking. System 1 is fast, intuitive, emotional – the gut feeling mentioned above. System 2 is slower, deliberate, logical. System 1, which functions very well most of the time, can lead to spectacular mistakes.

Kahneman is recommended reading for anyone involved in the decision-making process, whether Big Data is involved or not. And that, of course, means all of us.

Read the Full Story.

Resource Links: