Big Data Will Blind You

Not all of us are scientists, but all of us today are consumers of science. And I mean science, not technology. When we want to lose weight, or make more money, or find that perfect someone, we don’t go to gurus, and we don’t go with our guts. We look at the latest studies.

Zemanta Related Posts ThumbnailIt’s been said that Generation X has a deep need for data. Certainly a lot of people my age long ago lost our last vestiges of idealism, and are most interested in knowing, as pragmatically as possible, exactly what works and what doesn’t. We no longer believe in Dr. Spock’s intuitions or Oprah’s platitudes. We want to see what science says. We’re only interested in practical, proven methods. We haven’t given up trying to explain the world, but we’ve stopped trying to make beautiful, abstract theories workable. In the same vein, companies like Amazon, Google and Facebook are proud to call themselves ‘data-driven’: they make no claim to being led by ‘visionaries’, but act based on rigorous analysis of consumer activity. (Of course, there are a minority of companies, such as Apple, which do claim to be led by visionaries, but these are the exception, and their stock prices are more volatile.)

Part of this zeitgeist is the modern tech industry excitement about the possibilities of ‘Big Data’, a rapidly-emerging state in which we’ll have so much data on so many people and so many financial transactions that we’ll cross some kind of singularity into perfect knowledge, a threshold beyond which we’ll find new markets, new products, and vast new vistas of profit.

Maybe so. But there’s a big pitfall that comes with Big Data. If you’re given a big pile of facts, you start to imagine that you know more than you did before; that you can just crunch some equations and run some statistics, and the numbers will tell you what to do. You’re tempted to believe that you don’t need to get the ‘how’ and ‘why’ of things, as long as you have enough ‘what’.

A little knowledge is a dangerous thing. But knowledge without understanding is even more dangerous. Here’s some examples of why. Continue reading “Big Data Will Blind You”