Is Big Data all that?

Tuesday 18 June 2013

All of the hype and constant references to big data is great as there is something new to talk about but is it all that?

Sure there is more data. Social media such as Facebook and Twitter create loads of data but so what? I’ve looked at Twitter and tried to make sense of Tweets and work out sentiment and turn 140 characters of Twitter speak into useful, insightful information. Facebook… you’re having a laugh (a sponsored laugh these days).

We all get sucked in by the hype and the rhetoric.

Today is more about “now”. Getting the information you need about your business, your customers, and your staff when you want it and without having to go to someone to get it.

I’m in the big data arena.

We have software that is fundamentally geared around scale and performance, but it’s not big data we are promoting its self-service and ease of use.
In today’s society, size is irrelevant…its relevance that’s relevant.

Hadoop is a fine technology which has received a load of press and air time over the past couple of years. It’s great for storing vast amounts of data on banks of commodity hardware.

Accessing Hadoop requires code. Map Reduce and new products that are being or have been written to make data from Hadoop more accessible is driving a thriving community of developers but why?

The old fashioned data warehouse, staging areas, cubes and stuff is no longer in vogue. Everything these days is “search” for something and you’ll find it in seconds. Hadoop is an evolution of the database.

… Search, however is “throw away the old” and do things in a completely different way.

Google has been revolutionary. Hadoop is a warehouse without an interface. It’s Google without the ability to Google. Google is a gargantuan warehouse with the easiest to use interface you could possibly come up with… genius.

Big Data is a general realisation that there is so much more data than what you collect in your back office systems that might be useful, but even with the likes of Hadoop to persist that information in an accessible repository, what is the true value of this data and how do we get that value?

I find all of the current hype around big data and the promotion of technologies that require dozens of servers, armies of techies and even then professors of programming to get at the data a bit of a scam.

At the end of the day it’s about making money, customer satisfaction and self-improvement so it’s not the size that matters it’s being able to zero in on the facts… finding the information that matters.

I am concentrating on finding out the facts and masking out the noise. With all this “big data” we are creating a maelstrom where it’s increasingly difficult to see the facts through the mist.

Big data is for the likes of Coca-cola for McDonald’s for Walmart.

For everyone else big data is noise.

Concentrate on getting the most out of what we have before seeking solace from social chatter.

Get in touch to discuss your requirements further