Big Data Analytics: an Example

During virtually ever presentation I've made this year I've been asked about "big data"...

What is it?
Why is it?
Can IBM i do it?

My brief answer usually takes the form of both a description, followed by a question back.

The subjective and objective description of Big Data and the notion of analytics have been covered here and in many other places, so I'll spare you the repetition. Do be aware that Big Data is so hot right now, everyone is jumping on the band wagon - using the phrase as a way to associate their products and services with the latest trend. Combining "Big Data" and "Cloud" results in a hurricane of marketing hype that is vast and powerful.

In my quest to provide clarity and understanding, the question I pose back to the audience goes something like this:

"Do YOU have big data, and do YOU have a requirement to analyze it"?

Most folks stare back at me, wondering whether they do or not. To help them really understand my question, I offer a simple (and favorite) example of Big Data Analytics:

"Imagine the capture and analysis of every tweet flowing through Twitter on a daily basis".

To be sure, most folks in my audience are not taking full advantage of the data they have in house now, much less able and willing to make use of the truly massive amounts of data flowing past their internet doorstep.

A Real Live Example

Lo and behold while traveling this week, I read an article that crisply illustrates my default example of Big Data Analytics.  The article highlights a new startup, Social Market Analytics, that looks at all the tweets pulsing through the world wide network.

They analyze the messages to develop a profile based on the Twitter "chatter" about a given company traded publicly on the stock market. The resulting profile and corresponding scoring of the dialog will (hopefully) provide to investors a crystal ball of sorts. At least that's what they are telling potential users of the service.

If you want to read the article for yourself (and I recommend you take time to do so), it appears in the October 15, 2013 edition of American Way Magazine.

And if you are still wondering what it takes to handle big data, imagine trolling through 400 million tweets per day, looking for the few pearl laden oysters, in near real time!