When people think Big Data, they think Big Questions and Big Answers. But not every piece of data has to big, it’s more about questions and answers that range from small to medium to HUGE in: business, entertainment, politics, government, law, health, travel, genetics, biology, defense, history, friendship, social communications, public safety…the list goes on!
These answers come from the analytics and visualization side of Big Data. The data repositories created by Hadoop, Spark, Cloudera,Hortonworks, Hive, Splunk, Hunk, MapReduce and dozens of other infrastructure software products are important, however, once you get access to all the relevant internal data that in the cloud, that’s when the real solution formation process begins.
Did you know the most highly funded Big Data start up is not a software company – but is an analytics company named Palantir which has over $550 million in funding? Almost two times as much as the closest software company in the Big Data space.
There is a trend forming of companies jumping head first into the deep end of “Big Data” Analytics using tools from R, SAS, SPSS and MatLab to tackle the hard “use cases” (aka: important questions) organization have such as: how do we sell more to our clients and what do they want? In an effort to answer, hypotheses are made and tested. The answers are presented in dashboards to decision makers proving that if they make pro-active offers to Y customers about Z they will buy more and everyone is happy.
But there is another side to the hypothesis and testing formula, a new frontier if you will. To stay ahead of competition, companies have to ask themselves, what about the questions and hypothesis we don’t even suspect? New tools are on the rise that have the capability to visualize data in unique ways to find relationships that were previously unknown. On top of that, they can be seen in 3-D graphs to map out patterns. The other side of visualizations lies in the effective communication of complex answers that humans can quickly understand, because we understand visual information so quickly compared to rows of data.
The answers from Big Data, especially when it applies to public safety, medicine, engineering, or the all-important marketing reports to executives, require visuals that are understandable in real-time in order for people to make real decisions.
When the answers from Big Data all come together in an easily understandable visual format, everyone wins. This is the way Big Data is going. Companies are building, or have built, data warehouses, allowing the answers to flow for the big and small questions. But the key take-way is, all of that data doesn’t mean a thing if it can’t be communicated effectively to the people who need the answers – whether it’s a student, a doctor or a CEO.