BIG DATA – Beyond the HypeIf one were to run a Map Reduce job on a Hadoop Distributed File System (HDFS), consisting of all white-papers, articles and presentations created in the past couple of years on Business Intelligence &Analytics, to count the most frequently occurring 2-grams (2 words that occur together), BIG DATAwould most certainly trump others. And so, instead of attempting to define BIG DATA in this blog post, Iwould like to focus on the business value of BIG DATA.For many years, BI practitioners have dealt with structured data – managed and harnessed it forinsights. There have been remarkable improvements in business decision making and Analytics, as thedomain has taken a great leap forward. Nevertheless, the focus has always been on structured datawhich is typically 20-25% of data generated by any organization. The rest 75-80% is composed ofunstructured data (Text documents, Files etc.) and hitherto there has been no system / technique /platform to derive insights from this dataset.With the advent of BIG DATA techniques (in which Hadoop and Map Reduce play a big part), businessesfor the first time can confidently say that they can build the capability to manage large volumes of data(terabytes to exabytes to petabytes), different varieties of data (structured, semi-structured andunstructured), handle ever increasing data velocity and perform complex analysis that have highvariability.The diagram below illustrates the evolving architectural paradigm of combining structured andunstructured data analysis. The top half shows the unstructured data architecture while the bottomlayer shows the BI architecture corresponding to structured data analysis. But the real value is incombining the insights from the top & bottom layers for a variety of use cases that truly enableorganizations to compete on Analytics.
There are many interesting aspects in the diagram shown above and we at Hexaware have startedworking on proof of concepts for our customers, which combine the structured and unstructured worldof data. Each of the components mentioned above will be explained in subsequent blog posts.Thanks for reading. Please do provide your feedback.