Oceans of big data: Take the plunge or wade in slowly?
In a recent study, Deloitte identified some of the hurdles that keep organizations from making greater use of business analytics. These include poor technology infrastructure, the quality and amount of data being collected and leadership that may not support or even understand the use of analytics.
This presentation defines big data, explains why you should care about big data, and suggests when big data should be used. The potential of big data is immense, but it can also become an expensive distraction. Once you remove constraints on the size, type, source and complexity of useful data, you can ask the ‘crunchy’ questions that are critical to the success of your business.
Oceans of big data: Take the plunge or wade in slowly?
Oceans of big data:Take the plunge or wade in slowly?Jane GriffinNational Analytics Leader
OverviewWhat is big data?Why you should care about big dataWhat big data does not doBig data challengesMake sure you need big dataIdentify your ‘crunchy’ questions
The hype around big data is enough to giveanyone a headache. Some say it’s key tosustainable competitive advantage. Some worrythere could be more risk than reward. Many havecome to believe it can be tackled just bypurchasing the right hardware and software.
What is big data?It’s a dataset so big that…• It presents data storage, real-time processing and privacy problems.• It can’t be handled by traditional data management and analysis tools in a timely way.The 3 V’s of big dataVolumeThe sheer size of data in organizationsis exploding from TB to PBVarietyThe data formats, structures and semanticsare more diverse and inconsistentVelocityThe pace at which data is being generatedtoday is significantInternal + external dataStructured + unstructured data
So what do we really mean?• Purchasedetail• Purchaserecord• Paymentrecord• Segmentation• Offer details• Customertouches• Supportcontacts• Integrated view• Blogs• Dynamic pricing• Offer history• Click stream• Contact center• Search• Behaviourtargeting• Dynamic funnelsBIG DATA SignalsB2B/B2C CustomerInteractionsWeb TrafficInternalSystems• Social networks• User-generatedcontent• Monitoringdevices• Mobile Web• Demographics• Business feeds• Images• Audio• Video• Speech-to-text• Service logs• SMS/MMS• SentimentPetabyesTerabytesGigabytesMegabytesKilobytes
Big data is the new oil. The companies,governments, and organizations that are ableto mine this resource will have an enormousadvantage over those that don’t.The Future of Big Data, Pew Internet, July 20, 2012
Why you should care about big data• The sources for big data are numerous and growing.Big data sources include all stores of transactional information:• Data streams are exploding in number, size and complexityevery year.• For telecom, media and banking, big data collection has alreadybegun. Companies in these industries had no choice but to dive in.• In other industries, the move to big data is more of a choice.A choice to explore and seek competitive advantage throughgreater insight.Financial marketand e-commerceCell phoneconversationSocial network chatRFID signals andweather satellite dataWeb search andbrowsing patternsUrban trafficcameras andsurveillance cameras
Explore the potential of big data, but go in withyour eyes wide open and remember the goal ismore insight not more data.
What big data does not do• Bypass statistical reality or make the scientificmethod obsolete.• Absolve users of the need to ask the right questions.• Eliminate the need to find the right features.• Guarantee your ability to respond in a timely mannerjust because you can produce results in real time.• Make cost-benefit or ROI analysis obsolete.
TechnologyIssue Primary challengeScalability • Flexibility of infrastructure to interact with extreme volume using a varietyof data formatsIntegration • Cost of compiling, managing, and leveraging data across multiple platformsand systemsDeployment • Choosing between custom solutions or appliances, or cloud services• Transitioning from legacy systems to newer technologyAnalytics • Algorithms that scale, yet yield explainable resultsBig data challenges
DataIssue Primary challengeData quality • Maintaining quality when much data is external or unstructuredGovernance • Re-evaluation of internal and external data policies, standards and regulatoryenvironmentPrivacy • Privacy and security issues related to input data and resultsIssue Primary challengeTalent • Acquiring the skillsets required to leverage big dataPeopleBig data challenges
Big Data is not a silver bullet. It has enormouspromise, we’ve seen companies do great thingswith it. But you don’t want to use big data wheresmall data will do.
Where big data makes senseExploit faint signalsDisparate data sources are making it hard to see trends.Nurture experimentationPlay out different scenarios and distinguish between correlation and causation.Imagery and video analyticsAudio and video are unwieldy…the perfect assignment for big-data apps.Deliver real-time impactHave a major impact by analyzing data from many sources in real-time.
Deliver more precision fasterFocus to find small scale patterns and avoid spurious correlations.Work with constrained budgetsTake advantage of existing business intelligence tools and skill sets.Manage privacy and security risksControl and data management procedures are available for small databut not for big data.Basic performance management and forecastingFinancial and accounting data has lower volumes, is mostly structuredand is easier to analyze using traditional tools.Where small data makes sense
Identify your ‘crunchy’ questionsBig Data can become an important part of your strategy. It can also become anexpensive distraction. Start by identifying your most important business questions.Discuss critical business issues, specifically:1Demandsfor profitablegrowthGrowingcustomerexpectationsIncreasedregulatorypressureNew anddifferentsignalsSearch forhidden insight
Identify your ‘crunchy’ questionsIdentify decisions that need more insight:Evaluate potential datasets:23• Be specific• Link them to a measurable value ($)• Focus on optimizing or innovating as opposed to informing• Actionable (“do it” don’t “prove it”)• Use “what,” “where” and “how” questions versus “why”• Select the right collection of data (Example: Accounts payable data)• Identify how your crunchy questions interact with your selecteddataset (Example: Where can we save money through vendorconsolidation/sourcing?)
So remember, when wading into big datawaters, big data…holds enormous promise…provides fruitful areas for innovation…but it is not a panacea for all analytical challenges
Jane GriffinNational Analytics LeaderSend Jane Griffin an emailwww.linkedin.com/in/griffinjanewww.deloitte.ca/analyticsContact