Companies are seeing returns from big data Uses of Big Data 90% 80% 70% 60% 50% 40% 30% 20% Uses of Big Data 10% 0% Improved Improved Support of Not Business Current New Leveraged Decisions Revenue Revenue for Revenue Streams Streams GrowthSource: Avanade Inc. 2012 Big Data Survey
The Heart of a Data DrivenOrganization Data drives decisions and are the key to all decisions made within the organization People who think make decisions, not data! A data driven organization can not truly use data on its own, it takes people with the right skills and expertise in knowing how to use the data, to truly be data driven. Evidence based decisions + Reasoned Arguments is how an organization becomes data driven.“An organization’s data is found in its computer systems, but a company’sintelligence is found its biological and social systems” --- Valdis Krebs,researcher
Obtaining Data as a competitive Advantage Best in class data driven companies take 12 days on average to integrate new data sources into their analytical systems; industry avg companies take 60 days, laggards 143 days. Best-in-class companies can pursue new market opportunities faster Can take advantages quickly, newly emerging business opportunities Can bring high-value services and products to market faster Be proactive and create more information based insightsSource: Aberdeen Group: Data Management for BI: Fueling the analytical engine with high-octane information
To Put it Another Way Computational = Subconscious Strategic = Conscious
How to use Big Data to create adata driven culture Data • Data is the foundation • Insights improve Insights understanding • Actions, create Actions new experiences
Solving Problems with Big Data Hadoop-able Problems Complex data and lots of it Multiple data sources and highly unstructured Benefits of Analyzing with Hadoop Low cost Greater flexibility Ability to do previously impractical analysis
Where to Start with Big DataProblem Solving Text Mining (unstructured Modeling true risk (new data that was previously not available) data means better Pattern Recognition (find forecasts) previously unknown Recommendation patterns in the data) engines (engage Collaborative filtering customers) (power of the crowd) POS analysis (real-time Sentiment analysis (Beyond text mining) analysis) Prediction models (new Data “sandbox” (new data means new insights methods for testing new about what may come) products concepts)
Data Driven Decision Making Framework – Insights to ActionSource: Social Business By Design Dion Hinchcliffe
Signal Types Signals have attributes depending on their representation in time or frequency domain can also be categorized into multiple classes All signal types have certain qualities that describe how quickly signals can be generated (frequency), how often the signals vary (rate of change), whether they are forward looking (quality), and how responsive they are to stimulus (sensitivity)Rate of Change Quality Sensitivity Frequency (Slow or Fast) (Predictive or (Sensitive or Insensitive) (High or Low) Descriptive) Sentiment Behavior Event/Alert Expressed as Correlation These signals A discrete positive, Clusters Measures the identify signal neutral, or Signals based correlation of persistent generated when negative, the on an entity’s entities against trends or certain prevailing cohort their prescribed patterns in threshold attitude characteristics attributes over behavior over conditions are towards and time time met entity
Finding Signals in Unstructured DataHigh quality signals are necessary to distill the relationship among all the of theEntities across all records (including their time dimension) involving those Entities toturn Big Data into Small Data and capture underlying patterns to create useful inputsto be processed by a machine learning algorithm. For each dimension, develop meta- data, ontology, statistical measures, Clickstreams and models Social Timing/ Context Recency Articles Content Source Measure the Create Measure symbol Derive the freshness of Blogs sentiment sources’ the data and language to strength: describe and meaning of the insight Tweets from originality, environment importance, s in which tracking tools to quality, the data quantity, resides syntactic and semantics influence analysis
New Solutions Must Aid HumanInsight Big Data + Amplified Human Intelligence = Better Decisions Last Decade Next 5 Years - Structured Data - Any data, from - Conclusive Dashboards anywhere - Small scale / sampling - Intuitive exploration A data architect built a - Making sense of it view to reach a specific at scale conclusion Business users easily find, explore, visualize and navigate insights
Know Your EcosystemBusiness leaders must know the tools of the trade in order to know what is truly possible.
Data Driven Organizations Always Question the Data • How do we integrate the right data• What business opportunity/problem are together? we trying to solve? • How do we manage the quality of• What questions do we need to answer to the data? solve the problem? • What data does this relate to• What data do we need to answer the (master data)? questions? • Do we have all the data about this• What data do we have? (person, event, thing, etc.)?• How can data help differentiate us in the • What are the permissible purposes market? of the data? (compliance, regulatory environment)• What data is IP for us? Revenue generating for us? • Who is allowed to access the data? Use this data?
Data Driven Spider Graph Data Science Customer Big Data IT Care Data Driven Business Logistic Customer Strategists Experience Business Traditional Intelligence IT Tools Social
Always Remember: Data, Insights,Actions • Listen to the data streams Listen • Share the data with the rest of the organization Share • Engage to the data to find the insightsEngage • Innovate new ideas from the insights gained from the dataInnovate • Perform insightful actions from the data to create better customerPerform experiences