Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus Webinar

1,754 views
1,596 views

Published on

Impetus webcast ‘Big Data Use Cases for Different Verticals and Adoption Patterns’ available at http://lf1.me/7U/

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,754
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide
  •  About Impetus TechnologiesEnterprise and Partners leverage the thought leadership of our advisors, the experience of our architects, and our ability to create applications for accelerated business growth. Impetus, Innovation Architected.
  • NEED –ComScore, Web 2.0, Advertising industryThere are two answers to this – COST side and REVENUE side
  • Folks – I tried to draw a bar chart of this Traditional vs. Hadoop – the hadoop piece doesn’t even show on the scale – we are pretty much talking – close to zero in comparison with traditional numbers.
  • Talking pointsThe Y axis is really non-linearIntuition doesn’t begin at zero on the y axisMore measurement – better management
  • “big data strategy is a journey, not a destination. It’s not a product you’re going to buy; it’s not something you’re going to stand up there and be done with.”
  • "We analyzed very early that the problem in Democratic politics was you had databases all over the place," said one of the officials. "None of them talked to each other."
  • When we look at Big Data use-cases
  • Our own customer Neustar was able to eliminate 48 Oracle licenses with next-gen technologiesSimilarly we have many other customers whose first use case in Big Data was to replace commercial RDBMS licenses with HadoopELT/ ETL replacement – One of the Tier 1 investment banks is working on Enterprise wide replacement of a major commercial ETL product with Hadoop based applicationsM & M - Splunk and similar point solutions or tailored use-cases like Splunk – we have helped many companies implement Security: Zions bank - http://www.darkreading.com/security-monitoring/167901086/security/news/232602339/a-case-study-in-security-big-data-analysis.htmlNeeded months or years of data to train ML algo’s to become effective; @ 3TB / week – that could be 100s of TBs and SIEM tools couldn’t handle it. In fact they used to take a day just to load the data. With a fast and effective infrastructure set up and running, Zions uses the data for dozens of purposes. Database logs, firewall, antivirus, IDS logs, plus industry-specific logs like wire ACS deposit applications and credit data are all pulled together into a centralized syslog server.
  • Our own customer Neustar was able to eliminate 48 Oracle licenses with next-gen technologiesSimilarly we have many other customers whose first use case in Big Data was to replace commercial RDBMS licenses with HadoopELT/ ETL replacement – One of the Tier 1 investment banks is working on Enterprise wide replacement of a major commercial ETL product with Hadoop based applicationsM & M - Splunk and similar point solutions or tailored use-cases like Splunk – we have helped many companies implement Security: Zions bank - http://www.darkreading.com/security-monitoring/167901086/security/news/232602339/a-case-study-in-security-big-data-analysis.htmlNeeded months or years of data to train ML algo’s to become effective; @ 3TB / week – that could be 100s of TBs and SIEM tools couldn’t handle it. In fact they used to take a day just to load the data. With a fast and effective infrastructure set up and running, Zions uses the data for dozens of purposes. Database logs, firewall, antivirus, IDS logs, plus industry-specific logs like wire ACS deposit applications and credit data are all pulled together into a centralized syslog server.
  • Our own customer Neustar was able to eliminate 48 Oracle licenses with next-gen technologiesSimilarly we have many other customers whose first use case in Big Data was to replace commercial RDBMS licenses with HadoopELT/ ETL replacement – One of the Tier 1 investment banks is working on Enterprise wide replacement of a major commercial ETL product with Hadoop based applicationsM & M - Splunk and similar point solutions or tailored use-cases like Splunk – we have helped many companies implement Security: Zions bank - http://www.darkreading.com/security-monitoring/167901086/security/news/232602339/a-case-study-in-security-big-data-analysis.htmlNeeded months or years of data to train ML algo’s to become effective; @ 3TB / week – that could be 100s of TBs and SIEM tools couldn’t handle it. In fact they used to take a day just to load the data. With a fast and effective infrastructure set up and running, Zions uses the data for dozens of purposes. Database logs, firewall, antivirus, IDS logs, plus industry-specific logs like wire ACS deposit applications and credit data are all pulled together into a centralized syslog server.
  • Our own customer Neustar was able to eliminate 48 Oracle licenses with next-gen technologiesSimilarly we have many other customers whose first use case in Big Data was to replace commercial RDBMS licenses with HadoopELT/ ETL replacement – One of the Tier 1 investment banks is working on Enterprise wide replacement of a major commercial ETL product with Hadoop based applicationsM & M - Splunk and similar point solutions or tailored use-cases like Splunk – we have helped many companies implement Security: Zions bank - http://www.darkreading.com/security-monitoring/167901086/security/news/232602339/a-case-study-in-security-big-data-analysis.htmlNeeded months or years of data to train ML algo’s to become effective; @ 3TB / week – that could be 100s of TBs and SIEM tools couldn’t handle it. In fact they used to take a day just to load the data. With a fast and effective infrastructure set up and running, Zions uses the data for dozens of purposes. Database logs, firewall, antivirus, IDS logs, plus industry-specific logs like wire ACS deposit applications and credit data are all pulled together into a centralized syslog server.
  • Our own customer Neustar was able to eliminate 48 Oracle licenses with next-gen technologiesSimilarly we have many other customers whose first use case in Big Data was to replace commercial RDBMS licenses with HadoopELT/ ETL replacement – One of the Tier 1 investment banks is working on Enterprise wide replacement of a major commercial ETL product with Hadoop based applicationsM & M - Splunk and similar point solutions or tailored use-cases like Splunk – we have helped many companies implement Security: Zions bank - http://www.darkreading.com/security-monitoring/167901086/security/news/232602339/a-case-study-in-security-big-data-analysis.htmlNeeded months or years of data to train ML algo’s to become effective; @ 3TB / week – that could be 100s of TBs and SIEM tools couldn’t handle it. In fact they used to take a day just to load the data. With a fast and effective infrastructure set up and running, Zions uses the data for dozens of purposes. Database logs, firewall, antivirus, IDS logs, plus industry-specific logs like wire ACS deposit applications and credit data are all pulled together into a centralized syslog server.
  • Our own customer Neustar was able to eliminate 48 Oracle licenses with next-gen technologiesSimilarly we have many other customers whose first use case in Big Data was to replace commercial RDBMS licenses with HadoopELT/ ETL replacement – One of the Tier 1 investment banks is working on Enterprise wide replacement of a major commercial ETL product with Hadoop based applicationsM & M - Splunk and similar point solutions or tailored use-cases like Splunk – we have helped many companies implement Security: Zions bank - http://www.darkreading.com/security-monitoring/167901086/security/news/232602339/a-case-study-in-security-big-data-analysis.htmlNeeded months or years of data to train ML algo’s to become effective; @ 3TB / week – that could be 100s of TBs and SIEM tools couldn’t handle it. In fact they used to take a day just to load the data. With a fast and effective infrastructure set up and running, Zions uses the data for dozens of purposes. Database logs, firewall, antivirus, IDS logs, plus industry-specific logs like wire ACS deposit applications and credit data are all pulled together into a centralized syslog server.
  • Our own customer Neustar was able to eliminate 48 Oracle licenses with next-gen technologiesSimilarly we have many other customers whose first use case in Big Data was to replace commercial RDBMS licenses with HadoopELT/ ETL replacement – One of the Tier 1 investment banks is working on Enterprise wide replacement of a major commercial ETL product with Hadoop based applicationsM & M - Splunk and similar point solutions or tailored use-cases like Splunk – we have helped many companies implement Security: Zions bank - http://www.darkreading.com/security-monitoring/167901086/security/news/232602339/a-case-study-in-security-big-data-analysis.htmlNeeded months or years of data to train ML algo’s to become effective; @ 3TB / week – that could be 100s of TBs and SIEM tools couldn’t handle it. In fact they used to take a day just to load the data. With a fast and effective infrastructure set up and running, Zions uses the data for dozens of purposes. Database logs, firewall, antivirus, IDS logs, plus industry-specific logs like wire ACS deposit applications and credit data are all pulled together into a centralized syslog server.
  • One who wont
  • One who wont
  •  About Impetus TechnologiesEnterprise and Partners leverage the thought leadership of our advisors, the experience of our architects, and our ability to create applications for accelerated business growth. Impetus, Innovation Architected.
  • Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus Webinar

    1. 1. A review of multiple industry verticals and how they are using Big DataAnand Venugopal (AV)Director – Business DevelopmentBig Data Services
    2. 2. We will talk about…. • Definition • Value • Use-Cases and Patterns • How to Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    3. 3. Definition You have a Big Data situation… When traditional information systems cannot store process or analyze the volume, variety or velocity of data in a cost- effective, timely manner Store Volume Process Velocity COST Analyze Variety TIME Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    4. 4. Where is the value? How is Big Data monetized? Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    5. 5. Where is the value of Big Data? Example: Traditional DW/BI vs. Hadoop CAPEX savings of 96% to 99.56% $20,000 - $180,000 per TB vs. $800 per TB includes 24/7 support $550 per TB per year after that. Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    6. 6. Where is the value? Quality of Management Deci sions BIG DATA • Accurate • Relevant Data • Quick Intuition Amount of data analyzed Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    7. 7. A powerful example …that we all just witnessed Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    8. 8. Big Data Analytics won the election "We are going to measure every single thing in this campaign,” Campaign Manager Jim Messina said after taking the job. Analytics team = 5X of 2008 “Chief scientist" for the Chicago headquarters (Retail Analytics guru) The Obama campaign guarded what it believed to be its biggest institutional advantage over Mitt Romneys campaign: its data. Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    9. 9. Some facts from the Obama campaign The first 18 months – they aggregated ALL data silos Modeled everything – sources, appeals, swing-voters, crowd- pullers George Clooney pulled in 40 – 49 age females: fundraiser Millions In the final weeks, simulation 66000 times every night – results used to re-allocate resources 29000 sample from Ohio alone to model demographics $ 1 Billion raised; Optimally deployed 8 out of 8 swing states swept Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    10. 10. Big Data use cases? Learning from others experiences Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    11. 11. Analysis approach of use cases Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    12. 12. Analysis approach of use cases Company (CEO) BFSI Healthcare Marketing Engineering Sales (VP) Finance (VP) HR (VP) (VP) (VP) Telecom Hospitality West region (Director) Retail Travel East region (Director) Consumer durables IT South region (Director) Automobiles Media North region (Director) Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    13. 13. What are use case patterns? Abstraction of use cases across industry verticals with a set of common factors: • Type of data analyzed (3Vs) • Nature of analysis or processing • Technology stack Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    14. 14. PollWhich patterns are you seeing? Recorded version available at http://www.impetus.com/webinar_registration?event=archived&eid=65
    15. 15. Batch mode- Multi source Data Analytics • Flume, Scribe, Sqoop, Hiho • Custom connectors • HDFS/ NewSQL/ Hadapt/ MPP DBs • Hive / PIG/ MR/ Impala/ Apache Drill Analytics Applications Multiple Data Sources- Structured ETL/ ELTL- Connector based Big Data Store Multiple Data Sources- Unstructured Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    16. 16. Real Time or Near Real Time Analytics • Storm + Esper; Hstreaming • StreamBase, StreamInsight, IBM Streams, SQL Stream • JMS/ Messaging (e.g. Kafka)/ Streaming connectors • NoSQL (fast ingest) Alerts Event Processing Streams of real time Engine Big Data Store EVENTS Rules Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    17. 17. Social Media and Natural Language Processing • NLTK, R (text) • Apache Mahout, Apache Lucene • Apache UIMA (entity extraction) • GP Text, Madlib, Clarabridge, Text-miner (SAS, IBM) NLP & ML Multiple Data ETL/ ELTL Sources- Big Data Store Unstructured Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    18. 18. Analysis approach of use cases Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    19. 19. Analysis approach of use cases Company (CEO) BFSI Healthcare Marketing Engineering Sales (VP) Finance (VP) HR (VP) (VP) (VP) Telecom Hospitality West region (Director) Retail Travel East region (Director) Consumer durables IT South region (Director) Automobiles Media North region (Director) Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    20. 20. Analysis approach: Value drivers Functional areas of Big Data impact Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    21. 21. PollWhere do you see the most value? Recorded version available at http://www.impetus.com/webinar_registration?event=archived&eid=65
    22. 22. Analysis approach of use cases Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    23. 23. Analysis approach of use cases Company (CEO) BFSI Healthcare Marketing Engineering Sales (VP) Finance (VP) HR (VP) (VP) (VP) Telecom Hospitality West region (Director) Retail Travel East region (Director) Consumer durables IT South region (Director) Automobiles Media North region (Director) Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    24. 24. Vertical: Telecommunication SPs Value Driver: Cost reduce, capex optimize Functional Area Use-case pattern Use-case Correlate various devices logs. Network optimization Batch Analytics and Reporting Keep SLA, but lower cost of n/w. Small % change = Millions of $ DPI of Mobile apps data Batch processing of multi-source data Revenue share or throttle Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    25. 25. Vertical: Telecom Media Entertainment Value Driver: Increase revenue, new products Functional Area Use-case pattern Use-case Correlate data from: Customer Behavior and New Real-time, Near real time product analytics Analytics + Batch analytics TV viewing, Internet, Mobile app, Telephone records, Social Media updates and Customer service records Organizational All DATA ASSETS Consolidation DNS data, Transformation! and Analytics – Real time, Mobile transaction, Near Real time location, Geo-IP, Telecom  Information Number-portability Service Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    26. 26. Vertical: Financial services Value Driver: Increase revenue, new products Functional Area Use-case pattern Use-case Reducing False positives based on Fraud Accuracy Real time + Batch customer’s individual transaction history Monitoring and Management (Servers, Personalization and AD Batch Analytics Channeling the right targeting merchant offers to the right customer – by individual transaction analytics and profile scoring Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    27. 27. Vertical: Financial services Value Driver: Cost reduce, reduce risk Functional Area Use-case pattern Use-case Evaluating credit-worthiness and risk- Risk Batch Processing of multi-source profile of loan-targets using new data data sets Evaluate Auto-repair vendors and cost- Claims (Insurance) Batch analytics of very large files, optimization and quality improvement of unstructured data and social media claims processing data Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    28. 28. Vertical: Manufacturing/ Hi-tech Value Driver: New product development Functional Area Use-case pattern Use-case Find new patterns from product testing Product Engineering Batch Analytics and Reporting data from factories – failure analysis ? and Process Analytics Multi-format flexible parsing, Root-cause ? Prediction of faults search 2-3 TB / day – Statistical Process and Real-time + Batch Quality control – real-time responses plus pattern detection using batch analytics Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    29. 29. Vertical: Manufacturing/ Hi-tech Value Driver: Revenue++, up-sell/ cross-sell Functional Area Use-case pattern Use-case To alert customers and for proactive quality Field deployed Real time + Batch engineering and recommendations product analytics Up-sell/ New offering (New revenue) Machine to Machine carrier Sensor Data Analytics Real-time, Near real time – Cars, Wineries, Oil fields – Apply business (Billing and Analytics) Analytics, Business rules engine + logic in real-time to alert and prevent failure Batch analytics events; Value-added services for consumers Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    30. 30. Vertical: Retail Value Driver: Revenue++, up-sell Functional Area Use-case pattern Use-case Marketing: CLTV Batch analytics of Mobile, POS, E- Multi-channel commerce Propensity to buy - analytics Consumer spend modeling Real-time analytics Access the customer before the retail Location Analytics experience – NOT – after check-out And early alerts (TELCO + RETAIL) Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    31. 31. Vertical: Retail Value Driver: Revenue++ Horizontal Area: Social Media Analytics Functional Area Use-case pattern Use-case Retail staffing operations- based on Operation and Real time Social Media Analytics reactions to promotions Customer Service Proactive customer complaints resolution Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    32. 32. PollWhich vertical do you see in 2013? Recorded version available at http://www.impetus.com/webinar_registration?event=archived&eid=65
    33. 33. All Verticals Horizontal use-cases which are identically applicable to all or most industry verticals BFSI Telecom Retail Consumer Automobile Durables Healthcare Hospitality Travel IT Media Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    34. 34. Horizontal: IT Value Driver: Cost-, Risk-, Capex- Functional Area Use-case pattern Use-case IT Infrastructure Batch Analytics and Reporting DW Replacement with Hadoop for Transformation storage, reporting and BI Batch processing of multi-source ETL or ELT offload of DW with Hadoop data IT Operations Real Time, Near Real Time Analytics Monitoring and Management (Servers, (logs, events, alarms) Networks, Applications, Storage, Virtualization) IT Systems Security Real Time Analytics Security event detection and prevention (Application, in real time Infrastructure, Database Batch Analytics Security breach pattern detection and Network/ Firewall) machine learning to create rules-engine Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    35. 35. Horizontal: Contact Center Analytics Value Driver: Cost-, Revenue+, Risk Functional Area Use-case pattern Use-case HR: Employee Batch Analytics and Reporting Find what type of employees will stay for retention (Call-center) longer Experience not as important as personality Contact-center Natural Language Processing + Plus Loss reduction: Customer Churn Analytics Analytics multi- source batch analytics (email, voice, web) Revenue and Cost: Channel and (voice transcripts= 150x of campaign management- outbound call email/chat) analysis Loss Control: Proactive fault isolation in Telco networks using key-words as leading indicators of N/W faults Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    36. 36. Horizontal: Web Analytics/Digital Mktg Value Driver: Revenue+ Functional Area Use-case pattern Use-case Sales from E- Event Pattern Analysis+ Batch Increase conversion to buy (Rapid A-B commerce Analytics testing) Increase revenue up-sell/ cross-sell recommendation engine; (market basket analysis – correlated products) CLTV and Propensity to Buy analytics Digital Marketing and Natural Language Processing + Plus Customer behavior and segmentation Advertising multi- source batch analytics (email, Analytics voice, web) Personalization and Micro- Targeting Advertising, Ad Effectiveness Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    37. 37. Horizontal: Social Media Analytics Value Driver: Revenue+, Cost, Risk Functional Area Use-case pattern Use-case Marketing – Social Key-word search and Natural Competitive Analytics – Who and What are Media Analytics Language Processing people talking about the most? Sentiment Analysis using Natural Brand Analytics (sentiment)– What are they Language processing saying – positive, negative? Impacts on Brand perception, products Graph Analytics + Social Media Influencers analytics – Who knows whom? Analytics Who are the key influencers? What is the revenue impact of friends-of-friends? Operations and Real time text analytics Retail Staffing operations; Customer service Proactive customer complaints resolution Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    38. 38. We talked about • Big Data-Definition  • Value  • Use-Cases  • How to Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    39. 39. Suggesting a pathway for Big Data adoption “Big data strategy is a journey, not a destination. It’s not a product you’re going to buy; it’s not something you’re going to stand up there and be done with.” Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    40. 40. Big Data roll-out strategy Business Goals, PoV/ RoI Strategy Locate Collate BIG DATA Budget Technology Identify best sub-set Design and POC of use-cases Implement Implement Operations Production Integrate People Optimize Add use-cases Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65
    41. 41. Value Punch $112M New revenue from improving fraud accuracy 48 RDBMS licenses out IT transformation to Big Data 11 million+ articles converted into PDF Less than $300.00 New Business Launched for Hispanic Consumer Data Analytics© 2013 Impetus Technologies
    42. 42. Advisors. Experience Architects. Expertise Applications. Excellencebigdata.impetus.com bigdata@impetus.com
    43. 43. Legal • © 2013 Impetus Technologies. All rights reserved. • You are prohibited from making a copy or modification of, or from redistributing, rebroadcasting, or re-encoding of this content without the prior written consent of Impetus Technologies. • This presentation includes images from other products and services. These images are used for illustrative purposes only. There is no explicit or implied endorsement or sponsorship of these products by Impetus. All copyrights and trademarks are property of their respective owners. Recorded version available at© 2013 Impetus Technologies http://www.impetus.com/webinar_registration?event=archived&eid=65

    ×