Big Data Analytics Platform- Beyond Traditional EDW


Published on

Recorded version available at

Impetus Webinar on 'Big Data Analytics Platform: Beyond Enterprise DW'

Published in: Technology, Business

Big Data Analytics Platform- Beyond Traditional EDW

  1. 1. Big Data Analytics PlatformBeyond Traditional Enterprise Data Warehouse<br />1<br />Recorded version available at <br /><br />
  2. 2. Outline<br />What Is A Traditional Enterprise Data Warehouse?<br />What Is Required From A Big Data Warehouse?<br />Building Big Data Analytics Platform<br />How To Re-use Existing Investments?<br />Real-world Examples<br />Recorded version available at <br /><br />2<br />
  3. 3. The Answers We Seek<br />The kind of customer who will spend most with us next year ?<br />What is the most effective <br />Distribution channel?<br />In which area should we <br />open our new store next year?<br />What kind of products my customers are interested in ?<br />Customers that we are likely to lose ?<br />How much does my service impact my margin? <br />Recorded version available at <br /><br />3<br />3<br />
  4. 4. Traditional EDW<br />4<br />Recorded version available at <br /><br />
  5. 5. EDW Components<br />Extraction, Transformation and Loading - ETL<br />Data is extracted from a heterogeneous data sources<br />Transformed to match the data warehouse schema<br />Loaded into the data warehouse database <br />Analyze and Query - OLAP Tools<br />Active analysis - user queries<br />User guided data analysis<br />OLAP<br />Automated Analysis - Data Mining<br />Machine learning / NLP<br />Recommendations & forecasting<br />Recorded version available at <br /><br />5<br />
  6. 6. Enter Big Data<br />Recorded version available at <br /><br />6<br />
  7. 7. The Gap Area- Big Data v/s EDW<br />Large data volumes<br />Complex unstructured data<br />Deeper insights<br />Storing images, videos<br />The bottom-line - $/TB<br />Recorded version available at <br /><br />7<br />
  8. 8. Big Data in EDW<br />8<br />Recorded version available at <br /><br />
  9. 9. Key Characteristics - Big Data Platform<br /><ul><li>Highly scalable
  10. 10. Works on massive data sets
  11. 11. Support for multiple data sources
  12. 12. Easy deployment/ seamless integration
  13. 13. Deep analytics
  14. 14. Canned& customized reports as well as valuable BI
  15. 15. Support for real time analytics</li></ul>Recorded version available at <br /><br />9<br />
  16. 16. Building Big Data Analytics Platform<br />10<br />Recorded version available at <br /><br />
  17. 17. Building Big Data Analytics Platform<br />Recorded version available at <br /><br />
  18. 18. Building Big Data Analytics Platform<br />Recorded version available at <br /><br />
  19. 19. Building Big Data Analytics Platform<br />
  20. 20. Our Key Learnings<br /><ul><li>Open source yields better results for larger volumes of data
  21. 21. Parallel processing or faster mechanisms can be used for import/export of data
  22. 22. Real time is a myth in big data – needs careful design
  23. 23. Hadoop is the most cost effective option for big data
  24. 24. Reuse of existing EDW investments possible </li></ul>Recorded version available at <br /><br />14<br />
  25. 25. Impetus Big Data Analytics Platform- iLaDaP<br />15<br />Recorded version available at <br /><br />
  26. 26. iLadap- Technologies Used<br />Plug and Play Service Oriented Architecture<br />Workflow and ETL<br />Underlying PB Scale Store<br />BI and Analytics Query Engine<br />Real Time Analytics<br />Application Integration/ Development<br />16<br />
  27. 27. Reusing EDW Investments<br /><ul><li>Infrastructure
  28. 28. Code – logic and algorithm
  29. 29. Traditional data warehouse
  30. 30. RDBMS engine
  31. 31. Reporting tools
  32. 32. ETL tools
  33. 33. Development and testing strategy</li></ul>Recorded version available at <br /><br />17<br />
  34. 34. Case Study - 1<br />The Client<br />Leaders in internet services and media in Europe<br />Key Challenge<br />Very high volumes of data recorded each month<br />Near real time reporting engine needed <br />How much infrastructure needed?<br />Impetus Solution<br />Proposed Cloud for POC<br />Usage of Flume for collecting streaming data<br />Usage of Hbase/Hive for analysis<br />Benefits Realised<br /><ul><li>Highly scalable
  35. 35. Near real time analytics</li></li></ul><li>Web Analytics<br />19<br />
  36. 36. Case Study - 2<br />The Client<br />One of the key players in Telecom industry<br />Key Challenge<br />CDR Data Conversion<br />Customer churn analysis<br />Impetus Solution<br />Workflow based CDR data conversion<br />Canned reports for CDR data<br />Used Intellicus to generate customer churn analysis reports <br />Benefits Realised<br /><ul><li>Predefined canned reports for customer churn analysis
  37. 37. Better customer management</li></li></ul><li>Case Study - 3<br />The Client<br />Leading online product retailer<br />Key Challenge<br />Recommendation engine<br />Cross product customer analysis <br />Provide ‘Big Picture’ across business units<br />Impetus Solution<br />Proposed iLaDaP based solution<br />Apache Mahout based recommendation engine<br />Clickstream, Server log and OLTP cross analysis <br />Benefits Realised<br /><ul><li>Better product recommendations
  38. 38. True centralized business overview across product and business lines</li></li></ul><li>Summing up…<br />Big Data Analytics needs a well-thought of strategy<br />Any single vendor technology may not be sufficient to build a Big Data Analytics Platform<br />Hybrid solutions are effective due to their flexible cost model<br />Selecting the right tools is the key to build a successful Big Data Analytics Platform<br />Easy extension of the existing EDW infrastructure possible<br />Recorded version available at <br /><br />22<br />
  39. 39. Impetus Technologies <br />We offer innovative product engineering <br />and technology R&D services<br />23<br />Recorded version available at <br /><br />
  40. 40. Questions<br />Please send in your questions using the chat panel<br />24<br />Recorded version available at <br /><br />
  41. 41. Thank you<br />Mail us at<br />or visit<br />Recorded version available at <br /><br />