Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building Real-time Streaming Apps in Minutes- StreamAnalytix Webinar

781 views

Published on

On- demand webcast ‘Building Real-time Streaming Apps in Minutes’ available at http://bit.ly/1PwhobK

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Building Real-time Streaming Apps in Minutes- StreamAnalytix Webinar

  1. 1. © 2015 Impetus Technologies 1 Building Real-time Streaming Applications in minutes Apache Storm made easy Anand Venugopal Head of Product for StreamAnalytix Panelists: Punit Shah (Architect), Syed Bilal (Data Scientist) Recorded version available at http://bit.ly/1PwhobK
  2. 2. © 2015 Impetus Technologies 2 BRIEF INTRO • Big Data Solutions & Services company • Unique in depth, expertise – started Big Data implementations in 2008 • Proven with customer success • IP and Products • We deliver - Business Impact from Big Data Solutions • Technology expertise • Data Science • BusinessAnalytics • Serving Fortune 1000 companies since 1996 • Large-scale and mission critical software platforms • HQ: Los Gatos,CA; 1500 people • Offshore operations in 3 cities in India Recorded version available at http://bit.ly/1PwhobK
  3. 3. © 2015 Impetus Technologies 3 AGENDA • Modern Agile Enterprise • Business Drivers • Lambda Architecture • Enterprise “Nervous System” • StreamAnalytix Demo • Building real-time data pipelines on Apache Storm – easy and fast • Blending real-time and batch analytics in Social Media Analytics • Announcements • Q&A Recorded version available at http://bit.ly/1PwhobK
  4. 4. © 2015 Impetus Technologies 4 DRIVERS FORTHE MODERN AGILE ENTERPRISE Business Operations Business Analytics Real-time Streaming Analytics Recorded version available at http://bit.ly/1PwhobK
  5. 5. © 2015 Impetus Technologies 5 BUSINESS OPERATIONS INCREASINGLY REAL-TIME IN NATURE Fleet Operations & Logistics Security Mobile Devices andApps Energy Industry IT Operations Recorded version available at http://bit.ly/1PwhobK
  6. 6. © 2015 Impetus Technologies 6 DRIVERS FOR A MODERN AGILE ENTERPRISE –THE CUSTOMER You and II need it when I need it Stop sending me blanket Ads My income went up I can buy a house now I already bought it and wont need it for a few years Dear Call center agent – I wish you knew who I am and my recent issues – I am about to pull the plug! Recorded version available at http://bit.ly/1PwhobK
  7. 7. © 2015 Impetus Technologies 7 CONTEXT UNAWARE  BAD CUSTOMER EXPERIENCE Recorded version available at http://bit.ly/1PwhobK
  8. 8. © 2015 Impetus Technologies 8 CONTEXT AWARE  POSITIVE CUSTOMER EXPERIENCE Multi-channel engagement in real-time Context Sensitive service Happy customers, Loyalty, Revenue, Profits, Growth Recorded version available at http://bit.ly/1PwhobK
  9. 9. © 2015 Impetus Technologies 9 MODERN AGILE ENTERPRISE = IMPACT FROM DATA ANALYTICS DATA ANALYTICS IMPACT Recorded version available at http://bit.ly/1PwhobK
  10. 10. © 2015 Impetus Technologies 10 MODERN AGILE ENTERPRISE – DATA MANAGEMENT DATA ANALYTICS IMPACT • Sense/ Capture/Transport • Ingest,Transformation, Pre-process • Volume,Variety,Velocity – Big Data +Traditional merge • Quality, Governance, Security, Lineage, Life- cycle Recorded version available at http://bit.ly/1PwhobK
  11. 11. © 2015 Impetus Technologies 11 MODERN AGILE ENTERPRISE – ANALYTICS MATURITY DATA ANALYTICS IMPACT • Descriptive  Predictive  Prescriptive (ML ?) • Search and Exploration, Sandbox – Data Scientists • Across silos, Includes external data, Fast blending • Speed – Latency of access and Recency of data Recorded version available at http://bit.ly/1PwhobK
  12. 12. © 2015 Impetus Technologies 12 MODERN AGILE ENTERPRISE – REAL IMPACT DATA ANALYTICS IMPACT • Converting Insights to Impact, Decision culture • Measuring, closed-loop feedback, Improving • Customers Experience a difference • Financial measures are clearly impacted Recorded version available at http://bit.ly/1PwhobK
  13. 13. © 2015 Impetus Technologies 13 MODERN AGILE ENTERPRISE – SUMMARY DATA ANALYTICS IMPACT • Sense/ Capture/Transport • Ingest,Transformation, Pre-process • Volume,Variety,Velocity – Big Data +Traditional merge • Quality, Governance, Security, Lineage, Life- cycle • Descriptive  Predictive  Prescriptive (ML ?) • Search and Exploration, Sandbox – Data Scientists • Across silos, Includes external data, Fast blending • Speed – Latency of access and Recency of data • Converting Insights to Impact, Decision culture • Measuring, closed-loop feedback, Improving • Customers Experience a difference • Financial measures are clearly impacted Recorded version available at http://bit.ly/1PwhobK
  14. 14. © 2015 Impetus Technologies 14 BATCHVS. REAL-TIME BUSINESS PROCESS SENSE Days ANALYSE Weeks ACT SENSE ANALYSE ACT Sec/ ms Batch Real time Sec/ ms Recorded version available at http://bit.ly/1PwhobK
  15. 15. © 2015 Impetus Technologies 15 BARRIERTO BEING “AGILE” –THE “BATCH GAP” The batch workflow is too slow Views are out of date Not yet absorbed.Data absorbed into BatchViews Now Time Just a few hours of data. Recorded version available at http://bit.ly/1PwhobK
  16. 16. © 2015 Impetus Technologies 16 t now Hadoop works great back here RT-Ax works here BLENDED VIEW – HISTORICAL AND NOW Blended viewBlended viewBlended View Recorded version available at http://bit.ly/1PwhobK
  17. 17. © 2015 Impetus Technologies 17 LAMBDA ARCHITECTURE : BIGAND FAST DATA COMBINED Batch Layer All data Pre-computed information Batch re-compute Speed Layer All data Pre-computed information Real time increment Batch view Serving Layer Batch view Merge Real time view Real time view All Incoming Data Query Recorded version available at http://bit.ly/1PwhobK
  18. 18. © 2015 Impetus Technologies 18 THE FULL PICTURE OF A MODERN AND AGILE ARCHITECTURE Landing and ingestion Structured Unstructured External Social Machine Geospatial Time Series Streaming Provisioning,Workflow, Monitoring and Security Enterprise Data Lake Predictive applications Exploration & discovery Enterprise applications Real-Time applications Traditional data repositories RDBMS MPP Compliance, Governance, Information Lifecycle, Data Lineage, Enterprise Meta Data Management Recorded version available at http://bit.ly/1PwhobK
  19. 19. © 2015 Impetus Technologies 19 SMART ENTERPRISE BIG DATA BUS Recorded version available at http://bit.ly/1PwhobK
  20. 20. © 2015 Impetus Technologies 20 Enterprise Class Real time Streaming Analytics Platform A Product developed and offered by Recorded version available at http://bit.ly/1PwhobK
  21. 21. © 2015 Impetus Technologies 21 AT A GLANCE StreamAnalytix is a software platform that enables enterprises to analyze and respond to events in real-time at Big Data scale. It is designed to rapidly build and deploy streaming analytics applications for any industry vertical, any data format, and any use-case Recorded version available at http://bit.ly/1PwhobK
  22. 22. © 2015 Impetus Technologies 22 STREAMANALYTIX BLOCK DIAGRAM Recorded version available at http://bit.ly/1PwhobK
  23. 23. © 2015 Impetus Technologies 23 DEMO – SOCIAL MEDIA ANALYTICS Recorded version available at http://bit.ly/1PwhobK
  24. 24. © 2015 Impetus Technologies 24 TEXT ANALYTICS ALGORITHMS TEXT CLASSIFICATION TOPIC MODELINGSENTIMENTANALYSIS Recorded version available at http://bit.ly/1PwhobK
  25. 25. © 2015 Impetus Technologies 25 SENTIMENT ANALYSIS • Lexicon based approach; Unsupervised model • Lexicon prepared offline with Matrix decomposition based approach. • Custom language rules added: • Surrounding words could negate, amplify etc. • Dealing with Slang, abbreviations. Recorded version available at http://bit.ly/1PwhobK
  26. 26. © 2015 Impetus Technologies 26 TEXT CLASSIFICATION • 20 categories; Multiple labels if applicable • Semantic similarity approach based on Matrix Decomposition • Classification Model trained offline. Can be extended to more domains. Recorded version available at http://bit.ly/1PwhobK
  27. 27. © 2015 Impetus Technologies 27 BATCH PROCESS –TOPIC MODELING • Identification of main topics of conversation within text. • Model based on exploiting co-occurrence of words across documents. • Current model is specifically tuned for informal text like twitter to deal with issues like misspellings, slangs. Recorded version available at http://bit.ly/1PwhobK
  28. 28. © 2015 Impetus Technologies 28 DEMO – SOCIAL MEDIA ANALYTICS Recorded version available at http://bit.ly/1PwhobK
  29. 29. © 2015 Impetus Technologies 29 REAL-TIME MODEL BUILDING read Kafka Channel (Storm Spout) eventsKafka Topic Custom Processor (Model scoring in real-time) Kafka Channel (Storm Spout) Custom Processor (Model Building in real-time) read events Update model for scoring in real-time using the platform orchestration layer (functionality of StreamAnalytix) Scored event sent downstream Two copies of the event are read into two different topologies. One Topology is responsible for building/enhancing the model in real-time. This component will also access historical data to ensure model creation uses “all” the data. The second topology is responsible for applying the model in real-time for scoring. At a certain threshold, the ‘newer’ version of the model can be applied to the scoring component by using the platform orchestration layer without any down-time Another possible scenario: The real-time scoring and learning can be done by the same component. In such situations, a fork of the incoming message is not required. The system design/ configuration of doing learning and scoring in the same component is a matter of scale and performance tuning Historical/ training data read Recorded version available at http://bit.ly/1PwhobK
  30. 30. © 2015 Impetus Technologies 30 POLL Functionality Cores/ Capacity Duration Free 1 Limited Unlimited Forever Free 2 ALL Limited Forever Trial ALL Unlimited 60 days Which of these options for trying StreamAnalytix would you download ? Recorded version available at http://bit.ly/1PwhobK
  31. 31. © 2015 Impetus Technologies 31 Q&A (Use the chat/Q&A panel) Email us at inquiry@streamanalytix.com www.StreamAnalytix.com ? Request: On-premise and Cloud based trial and/or Proof of concept

×