Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How Bell Canada and Twilio Succeed with Real-Time Analytics

176 views

Published on

For enterprises with thousands of customers, optimizing their data analytics infrastructure is key to gaining competitive advantage. They need to make real-time analytics easily and quickly accessible to multiple teams, to monitor issues such as network performance and fraud detection.

Published in: Software
  • Be the first to comment

  • Be the first to like this

How Bell Canada and Twilio Succeed with Real-Time Analytics

  1. 1. Arcadia Data. Proprietary and Confidential How Bell Canada and Twilio Succeed with Scaling Real-Time Analytics
  2. 2. Arcadia Data. Proprietary and Confidential2 Featured Speakers Shant Hovsepian Co-Founder and CTO Arcadia Data David Vincent Senior Manager, Data Science Bell Canada Dave Burgess Senior Director, Data Platform & Data Science Twilio Joydeep Das Sr. Director, Product Cloudera
  3. 3. 3 Arcadia Data. Proprietary and Confidential  If you have any questions along the way, please type them into the chat window.  If you have audio problems, please chat us for help.  A recording of this presentation will be sent to you in a few days.  Please live tweet! @arcadiadata Before We Begin Our Presentation
  4. 4. 4 Arcadia Data. Proprietary and Confidential Arcadia Data Mission: To Connect Business Users to Big Data • Founding team from Teradata Aster, HPE 3PAR, IBM DB2 • Visual analytics and BI native to big data • Architected to solve big data analytics challenges • Large production customers in the Global 1000 Customers Strong Performer: Hadoop Native BI Wave Report, September 2016. ”Put your BI where your data is” Gartner names Arcadia Data a 2017 Cool Vendor for IoT Analytics, April 2017. Recent Awards CRN names Arcadia Data to both the 30 Coolest Business Analytics Vendors and the Coolest Emerging Big Data Vendors lists, May 2018. Leader and Technology Innovation Award Winner for 2017 Big Data Analytics Market Study
  5. 5. Arcadia Data. Proprietary and Confidential5 Advanced Visualizations and Semantic Layer Arcadia Is Visual Analytics Built from the Ground Up for Big Data • In-cluster for high performance, high concurrency • Distributed BI on every node • No data movement • Unified security • Single semantic layer • Blend with additional data sources, including S3 Data Lake Cluster Data Node Data Node Data Node Data Node Data Node … … … … … … … Streaming (via KSQL) Other external data sources
  6. 6. Arcadia Data. Proprietary and Confidential
  7. 7. 7 “I’m on the corner of 4th & Main.” “Your reservations are confirmed for Friday.” “I’ll help you resolve this issue right now.” The keys are under the mat. This new fall look is perfect for you. “Calling to check in on our newest product.” THE NEW STANDARD FOR CUSTOMER COMMUNICATIONS
  8. 8. © 2 0 1 8 T W I L I O , I N C . A L L R I G H T S R E S E R V E D . HOW DO YOU COMMUNICATE? YOUR CUSTOMERS YOUR PARTNERS YOUR TEAM DEPARTMENTS INDIVIDUALS SYSTEMS ?
  9. 9. © 2 0 1 8 T W I L I O , I N C . A L L R I G H T S R E S E R V E D . COMMUNICATIONS ARE TRANSFORMING CUSTOMER JOURNEY BETTER MARKETING text marketing lead dialers click-to-call Ads … BETTER CSAT Account Alerts web dialers Web Chat … REDUCE RISK TEXT AUTHORIZATION Phone Verification Two-Factor Auth … INCREASED REVENUE In-app Video … SMS SALES Order Notifications BETTER USER EXPERIENCE Arrival Alerts shift scheduling … anonymous calling
  10. 10. © 2 0 1 8 T W I L I O , I N C . A L L R I G H T S R E S E R V E D . RIGHT CHANNEL RIGHT CONTEXT RIGHT PERSON THE BEST COMMUNICATIONS ARE CUSTOMIZED FOR THE END CUSTOMER
  11. 11. © 2 0 1 8 T W I L I O , I N C . A L L R I G H T S R E S E R V E D . TWILIO CLOUD COMMUNICATIONS PLATFORM USE ANY CHANNEL DEPARTMENTS INDIVIDUALS SYSTEMS INTEGRATE WITH SOFTWARE APIS FOR CUSTOMIZING COMMUNICATIONS YOUR CUSTOMERS YOUR PARTNERS YOUR TEAM
  12. 12. © 2 0 1 8 T W I L I O , I N C . A L L R I G H T S R E S E R V E D . SECURITYCUSTOMER SERVICE LOGISTICSE-COMMERCE BUILT WITH TWILIO MARKETING
  13. 13. Confidential and Proprietary, Copyright 2018 Twilio Inc. 1. Developers want to see their Twilio Usage when they launch their apps 2. Marketers want to see how their marketing campaign is performing 3. Contact Center Managers want to see the status of their contact center 4. Twilio Operations and Customers want to see VOIP call quality and network health, and detect anomalies REAL-TIME STREAMING USE CASES WITH DATA PIPELINES
  14. 14. Network Big Data &AI SMS spammer detection usecase
  15. 15. SMSuse case - businessproblem Page • Spam is defined as unsolicited messages sent without the authorization of the recipient. Increasingly, spammers are turning to SMS to elicit revenue. SMS spam could be in form of unsolicited advertising, Inappropriate or adult-themed content, premium rate fraud, “Smishing”, Mobile Malware - SMS message open rate are 90% within 15min of receipt vs email open rate of 25% within 25 hrs of receipt – ITU - 90% of SMS spam falls into the scam/fraud category • SMS Spammers defraud users by phishing personal or financial information, impersonating Bell or Bell’s business customer • Most spammers use pre-paid accounts activated for the purpose of spamming only
  16. 16. Traditional solution limitations Page Spammers able to adapt approaches overcoming rules Applying and adjusting rules requires effort • Rules based with continuous need to adjust rules • • • Traditional solutions composed of many components with limited/targeted capabilities • Does not scale up and is not cost efficient • Visualization of interactions not real time
  17. 17. • Real-time log streaming • <3sec to dashboard • Operational dashboards • Single view with multiple datasets • Long term data retention • 1stsolution to handle all SMS logs (295 fields) • ~250 mil records per day @3K TPS • Faster troubleshooting + reporting (sec vs days) • AI Algorithm – Spammer detection • Anomaly based spammer detection Faster and more spammer detection than rules based Hadoop enables AI, Arcadia Data realtime visualization Page
  18. 18. Significant improvements Page Improvements Total spammer detected 250% Spam sent before detection - 90 % Detection Average of 24hrs faster Effort 99% reduction in manual work
  19. 19. SCALING REAL TIME ANALYTICS Joydeep Das | Senior Director, Products October 24, 2018
  20. 20. 20 © Cloudera, Inc. All rights reserved. BUSINESS VALUE FROM REAL TIME ANALYTICS Business Value From Operations Latency Of Analytics Low High High
  21. 21. 21 © Cloudera, Inc. All rights reserved. LOW LATENCY BUSINESS IMPERATIVES Network Performance Monitoring for Telecom Operators Cyber Threat Detection Quality Anomaly Detection in Manufacturing
  22. 22. 22 © Cloudera, Inc. All rights reserved. Recent Awards Enterprise customers Nodes deployed Partners THRIVING FOOTPRINT
  23. 23. 23 © Cloudera, Inc. All rights reserved. OPTIMIZED FOR REAL TIME ANALYTICS Source Ingestion Curation Partner Ingest Tools Curation, Text Indexing Security & Governance Queries Business Intelligence WORKLOAD XM SQL Queries Text Queries
  24. 24. 24 © Cloudera, Inc. All rights reserved. LOW LATENCY INSIGHTS REAL TIME ANALYTICS ON TIME SERIES DATA Time SeriesEvents Data Analytics from Logs Inserts, updates, scans, lookups Workload Examples Stream market data, fraud detection & prevention, risk monitoring Events data (typically from Logs) is most valuable if we can analyze it to change outcomes in real time. • Time series/Events data inserted/updated as it arrives • Analytic scans to find trends on latest data • Point in time lookups where an event occurred
  25. 25. 25 © Cloudera, Inc. All rights reserved. LOW LATENCY INSIGHTS WITH MINING EQUIPMENT IoT from mining trucks & equipment 400Mining Machines Produce 100s of thousands of metrics 5,000 220k New machines for data capture Writes per Second (Peak Volume) Handles complex out of order data arrivals as well
  26. 26. Social media: @arcadiadataarcadiadata.com 26 Education about Real-time Analytics See Arcadia on Kudu in Action Download Arcadia Instant https://www.arcadiadata.com/product/streami ng-visualizations/ https://www.arcadiadata.com/product/kudu-for-bi- analytics/ arcadiadata.com/instant

×