Moving Beyond Batch: Transactional Databases for Real-time Data

226 views

Published on

Join guest Forrester speaker, Principal Analyst Mike Gualtieri, and Dennis Duckworth Director of Product Marketing from VoltDB to learn how enterprises can create a real-time, “origin-zero” data architecture within transactional databases to become a real-time enterprise.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Moving Beyond Batch: Transactional Databases for Real-time Data

  1. 1. page MOVING BEYOND BATCH: TRANSACTIONAL DATABASES FOR REAL-TIME DATA 1
  2. 2. page© 2016 VoltDB OUR SPEAKERS Dennis Duckworth Dir. of Product Marketing, VoltDB 2 Mike Gualtieri Principal Analyst Forrester Research
  3. 3. Moving  Beyond  Batch:  Transac4onal  Databases   For  Real-­‐Time  Data     Mike Gualtieri, Principal Analyst July 26, 2016 Webinar Twitter: @mgualtieri
  4. 4. #Priority  
  5. 5. ©  2015  Forrester  Research,  Inc.  Reproduc4on  Prohibited   5   52%   53%   53%   54%   58%   64%   64%   65%   66%   73%   75%   0%   10%   20%   30%   40%   50%   60%   70%   80%   BeNer  leverage  big  data  and  analy4cs  in  business  decision-­‐making   Create  a  comprehensive  strategy  for  addressing  digital   Create  a  comprehensive  digital  marke4ng  strategy   BeNer  comply  with  regula4ons  and  requirements   Improve  differen4a4on  in  the  market   Increase  influence  and  brand  reach  in  the  market   Address  rising  customer  expecta4ons   Improve  our  ability  to  innovate   Reduce  costs   Improve  our  products  /services   Improve  the  experience  of  our  customers   Customer  experience  and  product  innova4on  are  top   priori4es.   ›  Base:  3,005  global  data  and  analy4cs  decision-­‐makers   ›  Source:  Global  Business  Technographics  Data  And  Analy4cs  Online  Survey,  2015  
  6. 6. •  Learn  individual  customer   characteris4cs  and  behaviors   •  Detect  customer  needs  and   desires  in  real-­‐4me   •  Adapt  applica4ons  to  serve  an   individual  customer   Customer  experiences  must:  
  7. 7. ©  2015  Forrester  Research,  Inc.  Reproduc4on  Prohibited   7   82%  of  enterprises  are  interested  in   IoT  
  8. 8. •  Learn  individual  device  and   systems  of  devices   characteris4cs  and  behaviors   •  Detect  context  in  real-­‐4me   •  Adapt  applica4ons  to  improve   the  applica4ons   IoT  applica4ons  must:  
  9. 9. 9  ©  2016  Forrester  Research,  Inc.  Reproduc4on  Prohibited   “As you look to improve your data processing and analytics capabilities, what aspect of the implementation is most important to your business? Please select one.” 11%   11%   12%   16%   24%   25%   Quick  turnaround  on  customer  requests   More  data  availability   Expanded  access  to  more  business  users  (i.e.,  self-­‐ service)   Low  cost   Advanced  analy4cs  capabili4es  (e.g.  predic4ve.   prescrip4ve,  streaming)   Faster  performance  (4me  to  value)   Faster  (me  to  value  and  advanced  analy(cs  is  most   important  to  business   Base: 100 data science and data analytics leaders at enterprises within the US Source: A commissioned study conducted by Forrester Consulting, April 2016
  10. 10. #Data  
  11. 11. Data is like a drop of rain
  12. 12. It originates in an instant
  13. 13. And travels far before it ripples
  14. 14. #Real-­‐4me  
  15. 15. All data originates in real-time!
  16. 16. #  
  17. 17. But, analytics to gain insights is usually done much, much later.
  18. 18. #Perishable  
  19. 19. Insights are perishable.
  20. 20. Real-­‐4me   insights   Opera4onal   insights   Performance   insights   Strategic   insights   Insight:  Shopping  for   furniture   Ac4on:  Recommend  cleaning   supplies   Insight:  Profit  lower  than  goal   Ac4on:  Op4mize  price   Insight:  Demand  forecast   strong   Ac4on:  Increase  inventory   Insight:  Furniture  demand  high   Ac4on:  Expand  product  line  Time  to  Act   Perishability   Sub-­‐second  to   seconds   Seconds  to   hours   Days  to  weeks   Weeks  to   years   Sub-­‐second  to   seconds   Seconds  to   hours   Hours  to   weeks   Weeks  to   years   Enterprises must act on a range of perishable insights to get value from data and analytics
  21. 21. Batch analytics operations take too long  Business  Value     Time  To  Ac(on   Data   originated   Analy4cs   performed   Insights   gleaned   Ac4on   taken   Outdated   insights   Impotent  or   harmful   ac4ons   Posi4ve  Nega4ve   Decision   made   Poor  decision  
  22. 22. Compress analytics operations to maximize the value of data  Business  Value     Time  To  Ac(on   Posi4ve  Nega4ve   Maximum   Business  Value  
  23. 23. ©  2015  Forrester  Research,  Inc.  Reproduc4on  Prohibited   23   Real-­‐4me  means  highly  perishable   ›  A customer walks into a shopping mall ›  A shopper clicks on an online add ›  A temperature sensor spikes ›  A stock price rises ›  A customer uses a credit card ›  A customer wakes up
  24. 24. How can you know if you should you make an offer or send a gentle nudge right now?
  25. 25. How can you warn other drivers that the road is slippery to avoid a crash right now?
  26. 26. Is this customer thinking about moving to a rival firm right now?
  27. 27. Modern  applica4ons  infuse  analy4cs  to  respond  in  real-­‐4me  and   become  smarter   Streaming  data   Applica4on   interface   App  Logic     Applica4ons   Context   Ac4ons   Real-­‐4me   Context   Programmed   Logic   Learned    Logic  Machine  learning     Learning   External   Ac4ons   External   Context   From  other  data   sources  of   applica4ons   To  other  data   sources  or   applica4ons  
  28. 28. 28  ©  2016  Forrester  Research,  Inc.  Reproduc4on  Prohibited   “If there were no drawbacks (e.g. SLA concerns, resource consumption concerns) how interested would you be in having real-time data to use for modeling?” 66%   25%   7%   1%  1%   Very   interested   4   Moderately   interested   2   Not  at  all   interested   91%  of  data  scien(sts  express  interest  in  real-­‐(me  data  use  for   modeling     91%  are   interested  or   very  interested   Base: 100 data science and data analytics leaders at enterprises within the US Source: A commissioned study conducted by Forrester Consulting, April 2016
  29. 29. Real-time analytics is necessary to detect and act on real-time perishable insights.
  30. 30. #Challenges  
  31. 31. 31  ©  2016  Forrester  Research,  Inc.  Reproduc4on  Prohibited   “What are the technological challenges impeding you from processing and analyzing data more effectively? Select all that apply.” 6%   18%   18%   22%   27%   29%   35%   35%   37%   We  have  no  technical  challenges   Lack  of  analy4cal  tools   Lack  of  data  management  tools   Difficulty  in  crea4ng  data  models  and/or  preparing  data   for  analy4cs   Too  many  data  formats  to  integrate  effec4vely   Data  is  difficult  to  access  from  mul4ple  sources   Difficulty  integra4ng  data  from  mul4ple  sources   Time  it  takes  to  assemble  data  for  analysis     Data  volume  is  too  large   Top  technological  challenges   Base: 100 data science and data analytics leaders at enterprises within the US Source: A commissioned study conducted by Forrester Consulting, April 2016
  32. 32. The  data  lake  approach  is  insufficient  because  it  takes  too   long   Customer   Reference   Data  Lake   Opera4onal   Transac4onal   Analy4cs  tools   Insights   Data   Scien4sts   Business   intelligence  
  33. 33. #Solu4on  
  34. 34. Data gravity approach performs analytics where the preponderance of the data originates.
  35. 35. Compute gravity approach performs analytics where the preponderance of the compute resides.
  36. 36. 36  ©  2016  Forrester  Research,  Inc.  Reproduc4on  Prohibited   “Thinking specifically about building predictive models, which of the following best describes the importance of the data needed to build accurate models?” 38%   29%   45%   46%   54%   63%   63%   20%   34%   27%   28%   27%   21%   22%   External  data  third-­‐par4es   IoT  data   Mobile  data   Web  behavior  data   Opera4onal  data  (from  enterprise  applica4ons)   Transac4onal  data   Customer  reference  data   Data  scien(sts  recognize  importance  of  transac(onal  data  in   building  predic(ve  models     Top  2     85%   84%   81%   74%   72%   63%   58%   Base: 100 data science and data analytics leaders at enterprises within the US Source: A commissioned study conducted by Forrester Consulting, April 2016
  37. 37. A capable transactional database is the ideal place to perform real-time analytics
  38. 38. In-memory (RAM) can access data 58,000 times faster than disk.
  39. 39. #Capabili4es  
  40. 40. Architecture   •  Workload  scalability   •  Inges4on  throughput   •  Analy4cal  throughput   •  Analy4cal  latency   •  Fault  tolerance   •  Opera4onal  management   Stream/event  handling   •  Event  sequencing   •  Enrichment   •  Business  logic   Analy(cal  operators   •  Transforma4on   •  Aggrega4on   •  Correla4on   •  Time  windows   •  PaNern  matching   Applica(ons  dev.   •  Development  tools   •  Data  connectors   •  Extensibility   •  Dynamic  deployment   Evaluate a transactional database’s ability to also provide analytics based on these criteria
  41. 41. 110010011011 0100100 0100110011 010 Historical   Transac4ons   Customer  data   Security   Ability to ingest structured and unstructured from multiple sources in real-time.
  42. 42. Scale to handle any volume & velocity of data.
  43. 43. Process and analyze in real-time.
  44. 44. Provide fault-tolerance for mission-critical business and customer applications.
  45. 45. Provide tools that make it easy to manage and monitor the platform and it’s interaction with other architecture components.
  46. 46. Offer tools to visualize insights from real-time data.
  47. 47. #Opportunity  
  48. 48. Enterprises must act on a range of perishable insights to get value from big data Real-­‐(me   Insights   Strategic   Insights   Opera(onal   Insights   Performance   Insights   Time  to  Act   Perishability   Sub-­‐second  to   seconds   Seconds  to   hours   Days  to  weeks   Weeks  to   years   Sub-­‐second  to   seconds   Seconds  to   hours   Hours  to   weeks   Weeks  to   years  
  49. 49. Use real-time analytics to create a whole new class of real-time customer experiences.
  50. 50. forrester.com Thank  you   Mike Gualtieri mgualtieri@forrester.com Twitter: @mgualtieri
  51. 51. page MOVING BEYOND BATCH: TRANSACTIONAL DATABASES FOR REAL-TIME DATA 51
  52. 52. page #BigData  
  53. 53. page© 2016 VoltDB Aggregate Data Value DataValue Interactive Real-time Analytics Record Lookup Historical Analytics Exploratory Analytics Data in Motion Data at Rest Big Data BIG DATA
  54. 54. page© 2016 VoltDB DIKW MODEL
  55. 55. page© 2016 VoltDB DIKUW VARIATION 55
  56. 56. page© 2016 VoltDB DIKW MODEL
  57. 57. page© 2016 VoltDB DIKW FOR NEXT BEST ACTION If we offer this player a free magic sword to get through the challenge, they will keep playing and are likely to buy a shield Historically, players who spend this much time at this level quit out of frustration This user has been at this challenge for over 10 minutes, which is above the high average amount of time of all users This user is playing our game, this user is at the cave challenge, this user is at the cave challenge, this user is at the cave challenge...
  58. 58. page #Ac4onableInsights  
  59. 59. page© 2016 VoltDB What good are “actionable insights” if you can’t or don’t act on them? 59
  60. 60. page #FastData  
  61. 61. page© 2016 VoltDB Aggregate Data Value DataValue Interactive Real-time Analytics Record Lookup Historical Analytics Exploratory Analytics Data in Motion Data at Rest Big Data BIG DATA
  62. 62. page© 2016 VoltDB Value of Individual Data Item Aggregate Data Value DataValue Interactive Real-time Analytics Record Lookup Historical Analytics Exploratory Analytics Data in Motion Data at Rest Fast Data Big Data FAST DATA + BIG DATA DatumValue(ActionValue)
  63. 63. page© 2016 VoltDB Value of Individual Data Item Aggregate Data Value TotalDataValue Data Warehouses Hadoop, etc.NoSQL Interactive Real-time Analytics Record Lookup Historical Analytics Exploratory Analytics Data in Motion Data at Rest Fast Data Big Data Feeds, Collectors CEP CEP + DB VoltDB FAST DATA + BIG DATA DatumValue(ActionValue)
  64. 64. page© 2016 VoltDB WHAT VOLTDB DOES REALLY WELL •  Ingest data/events really quickly (100K-1M+ events/sec) •  Allow action on data/events (in context) really quickly (under 10 millisecond response times) •  ...with “immediately consistent” and accurate data •  ...with strong isolation (strongly serializable) and durability •  Export the data to downline systems really quickly (allowing use as “fast data pipeline”) •  All highly scalable •  ... scaling out on commodity servers •  ... scaling more efficiently than many other systems 64
  65. 65. page© 2016 VoltDB DON’T JUST BELIEVE US (OR ANYONE ELSE)... VoltDB was subjected to the most stringent Jepsen test ever ...because VoltDB makes the most stringent claim (Strongly Serializable) VoltDB latest version (v6.4) passed the test https://voltdb.com/blog/voltdb-passes-official-jepsen-testing 65
  66. 66. page #Ac4ons  
  67. 67. page© 2016 VoltDB SOME TIME-SENSITIVE (REAL-TIME) USE CASES WE SEE... •  Telco/Mobile - Authorization •  Someone just opened browser on their phone. Do we allow them to connect to Internet? •  Gaming – Personalization •  User has spent over “high average” amount of time at a particular challenge. What should we do to keep them engaged? •  Financial Services - Arbitrage •  We’ve got a lot of IBM stock to sell off, with some of our clients wanting to buy. How do we get out of IBM in a way to maximize profit (and minimize market disruption)? •  Ad-Tech - Billing Management •  Good placement opportunity for our client but their remaining ad budget is very close to zero. Should we buy the placement for them or not? •  Across Verticals - SLA Management •  We have a complex (multiple step) process but a short fixed time in which to complete the process to meet our SLA; if we don’t, we have to pay penalty. How do we prioritize all those steps to maximize efficiency and minimize cost?
  68. 68. page #BusinessValue  
  69. 69. page© 2016 VoltDB KEY TAKEAWAYS •  Companies want to improve the experience of their customers and look to doing more analytics faster as one way of doing that: •  Process high volumes and velocity data in real time •  Extract actionable insights •  Act on those insights •  Fast Data solutions like VoltDB allow you to process data, extract insights, and act on them, all in real- time, to maximize the business value of that data.
  70. 70. page© 2015 VoltDB page THANK YOU 70

×