Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
page
DESIGNING AN ARCHITECTURE FOR
REAL-TIME EVENT PROCESSING
page© 2016 VoltDB
OUR SPEAKERS
2
Mike Stonebraker
Co-Founder of VoltDB
2014 Turing Award Winner
John Hugg
Founding Enginee...
page© 2016 VoltDB
WHAT IS VOLTDB?
ü In-Memory
ü Relational, SQL, ACID Compliant
ü Scale-out on commodity hardware
ü Re...
page© 2016 VoltDB page
MIKE STONEBRAKER
4
page© 2016 VoltDB
OUTLINE
•  Assumptions
•  A quadrant chart
•  Explanations
•  The details (Hugg)
5
page© 2016 VoltDB
ASSUMPTIONS
•  Data (messages) stream in from humans or devices
•  Massively multi-player games
•  Inter...
page© 2016 VoltDB
STREAMING SYSTEMS: QUADRANT CHART
7
Time CriticalNot Time Critical
Unimportant Data
Important Data
page© 2016 VoltDB
TIME AXIS
•  Not time critical
•  Seconds to minutes
•  Most any technology will work
•  Time Critical
•...
page© 2016 VoltDB
DATA AXIS
•  Unimportant data
•  Can lose messages without great harm
•  Partial processing without grea...
page© 2016 VoltDB
EXAMPLES
•  Unimportant data
•  Electronic trading
•  Important data
•  Risk assessment
10
page© 2016 VoltDB
STREAMING SYSTEMS: QUADRANT CHART
11
Time CriticalNot Time Critical
Unimportant Data
Important Data
Most...
page© 2016 VoltDB
STREAMING SYSTEMS: QUADRANT CHART
12
Time CriticalNot Time Critical
Unimportant Data
Important Data
Most...
page© 2016 VoltDB
STREAMING ENGINES
•  Exactly once semantics through as pipeline
of operations
•  With replication/failov...
page© 2016 VoltDB
STREAMING SYSTEMS: QUADRANT CHART
14
Time CriticalNot Time Critical
Unimportant Data
Important Data
Most...
page© 2016 VoltDB
VOLTDB
•  Main memory OLTP DBMS
•  Arbitrary transactions
•  Exactly once semantics
•  Transactional rep...
page© 2016 VoltDB page
JOHN HUGG
16
page© 2016 VoltDB 17
page© 2016 VoltDB 18
page© 2016 VoltDB 19
page© 2016 VoltDB 20
page© 2016 VoltDB 21
page© 2016 VoltDB 22
page© 2016 VoltDB 23
page© 2016 VoltDB 24
page© 2016 VoltDB 25
page© 2016 VoltDB 26
page© 2016 VoltDB 27
page© 2016 VoltDB 28
page© 2016 VoltDB 29
page© 2016 VoltDB 30
page© 2016 VoltDB 31
page© 2016 VoltDB 32
page© 2016 VoltDB 33
page© 2016 VoltDB 34
page© 2016 VoltDB 35
page© 2016 VoltDB 36
page© 2016 VoltDB 37
page© 2016 VoltDB 38
page© 2016 VoltDB 39
page© 2016 VoltDB 40
page© 2016 VoltDB 41
page© 2016 VoltDB 42
page© 2016 VoltDB 43
page© 2016 VoltDB 44
page© 2016 VoltDB 45
page© 2016 VoltDB 46
page© 2016 VoltDB 47
page© 2016 VoltDB 48
page© 2016 VoltDB 49
page© 2016 VoltDB 50
page© 2016 VoltDB 51
page© 2016 VoltDB 52
page© 2016 VoltDB 53
page© 2016 VoltDB 54
page© 2016 VoltDB 55
page© 2016 VoltDB 56
page© 2016 VoltDB 57
page© 2016 VoltDB 58
page© 2016 VoltDB 59
page© 2016 VoltDB 60
page© 2016 VoltDB 61
page© 2016 VoltDB 62
page© 2016 VoltDB 63
page© 2016 VoltDB 64
page© 2016 VoltDB 65
page© 2016 VoltDB 66
page© 2016 VoltDB 67
page© 2016 VoltDB 68
page© 2016 VoltDB
QUESTIONS?
•  Use the chat window to type in your questions
Ø  Download VoltDB:
•  www.voltdb.com/Downl...
Upcoming SlideShare
Loading in …5
×

Mike Stonebraker on Designing An Architecture For Real-time Event Processing

720 views

Published on

In this webinar, Mike Stonebraker explains the tradeoffs of performance, scale, and required programming “heroics” in capturing the value of fast data with different stream processing alternatives. Then hear from John Hugg, Founding Engineer of VoltDB, as he discusses the VoltDB approach to fast data. Learn what’s possible when systems integrate event processing with state management in a consistent, transactional way. You can also view the webinar recording here: http://learn.voltdb.com/WRStonebrakerRTeventprocessing.html

Published in: Software
  • Be the first to comment

Mike Stonebraker on Designing An Architecture For Real-time Event Processing

  1. 1. page DESIGNING AN ARCHITECTURE FOR REAL-TIME EVENT PROCESSING
  2. 2. page© 2016 VoltDB OUR SPEAKERS 2 Mike Stonebraker Co-Founder of VoltDB 2014 Turing Award Winner John Hugg Founding Engineer, VoltDB
  3. 3. page© 2016 VoltDB WHAT IS VOLTDB? ü In-Memory ü Relational, SQL, ACID Compliant ü Scale-out on commodity hardware ü Reliability, HA, fault tolerant ü Integration with OLAP, Hadoop, DW 3 An operational database purpose-built to run 100% in-memory at web scale Best use cases: operational and transactional workloads
  4. 4. page© 2016 VoltDB page MIKE STONEBRAKER 4
  5. 5. page© 2016 VoltDB OUTLINE •  Assumptions •  A quadrant chart •  Explanations •  The details (Hugg) 5
  6. 6. page© 2016 VoltDB ASSUMPTIONS •  Data (messages) stream in from humans or devices •  Massively multi-player games •  Internet of Things (IoT) •  At high volume •  Anybody can do 10 messages/sec •  100K messages/sec requires specialized software •  High Availability •  Nobody wants to go down these days 6
  7. 7. page© 2016 VoltDB STREAMING SYSTEMS: QUADRANT CHART 7 Time CriticalNot Time Critical Unimportant Data Important Data
  8. 8. page© 2016 VoltDB TIME AXIS •  Not time critical •  Seconds to minutes •  Most any technology will work •  Time Critical •  Msec to seconds •  Low latency along with assumed high throughput is essential 8
  9. 9. page© 2016 VoltDB DATA AXIS •  Unimportant data •  Can lose messages without great harm •  Partial processing without great harm •  Important data •  Exactly once processing required •  Though a possibly extensive pipeline •  Together with (perhaps multiple) external actions 9
  10. 10. page© 2016 VoltDB EXAMPLES •  Unimportant data •  Electronic trading •  Important data •  Risk assessment 10
  11. 11. page© 2016 VoltDB STREAMING SYSTEMS: QUADRANT CHART 11 Time CriticalNot Time Critical Unimportant Data Important Data Most anything, Spark, etc. Data warehouses, RDBMS, Transaction processing engines
  12. 12. page© 2016 VoltDB STREAMING SYSTEMS: QUADRANT CHART 12 Time CriticalNot Time Critical Unimportant Data Important Data Most streaming systems Most anything, Spark, etc. Data warehouses, RDBMS, Transaction processing engines
  13. 13. page© 2016 VoltDB STREAMING ENGINES •  Exactly once semantics through as pipeline of operations •  With replication/failover •  Either not present or ridiculously expensive •  i.e. logging state to disk 13
  14. 14. page© 2016 VoltDB STREAMING SYSTEMS: QUADRANT CHART 14 Time CriticalNot Time Critical Unimportant Data Important Data Most streaming systems (exactly once semantics is very expensive) Most anything, Spark, etc. Data warehouses, RDBMS, Transaction processing engines
  15. 15. page© 2016 VoltDB VOLTDB •  Main memory OLTP DBMS •  Arbitrary transactions •  Exactly once semantics •  Transactional replicas •  With automatic failover and recovery 15
  16. 16. page© 2016 VoltDB page JOHN HUGG 16
  17. 17. page© 2016 VoltDB 17
  18. 18. page© 2016 VoltDB 18
  19. 19. page© 2016 VoltDB 19
  20. 20. page© 2016 VoltDB 20
  21. 21. page© 2016 VoltDB 21
  22. 22. page© 2016 VoltDB 22
  23. 23. page© 2016 VoltDB 23
  24. 24. page© 2016 VoltDB 24
  25. 25. page© 2016 VoltDB 25
  26. 26. page© 2016 VoltDB 26
  27. 27. page© 2016 VoltDB 27
  28. 28. page© 2016 VoltDB 28
  29. 29. page© 2016 VoltDB 29
  30. 30. page© 2016 VoltDB 30
  31. 31. page© 2016 VoltDB 31
  32. 32. page© 2016 VoltDB 32
  33. 33. page© 2016 VoltDB 33
  34. 34. page© 2016 VoltDB 34
  35. 35. page© 2016 VoltDB 35
  36. 36. page© 2016 VoltDB 36
  37. 37. page© 2016 VoltDB 37
  38. 38. page© 2016 VoltDB 38
  39. 39. page© 2016 VoltDB 39
  40. 40. page© 2016 VoltDB 40
  41. 41. page© 2016 VoltDB 41
  42. 42. page© 2016 VoltDB 42
  43. 43. page© 2016 VoltDB 43
  44. 44. page© 2016 VoltDB 44
  45. 45. page© 2016 VoltDB 45
  46. 46. page© 2016 VoltDB 46
  47. 47. page© 2016 VoltDB 47
  48. 48. page© 2016 VoltDB 48
  49. 49. page© 2016 VoltDB 49
  50. 50. page© 2016 VoltDB 50
  51. 51. page© 2016 VoltDB 51
  52. 52. page© 2016 VoltDB 52
  53. 53. page© 2016 VoltDB 53
  54. 54. page© 2016 VoltDB 54
  55. 55. page© 2016 VoltDB 55
  56. 56. page© 2016 VoltDB 56
  57. 57. page© 2016 VoltDB 57
  58. 58. page© 2016 VoltDB 58
  59. 59. page© 2016 VoltDB 59
  60. 60. page© 2016 VoltDB 60
  61. 61. page© 2016 VoltDB 61
  62. 62. page© 2016 VoltDB 62
  63. 63. page© 2016 VoltDB 63
  64. 64. page© 2016 VoltDB 64
  65. 65. page© 2016 VoltDB 65
  66. 66. page© 2016 VoltDB 66
  67. 67. page© 2016 VoltDB 67
  68. 68. page© 2016 VoltDB 68
  69. 69. page© 2016 VoltDB QUESTIONS? •  Use the chat window to type in your questions Ø  Download VoltDB: •  www.voltdb.com/Download •  Open source version is available on github.com •  Developer  Guide  to  Streaming  Data  is in the resources section on the right side of your screen or download here: •  https://voltdb.com/resources/whitepapers •  Stored  Procedure  Superpowers:  A  Developer’s  Guide   Register: http://goo.gl/kqLUMW •  Join the conversation on Twitter #VoltDBFastData 69

×