Kyle Kingsbury Talks about the Jepsen Test: What VoltDB Learned About Data Accuracy and Consistency
•
2 likes•568 views
In this webinar Kyle Kingsbury will talk about the testing process, how Jepsen works and what he found in VoltDB. Then listen to John Hugg, the Founding Engineer at VoltDB to learn why these tests are important, how VoltDB reacted to test findings and the QA process moving forward.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (8)
Similar to Kyle Kingsbury Talks about the Jepsen Test: What VoltDB Learned About Data Accuracy and Consistency
Similar to Kyle Kingsbury Talks about the Jepsen Test: What VoltDB Learned About Data Accuracy and Consistency (20)
More from VoltDB
More from VoltDB (13)
Recently uploaded
Recently uploaded (20)
Kyle Kingsbury Talks about the Jepsen Test: What VoltDB Learned About Data Accuracy and Consistency
- 1. Kyle Kingsbury Talks about the Jepsen Test: ! ! What VoltDB Learned About Data Accuracy and Consistency! or! presents…!
- 2. Presenters! John Hugg, VoltDB Inc. Founding Engineer at VoltDB Was involved with both adding bugs found by Jepsen and fixing them. Kyle Kingsbury, Jepsen.io (aphyr) Creator of Jepsen, Riemann, Tesser Likes to break databases. Broke VoltDB, in fact.
- 3. Why did VoltDB pick Jepsen? What is VoltDB and what’s it good for? Learn More So What?
- 5. voltdb.com/jepsen jepsen.io pages VoltDB’s Jepsen-specific content Learn more about VoltDB Hacker News comments
- 6. What is VoltDB? • Scale-out, clustered, SQL Relational database • Strong serializable transactions by default, even at high scale • An excellent processing engine with rich import/ export functionality
- 7. Use Cases • Anything where decisions based on logic are made in real time for incoming events: • Policy Enforcement • Fraud Detection • Ad Tech • Real-time personalization • Payments • Anything where math / calculations are done: • Billing and reporting on live data • State tracking
- 8. Example: Telco Mobile phone is dialed. Request sent to VoltDB to decide if it should be let through. Single transaction looks at state and decides if this call: is fraudulent? is permitted under plan? has prepaid balance to cover? State Blacklists Fraud Rules Billing Info Recent Activity for both Numbers Export to OLAP
- 9. Example: Micro Personalization User clicks link on a website. This generates a request to VoltDB. VoltDB transaction scans a table of rules and checks which apply to this event. Eventually the transaction decides what to show the user next. That decision is exported to HDFS Spark ML is used to look at historical data in HDFS and generate new rules. These rules are loaded into VoltDB every few hours. User sees personalized content
- 10. StateProcessing
- 11. Complex, Transactional Business Logic Scale-Out Performance Streaming Events SQL Relational State
- 12. Why Jepsen? • We are always hungry for tests! • Could build VoltDB-Jepsen harness ourselves, but… Wouldn’t be as good and wouldn’t have Kingsbury’s credibility. • Customers have asked about it. • Kingsbury has a built-in audience (marketing)
- 61. VoltDB’s Takeaway Serious question: What’s the worst that could happen?
- 62. VoltDB Takeaway • Our policy: Consistency or data loss bugs are blocking bugs to be prioritized above all else. • So if Jepsen finds bugs, we need to fix them ASAP. • The risk is that Jepsen finds bugs that we have to fix, which might impact our schedule. • But that’s dumb. If our product has bugs, not knowing about them doesn’t make them not there.
- 63. VoltDB Takeaway Marketing & Perception: • Passing Jepsen is good. People talking about VoltDB is good. Showing we care about this stuff is good. • Having bugs is bad, but discussing and fixing issues openly and seriously can be positive.
- 65. Reproducible! • 100% reproducible test: • Set up Jepsen from Github • Clone Jepen VoltDB driver from Github • Run! • Can’t do this with systems you don’t control. github.com/jepsen-io/voltdb!
- 66. I want to learn more! chat.voltdb.com forum.voltdb.com askanengineer @voltdb.com @johnhugg @voltdb @aphyr voltdb.com/jepsen