Your SlideShare is downloading. ×
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
VoltDB - Stonebraker Live! - New York City 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

VoltDB - Stonebraker Live! - New York City 2013

1,832

Published on

VoltDB’s Dr. Michael Stonebraker of MIT, UC Berkeley, and Ingres and Postgres fame, presents the founding principles of solving modern data-velocity problems: “Data is growing faster than hard …

VoltDB’s Dr. Michael Stonebraker of MIT, UC Berkeley, and Ingres and Postgres fame, presents the founding principles of solving modern data-velocity problems: “Data is growing faster than hard drives,” “Move the computation to the data, never move the data to the computation,” “Bet on main memory because there's no other way to go fast,” and “Run transactions to completion, and you eliminate locking and multithreading,” are central to his beliefs.

Not-so-coincidentally, they’re also central concepts of VoltDB.

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,832
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
53
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • VoltDB was uniquely designed to let companies act on data automatically, the instant it's created, at its point of maximum value. No other database is constructed like this. We are the fastest, most scalable database on the planet. Hands down. Financial trading applications... e-commerce applications... billing applications in dynamic markets like telecommunications... sensor-driven applications in sectors like energy and healthcare. With VoltDB at the core, applications can decision against the freshest, most in-the-moment data, and they can do it at a fraction of the cost that they would with any other database on the market.Three million transactions per secondAn unprecedented drop in cost per transactionAll impacting a company's bottom lineVoltDB enables NOW.
  • done on the volt10'sDell R510 server2 x Intel(R) Xeon(R) (quad core) CPU X5670  @ 2.93GHz64GB RAM
  • Transcript

    • 1. Stonebraker Live!Navigating the Database UniverseVoltDB presents
    • 2. BRUCE READINGPresident and CEO
    • 3. • Traditional RDBMS is all wrong– Presented by Dr. Michael Stonebraker, Co-founder• Making sense of the database universe– Presented by Bruce Reading, President and CEO• Hello VoltDB 3.0– Presented by Ryan Betts, Field CTOAgenda
    • 4. TRADITIONAL RDBMS WISDOM IS ALLWRONGDr. Michael Stonebraker
    • 5. Traditional RDBMS Wisdom• Data is in disk block formatting (heavily encoded)• With a main memory buffer pool of blocks• Query plans– Optimize CPU, I/O– Fundamental operation is read a row• Indexing via B-trees– Clustered or unclustered
    • 6. Traditional RDBMS Wisdom• Dynamic row-level locking• Aries-style write-ahead log• Replication (asynchronous or synchronous)– Update the primary first– Then move the log to other sites– And roll forward at the secondary (s)
    • 7. Traditional RDBMS Wisdom• Describes MySQL, DB2, Postgres, SQLServer, Oracle…• Focus of most college-level DBMS courses– Including M.I.T.• Focus of most DBMS textbooks
    • 8. Traditional RDBMS Wisdom• Is completely wrong• (More charitably) is obsolete
    • 9. The DBMS Marketplace• About 1/3 “data warehouses”– Lots of big reads– Bulk-loaded from OLTP systems• About 1/3 “OLTP”– Lots of small updates– And a few reads• About 1/3 “everything else”– Hadoop, NoSQL, graph DBMS, Array DBMS…
    • 10. The DBMS Marketplace• Data warehouses– Market already moving strongly in the direction of column stores– Which have nothing to do with the traditional wisdom– Because column stores are 50 – 100 X row stores
    • 11. The Participants• Native column store vendors– HP/Vertica, SAP/Hana, Red Shift (Amazon/Paraccl), SAP/Sybase/IQ• Native row store vendors– Microsoft, Oracle, DB2, Netezza• In transition– Teradata, Asterdata, Greenplum• If you are running a row store, then be prepared to switch!
    • 12. The DBMS Marketplace• OLTP– NewSQL systems are wildly faster than the traditional wisdom• Everything else– Not an RDBMS market
    • 13. OLTP Databases – 3 Big Decisions• Main memory vs. disk orientation• Replication strategy• Concurrency control strategy
    • 14. Reality Check on OLTP Databases• TP database size grows at the rate transactions increase• 1 Tbyte of main memory buyable for around $30K (or less)– (say) 64 Gbytes per server in 16 servers• 10+ Tbytes possible• If your data doesn’t fit in main memory now, then wait acouple of years and it will…
    • 15. Reality Check – Main Memory Performance• TPC-C CPU cycles• On the Shore DBMSprototype• “Elephants” should besimilar
    • 16. To Go Fast• Must focus on overhead– B-trees affects a small fraction of the path length• Must get rid of all four pie slices– Anything less gives you a marginal win– TimesTen as an example16
    • 17. Buffer Pool Overhead• Get rid of the buffer pool• i.e., run a main-memory DBMS– Like VoltDB
    • 18. Single Threading• Hosed unless you do this– Unless you get rid of queuing (somehow)– Or eliminate shared data structures (somehow)• VoltDB statically divides shared memory among the cores– And cores are single threaded
    • 19. Concurrency Control• MVCC popular (NuoDB, Hekaton)• Time stamp order popular (VoltDB)• I don’t know anybody who is doing normal dynamic locking– It’s too slow!!!!
    • 20. Reality Check – High Availability (HA)• Requirement in today’s OLTP systems• Nobody will take down time• Must be solved through replication
    • 21. How to Implement HA• I am only interested in ACID outcomes!!!!• Eventual consistency actually means “creates garbage”– Consider 2 customers at 2 sites, each buying the last “widget”• Even Jeff Dean (Google) has come around to this pointof view
    • 22. How to Implement HA• Active-Passive– Effectively requires you to write a log– One of the four pie slices• Active-Active (VoltDB solution)– Send only the transaction, not the effect of the transaction– Allows read-queries to be sent to any replica
    • 23. Reality Check – Power Failures• What to do if you don’t have UPS…• Cannot lose data on a power failure!!!!• Two options– Bring back the log (and the pie slice)– Command log plus asynchronous checkpoints
    • 24. Some Data From Nirmesh Malvaiya• Implemented Aries in VoltDB• Compared against the VoltDB command logging• Command logging about 3X faster in total throughput
    • 25. The Nail in the Coffin• Time stamp order compatible with active-active– As are any deterministic schemes• Locking and MVCC are not– Need a 2 phase commit between the replicas– Slow, slow, slow
    • 26. Net-Net on OLTP• Main memory DBMS• Deterministic concurrency control• HA via active-active• Has nothing to do with the traditional wisdom• Even if your data is too big for main memory– The traditional wisdom is still wrong– Stay tuned for a paper on this topic
    • 27. Summary• What we teach our DBMS students is all wrong• Implementations from the “elephants” are all obsolete– One-size-does-not-fit-all– Several million lines of code per vendor are obsolete• I expect a lot of turmoil in the market off into the future
    • 28. MAKING SENSE OF THE DATABASEUNIVERSEBruce Reading
    • 29. The fact is…There’s only more andmore to come.And it’s not slowingdown…Record amounts of dataare being createdeveryday…
    • 30. And if that data is most valuable atthe moment it’s created, how do youput it to use NOW?How do you automate decisioningagainst it NOW?
    • 31. NOW
    • 32. Imagine…
    • 33. Nice story. So what?
    • 34. Large, busy bankRogue trader5 “Mistypednumber”-$Small sum lost9 “Mistypednumber”& “Mistypednumber-$Small sum lost-$Small sum lostOblivious-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$ -$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$ -$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$ -$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$-$ -$
    • 35. -$2BNLarge sum lostThird largest loss inbanking history
    • 36. UBS couldnt flag it among allthe data... until it was too late.
    • 37. This is our world now.
    • 38. Same old, same oldwon’t cut it.
    • 39. What’s a developer to do?
    • 40. Data Value ChainInteractive Real-time Analytics Record Lookup Historical Analytics Exploratory AnalyticsMilliseconds Hundredths of seconds Second(s) Minutes Hours• Place trade• Serve ad• Enrich stream• Examine packet• Approve trans.• Calculate risk• Leaderboard• Aggregate• Count• Retrieve clickstream• Show orders• Backtest algo• BI• Daily reports• Algo discovery• Log analysis• Fraud pattern matchAge of Data
    • 41. Data Value ChainInteractive Real-time Analytics Record Lookup Historical Analytics Exploratory AnalyticsMilliseconds Hundredths of seconds Second(s) Minutes Hours• Place trade• Serve ad• Enrich stream• Examine packet• Approve trans.• Calculate risk• Leaderboard• Aggregate• Count• Retrieve clickstream• Show orders• Backtest algo• BI• Daily reports• Algo discovery• Log analysis• Fraud pattern matchValue of IndividualData ItemDataValueAggregateData ValueAge of Data
    • 42. Traditional RDBMSSimple SlowSmallFastComplexLargeApplicationComplexityValue of Individual Data Item Aggregate Data ValueDataValueThe Database UniverseInteractive Real-time Analytics Record Lookup Historical AnalyticsExploratoryAnalyticsTransactional Analytic
    • 43. Traditional RDBMSSimple SlowSmallFastComplexLargeApplicationComplexityValue of Individual Data Item Aggregate Data ValueDataValueDataWarehouseHadoop, etc.NoSQLThe Database UniverseInteractive Real-time Analytics Record Lookup Historical AnalyticsExploratoryAnalyticsTransactional AnalyticNewSQLVelocity
    • 44. The fastest, most scalable database onthe market todayVoltDBIngest massive quantities of data andperform automated decisioning in real time3 MILLION transactionsper secondDramatically lowering your cost pertransactionVoltDB enablesNOW.A huge impact on the bottom lineNOW
    • 45. PREVENTACHIEVEAnything is possible…
    • 46. Electrical smart grids
    • 47. Micro-personalization
    • 48. Real-time display targeting
    • 49. Dynamic airline ticket purchasing
    • 50. State-of-the-art social networking
    • 51. Session management
    • 52. Network monitoring
    • 53. We enableNOW.www.VoltDB.com
    • 54. HELLO 3.0!Ryan Betts
    • 55. Introducing VoltDB 3.0VoltDB 3.0VoltDB: a modern OLTP database built for a high velocity world.– Horizontal scalability– Hundreds of thousands of transactions per second– Relational SQL
    • 56. Latency and Throughput, 50-50 Read/Write WorkloadLatency and Throughput, 50-50 Read/Write Workload0246810121416-50000 0 50000 100000 150000 200000 250000 300000Latency(ms)TPS3.02.8.4.1VoltDB 3.0 vs. v2.8.4.1Key/Value 50/50 read/write workload3 Node, K=1 Cluster
    • 57. Read/Write Workload Latency/ThroughputRead/Write WorkloadLatency/Throughput0123456789-50000 0 50000 100000 150000 200000 250000 300000 350000Avg.Latency(ms)TPS10% read/90% write50% read/50% write90% read/10% writeVoltDB 3.0Key/Value various read/write workload3 Node, K=1 Cluster
    • 58. Faster: Ad Hoc SQL Performance• Conversational SQL• Thousands to 10,000+ ad hoc SQL transactions/second• Single or multiple (batch) SQL statement transactionFaster: Ad Hoc SQLPerformance
    • 59. Easier Development: New SQL Support• SQL LIKE and NOT LIKE• UNION• Column Functions• Counting function (leaderboard ranking queries)• Ability to define index using column functionsEasier Development:New SQL Support
    • 60. • JSON values stored in a varchar column• Field() column function• Indexing on JSON elementsCREATE INDEX session_site_moderatorON user_session_table (field(json_data, site),field(json_data, moderator), username);• New JSON sample in kitEasier Development:JSON SupportEasier Development: JSON Support
    • 61. Easier Development:Online OperationsEasier Development: Online Operations• Ability to re-join a failed node to cluster with no impact toexisting operations• Online schema update• No service window
    • 62. Easier Development: Streamlined Development• Elimination of project.xml• VoltDB-specific configuration now defined in DDL• Defaulting of deployment.xml• New Volt Compiler CLI:voltdb compileEasier Development:Streamlined Development
    • 63. Expanded Reach: Cloud-Friendly• Reduce impact of variable node performance and latency• Elimination of strict NTP configuration• Scales to large # of nodesExpanded Reach:Cloud-Friendly
    • 64. Integration: High-Performance Export• Parallelized export• New connectors: JDBC, Netezza, VerticaIntegration: High-Performance Export
    • 65. Integration: Client Library Updates• New PHP Client• Node.js client v1.0• Go Client• Coming soon: updated Erlang clientIntegration: ClientLibrary Updateshttp://golang.org
    • 66. Other Notable New Features• Explain command• CSV loader utility• CSV snapshots• New Administration CLI: voltadmin– voltadmin save– voltadmin restore– voltadmin pause– voltadmin resume– voltadmin shutdownOther NotableNew Features
    • 67. More Samples Availablefor DownloadMore Samples Available for Downloadhttp://voltdb.com/community/volt-labs.php
    • 68. Volt University• Portfolio of instructional content, classes, tools, and otherresources to help them built applications quickly• Curriculum and supporting material range from beginner toadvanced• Three types of instruction:– Volt University Online– Volt University Classroom– Volt Vanguard CertificationVolt University
    • 69. Summary: VoltDB v3.0• Run faster: transactions at high velocity scale.• Create faster: write and scale your ACID application.• Learn faster: Volt Labs & VoltDB UniversityVoltDB v3.0
    • 70. DOWNLOAD 3.0atwww.voltdb.comImagine thePossibilities
    • 71. More Information?E-mailinfo@voltdb.comVisit our forumshttp://community.voltdb.com/forumRead the VoltDB “Getting Started Guide”http://community.voltdb.com/docs/GettingStarted/indexFollow@VoltDB on TwitterMore Information?
    • 72. QUESTIONS?
    • 73. THANK YOU

    ×