page
WHEN MORE IS LESS: EVALUATING NEW
DATABASE TECHNOLOGY IN 2016.
1
page© 2016 VoltDB PROPRIETARY page 2
• Tube map
page© 2016 VoltDB PROPRIETARY 3
page© 2016 VoltDB PROPRIETARY 4
page© 2016 VoltDB PROPRIETARY 5
page© 2016 VoltDB PROPRIETARY 6
page© 2016 VoltDB PROPRIETARY 7
http://i2.cdn.turner.com/money/2011/08/17/t
echnology/military_energy/fuel-
convoy.gi.top....
page© 2016 VoltDB PROPRIETARY
TRY TO DO EVERYTHING AND YOU’LL DO NOTHING WELL…
8
• The more ‘stuff’ you add the less likel...
page© 2016 VoltDB PROPRIETARY
A ”FEATURE MATRIX” FOR EVALUATING HADOOP
Feature “BloatCorp* Distributed FS” HDFS
Hardware A...
page© 2016 VoltDB PROPRIETARY
WHY NOSQL/NEWSQL HAPPENED…
10
• Existing products struggled with emerging needs:
• The Cloud...
page© 2016 VoltDB PROPRIETARY page 11
• Tube map
page© 2016 VoltDB PROPRIETARY
THE 2017 DILEMMA...
12
• Every possible architecture is now available.
• Everyone is claimin...
page© 2016 VoltDB PROPRIETARY
COLIN CHAPMAN, THE MAN BEHIND LOTUS CARS
13
“Simplify, then add lightness”
“Adding power mak...
page© 2016 VoltDB PROPRIETARY 14
page© 2016 VoltDB PROPRIETARY 15
page© 2016 VoltDB PROPRIETARY 16
page© 2016 VoltDB PROPRIETARY 17
page© 2016 VoltDB PROPRIETARY
THE “FEATURE MATRIX” IS DEAD, BUT..
• It’s easier to be 100x faster when you do virtually
no...
page© 2016 VoltDB PROPRIETARY
WHY THE “ZOO” ANALOGY?
• Nobody will pay to go to an ”All giraffe zoo”
• “Zoo” implies a bal...
page© 2016 VoltDB PROPRIETARY
DATABASE TECHNOLOGY SELECTION CHECKLIST
1. Do we understand all the requirements?
• We mean ...
page© 2016 VoltDB PROPRIETARY
DATABASE TECHNOLOGY SELECTION CHECKLIST
2. How will this fit into our Ecosystem?
• “Ecosyste...
page© 2016 VoltDB PROPRIETARY
DATABASE TECHNOLOGY SELECTION CHECKLIST
3. What is the TCO?
• Making decisions without regar...
page© 2016 VoltDB PROPRIETARY
DATABASE TECHNOLOGY SELECTION CHECKLIST
4. Will your chosen DB exist five years from now?
• ...
page© 2016 VoltDB PROPRIETARY
CONCLUSION
Know what you need.
• Do your own research.
• Be wary of ‘extra’ features/bloat.
...
page© 2016 VoltDB PROPRIETARY
WHO IS VOLTDB?
25
page© 2016 VoltDB PROPRIETARY page
DAVID ROLFE
DIRECTOR OF SOLUTIONS ENGINEERING, EMEA
DROLFE@VOLTDB.COM
@SR_MAD_SCIENCE
W...
page© 2016 VoltDB PROPRIETARY
THE VOLTDB TECHNOLOGY ORIGIN
page© 2016 VoltDB PROPRIETARY
BIG DATA
Non
Relational
Processing
BI
Reporting
Fast
Operational
Database
FAST DATA
Export
I...
Upcoming SlideShare
Loading in …5
×

When more is less: Evaluating new database technology in 2016, David Rolfe, VoltDB

83 views

Published on

This is the supporting slide deck for a presentation that was done at the Irish Hadoop Users Group on the 7th November 2016.

If you want further information please use the contact details on slide 26.

Summary

It used to be we didn’t have much of a choice, and that you used whatever mature RDBMS your ‘tribe’ was comfortable with, be it Oracle, DB2 or SQL*Server. Over that last decade that’s changed completely and you now have about 100 database-like technologies to choose from. Pretty much every possible architectural approach is now available to you, but how do you select the right one?
We're moving from an era where information is human generated and moves at human speeds to one where sources of data like IoT and M2M will simply swamp legacy technologies, both with increased volumes of data and much lower timescales in which you can extract value from it. It used to be that selecting software involved checking boxes in feature matrices. But if speed is such a fundamental requirement we're going to have to look at things differently, as every extra features 'adds slowness'. As a consequence there is no one 'silver bullet' solution that can be deployed everywhere - technologies such as Kafka, Apache Spark, Storm and even VoltDB are all very good in specific scenarios but can not replace a legacy DB.
In this presentation David will explain how you go about categorizing and understanding the new database and persistence technologies that exist.

Published in: Software
  • Be the first to comment

  • Be the first to like this

When more is less: Evaluating new database technology in 2016, David Rolfe, VoltDB

  1. 1. page WHEN MORE IS LESS: EVALUATING NEW DATABASE TECHNOLOGY IN 2016. 1
  2. 2. page© 2016 VoltDB PROPRIETARY page 2 • Tube map
  3. 3. page© 2016 VoltDB PROPRIETARY 3
  4. 4. page© 2016 VoltDB PROPRIETARY 4
  5. 5. page© 2016 VoltDB PROPRIETARY 5
  6. 6. page© 2016 VoltDB PROPRIETARY 6
  7. 7. page© 2016 VoltDB PROPRIETARY 7 http://i2.cdn.turner.com/money/2011/08/17/t echnology/military_energy/fuel- convoy.gi.top.jpg
  8. 8. page© 2016 VoltDB PROPRIETARY TRY TO DO EVERYTHING AND YOU’LL DO NOTHING WELL… 8 • The more ‘stuff’ you add the less likely it is that you can do everything properly. • Over time legacy products ended up full of features that you pay for but don’t need. • To add insult to injury the vendor will charge you more because it has more features
  9. 9. page© 2016 VoltDB PROPRIETARY A ”FEATURE MATRIX” FOR EVALUATING HADOOP Feature “BloatCorp* Distributed FS” HDFS Hardware Acceleration   Support for updates   Simultaneous updates   Support for small block sizes   Replication   9 Conclusion: Hadoop is a great example of how Feature Matrices are no longer relevant and can be highly misleading when it comes to choosing technology *“BloatCorp” is fictional
  10. 10. page© 2016 VoltDB PROPRIETARY WHY NOSQL/NEWSQL HAPPENED… 10 • Existing products struggled with emerging needs: • The Cloud / Virtualization • Commodity Hardware • New Use Cases • Cloud Economics • Distribution of data between multiple locations as standard • High Availability as standard • JSON, XML, CLOB, BLOB, Log files, >10KTPS OLTP…….. • Critically: A new generation of developers who couldn’t remember life before the RDBMS were unafraid to live without one.
  11. 11. page© 2016 VoltDB PROPRIETARY page 11 • Tube map
  12. 12. page© 2016 VoltDB PROPRIETARY THE 2017 DILEMMA... 12 • Every possible architecture is now available. • Everyone is claiming 10 to 100x faster. • We’ve gone from a chronic shortage of choice to a glut…. • How do you pick the right database?
  13. 13. page© 2016 VoltDB PROPRIETARY COLIN CHAPMAN, THE MAN BEHIND LOTUS CARS 13 “Simplify, then add lightness” “Adding power makes you faster on the straights; subtracting weight makes you faster everywhere” When it comes to databases “speed” is a consequence of architectural choices, not clever programming….
  14. 14. page© 2016 VoltDB PROPRIETARY 14
  15. 15. page© 2016 VoltDB PROPRIETARY 15
  16. 16. page© 2016 VoltDB PROPRIETARY 16
  17. 17. page© 2016 VoltDB PROPRIETARY 17
  18. 18. page© 2016 VoltDB PROPRIETARY THE “FEATURE MATRIX” IS DEAD, BUT.. • It’s easier to be 100x faster when you do virtually nothing… • Unneeded features == slowness, cost, complexity • Missing features == problem doesn’t get solved • Instead of one product that does everything you need a ‘Zoo’ of technologies. 18
  19. 19. page© 2016 VoltDB PROPRIETARY WHY THE “ZOO” ANALOGY? • Nobody will pay to go to an ”All giraffe zoo” • “Zoo” implies a balanced, rational collection.. • Nobody thinks Zoo animals look after themselves • Open source stops being free when you start using it.. • A Zoo is not an animal shelter • Not in the business of offering homes to stray technologies found on the internet… 19
  20. 20. page© 2016 VoltDB PROPRIETARY DATABASE TECHNOLOGY SELECTION CHECKLIST 1. Do we understand all the requirements? • We mean next year’s, not the next agile iteration. • A database is for life, not just for Christmas! • Need to distinguish between ‘need’ and ‘want’. • ‘want’ leads to feature bloat… 20
  21. 21. page© 2016 VoltDB PROPRIETARY DATABASE TECHNOLOGY SELECTION CHECKLIST 2. How will this fit into our Ecosystem? • “Ecosystem” includes people, skills, culture... • Depending on your business customer’s opinions may matter too… • Try and avoid overlap – having an Indian and an African elephant wastes resources. • Overlap also leads to office politics 21
  22. 22. page© 2016 VoltDB PROPRIETARY DATABASE TECHNOLOGY SELECTION CHECKLIST 3. What is the TCO? • Making decisions without regard to economics means you’ll end up trying to use a Rolls Royce as an Ice Cream van. • TCO includes zookeepers, cages, vets. Or trained staff, hardware and support contracts. 22
  23. 23. page© 2016 VoltDB PROPRIETARY DATABASE TECHNOLOGY SELECTION CHECKLIST 4. Will your chosen DB exist five years from now? • The ”Tube Map” is unsustainable in its current form. • An industry shakeout is unavoidable. • Deciding to support a dying Open Source platform has serious TCO implications. • Cynical view: Platform survivability is the issue you can’t get wrong. 23
  24. 24. page© 2016 VoltDB PROPRIETARY CONCLUSION Know what you need. • Do your own research. • Be wary of ‘extra’ features/bloat. • Ask “Is this the first DB these people have written?” • Find out why architectural choices were made. Do the reasons make sense to you? • Understand why it’s fast – something was left out…. • Think 5 years ahead. Make sure your vendor does. 24
  25. 25. page© 2016 VoltDB PROPRIETARY WHO IS VOLTDB? 25
  26. 26. page© 2016 VoltDB PROPRIETARY page DAVID ROLFE DIRECTOR OF SOLUTIONS ENGINEERING, EMEA DROLFE@VOLTDB.COM @SR_MAD_SCIENCE WWW.VOLTDB.COM 26
  27. 27. page© 2016 VoltDB PROPRIETARY THE VOLTDB TECHNOLOGY ORIGIN
  28. 28. page© 2016 VoltDB PROPRIETARY BIG DATA Non Relational Processing BI Reporting Fast Operational Database FAST DATA Export Ingestio n Engine Request & Response Applicatio n Streaming Analytics Data Warehous e Columnar Analytics OLAP DATA ARCHITECTURE FOR FAST + BIG DATA Data Lake (HDFS) OUTSIDE WORLD In Memory Data Real Time Data Sources End user or Device

×