This deck was presented at the Irish Hadoop Users Group on 7th November 2016. Note that it's very much designed to be used as a background to a spoken presentation, so the speaker notes are a critical part of it. Summary It used to be we didn’t have much of a choice, and that you used whatever mature RDBMS your ‘tribe’ was comfortable with, be it Oracle, DB2 or SQL*Server. Over that last decade that’s changed completely and you now have about 100 database-like technologies to choose from. Pretty much every possible architectural approach is now available to you, but how do you select the right one? We're moving from an era where information is human generated and moves at human speeds to one where sources of data like IoT and M2M will simply swamp legacy technologies, both with increased volumes of data and much lower timescales in which you can extract value from it. It used to be that selecting software involved checking boxes in feature matrices. But if speed is such a fundamental requirement we're going to have to look at things differently, as every extra features 'adds slowness'. As a consequence there is no one 'silver bullet' solution that can be deployed everywhere - technologies such as Kafka, Apache Spark, Storm and even VoltDB are all very good in specific scenarios but can not replace a legacy DB. In this presentation David will explain how you go about categorizing and understanding the new database and persistence technologies that exist.

1. page
WHEN MORE IS LESS: EVALUATING NEW
DATABASE TECHNOLOGY IN 2016.
1

2. page© 2016 VoltDB PROPRIETARY page 2
• Tube map

3. page© 2016 VoltDB PROPRIETARY 3

4. page© 2016 VoltDB PROPRIETARY 4

5. page© 2016 VoltDB PROPRIETARY 5

6. page© 2016 VoltDB PROPRIETARY 6

7. page© 2016 VoltDB PROPRIETARY 7
http://i2.cdn.turner.com/money/2011/08/17/t
echnology/military_energy/fuel-
convoy.gi.top.jpg

8. page© 2016 VoltDB PROPRIETARY
TRY TO DO EVERYTHING AND YOU’LL DO NOTHING WELL…
8
• The more ‘stuff’ you add the less likely it is that you can do
everything properly.
• Over time legacy products ended up full of features that you
pay for but don’t need.
• To add insult to injury the vendor will charge you more
because it has more features

9. page© 2016 VoltDB PROPRIETARY
A ”FEATURE MATRIX” FOR EVALUATING HADOOP
Feature “BloatCorp* Distributed FS” HDFS
Hardware Acceleration  
Support for updates  
Simultaneous updates  
Support for small block sizes  
Replication  
9
Conclusion: Hadoop is a great example of how Feature Matrices are no longer relevant
and can be highly misleading when it comes to choosing technology
*“BloatCorp” is fictional

10. page© 2016 VoltDB PROPRIETARY
WHY NOSQL/NEWSQL HAPPENED…
10
• Existing products struggled with emerging needs:
• The Cloud / Virtualization
• Commodity Hardware
• New Use Cases
• Cloud Economics
• Distribution of data between multiple locations as standard
• High Availability as standard
• JSON, XML, CLOB, BLOB, Log files, >10KTPS OLTP……..
• Critically: A new generation of developers who couldn’t
remember life before the RDBMS were unafraid to live
without one.

11. page© 2016 VoltDB PROPRIETARY page 11
• Tube map

12. page© 2016 VoltDB PROPRIETARY
THE 2017 DILEMMA...
12
• Every possible architecture is now available.
• Everyone is claiming 10 to 100x faster.
• We’ve gone from a chronic shortage of choice to a
glut….
• How do you pick the right database?

13. page© 2016 VoltDB PROPRIETARY
COLIN CHAPMAN, THE MAN BEHIND LOTUS CARS
13
“Simplify, then add lightness”
“Adding power makes you faster on the
straights; subtracting weight makes you faster
everywhere”
When it comes to databases “speed” is a
consequence of architectural choices, not
clever programming….

14. page© 2016 VoltDB PROPRIETARY 14

15. page© 2016 VoltDB PROPRIETARY 15

16. page© 2016 VoltDB PROPRIETARY 16

17. page© 2016 VoltDB PROPRIETARY 17

18. page© 2016 VoltDB PROPRIETARY
THE “FEATURE MATRIX” IS DEAD, BUT..
• It’s easier to be 100x faster when you do virtually
nothing…
• Unneeded features == slowness, cost, complexity
• Missing features == problem doesn’t get solved
• Instead of one product that does everything you
need a ‘Zoo’ of technologies.
18

19. page© 2016 VoltDB PROPRIETARY
WHY THE “ZOO” ANALOGY?
• Nobody will pay to go to an ”All giraffe zoo”
• “Zoo” implies a balanced, rational collection..
• Nobody thinks Zoo animals look after themselves
• Open source stops being free when you start using it..
• A Zoo is not an animal shelter
• Not in the business of offering homes to stray
technologies found on the internet…
19

20. page© 2016 VoltDB PROPRIETARY
DATABASE TECHNOLOGY SELECTION CHECKLIST
1. Do we understand all the requirements?
• We mean next year’s, not the next agile
iteration.
• A database is for life, not just for Christmas!
• Need to distinguish between ‘need’ and ‘want’.
• ‘want’ leads to feature bloat…
20

21. page© 2016 VoltDB PROPRIETARY
DATABASE TECHNOLOGY SELECTION CHECKLIST
2. How will this fit into our Ecosystem?
• “Ecosystem” includes people, skills, culture...
• Depending on your business customer’s
opinions may matter too…
• Try and avoid overlap – having an Indian and
an African elephant wastes resources.
• Overlap also leads to office politics
21

22. page© 2016 VoltDB PROPRIETARY
DATABASE TECHNOLOGY SELECTION CHECKLIST
3. What is the TCO?
• Making decisions without regard to economics
means you’ll end up trying to use a Rolls Royce
as an Ice Cream van.
• TCO includes zookeepers, cages, vets. Or trained
staff, hardware and support contracts.
22

23. page© 2016 VoltDB PROPRIETARY
DATABASE TECHNOLOGY SELECTION CHECKLIST
4. Will your chosen DB exist five years from now?
• The ”Tube Map” is unsustainable in its current form.
• An industry shakeout is unavoidable.
• Deciding to support a dying Open Source platform has
serious TCO implications.
• Cynical view: Platform survivability is the issue you can’t
get wrong.
23

24. page© 2016 VoltDB PROPRIETARY
CONCLUSION
Know what you need.
• Do your own research.
• Be wary of ‘extra’ features/bloat.
• Ask “Is this the first DB these people have written?”
• Find out why architectural choices were made. Do the
reasons make sense to you?
• Understand why it’s fast – something was left out….
• Think 5 years ahead. Make sure your vendor does.
24

25. page© 2016 VoltDB PROPRIETARY
WHO IS VOLTDB?
25

26. page© 2016 VoltDB PROPRIETARY page
DAVID ROLFE
DIRECTOR OF SOLUTIONS ENGINEERING, EMEA
DROLFE@VOLTDB.COM
@SR_MAD_SCIENCE
WWW.VOLTDB.COM
26

27. page© 2016 VoltDB PROPRIETARY
THE VOLTDB TECHNOLOGY ORIGIN

28. page© 2016 VoltDB PROPRIETARY
BIG DATA
Non
Relational
Processing
BI
Reporting
Fast
Operational
Database
FAST DATA
Export
Ingestio
n Engine
Request
&
Response
Applicatio
n
Streaming
Analytics
Data
Warehous
e Columnar
Analytics
OLAP
DATA ARCHITECTURE FOR FAST + BIG DATA
Data Lake
(HDFS)
OUTSIDE
WORLD
In Memory
Data
Real Time
Data
Sources
End user or
Device