This document discusses the increasing dominance of memory-oriented solutions for high-performance data access. It notes that database lookups are around 20 milliseconds while hashmap lookups are around 20 nanoseconds. It then discusses how abstraction improves software but hurts performance. It outlines the traditional database architecture with disk storage and compares it to newer in-memory and distributed in-memory architectures that can provide faster performance by avoiding disk I/O and leveraging memory and distribution.

1. A Paradigm Shift: The
Increasing Dominance of
Memory-Oriented Solutions for
High Performance Data Access!
Ben Stopford : RBS

2. How fast is a HashMap lookup?
~20 ns

3. That’s how long it takes light to
travel a room

4. How fast is a database lookup?
~20 ms

5. That’s how long it takes light to go
to Australia and back

6. 3 times

7. Computers really are very fast!

8. The problem is we’re quite good at
writing software that slows them down

9. Desktop Virtualization

10. We love
abstraction

11. There are many reasons
why abstraction is a
good idea…
…performance just isn’t
one of them

12. Question: is it fair to compare a
Database with a HashMap?

13. Not really…

14. Key Point
On one end of
..on the other sits
the scale sits the
the database…
HashMap…
…but it’s a very very long scale that
sits between them.

15. Times are changing

16. Database Architecture is
Aging

18. The Traditional Architecture

19. Traditional
Shared Shared
In Memory
Disk
Nothing
Distributed Simpler
In Memory
Contract

20. Simplifying the
Contract

21. How big is the internet?
5 exabytes
(which is 5,000 petabytes or
5,000,000 terabytes)

22. How big is an average enterprise
database
80% < 1TB
(in 2009)

23. Simplifying the Contract

24. Databases have huge operational
overheads
Taken from “OLTP Through
the Looking Glass, and What
We Found There”
Harizopoulos et al

25. Avoid that overhead with a simpler
contract and avoiding IO

26. Improving Database Performance !
Shared Disk Architecture
Shared
Disk

27. Improving Database Performance !
Shared Nothing Architecture

28. Each machine is responsible for a subset of the
records. Each record exists on only one
machine.!
1, 2, 3…
97, 98, 99…
765, 769…
169, 170…
Client
333, 334…
244, 245…

29. Improving Database Performance (3)!
In Memory Databases!
(single address-space)

30. Databases must cache subsets of
the data in memory
Cache

31. Not knowing what you don’t know
90% in Cache
Data on Disk

32. If you can fit it ALL in memory you
know everything!!

33. The architecture of an in memory
database

34. Memory is at least 100x faster than disk
ms
μs
ns
ps
1MB Disk/Network
1MB Main Memory
0.000,000,000,000
Cross Continental Main Memory
L1 Cache Ref
Round Trip
Ref
Cross Network L2 Cache Ref
Round Trip
* L1 ref is about 2 clock cycles or 0.7ns. This is
the time it takes light to travel 20cm

35. Memory allows random access.
Disk only works well for sequential
reads

36. This makes them very fast!!

37. The proof is in the stats. TPC-H
Benchmarks on a 1TB data set

38. So why haven’t in memory
databases taken off?

39. Address-Spaces are relatively small
and of a finite, fixed size

40. Durability

41. One solution is
distribution

42. Distributed In Memory (Shared
Nothing)

43. Again we spread our data but this time only
using RAM.
1, 2, 3…
97, 98, 99…
765, 769…
169, 170…
Client
333, 334…
244, 245…

44. Distribution solves our two
problems

45. We get massive amounts of parallel
processing

46. But at the cost of
loosing the single
address space

47. Traditional
Shared Shared
In Memory
Disk
Nothing
Distributed Simpler
In Memory
Contract

48. There are three key themes here:
Simplify the
Distribution
No Disk
contract
Improve
Gain
scalability by
scalability
picking All data is
through a
appropriate held in RAM
distributed
ACID
architecture
properties.

49. ODC

50. ODC – Distributed, Shared Nothing, In
Memory, Semi-Normalised, Graph DB
450 processes
2TB of RAM
Messaging (Topic Based) as a system of record
(persistence)

51. ODC represents a
balance between
throughput and
latency

52. What is Latency?

53. What is Throughput

54. Which is best for latency?
Shared
Nothing
(Distributed)
Traditional In-Memory
Database
Database
Latency?

55. Which is best for throughput?
Shared
Nothing
(Distributed)
Traditional In-Memory
Database
Database
Latency?

56. So why do we use distributed in
memory?
Plentiful
In Memory
hardware
Latency
Throughput

57. This is the technology of
the now.
So what is the technology
of the future?

58. Terabyte Memory Architectures

59. Fast Persistent Storage

60. New Innovations on the Horizon

61. These factors are remolding the
hardware landscape to one where
memory both vast and durable

62. This is changing the way we write
software

63. Huge servers in the
commodity space are
driving us towards single
process architectures that
utilise many cores and
large address spaces

64. We can attain hundreds of
thousands of executions
per second from a single
process if it is well
optimised.

65. “All computers wait at the
same speed”
!

66. We need to optimise for our CPU architecture
ms
μs
ns
ps
1MB Disk/Network
1MB Main Memory
0.000,000,000,000
Cross Continental Main Memory
L1 Cache Ref
Round Trip
Ref
Cross Network L2 Cache Ref
Round Trip
* L1 ref is about 2 clock cycles or 0.7ns. This is
the time it takes light to travel 20cm

67. Tools like Vtune allow us to
optimise software to truly leverage
our hardware

68. So what does this all mean?

69. Further Reading