Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1j8YoCp. Todd Montgomery challenges some of the common myths and misconceptions about high performance streaming data, and takes a look at what is really possible today. Filmed at qconnewyork.com. Todd Montgomery is a networking hacker who has researched, designed, and built numerous protocols, messaging-oriented middleware systems, and real-time data systems, done research for NASA, contributed to the IETF and IEEE, and cofounded two startups. He currently works for Kaazing as Chief Architect. @toddlmontgomery

1. Modern Fast
Streaming Data
Todd L. Montgomery
@toddlmontgomery

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/myths-data-streaming

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. Why Should We Care?
Myths & Misconceptions
You can’t escape the Math
Technologies & Techniques

5. Why Should We Care?

6. Human Knowledge is now
doubling every year*
* by discipline, 12-18 months

7. Fueled by Technology

8. Middle Ages - 1500 yrs
Renaissance - 250 yrs
Industrial Revolution - 150 yrs
WWII - 25 years
Buckminster Fuller - Critical Path 1981

9. IoT

10. IoT
Ubiquitous Computing

11. In the near future,
Human Knowledge could
double every 72 hours

12. What this could mean for
our systems…

14. Updates/Sec
=
Devices * Frequency * Market Share
Either ingest or streaming.
2x for Request/Response

15. Updates/Sec
=
Devices * Frequency * Market Share
9 Billion (Today)
50 Billion by 2020 (Cisco)
26 Billion by 2020 (Smartphone/Tablet - Gartner)
75 Billion by 2020 (Morgan Stanley)

16. Updates/Sec
=
50 Billion * 6/min * 1%
=
50 Million/sec

17. Bandwidth
=
50 Billion * 6/min * 1% * 200 bytes
=
9.3 GB/s (74.5 Gb/s)

18. And…
Geographic Distribution
30%
15%
10%
15%
20%
10%

20. Social & Societal demands
will require
processing an intense stream
of data in real-time

21. Myths & Misconceptions

22. Excuses, Excuses!

23. Myth
(CPUs, Storage, Networks) are not
capable of processing in real-time*
* for some unknown, unquantified data volume

24. Accumulated
Improvement
Time
Network
Bandwidth
Response Time
Storage
Capacity
CPU Cores
Memory
Capacity

25. http://en.wikipedia.org/wiki/Instructions_per_second
Year Processor MIPS
1974 Intel 8080 0.29
1982 Intel 286 1.28
1993 PowerPC 601 157
2003 Pentium 4 Extreme 9,726
2008
Intel Core i7 920
(Quad)
82,300
2011
Intel Core i7 2600K
(4/8) Sandy Bridge
128,300
2014
Intel Core i7 5960x
(8/16) Haswell
298,190

26. Raspberry Pi 2
(Quad)
1,186 MIPS!
http://en.wikipedia.org/wiki/Instructions_per_second

27. DDRSSD
PCIe - 3
100 GbE
…
OmniPath

28. 1 thread of awesome
>
128 cores of so-so
http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html
http://blog.acolyer.org/2015/06/05/scalability-but-at-what-cost/

29. Misconception
Data needs to come to rest to
be processed

30. Data at rest is a liability

31. Data at rest is a liability
MDMILM
WarehouseETL

32. Why would you want transient
data at rest?

33. But what about sort?
…REALLY?!

34. You can’t escape the Math

35. … and you don’t want to
The Math will guide you to a solution

36. "AmdahlsLaw" by Daniels220 at English Wikipedia - Own work based on: File:AmdahlsLaw.png. Licensed under CC BY-SA 3.0 via Wikimedia Commons

37. Setup & Scheduling
Work Unit Work UnitWork UnitWork Unit
Post Processing

38. Setup & Scheduling
Work Unit Work UnitWork UnitWork Unit
Post Processing
Contention
Contention

39. Contention isn’t the biggest enemy

40. Coherence is!

41. Universal Scalability Law
0
2
4
6
8
10
12
14
16
18
20
1 2 4 8 16 32 64 128 256 512 1024
Speedup
Processors
Amdahl USL

42. Setup & Scheduling
Work Unit Work UnitWork UnitWork Unit
Post Processing
Contention
Contention
Contention +
Coherence
Contention +
Coherence

43. Up Front Partitioning
Work Unit Work UnitWork UnitWork Unit

44. … and more
Queuing Theory ⭐
Complexity Theory
CAP Theorem

45. Technologies & Techniques

46. The Essence of Architecture
Data Structures
Protocols of Interaction
Mechanical Sympathy

47. Understanding is essential

48. Accumulated
Improvement
Time
Network
Bandwidth
Response Time
Storage
Capacity
CPU Cores
Memory
Capacity

49. Accumulated
Improvement
Time
Network
Bandwidth
Response Time
Storage
Capacity
CPU Cores
Memory
Capacity
Batching…

50. Technique
Smart Batching (Natural Batching)
http://mechanical-sympathy.blogspot.com/2011/10/smart-batching.html

51. Resource

52. Resource
Ring Buffer

53. Batching Thread
Resource
Pull off as much waiting
data as possible

54. Single Writer Principle
Avoid Resource Contention
Batching only when needed
Rate Decoupling
Back Pressure

55. Techniques
Freedom!
Lock-Free, Wait-Free
http://en.wikipedia.org/wiki/Non-blocking_algorithm

57. Words Matter

58. Obstruction-Freedom
Partially completed operations
aborted & changes made rolled back

59. Lock-Freedom
Individual thread may starve, but
guaranteed system-wide throughput
Lock-Free is Obstruction-Free

60. Wait-Freedom
Starvation free and guaranteed
system-wide throughput
Wait-Free is Lock-Free

61. These properties are
awesome!
Who wouldn’t want them?

62. System-wide properties start
at the lowest level

63. Essence
Just because we could take an
action right now, doesn’t mean
we should

65. Technology
CRDTs
http://en.wikipedia.org/wiki/Conflict-free_replicated_data_type

66. 0
0
0
0
Node
0
1
2
N
Value
sum(0,N) = 0
…

67. 0
1
0
0
Node
0
1
2
N
Value
sum(0,N) = 1
…

68. 1
1
0
0
Node
0
1
2
N
Value
sum(0,N) = 2
…

69. Gossip for visibility

70. [2] = 0 [1] = 2 [0] = 4[N] = 0
4 2 0 0…
4200
Shared View

71. Technology
Append-only Data Structures
https://github.com/real-logic/Aeron

72. Header
Message
Log

73. Header
Message
Header
Message
Header
Message
Log

74. Efficiently
Replicating an Append-only Log

75. What If…?
The Data Structure could be
directly sent to the “network”?

76. Header
Message

77. Header
Message
Position in Log
Length

78. Header
Message
Position in Log
Length
Version/Flags
Type
etc.
+

79. Header
Message
Fragment 0

80. Header
Message
Header
Message
Fragment 0

81. Header
Message
Header
Message
Header
Message
Header
Message
Fragment 0

82. Header
Message
Header
Message
Header
Message
Header
Message
Fragment 0
Fragment 1

83. Header
Message
Header
Message
Header
Message
Header
Message
Header
Message
Header
Message
Fragment 0
Fragment 1

84. Natural for broadcast replication

85. In Closing…

86. A flood of data is coming,
Many say it is already here,
How will you deal with it?

87. Ever seen a PitBull drink from a
Fire Hydrant?

89. It won’t give up…
Be the Pitbull at the Fire Hydrant!

90. @toddlmontgomery
Questions?
• Aeron https://github.com/real-logic/Aeron
• SlideShare http://www.slideshare.com/toddleemontgomery
• Twitter @toddlmontgomery
Thank You!

91. Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/myths-
data-streaming