0
The Sequence of Topics….
1

2
3
4
5

The Big Data Curve
The March of
Technology
Upheaval in the
Hardware Layer
Architectur...
1

The
Big Data
Curve
Moore’s Law and Its Consequences





Speed x10 every 6 years
Moore’s Law has about 10
years left (probably)
If Moore’...
The Visible “Big Data” Trend
Corporate data volumes
grow at about 55% per
annum - exponentially
 Data has been growing
at...
The Invisible Trend: Moore’s Law Cubed…
The biggest databases are new
databases
 They grow at the cube of Moore’s
Law
 M...
Moore’s Law’s Cubic Consequences
 Database

technology is
the most stressed
technology in the stack
 Scale-out architect...
2
Technology Evolution (Bloor Curve)
The Take Aways
Software architectures
change: centralized, C/S,
3 tier/web , SOA, etc.
 Applications migrate
according to...
Disruption on Disruption
We are no longer
certain that the pattern
still holds
 We used to encounter
new technologies tha...
Moore’s Law Does Somersault
 In 2004 chips got too hot
 That’s when the world
of parallel processing
suddenly emerged
 ...
Parallelism Will Become The Norm
 True parallelism involves
both data segmentation
and pipeline parallelism
 MapReduce i...
3
Upheaval
In the
Hardware
Layer
CPUs, GPUs and FPGA’s
 CPUs, GPUs and FPGAs
are commodities
 They can be harnessed
to deliver extreme
parallelism on a s...
The Network Latency
 In tests of DBMS
queries, Cisco found
about 90% of
latency was the
network
 Big network
switches vi...
The Memory Cascade
 On chip speed v RAM
 L1(32K) = 100x
 L2(246K) = 30x
 L3(8-20Mb) = 8.6x
 RAM v SSD
 RAM = 300x
 ...
In-Memory Disruption
 In-memory processing
will become the norm
 The latency matters
most for real-time
applications.
 ...
A Question
When will memory become the
primary source store for data?
Soon, probably.
Memory v SSD v Disk
It’s Over for Spinning Disk
 SSD is now on the
Moore’s Law curve.
 Disk is not and never was
(in respect of seek
time).
...
4

Architecture?
Tech Revolutions
Tech Revolution

Architecture

 Computer

 Batch

 On-line

 Centralized

 PC

 Client/server

 In...
Event Stream Processing or CEP
Event Driven/Big Data Architecture?
Some Architectural Principles
 The new atom of data
is the event
 SUSO, scale up before
scale out
 Take the processing ...
5

The
Flow
Of
Data
The Biological System
 Our human control system
works at different speeds:





Almost instant reflex
Swift response
C...
The Corporate Biological System
Right now this division
into different data flows
is already occurring
 Currently we can
...
In Summary…
1

2
3
4
5

The Big Data Curve
The March of
Technology
Upheaval in the
Hardware Layer
Architecture?
The Flow o...
Thank you
for your
attention
Technology Disruption
Technology Disruption
Upcoming SlideShare
Loading in...5
×

Technology Disruption

470

Published on

View Dr. Robin Bloor's presentation for the Dec. 2013 Big Data Conference in Rome.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
470
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
36
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Technology Disruption"

  1. 1. The Sequence of Topics…. 1 2 3 4 5 The Big Data Curve The March of Technology Upheaval in the Hardware Layer Architecture? The Flow of Data
  2. 2. 1 The Big Data Curve
  3. 3. Moore’s Law and Its Consequences     Speed x10 every 6 years Moore’s Law has about 10 years left (probably) If Moore’s Law stops there will be problems. Because of Moore’s Law, expensive technology is fairly affordable within 6 years and inexpensive within 12 years.
  4. 4. The Visible “Big Data” Trend Corporate data volumes grow at about 55% per annum - exponentially  Data has been growing at this rate for, maybe, 40 years  There is nothing new about big data. It clings to an established exponential trend 
  5. 5. The Invisible Trend: Moore’s Law Cubed… The biggest databases are new databases  They grow at the cube of Moore’s Law  Moore’s Law = 10x every 6 years  VLDB: 1000x every 6 years – 1991/2 megabytes – 1997/8 gigabytes – 2003/4 terabytes – 2009/10 petabytes – 2015/16 exabytes 
  6. 6. Moore’s Law’s Cubic Consequences  Database technology is the most stressed technology in the stack  Scale-out architecture has become a necessity  In-database analytics will become a necessity  In-memory database is the next iteration
  7. 7. 2
  8. 8. Technology Evolution (Bloor Curve)
  9. 9. The Take Aways Software architectures change: centralized, C/S, 3 tier/web , SOA, etc.  Applications migrate according to latencies  Dominant applications and software brands can die via “The innovator’s dilemma”  Wholly new applications appear because of lower latencies e.g. VMs, CEP. 
  10. 10. Disruption on Disruption We are no longer certain that the pattern still holds  We used to encounter new technologies that were 10x because of Moore’s Law  Now we encounter new technologies that are 100x or even 1000x  This is not because of Moore’s Law but because of parallelism 
  11. 11. Moore’s Law Does Somersault  In 2004 chips got too hot  That’s when the world of parallel processing suddenly emerged  Now CPUs miniaturize and add more cores  This changes software forever
  12. 12. Parallelism Will Become The Norm  True parallelism involves both data segmentation and pipeline parallelism  MapReduce is a halfway house.  This is about all software. Eventually everything will execute in parallel  Everything goes much faster
  13. 13. 3 Upheaval In the Hardware Layer
  14. 14. CPUs, GPUs and FPGA’s  CPUs, GPUs and FPGAs are commodities  They can be harnessed to deliver extreme parallelism on a single server  The use of such chips can deliver acceleration above 100x for some applications
  15. 15. The Network Latency  In tests of DBMS queries, Cisco found about 90% of latency was the network  Big network switches virtualize networks.  The network can no longer be ignored
  16. 16. The Memory Cascade  On chip speed v RAM  L1(32K) = 100x  L2(246K) = 30x  L3(8-20Mb) = 8.6x  RAM v SSD  RAM = 300x  SSD v Disk  SSD = 10x
  17. 17. In-Memory Disruption  In-memory processing will become the norm  The latency matters most for real-time applications.  However some businesses are using it for analytics  As such memory is an accelerator
  18. 18. A Question When will memory become the primary source store for data? Soon, probably.
  19. 19. Memory v SSD v Disk
  20. 20. It’s Over for Spinning Disk  SSD is now on the Moore’s Law curve.  Disk is not and never was (in respect of seek time).  All traditional databases were engineered for spinning disk and not for scale-out  This explains the new DBMS products…
  21. 21. 4 Architecture?
  22. 22. Tech Revolutions Tech Revolution Architecture  Computer  Batch  On-line  Centralized  PC  Client/server  Internet  Multi-tier  Mobile  Service  Internet of things Orientation  Event Driven/Big Data
  23. 23. Event Stream Processing or CEP
  24. 24. Event Driven/Big Data Architecture?
  25. 25. Some Architectural Principles  The new atom of data is the event  SUSO, scale up before scale out  Take the processing to the data, if you can  Hadoop is a component not a solution
  26. 26. 5 The Flow Of Data
  27. 27. The Biological System  Our human control system works at different speeds:    Almost instant reflex Swift response Considered response Organizations will gradually implement similar control systems  This suggests a data-flowbased architecture 
  28. 28. The Corporate Biological System Right now this division into different data flows is already occurring  Currently we can distinguish between:  Real-time/Business time applications  Analytical applications We should build specific architectures for this  
  29. 29. In Summary… 1 2 3 4 5 The Big Data Curve The March of Technology Upheaval in the Hardware Layer Architecture? The Flow of Data
  30. 30. Thank you for your attention
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×