2. Learn
Why we need fancy Big Data frameworks.
How the lambda architecture looks like.
How twitter used to do real-time analytics.
Why twitter created Storm.
How Storm works.
2
6. New Tools
Large scale computation systems such as Hadoop.
Scalable databases such as Casandra and Riak.
Easy to use frameworks such as Storm and Dempsy.
6
10. Batch Layer
Speed Layer
Serving Layer
Stores the immutable master dataset.
Precomputes arbitrary batch views.
Home of batch processing and map
reduce systems such as Hadoop.
10
Batch Layer
11. Serving Layer
Speed Layer
Serving Layer
Batch Layer
Read-only random-access to batch views.
Updated by batch layer.
Indexes batch views.
Home of real-time query systems
such as Cloudera Impala for Hadoop.
11
12. Speed Layer
Speed Layer
Serving Layer
Batch Layer
Compensates for high-latency batch views.
Fast, incremental algorithms.
More complex because of random-writes.
Home of Apache HBase or Storm.
12
20. Know
Why we need fancy Big Data frameworks.
How the lambda architecture looks like.
How twitter used to do real-time analytics.
Why twitter created Storm.
How Storm works.
20