This document discusses big data and how it can be analyzed. It defines big data as data that is too large, complex, and dynamic for conventional tools to handle due to its volume, velocity, and variety. It then lists some examples of the huge amounts of data created every day and discusses how organizations have benefited from improved risk management, increased sales, better management control, and other gains through big data analysis. The document also outlines some common Hadoop tools used for working with big data, like HDFS, MapReduce, Hive, HBase and Zookeeper. It notes that big data solutions can be implemented using vendors like Cloudera or Hortonworks or Amazon services and asks if results can be obtained even faster using tools

1. Mariusz Gil
BIG
data
ecosystem

2. / ABOUT ME /

3. This talk is about
BIG DATA

4. What is...
BIG DATA?

6. VOLUME
large amounts of data

7. VELOCITY
needs to be analyzed quickly

8. VARIETY
different types of structured and unstructured data

9. Big Data is data that is too large,
complex and dynamics for any conventional data tools
to capture, store, manage and analyze.

10. 30 billion pieces of content we added past month

11. more than 2 billion videos were watched yesterday

12. more than 58 millions messages were send yesterday

13. / MAIN QUESTIONS /

14. WHY?

15. 49
%
IMPROVED RISK
MANAGEMENT
32
%
INCREASED
SALES FIGURES
36 40
%
IMPROVED
MANAGEMENT
CONTROL
%
IT ANALYSIS
43
%
MARKET-ORIENTED
PRODUCT DEVELOPMENT
27
%
FINANCES AND
ECONOMICS

17. 690 nodes Hadoop cluster for predictions and analytics

18. HOW?

20. HDFS
YARN / MapReduce v2
HADOOP DISTRIBUTED FILE SYSTEM
DISTRIBUTED PROCESSING FRAMEWORK
COLUMNAR STORAGE
SQL DATA WAREHOUSE ENGINE
HIVE
DATA SERIALIZATION
AVRO
SCALABLE MACHINE LEARNING
MAHOUT
SCRIPTING FOR LARGE DATA SETS
PIG
WORKFLOWS ORCHESTRATION
PROVISIONING, MANAGING AND MONITORING CLUSTERS
HBASE
DATA EXCHANGE
SQOOP
OOZIE
DISTRIBUTED COORDINATION SERVICE
ZOOKEEPER
LOG COLLECTOR
FLUME
AMBARI
WHIRR
RUNNING CLOUD SERVICES

22. We can choose from multiple
VENDORS
like Cloudera, HortonWorks or Amazon

23. Even from...

24. Can we get results
FASTER?

25. Cloudera Impala
Storm
Apache Drill

26. thanks