1. Mariusz Gil
BIG
data
ecosystem

2. / ABOUT ME /

3. This talk is about
BIG DATA

4. What is...
BIG DATA?

6. VOLUME
large amounts of data

7. VELOCITY
needs to be analyzed quickly

8. VARIETY
different types of structured and unstructured data

9. Big Data is data that is too large,
complex and dynamics for any conventional data tools
to capture, store, manage and analyze.

10. 30 billion pieces of content we added past month

11. more than 2 billion videos were watched yesterday

12. more than 58 millions messages were send yesterday

13. / MAIN QUESTIONS /

14. WHY?

15. 49
%
IMPROVED RISK
MANAGEMENT
32
%
INCREASED
SALES FIGURES
36 40
%
IMPROVED
MANAGEMENT
CONTROL
%
IT ANALYSIS
43
%
MARKET-ORIENTED
PRODUCT DEVELOPMENT
27
%
FINANCES AND
ECONOMICS

17. 690 nodes Hadoop cluster for predictions and analytics

18. HOW?

20. HDFS
YARN / MapReduce v2
HADOOP DISTRIBUTED FILE SYSTEM
DISTRIBUTED PROCESSING FRAMEWORK
COLUMNAR STORAGE
SQL DATA WAREHOUSE ENGINE
HIVE
DATA SERIALIZATION
AVRO
SCALABLE MACHINE LEARNING
MAHOUT
SCRIPTING FOR LARGE DATA SETS
PIG
WORKFLOWS ORCHESTRATION
PROVISIONING, MANAGING AND MONITORING CLUSTERS
HBASE
DATA EXCHANGE
SQOOP
OOZIE
DISTRIBUTED COORDINATION SERVICE
ZOOKEEPER
LOG COLLECTOR
FLUME
AMBARI
WHIRR
RUNNING CLOUD SERVICES

22. We can choose from multiple
VENDORS
like Cloudera, HortonWorks or Amazon

23. Even from...

24. Can we get results
FASTER?

25. Cloudera Impala
Storm
Apache Drill

26. thanks