Vodafone, Cyberpark ve Türkiye Teknoloji Geliştirme Vakfı işbirliğinde düzenlen etkinlikte büyük veri kavramı, Apache Hadoop Ekosistemi ve Türkiye ve Dünyadaki örnek uygulamalar anlatıldı. - 1 Haziran 2016 - Onur Karadeli, Mustafa Murat Sever

1. Seeing the Big Picture
Through Big Data
Onur Karadeli
Mustafa Murat Sever
March-2016C1 - Public

2. Agenda
• What is Big Data?
• Use Cases
• Apache Hadoop Ecosystem
• Q&A
C1 - Public

3. Big Data ?
C1 - Public

4. What is Big Data ?
4
C1 - Public

5. Big Data is growing (Google Trends)
5
C1 - Public

6. Definition of Big Data
Big data is a term for data sets that are so large or complex
that traditional data processing applications are inadequate.
Challenges include analysis, capture, data curation, search,
sharing, storage, transfer, visualization, querying and
information privacy.
-BigData WIKIPEDIA
What is Big ?
6
C1 - Public

7. The ‘3V’ s
• Volume
• Velocity
• Variety
7
C1 - Public

8. Volume
• %40 Growth per year
• 50 Zettabytes by 2020
Ref:Where-is-your-data-FINAL-5a
8
C1 - Public

9. Volume Scalibility
Single TV
Multi TVs
9
C1 - Public

10. Velocity
Ref: http://wersm.com/how-much-data-is-generated-every-minute-on-social-media/
10C1 - Public

11. Velocity – Realtime triggers
11
C1 - Public

12. Variety
12
Ref: Relational Solutions
C1 - Public

13. Additional ‘V’ s
• Veracity
• Variability
• Visualization
• Value
Ref: http://blog.sqlauthority.com

14. Where to Use Big Data
C1 - Public

15. It’s a Big Family
15C1 - Public

16. 16
Human Being as a Big Data Source
C1 - Public

17. Every moment, new data...
17C1 - Public

18. 18
They are smart now
C1 - Public

19. Health
19
https://www.youtube.com/watch?v=Lyv0_GIGSbY
C1 - Public

20. Some professions will disappear
News reporters Sonographers Lawyers
Sports reporters Phlebotomists Compliance officers/workers
Wall street reporters Radiologists Bill collectors
Journalists Psychotherapists Meeting/event planners
Authors Counselors Fitness coaches
Psychologists Cost estimators Cryptographers
Military planners Accountants Financial planners/advisors
Logisticians Dietitians Tax advisors Interpreters/translators
Nutritionists Customer service reps Auditors
Doctors Teachers
* By Thomas Frey (Senior Futurist @ Da Vinci Institute)
C1 - Public

21. Not Only Humans ...
21C1 - Public

22. Not Only Humans: Connected Cows !
22C1 - Public

23. Not Only Humans: Connected Cows !
23C1 - Public

24. New opportunities for tech companies and new brands
24C1 - Public

25. Social Data
25C1 - Public

26. Visuality is important
26
https://www.youtube.com/watch?v=ujcrJZRSGkg
C1 - Public

27. Just Music
27C1 - Public

28. Not a rocket science, but ...
28
Discover Weekly Data Flow
C1 - Public

29. Not a rocket science, but ...
29
Implicit Matrix Factorization
C1 - Public

30. Other trend use cases
30C1 - Public

31. Apache Hadoop
C1 - Public

32. Apache Hadoop
• Open-Source Projects/Sub-projects of
Apache.
• Core projects
HDFS: Hadoop Distributed File System
MapReduce: Distributed Data processing
...
• Hadoop is not a database.
• Move computation to data !
• Now- %32 percent of all enterprise uses
Apache Hadoop.
32C1 - Public

33. Apache Hadoop History
• 2003 Google File system paper
• 2006 Hadoop subproject created
• 2008 Sort record: Running on a 910-node cluster, Hadoop sorted one
terabyte in 209 seconds
• 2009 Yahoo runs 17 clusters with 24,000 machines
• 2011 Facebook, LinkedIn, eBay and IBM collectively contribute 200,000
lines of code
Ref: https://en.wikipedia.org/wiki/Apache_Hadoop
33C1 - Public

34. Apache Hadoop Base Components & Enablers
Ref: http://synerzip.com - Innovation – It’s in our DNA
34C1 - Public

35. BI & Visualization example
35
Ref: http://forums.bsdinsight.com/articles/?page=4
C1 - Public

36. Hadoop Platforms
Ecosystem Management Software for Platform Management.
36
Examples:
• Cloudera
• Hortonworks
• IBM
• Pivotal
C1 - Public

37. HDFS File Storage Architecture
37C1 - Public

38. Hadoop Topology
38C1 - Public

39. Task Management
39C1 - Public

40. Map & Reduce
40C1 - Public

41. The Best Big Data Team should have ...
41
• Data Hygienists – for clean data
• Data Explorers – discover data to use
• Business Solution Architects – combine data for a use case
• Data Scientists – for the right model
• Campaign Expert – for the best benefit
* From HBR : https://hbr.org/2013/07/five-roles-you-need-on-your-bi
C1 - Public

42. Data Scientists Skills
42C1 - Public

43. Thank you
C1 - Public