This document discusses big data and Hadoop. It notes that in the last 5 years, more data has been generated than all of humanity previously. It then provides examples of the scale of data generated on Google, YouTube, and worldwide daily. The document goes on to discuss how big data is being used in various domains like politics, healthcare, banking, and more. It defines big data using IBM's 3V+1 framework and introduces Hadoop as an open source software framework for distributed storage and processing of large datasets across clusters of computers.

1. Phone : +91 9619 66 3272
Twitter : @aruforchange
1(C)2016 - 2017 Tamilboomi E-Learning

2. Last 5 years we have generated data more than
entire humanity has ever generated.!!
 After you start reading this line.
 More than 150k searches has been made on Google.
 More than 34k GB of Data generated in the world.
 More than 50 Hours video has been uploaded in
YouTube.
Big data is huge.!
Reference : http://www.internetlivestats.com/one-second/

3. (C)2016 - 2017 Aru Learning System 3Reference : Excelacom

4. (C)2016 - 2017 Aru Learning System 4

5. (C)2016 - 2017 Aru Learning System 5
 20 million requests.
 750 Gb Logs.
 60000 request sent in first one hour.
 30k users online.
 3 days seats sold in 6 mins.

6.  In 5 years, 21 billion connected devices, 6.1 billion will
be mobile phones.
 Big opportunities for Fortune 1000 companies to small
scale companies.
At the moment 0.5% data ever analyzed. Imagine
the potential.
 73% of organization is invested of going to invest in
hadoop by 2016.
Time will never be better than now to learn
hadoop..!
6(C)2016 - 2017 Aru Learning System
Reference : http://www.informationweek.com/mobile/mobile-devices/gartner-21-
billion-iot-devices-to-invade-by-2020/d/d-id/1323081

7. Bigdata successfully Implemented
Areas - Politics
 Worlds biggest democracy election 2014:
 1 .2 Billion people.
 24 crore internet users.
 100 Million smart phones
 Huge data to be analyzed in short manner.
 SAP, Oracle, InMobi partnered with BJP.
 Used cookies and Targeted advertisement.
 Obama used at 2012 in U.S Elections.
7(C)2016 - 2017 Aru Learning System
Reference : http://www.firstpost.com/india/big-data-analysis-how-it-firms-like-sap-
oracle-helped-modi-win-1576355.html

8. How Bigdata has used in politics?
(C)2016 - 2017 Aru Learning System 8
http://www.deccanherald.com/content/530569/big-data-tech-flavour-tn.html

9. Big data implemented in other
areas
 Digitalized industry,
 Health care –Siemens 800 million users
 Entertainment – Netflix
 Banking Sector – Credit risk simulation
 Telecom – Vodafone
 Mobile – Real time location based Promotion.
9(C)2016 - 2017 Aru Learning System
Reference : http://www-01.ibm.com/software/data/bigdata/use-cases.html

10. Bigdata Definition
 3 +1 V’s by IBM
 Volume
 Velocity
 Varity
 Veracity
 Anything beyond storage & Processing power is called
big data.
10(C)2016 - 2017 Aru Learning System

11. Meet Hadoop
 Hadoop is a framework written on java to process
Bigdata.
 Hadoop name derived from big elephant toy.
 Developed by Doug cutting.
11(C)2016 - 2017 Aru Learning System

12. (C)2016 - 2017 Aru Learning System 12

13. Working on a project nutch at yahoo.
 Faced scaling up problem.
 2003 Google published a white paper of how they
handle their data problem.
 Based on that idea hadoop is developed as an open
source system under apache license.
 It has two main concepts which made it the multi
billion dollar industry
• HDFS and Map Reduce
13(C)2016 - 2017 Aru Learning System
Link to white paper : http://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-
osdi04.pdf

14. Why Hadoop?
 Scale out instead of scale in.
 Commodity hardware.
 Unstructured data processing.
14(C)2016 - 2017 Aru Learning System

15. Current Career choices
15(C)2016 - 2017 Aru Learning System
Reference : karmasphere

16. What hadoop is not?
 Not going to replace RDBMS
 Not be suitable for Real time operational. Why?
 Hadoop is not suitable for all problems.
 Hadoop is not best solution for many small files.
16(C)2016 - 2017 Aru Learning System
Reference : https://wiki.apache.org/hadoop/HadoopIsNot

17. Assignment
 Think about your company data and how hadoop can
be used for that?
17(C)2016 - 2017 Aru Learning System