This document discusses the top 5 tasks of a Hadoop developer: 1) development and implementation of Hadoop applications, 2) loading data from disparate data sets, 3) performing analysis on big data, 4) securing data on Hadoop clusters, and 5) managing and deploying Hadoop clusters. It provides examples for tasks 2 and 3, discussing loading heterogeneous data from different sources and how companies analyze huge amounts of user data. The document is part of a presentation on Hadoop developer skills and roles.

1. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Top 5 Tasks of a
Hadoop Developer

2. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Session Objectives
This session will cover
ᗍ Introduction to Big Data and Hadoop
ᗍ Roles & Scope of a Hadoop Developer
ᗍ Top 5 Tasks of Hadoop Developers
ᗍ Introduction to Hadoop Clusters & HBase
ᗍ Job Trends for Hadoop

3. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Big Data Challenges

4. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Why Hadoop?

5. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
What is Hadoop?
ᗍ Hadoop is open source framework for big data. Both distributed storage and processing
ᗍ Hadoop is reliable and fault tolerant with no rely on hardware for these properties
ᗍ Hadoop has unique horizontal scalability

6. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Tasks of a Hadoop Developer
The following are the tasks of a Hadoop Developer:
ᗍ Development and implementation
ᗍ Loading from disparate data sets
ᗍ Pre-processing
ᗍ Designing, building, installing, configuring and supporting Hadoop
ᗍ Translate complex functional and technical requirements into detailed design
ᗍ Perform analysis on big data
ᗍ Securing Data
ᗍ Create scalable and high-performance web services for data tracking
ᗍ High-speed querying
ᗍ Managing and deploying

7. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Let us Look at the Top 5 Tasks of a Hadoop Developer with Examples

8. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Task 1: Development and Implementation
A Hadoop developer is responsible for the actual coding/programming of Hadoop applications
One of the most important component of Hadoop is MapReduce in which you need to write Java programs – all you need
is a basic Java background

9. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Task 2: Loading from Disparate Data Sets
Disparate Data are heterogeneous data
They are neither similar nor can be easily integrated with an organizations database management system. It differs
in one or more aspects of an information system
Disparate data may be characterized by these basic problems:
ᗍ Implementing a database system in an organization, there is no complete and integrated inventory of all its data
ᗍ High data redundancy all over the organization
ᗍ High variability of data formats and contents

10. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Task 2: Loading from Disparate Data Sets – Scenario
Consider a web application where a user can send a query on getting a variety of information about various aspects
such as crime statistics, weather, hotels, demographics, etc. in a particular city
Traditionally, the information must be stored in a single database with a single schema
But it would be difficult for any single enterprise to process and expensive to collect

11. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Task 3: Perform Analysis on Big Data
A Hadoop Developer perform and analyses the big data

12. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Task 3: Perform Analysis on Big Data – Example
Have you ever wondered how Google, Facebook or LinkedIn manages to store and utilize the huge data?
Today, it is becoming a problem for all of us to manage such BIG DATA….

13. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Task 4: Securing Data
One of the biggest concerns in our present age revolves around the security and protection of sensitive information
Network security breaches from internal and external attackers are on the rise, often taking months to be detected,
and affects the organizations terribly

14. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Task 5: Managing and Deploying
Managing the Hadoop Cluster. This is done using HBase

15. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Job Trends – Hadoop

16. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Course Topics
Module 1
Introduction to Big
Data and Hadoop
Module 2
HDFS Internals, Hadoop
Configurations and Data
Loading
Module 3
Introduction to Map
Reduce
Module 4
Advanced Map Reduce
Concepts
Module 5
Introduction to Pig
Module 6
Advanced Pig and
Introduction to Hive
Module 7
Advanced Hive Concepts
Module 8
Extending Hive and HBase
Introduction
Module 9
Advanced HBase and
Oozie Introduction
Module 10
Project Set-up Discussion

17. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Why SkillSpeed?
Course
Curriculum
from Industry
Experts
Instructor Led
Live Virtual
Sessions
Lifetime
access to
Course
Content via
LMS
100%
Placement
Assistance
24x7 Support

18. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Corporate Partners

19. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
Lines open 24/7
To know more about the course, Please contact:
IND +91-90660-20904 USA 1866-607-6547 (Toll Free)
Or reach us at
sales@skillspeed.com
Contact us..

20. Slide ‹#›© 2015 BlueCamphor Technologies (P) Ltd. www.skillspeed.com
References
http://bigdatascroll.com/what-is-hadoop-an-introduction/
http://www.rudraitservices.com/web-development.html
http://www.datameer.com/product/data-visualization.html