Submit Search
Upload
Map Reduce introduction
•
2 likes
•
1,533 views
M
Muralidharan Deenathayalan
Follow
Introduction about Map Reduce program
Read less
Read more
Technology
Report
Share
Report
Share
1 of 19
Recommended
Spark at-hackthon8jan2014
Spark at-hackthon8jan2014
Madhukara Phatak
Hadoop Map Reduce
Hadoop Map Reduce
VNIT-ACM Student Chapter
Introduction to Map Reduce
Introduction to Map Reduce
Apache Apex
Map Reduce
Map Reduce
Michel Bruley
MapReduce basic
MapReduce basic
Chirag Ahuja
Mapreduce by examples
Mapreduce by examples
Andrea Iacono
Introduction to Map-Reduce
Introduction to Map-Reduce
Brendan Tierney
Map Reduce
Map Reduce
Rahul Agarwal
Recommended
Spark at-hackthon8jan2014
Spark at-hackthon8jan2014
Madhukara Phatak
Hadoop Map Reduce
Hadoop Map Reduce
VNIT-ACM Student Chapter
Introduction to Map Reduce
Introduction to Map Reduce
Apache Apex
Map Reduce
Map Reduce
Michel Bruley
MapReduce basic
MapReduce basic
Chirag Ahuja
Mapreduce by examples
Mapreduce by examples
Andrea Iacono
Introduction to Map-Reduce
Introduction to Map-Reduce
Brendan Tierney
Map Reduce
Map Reduce
Rahul Agarwal
Map reduce presentation
Map reduce presentation
ateeq ateeq
Introduction to MapReduce
Introduction to MapReduce
Hassan A-j
MapReduce Paradigm
MapReduce Paradigm
Dilip Reddy
Map Reduce
Map Reduce
Vigen Sahakyan
Map Reduce
Map Reduce
schapht
An Introduction to MapReduce
An Introduction to MapReduce
Frane Bandov
Map Reduce
Map Reduce
Sri Prasanna
Analysing of big data using map reduce
Analysing of big data using map reduce
Paladion Networks
Mapreduce Algorithms
Mapreduce Algorithms
Amund Tveit
Mastering Hadoop Map Reduce - Custom Types and Other Optimizations
Mastering Hadoop Map Reduce - Custom Types and Other Optimizations
scottcrespo
Large Scale Data Analysis with Map/Reduce, part I
Large Scale Data Analysis with Map/Reduce, part I
Marin Dimitrov
An Introduction To Map-Reduce
An Introduction To Map-Reduce
Francisco Pérez-Sorrosal
Map reduce paradigm explained
Map reduce paradigm explained
Dmytro Sandu
MapReduce Algorithm Design
MapReduce Algorithm Design
Gabriela Agustini
Hadoop - Introduction to map reduce programming - Reunião 12/04/2014
Hadoop - Introduction to map reduce programming - Reunião 12/04/2014
soujavajug
Introduction to MapReduce
Introduction to MapReduce
Chicago Hadoop Users Group
Hadoop/MapReduce/HDFS
Hadoop/MapReduce/HDFS
praveen bhat
Introduction To Map Reduce
Introduction To Map Reduce
rantav
Topic 6: MapReduce Applications
Topic 6: MapReduce Applications
Zubair Nabi
Hadoop MapReduce Fundamentals
Hadoop MapReduce Fundamentals
Lynn Langit
Behm Shah Pagerank
Behm Shah Pagerank
gothicane
Hadoop ecosystem
Hadoop ecosystem
Ran Silberman
More Related Content
What's hot
Map reduce presentation
Map reduce presentation
ateeq ateeq
Introduction to MapReduce
Introduction to MapReduce
Hassan A-j
MapReduce Paradigm
MapReduce Paradigm
Dilip Reddy
Map Reduce
Map Reduce
Vigen Sahakyan
Map Reduce
Map Reduce
schapht
An Introduction to MapReduce
An Introduction to MapReduce
Frane Bandov
Map Reduce
Map Reduce
Sri Prasanna
Analysing of big data using map reduce
Analysing of big data using map reduce
Paladion Networks
Mapreduce Algorithms
Mapreduce Algorithms
Amund Tveit
Mastering Hadoop Map Reduce - Custom Types and Other Optimizations
Mastering Hadoop Map Reduce - Custom Types and Other Optimizations
scottcrespo
Large Scale Data Analysis with Map/Reduce, part I
Large Scale Data Analysis with Map/Reduce, part I
Marin Dimitrov
An Introduction To Map-Reduce
An Introduction To Map-Reduce
Francisco Pérez-Sorrosal
Map reduce paradigm explained
Map reduce paradigm explained
Dmytro Sandu
MapReduce Algorithm Design
MapReduce Algorithm Design
Gabriela Agustini
Hadoop - Introduction to map reduce programming - Reunião 12/04/2014
Hadoop - Introduction to map reduce programming - Reunião 12/04/2014
soujavajug
Introduction to MapReduce
Introduction to MapReduce
Chicago Hadoop Users Group
Hadoop/MapReduce/HDFS
Hadoop/MapReduce/HDFS
praveen bhat
Introduction To Map Reduce
Introduction To Map Reduce
rantav
Topic 6: MapReduce Applications
Topic 6: MapReduce Applications
Zubair Nabi
Hadoop MapReduce Fundamentals
Hadoop MapReduce Fundamentals
Lynn Langit
What's hot
(20)
Map reduce presentation
Map reduce presentation
Introduction to MapReduce
Introduction to MapReduce
MapReduce Paradigm
MapReduce Paradigm
Map Reduce
Map Reduce
Map Reduce
Map Reduce
An Introduction to MapReduce
An Introduction to MapReduce
Map Reduce
Map Reduce
Analysing of big data using map reduce
Analysing of big data using map reduce
Mapreduce Algorithms
Mapreduce Algorithms
Mastering Hadoop Map Reduce - Custom Types and Other Optimizations
Mastering Hadoop Map Reduce - Custom Types and Other Optimizations
Large Scale Data Analysis with Map/Reduce, part I
Large Scale Data Analysis with Map/Reduce, part I
An Introduction To Map-Reduce
An Introduction To Map-Reduce
Map reduce paradigm explained
Map reduce paradigm explained
MapReduce Algorithm Design
MapReduce Algorithm Design
Hadoop - Introduction to map reduce programming - Reunião 12/04/2014
Hadoop - Introduction to map reduce programming - Reunião 12/04/2014
Introduction to MapReduce
Introduction to MapReduce
Hadoop/MapReduce/HDFS
Hadoop/MapReduce/HDFS
Introduction To Map Reduce
Introduction To Map Reduce
Topic 6: MapReduce Applications
Topic 6: MapReduce Applications
Hadoop MapReduce Fundamentals
Hadoop MapReduce Fundamentals
Similar to Map Reduce introduction
Behm Shah Pagerank
Behm Shah Pagerank
gothicane
Hadoop ecosystem
Hadoop ecosystem
Ran Silberman
MAP REDUCE IN DATA SCIENCE.pptx
MAP REDUCE IN DATA SCIENCE.pptx
HARIKRISHNANU13
Map Reduce
Map Reduce
Prashant Gupta
Hadoop ecosystem
Hadoop ecosystem
Ran Silberman
Introduction to Map-Reduce Programming with Hadoop
Introduction to Map-Reduce Programming with Hadoop
Dilum Bandara
Big Data & Analytics MapReduce/Hadoop – A programmer’s perspective
Big Data & Analytics MapReduce/Hadoop – A programmer’s perspective
EMC
MapReduce basics
MapReduce basics
Harisankar H
MapReduce wordcount program
MapReduce wordcount program
Sarwan Singh
Spark what's new what's coming
Spark what's new what's coming
Databricks
Big-data-analysis-training-in-mumbai
Big-data-analysis-training-in-mumbai
Unmesh Baile
Intro to Apache Spark by Marco Vasquez
Intro to Apache Spark by Marco Vasquez
MapR Technologies
Advance Map reduce - Apache hadoop Bigdata training by Design Pathshala
Advance Map reduce - Apache hadoop Bigdata training by Design Pathshala
Desing Pathshala
A performance analysis of OpenStack Cloud vs Real System on Hadoop Clusters
A performance analysis of OpenStack Cloud vs Real System on Hadoop Clusters
Kumari Surabhi
Taste Java In The Clouds
Taste Java In The Clouds
Jacky Chu
Mapredtutorial
Mapredtutorial
Anup Mohta
Running Cognos on Hadoop
Running Cognos on Hadoop
Senturus
Cs267 hadoop programming
Cs267 hadoop programming
Kuldeep Dhole
Dart and Flutter Basics.pptx
Dart and Flutter Basics.pptx
DSCVSSUT
Intermachine Parallelism
Intermachine Parallelism
Sri Prasanna
Similar to Map Reduce introduction
(20)
Behm Shah Pagerank
Behm Shah Pagerank
Hadoop ecosystem
Hadoop ecosystem
MAP REDUCE IN DATA SCIENCE.pptx
MAP REDUCE IN DATA SCIENCE.pptx
Map Reduce
Map Reduce
Hadoop ecosystem
Hadoop ecosystem
Introduction to Map-Reduce Programming with Hadoop
Introduction to Map-Reduce Programming with Hadoop
Big Data & Analytics MapReduce/Hadoop – A programmer’s perspective
Big Data & Analytics MapReduce/Hadoop – A programmer’s perspective
MapReduce basics
MapReduce basics
MapReduce wordcount program
MapReduce wordcount program
Spark what's new what's coming
Spark what's new what's coming
Big-data-analysis-training-in-mumbai
Big-data-analysis-training-in-mumbai
Intro to Apache Spark by Marco Vasquez
Intro to Apache Spark by Marco Vasquez
Advance Map reduce - Apache hadoop Bigdata training by Design Pathshala
Advance Map reduce - Apache hadoop Bigdata training by Design Pathshala
A performance analysis of OpenStack Cloud vs Real System on Hadoop Clusters
A performance analysis of OpenStack Cloud vs Real System on Hadoop Clusters
Taste Java In The Clouds
Taste Java In The Clouds
Mapredtutorial
Mapredtutorial
Running Cognos on Hadoop
Running Cognos on Hadoop
Cs267 hadoop programming
Cs267 hadoop programming
Dart and Flutter Basics.pptx
Dart and Flutter Basics.pptx
Intermachine Parallelism
Intermachine Parallelism
More from Muralidharan Deenathayalan
What's new in C# 8.0 (beta)
What's new in C# 8.0 (beta)
Muralidharan Deenathayalan
Introduction to Jupyter notebook and MS Azure Machine Learning Studio
Introduction to Jupyter notebook and MS Azure Machine Learning Studio
Muralidharan Deenathayalan
Alfresco 5.0 features
Alfresco 5.0 features
Muralidharan Deenathayalan
Test drive on driven development process
Test drive on driven development process
Muralidharan Deenathayalan
Apache Hive - Introduction
Apache Hive - Introduction
Muralidharan Deenathayalan
Apache cassandra
Apache cassandra
Muralidharan Deenathayalan
Alfresco share 4.1 to 4.2 customisation
Alfresco share 4.1 to 4.2 customisation
Muralidharan Deenathayalan
Introduction about Alfresco webscript
Introduction about Alfresco webscript
Muralidharan Deenathayalan
Alfresco activiti workflows
Alfresco activiti workflows
Muralidharan Deenathayalan
Alfresco content model
Alfresco content model
Muralidharan Deenathayalan
More from Muralidharan Deenathayalan
(10)
What's new in C# 8.0 (beta)
What's new in C# 8.0 (beta)
Introduction to Jupyter notebook and MS Azure Machine Learning Studio
Introduction to Jupyter notebook and MS Azure Machine Learning Studio
Alfresco 5.0 features
Alfresco 5.0 features
Test drive on driven development process
Test drive on driven development process
Apache Hive - Introduction
Apache Hive - Introduction
Apache cassandra
Apache cassandra
Alfresco share 4.1 to 4.2 customisation
Alfresco share 4.1 to 4.2 customisation
Introduction about Alfresco webscript
Introduction about Alfresco webscript
Alfresco activiti workflows
Alfresco activiti workflows
Alfresco content model
Alfresco content model
Recently uploaded
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel Araújo
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
sudhanshuwaghmare1
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Maria Levchenko
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Delhi Call girls
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
Delhi Call girls
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
Antenna Manufacturer Coco
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Enterprise Knowledge
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
RTylerCroy
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Igalia
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Puma Security, LLC
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Katpro Technologies
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
Principled Technologies
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Drew Madelung
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
naman860154
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
The Digital Insurer
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
Recently uploaded
(20)
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Map Reduce introduction
1.
Confidential, Copyright ©
Quanticate Introduction to Map - Reduce Muralidharan Deenathayalan Technical Lead Muralidharan.deenathayalan@quanticate.com Apache logo are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.
2.
Confidential, Copyright ©
Quanticate Agenda What is Map-Reduce? Map-Reduce architecture Advantages of Map-Reduce Frameworks available for writing Map-Reduce? WordCount – Map-Reduce Program explained How to compile Map-Reduce program using Eclipse? How to deploy Map-Reduce program? How to run Map-Reduce program? Q & A
3.
Confidential, Copyright ©
Quanticate Who Am I ? 7+ years of experience in Microsoft technologies like Asp.net, C#, SQL server and SharePoint 2+ years of experience in open source technologies like Java, Alfresco and Apache Cassandra Author of Apache Cassandra Cookbook (In writing ) Csharpcorner MVP Frequent blogger
4.
Confidential, Copyright ©
Quanticate What is Map-Reduce? Generally called as Map-R program MapReduce Map() + Reduce() MapReduce is a programming approach to process large datasets in parallel, distributed on a cluster ( Divide and conquer). Map
5.
Confidential, Copyright ©
Quanticate What is Map-Reduce? • Map: – Receives input key/value pair – Outputs intermediate key/value pair • Reduce : – Receives intermediate key/value pair – Outputs key/value pair Input Data Map Reduce Reduce Map Map Input Data
6.
Confidential, Copyright ©
Quanticate Map-Reduce Architecture overview Job trackerJob tracker Task tracker Task tracker Task tracker Master node Slave node 1 Slave node 2 Slave node N Workers user Workers Workers
7.
Confidential, Copyright ©
Quanticate Advantages of Map-Reduce Distributed pattern-based searching Distributed sorting Web access logs Machine Learning
8.
Confidential, Copyright ©
Quanticate Framework available for writing Map-Reduce Courtesy & ©: http://blog.matthewrathbone.com/2013/01/05/a-quick-guide-to-hadoop-map-reduce-frameworks.html JAVA Cascading Crunch CLOJURE Cascalog SCALA Scrunch Scalding Scoobi R Rhadoop MICROSOFT .Net (C# / VB.net) SPECIAL (HIGH-LEVEL) Apache Hive Apache Pig RUBY Wukong Cascading Jruby PYTHON MR Job Dumbo Hadooppy Pydoop Luigi
9.
Confidential, Copyright ©
Quanticate WordCount – Map-Reduce Program public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> { private final static IntWritable one = new IntWritable(1); private Text word = new Text(); public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { String line = value.toString(); StringTokenizer tokenizer = new StringTokenizer(line); while (tokenizer.hasMoreTokens()) { word.set(tokenizer.nextToken()); output.collect(word, one); } } }
10.
Confidential, Copyright ©
Quanticate WordCount – Map-Reduce Program public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> { public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { int sum = 0; while (values.hasNext()) { sum += values.next().get(); } output.collect(key, new IntWritable(sum)); } }
11.
Confidential, Copyright ©
Quanticate WordCount – Map-Reduce Program public static void main(String[] args) throws Exception { JobConf conf = new JobConf(WordCount.class); conf.setJobName("wordcount"); conf.setOutputKeyClass(Text.class); conf.setOutputValueClass(IntWritable.class); conf.setMapperClass(Map.class); conf.setCombinerClass(Reduce.class); conf.setReducerClass(Reduce.class); conf.setInputFormat(TextInputFormat.class); conf.setOutputFormat(TextOutputFormat.class); FileInputFormat.setInputPaths(conf, new Path(args[0])); FileOutputFormat.setOutputPath(conf, new Path(args[1])); JobClient.runJob(conf); }
12.
Confidential, Copyright ©
Quanticate How to compile Map-Reduce program using Eclipse? Refer Hadoop jar file from your disk Maven is simple to use Eclipse Project Build Project No errors in the eclipse console
13.
Confidential, Copyright ©
Quanticate How to deploy Map-Reduce program?
14.
Confidential, Copyright ©
Quanticate How to run Map-Reduce program?
15.
Confidential, Copyright ©
Quanticate Summary What is Map-Reduce? Architecture of Map-Reduce? Advantages of Map-Reduce Frameworks available for Map-Reduce? WordCount – Map-Reduce Program explained Compiling WordCount Map-Reduce program using Eclipse Deploying Map-Reduce program Executing a Map-Reduce program
16.
Confidential, Copyright ©
Quanticate Q & A
17.
Confidential, Copyright ©
Quanticate References http://en.wikipedia.org/wiki/MapReduce http://hortonworks.com http://hadoop.apache.org
18.
Confidential, Copyright ©
Quanticate Coding-Freaks.Net www.codingfreaks.net Quanticate OPDev Twitter https://twitter.com/quanticateopdev Twitter www.Twitter.com/muralidharand
19.
Confidential, Copyright ©
Quanticate