+ Follow

MapReduce in Simple Terms

by Saliya Ekanayake

22,388 views

This presentation explains the idea of MapReduce in Data Intensive Computing in a nutshell

Accessibility

View text version

Upload Details

Uploaded via SlideShare as Microsoft PowerPoint

Usage Rights

File a copyright complaint

91 Embeds 4,876

http://nosql.mypopescu.com	1670
http://esaliya.blogspot.com	916
http://datasearch.ruc.edu.cn	563
http://pragmaticintegration.blogspot.com	324
http://blog.nosqlfan.com	275
https://twitter.com	228
http://calebesantos.wordpress.com	187
http://esaliya.blogspot.in	157
http://www.cxybase.com	44
http://www.slideshare.net	44
http://www.cols-code-snippets.co.uk	37
http://nagarun.wordpress.com	32
http://esaliya.blogspot.co.uk	29
http://www.techgig.com	28
http://pragmaticintegration.blogspot.co.uk	21
http://esaliya.blogspot.de	21
http://esaliya.blogspot.ca	20
http://pragmaticintegration.blogspot.de	20
http://www.scoop.it	19
http://esaliya.blogspot.com.au	18
http://pragmaticintegration.blogspot.in	17
http://blog.tjp.hu	12
http://paper.li	11
http://www.esaliya.blogspot.com	10
http://static.slidesharecdn.com	8
http://translate.googleusercontent.com	8
http://pragmaticintegration.blogspot.nl	8
http://pragmaticintegration.blogspot.ch	8
http://pragmaticintegration.blogspot.dk	7
http://pragmaticintegration.blogspot.com.br	7
http://confluence	7
http://esaliya.blogspot.fr	6
http://a0.twimg.com	6
http://esaliya.blogspot.fi	6
http://esaliya.blogspot.ru	5
http://esaliya.blogspot.sg	5
http://esaliya.blogspot.it	5
http://webcache.googleusercontent.com	5
http://pragmaticintegration.blogspot.kr	4
http://esaliya.blogspot.com.es	4
http://pragmaticintegration.blogspot.com.au	4
http://esaliya.blogspot.tw	4
http://esaliya.blogspot.se	3
http://esaliya.blogspot.com.br	3
http://esaliya.blogspot.hk	3
http://esaliya.blogspot.mx	2
http://esaliya.blogspot.nl	2
http://pragmaticintegration.blogspot.sk	2
http://esaliya.blogspot.kr	2
http://esaliya.blogspot.pt	2
http://esaliya.blogspot.ch	2

More...

Statistics

Likes: 44
Downloads: 654
Comments: 14
Embed Views: 4,876
Views on SlideShare: 17,512
Total Views: 22,388

MapReduce in Simple Terms Presentation Transcript

MapReduce
The Story of Sam
SaliyaEkanayake
SALSA HPC Group
http://salsahpc.indiana.edu
Pervasive Technology Institute, Indiana University, Bloomington

Believed “an apple a day keeps a doctor away”
Sam’s Mother
Mother
Sam
An Apple

Sam thought of “drinking” the apple
One day

He used a to cut the and a to make juice.

Sam applied his invention to all the fruits he could find in the fruit basket

(map ‘( ))

Next Day
A list of values mapped into another list of values, which gets reduced into a single value
( )

(reduce ‘( ))

Classical Notion of MapReduce in Functional Programming

18 Years Later
Sam got his first job in JuiceRUs for his talent in making juice
Wait!
Fruits

Now, it’s not just one basket but a whole container of fruits

Largedata and list of values for output

Also, they produce alist of juice types separately
But, Sam had just ONE and ONE

NOT ENOUGH !!

Implemented a parallelversion of his innovation
Brave Sam
Each input to a map is a list of <key, value> pairs
Fruits
A list of <key, value> pairs mapped into another list of <key, value> pairs which gets grouped by the key and reduced into a list of values
(<a, > , <o, > , <p, > , …)
Each output of a map is a list of <key, value> pairs
(<a’, > , <o’, > , <p’, > , …)
Grouped by key
The idea of MapReduce in Data Intensive Computing
Each input to a reduce is a <key, value-list> (possibly a list of these, depending on the grouping/hashing mechanism)
e.g. <a’, ( …)>
Reduced into a list of values

Sam realized,
To create his favorite mix fruit juice he can use a combiner after the reducers
If several <key, value-list> fall into the same group (based on the grouping/hashing algorithm) then use the blender (reducer) separately on each of them
The knife (mapper) and blender (reducer) should not contain residue after use – Side Effect Free
In general reducer should be associative and commutative
Afterwards

We think Sam was you 
That’s All Folks!

Long Nguyen Cong, Lecture at Ba Ria - Vung Tau Teacher Training College That is an interesting representation, thank you 3 months ago

Are you sure you want to

nxhoaf at nxhoaf What a nice presentation, thank you :) 7 months ago

Vishnu Rao, Software Engineer II at flipkart.com Very healthy :) thanks. 8 months ago

vozniuk, Computer Science PhD student at EPFL Wow!! Nice and clear presentation. Great analogies! 1 year ago

you1025 おもろい 1 year ago

Vijayakumar Ramdoss, Machine Learning /Big Data at EMC Nice one 1 year ago

Shay Sofer, Senior Developer at AlphaCSP Hi Saliya,

This is indeed a great presentation!
I'm currently working on a Hadoop\MapReduce presentation for TheEdge 2010 Conference (http://www.javaedge.com/jedge/) and I wondered if It's ok to use some of your slides.

Needless to say, I will reference you and SALSA group.

Thanks,
Shay. 2 years ago

Saliya Ekanayake, Grad Student, Research Assistant at Indiana University, Bloomington @ilay: You are welcome and good luck with the presentation 2 years ago

Ilayaraja Prabakaran, Product Engineer at Rediff.com hi saliya, appreciate your quick response. I wud be glad to acknowledge the ref.
Conference link: http://www.srmuniv.ac.in/icaet10/

Thanks
ilay 2 years ago

Saliya Ekanayake, Grad Student, Research Assistant at Indiana University, Bloomington @ilaybuzz: hi, yes you may take slides and please make a reference to SALSA HPC Group at Community Grids Labs headed by Prof. Geoffrey Fox in Pervasive Technology Institute, Indiana University, Bloomington.

Also we will be very glad to know the conference/university name that you may use this for our reference purposes.

Thank you for the interest and request.

saliya 2 years ago