More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (8)
Similar to Map Reduce
Similar to Map Reduce (20)
Recently uploaded
Recently uploaded (20)
Map Reduce
- 1. MapReduce A Gentle Introduction, In Four Acts
- 7. The Web Is Kind Of A Mess
- 8. But There Is Some Order <html> <head> <title> Marmots I’ve Loved </title> </head> <body> <h1> Marmot List </h1> <ul> <li> Marcy </li> <li> Stacy </li> </ul> </body> </html> 12:00:23 GET /marmots/index.html 12:00:55 GET /marmots/stacy.jpg 12:00:67 GET /marmots/marcy.jpg
- 10. Act II Enter Stage Left – MapReduce
- 14. MapReduce Pseudocode Distributed Word Count* *This example is legally required to be in all introductions to MapReduce map(record) words = split(record, ‘ ‘) for word in words emit(word, 1) reduce(key, values) int count = 0 for value in values count += 1 emit(key, count)
- 15. Act III Hadoop (Streaming Mode)
- 17. MapReduce Mapper Distributed Word Count* *This example is legally required to be in all introductions to MapReduce #!/usr/bin/ruby STDIN.each_line do |line| words = line.split(' ') words.each { |word| puts "#{word} 1" } end
- 18. MapReduce Reducer Distributed Word Count* *This example is legally required to be in all introductions to MapReduce #!/usr/bin/ruby count = 0 current_word = nil STDIN.each_line do |line| key, value = line.split("") current_word = key if nil == current_word if (key != current_word) then puts "#{current_word}#{count}" count = 0 current_word = key end count += value.to_i end puts "#{current_word}#{count}"
- 20. Act IV Amazon Elastic Map Reduce
- 21. So I’ve Got This Pile Of Data, Now What?
- 22. Buy A Bunch Of Servers?