Map Reduce in Hazelcast - Hazelcast User Group London Version
•
2 likes•1,780 views
A quick walktrough map-reduce and introduction into the Hazelcast implementation of Google's Whitepaper given at the Hazelcast User Group London on 11th June 2014
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (14)
Similar to Map Reduce in Hazelcast - Hazelcast User Group London Version
Similar to Map Reduce in Hazelcast - Hazelcast User Group London Version (20)
More from Christoph Engelbert
More from Christoph Engelbert (20)
Recently uploaded
Recently uploaded (20)
Map Reduce in Hazelcast - Hazelcast User Group London Version
- 1. BIG DATA - FAST DATA USING MAPREDUCE IN HAZELCAST Source: www.hazelcast.com
- 2. Christoph Engelbert(@noctarius2k) 8+ years of JavaWeirdoness Performance, GC, traffic topics Apache Committer Gaming, TravelManagement, ... CastMapRMapReduce for Hazelcast3 www.hazelcast.com
- 5. WHAT IS HAZELCAST? In-MemoryData-Grid DataPartioning(Sharding) JavaCollections Implementation Distributed ComputingPlatform www.hazelcast.com
- 6. WHY HAZELCAST? Automatic Partitioning FaultTolerance Sync /Async Backups FullyDistributed In-Memoryfor HighestSpeed www.hazelcast.com
- 12. DISTRIBUTED COMPUTING OR MULTICORE CPU ON STEROIDS www.hazelcast.com
- 13. THE IDEA OF DISTRIBUTED COMPUTING Source: www.hazelcast.com
- 15. MULTICORE IS NOT NEW Source: www.hazelcast.com
- 19. MAP & REDUCE THE BLACK MAGIC FROM PLANET GOOGLE www.hazelcast.com
- 20. USE CASES LogAnalysis DataQuerying Aggregation Distributed Sort ETL (ExtractTransform Load) and more... www.hazelcast.com
- 22. FULL STEPS Read Map /Transform Combine Group /Shuffle Reduce Collate www.hazelcast.com
- 24. Dataare mapped /transformed in asetof key-value pairs SOME PSEUDO CODE (1/3) MAPPING map(key:String,document:String):Void-> foreachw:Wordindocument: emit(w,1) www.hazelcast.com
- 25. Multiple values are combined to an intermediate resultto preserve traffic SOME PSEUDO CODE (2/3) COMBINING combine(word:Word,counts:List[Int]):Void-> emit(word,sum(counts)) www.hazelcast.com
- 26. Values are reduced /aggregated to the requested result SOME PSEUDO CODE (3/3) REDUCING reduce(word:String,counts:List[Int]):Int-> returnsum(counts) www.hazelcast.com
- 27. FOR MATHEMATICIANS Process: (KxV)*→(LxW)* ⇒ [(l1,w1),…,(lm,wm)] Mapping: (KxV) →(LxW)* ⇒ (k,v) →[(l1,w1),…,(ln,wn)] Reducing: LxW*→X* ⇒ (l,[w1,…,wn]) →[x1,…,xn] www.hazelcast.com
- 28. MAPREDUCE PROGRAMS IN GOOGLE SOURCE TREE Source: www.hazelcast.com