Downloaded 13 times
![www.hazelcast.com@noctarius2k
Parallelize Processing
int[] values = [0..100000].map(randomInt())
int[][] blocks = split(values, 10000)
int[] subresult = blocks.foreach(_.sum())
int result = subresult.sum()](https://image.slidesharecdn.com/distributedcomputing-brazil-150825192141-lva1-app6892/85/Distributed-Computing-with-Hazelcast-Brazil-Tour-21-320.jpg)
![www.hazelcast.com@noctarius2k
Parallelize Processing
int[][] blocks = split(values, 10000)
int[] subresult = blocks.foreach(_.sum())
int result = subresult.sum()
No mutation, only new values
Independently computable
Might split further down (fork-join, work-stealing)](https://image.slidesharecdn.com/distributedcomputing-brazil-150825192141-lva1-app6892/85/Distributed-Computing-with-Hazelcast-Brazil-Tour-22-320.jpg)
Uploaded byChristoph Engelbert
Distributed Computing with Hazelcast - Brazil Tour
The document discusses distributed computing and in-memory computing using Hazelcast. It covers how Hazelcast partitions and distributes data across nodes, allows parallel processing of distributed data, and provides distributed caching capabilities with features like TTL and auto-cleanup. Examples of using Hazelcast for distributed maps, parallel sums, and caching are shown.
More Related Content
Similar to Distributed Computing with Hazelcast - Brazil Tour
Distributed Computing with Hazelcast - Brazil Tour
- 1. www.hazelcast.com@noctarius2k /** * Distributed Computing *with Hazelcast */
- 2.
- 3.
- 4. www.hazelcast.com@noctarius2k Disclaimer # A fatalerror has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00002aaaaaf6d9c3, pid=2185, tid=1086892352 # # JRE version: 6.0_21-b06 # Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0-b16 mixed mode linux-amd64 ) # # An error report file with more information is saved as: # /path/to/hs_err_pid2185.log # # This presentation may include certain forward-looking statements and projections provided by the Company. Any such # statements and projections reflect various estimates and assumptions by the Company concerning anticipated # results. These statements involve risks, uncertainties and assumptions and are based on the current estimates and # assumptions of the management of the Company as of the date of this presentation and are subject to uncertainty and # changes. No representations or warranties are made by the Company as to the accuracy of any such statements or # projections. Whether or not any such forward-looking statements or projections are in fact achieved will depend upon # future events some of which are not within the control of the Company. Accordingly, actual results may vary from the # projected results and such variations may be material. Statements contained herein describing documents and # agreements are summaries only and such summaries are qualified in their entirety by reference to such documents # and agreements.
- 5. www.hazelcast.com@noctarius2k Who’s that dude? •Chris(toph) Engelbert • Twitter fanatic: @noctarius2k • Weird 9+ Java-Years • Performance, GC, Fairytales • Apache Committer • Gaming, Travel Management, …
- 6. www.hazelcast.com@noctarius2k • Chris(toph) Engelbert •Twitter fanatic: @noctarius2k • Weird 9+ Java-Years • Performance, GC, Fairytales • Apache Committer • Gaming, Travel Management, … The www.hazelcast.ninja Who’s that dude?
- 7.
- 8. www.hazelcast.com@noctarius2k Toaster Server 1 TV-Server 1 Power CoffeeMaker Server1 Kettle Server 1 Stove Server 1 Power Grid Server 1, 2 Distributed Computing…
- 9. www.hazelcast.com@noctarius2k … but whatmeans Distributed Computing?OOO ( )
- 10.
- 11.
- 12. www.hazelcast.com@noctarius2k ‘In-Memory Computing’ L1 CacheReference L2 Cache Reference Main Memory Reference 1K send over 1 Gbps Network 4K read from SSD Seek on Spinning Disk Send Packet CA->AMS->CA 0.5 ns 7 ns 100 ns 10,000 ns 150,000 ns 10,000,000 ns 150,000,000 ns 0.01 ms 0.15 ms 10 ms 150 ms
- 13.
- 14.
- 15. www.hazelcast.com@noctarius2k Data Distribution String code= “JCP-TOUR-2015” Map<String, TShirtDelivery> shirts = Maps.create() shirts.put(code, new TShirtDelivery(“Your Address”))
- 16. www.hazelcast.com@noctarius2k Data Distribution String code= “JCP-TOUR-2015” int hashcode = hash(code) partitionId = hashcode % partitionCount cluster.send(partitionId, code, delivery)
- 17. www.hazelcast.com@noctarius2k Data Distribution int hashcode= hash(code) partitionId = hashcode % partitionCount cluster.send(partitionId, code, delivery) Must be a consistent hash function Partition count must be constant Can be calculated, smart routing possible
- 18.
- 19. www.hazelcast.com@noctarius2k That was easy!Butdon’t tell anyone!
- 20.
- 21. www.hazelcast.com@noctarius2k Parallelize Processing int[] values= [0..100000].map(randomInt()) int[][] blocks = split(values, 10000) int[] subresult = blocks.foreach(_.sum()) int result = subresult.sum()
- 22. www.hazelcast.com@noctarius2k Parallelize Processing int[][] blocks= split(values, 10000) int[] subresult = blocks.foreach(_.sum()) int result = subresult.sum() No mutation, only new values Independently computable Might split further down (fork-join, work-stealing)
- 23.
- 24.
- 25.
- 26.
- 27. www.hazelcast.com@noctarius2k …access fast whennecessary!…access fast when necessary!
- 28.
- 29.
- 30.
- 31. www.hazelcast.com@noctarius2k Caches Are Simple! publicclass Cache<K, V> { private final Map<K, V> cache = new CHM<>(); public void put(K key, V value) { cache.put(key, value); } public V get(K key) { return cache.get(key); } }
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40. www.hazelcast.com@noctarius2k Proprietary Evolution Of Caching OpenSource Commercial Custom
- 41. www.hazelcast.com@noctarius2k Proprietary Evolution Of Caching OpenSource Commercial Stupidly Expensive Custom
- 42.
- 43.
- 44. www.hazelcast.com@noctarius2k Quick Shameless Plug:) Hazelcast!
- 45. www.hazelcast.com@noctarius2k Hazelcast in 5Facts • Java Collections API • Java Concurrency API • Transparent Data Distribution • Drop-In Replacement • Disruptively Simple (c) Rod Johnson (@springrod)
- 46.
- 47. www.hazelcast.com@noctarius2k Time to beHappy! Apache License 2
- 48. www.hazelcast.com@noctarius2k Time to beHappy! www.hazelcast.org
- 49.
- 50.
- 51.
- 52. www.hazelcast.com@noctarius2k Thank You! Any Questions? @noctarius2k http://www.sourceprojects.org http://github.com/noctarius @hazelcast http://www.hazelcast.com http://www.hazelcast.org http://github.com/hazelcast FreeHazelcast T-Shirt http://bit.ly/tshirtcast We’re hiring! www.hazelcast.com/careers