Bujok hazelcast 4developers
•
1 like•81 views
So, you have heard about in-memory computing but did not have time to deep dive into it nor had a chance to get acquainted with it in a commercial project? This session will try to identify the biggest challenges that are awaiting us in the world of distributed data processing. On the example of Hazelcast and its most popular features we will try to tackle the above-mentioned challenges right away. We will also have a closer look at the data structure offered by Hazelcast, their programming models and learn how to properly use them in our daily work. Finally, we will deep dive into caching. Don't miss this talk mate!
Recommended
Recommended
More Related Content
Similar to Bujok hazelcast 4developers
Similar to Bujok hazelcast 4developers (20)
Recently uploaded
Recently uploaded (20)
Bujok hazelcast 4developers
- 1. LET THE IN-MEMORY CLUSTER BE WITH YOU @TOM BUJOK Warsaw 11th of APRIL 2016 HAZELCAST
- 2. LET THE IN-MEMORY CLUSTER BE WITH YOU ABOUT ME Tom BUJOk SOFTWARE ENGINEER @HAZELCAST @TOMBUJOK WWW.REFICIO.ORG
- 5. SCALING UP vs. SCALING OUT UP OUT http://vinfrastructure.it/wp-content/uploads/2012/08/Storage-Scaleout-Scalein.png
- 6. CAN RDBMS HANDLE IT? ‣ RELATIONAL MODEL ‣ ACID ‣ (DISTRIBUTED?) TRANSACTIONS ‣ SHARDING / PARTITIONING ‣ BACKUPS
- 7. HAZELCAST ‣ LEADING OPEn-SOURCE in-MEMORY DATA / compute GRID ‣ OPEn-SOURCE - apache 2 license ‣ UNOPINIONATED LIBRARY (JAR) ‣ FREE 2 USE - no LIMITATIONS
- 8. HAZELCAST USED BY ‣ Vert.x ‣ Apache camel, shiro, KARAF ‣ Mule ESB ‣ WS02 ‣ ALFRESCO TELECOMMUNICATIONS BANKING & FINANCIAL SERVICES HIGH-TECH LOGISTICS INSURANCE CONSUMER & ECOMMERCE GAMING & ENTERTAINMENT
- 9. public class Employee { private final Boolean active; private final Integer age; private final String name; private final String surname; private final Double salary; } RDBMS USE-CASE
- 11. EVOLUTION NOT REVOLUTION WHAT IF...?
- 12. DAO MAP ...we could STORE THE DATA in a MAP HazelcastInstance hz = Hazelcast.newHazelcastInstance(); Map<Integer, Employee> empls = hz.getMap("employees"); users.put(1, bond); EMPLOYEE
- 13. DAO MAP ...IT's A DISTRIBUTED MAP Node 1 Node 2 Node 3 EMPLOYEE
- 14. PARTITIONING ... 271 PARTITIONs partition_id = hash (key) % partition_count Node 1 Node 2 Node 3 Step 1 Step 2 Step 3
- 15. BACKUP Node 1 Node 2 Node 3 PRIMARY BACKUP ‣ SYNC ‣ ASYNC ‣ PRIMARY vs. BACKUP
- 16. FAILOVER Node 1 Node 2 Node 3 Step 1 Step 2 Step 3
- 17. TOPOLOGY - EMBEDDED Node 1 Node 2 Node 3 Application Application Application JVM JVM JVM
- 18. TOPOLOGY - CLIENT-SERVER Node 1 Node 2 Node 3 JVM JVM JVM Application JVM
- 19. CLIENTS ‣ NATIVE CLIENT PROTOCOL (java, .*NET, *C++) ‣ MEMCACHE ‣ REST ‣ Open Binary Client Protocol (NEW!)
- 20. LET'S DO SOME QUERYING http://www-01.ibm.com/software/data/bigdata/what-is-big-data.html
- 21. QUERYING - PREDICATES public Set<Employee> getWithNameOrAge( int age, double salary ) { Predicate age = Predicates.between( "age", 25, 35 ); Predicate salary = Predicates.lessThan( "salary", 50000 ); Predicate query = Predicates.or( age, salary ); return employees.values( query ); }
- 22. QUERYING - PREDICATES public Set<Employee> getWithNameOrAge( int age, double salary ) { SqlPredicate query = new SqlPredicate( "(age BETWEEN 25 AND 35) OR (salary < 50000)" ) return employees.values( query ); }
- 23. INDEXES IMap map = hazelcastInstance.getMap( "employees" ); // ordered, since we have ranged queries for this field map.addIndex( "age", true ); // not ordered, because boolean field cannot have range map.addIndex( "active", false );
- 24. I NEED THE DATA IN THE RDBMS FOR REPORTING http://www-01.ibm.com/software/data/bigdata/what-is-big-data.html
- 25. MAP PERSISTENCE Node 1 Node 2 Node 3 RDBMS MAP MAp-STORE MAp-LOADER
- 26. MAP PERSISTENCE - MODES ‣ READ-THROUGH ‣ WRITE-THROUGH ‣ WRITE-BEHIND
- 27. MAP PERSISTENCE - GIMME MORE! ‣ INCREMENTAL KEY-LOADING ‣ POST-PROCESSOR
- 28. I NEED SOME CONSISTENCY
- 29. PESSIMISTIC LOCKING IMap<String, Value> map = hz.getMap( "map" ); map.lock( key ); try { Value value = map.get( key ); value.amount++; map.put( key, value ); } finally { map.unlock( key ); } }
- 30. OPTIMISTIC CONCURRENCY CONTROL String key = "007" Employee employee = map.get( key ); employee.salary *= 2; boolean success = map.replace( key, oldValue, newValue )
- 31. Let's DO SOME PROCESSING
- 32. DATA Bad: Send Data to Function DATA Good: Send Function to Data Read Data Write Data Write Function DATA PROCESSING THEORY ;-)
- 33. ENTRY PROCESSOR class SalaryProcessor extends AbstractEntryProcessor<> { int amount; SalaryProcessor(int amount) { this.amount = amount; } @Override public Object process(Map.Entry<> entry) { entry.getValue().salary += amount; return null; } } PRIMARY BACKUP
- 34. DATA AFFINITY Node 1 Node 2 Node 3 MAP Salary Address Notes Personal James Bond Salary Address Notes Personal Eathen Hunt
- 35. DATA AFFINITY: Storing public class SalaryKey implements Serializable, PartitionAware { private final long employeeId; private final long salaryId; @Override public Object getPartitionKey() { return employeeId; }
- 36. DATA AFFINITY: PROCESSING public static class SalaryTask implements Callable<Integer>, PartitionAware, Serializable { private long employeeId; private long salaryId; @Override public Integer call() { } @Override public Object getPartitionKey() { return employeeId; } }
- 37. INTERCEPTOR MAPINTERCEPTORS Node 1 Node 2 Node 3 PUT public interface MapInterceptor extends Serializable { Object interceptPut( Object oldValue, Object newValue ); void afterPut( Object value ); }
- 39. MAPNO NEAR CACHE Node 1 Node 2 Node 3 CALLER CLIENT GETGET JVM
- 40. MAPWITH NEAR CACHE Node 1 Node 2 Node 3 CALLER CLIENTGET JVM GET
- 41. GIMME CACHING!
- 42. EVICTION ‣ TIME-TO-LIVE ‣ MAX-IDLE-SECONDS ‣ EVICTION POLICY ‣ LRU, LFU ‣ MAX-SIZE ‣ ... <hazelcast> <map name="default"> ... <time-to-live-seconds>0</time-to-live-seconds> <max-idle-seconds>0</max-idle-seconds> <eviction-policy>LRU</eviction-policy> <max-size policy="PER_NODE">5000</max-size> <eviction-percentage>25</eviction-percentage> <min-eviction-check-millis>100</min-...> ... </map> </hazelcast>
- 43. JCACHE COMPLIANT
- 44. We'VE JUST SEEN THE MAP ONLY
- 45. HAZELCAST OPEN-SOURCE: ‣ Java Concurrency API ‣ ILock, ISemaphore, ICountDownLatch, ‣ ExecutorService, IdGenerator ‣ IAtomicLong, ISemaphore, IAtomicReference, ‣ Java Collection API ‣ Map, Queue, Set, List ‣ Other: MultiMap, RingBuffer, ReplicatedMap ‣ Messaging: Topic, ReliableTopic ‣ Caching: JCache compliant
- 46. SUBMIT MAP DISTRIBUTED EXECUTION Node 1 Node 2 Node 3 RUNNABLE HazelcastInstance hz = Hazelcast.newHazelcastInstance(); IExecutorService executor = hz.getExecutorService("service"); executor.execute(new EchoTask("foo"));
- 47. WRAP UP
- 48. ANY QUESTIONS?
- 49. THANK YOU Warsaw @TOMBUJOK HAZELCAST 11th of APRIL 2016