Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JavaFest. Grzegorz Piwowarek. Hazelcast - Hitchhiker’s Guide


Published on

Most of you probably heard about the distributed caching being the most common application of Hazelcast - but I bet not many of you heard about CRDT, HyperLogLog, or CP Subsystem.
During this talk, we'll go for quick journey around Hazelcast's ecosystem, revise basic functionality and have a look at some of the hidden flavours.
Even if that’s way too much for your use case, there’s still a lot for an engineer to learn from studying these concepts.

Published in: Education
  • Be the first to comment

  • Be the first to like this

JavaFest. Grzegorz Piwowarek. Hazelcast - Hitchhiker’s Guide

  1. 1. HazelcastHazelcast The Hitchhiker's GuideThe Hitchhiker's Guide Grzegorz PiwowarekGrzegorz Piwowarek @pivovarit@pivovarit
  2. 2. {{ }} Senior Software Engineer @Senior Software Engineer @ Trainer @Trainer @ HazelcastHazelcast Bottega IT MindsBottega IT Minds @pivovarit@pivovarit
  3. 3. Born in 2008 in Turkey as a simple caching solution... source: Hazelcast
  4. 4. The Hazelcast IMDG is an operational in-memory computing platform that manages data and distributes processing using memory and parallel execution for breakthrough application speed and scale. source: Hazelcast
  5. 5. source: Hazelcast
  6. 6. source: Hazelcast
  7. 7. source: Hazelcast
  8. 8. IMDGIMDG In-Memory Data GridIn-Memory Data Grid In-Memory (no persistence) Storage/Computation Multiple processes working as a group
  9. 9. Embedded Client-Server
  10. 10. Cache Access PatternsCache Access Patterns Read-through Write-through Write-behind Near Cache
  11. 11. Read-throughRead-through source: Hazelcast
  12. 12. Write-throughWrite-through source: Hazelcast
  13. 13. Write-behindWrite-behind source: Hazelcast
  14. 14. Near CacheNear Cache source: Hazelcast
  15. 15. <dependencies> <dependency> <groupid>com.hazelcast</groupid> <artifactid>hazelcast</artifactid> <version>4.0.1</version> </dependency> </dependencies>
  16. 16. ShadedShaded
  17. 17. EmbeddedEmbedded HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
  18. 18. Client-ServerClient-Server HazelcastInstance hazelcastInstance = HazelcastClient.newHazelcastClient();
  19. 19. ClientsClients JavaJava ScalaScala C++C++ C#/.NETC#/.NET PythonPython Node.jsNode.js GoGo Rust (work-in-progress)Rust (work-in-progress) Binary Client ProtocolBinary Client Protocol
  20. 20. // import; HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); IMap<String, Integer> simpleMap = hazelcastInstance.getMap("simpleMap"); simpleMap.put("foo", 42); simpleMap.get("foo"); // 42
  21. 21. // import java.util.Map; HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); Map<String, Integer> simpleMap = hazelcastInstance.getMap("simpleMap"); simpleMap.put("foo", 42); simpleMap.get("foo"); // 42
  22. 22. source: Hazelcast
  23. 23. Map is just the beginningMap is just the beginning MultiMap ReplicatedMap (non-partitioned Map) Set List Queue Topic
  24. 24. Data Structures is just the beginningData Structures is just the beginning
  25. 25. EntryProcessorEntryProcessor Instead of fetching the data, processing it and sending it back, let a cluster process it. A processed entry remains locked during processing @BinaryInterface @FunctionalInterface public interface EntryProcessor<K, V, R> extends Serializable { R process(Entry<K, V> entry); }
  26. 26. ExecutorServiceExecutorService public interface IExecutorService extends ExecutorService, DistributedObject { void execute(Runnable command, MemberSelector memberSelector); void executeOnAllMembers(Runnable command); // ... }
  27. 27. AggregationAggregation IMap<String, Employee> employees = hz.getMap("employees"); double avgSalary = employees.aggregate( new Aggregator<Map.Entry<String, Employee>, Double>() { protected long sum; protected long count; @Override public void accumulate(Map.Entry<String, Employee> entry) { count++; sum += entry.getValue().getSalaryPerMonth(); } // ... });
  28. 28. QueriesQueries IMap<String, Employee> map = hazelcastInstance.getMap("employee"); Set<Employee> employees = map.values(new SqlPredicate("active AND age < 30"));
  29. 29. CAP TheoremCAP Theorem ConsistencyConsistency AvailabilityAvailability Partition TolerancePartition Tolerance Pick two.
  30. 30. source: Hazelcast
  31. 31. CP SubsystemCP Subsystem source: Hazelcast Linearizable, distributed implementations of Java concurrency primitives Raft-based( Jepsen-veri ed Requires minimum 3 members
  32. 32. Semaphore CountDownLatch AtomicReference AtomicLong
  33. 33. Advanced Data StructuresAdvanced Data Structures
  34. 34. RingBu erRingBu er A xed-capacity data structure that overwrites oldest elements when capacity is exceeded
  35. 35. HyperLogLogHyperLogLog A probabilistic data structure used to estimate the cardinality(number of unique elements) of a data set. The HyperLogLog algorithm can estimate cardinalities well beyond 10^9 with a relative accuracy (standard error) of 2% while only using 1.5kb of memory. Fangjin Yang Fast, Cheap, and 98% Right: Cardinality Estimation for Big Data
  36. 36. Flake Id GeneratorFlake Id Generator Cluster-wide ordered and unique identi ers IDs without any coordination between members safe even in split-brain scenario
  37. 37. CRDT PN CounterCRDT PN Counter A con ict-free alternative to AtomicLong
  38. 38. Other IntegrationsOther Integrations
  39. 39. Hibernate 2LCHibernate 2LC Hazelcast provides a distributed second level cache for your Hibernate entities, collections and queries.
  40. 40. Spring DataSpring Data Spring Data Hazelcast Integration
  41. 41. Quarkus Hazelcast ClientQuarkus Hazelcast Client
  42. 42. Tomcat Session ManagerTomcat Session Manager Each HttpSession object is kept in the Hazelcast Map
  43. 43. Apache ShiroApache Shiro Apache Shiro is a security framework that performs authentication, authorization, cryptography, and session management. Shiro uses Hazelcast for caching and session clustering.
  44. 44. Envoy ProxyEnvoy Proxy Apache Shiro is a security framework that performs authentication, authorization, cryptography, and session management. Shiro uses Hazelcast for caching and session clustering.
  45. 45. Cloud IntegrationsCloud Integrations Zone-awareness + autodiscoveryZone-awareness + autodiscovery AWS GCP Azure
  46. 46. Hazelcast CloudHazelcast Cloud Hazelcast IMDG cluster as a serviceHazelcast IMDG cluster as a service https://cloud.hazelcast.com
  47. 47. Enterprise FeaturesEnterprise Features Striim Hot Cache Management Center Security Suite Disaster Recovery WAN Replication Rolling Upgrades HD Memory
  48. 48. Striim Hot CacheStriim Hot Cache source: Hazelcast
  49. 49. Disaster RecoveryDisaster Recovery source: Hazelcast
  50. 50. WAN ReplicationWAN Replication source: Hazelcast
  51. 51. HD Memory(o -heap store)HD Memory(o -heap store) Hot Restart StoreHot Restart Store
  52. 52. FutureFuture
  53. 53. Hazelcast JetHazelcast Jet
  54. 54. private static Pipeline buildPipeline() { Pipeline p = Pipeline.create(); p.readFrom(TestSources.itemStream(100, (ts, seq) -> nextRandomNumber())) .withIngestionTimestamps() .window(WindowDefinition.tumbling(1000)) .aggregate(AggregateOperations.topN(TOP, ComparatorEx.comparingLong(l -> l))) .map(WindowResult::result) .writeTo(Sinks.observable(RESULTS)); return p; }
  55. 55. Hazelcast Jet Design DocumentsHazelcast Jet Design Documents Hazelcast Jet ArchitectureHazelcast Jet Architecture
  56. 56. TutorialsTutorials https://training.hazelcast.com
  57. 57. Want to contribute but don't know where/how to start?Want to contribute but don't know where/how to start? @jholusa@jholusa
  58. 58. @pivovarit Thank You!Thank You! Twitter: @pivovaritTwitter: @pivovarit