Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
HazelcastHazelcast
The Hitchhiker's GuideThe Hitchhiker's Guide
Grzegorz PiwowarekGrzegorz Piwowarek
@pivovarit@pivovarit
{{ }}
Senior Software Engineer @Senior Software Engineer @
Trainer @Trainer @
4comprehension.com4comprehension.com
Hazelca...
Born in 2008 in Turkey as a simple caching solution...
source: Hazelcast
The Hazelcast IMDG is an operational in-memory computing platform that manages data and distributes
processing using memor...
source: Hazelcast
source: Hazelcast
source: Hazelcast
IMDGIMDG
In-Memory Data GridIn-Memory Data Grid
In-Memory (no persistence)
Storage/Computation
Multiple processes working ...
Embedded
Client-Server
Cache Access PatternsCache Access Patterns
Read-through
Write-through
Write-behind
Near Cache
Read-throughRead-through
source: Hazelcast
Write-throughWrite-through
source: Hazelcast
Write-behindWrite-behind
source: Hazelcast
Near CacheNear Cache
source: Hazelcast
<dependencies>
<dependency>
<groupid>com.hazelcast</groupid>
<artifactid>hazelcast</artifactid>
<version>4.0.1</version>
<...
ShadedShaded
EmbeddedEmbedded
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Client-ServerClient-Server
HazelcastInstance hazelcastInstance = HazelcastClient.newHazelcastClient();
ClientsClients
JavaJava
ScalaScala
C++C++
C#/.NETC#/.NET
PythonPython
Node.jsNode.js
GoGo
Rust (work-in-progress)Rust (wor...
// import com.hazelcast.map.IMap;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IMap<String, Int...
// import java.util.Map;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Map<String, Integer> simp...
source: Hazelcast
Map is just the beginningMap is just the beginning
MultiMap
ReplicatedMap (non-partitioned Map)
Set
List
Queue
Topic
Data Structures is just the beginningData Structures is just the beginning
EntryProcessorEntryProcessor
Instead of fetching the data, processing it and sending it back, let a cluster process it. A ...
ExecutorServiceExecutorService
public interface IExecutorService
extends ExecutorService, DistributedObject {
void execute...
AggregationAggregation
IMap<String, Employee> employees = hz.getMap("employees");
double avgSalary = employees.aggregate(
...
QueriesQueries
IMap<String, Employee> map = hazelcastInstance.getMap("employee");
Set<Employee> employees = map.values(new...
CAP TheoremCAP Theorem
ConsistencyConsistency
AvailabilityAvailability
Partition TolerancePartition Tolerance
Pick two.
source: Hazelcast
CP SubsystemCP Subsystem
source: Hazelcast
Linearizable, distributed implementations of Java concurrency primitives
Raft-b...
Semaphore
CountDownLatch
AtomicReference
AtomicLong
Advanced Data StructuresAdvanced Data Structures
RingBu erRingBu er
A xed-capacity data structure that overwrites oldest elements when capacity is exceeded
HyperLogLogHyperLogLog
A probabilistic data structure used to estimate the cardinality(number of unique elements) of a dat...
Flake Id GeneratorFlake Id Generator
Cluster-wide ordered and unique identi ers IDs without any coordination between membe...
CRDT PN CounterCRDT PN Counter
A con ict-free alternative to AtomicLong
Other IntegrationsOther Integrations
Hibernate 2LCHibernate 2LC
Hazelcast provides a distributed second level cache for your Hibernate entities, collections an...
Spring DataSpring Data
Spring Data Hazelcast Integration
https://github.com/hazelcast/spring-data-hazelcasthttps://github....
Quarkus Hazelcast ClientQuarkus Hazelcast Client
https://github.com/hazelcast/quarkus-hazelcast-clienthttps://github.com/h...
Tomcat Session ManagerTomcat Session Manager
Each HttpSession object is kept in the Hazelcast Map
https://github.com/hazel...
Apache ShiroApache Shiro
Apache Shiro is a security framework that performs authentication, authorization, cryptography, a...
Envoy ProxyEnvoy Proxy
Apache Shiro is a security framework that performs authentication, authorization, cryptography, and...
Cloud IntegrationsCloud Integrations
Zone-awareness + autodiscoveryZone-awareness + autodiscovery
AWS
GCP
Azure
Hazelcast CloudHazelcast Cloud
Hazelcast IMDG cluster as a serviceHazelcast IMDG cluster as a service
https://cloud.hazelc...
Enterprise FeaturesEnterprise Features
Striim Hot Cache
Management Center
Security Suite
Disaster Recovery
WAN Replication...
Striim Hot CacheStriim Hot Cache
source: Hazelcast
Disaster RecoveryDisaster Recovery
source: Hazelcast
WAN ReplicationWAN Replication
source: Hazelcast
HD Memory(o -heap store)HD Memory(o -heap store)
Hot Restart StoreHot Restart Store
FutureFuture
Hazelcast JetHazelcast Jet
private static Pipeline buildPipeline() {
Pipeline p = Pipeline.create();
p.readFrom(TestSources.itemStream(100, (ts, seq)...
Hazelcast Jet Design DocumentsHazelcast Jet Design Documents
Hazelcast Jet ArchitectureHazelcast Jet Architecture
https://...
TutorialsTutorials
https://training.hazelcast.comhttps://training.hazelcast.com
Want to contribute but don't know where/how to start?Want to contribute but don't know where/how to start?
@jholusa@jholusa
@pivovarit
Thank You!Thank You!
https://pivovarit.github.io/talks/hazelcasthttps://pivovarit.github.io/talks/hazelcast
Twi...
JavaFest. Grzegorz Piwowarek. Hazelcast - Hitchhiker’s Guide
JavaFest. Grzegorz Piwowarek. Hazelcast - Hitchhiker’s Guide
JavaFest. Grzegorz Piwowarek. Hazelcast - Hitchhiker’s Guide
Upcoming SlideShare
Loading in …5
×

JavaFest. Grzegorz Piwowarek. Hazelcast - Hitchhiker’s Guide

Most of you probably heard about the distributed caching being the most common application of Hazelcast - but I bet not many of you heard about CRDT, HyperLogLog, or CP Subsystem.
During this talk, we'll go for quick journey around Hazelcast's ecosystem, revise basic functionality and have a look at some of the hidden flavours.
Even if that’s way too much for your use case, there’s still a lot for an engineer to learn from studying these concepts.

  • Be the first to comment

  • Be the first to like this

JavaFest. Grzegorz Piwowarek. Hazelcast - Hitchhiker’s Guide

  1. 1. HazelcastHazelcast The Hitchhiker's GuideThe Hitchhiker's Guide Grzegorz PiwowarekGrzegorz Piwowarek @pivovarit@pivovarit
  2. 2. {{ }} Senior Software Engineer @Senior Software Engineer @ Trainer @Trainer @ 4comprehension.com4comprehension.com HazelcastHazelcast Bottega IT MindsBottega IT Minds @pivovarit@pivovarit
  3. 3. Born in 2008 in Turkey as a simple caching solution... source: Hazelcast
  4. 4. The Hazelcast IMDG is an operational in-memory computing platform that manages data and distributes processing using memory and parallel execution for breakthrough application speed and scale. source: Hazelcast
  5. 5. source: Hazelcast
  6. 6. source: Hazelcast
  7. 7. source: Hazelcast
  8. 8. IMDGIMDG In-Memory Data GridIn-Memory Data Grid In-Memory (no persistence) Storage/Computation Multiple processes working as a group
  9. 9. Embedded Client-Server
  10. 10. Cache Access PatternsCache Access Patterns Read-through Write-through Write-behind Near Cache
  11. 11. Read-throughRead-through source: Hazelcast
  12. 12. Write-throughWrite-through source: Hazelcast
  13. 13. Write-behindWrite-behind source: Hazelcast
  14. 14. Near CacheNear Cache source: Hazelcast
  15. 15. <dependencies> <dependency> <groupid>com.hazelcast</groupid> <artifactid>hazelcast</artifactid> <version>4.0.1</version> </dependency> </dependencies>
  16. 16. ShadedShaded
  17. 17. EmbeddedEmbedded HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
  18. 18. Client-ServerClient-Server HazelcastInstance hazelcastInstance = HazelcastClient.newHazelcastClient();
  19. 19. ClientsClients JavaJava ScalaScala C++C++ C#/.NETC#/.NET PythonPython Node.jsNode.js GoGo Rust (work-in-progress)Rust (work-in-progress) Binary Client ProtocolBinary Client Protocol
  20. 20. // import com.hazelcast.map.IMap; HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); IMap<String, Integer> simpleMap = hazelcastInstance.getMap("simpleMap"); simpleMap.put("foo", 42); simpleMap.get("foo"); // 42
  21. 21. // import java.util.Map; HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); Map<String, Integer> simpleMap = hazelcastInstance.getMap("simpleMap"); simpleMap.put("foo", 42); simpleMap.get("foo"); // 42
  22. 22. source: Hazelcast
  23. 23. Map is just the beginningMap is just the beginning MultiMap ReplicatedMap (non-partitioned Map) Set List Queue Topic
  24. 24. Data Structures is just the beginningData Structures is just the beginning
  25. 25. EntryProcessorEntryProcessor Instead of fetching the data, processing it and sending it back, let a cluster process it. A processed entry remains locked during processing @BinaryInterface @FunctionalInterface public interface EntryProcessor<K, V, R> extends Serializable { R process(Entry<K, V> entry); }
  26. 26. ExecutorServiceExecutorService public interface IExecutorService extends ExecutorService, DistributedObject { void execute(Runnable command, MemberSelector memberSelector); void executeOnAllMembers(Runnable command); // ... }
  27. 27. AggregationAggregation IMap<String, Employee> employees = hz.getMap("employees"); double avgSalary = employees.aggregate( new Aggregator<Map.Entry<String, Employee>, Double>() { protected long sum; protected long count; @Override public void accumulate(Map.Entry<String, Employee> entry) { count++; sum += entry.getValue().getSalaryPerMonth(); } // ... });
  28. 28. QueriesQueries IMap<String, Employee> map = hazelcastInstance.getMap("employee"); Set<Employee> employees = map.values(new SqlPredicate("active AND age < 30"));
  29. 29. CAP TheoremCAP Theorem ConsistencyConsistency AvailabilityAvailability Partition TolerancePartition Tolerance Pick two.
  30. 30. source: Hazelcast
  31. 31. CP SubsystemCP Subsystem source: Hazelcast Linearizable, distributed implementations of Java concurrency primitives Raft-based(http://thesecretlivesofdata.com/raft/) Jepsen-veri ed Requires minimum 3 members
  32. 32. Semaphore CountDownLatch AtomicReference AtomicLong
  33. 33. Advanced Data StructuresAdvanced Data Structures
  34. 34. RingBu erRingBu er A xed-capacity data structure that overwrites oldest elements when capacity is exceeded
  35. 35. HyperLogLogHyperLogLog A probabilistic data structure used to estimate the cardinality(number of unique elements) of a data set. The HyperLogLog algorithm can estimate cardinalities well beyond 10^9 with a relative accuracy (standard error) of 2% while only using 1.5kb of memory. Fangjin Yang Fast, Cheap, and 98% Right: Cardinality Estimation for Big Data
  36. 36. Flake Id GeneratorFlake Id Generator Cluster-wide ordered and unique identi ers IDs without any coordination between members safe even in split-brain scenario
  37. 37. CRDT PN CounterCRDT PN Counter A con ict-free alternative to AtomicLong
  38. 38. Other IntegrationsOther Integrations
  39. 39. Hibernate 2LCHibernate 2LC Hazelcast provides a distributed second level cache for your Hibernate entities, collections and queries. http://github.com/hazelcast/hazelcast-hibernate5/http://github.com/hazelcast/hazelcast-hibernate5/
  40. 40. Spring DataSpring Data Spring Data Hazelcast Integration https://github.com/hazelcast/spring-data-hazelcasthttps://github.com/hazelcast/spring-data-hazelcast
  41. 41. Quarkus Hazelcast ClientQuarkus Hazelcast Client https://github.com/hazelcast/quarkus-hazelcast-clienthttps://github.com/hazelcast/quarkus-hazelcast-client
  42. 42. Tomcat Session ManagerTomcat Session Manager Each HttpSession object is kept in the Hazelcast Map https://github.com/hazelcast/hazelcast-tomcat-sessionmanagerhttps://github.com/hazelcast/hazelcast-tomcat-sessionmanager
  43. 43. Apache ShiroApache Shiro Apache Shiro is a security framework that performs authentication, authorization, cryptography, and session management. Shiro uses Hazelcast for caching and session clustering.
  44. 44. Envoy ProxyEnvoy Proxy Apache Shiro is a security framework that performs authentication, authorization, cryptography, and session management. Shiro uses Hazelcast for caching and session clustering. https://github.com/envoyproxy/envoy/pull/10536https://github.com/envoyproxy/envoy/pull/10536
  45. 45. Cloud IntegrationsCloud Integrations Zone-awareness + autodiscoveryZone-awareness + autodiscovery AWS GCP Azure
  46. 46. Hazelcast CloudHazelcast Cloud Hazelcast IMDG cluster as a serviceHazelcast IMDG cluster as a service https://cloud.hazelcast.comhttps://cloud.hazelcast.com
  47. 47. Enterprise FeaturesEnterprise Features Striim Hot Cache Management Center Security Suite Disaster Recovery WAN Replication Rolling Upgrades HD Memory
  48. 48. Striim Hot CacheStriim Hot Cache source: Hazelcast
  49. 49. Disaster RecoveryDisaster Recovery source: Hazelcast
  50. 50. WAN ReplicationWAN Replication source: Hazelcast
  51. 51. HD Memory(o -heap store)HD Memory(o -heap store) Hot Restart StoreHot Restart Store
  52. 52. FutureFuture
  53. 53. Hazelcast JetHazelcast Jet
  54. 54. private static Pipeline buildPipeline() { Pipeline p = Pipeline.create(); p.readFrom(TestSources.itemStream(100, (ts, seq) -> nextRandomNumber())) .withIngestionTimestamps() .window(WindowDefinition.tumbling(1000)) .aggregate(AggregateOperations.topN(TOP, ComparatorEx.comparingLong(l -> l))) .map(WindowResult::result) .writeTo(Sinks.observable(RESULTS)); return p; }
  55. 55. Hazelcast Jet Design DocumentsHazelcast Jet Design Documents Hazelcast Jet ArchitectureHazelcast Jet Architecture https://jet-start.sh/docs/design-docs/https://jet-start.sh/docs/design-docs/ https://jet-start.sh/docs/architecture/distributed-computinghttps://jet-start.sh/docs/architecture/distributed-computing
  56. 56. TutorialsTutorials https://training.hazelcast.comhttps://training.hazelcast.com
  57. 57. Want to contribute but don't know where/how to start?Want to contribute but don't know where/how to start? @jholusa@jholusa
  58. 58. @pivovarit Thank You!Thank You! https://pivovarit.github.io/talks/hazelcasthttps://pivovarit.github.io/talks/hazelcast Twitter: @pivovaritTwitter: @pivovarit

×