Hazelcast - In-Memory DataGrid


Published on

Hazelcast is an easy to use but scalable in-memory datagrid and distributed executor framework. It enables you to build applications having a big requirement on memory or that needs to scale horizontally.

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Hazelcast - In-Memory DataGrid

  1. 1. HAZELCAST 3 IN-MEMORY DATAGRID www.hazelcast.com
  2. 2. WHO AM I Christoph Engelbert (@noctarius2k) 8+ years of professional Java development 5+ years of backend development Specialized to performance, GC, traffic topics Was working for int. companies as Ubisoft and HRS Since November 2013 official Hazelcast Hacker Apache DirectMemory / Lightning Committer and PMC Developer CastMapR - MapReduce on Hazelcast 3 www.hazelcast.com
  3. 3. HAZELCAST IN KEYWORDS In-Memory DataGrid Distributed Cache Distributed Execution Framework NoSQL Clustering Scalability Partitioning Cloud Ready OpenSource - Apache License 2.0 www.hazelcast.com
  4. 4. WHO USES HAZELCAST? and many more ... www.hazelcast.com
  5. 5. USECASES Scale your application Distribute and share data Partition your data Distribute messages Process in parallel on multiple machines Load balancing www.hazelcast.com
  6. 6. ALTERNATIVES? Oracle Coherence IBM eXtreme Scale VMware Gemfire Gigaspaces Redhat Infinispan Gridgain Terracotta www.hazelcast.com
  7. 7. DIFFERENCES? Easy to use License / Cost Lightweight Features www.hazelcast.com
  8. 8. FEATURES Java Collection API Map, Queue, Set, List MultiMap Topic (PubSub) Java Concurrency API Lock, Semaphore, CountDownLatch, ExecutorService Transactions Custom Serialization Off-Heap support Native client: C#, C++, Java, REST, memcached www.hazelcast.com
  9. 9. EASY API / Cetn anwHzlatnd / raig e aecs oe Hzlatntneh =HzlatnwaecsIsac(; aecsIsac z aecs.eHzlatntne) / GtigaMp Ls,Tpc .. / etn a, it oi, . Mpmp=h.eMp"aNm"; a a zgta(Mpae) Ls ls =h.eLs(LsNm"; it it zgtit"itae) Ioi tpc=h.eTpc"oiNm"; Tpc oi zgtoi(Tpcae) / Sutn dw tend / htig on h oe h.hton) zsudw(; www.hazelcast.com
  10. 10. HOW DOES IT WORK? www.hazelcast.com
  11. 11. DATA PARTITIONING (1/2) Multiple partitions per node Consistent Hashing: hash(key) % partitioncount Option to control partitioning: "key@partitionkey" Possibility to find key owner for every key Support for Near-Caching and executions on key owner Automatic Fault-Tolerance Synchronous and Asynchronous backups Define sync / async backup counts www.hazelcast.com
  12. 12. DATA PARTITIONING (2/2) With 4 cluster nodes every server holds 1/4 real data and 1/4 of backups www.hazelcast.com
  13. 13. A HAZELCAST NETWORK www.hazelcast.com
  14. 14. HAZELCAST IN NUMBERS Default partition amount 271 Any partition amount possible Biggest cluster 100+ members Handles 100k+/sec messages using a topic Max datasize depends on RAM Off-Heap for low GC overhead www.hazelcast.com
  15. 15. COMMUNITY VS. ENTERPRISE Feature Java Collection API Java Concurrency API SSL Socket Elastic Memory (Off-Heap) JAAS Security / Authentication Management Center Community X X X Enterprise X X X X X X www.hazelcast.com
  16. 16. CODE SAMPLES www.hazelcast.com
  17. 17. EASY TO UNITTEST pbi casSmTsCs { ulc ls oeetae piaeHzlatntne]isacs rvt aecsIsac[ ntne; @eoe Bfr pbi vi bfr( trw Ecpin{ ulc od eoe) hos xeto / Mlil isacso tesm JM / utpe ntne n h ae V isacs=nwHzlatntne2; ntne e aecsIsac[] isacs0 =HzlatnwaecsIsac(; ntne[] aecs.eHzlatntne) isacs1 =HzlatnwaecsIsac(; ntne[] aecs.eHzlatntne) } @fe Atr pbi vi atr)trw Ecpin{ ulc od fe( hos xeto HzlatsudwAl) aecs.htonl(; } } www.hazelcast.com
  18. 18. SERIALIZATION / jv.oSraial / aai.eilzbe pbi casUe ipeet Sraial { ulc ls sr mlmns eilzbe } / o jv.oEtraial / r aai.xenlzbe pbi casUe ipeet Etraial { ulc ls sr mlmns xenlzbe } / o (o.aecs.i.eilzto)DtSraial / r cmhzlatnosraiain.aaeilzbe pbi casUe ipeet DtSraial { ulc ls sr mlmns aaeilzbe } / o nwi Hzlat3(ut vrinspot Pral / r e n aecs mli eso upr) otbe pbi casUe ipeet Pral { ulc ls sr mlmns otbe } www.hazelcast.com
  19. 19. MAP itraecmhzlatcr.MpK V nefc o.aecs.oeIa<, > etnsjv.tlMp jv.tlCnurnMp xed aaui.a, aaui.ocreta Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) Ia<tig Ue>hMp=h.eMp"sr"; MpSrn, sr za zgta(ues) hMppt"ee" nwUe(Ptr,"enjr); za.u(Ptr, e sr"ee" Vete") MpSrn,Ue>mp=h.eMp"sr"; a<tig sr a zgta(ues) mppt"ee" nwUe(Ptr,"enjr); a.u(Ptr, e sr"ee" Vete") CnurnMpSrn,Ue>cnurnMp=h.eMp"sr"; ocreta<tig sr ocreta zgta(ues) cnurnMpptfbet"ee" nwUe(Ptr,"enjr); ocreta.uIAsn(Ptr, e sr"ee" Vete") Ue ptr=mpgt"ee"; sr ee a.e(Ptr) www.hazelcast.com
  20. 20. LIST itraecmhzlatcr.Ls<> nefc o.aecs.oeIitE etnsjv.tlLs xed aaui.it Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) Ls<sr ls =h.eLs(ues) i ​tUe> ​t i zgtit"sr"; ls.d(e Ue(Ptr,"enjr); i ​tadnw sr"ee" Vete") Ue ptr=ls.e() sr ee itgt0; www.hazelcast.com
  21. 21. QUEUE itraecmhzlatcr.Ls<> nefc o.aecs.oeIitE etnsjv.tlLs xed aaui.it Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) BoknQeeTs>qee=h.eQee"ak"; lciguu<ak uu zgtuu(tss) qeeofrnwak); uu.fe(eTs() qeeofrnwak) 50 TmUi.ILSCNS; uu.fe(eTs(, 0, ientMLIEOD) Ts ts =qeepl(; ak ak uu.ol) Ts ts =qeepl(0,TmUi.ILSCNS; ak ak uu.ol10 ientMLIEOD) Ts ts =qeetk(; ak ak uu.ae) www.hazelcast.com
  22. 22. LOCK (1/3) itraecmhzlatcr.Lc nefc o.aecs.oeIok etnsjv.tlcnurn.ok.ok xed aaui.ocretlcsLc www.hazelcast.com
  23. 23. LOCK (2/3) Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) / DsrbtdRetat / itiue enrn Lc lc =h.eLc(mLc"; o ​k ​k o zgtok"yok) lc.ok) o ​klc(; ty{ r / D smtig / o oehn }fnly{ ial lc.nok) o ​kulc(; } www.hazelcast.com
  24. 24. LOCK (3/3) Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) / Mp(o-lcs / a Rw)ok Ia<tig Ue>mp=h.eMp"sr"; MpSrn, sr a zgta(ues) mplc(Ptr) a.ok"ee"; ty{ r / D smtigwt Ptr / o oehn ih ee }fnly{ ial mpulc(Ptr) a.nok"ee"; } www.hazelcast.com
  25. 25. TOPIC / PUBSUB pbi casEapeipeet MsaeitnrSrn>{ ulc ls xml mlmns esgLsee<tig pbi vi snMsae{ ulc od edesg Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) Ioi<tig tpc=h.eTpc"oi"; TpcSrn> oi zgtoi(tpc) tpcadesgLsee(hs; oi.dMsaeitnrti) tpcpbih"el Wrd) oi.uls(Hlo ol"; } @vrie Oerd pbi vi oMsaeMsaeSrn>msae { ulc od nesg(esg<tig esg) Sse.u.rnl(Gtmsae "+msaegtesgOjc() ytmotpitn"o esg: esg.eMsaebet); } } www.hazelcast.com
  26. 26. EXECUTORSERVICE pbi itraecmhzlatcr.EeuoSrie ulc nefc o.aecs.oeIxctrevc etnsjv.tlcnurn.xctrevc xed aaui.ocretEeuoSrie Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) Ixctrevc e =h.eEeuoSrie"ae) EeuoSrie s zgtxctrevc(nm"; e.xctOAlebr(ulRnal() seeuenlMmesbidunbe); e.xctOKywe(ulRnal(,"ee"; seeueneOnrbidunbe) Ptr) e.xct(ulRnal() seeuebidunbe); Mp.>ftrs=e.umtolMmesbidalbe); a<. uue ssbiTAlebr(ulClal() Ftr<. ftr =e.umtoeOnrbidalbe) "ee"; uue.> uue ssbiTKywe(ulClal(, Ptr) e.umtolMmesbidalbe) bidalak); ssbiTAlebr(ulClal(, ulClbc() e.umtoeOnrbidalbe) "ee" bidalak); ssbiTKywe(ulClal(, Ptr, ulClbc() www.hazelcast.com
  27. 27. ADVANCED TECHNIQUES www.hazelcast.com
  28. 28. ADVANCED TECHNIQUES Indexing keys, values and value properties Distributed SQL-like query Write-Behind / Write-Through persistence Read-Through (if key not loaded use MapLoader) Transactions EntryListeners / EntryProcessors Automatic eviction Control partitioning (Version 3.1) and many more ... www.hazelcast.com
  29. 29. CODE SAMPLES www.hazelcast.com
  30. 30. INDEXING Ia<tig Ue>mp=Hzlatgta(ues) MpSrn, sr a aecs.eMp"sr"; mpadne(ae,tu) a.dIdx"g" re; / odrd / ree mpadne(atv" fle;/ ntodrd a.dIdx"cie, as) / o ree <a nm=epoes> mp ae"mlye" .. . <nee> idxs <ne odrd"re>g<idx idx ree=tu"ae/ne> <ne odrd"as"nm<idx idx ree=fle>ae/ne> <idxs /nee> <mp /a> www.hazelcast.com
  31. 31. DISTRIBUTED SQL-LIKE QUERIES Ia<tig Ue>mp=Hzlatgta(ues) MpSrn, sr a aecs.eMp"sr"; Peiaepeiae=nwSlrdct(atv ADae< 3"; rdct rdct e qPeiae"cie N g = 0) StUe>ues=mpvle(rdct) e<sr sr a.auspeiae; StEtySrn,Ue> etis=mpetye(rdct) e<nr<tig sr> nre a.nrStpeiae; www.hazelcast.com
  32. 32. MAPLOADER / MAPSTORE pbi casMptrg ulc ls aSoae ipeet Mptr<tig Ue> MpodrSrn,Ue>{ mlmns aSoeSrn, sr, aLae<tig sr / Sm mtosmsig.. / oe ehd isn . @vriepbi Ue la(tigky {rtr laVleBky;} Oerd ulc sr odSrn e) eun odauD(e) @vriepbi StSrn>laAles){rtr laKyD(;} Oerd ulc e<tig odlKy( eun odesB) @vriepbi vi dlt(tigky {dltD(e) } Oerd ulc od eeeSrn e) eeeBky; @vriepbi vi soeSrn ky Ue vle { Oerd ulc od tr(tig e, sr au) soeoaaaeky vle; trTDtbs(e, au) } } <a nm=ues> mp ae"sr" <a-tr eald"re> mpsoe nbe=tu" <ls-aecmhzlateapeMptrg<casnm> casnm>o.aecs.xml.aSoae/ls-ae <rt-ea-eod><wiedlyscns wiedlyscns0/rt-ea-eod> <mpsoe /a-tr> <mp /a> www.hazelcast.com
  33. 33. TRANSACTION (1/2) Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) fnlMpmp=h.eMp"eal"; ia a a zgta(dfut) fnlQeeqee=h.eQee"eal"; ia uu uu zgtuu(dfut) h.xctTascinnwTascinlakVi>){ zeeuernato(e rnatoaTs<od( @vrie Oerd pbi Vi eeueTascinlakotx cnet { ulc od xct(rnatoaTsCnet otx) Tettet=(we)qeepl(; we we Tet uu.ol) poeswe(we) rcsTettet; mpptbide(we) tet; a.u(ulKytet, we) rtr nl; eun ul } }; ) www.hazelcast.com
  34. 34. TRANSACTION (2/2) Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) Tascinotx cnet=h.eTascinotx(; rnatoCnet otx znwrnatoCnet) cnetbgnrnato(; otx.eiTascin) Tascinla mp=cnetgta(dfut) rnatoaMp a otx.eMp"eal"; Tascinluu qee=cnetgtuu(dfut) rnatoaQee uu otx.eQee"eal"; ty{ r Tettet=(we)qeepl(; we we Tet uu.ol) poeswe(we) rcsTettet; mpptbide(we) tet; a.u(ulKytet, we) cnetcmiTascin) otx.omtrnato(; }cth(xeto e { ac Ecpin ) cnetrlbcTascin) otx.olakrnato(; } www.hazelcast.com
  35. 35. CONTROL PARTITIONING Force location of corresponding data in the same partition by providing a special partition key Hzlatntneh =gtaecsIsac(; aecsIsac z eHzlatntne) Mpues=h.eMp"sr"; a sr zgta(ues) uespt"ee@ee" nwUe(Ptr,"enjr); sr.u(PtrPtr, e sr"ee" Vete") Mpfins=h.eMp"red"; a red zgta(fins) finspt"ee-hi@ee" nwUe(Crsoh,"nebr") red.u(PtrCrsPtr, e sr"hitp" Eglet); finspt"ee-udPtr,nwUe(Fa" "aio") red.u(PtrFa@ee" e sr"ud, Mlkv); www.hazelcast.com
  36. 36. WHAT ELSE? www.hazelcast.com
  37. 37. SPI (NEW IN HAZELCAST 3) Possibility to build own distributed datastructures Hook into datastructure events Implement your own services (like RemoteInvocation, MapReduce) React on membership events Manipulate migrations on your purpose Handle splitbrain events and many more ... www.hazelcast.com
  38. 38. EXTERNAL ADDONS hazelgrails - Hazelcast integration for Grails hazelmr / castmapr - MapReduce for Hazelcast 2.x / 3.x hazelblast - Remote Invocation hazelcast-actors - Actor framework maybe your's next? www.hazelcast.com
  39. 39. THANK YOU! ANY QUESTIONS? @noctarius2k @hazelcast hazelcast@googlegroups.com http://www.hazelcast.com http://github.com/hazelcast/hazelcast Images: www.clipartist.info, Gnome Nebula Theme, KDE theme www.hazelcast.com