• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Hazelcast and MongoDB at Cloud CMS
 

Hazelcast and MongoDB at Cloud CMS

on

  • 3,884 views

This is a presentation given on October 24 by Michael Uzquiano of Cloud CMS (http://www.cloudcms.com) at the MongoDB Boston conference. ...

This is a presentation given on October 24 by Michael Uzquiano of Cloud CMS (http://www.cloudcms.com) at the MongoDB Boston conference.

In this presentation, we cover Hazelcast - an in-memory data grid that provides distributed object persistence across multiple nodes in a cluster. When backed by MongoDB, objects are naturally written to Mongo by Hazelcast. The integration points are clean and easy to implement.

We cover a few simple cases along with code samples to provide the MongoDB community with some ideas of how to integrate Hazelcast into their own MongoDB Java applications.

Statistics

Views

Total Views
3,884
Views on SlideShare
3,735
Embed Views
149

Actions

Likes
6
Downloads
64
Comments
1

8 Embeds 149

https://twitter.com 128
http://localhost 8
http://cloudcms.tumblr.com 4
https://si0.twimg.com 3
http://assets.txmblr.com 3
http://twitter.com 1
https://www.cloudcms.com 1
http://www.gitanasoft.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Hazelcast and MongoDB at Cloud CMS Hazelcast and MongoDB at Cloud CMS Presentation Transcript

    • Distributed Caching with Hazelcast + MongoDB at Cloud CMS Michael Uzquiano uzi@cloudcms.com @uzquiano
    • Agenda• What is Cloud CMS?• Why we chose MongoDB• What is Hazelcast?• Code Samples• Implementation with MongoDB
    • http://www.cloudcms.com
    • • The fastest, easiest and most cost-effective way to build cloud-connected web and mobile applications.• An application server in the cloud for cloud- connected applications• Built to leverage the strengths of MongoDB• Hosted or On-Premise
    • Mobile Apps
    • Touch Apps
    • Application Experiences
    • Consumer Experiences
    • Cloud CMS provides• Content Management• Users, Groups, Roles, Permissions• Personalization (Behavioral Targeting)• Analytics, Reporting• Identity Management• Integrated Services• Email Campaigns, CRM, Integrated Billing
    • Silos
    • Keep things cost-effective
    • Mobile Ready• iOS, Android, Windows Mobile• JavaScript, Java, PHP, Ruby, Node.js• jQuery, jQuery Mobile, Dojo, YUI, Sencha Touch, Titanium, PhoneGap
    • The right tool for the job• JSON• Query and Indexing• Doesn’t overstep into application domain • No transactions • No referential integrity • No triggers, foreign keys, procedures• Gets out of the way so we can tackle these
    • Community + Momentum• Really great language drivers• Community contributors• Frequent release schedule• Exciting roadmap
    • Performance• Very fast • Anywhere from 2 to 10x faster than MySQL • About 50 times faster than CouchDB • Lots of benchmarks but the point is, it’s fast!• Sharding built-in, automatic, and *Just Works™ • *Just Works™ guarantee applies only if you have a cluster of shard replica sets with config servers and routing servers and you define your own shard key(s) with appropriate uniformity and granularity• Asynchronous replication for redundancy/failover
    • What is Hazelcast?• In-Memory Data Grid (IMDG)• Clustering and highly scalable data distribution solution for Java• Distributed Data Structures for Java• Distributed Hashtable (DHT) and more
    • What does Hazelcast do?• Scale your application• Share data across cluster• Partition your data• Send/receive messages• Balance load across cluster• Process in parallel on many JVM
    • Advantages• Open source (Apache License)• Super light, simple, no-dependency• Distributed/partitioned implementation of map, queue, set, list, lock and executor service• Transactional (JCA support)• Topic for pub/sub messaging• Cluster info and membership events• Dynamic clustering, backup, fail-over
    • Data Partitioning in a Cluster If you have 5 million objects in your 5-node cluster, then each node will carry 1 million objects and 1 million backup objects.Server1 Server2 Server3 Server4 Server5
    • SuperClient in a Cluster • -Dhazelcast.super.client=true • As fast as any member in the cluster • Holds no-dataServer1 Server2 Server3 Server4 Server5
    • Code Samples
    • Code Samples – Cluster Interfaceimportcom.hazelcast.core.*;importjava.util.Set;Cluster cluster = Hazelcast.getCluster();cluster.addMembershipListener(listener);Member localMember = cluster.getLocalMember();System.out.println (localMember.getInetAddress());Set setMembers = cluster.getMembers();
    • Code Samples – Distributed Mapimportcom.hazelcast.core.Hazelcast;importjava.util.Map;Map<String, User>map = Hazelcast.getMap(”users");map.put ("1", user);User user = map.get("1");
    • Code Samples – Distributed Queueimportcom.hazelcast.core.Hazelcast;importjava.util.concurrent.BlockingQueue;importjava.util.concurrent.TimeUnit;BlockingQueue<Task>queue = Hazelcast.getQueue(“tasks");queue.offer(task);Task t = queue.poll();Task t = queue.poll(5, TimeUnit.SECONDS);
    • Code Samples – Distributed Setimportcom.hazelcast.core.Hazelcast;importjava.util.Set;Set<Price>set= Hazelcast.getSet(“IBM-Quote-History");set.add (new Price (10, time1));set.add (new Price (11, time2));set.add (new Price (13, time3));for (Price price : set) {// process price}
    • Code Samples – Distributed Lockimportcom.hazelcast.core.Hazelcast;importjava.util.concurrent.locks.Lock;Lockmylock= Hazelcast.getLock(mylockobject);mylock.lock();try {// do something} finally {mylock.unlock();}
    • Code Samples – Distributed Topicimportcom.hazelcast.core.*;public class Sample implements MessageListener { public static void main(String[] args) { Sample sample = new Sample(); Topic topic = Hazelcast.getTopic ("default"); topic.addMessageListener(sample); topic.publish ("my-message-object"); } public void onMessage(Object msg) { System.out.println("Got msg :" + msg); }}
    • Code Samples – Distributed Eventsimportcom.hazelcast.core.IMap;importcom.hazelcast.core.Hazelcast;importcom.hazelcast.core.EntryListener;importcom.hazelcast.core.EntryEvent;publicclassSampleimplementsEntryListener{publicstaticvoidmain(String[]args){ Sample sample =newSample(); IMap map =Hazelcast.getMap("default"); map.addEntryListener(sample,true); map.addEntryListener(sample,"key"); } publicvoidentryAdded(EntryEventevent){System.out.println("Added "+event.getKey()+":"+event.getValue()); } publicvoidentryRemoved(EntryEventevent){ System.out.println("Removed "+event.getKey()+":"+event.getValue()); } publicvoidentryUpdated(EntryEventevent){ System.out.println("Updated "+event.getKey()+":"+event.getValue()); }}
    • Code Samples – Executor ServiceFutureTask<String>futureTask= newDistributedTask<String>(new Echo(input), member);ExecutorServicees=Hazelcast.getExecutorService();es.execute(futureTask);String result = futureTask.get();
    • Sample Configuration<hazelcast> <group> <name>dev</name> <password>dev-pass</password> </group> <network> <portauto-increment="true">5701</port> <join> <multicastenabled="true"> <multicast-group>224.2.2.3</multicast-group> <multicast-port>54327</multicast-port> </multicast> <tcp-ipenabled="false"> <interface>192.168.1.2-5</interface><hostname>istanbul.acme</hostname> </tcp-ip> </join> <interfacesenabled="false"> <interface>10.3.17.*</interface> </interfaces> </network> <executor-service> <core-pool-size>16</core-pool-size> <max-pool-size>64</max-pool-size> <keep-alive-seconds>60</keep-alive-seconds> </executor-service> <queuename="tasks"> <max-size-per-jvm>10000</max-size-per-jvm> </queue></hazelcast>
    • Distributed Job Queue Elasticity with Cloud CMS
    • Distributed Job Queue
    • Distributed Job Queue Upload of a 20 page PDF Write PDF to GridFS Add 2 jobs to Queue (each to build 10 pngs)
    • Distributed Job Queue Upload of a 20 page PDF Write PDF to GridFS Add 2 jobs to Queue (each to build 10 pngs)
    • Distributed Job Queue Upload of a 20 page PDF Write PDF to GridFS Add 2 jobs to Queue (each to build 10 pngs)
    • Distributed Job Queue Upload of a 20 page PDF Write PDF to GridFS Add 2 jobs to Queue (each to build 10 pngs)
    • Distributed Job Queue Jobs may run asynchronously (returns once transaction complete)Picks job from Picks Job fromqueue and works on it queue and works on it Job Scheduler determines which jobs get priority
    • Distributed Job Queue
    • Implementation with MongoDB com.hazelcast.core.MapStore
    • MapStorepublic interface MapStore<K,V> extends MapLoader<K,V>{ void store(Kk, V v); void storeAll(Map<K,V>kvMap); void delete(Kk); void deleteAll(Collection<K>ks);}public interface MapLoader<K,V>{ V load(Kk); Map<K,V>loadAll(Collection<K>ks); Set<K>loadAllKeys();}
    • MapLoaderLifecycleSupportpublic interface MapLoaderLifecycleSupport{ void init(HazelcastInstancehazelcastInstance, Properties properties, String mapName); void destroy(); Set<K>loadAllKeys();}
    • public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{}
    • public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ private Mongo mongo; private DBCollectioncol; public void init(HazelcastInstancehazelcastInstance, Properties properties, String mapName) {this.mongo = new Mongo(“localhost”, 27017); String dbname = properties.get(“db”); String cname = properties.get(“collection”); DB db = this.mongo.getDB(dbname);this.col = db.getCollection(cname); }}
    • public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ private Mongo mongo; private DBCollectioncol; ... public void destroy() {this.mongo.close(); } public Set<K>loadAllKeys() { return null; }}
    • public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ private Mongo mongo; private DBCollectioncol; ... public Set loadAllKeys() { Set keys = new HashSet();BasicDBList fields = new BasicDBList();fields.add(“_id”);DBCursor cursor = this.col.find(null, fields); while (cursor.hasNext()) {keys.add(cursor.next().get(“_id”)); } return keys; }}
    • public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ ... public void store(Kk, V v) {DBObjectdbObject = convert(v);dbObject.put(“_id”, k);this.col.save(dbObject); } public void delete(Kk) {DBObjectdbObject = new BasicDBObject();dbObject.put(“_id”, k);this.col.remove(dbObject); }}
    • public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ ... public void storeAll(Map map) { for (Object key : map.keySet()) {store(key, map.get(key)); } } public void deleteAll(Collection keys) { for (Object key: keys) {delete(key); } }}
    • public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ ... public void storeAll(Map map) { for (Object key : map.keySet()) {store(key, map.get(key)); } } public void deleteAll(Collection keys) {BasicDBListdbo = new BasicDBList(); for (Object key : keys) {dbo.add(newBasicDBObject("_id", key)); }BasicDBObjectdbb = new BasicDBObject("$or", dbo);this.col.remove(dbb); }}
    • Spring Config<beans xmlns="http://www.springframework.org/schema/beans"xmlns:hz="http://www.hazelcast.com/schema/spring"><hz:hazelcast id="hzInstance”><hz:config><hz:map name=”users” backup-count="1" max-size="2000” eviction-percentage="25" eviction-policy="LRU" merge-policy="hz.LATEST_UPDATE"><hz:map-store enabled="true" write-delay-seconds="0" implementation="mymap" /></hz:map></hz:config></hz:hazelcast><bean id=”mymap” class="org.sample.MongoUsersMapStore" />
    • Implementation with MongoDB com.hazelcast.core.EntryListener
    • EntryListenerpublic interface EntryListener<K,V>{ void entryAdded(EntryEvent<K, V> event); void entryUpdated(EntryEvent<K, V> event); void entryRemoved(EntryEvent<K, V> event); void entryEvicted(EntryEvent<K, V> event);}
    • EntryListenerpublic class UserCacheEntryListener<String, User>{ private Map<String, User> cache; public User getUser(String key) { return cache.get(key); } public void entryAdded(EntryEvent<String, User> event) {cache.put(event.getKey(), event.getValue()); }}
    • EntryListener
    • Spring Framework• Spring Data for MongoDB • http://www.springsource.org/spring-data/mongodb • com.hazelcast.spring.mongodb.MongoMapStore• Based on Spring API Template pattern • com.hazelcast.spring.mongodb.MongoTemplate• Easy to get started, base implementation• You still might want to roll your own
    • Questions?• Michael Uzquiano • uzi@cloudcms.com • @uzquiano• Cloud CMS • http://www.cloudcms.com • @cloudcms• Hazelcast • http://www.hazelcast.com • https://github.com/hazelcast/hazelcast