Hazelcast and MongoDB at Cloud CMS

5,181 views

Published on

This is a presentation given on October 24 by Michael Uzquiano of Cloud CMS (http://www.cloudcms.com) at the MongoDB Boston conference.

In this presentation, we cover Hazelcast - an in-memory data grid that provides distributed object persistence across multiple nodes in a cluster. When backed by MongoDB, objects are naturally written to Mongo by Hazelcast. The integration points are clean and easy to implement.

We cover a few simple cases along with code samples to provide the MongoDB community with some ideas of how to integrate Hazelcast into their own MongoDB Java applications.

Published in: Technology
1 Comment
8 Likes
Statistics
Notes
No Downloads
Views
Total views
5,181
On SlideShare
0
From Embeds
0
Number of Embeds
168
Actions
Shares
0
Downloads
88
Comments
1
Likes
8
Embeds 0
No embeds

No notes for slide

Hazelcast and MongoDB at Cloud CMS

  1. 1. Distributed Caching with Hazelcast + MongoDB at Cloud CMS Michael Uzquiano uzi@cloudcms.com @uzquiano
  2. 2. Agenda• What is Cloud CMS?• Why we chose MongoDB• What is Hazelcast?• Code Samples• Implementation with MongoDB
  3. 3. http://www.cloudcms.com
  4. 4. • The fastest, easiest and most cost-effective way to build cloud-connected web and mobile applications.• An application server in the cloud for cloud- connected applications• Built to leverage the strengths of MongoDB• Hosted or On-Premise
  5. 5. Mobile Apps
  6. 6. Touch Apps
  7. 7. Application Experiences
  8. 8. Consumer Experiences
  9. 9. Cloud CMS provides• Content Management• Users, Groups, Roles, Permissions• Personalization (Behavioral Targeting)• Analytics, Reporting• Identity Management• Integrated Services• Email Campaigns, CRM, Integrated Billing
  10. 10. Silos
  11. 11. Keep things cost-effective
  12. 12. Mobile Ready• iOS, Android, Windows Mobile• JavaScript, Java, PHP, Ruby, Node.js• jQuery, jQuery Mobile, Dojo, YUI, Sencha Touch, Titanium, PhoneGap
  13. 13. The right tool for the job• JSON• Query and Indexing• Doesn’t overstep into application domain • No transactions • No referential integrity • No triggers, foreign keys, procedures• Gets out of the way so we can tackle these
  14. 14. Community + Momentum• Really great language drivers• Community contributors• Frequent release schedule• Exciting roadmap
  15. 15. Performance• Very fast • Anywhere from 2 to 10x faster than MySQL • About 50 times faster than CouchDB • Lots of benchmarks but the point is, it’s fast!• Sharding built-in, automatic, and *Just Works™ • *Just Works™ guarantee applies only if you have a cluster of shard replica sets with config servers and routing servers and you define your own shard key(s) with appropriate uniformity and granularity• Asynchronous replication for redundancy/failover
  16. 16. What is Hazelcast?• In-Memory Data Grid (IMDG)• Clustering and highly scalable data distribution solution for Java• Distributed Data Structures for Java• Distributed Hashtable (DHT) and more
  17. 17. What does Hazelcast do?• Scale your application• Share data across cluster• Partition your data• Send/receive messages• Balance load across cluster• Process in parallel on many JVM
  18. 18. Advantages• Open source (Apache License)• Super light, simple, no-dependency• Distributed/partitioned implementation of map, queue, set, list, lock and executor service• Transactional (JCA support)• Topic for pub/sub messaging• Cluster info and membership events• Dynamic clustering, backup, fail-over
  19. 19. Data Partitioning in a Cluster If you have 5 million objects in your 5-node cluster, then each node will carry 1 million objects and 1 million backup objects.Server1 Server2 Server3 Server4 Server5
  20. 20. SuperClient in a Cluster • -Dhazelcast.super.client=true • As fast as any member in the cluster • Holds no-dataServer1 Server2 Server3 Server4 Server5
  21. 21. Code Samples
  22. 22. Code Samples – Cluster Interfaceimportcom.hazelcast.core.*;importjava.util.Set;Cluster cluster = Hazelcast.getCluster();cluster.addMembershipListener(listener);Member localMember = cluster.getLocalMember();System.out.println (localMember.getInetAddress());Set setMembers = cluster.getMembers();
  23. 23. Code Samples – Distributed Mapimportcom.hazelcast.core.Hazelcast;importjava.util.Map;Map<String, User>map = Hazelcast.getMap(”users");map.put ("1", user);User user = map.get("1");
  24. 24. Code Samples – Distributed Queueimportcom.hazelcast.core.Hazelcast;importjava.util.concurrent.BlockingQueue;importjava.util.concurrent.TimeUnit;BlockingQueue<Task>queue = Hazelcast.getQueue(“tasks");queue.offer(task);Task t = queue.poll();Task t = queue.poll(5, TimeUnit.SECONDS);
  25. 25. Code Samples – Distributed Setimportcom.hazelcast.core.Hazelcast;importjava.util.Set;Set<Price>set= Hazelcast.getSet(“IBM-Quote-History");set.add (new Price (10, time1));set.add (new Price (11, time2));set.add (new Price (13, time3));for (Price price : set) {// process price}
  26. 26. Code Samples – Distributed Lockimportcom.hazelcast.core.Hazelcast;importjava.util.concurrent.locks.Lock;Lockmylock= Hazelcast.getLock(mylockobject);mylock.lock();try {// do something} finally {mylock.unlock();}
  27. 27. Code Samples – Distributed Topicimportcom.hazelcast.core.*;public class Sample implements MessageListener { public static void main(String[] args) { Sample sample = new Sample(); Topic topic = Hazelcast.getTopic ("default"); topic.addMessageListener(sample); topic.publish ("my-message-object"); } public void onMessage(Object msg) { System.out.println("Got msg :" + msg); }}
  28. 28. Code Samples – Distributed Eventsimportcom.hazelcast.core.IMap;importcom.hazelcast.core.Hazelcast;importcom.hazelcast.core.EntryListener;importcom.hazelcast.core.EntryEvent;publicclassSampleimplementsEntryListener{publicstaticvoidmain(String[]args){ Sample sample =newSample(); IMap map =Hazelcast.getMap("default"); map.addEntryListener(sample,true); map.addEntryListener(sample,"key"); } publicvoidentryAdded(EntryEventevent){System.out.println("Added "+event.getKey()+":"+event.getValue()); } publicvoidentryRemoved(EntryEventevent){ System.out.println("Removed "+event.getKey()+":"+event.getValue()); } publicvoidentryUpdated(EntryEventevent){ System.out.println("Updated "+event.getKey()+":"+event.getValue()); }}
  29. 29. Code Samples – Executor ServiceFutureTask<String>futureTask= newDistributedTask<String>(new Echo(input), member);ExecutorServicees=Hazelcast.getExecutorService();es.execute(futureTask);String result = futureTask.get();
  30. 30. Sample Configuration<hazelcast> <group> <name>dev</name> <password>dev-pass</password> </group> <network> <portauto-increment="true">5701</port> <join> <multicastenabled="true"> <multicast-group>224.2.2.3</multicast-group> <multicast-port>54327</multicast-port> </multicast> <tcp-ipenabled="false"> <interface>192.168.1.2-5</interface><hostname>istanbul.acme</hostname> </tcp-ip> </join> <interfacesenabled="false"> <interface>10.3.17.*</interface> </interfaces> </network> <executor-service> <core-pool-size>16</core-pool-size> <max-pool-size>64</max-pool-size> <keep-alive-seconds>60</keep-alive-seconds> </executor-service> <queuename="tasks"> <max-size-per-jvm>10000</max-size-per-jvm> </queue></hazelcast>
  31. 31. Distributed Job Queue Elasticity with Cloud CMS
  32. 32. Distributed Job Queue
  33. 33. Distributed Job Queue Upload of a 20 page PDF Write PDF to GridFS Add 2 jobs to Queue (each to build 10 pngs)
  34. 34. Distributed Job Queue Upload of a 20 page PDF Write PDF to GridFS Add 2 jobs to Queue (each to build 10 pngs)
  35. 35. Distributed Job Queue Upload of a 20 page PDF Write PDF to GridFS Add 2 jobs to Queue (each to build 10 pngs)
  36. 36. Distributed Job Queue Upload of a 20 page PDF Write PDF to GridFS Add 2 jobs to Queue (each to build 10 pngs)
  37. 37. Distributed Job Queue Jobs may run asynchronously (returns once transaction complete)Picks job from Picks Job fromqueue and works on it queue and works on it Job Scheduler determines which jobs get priority
  38. 38. Distributed Job Queue
  39. 39. Implementation with MongoDB com.hazelcast.core.MapStore
  40. 40. MapStorepublic interface MapStore<K,V> extends MapLoader<K,V>{ void store(Kk, V v); void storeAll(Map<K,V>kvMap); void delete(Kk); void deleteAll(Collection<K>ks);}public interface MapLoader<K,V>{ V load(Kk); Map<K,V>loadAll(Collection<K>ks); Set<K>loadAllKeys();}
  41. 41. MapLoaderLifecycleSupportpublic interface MapLoaderLifecycleSupport{ void init(HazelcastInstancehazelcastInstance, Properties properties, String mapName); void destroy(); Set<K>loadAllKeys();}
  42. 42. public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{}
  43. 43. public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ private Mongo mongo; private DBCollectioncol; public void init(HazelcastInstancehazelcastInstance, Properties properties, String mapName) {this.mongo = new Mongo(“localhost”, 27017); String dbname = properties.get(“db”); String cname = properties.get(“collection”); DB db = this.mongo.getDB(dbname);this.col = db.getCollection(cname); }}
  44. 44. public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ private Mongo mongo; private DBCollectioncol; ... public void destroy() {this.mongo.close(); } public Set<K>loadAllKeys() { return null; }}
  45. 45. public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ private Mongo mongo; private DBCollectioncol; ... public Set loadAllKeys() { Set keys = new HashSet();BasicDBList fields = new BasicDBList();fields.add(“_id”);DBCursor cursor = this.col.find(null, fields); while (cursor.hasNext()) {keys.add(cursor.next().get(“_id”)); } return keys; }}
  46. 46. public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ ... public void store(Kk, V v) {DBObjectdbObject = convert(v);dbObject.put(“_id”, k);this.col.save(dbObject); } public void delete(Kk) {DBObjectdbObject = new BasicDBObject();dbObject.put(“_id”, k);this.col.remove(dbObject); }}
  47. 47. public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ ... public void storeAll(Map map) { for (Object key : map.keySet()) {store(key, map.get(key)); } } public void deleteAll(Collection keys) { for (Object key: keys) {delete(key); } }}
  48. 48. public class MongoUsersMapStore implements MapStore, MapLoaderLifecycleSupport{ ... public void storeAll(Map map) { for (Object key : map.keySet()) {store(key, map.get(key)); } } public void deleteAll(Collection keys) {BasicDBListdbo = new BasicDBList(); for (Object key : keys) {dbo.add(newBasicDBObject("_id", key)); }BasicDBObjectdbb = new BasicDBObject("$or", dbo);this.col.remove(dbb); }}
  49. 49. Spring Config<beans xmlns="http://www.springframework.org/schema/beans"xmlns:hz="http://www.hazelcast.com/schema/spring"><hz:hazelcast id="hzInstance”><hz:config><hz:map name=”users” backup-count="1" max-size="2000” eviction-percentage="25" eviction-policy="LRU" merge-policy="hz.LATEST_UPDATE"><hz:map-store enabled="true" write-delay-seconds="0" implementation="mymap" /></hz:map></hz:config></hz:hazelcast><bean id=”mymap” class="org.sample.MongoUsersMapStore" />
  50. 50. Implementation with MongoDB com.hazelcast.core.EntryListener
  51. 51. EntryListenerpublic interface EntryListener<K,V>{ void entryAdded(EntryEvent<K, V> event); void entryUpdated(EntryEvent<K, V> event); void entryRemoved(EntryEvent<K, V> event); void entryEvicted(EntryEvent<K, V> event);}
  52. 52. EntryListenerpublic class UserCacheEntryListener<String, User>{ private Map<String, User> cache; public User getUser(String key) { return cache.get(key); } public void entryAdded(EntryEvent<String, User> event) {cache.put(event.getKey(), event.getValue()); }}
  53. 53. EntryListener
  54. 54. Spring Framework• Spring Data for MongoDB • http://www.springsource.org/spring-data/mongodb • com.hazelcast.spring.mongodb.MongoMapStore• Based on Spring API Template pattern • com.hazelcast.spring.mongodb.MongoTemplate• Easy to get started, base implementation• You still might want to roll your own
  55. 55. Questions?• Michael Uzquiano • uzi@cloudcms.com • @uzquiano• Cloud CMS • http://www.cloudcms.com • @cloudcms• Hazelcast • http://www.hazelcast.com • https://github.com/hazelcast/hazelcast

×