Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Infinispan, transactional key value data grid and nosql database

1,189 views

Published on

Infinispan use case presentation made for BAKSIA meetup group.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Infinispan, transactional key value data grid and nosql database

  1. 1. • Sr. Consultant at Inmeta Consulting • Current project: Skattetaten Grid POC • Previous projects involving grid technologies: • Mattilsynet food authority system. • FrameSolution BPM framework used in Lovisa National Court Authority(Norway), Mattilsynet Food Authority • Other noteworthy projects • Coca Cola Basis ERP system – Coca Cola Bottler factories • mPower Mobilitec 300 million subscribers worldwide, and delivers over 500,000 pieces of content every day.
  2. 2. • Big data, Databases are slow. Memory is FAST! • Provides huge computing power. • Tax calculation  • Financial organizations • Government organizations use it for communication and data sharing between the different departments. • Scientific computations • MMORPG games
  3. 3. • General terminology relevant to Distributed Caching • Challenges related to introducing distributed caching to existing system • Metrics and tuning
  4. 4. • Cache JSR – 107 • Java Data Grid JSR - 347 • In memory Data Grid • Cluster • Distribution • Node – a member of a cluster • Transaction awareness • Colocation • Map / Reduce • Consistency
  5. 5. • Transaction scope • Lockingdeadlocking • Flushing policies • Mixing the technology stack. • Performance
  6. 6. • Wow we did it!
  7. 7. • Our Custom cache is super fast, but its cache hit ratio is rather low. • Our custom cache has a tendency of getting dirty as the updates to the shared data can not be propagated. At the same time the separation of the data regions is not full. • Marshaling is a rather slow and heavy process. • We are facing a technological cocktail and we need to keep integrity.
  8. 8. • Write through • Write Behind • Replication Queue
  9. 9. • Eviction • Least Recently Used • First In First Out • LIRS • Custom • Expiration • Invalidation
  10. 10. • Ref. Data vs Transactional • Reference data: Good. Max 30000 reads/sec 1k size • Transactional data: Good. Max 25000 writes/sec 1k size .
  11. 11. • Reference data: Good. 30000 reads/sec per server. Grow linearly by adding servers. • Transactional data: Not so good. Max 20000writes/second. Drops if you add 3rd server to 2500.
  12. 12. • Ref. Data vs Transactional • Reference data: Good. Max 30000 reads/sec 1k size • Transactional data: Good. Max 25000 writes/sec 1k size
  13. 13. • Reference data(1kb):Good. 30000 reads/sec per server. Grow linearly by adding servers. • Transactional data(1kb):Good. 20000 writes/sec per server. Grow linearly by adding servers.
  14. 14. • What is the size of our cluster? Reads vs. Writes • Communication inside our grid • UDP,TCP • Synchronous vs. Asynchronous. • What about the transaction isolation? • Repeatable Reads vs. Read Committed • What is the nature of our application? • Read intensive data • CMS systems • Write Intensive Data • Document Management System
  15. 15. • Level1 cache is Supported only for Distribution mode • Level 1 cache might have a performance Impact in certain systems
  16. 16. • Passivation • Activation • Hibernate
  17. 17. • Long running transactions need to be avoided. • What is a long running transaction? How long is actually long. • Read Committed vs Repeatable Reads
  18. 18. begin Update(A) Update(B) Update(C) Update(B) Begin Update(C) Update(B) Release(A) Lock(A) TX1 (Wants update A,B,C) TX2 (Wants to update C,B,A) C is locked by TX2 A is locked by TX1
  19. 19. begin get(k) - - Get(k) Begin Get(k) put(k, v2) commit What is returned?? TX1 TX2
  20. 20. • Java serialization • Java externalization • Impact on performance • Generic domain.
  21. 21. • Transaction scope • Lockingdeadlocking • Flushing policies • Mixing the technology stack. • Performance
  22. 22. • Wow we did it!
  23. 23. • Thank you for your attention
  24. 24. http://www.alachisoft.com/ncache/caching-topology.html http://www.infoq.com/news/2011/10/java-data-grid https://github.com/datagrids/spec/wiki http://www.jboss.org/infinispan/documentation http://code.google.com/p/thrift-protobuf- compare/wiki/Benchmarking

×