From I/O To RAM


Published on

IO is slow, memory is fast. For many applications, the main performance and scalability bottleneck is disk and network access. This session will cover strategies that can help you utilize your RAM efficiently even in a distributed environment.
Discover how a clustering solution like Terracotta can help you reduce overall application latency.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

From I/O To RAM

  1. 1. Riding The Memory Bus<br />Ophir Radnitz<br />
  2. 2. Agenda<br />RAM vs. IO<br />Tools of the Trade<br />Introducing Terracotta<br />RAM Patterns with Terracotta<br />
  3. 3. Anatomy of an Application<br />Get a DB connection, start transactioning<br />Start a request<br />Network<br />Get a bunch of data<br />Access disk,<br />Serialization<br />Release resources<br />Render stuff<br />
  4. 4. I/O Issues<br />Scalability issues<br />Excessive network chatter<br />How do you scale the database?<br />Only as fast as your slowest query<br />Implies serialization/deserialization<br />
  5. 5. Bandwidths (1 of 2)<br />
  6. 6. Bandwidths (1 of 2)<br />
  7. 7. RAM vs. Disk<br />Internet: ~80ms<br />
  8. 8. RAM Goodness<br />RAM = High Bandwidth & Low Latency<br />No serialization costs<br />Less network<br />
  9. 9. Use Cases<br />Low level cache<br />SQL, HTTP<br />Application cache<br />Hibernate 2nd level cache<br />Short lived entities<br />Queues<br />
  10. 10. Generic Tools<br />Memcached<br />A high-performance, distributed memory caching system<br />Used EVRYWHERE: Facebook, Twitter, Digg, Wikipedia, Slashdot, LiveJournal, Sourceforge etc.<br />Squid<br />Routing and load balancing <br />HTTP cache<br />
  11. 11. Memcached Limitations for Java<br />Remote cache<br />Serialization<br />2Gb limit<br />
  12. 12. Java In-Memory Tools<br />Prevayler, Space4J<br />Persist objects in memory, supports ACID and file system journaling<br />Db4O in memory, Perst<br />Object oriented database<br />Jofti<br />A high-performance object indexing and searching solution. Supports Map.<br />
  13. 13. In-Memory Considerations<br />Reliability requires redundancy<br />And/or durability<br />In a cluster state must be synced<br />Requires state distribution<br />
  14. 14. Java Distribution Tools<br />Coherence<br />GigaSpace<br />Ehcache<br />Infinispan<br />Terracotta<br />Hazelcast<br />
  15. 15. Introducing Terracotta<br />Network-Attached Memory infrastructure for the JVM<br />Open source<br />(TPL, based on Mozilla License)<br />Current version: 3.0.1<br />
  16. 16. Terracotta Customers<br />
  17. 17. Introducing Terracotta<br />Highly Available<br />Highly Scalable<br />Avoids excessive state replication<br />Hub and spoke architecture<br />
  18. 18. A Nice Diagram<br />Application<br />Application<br />Application<br />Terracotta<br />Server<br />Terracotta<br />Server<br />Terracotta<br />Server<br />Application<br />Application<br />
  19. 19. How Terracotta Works<br />Application<br />Terracotta<br />Client<br />Bootjarinstrumenting your application<br />Terracotta Libraries<br />JVM<br />Terracotta<br />Server<br />TC handles <br />JVM locks and <br />data access<br />
  20. 20. How Terracotta Works, take 2<br />Application<br />Application<br />Application<br /> Terracotta Server<br />Reference<br />Actual instance<br />
  21. 21. Terracotta Features<br />Cross-JVM object identity <br />Automatic Persistence to disk<br />Virtual Memory <br />spill heap to TC and/or to disk, transparently<br />Cluster profiling & visualization<br />JMX-based monitoring and management<br />
  22. 22. Terracotta Advantages<br />Easy to integrate, Maven support<br />Great scalability<br />No proprietary API (optional)<br />Ready made integration modules<br />Great diagnostics<br />In/out of process<br />
  23. 23. Terracotta Development Console<br />
  24. 24. Integrating Terracotta<br />Download & extract<br />Write a tc-config.xml<br />Create a bootjar<br />Run your app with the bootjar<br />Add elements & refactor<br />
  25. 25. TIM - Terracotta Integration Modules<br />Packaged functionality<br />Easy to integrate<br />Handles locks details<br />Examples:<br />Concurrent collections, Ehcache<br />Lucene/Compass, Spring Security<br />Pipes, Wicket, Tomcat, Jetty <br />Hibernate, JBoss, Spring etc.<br />
  26. 26. Terracotta Elements<br />HTTP Session Replication<br />Cache Evictor<br />Queues<br />Master-Worker<br />Asynchronous Processor<br />Cluster membership events<br />
  27. 27. Terracotta Patterns<br />Data Cache<br />Intermediate persistence<br />Queues / work distribution<br />Write behind to database<br />
  28. 28. Data Cache Considerations<br />Figure out data lifecycle<br />Read-only, metadata, configuration, session<br />Where cache is applicable<br />Single point of update<br />Set eviction policies<br />LRU, FIFO<br />
  29. 29. Data Cache<br />Session cache is easier, entity cache needs real distribution<br />Use a generic cache<br />ConcurrentMap + Map Evictor (+ Jofti)<br />Or a Hibernate 2nd level cache<br />ehcache, TC-Cache<br />
  30. 30. Where Should My Objects Live?<br />Memory<br />Database<br />Appropriateness<br />dies quickly<br />stays forever<br />Data Lifetime<br />
  31. 31. Intermediate Data Lifetime<br />Some data doesn’t have to be stored in the database<br />Session information<br />Messages / Tokens<br />Transaction<br />Unfinished stuff<br />Data Lifetime<br />
  32. 32. TIM Pipes: Queues<br />Execute time consuming tasks asynchronously<br />In a reliable way<br />Load balanced<br />Examples:<br />Send emails, process images <br />Supports pub/sub, routing<br />No need for JMS<br />
  33. 33. CommonJ – Work Distribution<br />A joint API spec by Oracle and IBM<br />Provides interfaces for:<br />Worker / WorkItem / WorkManager<br />Scheduler<br />etc…<br />TIM-Messaging provides an implementation<br />Pipes<br />Master / Worker<br />
  34. 34. Write Behind to System Of Record<br />Asynchronously persist to DB<br />Work queue<br />“1.5 phase commit”<br />Flags objects as dirty<br />Avoids syncing dirty objects<br />Flushes objects<br />Flags objects as clean<br />
  35. 35. Terracotta Drawbacks<br />Concurrency awareness<br />Massive scalability is a commercial feature<br />Requires another machine(s)<br />Sharing objects across applications requires configuration<br />TIMs documentation<br />
  36. 36. Terracotta Pros<br />Concurrency awareness<br />Fits into your application<br />Provides great visibility<br />Very versatile usage<br />Scales massively<br />Doesn’t use serialization<br />Great community<br />
  37. 37. Summary<br />Ask yourself:<br />Does it belong in the DB?<br />Do I have to go to the DB for that?<br />Do I have to do it now?<br />
  38. 38. Terracotta Pros<br />Any Questions?<br />