Building low latency java applications with ehcache

3,269 views
2,950 views

Published on

Dhruv Kumar's talk from http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/124095182/

Published in: Technology
0 Comments
17 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,269
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
17
Embeds 0
No embeds

No notes for slide

Building low latency java applications with ehcache

  1. 1. Building Low Latency Java Applications with Ehcache Dhruv Kumar Software Engineer, Terracotta Inc. Dhruv.Kumar@terracottatech.com
  2. 2. Why In-Memory? 2
  3. 3. 3 Data Center Machines 3
  4. 4. 4 Data Center Machines 4
  5. 5. 5 Data Center Machines 5
  6. 6. 6 Memory Capacity Trends 6
  7. 7. 7 Disk Capacity Trends 7
  8. 8. 8 SSD Capacity Trends 8
  9. 9. 9 CPU Speed Trends 9
  10. 10. 10 Network I/O Trends 10
  11. 11. 11 Memory Throughput Trends 11
  12. 12. 12 SSD Throughput Trends 12
  13. 13. 13 Disk Throughput Trends 13
  14. 14. 14 Trend Summary 14
  15. 15. 15 Trend Summary 15 §  Storage is cheap and increases exponentially §  Most transfer rates increase exponentially except disk throughput. –  Gap between disk capacity and transfer rate is increasing §  Accessing disk is very slow: transferring 100GB @ 100 MB/s takes 1000s, or approx. 17 minutes. –  Will get worse, as 512GB/node RAM will be common in 2 years. –  Faster to access Network Attached Memory! –  SSDs still not common due to unfavorable price performance
  16. 16. 16 In-Memory Computing Opportunity 16 §  256 GB RAM * 4 nodes = 1 TB In-Memory Cluster §  1 TB can hold many large datasets: 1 billion users @ 1 KB each §  Number of CPU cores double every 24 months: memory/core increases exponentially §  Bottomline: judiciously using RAM is key.
  17. 17. Ehcache System Architecture 17
  18. 18. 18 Terracotta Server Array Scale with Data and Processing Needs 18 BigMemory App Server Ehcache App Server Ehcache Traditional 3-Tier Architecture Scale Vertically Scale Horizontally Elastic Scale In the Cloud App Server Ehcache Quartz App Server Ehcache Quartz Database Database Database Database App Server Ehcache App Server Ehcache BigMemory Terracotta Server Array BigMemory Increase Data in Memory Reduce Database Reliance
  19. 19. 19 Hello Ehcache! Database Traditional 3-Tier Architecture App Server Ehcache §  Reduction in database reliance –  Store repetitively used data and access them in memory speeds §  Reduction of new objects created –  Stored objects will be reused constantly thus reducing the creation of new objects §  Less network traffic –  Stored data is already at the application layer so no need to make a network call to get it §  Reduce CPU usage –  Objects in cache are already in the format you need for use, no additional marshaling needed.
  20. 20. 20 Example Configuration Configuration via Ehcache.xml 20 Example Configuration <ehcache>! ! <cache name=”UserPreferencesCache"! maxElementsInMemory="10000"! timeToIdleSeconds="300”! ! ! !memoryStoreEvictionPolicy="LRU”/>! ! <cache name=”ShoppingCartCache"! maxElementsInMemory=”2500"! timeToLiveSeconds=”6000”! ! ! !memoryStoreEvictionPolicy=”LFU"/>! ! </ehcache>! 20
  21. 21. 21 Simple API 21 Example Code public Object testCache(String key) throws Exception {! !CacheManager cacheManager = ! ! !new CacheManager( “<path to my ehcache.xml>”);! !Cache myCache = cacheManager.getCache("MyCache");! !Object value;! ! !! !Element element = myCache.get(key);! !if (element == null) {! ! !value = "go get it from somewhere like DB or service, ! ! ! etc";! ! !myCache.put(new Element(key, value));! !} else {! ! !value = (Object) element.getValue();! !}! ! !! !return value;! }!
  22. 22. 22 Ehcache Search §  Intuitive, full-featured Search API §  Fast, efficient implementation §  Make cache searchable, then specify attributes 22 <cache name=”peopleCache”> <searchable> <searchAttribute name="age" expression="value.getAge()"/> <searchAttribute name="gender" expression="value.getGender()"/> </searchable> </cache> Example: Search for 32-year-old males results = cache.createQuery().includeKeys() .addCriteria(age.eq(32)).and (gender.eq("male")) .execute(); Example Configuration and Code
  23. 23. 23 Ehcache Search Using SQL-like Syntax §  Can also search Ehcache using a SQL like syntax §  String Queries mapped to Ehcache Search API 23 Example: Search for 32-year-old males results = cache.createQuery().includeKeys() .addCriteria(age.eq(32)).and (gender.eq("male")) .execute(); // or using Ehcache Query Language QueryManager qm = (new QueryManagerBuilder()).addEhcache(peopleCache).build(); String statement = “select age, gender from peopleCache where age = 32 and gender =‘male’;” results = qm.createQuery(statement).end().execute(); Example Configuration and Code
  24. 24. 24 Ehcache Persistence §  Persist actions in a log-append store (Fast Restart Store). §  Each mutate operation is recorded. Optionally synchronous. §  On restart, Ehcache entries and Index data are reconstructed from the log. 24 Example ConfigurationExample Configuration <ehcache>! <cache name=”UserPreferencesCache"! maxElementsInMemory="10000"! timeToIdleSeconds="300”! ! ! persistenceStrategy=“localRestartable”>! </ehcache>!
  25. 25. 25 In-memory Data and Speed 25 Milliseconds App Response Time Microseconds App Response Time Memory 90% of Data in Database Database 90% of Data in Memory
  26. 26. 26 Storing Data away from Java Heap (BigMemory) Off-heap memory allocator. Uses NIO DirectByteBuffers. Bypasses Java Garbage Collection. Only Serializable keys and values can be placed in Off-heap. Pure Java implementation compatible with all popular JVMs Requires no changes to application code, JVM or OS 26 Database Vertical Scaling App Server Ehcache BigMemory
  27. 27. 27 BigMemory Configuration <ehcache> <cache name= UserPreferencesCache" maxBytesOnHeap=”512M" timeToIdleSeconds="300 !memoryStoreEvictionPolicy="LRU !overflowToOffHeap="true ! !maxBytesOffHeap= 30G"/> <cache name= ShoppingCartCache" maxElementsInMemory= 2500" timeToLiveSeconds= 6000 !memoryStoreEvictionPolicy="LFU"/> </ehcache>! 27 Example Configuration
  28. 28. 28 Testing Offheap Performance 28 • Public URL: http://svn.terracotta.org/svn/forge/offHeap-test/ • Our Configuration: Six 4-core Xeon CPUs @ 2.93 GHz, 128 GB RAM, RHEL 5.1, Sun JDK 1.6.0_21 in 64 bit mode. • Load the cache and then run 50 threads doing 90% Read (get() call) calls and 10 % Writes (put() call) • Run in Heap and Offheap, and compare GC duration, latency, throughput.
  29. 29. 29 Offheap Perf Test: Garbage Collection Pauses 29
  30. 30. 30 Offheap Perf Test: Maximum Latency 30
  31. 31. 31 Offheap Perf Test: Mean Latency 31
  32. 32. Distributed Ehcache 32
  33. 33. 3333 Distributed Applications Consistency?
  34. 34. 3434 Distributed Ehcache using Terracotta Server Array §  Stateful Server Array §  Synchronous TCP based connections to clients §  Highly Available using Master Slave replication §  Linear Scale Out Characteristics §  Persistent §  Various consistency optionsDatabase Scale Out App Server Ehcache Terracotta Server Array BigMemory App Server Ehcache
  35. 35. 35 Terracotta Server Array 35 Terracotta ServerArray Stripe Pure Java cache server on commodity HW Transactional updated mirror for high availability Stripe Commodity Server Application Terracotta Driver Stripe Stripe Stripe Stripe Commodity Server Application Terracotta Driver Commodity Server Application Terracotta Driver TCP § Durability § Mirroring § Striping § Developer Console § Plug-in Monitoring § Operations Center Commodity Server Disk Active Server Commodity Server Disk BigMemory Mirror BigMemory
  36. 36. 36 Example Configuration Configuring Ehcache to use Terracotta 36 <ehcache> !<terracottaConfig url="someserver:9510"/> !<cache name= UserPreferencesCache ! !maxElementsInMemory="10000 ! !timeToIdleSeconds= 300 /> !<cache name= ShoppingCartCache" ! !maxElementsInMemory= 25000" ! !timeToLiveSeconds= 6000 />! !<terracotta /> </ehcache>! Simply change two lines of configuration
  37. 37. 37 Tiered Storage In-Memory Tiered Data Storage 37 Terracotta Heap Heap Store BigMemory Off-Heap Store 2,000,000+ 1,000,000 100,000 2 1,000 10,000+ Speed (TPS) 1,000s Size (GB) External Data Source (e.g., Database) Terracotta Offheap
  38. 38. 38 Example Configuration Consistency options in TSA 38 Strongly Consistent Fully Transactional Eventually Consistent More Consistency More Performance <cache name=”UserPreferencesCache" maxElementsInMemory="10000" timeToLiveSeconds="300”> <terracotta consistency=”eventual"/> </cache> <cache name=”ShoppingCartCache" maxElementsInMemory=”5000" timeToIdleSeconds=”6000”> <terracotta consistency=”strong"/> </cache> Example Configuration
  39. 39. 39 Terracotta Server Array Scale with Data and Processing Needs 39 BigMemory App Server Ehcache App Server Ehcache Traditional 3-Tier Architecture Scale Up Scale Out Elastic Scale In the Cloud App Server Ehcache Quartz App Server Ehcache Quartz Database Database Database Database App Server Ehcache App Server Ehcache BigMemory Terracotta Server Array BigMemory Increase Data in Memory Reduce Database Reliance
  40. 40. 40 Recent Developments 40 • JSR107 • Hadoop Integration: One way (EhcacheOutputFormat) • New Management and Monitoring
  41. 41. 4141 Thank You Dhruv.Kumar@terracottatech.com

×