Building Low Latency Java Applications with
Ehcache
Dhruv Kumar
Software Engineer, Terracotta Inc.
Dhruv.Kumar@terracottat...
Why In-Memory?
2
3
Data Center Machines
3
4
Data Center Machines
4
5
Data Center Machines
5
6
Memory Capacity Trends
6
7
Disk Capacity Trends
7
8
SSD Capacity Trends
8
9
CPU Speed Trends
9
10
Network I/O Trends
10
11
Memory Throughput Trends
11
12
SSD Throughput Trends
12
13
Disk Throughput Trends
13
14
Trend Summary
14
15
Trend Summary
15
§  Storage is cheap and increases exponentially
§  Most transfer rates increase exponentially except...
16
In-Memory Computing Opportunity
16
§  256 GB RAM * 4 nodes = 1 TB In-Memory Cluster
§  1 TB can hold many large datas...
Ehcache System Architecture
17
18
Terracotta Server Array
Scale with Data and Processing Needs
18
BigMemory
App Server
Ehcache
App Server
Ehcache
Traditi...
19
Hello Ehcache!
Database
Traditional
3-Tier Architecture
App Server
Ehcache
§  Reduction in database reliance
–  Store ...
20
Example Configuration
Configuration via Ehcache.xml
20
Example Configuration
<ehcache>!
!
<cache name=”UserPreferencesC...
21
Simple API
21
Example Code
public Object testCache(String key) throws Exception {!
!CacheManager cacheManager = !
! !ne...
22
Ehcache Search
§  Intuitive, full-featured Search API
§  Fast, efficient implementation
§  Make cache searchable, th...
23
Ehcache Search Using SQL-like Syntax
§  Can also search Ehcache using a SQL like syntax
§  String Queries mapped to E...
24
Ehcache Persistence
§  Persist actions in a log-append store (Fast Restart Store).
§  Each mutate operation is record...
25
In-memory Data and Speed
25
Milliseconds
App Response Time
Microseconds
App Response Time
Memory
90% of Data in
Databas...
26
Storing Data away from Java Heap
(BigMemory)
Off-heap memory allocator. Uses NIO
DirectByteBuffers.
Bypasses Java Garba...
27
BigMemory Configuration
<ehcache>
<cache name= UserPreferencesCache"
maxBytesOnHeap=”512M"
timeToIdleSeconds="300
!memo...
28
Testing Offheap Performance
28
• Public URL:
http://svn.terracotta.org/svn/forge/offHeap-test/
• Our Configuration: Six...
29
Offheap Perf Test: Garbage Collection Pauses
29
30
Offheap Perf Test: Maximum Latency
30
31
Offheap Perf Test: Mean Latency
31
Distributed
Ehcache
32
3333
Distributed Applications
Consistency?
3434
Distributed Ehcache using Terracotta Server
Array
§  Stateful Server Array
§  Synchronous TCP based connections to
...
35
Terracotta Server Array
35
Terracotta
ServerArray
Stripe
Pure Java
cache server on
commodity HW
Transactional
updated m...
36
Example Configuration
Configuring Ehcache to use Terracotta
36
<ehcache>
!<terracottaConfig url="someserver:9510"/>
!<c...
37
Tiered Storage
In-Memory
Tiered Data Storage
37
Terracotta Heap
Heap
Store
BigMemory
Off-Heap Store
2,000,000+
1,000,00...
38
Example Configuration
Consistency options in TSA
38
Strongly
Consistent
Fully
Transactional
Eventually
Consistent
More ...
39
Terracotta Server Array
Scale with Data and Processing Needs
39
BigMemory
App Server
Ehcache
App Server
Ehcache
Traditi...
40
Recent Developments
40
• JSR107
• Hadoop Integration: One way
(EhcacheOutputFormat)
• New Management and Monitoring
4141
Thank You
Dhruv.Kumar@terracottatech.com
Upcoming SlideShare
Loading in...5
×

Building low latency java applications with ehcache

2,128

Published on

Dhruv Kumar's talk from http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/124095182/

Published in: Technology
0 Comments
14 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,128
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
14
Embeds 0
No embeds

No notes for slide

Building low latency java applications with ehcache

  1. 1. Building Low Latency Java Applications with Ehcache Dhruv Kumar Software Engineer, Terracotta Inc. Dhruv.Kumar@terracottatech.com
  2. 2. Why In-Memory? 2
  3. 3. 3 Data Center Machines 3
  4. 4. 4 Data Center Machines 4
  5. 5. 5 Data Center Machines 5
  6. 6. 6 Memory Capacity Trends 6
  7. 7. 7 Disk Capacity Trends 7
  8. 8. 8 SSD Capacity Trends 8
  9. 9. 9 CPU Speed Trends 9
  10. 10. 10 Network I/O Trends 10
  11. 11. 11 Memory Throughput Trends 11
  12. 12. 12 SSD Throughput Trends 12
  13. 13. 13 Disk Throughput Trends 13
  14. 14. 14 Trend Summary 14
  15. 15. 15 Trend Summary 15 §  Storage is cheap and increases exponentially §  Most transfer rates increase exponentially except disk throughput. –  Gap between disk capacity and transfer rate is increasing §  Accessing disk is very slow: transferring 100GB @ 100 MB/s takes 1000s, or approx. 17 minutes. –  Will get worse, as 512GB/node RAM will be common in 2 years. –  Faster to access Network Attached Memory! –  SSDs still not common due to unfavorable price performance
  16. 16. 16 In-Memory Computing Opportunity 16 §  256 GB RAM * 4 nodes = 1 TB In-Memory Cluster §  1 TB can hold many large datasets: 1 billion users @ 1 KB each §  Number of CPU cores double every 24 months: memory/core increases exponentially §  Bottomline: judiciously using RAM is key.
  17. 17. Ehcache System Architecture 17
  18. 18. 18 Terracotta Server Array Scale with Data and Processing Needs 18 BigMemory App Server Ehcache App Server Ehcache Traditional 3-Tier Architecture Scale Vertically Scale Horizontally Elastic Scale In the Cloud App Server Ehcache Quartz App Server Ehcache Quartz Database Database Database Database App Server Ehcache App Server Ehcache BigMemory Terracotta Server Array BigMemory Increase Data in Memory Reduce Database Reliance
  19. 19. 19 Hello Ehcache! Database Traditional 3-Tier Architecture App Server Ehcache §  Reduction in database reliance –  Store repetitively used data and access them in memory speeds §  Reduction of new objects created –  Stored objects will be reused constantly thus reducing the creation of new objects §  Less network traffic –  Stored data is already at the application layer so no need to make a network call to get it §  Reduce CPU usage –  Objects in cache are already in the format you need for use, no additional marshaling needed.
  20. 20. 20 Example Configuration Configuration via Ehcache.xml 20 Example Configuration <ehcache>! ! <cache name=”UserPreferencesCache"! maxElementsInMemory="10000"! timeToIdleSeconds="300”! ! ! !memoryStoreEvictionPolicy="LRU”/>! ! <cache name=”ShoppingCartCache"! maxElementsInMemory=”2500"! timeToLiveSeconds=”6000”! ! ! !memoryStoreEvictionPolicy=”LFU"/>! ! </ehcache>! 20
  21. 21. 21 Simple API 21 Example Code public Object testCache(String key) throws Exception {! !CacheManager cacheManager = ! ! !new CacheManager( “<path to my ehcache.xml>”);! !Cache myCache = cacheManager.getCache("MyCache");! !Object value;! ! !! !Element element = myCache.get(key);! !if (element == null) {! ! !value = "go get it from somewhere like DB or service, ! ! ! etc";! ! !myCache.put(new Element(key, value));! !} else {! ! !value = (Object) element.getValue();! !}! ! !! !return value;! }!
  22. 22. 22 Ehcache Search §  Intuitive, full-featured Search API §  Fast, efficient implementation §  Make cache searchable, then specify attributes 22 <cache name=”peopleCache”> <searchable> <searchAttribute name="age" expression="value.getAge()"/> <searchAttribute name="gender" expression="value.getGender()"/> </searchable> </cache> Example: Search for 32-year-old males results = cache.createQuery().includeKeys() .addCriteria(age.eq(32)).and (gender.eq("male")) .execute(); Example Configuration and Code
  23. 23. 23 Ehcache Search Using SQL-like Syntax §  Can also search Ehcache using a SQL like syntax §  String Queries mapped to Ehcache Search API 23 Example: Search for 32-year-old males results = cache.createQuery().includeKeys() .addCriteria(age.eq(32)).and (gender.eq("male")) .execute(); // or using Ehcache Query Language QueryManager qm = (new QueryManagerBuilder()).addEhcache(peopleCache).build(); String statement = “select age, gender from peopleCache where age = 32 and gender =‘male’;” results = qm.createQuery(statement).end().execute(); Example Configuration and Code
  24. 24. 24 Ehcache Persistence §  Persist actions in a log-append store (Fast Restart Store). §  Each mutate operation is recorded. Optionally synchronous. §  On restart, Ehcache entries and Index data are reconstructed from the log. 24 Example ConfigurationExample Configuration <ehcache>! <cache name=”UserPreferencesCache"! maxElementsInMemory="10000"! timeToIdleSeconds="300”! ! ! persistenceStrategy=“localRestartable”>! </ehcache>!
  25. 25. 25 In-memory Data and Speed 25 Milliseconds App Response Time Microseconds App Response Time Memory 90% of Data in Database Database 90% of Data in Memory
  26. 26. 26 Storing Data away from Java Heap (BigMemory) Off-heap memory allocator. Uses NIO DirectByteBuffers. Bypasses Java Garbage Collection. Only Serializable keys and values can be placed in Off-heap. Pure Java implementation compatible with all popular JVMs Requires no changes to application code, JVM or OS 26 Database Vertical Scaling App Server Ehcache BigMemory
  27. 27. 27 BigMemory Configuration <ehcache> <cache name= UserPreferencesCache" maxBytesOnHeap=”512M" timeToIdleSeconds="300 !memoryStoreEvictionPolicy="LRU !overflowToOffHeap="true ! !maxBytesOffHeap= 30G"/> <cache name= ShoppingCartCache" maxElementsInMemory= 2500" timeToLiveSeconds= 6000 !memoryStoreEvictionPolicy="LFU"/> </ehcache>! 27 Example Configuration
  28. 28. 28 Testing Offheap Performance 28 • Public URL: http://svn.terracotta.org/svn/forge/offHeap-test/ • Our Configuration: Six 4-core Xeon CPUs @ 2.93 GHz, 128 GB RAM, RHEL 5.1, Sun JDK 1.6.0_21 in 64 bit mode. • Load the cache and then run 50 threads doing 90% Read (get() call) calls and 10 % Writes (put() call) • Run in Heap and Offheap, and compare GC duration, latency, throughput.
  29. 29. 29 Offheap Perf Test: Garbage Collection Pauses 29
  30. 30. 30 Offheap Perf Test: Maximum Latency 30
  31. 31. 31 Offheap Perf Test: Mean Latency 31
  32. 32. Distributed Ehcache 32
  33. 33. 3333 Distributed Applications Consistency?
  34. 34. 3434 Distributed Ehcache using Terracotta Server Array §  Stateful Server Array §  Synchronous TCP based connections to clients §  Highly Available using Master Slave replication §  Linear Scale Out Characteristics §  Persistent §  Various consistency optionsDatabase Scale Out App Server Ehcache Terracotta Server Array BigMemory App Server Ehcache
  35. 35. 35 Terracotta Server Array 35 Terracotta ServerArray Stripe Pure Java cache server on commodity HW Transactional updated mirror for high availability Stripe Commodity Server Application Terracotta Driver Stripe Stripe Stripe Stripe Commodity Server Application Terracotta Driver Commodity Server Application Terracotta Driver TCP § Durability § Mirroring § Striping § Developer Console § Plug-in Monitoring § Operations Center Commodity Server Disk Active Server Commodity Server Disk BigMemory Mirror BigMemory
  36. 36. 36 Example Configuration Configuring Ehcache to use Terracotta 36 <ehcache> !<terracottaConfig url="someserver:9510"/> !<cache name= UserPreferencesCache ! !maxElementsInMemory="10000 ! !timeToIdleSeconds= 300 /> !<cache name= ShoppingCartCache" ! !maxElementsInMemory= 25000" ! !timeToLiveSeconds= 6000 />! !<terracotta /> </ehcache>! Simply change two lines of configuration
  37. 37. 37 Tiered Storage In-Memory Tiered Data Storage 37 Terracotta Heap Heap Store BigMemory Off-Heap Store 2,000,000+ 1,000,000 100,000 2 1,000 10,000+ Speed (TPS) 1,000s Size (GB) External Data Source (e.g., Database) Terracotta Offheap
  38. 38. 38 Example Configuration Consistency options in TSA 38 Strongly Consistent Fully Transactional Eventually Consistent More Consistency More Performance <cache name=”UserPreferencesCache" maxElementsInMemory="10000" timeToLiveSeconds="300”> <terracotta consistency=”eventual"/> </cache> <cache name=”ShoppingCartCache" maxElementsInMemory=”5000" timeToIdleSeconds=”6000”> <terracotta consistency=”strong"/> </cache> Example Configuration
  39. 39. 39 Terracotta Server Array Scale with Data and Processing Needs 39 BigMemory App Server Ehcache App Server Ehcache Traditional 3-Tier Architecture Scale Up Scale Out Elastic Scale In the Cloud App Server Ehcache Quartz App Server Ehcache Quartz Database Database Database Database App Server Ehcache App Server Ehcache BigMemory Terracotta Server Array BigMemory Increase Data in Memory Reduce Database Reliance
  40. 40. 40 Recent Developments 40 • JSR107 • Hadoop Integration: One way (EhcacheOutputFormat) • New Management and Monitoring
  41. 41. 4141 Thank You Dhruv.Kumar@terracottatech.com

×