• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Building low latency java applications with ehcache
 

Building low latency java applications with ehcache

on

  • 1,475 views

Dhruv Kumar's talk from http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/124095182/

Dhruv Kumar's talk from http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/124095182/

Statistics

Views

Total Views
1,475
Views on SlideShare
1,475
Embed Views
0

Actions

Likes
6
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Building low latency java applications with ehcache Building low latency java applications with ehcache Presentation Transcript

    • Building Low Latency Java Applications with Ehcache Dhruv Kumar Software Engineer, Terracotta Inc. Dhruv.Kumar@terracottatech.com
    • Why In-Memory? 2
    • 3 Data Center Machines 3
    • 4 Data Center Machines 4
    • 5 Data Center Machines 5
    • 6 Memory Capacity Trends 6
    • 7 Disk Capacity Trends 7
    • 8 SSD Capacity Trends 8
    • 9 CPU Speed Trends 9
    • 10 Network I/O Trends 10
    • 11 Memory Throughput Trends 11
    • 12 SSD Throughput Trends 12
    • 13 Disk Throughput Trends 13
    • 14 Trend Summary 14
    • 15 Trend Summary 15 §  Storage is cheap and increases exponentially §  Most transfer rates increase exponentially except disk throughput. –  Gap between disk capacity and transfer rate is increasing §  Accessing disk is very slow: transferring 100GB @ 100 MB/s takes 1000s, or approx. 17 minutes. –  Will get worse, as 512GB/node RAM will be common in 2 years. –  Faster to access Network Attached Memory! –  SSDs still not common due to unfavorable price performance
    • 16 In-Memory Computing Opportunity 16 §  256 GB RAM * 4 nodes = 1 TB In-Memory Cluster §  1 TB can hold many large datasets: 1 billion users @ 1 KB each §  Number of CPU cores double every 24 months: memory/core increases exponentially §  Bottomline: judiciously using RAM is key.
    • Ehcache System Architecture 17
    • 18 Terracotta Server Array Scale with Data and Processing Needs 18 BigMemory App Server Ehcache App Server Ehcache Traditional 3-Tier Architecture Scale Vertically Scale Horizontally Elastic Scale In the Cloud App Server Ehcache Quartz App Server Ehcache Quartz Database Database Database Database App Server Ehcache App Server Ehcache BigMemory Terracotta Server Array BigMemory Increase Data in Memory Reduce Database Reliance
    • 19 Hello Ehcache! Database Traditional 3-Tier Architecture App Server Ehcache §  Reduction in database reliance –  Store repetitively used data and access them in memory speeds §  Reduction of new objects created –  Stored objects will be reused constantly thus reducing the creation of new objects §  Less network traffic –  Stored data is already at the application layer so no need to make a network call to get it §  Reduce CPU usage –  Objects in cache are already in the format you need for use, no additional marshaling needed.
    • 20 Example Configuration Configuration via Ehcache.xml 20 Example Configuration <ehcache>! ! <cache name=”UserPreferencesCache"! maxElementsInMemory="10000"! timeToIdleSeconds="300”! ! ! !memoryStoreEvictionPolicy="LRU”/>! ! <cache name=”ShoppingCartCache"! maxElementsInMemory=”2500"! timeToLiveSeconds=”6000”! ! ! !memoryStoreEvictionPolicy=”LFU"/>! ! </ehcache>! 20
    • 21 Simple API 21 Example Code public Object testCache(String key) throws Exception {! !CacheManager cacheManager = ! ! !new CacheManager( “<path to my ehcache.xml>”);! !Cache myCache = cacheManager.getCache("MyCache");! !Object value;! ! !! !Element element = myCache.get(key);! !if (element == null) {! ! !value = "go get it from somewhere like DB or service, ! ! ! etc";! ! !myCache.put(new Element(key, value));! !} else {! ! !value = (Object) element.getValue();! !}! ! !! !return value;! }!
    • 22 Ehcache Search §  Intuitive, full-featured Search API §  Fast, efficient implementation §  Make cache searchable, then specify attributes 22 <cache name=”peopleCache”> <searchable> <searchAttribute name="age" expression="value.getAge()"/> <searchAttribute name="gender" expression="value.getGender()"/> </searchable> </cache> Example: Search for 32-year-old males results = cache.createQuery().includeKeys() .addCriteria(age.eq(32)).and (gender.eq("male")) .execute(); Example Configuration and Code
    • 23 Ehcache Search Using SQL-like Syntax §  Can also search Ehcache using a SQL like syntax §  String Queries mapped to Ehcache Search API 23 Example: Search for 32-year-old males results = cache.createQuery().includeKeys() .addCriteria(age.eq(32)).and (gender.eq("male")) .execute(); // or using Ehcache Query Language QueryManager qm = (new QueryManagerBuilder()).addEhcache(peopleCache).build(); String statement = “select age, gender from peopleCache where age = 32 and gender =‘male’;” results = qm.createQuery(statement).end().execute(); Example Configuration and Code
    • 24 Ehcache Persistence §  Persist actions in a log-append store (Fast Restart Store). §  Each mutate operation is recorded. Optionally synchronous. §  On restart, Ehcache entries and Index data are reconstructed from the log. 24 Example ConfigurationExample Configuration <ehcache>! <cache name=”UserPreferencesCache"! maxElementsInMemory="10000"! timeToIdleSeconds="300”! ! ! persistenceStrategy=“localRestartable”>! </ehcache>!
    • 25 In-memory Data and Speed 25 Milliseconds App Response Time Microseconds App Response Time Memory 90% of Data in Database Database 90% of Data in Memory
    • 26 Storing Data away from Java Heap (BigMemory) Off-heap memory allocator. Uses NIO DirectByteBuffers. Bypasses Java Garbage Collection. Only Serializable keys and values can be placed in Off-heap. Pure Java implementation compatible with all popular JVMs Requires no changes to application code, JVM or OS 26 Database Vertical Scaling App Server Ehcache BigMemory
    • 27 BigMemory Configuration <ehcache> <cache name= UserPreferencesCache" maxBytesOnHeap=”512M" timeToIdleSeconds="300 !memoryStoreEvictionPolicy="LRU !overflowToOffHeap="true ! !maxBytesOffHeap= 30G"/> <cache name= ShoppingCartCache" maxElementsInMemory= 2500" timeToLiveSeconds= 6000 !memoryStoreEvictionPolicy="LFU"/> </ehcache>! 27 Example Configuration
    • 28 Testing Offheap Performance 28 • Public URL: http://svn.terracotta.org/svn/forge/offHeap-test/ • Our Configuration: Six 4-core Xeon CPUs @ 2.93 GHz, 128 GB RAM, RHEL 5.1, Sun JDK 1.6.0_21 in 64 bit mode. • Load the cache and then run 50 threads doing 90% Read (get() call) calls and 10 % Writes (put() call) • Run in Heap and Offheap, and compare GC duration, latency, throughput.
    • 29 Offheap Perf Test: Garbage Collection Pauses 29
    • 30 Offheap Perf Test: Maximum Latency 30
    • 31 Offheap Perf Test: Mean Latency 31
    • Distributed Ehcache 32
    • 3333 Distributed Applications Consistency?
    • 3434 Distributed Ehcache using Terracotta Server Array §  Stateful Server Array §  Synchronous TCP based connections to clients §  Highly Available using Master Slave replication §  Linear Scale Out Characteristics §  Persistent §  Various consistency optionsDatabase Scale Out App Server Ehcache Terracotta Server Array BigMemory App Server Ehcache
    • 35 Terracotta Server Array 35 Terracotta ServerArray Stripe Pure Java cache server on commodity HW Transactional updated mirror for high availability Stripe Commodity Server Application Terracotta Driver Stripe Stripe Stripe Stripe Commodity Server Application Terracotta Driver Commodity Server Application Terracotta Driver TCP § Durability § Mirroring § Striping § Developer Console § Plug-in Monitoring § Operations Center Commodity Server Disk Active Server Commodity Server Disk BigMemory Mirror BigMemory
    • 36 Example Configuration Configuring Ehcache to use Terracotta 36 <ehcache> !<terracottaConfig url="someserver:9510"/> !<cache name= UserPreferencesCache ! !maxElementsInMemory="10000 ! !timeToIdleSeconds= 300 /> !<cache name= ShoppingCartCache" ! !maxElementsInMemory= 25000" ! !timeToLiveSeconds= 6000 />! !<terracotta /> </ehcache>! Simply change two lines of configuration
    • 37 Tiered Storage In-Memory Tiered Data Storage 37 Terracotta Heap Heap Store BigMemory Off-Heap Store 2,000,000+ 1,000,000 100,000 2 1,000 10,000+ Speed (TPS) 1,000s Size (GB) External Data Source (e.g., Database) Terracotta Offheap
    • 38 Example Configuration Consistency options in TSA 38 Strongly Consistent Fully Transactional Eventually Consistent More Consistency More Performance <cache name=”UserPreferencesCache" maxElementsInMemory="10000" timeToLiveSeconds="300”> <terracotta consistency=”eventual"/> </cache> <cache name=”ShoppingCartCache" maxElementsInMemory=”5000" timeToIdleSeconds=”6000”> <terracotta consistency=”strong"/> </cache> Example Configuration
    • 39 Terracotta Server Array Scale with Data and Processing Needs 39 BigMemory App Server Ehcache App Server Ehcache Traditional 3-Tier Architecture Scale Up Scale Out Elastic Scale In the Cloud App Server Ehcache Quartz App Server Ehcache Quartz Database Database Database Database App Server Ehcache App Server Ehcache BigMemory Terracotta Server Array BigMemory Increase Data in Memory Reduce Database Reliance
    • 40 Recent Developments 40 • JSR107 • Hadoop Integration: One way (EhcacheOutputFormat) • New Management and Monitoring
    • 4141 Thank You Dhruv.Kumar@terracottatech.com