Blackboard DevCon 2013 - Advanced Caching in Blackboard Learn Using Redis Building Block

2,021 views
1,752 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,021
On SlideShare
0
From Embeds
0
Number of Embeds
28
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Good morning. My name is Noriaki Tatsumi, Director of engineering. I’d like to introduce you to a new building block called Redis Cache B2 that’s scheduled to come out next month.
  • Redis Cache B2 is a low level functionality that extends Blackboard Learn’s caching framework. It essentially adds an additional type of memory outside the Java heap.
  • It leverages a popular open source NoSQL server used by prominent companies like craigslist, github… It’s often seen as a competitor to Memcached but it’s much simpler and easier to use and has many more advanced functionalities. Redis has an excellent documentation on their website at http://redis.io so I highly encourage checking it out.
  • Because it’s the key to scaling large environment. Services like Facebook, Twitter, Craigslist would not work without caching.
  • So I want to give you a visual representation of how Redis is placed in the deployment architecture. This is what caching in Learn looks like out of the box.Each application node has its own copy of cacheTalks to each other to keep in sync using peer discovery service to find each otherDoes not copy data, only invalidations when updates and removal of cache happens
  • When using Redis Cache B2. A centralized instance of Redis server is deployed for all application nodes. Before going into more details, I’d like to show you some performance benchmark results of using Redis.
  • This benchmark test mimicked the scenario when database is pressured by load. For all of our performance tests, we used 2 to 1 ratio for database data volume and the JVM heap cache size.
  • Here’s another test case for when JVM heap is pressured.
  • This is statistics from Wikia site. It illustrates that performance and adoption rate are directly linked! Users don’t tolerate slow apps. You want your students to be kept engaged.2 second latency can double the abandonment5 seconds can triple
  • Now, I’d like to discuss the advantages that Redis cache B2 brings in detail.
  • As you saw in the diagram, Redis is deployed as a centralized caching server. This enables Redis to share cache among application nodes and also keep he hottest data local.
  • Blackboard Learn has a safety mechanism called, PanicThread.
  • Redis keeps JVM heap size smaller.
  • Redis offers network memory, which gives speed advantage over disk overflow.
  • We use a library called Kryo to make sure the serialization is efficient as possible.
  • Compared to disk overflow implementation we have in Blackboard Learn, Redis is very efficient at writing cache. It uses write-behind strategy, which means its writes are processed asynchronously. Therefore, even on SSD, Redis cache write operation blows away disk overflow by huge number.
  • For a very large Learn deployment with over 10 application nodes, we’re seeing big overhead from cluster invalidation in the field. P = probability
  • High max saturation and evicted cache require Redis
  • High max saturation and evicted cache require Redis
  • Blackboard DevCon 2013 - Advanced Caching in Blackboard Learn Using Redis Building Block

    1. 1. Blackboard Learn Redis Cache B2 Noriaki Tatsumi, Director, Engineering Blackboard, Inc.
    2. 2. 2 Redis Cache B2 What is it? Enables Blackboard Learn’s caching framework to use Redis as an additional type of memory store outside the Java heap.
    3. 3. 3 Redis What is it? • Open source @ http://redis.io • Networked advanced key-value store • Powerful but simple + easy • Written in ANSI C • Companies that use Redis – Craigslist – Github – Stackoverflow – Instagram
    4. 4. 4 Why Expand on Caching? • The key to scaling large environment • Database is hard to scale horizontally
    5. 5. 5 Caching in Learn Ehcache RMI Replicated Caching + Local Disk Overflow
    6. 6. 6 Redis Cache B2 Ehcache RMI Replicated Caching + Redis Overflow
    7. 7. 7 Performance Test Test case: Database is pressured Redis Cache B2 App Nodes %Fail Total Txns 95th% Resp Time (s) 90th% Resp Time (s) Stdev Disabled 11 0.40 235,852 14.2 10.61 6.09 Enabled 11 0.35 276,602 9.05 7.2 4.03 Delta 40,750 -5.15 -3.41 -2.06 Delta (%) 17.28% -36.27% -32.14% - 33.83% 0 5 10 15 95th% Resp Time(s) 90th% Resp Time(s) Ehcache Redis Cache B2 0 100,000 200,000 300,000 Ehcache Redis Cache B2 Total Txns
    8. 8. 8 Performance Test Test case: JVM heap is pressured Redis Cache B2 App Nodes %Fail Total Txns 95th% Resp Time (s) 90th% Resp Time (s) Stdev Disabled 11 0.61 292,864 5.86 4.41 18.49 Enabled 11 0.69 317,058 4.8 3.81 14.77 Delta 24,194 -1.06 -0.6 -3.72 Delta (%) 8.26% -18.09% -13.61% - 20.12% 0 2 4 6 8 95th% Resp Time(s) 90th% Resp Time(s) Ehcache Redis Cache B2 0 100,000 200,000 300,000 400,000 Ehcache Redis Cache B2 Total Txns
    9. 9. 9 Performance Test Test case: No pressure on any component Redis Cache B2App Nodes %Fail Total Txns 95th% Resp Time (s) 90th% Resp Time (s) Stdev Disabled 11 0.57 361,817 4.22 3.39 1.67 Enabled 11 0.47 363,220 4.02 3.28 1.69 Delta 1,403 -0.20 -0.11 +0.02 Delta (%) 0.39% -4.74% -3.24% -1.20% 0 1 2 3 4 5 95th% Resp Time(s) 90th% Resp Time(s) Ehcache Redis Cache B2 0 100,000 200,000 300,000 400,000 Ehcache Redis Cache B2 Total Txns
    10. 10. 10 Every Second Counts http://www.stevesouders.com/blog/2009/07/27/wikia-fast-pages-retain-users/
    11. 11. 11 Redis Cache B2 in Action No risk of performance loss – Redis fails safe
    12. 12. 12 Redis Cache B2 in Action Cache is shared and the hottest subset of data is kept local
    13. 13. 13 Redis Cache B2 in Action Cache is available under JVM heap pressure PanicRoom thread • Triggers when free heap memory is less than 129MB • Flushes cache to avoid running out of memory • Log message: actuallyFreeMemory: Starting to free up registered caches actuallyFreeMemory: Finished freeing registered caches
    14. 14. 14 Redis Cache B2 in Action Keeps a manageable JVM heap size From performanceterracotta.blogspot.com • GC pause increases with heap size • Compressed oops feature limited to 32 GB heap size
    15. 15. 15 Redis Cache B2 in Action Bigger cache and faster overflow storage JVM Heap Network Memory Disk Size Speed
    16. 16. 16 Redis Cache B2 in Action Fast and efficient object graph serialization Cache Java bytes Kryo bytes class blackboard.data.course.Course 4506 1837 class blackboard.data.navigation.CourseNavigationApplication 2551 621 class blackboard.data.user.User 7304 1735 class blackboard.platform.session.impl.BbSessionImpl 8107 2119 0 5000 10000 Java bytes Kryo bytes
    17. 17. 17 Redis Cache B2 in Action Asynchronous overflow operations with write-behind strategy Test case: 10,000 serial iterations per operation X 5 runs Test Put: Mean(s) Put: Min(s) Put: Max(s) Put: StdDev(s) Disk overflow on SSD 3.116242472 3.053443009 3.315864087 0.11230668 Redis overflow 0.033371059 0.030807671 0.038541155 0.003137204 -3.082871413 -3.022635338 -3.277322932 -0.109169476 -98.93% -98.99% -98.84% -97.21% 0 1 2 3 4 Put: Mean(s) Put: Min(s) Put: Max(s) Put: StdDev(s) Disk overflow on SSD Redis overflow
    18. 18. 18 Redis Cache B2 in Action Reduces on-heap cluster invalidation overhead - • On-heap cache size - • P(updates + removes) - • Cluster invalidations
    19. 19. 19 Redis Cache B2 in Action Cache survives application restarts and crashes
    20. 20. 20 Monitoring and Management Rich monitoring features for easy management
    21. 21. 21 Monitoring and Management Rich monitoring features for easy management
    22. 22. 22 Monitoring and Management Rich monitoring features for easy management
    23. 23. 23 Monitoring and Management Rich monitoring features for easy management
    24. 24. 24 Monitoring and Management Rich monitoring features for easy management
    25. 25. 25 Monitoring and Management Rich monitoring features for easy management
    26. 26. 26 Summary of Redis B2 • Provides better user experience from performance and availability boost • Easy to manage – Redis is simple and easy to install/operate – Redis cache fails safe – Redis Cache B2 has tools to monitor and diagnose problems • When do you need it? – High cache evictions – Database pressure – JVM heap pressure
    27. 27. 27 Release • Date: August 2013 • Requirements: 9.1 SP 11+ – Cumulative Patch 8 for 9.1 SP 11 – Cumulative Patch 2 for 9.1 SP 12
    28. 28. 28 Learn More! help.blackboard.com • Performance, scalability, and availability highlights • Installation guide for Redis server and Redis B2 • Hardening Redis server security
    29. 29. 29 Thank You! Noriaki Tatsumi Director, Engineering Blackboard, Inc. noriaki.tatsumi@blackboard.com

    ×