Clustered Architecture Patterns: Delivering Scalability and Availability Qcon San Francisco, 2007 Ari Zilka
This Session <ul><li>&quot;Clustered Architecture Patterns: Delivering Scalability and Availability&quot; </li></ul><ul><l...
Agenda <ul><li>The Clustering Patterns </li></ul><ul><li>Network Attached Memory / JVM-level clustering </li></ul><ul><li>...
The Clustering Patterns <ul><li>Types of clustering: </li></ul><ul><ul><li>Load-balanced Scale Out </li></ul></ul><ul><ul>...
Changing the Assumptions: JVM-level Clustering
JVM-level Clustering Addresses the Patterns <ul><li>SIMPLE </li></ul><ul><li>Honors Language Spec across JVM boundaries </...
The foundation for all this thinking… <ul><li>At Walmart.com we started like everyone else: stateless + load-balanced + Or...
The precarious path: These are common-place tools <ul><li>Stateless load-balanced architecture </li></ul><ul><li>Sticky Se...
Large publisher gets caught down the path with Oracle Scaling Out or Up?
Breaking the pattern without leaving “Load-Balanced” world <ul><li>$1.5 Million DB & HW savings </li></ul><ul><li>Doubled ...
Counter Examples <ul><li>Load Balanced Application </li></ul><ul><ul><li>Publishing Company </li></ul></ul><ul><ul><li>Hap...
Stateless Architecture
Stateless By Hand is Cumbersome and Inefficient <ul><li>Baseline Application </li></ul><ul><li>3 User Requests during one ...
So We Add Hibernate <ul><li>Add Hibernate </li></ul><ul><li>Eliminate Direct Connection to the DB via JDBC </li></ul><ul><...
Then We Turn on Caching Serialization is required BLOB Replication requirements are heavy User Conversation <ul><li>Enable...
So We Disconnect But Lose Availability <ul><li>Detached POJOs </li></ul><ul><li>Eliminates Intermediate Commits </li></ul>...
JVM-Level Clustering + Hibernate Together <ul><li>Cluster 2nd Level Cache - Hibernate Performance Curve Level 2 </li></ul>...
Demonstration Application <ul><li>Simple CRUD application </li></ul><ul><ul><li>Based on Hibernate Tutorial (Person, Event...
Source Code
Performance Tests <ul><li>ReadAgeHibernate </li></ul><ul><ul><li>25k iterations </li></ul></ul><ul><ul><ul><li>Reads a Per...
Results: Hibernate vs. Detached POJOs ~ 500,000 ops / sec Terracotta Read ~ 1800 ops / sec Hibernate + 2nd Level Cache Rea...
User Was Happy <ul><li>Database was still the SoR which kept reporting and backup simple </li></ul><ul><li>Scalability was...
Partitioned Architecture
Example Caching Service <ul><li>Reduce utilization of System of Record </li></ul><ul><li>Support 4 BUs </li></ul><ul><li>1...
BU #1 BU #2 BU #3 BU #4 Data center Websphere MQ Series - MOM Messaging Infrastructure  (JMS Topics) Caching Layer SoR MQ ...
BU #1 BU #2 BU #3 BU #4 Data center Websphere MQ Series - MOM Messaging Infrastructure  (JMS Topics) Caching Layer SoR MQ ...
BU #1 BU #2 BU #3 BU #4 Data center Websphere MQ Series - MOM Messaging Infrastructure  (JMS Topics) Caching Layer SoR MQ ...
Demo 2 Shared Queue
User Was Unhappy <ul><li>Simplicity was lost.  The Partitioning leaked up the application stack </li></ul><ul><li>Availabi...
Lessons Learned: Scalability + Availability + Simplicity <ul><li>Stop the Madness </li></ul><ul><ul><li>Stop the hacking! ...
Load-balanced Scale Out <ul><li>Simplicity    Ignore the impedance mismatch.  Don’t be afraid of the DB.  </li></ul><ul><...
Partitioned Scale Out <ul><li>Simplicity    Share concurrency events, not data.  Abstract as necessary (the caching servi...
Thank You <ul><li>Learn more at  http://www.terracotta.org/ </li></ul>
Upcoming SlideShare
Loading in …5
×

Clustered Architecture Patterns Delivering Scalability And Availability

2,062 views
1,970 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,062
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
91
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Clustered Architecture Patterns Delivering Scalability And Availability

    1. 1. Clustered Architecture Patterns: Delivering Scalability and Availability Qcon San Francisco, 2007 Ari Zilka
    2. 2. This Session <ul><li>&quot;Clustered Architecture Patterns: Delivering Scalability and Availability&quot; </li></ul><ul><li>Solution Track: Architecting for Performance & Scalability </li></ul><ul><li>Wednesday 14:30 - 15:30 </li></ul>
    3. 3. Agenda <ul><li>The Clustering Patterns </li></ul><ul><li>Network Attached Memory / JVM-level clustering </li></ul><ul><li>Database and Developer Abuse </li></ul><ul><li>Use Case #1: Hibernate disconnected mode </li></ul><ul><li>Use Case #2: Event-based architecture </li></ul><ul><li>Lessons Learned </li></ul>
    4. 4. The Clustering Patterns <ul><li>Types of clustering: </li></ul><ul><ul><li>Load-balanced Scale Out </li></ul></ul><ul><ul><li>Partitioned Scale Out </li></ul></ul><ul><li>Both Trade-off Scalability or availability (usually by hand) in different ways </li></ul><ul><li>Simplicity tends to get lost because frameworks rely on replication </li></ul><ul><ul><li>Copy-on-read </li></ul></ul><ul><ul><li>Copy-on-write </li></ul></ul><ul><ul><li>Examples: serialization, Hibernate, JMS </li></ul></ul>
    5. 5. Changing the Assumptions: JVM-level Clustering
    6. 6. JVM-level Clustering Addresses the Patterns <ul><li>SIMPLE </li></ul><ul><li>Honors Language Spec across JVM boundaries </li></ul><ul><li>Transparent to source code </li></ul><ul><li>Thread Coordination too </li></ul><ul><li>SCALABLE </li></ul><ul><li>Virtual Heap </li></ul><ul><li>Read from Heap </li></ul><ul><li>Write deltas, only where needed </li></ul><ul><li>AVAILABLE </li></ul><ul><li>Persist to disk at wire speed </li></ul><ul><li>Active / Passive and Active / Active strategies </li></ul><ul><li>Models </li></ul><ul><li>Load balanced stays naïve </li></ul><ul><li>Partitioned stays POJO (abstractions are easy) </li></ul><ul><li>Models </li></ul><ul><li>Load balanced scales through implicit locality </li></ul><ul><li>Partitioned scales by avoiding data sharing </li></ul><ul><li>Models </li></ul><ul><li>Load balanced apps made available by writing heap to disk </li></ul><ul><li>Partitioned made available by using the cluster to store workflow </li></ul>“ILITIES” Scale Out Model
    7. 7. The foundation for all this thinking… <ul><li>At Walmart.com we started like everyone else: stateless + load-balanced + Oracle (24 cpus, 24GB) </li></ul><ul><li>Grew up through distributed caching + partitioning + write behind </li></ul><ul><li>We realized that “ilities” conflict </li></ul><ul><ul><li>Scalability: avoid bottlenecks </li></ul></ul><ul><ul><li>Availability: write to disk ( and I/O bottleneck ) </li></ul></ul><ul><ul><li>Simplicity: No copy-on-read / copy-on-write semantics (relentless tuning, bug fixing) </li></ul></ul><ul><li>And yet we needed a stateless runtime for safe operation </li></ul><ul><ul><li>Start / stop any node regardless of workload </li></ul></ul><ul><ul><li>Cluster-wide reboot needed to be quick; could not wait for caches to warm </li></ul></ul><ul><li>The “ilities” clearly get affected by architecture direction and the stateless model leads us down a precarious path </li></ul>
    8. 8. The precarious path: These are common-place tools <ul><li>Stateless load-balanced architecture </li></ul><ul><li>Sticky Session with replicate-only-on-change </li></ul><ul><li>Clustered DB cache </li></ul><ul><li>Separate caching server </li></ul><ul><li>JMS for Async cache update </li></ul>
    9. 9. Large publisher gets caught down the path with Oracle Scaling Out or Up?
    10. 10. Breaking the pattern without leaving “Load-Balanced” world <ul><li>$1.5 Million DB & HW savings </li></ul><ul><li>Doubled business </li></ul><ul><li>More than halved database load </li></ul>
    11. 11. Counter Examples <ul><li>Load Balanced Application </li></ul><ul><ul><li>Publishing Company </li></ul></ul><ul><ul><li>Happy with availability and simplicity using Hibernate + Oracle </li></ul></ul><ul><ul><li>Not happy with scalability </li></ul></ul><ul><ul><li>SOLUTION: Hibernate disconnected mode </li></ul></ul><ul><li>Partitioned Application </li></ul><ul><ul><li>Travel Company </li></ul></ul><ul><ul><li>Happy with MQ-based availability, 4 dependent apps mean no API changes allowed </li></ul></ul><ul><ul><li>System of Record too expensive to keep scaling </li></ul></ul><ul><ul><li>SOLUTION: Proxy the System or Record; Partition for scale </li></ul></ul>
    12. 12. Stateless Architecture
    13. 13. Stateless By Hand is Cumbersome and Inefficient <ul><li>Baseline Application </li></ul><ul><li>3 User Requests during one Conversation </li></ul><ul><li>2 POJO Updates per Request </li></ul><ul><li>Total DB Load: 9 </li></ul>User Conversation
    14. 14. So We Add Hibernate <ul><li>Add Hibernate </li></ul><ul><li>Eliminate Direct Connection to the DB via JDBC </li></ul><ul><li>Eliminate Hand-Coded SQL </li></ul><ul><li>Eliminate Intermediate POJO Updates </li></ul><ul><li>Total DB Load: 6 </li></ul>User Conversation
    15. 15. Then We Turn on Caching Serialization is required BLOB Replication requirements are heavy User Conversation <ul><li>Enable 2nd Level cache </li></ul><ul><li>Eliminates Intermediate Loads </li></ul><ul><li>Total DB Load: 4 </li></ul>
    16. 16. So We Disconnect But Lose Availability <ul><li>Detached POJOs </li></ul><ul><li>Eliminates Intermediate Commits </li></ul><ul><li>Total DB Load: 2 </li></ul>Can lose state in case of failure! Replication is expensive Hibernate says to keep graphs small User Conversation
    17. 17. JVM-Level Clustering + Hibernate Together <ul><li>Cluster 2nd Level Cache - Hibernate Performance Curve Level 2 </li></ul><ul><ul><li>EHCache Support Built in to the product </li></ul></ul><ul><ul><li>Advantages </li></ul></ul><ul><ul><ul><li>Coherent Cache Across the cluster </li></ul></ul></ul><ul><ul><ul><li>Easy to integrate with existing applications </li></ul></ul></ul><ul><ul><ul><li>Performs very well </li></ul></ul></ul><ul><ul><ul><li>Eliminate the artificial cache misses in clustered environment </li></ul></ul></ul><ul><ul><li>Disadvantages </li></ul></ul><ul><ul><ul><li>Objects are represented as BLOBs by Hibernate </li></ul></ul></ul><ul><ul><ul><li>Doesn’t take direct advantage of Terracotta Scale-Out Features </li></ul></ul></ul><ul><li>Cluster Detached POJOs - Hibernate Performance Curve Level 3 </li></ul><ul><ul><li>Cluster Pure POJOs </li></ul></ul><ul><ul><li>Re-attach Session in the same JVM or a different JVM </li></ul></ul><ul><ul><li>Advantages </li></ul></ul><ul><ul><ul><li>Scales the best </li></ul></ul></ul><ul><ul><ul><li>Take Advantage of POJOs - Fine-grained changes, replicate only where resident </li></ul></ul></ul><ul><ul><li>Disadvantages </li></ul></ul><ul><ul><ul><li>Some code changes required to refactor Hibernate’s beginTransaction(), commit() </li></ul></ul></ul>
    18. 18. Demonstration Application <ul><li>Simple CRUD application </li></ul><ul><ul><li>Based on Hibernate Tutorial (Person, Event) </li></ul></ul><ul><ul><li>Already Refactored for Detached POJOs </li></ul></ul><ul><ul><li>Simple Session Management in Terracotta Environment - POJO wrapper </li></ul></ul><ul><ul><li>Detached Strategy requires a flush operation </li></ul></ul><ul><li>CREATE OPERATION </li></ul><ul><ul><li>Creates a new Person </li></ul></ul><ul><li>UPDATE OPERATION </li></ul><ul><ul><li>UpdateAge -> updates the age </li></ul></ul><ul><ul><li>UpdateEvent -> creates a new event and adds to Person </li></ul></ul><ul><li>READ OPERATION </li></ul><ul><ul><li>Sets the current object to a random Person </li></ul></ul><ul><li>DELETE OPERATION </li></ul><ul><ul><li>Not implemented </li></ul></ul><ul><li>FLUSH OPERATION </li></ul><ul><ul><li>Re-attaches Session and writes modified POJO to DB </li></ul></ul>
    19. 19. Source Code
    20. 20. Performance Tests <ul><li>ReadAgeHibernate </li></ul><ul><ul><li>25k iterations </li></ul></ul><ul><ul><ul><li>Reads a Person object, reads the age, commits </li></ul></ul></ul><ul><ul><li>Run with and without 2nd level cache </li></ul></ul><ul><li>UpdateAgeHibernate </li></ul><ul><ul><li>25k iterations </li></ul></ul><ul><ul><ul><li>Reads a Person object, updates the age, commits </li></ul></ul></ul><ul><ul><li>Run with and without 2nd level cache </li></ul></ul><ul><li>ReadAgeTC </li></ul><ul><ul><li>Reads a Person object </li></ul></ul><ul><ul><li>Sets person object into Terracotta clustered graph </li></ul></ul><ul><ul><li>25k iterations </li></ul></ul><ul><ul><ul><li>Reads the age </li></ul></ul></ul><ul><li>UpdateAgeTC </li></ul><ul><ul><li>Reads a Person object </li></ul></ul><ul><ul><li>Sets person object into Terracotta clustered graph </li></ul></ul><ul><ul><li>25k iterations </li></ul></ul><ul><ul><ul><li>Updates the age </li></ul></ul></ul><ul><ul><li>Commits </li></ul></ul>
    21. 21. Results: Hibernate vs. Detached POJOs ~ 500,000 ops / sec Terracotta Read ~ 1800 ops / sec Hibernate + 2nd Level Cache Read ~ 1000 ops / sec Hibernate Read Results Type Operation Results Type Operation ~ 7000 ops / sec Terracotta Update ~ 1800 ops / sec Hibernate + 2nd Level Cache Update ~ 1000 ops / sec Hibernate Update
    22. 22. User Was Happy <ul><li>Database was still the SoR which kept reporting and backup simple </li></ul><ul><li>Scalability was increased by over 10X </li></ul><ul><li>Availability was not compromised since test data was still on disk, but in memory-resident format instead of relational </li></ul>
    23. 23. Partitioned Architecture
    24. 24. Example Caching Service <ul><li>Reduce utilization of System of Record </li></ul><ul><li>Support 4 BUs </li></ul><ul><li>10K queries / second today </li></ul><ul><li>Headroom for 40K queries / second </li></ul><ul><li>(Note: all performance goals) </li></ul>
    25. 25. BU #1 BU #2 BU #3 BU #4 Data center Websphere MQ Series - MOM Messaging Infrastructure (JMS Topics) Caching Layer SoR MQ API MQ API Existing MQ Cache Node 1 Cache Node 27 Terracotta Server Terracotta Server . . . SoR API single pair
    26. 26. BU #1 BU #2 BU #3 BU #4 Data center Websphere MQ Series - MOM Messaging Infrastructure (JMS Topics) Caching Layer SoR MQ API MQ API Existing MQ Cache Node 1 Cache Node 27 Cache Node 14 Terracotta Server Terracotta Server Terracotta Server Terracotta Server Cache Node 15 . . . SoR API . . .
    27. 27. BU #1 BU #2 BU #3 BU #4 Data center Websphere MQ Series - MOM Messaging Infrastructure (JMS Topics) Caching Layer SoR MQ API MQ API Existing MQ Cache Node 1 Cache Node 27 Cache Node 14 Terracotta Server Terracotta Server Terracotta Server Terracotta Server Cache Node 15 . . . . . . SoR API Stateless Software Load Balancer
    28. 28. Demo 2 Shared Queue
    29. 29. User Was Unhappy <ul><li>Simplicity was lost. The Partitioning leaked up the application stack </li></ul><ul><li>Availability was no longer easy (failure had to partition as well) </li></ul><ul><li>Scalability was the only “Scale Out Dimension” delivered </li></ul>
    30. 30. Lessons Learned: Scalability + Availability + Simplicity <ul><li>Stop the Madness </li></ul><ul><ul><li>Stop the hacking! Stop the clustering! </li></ul></ul><ul><ul><li>Start naïve and get more sophisticated on demand </li></ul></ul><ul><li>Balancing the 3 requires scalable, durable memory across JVM boundaries (spread out to scale out) </li></ul><ul><li>Simplicity  Require no specific coding model and no hard-coded replication / persistence points </li></ul><ul><li>Scalability  Read from Local Memory; write only the deltas in batches </li></ul><ul><li>Availability  Write to external process + write to disk </li></ul>
    31. 31. Load-balanced Scale Out <ul><li>Simplicity  Ignore the impedance mismatch. Don’t be afraid of the DB. </li></ul><ul><li>Scalability  Just cache it! (EHCache, JBossCache, custom) Disconnect from the DB as often as you can </li></ul><ul><li>Availability  Distributed caches can be made durable / reliable and shared with JVM-level clustering </li></ul>
    32. 32. Partitioned Scale Out <ul><li>Simplicity  Share concurrency events, not data. Abstract as necessary (the caching service or SEDA or master / worker) </li></ul><ul><li>Scalability  Once in the correct JVM, don’t move data. push only what changes </li></ul><ul><li>Availability  Guarantee both the events and the data cannot be lost </li></ul><ul><li>Honestly. Punt on partitioning if you can. Most people who need it will know, based on the use case outrunning disk, network, CPU, etc. </li></ul><ul><ul><li>Example: Pushing more than 1GBit on a single node where multiple nodes could each push 1GBit </li></ul></ul>
    33. 33. Thank You <ul><li>Learn more at http://www.terracotta.org/ </li></ul>

    ×